Health Studies: Separating Ballyhoo From Breakthrough

Almost daily, headlines trumpet the findings of new medical studies, from drugs to cure cancer to diets that ensure good health and longevity. Then, in short order, contradictory evidence often follows. For nonscientific types, this wealth of medical information can be confounding. How do you separate the ballyhoo from the breakthroughs?

First, it pays to remember that in a study's crossover to the mainstream press, "a lot of caveats get lost in the translation," says Dr. Alan Garber, director of the Center of Primary Care & Outcomes Research at Stanford University in Palo Alto, Calif.

Most studies represent an evolution, not revolution, in medicine. "Sudden, dramatic discoveries happen incredibly rarely," says Drummond Rennie, a deputy editor for the Journal of the American Medical Association (JAMA) and professor of medicine at the University of California in San Francisco. So be skeptical about "breakthroughs."

To rate the significance of research, look for key details in reports on medical studies. If a study is epidemiological, then it's not much more than an educated guess. Epidemiological research looks at large groups or populations to make correlations between such things as diet or habits and health. Because the studies do not occur in a controlled environment, they "are good for generating hypotheses but they don't prove anything," says Garber. A case in point: Researchers still believe that the Mediterranean diet, which is rich in olive oil and vegetables, may cut the incidence of heart disease, as the famous 1960-85 Seven Countries Study suggested. But even after adjusting for numerous variables, researchers could not eliminate all the factors that might explain their results.

Only controlled experiments or clinical trials establish definite causal links. Still, a study's size and duration may greatly affect how dependable its findings are. "Giving 10 people a pill for two months is hardly going to provide solid evidence of its effectiveness," says Dr. Cynthia Mulrow, director of the San Antonio center of The Cochrane Collaboration, a nonprofit organization that analyzes and compiles medical research. The headline-grabbing research on the weight-loss drug leptin was a six-month study involving just 123 people, and the program was completed by only 76% of its 60 obese subjects.

Meanwhile, don't get too excited if the research was conducted on lab animals--as were the highly publicized studies of drugs that eradicated cancerous tumors in mice. Treatments that work very well in animals often have no effect on humans. Even studies with human subjects may not be relevant if the subjects differ markedly from you in age, ethnicity, gender, or health status.

Also look closely at the reported health outcome. Research published last month in JAMA suggests that individuals who take the cholesterol-lowering drug lovastatin have a 37% lower risk of developing a serious symptom of heart disease, such as chest pain. But the study does not show that the drug reduces the likelihood of death or hospitalization or even makes people feel better.

You can try to evaluate studies yourself by accessing them directly in the top medical journals, which are available online. They subject studies to rigorous scientific review by three or more experts before publication. You should check sources such as The Cochrane Collaboration to get a sense of the number of studies conducted on a given topic, and which direction the weight of the medical evidence is leaning. These days, you have to do some spadework to get the real story behind news reports of the latest medical miracle.