Linking Disease Markers to Medical Condition Often Exaggerated, Study Says

Studies that link genes and other biological markers to medical conditions such as heart disease or colon cancer may exaggerate the ties, casting doubt on some decisions doctors make for their patients, researchers said.

An analysis of 35 of the most widely cited medical papers on biomarker discovery showed that most of the findings didn’t hold up in larger studies, according to a review in today’s Journal of the American Medical Association. Findings not supported in larger analyses included those linking the BRCA1 breast tumor gene to colon cancer, and a measure of inflammation called c-reactive protein to heart disease.

Biomarkers like genes and blood proteins may help doctors determine who is at risk for disease, who is likely to respond to treatment and what course a disease might take. As biomarker findings are cited by other researchers in their papers, the information gets widely disseminated, even if untested in larger studies. As a result, doctors may be making decisions based on inaccurate information, said study author John Ioannidis.

“Single studies can be highly misleading,” said Ioannidis, a professor of medicine and health research and policy at Stanford University School of Medicine near Palo Alto, California, in a May 27 telephone interview. “When we do more studies, when we do larger studies, we see that the truth is a smaller magnitude of effects to what we thought when that highly cited study appeared.”

The researchers reviewed biomarker findings in 35 studies from 1991 to 2006 that had been cited by 400 or more other papers and had at least one analysis performed for the same association. The studies involved genes, infectious disease proteins and blood proteins linked mostly to cancer and heart disease.

Review Findings

The review found that 30 of the highly cited studies, or 86 percent, showed a stronger effect than the largest study investigating the same association. In only two cases was the effect stronger in the largest study than in the original, highly cited paper, researchers found.

For 29, or 83 percent, of the highly cited trials, an analysis of multiple studies for the same association showed a smaller effect, the authors said. Many of the highly cited studies were relatively small and among the first to find an association, they said.

Ioannidis, who conducted today’s study with Orestis Panagiotou of the University of Ioannina School of Medicine in Greece, said a more systemized approach to proving whether the biomarkers work is needed. For instance, if a biomarker is shown to work in a trial, then maybe a group of researchers could try to replicate the original findings. Only those biomarkers that are proven to work would continue forward in development.

Importance of Validation

“We have to learn to trust the bigger picture,” Ioannidis said in a statement. “And it’s better to demand this proof up front rather than waiting for it to happen on a case-by-case basis. It is vitally important to validate original published findings with subsequent large-scale evidence to make progress in the field of biomarkers and risk association.”

Patrick Bossuyt, a professor of clinical epidemiology at the University of Amsterdam, wrote in an accompanying editorial in the journal that it is “premature to doubt all scientific efforts at marker discovery and unwise to discount all future biomarker evaluation studies.”

Today’s findings “should convince clinicians and researchers to be careful to match personal hope with professional skepticism, to apply critical appraisal of study design and close scrutiny of findings where indicated, and to be aware of the findings of well-conducted systematic reviews and meta-analyses when evaluating evidence on biomarkers,” Bossuyt wrote.

To contact the reporter on this story: Nicole Ostrow in New York at nostrow1@bloomberg.net.

To contact the editor responsible for this story: Reg Gale at rgale5@bloomberg.net