Commentary: The Credibility Gap in Drug Research

By Paul Raeburn

It's supposed to be the gold standard of medical research: Doctors randomly split research subjects into two groups, with some getting a new drug, others a sugar pill. To avoid any bias, the researchers don't know who's getting what. This ritual--the randomized, double-blind clinical trial--often takes years and costs tens of millions of dollars, but it produces clear, unbiased data on the benefits of drugs.

At least that's the way it's supposed to work. In fact, the reports of those results are often misleading or incomplete. On June 1, the British Medical Journal published a report critical of a Pharmacia Corp.-funded study of its $3 billion-dollar-a-year arthritis drug, Celebrex. The report claimed the study's favorable results omitted contradictory data. And on June 5, The Journal of the American Medical Assn. (JAMA) published a series of reports critical of the way trials are done and reported, and the way conclusions are often couched to make new treatments seem better than they are.

The Celebrex case illustrates one such problem. The study, reported in JAMA by company-funded researchers two years ago, concluded the drug was associated with fewer ulcers and ulcer complications than two other drugs, ibuprofen and diclofenac. The British Medical Journal says the article was misleading because it omitted data that found no safety benefit for Celebrex. (The additional data were later made public at a Food & Drug Administration advisory committee meeting.)

G. Steven Geis, Pharmacia's (PHA ) vice-president for research, says information was omitted only because it was not reliable. On June 7, however, the FDA decided, using all the data, that the study "did not show a safety advantage in upper gastrointestinal events for Celebrex."

Critics blame industry sponsorship for these problems. It is not only the published studies that cause concern but also the studies that never appear, says Dr. Drummond Rennie, a deputy editor at JAMA and professor at the University of California at San Francisco. When a study starts going wrong, its sponsor may be tempted "to stomp on the investigators and say, `You must keep this quiet."' And "companies have yielded to that temptation," he says. Others agree. "In situations when there are large economic interests, you just cannot be sure whether the problems in the trial have been resolved in a fair way," says the author of the British Medical Journal article, Dr. Peter Jüni of the University of Bristol in Britain and the University of Berne in Switzerland.

"That's not true," says Mark Grayson, a spokesman for the Pharmaceutical Research & Manufacturers of America. Drugmakers support efforts to assure the independence of academic researchers who participate in industry-sponsored trials, he says.

Rennie thinks editors at the prestigious medical journals are getting better at policing these issues. He is still rankled, however, by JAMA's publication of the Celebrex study. The study's authors, including Pharmacia, "were not open with us," he says. "They signed letters saying the studies have all the relevant stuff," but "they had contradictory results when they sent us this paper, and they should have revealed them to us. And they didn't." Says Pharmacia's Geis: "We've tried to work real hard with JAMA to resolve any misunderstanding. We value JAMA."

Rennie says universities and their researchers must insist on guarantees from company sponsors that the results can be freely reported. Juni says the industry could fund an independent agency to finance studies. The FDA, which often has more data than the journals, does point out discrepancies, says Dr. Sandra Kweder, deputy director of the FDA's Office of New Drugs. But "it's difficult for us to police every potential journal," she says. "We don't have a department of journal review to catch those things."

The aim is to restore confidence in the findings of medical studies. As was the case with Celebrex, these studies are often widely circulated in the medical community. Many doctors begin prescribing drugs based on the findings, before the FDA rules on whether the studies have met its standards. If confidence in the veracity of medical research is lost, however, it is the pharmaceutical industry that will suffer.

Raeburn covers science and medicine in New York.

    Before it's here, it's on the Bloomberg Terminal.