What’s ‘Good for You’ Often Ends Up Being Bad

Nearly One-Third of Initial Medical Studies Misleading, Major Review Finds

The Associated Press

CHICAGO – Here’s some medical news you can trust: A new study confirms that what doctors once said was good for you often turns out to be bad — or at least not as great as initially thought.

The report is a review of major studies published in three influential medical journals between 1990 and 2003, including 45 highly publicized studies that initially claimed a drug or other treatment worked.

Subsequent research contradicted results of seven studies — 16 percent — and reported weaker results for seven others, an additional 16 percent. That means nearly one-third of the original results did not hold up, according to the study in Wednesday’s Journal of the American Medical Association.

“Contradicted and potentially exaggerated findings are not uncommon in the most visible and most influential original clinical research,” said study author Dr. John Ioannidis, a researcher at the University of Ioannina in Greece.

One study is not enough

Experts say the study is a reminder to doctors and patients that they should not put too much stock in a single study and understand that treatments often become obsolete with medical advances.

“A single study is not the final word, and that is an important message,” editors at the New England Journal of Medicine said in a statement about the study.

The refuted studies dealt with a wide range of drugs and treatments. Hormone pills were once thought to protect menopausal women from heart disease but later were shown to do the opposite. Contrary to initial results, Vitamin E pills have not been shown to prevent heart attacks.

Contradictions also included a study that found nitric oxide does not improve survival in patients with respiratory failure, despite earlier claims. And a study suggested that an antibody treatment did not improve survival in certain sepsis patients; a smaller previous study found the opposite.

Ioannidis acknowledged an important but not very reassuring caveat: “There’s no proof that the subsequent studies … were necessarily correct.” But he noted that in all 14 cases in which results were contradicted or softened, the subsequent studies were either larger or better designed. Also, none of the contradicted treatments is currently recommended by medical guidelines.

Nothing’s ‘black and white’

Ioannidis’ study examined research in the New England Journal of Medicine, Lancet and JAMA — prominent journals whose weekly studies help feed a growing public appetite for medical news.

Not by accident, this week’s JAMA also includes a study contradicting previous thinking that stomach-lying helped improve breathing in children hospitalized with acute lung injuries. The new study found they did no better than patients lying on their backs.

Dr. Catherine DeAngelis, JAMA’s editor, said she included the study with Ioannidis’ report to highlight the nature of medical research.

“The crazy part about science and yet the exciting part about science is you almost never have something that’s black and white,” she said.

Ioannidis said scientists and editors should avoid “giving selective attention only to the most promising or exciting results” and should make the public more aware of the limitations of science.

“The general public should not panic” about refuted studies, he said. “We all need to start thinking more critically.”

DeAngelis also noted that the media can complicate matters with misleading or exaggerated headlines about studies.

Ioannidis said that the studies most likely to be contradicted later were what scientists call nonrandomized studies. These are often based on observations of patients’ lifestyles rather than on results from a drug or other intervention assigned by researchers.