TCS Daily

When Peer Review Fails, Where Do We Turn?

By Henry I. Miller - February 8, 2008 12:00 AM

Those of us who regularly seek out information about scientific and medical subjects have learned that there's a hierarchy of reliability: the "facts" in a random blog or a New York Times' news article are on average less trustworthy than, say, those from the website of the National Institutes of Health. Likewise, the term "peer-reviewed journal" -- which means that articles have been evaluated anonymously by independent experts before being accepted for publication -- elicits instant respect from many people. Not from me, because I actually read many of those journals. I've found that it is not uncommon for articles that are egregiously, obviously flawed -- most often because the basic rules of scientific research have been violated -- to find their way into prominent scientific publications.

Some of the worst of these papers have conveyed false alarms about the safety of gene-spliced (or "genetically modified" plants that subsequently have been extensively reported in the popular press. In this way, misinformation is widely propagated and feeds the propagandizing by anti-technology activists.

A case in point is a 2001 paper in the British journal Nature that purported to show that genes from a pest-resistant, gene-spliced variety of corn had migrated into native corn plants in Mexico. However, colleagues of the authors had pointed out flaws in the methodology and results months before the article was submitted to the journal. The publication triggered criticism from major research groups that was published subsequently in Nature, and eventually the original paper was condemned by the editor-in-chief: "Nature has concluded that the evidence available is not sufficient to justify the publication of the original paper." No other research group has been able to confirm the original results.

In 2000, the American journal Science published an article in which the authors claimed to have evaluated the "ecological risks and benefits of genetically engineered plants." En route to concluding that they could draw no conclusion, the authors neglected the proven benefits of gene-spliced organisms, including enhanced yields, nutritional enhancements, less use of chemical pesticides and more no-till farming (which causes less soil erosion and runoff of chemicals and lower release of carbon dioxide to the environment). At the same time, no detrimental effects of gene-spliced plants had (or have) been described.

Another egregious and exceedingly harmful example of apparent anti-biotechnology bias appeared in an article in the British medical journal, The Lancet, which claimed to show that adding to potatoes a gene that codes for a substance toxic to insects caused damage to the immune system and stimulated abnormal cell division in the digestive tract of laboratory rats. But many research groups have shown that the experiments= research methodology was fundamentally flawed and that no conclusions about the safety of biotech foods can be drawn from them. After an extensive review, the British Royal Society issued a statement that detailed the ways in which the experiments were fatally flawed, concluding: "On the basis of this paper, it is wrong to conclude that there are human health concerns with the process of [gene-splicing] itself, or even with the particular genes inserted into these [gene-spliced] potatoes."

The editors of the journal remonstrated that in spite of the article's admittedly deficient methodology -- and over the strenuous objections of the paper's referees -- they published it to "make constructive progress in the debate between scientists, the media, and the general public" about a very politically charged issue. Unleashing such a sham has proved to be anything but constructive, because its publication is frequently cited as presumptive validation of its spurious conclusions. The irresponsible rationalization of the editors makes a mockery of the peer review process.

The most recent example of the failure of editorial and peer review occurred in an article published in September 2007 in the Proceedings of the National Academy of Sciences ("PNAS"). The authors claimed to show that pollen from gene-spliced corn was injurious to certain insects in a laboratory aquatic ecosystem, but their conclusions are dubious. Their methodology is atrociously sloppy and inadequately described. More important, the researchers reported elsewhere that they had failed to find these same effects in studies in the field -- which they neglected to reveal in the PNAS article. This is a critical omission because laboratory studies are designed to mimic what happens in the "real world." In other words, even if the laboratory studies had been performed correctly and carefully, positive results arguably would have been irrelevant because they don't mimic what happens in the field.

These kinds of failures of peer review corrupt the traditional process by which new scientific knowledge is obtained and reported; they inflict irreparable harm on the reporting and archiving of scientific developments for policy makers, the media, the public and the scientific community. Within weeks of the publication of the flawed PNAS article, for example, European Union environmental regulators cited it as justification for a ban on the sale of gene-spliced corn seeds.

Because science is (or is supposed to be) self-correcting -- a thesis is put forth, tested, and ultimately revised on the basis of new data -- misinformation conveyed to the scientific community distorts the entire process. Journals should request reviews of research articles from bona fide experts who do not have a known bias toward the subject of the research. These reviewers should be encouraged to ask probing, detailed questions, and the authors of the submitted article should be required to answer them satisfactorily before a paper is accepted.

Had such measures been taken in the case of the research articles described above, it is unlikely that any of them would have been published in a prominent journal. But what if editors lack the integrity and competence to undertake these measures? Will the scientific community take them to the woodshed? If they want to retain society's respect, they will.

Henry Miller, a physician and molecular biologist, is a fellow at Stanford University's Hoover Institution. He was at the N.I.H. and F.D.A. from 1977 to 1994. His most recent book (co-authored with Gregory Conko) is "The Frankenfood Myth." Note that a much longer version of this article, documented with references, will appear in the March issue of Trends in Biotechnology.


TCS Daily Archives