TCS Daily

Polluting the Debate

By Iain Murray - March 7, 2002 12:00 AM

Wednesday's front-page news that scientists have linked particulate air pollution with early mortality from lung cancer and cardiopulmonary causes has certainly caused a stir. Every major newspaper has carried the story (the Washington Post placed it squarely on its front page) and several, such as the LA Times and the Pittsburgh Post-Gazette, have used it as the basis for editorials calling on the President and the Environmental Protection Agency for action.

This should not be surprising. The study was published in a major journal, the Journal of the American Medical Association. It was large and longitudinal - half a million people were tracked over 16 years. It controlled for many confounding factors. There was a clear biological pathway identified for the particles to have some effect. In all but one aspect it was the very model of a well-argued study.

The problem is that the increased risk of death the study found to be associated with pollution was tiny. At a 12 percent increased risk of death, the association is barely noticeable. Nor is that 12 percent a definitive figure. These studies reach conclusions by means of what are called confidence intervals - a range of values, within which the researchers are reasonably certain that the true value lies. In this study's case, the confidence intervals ran from a maximum of about 20 percent increased risk to a level where there was no real increased risk at all. Indeed, according to the study, persons living in polluted areas may even be less likely to die from other causes than people living in unpolluted areas. The 12 percent figure merely represents the mid-point in those confidence intervals.

Nevertheless, people might argue, the study still found an increased rate of death attributable to air pollution. This begs the question of exactly what sort of level of increased risk one needs to be able to say the study has found clear evidence of causation, rather than simple correlation. Epidemiologists are quite clear on this. In medical terms, the paper found a relative risk of 1.12. The New England Journal of Medicine (NEJM) told Science magazine ("Epidemiology Faces Its Limits," Gary Taubes, July 14, 1995) that, "As a general rule of thumb, we are looking at a relative risk of 3 or more [before accepting a paper for publication]." A former statistical consultant backed this up, but said, "If it's a 1.5 relative risk, and it's only one study and even a very good one, you scratch your chin and say maybe."

This study, albeit being a good one, does not even approach the "maybe" threshold. It is worthwhile to compare this study to the extremely controversial claim that abortion can cause breast cancer. Several studies have shown a relative risk of 1.5 (i.e. a woman who has had an abortion appears 50 percent more likely to develop breast cancer). Very few epidemiologists, however, accept that the results of these studies go anywhere near proving causation. The relative risk is so small in epidemiological terms that it could easily be accounted for by other factors.

That is very often the case with studies that find small relative risks. This list of associations between various putative causal factors and cancers should underline how difficult it is to establish causation at these levels:

  • Pollution (this study): rr (relative risk) 1.14 (lung cancer)

  • High cholesterol diet: rr 1.65 (rectal cancer in men)

  • Eating yogurt once a month: rr 2 (ovarian cancer)

  • High fat diet: rr 2 (breast cancer)

  • Occupational exposure to dioxin: rr 1.5 (all cancers)

  • Regular use of high alcohol mouthwash: rr 1.5 (mouth cancer)

  • Vasectomy: rr 1.6 (prostate cancer)

  • Ever having used a sun lamp: rr 1.3 (melanoma)

  • Consuming olive oil once a day or less: rr 1.25 (breast cancer)
(Source: Science magazine, ibid).

Although this study controlled for the most obvious factors in relation to early death, such as age, weight, sex, race, occupational exposure, diet and so on, the question must be asked as to whether all potential confounding factors were accounted for fully. For example, the study did find that many of the early deaths associated with pollution were among the less well educated; the better educated (those with any education beyond high school) did not die earlier at any significant rate. Could there be, therefore, some factor associated with education levels that could have caused the correlation? Women also had a miniscule overall increased risk (and a decreased risk of dying from lung cancer). Why does the phenomenon only really affect men? Could there be genetics factors involved that were not controlled for?

These are all examples of the sort of questions that have to be answered before this study can really be taken as proof that particulate air pollution increases the rate or dying. It certainly does not prove, as the LA Times strangely claimed, that it "greatly increases the rate of dying." Most epidemiologists would agree that this study's findings are too small to be meaningful yet. The question therefore must be asked, why did JAMA publish this at all? By doing so, it may have polluted the public health debate with a contaminant.

TCS Daily Archives