TCS Daily

Dubious Data Awards

By Iain Murray - February 25, 2002 12:00 AM

Editor's Note: With this installment, Tech Central Station is pleased to present a new feature called "The Data Dump," an occasional look at the use, abuse, and misuse of numbers and statistics in journalism.

Every year there is a struggle between science and journalism that science normally loses. That's why the Statistical Assessment Service gives out Dubious Data awards to journalists who mangled statistics and subverted science in the way they reported stories.

Past winners have included the New York Times for claiming to have found the first free standing water at the North Pole in millions of years (it happens all the time) and National Public Radio for claiming that science had proved Thomas Jefferson fathered a child of a slave (it did nothing of the sort). This year, however, saw the award go to the entire press corps for one of the most astonishing displays of statistical illiteracy we've seen.

The winner was the story with no legs, but lots of bite. In July, August and the first few days of September headlines across America were dominated by one dangerous villain. No, not Gary Condit. This villain was unseen but threatening us all: The shark. With the exception of one excellent editorial in the New York Times, "The Statistical Shark," on Sept. 10, and the usual wise comments from statistician John Allen Paulos, it seemed that the whole U.S. media had fallen victim to the gaping maw of shark attack hype.

The frenzy was extraordinary. According to a NEXIS search, there were a mere 58 stories about shark attacks in the U.S. print media in June. This increased tenfold in July to 592, and it rose again in August to 684. September was the month the story would have consumed all others, with 624 entries up to and including September 11. The advent of another dangerous but unseen villain stopped all that.

What happened to produce all this attention? An eight year-old boy in Florida had his arm ripped off by a shark in July. Then two people were killed in North Carolina and Virginia within a few days. The last time two people had been killed by sharks in one year was in 1994. So 2001 seemed unusual, with two fatalities close together, and Virginia had never seen a shark attack fatality, while North Carolina had not seen one since 1957.

Reporters and pundits immediately jumped to conclusions. Some alleged there was an increasing trend of fatalities, others that sharks were being driven further north, possibly by global warming. One conservative commentator, according to the Times, theorized that "the whole thing [was] the fault of Bill Clinton, or at least of tighter shark-fishing limits passed during his presidency."

Of course, when we deal with such small numbers as fatal shark attacks in the U.S., two attacks rather than one or none is nothing unexpected. The fact that the two fatalities happened so close to one another is sheer coincidence. And the northernness of the attacks is also nothing particularly new: New Jersey has seen five fatal attacks in its history and even Massachusetts has had three (and as the last of those was in 1936, modern fishing limits cannot be to blame either).

The story, however, was more than a mere question of numbers. A host of factors multiplied each other. Shark attacks are seen more as violent attacks than as accidents. A violent attack is shocking; a violent attack on a child at play the more so. We have all seen "Jaws," so we can all visualize the events. Further examples compound the shock even more, and if the facts fit with a political prejudice (mankind is turning nature against him, or Bill Clinton was evil) then the shock can be raised to another power. The resulting equation turned two fatalities into thousands of column inches in the newspapers.

During the 1990s, when only five people were killed by sharks, 28 children were killed by falling TV sets. The Times editorial mentioned above concluded from STATS data that, loosely speaking, "watching 'Jaws' on TV is more dangerous than swimming in the Pacific."

After September 11, though, the media reflected on its summer obsession. There were 172 stories that included references to shark attacks in October. Many took the view that the coverage was an example of America contemplating its own navel ("The News that Got Away," Chicago Tribune, Oct. 30). In the end, the media realized that the story was toothless. As a footnote to the story, the University of Florida revealed its annual count of shark attacks worldwide for 2001 just last week. There were 76 attacks and five fatalities. Those numbers are down from 85 and 12 in the previous year.

Nevertheless, there is some hope. For the first time STATS awarded a Dubious Data Debunking award for a mainstream journalist questioning of shoddy research. It went to David Mehegan of the Boston Globe, who built on the work of writers like Linda Seebach of the Rocky Mountain News and Melissa Seckora of National Review Online in raising awkward questions about Professor Michael Bellesiles's award-winning book "Arming America." Mehegan drew attention to several problems with Bellesiles' research, which claimed to show that colonial America did not have a "gun culture."

To date, Bellesiles' replies have not satisfied his critics. Nor have they satisfied his university, Emory, which announced two weeks ago that they will be holding a formal investigation into the matter. We have no opinion as to the cause of the data discrepancies Mehegan and others identified, but the fact that they have been picked up on is cause for optimism.

So the struggle continues. Already this year we have seen the media blame the drug Accutane for the death of a teen who crashed a plane into a building in Florida. The Washington Times argued that a census report "found a threat to U.S. security" when all it did was estimate the number of illegal immigrants in America. And columnist Pat Buchanan claimed that gay men die at the average age of 43 (a figure that was debunked long ago in Slate by Walter Olson). This column will try to alert you to the latest example of dubious data that should be well and truly dumped.

Iain Murray is Director of Research at STATS. See for more daring demolitions of dubious data.

TCS Daily Archives