TCS Daily


The Nature of Things: War-o-pedia Continues

By Robert McHenry - March 28, 2006 12:00 AM

Just for fun, I posit as an axiom that, in the very nature of things, things aren't always as they appear. This ought to arouse no controversy. But it is precisely in controversy that we need to be reminded of this simple but profound rule.

There has been a certain amount of controversy in the last couple of years about the online free-for-all "encyclopedia" called Wikipedia. I have happily added to that controversy, here and here and here, among other places. Not long after the last of these appeared, the science journal Nature published a study that purported to pit Wikipedia against the Encyclopaedia Britannica. The editors at Nature selected a number of topics in the sciences, sent the relevant articles from the two reference sources to some expert reviewers, and collected their comments. The conclusion: Wikipedia came "close to" Britannica in terms of accuracy.

So, hats and horns and champagne all round on the Wikipedia side, shock and gloom at EB, eh? Well, not quite, at least not as far as EB was concerned. But before I get to that, please notice a couple of interesting aspects of the story.

First, the spin. The "close to" meme was picked up almost unanimously by the media. It wasn't long before "close to" had evolved (or was it part of an intelligent design?) into "pretty much equal to." Some examples:

"Wikipedia survives research test" -- BBC News

"Study: Wikipedia as accurate as Britannica" -- CnetNews.com

"Accuracy of Wikipedia matches Britannica" -- CBC News

"Wikipedia, Britannica: A Toss-Up" -- Wired online

An honorable exception to the just-print-the-PR-release-and-let's-get-some-lunch style of herd journalism was Andrew Orlowski of The Register, who was apparently alone in actually reading the study and looking at the numbers. For, as a matter of fact, the details of the report would have justified a headline that read

"Wikipedia: One-third More Errors, and Badly Written, Too."

But then, what sort of story would that have been? Reporters like man-bites-dog, David v. Goliath stories of spunky underdogs and easy O. Henry ironies.

Second, the Nature story fell like heaven-sent rain onto a Wikipedia parched by several stories that had seriously threatened its claims to respectability. The timing of the Nature study, and the cheerleading editorial that accompanied it, could not have been more fortuitous. And the Wikipedians made hay while the sun shone (this was after the rain ceased, of course), especially in calling attention to the speed with which they corrected all the errors identified by the Nature reviewers and thus inviting snickers at the relatively plodding pace of Britannica. Keep this point in mind for a bit.

Well, mirabile dictu, it turns out things ain't entirely what they seem. The editors at Britannica took the Nature study seriously enough to investigate how it was done, what the alleged errors were, and how conclusions were drawn. The results of their review of their review can be read here (pdf). I need not repeat them. Suffice it to say that the Nature study ought to be a source of shame for those who concocted and published it. But I suspect that, given the nature of things, it won't be.

Now let's think for a moment about the contrast between the two approaches to this study. The Nature reviewers identified what they said were a great many errors in both publications. On the one hand, the Wikipedia people accepted the reviewers' comments at face value, as they do virtually anyone's comments, expert or no, and hastened to incorporate them into their articles. Such is the Wikipedia way. Reference publishing is a NASCAR event, and truth is established, for any given nonce, by whoever has shouted loudest or last. This is less an editorial principle than a political one, and it is politics of a notably dubious kind.

On the other hand, the Britannica editors questioned and double-checked all, and challenged many, of the reviewers' criticisms. Even apart from such gross blunders on Nature's part as submitting only portions of articles (that were then criticized for omissions) or text not from the Encyclopaedia Britannica at all, the reviewers did all the human-nature-y things that editors are trained to resist: They assumed that their offhand knowledge was simply true and not in need of confirmation; they mistook difference of opinion for error; they mistook preference for knowledge.

Experts are not immune to this kind of behavior and sometimes are more than ordinarily prone to it -- having, as they do, the habit of being right, or at least of having it assumed that they are. Any experienced editor has seen these things a thousand times (and, truth be told, has committed them a few times, too, despite himself). The Britannica editors report that they have executed those necessary corrections that stood up to critical examination.

Credulity and cynicism have become second nature for so many of us. Credulity and cynicism -- the default, fraternal-twin attitudes of the lazy, ideological, or corrupt mind. If history is any guide, the comments to this essay will include some or all of the following:

"Pay no attention. He works for Britannica." (No, I used to.)

"Pay no attention. He used to work for Britannica." (Nice catch, Sherlock. You had to read all the way to the end to find that out.)

"Pay no attention. He doesn't get it." (Why is it up to me to get something? Do you get what I'm saying? Paraphrase my argument.)

How odd that the lazy, ideological, and corrupt so often turn out to be the loudest and last to shout the slogans that squelch thought. Apparently it's just in the nature of things.

Robert McHenry is Former Editor in Chief, the Encyclopaedia Britannica, and author of How to Know (Booklocker.com, 2004).
Categories:

19 Comments

Godel's revenge
While EB is a work of finte length,and can be both reviewed , and peer reviewed ,in its entirity The Wikipedia presents a literally macropedic paradox- having an unlimited number of authors writing simultaneously, it is not generated faster than any one reader can follow, but expanding at such a rate that no one could even read the reviews of all the new material as fast as they appear.

So only statistical measures of its overall quality exist pending the advent of AI software that emulates a hyperspeed-reading reviewer and enough levels of review text condensation to return an annual review of the Wikipedia short enough to be read in 364 days or less.

There may be a solution- there are plenty of contributors available to praise or damn anything foe $500 a pop , so why not save time with a bidding war between EB and WP?

What about the real groaners
I wonder what would have happened if someone started a contest to find the 20 worst articles in britannica and wikipedia. I think that is the true difference between the two.

Most of the "errors" discussed in the britannica rebuttal were close to nit-picking.

Some articles I've seen in wikipedia were so far off base I would have been ROTFL, were it not for the fact that many people take this as gospel.

The best is...
when someone is debating or just plain arguing with you and supplies some "evidence" with a wikipedia address.

Wikipedia is for those who are research-challenged or just plain lazy. I have never understood why people love the site or why they even bother to believe anything written within it. Perhaps it is just me but I like my information and facts to be accurate and supplied by people who I can track down to determine bias.

wikipedia as a source
I've personally suspected that a number of people who cite Wikipedia as a source in an argument, just finished modifying the Wikipedia entry that they are referencing.

Quit hitting the snooze button

Mr. McHenry, would you please wake up? EB is venerable, so I resect your efforts to champion it.

But it grows more absurd by the day even to compare EB and Wikipedia. People hardly care whether Wikipedia is accurate in close detail, since they typically don't refer to it for accuracy at all. Rather, people refer to Wikipedia to familiarize themselves with an alien topic, to take in an overview.

And it is EB's inability to supply even an overview of on a vast range of topics that renders it, for some purposes, wholly inferior and virtually incomparable to Wikipedia. What can EB tell me about Kinky Friedman or Rod Stewart or the Williams tower? Nothing, I suspsect.

No longer is EB the quickest or cheapest way to get exposure to a subject. Nor does EB serve well as an academic resource, since true scholarship has little to do with encyclopedias.

So, Mr. McHenry, you can write until you are blue, but if EB is to survive, it must radically alter its business model.

If you love EB, as I do, then you will send them that message.














Quit Hitting the Snooze Button: Amen.
Quit hitting the snooze button: Amen.

Right on. However, I would add that caution is required in reading anything. Granted, EB has earned a good deal of respect. But as to any entry, additional reading is recommended if time and importance permit.

In reading the news, comparison of sources always adds to understanding. Even reading the same NYT article twice greatly adds to understanding. Yeah, I know, most beat my limited reading comprehension and only need to glance at an article once to understand the story. But I have to put up with what I’ve got and so I do it my way: read most things of importance at least twice and check it for accuracy as time and chance permit.

The Immortals
Perhaps you should commend Kinky Friedman , Rod Stewart and the Williams tower to the syndics of Colliers Encyclopedia, as they cater to all tastes .

That's what Gutenberg Project is for

Why Colliers, when Wiki is free?

I've read Mortimer Adler opine about the "immortals," so I have sufficient reverance for them. But who needs EB to access the immortals, when there's the Gutenberg Project, also free?

My point is that EB needs a new business model. Do you disagree?


No Comment

War -o-pedia continues
Is author ofthis artcals have some prejudiced with encycopedia Britanica, he was retired formor editor of Britanica.I found so many mistakes in internet encycopidia, real fact is that today knowledge spreading so fastthat not a single pedia can gie correct information they to correct everyday otherwise they are bound to spread wrong information

Is this real
"Is author ofthis artcals have some prejudiced with encycopedia Britanica,"

Would you too like to write entries in Wikipedia?

All too real--Wiki errors included at no charge.
Yes, he would like to write articles (or maybe just articals) in the Wikipedia. I know I have seen a lot of his work at the Wikipedia site.

I rarely use Wikipedia for my own research, except to get a very, very, very rough idea of a topic with which I am not at all familiar. The info from Wiki gives me a little bit of an idea where I should start my search for accurate information, preferably from original source material. I never quote Wiki as a reliable source in any articles I write. To do so is to admit that one is not much of a researcher and should not be believed.

I have personally run across dozens of errors in Wiki, most of them apparently perpetrated for political correctness purposes.

The best feature of Wikipedia is their warranty: Satisfaction guaranteed or double your money back.

Really?
Just how, exactly, does one familiarize oneself with an alien topic by reading a description that is not accurate? In fact, is it not even MORE problematic to give the ignorant an ignorant source from which to "learn"? The idiocy of Wiki is that one cannot know how accurate it is (without going outside, but even here, there are often are no citations to check the accuracy), and in fact, the accuracy of any given article varies day by day, and thus cannot even be tracked. One hardly advances the state of knowledge by using such a source. In fact, one retards the state of knowledge by doing so.

-Bob

You can track down Wikipedia authors
It is possible that it is easier to track down Wikipedia contributors than it is EB contributors. The history of an article is available in its own tab and if you want to examine for bias, the fact that you can look at changes both from an article-centric pov as well as a contributor-centric pov means that bias should become easier to spot, not harder.

How, exactly, does one track down the author of an EB article or measure his output to the project for bias? I really don't know.

No, not real
Nobody by that name is a contributor.

Don't be silly
No contributor by that ID is contributing to the en.wikipedia.org project. Perhaps you were just making a joke, though.

"I have personally run across dozens of errors in Wiki, most of them apparently perpetrated for political correctness purposes."

I wonder if you fixed any of them.

The entire point of Wikipedia is that it's always improving. These days, if you see incorrect things being put in because of PC, you can flag that. Instructions on how to do that can be found here.

When I was a little boy, my mom would teach me to leave things better than I found them. Wikipedia is the encyclopedia for those that believe in that ethic.

Editors are fallible too
Robert McHenry says, "On the one hand, the Wikipedia people accepted the reviewers' comments at face value, as they do virtually anyone's comments, expert or no, and hastened to incorporate them into their articles. Such is the Wikipedia way. Reference publishing is a NASCAR event, and truth is established, for any given nonce, by whoever has shouted loudest or last."

However, a look through
(which is not all that easy to find) would have shown that although Wikipedians accepted most of the comments, they rejected some and tabled at least one (I didn't read the whole thing) till it could be researched.

We all make mistakes. I haven't scrutinized some other statements in McHenry's article that I have doubts about, but since I found only one obvious inaccuracy, I suspect it's up to Wikipedia or Britannica quality in accuracy. The sneers and outright insults wouldn't last long at either place.

Here's an example of Wikipedians not accepting the reviewer's comment at face value.

"Page 10: the term antifreeze is generally used for ethylene glycol not ethanol. Ethanol has a low melting point but this is not given.

"* The reviewer appears to be simply mistaken. The article does not equate ethanol with antifreeze (again, the reviewer appears to be confused about the fundamental nature of Wikipedia, which occasionally requires readers to follow links to other articles for more information). [Mark: The reviewers did not it was a Wikipedia article and all links were removed. Dan100 (Talk) 10:35, 23 December 2005 (UTC)] The melting point of ethanol has been there since early 2003."

Full disclosure: Though I don't speak for Wikipedia, I use the same moniker there that I chose here,

JerryFriedman

missing link
Sorry, I don't know how to format links here (and there's no preview). The discussion I cited is at http://en.wikipedia.org/wiki/Wikipedia:External_peer_review/Nature_December_2005/Errors Let's see whether that works.

Mea culpa, indeed
As someone who took the press release regurgitation as gospel, I can only say that I was wrong to do so. The fact that I did so on television makes it even worse. In this regard, I fully accept charges of laziness. Ideological? Well, I "believe" in the project insofar as I get a great deal of use and enjoyment out of Wikipedia, but I am certainly not blind to its (many) flaws. As for you "getting it"... well, I think you do, and I for one welcome any and all reasonable criticism of the Wikipedia project, but it doesn't mean that I'm going to end my connection with it. The search for perfection, be it in a single article or an entire encyclopedia (or "encyclopedia" :) ) is unrealistic, but coming as close as one can is not.

TCS Daily Archives