TCS Daily

Survey Says!?

By Arnold Kling - May 14, 2003 12:00 AM

When I teach high school statistics, I try to warn my students to be skeptical of survey results. The example that I use is what I call the omniscient voyeur.

Newspapers and magazines love to report sex surveys. With a straight face, a publication will report a finding like "The average male has five times the number of sex partners as the average female."

I ask students to imagine that there is an omniscient voyeur, who observes every heterosexual relationship that takes place. The voyeur counts the total number of relationships, which he calls N. Suppose that the number of males in the population is M, and the number of females in the population is F. Then the average number of heterosexual relationships per male is N/M, and the average number of heterosexual relationships per female is N/F. Assuming that the average number of males is close to the average number of females, then N/F has to be close to N/M, so that it is impossible for the average male to have five times as many heterosexual partners as the average female. The omniscient voyeur knows that the survey obtained bias results.

Revealed Preference

Recently on TCS, Iain Murray criticized the methods used in a survey of attitudes on America. The findings were broadly anti-American, and Murray pointed out a number of ways in which the design and execution of the survey influenced those results.

Another way to see the flaws in the survey is from the perspective of the omniscient voyeur. To assess people's real attitudes about America, we can look at immigration and emigration statistics. Unfortunately, the data available on the Internet from the U.S. Immigration and Naturalization Service are not fully up-to-date (why am I not surprised?).

Nonetheless, this table compares immigration to the United States with emigration from the United States by decade. For the latest period available, the 1980's, it shows 7.3 million immigrants and only 1.6 million emigrants. Somewhere between four and five times as many people chose to come into the United States as chose to leave. I would venture to guess that this ratio has held up in recent years.

The demand for immigration to the United States is even greater than the official statistics suggest. In the 1990's, the unauthorized immigrant population is estimated to have doubled, from 3.5 million to 7.0 million. In spite of these large figures, one can presume that immigration laws kept out some would-be immigrants, so that the true demand to live in the United States is even higher.

Inferring people's attitudes from the choices that they make rather than from their responses to a survey is a well-established doctrine in economics, known as revealed preference. Observing market behavior is the economist's equivalent of acting as an omniscient voyeur.

Another illustration of revealed preference is the behavior of teachers in public schools. As the Friedman Foundation reports, "50 percent of Milwaukee's and 40 percent of Cleveland's public school teachers send their own children to a private school."

Income and Happiness

Yet another example of questionable survey research is a study that purports to show that people obtain little incremental happiness beyond a certain level of income. As reported in the Atlantic Monthly by Don Peck and Ross Douthat, the results suggests that "Above about $20,000 per capita, increases in wealth [sic] yield at best minimal increases in happiness." (Presumably, the $20,000 per capita figure refers to annual income, not cumulative wealth.)

From the standpoint of revealed preference, the statement that income over $20,000 does not raise happiness simply falls apart. Observing the fact that even people with very high incomes choose to work, an economist would infer that for most people the point at which income brings sharply diminishing returns to happiness must be much higher than $20,000. If $20,000 were the point of diminishing returns, then people who earn more than that would reduce their work effort and consume more leisure.

If anything, market behavior under-estimates the relationship between income and happiness. That is because taxes drive a wedge between market earnings and consumer well-being. An individual who faces income taxes, payroll taxes, and sales taxes has an incentive to sit around and consume leisure rather than go into the market and earn taxable income in order to obtain taxable consumer goods and services.

Tax disincentives toward earning market income are even higher in other countries. For example, Edward Prescott argues that a significant portion of the difference in economic performance between France and the United States is caused by the higher tax rates faced by French workers. Because of this tax factor, Prescott says, "market time is about 30 percent lower in France than it is in the United States." In other words, if French tax rates were reduced to U.S. levels, French workers would increase their market labor significantly, because earning more income would increase their happiness.

The lesson here is that consumers' market behavior represents an important indicator of their preferences. When a survey produces reported attitudes that are inconsistent with behavior, skepticism is in order. Remember that the omniscient voyeur might be able to easily discredit the survey.

TCS Daily Archives