November 11, 2001

Go Figure

By JOHN SCHWARTZ

NUMBERS seem so solid. As taught in elementary school, they are as reliable as cold milk at lunch: 2 + 2 = 4, immutable and comforting in their certainty.

But try making sense of numbers these days. How many died in the World Trade Center collapse? The official estimate still hovers around 4,700, but other estimates are lower. How many children lost parents in the catastrophe? The figures have varied wildly, reaching as high as 15,000. Senator Hillary Rodham Clinton, quoting reports in the news media, recently said 10,000. The actual number can't be known until there is a final tally of casualities, but those estimates — which would require that almost every victim have multiple children — are almost certainly inaccurate.

These aren't the only statistics that seem to have gone mushy. How many anthrax spores does it take to cause illness? How many civilians have been killed in Afghanistan? How many Muslims live in the United States? (Even the number of virgins that greet Muslim martyrs isn't steady.) George W. Bush complained about "fuzzy math" during the 2000 campaign (and what was the Florida debacle, if not the ultimate fuzziness of numbers at the margins of statistical significance?), but in the confusion following disaster and in the fog of war, reliable numbers are harder than ever to come by.

It doesn't help that journalists and politicians tend to grab the most dramatic figures, giving inflated statistics a life of their own. The lower, less flashy estimates are slower to make their way into the national consciousness, and tend to be less hardy. Since, in many cases, the best estimates provide only a range of choices, those at the microphone and keyboard tend to pick the number that best underscores the point they hope to make. "It's not lying," said John Allen Paulos, a mathematician at Temple University. He says it's just that "people like to use numbers as decorations more than as real descriptors."

The desire for solid numbers and simple answers in troubled times is easy to understand. But that urge is more an emotional response than a scientific one — and the results are often unreliable. As Professor Paulos said: "71.3841674321 percent of statistics are made up on the spot."

Simple numbers are anything but, he said: "Most numbers that people are interested in aren't known with any degree of precision." Professor Paulos has written popular books on what he calls "innumeracy," the inability of many Americans, including journalists, to get their numbers right. Bogus figures don't usually bother people, he says, only surfacing as an issue when the statistics become important, as in the Sept. 11 attacks.

Some numbers are difficult to determine because there is no clear way to get the necessary data: even now, it isn't known how many people were inside the World Trade Towers on Sept. 11, and a final tally is still being prepared of the missing. No one would dream of doing the kind of research necessary to determine how many spores are required to cause an anthrax infection, since it would require giving people the disease.

In the case of the number of Muslims, the higher estimates of six million to seven million have generally come from Muslim organizations. But Tom W. Smith, director of the General Social Survey at the National Opinion Research Center at the University of Chicago, said there was "no basis for sustaining a number even half that high."

The most credible source of information, the census, is prohibited by federal law from asking about religious faith. Most surveys involve about 1,000 people, which makes measurement of any group, including Muslims, who constitute about 1 percent of the population, iffy at best. The larger estimates of the number of Muslims in America, Professor Smith said, are often based on immigration figures from Arab nations. But many Muslims do not come from Arab lands, he noted, and many immigrants from those places are not Muslim.

In recent weeks, both Professor Smith and researchers at the Graduate Center of the City University of New York have produced new estimates of 1.8 million to 2.8 million Muslims. The Chicago center's estimates, in particular, were criticized by Muslim organizations, which contended that they showed the biases of those who financed the study: the American Jewish Committee. The exercise, spokesmen for the groups said, was intended to diminish the influence of Muslims in America by underestimating their numbers.

The population confusion is part of a larger battle that works its way into any discussion of the census, voting rights and more, said David Murray of the Statistical Assessment Service, which is affiliated with the Center for Media and Public Affairs in Washington. "The symbolism of the number is starting to take priority over the academic validity of the count," Mr. Murray said.

The best prescription for statistical slipperiness was laid out nearly 50 years ago in a slim primer to the numbers game, "How to Lie With Statistics." The author, Darrell Huff, wrote that "the secret language of statistics, so appealing in a fact-minded culture, is employed to sensationalize, inflate, confuse and oversimplify."

He preached to readers that they were part of the chain of accountability, and needed to look for bias in the origin of the statistics or their treatment, and to ask the kinds of questions that poked holes in shoddy or dishonest work. "Without writers who use the words with honesty and understanding and readers who know what they mean," Mr. Huff wrote, "the result can only be semantic nonsense."


Copyright 2001 The New York Times Company | Privacy Information