Misunderstanding Margin of Error

Abstract

For years, scholars have documented the frequency to which the margin of error is provided in news reports of polls. What has not been systematically analyzed is whether the stories that report margin of error use this information correctly to interpret and present poll results. This study examines poll reporting on the network evening news during the 2000 presidential general election. Of the seventy-eight stories that reported findings from specific polls, forty-five (55 percent) included information about the sampling error. For the stories in which the accuracy of horse-race declarations could be assessed by the data used in the story, 47 percent were “inaccurate” because they misused the term statistical dead heat, claimed that one candidate was ahead when the results were within the margin of error (and not qualified as such), or said that the results were “outside of the margin of error” when they were not. These mistakes were most prevalent on NBC. Some of these inaccurate reports revealed a fundamental misunderstanding about what plus or minus means. As a result, changes in likely voters’ preferences were exaggerated during the general election campaign.

Get full access to this article

View all access options for this article.

References

American Association for Public Opinion Research. 1986. “Ethics Code.” Retrieved May 22, 2002, from http://www.aapor.org/ethics/code/html.

Bauman, S., and S. Herbst. 1994. “Managing Perceptions of Public Opinion: Candidates’ and Journalists’ Reactions to the 1992 Polls.” Political Communication 11:133–44.

Brady, H. E., and G. R. Orren. 1992. “Polling Pitfalls: Sources of Error in Public Opinion Surveys.” In Media Polls in American Politics, ed T. E. Mann and G. R. Orren. Washington, D.C.: Brookings Institution.

Broh, C. A. 1980. “Horse-Race Journalism: Reporting the Polls in the 1976 Presidential Election.” Public Opinion Quarterly 44:514–29.

Cantril, A. H. 1991. The Opinion Connection: Polling, Politics, and the Press. Washington, D.C.: CQ Press.

deClercy, C., and P. A. Ferguson. 2001. “Pollsters and Hucksters: A Comparative Analysis of Opinion Reporting in Canada and the United States.” Paper presented at the American Political Science Association Meetings, San Francisco, CA.

DeRosa, K. L., and J. C. Tedesco. 1999. “Surveying the Spin: Interpretation of the 1996 Presidential Polls.” In The Electronic Election, ed. L. L. Kaid and D. G. Bystrom. Mahwah, NJ: Lawrence Erlbaum.

Frankovic, K. A. 2000. “Election Polls: The Perils of Interpretation.” Media Studies Journal 14:104–09.

Gawiser, S. R., and G. E. Witt. 1994. A Journalist’s Guide to Public Opinion Polls. Westport, CT: Praeger.

10.

Gawiser, S. R., and G. E. Witt. 2002. “20 Questions a Journalist Should Ask about Poll Results.” Retrieved May 22 from http://www.ncpp.org/qajsa.htm.

11.

Hess, S. 1981. The Washington Reporters. Washington, D.C.: Brookings Institution.

12.

Hickman, H. 1991. “Public Polls and Election Participants.” In Polling and Presidential Election Coverage, ed. P. J. Lavrakas and J. K. Holley. Newbury Park, CA: Sage.

13.

Ladd, E. C. 1980. “Polling and the Press: The Clash of Institutional Imperatives.” Public Opinion Quarterly 44:574–84.

14.

Lavrakas, P. J., J. K. Holley, and P. V. Miller. 1991. “Public Reactions to Polling News during the 1988 Presidential Election Campaign.” In Polling and Presidential Election Coverage, ed. P. J. Lavrakas and J. K. Holley. Newbury Park, CA: Sage.

15.

Mayer, R. K. 2001. “What to Expect from Electoral Expectations.” Press/Politics 6:71–89.

16.

McBride, F. W. 1991. “Media Use of Preelection Polls.” In Polling and Presidential Election Coverage, ed. P. J. Lavrakas and J. K. Holley. Newbury Park, CA: Sage.

17.

Meyer, P. 1968. “Truth in Polling.” Columbia Journalism Review 7:20–23.

18.

Miller, M. M., and R. Hurd. 1982. “Conformity to AAPOR Standards in Newspaper Reporting of Public Opinion Polls.” Public Opinion Quarterly 46:243–49.

19.

Miller, P. V., D. M. Merkle, and P. Wang. 1991. “Journalism with Footnotes: Reporting the ‘Technical Details’ of Polls.” In Polling and Presidential Election Coverage, ed. P. J. Lavrakas and J. K. Holley. Newbury Park, CA: Sage.

20.

Mitofsky, W. J. 1995. “How Pollsters and Reporters Can Do a Better Job Informing the Public: A Challenge for Campaign ’96.” In Presidential Polls and the News Media, ed. P. J. Lavrakas, M. W. Traugott, and P. V. Miller. Boulder, CO: Westview.

21.

Norris, P. 2001. “Too Close to Call: Opinion Polls in Campaign 2000.” Press/Politics 6:3–8.

22.

Paletz, D. L., J. Y. Short, H. Baker, B. C. Campbell, R. J. Cooper, and R. M. Oeslander. 1980. “Polls in the Media: Content, Credibility, and Consequences.” Public Opinion Quarterly 44:495–513.

23.

Patterson, T. E. 1993. Out of Order. New York: Knopf.

24.

Rhee, J. W. 1996. “How the Polls Drive Campaign Coverage: The Gallup/CNN/USA Today Tracking Poll and USA Today’s Coverage of the 1992 Presidential Campaign.” Political Communication 13:213–29.

25.

Robinson, M. J., and M. A. Sheehan. 1983. Over the Wire and on TV: CBS and UPI in Campaign ’80. New York: Russell Sage Foundation.

26.

Salwen, M. B. 1985. “The Reporting of Public Opinion Polls during Presidential Years, 1968-1984.” Journalism Quarterly 62:272–77.

27.

Traugott, M. W., and M. Kang. 2000. “Public Attention to Polls in an Election Year.” In Election Polls, the News Media, and Democracy, ed. P. J. Lavrakas and M. W. Traugott. New York: Chatham House.

28.

Welch, R. L. 2002. “Polls, Polls, and More Polls: An Evaluation of How Public Opinion Polls Are Reported in Newspapers.” Press/Politics 7:102–14.