This article provides a review of recent research on the quality of survey data and serves as an introduction to this special issue of SMR. The article reviews standard statistical treatments of survey errors and discusses the importance of developing population models for measurement errors. A model is presented which nests measurement errors within other nonmeasurement survey errors—errors of coverage, sampling, and nonresponse. Promising new developments in research on survey measurement are reviewed and evaluated—applications of rational choice theories of response behavior, applications of cognitive theories of judgment and information processing, and experiments in the meaning of questions. The article concludes with a brief summary of the contributions included in this special issue of SMR.
Get full access to this article
View all access options for this article.
References
1.
Achen, C. H.1975. “Mass Political Attitudes and the Survey Response.”American Political Science Review69:1218-1231.
2.
Ajzen, I.
1990. “Attitude Theory and the Attitude-Behavior Relation.” Paper presented at conference on “Relevance of Attitude Measurement in Sociology.”Bad Homburg, Germany. July.
3.
Alwin, D. F.1973. “Making Inferences from Attitude-Behavior Correlations.”Sociometry36:253-278.
4.
Alwin, D. F.
1974. “Approaches to the Interpretation of Relationships in the Multitrait-Multimethod Matrix.” Pp. 79-105 in Sociological Methodology 1973-74, edited by H. L. Costner. San Francisco: Jossey-Bass.
5.
Alwin, D. F.1977. “Making Errors in Surveys: An Overview.”Sociological Methods and Research6:131-150.
6.
Alwin, D. F.1989. “Problems in the Estimation and Interpretation of the Reliability of Survey Data.”Quality and Quantity23:277-331.
7.
Alwin, D. F.
Forthcoming. “Information Transmission in the Survey Interview: Number of Response Categories and the Reliability of Attitude Measurement.”Sociological Methodology 1992.
8.
Alwin, D. F.
, and D. J. Jackson. 1979. “Measurement Models for Response Errors in Surveys: Issues and Applications.” Pp. 68-119 in Sociological Methodology 1980, edited by K. F. Schuessler. San Francisco: Jossey-Bass.
9.
Alwin, D. F.
, and J. A. Krosnick. 1985. “The Measurement of Values in Surveys: A Comparison of Ratings and Rankings.”Public Opinion Quarterly49:535-552.
10.
Alwin, D. F.
, and Thornton, A.1984. “Family Origins and the Schooling Process: Early vs. Late Influence of Parental Characteristics.”American Sociological Review49:784-802.
11.
American Psychological Association
. 1974. Standards for Educational and Psychological Tests. Washington, DC: Author.
12.
Anderson, R.
, J. Kasper, and M. R. Frankel. 1979. Total Survey Error: Applications to Improve Health Surveys. San Francisco: Jossey-Bass.
13.
Asher, H. B.1974. “Some Consequences of Measurement Error in Survey Data.”American Journal of Political Science28:468-485.
14.
Bachman, J. G.
, and P. M. O'Malley. 1981. “When Four Months Equal a Year: Inconsistencies in Student Reports of Drug Use.”Public Opinion Quarterly45:536-548.
15.
Belson, W.
1981. The Design and Understanding of Survey Questions. London: Gower.
16.
Bielby, W. T.
and R. M. Hauser. 1977. “Response Errors in Earnings Functions for Nonblack Males.”Sociological Methods & Research6:241-280.
17.
Bielby, W. T.
, R. M. Hauser, and D. L. Featherman. 1977a. “Response Errors of Nonblack Males in Models of the Stratification Process.”Journal of the American Statistical Association72:723-735.
18.
Bielby, W. T.
, R. M. Hauser, and D. L. Featherman. 1977b. “Response Errors of Black and Nonblack Males in Models of Status Inheritance and Mobility.”American Journal of Sociology82:1242-1288.
19.
Blalock, H. M., Jr.
1968. “The Measurement Problem: A Gap Between the Languages of Theory and Research.” Pp. 5-27 in Methodology in Social Research, edited by H. M. Blalock, Jr., and A. B. Blalock. New York: McGraw-Hill.
20.
Bohrnstedt, G. W.
1983. “Measurement.” Pp. 70-121 in Handbook of Survey Research, edited by P. H. Rossi, J. D. Wright, and A. B. Anderson. New York: Academic Press.
21.
Borus, M. E.
and G. Nestle. 1973. “Response Bias in Reports of Father's Education and Socioeconomic Status.”Journal of the American Statistical Association68:816-820.
22.
Bradburn, N.
and C. Danis. 1984. “Potential Contributions of Cognitive Research to Survey Questionnaire Design.” Pp. 101-129 in Cognitive Aspects of Survey Methodology: Building a Bridge Between Disciplines, edited by T. B. Jabine, M. L. Straf, J. M. Tanur, and R. Tourangeau. Washington, DC: National Academy Press.
23.
Cannell, C. F.
, F. J. Fowler, G. Kalton, L. Oksenberg, and K. Bischoping. 1989. Bulletin of the International Statistical Institute. Proceedings of the 47th Session. Paris, France.
24.
Cannell, C. F.
, P. V. Miller, and L. Oksenberg. 1981. “Research on Interviewing Techniques.” Pp. 389-437 in Sociological Methodology 1981, edited by S. Lienhardt. San Francisco: Jossey-Bass.
25.
Clark, H. H.
1985. “Language Use and Language Users.” Pp. 179-231 in The Handbook of Social Psychology. Vol. 2. 3rd ed., edited by G. Lindzey and E. Aronson. New York: Random House.
26.
Converse, P. E.
1964. “The Nature of Belief Systems in the Mass Public.” Pp. 206-261 in Ideology and Discontent, edited by D. E. Apter. New York: Free Press.
27.
Converse, P. E.
1970. “Attitudes and Non-Attitudes: Continuation of a Dialogue.” Pp. 168-189 in The Quantitative Analysis of Social Problems, edited by E. R. Tufte. Reading, MA: Addison-Wesley.
28.
Converse, P. E.1974. “Comment: The Status of Non-Attitudes.”American Political Science Review68:650-660.
29.
Converse, P. E.
and G. B. Markus. 1979. “Plus Ça Change...: The New CPS Election Study Panel.”American Political Science Review73:32-49.
30.
Crowne, D. P.
and D. Marlowe. 1960. “A New Scale of Social Desirability Independent of Psychopathology.”Journal of Consulting Psychology24:349-354.
31.
Duncan, G. J.
and N. A. Mathiowetz. 1985. A Validation Study of Economic Data. Survey Research Center, Institute for Social Research, University of Michigan, Ann Arbor.
32.
Erikson, R. S.1979. “The SRC Panel Data and Mass Political Attitudes.”British Journal of Political Science9:89-114.
33.
Esser, H.1986. “Können Befragte lügen? Zum Konzept des “wahren” Wertes im Rahmen der handlungstheoretischen Interpretation des Befragtenverhaltens.”Kölner Zeitschrift für Soziologie und Sozialpsychologie38:314-336.
34.
Esser, H.
1990. “Response Set: Habit, Frame or Rational Choice?” Paper presented at conference on Relevance of Attitude Measurement in Sociology. Bad Homburg, Germany. July.
35.
Groves, R. M.1988. “Research on Survey Data Quality.”Public Opinion Quarterly51:S156-S172.
36.
Groves, R. M.
1989. Survey Errors and Survey Costs. New York: Wiley.
37.
Groves, R. M.
and R. L. Kahn. 1979. Surveys by Telephone. New York: Academic Press.
38.
Hauser, R. M.
, S. L. Tsai, and W. M. Sewell. 1983. “A Model of Stratification with Response Error in Social and Psychological Variables.”Sociology of Education56:20-46.
39.
Heise, D. R.
and G. W. Bohrnstedt. 1970. “Validity, Invalidity and Reliability.” Pp. 104-129 in Sociological Methodology 1970, edited by E. F. Borgatta and G. W. Bohrnstedt. San Francisco: Jossey-Bass.
40.
Hippler, H. J.
, N. Schwarz, and S. Sudman, eds. 1987. Social Information Processing and Survey Methodology. New York: Springer-Verlag.
41.
Jabine, T. B.
, M. L. Straf, J. M. Tanur, and R. Tourangeau. 1984. Cognitive Aspects of Survey Methodology: Building a Bridge Between Disciplines. Report of the Advanced Research Seminar on Cognitive Aspects of Survey Methodology. Washington, DC: National Academy Press.
42.
Jagodzinski, W.
, and S. M. Kühnel. 1987. “Estimation of Reliability and Stability in Single-Indicator Multiple Wave Models.”Sociological Methods & Research15:219-259.
43.
Kalton, G.
and Schuman, H.1982. “The Effect of the Question on Survey Responses: A Review.”Journal of the Royal Statistical Association145:42-73.
44.
Krosnick, J. A.
and D. F. Alwin. 1987. “An Evaluation of a Cognitive Theory of Response Order Effects in Survey Measurement.”Public Opinion Quarterly51:201-219.
45.
Krosnick, J. A.
and D. F. Alwin. 1988. “A Test of the Form-Resistant Correlation Hypothesis: Ratings, Rankings and the Measurement of Values.”Public Opinion Quarterly52:526-538.
46.
Krosnick, J. A.
and D. F. Alwin. 1989. “Response Strategies for Coping with the Cognitive Demands of Attitude Measures in Surveys.” Unpublished paper. Institute for Social Research. University of Michigan, Ann Arbor.
47.
Lord, F. M.
and M. L. Novick. 1968. Statistical Theories of Mental Test Scores. Reading, MA: Addison-Wesley.
48.
Marquis, K. H.
1978. Record Check Validity of Survey Responses: A Reassessment of Bias in Reports of Hospitalization. Santa Monica, CA: RAND.
49.
Marquis, K. H.
1984. “Record Checks for Sample Surveys.” Pp. 130-147 in Cognitive Aspects of Survey Methodology: Building a Bridge Between Disciplines, edited by T. B. Jabine, M. L. Straf, J. M. Tanur, and R. Tourangeau. Washington, DC: National Academy Press.
50.
Marquis, K. H.
, N. Duan, M. S. Marquis, and J. M. Polich. 1981. Response Errors in Sensitive Topic Surveys: Estimates, Effects and Correction Options. Santa Monica, CA: RAND.
51.
Marquis, M. S.
and K. H. Marquis. 1977. Survey Measurement Design and Evaluation Using Reliability Theory. Santa Monica, CA: RAND.
52.
McClendon, M. J.1991. “Acquiescence and Recency Response-Order Effects in Interview Surveys.”Sociological Methods & Research20:60-103.
53.
McClendon, M. J.
and D. F. Alwin. 1990. “No Opinion Filters and Attitude Reliability.” Paper presented at the 1990 meetings of the American Association for Public Opinion Research. Lancaster, PA. May.
54.
Miller, P. V.
, and R. M. Groves. 1985. “Matching Survey Responses to Official Records: An Exploration of Validity in Victimization Reporting.”Public Opinion Quarterly49:366-380.
55.
Moser, C. A.
and G. Kalton. 1972. Survey Methods in Social Investigation. New York: Basic Books.
56.
Schuman, H.
and G. Kalton. 1985. “Survey methods.” Pp. 635-697 in The Handbook of Social Psychology. 3rd ed., edited by G. Lindzey and E. Aronson. New York: Random House.
57.
Schuman, H.
and S. Presser. 1981. Questions and Answers in Attitude Surveys. New York: Academic Press.
58.
Siegel, P. M.
and R. W. Hodge. 1968. “A Causal Approach to the Study of Measurement Error.” Pp. 28-59 in Methodology in Social Research, edited by H. M. Blalock, Jr., and A. B. Blalock. New York: McGraw-Hill.
59.
Simon, H. A.
1957. Models of Man. New York: Wiley.
60.
Simon, H. A.
and A. C. Stedry. 1968. “Psychology and Economics.” Pp. 269-314 in The Handbook of Social Psychology. Vol. 5. 2nd ed., edited by G. Lindzey and E. Aronson. Reading, MA: Addison-Wesley.
61.
Strack, F.
and L. L. Martin. 1987. “Thinking, Judging, and Communicating: A Process Account of Context Effects in Attitude Surveys.” Pp. 123-148 in Social Information Processing and Survey Methodology, edited by H. J. Hippler, N. Schwarz, and S. Sudman. New York: Springer-Verlag.
62.
Tourangeau, R.
1984. “Cognitive Sciences and Survey Methods.” Pp. 73-100 in Cognitive Aspects of Survey Methodology: Building a Bridge Between Disciplines, edited by T. B. Jabine, M. L. Straf, J. M. Tanur, and R. Tourangeau. Washington, DC: National Academy Press.
63.
Tourangeau, R.
1987. “Attitude Measurement: A Cognitive Perspective.” Pp. 149-162 in Social Information Processing and Survey Methodology, edited by H. J. Hippler, N. Schwarz, and S. Sudman. New York: Springer-Verlag.
64.
Tourangeau, R.
and K. A. Rasinski. 1988. “Cognitive Processes Underlying Context Effects in Attitude Measurement.”Psychological Bulletin103:299-314.
65.
Warner, S. L.1965. “Randomized Response: A Survey Technique for Eliminating Evasive Answer Bias.”Journal of the American Statistical Association60:63-69.
66.
Warner, S. L.1971. “The Linear Randomized Response Model.”Journal of the American Statistical Association66:884-888.
67.
Weaver, C. N.
, and C. L. Swanson. 1974. “Validity of Reports of Date of Birth, Salary and Seniority.”Public Opinion Quarterly38:69-80.
