This article reports the effect that the ranges presented in answer categories for survey questions can have on respondent answers. Response categories were manipulated in a split-ballot survey conducted in both telephone and mail modes. These categories, presented in the separate ballots, overlapped in one category; the other categories were unique to each ballot. The experiment was conducted on four questions: two frequent and mundane and two rare and salient. It was found that the response categories significantly affected the response for frequent and mundane questions. One question demonstrated a significant difference in response between the mail and telephone modes. For this question, a response scale with a limited number of socially desirable alternatives resulted in a social desirability effect in the telephone mode. Alternatively, the telephone mode demonstrated an extremeness effect when the response scale comprised a greater number of socially desirable alternatives.
Get full access to this article
View all access options for this article.
References
1.
Aquilino, William S.1994. “Interview Mode Effects in Surveys of Drug and Alcohol Use: A Field Experiment.”Public Opinion Quarterly58:210-240.
2.
Aquilino, William S.
and Leonard A. Lo Sciuto. 1990. “Effects of Interview Mode on Self-Reported Drug Use.”Public Opinion Quarterly54:362-395.
3.
Biemer, Paul P.
1988. “Measuring Data Quality.” Pp. 273-282 in Telephone Survey Methodology, edited by Robert Groves, Paul Biemer, Lars Lyberg, James Massey, William Nicholls, and Joseph Waksberg. New York: John Wiley.
4.
Bishop, George F.
, Hans-J. Hippler, Norbert Schwarz, and Fritz Strack. 1988. “A Comparison of Response Effects in Self-Administered and Telephone Surveys.” Pp. 321-340 in Telephone Survey Methodology, edited by Robert Groves, Paul Biemer, Lars Lyberg, James Massey, William Nicholls, and Joseph Waksberg. New York: John Wiley.
5.
Boudenhausen, Galen V.
and Wyer, Robert S., Jr. 1987. “Social Cognition and Social Reality: Information Acquisition and Use in the Laboratory and the Real World.” Pp. 6-41 in Social Information Processing and Survey Methodology, edited by Hans-J. Hippler, Norbert Schwarz, and Seymour Sudman. New York: Springer-Verlag.
6.
Colombotos, John
. 1969. “Personal Versus Telephone Interviews: Effect on Responses.”Public Health Reports84:773-782.
7.
de Leeuw, Edith
. 1992. Data Quality in Mail, Telephone, and Face to Face Surveys. Amsterdam, the Netherlands: TT Publikaties.
8.
de Leeuw, Edith
and Johannes van der Zouwen. 1988. “Data Quality in Telephone and Face to Face Surveys: A Comparative Meta-Analysis.” Pp. 283-299 in Telephone Survey Methodology, edited by Robert Groves, Paul Biemer, Lars Lyberg, James Massey, William Nicholls, and Joseph Waksberg. New York: John Wiley.
9.
Dillman, Don A.
1978. Mail and Telephone Surveys: The Total Design Method. New York: John Wiley.
10.
Dillman, Don A.1991. “The Design and Administration of Mail Surveys.”Annual Review of Sociology17:225-249.
11.
Dillman, Don A.
, Tommy L. Brown, John E. Carlson, Edwin H. Carpenter, Frederick O. Lorenz, Robert Mason, John Saltiel, and Roberta L. Sangster. 1995. “Effects of Category Order on Answers in Mail and Telephone Surveys.”Rural Sociology60:674-687.
12.
Dillman, Don A.
, Roberta L. Sangster, John Tarnai, and Todd H. Rockwood. 1996. “Understanding Differences in People's Answers to Telephone and Mail Surveys.” Pp. 45-61 in Advances in Survey Research: New Directions for Program Evaluation, No. 70, edited by Marc T. Braverman. San Francisco: Jossey-Bass.
13.
Dillman, Don A.
and John Tarnai. 1991. “Mode Effects of Cognitively Designed Recall Questions: A Comparison of Answers to Telephone and Mail Surveys.” Pp. 73-93 in Measurement Error in Surveys, edited by Paul P. Biemer, Robert M. Groves, Lars E. Lyberg, Nancy A. Mathiowetz, and Seymour Sudman. New York: John Wiley.
14.
Groves, Robert M.
1989. Survey Errors and Survey Costs. New York: John Wiley.
15.
Groves, Robert
and Robert L. Kahn. 1979. Surveys by Telephone: A National Comparison With Personal Interviews. New York: Academic Press.
16.
Hippler, Hans-J.
and Norbert Schwarz. 1987. “Response Effects in Surveys.” Pp. 102-121 in Social Information Processing and Survey Methodology, edited by Hans-J. Hippler, Norbert Schwarz, and Seymour Sudman. New York: Springer-Verlag.
17.
Hippler, Hans-J.
and Norbert Schwarz. 1992. “The Impact of Administration Modes on Response Effects in Surveys.” Paper presented at the 47th annual meeting of the American Association for Public Opinion Research, May, St. Petersburg, FL.
18.
Hippler, Hans-J.
, Norbert Schwarz, and Seymour Sudman, eds. 1987. Social Information Processing and Survey Methodology. New York: Springer-Verlag.
19.
Hochstim, Joseph R.1967. “A Critical Comparison of Three Strategies of Collecting Data from Households.”Journal of the American Statistical Society62:976-989.
20.
Krosnick, Jon A.
1992. “The Impact of Cognitive Sophistication and Attitude Importance on Response Order and Question Order Effects.” Pp. 203-218 in Context Effects in Social and Psychological Research, edited by Norbert Schwarz and Seymour Sudman. New York: Springer-Verlag.
21.
Krosnick, Jon A.
and Duane F. Alwin. 1987. “An Evaluation of a Cognitive Theory of Response-Order Effects in Survey Measurement.”Public Opinion Quarterly51:201-219.
22.
Locander, William
, Seymour Sudman, and Norman M. Bradburn. 1976. “An Investigation of Interview Method, Threat and Response Distortion.”Journal of the American Statistical Association71:269-275.
23.
Mingay, David J.
and Michael T. Greenwell. 1989. “Memory Bias and Response-Order Effects.”Journal of Official Statistics5:253-263.
24.
Schwarz, Norbert
, Herbert Bless, Gerd Bohner, Uwe Harlacher, and Margit Kellenbenz. 1991. “Response Scales as Frames of Reference: The Impact of Frequency Range on Diagnostic Judgements.”Applied Cognitive Psychology5:37-49.
25.
Schwarz, Norbert
and Hans-J. Hippler. 1987. “What Response Scales May Tell Your Respondents: Informative Functions of Response Alternatives.” Pp. 163-177 in Social Information Processing and Survey Methodology, edited by Hans-J. Hippler, Norbert Schwarz, and Seymour Sudman. New York: Springer-Verlag.
26.
Schwarz, Norbert
, Hans-J. Hippler, B. Deutsch, and Fritz Strack. 1985. “Response Scales: Effects of Category Range on Reported Behavior and Comparative Judgements.”Public Opinion Quarterly49:388-395.
27.
Schwarz, Norbert
, Hans-J. Hippler, and Elizabeth Noelle-Neumann. 1992. “A Cognitive Model of Response-Order Effects in Survey Measurement.” Pp. 187-201 in Context Effects in Social and Psychological Research, edited by Norbert Schwarz and Seymour Sudman. New York: Springer-Verlag.
28.
Schwarz, Norbert
, Fritz Strack, Hans-J. Hippler, and George F. Bishop. 1991. “The Impact of Administration of Mode on Response Effects in Survey Research.”Applied Cognitive Psychology5:193-212.
29.
Schwarz, Norbert
, Fritz Strack, Gesine Muller, and Brigitte Chassein. 1988. “The Range of Response Alternatives May Determine the Meaning of the Question: Further Evidence on Informative Functions of Response Alternatives.”Social Cognition6:107-117.
30.
Strack, Fritz
, Ralph Erber, and Robert A. Wickland. 1982. “Effects of Salience and Time Pressure on Ratings or Social Causality.”Journal of Experimental Psychology18:581-594.
31.
Tarnai, John
and Don A. Dillman. 1992. “Questionnaire Context as a Source of Response Differences in Mail and Telephone Surveys.” Pp. 116-129 in Context Effects in Social and Psychological Research, edited by Norbert Schwarz and Seymour Sudman. New York: Springer-Verlag.
32.
Tourangeau, Roger
and Kenneth A. Rasinski. 1988. “Cognitive Processes Underlying Context Effects in Attitude Measurement.”Psychological Bulletin103:299-314.
