This study examined the effects of questionnaire item grouping (thematic versus random) and placement of the “don't know” option (before or after the rating scale) on the frequency of nonattitude responses (checking don't know and omission) in rating attitude statements. Response consistency was also evaluated by item grouping. Findings supported the hypothesis that knowledge of an item's theme and of other items related to that theme encouraged attitude responses by reducing the frequency of don't know responses. The authors found no evidence that positioning the don't know option at the end of the rating scale reduced nonattitude responses. Furthermore, thematic grouping enhanced internal consistency reliability for four of six subscales and the total scale.
Get full access to this article
View all access options for this article.
References
1.
Alwin, D. F., and J. A. Krosnick. 1991. The reliability of survey attitude measurement: The influence of question and respondent attributes. Sociological Methods & Research20 (1): 139–181.
2.
American Marketing Association. 1937. The technique of marketing research. New York: McGraw-Hill.
3.
Budd, R. J.1987. Response bias and the theory of reasoned action. Social Cognition5 (2): 95–107.
4.
Converse, J. M., and S. Presser. 1986. Survey questions. Newbury Park, CA:Sage.
5.
Fazio, R. H., T. M. Lenn, and E. A. Effrein. 1984. Spontaneous attitude formation. Social Cognition2:217–234.
6.
Feick, L. F.1989. Latent class analysis of survey questions that include “don't know” responses. Public Opinion Quarterly53:525–547.
7.
Gilljam, M., and D. Granberg. 1993. Should we take “don't know” for an answer?Public Opinion Quarterly57:348–357.
8.
Hamilton, J. C., and T. R. Shuminsky. 1990. Self-awareness mediates the relationship between serial position and item reliability. Journal of Personality and Social Psychology59:1301–1307.
9.
Harris, R. J.1994. ANOVA: An analysis of variance primer. Itasca, IL:F. E. Peacock.
10.
Hoogstraten, J.1979. Pretesting as determinant of attitude change in evaluation research. Applied Psychological Measurement3:25–30.
11.
Howard, G. S., and P. R. Dailey. 1979. Response-shift bias: A source of contamination of self-report measures. Journal of Applied Psychology64:144–150.
12.
Johanson, G. A., C. J. Gips, and C. E. Rich. 1993. “If you can't say something nice”:A variation on the social desirability response set. Evaluation Review17 (1):116–122.
13.
Knauper, B.1999. The impact of age and education on response order effects in attitude measurement. Public Opinion Quarterly63:347–370.
14.
Knowles, E. S., and B. Byers. 1996. Reliability shifts in measurement reactivity:Driven by content engagement or self-engagement?Journal of Personality and Social Psychology70 (5): 1080–1090.
15.
Krosnick, J. A., and D. F. Alwin. 1987. An evaluation of a cognitive theory of response-order effects in survey measurement. Public Opinion Quarterly51:201–219.
16.
Lehman, D. R., H. A. Krosnick, R. L. West, and F. Li. 1992. The focus of judgment effect: A question wording effect due to hypothesis confirmation bias. Personality and Social Psychology Bulletin18 (6):690–699.
17.
Mayer, D. P.1999. Measuring instructional practice:Can policymakers trust survey data?Educational Evaluation and Policy Analysis21 (1):29–45.
18.
Metzner, H., and F. Mann. 1953. Effects of grouping related questions in questionnaires. Public Opinion Quarterly17 (1):136–140.
19.
Miller, J. M., and J. A. Krosnick. 1998. The impact of candidate name order on election outcomes. Public Opinion Quarterly62:291–330.
20.
Narayan, S., and J. A. Krosnick. 1996. Education moderates some response effects in attitude measurement. Public Opinion Quarterly60:58–88.
21.
Noelle-Neumann, E.1970. Wanted:Rules for wording structured questionnaires. Public Opinion Quarterly34:191–201.
22.
Poe, G. S., I. Seeman, J. McLauglin, E. Mehl, and M. Dietz. 1988. “Don't know” boxes in factual questions in a mail questionnaire:Effects on level and quality of response. Public Opinion Quarterly52:212–222.
23.
Presser, S., and H. Schuman. 1989. The management of a middle position in attitude surveys. In Survey research methods, edited by E. Singer and S. Presser, 108–123. Chicago:University of Chicago.
24.
Schroeder, S.1985. Towards a theory of how people answer questions. European Research13:82–90.
25.
Schuman, H., and S. Presser. 1996. Questions and answers in attitude surveys: Experiments on question form, wording, and context. Thousand Oaks, CA:Sage.
26.
Schwarz, N., and H. J. Hippler. 1995. Reverse context effects in mail surveys. Public Opinion Quarterly59:93–97.
27.
Sigelman, L.1981. Question-order effects on presidential popularity. Public Opinion Quarterly45:199–207.
28.
Smith, T. W.1981. Situational qualifications to generalized absolutes:An analysis of approval of hitting questions on the General Social Survey. Public Opinion Quarterly45:224–230.
29.
Steinberg, L.1994. Context and serial-order effects in personality measurement:Limits on the generality of measuring changes the measure. Journal of Personality and Social Psychology66:341–349.
30.
Sudman, S., and N. M. Bradburn. 1983. Asking questions. San Francisco:Jossey-Bass.
31.
Tenvergert, E., M. W. Gillespie, J. Kingma, and H. Klasen. 1992. Abortion attitudes, 1984–1987–1988: Effects of item order and dimensionality. Perceptual and Motor Skills74: 627–642.
32.
Todorov, A.2000. The accessibility and applicability of knowledge. Public Opinion Quarterly64 (4):429–451.
33.
Tourangeau, R., and K. A. Rasinski. 1986. Context effects in attitude surveys. Unpublished manuscript.
34.
Tourangeau, R., and K. A. Rasinski. 1988. Cognitive processes underlying context effects in attitude measurement. Psychological Bulletin103 (3):299–314.
