Previous research shows that in telephone surveys, 11–49% of respondents select a middle alternative when it is offered on dichotomous bipolar questions, although they do not volunteer a middle option when it is not mentioned. Furthermore, offering a middle option leads to different systematic response effects, including social desirability bias and satisficing effects, which are related to respondent characteristics. Research has shown that in telephone surveys, a question form with a branching middle response has a lower response validity compared to that of a non-branching middle response. However, compared to the non-branching form, the branching form could potentially motivate respondents to spend more time and effort in their responses in the absence of an interviewer. Therefore, validity and reliability differences between branching-question form responses and non-branching-question form responses in general population web surveys have become an area of interest for research. Using a randomized experiment, this study tests the impact of a branching middle alternative in a general population survey on the validity and reliability of responses. The branching middle alternative did not change the validity and reliability of responses, and compared to the alternative, it reduced satisficing behavior based on the proxies.
AlwinD. F.KrosnickJ. A. (1991). The reliability of survey attitude measurement. Sociological Methods & Research, 20(1), 139–181. https://doi.org/10.1177/0049124191020001005
2.
BendigA. W. (1953). The reliability of self-ratings as a function of the amount of verbal anchoring and of the number of categories on the scale. Journal of Applied Psychology, 37(1), 38–41. https://doi.org/10.1037/h0057911
3.
BishopG. F. (1987). Experiments with the middle response alternative in survey questions. Public Opinion Quarterly, 51(2), 220–232. https://doi.org/10.1086/269030
BohrnstedtG. W. (1983). Measurement, Chapter 3. In RossiP. H.WrightJ. D.AndersonA. B. (Eds.), Handbook of survey research (pp. 69–121): Academic Press.
ConverseJ. M.PresserS. (1986). Survey questions: Handcrafting the standardized questionnaire. Sage Publications, Inc.
8.
Federal Committee on Statistical Methodology. (2001). Measuring and reporting sources of error in surveys (statistical Policy Working paper 31). Washington, DC: Office of Management and Budget, Office of information and Regulatory Affairs, statistical Policy Office. Available at:https://nces.ed.gov/fcsm/pdf/spwp31.pdf (accessed 10 December 2022).
9.
GrovesR. M.FowlerF. J.CouperM. P.LepkowskiJ. M.SingerE.TourangeauR. (2009). Survey methodology (2nd ed.) Wiley Series in Survey Methodology.
10.
HuiC. H.TriandisH. C. (1989). Effects of Culture and response format on extreme response style. Journal of Cross-Cultural Psychology, 20(3), 296–309. https://doi.org/10.1177/0022022189203004
11.
IannacchioneV. G. (2011). The changing role of address-based sampling in survey research. Public Opinion Quarterly, 75(3), 556–575. https://doi.org/10.1093/poq/nfr017
12.
KaltonG.RobertsJ.HoltD. (1980). The effects of offering a middle response option with opinion questions. The Statistician, 29(1), 65–78. https://doi.org/10.2307/2987495
13.
KlingemannH. D. (1997). The left-right self-placement question in face to face and telephone surveys. In SarisW.KaaseM. (Eds.), Eurobarometer: Measurement Instruments for Opinions in Europe. Mannheim: Zentrum für Umfragen, Methoden und Analysen -ZUMA- (pp. 100–110).
14.
KlopferF.MaddenT. (1980). The middlemost choice on attitude items: Ambivalence, neutrality, or uncertainty?Personality and Social Psychology Bulletin, 6(1), 97–101. https://doi.org/10.1177/014616728061014
15.
KrosnickJ. A.AlwinD. F. (1987). An evaluation of a cognitive theory of response-order effects in survey measurement. Public Opinion Quarterly, 51(2), 201–219. https://doi.org/10.1086/269029
16.
KrosnickJ. A.BerentM. K. (1993). Comparisons of party Identification and Policy preferences: The impact of survey question format. American Journal of Political Science, 37(3), 941–964. https://doi.org/10.2307/2111580
17.
KrosnickJ. A.FabrigarL. R. (1997). Designing rating scales for effective measurement in surveys. In LybergL.BiemerP.CollinsM.De LeeuwE.DippoC.SchwarzN. (Eds.), Survey measurement and process quality. D. Trewin.
18.
KrosnickJ. A.SchumanH. (1988). Attitude intensity, importance, and certainty and Susceptibility to response effects. Journal of Personality and Social Psychology, 54(6), 940–952. https://doi.org/10.1037/0022-3514.54.6.940
19.
KulasJ. T.StachowskiA. A. (2009). Middle category endorsement in odd-numbered Likert response scales: Associated item characteristics, cognitive demands, and preferred meanings. Journal of Research in Personality, 43(3), 489–493. https://doi.org/10.1016/j.jrp.2008.12.005
LietzP. (2010). Research into questionnaire design: A summary of the literature. International Journal of Market Research, 52(2), 249–272. https://doi.org/10.2501/s147078530920120x
22.
LugtigP.Lensvelt-MuldersG. J. L. M.FrerichsR.GrevenA. (2011). Estimating nonresponse bias and mode effects in a mixed-mode survey. International Journal of Market Research, 53(5), 669–686. https://doi.org/10.2501/ijmr-53-5-669-686
23.
MalhotraN.KrosnickJ. A.ThomasR. K. (2009). Optimal design of branching questions to measure bipolar constructs. Public Opinion Quarterly, 73(2), 304–324. https://doi.org/10.1093/poq/nfp023
24.
NadlerJ. T.WestonR.VoylesE. C. (2015). Stuck in the middle: The Use and interpretation of Mid-Points in items on questionnaires. The Journal of General Psychology, 142(2), 71–89. https://doi.org/10.1080/00221309.2014.994590
25.
NarayanS.KrosnickJ. A. (1996). Education Moderates some response effects in attitude measurement. Public Opinion Quarterly, 60(1), 58–88. https://doi.org/10.1086/297739
O’MuircheartaighC. A.KrosnickJ. A.HelicA. (1999). Middle alternatives , acquiescence , and the quality of questionnaire data. AAPOR Presentation.
28.
RogersB.RyanV.Cibelli-HibbenK. (2022) “Why do you say that?” A case study in applying topic modeling to analyze open-ended survey probes. In: 2022 FCSM research & Policy Conference. Washington DC.
SchumanH.PresserS. (1981). Questions and answers in attitude surveys experiments on question form, Wording, and context. Academic Press, Inc.
31.
StatisticsNetherlands. (2022). About Blaise. Blaise Department. Heerlen, Netherlands. Available at:https://blaise.com/blaise/about-blaise(accessed 19 December 2022).
32.
SturgisP.RobertsC.SmithP. (2014). Middle alternatives revisited: How the neither/nor response acts as a way of saying “I don’t know”. Sociological Methods & Research, 43(1), 15–38. https://doi.org/10.1177/0049124112452527
33.
Suzer‐GurtekinZ. T.LepkowskiJ. M.CurtinR. T.ElkasabiM.LiuM.LavrakasP. J.TraugottM. W.KennedyC.HolbrookA. L. (2019). Randomized experiments for web-mail surveys conducted using address-based samples of the general population. In WestB. T. (Ed.), Experimental methods in survey research: Techniques that Combine random sampling with random Assignment (pp. 275–289): John Wiley and Sons.
34.
TourangeauR.RipsL.RasinskiK. (2000a). Selecting a response: Mapping judgments to survey answers. In The Psychology of survey response (pp. 230–254): Cambridge University Press.
35.
TourangeauR.RipsL. J.RasinskiK. (2000b). The Psychology of Survey Response. Cambridge University Press, pp.165–196.
WangR.KrosnickJ. A. (2020). Middle alternatives and measurement validity: A recommendation for survey researchers. International Journal of Social Research Methodology, 23(2), 169–184. https://doi.org/10.1080/13645579.2019.1645384
39.
WeijtersB.CabooterE.SchillewaertN. (2010). The effect of rating scale format on response styles: The number of response categories and response category labels. International Journal of Research in Marketing, 27(3), 236–247. https://doi.org/10.1016/j.ijresmar.2010.02.004
40.
WengL. (2004). Impact of the number of response categories and anchor labels on Coefficient Alpha and test-retest reliability. Educational and Psychological Measurement, 64(6), 956–972. https://doi.org/10.1177/0013164404268674
41.
WetzelhütterD. (2020). Scale-sensitive response behavior!? Consequences of offering versus omitting a ‘Don’t know’ option and/or a middle category. Survey Practice, 13(1), 47.
42.
ZhangC.ConradF. G. (2018). Intervening to reduce satisficing behaviors in web surveys: Evidence from two experiments on how it works. Social Science Computer Review, 36(1), 57–81. https://doi.org/10.1177/0894439316683923
