An important decision in online and mixed-mode questionnaire design is if and how to include a “do-not-know” (DK) option. Mandatory response is often a default option, but methodologists have advised against this. Several solutions for the DK category are suggested. These include (1) not explicitly offering a DK, but skipping questions is allowed, (2) explicitly offering a DK option with visual separation from the substantive responses, and (3) using the interactivity of the web to emulate interviewer probing after a DK answer. To test these solutions, experimental data were collected in a probability based online panel. Not offering DK, but allowing respondents to skip questions, followed by a polite probe when skips occurred, resulted in the lowest amount of missing information. To assess the effect of probing across different modes, a second experiment was carried out that compared explicitly and implicitly offering the DK option for web and telephone surveys.
Al BaghalT.LynnP. (2014). Using motivational statements in web instrument design to reduce item missing rates in a mixed-mode context. (Understanding Society Working Paper Series, 2014-02). Colchester, United Kingdom: Institute for Social and Economic Research, University of Essex.
2.
BehrD.KaczmirekL.BandillaW.BraunM. (2012). Asking probing questions in web surveys: Which factors have an impact on the quality of responses?Social Science Computer Review, 30, 487–498.
3.
BethlehemJ.BiffignandiS. (2012). Handbook of web surveys. New York, NY: John Wiley.
4.
BillietJ. B.McClendonM. J. (2000). Modeling acquiescence in measurement models for two balanced sets of items. Structural Equation Modeling, 7, 608–628.
5.
ConverseJ. M. (1976). Predicting don’t know in the polls. Public Opinion Quarterly, 40, 515–530.
6.
ConverseJ. M.PresserS. (1986). Survey questions: Handcrafting the standard questionnaire. Beverly Hills, CA: Sage.
7.
CouperM. P. (2008). Designing effective web surveys. Cambridge, MA: Cambridge University Press.
8.
DeRouvrayC.CouperM. P. (2002). Designing a strategy for reducing “no opinion” responses in web-based surveys. Social Science Computer Review, 20, 3–9.
9.
DillmanD. A.SmythJ. D.ChristianL. M. (2009). Implementation procedures. Internet, mail, and mixed mode surveys: the tailored design method. (3rd ed). Hoboken, NJ: John Wiley.
FrickerS.GalesicM.TourangeauR.YanT. (2005). An experimental comparison of web and telephone surveys. Public Opinion Quarterly, 69, 370–392.
12.
GrovesR. M. (1989). Survey errors and survey costs. New York, NY: John Wiley.
13.
GrovesR. M.DillmanD. A.EltingeJ. L.LittleR. J. A. (2002). Survey nonresponse. New York, NY: John Wiley.
14.
GrovesR. M.Mc GonagleK. A. (2001) A theory-guided interviewer training protocol regarding survey participation. Journal of Official Statistics, 17, 249–265.
15.
HeerweghD.LoosveldtG. (2008). Face-to-face versus web surveying in a high-internet-coverage population. Public Opinion Quarterly, 72, 836–846.
16.
HoxJ. J. (2010). Multilevel analysis: Techniques and applications. New York, NY: Routledge.
17.
JaeckleA.LynnP.BurtonJ. (2013). Going online with a face-to-face household panel: Initial results from an experiment on the Understanding Society Innovation Panel. (Understanding Society Working Paper Series, 2013-03). Colchester, United Kingdom: Institute for Social and Economic Research, University of Essex.
18.
KrosnickJ. A.HolbrookA. L.BerentM. K.CarsonR. T.HanemannW. M.KoppR. J.…ConawayM. (2002). The impact of ‘don’t know’ response options on data quality: Nonattitude reduction or an invitation to satisfice?Public Opinion Quarterly, 66, 371–403.
19.
de LeeuwE. D. (1992). Data quality in mail, telephone, and face-to-face surveys. Amsterdam, The Netherlands: TT-publikaties. Retrieved fromhttp://edithl.home.xs4all.nl/pubs/disseddl.pdf
de LeeuwE. D.HoxJ. J. (2011). Internet surveys as part of a mixed-mode design. In DasM.EsterP.KaczmirekL. (Eds.), Social and behavioral research and the Internet(pp. 45–76.) EAM methodology series. New York, NY: Routledge.
22.
de LeeuwE. D.HoxJ. J. (2014). Survey mode and mode effects. In EngelU.JannB.LynnP.ScherpenzeelA.SturgisP. (Eds.) Improving survey methods (pp. 22–34). EAM methodology series. New York, NY: Routledge.
23.
de LeeuwE.HoxJ.ScherpenzeelA. (2010). Emulating interviewers in an online survey: Experimental manipulation over the phone and on the web. Paper presented at AAPOR 2010, Chicago. Appeared in JSM-proceedings, Survey Methods Section of the American Statistical Association. Retrieved fromhttp://www.amstat.org/sections/SRMS/Proceedings/
OudejansM.ChristianL. M. (2011). Using interactive features to motivate and probe responses to open-ended questions. In DasM.EsterP.KaczmirekL. (Eds.), Social and behavioral research and the Internet (pp. 215–244). New York, NY: Routledge.
26.
RevillaM. A.SarisW. E. (2013). A comparison of the quality of questions in a face-to-face and a web survey. International Journal of Public Opinion Research, 25, 242–253.
27.
ScherpenzeelA. C.DasM. (2011). True longitudinal and probability-based Internet panels: Evidence from the Netherlands. In DasM.EsterP.KaczmirekL. (Eds), Social and behavioral research and the Internet (pp. 77–104) New York, NY: Routledge.
28.
SmythJ. D.DillmanD. A.ChristianL. M.McBrideM. (2009). Open-ended questions in web surveys. Can increasing the size of answer boxes and providing extra verbal instructions improve response quality?Public Opinion Quarterly, 73, 325–327.
29.
SteegersCh.DijstelbloemH.BromF. W. A. (2008). Meer dan status alleen. Burgerperspectieven op embryo-onderzoek. [In Dutch: Assessment and points of view of citizens on technology and embryo research]. The Hague, Netherlands: Rathenau-Instituut, TA rapport 0801. Retrieved fromhttp://www.rathenau.nl/uploads/tx_tferathenau/Meer_dan_status_alleen_april_2008.pdf
30.
SudmanS.BradburnN.M. (1982). Asking questions. San Francisco, CA: Jossey-Bass.
31.
TourangeauR.CouperM. P.ConradF. (2004). Spacing, position, and order. Interpretative heuristics for visual features of survey questions. Public Opinion Quarterly, 68, 368–393.
32.
TourangeauR.ConradF. G.CouperM. P. (2013). The science of web surveys. Oxford, England: Oxford University Press.
33.
WineJ. S.CominoleM. B.HeuerR. E.RiccobonoJ. A. (2006). Challenges of designing and implementing multimode instruments. Presented at the Second International Conference on Telephone Survey Methodology, Miami, FL, January 2006. Retrieved fromhttps://www.rti.org/pubs/TSM2006_Wine_paper.pdf
