The risk of multitasking is high in online surveys. However, knowledge on the effects of multitasking on answer quality is sparse and based on suboptimal approaches. Research reports inconclusive results concerning the consequences of multitasking on task performance. However, studies suggest that especially sequential-multitasking activities are expected to be critical. Therefore, this study focusses on sequential-on-device-multitasking activities (SODM) and its consequences for data quality. Based on probability-based data, this study aims to reveal the prevalence of SODM based on the javascript function OnBlur, to reflect the its determinants and to examine the consequences for data quality. Results show that SODM was detected for 25% of all respondents and that respondent attributes and the device used to answer the survey are related to SODM. Moreover, it becomes apparent that SODM is significantly correlated to data quality measures. Therefore, I propose SODM behavior as a new instrument for researching suboptimal response behavior.
AAPOR. 2019. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys, Vol. 9. Virginia: The American Association for Public Opinion Research.
2.
AdlerRachel F.Benbunan-FichRaquel. 2012. “Juggling on a High Wire: Multitasking Effects on Performance.” International Journal of Human-Computer Studies70(2):156‐68. doi:https://doi.org/10.1016/j.ijhcs.2011.10.003
3.
AizpuruaEvaHeidenEin O.ParkKi I.WittrockJillLoschMary E.. 2018. “Investigating Respondent Multitasking and Distraction Using Self-Reports and Interviewers’ Observations in a Dual-Frame Telephone Survey.” Survey Insights: Methods from the Field1‐10. doi:10.13094/SMIF-2018-00006
4.
AlvarezR. MichaelAtkesonLonna R.LevinInesLiYimeng. 2019. “Paying Attention to Inattentive Survey Respondents.” Political Analysis27(2):145‐62. doi:10.1017/pan.2018.57
5.
AnsolabehereStephenSchaffnerBrian F.. 2015. “Distractions: The Incidence and Consequences of Interruptions for Survey Respondents.” Journal of Survey Statistics and Methodology3(2):216‐39. doi:10.1093/jssam/smv003
6.
AntounChristopherCouperMick P.ConradFrederick G.. 2017. “Effects of Mobile Versus Pc Web on Survey Response Quality: A Crossover Experiment in a Probability Web Panel.” Public Opinion Quarterly81(S1):280‐306. doi:10.1093/poq/nfw088
7.
BaierTobiasFuchsMarek. 2020. “Page Switching in Mixed-Device Web Surveys: Prevalence and Data Quality.” Survey Insights: Methods from the Field Special issue: ‘Advancements in Online and Mobile Survey Methods1‐7. doi: 10.13094/SMIF-2020-00021
8.
BaumgartnerSusanne E.WeedaWouter D.van der HeijdenLisa L.HuizingaMariëtte. 2014. “The Relationship between Media Multitasking and Executive Function in Early Adolescents.” The Journal of Early Adolescence34(8):1120‐44.
9.
BoschOriolWeberWiebkeRevillaMelanie. 2018. Improving Web Panel Respondent Behaviour: The Effect of Encouragement Messages Throughout the Course of the Survey. London: SERISS.
10.
BowlingNathan A.HuangJason L.BrowerCheyna K.BraggCaleb B.. 2021. “The Quick and the Careless: The Construct Validity of Page Time as a Measure of Insufficient Effort Responding to Surveys.” Organizational Research Methods10‐30. doi:10.1177/10944281211056520
11.
CameronA. ColinTrivediPravin K.. 2013. Regression Analysis of Count Data, Vol. 53. Cambridge: Cambridge university press.
12.
CernatAlexandruRevillaMelanie. 2020. “Moving from Face-to-Face to a Web Panel: Impacts on Measurement Quality.” Journal of Survey Statistics and Methodology9(4):745‐63. doi:10.1093/jssam/smaa007
13.
de LeeuwEdith D.HoxJoop J.BoevéAnja. 2015. “Handling Do-Not-Know Answers: Exploring New Approaches in Online and Mixed-Mode Surveys.” Social Science Computer Review34(1):116‐32. doi:10.1177/0894439315573744
14.
DécieuxJean PhilippeHeinenAndreasWillemsHelmut. 2019. “Social Media and Its Role in Friendship-Driven Interactions among Young People: A Mixed Methods Study.” YOUNG27(1):18‐31. doi:10.1177/1103308818755516
15.
DécieuxJean PhilippeMergenerAlexandraSischkaPhilippNeufangKristina. 2015. “Implementation of the Forced Answering Option within Online Surveys: Do Higher Item Response Rates Come at the Expense of Participation and Answer Quality?” Psihologija48(4):311‐26. doi:10.2298/PSI1504311D
16.
DiedenhofenBirkMuschJochen. 2017. “Pagefocus: Using Paradata to Detect and Prevent Cheating on Online Achievement Tests.” Behavior Research Methods49(4):1444‐59. doi:10.3758/s13428-016-0800-7
17.
DillmanDon A.2017. “The Promise and Challenge of Pushing Respondents to the Web in Mixed-Mode Surveys.” Survey Methodology43(1):3‐30.
18.
ErlinghagenMarcelEtteAndreasGuedes AuditorJean. 2020. “The Demographics of German Emigration: Historical Developments, Geographical Contexts and Social Structures of Emigrants, Remigrants, Internal Migrants and Non-Migrants.” in Consequences of International Migration across the Life Course: Global Lives of German Migrants, edited by ErlinghagenM.EtteA.SchneiderN.WitteN.. Wiesbaden: Springer.
19.
ErlinghagenMarcelSchneiderNorbert F.. 2020 “Wave 1 of the German Emigration and Remigration Panelstudy (Gerps) 2018.” in German Emigration and Remigration Panelstudy (GERPS), edited by GESIS-Data-Archive. Collogne: GESIS.
20.
EtteAndreasDécieuxJean PhilippeErlinghagenMarcelGenoniAndreasGuedes AuditorJeanKnrischFrederikKühneSimonSandMatthiasMörchenLuisaNilsWitte. 2020. “German Emigration and Remigration Panel Study: Methodology and Data Manual of the Baseline Survey (Wave 1).” in BiB Daten- und Methodenberichte, Vol. 1/2020. Wiesbaden: Federal Institute for Population Research.
21.
EtteAndreasDécieuxJean PhilippeErlinghagenMarcelGuedes AuditorJeanSchneiderNorbert F.WitteNils. 2021 “Surveying across Borders: The Experiences of the German Emigration and Remigration Panel Study.” Pp. 21‐39 in The Global Lives of German Migrants. Consequences of International Migration across the Life Course, Imiscoe Research Series, edited by ErlinghagenM.EtteA.SchneiderN.WitteN.. Cham: Springer.
22.
GenoniAndreasDécieuxJean PhilippeEtteAndreasWitteNils. 2021 “Setting up Probability-Based Online Panels of Migrants with a Push-to-Web Approach: Lessons Learned from the German Emigration and Remigration Panel Study (Gerps).” Pp. 289‐307 in The Global Lives of German Migrants. Consequences of International Migration across the Life Course, Imiscoe Research Series, edited by ErlinghagenM.EtteA.SchneiderN.WitteN.. Cham: Springer.
23.
GibsonAnthonyBowlingNathan. 2020. “The Effects of Questionnaire Length and Behavioral Consequences on Careless Responding.” European Journal of Psychological Assessment36(2):410‐20. doi:10.1027/1015-5759/a000526
24.
GoldammerPhilippeAnnenHubertStöckliPeter L.JonasKlaus. 2020. “Careless Responding in Questionnaire Measures: Detection, Impact, and Remedies.” The Leadership Quarterly1‐30. doi:10.1177/1094428121105652
25.
GummerTobiasKunzTanja. 2019. “Relying on External Information Sources When Answering Knowledge Questions in Web Surveys.” Sociological Methods & Research1‐21. doi:10.1177/0049124119882470
26.
GummerTobiasKunzTanjaRettigTobiasHöhneJan K.. 2021. “Looking up Answers to Political Knowledge Questions: The Use of Different Instructions and Measures for Respondent Behavio.” Paper presented at the General Online Research 2021, Berlin.
27.
GummerTobiasRoßmannJossSilberHenning. 2021. “Using Instructed Response Items as Attention Checks in Web Surveys: Properties and Implementation.” Sociological Methods & Research50(1):238‐64. doi:10.1177/0049124118769083
28.
HenningerMirkaPlieningerHansjörg. 2021. “Different Styles, Different Times: How Response Times Can Inform Our Knowledge About the Response Process in Rating Scale Measurement.” Assessment28(5):1301‐19. doi:10.1177/1073191119900003
29.
HöhneJan K.SchlosserStephan. 2017. “Investigating the Adequacy of Response Time Outlier Definitions in Computer-Based Web Surveys Using Paradata Surveyfocus.” Social Science Computer Review36(3):369‐78. doi:10.1177/0894439317710450
30.
HöhneJan K.SchlosserStephanCouperMick P.BlomAnnelies G.. 2020. “Switching Away: Exploring on-Device Media Multitasking in Web Surveys.” Computers in Human Behavior 111: 106417. doi:10.1016/j.chb.2020.106417
31.
HwangYooriKimHyoung JeeJeongSe-Hoon. 2014. “Why Do Media Users Multitask?: Motives for General, Medium-Specific, and Content-Specific Types of Multitasking.” Computers in Human Behavior36:542‐48. doi: 10.1016/j.chb.2014.04.040
32.
JanssenTracyCartonJohn S.. 1999. “The Effects of Locus of Control and Task Difficulty on Procrastination.” The Journal of Genetic Psychology160(4):436‐42. doi:10.1080/00221329909595557
33.
KovalevaAnastassiya. 2012. “The Ie-4: Construction and Validation of a Short Scale for the Assessment of Locus of Control.” Vol. 9.GESIS-Schriftenreihe. Mannheim: GESIS. doi:10.21241/ssoar.37119
34.
LeAmandaHallerChiara S.LangerEllen J.CourvoisierDelphine S.. 2012. “Mindful Multitasking: The Relationship between Mindful Flexibility and Media Multitasking.” Computers in Human Behavior28(4):1526‐32. doi:10.1016/j.chb.2012.03.022
35.
LeinerDominik J.2019 “Too Fast, Too Straight, Too Weird: Non-Reactive Indicators for Meaningless Data in Internet Surveys.” Survey Research Methods13(3):229‐48. doi:10.18148/srm/2019.v13i3.7403
36.
LeónCarmen M.AizpuruaEvaDíaz de RadaVidal. 2021. “Multitarea En Una Encuesta Online: Prevalencia, Predictores E Impacto En La Calidad De Los Datos / Multitasking During an Online Survey: Prevalence, Predictors and Impact on Data Quality.” Revista Española de Investigaciones Sociológicas173:27‐46. doi:10.5477/cis/reis.173.27
37.
LugtigPeterGlasnerTinaBoevéAnja J.. 2016. “Reducing Underreports of Behaviors in Retrospective Surveys: The Effects of Three Different Strategies.” International Journal of Public Opinion Research28(4):583‐95. doi:10.1093/ijpor/edv032
38.
LugtigPeterToepoelVera. 2016. “The Use of Pcs, Smartphones, and Tablets in a Probability-Based Panel Survey:Effects on Survey Measurement Error.” Social Science Computer Review34(1):78‐94. doi:10.1177/0894439315574248
39.
LynnPeterKaminskaOlena. 2011 “Factors Affecting Measurement Error in Mobile Phone Interviews.” Pp. 211‐28 in Telephone Surveys in Europe, edited by HäderS.HäderM.KühneM.. Wiesbaden: Springer.
40.
McClainColleen A.CouperMick P.HuppAndrew L.KeuschFlorianPetersonGreggPiskorowskiAndrew D.WestBrady T.. 2019. “A Typology of Web Survey Paradata for Assessing Total Survey Error.” Social Science Computer Review37(2):196‐213. doi:10.1177/0894439318759670
41.
McKibbenW. BradleySilviaPaul J.. 2016. “Inattentive and Socially Desirable Responding.” Counseling Outcome Research and Evaluation7(1):53‐64. doi:10.1177/2150137815613135
42.
NicholsAustin L.EdlundJohn E.. 2020. “Why Don’t We Care More About Carelessness? Understanding the Causes and Consequences of Careless Participants.” International Journal of Social Research Methodology23(6):1‐14. doi:10.1080/13645579.2020.1719618
43.
ParryDouglas A.le RouxDaniel B.. 2019. “Media Multitasking and Cognitive Control: A Systematic Review of Interventions.” Computers in Human Behavior92:316‐27. doi:10.1016/j.chb.2018.11.031
44.
PeralesFrancisco. 2014. “How Wrong Were We? Dependent Interviewing, Self-Reports and Measurement Error in Occupational Mobility in Panel Surveys.” Longitudinal and Life Course Studies5(3):299‐316.
45.
RalphBrandon C. W.ThomsonDavid R.CheyneJames A.SmilekDaniel. 2014. “Media Multitasking and Failures of Attention in Everyday Life.” Psychological Research78(5):661‐69. doi:10.1007/s00426-013-0523-7
46.
ReevesByronCummingsJames J.YeykelisLeo. 2014. “Multitasking on a Single Device: Arousal and the Frequency, Anticipation, and Prediction of Switching between Media Content on a Computer.” Journal of Communication64(1):167‐92. doi:10.1111/jcom.12070
47.
RobertsCarolineGilbertEmilyAllumNickEisnerLéïla. 2019. “Research Synthesis: Satisficing in Surveys: A Systematic Review of the Literature.” Public Opinion Quarterly83(3):598‐626. doi:10.1093/poq/nfz035
48.
RoßmannJoss. 2015. “Rspeedindex: Stata Module to Compute a Response Speed Index and Perform Outlier Identification.”
49.
RoßmannJoss. 2017. “Respdiff: Stata Module for Generating Response Differentiation Indices.”
50.
RoßmannJossGummerTobiasKaczmirekLars. 2020. “Working with User Agent Strings in Stata: The Parseuas Command.” Journal of Statistical Software, Code Snippet92:1‐16. doi:10.18637/jss.v092.c01.
51.
SalvucciDario D.TaatgenNiels A.. 2010. The Multitasking Mind. Oxford: Oxford University Press.
52.
SariWinda L.FakhruddianaFuadah. 2019. “Internal Locus of Control, Social Support and Academic Procrastination among Students in Completing the Thesis.” International Journal of Evaluation and Research in Education8(2):363‐68.
53.
SchlosserStephanHöhneJan K.. 2018. “Ecsp – Embedded Client Side Paradata.”
54.
SchlosserStephanMaysAnja. 2018. “Mobile and Dirty:Does Using Mobile Devices Affect the Data Quality and the Response Process of Online Surveys?” Social Science Computer Review36(2):212‐30. doi:10.1177/0894439317698437
55.
SeberGeorge A.LeeAlan J.. 2012. Linear Regression Analysis, Vol. 329. Hoboken, NJ: John Wiley & Sons.
56.
SendelbahAnžeVehovarVasjaSlavecAnaPetrovčičAndraž. 2016. “Investigating Respondent Multitasking in Web Surveys Using Paradata.” Computers in Human Behavior55:777‐87. doi:10.1016/j.chb.2015.10.028
57.
ShinMyoungjuWebbAndrewKempsEva. 2019. “Media Multitasking, Impulsivity and Dual Task Ability.” Computers in Human Behavior92:160‐68. doi:10.1016/j.chb.2018.11.018
58.
SischkaPhilipp E.DécieuxJean PhilippeMergenerAlexandraNeufangKristina M.SchmidtAlexander F.. 2020. “The Impact of Forced Answering and Reactance on Answering Behavior in Online Surveys.” Social Science Computer Review1‐21. doi:10.1177/0894439320907067
59.
SzékelyLevente. 2015. “The Typology of Multitasking Activity.” European Journal of Communication30(2):209‐25. doi:10.1177/0267323114567842
60.
TurkleSherry. 2017. Alone Together: Why We Expect More from Technology and Less from Each Other. UK: Hachette.
61.
VésteinsdóttirVakaJoinsonAdamReipsUlf-DietrichDanielsdottirHilda B.ThorarinsdottirElin A.ThorsdottirFanney. 2018. “Questions on Honest Responding.” Behavior Research Methods51:811‐25. doi:10.3758/s13428-018-1121-9
62.
WenzAlexander. 2021. “Do Distractions During Web Survey Completion Affect Data Quality? Findings from a Laboratory Experiment.” Social Science Computer Review39(1):148‐61. doi:10.1177/0894439319851503
63.
WickensChristopher. 2008. “Multiple Resources and Mental Workload.” Human Factors50(3):449‐55.
64.
WickensChristopherGutzwillerRobertSantamariaAmy. 2015. “Discrete Task Switching in Overload: A Meta-Analyses and a Model.” International Journal of Human-Computer Studies79:79‐84. doi:10.1016/j.ijhcs.2015.01.002
65.
ZhangChanAntounChristopherYanH. YannaConradFrederick G.. 2020. “Professional Respondents in Opt-in Online Panels: What Do We Really Know?” Social Science Computer Review38(6):703‐19. doi:10.1177/0894439319845102
66.
ZhangTongdaSunXiaoChaiYuetingAghajanHamid. 2015. “A Look at Task-Switching and Multi-Tasking Behaviors: From the Perspective of the Computer Usage among a Large Number of People.” Computers in Human Behavior49:237‐44. doi:10.1016/j.chb.2015.03.012
67.
ZwarunLaraHallAlice. 2014. “What’s Going On? Age, Distraction, and Multitasking During Online Survey Taking.” Computers in Human Behavior41:236‐44. doi:10.1016/j.chb.2014.09.041
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
