The Representativeness of Online Time Use Surveys. Effects of Individual Time Use Patterns and Survey Design on the Timing of Survey Dropout

Abstract

Like other surveys, time use surveys are facing declining response rates. At the same time paper-and-pencil surveys are increasingly replaced by online surveys. Both the declining response rates and the shift to online research are expected to have an impact on the representativeness of survey data questioning whether they are still the most suitable instrument to obtain a reliable view on the organization of daily life. This contribution examines the representativeness of a self-administered online time use survey using Belgian data collected in 2013 and 2014. The design of the study was deliberately chosen to test the automated processes that replace interviewer support and its cost-efficiency. We use weighting coefficients, a life table and discrete-time survival analyses to better understand the timing and selectivity of dropout, with a focus on the effects of individual time use patterns and the survey design. The results show that there are three major hurdles that cause large groups of respondents to drop out. This dropout is selective, and this selectivity differs according to the dropout moment. The contribution aims to provide a better insight in dropout during the fieldwork and tries to contribute to the further improvement of survey methodology of online time use surveys.

Keywords

Nonresponse time use survey survey design online research

References

Abraham, K.G., A. Maitland, and S.M. Bianchi. 2006. “Nonresponse in the American Time Use Survey: Who Is Missing from the Data and How Much Does It Matter?” Public Opinion Quarterly 70(5): 676–703. DOI: https://doi.org/10.1093/poq/nfl037.

Bethlehem, J. 2009. “The Rise of Survey Sampling.” Statistics Netherlands. Discussion Paper (09015). Available at: https://www.cbs.nl/-/media/imported/documents/2009/07/2009-15-x10-pub.pdf?la=nl-nl&hash=B75A64DF0877B7FD089E796FCFE81145 (accessed August 2020).

Boström, G., J. Hallqvist, B.J.A. Haglund, A. Romelsjö, L. Svanström, and F. Diderichsen. 1993. “Socioeconomic Differences in Smoking in an Urban Swedish Population. The Bias Introduced by Non-Participation in a Mailed Questionnaire.” Scandinavian Journal of Public Health 21(2): 77–82. DOI: https://doi.org/10.1177/140349489302100204.

Connelly, N.A., T.L. Brown, and D.J. Decker. 2003. “Factors Affecting Response Rates to Natural Resource – Focused Mail Surveys: Empirical Evidence of Declining Rates Over Time.” Society & Natural Resources 16(6): 541–549. DOI: https://doi.org/10.1080/08941920309152.

Couper, M.P., A. Kapteyn, M. Schonlau, and J. Winter. 2007. “Noncoverage and Nonresponse in an Internet Survey.” Social Science Research 36(1): 131–148. DOI: https://doi.org/10.1016/j.ssresearch.2005.10.002.

Cull, W.L., K.G.O. Connor, S. Sharp, and S.S. Tang. 2005. “Methods Response Rates and Response Bias for 50 Surveys of Pediatricians.” HSR: Health Services Research 40(1): 213–226. DOI: https://doi.org/10.1111/j.1475-6773.2005.00350.x.

Curtin, R., S. Presser, and E. Singer. 2000. “The Effects of Response Rate Changes on the Index of Consumer Sentiment.” Public Opinion Quarterly 64(4): 413–428. DOI: https://doi.org/10.1086/318638.

Dillman, D.A., G. Phelps, R. Tortora, K. Swift, J. Kohrell, J. Berck, and B.L. Messer. 2009. “Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the Internet.” Social Science Research, 38(1), 1–18. DOI: https://doi.org/10.1016/j.ssresearch.2008.03.007.

Etter, J.-F., and T.V. Perneger. 1997. “Analysis of Non-Response Bias in a Mailed Health Survey.” Journal of Clinical Epidemiology 50(10): 1123–1128. DOI: https://doi.org/10.1016/S0895-4356(97)00166-2.

10.

Goyder, J. 1986. “Surveys on Surveys: Limitations and Potentialities.” Public Opinion Quarterly 50(1): 27–41. DOI: https://doi.org/10.1086/268957.

11.

Goyder, J., K. Warriner, and S. Miller. 2002. “Evaluating Socio-Economic Status (SES) Bias in Survey Nonresponse.” Journal of Official Statistics 18(1): 1–11. Available at: https://www.scb.se/contentassets/ca21efb41fee47d293bbee5bf7be7fb3/evaluating-socio-economic-status-ses-bias-in-survey-nonresponse.pdf (accessed September 2020).

12.

Groves, R.M., and M.P. Couper. 1998. Nonresponse in Household Interview Surveys. New York (NY): John Wiley. DOI: https://doi.org/10.1525/aa.1999.101.3.699.

13.

Hill, A., J. Roberts, P. Ewings, and D. Gunnell. 1997. “Non-Response Bias in a Lifestyle Survey.” Journal of Public Health 19(2): 203–207. DOI: https://doi.org/10.1093/oxfordjournals.pubmed.a024610.

14.

Hochschild, A., and A. Machung. 2003. The Second Shift: Working Families and the Revolution at Home. New York (NY): Penguin.

15.

Johnson, T.P., and J.S. Wislar. 2012. “Response Rates and Nonresponse Errors in Surveys.” JAMA: The Journal of the American Medical Association 307(17): 1805–1806. DOI: https://doi.org/10.1001/jama.2012.3532.

16.

Knulst, W., and A. van den Broek. 1998. “Do Time-Use Surveys Succeed in Measuring ‘Busyness’? Some Observations of the Dutch Case.” Loisir et Société / Society and Leisure 21(2): 563–572. DOI: https://doi.org/10.1080/07053436.1998.10753671.

17.

Kwak, N., and B. Radler. 2002. “A Comparison Between Mail and Web Surveys: Response Pattern, Respondent Profile, and Data Quality.” Journal of Official Statistics 18(2): 257–273. Available at: https://www.scb.se/contentassets/ca21efb41fee47d293bbee5bf7be7fb3/a-comparison-between-mail-and-web-surveys-response-pattern-respondent-profile-and-data-quality.pdf (accessed September 2020).

18.

Marcus, B., M. Bosnjak, S. Lindner, S. Pilischenko, and A. Schutz. 2007. “Compensating for Low Topic Interest and Long Surveys: A Field Experiment on Nonresponse in Web Surveys.” Social Science Computer Review 25(3): 372–383. DOI: https://doi.org/10.1177/0894439307297606.

19.

Minnen, J., I. Glorieux, T.P. van Tienoven, D. Weenas, J. Deyaert, S. van den Bogaert, and S. Rymenants. 2014. “Modular Online Time Use Survey (MOTUS) – Translating an existing method in the 21st century.” Electronic International Journal of Time Use Research, 11(1), 73–93. DOI: https://doi.org/10.13085/eIJTUR.11.1.73-93.

20.

Moore, D.L., and J. Tarnai. 2001. “Evaluating Nonresponse Error in Mail Surveys.” In Survey Nonresponse, edited by R.M. Groves, D.A. Dillman, J.L. Eltinge, and R.J.A. Little, 197–212. New York (NY): Wiley.

21.

Pääkkönen, H. 1998. “Are Busy People Under- or over-Represented in National Time Budget Surveys?” Loisir et Société / Society and Leisure 21(2): 573–582. DOI: https://doi.org/10.1080/07053436.1998.10753672.

22.

Porter, S.R., and M.E. Whitcomb. 2005. “Non-Response in Student Surveys: The Role of Demographics, Engagement and Personality.” Research in Higher Education 46(2): 127–152. DOI: https://doi.org/10.1007/s11162-004-1597-2.

23.

Savage, M., and R. Burrows. 2009. “Some Further Reflections on the Coming Crisis of Empirical Sociology.” Sociology 43(4): 762–772. DOI: https://doi.org/10.1177/0038038509105420.

24.

Sax, L.J., S.K. Gilmartin, and A.N. Bryant. 2003. “Assessing Response Rate and Nonreponse Bias in Web and Paper Surveys.” Research in Higher Education 44(4): 409–432. DOI: https://doi.org/10.1023/A:1024232915870.

25.

Singer, E., R.M. Groves, and A.D. Corning. 1999. “Differential Incentives: Beliefs About Practices, Perceptions of Equity, and Effects on Survey Participation.” Public Opinion Quarterly 63(2): 251–260. DOI: https://doi.org/10.1086/297714.

26.

Singer, E., J. van Hoewyk, and M.P. Maher. 2000. “Experiments with Incentives in Telephone Surveys.” Public Opinion Quarterly 64(2): 171–188. DOI: https://doi.org/10.1086/317761.

27.

Singer, J.D., and J.B. Willett. 2003. Applied Longitudinal Data Analysis. Modeling Change and Event Occurrence. New York (NY): Oxford University Press.

28.

Smith, C., and D. Nutbeam. 1990. “Assessing Non-Response Bias: A Case Study from the 1985 Welsh Heart Health Survey.” Health Education Research 5(3): 381–386. DOI: https://doi.org/10.1093/her/5.3.381.

29.

Smith, W.G. 2008. “Does Gender Influence Online Survey Participation? A Record-Linkage Analysis of University Faculty Online Survey Response Behavior.” Available at: https://files.eric.ed.gov/fulltext/ED501717.pdf (accessed August 2020).

30.

Szalai, A. 1966. “Trends in Comparative Time-Budget Research.” American Behavioral Scientist 9(9): 3–8. DOI: https://doi.org/10.1177/000276426600900901.

31.

Van Ingen, E., I. Stoop, and K. Breedveld. 2008. “Nonresponse in the Dutch Time Use Survey: Strategies for Response Enhancement and Bias Reduction.” Field Methods 21(1): 69–90. DOI: https://doi.org/10.1177/1525822X08323099.

32.

Van Kenhove, P., K. Wijnen, and K. de Wulf. 2000. “The Influence of Topic Involvement on Mail-Survey Response Behavior.” Psychology & Marketing 19(3): 293–301. DOI: https://doi.org/10.1002/mar.1053.

33.

Wallace, D. 1954. “A Case For- and Against- Mail Questionnaires.” Public Opinion Quarterly 18(1): 40–52. DOI: https://doi.org/10.1086/266484.

34.

Zuzanek, J. 1998. “Non-Response in Time-Use Surveys: Do the Two Ends Meet?” Loisir et Société / Society and Leisure 21(2): 547–549. DOI: https://doi.org/10.1080/07053436.1998.10753668.