Abstract
Previous research is inconclusive regarding the effects of paper and web surveys on response burdens. We conducted an establishment survey with random assignment to paper and web modes to examine this issue. We compare how the actual and perceived response burdens differ when respondents complete a survey in the paper mode, in the web mode and when they are allowed to choose between the two modes. Our results show that in the web mode, respondents have a lower estimated time to complete the questionnaire, while we do not find differences between paper and the web on the perceived response time and perceived burden. Even though the response burden in the web mode is lower, our study finds no evidence of an increased response burden when moving an establishment survey from paper to the web.
References
1.
AAPOR. 2016. Standard Definitions: Final Disposition of Case Codes and Outcome Rates for Surveys . Available at: http://www.aapor.org/AAPOR_Main/media/publications/Standard-Definitions20169theditionfinal.pdf (accessed March 2019).
2.
Bavdaž, M., D. Giesen, S.K.Černe, T. Löfgren, and V. Raymond-Blaess. 2015. “Response burden in official business surveys: Measurement and reduction practices of national statistical institutes.” Journal of Official Statistics 31: 559–588. https://doi.org/10.1515/jos-2015-0035.
3.
Berglund, F., G. Haraldsen, and Ø. Kleven. 2013. “Causes and consequences of actual and perceived response burden based on Norwegian data.” In Comparative report on integration of case study results related to reduction of response burden and motivation of business , edited by D. Giesen, M. Bavdaž, and I. Bolko: 29–35.
4.
Bradburn, N.M. 1978. “Respondent burden.” In Proceedings of the Section on Survey Research Methods Section: American Statistical Association, August 14–17, 1978. 35–40. San Diego, USA. American Statistical Association: 35–40. Available at: http://www.asasrms.org/Proceedings/y1978f.html (accessed March 2021).
5.
Bremner, C. 2011. “An investigation into the use of mixed mode data collection methods for UK business surveys.” In Proceedings of the BLUE-ETS Conference on Burden and Motivation in Official Business Surveys , edited by D. Giesen and M. Bavdaž. March 22–23, (pp. 217–220). Heerlen, The Netherlands: Statistics Netherlands. Available at: http://www.cbs.nl/NR/rdonlyres/23FD3DF5-6696-4A04-B8EF-1FAACEAD995C/0/2011proceedingsblueets.pdf (accessed March 2019).
6.
Cameron, A.C., and P.K. Trivedi. 2005. Microeconometrics: Methods and Applications . New York: Cambridge University Press.
7.
Conrad, F.G., M.F. Schober, and T. Coiner. 2007. “Bringing Features of Human Dialogue to Web Surveys.” Applied Cognitive Psychology 21: 165–187. DOI: https://doi.org/10.1002/acp.1335.
8.
Couper, M.P. 2008. Designing Effective Web Surveys . Cambridge: Cambridge University Press.
9.
Couper, M.P., and R.M. Groves. 1996. “Household-level determinants of survey nonresponse.” New Directions for Evaluation 70: 63–79. DOI: https://doi.org/10.1002/ev.1035.
10.
Dale, T., J. Erikson, J. Fosen, G. Haraldsen, J. Jones, and O. Kleven. 2007. Handbook for monitoring and evaluating business survey response burdens . Luxemburg: Eurostat. Available at: https://ec.europa.eu/eurostat/documents/64157/4374310/12-HANDBOOK-FOR-MONITORING-AND-EVALUATING-BUSINESS-SURVEY-RESONSE-BURDEN.pdf/600e3c6d-8e8d-44f7-a8f5-0931c71d9920 (accessed October 2021).
11.
Destatis. 2008. Klassifikation der Wirtschaftszweige . Wiesbaden, Germany: Statistisches Bundesamt. Available at: https://www.destatis.de/DE/Methoden/Klassifikationen/Gueter-Wirtschaftsklassifikationen/klassifikation-wz-2008.html (accessed June 2019).
12.
Downey, K., D. Mccarthy, and W. Mccarthy. 2007. “Encouraging the Use of Alternative Modes of Electronic Data Collection: Results of Two Field Studies.” In Proceedings of the Third International Conference on Establishment Surveys, June 18–21, 2007. Montréal, Canada: 517–524. Available at: https://ww2.amstat.org/meetings/ices/2007/proceedings/ICES2007-000168.PDF (accessed September 2019).
13.
Erikson, J. 2007. “Effects of offering web questionnaires as an option in enterprise surveys.” In Proceedings of the Third International Conference on Establishment Surveys, June 18–21, 2007. Montréal, Canada: 1431–1435. Available at: https://ww2.amstat.org/meetings/ices/2007/proceedings/ICES2007-000168.PDF (accessed September 2019)
14.
European Commission. 2011. European Statistics Code of Practice for the National and Community Statistical Authorities. Adopted by the European Statistical System Committee, September 28, 2011. Available at: https://ec.europa.eu/eurostat/documents/3859598/5921861/KS-32-11-955-EN.PDF/5fa1ebc6-90bb-43fa-888f-dde032471e15 (accessed September 2019).
15.
Gelman, A. 2007. “Struggles with Survey Weighting and Regression Modeling.” Statistical Science 22(2) : 153–164. DOI: https://doi.org/10.1214/088342306000000691.
16.
Giesen, D. 2007. “Does mode matter? Comparing response burden and data quality of paper and an electronic business questionnaire.” In Proceedings of the 6th Conference on Questionnaire Evaluation Standards (QUEST), April 24–26, 2007. Ottawa, Canada: 150–161. Available at: https://wwwn.cdc.gov/QBank/QUEST/2007/QUEST%202007%20Proceedings-all%20papers.pdf (accessed September 2019).
17.
Giesen, D. 2011. “Burden reduction by communication.” In Response Burden in Official Business Surveys: Measurement and Reduction Practices of National Statistical Institutes , edited by D. Giesen: 33–42.
18.
Giesen, D. 2012. “Exploring Causes and Effects of Perceived Response Burden.” In Proceedings of the Fourth International Conference on Establishment Surveys, June 11–14, 2012. Montréal, Canada. Available at: https://ww2.amstat.org/meetings/ices/2012/papers/302171.pdf (accessed September 2019).
19.
Giesen, D. 2013a. “Causes and Consequences of Actual and Perceived Response Burden Based on Dutch Data.” In Comparative report on integration of case study results related to reduction of response burden and motivation of business , edited by D. Giesen, M. Bavdaž, and I. Bolko: 33–39.
20.
Giesen, D. 2013b. “Reducing Response Burden by Questionnaire Redesign.” In Comparative report on integration of case study results related to reduction of response burden and motivation of business , edited by D. Giesen, M. Bavdaž, and I. Bolko: 63–68.
21.
Giesen, D., M. Bavdaž, and G. Haraldsen. 2011. “Response burden measurement: Current diversity and proposal for moving towards standardisation.” In Proceedings of the BLUE-ETS Conference on Burden and Motivation in Official Business Surveys , edited by D. Giesen and M. Bavdaž. March 22–23: 125–134. Heerlen, The Netherlands: Statistics Netherlands. Available at: https://www.cbs.nl/-/media/imported/documents/2011/14/2011-4-4-4-2-giesen-et-al-presentation-blue-ets-2011.pdf (accessed March 2021).
22.
Giesen, D., and J. Burger. 2013. “Measuring and understanding response quality in the Structural Business Survey questionnaires.” In Proceedings of the European Establishment Statistics Workshop, September 9–11, 2013. Nuremberg, Germany. Available at: http://doku.iab.de/fdz/events/2013/Session5%20Giesen.pdf (accessed September 2019).
23.
Giesen, D., M. Morren, and G. Snijkers. 2009. “The effect of survey redesign on response burden: An evaluation of the redesign of the SBS questionnaires.” Paper presented at the European Survey Research Association Conference 2009, June 29–July 3, 2009. Warsaw, Poland. European Survey Research Association.
24.
Giesen, D., M. Vella, and C. Brady. 2018. “Response Burden Management for Establishment Surveys at Four National Statistical Institutes.” Journal of Official Statistics 34(2): 397–418. DOI: https://doi.org/10.2478/jos-2018-0018.
25.
Groves, R.M., R.B. Cialdini, and M.P. Couper. 1992. “Understanding the decision to participate in a survey.” Public Opinion Quarterly 56: 475–495. DOI: https://doi.org/10.1086/269338.
26.
Gregory, G., and M. Earp. 2007. “Evolution of Web at USDA’ National Agricultural Statistics Service.” In Proceedings of the Third International Conference on Establishment Surveys (ICES-III), June 18–21, 2007. Montréal, Canada: 1442–1445. Available at: https://ww2.amstat.org/meetings/ices/2007/proceedings/ICES2007-000192.PDF (accessed September 2019).
27.
Gravem, D. 2011. “Response burden trends and consequences.” In Proceedings of the BLUE-ETS Conference on Burden and Motivation in Official Business Surveys , edited by D. Giesen and M. Bavdaž. March 22–23: 221–236. Heerlen, The Netherlands: Statistics Netherlands. Available at: http://www.cbs.nl/NR/rdonlyres/23FD3DF5-6696-4A04-B8EF-1FAACEAD995C/0/2011proceedingsblueets.pdf (accessed March 2019).
28.
Haas, G.-C., S. Eckman, R. Bach, and F. Kreuter. 2016. “Is Moving Establishment Surveys from Mail to Web a Good or Bad Decision in Terms of Performance and Data Quality?” In Proceedings of the International Conference for Establishment Surveys 2016 (ICES-V). June 21–23, 2016. Geneva, Switzerland. Available at: https://ww2.amstat.org/meetings/ices/2016/proceedings/ICESV_TOC.pdf (accessed February 2020).
29.
Haraldsen, G., and J. Jones. 2007. “Paper and Web Questionnaires Seen from the Business Respondent’s Perspective.” In Proceedings of the Third International Conference on Establishment Surveys (ICES-III), June 18–21, 2007, Montreal, Canada: 1040–1047. Available at: https://ww2.amstat.org/meetings/ices/2007/proceedings/ICES2007-000259.PDF (accessed September 2019).
30.
Haraldsen, G., J. Jones, D. Giesen, and L.C. Zhang. 2013. “Understanding and coping with response burden.” In Designing and Conducting Business Surveys , edited by G. Snijkers, G. Haraldsen, J. Jones, and D. Willimack: 219–252. Hoboken, NJ: John Wiley and Sons.
31.
Harrell, L., H. Yu, and R. Rosen. 2007. “Respondent acceptance of web and E-mail data reporting for an establishment survey.” In Proceedings of the Third International Conference on Establishment Surveys (ICES-III), June 18–21, 2007, Montreal, Canada: 1442–1445. Available at: https://ww2.amstat.org/meetings/ices/2007/proceedings/ICES2007-000230.PDF (accessed September 2019).
32.
Hedlin, D., T. Dale, G. Haraldsen, and J. Jones. 2005. Developing Methods for Assessing Perceived Response Burden . Luxembourg: Eurostat. Available at: https://ec.europa.eu/eurostat/documents/64157/4374310/10-DEVELOPING-METHODS-FOR-ASSESSING-PERCEIVED-RESPONSE-BURDEN.pdf/1900efc8-1a07-4482-b3c9-be88ee71df3b (accessed October 2021).
33.
Jones, J. 2012. “Response Burden: Introductory Overview Lecture.” In Proceedings of the Fourth International Conference on Establishment Surveys, June 11–14, 2012. Montréal, Canada. Available at: http://www.amstat.org/meetings/ices/2012/papers/302289.pdf (accessed September 2019).
34.
Jones, J., G. Snijkers, and G. Haraldsen. 2013. “Surveys and Business Surveys.” In Designing and Conducting Business Surveys , edited by G. Snijkers, G. Haraldsen, J. Jones, and D.K. Willimack: 1–33. Hoboken, NJ: John Wiley & Sons.
35.
Kaplowitz, M.D., T.D. Hadlock, and R. Levine. 2004. “A Comparison of Web and Mail Survey Response Rates.” Public Opinion Quarterly 68: 94 – 101. DOI: https://doi.org/10.1093/poq/nfh006.
36.
Krosnick, J.A. 1991. “Response strategies for coping with the cognitive demands of attitude measures in surveys.” Applied Cognitive Psychology 5: 213–236. DOI: https://doi.org/10.1002/acp.2350050305.
37.
Kwak, N., and B. Radler. 2002. “A Comparison between Mail and Web Surveys: Response Pattern, Respondent Profile, and Data Quality.” Journal of Official Statistics 18: 257–273. DOI: https://doi.org/10.1177/1525822X08317085.
38.
Lagerstøm, B. 2018. “Chatbots as digital interviewers.” Paper presented at the International Household Nonresponse Workshop, August 22–24, 2018. Budapest, Hungary
39.
Löfgren, T. 2011. “Burden reduction by instrument design.” In Response Burden in Official Business Surveys: Measurement and Reduction Practices of National Statistical Institutes , edited by D. Giesen: 43–50.
40.
Lyly-Yrjänäinen, M., and G. van Houten. 2011. “Reduce burden, increase motivation. Main findings of the quality assessment of the second European company survey.” In Proceedings of the BLUE-ETS Conference on Burden and Motivation in Official Business Surveys March 22–23, 2011, Heerlen, Netherlands, edited by D. Giesen and M. Bavdaž: 107–118. Available at: https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&ved=2ahUKEwiFvfTm2OrzAhU8BWMBHYOlDv8QFnoECAgQAQ&url=https%3A%2F%2Fwww.cbs.nl%2F-%2Fmedia%2Fimported%2Fdocuments%2F2011%2F25%2F2011-06-20-lyly-yrjanainen-van-houten-2011-reduce-burden-increase-motivation.pdf&usg=AOvVaw0n4dUPG4Ectn1K-baV7hE0 (accessed October 2021).
41.
Medway, R.L., and J. Fulton. 2012. “When More Gets You Less: A Meta-Analysis of the Effect of Concurrent Web Options on Mail Survey Response Rates.” Public Opinion Quarterly 76: 733–746. DOI: https://doi.org/10.1093/poq/nfs047.
42.
Messer, B.L., and D.A. Dillman. 2010. Using Address Based Sampling to Survey the General Public by Mail vs. “Web plus Mail”. Technical Report 10–13. Social and Economic Sciences Research Center , Washington State University, Pullman. Available at: http://www.sesrc.wsu.edu/dillman/papers/2010/Messer%20Dillman%20WCSTechReport.pdf (accessed October 2021).
43.
Millar, M.M., O.A.C. Neill, and D.A. Dillman. 2009. “Are Mode Preferences Real?” Technical Report of the Social and Economic Sciences Research Center. Pullman, Washington: Washington State University
44.
Sear, J. 2011. “Response burden measurement and motivation at Statistics Canada.” In Proceedings of the BLUE-ETS Conference on Burden and Motivation in Official Business Surveys Heerlen, March 22–23, 2011, Heerlen, Netherlands. edited by D. Giesen and M. Bavdaž: 151–160. Available at: https://www.cbs.nl/-/media/imported/-documents/2011/25/2011-06-20-sear-2011-response-burden-measurement-and-motivation-at-statistics-canada.pdf (accessed October 2021).
45.
Snedecor, G.W., and W.G. Cochran. 1989. Statistical Methods , (8th edition). Ames, IA: Iowa State University Press.
46.
Snijkers, G., G. Haraldsen, A. Sundvoll, T. Vik, and H.P. Stax. 2011. “Utilizing web technology in business data collection: Some Norwegian, Dutch and Danish experiences.” In Proceeding of the European Conference on New Techniques and Technologies for Statistics (NTTS) , Brussels, Belgium
47.
Snijkers, G., E. Onat, and R. Vis-Visschers. 2007. “The Annual Structural Business Survey: Developing and Testing an Electronic Form.” In Proceedings of the International Conference for Establishment Surveys 2007 (ICES-III) June 18–21, 2007, Montreal, Canada: 456–463. Available at: https://ww2.amstat.org/meetings/ices/2007/proceedings/ICES2007-000095.PDF (accessed October 2021).
48.
Verkruyssen, F., and S. Moens. 2011. “Communication as a tool to reduce perceived response burden: Tips and tricks.” In Proceedings of the BLUE-ETS Conference on Burden and Motivation in Official Business Surveys , March 22–23, 2011, Heerlen, Netherlands edited by D. Giesen and M. Bavdaž. 237 – 242. Available at: https://www.cbs.nl/-/media/imported/documents/2011/14/2011-4-4-8-1-verkruyssenmoens-presentation-blue-ets-2011.ashx?la=en-gb (accessed October 2021).
49.
Willimack, D., and E. Nichols. 2010. “A Hybrid Response Process Model for Business Surveys.” Journal of Official Statistics 26: 3–24. Available at: https://www.scb.se/-contentassets/ca21efb41fee47d293bbee5bf7be7fb3/a-hybrid-response-process-model-for-business-surveys.pdf (accessed September 2021).
50.
Van Loon, A.J.M., M. Tijhuis, H.S. Picavet, P.G. Surtees, and J. Ormel. 2003. “Survey Non-response in the Netherlands: Effects on Prevalence Estimates and Associations.” Ann Epidemiol 13: 105–110. DOI: https://doi.org/10.1016/S1047-2797(02)00257-0.
51.
Yan, T., S. Fricker, and S. Tsai. 2020. “Response Burden: What Is It and What Predicts It?” In Advances in Questionnaire Design, Development, Evaluation and Testing , edited by P.C. Beatty, D. Collins, L. Kaye, J.-L. Padilla, G.B. Willis, and A. Wilmot: 193–212.