Abstract
Response burden has long been a concern for data producers. In this article, we investigate the relationship between some measures of actual and perceived burden and we provide empirical evidence of their association with data quality. We draw on two business surveys conducted by Banca d’Italia since 1970, which provide a very rich and unique source of information. We find evidence that the perceived burden is affected by actual burden but the latter is not the only driver. Our results also show a clear link between a respondent’s perceived effort and the probability of not answering some important questions (such as those relating to expectations of future investments and turnover) or of dropping out of the survey. On the contrary, we do not find significant effects on the quality of answers to quantitative questions such as business turnover and investments. Overall, these findings have implications for data producers that should target the perceived burden, besides the actual burden, to increase data quality.
References
1.
Bavdaž, M. 2010. “Sources of Measurement Errors in Business Surveys.” Journal of Official Statistics 26 (1): 24–42. Available at: https://www.scb.se/contentassets/ca21efb41fee47d293bbee5bf7be7fb3/sources-of-measurement-errors-in-business-surveys.pdf (accessed September 2021).
2.
Bavdaž, M., D. Giesen, S.K. Černe, T. Löfgren, and V. Raymond-Blaess. 2015. “Response Burden in Official Business Surveys: Measurement and Reduction Practices of National Statistical Institutes.” Journal of Official Statistics 31 (4): 559–588. DOI: https://doi.org/10.1515/jos-2015-0035..
3.
Bergman, L.R., and R. Brage. 2008. “Survey Experience and Later Survey Attitudes, Intention and Behavior.” Journal of Official Statistics 24 (1): 99–113. Available at: https://www.scb.se/contentassets/ca21efb41fee47d293bbee5bf7be7fb3/survey-experiences-and-later-survey-attitudesintentions-and-behaviour.pdf (accessed September 2021).
4.
Bradburn, N.M. 1978. “Respondent burden.” In Proceedings of the Survey Research Methods Section of the American Statistical Association , 35–40. Available at: http://www.asasrms.org/Proceedings/papers/1978_007.pdf (accessed March 2021)
5.
Crawford, S.D., M.P. Couper, and M.J. Lamias. 2001. “Web surveys: Perceptions of burden.” Social Science Computer Review 19 (2): 146–162. DOI: https://doi.org/10.1177/089443930101900202.
6.
Dale, T., J. Erikson, J. Fosen, G. Haraldsen, J. Jones, and Ø. Kleven. 2007. Handbook for Monitoring and Evaluating Business Survey Response Burdens . European Commission. Available at: https://ec.europa.eu/eurostat/documents/64157/4374310/12-HANDBOOK-FORMONITORING-AND-EVALUATING-BUSINESS-SURVEY-RESONSE-BURDEN.pdf/600e3c6d-8e8d-44f7-a8f5-0931c71d9920 (accessed March 2021)
7.
D’Aurizio, L., and G. Papadia. 2019. “Using Administrative Data to Evaluate Sampling Bias in a Business Panel Survey.” Journal of Official Statistics 35 (1): 67–92. DOI: https://doi.org/10.2478/jos-2019-0004.
8.
Edwards, P., I. Roberts, M. Clarke, C. DiGuiseppi, S. Pratap, R. Wentz, and I. Kwan. 2002. “Increasing response rates to postal questionnaires: systematic review.” BMJ 324 (7347): 1183. DOI: https://doi.org/10.1136/bmj.324.7347.1183.
9.
European Commission. Statistical Office of the European Union. 2014. Manuals and guidelines. ESS Handbook for Quality Reports . DOI: https://doi.org/10.2785/983454.
10.
Fisher, S., and L. Kydoniefs. 2001. “Using a theoretical model of response burden to identify sources of burden in surveys.” Paper presented at the 12th International Workshop on Household Survey Nonresponse, September 12–14, Oslo, Norway.
11.
Galesic, M. 2006. “Dropouts on the Web: Effects of Interest and Burden Experienced During an Online Survey.” Journal of Official Statistics 22: 313–328. Available at: https://www.scb.se/contentassets/ca21efb41fee47d293bbee5bf7be7fb3/dropouts-onthe-web-effects-of-interest-and-burden-experienced-during-an-online-survey.pdf (accessed September 2021).
12.
Galesic, M., and M. Bosnjak. 2009. “Effects of questionnaire length on participation and indicators of response quality in a web survey.” Public Opinion Quarterly 73 (2): 349–360. DOI: https://doi.org/10.1093/poq/nfp031.
13.
Giesen, D. 2012. “Exploring causes and effects of perceived response burden.” In Proceedings of the Fourth International Conference on Establishment Surveys: American Statistical Association, June 11–14, Montréal, Canada. Available at: https://ww2.amstat.org/meetings/ices/2012/papers/302171.pdf (accessed March 2021)
14.
Groves, R.M., M.P. Couper, S. Presser, E. Singer, R. Tourangeau, G.P. Acosta, and L. Nelson. 2006. “Experiments in producing nonresponse bias.” Public Opinion Quarterly 70 (5): 720–736. DOI: https://doi.org/10.1093/poq/nfl036.
15.
Haraldsen, G. 2004. “Identifying and Reducing Response Burden in Internet Business Surveys.” Journal of Official Statistics 20 (2): 393–410. Available at: https://www.scb.se/contentassets/ca21efb41fee47d293bbee5bf7be7fb3/identifying-and-reducing-response-burdens-in-internet-business-surveys.pdf (accessed September 2021).
16.
Haraldsen, G. 2013. “Quality issues in business surveys.” Designing and Conducting Business Surveys , 83–125. John Wiley & Sons. DOI: https://doi.org/10.1002/9781118447895.ch03.
17.
Haraldsen, G., and J. Jones. 2007. “Web and paper questionnaires seen from the business respondent’s perspective.” In Proceedings of the Third International Conference for Establishments Surveys, JUNE 18–21, Montréal, Canada. Available at: https://ww2.amstat.org/meetings/ices/2007/proceedings/ICES2007-000259.PDF (accessed March 2021)
18.
Haraldsen, G., J. Jones, D. Giesen, and L.-C. Zhang. 2013. “Understanding and coping with response burden.” In Designing and Conducting Business Surveys , 219–252. John Wiley & Sons. DOI: https://doi.org/10.1002/9781118447895.ch06.
19.
Heerwegh, D., and G. Loosveldt. 2002. “An evaluation of the effect of response formats on data quality in web surveys.” Social Science Computer Review 20 (4): 471–484. DOI: https://doi.org/10.1177/089443902237323.
20.
Holbrook,A.L.,J.A.Krosnick,D.Moore,andR. Tourangeau. 2007. “Response order effects in dichotomous categorical questions presented orally: The impact of question and respondent attributes.” Public Opinion Quarterly 71 (3): 325–348. DOI: https://doi.org/10.1093/poq/nfm024.
21.
Jones, J., J. Rushbrooke, G. Haraldsen, T. Dale, and D. Hedlin. 2005. “Conceptualising total business survey burden.” In Survey Methodology Bulletin 55. UK Office for National Statistics, 1–10. Available at: http://www.ons.gov.uk/ons/guide-method/method-quality/survey-methodology-bulletin/smb-55/index.html (accessed March 2021)
22.
Peytchev, A. 2011. “Breakoff and Unit Nonresponse Across Web Surveys.” Journal of Official Statistics 27 (1): 33. Available at: https://www.scb.se/contentassets/ca21efb41-fee47d293bbee5bf7be7fb3/breakoff-and-unit-nonresponse-across-web-surveys.pdf (accessed September 2021).
23.
Presser, S., J. Blair, and T. Triplett. 1992. “Survey sponsorship, response rates, and response effects.” Social Science Quarterly 73 (5): 699–702. Available at: http://www.jstor.org/stable/42863089.
24.
Snijkers, G., G. Haraldsen, J. Jones, and D.K. Willimack. 2013. Designing and conducting business surveys . New York: Wiley.
25.
Tomaskovic-Devey, D., J. Leiter, and S. Thompson. 1994. “Organizational survey nonresponse.” Administrative Science Quarterly 39 (3): 439–457. DOI: https://doi.org/10.2307/2393298.
26.
Tourangeau, R., T. Yan, and H. Sun. 2019. “Who Can You Count On? Understanding The Determinants of Reliability.” Journal of Survey Statistics and Methodology 8 (5): 903–931. DOI: https://doi.org/10.1093/jssam/smz034.
27.
Willeboordse, A. 1997. “Minimizing response burden.” In Handbook on Design and Implementation of Business Surveys , edited by Willeboordse. 111–118. Available at: https://ec.europa.eu/eurostat/documents/3859598/5825949/CA-09-97-818-EN.PDF/f5ee3198-6fc0-4672-96a1-8fdb4a81ca93?version=1.0 (accessed March 2021).
28.
Yan, T., F.G. Conrad, R. Tourangeau, and M.P. Couper. 2010a. “Should I stay or should I go: The effects of progress feedback, promised task duration, and length of questionnaire on completing web surveys.” International Journal of Public Opinion Research 23 (2): 131–147. DOI: https://doi.org/10.1093/ijpor/edq046.
29.
Yan, T., R. Curtin, and M. Jans. 2010b. “Trends in Income Nonresponse Over Two Decades.” Journal of Official Statistics 26 (1): 145 – 164. Available at: https://www.scb.se/contentassets/ca21efb41fee47d293bbee5bf7be7fb3/trends-in-income-nonresponse-over-two-decades.pdf (accessed September 2021).
30.
Yan, T., S. Fricker, and S. Tsai. 2014. “The impact of response burden on data quality in a longitudinal survey.” In Proceedings of the International Total Survey Error Workshop, October 1–3, Washington, D.C. Available at: https://www.niss.org/events/2014-international-total-survey-error-workshop-itsew-2014 (accessed March 2021)
31.
Yan, T., S. Fricker, and S. Tsai. 2019. “Response burden: What is it and what predicts it?” In Advances in Questionnaire Design, Development, Evaluation and Testing , 193–212. John Wiley & Sons. DOI: https://doi.org/10.1002/9781119263685.ch8.