The Influence of an Up-Front Experiment on Respondents’ Recording Behaviour in Payment Diaries: Evidence from Germany

Abstract

In this article, we analyse the effect of an incentive experiment on German consumers’ recording behaviour on the basis of a one-week diary of their point-of-sale expenditure. Part of the experiment, which was carried out shortly before the consumers began filling in their payment diaries, involved consumers rolling a die with a chance of winning either EUR 20 or nothing, that is, they were randomly assigned an incentive. We ask whether respondents’ recording behaviour differs depending on whether individuals win or lose. We argue that winners attach a more positive feeling to the survey than losers and therefore show a stronger commitment to the diary. As the incentive experiment is part of a larger experiment to elicit respondents’ risk preferences, we also provide evidence on the effect of conducting up-front behavioural experiments in representative surveys. Our results indicate that the outcome of the lottery (rolling of the die) has an impact on the quantity of transactions recorded, but does not affect other aspects of respondents’ recording behaviour, such as item nonresponse or rounding. It also has a negligible impact on substantive measures, such as the cash share.

Keywords

Incentives risk experiments data quality

References

Awel, Y.M. and T.T. Azamahou. 2015. “Risk Preference or Financial Literacy? Behavioural Experiment on Index Insurance Demand.” UNU-MERIT Working Papers 2015-005, Maastricht. Available at: http://www.merit.unu.edu/publications/working-papers/abstract/?id=5626 (accessed August 2016).

Axhausen, K.W., A. Zimmermann, S. Schönfelder, G. Rindsfüser, and T. Haupt. 2002. “Observing the Rhythms of Daily Life: A Six-Week Travel Diary.” Transportation 29(2): 95–124. Doi: http://dx.doi.org/10.1023/A:1014247822322.

Bagnall, J., D. Bounie, K.P. Huynh, A. Kosse, T. Schmidt, S.D. Schuh, and H. Stix. 2016. “Consumer Cash Usage: A Cross-Country Comparison with Payment Diary Survey Data.” International Journal of Central Banking 12(4): 1–62. Available at: http://www.ijcb.org/journal/ijcb16q4a1.htm (accessed December 2016).

Biemer, P.P. and L.E. Lyberg. 2003. Introduction to Survey Quality. New Jersey: John Wiley & Sons.

Bonke, J. and P. Fallesen. 2010. “The Impact of Incentives and Interview Methods on Response Quantity and Quality in Diary- and Booklet-Based Surveys.” Survey Research Methods 4(2): 91–101. Doi: http://dx.doi.org/10.18148/srm/2010.v4i2.3614.

Borghans, L., J.J. Heckman, B.H. Golsteyn, and H. Meijers. 2009. “Gender Differences in Risk Aversion and Ambiguity Aversion.” Journal of the European Economic Association 7(2–3): 649–658. Doi: http://dx.doi.org/10.1007/s11238-016-9565-9.

Charness, G. and U. Gneezy. 2010. “Portfolio Choice and Risk Attitudes: An Experiment.” Economic Inquiry 48(1): 133–146. Doi: http://dx.doi.org/10.1111/j.1465-7295.2009.00219.x.

Charness, G., U. Gneezy, and A. Imas. 2013. “Experimental Methods: Eliciting Risk Preferences.” Journal of Economic Behavior & Organization 87: 43–51. Doi: http://dx.doi.org/10.1016/j.jebo.2012.12.023.

Davern, M., T.H. Rockwood, R. Sherrod, and S. Campbell. 2003. “Prepaid Monetary Incentives and Data Quality in Face-to-Face Interviews: Data from the 1996 Survey of Income and Program Participation Incentive Experiment.” Public Opinion Quarterly 67(1): 139–147. Doi: http://dx.doi.org/10.1086/346012.

10.

Deutsche Bundesbank. 2015. Payment Behaviour in Germany in 2014. Third Study on the Utilisation of Cash and Cashless Payment Instruments. Frankfurt am Main: Deutsche Bundesbank. Available at: http://www.bundesbank.de/Redaktion/EN/Downloads/Publications/Studies/payment_behaviour_in_germany_in_2014.pdf?__blob=publicationFile (accessed December 2015).

11.

Dohmen, T., A. Falk, D. Huffman, and U. Sunde. 2010. “Are Risk Aversion and Impatience Related to Cognitive Ability?” The American Economic Review 100(3): 1238–1260. Doi: http://dx.doi.org/10.1257/aer.100.3.1238.

12.

Dohmen, T., A. Falk, D. Huffman, U. Sunde, J. Schupp, and G.G. Wagner. 2011. “Individual Risk Attitudes: Measurement, Determinants and Behavioral Consequences.” Journal of the European Economic Association 9(3): 522–550. Doi: http://dx.doi.org/10.1111/j.1542-4774.2011.01015.x.

13.

Eckel, C.C. and P.J. Grossman. 2002. “Sex Differences and Statistical Stereotyping in Attitudes Toward Financial Risk.” Evolution and Human Behavior 23(4): 281–295. Doi: http://dx.doi.org/10.1016/S1090-5138(02)00097-1.

14.

Eckel, C.C. and P.J. Grossman. 2008. “Men, Women and Risk Aversion: Experimental Evidence.” In Handbook of Experimental Economics Results 1(113), edited by C.R. Plott and V.L. Smith. 1061–1073. New York: Elsevier.

15.

Fricker, S. and R. Tourangeau. 2010. “Examining the Relationship Between Nonresponse Propensity and Data Quality in Two National Household Surveys.” Public Opinion Quarterly 74(5): 934–955. Doi: http://dx.doi.org/10.1093/poq/nfq064.

16.

Godwin, R.K. 1979. “The Consequences of Large Monetary Incentives in Mail Surveys of Elites.” Public Opinion Quarterly 43(3): 378–387. Doi: http://dx.doi.org/10.1086/268528.

17.

Goetz, E.G., T.R. Tyler, and F.L. Cook. 1984. “Promised Incentives in Media Research: A Look at Data Quality, Sample Representativeness and Response Rates.” Journal of Marketing Research 21(2): 148–154. Doi: http://dx.doi.org/10.2307/3151697.

18.

Goldenberg, K., D. McGrath, and L. Tan. 2009. “The Effects of Incentives on the Consumer Expenditure Interview Survey.” Paper presented at the 64^th Annual Conference of the American Association for Public Opinion Research, May 14 – 17, 2009, Hollywood, Florida. Available at: https://stats.bls.gov/osmr/pdf/st090100.pdf (accessed December 2015).

19.

Goldenberg, K. and J. Ryan. 2009. “Evolution and Change in the Consumer Expenditure Surveys: Adapting Methodologies to Meet Changing Needs.” Paper presented at the NBER Summer Institute 2009 – Conference on Research on Income and Wealth, July 13, 2009, Cambridge, Massachusetts. Available at: https://www.bls.gov/cex/research_papers/pdf/Evolution-and-Change-in-the-Consumer-Expenditure-Surveys-Adapting-Methodologies-to-Meet-Changing-Needs.pdf (accessed December 2015).

20.

Göritz, A.S. 2004. “The Impact of Material Incentives on Response Quantity, Response Quality, Sample Composition, Survey Outcome, and Cost in Online Access Panels.” International Journal of Market Research 46(3): 327–345. Available at: https://www.mrs.org.uk/ijmr_article/article/79180 (accessed January 2016).

21.

Göritz, A.S. 2005. “Contingent versus Unconditional Incentives in WWW-Studies.” Metodolosky Zvezki 2(1): 1–14. Available at: http://mrvar.fdv.uni-lj.si/pub/mz/mz2.1/goritz.pdf (accessed January 2016).

22.

Groves, R.M., R.B. Cialdini and M.P. Couper. 1992. “Understanding the Decision to Participate in a Survey.” Public Opinion Quarterly 56(4): 475–495. Doi: http://dx.doi.org/10.1086/269338.

23.

Groves, R.M. 2004. Survey Errors and Survey Costs. New Jersey: John Wiley & Sons.

24.

Groves, R.M. and L. Lyberg. 2010. “Total Survey Error: Past, Present, and Future.” Public Opinion Quarterly 74(5): 849–879. Doi: http://dx.doi.org/10.1093/poq/nfq065.

25.

Halek, M. and J.G. Eisenhauer. 2001. “Demography of Risk Aversion.” Journal of Risk and Insurance 68(1): 1–24. Doi: http://dx.doi.org/10.2307/2678130.

26.

Hedlin, D., T. Dale, G. Haraldsen, and J. Jones (eds.). 2005. “Developing Methods for Assessing Perceived Response Burden. Research Report.” Stockholm: Statistics Sweden, Oslo: Statistics Norway, and London: Office for National Statistics. Available at: http://ec.europa.eu/eurostat/documents/64157/4374310/10-DEVELOPING-METHODS-FOR-ASSESSING-PERCEIVED-RESPONSE-BURDEN.pdf/1900efc8-1a07-4482-b3c9-be88ee71df3b (accessed November 2015).

27.

Hoffmeyer-Zlotnik, J.H. 2003. “New Sampling Designs and the Quality of Data.” In Developments in Applied Statistics. Methodoloski zvezki 19, edited by A. Ferligoj and A. Mrvar. 205–217. Ljubljana: FDV.

28.

James, J.M. and R. Bolstein. 1990. “The Effect of Monetary Incentives and Follow-Up Mailings on the Response Rate and Response Quality in Mail Surveys.” Public Opinion Quarterly 54(3): 346–361. Doi: http://dx.doi.org/10.1086/269211.

29.

Jones, J. 2012. “Response Burden: Introductory Overview Lecture.” Paper presented at the 4th International Establishment Surveys Conference, June 11–14, 2012, Montreal, Canada. Available at: https://ww2.amstat.org/meetings/ices/2012/papers/302289.pdf (accessed July 2016).

30.

Jonker, N. and A. Kosse. 2013. “Estimating Cash Usage: The Impact of Survey Design on Research Outcomes.” De Economist 161(1): 19–44. Doi: http://dx.doi.org/10.1007/s10645-012-9200-2.

31.

Laurie, H. and P. Lynn. 2009. “The Use of Respondent Incentives on Longitudinal Surveys.” In Methodology of Longitudinal Surveys, edited by P. Lynn, 205–234. Chichester, UK: John Wiley & Sons.

32.

Schwarz, N. and G.L. Clore. 1996. “Feelings and Phenomenal Experiences.” In Social Psychology: Handbook of Basic Principles, edited by E.T. Higgins and A. Kruglanski, 433–465. New York: Guilford.

33.

Sharp, L.M. and J. Frankel. 1983. “Respondent Burden: A Test of Some Common Assumptions.” Public Opinion Quarterly 47(1): 36–53. Doi: http://dx.doi.org/10.1086/268765.

34.

Shettle, C. and G. Mooney. 1999. “Monetary Incentives in US Government Surveys.” Journal of Official Statistics 15(2): 231–250. Available at: http://www.jos.nu/Articles/abstract.asp?article=152231 (accessed January 2016).

35.

Shuttleworth, F.K. 1931. “A Study of Questionnaire Technique.” Journal of Educational Psychology 22(9): 652–658. Doi: http://dx.doi.org/10.1037/h0074591.

36.

Singer, E., J. Van Hoewyk, and M. Maher. 2000. “Experiments with Incentives in Telephone Surveys.” Public Opinion Quarterly 64(2): 171–188. Doi: http://dx.doi.org/10.1086/317761.

37.

Tzamourani, P. and P. Lynn. 1999. “The Effect of Monetary Incentives on Data Quality – Results from the Social Attitudes Survey 1998 experiment.” CREST Working Paper 73, Oxford, UK. Available at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.573.5965&rep=rep1&type=pdf (accessed January 2016).

38.

Weisberg, H.F. 2005. The Total Survey Error Approach. A Guide To The New Science Of Survey Research. Chicago/London: The University of Chicago Press.

39.

Willimack, D.K., H. Schuman, B.E. Pennell, and J.M. Lepkowski. 1995. “Effects of a Prepaid Nonmonetary Incentive on Response Rates and Response Quality in a Face-to-Face Survey.” Public Opinion Quarterly 59(1): 78–92. Doi: http://dx.doi.org/10.1086/269459.