Restricted accessResearch articleFirst published online 2018-7
Are Prepaid Monetary Incentives Sufficient for Reducing Panel Attrition and Optimizing the Response Rate? An Experiment in the Context of a Multi-Wave Panel with a Sequential Mixed-Mode Design
In this contribution, we evaluate the short- and long-term effects of a prepaid cash incentive on young people’s cooperation and response rate in the fourth and fifth wave of a panel with sequential mixed-mode design (online questionnaire, CATI). Analyses are based on a survey experiment of students from randomly selected school classes of equal shares, which have participated in the third wave. Findings show that a monetary incentive has a direct and positive effect on the response rate in the fourth but not in the subsequent wave. However, the effect of the incentive is not persistent, since the effect weakens and fades away during the field phase and cannot be directly transferred to the second survey mode. As emphasized in the tailored design method (TDM), a monetary incentive can contribute to a shorter field phase and hence lower costs, but it is an insufficient instrument against panel attrition and the optimization of the retention rate when other strategies are disregarded.
AAPOR (The American Association for Public Opinion Research) (2015) Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys(eight edition). AAPOR.
2.
AndreßHJGolschKSchmidtAW (2013) Applied Panel Data Analysis for Economic and Social Surveys. Heidelberg: Springer.
3.
ArzheimerKKleinM (1998) Die Wirkung materieller Incentives auf den Rücklauf einer schriftlichen Befragung. ZA–Information43: 6–31.
4.
AxelrodR (1984) The Evolution of Cooperation. New York, NY: Basic Books.
5.
BanduraA (1986) Social Foundations of Thought and Action: A Social Cognitive Theory. Englewood Cliffs, NJ: Prentice–Hall.
6.
BanduraA (1995) Self-Efficacy in Changing Societies. Cambridge: Cambridge University Press.
7.
BeckerR (2006) Selective response to questions about delinquency. Quality & Quantity40(4): 483–498.
8.
BeckerRMehlkopG (2011) Effects of prepaid monetary incentives on mail survey response rates and on self–reporting about delinquency – empirical findings. Bulletin of Sociological Methodology111(1): 5–25.
9.
BergerR (2013) Altruistische Reziprozität. Theoretische Überlegungen und experimentelle Evidenz. Kölner Zeitschrift für Soziologie und Sozialpsychologie65(1): 31–48.
10.
BergerRRauhutH (2015) Reziprozität und Reputation. In: BraunNSaamNJ (eds) Handbuch Modellbildung und Simulation in den Sozialwissenschaften. Wiesbaden: Springer, 715–742.
11.
BestHWolfC (2012) Modellvergleich und Ergebnisinterpretation in Logit– und Probit–Regressionen. Kölner Zeitschrift für Soziologie und Sozialpsychologie64(2): 377–395.
12.
BlossfeldHPGolschKRohwerG (2007) Event History Analysis with Stata. New York, NY: Lawrence Erlbaum.
13.
BlossfeldHPRossbachHGvon MauriceJ (eds) (2011) Education as a Lifelong Process: The German National Educational Panel Study (NEPS). Wiesbaden: Springer.
14.
BosnjakMTutenTL (2003) Prepaid and promised incentives in web surveys. Social Science Computer Review21(2): 208–217.
15.
CastiglioniLPforrKKriegerU (2008) The effect of incentives on response rates and panel attrition: results of a controlled experiment. Survey Research Methods2(3): 151–158.
16.
CernatA (2015) The impact of mixing modes on reliability in longitudinal studies. Sociological Methods & Research44(3): 427–457.
17.
ChurchAH (1993) Estimating the effect of incentives on mail survey response rates: a meta-analysis. Public Opinion Quarterly57(1): 62–79.
18.
CouperMP (2000) Web surveys: a review of issues and approaches. Public Opinion Quarterly64(4): 464–494.
19.
CouperMP (2011) The future of modes of data collection. Public Opinion Quarterly75(5): 889–908.
20.
CouperMPBosnjakM (2010) Internet surveys. In: MarsdenPVWrightJD (eds) Handbook of Survey Research (second edition). Bingley: Emerald, 527–549.
21.
CouperMPKapteynASchonlauMWinterJ (2007) Noncoverage and nonresponse in an Internet survey. Social Science Research36(1): 131–148.
22.
De LeeuwED (2005) To mix or not to mix data collection modes in surveys. Journal of Official Statistics21(2): 233–255.
23.
De LeeuwEDHoxJJDillmanDA (2008) Mixed-mode surveys: when and why. In: De LeeuwEDHoxJJDillmanDA (eds) International Handbook of Survey Methodology. New York, NY: Lawrence Erlbaum, 299–316.
24.
De LeeuwEDHoxJJScherpenzeelA (2011) Mode effect or question wording? Measurement error in mixed mode surveys. In: JSM (Joint Statistical Meetings) Proceedings, Miami Beach, Florida, July 30–August 4, 2011 (Section on Survey Research Methods, ASA), 5959–5967. Alexandria, VA: American Statistical Association.
25.
De LeeuwEDMellenbergGJHoxJJ (1996) The influence of data collection method on structural models. Sociological Methods & Research24(4): 443–472.
26.
DiekmannA (2004) The power of reciprocity: fairness, reciprocity, and stakes in variants of the dictator game. Journal of Conflict Resolution48(4): 487–505.
27.
DiekmannAJannB (2001) Anreizformen und Ausschöpfungsquoten bei postalischen Befragungen. Eine Prüfung der Reziprozitätshypothese. ZUMA–Nachrichten25(48): 19–27.
28.
DiekmannAVossT (2008) Soziale Normen und Reziprozität – Die Bedeutung „sozialer” Motive für die Rational–Choice–Erklärung sozialer Normen. In: DiekmannAEichnerKSchmidtPVossT (eds) Rational Choice: Theoretische Analysen und empirische Resultate. Wiesbaden: Springer, 83–100.
29.
DillmanDA (2000) Mail and Internet Surveys. The Tailored Design Method. New York, NY: Wiley.
30.
DillmanDAPhelpsGTortoraRSwiftKKohrellJBerckJMesserBL (2009) Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the Internet. Social Science Research38(1): 1–18.
31.
DillmanDASmythJDChristianLM (2014) Internet, Phone, Mail, and Mixed–Mode Surveys: The Tailored Design Method. New York, NY: John Wiley & Sons.
32.
EdwardsMLDillmanDASmythJD (2014) An experimental test of the effects of survey sponsorship on Internet and mail survey response. Public Opinion Quarterly78(3): 734–750.
33.
EriksonRGoldthorpeJH (1992) The Constant Flux. A Study of Class Mobility in Industrial Societies. Oxford: Clarendon Press.
34.
FalkAFischbacherUA (2006) A theory of reciprocity. Games and Economic Behavior54(2): 293–315.
35.
FestingerL (1957) A Theory of Cognitive Dissonance. Stanford, CA: Stanford University Press.
36.
GajicACameronDHurleyJ (2012) The cost-effectiveness of cash versus lottery incentives for a web-based, stated-preference community survey. European Journal of Health Economics13(6): 789–799.
37.
GlauserD (2015) Berufsausbildung oder Allgemeinbildung. Wiesbaden: Springer.
38.
GlauserDBeckerR (2016) VET or general education? Effects of regional opportunity structures on educational attainment in German-speaking Switzerland. Educational Research in Vocational Education and Training8(8): 1–25.
39.
GöritzA (2004) The impact of material incentives on response quantity, response quality, sample composition, survey outcome, and cost in online access panels. International Journal of Market Research46(3): 327–345.
40.
GöritzA (2006) Incentives in web studies: methodological issues and a review. International Journal of Internet Science1(1): 58–70.
41.
GouldnerAW (1960) The norm of reciprocity: a preliminary statement. American Sociological Review25(2): 161–178.
42.
GreeneJSpeizerHWiitalaW (2008) Telephone and web: mixed-mode challenge. Health Research and Educational Trust43(1): 230–248.
43.
GrovesRMCialdiniRBCouperMP (1992) Understanding the decision to participate in a survey. Public Opinion Quarterly56(4): 475–495.
44.
GrovesRMPeytchevaE (2008) The impact of nonresponse rates on nonresponse bias: a meta-analysis. Public Opinion Quarterly72(2): 167–189.
45.
GrovesRMPresserSDipkoS (2004) The role of topic interest in survey participation decisions. Public Opinion Quarterly68(1): 2–31.
46.
GrovesRMSingerECorningA (2000) Leverage-saliency theory of survey participation: description and an illustration. Public Opinion Quarterly64(3): 299–308.
47.
HeerweeghD (2009) Mode differences between face-to--ace and web surveys: an experimental investigation of data quality and social desirability effects. International Journal of Public Opinion Research21(1): 111–121.
48.
HeerweghDLoosveldtG (2008) Face-to-face versus web surveying in a high-Internet-coverage population. Public Opinion Quarterly72(5): 836–846.
49.
HinzTAuspurgK (2011) Gruppenvergleiche bei Regressionen mit binären abhängigen Variablen – Probleme und Fehleinschätzungen am Beispiel von Bildungschancen im Kohortenverlauf. Zeitschrift für Soziologie40(1): 62–73.
50.
HomansGC (1961) Social Behavior: Its Elementary Forms. New York: Harcourt, Brace & World.
51.
HoxJJDe LeeuwEDJijmansEAO (2015) Measurement equivalence in mixed mode surveys. Frontiers in Psychology6(87): 1–11.
52.
KretschmerSMüllerG (2017) The Wave 6 NEPS adult study incentive experiment. Methods, Data, Analyses11(1): 7–28.
53.
KreuterFMüllerGTrappmannM (2010) Nonresponse and measurement error in employment research: making use of administrative data. Public Opinion Quarterly74(5): 880–906.
54.
KreuterFPresserSTourangeauR (2008) Social desirability bias in CATI, IVR, and web surveys: the effects of mode and question sensitivity. Public Opinion Quarterly72(5): 847–865.
55.
KrugGCarstensenJKriwyP (2017) Die richtige Mischung? Ein randomisiertes Experiment zur Datenqualität bei der Kombination von Telefon– und Onlineerhebung in der empirischen Sozialforschung. Zeitschrift für Soziologie46(2): 89–106.
56.
LaRoseRTsaiHS (2014) Completion rates and non-response error in online surveys: comparing sweepstakes and pre-paid cash incentives in studies of online behavior. Computers in Human Behavior34: 110–119.
57.
LaurieHLynnP (2009) The use of respondent incentives on longitudinal surveys. In: LynnP (ed.) Methodology of Longitudinal Surveys. Chichester: Wiley & Sons, 205–233.
58.
LippsO (2010) Effects of different incentives on attrition and fieldwork effort in telephone household panels surveys. Survey Research Methods4(2): 81–90.
59.
LongJS (1997) Regression models for categorical and limited dependent variables. Thousand Oaks, CA: SAGE Publications.
60.
MaussM (1984) Die Gabe: Form und Funktion des Austauschs in archaischen Gesellschaften (The Gift: The Form and Reason for Exchange in Archaic Societies). Frankfurt am Main: Suhrkamp.
61.
MehlkopGBeckerR (2007) Zur Wirkung monetärer Anreize auf die Rücklaufquote in postalischen Befragungen zu kriminellen Handlungen: Theoretische Überlegungen und empirische Befunde eines Methodenexperiments. Methods, data, analyses1(1): 5–24.
62.
MercerACaporasoACantorDTownsendR (2015) How much gets you how much? Monetary incentives and response rates in household surveys. Public Opinion Quarterly79(1): 105–129.
63.
MillarMMDillmanDA (2011) Improving response to web and mixed-mode surveys. Public Opinion Quarterly75(2): 249–269.
64.
PforrK, et al. (2015) Are incentive effects on response rates and nonresponse bias in large-scale, face-to-face surveys generalizable to Germany? Evidence from ten experiments. Public Opinion Quarterly79(3): 740–768.
65.
RyuECouperMPMaransRW (2005) Survey incentives: cash vs. in-kind; face–to–face vs. mail; response rate vs. nonresponse error. International Journal of Public Opinion Research18(1): 89–121.
66.
Sanchez–FernándenzJMunoz–LeivaFMontoros–RiosFJIbanez–ZapataJA (2010) An analysis of the effect of pre-incentives and post-incentives based on draws on response to web surveys. Quality & Quantity44(2): 357–373.
67.
ScherpenzeelAToepoelV (2012) Recruiting a probability sample for an online panel: effects of contact mode, incentives, and information. Public Opinion Quarterly76(3): 470–490.
68.
SchonlauMVan SoestAKapteynACouperMP (2009) Selection bias in web surveys and the use of propensity scores. Sociological Methods & Research37(3): 291–318.
69.
SchoutenBVan den BrakelJBuelensBVan der LaanJKlauschT (2013) Disentangling mode-specific selection and measurement bias in social surveys. Social Science Research42(6): 1555–1570.
70.
SingerEVan HoewykJGeblerNRaghunathanTMcGonagleK (1999) The effect of incentives on response rates in interviewer-mediated surveys. Journal of Official Statistics15(2): 217–230.
71.
SingerEYeC (2013) The use and effects of incentives in surveys. The Annals of the American Academy of Political and Social Science645(1): 112–141.
72.
SingerEVan HoewykJMaherMP (2000) Experiments with incentives in telephone surveys. Public Opinion Quarterly64(2): 171–188.
73.
SmythJDOlsonKMillarMM (2014) Identifying predictors of survey mode preference. Social Science Research48: 135–144.
74.
SzelényiKBryantANLindholmJA (2005) What money can buy: examining the effects of prepaid monetary incentives on survey response rates among college students. Educational Research and Evaluation11(4): 385–404.
75.
TREE (2016) Documentation on the first TREE cohort (TREE1), 2000–2016. Bern: TREE.
76.
VannieuwenhuyzeJTLoosveldtGMolenberghsG (2012) A method to evaluate mode effects on the mean and variance of a continuous variable in mixed-mode surveys. International Statistical Review80(2): 306–322.
77.
YoungJMO’HalloranAMcAulayCPirottaMForsdikeKStaceyICurrowD (2014) Unconditional and conditional incentives differentially improved general practitioners’ participation in an online survey: randomized controlled trial. Journal of Clinical Epidemiology68(6): 693–697.
