To examine the effects of lotteries on response behavior, five experiments were conducted in a nonprofit online panel. The type of lottery offered for study participation was a cash lottery, a voucher lottery, or a lottery of surprise gifts of either known or unknown value. The control group was not offered a lottery. Dependent measures were response, retention, and several facets of response quality. None of the lotteries significantly increased response, retention, or response quality. However, looking at effect sizes reveals a pattern across the five experiments that can inform and refine the practice of employing lotteries in nonprofit online panels. This pattern suggests that lotteries are more effective with panelists low in motivation than high in motivation.
BakerR.BlumbergS. J.BrickJ. M.CouperM. P.CourtrightM.DennisJ. M.. 2010. AAPOR report on online panels. Public Opinion Quarterly74:711–81. doi:10.1093/poq/nfq048.
3.
BošnjakM.BatinicB.. 2002. Understanding the willingness to participate in online-surveys. In Online social sciences, ed. BatinicB.ReipsU.-D.BošnjakM., 81–92. Seattle: Hogrefe & Huber.
4.
CallegaroM.DiSograC.. 2008. Computing response metrics for online panels. Public Opinion Quarterly 72:1008–32. doi:10.1093/poq/nfn065.
EysenbachG.2004. Improving the quality of web surveys: The checklist for reporting results from Internet e-surveys (cherries). Journal of Medical Internet Research6:e34. doi:10.2196/jmir.6.3.e34.
7.
GöritzA. S.2004. The impact of material incentives on response quantity, response quality, sample composition, survey outcome, and cost in online access panels. International Journal of Market Research46:327–45.
8.
GöritzA. S.2006a. Cash lotteries as incentives in online panels. Social Science Computer Review24:445–59. doi:10.1177/0894439305286127.
9.
GöritzA. S.2006b. Incentives in web studies: Methodological issues and a review. International Journal of Internet Science1:58–70.
10.
GöritzA. S.2008. The long-term effect of material incentives on participation in online panels. Field Methods20:211–25. doi:10.1177/1525822X08317069.
11.
GöritzA. S.2010. Using lotteries, loyalty points, and other incentives to increase participant response and completion. In Advanced methods for conducting online behavioural research, ed. GoslingS. D.JohnsonJ. A., 219–33. Washington, DC: American Psychological Association.
12.
GöritzA. S.WolffH.-G.. 2007. Lotteries as incentives in longitudinal web studies. Social Science Computer Review25:99–110. doi:10.1177/0894439306292268.
13.
HillD. H.WillisR. J.. 2001. Reducing panel attrition. The Journal of Human Resources36:417–38.
14.
HinesD. A.DouglasE. M.MahmoodS.. 2010. The effects of survey administration on disclosure rates to sensitive items among men: A comparison of an Internet panel sample with an RDD telephone sample. Computers in Human Behavior26:1327–35. doi:10.1016/j.chb.2010.04.006.
15.
HolbrookA. L.GreenM. C.KrosnickJ. A.. 2003. Telephone versus face-to-face interviewing of national probability samples with long questionnaires: Comparisons of respondent satisficing and social desirability response bias. Public Opinion Quarterly67:79–125.
16.
LauriolaM.LevinI. P.. 2001. Personality traits and risky decision-making in a controlled experimental task: An exploratory study. Personality and Individual Differences31:215–26. doi:10.1016/S0191-8869(00)00130-6.
17.
MarcusB.BošnjakM.LindnerS.PilischenkoS.SchützA.. 2007. Compensating for low topic interest and long surveys: A field experiment on nonresponse in web surveys. Social Science Computer Review25:372–83. doi:10.1177/0894439307297606.
18.
PorterS. R.WhitcombM. E.. 2003. The impact of lottery incentives on student survey response rates. Research in Higher Education44:389–407. doi:10.1023/A:1024263031800.
19.
TourangeauR.CouperM.ConradF.. 2004. Spacing, position, and order: Interpretive heuristics for visual features of survey questions. Public Opinion Quarterly68:368–93. doi:10.1039/poq/nfh035.
20.
TutenT. L.GalešićM.BošnjakM.. 2004. Effects of immediate versus delayed notification of prize draw results on response behaviour in web surveys: —An experiment. Social Science Computer Review22:377–84. doi:10.1177/0894439304265640.
