This article describes the results of an experiment designed to examine the impact of the use and amount of delayed unconditional incentives in a mixed mode (push to web) supplement on response rates, response mode, data quality, and sample bias. The supplement was administered to individuals who participate in the U.S. Panel Study of Income Dynamics, the longest running national household panel in the world. After 10 weeks of data collection, individuals who had not yet completed the interview were sent a final survey request and randomly assigned to one of three treatment conditions: no incentive, US$5, and US$10. The impact of the incentives on response rates and mode, effects on data quality, and sample bias are described. The implications for the use of incentives in mixed mode surveys and directions for future research are discussed.
Get full access to this article
View all access options for this article.
References
1.
AduaL.SharpJ. S.. 2010. Examining survey participation and response quality: The significance of topic salience and incentives. Survey Methodology36:95–109.
2.
CantorD.O’HareB.O’ConnorK.. 2008. The use of monetary incentives to reduce nonresponse in random digit dial telephone surveys. In Advances in telephone survey methodology, eds. LepkowskiJ. M.TuckerC.BrickJ. M.de LeeuwE.JapecL.LavrakasP. J.LinkM. W.SangsterR. L., 471–98. New York: Wiley.
3.
ChurchA. H.1993. Incentives in mailed surveys: A meta-analysis. Public Opinion Quarterly57:62–79.
4.
DillmanD. A.SmythJ. D.ChristianL. M.. 2009. Internet, mail and mixed-mode survey: The tailored design method, 3rd ed. Hoboken, NJ: John Wiley.
5.
EdwardsP.RobertsL.ClarkeM.DiGuiseppiC.PratapS.WentzR.KwanI.. 2002. Increasing response rates to postal questionnaires: Systematic review. British Medical Journal324:1183.
6.
FombyP.SastryN.McGonagleK. A.. 2015. Effectiveness of a time-limited incentive on participation by hard-to-reach respondents in a panel study. Paper presented at the 6th Conference of the European Survey Research Association, Reykjavik, Iceland, July 14–17.
7.
FumagalliL.LaurieH.LynnP.. 2013. Experiments with methods to reduce attrition in longitudinal surveys. Journal of the Royal Statistical Society: Series A (Statistics in Society)176:499–519.
8.
GoldenbergK. L.McGrathD.TanL.. 2009. The effects of incentives on the consumer expenditure interview survey. Joint Statistical Meetings Proceedings:5985–99.
9.
GöritzA. S.2006. Incentives in web studies: Methodological issues and a review. International Journal of Internet Science1:58–70.
10.
JäckleA.LynnP.. 2008. Respondent incentives in a multi-mode panel survey: Cumulative effects on nonresponse and bias. Survey Methodology34:105–17.
11.
LaurieH.LynnP.. 2009. The use of respondent incentives on longitudinal surveys. In Methodology of longitudinal surveys, ed. LynnP., 205–33. New York: Wiley.
12.
McGonagleK. A.SchoeniR. F.CouperM. P.. 2013. The effects of a between-wave incentive experiment on contact update and production outcomes. Journal of Official Statistics29:1–17.
13.
McGonagleK. A.SchoeniR. F.SastryN.FreedmanV. A.. 2012. The Panel Study of Income Dynamics: Overview, recent innovations, and potential for life course research. Longitudinal and Life Course Studies3:268–84.
14.
MedwayR. L.TourangeauR.. 2015. Response quality in telephone surveys: Do prepaid cash incentives make a difference?Public Opinion Quarterly79:524–43.
15.
MesserB. L.2012. Pushing households to the web: Results from webmail experiments using address based samples of the general public and mail contact procedures. PhD dissertation, Washington State University, Pullman.
16.
MesserB. L.DillmanD. A.. 2011. Surveying the general public over the Internet using address-based sampling and mail contact procedures. Public Opinion Quarterly75:429–57.
17.
MillarM.DillmanD. A.. 2011. Improving response to web and mixed-mode surveys. Public Opinion Quarterly75:249–69.
18.
ParsonsN. L.ManierreM. J.. 2014. Investigating the relationship among prepaid token incentives, response rates, and nonresponse bias in a web survey. Field Methods26:191–204.
19.
PatrickM. E.SingerE.BoydC. J.CranfordJ. A.McCabeS. E.. 2013. Incentives for college student participation in web-based substance use surveys. Addictive Behaviors38:1710–14.
20.
PetroliaD. R.BhattacharjeeS.. 2009. Revisiting incentive effect: Evidence from a random sample mail survey on consumer preferences for fuel ethanol. Public Opinion Quarterly73:537–50.
21.
RodgersW.2002. Size of incentive effects in a longitudinal study. Paper presented at the American Association for Public Opinion Research, St. Petersburg, FL, May 16–19.
22.
RyuE.CouperM. P.MaransR. W.. 2006. Survey incentives: Cash vs. in-kind; face-to-face vs. mail; response rate vs. nonresponse error. International Journal of Public Opinion Research18:89–106.
23.
SAS Institute. 2011. Base SAS® 9.3 Procedures Guide. Cary, NC: SAS Institute.
24.
SchoeniR. F.StaffordF. P.McGonagleK. A.AndreskiP.. 2013. Response rates in national panel surveys. The Annals of the Academy of Political and Social Science645:60–87.
25.
SingerE.GeblerN.RaghunathanT.Van HoewykJ.McGonagleK.. 1999. The effect of incentives in telephone and face-to-face surveys. Journal of Official Statistics15:217–30.
26.
SingerE.YeC.. 2013. The use and effects of incentives in surveys. The Annals of the American Academy of Political and Social Science645:112–41.
27.
SternM. J.AdamsA.ElsasserS.. 2009. Digital inequality and place: The effects of technological diffusion on Internet proficiency and usage across rural, suburban, and urban counties. Sociological Inquiry79:391–417.
28.
SternM. J.BilgenI.DillmanD. A.. 2014. The state of survey methodology challenges, dilemmas, and new frontiers in the era of the tailored design. Field Methods26:284–301.
