Previous research has shown that increasing the size of incentives can increase response rates for probability-based, cross-sectional surveys. However, the effects of incentives on web panels have not been extensively studied. We sought to answer the question: What is the effect of larger, postpaid incentives on (1) response, (2) data quality, and (3) nonresponse bias for individuals in a web panel? We analyzed data from the 2015 and 2016 National Internet Flu Survey, a survey that uses the GfK KnowledgePanel® as its sampling frame. We compare panel members who received a postpaid, standard 1,000-point (the equivalent of US$1) incentive in 2015 to panelists who received a larger, 5,000-point (the equivalent of US$5) incentive in 2016. We found that larger incentives were associated with increased interview completion rates with minimal impact on data quality or bias.
Get full access to this article
View all access options for this article.
References
1.
BakerR.BlumbergS. J.BrickJ. M.CouperM. P.CourtrightM.DennisJ. M.DillmanD., et al.2010. Research synthesis: AAPOR report on online panels. Public Opinion Quarterly74:711–81.
2.
BargeS.GehlbachH.. 2012. Using the theory of satisficing to evaluate the quality of survey data. Research in Higher Education53:182–200.
3.
BlomA. G.GathmannC.KriegerU.. 2015. Setting up an online panel representative of the general population. Field Methods27:391–408.
4.
BrickJ. M.BrickP. D.DipkoS.PresserS.TuckerC.YuanY.. 2007. Cell phone survey feasibility in the U.S.: Sampling and calling cell numbers versus landline numbers. Public Opinion Quarterly71:23–39.
5.
BrickJ. M.TourangeauR.. 2017. Responsive survey designs for reducing nonresponse bias. Journal of Official Statistics33:735–52.
6.
CallegaroM.BakerR.BethlehemJ.GoritzA. S.KrosnickJ. A.LavrakaP., eds. 2014. Online panel research: A data quality perspective. London: John Wiley.
7.
CallegaroM.DisograC.. 2008. Computing response metrics for online panels. Public Opinion Quarterly72:1008–32.
8.
DillmanD. A.SmythJ. D.ChristianJ. M.. 2014. Internet, mail, and mixed-mode surveys: The tailored design method, 4th ed. Hoboken, NJ: John Wiley.
9.
EnglishN.KennelT.BuskirkT.HarterR.. 2018. The construction, maintenance, and enhancement of address-based sampling frames. Journal of Survey Statistics and Methodology7:66–92.
10.
FoxR. J.CraskM. R.JonghoonK.. 1988. Mail survey response rate: A meta-analysis of selected techniques for inducing response. Public Opinion Quarterly52:467.
GoritzA. S.2006. Incentives in web studies: Methodological issues and a review. International Journal of Information Security1:58–70.
13.
HalpernS. D.UbelP. A.BerlinJ. A.AschD. A.. 2002. Randomized trial of $5 versus $10 monetary incentives, envelope size, and candy to increase physician response rates to mailed questionnaires. Medical Care40:834–39.
14.
HsuJ. W.SchmeiserM. D.HaggertyC.NelsonS.. 2017. The effect of large monetary incentives on survey completion. Public Opinion Quarterly81:736–47.
15.
JäckleA.LynnP.. 2008. Respondent incentives in a multi-mode panel survey: Cumulative effects on nonresponse and bias. Survey Methodology34:105–17.
LuPeng-JunSrivastavA.SantibanezT. A.StringerC. M.BostwickM.DeverJ. A.KurtzM. S.WilliamsW. W.. 2017. Knowledge of influenza vaccination recommendation and early vaccination uptake during the 2015–16 season among adults aged ≥18 years—United States. Vaccine35:4346–54.
18.
MercerA.CaporasoA.CantorD.TownsendR.. 2015. How much gets you how much? Monetary incentives and response rates in household surveys. Public Opinion Quarterly79:105–29.
19.
MillerJ.2008. Burke panel quality R and D. Cincinnati, OH: Burke.
20.
ShettleC.MooneyG.. 1999. Monetary incentives in U.S. government surveys. Journal of Official Statistics15:231–50.
21.
SingerE.KulkaR. A.. 2002. Paying respondents for survey participation. In Studies of welfare populations: Data collection and research issues, eds. PloegM. V.MoffittR. A.CitroC. F., 105–28. Washington, DC: National Academy Press.
22.
SingerE.van HoewykJ.GeblerN.RaghunathanT.McGonagleK.. 1999. The effect of incentives on response rates in interviewer-mediated surveys. Journal of Official Statistics15:217–30.
23.
SingerE.van HoewykJ.MaherM. P.. 1998. Does the payment of incentives create expectation effects?Public Opinion Quarterly62:152.
24.
SingerE.YeC.. 2013. The use and effects of incentives in surveys. The Annals of the American Academy of Political and Social Science645:112–41.
25.
ToepoelV.2012. Effects of incentives in surveys. In Handbook of survey methodology for the social sciences, eds. GideonL., 209–23. New York: Springer.
26.
TrussellN.LavrakasPaul J.. 2004. The influence of incremental increases in token cash incentives on mail survey response: Is there an optimal amount?Public Opinion Quarterly68:349–67.
27.
UnangstJ.AmayaA.SandersH.HowardJ.FerrellA.KaronS.DeverJ. A.. 2019. A process for decomposing total survey error in probability and nonprobability surveys: A case study comparing health statistics in U.S. Internet panels. Journal of Survey Statistics and Methodology. doi: 10.1093/jssam/smz040.
28.
VonkT.van OsenbruggenR.WilliamsP.. 2006. The effects of panel recruitment and management on research results: A study across 19 online panels. Proceedings of the ESOMAR Panel Research, ESOMAR, Amsterdam, the Netherlands.
29.
WatsonN.LeissouE.GuyerH.WoodenM.. 2018. Best practices for panel maintenance and retention. In Advances in comparative survey methods: Multinational, multiregional, and multicultural contexts (3MC), eds. JohnsonT. P.PennellB.-E.StoopI. A. L.DorerB., 583–96. Hoboken, NJ: John Wiley.
