Scratch the Scratch-off: Testing Prepaid and Conditional Incentives with Postcard and Letter Invitations in a Web-push Design with an Address-based Sample

Abstract

We tested a novel extension to mailed invitations to a web-push survey, using a postcard invitation to deliver a scratch-off giftcode incentive similar to an instant-win lottery ticket. Scratch-off postcards were included as one of five conditions in randomized survey experiment varying two mailing types (letter and postcard) and three incentive types (prepaid cash, prepaid giftcodes, and conditional giftcodes). Invitations were sent to a sample of 17,808 addresses in Boston, Massachusetts, recruiting for a new online panel study of city residents. We report response rates and costs for each condition. Findings suggest that letters achieve higher response rates than postcards and are more cost effective overall. We also find that conditional incentives achieve higher response rates and are more cost effective, although conflating factors do not permit clear inferences. Notably, the novel scratch-off postcard condition achieved the lowest response rate and the highest costs per completed survey.

Get full access to this article

View all access options for this article.

References

American Association for Public Opinion Research (AAPOR) . 2016. Standard definitions: Final dispositions of case codes and outcome rates for surveys. 9th ed. Washington, DC: AAPOR.

Barnes

G. M.

Welte

J. W.

Tidwell

M. C. O.

Hoffman

J. H.

. 2011. Gambling on the lottery: Sociodemographic correlates across the lifespan. Journal of Gambling Studies 27:286–575.

Battaglia

M. P.

Dillman

D. A.

Frankel

M. R.

Harter

Buskirk

T. D.

McPhee

C. B.

DeMatteis

J. M.

Yancey

. 2016. Sampling, data collection, and weighting procedures for address-based sample surveys. Journal of Survey Statistics and Methodology 4:476–500.

Birnholtz

J. P.

Horn

D. B.

Finholt

T. A.

Sung Joo

. 2004. The effects of cash, electronic, and paper gift certificates as respondent incentives for a web-based survey of technologicaly sophisticated respondents. Social Science Computer Review 22:355–62.

Broughton

2011. Massachusetts sets pace in team lottery games. Sports Business Journal. Accessed September 10, 2019. https://www.sportsbusinessdaily.com/Journal/Issues/2011/02/07/Franchises/Lottery-tickets.aspx.

DeBell

Maisel

Edwards

Amsbary

Meldener

. 2020. Improving response rates with visible money. Journal of Survey Statistics and Methodology 8:821–31.

de Leeuw

E. D.

2018. Mixed-mode: Past, present, and future. Survey Research Methods 12:75–89.

Dillman

D. A.

2017. The promise and challenge of pushing respondents to the web in mixed-mode surveys. Survey Methodology 43:2–30.

Dillman

D. A.

Smyth

J. D.

Christian

L. M.

. 2014. Internet, phone, mail, and mixed-mode surveys. New York: Wiley.

10.

Dutwin

Buskirk

T. D.

. 2017. Apples to oranges or gala versus golden delicious? Comparing data quality of nonprobability Internet samples to low response rate probability samples. Public Opinion Quarterly 81:213–39.

11.

Finsen

Storeheier

A. H.

. 2006. Scratch lottery tickets are a poor incentive to respond to mailed questionnaires. BMC Medical Research Methodology 6:19.

12.

Göritz

A. S.

2006. Cash lotteries as incentives in online panels. Social Science Computer Review 24:445–59.

13.

Kaplowitz

M. D.

Lupi

Couper

M. P.

Thorp

. 2012. The effect of invitation design on web survey response rates. Social Science Computer Review 30:339–49.

14.

Lynn

2020. Evaluating push-to-web methodology for mixed-mode surveys using address-based samples. Survey Research Methods 14:19–30.

15.

Martinsson

Riedel

. 2015. Postal recruitment to a probability based web panel: Long term consequences for response rates, representativeness and costs. LORE methodological notes 2015:1. Gothenburg, Sweden: University of Gothenburg.

16.

Massachusetts State Lottery Commission . 2016. Information Packet. Accessed September 10, 2019. https://www.masslottery.com/lib/downloads/about/MSLC%20Info%20Pack%201216.pdf.

17.

Mercer

Caporaso

Cantor

Townsend

. 2015. How much gets you how much? Monetary incentives and response rates in household surveys. Public Opinion Quarterly 79:105–29.

18.

Messer

B. L.

Dillman

D. A.

. 2011. Surveying the general public over the internet using address-based sampling and mail contact procedures. Public Opinion Quarterly 75:429–57.

19.

Millar

M. M.

Dillman

D. A.

. 2011. Improving response to web and mixed-mode surveys. Public Opinion Quarterly 75:249–69.

20.

Olsen

Abelsen

Olsen

J. A.

. 2012. Improving response rate and quality of survey data with a scratch lottery ticket incentive. BMC Medical Research Methodology 12:52. https://doi.org/10.1186/1471-2288-12-52.

21.

Pew Research Center . 2019. Mobile technology and home broadband. Accessed July 10, 2020. https://www.pewresearch.org/internet/wp-content/uploads/sites/9/2019/06/PI_2019.06.13_Mobile-Technology-and-Home-Broadband_FINAL2.pdf.

22.

Shaw

M. J.

Beebe

T. J.

Jensen

H. L.

Adlis

S. A.

. 2001. The use of monetary incentives in a community survey: Impact on response rates, data quality, and cost. Health Services Review 35:1339–46.

23.

Shin

Johnson

T. P.

Rao

. 2012. Survey mode effects on data quality: Comparison of web and mail modes in a U.S. national panel study. Social Science Computer Review 30:212–28.

24.

Sinclair

O’Toole

Malawaraarachchi

Leder

. 2012. Comparison of response rates and cost-effectiveness for a community-based survey: Postal, internet, and telephone modes with generic or personalized recruitment approaches. BMC Medical Research Methodology 12:132.

25.

Singer

Groves

R. M.

Corning

A. D.

. 1999. Differential incentives: Beliefs about practices, perceptions of equity, and effects on survey participation. Public Opinion Quarterly 63:251–60.

26.

Singer

van Hoewyk

Maher

M. P.

. 2000. Experiments with incentives in telephone surveys. Public Opinion Quarterly 64:171–88.

27.

Singer

. 2013. The use and effects of incentives in surveys. The Annals of the American Academy of Political and Social Science 645:112–41.

28.

Smith

M. G.

Witte

Rocha

Basner

. 2019. Effectiveness of incentives and follow-up on increasing survey response rates and participation in field studies. BMC Medical Research Methodology 19:230.

29.

Sterrett

Malato

Benz

Tompson

English

. 2017. Assessing changes in coverage bias of web surveys in the United States. Public Opinion Quarterly 81:338–56.

30.

Tourangeau

2017. Mixing modes: Tradeoffs among coverage, nonresponse, and measurement error. In Total survey error in practice, eds. Biemer

P. P.

de Leeuw

E. D.

Eckman

Edwards

Kreuter

Lyberg

L. E.

Tucker

N. C.

West

B.T.

, 115–32. New York: Wiley.

31.

Tourangeau

Conrad

F. G.

Couper

M. P.

. 2013. The science of web surveys. New York: Oxford University Press.

32.

U.S. Census Bureau . 2018. 2018 ACS 5-year estimates data profiles, Table DP02. Accessed July 10, 2020. https://data.census.gov/cedsci/table?g=0400000US25_1600000US2507000&y=2018&d=ACS%205-Year%20Estimates%20Data%20Profiles&tid=ACSDP5Y2018.DP02.

33.

Volberg

R. A.

Williams

R. J.

Stanek

E. J.

Houpt

Zorn

Rodriguez-Monguio

. 2017. Gambling and problem gambling in Massachusetts: Results of a baseline population survey. Amherst: University of Massachusetts.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

4.30 MB