Over the past few decades, the survey industry has experienced a steady decline in response rates, which has posed numerous challenges for researchers, most notably concerns about nonresponse bias. We present results from three studies conducted in an attempt to increase response rates and reduce nonresponse bias for a U.S. national household survey. We examined design features related to mode (i.e., mail-only vs. web-push), incentive, reminder type, nonresponse follow-up, and messaging. Results showed that a combination of a mail-only approach and the use of US$2 prepaid incentives increased response rates. However, using incentives did not bring into the respondent pool sample members who were less likely to respond. Additionally, we found that the use of a seventh follow-up contact only slightly increased response rates. Finally, we found that the use of a friendly message not only suppressed response rates but also increased potential nonresponse bias for sample members aged 35–48.
Get full access to this article
View all access options for this article.
References
1.
BrickJ. M.AndrewsW. R.BrickP. D.KingH.MathiowetzN. A.StokesL.. 2012. Methods for improving response rates in two-phase mail surveys. Survey Practice5:3093.
2.
BrickJ. M.TourangeauR.. 2017. Responsive survey designs for reducing nonresponse bias. Journal of Official Statistics33:735–52.
3.
BrickJ. M.WilliamsD.. 2013. Explaining rising nonresponse rates in cross-sectional surveys. The Annals of the American Academy of Political and Social Science645:36–59.
4.
ComptonE.BentleyM.DuschG.. 2012. Census nonresponse follow-up contact strategy experiment. Presented at the Joint Statistical Meetings, San Diego, CA, July 28–August 2.
5.
CurtinR.PresserS.SingerE.. 2000. The effects of response rate changes on the index of consumer sentiment. Public Opinion Quarterly64:413–28.
6.
CzajkaJ. L.BeylerA.. 2016. Background paper-declining response rates in federal surveys: Trends and implications. Mathematica Policy Research1:1–86.
7.
DillmanD. A.2000. Mail and telephone surveys: The tailored design method. New York: John Wiley & Sons.
8.
DillmanD. A.ClarkJ. R.SinclairM. D.. 1995. How prenotice letters, stamped return envelopes and reminder postcards affect mailback response rates for census questionnaires. Survey Methodology21:159–66.
9.
DillmanD. A.SingerE.ClarkJ. R.TreatJ. B.. 1996. Effects of benefits appeals, mandatory appeals, and variations in statements of confidentiality on completion rates for census questionnaires. Public Opinion Quarterly60:376–89.
10.
DillmanD. A.SmythJ. D.ChristianL. M.. 2014. Internet, phone, mail, and mixed-mode surveys: The tailored design method. Hoboken, NJ: John Wiley & Sons.
11.
DillmanD. A.WestK. K.ClarkJ. R.. 1994. Influence of an invitation to answer by telephone on response to census questionnaires. Public Opinion Quarterly58:557–68.
12.
DoddD. K.MarkwieseB. J.. 1987. Survey response rate as a function of personalized signature on cover letter. The Journal of Social Psychology127:97–98.
13.
DolsonD.2013. 2011 Canadian census. Paper presented at the 5th International Workshop on Internet Survey and Survey Methodology, Daejeon, Korea, September 11–12.
14.
DykemaJ.JaquesK.CyffkaK.AssadN.HammersR. G.ElverK.MaleckiK. C.StevensonJ.. 2015. Effects of sequential prepaid incentives and envelope messaging in mail surveys. Public Opinion Quarterly79:906–31.
15.
EdwardsM.DillmanD. A.SmythJ.. 2014. An experimental test of the effects of survey sponsorship on Internet and mail survey response. Public Opinion Quarterly78:734–50.
16.
EdwardsP.RobertsI.ClarkeM.DiGuiseppiC.PratapS.WentzR.KwanI.. 2002. Increasing response rates to postal questionnaires: Systematic review. British Medical Journal324:1183.
17.
FinnA.GendallP.HoekJ.. 2004. Two attempts to increase the response to a mail survey. Marketing Bulletin15:1–5.
18.
FowlerF. J.Jr. 2013. Survey research methods. Thousand Oaks, CA: Sage.
19.
FuchsM.BossertD.StukowskiS.. 2013. Response rate and nonresponse bias-impact of the number of contact attempts on data quality in the European Social Survey. Bulletin of Sociological Methodology/Bulletin de Méthodologie Sociologique117:26–45.
20.
GoldenbergK. L.McGrathD.TanL.. 2009. The effects of incentives on the Consumer Expenditure Interview Survey. Presented at the 64th American Association for Public Opinion Research Annual Conference, Ft. Lauderdale, FL, May 14–17.
21.
GrovesR. M.2006. Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly70:646–75.
22.
GrovesR. M.2011. Three eras of survey research. Public Opinion Quarterly75:861–71.
23.
GrovesR. M.CouperM. P.PresserS.SingerE.TourangeauR.AcostaG. P.NelsonL.. 2006. Experiments in producing nonresponse bias. International Journal of Public Opinion Quarterly70:720–36.
24.
GrovesR. M.KahnR.. 1979. Surveys by telephone; A national comparison with personal interview. New York: Academic Press.
25.
GrovesR. M.PeytchevaE.. 2008. The impact of nonresponse rates on nonresponse bias: A meta-analysis. Public Opinion Quarterly72:167–89.
26.
GrovesR. M.PresserS.DipkoS.. 2004. The role of topic interest in survey participation decisions. Public Opinion Quarterly68:2–31.
27.
GrovesR. M.PresserS.TourangeauR.WestB.CouperM. P.SingerE.ToppeC.. 2012. Support for the survey sponsor and nonresponse bias. Public Opinion Quarterly76:512–24.
28.
GrovesR. M.SingerE.CorningA.. 2000. Leverage-saliency theory of survey participation: Description and an illustration. Public Opinion Quarterly64:299–308.
29.
JamesJ. M.BolsteinR.. 1990. The effect of monetary incentives and follow-up mailings on the response rate and response quality in mail surveys. Public Opinion Quarterly54:346–61.
30.
JansM.ParkR.RauchJ.GrantD.EdwardsS.. 2015. Logos and inserts can reduce survey return rates: An experiment in California. Survey Practice8:1–11. https://doi.org/10.29115/SP-2015-0017 (accessed November 28, 2017).
31.
McLeanS.PaxtonS.MasseyR.MondJ.RodgersB.HayP.. 2014. Prenotification but not envelope teaser increased response rates in a bulimia nervosa mental health literacy survey: A randomized controlled trial. Journal of Clinical Epidemiology67:870–76.
32.
MedwayR. L.FultonJ.. 2012. When more gets you less: A meta-analysis of the effect of concurrent web options on mail survey response rates. Public Opinion Quarterly76:733–46.
33.
MedwayR. L.TourangeauR.. 2015. Response quality in telephone surveys: Do prepaid cash incentives make a difference?Public Opinion Quarterly79:524–43.
34.
MercerA.CaporasoA.CantorD.TownsendR.. 2015. How much gets you how much? Monetary incentives and response rates in household surveys. Public Opinion Quarterly79:105–29.
35.
MesserB. L.DillmanD. A.. 2011. Surveying the general public over the Internet using address-based sampling and mail contact procedures. Public Opinion Quarterly75:429–57.
36.
RobertsP. J.RobertsC.SibbaldB.TorgersonD. J.. 2000. The effect of a direct payment or a lottery on questionnaire response rates: A randomised controlled trial. Journal of Epidemiology & Community Health54:71–72.
37.
SingerE.2002. The use of incentives to reduce nonresponse in household surveys. In Survey nonresponse, eds. GrovesR.DillmanD.EltingeJ.LittleR., 163–77. New York: Wiley.
38.
SingerE.BossarteR. M.. 2006. Incentives for survey participation: When are they “coercive”?American Journal of Preventive Medicine31:411–18.
39.
SingerE.Van HoewykJ.GeblerN.McGonagleK.. 1999. The effect of incentives on response rates in interviewer-mediated surveys. Journal of Official Statistics15:217.
40.
SingerE.Van HoewykJ.MaherM. P.. 2000. Experiments with incentives in telephone surveys. Public Opinion Quarterly64:171–88.
41.
SingerE.YeC.. 2013. The use and effects of incentives in surveys. The Annals of the American Academy of Political and Social Science645:112–41.
42.
SmithK. T.2011. Digital marketing strategies that millennials find appealing, motivating, or just annoying. Journal of Strategic Marketing19:489–99.
43.
TancretoJ.ZelenakM. F.DavisM.RuiterM.MatthewsB.. 2012. 2011 American Community Survey Internet tests: Results from first test in April 2011. Final Report. Washington, DC: U.S. Census Bureau.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
