Falling participation rates is one of the most significant challenges facing survey research today. To curb this negative trend, scholars have searched for factors that can increase and decrease citizens’ willingness to participate in surveys. In this article, we investigate the timing effects of survey invitation e-mails on participation rates in a university-based online panel with members of the Swedish public. Through two large-scale experimental studies, we examine whether the day of week (N = 11,294) and time of day (N = 47,279) for sending out survey invitations impact participation rates. We also ask respondents when they prefer to answer surveys. We find that the timing of survey invitations affects participation rates, however, the effects are small, short-lived, and even out within a week. We also find that the effects of timing vary by employment status and age. The results have implications for scholars and practitioners who utilize online panels for web surveys. When quick answers are important, there may be some limited gains of tailoring the timing of the survey invitation to different individuals. In surveys with more extended field periods, however, such efforts seem less warranted.
BennettS.MatonK.KervinL. (2008). The “digital natives” debate: A critical review of the evidence. British Journal of Educational Technology, 39, 775–786. doi:10.1111/j.1467-8535.2007.00793.x
BethlehemJ.CobbenF.SchoutenB. (2011). Handbook of nonresponse in household surveys. Hoboken, NJ: Wiley.
4.
BrüggenE.DholakiaU. (2010). Determinants of participation and response effort in web panel surveys. Journal of Interactive Marketing, 24, 239–250. doi:10.1016/j.intmar.2010.04.004
5.
ChangL.KrosnickJ. (2009). National surveys via RDD telephone interviewing versus the internet: Comparing sample representativeness and response quality. Public Opinion Quarterly, 73, 641–678. doi:10.1093/poq/nfp075
6.
CheongP. (2008). The young and the techless? Investigating Internet use and problem solving behaviors of young adults in Singapore. New Media and Society, 10, 771–791. doi:10.1177/1461444808094356
7.
CohenJ. (1992). Statistical power analysis current directions. Psychological Science, 1, 98–101.
8.
CouperM. (2000). Web surveys: A review of issues and approaches. Public Opinion Quarterly, 64, 464–494.
9.
DeutskensE.De RuyterK.WetzelsM.OosterveldP. (2004). Participation rate and response quality of internet-based surveys: An experimental study. Marketing Letters, 15, 21–36. doi:10.1023/b:mark.0000021968.86465.00
10.
DillmanD. (2011). Mail and internet surveys: The tailored design method (2nd ed.). Hoboken, NJ: John Wiley.
11.
DrèzeX.BonfrerA. (2008). An empirical investigation of the impact of communication timing on customer equity. Journal of Interactive Marketing, 22, 36–50. doi:10.1002/dir.20105
12.
DurrantG.D’ArrigoJ.SteeleF. (2011). Using paradata to predict best times of contact, conditioning on household and interviewer influences. Journal of the Royal Statistical Society: Series A (Statistics in Society), 174, 1029–1049. doi:10.1111/j.1467-985X.2011.00715.x
FanW.YanZ. (2010). Factors affecting participation rates of the web survey: A systematic review. Computers in human behavior, 26, 132–139. doi:10.1016/j.chb.2009.10.015
17.
FaughtK.WhittenD.GreenK. (2004). Doing survey research on the internet: Yes, timing does matter. Journal of Computer Information Systems, 44, 26–34. doi:10.1080/08874417.2004.11647579
FrippiatD.MarquisN. (2010). Web surveys in the social sciences: An overview. Population-E, 65, 252–285. doi:10.3917/pope.1002.0285
20.
GrovesR. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70, 646–675. doi:10.1093/poq/nfl033
21.
GrovesR.PeytchevaE. (2008). The impact of nonresponse rates on nonresponse bias: A meta-analysis. Public Opinion Quarterly, 72, 167–189. doi:10.1093/poq/nfn011
22.
GrovesR.PresserS.DipkoS. (2004). The role of topic interest in survey participation decisions. Public Opinion Quarterly, 68, 2–31. doi:10.1093/poq/nfh002
23.
GöritzA. (2004). Recruitment for online access panels. International Journal of Market Research, 46, 411–426.
24.
GöritzA.MoserK. (2000). Repräsentativität im online panel. Der Markt, 39, 156–162. doi:10.1007/BF03033955
25.
GöritzA.ReinholdN.BatinicB. (2002). Online panels. In BatinicB.ReipsU.-D.BosnjakM. (Eds.), Online Social Sciences (pp. 27–47). Göttingen, DE: Hogrefe & Huber.
26.
HahnU.HarrisA. (2014). What does it mean to be biased: Motivated reasoning and rationality. In RossB. (Ed.), Psychology of Learning and Motivation (pp. 41–102). Cambridge, MA: Academic Press.
27.
IlievaJ.BaronS.HealeyN. (2002). Online surveys in marketing research: Pros and cons. International Journal of Market Research, 44, 361–376.
28.
MannukkaJ. (2007). Characteristics of early adopters in mobile communications markets. Marketing Intelligence & Planning, 25, 719–731. doi:10.1108/02634500710834188
KeuschF. (2013). The role of topic interest and topic salience in online panel web surveys. International Journal of Market Research, 55, 59–80. doi:10.2501/ijmr-2013-007
31.
KeuschF. (2015). Why do people participate in web surveys? Applying survey participation theory to internet survey data collection. Management Review Quarterly, 65, 183–216. doi:10.1007/s11301-014-0111-y
32.
KreuterF.MüllerG. (2015). A note on improving process efficiency in panel surveys with paradata. Field Methods, 27, 55–65. doi:10.1177/1525822X14538205
33.
KreuterF.PresserS.TourangeauR. (2008). Social desirability bias in CATI, IVR, and web surveys: The effects of mode and question sensitivity. Public Opinion Quarterly, 72, 847–865. doi:10.1093/poq/nfn063
34.
LewisT.HessK. (2017). The effect of alternative e-mail contact timing strategies on response rates in a self-administered web survey. Field Methods, 29, 351–364. doi:10.1177/1525822x17715865
35.
LippsO. (2012). A note on improving contact times in panel surveys. Field Methods, 24, 95–111. doi:10.1177/1525822X11417966
36.
LuitenA.SchoutenB. (2013). Tailored fieldwork design to increase representative household survey response: An experiment in the survey of consumer satisfaction. Journal of the Royal Statistical Society: Series A (Statistics in Society), 176, 169–189. doi:10.1111/j.1467-985X.2012.01080.x
37.
PaoloA.BonaminioG.GibsonC.PartridgeT.KallailK. (2000). Participation rate comparisons of e-mail-and mail-distributed student evaluations. Teaching and learning in medicine, 12, 81–84. doi:10.1207/S15328015TLM1202_4
PrenskyM. (2001). Digital natives, digital immigrants, part 1. On the Horizon, 9, 1–6.
40.
RogelbergS.StantonJ. (2007). Introduction: Understanding and dealing with organizational survey nonresponse. Organizational Research Methods, 10, 195–209. doi:10.1177/1094428106294693
41.
SauermannH.RoachM. (2013). Increasing web survey participation rates in innovation research: An experimental study of static and dynamic contact design features. Research Policy, 42, 273–286. doi:10.1016/j.respol.2012.05.003
42.
SaxL.GilmartinS.BryantA. (2003). Assessing participation rates and nonresponse bias in web and paper surveys. Research in Higher Education, 44, 409–432. doi:10.1023/A:102423291
SheehanK. (2001). E-mail survey participation rates: A review. Journal of Computer-Mediated Communication, 6, 0–0. doi:10.1111/j.1083-6101.2001.tb00117.x
45.
ShinnG.BakerM.BriersG. (2007). Response patterns: Effect of day of receipt of an e-mailed survey instrument on response rate, response time, and response quality. Journal of Extension, 45(2).
46.
SmythJ.DillmanD.ChristianL.McBrideM. (2009). Open-ended questions in web surveys: Can increasing the size of answer boxes and providing extra verbal instructions improve response quality?Public Opinion Quarterly, 73, 325–337. doi:10.1093/poq/nfp029
47.
TaberC.LodgeM. (2006). Motivated skepticism in the evaluation of political beliefs. American Journal of Political Science, 50, 755–769. doi:10.1111/j.1540-5907.2006.00214.x
48.
van DeursenA.van DijkJ. (2010). Measuring internet skills. International Journal of Human-Computer Interaction, 26, 891–916. doi:10.1080/10447318.2010.496338
49.
WagnerJ. (2013). Adaptive contact strategies in telephone and face-to-face surveys. Survey Research Methods, 7, 45–55. doi:10.18148/srm/2013.v7i1.5037
50.
WansinkB. (2001). Editorial: The power of panels. Journal of Database Marketing & Customer Strategy Management, 8, 190–194. doi:10.1057/palgrave.jd
51.
WeeksM.JonesB.FolsomJ.BenrudC. (1980). Optimal times to contact sample households. Public Opinion Quarterly, 44, 101–114. doi:10.1086/268569
52.
WeeksM.KulkaR.PiersonS. (1987). Optimal call scheduling for a telephone survey. Public Opinion Quarterly, 51, 540–549. doi:10.1086/269056
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
