This paper investigates the effects on response and breakoff rates in Web surveys of e-mail invitation design elements including: subject line, estimated survey duration, and invitation length. The analysis is focused not only on the main effects, but also on an issue which has not yet been studied systematically: interaction effects between these design elements. A factorial complete block design Web experiment – among students, faculty, and administrative staff at the Higher School of Economics in Russia – varying the estimated length of the survey (10 vs. 20 minutes), subject line (formal “monitoring” vs. informal “help” request), and invitation length (short vs. long) was conducted. We discuss the results of the experiment and argue that we should not ignore interaction effects between design elements to understand response in Web surveys more thoroughly.
BosnjakMTutenTL (2003) Prepaid and Promised Incentives in Web Surveys. Social Science Computer Review21(2): 208–17.
3.
CannellCFFowlerFJ (1963) A Study of the Reporting of Visits to Doctors in the National Health Survey. Research Report. Ann Arbor: Survey Research Center, University of Michigan.
4.
CookCHeathFThompsonRL (2000) A Meta-Analysis of Response Rates in Web- or Internet-Based Surveys. Educational and Psychological Measurement60(6): 821–36.
5.
CrawfordSDCouperMPLamiasMJ (2001) Web Surveys. Perceptions of Burden. Social Science Computer Review19(2): 146–62.
6.
DillmanDASmythJDChristianLM (2009) Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method. New York: John Wiley and Sons.
7.
GalesicMBosnjakM (2009) Effects of Questionnaire Length on Participation and Indicators of Response Quality in a Web Survey. Public Opinion Quarterly73(2): 349–60.
8.
GöritzAS (2006) Incentives in Web Studies: Methodological Issues and a Review. International Journal of Internet Science1(1): 58–70.
9.
GueguenN (2003) Help on the Web: The Effect of the Same First Name between the Sender and the Receptor in a Request Made by E-Mail. Psychological Record53: 459–66.
10.
JacksonDNMessickS (1958) Content and Style in Personality Assessment. Psychological Bulletin55(4): 243–252.
11.
JoinsonANReipsUD (2007) Personalized Salutation, Power of Sender and Response Rates to Web-Based Surveys. Computers in Human Behavior23(3): 1372–83.
12.
HendersonV (2011) Increasing (or Decreasing) Response Rate by Changing the Subject of Email Invitations. Paper presented at AAPOR Annual Conference, 12-15 May 2011. Phoenix, USA.
13.
HeerweghDLoosveldtG (2006) An Experimental Study on the Effects of Personalization, Survey Length Statements, Progress Indicators, and Survey Sponsor Logos in Web Surveys. Journal of Official Statistics22(2): 191–210.
14.
KaplowitzMDLupiFCouperMPThorpL (2012) The Effect of Invitation Design on Web Survey Response Rates. Social Science Computer Review30(3): 339–49.
15.
KeuschF (2012) How to Increase Response Rates in List-Based Web Survey Samples. Social Science Computer Review30: 380–88.
16.
KlofstadCABoulianneSBassonD (2008) Matching the Message to the Medium: Results from an Experiment on Internet Survey Email Contacts. Social Science Computer Review26(4): 498–509.
17.
MarcusBBosnjakMLindnerSPilischenkoSSchützA (2007) Compensating for Low Topic Interest and Long Surveys: A Field Experiment on Nonresponse in Web Surveys. Social Science Computer Review25(3): 372–83.
18.
Muñoz-LeivaFSánchez-FernándezJMontoro-RíosFIbáñez-ZapataJA (2010) Improving the Response Rate and Quality in Web-Based Surveys through the Personalization and Frequency of Reminder Mailings. Quality & Quantity44(5): 1037–52
19.
PaulhusDL (1991) Measurement and Control of Response Bias. In RobinsonJPShaverPRWrightsmanLS (eds) Measures of Personality and Social Psychological Attitudes. San Diego, CA: Academic Press, 17–59.
20.
PorterSRWhitcombME (2003) The Impact of Contact Type on Web Survey Response Rates. Public Opinion Quarterly67(4): 579–88.
21.
PorterSRWhitcombME (2005) E-mail Subject Lines and Their Effect on Web Survey Viewing and Response. Social Science Computer Review23(3): 380–87.
22.
TrouteaudAR (2004) How You Ask Counts: A Test of Internet-Related Components of Response Rates to a Web-Based Survey. Social Science Computer Review22(3): 385–92.
VinkJMBoomsmaDI (2008) A Comparison of Early and Late Respondents in a Twin-Family Survey Study. Twin Research and Human Genetics11(2): 165–73.
25.
YanTConradFGTourangeauRCouperMP (2010) Should I Stay or Should I Go: The Effects of Progress Feedback, Promised Task Duration, and Length of Questionnaire on Completing Web Surveys. International Journal of Public Opinion Research23(2): 131–47.
