A field experiment was conducted to test the effectiveness of norm-based persuasive messages for increasing response rates in online survey research. Participants in an interdisciplinary conference were asked to complete two successive postconference surveys and randomly assigned to one of two groups at each time point. The experimental group received a message asking them to complete an online survey that highlighted a descriptive social norm indicating typical response rates among attendees of prior similar conferences. The control group received a generic request to complete the online survey without any norm-based appeals. As expected, participants receiving a message highlighting the descriptive social norm, when asked to complete an online survey were more likely to comply with the request as compared to those who received the generic message without normative information. Participants receiving a double dose of the descriptive norm treatment at both time points were more likely to complete the second online survey than those who did not receive the descriptive norm intervention at either time point. Implications for future research on evaluation are discussed.
ArcherT. M. (2003). Web-based surveys. Journal of Extension, 41, 1–5.
2.
BatorR. J.CialdiniR. B. (2000). The application of persuasion theory to the development of effective pro-environmental public service announcements. Journal of Social Issues, 56, 527–541.
3.
BiererS. B.FishlederA. J.DanneferE.FarrowN.HullA. L. (2004). Psychometric properties of an instrument designed to measure the educational quality of graduate training programs. Evaluation & The Health Professions, 27, 410–424.
4.
BirnbaumM. H. (2004). Human research and data collection via the Internet. Annual Review of Psychology, 55, 803–832.
5.
BowlingJ. M.RimerB. K.LyonsE. J.GolinC. E.FrydmanG.RibislK. M. (2006). Methodologic challenges of e-health research. Evaluation and Program Planning, 29, 390–396.
6.
BradburnN. M. (1992). Presidential address: A response to the non-response problem. Public Opinion Quarterly, 56, 391–398.
7.
CialdiniR. B. (2003). Crafting normative messages to protect the environment. Current Directions in Psychological Science, 12, 105–109.
8.
CialdiniR. B.RenoR. R.KallgrenC. A. (1990). A focus theory of normative conduct: Recycling the concept of norms to reduce littering in public places. Journal of Personality and Social Psychology, 58, 1015–1026.
9.
CobanogluC.WardeB.MoreoP. J. (2001). A comparison of mail, fax and web-based survey methods. International Journal of Market Research, 43, 441–452.
10.
ConverseP. D.WolfeE. W.HuangX.OswaldF. L. (2008). Response rates for mixed-mode surveys using mail and e-mail/web. American Journal of Evaluation, 29, 99–107.
11.
CookC.HeathF.ThompsonR. L. (2000). A meta-analysis of response rates in web- or Internet-based surveys. Educational and Psychological Measurement, 60, 821–836.
12.
De LeeuwE.HeerW. (2002). Trends in household survey non-response: A longitudinal and international comparison. In GrovesR. M.DillmanD. A.EltingeJ. L.LittleR. J. A. (Eds.), Survey non-response (pp. 41–54). New York, NY: John Wiley.
13.
DeutskensE.RuyterK. D.WetzelsM.OosterweldP. (2004). Response rate and response quality of Internet-based surveys: An experimental study. Marketing Letters, 15, 21–36.
14.
DeyE. L. (1997). Working with low survey response rates: The efficacy of weighing adjustments. Research in Higher Education, 38, 215–227.
15.
DillmanD. (2000).Mail and Internet surveys: The total design method. (2nd ed.).
New York, NY: John Wiley.
16.
DillmanD. A. (2007). Mail and Internet surveys: The tailored design method. (2nd ed., 2007 update)
Hoboken, NJ: John Wiley.
17.
DommeyerC. J.BaumP.HannaR. W.ChapmanK. S. (2004). Gathering faculty teaching evaluations by in-class and online surveys: Their effects on response rates and evaluations. Assessment & Evaluation in Higher Education, 29, 611–623.
18.
FestingerL. (1954). A theory of social comparison processes. Human Relations, 7, 117–140.
19.
FraenkelJ. R.WallenN. E. (1993). How to design and evaluate research in education. (2nd ed.).
New York, NY: McGraw Hill.
20.
FrickerR. D.SchonlauM. (2002). Advantages and disadvantages of Internet research surveys: Evidence from the literature. Field Methods, 14, 347–367.
21.
GoldsteinN.CialdiniR. B.GriskeviciusV. (2008). A room with a viewpoint: Using norm-based appeals to motivate conservation behaviors in a hotel setting. Journal of Consumer Research, 35, 472–482.
22.
GoslingS. D.VazireS.SrivastavaS.JohnO. P. (2004). Should we trust web-based studies? A comparative analysis of six preconceptions about Internet questionnaires. American Psychologist, 59, 93–104.
23.
GrovesR. M.CialdiniR. B.CouperM. P. (1992). Understanding the decision to participate in a survey. Public Opinion Quarterly, 56, 475–495.
24.
HenryG. T.MarkM. M. (2003). Toward an agenda for research on evaluation. New Directions for Evaluation, 97, 69–80.
25.
KallgrenC. A.RenoR. R.CialdiniR. B. (2000). A focus theory of normative conduct: When norms do and do not affect behavior. Personality and Social Psychology Bulletin, 26, 1002–1012.
26.
KaplowitzM. D.HadlockT. D.LevineR. (2004). A comparison of web and mail survey response rates. Public Opinion Quarterly, 68, 94–101.
27.
KarrasB. T.TufanoJ. T. (2006). Multidisciplinary eHealth survey evaluation methods. Evaluation and Program Planning, 29, 413–418.
28.
KayeB. K.JohnsonT. J. (1999). Research methodology: Taming the cyber frontier. Social Science Computer Review, 17, 323–337.
29.
KiernanN. E.KiernanM.OylerM. A.GillesC. (2005). Is a web survey as effective as a mail survey? A field experiment among computer users. American Journal of Evaluation, 26, 245–252.
30.
KittlesonM. (1997). Determining effective follow-up of e-mail surveys. American Journal of Health Behavior, 21, 193–196.
31.
MarkM. M. (2001). Evaluation’s future: Furor, futile, or fertile?American Journal of Evaluation, 22, 457–479.
32.
McCabeS. E. (2004). Comparison of web and mail surveys in collecting illicit drug use data: A randomized experiment. Journal of Drug Education, 34, 61–72.
33.
MowenJ. C.CialdiniR. B. (1980). On implementing the door-in-the-face compliance technique in a business context. Journal of Marketing Research, 17, 253–258.
34.
RitterL. A.SueV. M. (2007). Conducting the survey. New Directions for Evaluation, 115, 47–50.
35.
SaxL. J.GilmartinS. K.BryantA. N. (2003). Assessing response rates and non-response bias in web and paper surveys. Research in Higher Education, 44, 409–432.
36.
SchultzP. W. (1999). Changing behavior with normative feedback interventions: A field experiment on curbside recycling. Basic and Applied Social Psychology, 21, 25–38.
37.
ShadishW. R.JrCookT. D.LevitonL. C. L. (1991). Foundations of program evaluation. Thousand Oaks, CA: Sage.
SheehanK.McMillanS. (1999). Response variation in e-mail surveys: An exploration. Journal of Advertising Research, 39, 45–54.
40.
SmithT. (1995). Trends in non-response rates. International Journal of Public Opinion Research, 7, 157–171.
41.
SolomonD. J. (2001). Conducting Web-based Survey. Practical Assessment, Research & Evaluation, 7(19). Retrieved fromhttp://cogprints.org/2357/.
42.
TruellA. D.BartlettJ. E.AlexanderM. W. (2002). Response rate, speed, & completeness: A comparison of Internet-based and mail surveys. Behavior Research Methods, Instruments, & Computers, 34, 50–54.
43.
VehovarV.BatageljZ.ManfredaK. L.ZaletelM. (2002). Nonresponse in web surveys. In GrovesR. M.DillmanD. A.EltingeJ. L.LittleR. J. A. (Eds.), Survey nonresponse (pp. 229–242). New York, NY: John Wiley.
44.
YunG. W.TrumboC. W. (2000). Comparative response to a survey executed by post, e-mail, & web form. Journal of Computer-Mediated Communication, 6. Retrieved March 27, 2011, fromhttp://jcmc.indiana.edu/vol6/issue1/yun.html
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
