Findings from existing research that compares Web-based survey and traditional survey methods have been inconclusive. This study compares responses from a Web-based survey and those from a paper-and-pencil survey in terms of response rates, data quality, demographic profiles of respondents, internal consistency of scales, and responses to items among American Society of Travel Agents members. Two samples from the same population were randomly assigned to receive the questionnaire either by e-mail notices or by postal mail. Several differences were found between the Web-based survey and the paper-and-pencil survey. The response rate to the Web-based survey was lower than that to the mail survey. The Web-based survey had more missing data fields than the mail survey. The two groups differed in 15.3% of the total items in the data set. Web respondents’ mean scores to the five scales were consistently lower than those of mail respondents.
Basi, R. K. (1999). “WWW Response Rates to Socio-Demographic Items.”Journal of the Market Research Society, 41 (4): 397-401.
2.
Buchanan, T., and J. L. Smith (1999). “Using the Internet for Psychological Research: Personality Testing on the World Wide Web.”British Journal of Psychology, 90: 125-144.
3.
Carini, R. M., J. C. Hayek, G. D. Kuh, J. M. Kennedy, and J.A. Ouimet (2003). “College Student Responses to Web and Paper Surveys: Does Mode Matter?”Research in Higher Education, 44 (1): 1-19.
4.
Cobanoglu, C., B. Warde, and P. Moreo (2001). “A Comparison of Mail, Fax, and Web-Based Survey Methods.”International Journal of Market Research, 43 (4): 441-452.
5.
Connelly, N. A., T. L. Brown, and D. J. Decker (2003). “Factors Affecting Response Rates to Natural Resource-Focused Mail Surveys: Empirical Evidence of Declining Rates over Time.”Society and Natural Resources, 16, 541-549.
6.
Couper, M. P. (2000). “Web Surveys: A Review of Issues and Approaches.”Public Opinion Quarterly, 6: 464-494.
7.
Couper, M. P., M. W. Traugott, and M. J. Lamias (2001). “Web Survey Design and Administration.”Public Opinion Quarterly, 65: 230-253.
8.
Crompton, J. L., and S. Tian-Cole (1999). “What Response Rate Can Be Expected from Questionnaire Surveys That Address Park and Recreation Issues?”Journal of Park and Recreation Administration, 17(1): 60-72.
9.
Dey, E. L. (1997). “Working with Low Survey Response Rates: The Efficacy of Weighting Adjustments.”Research in Higher Education,38 (2): 215-227.
10.
Dillman, D. A. (1978). Mail and Telephone Surveys: The Total Design Method. New York: John Wiley.
11.
Dillman, D. A. (2000). Mail and Internet Surveys: The Tailored Design Method. New York: John Wiley.
12.
Dillman, D. A., R. D. Tortora, J. Conradt, and D. Bowker (1998). “Influence of Plain vs. Fancy Design on Response Rates for Web Surveys.” Paper presented at the Joint Statistical Meetings of the American Statistical Association, Dallas, TX.
13.
Hatten, T. S. (1997). Small Business: Entrepreneurship and Beyond. Upper Saddle River, NJ: Prentice Hall.
14.
Fricker, R. D., and M. Schonlau (2002). “Advantages and Disadvantages of Internet Research Surveys: Evidence from the Literature.”Field Methods, 14 (4): 347-367.
15.
Idleman, L. (2003). “Comparing Responses to Mail and Web-Based Surveys.” Paper presented at the annual meeting of the American Educational Research Association, Chicago.
16.
Ilieva, J., S. Baron, and N. M. Healey (2002). “Online Surveys in Marketing Research: Pros and Cons.”International Journal of Market Research, 44 (3): 361-382.
17.
Kehoe, C. M., and J. E. Pitkow (1996). “Surveying the Territory: GVU’s Five WWW User Surveys.”The World Wide Web Journal, 1 (3): 77-84.
18.
Mertler, C. A., and M. A. Earley (2003). “A Comparison of the Psychometric Qualities of Surveys Administered by Web and Traditional Methods.” Paper presented at the annual meeting of the American Educational Research Association, Chicago.
19.
Palmquist, J., and A. Stueve (1996). “Stay Plugged in to New Opportunities.”Marketing Research, 8 (1): 13-15.
20.
Porter, S. R., and P. D. Umbach (2001). “What Works Best? Collecting Alumni Data with Multiple Technologies.” Paper presented at the 41st Forum of the Association of Institutional Research, Long Beach, CA.
21.
Raziano, D. B., R. Jayadevappa, D. Valenzula, M. Weiner, and R. Lavizzo-Mourey (2001). “E-mail versus Conventional Postal Mail Survey of Geriatric Chiefs.”The Gerontologist, 41 (6): 799-804.
22.
Sax, L. J., S. K. Gilmartin, and A. N. Bryant (2003). “Assessing Response Rates and Nonresponse Bias in Web and Paper Surveys.”Research in Higher Education, 44 (4): 409-432.
23.
Schaefer, D. R., and D. A. Dillman (1998). “Development of a Standard E-Mail Methodology: Results of an Experiment.”Public Opinion Quarterly, 62: 378-397.
24.
Schillewaert, N., F. Langerak, and T. Duhamel (1998). “Non-Probability Sampling for WWW Surveys: A Comparison of Methods.”Journal of the Market Research Society, 40 (4): 307-322.
25.
Schleyer, T., and J. L. Forrest (2000). “Methods for the Design and Administration of Web-Based Surveys.”Journal of the American Medical Informatics Association, 7 (4): 416-425.
26.
Shannon, D. M., and C. C. Bradshaw (2002). “A Comparison of Response Rate, Response Time, and Costs of Mail and Electronic Surveys.”The Journal of Experimental Education, 70 (2): 179-192.
27.
Stanton, J. M. (1998). “An Empirical Assessment of Data Collection Using the Internet.”Personnel Psychology, 51: 709-725.
28.
Swoboda, W. J., N. Muehlberger, R. Weitkunat, and S. Schneeweib (1997). “Internet Surveys by Direct Mailing: An Innovative Way of Collecting Data.”Social Science Computer Review, 15 (3): 242-255.
29.
Taylor, H. (2000). “Does Internet Research Work?”International Journal of Market Research, 42 (1): 51-63.
30.
Tomsic, M. L., D. D. Hendel, and R. P. Matross (2000). “A World Wide Web Response to Student Satisfaction Surveys: Comparisons Using Paper and Internet Formats.” Paper presented at the 40th Annual Meeting of the Association of Institutional Research, Cincinnati, OH.
31.
Yun, G. W., and C. W. Trumbo (2000). “Comparative Response to a Survey Executed by Post, E-Mail, & Web Form.”Journal of Computer-Mediated Communication, 6 (1). Available at http://www.ascusc.org/jcmc/vol6/issue1/yun.html (accessed August 19, 2004).
