Web versus Mail

Abstract

Despite organizations’ widespread Internet use and ready availability of Web survey software, few studies have examined explicitly the impact on employees’ responses of using the Web as opposed to mail-based surveys (MBS). This article addresses this gap using an employee attitude survey distributed to a 50% systematic sample of 3,338 employees by mail, the remaining employees receiving the survey via a Web link. Although the return rate for the Web (49.1%) was higher than for mail (33.5%), the quality of Web returns was reduced by a higher number of partial responses and abandonments. Taking into account effect size, significant differences in response were small other than for open question content. Recommendations regarding use of Web-based surveys (WBS) are offered and areas for future research suggested.

Keywords

survey distribution mail survey Web survey response rate organizational survey

Get full access to this article

View all access options for this article.

References

American Association for Public Opinion Research (AAPOR). 2008. Standard definitions: Final dispositions of case codes and outcome rates for surveys. (5th ed). Lenexa, KS: AAPOR.

Baruch

Holtom

B. C.

2008. Survey response rate levels and trends in organizational research. Human Relations, 61, 1139–60.

Brockner

Siegel

P. A.

Daly

J. R.

Tyler

Martin

1997. When trust matters: The moderating effect of outcome favourability. Administrative Science Quarterly, 42, 558–83.

Cohen

1988. Statistical power analysis for the behavioral sciences. (2nd ed). Hillside, NJ: Lawrence Erlbaum.

Commission of the European Communities. 2009. Europe’s digital competitiveness report: Volume 1: i2010—Annual Information Society Report 2009. http://ec.europa.eu/information_society/eeurope/i2010/docs/annual_report/2009/sec_2009_1103.pdf (accessed August 24, 2010).

Converse

P. D.

Wolfe

E. W.

Huang

Oswald

F. L.

2008. Response rates for mixed-mode surveys using mail and e-mail/web. American Journal of Evaluation, 29, 99–107.

Couper

M. P.

2000. Web surveys: A review of issues and approaches. Public Opinion Quarterly, 64, 464–94.

Couper

M. P.

Blair

Triplett

1999. A comparison of mail and e-mail for a survey of employees in federal statistical agencies. Journal of Official Statistics, 15, 39–56.

Crawford

S. D.

Couper

M. P.

Lamias

M. J.

2001. Web surveys: Perceptions of burden. Social Science Computer Review, 19, 146–62.

10.

Dillman

D. A.

2009. Internet, mail and mixed mode surveys: The tailored design method. (3rd ed.). New York: Wiley.

11.

Eisenberger

Huntington

Hutchinson

Sowa

1986. Perceived organizational support. Journal of Applied Psychology, 71, 500–7.

12.

Groves

R. M.

2006. Non-response rates and non-response bias in household surveys. Public Opinion Quarterly, 70, 646–75.

13.

Hoonakker

Carayon

2009. Questionnaire survey nonresponse: A comparison of postal mail and Internet surveys. International Journal of Human-Computer Interaction, 25, 348–73.

14.

Kaplowitz

M. D.

Hadlock

T. D.

Levine

2004. A comparison of web and mail survey response rates. Public Opinion Quarterly, 68, 94–101.

15.

Kraut

A. I.

2006. Getting action from organizational surveys: New concepts, technologies and applications. San Francisco: Jossey-Bass.

16.

Kulesa

Bishop

R. J.

2006. What did they mean? New and emerging methods for analyzing open-ended comments. In Getting action from organizational surveys: New concepts, technologies and applications, eds. Kraut

A. I.

238–63. San Francisco: Jossey-Bass.

17.

Manfreda

K. L.

Vehovar

2008. Internet surveys. In International handbook of survey methodology, eds. de Leeuw

E. D.

Hox

J. J.

Dillman

D. A.

264–84. New York: Lawrence Erlbaum.

18.

Meyer

J. P.

Allen

W. T.

1997. Commitment in the workplace: Theory research and application. Thousand Oaks, CA: SAGE.

19.

Poncheri

R. M.

Lindberg

J. T.

Thompson

L. F.

Surface

E. A.

2008. A comment on employee surveys: Negative bias in open responses. Organizational Research Methods, 11, 614–30.

20.

Provalis Research. 2005. WordStat Content Analysis Module for SIMSTAT and QDA Miner. http://www.provalisresearch.com/Download/Manuals.html (accessed August 26, 2010).

21.

Rogelberg

S. G.

Luong

Sederberg

M. E.

Cristol

D. S.

2000. Employee attitude surveys: Examining the attitudes of noncompliant employees. Journal of Applied Psychology, 85, 284–93.

22.

Rogelberg

S. G.

Stanton

J. M.

2007. Introduction: Understanding and dealing with organizational survey non-response. Organizational Research Methods, 10, 195–209.

23.

Shih

T. H.

Fan

2008. Comparing response rates from web and mail surveys: A meta-analysis. Field Methods, 20, 249–71.

24.

Thompson

L. F.

Surface

E. A.

2007. Employee surveys administered online: attitudes towards the medium, non-response, and data representativeness. Organizational Research Methods, 10, 241–61.

25.

Toepoel

Das

Van Soest

2009. Design of web questionnaires: The effects of the number of items per screen. Field Methods, 21, 200–13.

26.

Tomaskovic-Devey

Leiter

Thompson

1994. Organizational survey non-response. Administrative Science Quarterly, 39, 439–57.

27.

U.S. Census Bureau. 2009. Statistical Abstract of the United States: 2010. 129th ed. http://www.census.gov/compendia/statab/2010edition.html (accessed August 24, 2010).

28.

Von Eye

Mun

E. Y.

2005. Analyzing rater agreement: Manifest variable methods. Mahwah, NJ: Lawrence Erlbaum.