Comparing Two Web/Mail Mixed-mode Contact Protocols to a Unimode Mail Survey

Abstract

Recent research has shown mixed-mode surveys are advantageous for organizations to use in collecting data. Previous research explored web/mail mode effects for four-contact waves. This study explores the effect of web/mail mixed-mode systems over a series of contacts on the customer satisfaction data from the Florida Cooperative Extension Service during 2012–2013. The experimental design involved a group of clients who provided e-mail and mail contact information randomly assigned to two mixed-mode treatment groups and a mail-only control. Demographic and service utilization data were compared to assess response rates and nonresponse bias. Logistic regression showed the treatment groups had similar response rates and nonresponse bias. The fifth contact was statistically significant in increasing response rates but did not reduce nonresponse bias. Future research should continue exploring optimizing the number of contacts in mixed-mode survey methodology.

Get full access to this article

View all access options for this article.

References

American Association for Public Opinion Research. 2015. Standard definitions: Final dispositions of case codes and outcome rates for surveys. http://www.aapor.org/AAPOR_Main/media/publications/Standard-Definitions20169theditionfinal.pdf (accessed April 27, 2017).

Baines

A. D.

Partin

M. R.

Davern

Rockwood

T. H.

. 2007. Mixed-mode administration reduced bias and enhanced post stratification adjustments in a health behavior survey. Journal of Clinical Epidemiology 60:1246–55.

Biemer

P. P.

Christ

S. L.

. 2008. Weighting survey data. In International handbook of survey methodology, edited by de Leeuw

Hox

J. J.

Dillman

D. D.

, 317–41. New York: Lawrence Erlbaum.

Cook

Heath

Thompson

R. L.

. 2000. A meta-analysis of response rates in web- or Internet-based surveys. Educational and Psychological Measurement 60:821–36.

de Bernardo

D. H.

Curtis

. 2012. Using online and paper surveys: The effectiveness of mixed-mode methodology for populations over 50. Research on Aging 35:220–40.

de Leeuw

E. D.

2005. To mix or not to mix data collection modes in surveys. Journal of Official Statistics 21:233–55.

Dillman

D. A.

Phelps

Tortora

Swift

Kohrell

Berck

Messer

B. L.

. 2009. Response rate and measurement differences in mixed-mode surveys using mail, telephone, Interactive Voice Response (IVR) and the Internet. Social Science Research 38:1–18.

Dillman

D. A.

Smyth

J. D.

Christian

L. M.

. 2014. Internet, phone, mail, and mixed-mode surveys: The tailored design method, 4th ed. Hoboken, NJ: John Wiley.

Dixon

2005. Comparison of item and unit nonresponse in household surveys. Washington, DC: Bureau of Labor Statistics. http://www.bls.gov/ore/pdf/st050080.pdf (accessed July 2, 2014).

10.

Dykema

Stevenson

Klein

Kim

Day

. 2013. Effects of e-mailed versus mailed invitations and incentives on response rates, data quality, and costs in a web survey of university faculty. Social Science Computer Review 31:359–70.

11.

Fan

Yan

. 2010. Factors affecting response rates of the web survey: A systematic review. Computers in Human Behavior 26:132–39.

12.

Groves

R. M.

2006. Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly 70:646–75.

13.

Groves

R. M.

Fowler

Jr Couper

M. P.

Lepkowski

J. M.

Singer

Tourangeau

. 2009. Survey methodology, 2nd ed. Hoboken, NJ: John Wiley.

14.

Holmberg

Lorenc

Werner

. 2010. Contact strategies to improve participation via the web in a mixed-mode mail and web survey. Journal of Official Statistics 26:465–80.

15.

Israel

G. D.

2010. Using web-hosted surveys to obtain responses from extension clients: A cautionary tale. Journal of Extension 48:4. http://www.joe.org/joe/2010august/a8.php (accessed June 16, 2014).

16.

Israel

G. D.

2013a. Combining mail and e-mail contacts to facilitate participation in mixed-mode surveys. Social Science Computer Review 31:346–58.

17.

Israel

G. D.

2013b. Using mixed-mode contacts in client surveys: Getting more bang for your buck. Journal of Extension 51:3. http://www.joe.org/joe/2013june/a1.php (accessed June 16, 2014).

18.

Israel

G. D.

Lamm

A. J.

. 2012. Item non-response in a client survey of the general public. Survey Practice. http://surveypractice.wordpress.com/2012/04/17/item-nonresponse-in-a-client-survey-of-the-general-public/ http://surveypractice.wordpress.com/2012/04/17/item-nonresponse-in-a-client-survey-of-the-general-public/#more-6070 (accessed July 2, 2014).

19.

Izrael

Battaglia

M. P.

Frankel

M. R.

. 2009. Extreme survey weight adjustment as a component of sample balancing (a.k.a. raking). Proceedings of the 2009 SAS Global Forum. http://support.sas.com/resources/papers/proceedings09/247-2009.pdf (accessed October 9, 2014).

20.

Kaplowitz

M. D.

Lupi

Couper

M. P.

Thorp

. 2012. The effect of invitation design on web survey response rates. Social Science Computer Review 30:339–49.

21.

Lesser

V. M.

Newtown

L. D.

Yang

D. K.

. 2012. Comparing item nonresponse across different delivery modes in general population survey. Survey Practice. http://surveypractice.wordpress.com/2012/04/17/comparing-item-nonresponse-across-different-delivery-modes-in-general-population-surveys-2/ (accessed June 16, 2014).

22.

Lesser

V. M.

Yang

D. K.

Newton

L. D.

. 2011. Assessing hunters’ opinions based on a mail and a mixed-mode survey. Human Dimensions of Wildlife 16:164–73.

23.

Manfreda

K. L.

Bosnjak

Berzelak

Haas

Vehovar

. 2008. Web surveys versus other survey modes. International Journal of Market Research 50:79–104.

24.

Mason

Lesser

Traugott

. 2002. Effect of item nonresponse on nonresponse error and inference. In Survey nonresponse, edited by Groves

Dillman

Eltinge

Little

, 149–62. New York: Wiley.

25.

Medway

R. L.

Fulton

. 2012. When more gets you less: A meta-analysis of the effect of concurrent web options on mail survey response rates. Public Opinion Quarterly 76:733–46.

26.

Messer

B. L.

Dillman

D. A.

. 2011. Surveying the general public over the Internet using address-based sampling and mail contact procedures. Public Opinion Quarterly 75:429–57.

27.

Messer

B. L.

Edwards

M. L.

Dillman

D. A.

. 2012. Determinants of item web and mail respondents in three address-based mixed-mode surveys of the general public. Survey Practice. http://surveypractice.wordpress.com/2012/04/17/determinants-of-item-nonresponse-to-web-and-mail-respondents-in-three-address-based-mixed-mode-surveys-of-the-general-public/ (accessed July 9, 2015).

28.

Millar

M. M.

Dillman

D. A.

. 2011. Improving response to web and mixed-mode surveys. Public Opinion Quarterly 75:249–68.

29.

National Telecommunications and Information Administration, & Economics and Statistics Administration. 2013. Exploring the digital nation: America’s emerging online experience. http://www.ntia.doc.gov/files/ntia/publications/exploring_the_digital_nation_-_americas_emerging_online_experience.pdf (accessed July 9, 2015).

30.

Porter

S. R.

Whitcomb

M. E.

. 2007. Mixed-mode contacts in web surveys: Paper is not necessary better. Public Opinion Quarterly 71:635–48.

31.

Qualtrics. 2015. Qualtrics software, Version Insight Platform. www.qualtrics.com.

32.

Schafer

Joseph L.

Graham

John W.

. 2002. Missing data: Our view of the state of the art. Psychological Methods 7:147–77.

33.

Seevers

Graham

. 2012. Education through cooperative extension, 3rd ed. Fayetteville: University of Arkansas.

34.

Shih

T. H.

Fan

. 2008. Comparing response rates from web and mail surveys: A meta-analysis. Field Methods 20:249–71.

35.

Smyth

J. D.

Dillman

D. A.

Christian

L. M.

O’Neill

A. C.

. 2010. Using the Internet to survey small towns and communities: Limitations and possibilities in the early 21st century. American Behavioral Scientist 53:1423–48.

36.

Stern

Michael J.

Bilgen

Dillman

D. A.

. 2014. The state of survey methodology: Challenges, dilemmas, and new frontiers in the era of the tailored design. Field Methods 24:284–301.

37.

Yuan

Y. C

. 2010. Multiple imputation for missing data: Concepts and new development (Version 9.0). Rockville, MD: SAS Institute. https://connect.ssri.duke.edu/sites/connect.ssri.duke.edu/files/upload/help-resource/multipleimputation%20missing%20data%20-%20sas.pdf (accessed November 15, 2013).

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.18 MB