A Randomized Experiment Investigating the Suitability of Speech-Enabled IVR and Web Modes for Publicly Reported Surveys of Patients’ Experience of Hospital Care

Abstract

The HCAHPS Survey obtains hospital patients’ experiences using four modes: Mail Only, Phone Only, Mixed (mail/phone follow-up), and Touch-Tone (push-button) Interactive Voice Response with option to transfer to live interviewer (TT-IVR/Phone). A new randomized experiment examines two less expensive modes: Web/Mail (mail invitation to participate by Web or request a mail survey) and Speech-Enabled IVR (SE-IVR/Phone; speaking to a voice recognition system; optional transfer to an interviewer). Web/Mail had a 12% response rate (vs. 32% for Mail Only and 33% for SE-IVR/Phone); Web/Mail respondents were more educated and less often Black than Mail Only respondents. SE-IVR/Phone respondents (who usually switched to an interviewer) were less often older than 75 years, more often English-preferring, and reported better care than Mail Only respondents. Concerns regarding inconsistencies across implementations, low adherence to primary modes, or low response rate may limit the applicability of the SE-IVR/Phone and Web/Mail modes in HCAHPS and similar standardized environments.

Keywords

speech-enabled IVR web surveys patient satisfaction HCAHPS

Get full access to this article

View all access options for this article.

References

Ball

Bonnewell

Danielsen

Mataga

Rehor

(2000). Speech-enabled services using TelePortal™ software and VoiceXML. Bell Labs Technical Journal, 5(3), 98-111.

Centers for Medicare & Medicaid Services. (2010, June 13). HCAHPS fact sheet 2010. Retrieved from http://www.hcahpsonline.org/files/HCAHPS%20Fact%20Sheet%202010.pdf

Centers for Medicare & Medicaid Services. (2012, September 18). HCAHPS Quality Assurance Guidelines V7.0. Retrieved from http://www.hcahpsonline.org/files/HCAHPS%20Quality%20Assurance%20Guidelines%20V7.0%20March%202012.pdf

Christian

L. M.

Dillman

D. A.

Smyth

J. D.

(2008). The effects of mode and format on answers to scalar questions in telephone and web surveys. In Lepkowski

Tucker

Brick

de Leeuw

Japec

Lavrakas

Sangster

(Eds.), Advances in telephone survey methodology (pp. 250-275). New York, NY: Wiley-Interscience.

Corkrey

Parkinson

(2002). Interactive voice response: Review of studies 1989-2000. Behavioral Research Methods, Instruments, & Computers, 34, 342-353.

Couper

(2000). Web surveys: A review of issues and approaches. Public Opinion Quarterly, 64, 464-494.

Damiano

P. C.

Elliott

M. N.

Tyler

Hays

R. D.

(2004). Differential use of the CAHPS^® 0-10 global gating scale by Medicaid and commercial populations. Health Services and Outcomes Research Methodology, 5(3-4), 193-205.

Davis

R. E.

Couper

M. P.

Janz

N. K.

Caldwell

C. H.

Resnicow

(2010). Interviewer effects in public health surveys. Health Education Research, 25(1), 14-26.

Dillman

D. A.

Phelps

Tortora

Swift

Kohrell

Berck

Messer

B. L.

(2009). Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the internet. Social Science Research, 38(1), 3-20.

10.

Dulude

(2002). Automated telephone answering systems and aging. Behaviour & Information Technology, 21, 171-184.

11.

Elliott

M. N.

Haviland

A. M.

Kanouse

D. E.

Hambarsoomian

Hays

R. D.

(2009). Adjusting for subgroup differences in extreme response tendency in ratings of health care: impact on disparity estimates. Health Services Research, 44(2 Pt 1), 542-561.

12.

Elliott

M. N.

Zaslavsky

A. M.

Goldstein

Lehrman

Hambarsoomians

Beckett

M. K.

Giordano

(2009). Effects of survey mode, patient mix, and nonresponse on CAHPS hospital survey scores. Health Services Research, 44(2 Pt 1), 501-518.

13.

Fowler

F. J.

Gallagher

P. M.

Stringfellow

V. L.

Zaslavsky

A. M.

Thompson

J. W.

Cleary

P. D.

(2002). Using telephone interviews to reduce nonresponse bias to mail surveys of health plan members. Medical Care, 40, 190-200.

14.

Giordano

L. A.

Elliott

M. N.

Goldstein

Lehrman

W. G.

Spencer

P. A.

(2010). Development, implementation, and public reporting of the HCAHPS survey. Medical Care Research and Review, 67, 27-37.

15.

Goldstein

Elliott

M. N.

Lehrman

W. G.

Hambarsoomian

Giordano

L. A.

(2010). Racial/ethnic differences in patients’ perceptions of inpatient care using the HCAHPS survey. Medical Care Research and Review, 67, 74-92.

16.

Goldstein

Farquhar

Crofton

Darby

Garfinkel

(2005). Measuring hospital care from the patients’ perspective: An overview of the CAHPS hospital survey development process. Health Services Research, 40(6 Pt 2), 1977-1995.

17.

Griffin

D. H.

Obenski

S. M.

(2002). Meeting 21st century demographic needs implementing the American Community Survey: May 2002, Report 2: Demonstrating survey quality. Washington, DC: U.S. Department of Commerce, Economics, and Statistics.

18.

Kaplowitz

M. D.

Hadlock

T. D.

Levine

(2004). A comparison of web and mail survey response rates. Public Opinion Quarterly, 68, 94-101.

19.

Kennedy

J. M.

Kuh

Carini

(2000). Web and mail surveys: preliminary results of comparisons based on a large-scale project. Paper presented at the annual meeting of the American Association for Public Opinion Research, Portland, OR.

20.

Klingner

Tversky

Hanrahan

(2011). Effects of visual and verbal presentation on cognitive load in vigilance, memory, and arithmetic tasks. Psychophysiology, 48, 323-332.

21.

Krosnick

J. A.

Alwin

D. F.

(1987). An evaluation of a cognitive theory of response-order effects in survey measurement. Public Opinion Quarterly, 51, 201-219.

22.

Krysan

Schuman

Scott

L. J.

Beatty

(1994). Response rates and response content in mail versus face-to-face surveys. Public Opinion Quarterly, 58, 381-399.

23.

Link

Mokdad

(2005). Use of alternative modes for health surveillance surveys: Results from a web/mail/telephone experiment. Epidemiology, 16, 701-704.

24.

Miller

D. I.

Bruce

Gagnon

Talbot

Messier

(2011). Improving older adults’ experience with interactive voice response systems. Telemedicine and e-Health, 17, 452-455.

25.

Rodriguez

H. P.

von Glahn

Rogers

W. H.

Chang

Fanjiang

Safran

D. G.

(2006). Evaluating patients’ experiences with individual physicians: A randomized trial of mail, internet, and interactive voice response telephone administration of surveys. Medical Care, 44, 167-174.

26.

Rogers

S. M.

Miller

H. G.

Turner

C. F.

(1998). Effects of interview mode on bias in survey measurements of drug use: Do respondent characteristics make a difference? Substance Use & Misuse, 33, 2179-2200.

27.

Schneider

S. J.

Cantor

Heller

T. H.

Brick

P. D.

(2002, November). Pretesting interactive voice response/automated speech recognition surveys. Paper presented at the International Conference on Questionnaire Development, Evaluation, and Testing Methods, Charleston, SC.

28.

Shettle

Mooney

(1999). Monetary incentives in U.S. government surveys. Journal of Official Statistics, 15, 231-250.

29.

Tarnai

Dillman

D. A.

(1992). Questionnaire context as a source of response differences in mail and telephone surveys. In Schwarz

Hippler

H. J.

Sudman

(Eds.), Context effects in social and psychological research (pp. 115-130). New York, NY: Springer-Verlag.

30.

Toepoel

Das

Soest

(2009). Effects of design in web Surveys: Comparing trained and fresh respondents. Public Opinion Quarterly, 72, 985-1007.

31.

Tourangeau

Steiger

D. M.

Wilson

(2002). Self-administered questions by telephone—Evaluating interactive voice response. Public Opinion Quarterly, 66, 265-278.

32.

Tourkin

Cox

Parmer

Zukerberg

(2005). (Inter) net gain? Experiments to increase web based response. Paper presented at the American Association for Public Opinion Research Conference, Miami, FL.

33.

Turner

C. F.

Forsyth

B. H.

O’Reilly

Cooley

P. C.

Smith

T. K.

Rogers

S. M.

Miller

(1998). Automated self-interviewing and the survey measurement of sensitive behaviors. In Couper

M. P.

Baler

R. P.

Bethlehem

Clark

C. Z.

Martin

Nicholls

O’Reilly

J. M.

(Eds.), Computer-assisted survey information collection (pp. 455-473). New York, NY: Wiley.

34.

Weech-Maldonado

Elliott

M. N.

Oluwole

Schiller

K. C.

Hays

R. D.

(2008). Survey response style and differential use of CAHPS rating scales by Hispanics. Medical Care, 46, 963-968.

35.

Ziegenfuss

J. Y.

Beebe

T. J.

Rey

Schleck

Locke

G. R.

Talley

N. J.

(2010). Internet option in a mail survey: More harm than good? Epidemiology, 21, 585-586.