Effects of Recognition Accuracy and Vocabulary Size of a Speech Recognition System on Task Performance and User Acceptance

Abstract

The purpose of this study was to determine the effects of recognizer accuracy and vocabulary size on performance of a speech recognition system. Subjects ranging in age from 20 to 55 years performed a data-entry task using a simulated speech recognizer that simulated three accuracy levels and three levels of available vocabulary. The accuracy level at which the recognizer was performing significantly influenced the time to complete the task as well as the user's acceptability ratings but had only a small effect on the number of errors left uncorrected. Available word-level vocabulary size also significantly affected the task completion time; however, its effect on the average number of uncorrected errors and on the acceptability ratings was negligible. Age of the subject was found to influence both objective and subjective measures. Older subjects required more time to complete the tasks, though they consistently rated the speech input systems more favorably than did the younger subjects. Minimal experience with computer text editors may account in part for the speed differences between older and younger subjects.

Get full access to this article

View all access options for this article.

References

Brickfield, C. F. (1984). Attitudes and perceptions of older persons toward technology. In P. K. Robinson , J. Livingston , and J. E. Birren (Ed.), Aging and technological advances. New York: Plenum.

Clark, M. C. (1986). The use of technology in the home by older adults. In Proceedings of the Human Factors Society 30th Annual Meeting (pp. 1164–1166). Santa Monica, CA: Human Factors Society.

Cochran, W. G. , and Cox, G. M. (1957). Experimental designs. New York: Wiley.

Coleman, W. D. (1985). Examining the relationship between performance measures and user evaluations in a transfer-of-training paradigm. Unpublished master's thesis, Virginia Polytechnic Institute and State University, Blacksburg, VA.

Coler, C. (1982). Helicopter speech-command systems: Recent noise tests are encouraging. Speech Technology 1(3), 76–81.

Gomez, L. M. , Egan, D. E. , and Bowers, C. (1986). Learning to use a text editor: Some learner characteristics that predict success. Human-Computer Interaction, 2, 1–23.

Gould, J. D. , Conti, J. , and Hovanyecz, J. (1983). Composing letters with a stimulated listening typewriter. Communications of the ACM, 26, 295–308.

Grady, M. W. (1982). A systems engineering view of advanced speech technology performance standards. In D. Pallett (Ed.), Proceedings of the Workshop on Standardization for Speech I/O Technology (pp. 155–158). Gaithersburg, MD: National Bureau of Standards.

Harris, S. D. (1982). Voice-controlled avionics: Programs, progress, and prognosis (A case for holistic engineering). In D. Pallett (Ed.), Proceedings of the Workshop on Standardization for Speech I/O Technology (pp. 113–129). Gaithersburg, MD: National Bureau of Standards.

10.

Lea, W. A. (1983). Problems in predicting performances of speech recognizers. In D. Pallett (Ed.), Proceedings of the Workshop on Standardization for Speech I/O Technology (pp. 15–24). Gaithersburg, MD: National Bureau of Standards.

11.

Rosson, M. B. , and Mellon, N. M. (1985). Behavioral issues in speech-based remote information retrieval. (Report No. RC11028 [#49528]). Yorktown Heights, NY: IBM Watson Research Center.

12.

Schmandt, C. (1982). Speech communication, a systems approach. In Proceedings of the Voice Data Entry Systems Applications Conference (pp. 3–12). San Mateo, CA: Lockheed Missiles & Space Co.

13.

Schurick, J.M. (1986). Efficiency of limited vocabulary speech recognition for data entry tasks. In Proceedings of the Human Factors Society 30th Annual Meeting (pp. 931–935). Santa Monica, CA: Human Factors Society.

14.

Simpson, C. A. , McCauley, M. E. , Roland, E. F. , Ruth, J. C. , and Williges, B. H. (1985). Systems design for speech recognition and generation. Human Factors, 27, 115–141.

15.

Williges, B. H. , and Dryden, R. D. (1987, January). Investigation of high-technology devices for the motor-impaired (Report No. INF-85-003-01). Blacksburg, VA: Virginia Polytechnic Institute and State University.