This article reviews human factors research on the design of systems that use speech recognition for human control of the system or that use speech generation for the display of information. Speech technology terms are defined and the current status of the field is reviewed. Included are the performance of current speech recognition and generation algorithms, descriptions of several applications of the technology to particular tasks, and a discussion of research on design principles for speech interfaces. Finally, directions for further research are suggested. The need for better simulation techniques and performance measures is stressed, as is the importance of considering the entire system in which speech technology will function.
Get full access to this article
View all access options for this article.
References
1.
Allen, H. B., and Underwood, G. N. (Eds.). (1971). Readings in American dialectology. New York: Appleton-Century-Crofts.
2.
Armstrong, J. W. (1980). The effects of concurrent motor tasking on performance of a voice recognition system. Unpublished masters thesis, Naval Postgraduate School, Monterey, CA.
3.
Bierman, A., Rodman, R., Rubin, D., and Heidlage, F. (1984). Natural language with discrete speech as a mode for human to machine communication. Durham, NC: Duke University, Computer Science Department.
4.
Bond, Z. S., and Games, S. (1980). Misperceptions of fluent speech. In R. A. Cole (Ed.), Perception and production affluent speech (pp. 115–113). Hillsdale NJ: Erlbaum.
5.
Breaux, R. (1977). Laboratory demonstration of computer speech recognition in training. In R. Breaux, M. Curran, and E. M. Huff (Eds.), Voice technology of interactive real-time command/control systems applications. Moffett Field, CA: NASA-Ames Research Center.
6.
Brokx, J. P. L., and Nooteboom, S. G. (1982). Intonation and the perception of simultaneous voices. Journal of Phonetics, 10, 23–26.
7.
Brown, J. E., Bertone, C. M., and Obermeyer, R. W. (1968, February). Army aircraft warning system study (U.S. Army Technical Memorandum 6–68). Aberdeen Proving Ground. MD: U.S. Army Engineering Laboratories.
8.
Butler, F., Manaker, E., and Obert-Thorn, W. (1981, June). Investigation of a voice synthesis system for the F-14 aircraft: Final report (Report No. ACT 81–001). Bethpage, NY: Grumman Aerospace Corp.
9.
Bucher, N. M., Karl, R., Voorhees, J., and Werner, E. (1984, May). Alerting prefixes for speech warning messages. In Proceedings of the National Aerospace & Electronics Conference (NAECON) (pp. 924–931). New York: IEEE.
10.
Chapanis, A. (1975). Interactive human communication. Scientific American, 232(3), 36–42.
11.
Chapanis, A. (1976). Engineering psychology. In M. D. Dunnette (Ed.), Handbook of industrial and organizational psychology. Chicago: Rand McNally.
12.
Cole, R. A. (Ed.). (1980). Perception and production of fluent speech. Hillsdale NJ: Erlbaum.
13.
Coler, C. (1982, September/October). Helicopter speech-command systems: Recent noise tests are encouraging. Speech Technology1(3), 76–81.
14.
Coler, C. R., and Plummer, R. P. (1974, May). Development of a computer speech recognition system for flight systems applications. In Preprints of the 45th Annual Scientific Meeting (pp. 116–117). Washington, DC: Aerospace Medical Association.
15.
Coler, C., Plummer, R., Huff, E., and Hitchcock, M. (1977, December). Automatic speech recognition research at NASA-Ames Research Center. In Proceedings of the Voice-Interactive Real-Time Command/Control Systems Application Conference (pp. 143–163). Moffett Field, CA: NASA-Ames Research Center.
16.
Cotton, J. C., and McCauley, M. E. (1983, March). Voice technology design guides for Navy training systems: Final report for the period 23 April, 1980–2 January, 1982 (Report No. NAVTRAEQUIPCEN 80-C-0057–1). Orlando, FL: Naval Training Equipment Center.
17.
Cotton, J. C., McCauley, M. E., North, R. A., and Strieb, M. (1983). Development of speech inputloutput interfaces for tactical aircraft (AFWAL-TR-83–3073). Wright-Patterson AFB, OH: Flight Dynamics Laboratory.
18.
Deathridge, B. H. (1972). Auditory and other sensory forms of information presentation. In H. P. Van Cott and R. G. Kinkade (Eds.), Human engineering guide to equipment design. Washington, D.C.: U.S. Government Printing Office.
19.
Doddington, G., and Schalk, T. (1981). Speech recognition: Turning theory to practice. IEEE Spectrum, 18, 26–32.
20.
DuBord, M. J. (1982, June). An investigation of response time to collision avoidance commands with a cockpit display of traffic information. Unpublished masters thesis, San Jose State University, San Jose, CA.
21.
Egan, J. P. (1948). Articulation testing methods. Laryngoscope, 58, 955–991.
22.
Flanagan, J. L. (1972). Speech analysis, synthesis, and perception (2nd ed). New York: Springer Verlag.
23.
Fujimura, O. (1984, June). The role of linguistics for future speech technology. In LSA Bulletin 104 (pp. 4–7). Baltimore, MD: Linguistic Society of America.
24.
Garnes, S., and Bond, Z. S. (1977). The influence of semantics on speech perception. Paper presented at the 93rd Meeting of the Acoustical Society of America, University Park, PA.
25.
Gould, J. D., Conti, J., and Hovanyecz, J. (1983, April). Composing letters with a simulated listening typewriter. Communications of the ACM, 295–308.
26.
Grady, M. W.(1982). Air intercept controller prototype training system (NAVTRAEQUIPCEN 78-C-0182–14). Orlando, FL: Naval Training Equipment Center.
27.
Harris, S.D., North, R.A., and Owens, J.M. (1977, November). A system for the assessment of human performance in Concurrent verbal and manual control tasks. Paper presented at the 7th Annual Meeting of the National Conference on the Use of On-Line Computers in Psychology, Washington, DC.
28.
Hakkinen, M. T., and Williges, B. H. (1984). Synthesized warning messages: Effects of an alerting cue in single and multiple-function voice synthesis systems. Human Factors, 26, 185–195.
29.
Hecker, M. H., Stevens, K. N., von Bismark, G., and Williams, C. E. (1968). Manifestations of task-induced stress in the acoustical speech signal. Journal of the Acoustical Society of America, 44, 993–1001.
30.
Hilborn, E. H. (1975). Human factors experiments for data link: Final report (FAA-RD-75–170). Cambridge, MA: Department of Transportation Systems Center.
31.
Kemmerling, P., Geiselhart, R., Thornburn, D. E., and Cronburg, J. G. (1969). A comparison of voice and tone warning systems as a function of task loading (Technical Report ASD-TR-69–104). Wright-Patterson AFB, OH: U.S. Air Force, ASD.
32.
Kersteen, Z., and Damos, D. (1983, December). Human factors issues associated with the use of speech technology in the cockpit (Final Tech. Report, U.S. Army Grant No. NAG2–217). (Available from J. Voorhees, U.S. Army Aeromechanics Laboratory, NASA-Ames Research Center, Moffett Field, CA 94035.)
33.
Kryter, K. (1972). Speech communication. In H. P. Van Cott and R. G. Kinkade (Eds.), Human engineering guide to equipment design (pp. 161–226). Washington, DC: U.S. Government Printing Office.
34.
Lilleboe, M. L. (1963, June). Final Report: evaluation of As-tropower, Inc. auditory information display installed in the VA-3B airplane (Technical Report ST 31–22R-63; AD-83 1823). Patuxent River, MD: U.S. Naval Air Station, Naval Air Test Center.
35.
Luce, P. A., Feustel, T. C., and Pisoni, D. B. (1983). Capacity demands in short-term memory for synthetic and natural speech. Human Factors, 25, 17–32.
36.
McCauley, M. E. (1984). Human factors in voice technology. In F. A. Muckler (Ed.), Human Factors Review: 1984. Santa Monica, CA: Human Factors Society.
37.
McCauley, M. E., Root, R. W., and Muckler, F. A. (1982). Training evaluation of all automated air intercept controller training system (NAVTRAEQUIPCEN 81-C-0055–1). Orlando, FL: Naval Training Equipment Center.
38.
McCauley, M. E., and Semple, C. A. (1980). Precision approach radar training system (PARTS) (NAVTRAEQUIPCEN 79-C-0042–1). Orlando, FL: Naval Training Equipment Center.
39.
McSorley, W. J. (1981, March). Using voice recognition equipment to run the Warfare Environmental Simulator (WES). Unpublished masters thesis, Naval Postgraduate School, Monterey, CA.
40.
Miller, G. A., Heise, G. A., and Lichten, W. (1951). The intelligibility of speech as a function of the context of the test materials. Journal of Experimental Psychology, 41, 329–335.
41.
Moore, C. A., Moore, D. R., and Ruth, J. C. (1984, December). Applications of voice interactive systems-Military flight test and the future. In Proceedings of the Sixth Digital Avionics Systems Conference (pp. 301–308). New York: IEEE.
42.
Mountford, S, J., North, R. A., Metz, S. V., and Graffunder, K., (1982, April). Methodology for identifying voice functions for airborne voice-interactive control systems (Contract No. N62269–81-R-0344). Minneapolis, MN: Honeywell Systems Research Center.
43.
National Research Council. Committee on Computerized Speech Recognition Technologies. (1984). Automatic speech recognition in severe environments. Washington, DC: National Research Council, Commission on Engineering and Technical Systems.
44.
North, R. A., and Lea, W. (1982). Application of advanced speech technology in manned penetration bombers (AFWAL-TR-82–3004). Wright-Patterson AFB, OH: Flight Dynamics Laboratory.
45.
Nusbaum, H. C., and Schwab, E. C. (1983. May). The effects of training on intelligibility of synthetic speech: II the learning curve for synthetic speech. Paper presented at the 105th meeting of the Acoustical Society of America, Cincinnati, OH.
46.
Nye, P. W., and Gaitenby, J. (1974). The intelligibility of synthetic monosyllabic words in short, syntactically normal sentences. In Status report on speech research (SR-37/38) (pp. 169–190). New Haven CT: Haskins Laboratories.
47.
Pallett, D. (Ed.). (1982, March). Proceedings of the workshop on standardization for speech IIO technology. Gaithersburg, MD: National Bureau of Standards.
48.
Palmer, E., and Ellis, S. R. (1983, October). Potential interactions of collision avoidance advisories and cockpit displays of traffic information (SAE Technical Paper Series 831544). In Proceedings of the Second Aerospace Behavioral Engineering Technology Conference, Aerospace Congress & Exposition (pp. 433–443). Warrendale, PA: SAE.
49.
Pisoni, D. B., and Hunnicutt, S. (1980, April). Perceptual evaluation of MITalk: The MIT unrestricted text-to-speech system. In IEEE International Conference Record on Acoustics, Speech, and Signal Processing (pp. 572–575). New York: IEEE.
50.
Poock, G. K. (1981a). A longitudinal study of computer recognition performance and vocabulary size (NPS-55–81–013). Monterey, CA: Naval Postgraduate School.
51.
Poock, G. K. (1981b, October). To train randomly or all at once-That is the question. In Proceedings of the Voice Data Entry Systems Applications Conference. Sunnyvale, CA: Lockheed Missiles & Space Co.
52.
Poock, G. K., Martin, B. J., and Roland, E. F. (1983, February). The effect of feedback to users of voice recognition equipment (NPS Technical Report NPS-55–83–003). Monterey, CA: Naval Postgraduate School.
53.
Poock, G. K., and Roland, E. F. (1982, November). Voice recognition accuracy: What is acceptable? (NPS Technical Report, NPS55–82–030). Monterey, CA: Naval Postgraduate School.
54.
Poock, G. K., and Roland, E. F. (1984). A feasibility study for integrated voice recognition input into the Integrated Information Display system (IID) (NPS Technical Report NPS-55–84–008). Monterey, CA: Naval Postgraduate School.
55.
Poock, G. K., Schwalm, N. D., Martin, B. J., and Roland, E. F. (1982, December). Trying for speaker independence in the use of speaker dependent voice recognition equipment (NPS Technical Report NPS-55–82–032). Monterey, CA: Naval Postgraduate School.
56.
Rollins, A. M., (1984). “Composite” templates for speech recognition for small groups. In Proceedings of the Human Factors Society 28th Annual Meeting (Vol. 2)(pp. 758–762). Santa Monica, CA: Human Factors Society.
57.
Schurick, J. M., Williges, B. H., and Maynard, J. F. (in press). User feedback requirements with automatic speech recognition. Ergonomics.
58.
Searle, J. (1969). Speech acts. London: Cambridge University press.
59.
Sherwood, B. A. (1979, August). The computer speaks. IEEE Spectrum, pp. 18–25.
60.
Simpson, C. A. (1975). Occupational experience with a specific phraseology: Group differences in intelligibility for synthesized and human speech. Journal of the Acoustical Society of America, 58(Supplement 1),57.
61.
Simpson, C. A. (1981a, April). Access to speech synthesis and its applications. In J. C. Warren (Ed.), The best of the Computer Faires. Volume VI: Conference proceedings of the Sixth West Coast Computer Faire (pp. 74–79). Woodside, CA: West Coast Computer Faire.
62.
Simpson, C.A. (1981b). Evaluation of synthesized voice approach callouts (SYNCALL). In J. Moraal and K. F. Kraiss (Eds.), Manned systems design: Methods, equipment, and applications (pp. 375–393). New York: Plenum.
63.
Simpson, C. A. (1981c, April). Programming “phoneme” voice synthesizers phonetically. In J. C. Warren (Ed.), The Best of the Computer Faires, Volume VI: Conference proceedings of the Sixth West Coast Computer Faire (pp. 84–90). Woodside, CA: West Coast Computer Faire.
64.
Simpson, C. A. (1983a, December). Advanced technology-New fixes or new problems? Verbal communications in the aviation system. Paper presented at Beyond Pilot Error: A Symposium of Scientific Focus, sponsored by the Air Line Pilots Association, Washington, DC.
65.
Simpson, C. A. (1983b, March). Evaluating computer speech devices for your application. In J. C. Warren (Ed.), Proceedings of the Seventh West Coast Computer Faire (pp. 395–401). Woodside, CA: West Coast Computer Faire.
66.
Simpson, C. A. (1984). Integrated voice controls and speech displays for rotorcraft mission management (SAE Technical Paper Series 831523). SAE 1983 Transactions, 92(4), 271–280. Warrendale, PA: SAE.
67.
Simpson, C. A. (in press). Voice displays for single pilot IFR (NASA Contract Report CR-I72422). Hampton, VA: NASA-Langley Research Center.
68.
Simpson, C. A., Coler, C. R., and Huff, E. M. (1982, March). Human factors of voice I/O for aircraft cockpit controls and displays. In D. Pallett (Ed.), Proceedings of the Workshop on Standardization for Speech I/O Technology(pp. 159–166). Gaithersburg, MD: National Bureau of Standards.
69.
Simpson, C. A., and Marchionda-Frost, K. (1984). Synthesized speech rate and pitch effects on intelligibility of warning messages for pilots. Human Factors, 26, 509–517.
70.
Simpson, C. A., Marchionda-Frost, K., and Navarro, T. N. (1984, October). Comparison of voice types for helicopter voice warning systems (SAE Technical Paper series 841611). In Proceedings of the Third Aerospace Behavioral Engineering Technical Conference, 1984 SAE Aerospace Congress and Exposition. Warrendale, PA: SAE.
71.
Simpson, C. A., and Navarro, T. N. (1984, May). Intelligibility of computer generated speech as a function of multiple factors. In Proceedings of the National Aerospace and Electronics Conference (84CH1984–7 NAECON) (pp. 932–940). New York: IEEE.
72.
Simpson, C. A., and Williams, D. H. (1975, May). Human factors research problems in electronic voice warning system design. In Proceedings of the 11th Annual Conference on Manual Control (NASA TMX-62,464) (pp. 94–106). Moffett Field, CA: NASA-Ames Research Center.
73.
Simpson, C. A., and Williams, D. H. (1980). Response time effects of alerting tone and semantic context for synthesized voice cockpit warnings. Human Factors, 22, 319–320.
74.
Smith, C. (1983, September). Relating the performance of speech processors to the bit error rate. Speech Technology, 2(1), pp. 41–53.
75.
Smith, G. (1984). Five voice synthesizers. Byte: The Small Systems Journal, 9(10),337–347.
76.
Spine, T. M., Maynard, J. F., and Williges, B. H. (1983, September). Error correction strategies for voice recognition. ln Proceedings of the Voice Data Entry Systems Application Conference, Chicago, IL: American Voice I/O Society.
77.
Stem, K. R. (1984). An evaluation of written, graphics, and voice messages in proceduralized instructions. In Proceedings of the Human Factors Society 28th Annual Meeting (Vol. 1) (pp. 314–318). Santa Monica, CA: Human Factors Society.
78.
Thomas, J. C., Rosson, M. B., and Chodorow, M. (1984, October). Human factors and synthetic speech. In Proceedings of the Human Factors Society 28th Annual Meeting (Vol. 2) (pp. 763–767). Santa Monica, CA: Human Factors Society.
79.
U.S. Department of Defense. (1981, May). Human engineering design criteria for military systems, equipment, and facilities (MIL-STD-1472C). Washington, DC: Author.
80.
Van Cott, H. P., and Kinkade, R. G. (Eds.). (1972). Human engineering guide to equipment design (rev. ed). Washington, DC: U.S. Government Printing Office.
81.
Voorhees, J. W., Bucher, N. M., Huff, E. M., Simpson, C. A., and Williams, D. H. (1983). Voice interactive electronic warning system (VIEWS). In Proceedings of the IEEEI AIAA 5th Digital Avionics Systems Conference (83CHI839–0) (pp. 3.5.1–3.5.8). New York: IEEE.
82.
Voorhees, J. W., Marchionda, K., and Atchison, V. (1982, March). Auditory display of helicopter cockpit information. In Proceedings of the Workshop on standardization for Speech IIO Technology. Gaithersburg MD: National Bureau of Standards.
83.
Welch, J. R. (1977, September). Automated data entry analysis (RADC TR-77–306). Griffiss AFB, New York: Rome Air Development Center.
84.
Wickens, C. D., Sandry, D. L., and Vidulich, M. (1983). Compatibility and resource competition between modalities of input, central processing, and output. Human Factors, 25, 227–248.
85.
Wierwille, W. W., and Connor, S. A. (1983). Evaluation of 20 pilot workload measures using a psychomotor task in a moving-base aircraft simulator, Human Factors, 25,1–16.
86.
Williams, D. H., and Simpson, C. A. (1976). A systematic approach to advanced cockpit warning systems for air transport operations: Line pilot preferences. In Proceedings of the Aircraft Safety and Operating Problems Conference (NASA SP-416) (pp. 617–644). Norfolk, VA: NASA-Langley Research Center.
87.
Williges, B. H., and Williges, R. C. (1982). Structuring human/computer dialogue using speech technology. In Proceedings of the Workshop on Standardization for Speech IIO Technology. (pp. 143–151). Gaithersburg, MD: National Bureau of Standards.
88.
Woodson, W. E. (1981). Human factors design handbook. New York: McGraw-Hill.
89.
Zoltan-Ford, E. (1984). Reducing variability in natural language interactions with computers. In Proceedings of the Human Factors Society 28th Annual Meeting (Vol. 2.) (pp. 768–772). Santa Monica, CA: Human Factors Society.
