Previous comprehension studies using postperceptual memory tests have often reported negligible differences in performance between natural speech and several kinds of synthetic speech produced by rule, despite large differences in segmental intelligibility. The present experiments investigated the comprehension of natural and synthetic speech using two different on-line tasks: word monitoring and sentence-by-sentence listening. On-line task performance was slower and less accurate for passages of synthetic speech than for passages of natural speech. Recognition memory performance in both experiments was less accurate following passages of synthetic speech than of natural speech. Monitoring performance, sentence listening times, and recognition memory accuracy all showed moderate correlations with intelligibility scores obtained using the Modified Rhyme Test. The results suggest that poorer comprehension of passages of synthetic speech is attributable in part to the greater encoding demands of synthetic speech. In contrast to earlier studies, the present results demonstrate that on-line tasks can be used to measure differences in comprehension performance between natural and synthetic speech.
Get full access to this article
View all access options for this article.
References
1.
Allen, J., Klatt, D. H., and Hunnicutt, S. (1987). From text to speech: The MITalk system. Cambridge, England: Cambridge University Press.
2.
Bartlett, F. C. (1932). Remembering. Cambridge, England: Cambridge University Press.
3.
Blank, M. A., Pisoni, D. B., and McClaskey, C. L. (1981). Effects of target monitoring on understanding fluent speech. Perception and Psychophysics, 29, 383–388.
4.
Britton, B. K., Westbrook, R. D., and Holdredge, T. (1978). Reading and cognitive capacity: Effects of text difficulty. Journal of Experimental Psychology: Human Learning and Memory, 4, 582–591.
5.
Brunner, H., and Pisoni, D. B. (1982). Some effects of perceptual load on spoken text comprehension. Journal of Verbal Learning and Verbal Behavior, 21, 186–195.
6.
Cirilo, R. K. (1981). Referential coherence and text structure in story Comprehension. Journal of Verbal Learning and Verbal Behavior, 20, 358–367.
7.
Cirilo, R. K., and Foss, D. J. (1980). Text structure and reading time for sentences. Journal of Verbal Learning and Verbal Behavior, 19, 96–109.
8.
Cutler, A. (1976). Phoneme-monitoring reaction time as a function of preceding intonation contour. Perception and Psychophysics, 20, 55–60.
9.
Educational Testing Service. (1960). Cooperative English tests: Reading comprehension, Form IB. Princeton, NJ: Author.
10.
Flesch, R. (1949). The art of readable writing. New York: Harper.
11.
Foss, D. J. (1969). Decision processes during sentence comprehension: Effects of lexical item difficulty and position upon decision times. Journal of Verbal Learning and Verbal Behavior, 8, 457–462.
12.
Graesser, A. C., Hoffman, N. L., and Clark, L. F. (1980). Structural components in reading times. Journal of Verbal Learning and Verbal Behavior, 19, 135–151.
13.
Gunning, R. (1968). The technique of clear writing. New York: McGraw-Hill.
14.
Haberlandt, K. (1984). Components of sentence and reading times. In D. E. Kieras and M. A. Just (Eds.), New methods in reading comprehension research (pp. 219–251). Hillsdale, NJ: Erlbaum.
15.
Harcourt Brace Jovanovich. (1972a). Iowa Silent Reading Tests: Level 3, Form E. New York: Author.
16.
Harcourt Brace Jovanovich. (1972b). Stanford Test of Academic Skills: Reading, College Level, 11-A. New York: Author.
17.
Havil, S. E., and Clark, H. H. (1974). What's new? Acquiring new information as a process in comprehension. Journal of Verbal Learning and Verbal Behavior, 13, 512–521.
18.
Hersch, H. M., and Tartarglia, L. (1983). Understanding synthetic speech. Maynard, MA: Digital Equipment Corporation, User Research Group, Corporate Research and Architecture.
19.
Hoover, J., Reichle, J., van Tasell, D., and Cole, D. (1987). The intelligibility of synthesized speech: Echo II versus Votrax. Journal of Speech and Hearing Research, 30, 425–431.
20.
Houghton-Mifflin. (1973). The Nelson-Denny Reading Test: Form D. Boston: Author.
21.
House, A. S., Williams, C. E., Hecker, M. H., and Kryter, K. D. (1965). Articulation-testing methods: Consonantal differentiation with a closed-response set. Journal of the Acoustical Society of America, 37, 158–166.
22.
Jenkins, J. J., and Franklin, L. D. (1981. November). Recall of passages of synthetic speech. Paper presented at the Psychonomics Society Meeting, Philadelphia, PA.
23.
Just, M. A., and Carpenter, P. A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87, 329–354.
24.
Kahneman, D. (1973). Attention and effort. Englewood Cliffs, NJ: Prentice-Hall.
25.
Kieras, D. E. (1974). Analysis of the effect of word properties and limited reading time in a sentence comprehension and verification task. Unpublished doctoral dissertation, University of Michigan, Ann Arbor, MI.
26.
Kintsch, W., and Keenan, J. M. (1973). Reading rate as a function of the number of propositions in the base structure of sentences. Cognitive Psychology, 5, 257–274.
27.
Kintsch, W., and van Dijk, T. A. (1978). Toward a model of text comprehension and production. Psychological Review, 85, 363–394.
28.
Klatt, D. H. (1987). Review of text-to-speech conversion for English. Journal of the Acoustical Society of America, 82, 737–793.
29.
LaBerge, D., and Samuels, S. L. (1974). Toward a theory of automatic information processing in reading. Cognitive Psychology, 6, 293–323.
30.
Lee, L., and Nusbaum, H. C. (1989). The effects of perceptual learning on capacity demands for recognizing synthetic speech. Journal of the Acoustical Society of America, 85(Suppl. 1), S25.
31.
Lesgold, A. M., Roth, S. F., and Curtis, M. E. (1979). Foregrounding effects in discourse comprehension. Journal of Verbal Learning and Verbal Behavior, 18, 291–308.
32.
Levelt, W. J. M. (1978). A survey of studies in sentence perception. In W. J. M. Levelt and G. B. Flores d'Arcais (Eds.), Studies in the perception of language (pp. 1–74). New York: Wiley.
33.
Logan, J. S., Greene, B. G., and Pisoni, D. B. (1989). Intelligibility of eight text-to-speech systems. Journal of the Acoustical Society of America, 86, 566–587.
34.
Luce, P. A. (1981). Comprehension of fluent synthetic speech produced by rule (Research on Speech Perception, Progress Report 7). Bloomington: Indiana University, Speech Research Laboratory.
35.
Luce, P. A., and Carrell, T. D. (1981). Creating and editing waveforms using WAVES (Research on Speech Perception, Progress Report 7). Bloomington: Indiana University, Speech Research Laboratory.
36.
Luce, P. A., Feustel, T. C., and Pisoni, D. B. (1983). Capacity demands in short-term memory for synthetic and natural speech. Human Factors, 25, 17–32.
37.
Manous, L. M., Pisoni, D. B., Dedina, M. J., and Nusbaum, H. C. (1985). Comprehension of natural and synthetic speech using a sentence verification task (Research on Speech Perception, Progress Report 11). Bloomington: Indiana University, Speech Research Laboratory.
38.
McCauley, M. E. (1984). Human factors in voice technology. In F. A. Muckler (Ed.), Human factors review (pp. 131–166). Santa Monica, CA: Human Factors Society.
39.
McHugh, A. (1976). Listener preference and comprehension tests of stress algorithms for a text-to-phonetic speech synthesis program (Naval Research Laboratory Report 8015). Washington, DC: Naval Research Laboratory.
40.
Miller, G. A., and McKean, K. O. (1964). A chronometric study of some relations between sentences. Quarterly Journal of Experimental Psychology, 16, 297–308.
41.
Mimmack, P. C. (1982). Sentence-by-sentence listening times for spoken passages: Text structure and listeners' goals. Unpublished master's thesis, Indiana University, Bloomington, IN.
42.
Moody, T. S., and Joost, M. G. (1987). The effects of various types of speech output on comprehension rates. In H. J. Bullinger and B. Shackel (Eds.), Human computer interactions (pp. 573–579). Amsterdam: Elsevier.
43.
Nye, P. W., and Gaitenby, J. (1973). Consonant intelligibility in synthetic speech and in a natural speech control (Modified Rhyme Test results) (Haskins Laboratory Status Report on Speech Research, SR-33). New Haven, CT: Haskins Laboratories.
44.
Pauk, W. (1983). Six-way paragraphs: Middle level. Providence, RI: Jamestown.
45.
Pisoni, D. B., and Hunnicutt, S. (1980). Perceptual evaluation of MITalk: The MIT unrestricted text-to-speech system. In 1980 IEEE International Conference Record on Acoustics, Speech, and Signal Processing (pp. 572–575). New York: IEEE.
46.
Pisoni, D. B., Nusbaum, H. C., and Greene, B. G. (1985). Perception of synthetic speech generated by rule. Proceedings of the IEEE, 11, 1665–1676.
47.
Posner, M. I., and Boies, S. J. (1971). Components of attention. Psychological Review, 78, 391–408.
48.
Reed, A. V. (1976). List length and the time-course of recognition in immediate memory. Memory and Cognition, 4, 16–30.
49.
Schwab, E. C., Nusbaum, H. C., and Pisoni, D. B. (1985). Some effects of training on the perception of synthetic speech. Human Factors, 27, 395–408.
50.
Wickens, C. D. (1987). Information processing, decision making, and cognition. In G. Salvendy (Ed.), Handbook of human factors (pp. 72–107). New York: Wiley.
