Face-to-face conversations in every day life are conducted over a range of distances. However, previous research provides only limited indications of the effects of distance on visual and audiovisual speech recognition. We report an experiment which investigated effects of distance on perception of unimodal visual speech and congruent and incongruent audiovisual speech using a talking face presented at distances of 1, 5, 10, 20, and 30m and auditory, visual, congruent, and incongruent forms of the syllables /ba/, /bi/, /ga/, and /gi/. Identification of unimodal visual speech was unaffected by increasing distance to 10m, but was impaired at 20 and 30m. However, despite these drops in unimodal visual speech identification, visual speech improved performance with congruent auditory speech at all distances and impaired performance with incongruent auditory speech at distances up to 20m, indicating that auditory speech recognition is influenced by visual speech even when encoded from distant faces. Implications of these findings for understanding visual and audiovisual speech recognition are discussed.
Aubert, H. (1886). Die Bewegungsempfindung. Archiv fur die Gesamte Psychologie, 39, 347-370.
2.
Binnie, C.A., Montgomery, A.A., & Jackson, P.L. (1974). Auditory and visual contributions to the perception of selected English consonants for normally hearing and hearing-impaired listeners . In H. Birk Nielson & E. Kampp (Eds.), Visual and audiovisual perception of speech (Scandinavian Supplementum 4, pp. 181-209). Stockholm: Almquist and Wiksell.
3.
Burr, D.C. (1991). Human sensitivity to flicker and motion. In J.J. Kulikowski, V. Walsh & I.J. Murray (Eds.), Vision and visual dysfunction, Vol 5 (pp.147-159). London: MacMillan.
4.
Bertelson, P., Vroomen, J., Wiegard, G., & De Gelder, B. (1994). Exploring the relation between McGurk interference and ventriloquism. Paper presented at the International Conference on Spoken Language Processing, September, Yokohama , Japan.
5.
Campbell, R., Dodd, B., & Burnham, D. (1998). Hearing by eye II: Advances in the psychology of speech reading and auditory-visual speech. London: Psychology Press.
6.
Conrad, R. (1979). The deaf schoolchild. London : Harper and Row.
7.
Dixon, N., & Spitz, L. (1980). The detection of audiovisual desynchrony. Perception, 9, 719-721.
8.
Erber, N. (1969). Interaction of audition and vision in the recognition of oral speech stimuli. Journal of Speech and Hearing Research , 12, 423-425.
9.
Erber, N.P. (1971). Effects of distance on the visual reception of speech. Journal of Speech and Hearing Research, 14, 848-857.
10.
Erber, N.P. (1974). Effects of angle, distance and illumination on visual reception of speech by profoundly deaf children. Journal of Speech and Hearing Research, 17, 99-112.
11.
Gailey, L. (1987). Psychological parameters of lip-reading skill . In B. Dodd & R. Campbell (Eds.), Hearing by eye: The psychology of lip-reading (pp.115-141). Hillsdale, NJ: Erlbaum.
12.
Gordon, P.C., Eberhardt, J.L., & Rueckl, J.G. (1993). Attentional modulation of the phonetic significance of acoustic cues. Cognitive Psychology, 25, 1-42.
13.
Green, K.P. (1994). The influence of inverted face on the McGurk effect . Poster presented at the Spring meeting of the Acoustical Society of America, Cambridge, Massachusetts. Journal of the Acoustical Society of America, 95, 3014 (Abstract).
14.
Green, K.P. (1996). The use of auditory and visual information in phonetic perception. In D. Stork & M. E. Hennecke (Eds.), Speechreading by humans and machines (pp. 55-78). Berlin: SpringerVerlag.
15.
Green, K.P. (1998). The use of auditory and visual information during phonetic processing: Implications for theories of speech perception. In R. Campbell, B. Dodd, & D. Burnham (Eds.), Hearing by eye II: Advances in the psychology of speechreading and auditory-visual speech (pp. 3-25). London: Psychology Press.
16.
Green, K.P., Kuhl P. K., Meltzoff, A.N., & Stevens, E.B. (1991). Integrating speech information across talkers, gender and sensory modality: Female faces and male voices in the McGurk effect . Perception and Psychophysics, 50, 524-536.
17.
Johnson, J.A., Rosenblum, L.D., & Saldaña, H.M. (1994). The contribution of a reduced visual image to speech perception in noise. Poster presented at the Spring, 1994, meeting of the Acoustical Society of America, Cambridge, Massachusetts. Journal of the Acoustical Society of America , 95, 3009 (Abstract).
18.
Jordan, T.R., & Bevan, K.M. (1997). Seeing and hearing rotating faces: Influences of facial orientation on visual and audio-visual speech recognition. Journal of Experimental Psychology: Human Perception and Performance, 23, 388-403.
19.
Jordan, T.R., & Sergeant, P.C. (1998). Effects of facial image size on visual and audio visual speech recognition. In R. Campbell, B. Dodd, & D. Burnham (Eds.), Hearing by eye II: Advances in the psychology of speech reading and audiovisual speech (pp.155-176). London: Taylor and Francis Psychology Press.
20.
Jordan, T.R., McCotter, M.V., & Thomas, S.M. (2000). Visual and audiovisual speech perception with color and gray scale facial images. Perception and Psychophysics , in press.
21.
MacLeod, A., & Summerfield, Q. (1987). Quantifying the contribution of vision to speech perception in noise. British Journal of Audiology, 12, 131-141.
22.
MacLeod, A., & Summerfield, Q. (1990). A procedure for measuring auditory and audio-visual speech-reception thresholds for sentences in noise: Rationale, evaluation, and recommendations for use. British Journal of Audiology, 24, 29-43.
23.
Massaro, D.W. (1998). Perceiving talking faces: From speech perception to a behavioral principle. Cambridge, MA: MIT Press.
24.
Massaro, D.W., & Cohen, M.M. (1990). Perception of synthesized audible and visible speech. Psychological Science, 1, 55-63.
25.
Massaro, D.W., & Cohen, M.M. (1996). Perceiving speech from inverted faces. Perception and Psychophysics, 58, 1047-1065.
26.
Massaro, D.W., Cohen, M.M., & Smeele, P.M.T. (1996). Perception of asynchronous and conflicting visual and auditory speech. Journal of the Acoustical Society of America. 100, 1777- 1786.
27.
McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746-748.
28.
McKee, S.P. (1991). The physical constraints on visual hyperacuity . In J. J. Kulikowski, V. Walsh, & I. J. Murray (Eds.), Vision and visual dysfunction, Vol 5 (pp.221-233). London: MacMillan.
29.
Middleweerd, M.J., & Plomp, R. (1987). The effect of speechreading on the speech-reception threshold of sentences in noise. Journal of the Acoustical Society of America, 82, 2145- 2147.
30.
Montgomery, A.A., & Jackson, P.L. (1983). Physical characteristics of the lips underlying vowel lipreading performance. Journal of the Acoustical Society of America , 73, 2134-2144.
31.
Munhall, K.G., Gribble, P., Sacco, L., & Ward, M. (1996). Temporal constraints on the McGurk effect. Perception and Psychophysics, 58, 351-362.
32.
Neely, K.K. (1956). Effect of visual factors on the intelligibility of speech. Journal of the Acoustical Society of America, 28, 1275-1277.
33.
O'Neill, J.J. (1954). Contributions of the visual components of oral symbols to speech comprehension. Journal of Speech and Hearing Disorders, 19, 429-439.
34.
Petajan, E.D. (1984). Automatic lipreading to enhance speech recognition . Proceedings of the IEEE Communications Society , 265-272.
35.
Reisberg, D., McLean, J., & Goldfield, A. (1987). Easy to hear but hard to understand: A lipreading advantage with intact auditory stimuli. In B. Dodd & R. Campbell (Eds.), Hearing by eye: The psychology of lip reading (pp. 97-113). London : Lawrence Erlbaum Associates Ltd.
36.
Repp, B.H., Manuel, S.Y., Liberman, A.M., & Studdert-Kennedy, M. (1983). Exploring the McGurk Effect. Paper presented at the 24th annual meeting of the Psychonomic Society, San Diego, California, November, 1983.
37.
Ronnberg, J. (1993). Cognitive characteristics of skilled tactiling: The case of GS. European Journal of Cognitive Psychology, 5, 19-33.
38.
Rosenblum, L.D., & Saldaña, H.M. (1992). Discrimination tests of visually influenced syllables . Perception and Psychophysics , 52, 461-473.
39.
Rosenblum, L.D., & Saldaña, H.M. (1996). An audiovisual test of kinematic primitives for visual speech perception. Journal of Experimental Psychology: Human Perception and Performance, 22, 318-331.
40.
Rosenblum, L.D., Johnson, J.A., & Saldaña, H.M. (1995). Determining the kinematic features for visual speech perception. Unpublished manuscript.
41.
Saldaña, H.M., Rosenblum, L.D., & Osinga, T. (1992). Visual influence on heard speech syllables with a reduced visual image. Journal of the Acoustical Society of America, 92, 2340 (Abstract).
42.
Samuelsson, S., & Ronnberg, J. (1993). Implicit and explicit use of scripted constraints in lipreading. European Journal of Cognitive Psychology, 5, 201-233.
43.
Sekiyama, K., & Tohkura, Y. (1991). McGurk effect in non-English listeners: Few visual effects for Japanese subjects hearing Japanese syllables of high auditory intelligibility. Journal of the Acoustical Society of America , 90, 1797-1805.
44.
Sudman, J.A., & Berger, K.W. (1971). Two-dimension versus three-dimension viewing in speechreading. Journal of Communication Disorders, 4, 195-198.
45.
Sumby, W.H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212-215.
46.
Summerfield, A.Q. (1987). Some preliminaries to a comprehensive account of audio-visual speech perception. In B. Dodd & R. Campbell (Eds.), Hearing by eye: The psychology of lip-reading (pp. 3-51). Hillsdale, NJ: Erlbaum.
47.
Summerfield, A.Q. (1992). Lipreading and audio-visual speech perception . In V. Bruce, A. Cowey, A. W. Ellis, & D. I. Perrett (Eds.), Processing the facial image (pp.71-78). Oxford: Oxford University Press.
48.
Summerfield, A.Q., & McGrath, M. (1984). Detection and resolution of audio-visualincompatibility in the perception of vowels. Quarterly Journal of Experimental Psychology, 36A, 51-74.
49.
Summerfield, A.Q., MacLeod, P., McGrath, M., & Brooke, N.M. (1989). Lips, teeth, and the benefits of lipreading. In A. W. Young & H. B. Ellis (Eds.), Handbook of research on face processing (pp.223-233). Amsterdam: North Holland.
50.
Utley, J. (1946). A test of lip-reading ability. Journal of Speech Disorders, 11, 109-116.
51.
Vandegrind, W.A., Koenderink, J.J., & Vandoorn, A.J. (1992). Viewing distance invariance of movement detection. Experimental Brain Research, 91, 135-150.
52.
Walden, B.E., Prosek, R.A., Montgomery, A.A., Scherr, C.K., & Jones, C.J. (1977). Effects of training on the visual recognition of consonants. Journal of Speech and Hearing Research, 20, 130-145.
53.
Wandell, B.A. (1995). Foundations of vision. Sunderland, MA: Sinauer Associates.
54.
Woodward, M.F., & Barber, C.G. (1960). Phoneme perception in lipreading. Journal of Speech and Hearing Research, 3, 212-222.
