In the McGurk effect (McGurk and MacDonald, 1976 Nature264 746–748), illusory auditory perception is produced if the visual information from lip movements is discrepant from the auditory information from the voice. A study is reported of the tolerance of the effect to varying levels of spatial degradation (videotaped images of a speaker's face were quantised by a mosaic transform). The illusory effect systematically decreased with an increase in the coarseness of the spatial quantisation. However, even with the coarsest level (11.2 pixels/face) the illusion did not completely disappear. In addition, those participants who did not experience the illusion nevertheless showed the effects of auditory–visual interaction in their clarity ratings of the auditory stimulus. It is concluded that auditory – visual interaction in visible speech perception is based on relatively coarse-spatial-scale information.
Get full access to this article
View all access options for this article.
References
1.
BachmannT, 1989“Microgenesis as traced by the transient paired-forms paradigm”Acta Psychologica703–17.
2.
BachmannT, 1991“Identification of spatially quantised tachistoscopic images of faces: How many pixels does it take to carry identity?”European Journal of Cognitive Psychology387–103.
3.
BachmannTKahuskN, 1997“The effects of coarseness of quantisation exposure duration and selective spatial attention on the perception of spatially quantised (‘blocked’) visual images”Perception261181–1196.
4.
BassiliJ M, 1978“Facial motion in the perception of faces and of emotional expression”Journal of Experimental Psychology: Human Perception and Performance4373–379.
5.
BassiliJ M, 1979“Emotion recognition: The role of facial movement and the relative importance of upper and lower areas of the face”Journal of Personality and Social Psychology372049–2058.
6.
BenoitCGuiard-MarignyTLe GoffBAdjoudaniA, 1996“Which components of the face do humans and machines best speechread?”, in Speechreading by Humans and Machines: Models, Systems and Applications Eds StorkD GHenneckeM E, NATO ASI Series (Berlin: Springer) pp 315–328.
7.
BertelsonP, 1998“Starting from the ventriloquist: The perception of multimodal events”, in Advances in Psychological Science volume II Biological and Cognitive Aspects Eds SabourinMraikF I M CRobertM (Hove, Sussex: Psychology Press) pp 419–439.
8.
BhatiaS KLakshminarayananVSamalAWellandG V, 1995“Human face perception in degraded images”Journal of Visual Communication and Image Representation6280–295.
9.
BrookeN MSummerfieldA Q, 1983“Analysis, synthesis and perception of visible articulatory movements”Journal of Phonetics1163–76.
10.
BruceV, 1988Recognising Faces (Hove, Sussex: Lawrence Erlbaum Associates).
11.
BruceVLangtonS, 1994“The use of pigmentation and shading information in recognising the sex and identities of faces”Perception23803–822.
12.
BruceVValentineT, 1988“When a nod's as good as a wink: The role of dynamic information in facial recognition”, in Practical Aspects of Memory: Current Research and Issuesvolume 1, Eds GrunebergM MMorrisP ESykesR N (Chichester, Sussex: John Wiley) pp 169–174.
13.
CaelliTYuzykJ, 1985“What is perceived when two images are combined”Perception1441–48.
14.
CalisGSterenborgJMaarseF, 1984“Initial microgenetic steps in single-glance face recognition”Acta Psychologica55215–230.
15.
CampbellC SMassaroD W, 1997“Perception of visible speech: Influence of spatial quantization”Perception26627–644.
16.
CostenN PParkerD MCrawI, 1994a“Spatial content and spatial quantisation effects in face recognition”Perception23129–146.
17.
CostenN PParkerD MCrawI, 1996“Effects of high-pass and low-pass spatial filtering on face identification”Perception & Psychophysics58602–612.
18.
CostenN PShepherdJ WEllisH DCrawI, 1994b“Masking of faces by facial and non-facial stimuli”Visual Cognition1227–251.
19.
DoddB, 1977“The role of vision in the perception of speech”Perception631–40.
20.
EastonR DBasalaM, 1982“Perceptual dominance during lipreading”Perception & Psychophysics32562–570.
21.
FiorentiniAMaffeiLSandiniG, 1983“The role of higher spatial frequencies in face perception”Perception12195–201.
22.
FowlerC A, 1996“Listeners do hear sounds, not tongues”Journal of the Acoustical Society of America991730–1741.
23.
GeorgeP AHoleG J, 1998“The influence of feature-based information in the age processing of unfamiliar faces”Perception27295–312.
HarmonL DJuleszB, 1973“Masking in visual recognition: Effects of two-dimensional filtered noise”Science1801194–1197.
26.
HughesH CNozawaGKitterleF, 1996“Global precedence, spatial frequency channels, and the statistics of natural images”Journal of Cognitive Neuroscience8197–230.
27.
LederH, 1996“Line drawings of faces reduce configural processing”Perception25355–366.
28.
MacDonaldJMcGurkH, 1978“Visual influences on speech perception processes”Perception & Psychophysics24253–257.
29.
McGurkHMacDonaldJ, 1976“Hearing lips and seeing voices”Nature (London)264746–748.
30.
MassaroD W, 1998Perceiving Talking Faces, from Speech Perception to a Behavioral Principle (Cambridge, MA: Bradford/MIT Press).
31.
MassaroD WCohenM M, 1983“Evaluation and integration of visual and auditory information in speech perception”Journal of Experimental Psychology: Human Perception and Performance9753–771.
32.
MassaroD WCohenM M, 1995“Perceiving talking faces”Current Directions in Psychological Science4104–109.
33.
MattinglyI GStuddert-KennedyM (Eds), 1991Modularity and the Motor Theory of Speech Perception (Hillsdale, NJ: Lawrence Erlbaum Associates).
34.
MunhallK GGribblePSaccoLWardM, 1996“Temporal constraints on the McGurk effect”Perception & Psychophysics58351–362.
35.
ParkerD MLishmanJ RHughesJ, 1996“Role of coarse and fine spatial information in face and object processing”Journal of Experimental Psychology: Human Perception and Performance221448–1466.
36.
ParkerD MLishmanJ RHughesJ, 1997“Evidence for the view that temporospatial integration in vision is temporally anisotropic”Perception261169–1180.
37.
RobertsMSummerfieldA Q, 1981“Audio-visual adaptation in speech perception”Perception & Psychophysics30309–314.
38.
SergentJ, 1986“Microgenesis of face perception”, in Aspects of Face Processing Eds EllisHJeevesMNewcombeFYoungA (Dordrecht: Martinus Nijhoff) pp 17–33.
39.
SpenceCDriverJ, 1997“Audiovisual links in exogenous covert spatial orienting”Perception & Psychophysics591–22.
40.
SpenceCNichollsM E RGillespieNDriverJ, 1998“Cross-modal links in exogenous covert spatial orienting between touch, audition, and vision”Perception & Psychophysics60544–557.
41.
StevenageS V, 1998“Which twin are you? A demonstration of induced categorical perception of identical twin faces”British Journal of Psychology8939–57.
42.
SummerfieldQ, 1979“Use of visual information for phonetic perception”Phonetics36314–331.
43.
SummerfieldA Q, 1992“Lipreading and audio-visual speech perception”Philosophical Transactions of the Royal Society of London, Series B33571–78.
44.
TrojeN FBülthoffH H, 1996“Face recognition under varying poses: The role of texture and shape”Vision Research361761–1771.
45.
UttalW RBaruchTAllenL, 1997“A parametric study of face recognition when image degradations are combined”Spatial Vision11179–204.
46.
Vatikiotis-BatesonEEigstiI MYanoSMunhallK G, 1998“Eye movement of perceivers during audiovisual speech perception”Perception & Psychophysics60926–940.
47.
WaldenB EProsekR AMontgomeryAScherrC KJonesC J, 1977“Effects of training on the visual recognition of consonants”Journal of Speech and Hearing Research20130–145.
48.
WalkerSBruceVO'MalleyC, 1995“Facial identity and facial speech processing: Familiar faces and voices in the McGurk effect”Perception & Psychophysics571124–1133.
49.
WallbottH G, 1991“The robustness of communication of emotion via facial expression: Emotion recognition from photographs with deteriorated pictorial quality”European Journal of Social Psychology2189–98.
50.
WallbottH G, 1992“Effects of distortion of spatial and temporal resolution of video stimuli on emotion attributions”Journal of Nonverbal Behavior165–20.
