Own-race faces promote integrated audiovisual speech information

Abstract

The other-race effect indicates a perceptual advantage when processing own-race faces. This effect has been demonstrated in individuals’ recognition of facial identity and emotional expressions. However, it remains unclear whether the other-race effect also exists in multisensory domains. We conducted two experiments to provide evidence for the other-race effect in facial speech recognition, using the McGurk effect. Experiment 1 tested this issue among East Asian adults, examining the magnitude of the McGurk effect during stimuli using speakers from two different races (own-race vs. other-race). We found that own-race faces induced a stronger McGurk effect than other-race faces. Experiment 2 indicated that the other-race effect was not simply due to different levels of attention being paid to the mouths of own- and other-race speakers. Our findings demonstrated that own-race faces enhance the weight of visual input during audiovisual speech perception, and they provide evidence of the own-race effect in the audiovisual interaction for speech perception in adults.

Keywords

Other-race effect the McGurk effect audiovisual speech

Get full access to this article

View all access options for this article.

References

Aruffo

Shore

D. I.

(2012). Can you McGurk yourself? Self-face and self-voice in audiovisual speech. Psychonomic Bulletin & Review, 19(1), 66–72.

Beauchamp

M. S.

Argall

B. D.

Bodurka

Duyn

J. H.

Martin

(2004). Unraveling multisensory integration: Patchy organization within human STS multisensory cortex. Nature Neuroscience, 7, 1190–1192.

Bortolon

Raffard

(2018). Self-face advantage over familiar and unfamiliar faces: A three-level meta-analytic approach. Psychonomic Bulletin & Review, 25(4), 1287–1300. https://doi.org/10.3758/s13423-018-1487-9

Bothwell

R. K.

Bringham

J. C.

Malpass

R. S.

(1989). Cross-racial identification. Personality and Social Psychology Bulletin, 15(1), 19–25.

Brigham

J. C.

Bennett

L. B.

Meissner

C. A.

Mitchell

T. L.

(2007). The influence of race on eyewitness memory. In Lindsay

R. C. L.

Ross

D. F.

Read

J. D.

Toglia

M. P.

(Eds.), The handbook of eyewitness psychology: Vol. 2. Memory for people (pp. 257–281). Lawrence Erlbaum.

Bruce

Young

(1986). Understanding face recognition. British Journal of Psychology, 77, 305–327.

Butler

D. L.

Mattingley

J. B.

Cunnington

Suddendorf

(2013). Different neural processes accompany self-recognition in photographs across the lifespan: An ERP study using dizygotic twins. PLOS ONE, 8(9), Article e72586.

Byatt

Rhodes

(1998). Recognition of own-race and other-race caricatures: Implications for models of face recognition. Vision Research, 38(15), 2455–2468.

Calvert

G. A.

Campbell

Brammer

M. J.

(2000). Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Current Biology, 10, 649–657.

10.

De Gelder

Bertelson

(2003). Multisensory integration, perception, and ecological validity. Trends in Cognitive Sciences, 7(10), 460–467.

11.

De Gelder

Vroomen

van der Heide

(1991). Face recognition and lip-reading in autism. European Journal of Cognitive Psychology, 3, 69–86.

12.

C. S.

Wang

Quinn

P. C.

Lee

(2012). Adults scan own- and other-race faces differently. PLOS ONE, 7(6), Article e37688. https://doi.org/10.1371/journal.pone.0037688

13.

Gurler

Doyle

Walker

Magnotti

Beauchamp

(2015). A link between individual differences in multisensory speech perception and eye movements. Attention, Perception, & Psychophysics, 77, 1333–1341.

14.

Irwin

J. R.

Whalen

D. H.

Fowler

C. A.

(2006). A sex difference in visual influence on heard speech. Perception & Psychophysics, 68(4), 582–592. https://doi.org/10.3758/BF03208760

15.

Kelly

D. J.

Liu

Lee

Quinn

P. C.

Pascalis

Slater

A. M.

(2009). Development of the other-race effect during infancy: Evidence toward universality? Journal of Experimental Child Psychology, 104, 105–114.

16.

Kelly

D. J.

Quinn

P. C.

Slater

A. M.

Lee

Pascalis

(2007). The other-race effect develops during infancy: Evidence of perceptual narrowing. Psychological Science, 18, 1084–1089.

17.

Lewkowicz

D. J.

Ghazanfar

A. A.

(2009). The emergence of multisensory systems through perceptual narrowing. Trends in Cognitive. Science, 13, 470–478.

18.

Lewkowicz

D. J.

Hansen-Tift

A. M.

(2012). Infants deploy selective attention to the mouth of a talking face when learning speech. Proceedings of the National Academy of Sciences of the United States of America, 109(5), 1431–1436. https://doi.org/10.1073/pnas.1114783109

19.

Liberman

A. M.

Mattingly

I. G.

(1985). The motor theory of speech perception revised. Cognition, 21, 1–36.

20.

Liu

Quinn

P. C.

Wheeler

Xiao

Lee

(2011). Similarity and difference in the processing of same- and other-race faces as revealed by eye tracking in 4- to 9-month-olds. Journal of Experimental Child Psychology, 108, 180–189.

21.

Mallick

D. B.

Magnotti

J. F.

Beauchamp

M. S.

(2015). Variability and stability in the McGurk effect: Contributions of participants, stimuli, time, and response type. Psychonomic Bulletin & Review, 22(5), 1299–1307. https://doi.org/10.3758/s13423-015-0817-4

22.

Malpass

R. S.

Kravitz

(1969). Recognition for faces of own and other “race.” Journal of Personality & Social Psychology, 13, 330–335.

23.

McGurk

MacDonald

J. W.

(1976). Hearing lips and seeing voices. Nature, 264, 746–748.

24.

Minar

N. J.

Lewkowicz

D. J.

(2017). Overcoming the other-race effect in infancy with multisensory redundancy: 10–12-month-olds discriminate dynamic other-race faces producing speech. Developmental Science, 21(4), Article e12604.

25.

Nath

A. R.

Beauchamp

M. S.

(2012). A neural basis for interindividual differences in the McGurk effect, a multisensory speech illusion. NeuroImage, 59(1), 781–787.

26.

Nelson

C. A.

(2001). The development and neural bases of face recognition. Infant and Child Development, 10, 3–18.

27.

O’Toole

Peterson

Deffenbacher

K. A.

(1996). An “other-race effect” for categorizing faces by sex. Perception, 25, 669–676.

28.

Pascalis

de Haan

Nelson

C. A.

(2002). Is face processing species-specific during the first year of life? Science, 296, 1321–1323.

29.

Pascalis

Loevenbruck

Quinn

P. C.

Kandel

Tanaka

J. W.

Lee

(2014). On the links among face processing, language processing, and narrowing during development. Child Development Perspective, 8(2), 65–70.

30.

Saalasti

Kätsyri

Tiippana

Laine-Hernandez

von Wendt

Sams

(2012). Audiovisual speech perception and eye gaze behavior of adults with Asperger syndrome. Journal of Autism and Developmental Disorders, 42(8), 1606–1615.

31.

Schwartz

J.-L.

Tiippana

Andersen

(2010). Disentangling unisensory from fusion effects in the attentional modulation of McGurk effects: A Bayesian modeling study suggests that fusion is attention-dependent. In AVSP 2010: 9th International Conference on Auditory-Visual Speech Processing (pp. 23–27). https://hal.archives-ouvertes.fr/hal-00941284/document

32.

Segal

S. C.

Reyes

Gobin

K. C.

Moulson

M. C.

(2019). Children’s recognition of emotion expressed by own-race versus other-race faces. Journal of Experimental Child Psychology, 182, 102–113.

33.

Sekiyama

(1994). Differences in auditory-visual speech perception between Japanese and Americans: McGurk effect as a function of incompatibility. Journal of Acoustic Society of Japan, 15, 143–158.

34.

Sekiyama

Burnham

(2008). Impact of language on development of auditory-visual speech perception. Developmental Science, 11, 303–317.

35.

Sekiyama

Tohkura

(1991). McGurk effect in non-English listeners: Few visual effects for Japanese subjects hearing Japanese syllables of high auditory intelligibility. Journal of the Acoustical Society of America, 90, 1797–1805.

36.

Seltman

H. J.

(2018). Experimental design and analysis. http://www.stat.cmu.edu/~hseltman/309/Book/Book.pdf

37.

Sumby

W. H.

Pollack

(1954). Visual contribution to speech intelligibility in noise. The Journal of the Acoustical Society of America, 26(2), 212–215.

38.

Tanaka

J. W.

Curran

Porterfield

A. L.

Collins

(2006). Activation of preexisting and acquired face representations: The N250 event-related potential as an index of face familiarity. Journal of Cognitive Neuroscience, 18(9), 1488–1497.

39.

Tiippana

Andersen

T. S.

Sams

(2004). Visual attention modulates audiovisual speech perception. European Journal of Cognitive Psychology, 16(3), 457–472. https://doi.org/10.1080/09541440340000268

40.

Ujiie

Takahashi

(2021). Weaker McGurk Effect for Rubin’s Vase-Type Speech in People With High Autistic Traits. Multisensory Research, 34(6), 663–679.

41.

Ujiie

Asai

Wakabayashi

(2015). The relationship between level of autistic traits and local bias in the context of the McGurk effect. Frontiers in Psychology, 6, Article 891.

42.

Ujiie

Asai

Wakabayashi

(2018). Individual differences and the effect of face configuration contexts in the McGurk effect. Experimental Brain Research, 236(4), 973–984.

43.

Ujiie

Kanazawa

Yamaguchi

M. K.

(2020). The other-race-effect on audiovisual speech integration in infants: A NIRS study. Frontiers in Psychology, 11, Article 971.

44.

Valentine

(1991). A unified account of the effects of distinctiveness, inversion, and race in face recognition. The Quarterly Journal of Experimental Psychology Section A: Human Experimental Psychology, 43(2), 161–204.

45.

Walker

Bruce

O’Malley

(1995). Facial identity and facial speech processing: Familiar faces and voices in the McGurk effect. Perception & Psychophysics, 57(8), 1124–1133.

46.

Zhou

Cheng

Yue

Tredoux

Wang

(2015). Own-race faces capture attention faster than other-race faces: Evidence from response time and the N2pc. PLOS ONE, 10(6), Article e0127709. https://doi.org/10.1371/journal.pone.0127709