The role of visual cues on word recognition in noise in monolingual and bilingual toddlers

Abstract

Aims and objectives/purpose/research questions:

The purpose of this study was to examine how visual speech cues impact the ability to recognize words in the presence of background noise in toddlers who vary in language exposure (i.e., being raised bilingual or monolingual).

Design/methodology/approach:

Seventy-two children (36 monolinguals and 36 bilinguals) between the ages of 24 and 35 months completed an online Preferential Looking task where they saw picture-pairs of familiar objects (e.g., balloon and apple) on a screen and heard sentences instructing them to locate one of the objects (e.g., look at the apple!). Speech was heard in quiet and in the presence of competing noise, and with or without visual speech cues.

Data and analysis:

Children’s eye movements were coded offline. We then identified the proportion of time they fixated on the correct object on the screen. Performance within and between groups was compared using linear mixed effects models.

Findings/conclusions:

Bilinguals performed worse than monolinguals when there was both noise and no visual speech cues present. Within group differences showed that monolinguals performed similarly in every condition while bilinguals had significantly lower accuracy in every condition compared to the easiest condition (i.e., visual speech cues present without noise).

Originality:

This is the first study to examine how the presence of visual speech cues impacts word recognition in noise in both monolingual and bilingual infants.

Significance/implications:

High noise levels are present in everyday environments and children might not always have access to visual speech cues while hearing speech. Thus, this work has implications for learning and education practices with young bilinguals.

Limitations (optional):

We used an indirect measure of noise in the home to explore how this related to processing in noise in a laboratory task. Future work should use a direct measure of noise in the home to further explore this.

Keywords

Audiovisual speech perception word recognition bilingualism adverse listening conditions infant development

Get full access to this article

View all access options for this article.

References

Anusha

P. V.

Anuradha

Murty

P. C.

Kiran

C. S.

(2019). Detecting outliers in high dimensional data sets using Z-score methodology. International Journal of Innovative Technology and Exploring Engineering, 9(1), 48–53.

Bsharat-Maalouf

Degani

Karawani

(2023). The involvement of listening effort in explaining bilingual listening under adverse listening conditions. Trends in Hearing, 27, 23312165231205107. https://doi.org/10.1177/23312165231205107

Burnham

Dodd

(2004). Auditory–visual speech integration by prelinguistic infants: Perception of an emergent consonant in the McGurk effect. Developmental Psychobiology, 45(4), 204–220. https://doi.org/10.1002/dev.20032

Byers-Heinlein

Morin-Lessard

Lew-Williams

(2017). Bilingual infants control their languages as they listen. Proceedings of the National Academy of Sciences, 114(34), 9032–9037. https://doi.org/10.1073/pnas.1703220114

Datavyu Team (2014). Datavyu: A video coding tool. Databrary Project, New York University. http://datavyu.org.

Desjardins

R. N.

Werker

J. F.

(2004). Is the integration of heard and seen speech mandatory for infants? Developmental Psychobiology, 45(4), 187–203. https://doi.org/10.1002/dev.20033

Erickson

L. C.

Newman

R. S.

(2017). Influences of background noise on infants and children. Current Directions in Psychological Science, 26(5), 451–457. https://doi.org/10.1177/0963721417709087

Fennell

C. T.

Byers-Heinlein

Werker

J. F.

(2007). Using speech sounds to guide word learning: The case of bilingual infants. Child Development, 78(5), 1510–1525. https://doi.org/10.1111/j.1467-8624.2007.01080.x

Gómez-Merino

Gheller

Spicciarelli

Trevisi

(2020). Pupillometry as a measure for listening effort in children: A review. Hearing Balance and Communication, 18(3), 152–158. https://doi.org/10.1080/21695717.2020.1807256

10.

Haith

M. M.

Wentworth

Canfield

R. L.

(1993). The formation of expectations in early infancy. Advances in Infancy Research, 8, 251–297.

11.

Höhle

Bijeljac-Babic

Nazzi

(2020). Variability and stability in early language acquisition: Comparing monolingual and bilingual infants’ speech perception and word recognition. Bilingualism: Language and Cognition, 23(1), 56–71. https://doi.org/10.1017/S1366728919000348

12.

Hollich

Newman

R. S.

Jusczyk

P. W.

(2005). Infants’ Use of Synchronized Visual Information to Separate Streams of Speech. Child Development, 76(3), 598–613. https://doi.org/10.1111/j.1467-8624.2005.00866.x

13.

Klatte

Bergström

Lachmann

(2013). Does noise affect learning? A short review on noise effects on cognitive performance in children. Frontiers in Psychology, 4. https://doi.org/10.3389/fpsyg.2013.00578

14.

Krizman

Bradlow

A. R.

Lam

S. S.-Y.

Kraus

(2017). How bilinguals listen in noise: Linguistic and non-linguistic factors. Bilingualism: Language and Cognition, 20(4), 834–843. https://doi.org/10.1017/S1366728916000444

15.

Król

M. E.

(2018). Auditory noise increases the allocation of attention to the mouth, and the eyes pay the price: An eye-tracking study. PLoS ONE, 13(3), Article e0194491. https://doi.org/10.1371/journal.pone.0194491

16.

Lalonde

Holt

R. F.

(2015). Preschoolers benefit from visually salient speech cues. Journal of Speech, Language, and Hearing Research, 58(1), 135–150. https://doi.org/10.1044/2014_JSLHR-H-13-0343

17.

Morini

Newman

R. S.

(2020). Monolingual and bilingual word recognition and word learning in background noise. Language and Speech, 63(2), 381–403. https://doi.org/10.1177/0023830919846158

18.

Morini

Newman

R. S.

(2021). A comparison of monolingual and bilingual toddlers’ word recognition in noise. International Journal of Bilingualism, 25(5), 1446–1459. https://doi.org/10.1177/13670069211028664

19.

Newman

R. S.

(2005). The cocktail party effect in infants revisited: Listening to one’s name in noise. Developmental Psychology, 41(2), 352–362. https://doi.org/10.1037/0012-1649.41.2.352

20.

Newman

R. S.

(2006). Perceptual restoration in toddlers. Perception & Psychophysics, 68(4), 625–642. https://doi.org/10.3758/BF03208764

21.

Newman

R. S.

(2009). Infants’ listening in multitalker environments: Effect of the number of background talkers. Attention, Perception, & Psychophysics, 71(4), 822–836. https://doi.org/10.3758/APP.71.4.822

22.

Newman

R. S.

(2011). 2-year-olds’ speech understanding in multitalker environments. Infancy, 16(5), 447–470. https://doi.org/10.1111/j.1532-7078.2010.00062.x

23.

Newman

R. S.

Jusczyk

P. W.

(1996). The cocktail party effect in infants. Perception & Psychophysics, 58(8), 1145–1156.

24.

Ovans

Ayala

M. R.

Asmah

Montoute

Horne

A. O. V.

Morini

Huang

Y. T.

(2025). The feasibility of remote visual-world eye-tracking with young children. Open Mind, 9, 992–1019.

25.

Pearson

B. Z.

Fernández

S. C.

Oller

D. K.

(1993). Lexical development in bilingual infants and toddlers: Comparison to monolingual norms. Language Learning, 43(1), 93–120. https://doi.org/10.1111/j.1467-1770.1993.tb00174.x

26.

Pons

Bosch

Lewkowicz

D. J.

(2015). Bilingualism modulates infants’ selective attention to the mouth of a talking face. Psychological Science, 26(4), 490–498. https://doi.org/10.1177/0956797614568320

27.

Reetzke

Lam

B. P.-W.

Xie

Sheng

Chandrasekaran

(2016). Effect of simultaneous bilingualism on speech intelligibility across different masker types, modalities, and signal-to-noise ratios in school-age children. PLoS ONE, 11(12), Article e0168048. https://doi.org/10.1371/journal.pone.0168048

28.

Sebastián-Gallés

Albareda-Castellot

Weikum

W. M.

Werker

J. F.

(2012). A bilingual advantage in visual language discrimination in infancy. Psychological Science, 23(9), 994–999. https://doi.org/10.1177/0956797612436817

29.

Sebastian-Galles

Santolin

(2020). Bilingual acquisition: The early steps. Annual Review of Developmental Psychology, 2(1), 47–68. https://doi.org/10.1146/annurev-devpsych-013119-023724

30.

Shook

Marian

(2013). The bilingual language interaction network for comprehension of speech. Bilingualism: Language and Cognition, 16(2), 304–324. https://doi.org/10.1017/S1366728912000466

31.

Tan

S. H. J.

Kalashnikova

Burnham

(2023). Seeing a talking face matters: Infants’ segmentation of continuous auditory-visual speech. Infancy, 28(2), 277–300. https://doi.org/10.1111/infa.12509

32.

Teinonen

Aslin

R. N.

Alku

Csibra

(2008). Visual speech contributes to phonetic learning in 6-month-old infants. Cognition, 108(3), 850–855. https://doi.org/10.1016/j.cognition.2008.05.009

33.

Van Der Wel

Van Steenbergen

. (2018). Pupil dilation as an index of effort in cognitive control tasks: A review. Psychonomic Bulletin & Review, 25(6), 2005–2015. https://doi.org/10.3758/s13423-018-1432-y

34.

Weatherhead

Arredondo

M. M.

Nácar Garcia

Werker

J. F.

(2021). The role of audiovisual speech in fast-mapping and novel word retention in monolingual and bilingual 24-month-olds. Brain Sciences, 11(1), 114. https://doi.org/10.3390/brainsci11010114

35.

Weatherhead

White

K. S.

(2017). Read my lips: Visual speech influences word processing in infants. Cognition, 160, 103–109. https://doi.org/10.1016/j.cognition.2017.01.002

36.

Weikum

W. M.

Vouloumanos

Navarra

Soto-Faraco

Sebastián-Gallés

Werker

J. F.

(2007). Visual language discrimination in infancy. Science, 316(5828), 1159–1159. https://doi.org/10.1126/science.1137686

37.

Werker

J. F.

Byers-Heinlein

Fennell

C. T.

(2009). Bilingual beginnings to learning words. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1536), 3649–3663. https://doi.org/10.1098/rstb.2009.0105

38.

Werner

Fay

R. R.

Popper

A. N.

(Eds.). (2012). Human auditory development (Vol. 42). Springer. https://doi.org/10.1007/978-1-4614-1421-6

39.

Zekveld

A. A.

Koelewijn

Kramer

S. E.

(2018). The pupil dilation response to auditory stimuli: Current state of knowledge. Trends in Hearing, 22, 2331216518777174. https://doi.org/10.1177/2331216518777174

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB