Acoustic cues are short-lived and highly variable, which makes speech perception a difficult problem. However, most listeners solve this problem effortlessly. In the present experiment, we demonstrated that part of the solution lies in predicting upcoming speech sounds and that predictions are modulated by high-level expectations about the current sound. Participants heard isolated fricatives (e.g., “s,” “sh”) and predicted the upcoming vowel. Accuracy was above chance, which suggests that fine-grained detail in the signal can be used for prediction. A second group performed the same task but also saw a still face and a letter corresponding to the fricative. This group performed markedly better, which suggests that high-level knowledge modulates prediction by helping listeners form expectations about what the fricative should have sounded like. This suggests a form of data explanation operating in speech perception: Listeners account for variance due to their knowledge of the talker and current phoneme, and they use what is left over to make more accurate predictions about the next sound.
BeddorP. S.HarnsbergerJ. D.LindemannS. (2002). Language-specific patterns of vowel-to-vowel coarticulation: Acoustic structures and their perceptual correlates. Journal of Phonetics, 30, 591–627.
4.
BlankH.von KriegsteinK. (2013). Mechanisms of enhancing visual–speech recognition by prior auditory information. NeuroImage, 65, 109–118. doi:10.1016/j.neuroimage.2012.09.047
5.
BlumsteinS. E.StevensK. N. (1979). Acoustic invariance in speech production: Evidence from measurements of the spectral characteristics of stop consonants. Journal of the Acoustical Society of America, 66, 1001–1017.
6.
ClarkA. (2013). Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behavioral & Brain Sciences, 36, 181–204.
7.
ColeJ. S.LinebaughG.MunsonC.McMurrayB. (2010). Unmasking the acoustic effects of vowel-to-vowel coarticulation: A statistical modeling approach. Journal of Phonetics, 38, 167–184.
8.
DaniloffR.MollK. (1974). Coarticulation of lip rounding. In LassN. J. (Ed.), Experimental phonetics (pp. 100–114). New York, NY: MSS Information Corp.
9.
FowlerC. A.DekleD. J. (1991). Listening with eye and hand: Cross-modal contributions to speech perception. Journal of Experimental Psychology: Human Perception and Performance, 17, 816–828.
10.
FowlerC. A.SmithM. (1986). Speech perception as “vector analysis”: An approach to the problems of segmentation and invariance. In PerkellJ. S.KlattD. (Eds.), Invariance and variability in speech processes (pp. 123–136). Hillsdale, NJ: Erlbaum.
11.
GagnepainP.HensonR. N.DavisM. H. (2012). Temporal predictive codes for spoken words in auditory cortex. Current Biology, 22, 615–621.
12.
GoldingerS. D. (1998). Echoes of echoes? An episodic theory of lexical access. Psychological Review, 105, 251–279.
13.
GowD. W.Jr. (2001). Assimilation and anticipation in continuous spoken word recognition. Journal of Memory and Language, 45, 133–159.
14.
GowD. W.Jr. (2003). Feature parsing: Feature cue mapping in spoken word recognition. Perception & Psychophysics, 65, 575–590.
15.
GowD. W.Jr.McMurrayB. (2007). Word recognition and phonology: The case of English coronal place assimilation. In ColeJ.HualdeJ. I (Eds.), Laboratory phonology 9 (pp. 173–200). New York, NY: Mouton de Gruyter.
16.
HayJ.DragerK. (2010). Stuffed toys and speech perception. Linguistics, 48, 865–892.
17.
HoudeJ. F.NagarajanS. S.SekiharaK.MerzenichM. M. (2002). Modulation of the auditory cortex during speech: An MEG study. Journal of Cognitive Neuroscience, 14, 1125–1138.
18.
JohnsonK. C.StrandE. A.D’ImperioM. (1999). Auditory-visual integration of talker gender in vowel perception. Journal of Phonetics, 24, 359–384.
19.
JongmanA.WaylandR.WongS. (2000). Acoustic characteristics of English fricatives. Journal of the Acoustical Society of America, 106, 1252–1263.
20.
KleinschmidtD.JaegerF. (2015). Robust speech perception: Recognize the familiar, generalize to the similar, and adapt to the novel. Psychological Review, 122, 148–203.
21.
LibermanA. M.MattinglyI. (1985). The motor theory of speech perception revised. Cognition, 21, 1–36.
22.
MartinJ. G.BunnellH. T. (1981). Perception of anticipatory coarticulation effects. Journal of the Acoustical Society of America, 69, 559–567.
23.
McClellandJ. L.ElmanJ. L. (1986). The TRACE model of speech perception. Cognitive Psychology, 18, 1–86.
24.
McClellandJ. L.MirmanD.HoltL. L. (2006). Are there interactive processes in speech perception?Trends in Cognitive Sciences, 10, 363–369.
25.
McMurrayB.JongmanA. (2011). What information is necessary for speech categorization? Harnessing variability in the speech signal by integrating cues computed relative to expectations. Psychological Review, 118, 219–246.
26.
MiallR. C.WolpertD. M. (1996). Forward models for physiological motor control. Neural Networks, 9, 1265–1279. doi:10.1016/S0893-6080(96)00035-4
27.
NeareyT. M. (1990). The segment as a unit of speech perception. Journal of Phonetics, 18, 347–373.
28.
NorrisD.McQueenJ.CutlerA. (2000). Merging information in speech recognition: Feedback is never necessary. Behavioral & Brain Sciences, 23, 299–370.
29.
NygaardL. C.SommersM. S.PisoniD. B. (1994). Speech perception as a talker-contingent process. Psychological Science, 5, 42–46.
30.
OdenG.MassaroD. W. (1978). Integration of featural information in speech perception. Psychological Review, 85, 172–191.
31.
PerrachioneT. K.Del TufoS. N.GabrieliJ. D. E. (2011). Human voice recognition depends on language ability. Science, 333, 595. doi:10.1126/science.1207327
32.
PickeringM. J.GarrodS. (2013). An integrated theory of language production and comprehension. Behavioral & Brain Sciences, 36, 329–347. doi:10.1017/S0140525X12001495
33.
RaoR. P. N.BallardD. H. (1999). Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience, 2, 79–87.
34.
R Development Core Team. (2008). R: A language and environment for statistical computing. Retrieved from http://www.R-project.org
35.
RhodesG.LeopoldD. A. (2011). Adaptive norm-based coding of face identity. In CalderA. J.RhodesG.JohnsonM. H.HaxbyJ. V. (Eds.), The Oxford handbook of face perception (pp. 263–286). Oxford, England: Oxford University Press.
36.
SalverdaA. P.KleinschmidtD.TanenhausM. K. (2014). Immediate effects of anticipatory coarticulation in spoken-word recognition. Journal of Memory and Language, 71, 145–163.
37.
SmitsR. (2001). Evidence for hierarchical categorization of coarticulated phonemes. Journal of Experimental Psychology: Human Perception and Performance, 27, 1145–1162.
38.
StrandE. (1999). Uncovering the role of gender stereotypes in speech perception. Journal of Language and Social Psychology, 18, 86–100.
39.
von KriegsteinK.SmithD. R. R.PattersonR. D.KiebelS. J.GriffithsT. D. (2010). How the human brain recognizes speech in the context of changing speakers. The Journal of Neuroscience, 30, 629–638. doi:10.1523/jneurosci.2742-09.2010
40.
WolpertD. M.FlanaganJ. R. (2001). Motor prediction. Current Biology, 11, R729–R732.
41.
Yeni-KomshianG. H.SoliS. D. (1981). Recognition of vowels from information in fricatives: Perceptual evidence of fricative-vowel coarticulation. Journal of the Acoustical Society of America, 70, 966–975.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
