Although it is generally accepted that hierarchical phrase structures are instrumental in describing human language, their role in cognitive processing is still debated. We investigated the role of hierarchical structure in sentence processing by implementing a range of probabilistic language models, some of which depended on hierarchical structure, and others of which relied on sequential structure only. All models estimated the occurrence probabilities of syntactic categories in sentences for which reading-time data were available. Relating the models’ probability estimates to the data showed that the hierarchical-structure models did not account for variance in reading times over and above the amount of variance accounted for by all of the sequential-structure models. This suggests that a sentence’s hierarchical structure, unlike many other sources of information, does not noticeably affect the generation of expectations about upcoming words.
ArnonI.SniderN. (2010). More than words: Frequency effects for multi-word phrases. Journal of Memory and Language, 62, 67–82.
2.
BaayenR. H.DavidsonD. J.BatesD. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390–412.
3.
BannardC.MatthewsD. (2008). Stored word sequences in language learning: The effect of familiarity on children’s repetition of four-word combinations. Psychological Science, 19, 241–248.
4.
BostonM. F.HaleJ.PatilU.KlieglR.VasishthS. (2008). Parsing costs as predictors of reading difficulty: An evaluation using the Potsdam Sentence Corpus. Journal of Eye Movement Research, 2, 1–12.
ChomskyN. (1957). Syntactic structures. The Hague, The Netherlands: Mouton.
7.
ChristiansenM. H.ChaterN. (1999). Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, 23, 157–205.
8.
ChristiansenM. H.MacDonaldM. C. (2009). A usage-based approach to recursion in sentence processing. Language Learning, 59, 129–164.
9.
ConwayC. M.BauernschmidtA.HuangS. S.PisoniD. B. (2010). Implicit statistical learning in language processing: Word predictability is the key. Cognition, 114, 356–371.
10.
DembergV.KellerF. (2008). Data from eye-tracking corpora as evidence for theories of syntactic processing complexity. Cognition, 109, 193–210.
11.
DembergV.KellerF. (2009). A computational model of prediction in human parsing: Unifying locality and surprisal effects. In TaatgenN. A.van RijnH. (Eds.), Proceedings of the 31st Annual Conference of the Cognitive Science Society (pp. 1888–1893). Austin, TX: Cognitive Science Society.
12.
ElmanJ. L. (1991). Distributed representations, simple recurrent networks, and grammatical structure. Machine Learning, 7, 195–225.
13.
FrankS. L. (2009). Surprisal-based comparison between a symbolic and a connectionist model of sentence processing. In TaatgenN. A.van RijnH. (Eds.), Proceedings of the 31st Annual Conference of the Cognitive Science Society (pp. 1139–1144). Austin, TX: Cognitive Science Society.
14.
FrankS. L.ČerňanskýM. (2008). Generalization and systematicity in echo state networks. In LoveB. C.McRaeK.SloutskyV. M. (Eds.), Proceedings of the 30th Annual Conference of the Cognitive Science Society (pp. 733–738). Austin, TX: Cognitive Science Society.
15.
GaleW. A.SampsonG. (1995). Good-Turing frequency estimation without tears. Journal of Quantitative Linguistics, 2, 217–237.
16.
GillespieM.PearlmutterN. J. (2011). Hierarchy and scope of planning in subject-verb agreement production. Cognition, 118, 377–397.
17.
HaleJ. (2001). A probabilistic Earley parser as a psycholinguistic model. In Proceedings of the Second Conference of the North American Chapter of the Association for Computational Linguistics (Vol. 2, pp. 159–166). Pittsburgh, PA: Association for Computational Linguistics.
18.
HauserM. D.ChomskyN.FitchW. T. (2002). The faculty of language: What is it, who has it, and how did it evolve?Science, 298, 1569–1579.
19.
JaegerH.HaasH. (2004). Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication. Science, 304, 78–80.
20.
JohnsonM. (1998). The effect of alternative tree representations on tree bank grammars. In Proceedings of the Joint Conferences on New Methods in Language Processing and Computational Natural Language Learning (pp. 39–48). Pittsburgh, PA: Association for Computational Linguistics.
21.
JurafskyD.MartinJ. H. (2009). Speech and language processing. Upper Saddle River, NJ: Pearson Education.
22.
KamideY.AltmannG. T. M.HaywoodS. L. (2003). The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye movements. Journal of Memory and Language, 49, 133–156.
23.
KellerF. (2003). A probabilistic parser as a model of global processing difficulty. In AltermanR.KirshD. (Eds.), Proceedings of the 25th Annual Conference of the Cognitive Science Society (pp. 646–651). Boston, MA: Cognitive Science Society.
24.
KennedyA.PynteJ. (2005). Parafoveal-on-foveal effects in normal reading. Vision Research, 45, 153–168.
ManningC. D.SchützeH. (1999). Foundations of statistical natural language processing. Cambridge, MA: MIT Press.
27.
MarcusM.SantoriniB.MarcinkiewiczM. A. (1993). Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19, 313–330.
28.
McDonaldS. A.ShillcockR. C. (2003). Low-level predictive inference in reading: The influence of transitional probabilities on eye movements. Vision Research, 43, 1735–1751.
29.
OttenM.Van BerkumJ. J. A. (2008). Discourse-based word anticipation during language processing: Prediction or priming?Discourse Processes, 45, 464–496.
30.
RoarkB. (2001). Probabilistic top-down parsing and language modeling. Computational Linguistics, 27, 249–276.
31.
RoarkB.BachrachA.CardenasC.PallierC. (2009). Deriving lexical and syntactic expectation-based measures for psycholinguistic modeling via incremental top-down parsing. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing (pp. 324–333). Pittsburgh, PA: Association for Computational Linguistics.
32.
SantoriniB. (1991). Part-of-speech tagging guidelines for the Penn Treebank Project (Technical Report No. MS-CIS-90-47). Philadelphia: University of Pennsylvania.
33.
SmithN. J.LevyR. (2008). Optimal processing times in reading: A formal model and empirical investigation. In LoveB. C.McRaeK.SloutskyV. M. (Eds.), Proceedings of the 30th Annual Conference of the Cognitive Science Society (pp. 595–600). Austin, TX: Cognitive Science Society.
34.
StaubA.CliftonC.Jr. (2006). Syntactic prediction in language comprehension: Evidence from either … or. Journal of Experimental Psychology: Learning, Memory, and Cognition, 32, 425–436.
35.
TaborW.GalantucciB.RichardsonD. (2004). Effects of merely local syntactic coherence on sentence processing. Journal of Memory and Language, 50, 355–370.
36.
TongM. H.BickettA. D.ChristiansenE. M.CottrellG. W. (2007). Learning grammatical structure with echo state networks. Neural Networks, 20, 424–432.
37.
TremblayA.DerwingB.LibbenG.WestburyC. (2011). Processing advantages of lexical bundles: Evidence from self-paced reading and sentence recall tasks. Language Learning. Advance online publication. doi:10.1111/j.1467-9922.2010.00622.x
38.
Van BerkumJ. J. A.BrownC. M.ZwitserloodP.KooijmanV.HagoortP. (2005). Anticipating upcoming words in discourse: Evidence from ERPs and reading times. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31, 443–467.
39.
WittenI. H.BellT. C. (1991). The zero-frequency problem: Estimating the probabilities of novel events in adaptive text compression. IEEE Transactions on Information Theory, 37, 1085–1094.
40.
WuS.BachrachA.CardenasC.SchulerW. (2010). Complexity metrics in an incremental right-corner parser. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (pp. 1189–1198). Pittsburgh, PA: Association for Computational Linguistics.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
