In this paper we present three design principles of language – experience, heterogeneity and redundancy – and present recent developments in a family of models incorporating them, namely Data-Oriented Parsing/Unsupervised Data-Oriented Parsing. Although the idea of some form of redundant storage has become part and parcel of parsing technologies and usage-based linguistic approaches alike, the question how much of it is cognitively realistic and/or computationally optimally efficient is an open one. We argue that a segmentation-based approach (Bayesian Model Merging) combined with an all-subtrees approach reduces the number of rules needed to achieve an optimal performance, thus making the parser more efficient. At the same time, starting from unsegmented wholes comes closer to the acquisitional situation of a language learner, and thus adds to the cognitive plausibility of the model.
AbneyS. (1996). Statistical methods and linguistics. In KlavansJ.ResnikP. (Eds.), The balancing act: Combining symbolic and statistical approaches to language (pp. 1–26). Cambridge, MA: The MIT Press.
2.
AlishahiA.StevensonS. (2008). A computational model for early argument structure acquisition. Cognitive Science, 32, 789–834.
3.
ArnonI.Cohen-Priva (this issue). More than words: The effect of multi-word frequency and constituency on phonetic duration. Language and Speech
4.
ArnonI.SniderN. (2010). More than words: Frequency effects for multi-word phrases. Journal of Memory and Language, 62, 67–82.
5.
BannardC.MatthewsD. (2008). Stored word sequences in language learning: The effect of familiarity on children’s repetition of four-word combinations. Psychological Science, 19, 241–248.
6.
BansalM.KleinD. (2010). Simple, accurate parsing with an all-fragments grammar. Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pp. 1098–1107.
7.
BodR. (1992). A computational model of language performance: data-oriented parsing. Proceedings of the fifteenth International Conference on Computational Linguistics, pp. 855–859.
8.
BodR. (1998). Beyond grammar: An experienced-based theory of language. Stanford, CA: CSLI Publications.
9.
BodR. (2001). Sentence memory: Storage vs. computation of frequent sentences. Proceedings of the CUNY Conference on Sentence Processing 2001, Philadelphia, PA.
10.
BodR. (2006a). Exemplar-based syntax: How to get productivity from examples. The Linguistic Review, 23, 291–320.
11.
BodR. (2006b). An all-subtrees approach to unsupervised parsing. Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pp. 865–872.
12.
BodR. (2007). Is the end of supervised parsing in sight?Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, pp. 400–407.
13.
BodR. (2009). From exemplar to grammar: A probabilistic analogy-based model of language learning. Cognitive Science, 33, 752–793.
14.
BodR.HayJ.JannedyS. (Eds.) (2003). Probabilistic linguistics. Cambridge, MA: The MIT Press.
BodR.SmetsM. (2012). Empiricist solutions to nativist puzzles by means of unsupervised TSG. Proceedings of the Workshop on Computational Models of Language Acquisition and Loss at the 13th Conference of the European Chapter of the Association for Computational Linguistics, pp. 10–18.
17.
BonnemaR.BodR.SchaR. (1997). A DOP model for semantic interpretation. Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics, pp. 159–167.
18.
BorensztajnG.ZuidemaW.BodR. (2009). Children’s grammars grow more abstract with age. Topics in Cognitive Science, 1, 175–188.
19.
BresnanJ. (2007). Is syntactic knowledge probabilistic? Experiments with the English dative alternation. In FeatherstonS.SternefeldW. (Eds.), Roots: Linguistics in search of its evidential base (pp. 77–96). Berlin, Germany: Mouton de Gruyter.
20.
BrownR. (1973). A first language: The early stages. London, UK: George Allen & Unwin Ltd.
21.
BybeeJ. (2006). From usage to grammar: The mind’s response to repetition. Language, 82, 711–733.
22.
CharniakE.JohnsonM. (2005) Coarse-to-fine n-best parsing and maxent discriminative reranking. Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics.
23.
ChomskyN. (1957). Syntactic structures. The Hague, The Netherlands: Mouton De Gruyter.
24.
ClarkA.EyraudR. (2006). Learning auxiliary fronting with grammatical inference. Proceedings of the Tenth Workshop on Natural Language Learning.
CohnT.GoldwaterS.BlunsomP. (2009). Inducing compact but accurate tree-substitution grammars. Proceedings of Human Language Technologies: The 2009 Annual Meeting of the North American Chapter of the Association for Computational Linguistics, pp. 548–556.
27.
CollinsM.DuffyN. (2001). Convolution kernels for natural language. Advances in Neural Information Processing Systems, 13, 625–632.
28.
CrainS. (1991). Language acquisition in the absence of experience. Behavorial and Brain Sciences, 14, 597–612.
29.
CrainS.NakayamaM. (1987). Structure dependence in grammar formation. Language, 63, 522–543.
30.
CulicoverP.JackendoffR. (2005). Simpler syntax. Oxford, UK: Oxford University Press.
31.
FrankS.BodR.ChristiansenM. (2012). How hierarchical is language use?Proceedings of the Royal Society B, 279, 4522–4531.
32.
GahlS.GarnseyS. (2004). Knowledge of grammar, knowledge of usage: Syntactic probabilities affect pronunciation variation. Language, 80, 748–775.
33.
GahlS.GarnseyS. (2006). Knowledge of grammar includes knowledge of syntactic probabilities. Language, 82, 405–410.
34.
GoldbergA. (2006). Constructions at work. The nature of generalization in language. Oxford, UK: Oxford University Press.
35.
GoodmanJ. (2003). Efficient parsing of DOP with PCFG-reductions. In BodR.SchaR.Sima’anK. (Eds.), Data-Oriented Parsing (pp. 125–146). Stanford, CA: CSLI Publication
36.
HopperP.TraugottE. (2003). Grammaticalization. Cambridge, UK: Cambridge University Press.
37.
JohnsonM.GriffithsT.GoldwaterS. (2007). Adaptor grammars: A framework for specifying compositional nonparametric Bayesian models. Advances in Neural Information Processing Systems, 19, 641–648.
38.
KamX.StoyneshkaL.TornyovaL.FodorJ.SakasW. (2008). Bigrams and the richness of the stimulus. Cognitive Science, 32, 771–787.
39.
KaplanR. (1996). A probabilistic approach to lexical-functional analysis. Proceedings of the 1996 LFG Conference and Workshops.
40.
LewisJ.ElmanJ. (2001). Learnability and the statistical structure of language: Poverty of stimulus arguments revisited. Proceedings of 26th annual Boston University Conference on Language Development, pp. 359–370.
41.
MacWhinneyB. (2000). The CHILDES project: Tools for analyzing talk. Mawah, NJ: Lawrence Erlbaum Associates.
42.
MacWhinneyB. (2005). Item-based constructions and the logical problem. Proceedings of the Second Workshop on Psychocomputational Models of Human Language Acquisition, Ann Arbor, MI.
43.
ManningC.SchützeH. (1999). Foundations of statistical natural language processing. Cambridge, MA: The MIT Press.
44.
MarrD. (1982). Vision. A computational approach. San Francisco, CA: Freeman & Co.
45.
NewmeyerF. (2006). On Gahl and Garnsey on grammar and usage. Language, 82, 399–404.
46.
O’DonnellT. (2011). Productivity and reuse in language. PhD thesis, Harvard University.
47.
PerforsA.TenenbaumJ. B.RegierT. (2011). The learnability of abstract syntactic principles. Cognition, 118, 306–338.
PostM.GildeaD. (2009). Bayesian learning of a tree substitution grammar. Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP.
50.
PostM.GildeaD. (this issue). Bayesian tree substitution grammars as a usage-based approach. Language and Speech
51.
RealiF.ChristiansenM. (2005). Uncovering the richness of the stimulus: structure dependence and indirect statistical evidence. Cognitive Science, 29, 1007–1028.
52.
RissanenJ. (1989). Stochastic complexity in statistical inquiry. Singapore: World Scientific.
53.
SangatiF.ZuidemaW. (2011). Accurate parsing with compact tree-substitution grammars: double-DOP. Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 84–95.
54.
SchaR. (1990). Taaltheorie en taaltechnologie; Competence en performance. In de KortQ.LeerdamG. (Eds.), Computertoepassingen in de Neerlandistiek (pp. 7–22). Almere, The Netherlands: Landelijke Vereniging van Neerlandici.
55.
Sima’anK. (1996). Computational complexity of probabilistic disambiguation by means of tree grammars. Proceedings of the 16th International Conference on Computational Linguistics, pp. 1175–1180.
56.
SpiveyM. J.TanenhausM. K. (1998). Syntactic ambiguity resolution in discourse: Modeling the effects of referential context and lexical frequency. Journal of Experimental Psychology: Learning, Memory and Cognition, 24, 1521–1543.
57.
StolckeA.OmohundroS. (1994). Inducing probabilistic grammars by Bayesian model merging. In CarrascoR. C.OncinaJ. (Eds.), Grammatical inference and applications (pp. 106–118). Berlin, Germany: Springer.
58.
TremblayA.BaayenR. H. (2010). Holistic processing of regular four-word sequences: A behavioral and ERP study of the effects of structure, frequency, and probability on immediate free recall. In WoodD. (Ed.), Perspectives on formulaic language: Acquisition and communication (pp. 151–173). London, UK: The Continuum International Publishing Group.
59.
VerhagenA. (2005). Constructions of intersubjectivity: Discourse, syntax and cognition. Oxford, UK: Oxford University Press.
60.
WagnerR. A.FisherM. J. (1974). The string-to-string correction problem. Journal of the Association for Computing Machinery, 21, 168–173.
61.
ZuidemaW. (2006). Theoretical evaluation of estimation methods for Data-Oriented Parsing. Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics, pp. 183–186 (corrigendum at staff.science.uva.nl/~jzuidema).
62.
ZuidemaW. (2007). Parsimonious data-oriented parsing. Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 551–560.
