In this study, a corpus of essays stratified by level (9th grade, 11th grade, and college freshman) are analyzed computationally to discriminate differences between the linguistic features produced in essays by adolescents and young adults. The automated tool Coh-Metrix is used to examine to what degree essays written at various grade levels can be distinguished from one another using a number of linguistic features related to lexical sophistication (i.e., word frequency, word concreteness), syntactic complexity (i.e., the number of modifiers per noun phrase), and cohesion (i.e., word overlap, incidence of connectives). The analysis demonstrates that high school and college writers develop linguistic strategies as a function of grade level. Primarily, these writers produce more sophisticated words and more complex sentence structure as grade level increases. In contrast, these writers produce fewer cohesive features in text as a function of grade level. This analysis supports the notion that linguistic development occurs in the later stages of writing development and that this development is primarily related to producing texts that are less cohesive and more elaborate.
Abbott, R., & Berninger, V.W. (1993). Structural equation modeling of relationships among developmental skills and writing skills in primary- and intermediate-grade writers. Journal of Educational Psychology , 85, 478-508.
2.
Abbott, R., Berninger, V.W., & Fayol, M. ( 2010). Longitudinal relationships of levels of language in writing and between writing and reading in grades 1 to 7. Journal of Educational Psychology, 102, 281-298.
3.
Akiguet, S., & Piolat, A. ( 1996). Insertion of connectives by 9- to 11-year-old children in an argumentative text. Argumentation, 10, 253-270.
4.
Baayen, R.H., Piepenbrock, R., & Gulikers, L. ( 1995). The CELEX lexical database (release 2) [CD]. Philadelphia, PA: Linguistic Data Consortium, University of Pennsylvania.
5.
Berninger, V., Abbott, R., Swanson, H.L., Lovitt, D., Trivedi, P., Lin, S., . . . Amtmann, D. ( 2010). Relationship of word- and sentence-level working memory to reading and writing in second, fourth, and sixth grade. Language, Speech, and Hearing Services in Schools, 41, 179-193.
6.
Berninger, V., Abbott, R., Thomson, J., Wagner, R., Swanson, H.L., Wijsman, F., & Raskind, W. ( 2006). Modeling developmental phonological core deficits within a working-memory architecture in children and adults with developmental dyslexia . Scientific Studies in Reading, 10, 165-198.
7.
Berninger, V., Cartwright, A., Yates, C., Swanson, L., & Abbott, R. ( 1994). Developmental skills related to writing and reading acquisition in the intermediate grades. Reading and Writing: An Interdisciplinary Journal, 6, 161-196.
8.
Berninger, V., Mizokawa, D., & Bragg, R. ( 1991). Theory-based diagnosis and remediation of writing disabilities . Journal of School Psychology, 29, 57-79.
9.
Berninger, V., Yates, C., Cartwright, A., Rutberg, J., Remy, E., & Abbott, R. ( 1992). Lower-level developmental skills in beginning writing . Reading and Writing: An Interdisciplinary Journal, 4, 257-280.
10.
Brace, N., Kemp, R., & Snelgar, R. ( 2006). SPSS for psychologists. Mahwah, NJ : Lawrence Erlbaum.
11.
Costerman, J., & Fayol, M. ( 1997). Processing interclausal relationships: Studies in production and comprehension of text. Hillsdale, NJ: Lawrence Erlbaum.
12.
Crismore, A., Markkanen, R., & Steffensen, M.S. (1993). Metadiscourse in persuasive writing: A study of texts written by American and Finnish university students . Written Communication, 10, 39-71.
13.
Crossley, S.A., Greenfield, J., & McNamara, D.S. (2008). Assessing text readability using psycholinguistic indices. TESOL Quarterly, 42, 475-493.
14.
Crossley, S.A., & McNamara, D.S. (2010). Cohesion, coherence, and expert evaluations of writing proficiency. In R. Catrambone & S. Ohlsson (Eds.), Proceedings of the 32nd annual conference of the Cognitive Science Society (pp. 984-989). Austin, TX: Cognitive Science Society.
15.
Crossley, S.A., & McNamara, D.S. (in press). Predicting second language writing proficiency: The role of cohesion, readability, and lexical difficulty. Journal of Research in Reading.
16.
Douglas, D. ( 1981). An exploratory study of bilingual reading proficiency . In S. Hudelson (Ed.), Learning to read in different languages (pp. 33-102). Washington, DC: Center for Applied Linguistics.
17.
Durst, R.K. ( 1984). The development of analytic writing. In A. N. Applebee (Ed.), Contexts for learning to write: Studies of secondary school instruction (pp. 79-102). Norwood, MA: Ablex Press.
18.
Fellbaum, C. ( 1998). WordNet: An electronic lexical database. Cambridge, MA: MIT Press.
19.
Field, A. ( 2005). Discovering statistics using SPSS. London, UK: SAGE.
20.
Freedman, A., & Pringle, I. ( 1980). Writing in the college years: Some indices of growth . College Composition and Communication, 31, 311-324.
21.
Grabowski, J., Weinzierl, C., & Schmitt, M. ( 2010). Second and fourth graders’ copying ability: From graphical to linguistic processing. Journal of Research in Reading, 33, 39-53.
22.
Graesser, A.C., McNamara, D.S., Louwerse, M., & Cai, Z. ( 2004). Coh-Metrix: Analysis of text on cohesion and language . Behavior Research Methods, Instruments, & Computers , 36, 193-202.
23.
Haberlandt, K., & Graesser, A.C. (1985). Component processes in text comprehension and some of their interactions. Journal of Experimental Psychology: General, 114, 357-374.
24.
Halliday, M.A.K., & Hasan, R. ( 1976). Cohesion in English. London, UK: Longman.
25.
Haswell, R.H. ( 1986). Change in undergraduate and post-graduate writing performance: Quantified findings (Technical Report). Retrieved fromERIC database. (ED269780)
26.
Haswell, R.H. ( 1989). Textual research and coherence: Findings, intuition, application. College English, 51, 305-319.
27.
Haswell, R.H. ( 2000). Documenting improvement in college writing: A longitudinal approach. Written Communication, 17, 307-352.
28.
Hayes, J., & Flower, L. ( 1980). Identifying the organization of the writing process. In L. W. Gregg & E. R. Steinberg (Eds.), Cognitive processes in writing (pp. 3-30). Hillsdale, NJ: Erlbaum.
29.
Hillocks, G. ( 2002). The testing nap: How state writing assessments control learning. New York, NY: Teachers College Press.
30.
Hunt, K.W. ( 1965). Grammatical structures written at three grade levels (NCTE Research Report Number 3). Urbana, IL: National Council of Teachers of English.
31.
Huot, B. ( 1990). The literature of direct writing assessment: Major concerns and prevailing tends. Review of Educational Research, 60, 237-263.
32.
Institute of Education Sciences. (2003). The nation’s report card: Writing 2002 (NCES2003-529). Retrieved from http://nces.ed.gov/nationsreportcard/pubs/main2002/2003529.asp
33.
Just, M.A., & Carpenter, P.A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87, 329-354.
34.
Just, M.A., & Carpenter, P.A. (1992). A capacity theory of comprehension: Individual differences in working memory. Psychological Review , 99, 122-149.
35.
Kellogg, R.T. ( 2008). Training writing skills: A cognitive developmental perspective . Journal of Writing Research, 1, 1-26.
36.
King, M., & Rentel, V. ( 1979). Toward a theory of early writing development. Research in the Teaching of English, 13, 243-253.
37.
Kintsch, W., & van dijk, T.A. (1978). Toward a model of text comprehension and production. Psychological Review, 85, 363-394.
38.
Landauer, T., McNamara, D. S., Dennis, S., & Kintsch, W. (Eds.). (2007). Handbook of latent semantic analysis. Mahwah, NJ: Erlbaum.
39.
Loban, W.D. ( 1976). Language development: Kindergarten through grade twelve (Research Report Number 18). Urbana, IL: National Council of Teachers of English.
40.
Longo, B. ( 1994). Current research in technical communication: The role of metadiscourse in persuasion. Technical Communication, 41, 348-352.
41.
Louwerse, M.M. ( 2001). An analytic and cognitive parameterization of coherence relations. Cognitive Linguistics, 12, 291-315.
42.
Malvern, D.D., Richards, B., Chipere, N., & Duran, P. ( 2004). Lexical diversity and language development: Quantification and assessment. Basingstoke, UK: Pal-grave Macmillan.
43.
McCarthy, P.M., & Jarvis, S. ( 2010). MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment. Behavior Research Methods, 42, 381-392.
44.
McCutchen, D. ( 1986). Domain knowledge and linguistic knowledge in the development of writing ability. Journal of Memory and Language, 25, 431-444.
45.
McCutchen, D. ( 2000). Knowledge, processing, and working memory: Implications for a theory of writing. Educational Psychologist, 35, 13-23.
46.
McCutchen, D., & Perfetti, C. ( 1982). Coherence and connectedness in the development of discourse production. Text, 2, 113-139.
47.
McNamara, D.S. ( 2001). Reading both high-coherence and low-coherence texts: Effects of text sequence and prior knowledge. Canadian Journal of Experimental Psychology, 55, 51-62.
48.
McNamara, D.S., Crossley, S.A., & McCarthy, P.M. (2010). Linguistic features of writing quality. Written Communication, 27, 57-86.
49.
McNamara, D.S., & Graesser, A.C. (in press). Coh-Metrix. In P. M. McCarthy & C. Boonthum (Eds.), Applied natural language processing and content analysis: Identification, investigation, and resolution. Hershey, PA: IGI Global.
50.
McNamara, D.S., Kintsch, E., Butler-Songer, N., & Kintsch, W. ( 1996). Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction, 14, 1-43.
51.
Miller, G.A., Beckwith, R., Fellbaum, C., Gross, D., & Miller, K. ( 1990). Five papers on WordNet. Princeton, NJ: Princeton University, Cognitive Science Laboratory.
52.
O’Donnell, R.C., Griffin, W.J., & Norris, R.C. ( 1967). Syntax of kindergarten and elementary school children: A transformational analysis (NCTE Research Report Number8). Urbana, IL: National Council of Teachers of English.
53.
Oliver, E. ( 1995). The writing quality of seventh, ninth, and eleventh graders, and college freshmen: Does rhetorical specification in writing prompts make a difference?Research in the Teaching of English, 29, 422-450.
54.
O’Reilly, T., & McNamara, D.S. (2007). Reversing the reverse cohesion effect: Good texts can be better for strategic, high-knowledge readers. Discourse Processes, 43, 121-152.
55.
Pearson, P.D. ( 1974-1975). The effects of grammatical complexity on children’s comprehension, recall, and conception of certain semantic. Reading Research Quarterly, 10, 155-192.
56.
Perfetti, C.A., Landi, N., & Oakhill, J. ( 2005). The acquisition of reading comprehension skill. In M. J. Snowling & C. Hulme (Eds.), The science of reading: A handbook (pp. 227-247). Oxford, UK: Blackwell.
57.
Perfetti, C., & McCutchen, D. ( 1987). Schooled language competence: Linguistic abilities in reading and writing. In S. Rosenberg (Ed.), Advances in applied psycholinguistics (pp. 105-141). Cambridge, UK: Cambridge University Press.
58.
Rashotte, C.A., & Torgesen, J.K. (1985). Repeated reading and reading fluency in learning disabled children. Reading Research Quarterly , 20, 180-188.
59.
Rayner, K. & Pollatsek, A. ( 1994). The psychology of reading. Englewood Cliffs, NJ: Prentice Hall.
60.
Stewart, M.F. ( 1978). Syntactic maturity from high school to university: A first look. Research in the Teaching of English, 12(1), 37-46.
61.
Swanson, H.L., & Berninger, V.W. (1996). Individual differences in children’s working memory and writing skill. Journal of Experimental Child Psychology, 63, 358-385.
62.
Tabachnick, B.G., & Fidell, L.S. ( 2001). Using multivariate statistics (4th ed.). Needham Heights, MA: Allyn & Bacon.
63.
Van de Kopple, W. ( 1985). Some exploratory discourse on metadiscourse. College Composition and Communication, 36, 82-93.
64.
Weston, J.L., Crossley, S.A., McCarthy, P.M., & McNamara, D.S. (2011). Number of words versus number of ideas: Finding a better predictor of writing quality. Proceedings of the 24th International Florida Artificial Intelligence Research Society. Retrieved from http://www.aaai.org/ocs/index.php/FLAIRS/FLAIRS11/paper/download/2618/3182
65.
Wilson, M.D. ( 1988). The MRC psycholinguistic database: Machine-readable dictionary, version 2. Behavioural Research Methods, Instruments, and Computers, 201, 6-11.
66.
Witten, I.H., & Frank, E. ( 2005). Data mining: Practical machine learning tools and techniques . San Francisco, CA: Elsevier.
67.
Witte, S.P., & Faigley, L. ( 1981). Coherence, cohesion, and writing quality. College Composition and Communication, 32, 189-204.
