Idea generation is an important component of most major theories of writing. However, few studies have linked idea generation in writing samples to assessments of writing quality or examined links between linguistic features in a text and idea generation. This study uses human ratings of idea generation, such as idea fluency, idea flexibility, idea originality, and idea elaboration, to analyze the extent to which idea generation relates to human judgments of essay quality in a corpus of college student essays. In conjunction with this analysis, linguistic features extracted from the essays are used to develop a predictive model of idea generation to further understand relations between the language features in an essay and the idea generation scores assigned to that essay. The results indicate that essays rated as containing a greater number of ideas that were flexible, original, and elaborated were judged to be of higher quality. Two of these features (elaboration and originality) were significant predictors of essay quality scores in a regression analysis that explained 33% of the variance in human scores. The results also indicate that idea generation is strongly linked to language features in essays. Specifically, the use of unique multiword units, more difficult words, semantic but not lexical similarities between paragraphs, and fewer word repetitions explained 80% of the variance in human scores of idea generation. These results have implications for writing theories and writing practice.
AlamargotD.ChanquoyL. (2001). Development of expertise in writing. In AlamargotD.ChanquoyL. (Eds.), Through the models of writing (pp. 185-218). Dordrecht, Netherlands: Kluwer.
2.
AlamargotD.FayolM. (2009). Modelling the development of written composition. In BeardR.MyhillD.RileyJ.NystrandM. (Eds.), The Sage handbook of writing development (pp. 23-47). Thousand Oaks, CA: Sage.
3.
AllenL. K.SnowE. L.JacksonG. T.CrossleyS. A.McNamaraD. S. (2014). Reading components and their relation to writing. L’Année psychologique/Topics in Cognitive Psychology, 114(4), 663-691.
4.
AmabileT. (2013). Componential theory of idea generation. In KesslerE. (Ed.), Encyclopedia of management theory (pp. 135-140). Thousand Oaks, CA: Sage.
5.
AndersonJ. M. (1996). Historical linguistics. In MalmkjaerK. (Ed.), The linguistics encyclopedia (pp. 189-216). London, UK: Routledge.
6.
AntesA.MumfordM. (2009). Effects of time frame on creative thought: Process versus problem-solving effects. Idea Generation Research Journal, 21, 166-182.
7.
BabayigitS.StainthorpR. (2011). Modeling the relationships between cognitive-linguistic skills and literacy skills: New insights from a transparent orthography. Journal of Educational Psychology, 103(1), 169-189.
8.
BaerJ.McKoolS. (2009). Assessing creativity using the consensual assessment technique. In SchreinerC. (Ed.), Handbook of assessment technologies, methods, and applications in higher education (pp. 1-13). Hershey, PA: IGI Global.
9.
BereiterC.ScardamaliaM. (1987). The psychology of written composition. Hillsdale, NJ: Lawrence Erlbaum.
10.
BerningerV. W.AbbottR. D.AbbottS. P.GrahamS.RichardsT. (2002). Writing and reading: Connections between language by hand and language by eye. Journal of Learning Disabilities, 35, 39-56.
11.
BerningerV. W.FullerF.WhitakerD. (1996). A process model of writing development across the life span. Educational Psychology Review, 8, 193-218.
12.
BerningerV. W.RichardsT.StockP.AbbottR.TrivediP.AltemeierL.HayesJ. R. (2009). fMRI activation related to nature of ideas generated and differences between good and poor writers during idea generation. British Journal of Educational Psychology Monograph Series II, 6, 77-93.
13.
BirdS.LoperE.KleinK. (2009). Natural language processing with Python. Sebastopol, CA: O’Reilly Media.
14.
BourdinB.FayolM.DarciauxS. (1996). The comparison of oral and written modes on adults’ and children’s narrative recall. In RijlaarsdamG.Van den BerghH.CouzijnM. (Eds.), Theories, models and methodology in writing research (pp. 121-141). Amsterdam, Netherlands: Amsterdam University Press.
15.
BrownG. D. A. (1984). A frequency count of 190,000 words in the London-Lund Corpus of English Conversation. Behavioural Research Methods Instrumentation and Computers, 16, 502-532.
16.
BrysbaertM.NewB. (2009). Moving beyond Kucera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Behavior Research Methods, 41(4), 977-990.
17.
BrysbaertM.WarrinerA. B.KupermanV. (2014). Concreteness ratings for 40 thousand generally known English word lemmas. Behavior Research Methods, 46(3), 904-911.
18.
CarterR. (2016). Language and creativity: The art of common talk (2nd ed.). New York, NY: Routledge.
19.
CheungW. M.TseS. K.TsangW. H. H. (2001). Development and validation of the Chinese creative writing scale for primary school students in Hong Kong. Journal of Creative Behavior, 35, 249-260.
20.
ColtheartM. (1981). The MRC psycholinguistic database. Quarterly Journal of Experimental Psychology, 33A, 497-505.
21.
CoxheadA. (2000). A new academic word list. TESOL Quarterly, 34(2), 213-238.
22.
CrossleyS. A.KyleK.McNamaraD. S. (in press). The Tool for the Automatic Analysis of Text Cohesion (TAACO): Automatic assessment of local, global, and text cohesion. Behavior Research Methods.
23.
CrossleyS. A.McNamaraD. S. (2011). Text coherence and judgments of essay quality: Models of quality and coherence. In CarlsonL.HoelscherC.ShipleyT. F. (Eds.), Proceedings of the 29th annual conference of the Cognitive Science Society (pp. 1236-1241). Austin, TX: Cognitive Science Society.
24.
CrossleyS. A.McNamaraD. S. (2016). Say more and be more coherent: How text elaboration and cohesion can increase writing quality. Journal of Writing Research, 7(3), 351-370.
25.
CrossleyS. A.RoscoeR.McNamaraD. S. (2013). Using automatic scoring models to detect changes in student writing in an intelligent tutoring system. In McCarthyP. M.YoungbloodG. M. (Eds.), Proceedings of the 26th International Florida Artificial Intelligence Research Society (FLAIRS) conference (pp. 208-213). Menlo Park, CA: AAAI Press.
26.
DiakidoyI.ConstantinouC. (2000-2001). Idea generation in physics: Response fluency and task specificity. Idea Generation Research Journal, 13(3-4), 401-410.
27.
ElbowP. (1973). Writing without teachers. New York, NY: Oxford University Press.
28.
FeldhusenJ.GohB. E. (1995). Assessing and accessing idea generation: An integrative review of theory, research, and development. Idea Generation Research Journal, 8(3), 231-247.
29.
FlowerL. S.HayesJ. R. (1980). The cognition of discovery: Defining a rhetorical problem. College Composition and Communication, 31, 21-32.
30.
FlowerL. S.HayesJ. R. (1981). A cognitive process theory of writing. College Composition and Communication, 32, 365-387.
31.
GalbraithD. (2009). Writing as discovery. Teaching and Learning Writing, 2(6), 5-26. doi:10.1348/978185409X421129
32.
GalbraithD.FordS.WalkerG.FordJ. (2005). The contribution of different components of working memory to planning in writing. L1 Educational Studies in Language and Literature, 15, 113-145.
33.
GistM. E. (1989). The influence of training method on self-efficacy and idea generation among managers. Personnel Psychology, 42, 787-805.
34.
GrahamS.PerrinD. (2007). A meta-analysis of writing instruction for adolescent students. Journal of Educational Psychology, 99, 445-476.
35.
HallidayM. A. K.HasanR. (1976). Cohesion in English. London, UK: Longman.
36.
HayesJ. (1996). A new framework for understanding cognition and affect in writing. In LevyM.RansdellS. (Eds.), The science of writing: Theories, methods, individual differences, and applications (pp. 1-27). Mahwah, NJ: Lawrence Erlbaum.
37.
HayesJ. R. (2006). New directions in writing theory. In MacArthurC. A.GrahamS.FitzgeraldJ. (Eds.), Handbook of writing research (pp. 28-40). New York, NY: Guilford.
38.
HayesJ. R.BerningerV. W. (2010). Relation between idea generation and transcription. In BazermanC.KrutR.LunsfordK.McLeodS.NullS.RogersP.StansellA. (Eds.), Traditions of writing research (pp. 166-180). New York, NY: Routledge.
39.
HayesJ.FlowerL. (1980). Identifying the organization of writing processes. In GreggL.SteinbergE. (Eds.), Cognitive processes in writing: An interdisciplinary approach (pp. 3-30). Hillsdale, NJ: Lawrence Erlbaum.
40.
HinkleS.HinkleA. (1990). An experimental comparison of the effects of focused freewriting and other study strategies on lecture comprehension. Teaching of Psychology, 17, 31-35.
41.
JohnsonT. S.ThompsonL.SmagorinskyP.FryP. G. (2003). Learning to teach the five-paragraph theme. Research in the Teaching of English, 38, 136-176.
42.
KelloggR. (1988). Attentional overload and writing performance: Effects of rough draft and outline strategies. Journal of Experimental Psychology: Learning, Memory and Cognition, 14, 355-365.
43.
KelloggR. T. (1994). The psychology of writing. New York, NY: Oxford University Press.
44.
KelloggR. T. (1996). A model of working memory in writing. In LevyM. C.RansdellS. E. (Eds.), The science of writing: Theories, methods, individual differences and applications (pp. 57-71). Hillsdale, NJ: Lawrence Erlbaum.
45.
KelloggR. T. (2001). Long-term working memory in text production. Memory & Cognition, 29, 43-52.
46.
KuceraH.FrancisW. N. (1967). Computational analysis of present-day American English. Providence, RI: Brown University Press.
47.
KupermanV.Stadthagen-GonzalesH.BrysbaertM. (2012). Age-of-acquisition ratings for 30 thousand English words. Behavior Research Methods, 44(4), 978-990.
48.
KyleK.CrossleyS. A. (2015). Automatically assessing lexical sophistication: Indices, tools, findings, and application. TESOL Quarterly, 49(4), 757-786.
49.
Levav-WaynbergA.LeikinR. (2012). The role of multiple solution tasks in developing knowledge and idea generation in geometry. Journal of Mathematical Behavior, 31, 73-90.
50.
MajidD.TanA.SohK. (2003). Enhancing children’s idea generation: An exploratory study on using the Internet and SCAMPER as creative writing tools. Korean Journal of Thinking & Problem Solving, 13(2), 67-81.
51.
MalvernD. D.RichardsB. J.ChipereN.DuránP. (2004). Lexical diversity and language development: Quantification and assessment. Houndmills, UK: Palgrave Macmillan.
52.
MayerR. E. (1999). Fifty years of idea generation research. In SternbergR. J. (Ed.), Handbook of idea generation (pp. 449-458). Cambridge, UK: Cambridge University Press.
53.
McCarthyP.JarvisS. (2010). MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment. Behavior Research Methods, 42, 381-392.
54.
McCutchenD. (1996). A capacity theory of writing: Working memory in composition. Educational Psychology Review, 8, 299-325.
55.
McCutchenD. (2000). Knowledge, processing, and working memory: Implication for a theory of writing. Educational Psychologist, 35, 13-23.
56.
McCutchenD.CovillA.HoyneS. H.MildesK. (1994). Individual differences in writing: Implications of translating fluency. Journal of Educational Psychology, 86, 256-266.
57.
McNamaraD. S.CrossleyS. A.RoscoeR. (2013). Natural language processing in an intelligent writing strategy tutoring system. Behavior Research Methods, 45(2), 499-515.
58.
MillerG. A. (1995). WordNet: A Lexical Database for English. Communications of the ACM, 38(11), 39-41.
59.
MuldnerK.BurlesonW. (2015). Utilizing sensor data to model students’ creativity in a digital environment. Computers in Human Behavior, 42, 127-137.
60.
NunnallyT. E. (1991). Breaking the five-paragraph-theme barrier. English Journal, 80, 67-71.
61.
PaulusP. B.YangH.-C. (2000). Idea generation in groups: A basis for idea generation in organizations. Organizational Behavior and Human Decision Processes, 82, 76-87.
62.
ReynoldsM. (1984). Freewriting’s origin. English Journal, 73, 81-82.
63.
RuncoM. A.OkudaS. M. (1991). The instructional enhancement of the flexibility and originality scores of divergent thinking tests. Applied Cognitive Psychology, 5, 435-441.
64.
SilverE. (1997). Fostering idea generation through instruction rich in mathematical problem solving and problem posing. ZDM, 29(3), 75-80.
65.
Simpson-VlachR.EllisN. C. (2010). An academic formulas list: New methods in phraseology research. Applied Linguistics (Oxford), 31(4), 487-512.
66.
SnyderA.MitchellJ.BossomaierT.PallierG. (2004). The idea generation quotient: An objective scoring of ideational fluency. Idea Generation Research Journal, 16, 415-420.
67.
SosikJ.KahaiS.AvolioB. (1998). Transformational leadership and dimensions of idea generation: Motivating idea generation in computer-mediated groups. Idea Generation Research Journal, 11(2), 111-121.
68.
SternG. (1965). Meaning and change of meaning: With special reference to the English language. Bloomington: Indiana University Press.
69.
SvartvikJ.QuirkR. (1980). A corpus of English conversation. Lund, Sweden: Gleerup.
70.
ThorndikeE. L.LorgeI. (1944). The teacher’s word book of 30,000 words. New York, NY: Teachers College, Columbia University.
71.
TorranceM.ThomasG. V.RobinsonE. J. (1996). Finding something to write about: Strategic and automatic processes in idea generation. In LevyM.RansdellS. (Eds.), The science of writing (pp. 189-206). Mahwah, NJ: Lawrence Erlbaum.
72.
The British National Corpus. (2007). Distributed by Oxford University Computing Services on behalf of the BNC Consortium. Retrieved from http://www.natcorp.ox.ac.uk/ (accessed October, 2013).
73.
van den BerghH.RijlaarsdamG. (1999). The dynamics of idea generation during writing: An online study. In TorranceM.WaesL.GalbraithD. (Eds.), Writing and cognition (pp. 125-150). Amsterdam, Netherlands: Elsevier.
74.
WittenI. A.FrankE.HallM. A. (2011). Data mining. San Francisco, CA: Elsevier.
