We examined whether an automated essay feedback system could improve the writing performance of college students. Students in a freshmen composition course received no feedback or varying amounts of formative feedback on their first drafts of three practice essays (none, intermittent, or continuous). The students wrote a first and then a revised final draft of an essay in a writing lab held once per week. After a retention interval of 2 weeks, a test essay was written without the use of feedback to assess transfer of learning. Holistic scores showed no reliable gains, but the transfer test showed that students learned to reduce errors of mechanics, usage, grammar, and style. The benefit was found for continuous but not intermittent feedback.
Get full access to this article
View all access options for this article.
References
1.
AttaliY., & BursteinJ. (2006). Automated essay scoring with E-rater V. 2.0. The Journal of Technology, Learning, and Assessment, 4, 13–18.
2.
Bangert-DrownsR. L.KulikC. C.KulikJ. A., & MorganM. T. (1991). The instructional effect of feedback in test-like events. Review of Educational Research, 61, 213–238.
3.
BeachR., & FriedrichT. (2006). Response to writing. In MacArthurC. A.GrahamS., & FitzgeraldJ. (Eds.), Handbook of writing research (pp. 222–234). New York: Guilford.
4.
BokD. (2006). Our underachieving colleges: A candid look at how students learn and why they should be learning more.Princeton, NJ: Princeton University Press.
5.
BrelandH. M.CampR.JonesR. J.MorrisM. M., & RockD. A. (1987). Assessing writing skill (No. 0–87447–280–6). New York: College Entrance Examination Board.
6.
BursteinJ. (2003). The e-rater scoring engine: Automated essay scoring with natural language processing. In ShermisM. D. & BursteinJ. (Eds.), Automated essay scoring: A cross-disciplinary perspective (pp. 113–122). Hillsdale, NJ: Lawrence Erlbaum.
7.
BursteinJ., & WolskaM. (2003, April). Toward evaluation of writing style: Finding overly repetitive word use in student essays. In Proceedings of the 10th Conference of the European Chapter of the Association for Computational Linguistics, Budapest, Hungary.
8.
CalfeeR. C., & MillerR. G. (2007). Best practices in writing assessment. In GrahamS.MacArthurC. A. & FitzgeraldJ. (Eds.), Best practices in writing instruction (pp. 265–286). New York: Guilford Press.
9.
Center for Survey Research and Analysis. (2002, November). History Research Paper Study. Retrieved December 15, 2003 from the Concord Review website: www.tcr.org/historytcr.pdf
10.
ChoK.SchunnC. D., & WilsonR. W. (2006). Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives. Journal of Educational Psychology, 98, 891–901.
11.
DiederichP. B. (1974). Measuring growth in English.Urbana, IL: National Council of Teachers of English.
12.
DiederichP. B.FrenchJ. W., & CarltonS. T. (1961). Factors in the judgement of writing quality (No. RB 6, 1–15). Princeton: Educational Testing Service.
13.
DorowL. G., & BoyleM. E. (1998). Instructor feedback for college writing assignments in introductory classes. Journal of Behavioral Education, 8, 115–129.
14.
EliottN. (2005). On a scale: A social history of writing assessment in America.New York: Peter Lang Publishing.
15.
EricssonP. F. (2006). The meaning of meaning: Is a paragraph more than an equation. In EricssonP. F. & HaswellR. H. (Eds.), Machine scoring of student essays (pp. 28–37). Logan, UT: Utah State University Press.
16.
EricssonP. F., & HaswellR. H. (Eds.). (2006). Machine scoring of student essays.Logan, UT: Utah State University Press.
17.
FoltzP.W.KintschW., & LandauerT. K. (1998). The measurement of textual coherence with latent semantic analysis. Discourse Processes, 25(2–3), 285–307.
18.
FoltzP. W., & WellsA. D. (1999). Automatically deriving readers' knowledge structures from texts. Behavior Research Methods, Instruments and Computers, 31, 208–214.
19.
FreedmanS. W., & CalfeeR. C. (1983). Holistic assessment of writing: Experimental design and cognitive theory. In MosenthalP.TamorL., & WalmsleyS. A. (Eds.), Research in writing: Principles and methods (pp. 75–98). New York: Longman.
20.
GraesserA. C.McNamaraD. S.LouwerseM. M., & CaiZ. (2004). Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments, and Computers, 36, 193–202.
21.
HattieJ., & TimperleyH. (2007). The power of feedback. Review of Educational Research, 77, 81–113.
22.
HayesJ. R. (1996). A new framework for understanding cognition and affect in writing. In LevyC. M. & RansdellS. (Eds.), The science of writing: Theories, methods, individual differences, and applications (pp. 1–27). Mahwah, NJ: Lawrence Erlbaum Associate.
23.
HayesJ. R., & FlowerL. (1980). Identifying the organization of writing processes. In GreggL. & SteinbergE. R. (Eds.), Cognitive processes in writing (pp. 3–30). Hillsdale, NJ: Lawrence Erlbaum.
24.
HuotB. (1990). The literature of direct writing assessment: Major concerns and prevailing trends. Review of Educational Research, 60, 237–263.
25.
KiuharaS. A.GrahamS., & HawkenL. S. (2009). Teaching writing to high school students: A national survey. Journal of Educational Psychology, 101(1), 136–160.
26.
LandauerT. K.FoltzP. W., & LahamD. (1998). An introduction to latent semantic analysis. Discourse Processes, 25(2–3), 259–284.
27.
LandauerT. K.LahamD., & FoltzP. W. (2003). Automatic essay assessment. Assessment in Higher Education, 10, 295–308.
28.
McGeeT. (2006). Taking a spin on the Intelligent Essay Assessor. In EricssonP. F. & HaswellR. H. (Eds.), Machine scoring of student essays (pp. 79–92). Logan, UT: Utah State University Press.
PageE. B. (2003). Project essay grade: PEG. In ShermisM. D. & BursteinJ. C. (Eds.), Automated essay scoring: A cross-disciplinary perspective. (pp. 43–54). Mahwah, NJLawrence Erlbaum.
32.
SchmidtR. A., & BjorkR. A. (1992). New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psychological Science, 3, 207–217.
33.
ShuteV. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189.
