This paper presents Apex, a system that can automatically assess a student essay based on its content. It relies on Latent Semantic Analysis, a tool which is used to represent the meaning of words as vectors in a high-dimensional space. By comparing an essay and the text of a given course on a semantic basis, our system can measure how well the essay matches the text. Various assessments are presented to the student regarding the topic, the outline and the coherence of the essay. Our experiments yield promising results.
Get full access to this article
View all access options for this article.
References
1.
LandauerT. K. and DumaisS. T., A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction and Representation of Knowledge, Psychological Review, 104: 2, pp. 211–240, 1997.
2.
HayesJ. R. and FlowerL. S., Identifying the Organization of Writing Processes, in Cognitive Processes in Writing, GreggL. W. and SteinbergE. R. (eds.), Erlbaum, Hillsdale, pp. 3–30, 1980.
3.
LandS. M. and HannafinM. J., A Conceptual Framework for the Development of Theories in Action with Open-Ended Learning Environments, Educational Technology Research and Development, 44: 3, pp. 37–53, 1996.
4.
EvensM.VandendorpeJ., and WangY. C., Lexical-Semantic Relations for Information Retrieval, in Humans and Machines, WilliamsS. (ed.), Ablex, Norwood, pp. 73–100, 1995.
5.
DeerwesterS.DumaisS. T.FurnasG. W.LandauerT. K., and HarshmannR., Indexing by Latent Semantic Analysis, Journal of the American Society for Information Science, 41, pp. 391–407, 1990.
6.
LandauerT. K.FoltzP. W., and LahamD., An Introduction to Latent Semantic Analysis, Discourse Processes, 25: 2–3, pp. 259–284, 1998.
7.
DumaisS. T., Improving the Retrieval of Information From External Sources, Behavior Research Methods, Instruments, & Computers, 23: 2, pp. 229–236, 1991.
8.
FoltzP. W., Latent Semantic Analysis for Text-Based Research, Behavior Research Methods, Instruments, & Computers, 28: 2, pp. 197–202, 1996.
9.
DessusP., Semantic Checking of Hypertext Links, in Hypertextes, Hypermédias et Internet, BalpeJ.-P.LeluA.NatkinS., and SalehI. (eds.), Hermès, Paris, pp. 119–129, 1999 (in French).
10.
FukkinkR. G. and de GlopperK., Effects of Instruction in Deriving Word Meaning from Context: A Meta-Analysis, Review of Educational Research, 68: 4, pp. 450–469, 1998.
11.
RehderB.SchreinerM. E.WolfeM. B.LahamD.LandauerT. K., and KintschW., Using Latent Semantic Analysis to Assess Knowledge: Some Technical Considerations, Discourse Processes, 25: 2–3, pp. 337–354, 1998.
12.
StahlG., Allowing Learners to be Articulate: Incorporating Automated Text Evaluation into Collaborative Software Environments, Proposal to the McDonnell Foundation, 1997.
13.
Wiemer-HastingsP.Wiemer-HastingsK., and GraesserA., Improving an Intelligent Tutor's Comprehension of Students with Latent Semantic Analysis, in Artificial Intelligence in Education (AIED'99), LajoieS. and VivetM. (eds.), IOS Press, Amsterdam, pp. 535–542, 1999.
14.
Wiemer-HastingsP.Wiemer-HastingsK., and GraesserA., Approximate Natural Language Understanding for an Intelligent Tutor, in Proceedings of the 12th Florida Artificial Intelligence Research Symposium (FLAIRS'99), 1999.
15.
Wiemer-HastingsP., How Latent is Latent Semantic Analysis? in Proceedings of the 16th International Joint Conference on Artificial Intelligence (IJCAI'99), Stockholm, 1999.
16.
LemaireB., Models of High-Dimensional Semantic Spaces, in Proceedings of the 4th International Workshop on MultiStrategy Learning (MSL'98), Desenzano, 1998.
17.
LemaireB., Tutoring Systems Based on Latent Semantic Analysis, in Artificial Intelligence in Education (AIED'99), LajoieS. and VivetM. (eds.), IOS Press, Amsterdam, pp. 527–534, 1999.
18.
BlankD.HolmesG.WellsR., and WolinskiP., Interactive Gradebook: The Missing (Hyper)Link, ACM SIGCSE, 1998.
19.
StockburgerD. W., Automated Grading of Homework Assignments and Tests in Introductory and Intermediate Statistics Courses Using Active Server Pages, Behavior Research Methods, Instruments, & Computers, 31: 2, pp. 252–262, 1999.
20.
WreschW., The Imminence of Grading Essays by Computer—25 Years Later, Computers and Composition, 10: 2, pp. 45–58, 1993.
21.
PageE., The imminence of Grading Essays by Computer, Phi Delta Kappan, 47, pp. 238–243, 1966.
22.
FoltzP. W.LahamD., and LandauerT. K., Automated Essay Scoring: Applications to Educational Technology, Proceedings of the ED-MEDIA'99 Conference, AACE, Charlottesville, 1999.
23.
KintschE.SteinhartD., G. Stahl, and the LSA Research Group, Developing Summarization Skills through the Use of LSA-Based Feedback, Interactive Learning Environments, 8: 2, pp. 87–109, 2000.
24.
StahlG., R. dePaula, and the LSA Research Group, Evolution of an LSA-Based Interactive Environment for Learning to Write Summaries, Interactive Learning Environments, to appear.
25.
Wiemer-HastingsP. and GraesserA., Select-a-Kibitzer: A Computer Tool that Gives Meaningful Feedback on Student Compositions, Interactive Learning Environments, 8: 2, pp. 149–169, 2000.
26.
FoltzP. W.KintschW., and LandauerT. K., The Measurement of Textual Coherence with Latent Semantic Analysis, Discourse Processes, 25: 2–3, pp. 285–307, 1998.
27.
WolfeM. B.SchreinerM. E.RehderB., and LahamD., Learning from Text: Matching Readers and Texts by Latent Semantic Analysis, Discourse Processes, 25: 2–3, pp. 309–336, 1998.
28.
ZellermayerM.SalomonG.GlobersonT., and GivonH., Enhancing Writing-Related Metacognitions through a Computerized Writing Partner, American Educational Research Journal, 28: 2, pp. 373–391, 1991.
29.
KozmaR. B., The Impact of Computer-Based Tools and Embedded Prompts on Writing Processes and Products of Novice and Advanced College Writers, Cognition and Instruction, 8: 1, pp. 1–27, 1991.
30.
FattonA. and Severinson-EklundhK., How to Support In-Process Planning in a Computer-Based Writing Environment, in Proceedings of the European Writing Conference, Barcelona, 1996.
