Latent semantic analysis (LSA) is an automated, statistical technique for comparing the semantic similarity of words or documents. In this article, I examine the application of LSA to automated essay scoring. I compare LSA methods to earlier statistical methods for assessing essay quality, and critically review contemporary essay-scoring systems built on LSA, including the Intelligent Essay Assessor, Summary Street, State the Essence, Apex, and Select-a-Kibitzer. Finally, I discuss current avenues of research, including LSA's application to computer-measured readability assessment and to automatic summarization of student essays.
Get full access to this article
View all access options for this article.
References
1.
AbramM. J. (1981). Readability: Its use in adult education. Lifelong Learning: The Adult Years, 4(5), 8–9, 30.
2.
BereiterC.ScardamaliaM. (1987). The psychology of written composition. Hillsdale, NJ: Lawrence Erlbaum Associates.
3.
BursteinJ.KukichK.WolffS.LuC.ChodorowM. (1998). Enriching automated scoring using discourse marking. Proceedings of the Workshop on Discourse Relations and Discourse Marking, 36th Annual Meeting of the Association for Computational Linguistics and the 17th International Conference on Computational Linguistics.
4.
BursteinJ.KukichK.WolffS.LuC.ChodorowM.Braden-HarderL.HarrisM. D. (1998). Automated essay scoring using a hybrid feature identification technique. Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and the 17th International Conference on Computation Linguistics, pp. 206–210.
5.
BursteinJ.MarcuD. (2000). Benefits of modularity in an automated scoring system. Proceedings of the Workshop on Using Toolsets and Architectures to Build NLP Systems, 18th International Conference on Computational Linguistics.
6.
CoffmanW. E. (1971). Essay examinations. In ThorndikeR. L. (Ed.), Educational measurement (2nd ed., pp. 271–302). Washington, DC: American Council on Education.
7.
DaleE.ChallJ. S. (1948). A formula for predicting readability. Educational Research Bulletin, 87, 11–20.
8.
DeerwesterS.DumaisS. T.FurnasG. W.LandauerT. K.HarshmanR. (1990). Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41, 391–407.
9.
DessusP.LemaireB. (1999). Apex, un système d'aide à la prèparation d'examens [Apex, a system to assist in the preparation of exams]. Sciences et Techniques Éducatives, 6(2), 409–415.
10.
DessusP.LemaireB.VernierA. (2000). Free-text assessment in a virtual campus. Proceedings of the 3rd International Conference on Human-Learning Systems.
11.
FinlaysonD. S. (1951). The reliability of the marking of essays. British Journal of Educational Psychology, 21(2), 126–134.
12.
FleschR. (1948). A new readability yardstick. Journal of Applied Psychology, 32, 221–233.
13.
FoltzP. W. (1996). Latent semantic analysis for text-based research. Behavior Research Methods, Instruments, and Computers, 28(2), 197–202.
14.
FoltzP. W.KitschW.LandauerT. K. (1998). The measurement of textual coherence with latent semantic analysis. Discourse Processes, 25(2&3), 285–307.
15.
FoltzP. W.LahamD.LandauerT. K. (1999). Automated essay scoring: Applications to educational technology. Proceedings of the ED-MEDIA 1999 World Conference on Educational Multimedia, Hypermedia, and Telecommunications.
16.
FurnasG. W.LandauerT. K.GomezL. M.DumaisS. T. (1983). Statistical semantics: Analysis of the potential performance of key-word information systems. Bell System Technical Journal, 62(6), 1753–1806.
17.
HopkinsK. D.StanleyJ. C.HopkinsB. R. (1990). Constructing and using essay tests. In Educational and psychological measurement and evaluation (7th ed., pp. 193–223). Englewood Cliffs, NJ: Prentice-Hall.
18.
JoanisE. J. S. (1999). Review of the literature on aggregation in natural language generation (Tech. Rep. No. CSRG-398). Toronto, Canada: University of Toronto, Department of Computer Science.
19.
KintschE.SteinhartD., Stahl, G., the LSA Research Group, MatthewsC.LambR. (2000). Developing summarization skills through the use of LSA-based feedback. Interactive Learning Environments, 8(2), 87–109.
20.
LandauerT. K.FoltzP. W.LahamD. (1998). An introduction to latent semantic analysis. Discourse Processes, 25(2&3), 259–284.
21.
LandauerT. K.LahamD.FoltzP. W. (2000). The Intelligent Essay Assessor. IEEE Intelligent Systems, 15(5), 27–31.
22.
LandauerT. K.LahamD.RehderB.SchreinerM. E. (1997). How well can passage meaning be derived without using word order? A comparison of latent semantic analysis and humans. Proceedings of the 19th Annual Conference of the Cognitive Science Society, pp. 412–417.
23.
LemaireB.DessusP. (2001). A system to assess the semantic content of student essays. Journal of Educational Computing Research, 24(3), 305–320.
24.
MiltsakakiE.KukichK. (2000a). Automated evaluation of coherence in student essays. Proceedings of the Workshop on Language Resources and Tools in Educational Applications, 2nd International Conference on Language Resources and Evaluation.
25.
MiltsakakiE.KukichK. (2000b). The role of centering theory's rough-shift in the teaching and evaluation of writing skills. Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics.
26.
PageE. B. (1966). The imminence of grading essays by computer. Phi Delta Kappan, 47, 238–243.
27.
PageE. B. (1994). Computer grading of student prose, using modern concepts and software. Journal of Experimental Education, 62(2), 127–142.
28.
PageE. B.PetersenN.S. (1995). The computer moves into essay grading: Updating the ancient test. Phi Delta Kappan, 76, 561–565.
29.
RehderB.SchreinerM. E.WolfeM. B. W.LahamD.LandauerT. K.KitschW. (1998). Using latent semantic analysis to assess knowledge: Some technical considerations. Discourse Processes, 25(2&3), 337–354.
30.
JonesK. Spärck (1972). A statistical interpretation of term specificity and its application in retrieval. Journal of Documentation, 28(1), 11–20.
31.
StandalT. C. (1987). Computer-measured readability. Computers in the Schools, 4(1), 123–132.
32.
TaylorM. C.WahlstromM. W. (1986). Readability as applied to an ABE assessment instrument. Adult Literacy and Basic Education: An International Journal for Basic Education, 10(3), 155–170.
33.
Wiemer-HastingsP. (2000). Adding syntactic information to LSA. Proceedings of the 22nd Annual Conference of the Cognitive Science Society, pp. 989–993.
34.
Wiemer-HastingsP.GraesserA. (2000). Select-a-Kibitzer: A computer tool that gives meaningful feedback on student compositions. Interactive Learning Environments, 8(2), 149–169.
35.
WolfeM. B. W.SchreinerM. E.RehderB.LahamD. (1998). Learning from text: Matching readers and texts by latent semantic analysis. Discourse Processes, 25(2&3), 309–336.
