Most automated essay scoring programs use a linear regression model to predict an essay score from several essay features. This article applied a cumulative logit model instead of the linear regression model to automated essay scoring. Comparison of the performances of the linear regression model and the cumulative logit model was performed on a large variety of data sets. It appears that the cumulative logit model performed somewhat better than did the linear regression model.
Cohen J..A coefficient of agreement for nominal scales.Educational and Psychological Measurement. 1960;20:37-46
5.
Davey T...
Paper presented at the Annual meeting of the National Council of Measurement in Education, April; 2009San Diego, CA; 2009. .
6.
Draper N. R.,Smith H.Applied regression analysis. 3rd ed.New York, NY: Wiley; 1998:
7.
Feng X.,Dorans N. J.,Patsula L. N.,Kaplan B.Improving the statistical aspects of e-rater®: Exploring alternative feature reduction and combination rules (ETS Research Rep. No. RR-03-15). Princeton, NJ: ETS; 2003:
8.
Gilula Z.,Haberman S. J..Models for analyzing categorical panel data.Journal of the American Statistical Association. 1994;89:645-656
9.
Haberman S. J..Concavity and estimation.The Annals of Statistics. 1989;17:1631-1661
10.
Haberman S. J.Handbook of statistics. Rao C. R.Sinharay S., ed. Amsterdam, Netherlands: North-Holland; 2007:205-233.
11.
Haberman S. J..When can subscores have value?.Journal of Educational and Behavioral Statistics. 2008;33:204-229
12.
Johnson V. E..On Bayesian analysis of multirater ordinal data: An application of automated essay scoring.Journal of the American Statistical Association. 1996;91:42-51
13.
McCullagh P.,Nelder J. A.Generalized linear models. 2nd ed.Boca Raton, FL: Chapman and Hall; 1989:
14.
Neter J.,Kutner M. H.,Nachtsheim C. J.,Wasserman W.Applied linear regression models. Homewood, IL: Irwin; 1996:
15.
Pratt J. W..Concavity of the log likelihood.Journal of the American Statistical Association. 1981;76:103-109
16.
Spitzer R. L.,Cohen J.,Fleiss J. L.,Endicott J..Quantification of agreement in psychiatric diagnosis.Archives of General Psychiatry. 1967;17:83-87
