Sage Journals: Discover world-class research

Abstract

This article promotes the use of modern test theory in testing situations where sum scores for binary responses are now used. It directly compares the efficiencies and biases of classical and modern test analyses and finds an improvement in the root mean squared error of ability estimates of about 5% for two designed multiple-choice tests and about 12% for a classroom test. A new parametric density function for ability estimates, the tilted scaled β, is used to resolve the nonidentifiability of the univariate test theory model. Item characteristic curves (ICCs) are represented as basis function expansions of their log-odds transforms. A parameter cascading method along with roughness penalties is used to estimate the corresponding log odds of the ICCs and is demonstrated to be sufficiently computationally efficient that it can support the analysis of large data sets.

Keywords

parameter cascading item characteristic curves sum score distribution performance manifold

Get full access to this article

View all access options for this article.

References

Bates

D. M.

Watts

D. B.

(1988). Nonlinear regression analysis and its applications. New York: Wiley.

Birnbaum

(1968). Some latent trait models and their use in inferring an examinee’s ability. In Lord

Novick

M. R.

(Eds.), Statistical theories of mental tests (pp. 395–479). Reading, MA: Addison-Wesley.

Cover

T. M.

Thomas

T. M.

(2006). Elements of information theory (2nd ed.). Hoboken, NJ: Wiley.

Gulliksen

(1950). The theory of mental tests. New York, NY: John Wiley.

Lee

Y.-S.

(2007). A comparison of methods for nonparametric estimation of item characteristic curves for binary items. Applied Psychological Measurement, 37, 121–134.

Mokken

R. J.

(1997). Nonparametric models for dichotomous responses. In van der Linden

W. J.

Hambleton

R. K.

(Eds.), Handbook of modern item response theory (pp. 351–367). New York, NY: Springer.

Murphy

S. A.

van der Vaart

A. W.

(2000). On profile likelihood. Journal of the American Statistical Association, 95, 449–485.

Ramsay

J. O.

(1991). Kernel smoothing approaches to nonparametric item characteristic curve estimation. Psychometrika, 56, 611–630.

Ramsay

J. O.

(1997). A functional approach to modeling test data. In van der Linden

W. J.

Hambleton

R. K.

(Eds.), Handbook of modern item response theory (pp. 351–367). New York, NY: Springer.

10.

Ramsay

J. O.

Hooker

Graves

(2009). Functional data analysis with r and MATLAB. New York: Springer.

11.

Ramsay

J. O.

Silverman

B. W.

(2002). Functional models for test items. Applied functional data analysis. New York, NY: Springer-Verlag.

12.

Ramsay

J. O.

Silverman

B. W.

(2005). Functional data analysis (2nd ed.). New York, NY: Springer-Verlag.

13.

Rossi

Wang

Ramsay

J. O.

(2002). Nonparametric item response function estimates with the EM algorithm. Journal of the Behavioral and Educational Sciences, 27, 291–317.

14.

Severini

Wong

W. H.

(1992). Profile likelihood and conditionally parametric models. Annals of Statistics, 20, 1768–1802.

15.

Thissen

Orlando

(2001). Item response theory for items scored in two categories. In Thissen

Wainer

(Eds.), Test scoring (pp. 73–140). Mahwah, NJ: Lawrence Erlbaum.

16.

Thurstone

L. L.

(1925). A method of scaling psychological and educational tests. Journal of Educational Psychology, 16, 433–449.

17.

Woods

C. M.

(2006). Ramsay-curve item response theory (RC-IRT) to detect and correct for nonnormal latent variables. Psychological Methods, 11, 253–270.

18.

Woods

C. M.

Thissen

(2006). Item response theory with estimation of the latent population distribution using spline-based densities. Psychometrika, 71, 281–301.

A Strategy for Replacing Sum Scoring

Abstract

Keywords

Get full access to this article

References