Sage Journals: Discover world-class research

Abstract

Selection of an appropriate item response model is critical in the measurement of latent examinee ability. The one-, two-, and three-parameter logistic (1PL, 2PL, and 3PL) models are nested, and as such can be compared using likelihood ratio (LR) tests. The null hypothesis in the LR test for selection among the 2PL and 3PL models sets the guessing parameters to their lower bound of 0. This violates one of the assumptions of the LR test and renders the usual $χ^{2}$ reference distribution inappropriate for the comparison. A review of the current literature revealed that this problem is not well understood in the educational measurement field. Ignoring this issue can lead to selection of an overly simplified model, with implications for the ability estimates. In this article, the use of the LR test for item response model selection is investigated, with the goal of providing practitioners with an appropriate method of selecting the most parsimonious model. The results of simulation studies indicate the nature of the problem, with inaccurate Type I error rates for cases where the inappropriate null distribution was used. An analysis of data from a statewide mathematics test showed differences pertinent to subsequent analyses.

Keywords

item response theory model selection likelihood ratio test boundary issue

Get full access to this article

View all access options for this article.

References

Akaike

(1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19, 716-723.

Andersen

E. B.

(1973). A goodness of fit test for the Rasch model. Psychometrika, 38, 123-140.

Bachman

L. F.

Davidson

Ryan

Choi

I.-C.

(1995). An investigation into the comparability of two tests of English as a foreign language: The Cambridge-TOEFL comparability study. Cambridge, UK: University of Cambridge Local Examinations Syndicate and Cambridge University Press.

Bergan

J. R.

(2010). Assessing the relative fit of alternative item response theory models to the data. Retrieved from http://www.ati-online.com/pdfs/researchK12/AlternativeIRTModels.pdf

Bock

R. D.

(1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29-51.

Bock

R. D.

Aitkin

(1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443-459.

De Ayala

R. J.

(2009). The theory and practice of item response theory. New York, NY: Guilford Press.

DeMars

(2010). Item response theory. Oxford, UK: Oxford University Press.

du Toit

(Ed.). (2003). IRT from SSI. Lincolnwood, IL: Scientific Software International.

10.

Edelen

Reeve

(2007). Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement. Quality of Life Research, 16, 5-18.

11.

Embretson

S. E.

Reise

S. P.

(2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.

12.

Florida Department of Education. (2003). Florida Comprehensive Assessment Test. Unpublished instrument.

13.

Glas

C. A. W.

(1999). Modification indices for the 2-PL and the nominal response model. Psychometrika, 64, 273-294.

14.

Hambleton

R. K.

Swaminathan

Rogers

H. J.

(1991). Fundamentals of item response theory. Newbury Park, CA: Sage.

15.

Kang

Cohen

A. S.

(2007). IRT model selection methods for dichotomous items. Applied Psychological Measurement, 31, 331-358.

16.

Kline

(2005). Psychological testing: A practical approach to design and evaluation. Thousand Oaks, CA: Sage.

17.

Kutner

M. H.

Nachtsheim

C. J.

Neter

(2005). Applied linear statistical models (5th ed.). New York, NY: McGraw-Hill.

18.

Lord

F. M.

(1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.

19.

Lord

F. M.

Novick

M. R.

(1968). Statistical theories of mental test scores (with contributions by A. Birnbaum). Reading, MA: Addison-Wesley.

20.

Maydeu-Olivares

Cai

(2006). A cautionary note on using G2(dif) to assess relative model fit in categorical data analysis. Multivariate Behavioral Research, 41, 55-64.

21.

Oehlert

G. W.

(2000). A first course in design and analysis of experiments. New York, NY: Freeman.

22.

Orlando

Thissen

(2000). Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24, 50-64.

23.

Reise

S. P.

Waller

N. G.

(2003). How many IRT parameters does it take to model psycholopathology items? Psychological Methods, 8, 164-184.

24.

Rizopoulos

(2006). ltm: An R package for latent variable modeling and item response theory analyses. Journal of Statistical Software, 17, 1-25.

25.

Schwarz

G. E.

(1978). Estimating the dimension of a model. Annals of Statistics, 6, 461-464.

26.

Self

S. G.

Liang

K.-Y.

(1987). Asymptotic properties of maximum likelihood estimators and likelihood ratio tests under nonstandard conditions. Journal of the American Statistical Association, 82, 605-610.

27.

Silvapulle

M. J.

Sen

P. K.

(2005). Constrained statistical inference: Order, inequality, and shape constraints. Hoboken, NJ: Wiley.

28.

Stram

D. O.

Lee

J. W.

(1994). Variance components testing in the longitudinal mixed effects model. Biometrics, 50, 1171-1177.

29.

Thissen

(1991). MULTILOG user’s guide: Multiple categorical item analysis and test scoring using item response theory. Chicago, IL: Scientific Software International.

30.

Waller

M. I.

(1981). A procedure for comparing logistic latent trait models. Journal of Educational Measurement, 18, 119-125.

31.

Wilks

S. S.

(1938). The large-sample distribution of the likelihood ratio for testing composite hypotheses. The Annals of Mathematical Statistics, 9, 60-62.

32.

Yen

W. M.

(1981). Using simulation results to choose a latent trait model. Applied Psychological Measurement, 5, 245-262.

33.

Yen

W. M.

(1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8, 125-145.

34.

Zimowski

M. F.

Muraki

Mislevy

R. J.

Bock

R. D.

(1996). BILOG MG: Multiple-group IRT analysis and test maintenance for binary items. Chicago, IL: Scientific Software International.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.12 MB

Comparing the Two- and Three-Parameter Logistic Models via Likelihood Ratio Tests

Abstract

Keywords

Get full access to this article

References

Supplementary Material