Using the Bootstrap Method to Evaluate the Critical Range of Misfit for Polytomous Rasch Fit Statistics

Abstract

The purpose of this study was to apply the bootstrap procedure to evaluate how the bootstrapped confidence intervals (CIs) for polytomous Rasch fit statistics might differ according to sample sizes and test lengths in comparison with the rule-of-thumb critical value of misfit. A total of 25 simulated data sets were generated to fit the Rasch measurement and then a total of 1,000 replications were conducted to compute the bootstrapped CIs under each of 25 testing conditions. The results showed that rule-of-thumb critical values for assessing the magnitude of misfit were not applicable because the infit and outfit mean square error statistics showed different magnitudes of variability over testing conditions and the standardized fit statistics did not exactly follow the standard normal distribution. Further, they also do not share the same critical range for the item and person misfit. Based on the results of the study, the bootstrapped CIs can be used to identify misfitting items or persons as they offer a reasonable alternative solution, especially when the distributions of the infit and outfit statistics are not well known and depend on sample size.

Keywords

infit and outfit Rasch model bootstrap method fit statistics

Get full access to this article

View all access options for this article.

References

Ames

Penfield

(2015) An NCME instructional module on item-fit statistics for item response theory models. Educational Measurement and Practice 34: 39–48.

Andrich

(1978) A rating formulation for ordered response categories. Pyshometrika 31: 84–98.

Andrich

Sheridan

Luo

(2010) RUMM 2030 [Computer Program] , Perth, PA: RUMM Laboratory Pty Ltd.

Bond

T. G.

Fox

C. M.

(2015) Applying the Rasch model: Fundamental measurement in the human sciences, New York, NY: Routledge.

Boone

W. J.

Staver

J. R.

Yale

M. S.

(2014) Rasch analysis in the human sciences, New York, NY: Springer Verlag.

Dimitrov

D. M.

Smith

R. M.

(2006) Adjusted Rasch person-fit statistics. Journal of Applied Measurement 7: 170–183.

Efron

(1979) Bootstrap methods: Another look at the jackknife. The Annals of Statistics 7(1): 1–26.

Efron

Tibshirani

R. J.

(1993) An introduction to the bootstrap, New York, NY: Chapman & Hall.

Engelhard

(2013) Invariant measurement: Using Rasch models in the social behavioral, and health sciences, New York, NY: Routledge.

10.

Fischer

G. H.

Molenaar

I. W.

(1995) Rasch models: Foundations, recent developments, and applications, New York, NY: Springer Verlag.

11.

Han

K. T.

(2007) WinGen: Windows software that generates IRT parameters and item responses. Applied Psychological Measurement 31: 457–459.

12.

Han

K. T.

Hambleton

R. K.

(2014) WinGen3: Windows software that generates IRT parameters and item responses [Computer Program] , Amherst: University of Massachusetts, Center for Educational AssessmentRetrieved from http://www.hantest.net/wingen.

13.

Karabatos

(2003) Comparing the aberrant response detection performance of thirty-six person fit statistics. Applied Measurement in Education 16: 277–298.

14.

Linacre

J. M.

(1994) Many-facet Rasch measurement, Chicago, IL: MESA press.

15.

Linacre

J. M.

(2003) Rasch power analysis size vs. significance standardized chi-square fit statistics. Rasch Measurement Transactions 17: 918.

16.

Linacre

J. M.

(2014a) Winsteps® Rasch measurement computer program, Beaverton, OR: Winsteps.com.

17.

Linacre

J. M.

(2014b) Winsteps® Rasch measurement computer program user’s guide, Beaverton, OR: Winsteps.com.

18.

Masters

G. N.

(1982) A Rasch model for partial credit scoring. Psychometrika 47: 149–174.

19.

Meijer

R. R.

(1996) Person-fit research: An introduction. Applied Measurement in Education 9: 3–8.

20.

Meyer, J. P. (2014). jMetrik: Open source psychometric software [Computer Program]. Retrieved from http://www.itemanalysis.com/about.php.

21.

Smith

R. M.

(1991a) The distributional properties of Rasch item fit statistics. Educational and Psychological Measurement 51: 541–565.

22.

Smith

R. M.

(1991b) IPARM: Item and person analysis with the Rasch model, Chicago, IL: MESA Press.

23.

Smith

R. M.

Plackner

(2009) The family approach to assessing fit in Rasch measurement. Journal of Applied Measurement 10: 424–437.

24.

Smith

R. M.

Schumacker

R. E.

Bush

M. J.

(1998) Using item mean squares to evaluate fit to the Rasch model. Journal of Outcome Measurement 2: 66–78.

25.

Stone

C. A.

(2003) Empirical power and Type I error rates for an IRT fit statistics that considers the precision of ability estimates. Educational and Psychological Measurement 63: 566–583.

26.

Stone

C. A.

Zhang

(2003) Assessing goodness of fit of item response theory models: A comparison of traditional and alternative procedures. Journal of Educational Measurement 40: 331–352.

27.

Y. H.

Sheu

C. F.

Wang

W. C.

(2007) Computing confidence intervals of item fit statistics in the family of Rasch models using the bootstrap method. Journal of Applied Measurement 8: 190–203.

28.

Wang

W. C.

Chen

C. T.

(2005) Item parameter recovery, standard error estimates, and fit statistics of the WINSTEPS program for the family of Rasch models. Educational and Psychological Measurement 65: 376–404.

29.

Wilson

E. B.

Hilferty

M. M.

(1931) The distribution of chi-square. Proceeding of the National Academy of Sciences of the United States of America 17: 684–688.

30.

Wolfe

E. W.

(2008) RBF. sas (Rasch bootstrap Fit): A SAS macro for estimating critical values for Rasch model fit statistics. Applied Psychological Measurement 32: 585–586.

31.

Wolfe

E. W.

(2013) A bootstrap approach to evaluating person and item fit to the Rasch model. Journal of Applied Measurement 14(1): 1–9.

32.

Wright

B. D.

Linacre

(1994) Reasonable mean-square fit values. Rasch Measurement Transactions 8: 370.

33.

Wright

B. D.

Masters

(1982) Rating scale analysis, Chicago, IL: MESA Press.

34.

Wright

Stone

M. H.

(1979) Best test design, Chicago, IL: MESA Press.

35.

Adams

(2013) Properties of Rasch residual fit statistics. Journal of Applied Measurement 14: 400–413.

36.

M. L.

Adams

R. J.

Wilsons

M. R.

(1998) ConQuest Generalized item response modeling software [Computer Program] , Camberwell, VIC, Australia: Australian Council for Educational Research.