Sage Journals: Discover world-class research

Abstract

It has been widely known that the Type I error rates of goodness-of-fit tests using full information test statistics, such as Pearson’s test statistic χ² and the likelihood ratio test statistic G², are problematic when data are sparse. Under such conditions, the limited information goodness-of-fit test statistic M₂ is recommended in model fit assessment for models with binary response data. A simulation study was conducted to investigate the power and Type I error rate of M₂ in fitting unidimensional models to many different types of multidimensional data. As an additional interest, the behavior of RMSEA₂ was also examined, which is the root mean square error approximation (RMSEA) based on M₂. Findings from the current study showed that M₂ and RMSEA₂ are sensitive in detecting the misfits due to varying slope parameters, the bifactor structure, and the partially (or completely) simple structure for multidimensional data, but not the misfits due to the within-item multidimensional structures.

Keywords

item response theory limited information statistic multidimensional structures

Get full access to this article

View all access options for this article.

References

Adams

R. J.

Wilson

Wang

W.-C.

(1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21, 1-23.

Ansley

T. N.

Forsyth

R. A.

(1985). An examination of the characteristics of unidimensional IRT parameter estimates derived from two-dimensional data. Applied Psychological Measurement, 9, 37-48.

Bartholomew

D. J.

Leung

S. O.

(2002). A goodness of fit test for sparse 2p contingency tables. British Journal of Mathematical and Statistical Psychology, 55, 1-15.

Bartholomew

D. J.

Tzamourani

(1999). The goodness of fit of latent trait models in attitude measurement. Sociological Methods & Research, 27, 525-546.

Barton

M. A.

Lord

F. M.

(1981). An upper asymptote for the three-parameter logistic item-response model. Princeton, NJ: Educational Testing Service.

Bolt

D. M.

Lall

V. F.

(2003). Estimation of compensatory and noncompensatory multidimensional item response models using Markov Chain Monte Carlo. Applied Psychological Measurement, 27, 395-414.

Browne

M. W.

Cudeck

(1993). Alternative ways of assessing model fit. In Bollen

Long

(Eds.), Testing structural equation models (pp. 136-162). Newbury Park, CA: SAGE.

Cai

(2013). flexMIRT: A numerical engine for flexible multilevel multidimensional item analysis and test scoring (Version 2.0) [Computer software]. Chapel Hill, NC: Vector Psychometric Group.

Cai

Hansen

(2013). Limited-information goodness-of-fit testing of hierarchical item factor models. British Journal of Mathematical and Statistical Psychology, 66, 245-276.

10.

Cai

Maydeu-Olivares

Coffmanand

D. L.

Thissen

(2006). Limited-information goodness-of-fit testing of item response theory models for sparse 2P tables. British Journal of Mathematical and Statistical Psychology, 59, 173-194.

11.

Collins

L. M.

Fidler

P. L.

Wugalter

S. E.

Long

J. D.

(1993). Goodness-of-fit testing for latent class models. Multivariate Behavioral Research, 28, 375-389.

12.

Hambleton

R. K.

Murray

L. N.

(1983). Some goodness of fit investigations for item response models. In Hambleton

R. K.

(Ed.), Applications of item response theory (pp. 71-94). Vancouver, Canada: Educational Research Institute of British Columbia.

13.

Hansen

Cai

Monroe

(2014). Limited-information goodness-of-fit testing of diagnostic classification item response theory models (CRESST Report No. 840). Los Angeles: National Center for Research on Evaluation, Standards, and Student Testing, University of California.

14.

L.-t.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.

15.

Koehler

Larntz

(1980). An empirical investigation of goodness-of-fit statistics for sparse multinomials. Journal of the American Statistical Association, 75, 336-344.

16.

M. F.

Olejnik

(1997). The power of Rasch person-fit statistics in detecting unusual response patterns. Applied Psychological Measurement, 21, 215-231.

17.

Liao

W.-W.

R.-G.

Yen

Y.-C.

Cheng

H.-C.

(2012). The four-parameter logistic item response theory model as a robust method of estimating ability despite aberrant responses. Social Behavior and Personality, 40, 1679-1694.

18.

Maris

(1999). Estimating multiple classification latent class models. Psychometrika, 64, 187-212.

19.

Maydeu-Olivares

(2013). Goodness-of-fit assessment of item response theory models. Measurement, 11, 71-101.

20.

Maydeu-Olivares

Cai

Hernández

(2011). Comparing the fit of item response theory and factor analysis models. Structural Equation Modeling, 18, 333-356.

21.

Maydeu-Olivares

Garca-Forero

(2010). Goodness-of-fit testing. International Encyclopedia of Education, 7, 190-196.

22.

Maydeu-Olivares

Joe

(2005). Limited- and full-information estimation and goodness-of-fit testing in 2ⁿ contingency tables: A unified framework. Journal of the American Statistical Association, 100, 1009-1020.

23.

Maydeu-Olivares

Joe

(2006). Limited information goodness-of-fit testing in multidimensional contingency tables. Psychometrika, 71, 713-732.

24.

Maydeu-Olivares

Joe

(2014). Assessing approximate fit in categorical data analysis. Multivariate Behavioral Research, 49, 305-328. doi:10.1080/00273171.2014.911075

25.

Marsh

H. W.

Hau

K. T.

Wen

(2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling, 11, 320-341.

26.

Orlando

Thissen

(2000). Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24, 50-64.

27.

Orlando

Thissen

(2003). Further investigation of the performance of S - X2: An item fit index for use with dichotomous item response theory models. Applied Psychological Measurement, 27, 289-298.

28.

Reckase

M. D.

(2009). Multidimensional item response theory (Statistics for social and behavioral sciences). Dordrecht, The Netherlands: Springer.

29.

Steiger

J. H.

Lind

J. C.

(1980, May). Statistically based tests for the number of common factors. Paper presented at the Psychometrika Society meeting, Iowa City, IA.

30.

Tollenaar

Mooijaart

(2003). Type I errors and power of the parametric bootstrap goodness-of-fit test: Full and limited information. British Journal of Mathematical and Statistical Psychology, 56, 271-288.

31.

Zhang

Stone

C. A.

(2008). Evaluating item fit for multidimensional item response models. Educational and Psychological Measurement, 68, 181-196.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.69 MB

Investigating the Behaviors of M 2 and RMSEA 2 in Fitting a Unidimensional Model to Multidimensional Data

Abstract

Keywords

Get full access to this article

References

Supplementary Material