Residual-based fit statistics, which compare observed item statistics (e.g., proportions) with model-implied probabilities, are widely used to evaluate model fit, item fit, and local dependence in item response theory (IRT) models. Despite the prevalence of item non-responses in empirical studies, their impact on these statistics has not been systematically examined. Existing software (package) often applies heuristic treatments (e.g., listwise or pairwise deletion), which can distort fit statistics because missing data further inflate discrepancies between observed and expected proportions. This study evaluates the appropriateness of such treatments through extensive simulation. Results show that deletion methods degrade the accuracy of fit testing: fit indices are inflated under both null and power conditions, with the bias worsening as missingness increases. In addition, the impact of missing data exceeds that of model misspecification. Practical recommendations and alternative methods are discussed to guide applied researchers.
AllisonP. D. (2009). Missing data. In MilsapR. E.Maydeu-OlivaresA. (Eds.), The SAGE handbook of quantitative methods in psychology (pp. 72–89). Sage.
2.
BockR. D.AitkinM. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46(4), 443–459.
3.
BrowneM. W. (1984). Asymptotically distribution-free methods for the analysis of covariance structures. British Journal of Mathematical and Statistical Psychology, 37(1), 62–83.
4.
CaiL. (2015). Lord–Wingersky algorithm version 2.0 for hierarchical item factor models with applications in test scoring, scale alignment, and model fit testing. Psychometrika, 80(2), 535–559.
5.
CaiL. (2024). flexMIRT: Flexible multilevel multidimensional item analysis and test scoring (Version 3.7) [Computer software]. Vector Psychometric Group, LLC.
6.
CaiL.HansenM. (2013). Limited-information goodness-of-fit testing of hierarchical item factor models. British Journal of Mathematical and Statistical Psychology, 66(2), 245–276.
7.
CaiL.MonroeS. (2014). A new statistic for evaluating item response theory models for ordinal data (CRESST Report No. 839). University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
8.
ChalmersR. P. (2012). Mirt: A multidimensional item response theory package for the R Environment. Journal of Statistical Software, 48(6), 1–29.
9.
ChenW.-H.ThissenD. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22(3), 265–289.
10.
ChonK. H.LeeW.-C.DunbarS. B. (2010). A comparison of item fit statistics for mixed IRT models. Journal of Educational Measurement, 47(3), 318–338.
11.
ChungS.CaiL. (2019). Alternative multiple imputation inference for categorical structural equation modeling. Multivariate Behavioral Research, 54(3), 323–337.
12.
EdwardsM. C.HoutsC. R.CaiL. (2018). A diagnostic procedure to detect departures from local independence in item response theory models. Psychological Methods, 23(1), 138–149.
13.
EndersC. K. (2022). Applied missing data analysis (2nd ed.). Guilford Press.
14.
FinchH. (2008). Estimation of item response theory parameters in the presence of missing data. Journal of Educational Measurement, 45(3), 225–245.
15.
GalimardJ.-E.ChevretS.CurisE.Resche-RigonM. (2018). Heckman imputation models for binary or continuous MNAR outcomes and MAR predictors. BMC Medical Research Methodology, 18(1), 90.
16.
GrahamJ. W. (2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 60, 549–576.
17.
HanZ.SinharayS.JohnsonM. S.LiuX. (2023). The standardized S-X2 statistic for assessing item fit. Applied Psychological Measurement, 47(1), 3–18.
18.
HansenM.CaiL.StuckyB. D.TuckerJ. S.ShadelW. G.EdelenM. O. (2014). Methodology for developing and evaluating the PROMIS smoking item banks. Nicotine & Tobacco Research, 16(Suppl. 3), S175–S189.
19.
HeitjanD. F.BasuS. (1996). Distinguishing “missing at random” and “missing completely at random.”The American Statistician, 50(3), 207–213.
20.
HuangS.CaiL. (2021). Lord–Wingersky algorithm version 2.5 with applications. Psychometrika, 86(4), 973–993.
21.
JoeH.Maydeu-OlivaresA. (2010). A general family of limited information goodness-of-fit statistics for multinomial data. Psychometrika, 75(3), 393–419.
22.
KangH. (2013). The prevention and handling of the missing data. Korean Journal of Anesthesiology, 64(5), 402–406.
23.
KangT.ChenT. T. (2008). Performance of the generalized S-X2 item fit index for polytomous IRT models. Journal of Educational Measurement, 45(4), 391–406.
LittleR. J. A. (1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83(404), 1198–1202.
26.
LittleR. J. A.RubinD. B. (2019). Statistical analysis with missing data (3rd ed.). Wiley.
27.
LiuY.Maydeu-OlivaresA. (2013). Local dependence diagnostics in IRT modeling of binary data. Educational and Psychological Measurement, 73(2), 254–274.
28.
LiuY.SriutaisukS. (2020). Evaluation of model fit in structural equation models with ordinal missing data: An examination of the D2 method. Structural Equation Modeling: A Multidisciplinary Journal, 27(4), 561–583.
29.
LiuY.ThissenD. (2012). Identifying local dependence with a score test statistic based on the bifactor logistic model. Applied Psychological Measurement, 36(8), 670–688.
30.
LiuY.ThissenD. (2014). Comparing score tests and other local dependence diagnostics for the graded response model. British Journal of Mathematical and Statistical Psychology, 67(3), 496–513.
31.
LordF. M.WingerskyM. S. (1984). Comparison of IRT true-score and equipercentile observed-score “equatings.”Applied Psychological Measurement, 8(4), 453–461.
32.
Maydeu-OlivaresA. (2013). Goodness-of-fit assessment of item response theory models. Measurement: Interdisciplinary Research and Perspectives, 11(3), 71–101.
33.
Maydeu-OlivaresA.JoeH. (2005). Limited- and full-information estimation and goodness-of-fit testing in 2n contingency tables. Journal of the American Statistical Association, 100(471), 1009–1020.
34.
Maydeu-OlivaresA.JoeH. (2006). Limited information goodness-of-fit testing in multidimensional contingency tables. Psychometrika, 71(4), 713–732.
35.
MislevyR. J.JohnsonE. G.MurakiE. (1992). Chapter 3: Scaling procedures in NAEP. Journal of Educational Statistics, 17(2), 131–154.
36.
MyersT. A. (2011). Goodbye, listwise deletion: Presenting hot deck imputation as an easy and effective tool for handling missing data. Communication Methods and Measures, 5(4), 297–310.
37.
NewmanD. A. (2014). Missing data: Five practical guidelines. Organizational Research Methods, 17(4), 372–411.
38.
Organisation for Economic Co-operation and Development. (2024). PISA 2022 technical report. PISA, OECD Publishing.
39.
OrlandoM.ThissenD. (2000). Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24(1), 50–64.
40.
OrlandoM.ThissenD. (2003). Further investigation of the performance of S—X2: An item fit index for use with dichotomous item response theory models. Applied Psychological Measurement, 27(4), 289–298.
41.
PepinskyT. B. (2018). A note on listwise deletion versus multiple imputation. Political Analysis, 26(4), 480–488.
42.
R Core Team. (2025). R: A language and environment for statistical computing (Version 4.5.1) [Computer software]. R Foundation for Statistical Computing. https://www.R-project.org
43.
ReiserM. (1996). Analysis of residuals for the multinomial item response model. Psychometrika, 61(3), 509–528.
44.
RubinD. B. (1976). Inference and missing data. Biometrika, 63(3), 581–592.
45.
RubinD. B. (1987). Multiple imputation for nonresponse in surveys. Wiley-Interscience.
46.
SamejimaF. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika, 34(S1), 1–97.
47.
SinharayS.MonroeS. (2025). Assessment of fit of item response theory models: A critical review of the status quo and some future directions. British Journal of Mathematical and Statistical Psychology.
48.
SriutaisukS.LiuY.ChungS.KimH.GuF. (2025). Evaluating imputation-based fit statistics in structural equation modeling with ordinal data: The MI2S approach. Educational and Psychological Measurement, 85(1), 82–113.
49.
ThissenD.PommerichM.BilleaudK.WilliamsV. S. L. (1995). Item response theory for scores on tests including polytomous items with ordered responses. Applied Psychological Measurement, 19(1), 39–49.
50.
van BuurenS.Groothuis-OudshoornK. (2011). Mice: Multivariate imputation by chained equations in R. Journal of Statistical Software, 45(3), 1–67.
51.
WaterburyG. T. (2019). Missing data and the Rasch model: The effects of missing data mechanisms on item parameter estimation. Journal of Applied Measurement, 20(2), 154–166.
52.
YuL.BuysseD. J.GermainA.MoulD. E.StoverA.DoddsN. E.JohnstonK. L.PilkonisP. A. (2012). Development of short forms from the PROMISTM sleep disturbance and sleep-related impairment item banks. Behavioral Sleep Medicine, 10(1), 6–24.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
