Person-fit statistics are widely used to detect aberrant response patterns in educational and psychological measurement. Snijders suggested an asymptotically correct standardization for a broad class of such statistics. This paper presents an alternative derivation of this standardization. The derivation yields several advantages including a simpler formula and simpler description of several person-fit statistics including the statistic and a theoretical explanation of simulation findings reported by Snijders and van Krimpen-Stoop and Meijer, among others.
AlbersC. J.MeijerR. R.TendeiroJ. N. (2016). Derivation and applicability of asymptotic results for multiple subtests person-fit statistics. Applied Psychological Measurement, 40(4), 274–288. https://doi.org/10.1177/0146621615622832
2.
BockR. D.MislevyR. J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6, 431–444.
3.
BussW. G.NovickM. R. (1980). The detection of cheating on standardized tests: Statistical and legal analysis. The Journal of Law and Education, 9(1), 1–64.
4.
ChangH. H.StoutW. (1993). The asymptotic posterior normality of the latent trait in an IRT model. Psychometrika, 58, 37–52.
5.
de AyalaR. J. (2022). The theory and practice of item response theory (2nd ed.). Guilford Press.
6.
DrasgowF.LevineM. V.WilliamsE. A. (1985). Appropriateness measurement with polychotomous item response models and standardized indices. British Journal of Mathematical and Statistical Psychology, 38, 67–86. https://doi.org/10.1111/j.2044-8317.1985.tb00817.x
7.
FerrandoP. J. (2014). A general approach for assessing person fit and person unreliability in typical-response measurement. Applied Psychological Measurement, 38, 166–183. https://doi.org/https://doi.org/10.1177/0146621613497532
8.
FerrandoP. J.Vigil-ColetA.Lorenzo-SevaU. (2016). Practical person-fit assessment with the linear fa model: New developments and a comparative study. Frontiers in Psychology, 7, Article 1973. https://doi.org/10.3389/fpsyg.2016.01973
HongM. R.ChengY. (2019). Robust maximum marginal likelihood (rmml) estimation for item response theory models. Behavior Research Methods, 51(2), 573–588. https://doi.org/10.3758/s13428-018-1150-4
13.
HongM. R.LinL.ChengA. (2021). Asymptotically corrected person fit statistics for multidimensional constructs with simple structure and mixed item types. Psychometrika, 86, 464–488. https://doi.org/10.31234/osf.io/6wv8p
14.
KimJ. K.NicewanderW. A. (1993). Ability estimation for conventional tests. Psychometrika, 58, 587–599.
15.
KlauerK. C. (1995). The assessment of person fit. In FischerG. H.MolenaarI. W. (Eds.), Rasch models: Foundations, recent developments, and applications (pp. 97–110). Springer Verlag.
16.
LinZ.JiangT.RijmenF.Van WamelenP. (2024). Asymptotically correct person fit z-statistics for the Rasch testlet model. Psychometrika, 89(4), 1230–1260. https://doi.org/10.1007/s11336-024-09997-y
17.
LordF. M. (1983). Unbiased estimators of ability parameters, of their variance, and of their parallel-forms reliability. Psychometrika, 48, 233–245.
18.
MagisD. (2014). On the asymptotic standard error of a class of robust estimators of ability in dichotomous item response models. British Journal of Mathematical and Statistical Psychology, 67, 430–450.
19.
MagisD. (2016). Efficient standard error formulas of ability estimators with dichotomous item response models. Psychometrika, 81, 184–200. https://doi.org/10.1007/s11336-015-9443-3
20.
MagisD.BelandS.RaicheG. (2014). Snijders’s correction of infit and outfit indexes with estimated ability level: An analysis with the Rasch model. Journal of Applied Measurement, 15, 82–93.
21.
MeijerR. R.EgberinkI. J.EmonsW. H.SijtsmaK. (2008). Detection and validation of unscalable item score patterns using item response theory: An illustration with Harter’s self-perception profile for children. Journal of Personality Assessment, 90, 227–238. https://doi.org/10.1080/00223890701884921
SchusterC.YuanK. H. (2011). Robust estimation of latent ability in item response models. Journal of Educational and Behavioral Statistics, 36, 720–735.
27.
SerflingR. J. (1980). Approximation theorems of mathematical statistics. Wiley.
28.
SinharayS. (2015). The asymptotic distribution of Ability estimates: Beyond dichotomous items and unidimensional IRT models. Journal of Educational and Behavioral Statistics, 40, 511–528. https://doi.org/10.3102/1076998615606115
SinharayS. (2016c). Person fit analysis in computerized adaptive testing using tests for a change point. Journal of Educational and Behavioral Statistics, 41, 521–549. https://doi.org/10.3102/1076998616658331
32.
SinharayS. (2016d). The choice of the ability estimate with asymptotically correct standardized person-fit statistics. British Journal of Mathematical and Statistical Psychology, 69, 175–193. https://doi.org/10.1111/bmsp.12067
33.
SinharayS.JensenJ. L. (2019). Higher-order asymptotics and its application to testing the equality of the examinee ability over two sets of items. Psychometrika, 84(2), 484–510. https://doi.org/10.1007/s11336-018-9627-8
34.
SnijdersT. (2001). Asymptotic null distribution of person-fit statistics with estimated person parameter. Psychometrika, 66, 331–342. https://doi.org/10.1007/bf02294437
TendeiroJ. N. (2016). The person-fit statistic in an unfolding model context. Applied Psychological Measurement, 41, 44–59. https://doi.org/10.1177/0146621616669336
37.
TendeiroJ. N.MeijerR. R. (2014). Detection of invalid test scores: The usefulness of simple nonparametric statistics. Journal of Educational Measurement, 51, 239–259. https://doi.org/10.1111/jedm.12046
38.
TendeiroJ. N.MeijerR. R.NiessenA. S. M. (2016). PerFit: An R package for person-fit analysis in IRT. Journal of Statistical Software, 74(5), 1–27. https://doi.org/10.18637/jss.v074.i05
39.
van Krimpen-StoopE. M. L. A.MeijerR. R. (1999). The null distribution of person-fit statistics for conventional and adaptive tests. Applied Psychological Measurement, 23, 327–345. https://doi.org/10.1177/01466219922031446
40.
WarmT. A. (1989). Weighted likelihood estimation of ability in item response theory. Psychometrika, 54, 427–450. https://doi.org/10.1007/bf02294627
41.
WeinsteinM. J. (2016). When numbers are not enough. In CizekG. J.WollackJ. A. (Eds.), Handbook of quantitative methods for detecting cheating on tests (pp. 358–369). Routledge. https://doi.org/10.4324/9781315743097-19
42.
WollackJ. A.CohenA. S.EckerlyC. A. (2015). Detecting test tampering using item response theory. Educational and Psychological Measurement, 75, 931–953. https://doi.org/10.1177/0013164414568716
43.
WollackJ. A.SchoenigR. W. (2018). Cheating. In FreyB. B. (Ed.), The SAGE encyclopedia of educational research, measurement, and evaluation (pp. 260–265). Sage.
WrightB. D.StoneM. H. (1979). Best test design. Mesa Press.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
