This article proposes a general parametric item response theory approach for identifying sources of misfit in response patterns that have been classified as potentially inconsistent by a global person-fit index. The approach, which is based on the weighted least squared regression of the observed responses on the model-expected responses, can be used with a variety of unidimensional and multidimensional models intended for binary, graded, and continuous responses and consists of procedures for identifying (a) general deviation trends, (b) local inconsistencies, and (c) single response inconsistencies. A free program called REG-PERFIT that implements most of the proposed techniques has been developed, described, and made available for interested researchers. Finally, the functioning and usefulness of the proposed procedures is illustrated with an empirical study based on a statistics-anxiety scale.
ArmstrongR. D.StoumbosZ. G.KungM. T.ShiM. (2007). On the performance of lz statistic in person fit measurement. Practical Assessment, Research & Evaluation, 12. Retrieved from http://pareonline.net/getvn.asp?v=12&n=16
2.
AzzaliniA.BowmanA. W.HärdleW. (1989). On the use of nonparametric regression for model checking. Biometrika, 76, 1-11.
3.
ChangH.MazzeoJ. (1994). The unique correspondence of the item response function and item category response function in polytomously scored item response models. Psychometrika, 59, 391-404.
4.
CochranW. G. (1963). Sampling techniques. New York, NY: Wiley.
5.
ConijnJ. M.EmonsW. H. M.De JongK.SitjsmaK. (2015). Detecting and explaining aberrant responding on the Outcome Questionnaire-45. Assessment. Advance online publication. doi:10.1177/1073191114560882
6.
ConijnJ. M.EmonsW. H. M.SijtsmaK. (2014). Statistic lz-based person-fit methods for noncognitive multiscale measures. Applied Psychological Measurement, 38, 122-136.
7.
ConradK. J.BezruczkoN.ChanY.RileyB.DiamondG.DennisM. L. (2010). Screening for atypical suicide risk with person fit statistics among people presenting to alcohol and other drug treatment. Drug and Alcohol Dependence, 106, 92-100.
8.
CronbachL. J. (1950). Further evidence on response sets and test design. Educational and Psychological Measurement, 10, 3-31.
9.
DodeenH.DarabiM. (2009). Person-fit: Relationship with four personality tests in mathematics. Research Papers in Education, 24, 115-126.
10.
DraperN.SmithH. (1966). Applied regression analysis. New York, NY: Wiley.
11.
EgberinkI. J. L.MeijerR. R. (2011). An item response theory analysis of Harter’s self-perception profile for children or why strong clinical scales should be distrusted. Assessment, 18, 201-212.
12.
EmonsW. H. M.SijtsmaK.MeijerR. R. (2004). Testing hypotheses about the person-response function in person-fit analysis. Multivariate Behavioral Research, 39, 1-35.
13.
EmonsW. H. M.SijtsmaK.MeijerR. R. (2005). Global, local and graphical person-fit analysis using person-response functions. Psychological Methods, 10, 101-119.
14.
FerrandoP. J. (2012). Assessing inconsistent responding in E and N measures: An application of person-fit analysis in personality. Personality and Individual Differences, 52, 718-722.
15.
FerrandoP. J. (2013). A factor-analytic model for assessing individual differences in response scale usage. Multivariate Behavioral Research, 49, 390-405.
16.
FerrandoP. J. (2015). Assessing person fit in typical-response measures. In ReiseS. P.RevickiD. A. (Eds.), Handbook of item response theory modeling: Applications to typical performance assessment (pp. 128-155). New York, NY: Routledge.
17.
GoldbergerA. S. (1964). Econometric theory. New York, NY: Wiley.
KarabatsosG. (2000). A critique of Rasch residual fit statistics. Journal of Applied Measurement, 1, 152-176.
20.
LordF. M.NovickM. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
21.
MeijerR. R.EgberinkI. J. L.EmonsW. H. M.SijtsmaK. (2008). Detection and validation of unscalable item score patterns using item response theory: An illustration with Harter’s self-perception profile for children. Journal of Personality Assessment, 90, 1-14.
22.
MeijerR. R.SijtsmaK. (2001). Methodology review: Evaluating person fit. Applied Psychological Measurement, 25, 107-135.
23.
OnwuegbuzieA. J.DaleyC. E. (1999). Perfectionism and statistics anxiety. Personality and Individual Differences, 26, 1089-1102.
24.
ParsonsCh. K. (1983).The identification of people for whom job descriptive index scores are inappropriate. Organizational Behavior and Human Performance, 31, 365-393.
25.
PeabodyD. (1962). Two components in bipolar scales: Direction and extremeness. Psychological Review, 69, 65-73.
26.
ReiseS. P.FlanneryW. P. (1996). Assessing person-fit on measures of typical performance. Applied Measurement in Education, 9, 9-26.
27.
ReiseS. P.RevickiD. A. (2015). Handbook of item response theory modeling: Applications to typical performance assessment. New York, NY: Routledge.
28.
SamejimaF. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement, 34(4, Pt 2, Monograph No. 17).
29.
SijtsmaK.MeijerR. R. (2001). The Person Response Function as a tool in person-fit research. Psychometrika, 66, 191-207.
30.
SmithR. M. (1986). Person fit in the Rasch model. Educational and Psychological Measurement, 46, 359-372.
31.
SmithR. M. (1990). Theory and practice of fit. Rasch Measurement Transactions, 3, 78-80.
32.
TendeiroJ. N.MeijerR. R. (2014). Detection of invalid test scores: The usefulness of simple nonparametric statistics. Journal of Educational Measurement, 51, 239-259.
33.
Vigil-ColetA.Lorenzo-SevaU.CondonL. (2008). Development and validation of the statistical anxiety scale. Psicothema, 20, 174-180.
34.
WrightB. D.LinacreJ. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8, 370.
35.
ZimowskiM.MurakiE.MislevyR. J.BockR. D. (2003). BILOG-MG 3: Item analysis and test scoring with binary logistic models. Chicago, IL: Scientific Software.
