In constant-θ IRT models, person unreliability is regarded as a source of person misfit. In Variable-θ IRT models, on the other hand, it is regarded as a relevant individual characteristic, which is modeled as an additional person parameter. As in any IRT application, person fit must be assessed when a variable-θ model is fitted to real data. This assessment aims to detect inconsistent response patterns once person unreliability has been taken into account. The present paper makes a general proposal for assessing person fit based on variable-θ models that are intended to be used with personality and attitude measures. For the three types of variable-θ IRT models developed to date—binary, graded, and (approximately) continuous response models—(a) graphical procedures, (b) global person-fit indices, and (c) residual indices at the item level are proposed and discussed. A multistage approach for using the indices and procedures is also proposed and illustrated using an empirical example.
AndrichD. (1978). Relationships between the Thurstone and Rasch approaches to item scaling. Applied Psychological Measurement, 2, 449-460.
2.
BellR. C. (1982). Person fit and person reliability. Educational Research & Perspectives, 9, 105-113.
3.
ChangH.MazzeoJ. (1994). The unique correspondence of the item response function and item category response function in polytomously scored item response models. Psychometrika, 59, 391-404.
4.
CoombsC. H. (1948). A rationale for the measurement of traits in individuals. Psychometrika, 13, 59-68.
5.
CoombsC. H. (1964). A theory of data. New York, NY: Wiley.
6.
DrasgowF.LevineM. V.WilliamsE. A. (1985). Appropriateness measurement with polychotomous item response models and standardized indices. British Journal of Mathematical and Statistical Psychology, 38, 67-86.
7.
DouglasJ.CohenA. (2001). Nonparametric item response function estimation for assessing parametric model fit. Applied Psychological Measurement, 25, 234-243.
8.
EmonsW. H. M.SijtsmaK.MeijerR. R. (2004). Testing hypotheses about the "person-response" function in "person-fit" analysis. Multivariate Behavioral Research, 39, 1-35.
9.
EmonsW. H. M.SijtsmaK.MeijerR. R. (2005). Global, local and graphical person-fit analysis using person-response functions. Psychological Methods, 10, 101-119.
10.
FerrandoP. J. (2004). Person reliability in personality measurement: An item response theory analysis. Applied Psychological Measurement, 28, 126-140.
11.
FerrandoP. J. (2007). A Pearson-type-VII item response model for assessing person fluctuation. Psychometrika, 72, 25-41.
12.
FerrandoP. J. (2009). A graded response model for measuring person reliability. British Journal of Mathematical and Statistical Psychology, 62, 641-662.
13.
FerrandoP. J. (2010). Some statistics for assessing person-fit based on continuous-response models. Applied Psychological Measurement, 34, 219-237.
14.
FerrandoP. J. (2011). A linear variable-theta model for measuring individual fluctuation. Applied Psychological Measurement, 35, 200-216.
HernándezJ. M.RubioV. J.RevueltaJ.SantacreuJ. (2006). A procedure for estimating intra-subject consistency of behavior. Educational and Psychological Measurement, 66, 417-434.
18.
KarabatsosG. (2000). A critique of Rasch residual fit statistics. Journal of Applied Measurement, 1, 152-176.
19.
LevineM. V.DrasgowF. (1983). Appropriateness measurement: Validating studies and variable ability models. In WeissD. J. (Ed.), New horizons in testing (pp. 109-131). New York, NY: Academic press.
20.
LevineM. V.RubinD. B. (1979). Measuring the appropriateness of multiple choice test scores. Journal of Educational Statistics, 4, 269-290.
21.
LumsdenJ. (1977). Person reliability. Applied Psychological Measurement, 1, 477-482.
22.
LumsdenJ. (1978). Tests are perfectly reliable. British Journal of Mathematical and Statistical Psychology, 31, 19-26.
23.
LumsdenJ. (1980). Variations on a theme by Thurstone. Applied Psychological Measurement, 4, 1-7.
24.
MeijerR. R.EgberinkI. J. L.EmonsW. H. M.SijtsmaK. (2008). Detection and validation of unscalable item score patterns using item response theory: An illustration with Harter’s self-perception profile for children. Journal of Personality Assessment, 90, 1-14.
25.
MislevyR. J. (1984). Estimating latent distributions. Psychometrika, 49, 359-381.
26.
MosierC. I. (1942). Psychophysics and mental test theory II: The constant process. Psychological Review, 48, 235-249.
27.
PaunonenS. V. (1988). Trait relevance and the differential predictability of behavior. Journal of Personality, 56, 599-619.
28.
ReiseS. P.FlanneryW. P. (1996). Assessing person-fit on measures of typical performance. Applied Measurement in Education, 9, 9-26.
29.
ReiseS. P.WallerN. G. (1993). Traitedness and the assessment of response pattern scalability. Journal of Personality and Social Psychology, 65, 143-151.
30.
ReiseS. P.WallerN. G. (2009). Item response theory and clinical psychology. Annual Review of Clinical Psychology, 5, 27-48.
31.
SamejimaF. (1969). Estimation of latent ability using a response pattern of graded scores (Psychometrika Monograph No. 17). Iowa City, Iowa: Psychometric Society.
32.
SiemF. M. (1998). Metatraits and self-schemata: Same or different?Journal of Personality, 66, 783-803.
33.
SmithR. M. (1990). Theory and practice of fit. Rasch Measurement Transactions, 3, 78-80.
34.
SueiroM. J.AbadF. J. (2011). Assessing goodness of fit in item response theory with nonparametric models: A comparison of posterior probabilities and kernel smoothing approaches. Educational and Psychological Measurement, 71, 834-848.
35.
TrabinT. E.WeissD. J. (1983). The person response curve fit of individuals to item response theory models. In WeissD. J. (Ed.), New horizons in testing (pp. 83-108). New York, NY: Academic Press.
36.
van der MaasH. L. J.MolenaarD.MarisG.KievitR. A.BorsboomD. (2011). Cognitive psychology meets psychometric theory: On the relation between process models for decision making and latent variable models for individual differences. Psychological Review, 118, 339-356.
37.
van Krimpen-StoopE. M. L. A.MeijerR. R. (2002). Detection of person misfit in computerized adaptive tests with polytomous items. Applied Psychological Measurement, 26, 164-180.
38.
WallerN. G.ReiseS. P. (1992). Genetic and environmental influences on item response pattern scalability. Behavior Genetics, 22, 135-152.
39.
WrightB. D. (1977). Solving measurement problems with the Rasch model. Journal of Educational Measurement, 14, 97-116.
