Collapsing categories is a commonly used data reduction technique; however, to date there do not exist principled methods to determine whether collapsing categories is appropriate in practice. With ordinal responses under the partial credit model, when collapsing categories, the true model for the collapsed data is no longer a partial credit model, and therefore refitting a partial credit model may result in model misspecification. This article details the implementation and performance of an information matrix test (IMT) to assess the implications of collapsing categories for a given data set under the partial credit model and compares its performance to the application of a nominal response model (NRM) and the S − X2 goodness-of-fit statistic. The IMT and NRM-based test are able to correctly determine the true number of categories for an item, given reasonable power through this goodness-of-fit test. We conclude by applying the test to a well-studied data set from the literature.
AbadA. A.LitièreS.MolenberghsG. (2010). Testing for misspecification in generalized linear mixed models. Biostatistics, 11, 771–786.
2.
AdamsR. J.WuM. L.WilsonM. (2012). The Rasch rating model and the disordered threshold controversy. Educational and Psychological Measurement, 72, 547–573.
3.
AndrichD. (1995). Models for measurement, precision, and the nondichotomization of graded responses. Psychometrika, 60, 7–26.
4.
BatesD.MächlerM.BolkerB.WalkerS. (2014). Fitting linear mixed-effects models using lme4. arXiv preprint arXiv:1406.5823.
5.
BenjaminiY.HochbergY. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological), 1, 289–300.
6.
BockR. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29–51.
7.
BondT.FoxC. (2007). Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.
8.
BrownR. (1991). The effect of collapsing ordered polytomous scales on parameter estimates in structural equation measurement models. Educational and Psychological Measurement, 51, 317–328.
9.
ChalmersR. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48, 1–29.
10.
ChalmersR. P. (2018). Numerical approximation of the observed information matrix with Oakes’ identity. British Journal of Mathematical and Statistical Psychology. Advance online publication. doi:10.1111/bmsp.12127
11.
ChengY.YuanK.-H. (2010). The impact of fallible item parameter estimates on latent trait recovery. Psychometrika, 75, 280–291.
12.
ChesherA. (1983). The information matrix test: Simplified calculation via a score test interpretation. Economics Letters, 13, 45–48.
13.
De BoeckP.WilsonM. (Eds.). (2004). A framework for item response models. In Explanatory item response models (pp. 3–41). New York: Springer.
14.
EklundM.ErlandssonL.-K.HagellP. (2012). Psychometric properties of a Swedish version of the Pearlin Mastery Scale in people with mental illness and healthy people. Nordic Journal of Psychiatry, 66, 380–388.
15.
ForjazM. J.Martinez-MartinP.DujardinK.MarshL.RichardI. H.StarksteinS. E.LeentjensA. F. (2013). Rasch analysis of anxiety scales in Parkinson’s disease. Journal of Psychosomatic Research, 74, 414–419.
16.
GrondinJ.BlaisJ.-G. (2010, 04-05). A Rasch analysis on collapsing categories in item’s response scales of survey questionnaire: Maybe it’s not one size fits all. Paper presented at the annual meeting of the American Educational Research Association, Denver, CO.
17.
HarelD. (2014). The effect of model misspecification for polytomous logistic adjacent category item response theory models(PhD thesis). Montreal, Quebec: McGill University.
18.
HemkerB. T.SijtsmaK.MolenaarI. W.JunkerB. W. (1996). Polytomous IRT models and monotone likelihood ratio of the total score. Psychometrika, 61, 679–693.
19.
HemkerB. T.SijtsmaK.MolenaarI. W.JunkerB. W. (1997). Stochastic ordering using the latent trait and the sum score in polytomous IRT models. Psychometrika, 62, 331–347.
20.
HendriksJ.FyfeS.StylesI.SkinnerS. R.MerrimanG. (2012). Scale construction utilising the Rasch unidimensional measurement model: A measurement of adolescent attitudes towards abortion. The Australasian Medical Journal, 5, 251.
21.
JacksonL. F.GrayA. G.FienbergS. E. (2008). Sequential category aggregation and partitioning approaches for multi-way contingency tables based on survey and census data. The Annals of Applied Statistics, 2, 955–981.
22.
JansenP. G.RoskamE. E. (1986). Latent trait models and dichotomization of graded responses. Psychometrika, 51, 69–91.
23.
KangT.ChenT. T. (2008). Performance of the generalized S-X2 item fit index for polytomous IRT models. Journal of Educational Measurement, 45, 391–406.
24.
LancasterT. (1984). The covariance matrix of the information matrix test. Econometrica: Journal of the Econometric Society, 52, 1051–1053.
25.
LecointeD. A. (1995, 4). How the collapsing of categories impacts the item information function in polytomous item response theory. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.
26.
MagisD.BélandS.TuerlinckxF.De BoeckP. (2010). A general framework and an R package for the detection of dichotomous differential item functioning. Behavior Research Methods, 42, 847–862.
27.
MastersG. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149–174.
28.
MeadeA. W.LautenschlagerG. J.HechtJ. E. (2005). Establishing measurement equivalence and invariance in longitudinal data with item response theory. International Journal of Testing, 5, 279–300.
29.
MuradH.FleischmanA.SadetzkiS.GeyerO.FreedmanL. S. (2003). Small samples and ordered logistic regression: Does it help to collapse categories of outcome?The American Statistician, 57, 155–160.
30.
MurakiE. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159–176.
31.
MurakiE. (1993). Information functions of the generalized partial credit model. Applied Psychological Measurement, 17, 351–363.
32.
MuthenB. (1984). A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika, 49, 115–132.
33.
OrlandoM.ThissenD. (2000). Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24, 50–64.
34.
OrmeC. (1990). The small-sample performance of the information-matrix test. Journal of Econometrics, 46, 309–331.
35.
PrestonK.ReiseS.CaiL.HaysR. D. (2011). Using the nominal response model to evaluate response category discrimination in the PROMIS emotional distress item pools. Educational and Psychological Measurement, 71, 523–550.
36.
R Development Core Team. (2011). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.
37.
RangerJ.KuhnJ.-T. (2012). Assessing fit of item response models using the information matrix test. Journal of Educational Measurement, 49, 247–268.
38.
RangerJ.KuhnJ.-T. (2015). Assessing person fit with the information matrix test. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 11, 3.
39.
RizopoulosD. (2006). ltm: An R package for latent variable modelling and item response theory analyses. Journal of Statistical Software, 17, 1–25.
40.
RoskamE. E. (1995). Graded responses and joining categories: A rejoinder to Andrich “models for measurement, precision, and nondichotomization of graded responses.”Psychometrika, 60, 27–35.
41.
SijtsmaK.HemkerB. T. (2000). A taxonomy of IRT models for ordering persons and items using simple sum scores. Journal of Educational and Behavioral Statistics, 25, 391–415.
42.
StrombergU. (1996). Collapsing ordered outcome categories: A note of concern. American Journal of Epidemiology, 144, 421–424.
43.
TeresiJ. A.Ocepek-WeliksonK.KleinmanM.EimickeJ. P.CraneP. K.JonesR. N.…CellaD. (2009). Analysis of differential item functioning in the depression item bank from the Patient Reported Outcome Measurement Information System (PROMIS): An item response theory approach. Psychology Science Quarterly, 51, 148.
44.
ThissenD.CaiL.BockR. D. (2010). The nominal categories item response model. In NeringM.OstiniR. (Eds.). Handbook of polytomous item response theory models (pp. 43–75). New York, NY: Routledge.
45.
van der ArkL. A. (2005). Stochastic ordering of the latent trait by the sum score under various polytomous IRT models. Psychometrika, 70, 283–304.
46.
van der ArkL. A.BergsmaW. P. (2010). A note on stochastic ordering of the latent trait using the sum of polytomous item scores. Psychometrika, 75, 272–279.
47.
VansteelandtK. (2000). Formal models for contextualized personality psychology (Unpublished doctoral dissertation). K. U. Leuven, Belgium.
48.
von DavierA. A.WilsonC. (2007). IRT true-score test equating: A guide through assumptions and applications. Educational and Psychological Measurement, 67, 940–957.
49.
WeissA. A. (1997). Specification tests in ordered logit and probit models. Econometric Reviews, 16, 361–391.
50.
WhiteH. (1982). Maximum likelihood estimation of misspecified models. Econometrica, 50, 1–25.
51.
WilsonM.MastersG. N. (1993). The partial credit model and null categories. Psychometrika, 58, 87–99.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
