Collateral information has been used to address subpopulation heterogeneity and increase estimation accuracy in some large-scale cognitive assessments. The methodology that takes collateral information into account has not been developed and explored in published research with models designed specifically for noncognitive measurement. Because the accurate noncognitive measurement is becoming increasingly important, we sought to examine the benefits of using collateral information in latent trait estimation with an item response theory model that has proven valuable for noncognitive testing, namely, the generalized graded unfolding model (GGUM). Our presentation introduces an extension of the GGUM that incorporates collateral information, henceforth called Explanatory GGUM. We then present a simulation study that examined Explanatory GGUM latent trait estimation as a function of sample size, test length, number of background covariates, and correlation between the covariates and the latent trait. Results indicated the Explanatory GGUM approach provides scoring accuracy and precision superior to traditional expected a posteriori (EAP) and full Bayesian (FB) methods. Implications and recommendations are discussed.
AdamsR. J.WilsonM.WuM. (1997). Multilevel item response models: An approach to errors in variables regression. Journal of Educational and Behavioral Statistics, 22, 47–76.
2.
AndrichD. (1988). The application of an unfolding model of the PIRT type to the measurement of attitude. Applied Psychological Measurement, 12, 33–51.
3.
AndrichD.StylesI. M. (1998). The structural relationship between attitude and behavior statements from the unfolding perspective. Psychological Methods, 3, 454–469.
4.
BirnbaumA (1968). Some latent trait models and their use in inferring an examinee’s ability. In LordF.M.NovickM.R.. Statistical theories of mental test scores: Addison-Wesley.
5.
BolsinovaM.TijmstraJ. (2018). Improving precision of ability estimation: Getting more from response times. British Journal of Mathematical and Statistical Psychology, 71, 13–38.
6.
BormanW. C.BuckD. E.HansonM. A.MotowidloS. J.StarkS.DrasgowF. (2001). An examination of the comparative reliability, validity, and accuracy of performance ratings made using computerized adaptive rating scales. Journal of Applied Psychology, 86, 965–973.
7.
BriggsD. C. (2008). Using explanatory item response models to analyze group differences in science achievement. Applied Measurement in Education, 21, 89–118.
8.
CarterN. T.ZickarM. J. (2011). A comparison of the LR and DFIT frameworks of differential functioning applied to the generalized graded unfolding model. Applied Psychological Measurement, 35, 623–642.
9.
ChernyshenkoO. S.StarkS.DrasgowF.RobertsB. W. (2007). Constructing personality scales under the assumptions of an ideal point response process: Toward increasing the flexibility of personality measures. Psychological Assessment, 19, 88–106.
10.
CohenJ. (1992). Statistical power analysis, Current Directions in Psychological Science, 1, 98–101.
11.
De BoeckP.ChoS. J.WilsonM. (2011). Explanatory secondary dimension modeling of latent differential item functioning. Applied Psychological Measurement, 35, 583–603.
12.
de la TorreJ. (2009). Improving the quality of ability estimates through multidimensional scoring and incorporation of ancillary variables. Applied Psychological Measurement, 33, 465–485.
13.
de la TorreJ.StarkS.ChernyshenkoO. S. (2006). Markov chain Monte Carlo estimation of item parameters for the generalized graded unfolding model. Applied Psychological Measurement, 30, 216–232.
14.
FerrandoP. J.Lorenzo-SevaU. (2007). An item response theory model for incorporating response time data in binary personality items. Applied Psychological Measurement, 31, 525–543.
15.
GelmanA.RubinD. B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7, 457–472.
16.
JooS. H.ChunS.StarkS.ChernyshenkoO. S. (2019). Item parameter estimation with the general hyperbolic cosine ideal point IRT model. Applied Psychological Measurement, 43, 18–33.
17.
JooS. H.LeeP.StarkS. (2017). Evaluating anchor-item designs for concurrent calibration with the GGUM. Applied Psychological Measurement, 41, 83–96.
18.
KubingerK. D. (2009). Applications of the linear logistic test model in psychometric research. Educational and Psychological Measurement, 69, 232–244.
19.
MislevyR. J. (1986). Bayes modal estimation in item response models. Psychometrika, 51, 177–195.
20.
MislevyR. J.BeatonA. E.KaplanB.SheehanK. M. (1992). Estimating population characteristics from sparse matrix samples of item responses. Journal of Educational Measurement, 29, 133–161.
21.
MislevyR. J.JohnsonE. G.MurakiE. (1992). Chapter 3: Scaling procedures in NAEP. Journal of Educational Statistics, 17, 131–154.
22.
MislevyR. J.SheehanK. M. (1989). The role of collateral information about examinees in item parameter estimation. Psychometrika, 54, 661–679.
23.
ParkY. S.XingK.LeeY. S. (2018). Explanatory cognitive diagnostic models: Incorporating latent and observed predictors. Applied Psychological Measurement, 42, 376–392.
24.
PoinstinglH. (2009). The Linear Logistic Test Model (LLTM) as the methodological foundation of item generating rules for a new verbal reasoning test. Psychological Test and Assessment Modeling, 51, 123–134.
25.
RandallJ.CheongY. F.EngelhardG. (2011). Using explanatory item response theory modeling to investigate context effects of differential item functioning for students with disabilities. Educational and Psychological Measurement, 71, 129–147.
26.
RangerJ.KuhnJ. T. (2012). Improving item response theory model calibration by considering response times in psychological tests. Applied Psychological Measurement, 36, 214–231.
27.
RobertsJ. S.DonoghueJ. R.LaughlinJ. E. (2000). A general item response theory model for unfolding unidimensional polytomous responses. Applied Psychological Measurement, 24, 3–32.
28.
RobertsJ. S.DonoghueJ. R.LaughlinJ. E. (2002). Characteristics of MML/EAP parameter estimates in the generalized graded unfolding model. Applied Psychological Measurement, 26, 192–207.
29.
RobertsJ. S.LaughlinJ. E.WedellD. H. (1999). Validity issues in the Likert and Thurstone approaches to attitude measurement. Educational and Psychological Measurement, 59, 211–233.
30.
SamejimaF. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometric Monograph, 34, 100.
31.
StarkS.ChernyshenkoO. S.DrasgowF.WilliamsB. A. (2006). Examining assumptions about item responding in personality assessment: Should ideal point methods be considered for scale development and scoring?Journal of Applied Psychology, 91, 25–39.
32.
TayL.DrasgowF.RoundsJ.WilliamsB. A. (2009). Fitting measurement models to vocational interest data: Are dominance models ideal?Journal of Applied Psychology, 94, 1287–1304.
33.
UsamiS. (2011). Generalized graded unfolding model with structural equation for subject parameters. Japanese Psychological Research, 53, 221–232.
34.
van den NoortgateW.De BoeckP. (2005). Assessing and explaining differential item functioning using logistic mixed models. Journal of Educational and Behavioral Statistics, 30, 443–464.
35.
van der LindenW. J. (2007). A hierarchical framework for modeling speed and accuracy on test items. Psychometrika, 72, 287–308.
36.
von DavierM.GonzalezE.MislevyR. (2009). What are plausible values and why are they useful. IERI Monograph Series, 2, 9–36.
37.
von DavierM.SinharayS. (2013). Analytics in international large-scale assessments: Item response theory and population models. In RutkowskiL.von DavierM.RutkowskiD. (Eds.), Handbook of international large-scale assessment (pp. 155–274): CRC Press.
38.
von DavierM.SinharayS.OranjeA.BeatonA. (2006). The statistical procedures used in National Assessment of Educational Progress: Recent developments and future directions. In RaoC. R.SinharayS. (Eds.), Handbook of statistics: Vol. 26. Psychometrics (pp. 1039–1055). Elsevier.
39.
WangW. C.WuS. L. (2016). Confirmatory multidimensional IRT unfolding models for graded-response items. Applied Psychological Measurement, 40, 56–72.
40.
WilsonM.De BoeckP. (2004). Descriptive and explanatory item response models. In explanatory item response models (pp. 43–74). Springer.
41.
WilsonM.ZhengX.McGuireL. (2012). Formulating latent growth using an explanatory item response model approach. Journal of Applied Measurement, 13, 1–22.
42.
ZengL. (1997). Implementation of marginal Bayesian estimation with four-parameter beta prior distributions. Applied Psychological Measurement, 21, 143–156.
