Sage Journals: Discover world-class research

Abstract

To provide more insight into an individual’s response process and cognitive process, this study proposed three mixed sequential item response models (MS-IRMs) for mixed-format items consisting of a mixture of a multiple-choice item and an open-ended item that emphasize a sequential response process and are scored sequentially. Relative to existing polytomous models such as the graded response model (GRM), generalized partial credit model (GPCM), or traditional sequential Rasch model (SRM), the proposed models employ an appropriate processing function for each task to improve conventional polytomous models. Simulation studies were carried out to investigate the performance of the proposed models, and the results indicated that all proposed models outperformed the SRM, GRM, and GPCM in terms of parameter recovery and model fit. An application illustration of the MS-IRMs in comparison with traditional models was demonstrated by using real data from TIMSS 2007.

Keywords

sequential response item response theory item information

Get full access to this article

View all access options for this article.

References

Agresti

(2013). Categorical data analysis. Wiley.

Bolt

D. M.

Wollack

J. A.

Suh

(2012). Application of a multidimensional nested logit model to multiple-choice test items. Psychometrika, 77(2), 339–357. https://doi.org/10.1007/s11336-012-9257-5

Brooks

S. P.

Gelman

(1998). General methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics, 7(4), 434–455. https://doi.org/10.2307/1390675

Darrell Bock

(1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37(1), 29–51. https://doi.org/10.1007/BF02291411

Deng

Bolt

D. M.

(2016). A sequential IRT model for multiple-choice items and a multidimensional extension. Applied Psychological Measurement, 40(4), 243–257. https://doi.org/10.1177/0146621616631518

Gelman

Carlin

J. B.

Stern

H. S.

Dunson

D. B.

Vehtari

Rubin

D. B.

(2014a). Bayesian data analysis (3rd edn). Taylor and Francis.

Gelman

Hwang

Vehtari

(2014b). Understanding predictive information criteria for Bayesian models. Statistics and Computing, 24(6), 997–1016. https://doi.org/10.1007/s11222-013-9416-2

Gelman

Lee

Guo

(2015). Stan: A probabilistic programming language for bayesian inference and optimization. Journal of Educational and Behavioral Statistics, 40(5), 530–543. https://doi.org/10.3102/1076998615606113

Gilks

W. R.

Richardson

Spiegelhalter

D. J.

(1996). Markov chain Monte Carlo in Practice. Chapman and Hall/CRC.

10.

Jeon

De Boeck

(2016). A generalized item response tree model for psychological assessments. Behavior Research Methods, 48(3), 1070–1085. https://doi.org/10.3758/s13428-015-0631-y

11.

Lee

Kim

S. Y.

Choi

Kang

(2019). IRT approaches to modeling scores on mixed format tests. Journal of Educational Measurement, 57(2), 230–254. https://doi.org/10.1111/jedm.12248

12.

de la Torre

(2016). A sequential cognitive diagnosis model for polytomous responses. The British Journal of Mathematical and Statistical Psychology, 69(3), 253–275. https://doi.org/10.1111/bmsp.12070

13.

Martín-Fernández

Revuelta

(2017). Bayesian estimation of multidimensional item response models: A comparison of analytic and simulation algorithms. Psicológica, 38, 25–55.

14.

Masters

G. N.

(1982). A Rasch model for partial credit scoring. Psychometrika, 47(2), 149–174. https://doi.org/10.1007/BF02296272

15.

Muraki

(1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16(2), 159–176. https://doi.org/10.1177/014662169201600206

16.

Patz

R. J.

Junker

B. W.

(1999). Applications and extensions of MCMC in IRT: Multiple item types, missing data, and rated responses. Journal of Educational and Behavioral Statistics, 24(4), 342–366. https://doi.org/10.2307/1165367

17.

Plummer

(2015). JAGS Version 4.0.0 user manual.

18.

Roediger

H. L.

III Marsh

E. J.

(2005). The positive and negative consequences of multiple-choice testing. Journal of Experimental Psychology. Learning, Memory, and Cognition, 31(5), 1155–1159. https://doi.org/10.1037/0278-7393.31.5.1155

19.

Samejima

(1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika, 34(S1), 1–97. https://doi.org/10.1007/bf03372160

20.

Spiegelhalter

D. J.

Best

N. G.

Carlin

B. P.

van der Linde

(2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 64(4), 583–639. https://doi.org/10.1111/1467-9868.00353

21.

Y. S.

Yajima

(2015). R2jags: Using R to run “JAGS” (R package version 0.5-7).

22.

Suh

Bolt

D. M.

(2010). Nested logit models for multiple-choice item response data. Psychometrika, 75(3), 454–473. https://doi.org/10.1007/s11336-010-9163-7

23.

Tutz

(1990). Sequential item response models with an ordered response. British Journal of Mathematical and Statistical Psychology, 43(1), 39–55. https://doi.org/10.1111/j.2044-8317.1990.tb00925.x

24.

Tutz

(1997). Sequential models for ordered responses. In van der Linden

W. J.

Hambleton

R. K.

(Eds), Handbook of modern item response theory (pp. 139–152). Springer-Verlag. https://doi.org/10.1007/978-1-4757-2691-6

25.

Vehtari

Gelman

Gabry

(2016a). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC (arXiv preprint arXiv:1507.04544). https://arxiv.org/abs/1507.04544

26.

Wainer

Thissen

(1993). Combining multiple-choice and constructed-response test scores: Toward a Marxist theory of test construction. Applied Measurement in Education, 6, 103–118. https://doi.org/10.1207/s15324818ame0602_1

27.

Watanabe

(2010). Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory. Journal of Machine Learning Research, 11, 3571–3594.

28.

Watanabe

(2013). A widely applicable Bayesian information criterion. Journal of Machine Learning Research, 14(1), 867–897.

29.

Whittaker

T. A.

Chang

Dodd

B. G.

(2012). The performance of IRT model selection methods with mixed-format tests. Applied Psychological Measurement, 36(3), 159–180. https://doi.org/10.1177/014662161244030

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.61 MB

A Mixed Sequential IRT Model for Mixed-Format Items

Abstract

Keywords

Get full access to this article

References

Supplementary Material