Sage Journals: Discover world-class research

Abstract

This article presents a multilevel longitudinal nested logit model for analyzing correct response and error types in multilevel longitudinal intervention data collected under a pretest–posttest, cluster randomized trial design. The use of the model is illustrated with a real data analysis, including a model comparison study regarding model complexity and cluster bias. Two substantive research questions regarding the intervention effect on correct response probability and error patterns are investigated using the proposed model. The recovery of item parameters for the proposed model using two sample size conditions is examined via a simulation study. The accuracy of the parameter estimates is comparable with those found in previous studies for the same family of models, except for the intercept parameters of correct responses. Finally, the impact of ignoring cluster membership in the model on the parameter estimation is also studied by fitting a single-level model to multilevel data. Ignoring cluster membership in the model adversely affects the estimation of intercept parameters in correct and error responses.

Keywords

multilevel longitudinal model nested logit model Bayesian data analysis error analysis

Get full access to this article

View all access options for this article.

References

Andersen

E. B.

(1985). Estimating latent correlations between repeated testings. Psychometrika, 50, 3-16.

Baker

F. B.

Kim

S.-H.

(2004). Item response theory: Parameter estimation techniques (2nd ed.). New York, NY: Dekker.

Bock

R. D.

(1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29-51.

Bolt

D. M.

Wollack

J. A.

Suh

(2012). Application of a multidimensional nested logit model to multiple-choice test items. Psychometrika, 77, 339-357.

Bottge

B. A.

Gassaway

Butler

Toland

M. D.

(2014). Detecting and correcting fractions computation error patterns. Exceptional Children, 80, 236-254.

Bottge

B. A.

Gassaway

Toland

M. D.

Butler

Cho

S.-J.

(2014). Effects of blended instructional models on math performance. Exceptional Children, 80, 423-437.

Cai

(2010). A two-tier full-information item factor analysis model with applications. Psychometrika, 75, 581-612.

Charalambous

C. Y.

Pitta-Pantazi

(2007). Drawing on a theoretical model to study students’ understandings of fractions. Educational Studies in Mathematics, 64, 293-316. doi:10.1007/ s10649-006-9036-2

Cho

S.-J.

Bottge

B. A.

(2015). Multilevel multidimensional item response model with a multilevel latent covariate. British Journal of Mathematical and Statistical Psychology, 68, 410-433.

10.

Clarke

D. M.

Roche

(2009). Students’ fraction comparison strategies as a window into robust understanding and possible pointers for instruction. Educational Studies in Mathematics, 72, 127-138. doi:10.1007/s10649-009-9198-9

11.

Cole

D. A.

Preacher

K. J.

(2014). Manifest variable path analysis: Potentially serious and misleading consequences due to uncorrected measurement error. Psychological Methods, 19, 300-315.

12.

De Boeck

Wilson

. (2004). Explanatory item response models: A generalized linear and nonlinear approach. New York, NY: Springer.

13.

Embretson

S. E.

(1991). A multidimensional latent trait model for measuring learning and change. Psychometrika, 56, 495-515.

14.

Forero

C. G.

Maydeu-Olivares

(2009). Estimation of IRT graded response models: Limited versus full information methods. Psychological Methods, 14, 275-299.

15.

Fox

J.-P.

(2010). Bayesian item response modeling. New York, NY: Springer.

16.

Fox

J.-P.

Glas

C. A. W.

(2001). Bayesian estimation of a multilevel IRT model using Gibbs sampling. Psychometrika, 66, 271-288.

17.

Fox

J.-P.

Glas

C. A. W.

(2003). Bayesian modeling of measurement error in predictor variables using item response theory. Psychometrika, 68, 169-191.

18.

Fox

J.-P.

Glas

C. A. W.

(2016). Multilevel item response models with covariates and multiple groups. In van der Linden

W. J.

(Ed.), Handbook of item response theory, models, statistical tools, and applications (Vol. 1., pp. 407-420). Boca Raton, FL: Chapman & Hall.

19.

Gelman

Rubin

D. B.

(1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7, 457-472.

20.

Jak

Oort

F. J.

Dolan

C. V.

(2013). A test for cluster bias: Detecting violations of measurement invariance across clusters in multilevel data. Structural Equation Modeling: A Multidisciplinary Journal, 20, 265-282.

21.

Kamata

(2001). Item analysis by the hierarchical generalized linear model. Journal of Educational Measurement, 38, 79-93.

22.

Kim

J.-S.

(2006). Using the distractor categories of multiple-choice items to improve IRT linking. Journal of Educational Measurement, 43, 193-213.

23.

Lee

W.-y.

Cho

S.-J.

(2015, April). Detecting cluster bias in a multilevel item response model. Annual Meeting of National Council on Measurement in Education, Chicago, IL.

24.

Longford

N. T.

Muthén

B. O.

(1992). Factor analysis for clustered observations. Psychometrika, 57, 581-597.

25.

Mislevy

R. J.

Bock

R. D.

(1989). A hierarchical item response model for educational testing. In Bock

R. D.

(Ed.), Multilevel analysis of educational data (pp. 57-74). San Diego, CA: Academic Press.

26.

Muthén

(1990). Mean and covariance structure analysis of hierarchical data. Los Angeles: University of California, Los Angeles.

27.

Muthén

Asparouhov

(2016). Multidimensional, multilevel, and multi-time point item response modeling. In van der Linden

W.J.

(Ed.), Handbook of item response theory, models, statistical tools, and applications (Vol. 1. pp. 527-540). Boca Raton, FL: Chapman & Hall.

28.

Muthén

L. K.

Muthén

B. O.

(1998-2015). Mplus [Computer program]. Los Angeles, CA: Author.

29.

Patz

R. J.

Junker

B. W.

(1999). A straightforward approach to Markov chain Monte Carlo methods for item response models. Journal of Educational and Behavioral Statistics, 24, 146-178.

30.

Rabe-Hesketh

Skrondal

Pickles

(2004). Generalized multilevel structural equation modeling. Psychometrika, 69, 167-190.

31.

Raudenbush

S. W.

(1997). Statistical analysis and optimal design for cluster randomized trials. Psychological Methods, 2, 173-185.

32.

Raudenbush

S. W.

Bryk

A. G.

(2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage.

33.

Spiegelhalter

D. J.

Best

N. G.

Carlin

B. P.

van der Linde

(2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society, Series B, 64, 583-616.

34.

Spiegelhalter

D. J.

Best

N. G.

Carlin

B. P.

van der Linde

(2014). The deviance information criterion: 12 years on. Journal of the Royal Statistical Society, Series B, 76, 485-493.

35.

Spiegelhalter

D. J.

Thomas

Best

N. G.

Lunn

(2003). WinBUGS user manual. Cambridge, UK: MRC Biostatistics Unit, Institute of Public Health.

36.

Suh

Bolt

D. M.

(2010). Nested logit models for multiple-choice item response data. Psychometrika, 75, 454-473.

37.

Suh

Bolt

D. M.

(2011). A nested logit approach for investigating distractors as causes of differential item functioning. Journal of Educational Measurement, 48, 188-205.

38.

Thissen

Steinberg

(1986). A taxonomy of item response models. Psychometrika, 51, 567-577.

39.

Verhagen

Fox

J.-P.

(2013). Longitudinal measurement in health-related surveys: A Bayesian joint growth model for multivariate ordinal responses. Statistics in Medicine, 32, 2988-3005.

40.

von Davier

Carstensen

C. H

. (2011). Measuring growth in a longitudinal large-scale assessment with a general latent variable model. Psychometrika, 76, 318-336.

41.

Wollack

J. A.

Bolt

D. M.

Cohen

A. S.

Lee

Y.-S.

(2002). Recovery of item parameters in the nominal response model: A comparison of marginal maximum likelihood estimation and Markov chain Monte Carlo estimation. Applied Psychological Measurement, 26, 339-352.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.11 MB

A Multilevel Longitudinal Nested Logit Model for Measuring Changes in Correct Response and Error Types

Abstract

Keywords

Get full access to this article

References

Supplementary Material