Explanatory Cognitive Diagnostic Models: Incorporating Latent and Observed Predictors

Abstract

Large-scale educational testing data often contain vast amounts of variables associated with information pertaining to test takers, schools, or access to educational resources—information that can help explain relationships between test taker performance and their learning environment. This study examines approaches to incorporate latent and observed explanatory variables as predictors for cognitive diagnostic models (CDMs). Methods to specify and simultaneously estimate observed and latent variables (estimated using item response theory) as predictors affecting attribute mastery were examined. Real-world data analyses were conducted to demonstrate the application using large-scale international testing data. Simulation studies were conducted to examine the recovery and classification for simultaneously estimating multiple latent (using dichotomous and polytomous items as indicators for the latent construct) and observed predictors for varying sample sizes and number of attributes. Results showed stable parameter recovery and consistency in attribute classification. Implications for latent predictors and attribute specifications are discussed.

Keywords

cognitive diagnostic models explanatory models item response theory covariates

Get full access to this article

View all access options for this article.

References

Bedeian

A. G.

Day

D. V.

Kelloway

E. K.

(1997). Correcting for measurement error attenuation in structural equation models: Some important reminders. Educational and Psychological Measurement, 57, 785-799.

Chang

Kim

(2009). Computer access and computer use for science performance of racial and linguistic minority students. Journal of Educational Computing Research, 40, 469-501.

Dayton

C. M.

Macready

G. B.

(1988). A latent class covariate model with applications to criterion-referenced testing. In Langeheine

Rost

(Eds.), Latent trait and latent class models (pp. 129-143). New York, NY: Plenum Press.

De Boeck

Wilson

. (2004). Explanatory item response models: A generalized linear and nonlinear approach. New York, NY: Springer.

DeCarlo

L. T.

(2005). On the use of covariates in a latent class signal detection model for essay grading. Unpublished manuscript.

DeCarlo

L. T.

(2011). On the analysis of fraction subtraction data: The DINA model, classification, latent class sizes, and the Q-matrix. Applied Psychological Measurement, 35, 8-26.

de la Torre

(2009). DINA model and parameter estimation: A didactic. Journal of Educational and Behavioral Statistics, 34, 115-130.

de la Torre

(2011). The generalized DINA model framework. Psychometrika, 76, 179-199.

DiBello

L. V.

Roussos

L. A.

Stout

(2007). Review of cognitively diagnostic assessment and a summary of psychometric models. In Rao

C. R.

Sinharay

(Eds.), Handbook of Statistics: Vol. 26. Psychometrics (pp. 979-1030). Amsterdam, The Netherlands: Elsevier.

10.

Haertel

E. H.

(1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 333-352.

11.

Henson

R. A.

Templin

J. L.

Willse

J. T.

(2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191-210.

12.

Huang

G. H.

Bandeen-Roche

(2004). Building an identifiable latent class model with covariate effects on underlying and measured variables. Psychometrika, 69, 5-32.

13.

Junker

B. W.

Sijtsma

(2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258-272.

14.

Lee

Y.-S.

Park

Y. S.

Taylan

(2011). A cognitive diagnostic modeling of attribute mastery in Massachusetts, Minnesota, and the U.S. national sample using the TIMSS 2007. International Journal of Testing, 11, 144-177.

15.

Park

Y. S.

Lee

Y.-S.

(2014). An extension of the DINA model using covariates: Examining factors affecting response probability and latent classification. Applied Psychological Measurement, 38, 376-390.

16.

Rupp

A. A.

Templin

Henson

R. A.

(2010). Diagnostic measurement: Theory, methods, and applications. New York, NY: Guilford Press.

17.

Samejima

(1969). Estimation of latent ability using a response pattern of graded scores (Psychometric Monograph No. 17). Richmond, VA: Psychometric Society.

18.

Tienken

C. H.

Wilson

M. J.

(2007). The impact of computer assisted instruction on seventh-grade students’ mathematics achievement. Planning and Changing, 38, 181-190.

19.

Vermunt

J. K.

Magidson

(2013). Technical guide for latent GOLD 5.0: Basic, advanced, and syntax. Belmont, MA: Statistical Innovations.

20.

von Davier

. (2005). A general diagnostic model applied to language testing data (Research Report RR-05-16). Princeton, NJ: Educational Testing Service.

21.

von Davier

. (2007). Mixture general diagnostic models (Research Report RR-07-32). Princeton, NJ: Educational Testing Service.

22.

von Davier

. (2008). A general diagnostic model applied to language testing data. British Journal of Mathematical and Statistical Psychology, 61, 287-307.

23.

von Davier

. (2010). Hierarchical mixtures of diagnostic models. Psychological Test and Assessment Modeling, 52, 8-28.

24.

von Davier

Carstensen

C. H

. (2011). Measuring growth in a longitudinal large-scale assessment with a general latent variable model. Psychometrika, 76, 318-336.

25.

von Davier

(2008). Fitting the structured general diagnostic model to NAEP data (RR-08-27, ETS Research Report). Princeton, NJ: ETS.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.06 MB