A Sequential Process Model for Cognitive Diagnostic Assessment With Repeated Attempts

Abstract

When diagnostic assessments are administered to examinees, the mastery status of each examinee on a set of specified cognitive skills or attributes can be directly evaluated using cognitive diagnosis models (CDMs). Under certain circumstances, allowing the examinees to have at least one opportunity to correctly answer the questions and assessments, with repeated attempts on the items, provides many potential benefits. A sequential process model can be extended to model repeated attempts in diagnostic assessments. Two formulations of the sequential generalized deterministic-input noisy-“and”-gate (G-DINA) model were developed in this study. The first extension uses the latent transition analysis (LTA) approach to model changes in the attributes over attempts, and the second extension constructs a higher order structure of latent continuous variables and latent attributes to account for the dependences of the attributes over attempts. Accurate model parameter estimation and correct classifications of attributes were observed in a series of simulations using Bayesian estimation. The effectiveness of the developed sequential G-DINA model was demonstrated by fitting real data from a longitudinal mathematical test to the developed model and the longitudinal G-DINA model using the LTA approach. Finally, this article closes by discussing several important issues associated with the developed models and providing suggestions for future directions.