Flexible Computerized Adaptive Tests to Detect Misconceptions and Estimate Ability Simultaneously

Abstract

The Scaling Individuals and Classifying Misconceptions (SICM) model is an advanced psychometric model that can provide feedback to examinees’ misconceptions and a general ability simultaneously. These two types of feedback are represented by a discrete and a continuous latent variable, respectively, in the SICM model. The complex structure of the SICM model brings difficulties in estimating both misconception profile and ability efficiently in a linear test. To overcome this challenge, this study proposes a flexible computerized adaptive test (FCAT) design as a new test delivery method to increase test efficiency by administering an individualized test to examinees. We propose three item selection methods and two transition criteria to determine adaptive steps based on the needs of estimating one or two latent variables. Through two simulation studies, we demonstrate how to select an appropriate item selection method for an adaptive step and what transition criterion should be used between two adaptive steps. Results reveal the combination of the item selection method and the transition criterion could improve the estimation accuracy of a specific latent variable to a different extent and thus provide further guidance in designing an FCAT.