Multistage Testing for Cognitive Diagnosis Based on Skill-Space Partitioning

Abstract

The growing demand for personalized online learning underscores the necessity for diagnostic assessments that are tailored to the cognitive abilities of individual examinees. The combination of cognitive diagnostic models (CDMs) with computerized multistage testing (MST) holds potential for meeting these educational needs. However, research on the integration of MST with cognitive diagnosis (CD-MST) has been limited, largely due to the challenges in establishing criteria for constructing test modules and defining routing rules prior to administering the test. This study aims to introduce an innovative design approach for CD-MST that employs a strategy of partitioning the skill-space, which encompasses all possible attribute mastery profiles, to address the challenges. By partitioning the skill-space into groups of attribute profiles, distinct modules tailored to each partitioned group can be constructed, ensuring that each examinee is adaptively routed to the most suitable module at each testing stage. Item information functions for CD-MST are also proposed by defining the information conditionally on an attribute profile, in order to quantify an item’s discrimination power for each profile. Furthermore, a strategic approach for automated module assembly in CD-MST is developed to construct modules that maximize information for each attribute profile group while satisfying all practical constraints. Simulation results indicate that the proposed CD-MST improves estimation accuracy compared to traditional linear test and can effectively utilize a wider range of item types from the item bank.