Multidimensional Linking for Domain Scores and Overall Scores for Nonequivalent Groups

Abstract

The No Child Left Behind Act requires state assessments to report not only overall scores but also domain scores. To see the information on students’ overall achievement, progress, and detailed strengths and weaknesses, and thereby identify areas for improvement in educational quality, students’ performances across years or across forms need to be made comparable in terms of both overall scores and domain scores. Multidimensional item response theory (MIRT) and its linking procedures for item parameter recovery have been studied; however, the effects of those linking procedures on the recovery of domain scores and overall scores have not been addressed, especially with regard to MIRT models of higher dimensions. The relationship between an examinee’s overall score and domain scores can be complex and may not be adequately represented by a linear function. This article proposes using the MIRT maximum information method to obtain the overall scores from MIRT domain scores. A simulation study was conducted to investigate the accuracy and effects of using the MIRT matching test response function (TRF) linking procedure to link the five-dimensional domain scores and their overall scores under varying conditions of anchor set lengths, population distribution types, and sample sizes. The results show that the TRF method recovered the domain scores and overall scores well for an anchor set of 10 items, with 2 items in each domain. Domain scores and overall scores have high reliabilities when the correlations between domains are high; reliability is higher than .9 for overall scores and higher than .8 for domain scores under all conditions.