Optimal Calibration Designs for Tests of Polytomously Scored Items Described by Item Response Theory Models

Abstract

This article examines calibration designs, which maximize the determinant of Fisher’s information matrix on the item parameters (D-optimal), for sets of polytomously scored items. These items were analyzed using a number of item response theory (IRT) models, which are members of the “divide-by-total” family, including the nominal categories model, the rating scale model, the unidimensional polytomous Rasch model and the partial credit model. We extend the known results for dichotomous items, both singly and in tests to polytomous items. The structure of Fisher’s information matrix is examined in order to gain insights into the structure of D-optimal calibration designs for IRT models. A theorem giving an upper bound for the number of support points for such models is proved. A lower bound is also given.

Finally, we examine a set of items, which have been analyzed using a number of different models. The locally D-optimal calibration design for each analysis is calculated using an exact numerical and a sequential procedure. The results are discussed both in general and in relation to each other.