Numerical Differentiation Methods for Computing Error Covariance Matrices in Item Response Theory Modeling

Abstract

In item response theory (IRT) modeling, the item parameter error covariance matrix plays a critical role define? in statistical inference procedures. When item parameters are estimated using the EM algorithm, the parameter error covariance matrix is not an automatic by-product of item calibration. Cai proposed the use of Supplemented EM algorithm for computing the item parameter error covariance matrix. This method has been subsequently implemented in commercial IRT software programs such as IRTPRO and flexMIRT. Jamshidian and Jennrich noted that Supplemented EM is among a class of methods based on numerically differentiating the EM map, and they proposed noniterative alternatives, such as forward difference and Richardson extrapolation, that are mathematically simpler and may lead to a reduction in computational burden when compared with Supplemented EM. However, the relative merits of the various numerical differentiation methods have not been evaluated in the context of IRT modeling. We perform such an evaluation, using both simulated and empirical data. It is found that the accuracy of the simpler noniterative alternatives is heavily dependent on the choice of the numerical differentiation perturbation constants. On the other hand, Supplemented EM consistently maintains accuracy and does not require the selection of perturbation constants. Furthermore, when implemented with an adaptive iteration scheme, an updated Supplemented EM algorithm can be as computationally efficient as the alternatives. The expected (Fisher) information matrix, while accurate, requires too heavy computation for realistic test lengths. Therefore, we recommend the routine use of the updated Supplemented EM algorithm in IRT applications.