Modification of the Mantel-Haenszel and Logistic Regression DIF Procedures to Incorporate the SIBTEST Regression Correction

Abstract

The Mantel-Haenszel (MH) and logistic regression (LR) differential item functioning (DIF) procedures have inflated Type I error rates when there are large mean group differences, short tests, and large sample sizes. When there are large group differences in mean score, groups matched on the observed number-correct score differ on true score, contributing to inflated Type I error rates. The simultaneous item bias test procedure has incorporated an adjustment for this difference, originally using a linear regression correction and later using a nonlinear correction. In this study, these adjustments are applied to the MH and LR procedures. They effectively reduce the Type I error inflation for the MH and the LR test of uniform DIF, but not the LR test of nonuniform DIF. For large samples and large group mean differences, the Δ effect size is estimated with greater accuracy using these adjustments.