Identification of Nonuniform Differential Item Functioning: A Comparison of Mantel-Haenszel and Item Response Theory Analysis Procedures

Abstract

Although practical and one of the most popular methods, the Mantel-Haenszel (MH) procedure was not initially designated for detecting nonuniform differential item functioning (DIF). To improve detection rates for items showing nonuniform DIF, a modification of the standard MH procedure has been proposed by breaking the full sample at approximately the middle of the test score distribution and then rerunning the MH procedure on each subsample separately. The purpose of the present study was to compare this MH variation to a residual procedure based on item response theory analysis to test its capacity for the detection of nonuniform DIF. Different sets of examinee responses were generated to simulate both uniform and nonuniform DIF. Results from this study suggest that although the MH procedure may be effective for detecting nonuniform DIF, and although this capacity may be increased with a simple modification to the MH standard procedure, the residual analysis procedure achieves a higher detection rate with both uniform and nonuniform DIF.

Get full access to this article

View all access options for this article.

References

Barbero, M. I. , & Prieto, P. (1996). Detección de funcionamiento diferencial de los items mediante análisis de residuales: Una aplicación de la TRI. Psicothema, 8(1), 179-187.

Hambleton, R. K. , & Rogers, H. J. (1989). Detecting potentially biased test items: Comparison of IRT area and the Mantel-Haenszel methods. Applied Measurement in Education, 2, 313-334.

Hambleton, R. K. , Swaminathan, H. , & Rogers, H. J. (1991). Fundamentals of item response theory. London: Sage.

Holland, P. W. , & Thayer, D. T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 129-145). Hillsdale, NJ: Lawrence Erlbaum.

Linn, R. L. , & Harnisch, D. L. (1981). Interactions between item content and group membership on achievement test items. Journal of Educational Measurement, 18, 109-118.

Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.

Mazor, K. M. , Clauser, B. E. , & Hambleton, R. K. (1994). Identification of non-uniform differential item functioning using a variation of the Mantel-Haenszel procedure. Educational and Psychological Measurement, 54, 284-291.

Mellenberg, G. J. (1982). Contingency table models for assessing item bias. Journal of Educational Statistics, 7, 105-108.

San Luis, C. , Prieto, P. , Barbero, M. , & Sánchez, J. A. (1995). GENESTE: Un programa de control para TRI. Psicológica, 16, 297-304.

10.

Shepard, G. , Camilli, L. A. , & Williams, D. (1985). Validity for approximation techniques for detecting item bias. Journal of Educational Measurement, 22, 77-105.

11.

Swaminathan, H. , & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361-370.

12.

Wright, B. D. , & Panchapakesan, N. (1969). A procedure for sample-free item analysis. Educational and Psychological Measurement, 29, 23-48.