This article compares several procedures in their efficacy for detecting differential item functioning (DIF): logistic regression analysis, the Mantel-Haenszel (MH) procedure, and the modified Mantel-Haenszel procedure by Mazor, Clauser, and Hambleton. It also compares the effect size measures that these procedures provide. In this study, different conditions of item parameters (difficulty and discrimination) and DIF magnitude were manipulated. Furthermore, both uniform and nonuniform DIF conditions were simulated. Results suggest that logistic regression analysis generally detected more items with DIF than the standard MH procedure and the modified MH procedure for symmetrical nonuniform DIF. The DIF effect size measures based on logistic regression, however, appeared to be insensitive to the specified DIF conditions.
Bennett, R. E., Rock, D. A.,& Kaplan, B. A. (1987). SAT differential item performance for nine handicapped groups. Journal of Educational Measurement, 24, 56-64.
2.
Clauser, B. E., & Mazor, K. M. (1998). Using statistical procedures to identify differential item functioning test items. Educational Measurement: Issues and Practice, 17, 31-44.
3.
Clauser, B. E., Mazor, K. M., & Hambleton, R. K. (1993). The effects of purification of the matching criterion on the identification of DIF using the Mantel-Haenszel procedure. Applied Measurement in Education, 6, 269-279.
4.
Clauser, B. E., Mazor, K. M.,& Hambleton, R. K. (1994). The effects of score group width on the Mantel-Haenszel procedure. Journal of Educational Measurement, 31, 67-78.
5.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.
6.
Ellis, B. (1989). Differential item functioning: Implications for test translations. Journal of Applied Psychology, 74, 912-921.
7.
Fidalgo, A. M. (1994). MHDIF: Acomputer program for detecting uniform and nonuniform differential item functioning with the Mantel-Haenszel procedure. Applied Psychological Measurement, 18, 300-300.
8.
Hambleton, R. K.,& Rogers, H. J. (1989). Detecting potentially biased test items: Comparison of IRT area and Mantel-Haenszel methods. Applied Measurement in Education, 2, 313-334.
9.
Hambleton, R. K.,& Rovinelli, R. (1973). AFORTRANIV program for generating examinee response data from logistic test models. Behavioral Science, 18, 73-74.
10.
Holland, P. W., & Thayer, D. T. (1988). Differential item performance and Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 129-145). Hillsdale, NJ: Lawrence Erlbaum.
11.
Jodoin, M. G.,& Gierl, M.J. (2001). Evaluating Type I error and power rates using an effect size measure with logistic regression procedure for DIF detection. Applied Measurement in Education, 14, 329-349.
12.
Kirk, R. E. (1996). Practical significance: A concept whose time has come. Educational and Psychological Measurement, 56, 746-759.
13.
Mazor, K. M., Clauser, B. E.,& Hambleton, R. K. (1992). The effect of sample size on the functioning of the Mantel-Haenszel statistic. Educational and Psychological Measurement, 52, 443-451.
14.
Mazor, K. M., Clauser, B. E.,& Hambleton, R. K. (1994). Identification of nonuniform differential item functioning using a variation of the Mantel-Haenszel procedure. Educational and Psychological Measurement, 54, 284-291.
15.
Mellenbergh, G. J. (1982). Contingency table models for assessing item bias. Journal of Educational Statistics, 7, 105-118.
16.
Narayan, P.,& Swaminathan, H. (1994). Performance of the Mantel-Haenszel and simultaneous item bias procedures for detecting differential item functioning. Applied Psychological Measurement, 18, 315-338.
17.
Narayan, P., & Swaminathan, H. (1996). Identification of items that show nonuniform DIF. Applied Psychological Measurement, 20, 257-274.
18.
Raju, N. S. (1988). The area between two item characteristic curves. Psychometrika, 53, 284-291.
19.
Rogers, H. J. (1989). Item bias investigation with logistic regression (Doctoral dissertation, University of Massachusetts at Amherst, 1989). Dissertation Abstract International, 50, 3928.
20.
Rogers, H. J., & Swaminathan, H. (1993). A comparison of logistic regression and Mantel Haenszel procedures for detecting differential item functioning. Applied Psychological Measurement, 17, 105-116.
21.
Roussos, L. A.,& Stout, W. F. (1996). Simulation studies of the effects of small sample size and studied item parameters on SIBTEST and Mantel-Haenszel Type I error performance. Journal of Educational Measurement, 33, 215-230.
22.
Swaminathan, H., & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361-370.
23.
Zumbo, B. D. (1999). A handbook on the theory and methods of differential item functioning (DIF): Logistic regression modeling as a unitary framework for binary and Likert-type (ordinal) item scores. Ottawa, Canada: Directorate of Human Resources Research and Evaluation, Department of National Defense.
24.
Zumbo, B. D.,& Thomas, D. R. (1997). A measure of effect size for a model-based approach for studying DIF(Working paper of the Edgeworth Laboratory for Quantitative Behavioral Science). Prince George, Canada: University of Northern British Columbia.
25.
Zwick, R., & Ercikan, K. (1989). Analysis of differential item functioning in the NAEP history assessment. Journal of Educational Measurement, 26, 55-66.
