Sage Journals: Discover world-class research

Abstract

The asymptotic power of the Mantel–Haenszel (MH) test for the differential item function (DIF) is derived. The formula describes the behavior of the power when the number of items is large, so that the measured latent trait can be considered as the matching variable in the MH test. As shown in the derived formula, the power is related to the sample size, effect size of DIF, the item response function (IRF), and the distribution of the latent trait in the reference and the focal groups. The formula provides an approximation of the power of the MH test in practice and thus provides a guideline for DIF detection in practice. It also suggests analytical explanations of the behavior of the MH test as observed in many previous simulation studies. Based on the formula, this study shows how to conduct the sample size calculation. The power of MH test under some practical models such as the two-parameter logistic (2PL) and three-parameter logistic (3PL) item response theory (IRT) models is discussed.

Keywords

DIF Mantel-Haenszel test item response theory power

Get full access to this article

View all access options for this article.

References

Agresti

(2002). Categorical data analysis (2nd ed.). New York, NY: John Wiley.

Birch

M. W.

(1964). The detection of partial association, I: The 2 × 2 case. Journal of the Royal Statistical Society: Series B (Methodological), 26, 313-324.

Cheng

Chen

Qian

Chang

H.-H.

(2013). Equated pooled booklet method in DIF testing. Applied Psychological Measurement, 37, 276-288.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.

DeMars

C. E.

(2010). Type I error inflation for detecting DIF in the presence of impact. Educational and Psychological Measurement, 70, 961-972.

Donoghue

J. R.

Holland

P. W.

Thayer

D. T.

(1993). A Monte Carlo study of factors that affect the Mantel-Haenszel and standardization measures of differential item functioning. In Holland

P. W.

Wainer

(Eds.), Differential item functioning: Theory and practice (pp. 137-166). Hillsdale, NJ: Lawrence Erlbaum.

Finch

W. H.

(2012). The MIMIC model as a tool for differential bundle functioning detection. Applied Psychological Measurement, 36, 40-59.

Holland

P. W.

Thayer

D. T.

(1988). Differential item performance and the Mantel-Haenszel procedure. In Wainer

Braun

(Eds.), Test validity (pp. 129-145). Hillsdale, NJ: Lawrence Erlbaum.

Holland

P. W.

Wainer

H. E.

(1993). Differential item functioning: Theory and practice. Hillsdale, NJ: Lawrence Erlbaum.

10.

Lehmann

E. L.

(1999). Elements of large-sample theory. New York, NY: Springer.

11.

(2014a). Power and sample size calculations for logistic regression tests for differential item functioning. Journal of Educational Measurement, 51, 441-462.

12.

(2014b). A power formula for the SIBTEST procedure for differential item functioning. Applied Psychological Measurement, 38, 311-328.

13.

Lord

F. M.

(1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.

14.

Mantel

Haenszel

(1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22, 719-748.

15.

Millsap

R. E.

Everson

H. T.

(1993). Methodology review: Statistical approaches for assessing measurement bias. Applied Psychological Measurement, 17, 297-334.

16.

Paek

(2010). Conservativeness in rejection of the null hypothesis when using the continuity correction in the MH chi-square test in DIF applications. Applied Psychological Measurement, 34, 539-548.

17.

Paek

Guo

(2011). Accuracy of DIF estimates and power in unbalanced designs using the Mantel-Haenszel DIF detection procedure. Applied Psychological Measurement, 35, 518-535.

18.

Phillips

Holland

P. W.

(1987). Estimators of the variance of the Mantel-Haenszel log-odds-ratio estimate. Biometrics, 43, 425-431.

19.

R Core Team. (2012). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Available from http://www.r-project.org

20.

Roussos

L. A.

Schnipke

D. L.

Pashley

P. J.

(1999). A generalized formula for the Mantel-Haenszel differential item functioning parameter. Journal of Educational and Behavioral Statistics, 24, 293-322.

21.

Roussos

L. A.

Stout

W. F.

(1996). Simulation studies of the effects of small sample size and studied item parameters on SIBTEST and Mantel-Haenszel type I error performance. Journal of Educational Measurement, 33, 215-230.

22.

Swaminathan

Rogers

H. J.

(1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361-370.

23.

Uttaro

Millsap

R. E.

(1994). Factors influencing the Mantel-Haenszel procedure in the detection of differential item functioning. Applied Psychological Measurement, 18, 15-25.

24.

Wang

Tay

Drasgow

(2013). Detecting differential item functioning of polytomous items for an ideal point response process. Applied Psychological Measurement, 37, 316-335.

25.

Zwick

R. J.

(2012). A review of ETS differential item functioning assessment procedures: Flagging rules, minimum sample size requirements, and criterion refinement (ETS Research Report No. RR-12-08). Princeton, NJ: ETS.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.60 MB

A Power Formula for the Mantel–Haenszel Test for Differential Item Functioning

Abstract

Keywords

Get full access to this article

References

Supplementary Material