A nonparametric tree classification procedure is used to detect differential item functioning for items that are dichotomously scored. Classification trees are shown to be an alternative procedure to detect differential item functioning other than the use of traditional Mantel—Haenszel and logistic regression analysis. A nonparametric classification rule is examined through simulation and real data, and Type I error and power are compared with equivalent Mantel—Haenszel, logistic regression, and discriminant analyses.
Borsboom, D., Mellenbergh, G.J., & van der Linden, W.J. (2002). Different kinds of DIF: A distinction between absolute and relative forms of measurement invariance and bias. Applied Psychological Measurement, 26, 433-450.
Clark, L.A., & Pregibon, D. ( 1992). Tree-based models. In T. J. Hastie (Ed.), Statistical models in S (pp. 377-419). Pacific Grove, CA: Wadsworth.
4.
French, B.F. ( 2003). Iterative purification and effect size use with logistic regression for DIF detection (Unpublished dissertation). Purdue University, West Lafayette, IN.
5.
Hair, J.F., Anderson, R.E., Tatham, R.L., & Black, W.C. ( 1988). Multivariate data analysis. Upper Saddle River, NJ: Prentice Hall.
6.
Holland, P.W. ( 1985, October). On the study of differential item performance without IRT. In Proceedings of the 27th Annual Conference of the Military Testing Association (Vol. 1, pp. 282-287). San Diego, CA: Navy Personnel Research and Development Center.
7.
Jodoin, M.G., & Gierl, M.J. ( 2001). Evaluating Type I error and power rates using an effect size measure with the logistic regression procedure for DIF detection. Applied Measurement in Education, 14, 329-349.
8.
Li, H., & Stout, W. ( 1996). A new procedure for detection of crossing DIF. Psychometrika, 61, 647-677.
Mantel, N. ( 1963). Chi-square tests with one degree of freedom: Extensions of the Mantel-Haenszel procedure. Journal of the American Statistical Association, 58, 690-700.
11.
Mantel, N., & Haenszel, W. ( 1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute , 22, 719-748.
12.
Mellenbergh, G.J. ( 1982). Contingency table models for assessing item bias. Journal of Educational Statistics, 7, 105-118.
13.
Miller, T.R., & Spray, J.A. ( 1993). Logistic discriminant function analysis for DIF identification of polytomously scored items. Journal of Educational Measurement , 30, 107-122.
14.
Paek, I. ( 2002). Investigations of differential item functioning: Comparisons among approaches, and extensions to a multidimensional context (Unpublished doctoral dissertation). University of California at Berkeley , Berkeley.
15.
Rogers, H.J., & Swaminathan, H. (1993). A comparison of logistic regression and Mantel-Haenszel procedures for detecting differential item functioning. Applied Psychological Measurement, 17, 105-116.
Roussos, L.A., & Stout, W.F. ( 1996b). Simulation studies of the effects of small sample size and studied item parameters on SIBTEST and Mantel-Haenszel Type I error performance . Journal of Educational Measurement, 33, 215-230.
Tabachnick, B.G., & Fidell, L.S. ( 2001). Using multivariate statistics. Needham Heights, MA: Allyn & Bacon.
20.
Vaughn, B.K. ( 2006). A hierarchical generalized linear model of random differential item functioning for polytomous items: A Bayesian multilevel approach (Unpublished doctoral dissertation). Florida State University, Tallahassee.
21.
Vaughn, B.K. ( 2008). Better quality in assessments: Consideration of contextual effects on item bias and differential item functioning. Journal on School Educational Technology, 4(2), 29-39.
22.
Vaughn, B.K., & Wang, Q. ( 2008). Classification based on regression trees. Journal of Experimental Education, 65, 1-18.
23.
Venables, W.N., & Ripley, B.D. ( 2002). Modern applied statistics with S. New York: Springer-Verlag.
