In spite of the growing interest in cross-cultural research and assessment, there is little research on statistical procedures that can be used to simultaneously assess the differential item functioning (DIF) across multiple groups. The chief objective of this work is to show a unified framework for the analysis of DIF in multiple groups using one of the most popular methodologies for DIF assessment: the Mantel-Haenszel (MH) methods. The MH statistics proposed to date with this purpose only permitted analysis of the DIF for dichotomous items. In contrast, the statistics presented here permit, through a single significance test, simultaneous evaluation of the DIF in several groups, being applicable to both dichotomous and polytomous items. Specific software to detect DIF using this methodology is available free of charge.
Angoff, W.H., & Sharon, A.T. ( 1974). The evaluation of differences in test performance of two or more groups. Educational and Psychological Measurement , 34, 807-816.
2.
Aptech Systems (1993). GAUSS (Version 3.1.4) [Computer programming language]. Maple Valley, WA: Author.
3.
Bradley, J.V. ( 1978). Robustness?The British Journal of Mathematical & Statistical Psychology, 33, 144-152.
4.
Chang, H.H., Mazzeo, J., & Roussos, J. ( 1996). Detecting DIF for politomously scored items: An adaptation of the SIBTEST procedure. Journal of Educational Measurement , 33, 333-353.
5.
Ellis, B.B., & Kimmel, H.D. ( 1992). Identification of unique cultural response patterns by means of item response theory. Journal of Applied Psychology , 77, 177-184.
6.
Fidalgo, A.M. ( 2005). Mantel-Haenszel methods. In B. S. Everitt & D. C. Howell (Eds.), Encyclopedia of statistics in behavioral science (Vol. 3, pp. 1120-1126). Chichester, United Kingdom: John Wiley & Sons.
7.
Fidalgo, A.M., Bartram, D., Quintanilla, L., Fernández, R., & Pons, F. ( 2008, July). Effect of the choice of scores assigned to the response variable on the Mantel test. Paper presented at the Third European Congress of Methodology, Oviedo, Spain.
8.
Fidalgo, A.M., & Madeira, J.M. ( 2008). Generalized Mantel-Haenszel methods for DIF detection . Educational and Psychological Measurement, 68, 940-958.
9.
Gierl, M.J., & Blot, D.M. ( 2001). Illustrating the use of nonparametric regression to assess differential item and bundle functioning between multiple groups. International Journal of Testing, 1, 249-270.
10.
Kim, S.H., Cohen, A.H., & Park, T.H. ( 1995). Detection of differential item functioning in multiple groups. Journal of Educational Measurement, 32, 261-276.
11.
Kristjansson, E., Aylesworth, R., McDowell, I., & Zumbo, B.D. ( 2005). A comparison of four methods for detecting differential item functioning in ordered response items. Educational and Psychological Measurement, 65, 935-953.
12.
Landis, J.R., Heyman, E.R., & Koch, G.G. ( 1978). Average partial association in three-way contingency tables: A review and discussion of alternative tests. International Statistical Review, 46, 237-254.
13.
Lord, F.M. ( 1977). A study of item bias, using item characteristic curve theory. In Y. H. Poortinga (Ed.), Basic problems in cross-cultural psychology (pp. 19-29). Amsterdam , The Netherlands: Swets and Zeitlinger B. V.
14.
Lord, F.M. ( 1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
15.
Mantel, N. ( 1963). Chi-square tests with one degree of freedom; extension of the Mantel-Haenszel procedure. Journal of the American Statistical Association, 58, 690-700.
16.
Mantel, N., & Haenszel, W. ( 1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute , 22, 719-748.
17.
Masters, G.N. ( 1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.
18.
Muraki, E. ( 1999). Stepwise analysis of differential item functioning based on multiple-group partial credit model. Journal of Educational Measurement, 36(3), 217-232.
19.
Penfield, R.D. ( 2001). Assessing differential item functioning among multiple groups: A comparison of three Mantel-Haenszel procedures. Applied Measurement in Education, 14, 235-259.
20.
Penfield, R.D., & Algina, J. ( 2003). Applying the Liu-Agresti estimator of the cumulative common odds ratio to DIF detection in polytomous items. Journal of Educational Measurement, 40(4), 353-370.
21.
Ramsay, J.O. ( 2000). TESTGRAF: A program for the graphical analysis of multiple-choice test and questionnaire data [Computer program and manual]. Retrieved February, 12, 2009, from http://ego.psych.mcgill.ca/faculty/ramsay/TestGraf.html
22.
Schmitt, A.P. ( 1988). Language and cultural characteristics that explain differential item functioning for Hispanic examinees on the Scholastic Aptitude Test. Journal of Educational Measurement, 25, 1-13.
23.
Schmitt, A.P., & Dorans, N.J. ( 1990). Differential item functioning for minority examinees on the SAT. Journal of Educational Measurement, 27, 67-81.
24.
Wang, W.C. ( 1998, April). Decomposition of DIF effects for polytomous items . Paper presented at the Annual Meeting of the American Educational Research Association, San Diego, CA .
25.
Wang, W.C., & Su, Y.H. ( 2004). Factors influencing the Mantel and Generalized Mantel-Haenszel methods for the assessment of differential item functioning in polytomous items. Applied Psychological Measurement, 28, 450-480.
26.
Zwick, R., & Ercikan, K. ( 1989). Analysis of differential item functioning in the NAEP history assessment. Journal of Educational Measurement, 26, 55-66.
27.
Zwick, R., Donoghue, J.R., & Grima, A. ( 1993). Assessment of differential item functioning for performance tasks. Journal of Educational Measurement, 30, 233-251.
