Differential item functioning (DIF) is not often seen in the literature on attitude assessment. A brief discussion of DIF and methods of implementation is followed by an illustrative example from a program evaluation, using an attitude-towards-science scale with 1550 children in grades one through six. An item exhibiting substantial DIF with respect to gender was detected using the Mantel-Haenszel procedure. In a second example, data from workshop evaluations with 1682 adults were recoded to a binary format, and it was found that an item suspected of functioning George A. Johanson differentially with respect to age groups was, in fact, not doing so. Implications for evaluation practice are discussed.
Get full access to this article
View all access options for this article.
References
1.
Agresti, A. (1990). Categorical data analysis. New York: John Wiley & Sons.
2.
Benson, J. (1987). Detecting item bias in affective scales. Educational and Psychological Measurement, 47, 55-67.
3.
Camilli, G., & Shepard, L.A. (1994). Methods for identifying biased test items. Thousand Oaks, CA: Sage Publications.
4.
Chang, H., Mazzeo, J., & Roussos, L. (1996). Detecting DIF for polytomously scored items: An adaptation of the SIBTEST procedure. Journal of Educational Measurement, 33, 333-353.
5.
Clauser, B., Mazor, K., & Hambleton, R.K. (1993). The effects of purification of the matching criterion on the identification of DIF using the Mantel-Haenszel procedure. Applied Measurement in Education, 6, 269-279.
6.
Cochran, W.G. (1954). Some methods of strengthening the common X2 tests . Biometrics, 10, 417-451.
7.
Cohen, A.S., Kim, S.H., & Baker, F.B. (1993). Detection of differential item functioning in the graded response model. Applied Psychological Measurement , 17, 335-350.
8.
Converse, J.M., & Presser, S. (1986). Survey questions: Handcrafting the standardized questionnaire. Beverly Hills, CA: Sage Publications.
9.
Donoghue, J.R., Holland, P.W., & Thayer, D.T. (1993). A Monte-Carlo study of factors that affect the Mantel-Haenszel and standardization measures of differential item functioning . In P. W. Holland, & H. Wainer (Eds.), Differential item functioning (pp. 137-166). Hillsdale, NJ: Lawrence Erlbaum Associates .
10.
Dorans, N.J. (1989). Two new approaches to assessing differential item functioning: Standardization and the Mantel-Haenszel method. Applied Measurement in Education, 2, 217-233.
11.
Dorans, N.J., & Holland, P.W. (1993). DIF detection and description: Mantel-Haenszel and standardization. In P. W. Holland, & H. Wainer (Eds.), Differential item functioning (pp. 35-66). Hillsdale, NJ: Lawrence Erlbaum Associates.
12.
Engelhard, G., Hansche, L., & Rutledge, K.E. (1990). Accuracy of bias review judges in identifying differential item functioning on teacher certification tests. Applied Measurement in Education, 3, 347-360.
13.
Fidalgo, A.M. (1994). MHDIF: A computer program for detecting uniform and nonuniform differential item functioning with the Mantel-Haenszel procedure . Applied Psychological Measurement, 18, 300.
14.
French, A., & Miller, T. (1996). Logistic regression and its use in detecting differential item functioning in polytomous items. Journal of Educational Measurement, 33, 315-332.
15.
Holland, P.W., & Thayer, D.T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer, & H. I. Braun (Eds.), Test validity (pp. 129-145). Hillsdale, NJ: Lawrence Erlbaum Associates .
16.
Ironson, G.H. (1983). Using item response theory to measure bias. In R. K. Hambleton (Ed.), Applications of item response theory (pp. 155-174). Vancouver, B. C., Canada: Educational Research Institute of British Columbia.
17.
Mantel, N., & Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22, 719-748.
18.
Martin, R. (1989). The lead teacher project: K-6 mathematics and science elementary teacher enhancement. Ohio University , Athens, OH. NSF grant number 91-47392.
Mueller, D.J. (1986). Measuring social attitudes: A handbook for researchers and practitioners. New York: Teachers College Press.
21.
Nichols, D.P. (1994). The Mantel-Haenszel statistic for 2 x 2 x K tables . Keywords: Tips and news for statistical software users , 54, 10-12.
22.
Raju, N.S., Bode, R.K., & Larsen, V.S. (1989). An empirical assessment of the Mantel-Haenszel statistic for studying differential item performance. Applied Measurement in Education, 2, 1-13.
23.
Uttaro, T., & Millsap, R.E. (1994). Factors influencing the Mantel-Haenszel procedure in the detection of differential item functioning. Applied Psychological Measurement, 18, 15-25.
24.
Welch, C., & Hoover, H.D. (1993). Procedures for extending item bias detection techniques to polytomously scored items. Applied Measurement in Education , 6, 1-19.
25.
Zieky, M. (1993). Practical questions in the use of DIF statistics in test development. In P. W. Holland, & H. Wainer (Eds.), Differential item functioning (pp. 337-347). Hillsdale, NJ: Lawrence Erlbaum Associates.
