Differential item functioning (DIF) occurs when an item on a test or questionnaire has different measurement properties for one group of people versus another, irrespective of mean differences on the construct. There are many methods available for DIF assessment. The present article is focused on indices of partial association. A family of average conditional ordinal association measures described by Quade is described and empirically compared to the Mantel-Haenszel (MH) test, the partial Pearson correlation pr, and the Spearman rank correlation prs. Because coefficients of linear association are not meaningful for the binary and ordinal variables usually used in DIF applications, practitioners are urged to seek alternatives to pr and prs. Some of the Quade-family measures are viable alternatives and performed as well as, or better than, the established MH test. Computer code for calculating the average conditional ordinal measures using the free R program is given.
Agresti, A. ( 1977). Considerations in measuring partial association for ordinal categorical data. Journal of the American Statistical Association , 72, 37-45.
2.
Agresti, A. ( 1984). Analysis of ordinal categorical data. New York: John Wiley.
3.
Bolt, D.M., Hare, R.D., Vitale, J.E., & Newman, J.P. ( 2004). A multigroup item response theory analysis of the Psychopathy Checklist-Revised. Psychological Assessment , 16, 155-168.
4.
Camilli, G., & Shepard, L.A. ( 1994). Methods for identifying biased test items. Thousand Oaks, CA: Sage.
5.
Clauser, B.E., Nungester, R.J., & Swaminathan, H. (1996). Improving the matching for DIF analysis by conditioning on both test score and an educational background variable. Journal of Educational Measurement, 33, 453-464.
6.
Cliff, N. ( 1996). Ordinal methods for behavioral data analysis. Mahwah, NJ: Lawrence Erlbaum.
7.
Daniels, H.E. ( 1944). The relation between measures of correlation in the universe of sample permutations. Biometrika, 33, 129-135.
8.
Davis, J.A. ( 1967). A partial coefficient for Goodman and Kruskal’s gamma. Journal of the American Statistical Association, 62, 189-193.
9.
Donoghue, J.R., & Cliff, N. ( 1991). An investigation of ordinal true score test theory . Applied Psychological Measurement, 15, 335-351.
10.
Elfving, G., & Whitlock, J.H. (1950). A simple trend test with application to erythrocyte size data. Biometrics, 6, 282-288.
Goodman, L.A., & Kruskal, W.H. ( 1954). Measures of association for cross classifications . Journal of the American Statistical Association, 49, 732-764.
13.
Hawkes, R.K. ( 1971). The multivariate analysis of ordinal measures . American Journal of Sociology, 76, 908-926.
14.
Hill, C.D. ( 2004). Precision of parameter estimates for the graded item response model. Unpublished master’s thesis, University of North Carolina at Chapel Hill.
15.
Höeffding, W. ( 1948). A class of statistics with asymptotically normal distribution . Annals of Mathematical Statistics, 19, 293-325.
16.
Holland, P.W., & Thayer, D.T. ( 1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 129-145). Hillsdale, NJ: Lawrence Erlbaum.
17.
Holland, P. W., & Wainer, H. (Eds.). (1993). Differential item functioning . Hillsdale, NJ: Lawrence Erlbaum .
18.
Iwata, N., & Buka, S. ( 2002). Race/ethnicity and depressive symptoms: A cross-cultural/ethnic comparison among university students in East Asia, North and South America . Social Science and Medicine, 55, 2243-2252.
19.
Iwata, N., Turner, R.J., & Lloyd, D.A. ( 2002). Race/ethnicity and depressive symptoms in community-dwelling young adults: A differential item functioning analysis. Psychiatry Research, 110, 281-289.
20.
Jang, Y., Kim, G., & Chiriboga, D. ( 2005). Acculturation and manifestation of depressive symptoms among Korean-American older adults. Aging and Mental Health , 9, 500-507.
Kendall, M.G. ( 1938). A new measure of rank correlation. Biometrika , 30, 81-93.
23.
Kendall, M.G. ( 1945). The treatment of ties in ranking problems. Biometrika, 33, 239-251.
24.
Kim, J. (1975). Multivariate analysis of ordinal variables. The American Journal of Sociology, 81, 261-298. Korn, E. L. (1984). Kendall’s tau with a blocking variable. Biometrics, 40, 209-214.
25.
Kreiner, S. ( 2003). Introduction to DIGRAM. Unpublished research report , Department of Biostatistics, University of Copenaghen , Denmark.
26.
Kreiner, S., & Christensen, K.B. (2002). Graphical Rasch models . In M. Mesbah, B. F. Cole, & M. T. Lee (Eds.), Statistical methods for quality of life studies (pp. 187-203). Dordrecht, the Netherlands: Kluwer Academic.
27.
Kreiner, S., & Christensen, K.B. (2004). Analysis of local dependence and multidimensionality in graphical loglinear Rasch models. Communications in Statistics, 6, 1239-1276.
28.
Mantel, N. ( 1963). Chi-square tests with one degree of freedom: Extensions of the Mantel-Haenszel procedure. Journal of the American Statistical Association, 58, 690-700.
29.
Mantel, N., & Haenszel, W. ( 1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute , 22, 719-748.
30.
Mazor, K.M., Hambleton, R.K., & Clauser, B.E. ( 1998). Multidimensional DIF analyses: The effects of matching on unidimensional subtest scores. Applied Psychological Measurement , 22, 357-367.
31.
Mellenbergh, G.J. ( 1989). Item bias and item response theory. International Journal of Educational Research, 13, 127-143.
Nijenhuis, J., van der Flier, H., & van Leeuwen, L. (1997). Comparability of personality test scores for immigrants and majority group members: Some Dutch findings . Personality and Individual Differences, 23, 849-859.
36.
Nijenhuis, J., van der Flier, H., & van Leeuwen, L. (2003). The use of a test for neuroticism, extraversion, and rigidity for Dutch immigrant job applicants . Applied Psychology: An International Review, 52, 630-647.
37.
Nolan, R.F., Watlington, D.K., & Willson, V.L. ( 1989). Gifted and nongifted race and gender effects on item functioning in the Kaufman assessment battery for children. Journal of Clinical Psychology, 45, 645-650.
38.
Quade, D. ( 1974). Nonparametric partial correlation. In H. M. Blalock (Ed.), Measurement in the social sciences: Theories and strategies (pp. 369-398). Chicago: Aldine.
39.
R Development Core Team. (2005). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing . Available at http://www.R-project.org
40.
Reynolds, C.R., Willson, V.L., & Chatman, S.R. ( 1984). Item bias on the 1981 revision of the Peabody picture vocabulary test using a new method of detecting bias. Journal of Psychoeducational Assessment, 2, 219-224.
41.
Reynolds, H.T. ( 1973). On ‘‘The multivariate analysis of ordinal measures.’’ American Journal of Sociology , 78, 1513-1516.
42.
Reynolds, H.T. ( 1974). Ordinal partial correlation and causal inferences . In H. M. Blalock (Ed.), Measurement in the social sciences: Theories and strategies (pp. 399-422). Chicago: Aldine.
43.
Shealy, R., & Stout, W. ( 1993). A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58, 159-194.
44.
Somers, R.H. ( 1959). The rank analogue of product-moment partial correlation and regression, with application to manifold, ordered contingency tables. Biometrika, 46, 241-246.
45.
Somers, R.H. ( 1962). A new asymmetric measure of association for ordinal variables . American Sociological Review, 27, 799-811.
46.
Somers, R.H. ( 1964). Simple measures of association for the triple dichotomy . Journal of the Royal Statistical Society, Series A, 127, 409-415.
47.
Somers, R.H. ( 1968). An approach to the multivariate analysis of ordinal data . American Sociological Review, 33, 971-977.
48.
Somers, R.H. ( 1970). A partitioning of ordinal information in a three-way cross-classification. Multivariate Behavioral Research, 5, 217-239.
49.
Somers, R.H. ( 1974). Analysis of partial rank correlation measures based on the product-moment model: Part one. Social Forces , 53, 229-246.
50.
Steinberg, L., & Thissen, D. ( 2006). Using effect sizes for research reporting: Examples using item response theory to analyze differential item functioning. Psychological Methods, 11, 402-415.
51.
Stricker, L.J. ( 1982). Identifying test items that perform differently in population subgroups: A partial correlation index. Applied Psychological Measurement, 6, 261-273.
52.
Stricker, L.J. ( 1984). The stability of a partial correlation index for identifying items that perform differentially in subgroups. Educational and Psychological Measurement, 44, 831-837.
Thissen, D., Steinberg, L., & Gerrard, M. ( 1986). Beyond group-mean differences: The concept of item bias . Psychological Bulletin, 99, 118-128.
55.
Thissen, D., Steinberg, L., & Wainer, H. ( 1988). Use of item response theory in the study of group difference in trace lines. In H. Wainer & H. Braun (Eds.), Test validity (pp. 147-169). Hillsdale, NJ: Lawrence Erlbaum.
56.
Thissen, D., Steinberg, L., & Wainer, H. ( 1993). Detection of differential item functioning using the parameters of item response models. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 67-113). Hillsdale, NJ : Lawrence Erlbaum.
57.
van Leest, P.F. ( 1997). Bias and equivalence research in the Netherlands . European Review of Applied Psychology, 47, 319-328.
58.
Vargha, A., Rudas, T., Delaney, H.D., & Maxwell, S.E. ( 1996). Dichotomization, partial correlation, and conditional independence. Journal of Educational and Behavioral Statistics , 21, 264-282.
59.
Wainer, H. ( 1993). Model-based standardized measurement of an item’s differential impact. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 123-135). Hillsdale, NJ: Lawrence Erlbaum.
60.
Wainer, H., Sireci, S.G., & Thissen, D. ( 1991). Differential testlet functioning: Definitions and detection . Journal of Educational Measurement, 28, 197-219.
61.
Wang, W., & Yeh, Y. ( 2003). Effects of anchor item methods on differential item functioning detection with the likelihood ratio test. Applied Psychological Measurement, 27, 479-498.
62.
Wilson, T.P. ( 1974a). Measures of association for bivariate ordinal hypotheses . In H. M. Blalock (Ed.), Measurement in the social sciences: Theories and strategies (pp. 327-342). Chicago: Aldine.
63.
Wilson, T.P. ( 1974b). On interpreting ordinal analogies to multiple regression and path analysis. Social Forces, 53, 196-199.
64.
Wilson, T.P. ( 1974c). Reply to Somers andSmith. Social Forces , 53, 247-251.
65.
Willson, V.L., Nolan, R.F., Reynolds, C.R., & Kamphaus, R.W. (1989). Race and gender effects on item functioning on the Kaufman assessment battery for children. Journal of School Psychology, 27, 289-296.
66.
Woods, C.M. ( 2007). Confidence intervals for gamma-family measures of ordinal association. Psychological Methods, 12, 185-204.
67.
Woods, C.M. (in press). Empirical selection of anchors for tests of differential item functioning. Applied Psychological Measurement.
68.
Yule, G.U. ( 1900). On the association of attributes in statistics: With illustrations from the material of the childhood society. Philosophical Transactions of the Royal Society of London, Series A, 194, 257-319.
69.
Zwick, R., Donoghue, J., & Grima, A. ( 1993). Assessment of differential item functioning for performance tasks. Journal of Educational Measurement, 30, 233-251.
