A multidimensionality-based differential item func tioning (DIF) analysis paradigm is presented that unifies the substantive and statistical DIF analysis approaches by linking both to a theoretically sound and mathematically rigorous multidimensional conceptualization of DIF. This paradigm has the potential (1) to improve under standing of the causes of DIF by formulating and testing substantive dimensionality-based DIF hypotheses; (2) to reduce Type 1 error through a better understanding of the possible multidimensionality of an appropriate matching criterion; and (3) to increase power through the testing of bundles of items measuring similar dimen sions. Using this approach, DIF analysis is shown to have the potential for greater integration in the overall test development process.
Ackerman, T. (1992). A didactic explanation of item bias, item impact, and item validity from a multidimensional perspective. Journal of Educational Measurement, 29, 67-91.
2.
American College Testing. (1996). Test preparation: Reference manual for teachers and counselors. Iowa City IA: Author.
3.
Anastasi, A. (1986). Evolving concepts of test validation. Annual Review of Psychology, 37, 1-15.
4.
Angoff, W.H. (1993). Perspectives on differential item functioning methodology. In P W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 3-23). Hillsdale NJ: Erlbaum.
5.
Berk, R. A. (Ed.). (1982). Handbook of methods for detecting test bias. Baltimore MD: The Johns Hopkins University Press.
6.
Cohen, A.S., & Kim, S.-H. (1992). Detecting calculator effects on item performance . Applied Measurement in Education, 5, 303-320.
7.
Cole, N.S. (1981). Bias in testing. American Psychologist , 36, 1067-1077.
8.
College Board. (1995). Counselor's handbook for the SAT program. New York: Author.
9.
Cronbach, L.J. (1990). Essentials of psychological testing (5th ed.). New York: Harper & Row.
Dorans, N.J., & Schmitt, A.P. (1989, March). The methods for dimensionality assessment and DIF detection. Paper presented at the annual meeting of the National Council on Measurement in Education, San Francisco.
13.
Douglas, J., Roussos, L.A., & Stout, W.F. (1996). Item bundle DIF hypothesis testing: Identifying suspect bundles and assessing their DIF. Journal of Educational Measurement, 33, 465-485.
Embretson, S. (1983). Construct validity: Construct representation versus nomothetic span. Psychological Bulletin, 93, 179-197.
16.
Engelhard, G., Hansche, L., & Rutledge, K.E. (1990). Accuracy of bias review judges in identifying differential item functioning on teacher certification tests. Applied Measurement in Education, 3, 347-360.
17.
Holland, P.W., & Thayer, D.T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. Braun (Eds.), Test validity (pp. 129-145). Hillsdale NJ: Erlbaum.
18.
Jensen, A.R. (1980). Bias in mental testing. New York: Macmillan.
Kok, F. ( 1988). Item bias and test multidimensionality. In R. Langeheine & J. Rost (Eds.), Latent trait and latent class models (pp. 263-275). New York: Plenum.
21.
Lehmann, E.L. (1959). Testing statistical hypotheses. New York: Wiley.
22.
Li, H.-H., & Stout, W.F. (1996). A new procedure for detection of crossingDIF Psychometrika, 61, 647-677.
23.
Linn, R.L. (1986). Bias in college admissions. In Measures in the college admissions process: A College Board colloquium (pp. 80-86). New York: The College Entrance Examination Board.
24.
Linn, R.L. (1993). The use of differential item functioning statistics: A discussion of current practice and future implications. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 337-347). Hillsdale NJ: Erlbaum.
25.
Loevinger, J. (1957). Objective tests as instruments of psychological theory. Psychological Reports, 3, 635-694 (Monograph Supplement, 9).
26.
Lord, F.M. (1980). Applications of item response theory to practical testing problems. Hillsdale NJ: Erlbaum.
27.
Mazor, K.M., Kanjee, A., & Clauser, B.E. (1995). Using logistic regression and the Mantel-Haenszel with multiple ability estimates to detect differential item functioning. Journal of Educational Measurement, 32, 131-144.
28.
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed.; pp. 13-103). New York: Macmillan.
29.
Nandakumar, R., & Stout, W.F. (1993). Refinements of Stout's procedure for assessing latent trait unidimensionality. Journal of Educational Statistics , 18, 41-68.
30.
O'Neill, K.A., & McPeek, W.M. (1993). Item and test characteristics that are associated with differential item functioning. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 255-279). Hillsdale NJ: Erlbaum.
31.
Oshima, T.C., Raju, N.S., Flowers, C.P., & Slinde, J.A. (1996, April). Differential bundle functioning (DBF) using the DFIT framework: Procedures for identifying possible sources of DIF. Paper presented at the annual meeting of the National Council on Measurement in Education, New York.
32.
Plake, B.S. (1980). A comparison of statistical and subjective procedures to ascertain item validity: One step in the validation process. Educational and Psychological Measurement, 40, 397-404.
33.
Raju, N.S., van der Linden, W.J., & Fleer, P.F. (1995). IRT-based internal measures of differential functioning of items and tests. Applied Psychological Measurement, 19, 353-368.
34.
Ramsey, P.A. (1993). Sensitivity review: The ETS experience as a case study. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 367-388). Hillsdale NJ: Erlbaum.
35.
Reynolds, C.R. (1982). The problem of bias in psychological measurement . In C. R. Reynolds & T. B. Gutkin (Eds.), The handbook of school psychology (pp. 178-201). New York: Wiley .
36.
Scheuneman, J.D. (1982). A posteriori analyses of biased items. In R. A. Berk (Ed.), Handbook of methods for detecting test bias (pp. 180-198). Baltimore MD: The Johns Hopkins University Press.
37.
Scheuneman, J.D. (1987). An experimental exploratory study of causes of bias in test items. Journal of Educational Measurement, 24, 97-118.
38.
Schmitt, A.P. (1988). Language and cultural characteristics that explain differential item functioning for Hispanic examinees on the Scholastic Aptitude Test. Journal of Educational Measurement, 25, 1-13.
39.
Schmitt, A.P., Holland, P.W., & Dorans, N.J. (1993). Evaluating hypotheses about differential item functioning. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 281-315). Hillsdale NJ: Erlbaum.
40.
Shealy, R., & Stout, W.F. (1993a). An item response theory model for test bias. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 197-239). Hillsdale NJ: Erlbaum.
41.
Shealy, R., & Stout, W.F. (1993b). A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTT as well as item bias/DIF. Psychometrika, 58, 159-194.
42.
Shepard, L.A. (1981). Identifying bias in test items. In B. F. Green (Ed.), New directions in testing and measurement : Issues in testing—Coaching, disclosure, and test bias (No. 11; pp. 79-104). San Francisco : Jossey-Bass.
43.
Shepard, L.A. (1982). Definitions of bias. In R. A. Berk (Ed.), Handbook of methods for detecting test bias (pp. 9-30). Baltimore MD: The Johns Hopkins University Press.
44.
Shepard, L.A. (1987). Discussant comments on the NCME symposium: Unexpected differential item performance and its assessment among black, Asian-American, and Hispanic students. In A. P. Schmitt & N. J. Dorans (Eds.), Differential item functioning on the Scholastic Aptitude Test (RM-87-1. Princeton NJ: Educational Testing Service.
45.
Skaggs, G., & Lissitz, R.W. (1992). The consistency of detecting item bias across different test administrations: Implications of another failure. Journal of Educational Measurement, 29, 227-242.
46.
Stout, W.F. (1987). A nonparametric approach for assessing latent trait dimensionality. Psychometrika, 52, 589-617.
47.
Stout, W.F., Douglas, J., Habing, B., Kim, H.R., Roussos, L.A., & Zhang, J. (1996). Conditional covariance based nonparametric multidimensionality assessment. Applied Psychological Measurement, 20, 331-354.
48.
Stout, W.F., Li., H.-H., & Nandakumar, R. (1996). Use of SIBTEST to do DIF when the matching subtest is intentionally multidimensional. Manuscript submitted for publication.
49.
Stout, W.F., & Roussos, L.A. (1995). SIBTEST users manual (2nd ed.) [Computer program manual]. Urbana-Champaign : University of Illinois, Department of Statistics .
50.
Tittle, C.K. (1982). Use of judgemental methods in item bias studies . In R. A. Berk (Ed.), Handbook of methods for detecting test bias (pp. 31-63). Baltimore MD: The Johns Hopkins University Press.
51.
Wiley, D.E. (1990). Test validity and invalidity reconsidered. In R. Snow & D. E. Wiley (Eds.), Improving inquiry in social science (pp. 75-107). Hillsdale NJ: Erlbaum.
