Oshima, Raju, and Flowers demonstrated the use of an item response theory—based technique for analyzing differential item function (DIF) and differential test function for dichotomously scored data that are intended to be multidimensional. Their study assumed that the number of intended-to-be measured dimensions was correctly identified. In practice, however, the number of dimensions may be misidentified. Therefore, the purpose of this study was to demonstrate the effects of both underestimation and overestimation of the number of intended-to-be measured dimensions on the multidimensional DIF analysis using simulated two-dimensional data with known DIF items. Results show that overestimation of the number of y traits had a consequence of decreased power. Underestimation resulted in missing a certain type of nonuniform DIF, as well as confounding the impact with DIF. Recommendations are made on how to conduct a DIF investigation with a multidimensional within-item test.
Ackerman, T., Gierl, M., & Walker, C.M. ( 2003). Using multidimensional item response theory to evaluate educational and psychological tests. Educational Measurement: Issues and Practice, 22(3), 37-53.
2.
Ackerman, T.A. ( 1989). Unidimensional IRT calibration of compensatory and noncompensatory multidimensional items. Applied Psychological Measurement , 13, 113-127.
3.
Ackerman, T.A. ( 1992). A didactic explanation of item bias, item impact, and item validity from a multidimensional perspective. Journal of Educational Measurement, 29, 67-91.
4.
Camilli, G., Wang, M.-M., & Fesq, J. ( 1995). The effects of dimensionality on equating the law school admission test. Journal of Educational Measurement, 32, 79-96.
5.
Frasier, C. ( 1988). NOHARM [Computer program]. New South Wales, Australia: University of New England, Center for Behavioral Studies.
6.
Huang, C.D., Church, A.T., & Katigbak, M.S. (1997). Identifying cultural differences in items and traits-Differential item functioning in the NEO Personality Inventory . Journal of Cross-Cultural Psychology, 28, 192-218.
7.
Lee, K., & Oshima, T.C. ( 1996). IPLINK: Multidimensional and unidimensional item parameter linking in item response theory. Applied Psychological Measurement , 20, 230.
8.
Mazor, K.M., Hambleton, R.K., & Clauser, B.E. ( 1998). Multidimensional DIF analyses: The effects of matching on unidimensional subtest scores. Applied Psychological Measurement , 22, 357-367.
9.
McDonald, R.P. ( 2000). A basis for multidimensional item response theory. Applied Psychological Measurement, 24, 99-114.
10.
Muthén, L.K. & Muthén, B. ( 2001). Mplus (version 2.0) [Computer software]. Los Angeles, CA: Muthén & Muthén .
11.
Oshima, T.C., Davey, T.C., & Lee, K. ( 2000). Multidimensional linking: Four practical approaches. Journal of Educational Measurement, 37, 357-373.
12.
Oshima, T.C., & Morris, S.B. ( 2008). An NCME instructional module on Raju’s differential functioning of items and tests (DFIT). Educational Measurement: Issues and Practice, 27, 43-50.
13.
Oshima, T.C., Raju, N.S., & Flowers, C.P. ( 1997). Development and demonstration of multidimensional IRT-based internal measures of differential functioning of items and tests. Journal of Educational Measurement, 34, 253-272.
14.
Oshima, T.C., Raju, N.S., & Nanda, A. ( 2006). A new method for assessing statistical significance in the differential functioning of items and tests (DFIT) framework. Journal of Educational Measurement, 43, 1-17.
15.
Raju, N.S. ( 1997). DFIT [Computer program]. Atlanta: Georgia Institute of Technology.
16.
Raju, N.S., & Ellis, B.B. ( 2000). Differential item functioning. In F. Drasgow & N. Schmitt (Eds.), Measuring and analyzing behavior in organizations: Advances in measurement and data analysis (pp. 156-188). San Francisco: Jossey-Bass.
17.
Raju, N.S., Oshima, T.C., & Walach, A.H. ( 2009). DFIT8 [Computer software]. St. Paul, MN: Assessment Systems Corporation.
18.
Raju, N.S., van der Linden, W.J., & Fleer, P.F. ( 1995). An IRT-based internal measure of test bias with applications for differential item functioning. Applied Psychological Measurement , 19, 353-368.
19.
Reckase, M.D. ( 1979). Unifactor latent trait models applied to multifactor tests: Results and implications. Journal of Educational Statistics , 4, 207-230.
20.
Shealy, R., & Stout, W. ( 1993). A model-based standardization approach that separates true bias / DIF from group ability differences and detects test bias / DTF as well as item bias / DIF. Psychometrika, 58, 159-194.
21.
Stone, C., & Yeh, C.C. ( 2006). Assessing the dimensionality and factor structure of multiple-choice exams: An empirical comparison of methods using the multistate bar examination. Educational and Psychological Measurement , 66, 193-214.
22.
Stout, W. ( 1987). A nonparametric approach for assessing latent trait unidimensionality . Psychometrika, 52, 589-617.
23.
Stout, W., Li, H., Nandakumar, R., & Bolt, D. ( 1997). MULTISIB-A procedure to investigate DIF when a test is intentionally multidimensional. Applied Psychological Measurement , 21, 195-213.
24.
Wang, W.-C., Wilson, M.R., & Adams, R.J. ( 1997). Rasch models for multidimensionality between items and within items. In M. Wilson, G. Engelhard, & K. Draney (Eds.), Objective measurement: Vol. 4. Theory into practice (pp. 139-155). Norwood, NJ: Ablex.
25.
Wilson, D., Wood, R., Schilling, S., & Gibbons, R. ( 2003). TESTFACT (Version 4.0) [Computer software]. Lincolnwood, IL: Scientific Software International .
26.
Zhang, J. & Stout, W. ( 1999). The theoretical DETECT index of dimensionality and its application to approximate simple structure. Psychometrika , 64, 213-249.
