The differential functioning of items and tests (DFIT) framework was used to examine the measurement equivalence of a Spanish translation of the Sixteen Personality Factor (16PF) Questionnaire. The questionnaire was administered in English to English-speaking Anglo-Americans and English-dominant Hispanic Americans and in Spanish to Spanish-dominant Hispanic Americans and Spanish-speaking Mexican nationals. As expected, the compensatory differential item functioning/differential test functioning (CDIF/DTF) procedure, which accounts for CDIF at the scale level, flagged fewer items as differential functioning than did the noncompensatory differential item functioning (NCDIF) procedure. Results did not support the hypothesis that DIF would be greatest in the Anglo versus Spanish-speaker comparison followed by the Hispanic versus Spanish-speaker comparison and least in the Anglo versus Hispanic comparison. Advantages of using the DFIT framework in assessing test translations, especially for test developers, are discussed.
Get full access to this article
View all access options for this article.
References
1.
Baker, F. B. (1993). EQUATE 2.1: Computer program for equating two metrics in item response theory [Computer program]. Madison: University of Wisconsin, Laboratory of Experimental Design.
2.
Birnbaum, A. (1968). Some latent trait models and their use in inferring examinee’s ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test scores (pp. 397-479). Reading, MA: Addison-Wesley.
3.
Budgell, G. R. , Raju, N. S., & Quartetti, D. A. (1995). Analysis of differential item functioning in translated assessment instruments. Applied Psychological Measurement, 19, 309-321.
4.
Candell, G. L. , & Drasgow, F. (1988). An iterative procedure for linking metrics and assessing item bias in item response theory. Applied Psychological Measurement, 12, 253-260.
5.
Candell, G. L. , & Hulin, C. L. (1987). Cross-language and cross-cultural comparisons in scale translations: Independent sources of information about item nonequivalence. Journal of Cross-Cultural Psychology, 17, 417-440.
6.
Cattell, R. B. (1957). Personality and motivation: Structure and measurement. New York: World Book.
7.
Cattell, R. B. , Cattell, A. K., & Cattell, H.E.P. (1993). Sixteen Personality Factor Questionnaire, Fifth Edition. Champaign, IL: Institute for Personality and Ability Testing.
8.
Chan, K. , Chernyshenko, O. S., & Stark, S. (1997). Factor analysis of the fifth edition of 16PF. Champaign, IL: Institute of Personality and Ability Testing.
9.
Cohen, A. S. , & Kim, S. H. (1993). A comparison of Lord’s χ2 and Raju’s area measures in detection of DIF. Applied Psychological Measurement, 17, 39-52.
10.
Collins, W. C. , Raju, N. S., & Edwards, J. E. (1997, April). Assessing DIF in a polytomously-scored satisfaction scale. Paper presented at the “Recent Developments in IRT” symposium conducted at the annual meeting of the Society for Industrial and Organizational Psychology, St. Louis, MO.
11.
Conn, S. R. , & Rieke, M. L. (1994). The 16PF Fifth Edition technical manual. Champaign, IL: Institute for Personality and Ability Testing.
12.
Dorans, N. J. , & Kulick, E. (1983). Assessing unexpected differential item performance of female candidates on SAT and TSWE forms administered in December 1977: An application of the standardization approach (Research Report No. 83-9). Princeton, NJ: Educational Testing Service.
13.
Dorans, N. J. , & Kulick, E. (1988). The standardization approach to assessing differential speededness (Research Report No. 88-31). Princeton, NJ: Educational Testing Service.
14.
Drasgow, F. (1984). Scrutinizing psychological tests: Measurement equivalence and equivalent relations with external variables are the central issues. Psychological Bulletin, 95, 134-135.
15.
Drasgow, F. , & Hulin, C. L. (1990). Item response theory. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial & organizational psychology: Vol. 1 (2nd ed., pp. 577-636). Palo Alto, CA: Consulting Psychologists Press.
16.
Drasgow, F. , & Lissak, R. I. (1983). Modified parallel analysis: A procedure for examining the latent dimensionality of dichotomously scored item responses. Journal of Applied Psychology, 68, 363-373.
17.
Ellis, B. B. (1989). Differential item functioning: Implications for test translations. Journal of Applied Psychology, 74, 912-921.
18.
Ellis, B. B. , & Kimmel, H. D. (1992). Identification of unique cultural response patterns by means of item response theory. Journal of Applied Psychology, 77, 177-184.
19.
Ellis, B. B. , Minsel, B., & Becker, P. (1989). Evaluation of attitude survey translations: An investigation using item response theory. International Journal of Psychology, 24, 665-684.
20.
Flanagan, W. J. (1997). Measurement equivalence between high and average impression management groups: An IRT analysis of personality factors. Unpublished doctoral dissertation, Georgia Institute of Technology, Atlanta.
21.
Flanagan, W. J. , Raju, N. S., & Haygood, J. M. (1998, April). Impression management, measurement equivalence, and personality factors: Can IRT be used to determine the impact of faking. Paper presented at the “Improvements in Measurement: Applications of Item Response Theory” symposium conducted at the annual meeting of the Society for Industrial and Organizational Psychology, Dallas, TX.
22.
Fleer, P. F. (1993). A Monte Carlo assessment of a new measure of item and test bias. Unpublished doctoral dissertation, Illinois Institute of Technology.
23.
Fleer, P. F. , Raju, N. S., & van der Linden, W. J. (1995, April). A Monte Carlo assessment of DFIT with dichotomously-scored unidimensional tests. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.
24.
Flowers, C. P. , Oshima, T. C., & Raju, N. S. (1995, April). A Monte Carlo assessment of DFIT with dichotomously-scored unidimensional tests. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.
25.
Goldberg, L. R. (in press). The comparative validity of adult personality inventories. In S. R. Briggs, J. M. Cheeck, & E. M. Donahue (Eds.), Handbook of adult personality inventories. New York: Plenum.
26.
Hambleton, R. K. (1998, August). Translation and adaptation of psychological tests: Issues, research, designs, statistical approaches, and practical steps. Paper presented at the “Cross-Cultural Assessment and Test Adaptations” symposium conducted at the meeting of the International Congress of Applied Psychology, San Francisco.
27.
Hambleton, R. K. , & Kanjee, A. (1995). Increasing the validity of cross-cultural assessments: Use of improved methods for test adaptations. European Journal of Psychological Assessment, 11, 147-157.
28.
Holland, P. W. , & Thayer, D. T. (1988). Differential item functioning and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 129-145). Hillsdale, NJ: Lawrence Erlbaum.
29.
Hulin, C. L. , Drasgow, F., & Komocar, J. (1982). Applications of item response theory to analysis of scale translations. Journal of Applied Psychology, 67, 818-825.
30.
Hulin, C. L. , & Mayer, L. J. (1986). Psychometric equivalence of a translation of the Job Descriptive Index into Hebrew. Journal of Applied Psychology, 71, 83-94.
31.
Humphreys, L. G. (1986). Analysis and evaluation of test and item bias in the prediction context. Journal of Applied Psychology, 71, 327-333.
32.
Kim, S. H. , & Cohen, A. S. (1991). A comparison of two area measures for detecting differential item functioning. Applied Psychological Measurement, 15, 269-278.
33.
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
34.
Mershon, B. , & Gorsuch, R. L. (1988). Number of factors in the personality sphere. Journal of Personality and Social Psychology, 55, 675-680.
35.
Mislevy, R. J. , & Bock, R. D. (1990). BILOG 3: Item analysis and test scoring with binary logistic models [Computer program]. Chicago: Scientific Software, Inc.
36.
Raju, N. S. (1990). Determining the significance of estimated signed and unsigned areas between two item response functions. Applied Psychological Measurement, 14, 197-207.
37.
Raju, N. S. (1999). Some notes on the DFIT framework. Unpublished manuscript, Illinois Institute of Technology, Chicago.
38.
Raju, N. S. , van der Linden, W. J., & Fleer, P. F. (1992, April). An IRT-based internal measure of test bias with applications for differential item functioning. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.
39.
Raju, N. S. , van der Linden, W. J., & Fleer, P. F. (1995). IRT-based internal measure of differential functioning of items and tests. Applied Psychological Measurement, 19, 353-368.
40.
Roznowski, M. (1987). Use of tests manifesting sex differences as measures of intelligence: Implications for measurement bias. Journal of Applied Psychology, 72, 480-483.
41.
Roznowski, M. , & Reith, J. (1999). Examining the measurement quality of tests containing differentially functioning items: Do biased items result in poor measurement?Educational and Psychological Measurement, 59, 248-269.
42.
Segall, D. O. (1983). Test characteristic curves, item bias, and transformation to a common metric in item response theory: A methodological artifact with serious consequences and a simple solution. Unpublished manuscript, University of Illinois, Department of Psychology.
43.
Stocking, M. L. , & Lord, F. M. (1983). Developing a common metric in item response theory. Applied Psychological Measurement, 7, 201-210.
44.
Van de Vijver, F. , & Hambleton, R. K. (1996). Translating tests: Some practical guidelines. European Psychologist, 1, 89-99.
