The authors investigated the cross-cultural measurement equivalence of items in the English-language version of the NEO Personality Inventory, a measure of the five-factor personality model, in a different cultural context in which English is a language of instruction. The item responses of 432 Filipino and 610 American college students were analyzed. Three methods for detecting differential item functioning (DIF)-two based on item response theory-plus the Mantel-Haenszel method showed fairly good agreement in the detection of DIF items. Nearly 40% of the 180 items showed DIF. Several significant cultural mean differences that were found with the original raw scores were no longer significant when the authors analyzed "purified" scales obtained by deleting DIF items.
Get full access to this article
View all access options for this article.
References
1.
Ben-Porath, Y S. , Almagor, M., Hoffman-Chemi, A., & Tellegen, A. (1995). A cross-cultural study of personality with the Multidimensional Personality Questionnaire. Journal of Cross-cultural Psychology, 26, 360-373.
2.
Berk, R. A. (1982). Handbook of methods for detecting test bias. Baltimore: Johns Hopkins University Press.
3.
Bijnen, E. J. , Van Der Net, T.Z.J., & Poortinga, Y H. (1986). On cross-cultural comparative studies with the Eysenck Personality Questionnaire. Journal of Cross-cultural Psychology, 17, 3-16.
4.
Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee's ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test score. (chapters 17-20). Reading, MA: Addison-Wesley.
5.
Bjorgvinsson, T. , & Thompson, A. P. (1994). Psychometric properties of an Icelandic translation of the Basic Personality Inventory: Cross-cultural invariance of a three-factor solution. Personality and Individual Differences, 16, 47-56.
6.
Bontempo, R. (1993). Translation fidelity of psychological scales: An item response theory analysis of an individualism-collectivism scale. Journal of Cross-cultural Psychology, 24, 149-166.
7.
Borkenau, P. , & Ostendorf, F. (1990). Comparing exploratory and confirmatory factor analysis: A study on the 5-factor model of personality. Personality and Individual Differences, 11, 515-524.
8.
Bulatao, J. C. (1963). Personal preferences of Filipino students. Philippine Sociological Review, 11, 168-178.
9.
Bulatao, J. C. (1964). Hiya. Philippine Studies, 12, 424-438.
10.
Camilli, G. , & Shepard, L. A. (1994). Methods for identifying biased test items. Thousand Oaks, CA: Sage.
11.
Candell, G. L. , & Drasgow, F. (1988). An iterative procedure for linking metrics and assessing item bias in item response theory. Applied Psychological Measurement, 12, 253-260.
12.
Candell, G. L. , & Hulin, C. L. (1987). Cross-language and cross-cultural comparisons in scale translations: Independent sources of information about item equivalence. Journal of Cross-Cultural Psychology, 17,417-440.
13.
Church, A. T. (1987). Personality research in a non-Western culture: The Philippines. Psychological Bulletin, 102, 272-292.
14.
Church, A. T. , & Burke, P. J. (1994). Exploratory and confirmatory tests of the Big Five and Tellegen's three- and four-dimensional models. Journal of Personality and Social Psychology, 66,93-114.
15.
Church, A. T. , Reyes, J.A.S., Katigbak, M. S., & Grimm, S. D. (1995, August). Indigenous Filipino personality dimensions and the Big Five model. In 0. P. John (Chair), Culture, personality, and the Big Five. Symposium conducted at the 103rd Annual Convention of the American Psychological Association, New York.
16.
Clauser, B. E. , Mazor, K. M., & Hambleton, R. K. (1994). The effects of score group width on the Mantel-Haenszel procedure. Journal of Educational Measurement, 31, 67-78.
17.
Cohen, A. S. , & Kim, S-H. (1993). A comparison of Lord's x2 and Raju's area measures in detection of DIF. Applied Psychological Measurement, 17, 39-52.
18.
Costa, P. T., Jr. , & McCrae, R. R. (1985). The NEO Personality Inventory manual. Odessa, FL: Psychological Assessment Resources.
19.
Costa, P. T., Jr. , & McCrae, R. R. (1992). Revised NEO Personality Inventory and NEO Five-Factory Inventory (NEO-FFI). Odessa, FL: Psychological Assessment Resources.
20.
Costa, P. T., Jr. , McCrae, R. R., Bond, M. H., & Paunonen, S. V. (1994, August). Evaluating factor replicability with targeted rotation: A Chinese example. Paper presented at the 102nd Annual Convention of the American Psychological Association, Los Angeles.
21.
De Vera, M.V.M. (1986). Establishing cultural relevance and measurement equivalence using emic and etic items (Doctorl dissertation, University of Illinois at Urbana-Champaign, 1985). Dissertation Abstracts International, 46(07), 2485B.
22.
Devine, P. J. , & Raju, N. S. (1982). Extent of overlap among four item bias methods. Educational and Psychological Measurement, 42, 1049-1066.
23.
Digman, J. M. (1990). Personality structure: Emergence of the five-factor model. Annual Review of Psychology, 41, 417-440.
24.
Donoghue, J. R. , & Allen, N. L. (1993). Thin versus thick matching in the Mantel-Haenszel procedure for detecting DIF. Journal of Educational Statistics, 18, 131-154.
25.
Dorans, N. J. , & Holland, P. W. (1993). DIF detection and description: Mantel-Haenszel and standardization. In P. W. Holland & H. Wainer (Eds.), Differential item functioning. (pp. 35-66). Hillsdale, NJ: Lawrence Erlbaum.
26.
Ellis, B. B. (1990). Extension and evaluation of the Humphreysl Hulin emic item identification hypothesis using item response theory. In P.J.D. Drenth, J. A. Sergeant, & R. J. Takens (Eds.), European perspectives in psychology: Vol. I (pp. 43-55). New York: John Wiley.
27.
Ellis, B. B. , Becker, P., & Kimmnel, H. D. (1993). An item response theory evaluation of an English version of the Trier Personality Inventory (TPI). Journal of Cross-cultural Psychology, 24, 133-148.
28.
Ellis, B. B. , Minsel, B., & Becker, P. (1989). Evaluation of attitude survey translations: An investigation using item response theory. International Journal of Psychology, 24, 665-684.
29.
Engelhard, G. , Hansche, L., & Rudledge, K. E. (1990). Accuracy of bias review judges in identifying differential item functioning on teacher certification tests. Applied Measurement in Education, 3, 347-360.
30.
Eysenck, S.B.G. , Barrett, P. T., & Barnes, G. E. (1993). A cross-cultural study of personality: Canada and England. Personality and Individual Differences, 14, 1-9.
31.
Eysenck, S.B.G. , Makaremi, A., & Barrett, P. T. (1994). A cross-cultural study of personality: Iranian and English children. Personality and Individual Differences, 16, 203-211.
32.
Guthrie, G. M. , & Jacobs, P. J. (1966). Child rearing and personality development in the Philippines. University Park: Pennsylvania State University Press.
33.
Hambleton, R. K. , Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage.
34.
Hanisch, K. A. (1992). The Job Descriptive Index revisited: Questions about the question mark. Journal of Applied Psychology, 77, 377-382.
35.
Hofstede, C. (1980). Culture's consequences: International differences in work-related values. Beverly Hills, CA: Sage.
36.
Holland, P. W. , & Thayer, D. T. (1988). Differential item functioning and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity. (pp. 129-145). Hillsdale, NJ: Lawrence Erlbaum.
37.
Hoover, H. D. , & Kolen, M. J. (1984). The reliability of six item bias indices. Applied Psychological Measurement, 8, 173-181.
38.
Hui, C. H. , Drasgow, F., & Chang, B. H. (1983). Analysis of the modernity scale: An item response theory approach. Journal of Cross-cultural Psychology, 14, 259-278.
39.
Hui, C. H. , & Triandis, H. C. (1983). Multistrategy approach to cross-cultural research: The case of locus of control. Journal of Cross-cultural Psychology, 14, 65-83.
40.
Hui, C. H. , & Triandis, H. C. (1985). Measurement in cross-cultural psychology: A review and comparison of strategies. Journal of Cross-cultural Psychology, 16, 131-152.
41.
Hulin, C. L. (1987). A psychometric theory of evaluation of item and scale translations: Fidelity across languages. Journal of Cross-cultural Psychology, 18, 115-142.
42.
Hulin, C. L. , Drasgow, F., & Komocar, J. (1982). Application of item response theory to analysis of attitude scale translations. Journal of Applied Psychology, 67, 818-825.
43.
Hulin, C. L. , & Mayer, L. J. (1986). Psychometric equivalence of a translation of the Job Description Index into Hebrew. Journal of Applied Psychology, 71, 83-94.
44.
Ironson, G. H. , & Subkoviak, M. J. (1979). A comparison of several methods of assessing item bias. Journal of Educational Measurement, 16, 209-225.
45.
John, O. P. (1990). The "Big Five" factor taxonomy: Dimensions of personality in the natural language and in questionnaires. In L. A. Pervin (Ed.), Handbook of personality: Theory and research. (pp. 66-100). New York: Guilford.
46.
Katigbak, M. S. , Cb'irch, A. T., & Akamine, T. X. (1996). Cross-cultural generalizability of personality dimensions: Relating indigenous and imported dimensions in two cultures. Journal of Personality and Social Psychology, 70, 99-114.
47.
Kim, S-H. , & Cohen, A. S. (1991). IRTDIF: A computer program for IRT differential item functioning analysis. Madison: University of Wisconsin at Madison.
48.
Kim, S-H. , Cohen, A. S., & Kim, H-O. (1994). An investigation of Lord's procedure for the detection of differential item functioning. Applied Psychological Measurement, 18, 217-228.
49.
Kluckhohm, F , & Strodtbeck, F. (1973). Variations in value orientations. Westport, CT: Greenwood Press. (Original work published 1961)
50.
Lazo, L. S. (1976). Transplanting personality inventories. Philippine Journal of Psychology, 9, 35-37.
51.
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
52.
Lynch, F. (1973). Social acceptance reconsidered. In F. Lynch & A. de Guzman II (Eds.), Four readings on Philippine value. (4th ed., enlarged, Institute of Philippine Culture Papers No. 2, pp. 1-68). Quezon City, Philippines: Ateneo de Manila University Press.
53.
Mantel, N. , & Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22, 719-748.
54.
Mazor, K. M. , Clauser, B. E., & Hambleton, R. K. (1992). The effect of sample size on the functioning of the Mantel-Haenszel statistic. Educational and Psychological Measurement, 52,443-451.
55.
McCrae, R. R. , & Costa, P. T., Jr. (1985). Updating Norman's adequate taxonomy: Intelligence and personality in natural language and in questionnaires. Journal of Personality and Social Psychology, 49, 710-721.
56.
McCrae, R. R. , Costa, P. T., Jr., & Yik, M.S.M. (in press). Universal aspects of Chinese personality structure. In M. H. Bond (Ed.), The handbook of Chinese psychology. Hong Kong: Oxford University Press.
57.
Miller, M. D. , & Oshima, T. C. (1992). Effect of sample size, number of biased items, and magnitude of bias on a two-stage item bias estimation method. Applied Psychological Measurement, 16, 381-388.
58.
Miller, T. R. , & Spray, J. A. (1993). Logistic discriminant function analysis for DIF identification of polytomously scored items. Journal of Educational Measurement, 30, 107-122.
59.
Mislevy, R. J. , & Bock, R. D. (1990). BILOG: Item analysis and test scoring with binary logistic model. (2nd ed.). Mooresville, IN: Scientific Software.
60.
Montag, I. , & Levin, J. (1994). The five-factor personality model in applied settings. European Journal of Personality, 8, 1-11.
61.
Narayanan, R , & Swaminathan, H. (1995). Performance of the Mantel-Haenszel and simultaneous item bias procedures for detecting differential item functioning. Applied Psychological Measurement, 18, 315-328.
62.
Nunnally, J. C. (1967). Psychometric theory. New York: McGraw-Hill.
63.
Park, D. G. , & Lautenschlager, G. J. (1990). Improving IRT item bias detection with iterative linking and ability scale purification. Applied Psychological Measurement, 14, 163-173.
64.
Paunonen, S. V. , Jackson, D. N., Trzebinski, J., & Forsterling, F. (1992). Personality structure across cultures: A multimethod evaluation. Journal of Personality and Social Psychology, 62, 447-456.
65.
Perera, M. , & Eysenck, S.B.G. (1984). A cross-cultural study of personality: Sri Lanka and England. Journal of Cross-cultural Psychology, 15, 353-371.
66.
Plake, B. S. (1980). A comparison of a statistical and subjective procedure to ascertain item validity: One step in the validation process. Educational and Psychological Measurement, 40,397-404.
67.
Raju, N. S. (1988). The area between two item characteristic curves. Psychometrika, 53, 495-502.
68.
Raju, N. S. (1990). Determining the significance of estimated signed and unsigned areas between two item response functions. Applied Psychological Measurement, 14, 197-207.
69.
Raju, N. S. , Drasgow, F., & Slinde, J. A. (1993). An empirical comparison of the area methods, Lord's chi-square test, and the Mantel-Haenszel technique for assessing differential item functioning. Educational and Psychological Measurement, 53, 301-314.
70.
Raju, N. S. , & Normand, J. (1985). The regression bias method: A unified approach for detecting item bias and selection bias. Educational and Psychological Measurement, 45, 37-54.
71.
Ramos, E. C. (1977). Assessment of psychological testing in the Philippines: Focus on industries and national education. Philippine Journal of Psychology, 10, 19-22.
72.
Reise, S. P. , Widaman, K. F., & Pugh, R. H. (1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin, 114, 552-566.
73.
Rogers, H. J. , & Swaminathan, H. (1993). A comparison of logistic regression and Mantel-Haenszel procedures for detecting differential item functioning. Applied Psychological Measurement, 17, 105-116.
74.
Rudner, L. M. , Getson, P. R., & Knight, D. L. (1980). Biased items detection techniques. Journal of Educational Statistics, 5, 213-233.
75.
Sandoval, J. , & Miille, M.P.W. (1980). Accuracy of judgments of WISC-R item difficulty for minority groups. Journal of Consulting and Clinical Psychology, 48, 249-253.
76.
SAS Institute, Inc. (1985). SAS user's guide: Statistics. (version 5). Cary, NC: Author.
77.
Scheuneman, J. D. (1979). A method for assessing bias in test items. Journal of Educational Measurement, 16, 143-152.
78.
Shepard, L. , Camilli, G., & Averill, M. (1981). Comparison of procedures for detecting test-item bias with both internal and external ability criteria. Journal of Educational Statistics, 6, 317-375.
79.
Shepard, L. A. , Camilli, G., & Williams, D. M. (1985). Validity of approximation techniques for detecting item bias. Journal of Educational Measurement, 22, 77-105.
80.
Silva, F. , Avia, M. D., Sanz, J., & Grana, J. L. (1993, July). The five-factor model in Spain: Contributions to the structure of the NEO-PL. Paper presented at the Sixth Meeting of the International Society for the Study of Individual Differences, Baltimore.
81.
Skaggs, G. , & Lissitz, R. W. (1992). The consistency of detecting item bias across different test administrations: Implications of another failure. Journal of Educational Measurement, 29, 227-242.
82.
Smith, P. C. , Kendall, L. M., & Hulin, C. L. (1969). The measurement of satisfaction in work and retirement: A strategy for the study of attitudes. Chicago: Rand McNally.
83.
Stocking, M. L. , & Lord, F. M. (1983). Developing a common metric in item response theory. Applied Psychological Measurement, 7, 201-210.
84.
Swaminathan, H. , & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361-370.
85.
Thissen, D. , Steinberg, L., & Wainer, H. (1988). Use of item response theory in the study of group differences in trace lines. In H. Wainer & H. I. Braun (Eds.), Test validity. (pp. 147-169). Hillsdale, NJ: Lawrence Erlbaum.
86.
Thissen, D. , Steinberg, L., & Wainer, H. (1993). Detection of differential item functioning using the parameters of item response models. In P. W. Holland & H. Wainer (Eds.), Differential item functioning: Theory and practice. (pp. 67-113). Hillsdale, NJ: Lawrence Erlbaum.
87.
Triandis, H. C. (1993). Collectivism and individualism as cultural syndromes. Cross-cultural Research, 27, 155-180.
88.
Uttaro, T. , & Millsap, R. E. (1994). Factors influencing the Mantel-Haenszel procedure in the detection of differential item functioning. Applied Psychological Measurement, 18, 15-25.
89.
Zwick, R. (1990). When do item response function and Mantel-Haenszel definitions of differential item functioning coincide?Journal of Educational Statistics, 15, 185-197.
90.
Zwick, R. , Thayer, D. T., & Wingersky, M. (1994). A simulation study of methods for assessing differential item functioning in computerized adaptive test. Applied Psychological Measurement, 18, 121-140.
