Restricted accessResearch articleFirst published online 2005-08
Measurement Equivalence of the Job Descriptive Index Across Chinese and American Workers: Results from Confirmatory Factor Analysis and Item Response Theory
Despite increased awareness of practical issues in multinational data collection, few studies have addressed the issue of measurement equivalence across Western and Eastern cultures, especially using measures of job attitudes. Therefore, the measurement equivalence of the Job Descriptive Index (JDI) was examined across 2,638 Chinese workers and 1,664 American workers. Confirmatory factor analysis (CFA) and item response theory were used to ensure a comprehensive test of measurement equivalence at the facet level. A cross-sample validation procedure was also used with the CFA approach to increase the generalizability of the findings. The measurement equivalence of the JDI was established using both approaches. For human resource management practitioners, this study provides psychometric justification for direct comparisons of job satisfaction scores across Chinese and American employees of multinational companies.
Baker, F. B., Al-Karni, A., & Al-Dosary, I. M. (1992). EQUATE (Version 2.0) [Computer program]. Madison: University of Wisconsin.
2.
Balzer, W. K., Kihm, J. A., Smith, P. C., Irwin, J. L., Bachiochi, P. D., Robie, C., et al. (1997). Users’ manual for the Job Descriptive Index (JDI; 1997 revision) and the Job in General (JIG) scales. Bowling Green, OH: Bowling Green State University.
3.
Beamish, P. W. (1993). The characteristics of joint ventures in the People’s Republic of China. Journal of International Marketing, 1, 29-48.
4.
Bollen, K. A. (1989). Structural equations with latent variables. New York: John Wiley.
5.
Bollen, K. A. (1990). Overall fit in covariance structure models: Two types of sample size effects. Psychological Bulletin, 107, 256-259.
6.
Brannick, M. T. (1995). Critical comments on applying covariance structure modeling. Journal of Organizational Behavior, 16, 201-213.
7.
Browne M. W., & Cudeck R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136-162). London: Sage.
8.
Buckley, M. R., Carraher, S. M., & Cote, J. A. (1992). Measurement issues concerning the use of inventories of job-satisfaction. Educational and Psychological Measurement, 52, 529-543.
9.
Byrne, B. M. (1998). Structural equation modeling with LISREL, PRELIS, and SIMPLIS: Basic concepts, applications, and programming. Mahwah, NJ: Lawrence Erlbaum.
10.
Caligiuri, P., Phillips, J., Lazarova, M., Tarique, I., & Biiri, P. (2001). The theory of met expectations applied to expatriate adjustment: The role of cross-cultural training. International Journal of Human Resource Management, 12, 357-372.
11.
Candell, G. L., & Hulin, C. L. (1986). Cross-language and cross-cultural comparisons in scale translations: Independent sources of information about item nonequivalence. Journal of Cross-Cultural Psychology, 17, 417-440.
12.
Chan, D. (2000). Detection of differential item functioning on the Kirton Adaption-Innovation Inventory using multi-group mean and covariance structure analyses. Multivariate Behavioral Research, 35, 169-199.
13.
Chan, D., & Schmitt, N. (1997). Video-based versus paper-and-pencil method of assessment in situational judgment test: Subgroup differences in test performance and face validity perceptions. Journal of Applied Psychology, 82, 143-159.
14.
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233-255.
15.
Donovan, M. A., Drasgow, F., & Probst, T. M. (2000). Does computerizing paper-and-pencil job attitude scales make a difference? New IRT analyses offer insight. Journal of Applied Psychology, 85, 305-313.
16.
Drasgow, F., & Kanfer, R. (1985). Equivalence of psychological measurement in heterogeneous populations. Journal of Applied Psychology, 70, 662-680.
17.
Duane, M. J. (2003). Policies and practices in global human resource systems. Personnel Psychology, 56, 248-251.
18.
Ellis, B. B., & Mead, A. D. (2000). Assessment of the measurement equivalence of a Spanish translation of the 16PF Questionnaire. Educational and Psychological Measurement, 60, 787-807.
19.
Flowers, C. P., Oshima, T. C., & Raju, N. S. (1999). A description and demonstration of the polytomous-DFIT framework. Applied Psychological Measurement, 23, 309-326.
20.
Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory, Vol. 2. Newbury Park, CA: Sage.
21.
Hanisch, K. A. (1992). The Job Descriptive Index revisited: Questions about the question mark. Journal of Applied Psychology, 77, 377-382.
22.
Hofstede, G. (1991). Cultures and organizations: Software of the mind. London: McGraw-Hill.
23.
Hui, C. H., & Triandis, H. C. (1985). Measurement in cross-culture psychology: A review and comparison of strategies. Journal of Cross-Cultural Psychology, 16, 131-152.
24.
Hulin, C. L., Drasgow, F., & Komocar, J. (1982). Applications of item response theory to analysis of attitude scale translations. Journal of Applied Psychology, 67, 818-825.
25.
Ironson, G. H., Smith, P. C., Brannick, M. T., Gibson, W. M., & Paul, K. B. (1989). Construction of a “Job in General” scale: A comparison of global, composite, and specific measures. Journal of Applied Psychology, 74, 193-200.
26.
Jöreskog, K. G., & Sörbom, D. (1996). LISREL 8: User’s reference guide. Chicago: Scientific Software International.
27.
Jöreskog, K. G., & Sörbom, D. (2003). LISREL (Version 8.54) [Computer software]. Lincolnwood, IL: Scientific Software International.
28.
Katigbak, M. S., Church, A. T., & Akamine, T. X. (1996). Cross-culture generalizability of personality dimensions: Relating indigenous and imported dimensions in two cultures. Journal of Personality and Social Psychology, 70, 99-114.
29.
Kelloway, E. K. (1995). Structural equation modeling in perspective. Journal of Organizational Behavior, 16, 215-224.
30.
Kinicki, A. J., McKee-Ryan, F. M., Schriesheim, C. A., & Carson, K. P. (2002). Assessing the construct validity of the Job Descriptive Index: A review and meta-analysis. Journal of Applied Psychology, 87, 14-32.
31.
Leung, K., Wang, Z., & Smith, P. B. (2001). Job attitudes and organizational justice in joint venture hotels in China: The role of expatriate managers. International Journal of Human Resource Management, 12, 926-945.
32.
Liang, J. (2001). Assessing cross-cultural comparability in mental health among older adults. Journal of Mental Health and Aging, 7, 21-30.
33.
Lincoln, J. R., Hanada, M., & Olson J. (1981). Cultural orientations and individual reactions to organizations: A study of employees of Japanese-owned firms. Administrative Science Quarterly, 26, 93-115.
34.
Lytle A. L., Brett J. M., Barsness, Z. I., Tinsley, C. H., & Janssens, M. (1995). A paradigm for confirmatory cross-cultural research in organizational behavior. Research in Organizational Behavior, 17, 167-214.
35.
McDonald, R. P. (1997). Normal ogive multidimensional model. In W. J. Van der Linden & R. K. Hambleton (Eds.), Handbook of item response theory (pp. 258-269). New York: Spring-Verlag.
36.
McDonald, R. P. (2000). A basis for multidimensional item response theory. Applied Psychological Measurement, 24, 99-114.
37.
Meredith W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58, 525-543.
38.
Muraki, E., & Bock, R. D. (1997). PARSCALE: IRT based test scoring and item analysis for graded open-ended exercises and performance tasks. Chicago: Scientific Software International.
39.
Oshima, T. C., Raju, N. S., & Flowers, C. P. (1997). Development and demonstration of multidimensional IRT-based internal measures of differential functioning of items and tests. Journal of Educational Measurement, 34, 253-272.
40.
Pan, Y. (1997). Formation of Japanese and U.S. equity joint ventures in China. Strategic Management Journal, 18, 247-254.
41.
Raju, N. S. (1999). DFITPS6: Differential functioning of items and tests for the polytomous case (Samejima’s GRM) [Computer program]. Chicago: Illinois Institute of Technology.
42.
Raju, N. S., Laffitte, L. J., & Byrne, B. M. (2002). Measurement equivalence: A comparison of methods based on confirmatory factor analysis and item response theory. Journal of Applied Psychology, 87, 517-529.
43.
Raju, N. S., van der Linden, W., & Fleer, P. (1995). An IRT-based internal measure of test bias with applications for differential item functioning. Applied Psychological Measurement, 19, 353-368.
44.
Raykov, T., & Penev, S. (1998). Nested structural equation models: Noncentrality and power of restriction test. Structural Equation Modeling, 5, 229-246.
45.
Reckase, M. D. (1997). The past and future of multidimensional item response theory. Applied Psychological Measurement, 21, 25-36.
46.
Reise, S. P., Widaman, K. F., & Pugh, R. H. (1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement equivalence. Psychological Bulletin, 114, 552-566.
47.
Rensvold, R. B., & Cheung, G. W. (1998). Testing measurement model for factorial invariance: A systematic approach. Educational and Psychological Measurement, 58, 1017-1034.
48.
Rogelberg, S. G., Conway, J. M., Sederberg, M. E., Spitzmueller, C., Aziz, S., & Knight, W. E. (2003). Profiling active and passive nonrespondents to an organizational survey. Journal of Applied Psychology, 88, 1104-1114.
49.
Ryan, A. M., Chan, D., Ployhart, R. E., & Slade, L. A. (1999). Employee attitude surveys in a multinational organization: Considering language and culture in assessing measurement equivalence. Personnel Psychology, 52, 37-58.
50.
Ryan, A. M., Horvath, M., Ployhart, R. E., Schmitt, N., & Slade, L. A. (2000). Hypothesizing differential item functioning in global employee opinion surveys. Personnel Psychology, 53, 531-562.
51.
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph, 17, 1-100.
52.
Schafer, J. L. (1997). Analysis of incomplete multivariate data. Monographs on Statistics and Applied Probability, 72, 401-414.
53.
Selmer, J. (2001). Adjustment of Western European vs North American expatriate managers in China. Personnel Review, 30, 6-21.
54.
Smith, P. C., Kendall, L. M., & Hulin, C. L. (1969). Measurement of satisfaction in work and retirement. Chicago: Rand McNally.
55.
Smith, P. C., Smith, O. W., & Rollo, J. (1974). Factor structure for Blacks and Whites of the Job Descriptive Index and its discriminations of job satisfaction among three groups of subjects. Journal of Applied Psychology, 59, 99-100.
56.
Smith, P. C., & Stanton, J. M. (1998). Perspectives on the measurement of job attitudes: The long view. Human Resource Management Review, 8, 367-386.
57.
Spector, P. E., & Wimalasiri, J. (1986). A cross-cultural comparison of job satisfaction dimensions in the United States and Singapore. International Review of Applied Psychology, 35, 147-158.
58.
Stanton, J. M., Sinar, E. F., Balzer, W. K., Julian, A. L., Thoresen, P., Aziz, S., et al. (2001). Development of a compact measure of job satisfaction: The abridged Job Descriptive Index. Educational and Psychological Measurement, 61, 1104-1122.
59.
Stark, S., Chernyshenko, S., Chuah, D., Lee, W., & Wadlington, P. (2001). University of Illinois IRT Laboratory. Retrieved July 1, 2003, from http://work.psych.uiuc.edu/irt
60.
Teresi, J. A. (2001). Statistical methods for examination of differential item functioning (DIF) with applications to cross-cultural measurement of functional, physical and mental health. Journal of Mental Health and Aging, 7, 31-40.
61.
Thissen, D. (1991). MULTILOG user’s guide: Multiple, categorical item analysis and test scoring using item response theory (Version 6.0). Chicago: Scientific Software International.
62.
Thissen, D. (2001). IRTLRDIF v.2.0b: Software for the computation of the statistics involved in item response theory likelihood-ratio tests for differential item functioning. Unpublished manuscript, University of North Carolina, Chapel Hill.
63.
Thissen, D., Steinberg, L., & Wainer, H. (1988). Use of item response theory in the study of group differences in trace lines. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 147-169). Hillsdale, NJ: Lawrence Erlbaum.
64.
Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4-69.
65.
Wilson, D., Wood, R., & Gibbons, R. D. (1991). TESTFACT: Test scoring, item statistics, and item factor analysis. Mooresville, IN: Scientific Software International.
