Common methods variance often is a problem with psychological measures that require respondent self-reports of attitudes, beliefs, perceptions, and the like. The present study examined this problem by comparing multiple-item, Likert-type measures of psychological constructs to single-item, non-Likert-type measures of the same constructs. Using confirmatory factor analysis, the alternative forms were compared on criteria of methods variance and construct validity. Neither method appeared to be empirically better than the other. Unusual situations in which well-developed single-item measures might be appropriate are discussed.
Get full access to this article
View all access options for this article.
References
1.
Aguinis, H. (1995). Statistical power problems with moderated multiple regression in management research. Journal of Management, 21, 1141-1158.
2.
Bagozzi, R. P. , & Yi, Y (1990). Assessing method variance in multitrait-multimethod matrices: The case of self-reported affect and perceptions at work. Journal of Applied Psychology, 75, 547-560.
3.
Becker, T. E. , & Cote, J. A. (1994). Additive and multiplicative method effects in applied psychological research: An empirical assessment of three models. Journal of Management, 20, 625-642.
4.
Bentler, P. , & Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of co-variance structures. Psychological Bulletin, 88, 588-606.
5.
Campbell, D. T. , & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105.
6.
Cheung, K. C. , & Mooi, L. C. (1994). A comparison between the rating scale model and dual scaling for Likert scales. Applied Psychological Measurement, 18, 1-13.
7.
Gardner, D. G. , Dunham, R. B., Cummings, L. L., & Pierce, J. L. (1987). Focus of attention at work and leader-follower relations. Journal of Occupational Behavior, 8, 277-294.
8.
Gardner, D. G. , Dunham, R. B., Cummings, L. L., & Pierce, J. L. (1989). Focus of attention at work: Construct definition and empirical validation. Journal of Occupational Psychology, 62, 61-77.
9.
Hackman, J. R. , & Oldham, G. R. (1975). Development of the Job Diagnostic Survey. Journal of Applied Psychology, 60, 159-170.
10.
Harris, M. M. , & Bladen, A. (1994). Wording effects in the measurement of role conflicts and role ambiguity: A multitrait-multimethod analysis. Journal of Management, 20, 887-902.
11.
Harrison, D. A. , & McLaughlin, M. E. (1993). Cognitive processes in self-report responses: Test of item context effects in work attitude measures. Journal of Applied Psychology, 78, 129-140.
12.
Hinkin, T. R. (1995). A review of scale development practices in the study of organizations. Journal of Management, 21, 967-988.
13.
Jöreskog, K. G. , & Sörbom, D. (1993). LISREL 8: Structural equation modeling with the SIM-PLIS command language. Hillsdale, NJ: Lawrence Erlbaum.
14.
Jöreskog, K. G. , & Sörbom, D. (1989). Lisrel 7 user's reference guide. Mooresville, IN: Scientific Software Inc.
15.
Kenny, D. A. (1979). Correlation and causality. New York: John Wiley.
16.
Lance, C. E. , Teachout, M. S., & Donnelly, T. M. (1992). Specification of the criterion construct space: An application of hierarchical confirmatory factor analysis. Journal of Applied Psychology, 77, 437-452.
17.
Marsh, H. W. , & Hocevar, D. (1985). Applications of confirmatory factor analysis to the study of self-concept: First-and higher-order factor models and their invariance across groups. Psychological Bulletin, 103, 391-410.
18.
Morrow, P. (1983). Concept redundancy in organizational research: The case of work commitment. Academy of Management Review, 8, 486-500.
19.
Mulaik, S. A. , James, L. R., Van Alstine, J., Bennett, N., Lind, S., & Stilwell, C. D. (1989). Evaluation of goodness-of-fit indices for structural equation models. Psychological Bulletin, 105, 430-445.
20.
Nunnally, J. C. (1978). Psychometric theory. New York: McGraw-Hill.
21.
Podsakoff, P. M. , & Organ, D. W. (1986). Self-reports in organizational research: Problems and prospects. Journal of Management, 12, 531-544.
22.
Pryor, R.G.L. (1987). Differences among differences: In search of general work preference dimensions. Journal of Applied Psychology, 72, 426-438.
23.
Russell, C. J. , & Bobko, P. (1992). Moderated regression analysis and Likert scales: Too coarse for comfort?Journal of Applied Psychology, 77, 336-342.
24.
Russell, J. A. , Weiss, A., & Mendelsohn, G. A. (1989). Affect grid: A single-item scale of pleasure and arousal. Journal of Personality and Social Psychology, 57, 493-502.
25.
Scarpello, V. , & Campbell, J. P. (1983). Job satisfaction: Are all the parts there?Personnel Psychology, 36, 577-600.
26.
Schmitt, N. , Nason, E., Whitney, D. J., & Pulakos, E. D. (1995). The impact of method effects on structural parameters in validation research. Journal of Management, 21, 159-174.
27.
Schmitt, N. , & Stults, D. M. (1986). Methodology review: Analysis of multitrait-multimethod matrices. Applied Psychological Measurement, 10, 1-22.
28.
Siegall, M. , & McDonald, T. (1995). Focus of attention and reactions to job change. Journal of Applied Social Psychology, 25, 1121-1141.
29.
Smith, P. C. , Kendall, L. M., & Hulin, C. L. (1969). The measurement of satisfaction in work and retirement. Chicago: Rand McNally.
30.
Wanous, J. P. , Reichers, A. E., & Hudy, M. J. (1997). Overall job satisfaction: How good are single-item measures?Journal of Applied Psychology, 82, 247-252.
31.
Widaman, K. F. (1985). Hierarchically nested covariance structure models for multitrait-multimethod data. Applied Psychological Measurement, 9, 1-26.
32.
Williams, L. J. , & Anderson, S. E. (1994). An alternative approach to method effects by using latent-variable models: Applications in organizational behavior research. Journal of Applied Psychology, 79, 323-331.
33.
Williams, L. J. , & Brown, B. K. (1994). Method variance in organizational behavior and human resources research: Effects on correlations, path coefficients, and hypothesis testing. Organizational Behavior and Human Decision Processes, 57, 185-209.
34.
Williams, L. J. , Cote, J. A., & Buckley, M. R. (1989). Lack of method variance in self-reported affect and perceptions at work: Reality or artifact?Journal of Applied Psychology, 74, 462-468.
35.
Williams, L. J. , & James, L. R. (1995). Causal models in organizational behavior research: From path analysis to LISREL and beyond. In J. Greenberg (Ed.), Organizational behavior: The state of the science (pp. 181-205). Hillsdale, NJ: Lawrence Erlbaum.
36.
Wothke, W. (1996). Models for multitrait-multimethod matrix analysis. In G. A. Marcoulides & R. E. Schumaker (Eds.), Advanced structural equation modeling: Issues and techniques. Mahwah, NJ: Lawrence Erlbaum.
