A key assumption underlying methods of construct validation is that constructs and their indicators are represented at the appropriate depth (i.e., the specificity versus generality of constructs and their indicators). This article presents a framework that depicts constructs and indicators at various depths and provides guidelines for choosing from among these depths. The framework is then integrated with methods of construct validation based on the confirmatory factor analysis of multitrait-multimethod (MTMM) matrices. The authors apply these methods to the measurement of work values, using the Work Aspect Preference Scale (WAPS). Results show that the WAPS performs better when used to represent relatively specific work values as opposed to more global, general values. Further analyses supported the generalizability of the WAPS factor structure for men and women, although gender differences were found on structured means for several latent value dimensions.
Get full access to this article
View all access options for this article.
References
1.
Allen, N. J.
, & Meyer, J. P. (1990). The measurement and antecedents of affective, continuance, and normative commitment to the organization. Journal of Occupational Psychology, 63, 1-18.
2.
Anderson, J. C.
, & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103, 411-423.
3.
Bagozzi, R. P.
(1993). Assessing construct validity in personality research: Applications to measures of self-esteem. Journal of Research in Personality, 27, 49-87.
4.
Bagozzi, R. P.
, & Heatherton, T. F. (1994). A general approach to representing multifaceted personality constructs: Application to state self-esteem. Structural Equation Modeling, 1, 35-67.
5.
Bagozzi, R. P.
, & Phillips, L. W. (1982). Representing and testing organizational theories: A holistic construal. Administrative Science Quarterly, 27, 459-489.
6.
Bagozzi, R. P.
, Yi, Y., & Nassen, K. D. (in press). Representation of measurement error in marketing variables: Review of approaches and extension to three-facet designs. Journal of Econometrics.
7.
Becker, T. E.
, & Vance, R. J. (1993). Construct validity of three types of organizational citizenship behavior: An Illustration of the direct product model with refinements. Journal of Management, 19, 663-682.
8.
Bentler, P. M.
(1989). EQS: Structural equations program manual [Computer program manual]. Los Angeles: BMDP Statistical Software.
9.
Bentler, P. M.
(1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238-246.
10.
Bentler, P. M.
, & Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88, 588-606.
11.
Bollen, K. A.
(1989). Structural equations with latent variables. New York: John Wiley.
12.
Bollen, K. A.
, & Ting, K. (1993). Confirmatory tetrad analysis. In P. V. Marsden (Ed.), Sociological methodology 1993 (pp. 147-175). Washington, DC: American Sociological Association.
13.
Browne, M. W.
(1984). The decomposition of multitrait-multimethod matrices. British Journal of Mathematical and Statistical Psychology, 37, 1-21.
14.
Browne, M. W.
(1989). Relationships between an additive model and a multiplicative model for multitrait-multimethod matrices. In R. Coppi & S. Bolasco (Eds.), Multiway data analysis (pp. 507-520). Amsterdam, The Netherlands: North-Holland.
15.
Browne, M. W.
(1990). MUTMUM pc user's guide. Unpublished manuscript, Ohio State University.
16.
Byrne, B. M.
(1994). Structural equation modeling with EQS and EQS/Windows. Thousand Oaks, CA: Sage.
17.
Campbell, D. T.
, & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105.
18.
Chusmir, L. H.
, & Parker, B. (1991). Gender and situational differences in managers' values: A look at work and home lives. Journal of Business Research, 23, 325-335.
19.
Chusmir, L. H.
, & Parker, B. (1992). Success strivings and their relationship to affective work behaviors: Gender differences. Journal of Social Psychology, 132, 87-99.
20.
Cook, T. D.
, & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field setting. Chicago: Rand McNally.
21.
Cudeck, R.
(1989). Analysis of correlation matrices using covariance structure models. Psychological Bulletin, 105, 317-327.
22.
Dawis, R. V.
(1991). Vocational interests, values, and preferences. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (Vol. 2, pp. 833-872). Palo Alto, CA: Consulting Psychologists Press.
23.
Eagly, A. H.
(1987). Sex differences in social behavior: A social-role interpretation. Hillsdale, NJ: Lawrence Erlbaum.
24.
Gati, I.
(1991). The structure of vocational interests. Psychological Bulletin, 109, 309-324.
25.
Gay, E. G.
, Weiss, D. J., Hendel, D. D., Dawis, R. V., & Lofquist, L. H. (1971). Manual for the Minnesota importance questionnaire. Minnesota Studies in Vocational Rehabilitation, 28, 1-83.
26.
Gerbing, D. W.
, & Anderson, J. C. (1988). An updated paradigm for scale development incorporating unidimensionality and its assessment. Journal of Marketing Research, 25, 186-192.
27.
Gerbing, D. W.
, & Anderson, J. C. (1992). Monte Carlo evaluations of goodness of fit indices for structural equation models. Sociological Methods & Research, 21, 132-160.
28.
Gleason, J.
(1989). Sex differences in parent-child interaction. In S. Philips, S. Steele, & C. Tanz (Eds.), Language, gender, and sex in comparative perspective (pp. 189-199). Cambridge, UK: Cambridge University Press.
29.
Hackman, J. R.
, & Oldham, G. R. (1975). Development of the job diagnostic survey. Journal of Applied Psychology, 60, 159-170.
30.
Hall, J. A.
(1987). On explaining gender differences: The case of nonverbal communication. Review of Personality and Social Psychology, 7, 177-200.
31.
Hattie, J.
(1985). Methodology review: Assessing unidimensionality of tests and items. Applied Psychological Measurement, 9, 139-164.
32.
Holland, J. L.
(1985). Making vocational choices (2nd ed.). Englewood Cliffs, NJ: Prentice Hall.
33.
House, R.
, Rousseau, D. M., & Thomas-Hunt, M. (1995). The meso paradigm: A framework for the integration of micro and macro organizational behavior. In L. L. Cummings & B. M. Staw (Eds.), Research in organizational behavior (Vol. 17, pp. 71-114). Greenwich, CT: JAI.
34.
Hull, J. G.
, Lehn, D. A., & Tedlie, J. C. (1991). A general approach to testing multifaceted personality constructs. Journal of Personality and Social Psychology, 61, 932-945.
35.
Ironson, G. H.
, Smith, P. C., Brannick, M. T., Gibson, W. M., & Paul, K. B. (1989). Construction of a job in general scale: A comparison of global, composite, and specific measures. Journal of Applied Psychology, 74, 193-200.
36.
James, L. R.
(1982). Aggregation bias in estimates of perceptual agreement. Journal of Applied Psychology, 67, 219-229.
37.
Jensen, T. D.
, White, D. D., & Singh, R. (1990). Impact of gender, hierarchical position, and leadership styles on work-related values. Journal of Business Research, 20, 145-152.
38.
Jöreskog, K. G.
, & Sörbom, D. (1989). LISREL7 user's reference guide (1st ed.). Mooresville, IN: Scientific Software.
39.
Jöreskog, K. G.
, & Sörbom, D. (1994). LISREL8: Structural equation modeling with the SIMPLIS command language. Chicago: Scientific Software.
40.
Judge, T. A.
, & Bretz, R. D. Jr. (1992). Effects of work values on job choice decisions. Journal of Applied Psychology, 77, 261-271.
41.
Kenny, D. A.
(1976). An empirical application of confirmatory factor analysis to the multitrait-multimethod matrix. Journal of Experimental Social Psychology, 12, 247-252.
42.
Klein, K. J.
, Dansereau, F., & Hall, R. J. (1994). Levels issues in theory development, data collection, and analysis. Academy of Management Review, 19, 195-229.
43.
Landy, F. J.
, & Farr, J. L. (1980). Performance rating. Psychological Bulletin, 87, 72-107.
44.
Lobel, S. A.
(1991). Allocation of investment in work and family roles: Alternative theories and implications for research. Academy of Management Review, 16, 507-521.
45.
Locke, E. A.
(1976). The nature and causes of job satisfaction. In M. Dunnette (Ed.), Handbook of industrial and organizational psychology (pp. 1297-1350). Chicago: Rand McNally.
46.
Macnab, D.
, & Fitzsimmons, G. W. (1987). A multitrait-multimethod study of work-related needs, values, and preferences. Journal of Vocational Behavior, 30, 1-15.
47.
Mael, F. A.
, & Ashforth, B. (1992). Alumni and their alma mater: A partial test of the reformulated model of organizational identification. Journal of Organizational Behavior, 13, 103-123.
48.
Marsh, H. W.
(1989). Confirmatory factor analyses of multitrait-multimethod data: Many problems and a few solutions. Applied Psychological Measurement, 13, 335-361.
49.
Marsh, H. W.
(1990). The structure of academic self-concept: The Marsh/Shavelson model. Journal of Educational Psychology, 82, 623-636.
50.
Marsh, H. W.
(1994). Confirmatory factor analysis models of factorial invariance: A multifaceted approach. Structural Equation Modeling, 1, 5-34.
51.
Marsh, H. W.
, & Bailey, M. (1991). Confirmatory factor analyses of multitrait-multimethod data: A comparison of the behavior of alternative models. Applied Psychological Measurement, 15, 47-70.
52.
Marsh, H. W.
, & Hocevar, D. (1985). Application of confirmatory factor analysis to the study of self-concept: First- and higher-order factor models and their invariance across groups. Psychological Bulletin, 97, 562-582.
53.
Marsh, H. W.
, & Hocevar, D. (1988). Anew, more powerful approach to multitrait-multimethod analyses: Application of second-order confirmatory factor analysis. Journal of Applied Psychology, 73, 107-117.
54.
McDonald, R. P.
, & Marsh, H. W. (1990). Choosing a multivariate model: Noncentrality and goodness of fit. Psychological Bulletin, 107, 247-255.
55.
Meredith, W.
(1993). Measurement invariance, factor analysis, and factorial invariance. Psychometrika, 58, 525-543.
56.
Mottaz, C.
(1986). Gender differences in work satisfaction, work-related rewards and values, and the determinants of work satisfaction. Human Relations, 39, 359-378.
57.
Mowday, R. T.
, Steers, R. M., & Porter, L. W. (1979). The measurement of organizational commitment. Journal of Vocational Behavior, 14, 224-247.
58.
Newcomb, M. D.
, & Bentler, P. M. (1988). Consequences of adolescent drug use. Newbury Park, CA: Sage.
59.
Nord, W. R.
, Brief, A. P., Atieh, J. M., & Doherty, E. M. (1988). Work values and the conduct of organizational behavior. In B. M. Staw & L. L. Cummings (Eds.), Research in organizational behavior (Vol. 10, pp. 1-42). Greenwich, CT: JAI.
60.
Organ, D. W.
, & Konovsky, M. (1989). Cognitive versus affective determinants of organizational citizenship behavior. Journal of Applied Psychology, 74, 157-164.
61.
Ostroff, C.
(1993). Comparing correlations based on individual-level and aggregated data. Journal of Applied Psychology, 78, 569-582.
62.
Paley, V. G.
(1984). Boys and girls: Superheroes in the doll corner. Chicago: University of Chicago Press.
63.
Pierce, J. L.
, McTavish, D. G., & Knudsen, K. R. (1986). The measurement of job characteristics: A content and contextual analytic look at scale validity. Journal of Occupational Behavior, 7, 299-314.
64.
Pryor, R.G.L.
(1983a). Manual for the Work Aspect Preference Scale. Melbourne: Australian Council for Education Research.
65.
Pryor, R.G.L.
(1983b). Sex differences in the levels of generality of values/preferences related to work. Journal of Vocational Behavior, 23, 233-241.
66.
Pryor, R.G.L.
(1987). Differences among differences: In search of general work preference dimensions. Journal of Applied Psychology, 72, 426-433.
67.
Ravlin, E. C.
, & Meglino, B. M. (1987). Effect of values on perception and decision making: A study of alternative work values measures. Journal of Applied Psychology, 72, 666-673.
68.
Ravlin, E. C.
, & Meglino, B. M. (1989). The transitivity of work values: Hierarchical preference ordering of socially desirable stimuli. Organizational Behavior and Human Decision Process, 44, 494-508.
69.
Robinson, J. P.
, Shaver, P. R., & Wrightsman, L. S. (Eds.). (1991). Measures of personality and social psychological attitudes. New York: Academic Press.
70.
Rosch, E.
(1978). Principles of categorization. In E. Rosch & B. B. Lloyd (Eds.), Cognition and categorization (pp. 27-71). Hillsdale, NJ: Lawrence Erlbaum.
71.
Rousseau, D. M.
(1985). Issues of level in organizational research: Multi-level and cross-level perspectives. In B. M. Staw & L. L. Cummings (Eds.), Research in organizational behavior (Vol. 7, pp. 1-37). Greenwich, CT: JAI.
72.
Schwab, D. P.
(1980). Construct validity in organizational behavior. In L. L. Cummings & B. M. Staw (Eds.), Research in organizational behavior (Vol. 2, pp. 3-43). Greenwich, CT: JAI.
73.
Schwartz, S. H.
, & Bilsky, W. (1987). Toward a universal psychological structure of human values. Journal of Personality and Social Psychology, 53, 550-562.
74.
Super, D. E.
(1970). Work values inventory manual. Boston: Houghton Mifflin.
75.
Super, D. E.
(1973). The work values inventory. In D. G. Zytowski (Ed.), Contemporary approaches to interest measurement. Minneapolis: University of Minnesota Press.
76.
Vance, R. J.
, MacCallum, R. C., Coovert, M. D., & Hedge, J. W. (1988). Construct validity of multiple job performance measures using confirmatory factor analysis. Journal of Applied Psychology, 73, 74-80.
77.
Vandenberg, R. J.
, & Self, R. M. (1993). Assessing newcomers' changing commitments to the organization during the first 6 months of work. Journal of Applied Psychology, 78, 557-568.
78.
Warr, P. B.
, Cook, J. D., & Wall, T. D. (1979). Scales for the measurement of some work attitudes and aspects of psychological well-being. Journal of Occupational Psychology, 52, 129-148.
79.
Widaman, K. E.
(1985). Hierarchically nested covariance structure models for multitrait-multimethod data. Applied Psychological Measurement, 9, 1-26.
80.
Williams, L. J.
, Cote, J. A., & Buckley, M. R. (1989). Lack of method variance in self-reported affect and perceptions at work: Reality or artifact?Journal of Applied Psychology, 74, 462-468.
81.
Wolins, L.
(1982). Research mistakes in the social and behavioral sciences. Ames: Iowa State University Press.
82.
Wothke, W.
(1996). Models for multitrait-multimethod matrix analysis. In G. A. Marcoulides & R. E. Schumacker (Eds.), Advanced structural equation modeling: Issues and techniques (pp. 7-56). Mahwah, NJ: Lawrence Erlbaum.
83.
Wothke, W.
, & Browne, M. W. (1990). The direct product model for the MTMM matrix parameterized as a second order factor analysis model. Psychometrika, 55, 255-262.
