Measurement artifacts, including measurement errors and scale-specific factors, distort observed correlations between measures of psychological and organizational constructs. The authors discuss two alternative procedures, one using the generalized coefficient of equivalence and stability (GCES) and one based on structural equation modeling, to correct for the biasing effect of measurement artifacts in order to estimate construct-level relationships. Assumptions underlying the procedures are discussed and the degrees of biases resulting from violating the assumptions are examined by means of Monte Carlo simulation. They then propose an approach using cumulative knowledge in the literature about properties of measures of a construct to estimate the GCES. That approach can allow researchers to estimate relationships between constructs in most research situations. The authors apply the approach to estimate the GCES for overall job satisfaction, an important organizational construct.
Bagozzi, R.P., & Yi, Y. (1990). Assessing method variance in multitrait-multimethod matrices: The case of self-reported affects and perceptions at work. Journal of Applied Psychology, 75, 547-560.
2.
Becker, G. (2000). How important is transient error in estimating reliability? Going beyond simulation studies. Psychological Methods, 5, 370-379.
3.
Becker, T.E., & Cote, J.A. (1994). Additive and multiplicative method effects in applied psychological research: An empirical assessment of three models. Journal of Management, 20, 625-641.
4.
Brayfield, A.H., & Rothe, H.F. (1951). An index of job satisfaction. Journal of Applied Psychology, 35, 307-311.
5.
Brennan, R.L. (2001). Generalizability theory. New York: Springer-Verlag.
6.
Brooke, P.P., Russell, D.W., & Price, J.L. (1988). Discriminant validation of measures of job satisfaction, job involvement, and organizational commitment. Journal of Applied Psychology, 73, 139-145.
7.
Campbell, D.T., & Fiske, M.R. (1959). Convergent and discriminant validation by the multitrait multimethod matrix. Psychological Bulletin, 56, 818-850.
8.
Campbell, J.P. (1982). Editorial: Some remarks from the outgoing editor . Journal of Applied Psychology, 67, 691-700.
Cohen, P., Cohen, J., Teresi, J., Marchi, M., & Velez, C.N. (1990). Problems in the measurement of latent variables in structural equations causal models. Applied Psychological Measurement, 14, 183-196.
11.
Conway, J.M. (1998). Estimation and uses of the proportion of method variance for multitrait-multimethod data. Organizational Research Methods, 1, 209-222.
12.
Cook, T.D. (1985). Postpositivist critical multiplism. In R. L. Shotland & M. M. Mark (Eds.), Social science and social policy (pp. 21-62). Newbury Park, CA: Sage.
13.
Cook, T.D., & Campbell, D.T. (1979). Quasi-experimentation: Design and analysis for field settings. Chicago: Rand McNally.
14.
Cook, J.D., Hepworth, S.J., Wall, T.D., & Warr, P.B. (1981). The experience of work: A compendium and review of 249 measures and their use. London: Academic Press.
15.
Cronbach, L.J. (1947). Test reliability: Its meaning and determination . Psychometrika, 12, 1-16.
16.
Cronbach, L.J., Gleser, G.C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York: John Wiley.
17.
DeShon, R.P. (1998). A cautionary note on measurement error corrections in structural equation models. Psychological Methods, 3, 412-423.
18.
Doty, H.D., & Glick, W.H. (1998). Common methods bias: Does common methods variance really bias results? Organizational Research Methods, 4, 374-406.
19.
Feldt, L.S., & Brennan, R.L. (1989). Reliability. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 105-146). New York: Macmillan .
20.
Fuller, W.A. (1987).Measurement error models. New York: John Wiley.
21.
Gerbing, D.W., & Anderson, J.C. (1984). On the meaning of within-factor correlated measurement errors. Journal of Consumer Research , 11, 572-580.
22.
Green, D.P., Goldman, S.L., & Salovey, P. (1993). Measurement error masks bipolarity in affect ratings . Journal of Personality and Social Psychology, 64, 1029-1041.
23.
Harris, M.M., & Bladen, A. (1994). Wording effects in the measurement of role conflict and role ambiguity: A multitrait-multimethod analysis. Journal of Management, 20, 887-901.
24.
Harrison, D.A., & Martocchio, J.J. (1998). Time for absenteeism: A 20-year review of origins, offshoots, and outcomes. Journal of Management, 24, 305-350.
25.
Hom, P.W., Caranikas-Walker, F., Prussia, G.E., & Griffeth, R.W. (1992). A meta-analytical structural equations analysis of a model of employee turnover. Journal of Applied Psychology, 77, 890-909.
26.
Hom, P.W., & Griffeth, R.W. (1991). Structural equations modeling test of a turnover theory: Cross-sectional and longitudinal analyses. Journal of Applied Psychology, 76, 350-366.
27.
Hoppock, R. (1935). Job satisfaction. New York : Harper Brothers.
28.
Houts, A.C., Cook, T.D., & Shadish, W.R. (1986). The person-situation debate: A critical multiplist perspective. Journal of Personality, 54, 52-105.
29.
Hunter, J.E., & Schmidt, F.L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage.
30.
Ilies, R., & Judge, T.A. (2004). An experience-sampling measure of job satisfaction and its relationships with affectivity, mood at work, job beliefs, and general job satisfaction. European Journal of Work and Organizational Psychology, 13, 367-389.
31.
Ironson, G.H., Smith, P., Brannick, M.T., & Gibson, W.M. (1989). Construction of a Job In General scale: A comparison of global, composite, & specific measures. Journal of Applied Psychology, 74, 193-200.
32.
James, L.R., Demaree, R.G., Mulaik, S.A., & Ladd, R.T. (1992). Validity generalization in the context of situational models. Journal of Applied Psychology, 77, 3-14.
33.
Joreskog, K.G. (1971). Statistical analysis of sets of congeneric tests . Psychometrika, 36, 109-133.
34.
Kenny, D.A. (1995). The multitrait-multimethod matrix: Design, analysis, and conceptual issues. In P. E. Shrout & S. T. Fiske (Eds.), Personality research, methods, and theory: A festschrift honoring Donald W. Fiske (pp. 111-124). Hillsdale, NJ: Lawrence Erlbaum.
35.
Kunin, T. (1955). The construction of a new type of job satisfaction measure. Personnel Psychology, 8, 65-77.
36.
Lance, C.E., Noble, C.L., & Scullen, S.E. (2002). A critique of the correlated trait correlated method and correlated uniqueness models for multitrait-multimethod data. Psychological Methods, 7, 228-244.
37.
Lord, F.M., & Novick, M.R. (1968). Statistical theories of test scores. Reading, MA: Addison-Wesley.
38.
Marsh, H.W. (1989). Confirmatory factor analysis of multitrait-multimethod data: Many problems and a few solutions.Applied Psychological Measurement, 13, 335-361.
39.
Marsh, H.W., & Grayson, D. (1994). Longitudinal confirmatory factor analysis: Common, time-specific, item-specific, and residual-error components of variance. Structural Equation Modeling, 1, 116-146.
40.
Marsh, H.W., & Hocevar, D. (1988). A new, more powerful approach to multitrait-multimethod analyses: Application of second-order confirmatory factor analysis. Journal of Applied Psychology, 73, 107-117.
41.
Mathieu, J.E., & Farr, J.L. (1991). Further evidence for the discriminant validity of measures of organizational commitment, job involvement, and job satisfaction . Journal of Applied Psychology, 76, 127-133.
42.
Meyer, J.P., Allen, N.J., & Smith, C.A. (1993). Commitment to organizations and occupations: Extension and test of a three-component conceptualization. Journal of Applied Psychology, 78, 538-551.
43.
Mowday, R., Steers, R., & Porter, L. (1979). The measurement of organizational commitment. Journal of Vocational Behavior, 14, 224-247.
44.
Nunnally, J.C., & Bernstein, I.H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.
45.
Peterson, R.A. (1994). A meta-analysis of Cronbach's coefficient alpha . Journal of Consumer Research, 21, 381-391.
46.
Price, J.L. (1997). Handbook of organizational measurement. International Journal of Manpower, 18, 303-558.
47.
Ree, M.J., & Carretta, T.R. (2006). The role of measurement error in familiar statistics. Organizational Research Methods , 9, 99-112.
48.
Rothstein, H.R. (1990). Interrater reliability of job performance ratings: Growth to asymptote level with increasing opportunity to observe. Journal of Applied Psychology, 75, 322-327.
49.
Schmidt, F.L., & Hunter, J.E. (1996). Measurement error in psychological research: Lessons from 26 research scenarios. Psychological Methods, 1, 199-223.
50.
Schmidt, F.L., & Hunter, J.E. (1999). Theory testing and measurement error. Intelligence, 27, 183-198.
51.
Schmidt, F.L., Le, H., & Ilies, R. (2003). Beyond alpha: An empirical examination of the effects of different sources of measurement error on reliability estimates for measures of individual difference constructs. Psychological Methods, 8, 206-224.
52.
Schmidt, F.L., Viswesvaran, C., & Ones, D.S. (2000). Reliability is not validity and validity is not reliability. Personality Psychology, 53, 901-912.
53.
Schmitt, M.J., & Steyer, R. (1993). A latent state-trait model (not only) for social desirability. Personality and Individual Differences, 14, 519-529.
54.
Shavelson, R.J., & Webb, N.M. (1991). Generalizability theory: A primer. Newbury Park, CA: Sage.
55.
Smith, P.C., Kendall, L.M., & Hulin, C.L. (1969). The measurement of satisfaction in work and retirement . Chicago: Rand McNally.
56.
Stanley, J.C. (1971). Reliability. In R. L. Thorndike (Ed.), Educational measurement (pp. 356-442). Washington, DC: American Council on Education.
57.
Thorndike, R.L. (1949). Personnel selection: Test and measurement techniques . New York: John Wiley.
58.
Thorndike, R.L. (1951). Reliability. In E. F. Lindquist (Ed.), Educational measurement (pp. 560-620). Washington, DC: American Council on Education.
59.
Vandenberg, R.J., & Lance, C.E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4-69.
60.
Viswesvaran, C., & Ones, D. (2000). Measurement error in `Big Five factors` personality assessment: Reliability generalization across studies and measures. Educational and Psychological Measurement, 60, 224-235.
61.
Weiss, D.J., Dawis, R.V., England, G.W., & Lofquist, L.H. (1967). Manual for the Minnesota Satisfaction Questionnaire. Minneapolis: University of Minnesota, Industrial Relations Center.
62.
Williams, L.J. (1995). Covariance structure modeling in organizational research: Problems with the method versus applications of the method. Journal of Organizational Behavior, 16, 225-233.
63.
Williams, L.J., Cote, J.A., & Buckley, M.R. (1989). Lack of method variance in self-reported affect and perceptions at work: Reality or artifact? Journal of Applied Psychology, 74, 462-468.
