A New Perspective on Method Variance: A Measure-Centric Approach

Abstract

A widespread methodological concern in the organizational literature is the possibility that observed results are due to the influence of common-method variance or mono-method bias. This concern is based on a conception of method variance as being produced by the nature of the method itself, and therefore, variables assessed with the same method would share common-method variance that inflates observed correlations. In this paper, we argue for a more complex view of method variance that consists of multiple sources that affect each measured variable in a potentially unique way. Shared sources among measures (common-method variance) act to inflate correlations, whereas unshared sources (uncommon-method variance) act to attenuate correlations. Two empirical examples, one from a simulation study and the other from a single-source survey, are presented to illustrate the complex action of multiple sources of method variance. A five-step approach is suggested whereby a theory of the measure is generated for each measured variable that would inform strategies to control for method variance by assessing and modeling the actions of identified method variance sources.

Keywords

common-method variance uncommon-method variance method bias psychometrics construct validity

Get full access to this article

View all access options for this article.

References

Alessandri

Vecchione

Donnellan

B. M.

Tisak

2013. An application of the LC-LSTM framework to the self-esteem instability case. Psychometrika, 78(4): 769-792.

Allen

M. J.

Yen

W. M.

1979. Introduction to measurement theory. Monterey, CA: Brooks/Cole.

Baumgartner

Steenkamp

J.-B. E.

2001. Response styles in marketing research: A cross-national investigation. Journal of Marketing Research, 38(2): 143-156.

Baumgartner

Weijters

2012. Commentary on “Common Method Bias in Marketing: Causes, Mechanisms, and Procedural Remedies.” Journal of Retailing, 88(4): 563-566.

Binning

J. F.

Barrett

G. V.

1989. Validity of personnel decisions: A conceptual analysis of the inferential and evidential bases. Journal of Applied Psychology, 74(3): 478-494.

Bowling

N. A.

Beehr

T. A.

2006. Workplace harassment from the victim’s perspective: A theoretical model and meta-analysis. Journal of Applied Psychology, 91(5): 998-1012.

Brannick

M. T.

Spector

P. E.

1990. Estimation problems in the block-diagonal model of the multitrait-multimethod matrix. Applied Psychological Measurement, 14(4): 325-339.

Campbell

D. T.

Fiske

D. W.

1959. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2): 81-105.

Carpenter

N. C.

Berry

C. M.

Houston

2014. A meta-analytic comparison of self-reported and other-reported organizational citizenship behavior. Journal of Organizational Behavior, 35(4): 547-574.

10.

Chen

P. Y.

Dai

Spector

P. E.

Jex

S. M.

1997. Relation between negative affectivity and positive affectivity: Effects of judged desirability of scale items and respondents’ social desirability. Journal of Personality Assessment, 69(1): 183-198.

11.

Crowne

D. P.

Marlowe

1964. The approval motive. New York: John Wiley.

12.

Doty

Glick

W. H.

1998. Common methods bias: Does common methods variance really bias results? Organizational Research Methods, 1(4): 374-406.

13.

Farh

J.-l.

Dobbins

G. H.

Cheng

B.-S.

1991. Cultural relativity in action: A comparison of self-rating made by Chinese and U.S. workers. Personnel Psychology, 44(1): 129-147.

14.

Fiske

D. W.

1982. Convergent-discriminant validation in measurements and research strategies. New Directions for Methodology of Social & Behavioral Science, 12: 77-92.

15.

Fiske

D. W.

1987a. Construct invalidity comes from method effects. Educational and Psychological Measurement, 47(2): 285-307.

16.

Fiske

D. W.

1987b. On understanding our methods and their effects. Diagnostica, 33(3): 188-194.

17.

Fried

Rowland

K. M.

Ferris

G. R.

1984. The physiological measurement of work stress: A critique. Personnel Psychology, 37(4): 583-615.

18.

Ganster

D. C.

Rosen

C. C.

2013. Work stress and employee health: A multidisciplinary review. Journal of Management, 39(5): 1085-1122.

19.

Hoge

Bussing

2004. The impact of sense of coherence and negative affectivity on the work stressor-strain relationship. Journal of Occupational Health Psychology, 9(3): 195-205.

20.

Johnson

R. E.

Rosen

C. C.

Djurdjevic

2011. Assessing the impact of common method variance on higher order multidimensional constructs. Journal of Applied Psychology, 96(4): 744-761.

21.

Johnson

Kulesa

Cho

Y. I.

Shavitt

2005. The relation between culture and response styles: Evidence from 19 countries. Journal of Cross-Cultural Psychology, 36(2): 264-277.

22.

Jöreskog

K. G.

Sörbom

1996. LISREL 8 user’s reference guide. Chicago: Scientific Software.

23.

Judge

T. A.

Bretz

R. D.

1993. Report on an alternative measure of affective disposition. Educational and Psychological Measurement, 53(4): 1095-1104.

24.

Levin

Stokes

J. P.

1989. Dispositional approach to job satisfaction: Role of negative affectivity. Journal of Applied Psychology, 74(5): 752-758.

25.

MacKenzie

S. B.

Podsakoff

P. M.

2012. Common method bias in marketing: Causes, mechanisms, and procedural remedies. Journal of Retailing, 88(4): 542-555.

26.

Meade

A. W.

Craig

S. B.

2012. Identifying careless responses in survey data. Psychological Methods, 17(3): 437-455.

27.

Moorman

R. H.

Podsakoff

P. M.

1992. A meta-analytic review and empirical test of the potential confounding effects of social desirability response sets in organizational behaviour research. Journal of Occupational and Organizational Psychology, 65(2): 131-149.

28.

Murphy

K. R.

Jako

R. A.

Anhalt

R. L.

1993. Nature and consequences of halo error: A critical analysis. Journal of Applied Psychology, 78(2): 218-225.

29.

Nunnally

J. C.

Bernstein

I. H.

1994. Psychometric theory. New York: McGraw Hill.

30.

Podsakoff

P. M.

MacKenzie

S. B.

Lee

J.-Y.

Podsakoff

N. P.

2003. Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5): 879-903.

31.

Podsakoff

P. M.

MacKenzie

S. B.

Podsakoff

N. P.

2012. Sources of method bias in social science research and recommendations on how to control it. Annual Review of Psychology, 63: 539-569.

32.

Richardson

H. A.

Simmering

M. J.

Sturman

M. C.

2009. A tale of three perspectives: Examining post hoc statistical techniques for detection and correction of common method variance. Organizational Research Methods, 12(4): 762-800.

33.

Rorer

L. G.

1965. The great response-style myth. Psychological Bulletin, 63(3): 129-156.

34.

Schmitt

1994. Method bias: The importance of theory and measurement. Journal of Organizational Behavior, 15(5): 393-398.

35.

Simmering

M. J.

Fuller

C. M.

Richardson

H. A.

Ocal

Atinc

G. M.

2015. Marker variable choice, reporting, and interpretation in the detection of common method variance: A review and demonstration. Organizational Research Methods, 18(3): 473-511.

36.

Soubelet

Salthouse

T. A.

2011. Influence of social desirability on age differences in self-reports of mood and personality. Journal of Personality, 79(4): 741-762.

37.

Spector

P. E.

1988. Development of the Work Locus of Control Scale. Journal of Occupational Psychology, 61(4): 335-340.

38.

Spector

P. E.

1992. A consideration of the validity and meaning of self-report measures of job conditions. In Cooper

C. L.

Robertson

I. T.

(Eds.), International review of industrial and organizational psychology 1992: 123-151. West Sussex, UK: Wiley.

39.

Spector

P. E.

Bauer

J. A.

Fox

2010. Measurement artifacts in the assessment of counterproductive work behavior and organizational citizenship behavior: Do we know what we think we know? Journal of Applied Psychology, 95(4): 781-790.

40.

Spector

P. E.

Brannick

M. T.

1995. The nature and effects of method variance in organizational research. In Cooper

C. L.

Robertson

I. T.

(Eds.), International review of industrial and organizational psychology 1995: 249-274. West Sussex, UK: Wiley.

41.

Spector

P. E.

Brannick

M. T.

2009. Common method variance or measurement bias? The problem and possible solutions. In Buchanan

Bryman

(Eds.), The Sage handbook of organizational research methods: 346-362. Thousand Oaks, CA: Sage.

42.

Spector

P. E.

Brannick

M. T.

2011. Methodological urban legends: The misuse of statistical control variables. Organizational Research Methods, 14(2): 287-305.

43.

Spector

P. E.

Dwyer

D. J.

Jex

S. M.

1988. Relation of job stressors to affective, health, and performance outcomes: A comparison of multiple data sources. Journal of Applied Psychology, 73(1): 11-19.

44.

Spector

P. E.

Jex

S. M.

1991. Relations of job characteristics from multiple data sources with employee affect, absence, turnover intentions, and health. Journal of Applied Psychology, 76(1): 46-53.

45.

Spector

P. E.

Liu

Sanchez

J. I.

2015. Methodological and substantive issues in conducting multinational and cross-cultural research. Annual Review of Organizational Psychology and Organizational Behavior, 2(1): 101-131.

46.

Spector

P. E.

Van Katwyk

P. T.

Brannick

M. T.

Chen

P. Y.

1997. When two factors don’t reflect two constructs: How item characteristics can produce artifactual factors. Journal of Management, 23(5): 659-677.

47.

Spector

P. E.

Zapf

Chen

P. Y.

Frese

2000. Why negative affectivity should not be controlled in job stress research: Don’t throw out the baby with the bath water. Journal of Organizational Behavior, 21(1): 79-95.

48.

Spector

P. E.

Zhou

Z. E.

2011. Individual differences in counterproductive work behavior: The interaction of gender and personality. Tampa: University of South Florida.

49.

Watson

Clark

L. A.

1984. Negative affectivity: The disposition to experience aversive emotional states. Psychological Bulletin, 96(3): 465-490.

50.

Watson

Pennebaker

J. W.

Folger

1986. Beyond negative affectivity: Measuring stress and satisfaction in the workplace. Journal of Organizational Behavior Management, 8(2): 141-157.

51.

Weiss

H. M.

Cropanzano

1996. Affective events theory: A theoretical discussion of the structure, causes and consequences of affective experiences at work. In Staw

B. M.

Cummings

L. L.

(Eds.), Research in organizational behavior: 1-74. Greenwich, CT: JAI Press.

52.

Williams

L. J.

Anderson

S. E.

1994. An alternative approach to method effects by using latent-variable models: Applications in organizational behavior research. Journal of Applied Psychology, 79(3): 323-331.

53.

Williams

L. J.

Brown

B. K.

1994. Method variance in organizational behavior and human resources research: Effects on correlations, path coefficients, and hypothesis testing. Organizational Behavior and Human Decision Processes, 57(2): 185-209.

54.

Williams

L. J.

Gavin

M. B.

Williams

M. L.

1996. Measurement and nonmeasurement processes with negative affectivity and employee attitudes. Journal of Applied Psychology, 81(1): 88-101.

55.

Williams

L. J.

McGonagle

A. K.

2016. Four research designs and a comprehensive analysis strategy for investigating common method variance with self-report measures using latent variables. Journal of Business and Psychology, 31(3): 339-359.

56.

Williams

L. J.

O’Boyle

E. H.

2015. Ideal, nonideal, and no-marker variables: The confirmatory factor analysis (CFA) marker technique works when it matters. Journal of Applied Psychology, 100(5): 1579-1602.