Researchers often turn to post hoc statistical techniques to identify common method variance (CMV) in same source data and one viable option is to use a marker variable. The choice of marker variable is important, yet these variables are difficult to find, primarily because they must be theoretically unrelated to study variables but measured in the same way (e.g., perceptual; on a Likert scale). This manuscript uses scale development best practices to create a marker variable—attitude toward the color blue—that can be applied in a wide variety of social science research. Scale reliability and validity are addressed, discriminant validity with other measures that detect CMV is tested, and the Confirmatory Factor Analysis Marker Technique is applied with this scale. An experiment designed to analyze the effect of the placement of the scale in surveys is reported. Recommendations to researchers for use of this new scale to detect CMV are provided.
AguinisH.VillamorI.RamaniR. S. (2021). MTurk research: Review and recommendations. Journal of Management, 47(4), 823–837. https://doi.org/10.1177/0149206320969787
2.
AshkanasyN. M. (2008). Submitting your manuscript. Journal of Organizational Behavior, 29(3), 263–264. https://doi.org/10.1002/job.524
3.
BatemanT. S.CrantJ. M. (1993). The proactive component of organizational behavior. Journal of Organizational Behavior, 14(2), 103–118. https://doi.org/10.1002/job.4030140202
4.
BehrendT. S.SharekD. J.MeadeA. W.WiebeE. N. (2011). The viability of crowdsourcing for survey research. Behavior Research Methods, 43(3), 800–813. https://doi.org/10.3758/s13428-011-0081-0
5.
BemS. L. (1974). The measure of psychological androgyny. Journal of Consulting and Clinical Psychology, 42(2), 155–162. https://doi.org/10.1037/h0036215
BentlerP. M.WuE. J. (2002). EQS for Windows user’s guide. Multivariate Software.
8.
BerinskyA. J.HuberG. A.LenzG. S. (2012). Evaluating online labor markets for experimental research: Amazon.com’s Mechanical Turk. Political Analysis, 20(3), 351–368. https://doi.org/10.1093/pan/mpr057
9.
CastilleC.CrawfordW.SimmeringM. J. (2017). You gotta keep ‘em separated: The efficacy of proximal remedies for method variance. Paper Presented at the Southern Management Association Meeting, St. Pete Beach, FL.
10.
CampbellD. T.FiskeD. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81–105. https://doi.org/10.1037/h0046016
11.
ChanD. (2009). So why ask me? Are self-report data really that bad? In LanceC. E.VandenbergR. J. (Eds.), Statistical and methodological myths and urban legends (pp. 309–336). Routledge, Taylor, and Francis.
12.
ChangS. J.Van WitteloostuijnA.EdenL. (2010). From the editors: Common method variance in international business research. Journal of International Business Studies, 41, 178–184. https://doi.org/10.1057/jibs.2009.88
13.
ChinW. W.ThatcherJ. B.WrightR. T. (2012). Assessing common method bias: problems with the ULMC technique. MIS Quarterly, 36(3), 1003–1019. https://doi.org/10.2307/41703491
14.
ComreyA. L. (1973). A first course in factor analysis. Academic Press.
15.
CummingG. (2012). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. Routledge.
16.
DalalD. K.CarterN. T. (2015). Negatively worded items negatively impact survey research. In LanceC. E.VandenbergR. J. (Eds.), More statistical and methodological myths and urban legends (pp. 112–123). Routledge, Taylor, and Francis.
EschlemanK. J.BowlingN. A. (2011). A construct validation of the neutral objects satisfaction questionnaire (NOSQ). Journal of Business and Psychology, 26(4), 501–515. https://doi.org/10.1007/s10869-010-9206-1
19.
FullerB.Jr.MarlerL. E. (2009). Change driven by nature: A meta-analytic review of the proactive personality literature. Journal of Vocational Behavior, 75(3), 329–345. http://dx.doi.org/10.1016/j.jvb.2009.05.008
20.
FullerC. M.SimmeringM. J.AtincG.AtincY.BabinB. J. (2016). Common methods variance detection in business research. Journal of Business Research, 69(8), 3192–3198. https://doi.org/10.1016/j.jbusres.2015.12.008
21.
FullerJ. B.PattersonC. E.HesterK. I. M.StringerD. Y. (1996). A quantitative review of research on charismatic leadership. Psychological reports, 78(1), 271–287. https://doi.org/10.2466/pr0.1996.78.1.271
22.
GorsuchR. L. (1983). Factor analysis (2nd ed.). Erlbaum.
23.
HackmanJ. R.OldhamG. R. (1975). Development of the job diagnostic survey. Journal of Applied Psychology, 60(2), 159. https://doi.org/10.1037/h0076546
24.
HairJ. E.AndersonR. E.TathamR. L.BlackW. C. (1998). Multivariate data analysis (5th ed.). Prentice Hall.
25.
HensonR. K. (1999). Multivariate normality? What is it and how is it assessed. In ThompsonB. (Ed.), Advances in social science methodology (pp. 193–211). JAI Press.
26.
HinkinT. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1(1), 104–121. https://doi.org/10.1177/109442819800100106
27.
HuL. T.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
28.
HuangJ. L.CurranP. G.KeeneyJ.PoposkiE. M.DeShonR. P. (2012). Detecting and deterring insufficient effort responding to surveys. Journal of Business and Psychology, 27, 99–114. https://doi.org/10.1007/s10869-011-9231-8
29.
JohnsonR. E.RosenC. C.DjurdjevicE. (2011). Assessing the impact of common method variance on higher order multidimensional constructs. Journal of Applied Psychology, 96(4), 744–761. https://doi.org/10.1037/a0021504
JudgeT.A.BretzR.D. (1993). Report on an alternative measure of affective disposition. Educational and Psychological Measurement,53, 1095–1104. 10.1177/0013164493053004022
32.
KlineR. B. (2011). Principles and practice of structural equation modeling (3rd ed.). Guilford Press.
33.
LanceC. E.DawsonB.BirkelbachD.HoffmanB. J. (2010). Method effects, measurement error, and substantive conclusions. Organizational Research Methods, 13(3), 435–455. https://doi.org/10.1177/1094428109352528
34.
LindellM. K.WhitneyD. J. (2001). Accounting for common method variance in cross sectional research designs. Journal of Applied Psychology, 86(1), 114–121. https://doi.org/10.1037/0021-9010.86.1.114
35.
LovettM.BajabaS.LovettM.SimmeringM. J. (2018). Data quality from crowdsourced surveys: A mixed method inquiry into perceptions of Amazon’s Mechanical Turk Masters. Applied Psychology: An International Review, 67(2), 339–366. https://doi.org/10.1111/apps.12124
36.
MathieuJ. E.ZajacD. M. (1990). A review and meta-analysis of the antecedents, correlates, and consequences of organizational commitment. Psychological Bulletin, 108(2), 171. https://doi.org/10.1037/0033-2909.108.2.171
37.
MillerB. K.ChiodoB. (2008). Academic entitlement: Adapting the equity preference questionnaire for a university setting. Paper presented at the Southern Management Association Meeting, St. Pete Beach, FL.
38.
MowdayR. T.SteersR. M.PorterL. W. (1979). The measurement of organizational commitment. Journal of Vocational Behavior, 14(2), 224–247. https://doi.org/10.1016/0001-8791(79)90072-1
39.
MuthénL. K.MuthénB. (2018). Mplus. The comprehensive modelling program for applied researchers: User’s guide, 5. Muthén and Muthén.
40.
OsakweC. N.Palamidovska-SterjadovskaN.MihajlovM.Ciunova-ShuleskaA. (2020). Brand orientation, brand-building behavior and brand identity in SMEs: An empirical evaluation. Marketing Intelligence & Planning, 38(7), 813–828. https://doi.org/10.1108/MIP-07-2019-0370
41.
PaulhusD. L. (1984). Two-component models of socially desirable responding. Journal of Personality and Social Psychology, 46(3), 598–609. https://doi.org/10.1037/0022-3514.46.3.598
42.
PaulhusD. L. (1988). Manual for the balanced inventory of desirable responding (BIDR-6). University of British Columbia. Unpublished manual.
43.
PodsakoffM. P.OrganW. D. (1986). Self-reports in organizational research: Problems and prospects. Journal of Management, 12(4), 531–544. https://doi.org/10.1177/014920638601200408
44.
PodsakoffP. M.MacKenzieS. B.LeeJ.PodsakoffN. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903. https://doi.org/10.1037/0021-9010.88.5.879
45.
PodsakoffP. M.MacKenzieS. B.PodsakoffN. P. (2012). Sources of method bias in social science research and recommendations on how to control it. Annual Review of Psychology, 63, 539–569. https://doi.org/10.1146/annurev-psych-120710-100452
46.
RandrianasoloA.SemenovA.ArnoldM.ReynoldsK. (2020). A model of cultural intelligence and global identity. Journal of Consumer Marketing, 37(7), 821–832. https://doi.org/10.1108/JCM-05-2019-3244
47.
RichardsonH. A.SimmeringM. J.SturmanM. C. (2009). A tale of three perspectives: Examining post hoc statistical techniques for detection and correction of common method variance. Organizational Research Methods, 12(4), 762–800. https://doi.org/10.1177/1094428109332834
48.
SatorraA. (2000). Scaled and adjusted restricted tests in multi-sample analysis of moment structures. In HeijmansR. D. H.PollockD. S. G.SatorraA. (Eds.), Innovations in multivariate statistical analysis. A Festschrift for Heinz Neudecker (pp. 233–247). Kluwer Academic Publishers.
49.
SatorraA.BentlerP. M. (2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66(4), 507–514. https://doi.org/10.1007/bf02296192
50.
SchwarzN.BohnerG. (2001). The construction of attitudes. In TesserA.SchwarzN. (Eds.). Intrapersonal processes (pp. 436–457). Blackwell.
51.
SeibertS. E.KraimerM. L.CrantJ. M. (2001). What do proactive people do? A longitudinal model linking proactive personality and career success. Personnel Psychology, 54(4), 845–874. https://doi.org/10.1111/j.1744-6570.2001.tb00234.x
52.
SimmeringM. J.FullerC. M.RichardsonH. A.OcalY.AtincG. M. (2015). Marker variable choice, reporting, and interpretation in the detection of common method variance: A review and demonstration. Organizational Research Methods, 18(3), 473–511. https://doi.org/10.1177/1094428114560023
53.
SpectorP. E. (1987). Method variance as an artifact in self-reported affect and perceptions at work: Myth or significant problem?Journal of Applied Psychology, 72(3), 438–443. https://doi.org/10.1037/0021-9010.72.3.438
54.
SpectorP. E. (1994). Using self-report questionnaires in OB research: A comment on the use of a controversial method. Journal of Organizational Behavior, 15(5), 385–392. https://doi.org/10.1002/job.4030150503
55.
SpectorP. E. (2006). Method variance in organizational research: Truth or urban legend?Organizational Research Methods, 9(2), 221–232. https://doi.org/10.1177/1094428105284955
56.
SpectorP. E.BrannickM. T. (1995). The nature and effects of method variance in organizational research. In CooperC. L.RobertsonI. T. (Eds.). International review of industrial and organizational psychology (pp. 249–274). John Wiley.
57.
SpectorP. E.RosenC. C.RichardsonH. A.WilliamsL. J.JohnsonR. E. (2019). A new perspective on method variance: A measure-centric approach. Journal of Management, 45(3), 855–880. https://doi.org/10.1177/0149206316687295
58.
TabachnikB. G.FidellL. S. (2007). Using multivariate statistics (5th ed.). Pearson.
59.
TettR. P.MeyerJ. P. (1993). Job satisfaction, organizational commitment, turnover intention, and turnover: Path analyses based on meta-analytic findings. Personnel Psychology, 46(2), 259–293. https://doi.org/10.1111/j.1744-6570.1993.tb00874.x
60.
ThompsonB. (2004). Exploratory and confirmatory factor analysis: Understanding concepts and applications. American Psychological Association.
61.
ThompsonB.DanielL. (1996). Factor analytic evidence for the construct validity of scores: An historical overview and some guidelines. Educational and Psychological Measurement, 56(2), 197–208. https://doi.org/10.1177/0013164496056002001
62.
ThurstoneL. L. (1935). The vectors of the mind. University of Chicago Press.
63.
TourangeauR.CouperM. P.ConradF. G. (2013). “Up means good” the effect of screen position on evaluative ratings in web surveys. Public Opinion Quarterly, 77(S1), 69–88. https://doi.org/10.1093/poq/nfs063
64.
TourangeauR.RipsL.J., & RasinskiK. (2000). The psychology of survey response. Cambridge University Press.
65.
TrafimowD.MarksM. (2015). Editorial. Basic and Applied Social Psychology, 37(1): 1–2. 10.1080/01973533.2015.1012991
66.
VandenbergR. J. (2006). Introduction: Statistical and methodological myths and urban legends: Where, pray tell, did they get this idea?Organizational Research Methods, 9(2), 194–201. https://doi.org/10.1177/1094428105285506
67.
Van DyneL.LePineJ. A. (1998). Helping and voice extra-role behavior: Evidence of construct and predictive validity. Academy of Management Journal, 41(1), 108–119. https://doi.org/10.5465/256902
68.
WallA. E.FullerC. M.SimmeringM. J. (2015). Manipulating and measuring common method variance in same source data: An experiment. Paper Presented at the Academy of Management Meeting, Philadelphia, PA.
69.
WatsonD.ClarkL. A.TellegenA. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54(6), 1063–1107. https://doi.org/10.1037/0022-3514.54.6.1063
WertsC. E.LinnR. L.JoreskogK. G. (1974). Interclass reliability estimates: Testing structural assumptions. Educational and Psychological Measurement, 34(1), 25–33. https://doi.org/10.1177/001316447403400104
72.
WestS. G.FinchJ. D. F.CurranP. J. (1995). Structural equation models with nonnormal data. In HoyleR. H. (Ed.), Structural equation modeling (pp. 56–75). Sage.
73.
WilliamsL. J.AndersonS. E. (1994). An alternative approach to method effects by using latent-variable models: Applications in organizational behavior research. Journal of Applied Psychology, 79(3), 323–331. https://doi.org/10.1037/0021-9010.79.3.323
74.
WilliamsL. J.BrownB. K. (1994). Method variance in organizational behavior and human resources research: Effects on correlations, path coefficients, and hypothesis testing. Organizational Behavior and Human Decision Processes, 57(2), 185–209. https://doi.org/10.1006/obhd.1994.1011
75.
WilliamsL. J.CoteJ. A.BuckleyM. R. (1989). Lack of method variance in self-reported affect and perceptions at work: Reality or artifact?Journal of Applied Psychology, 74(3), 462–468. https://doi.org/10.1037/0021-9010.74.3.462
76.
WilliamsL. J.HartmanN.CavazotteF. (2010). Method variance and marker variables: A review and comprehensive CFA marker technique. Organizational Research Methods, 13, 477–514. https://doi.org/10.1177/1094428110366036
77.
WilliamsL. J.McGonagleA. K. (2016). Four research designs and a comprehensive analysis strategy for investigating common method variance with self-report measures using latent variables. Journal of Business and Psychology, 31(3), 339–359. https://doi.org/10.1007/s10869-015-9422-9
78.
WilliamsL. J.O’BoyleE. H. (2015). Ideal, nonideal, and no-marker variables: The confirmatory factor analysis (CFA) marker technique works when it matters. Journal of Applied Psychology, 100(5), 1579. https://doi.org/10.1037/a0038855
79.
WilsonT. D.HodgesS. D. (1992). Attitudes as temporary constructions. In L. L. Martin, & A. Tesser (Eds.). The construction of social judgments (pp. 37–65). Erlbaum.
