Using different measures of constructs in research to develop robust evidence of relationships and effects is seen as good methodological practice. This assumes these measures possess high convergent validity. However, proxies—alternative measures of the same construct—are rarely perfectly convergent. Although some convergence is preferred to none, this study demonstrates that even modest departures from perfect convergent validity can result in substantial differences in the magnitudes of findings, creating challenges for the accumulation and interpretation of research. Using data from published research, the authors find that substantial differences in findings between studies using desired and proxy variables occur even at levels of convergent validity as high as r = .85. Implications of using measures with less-than-ideal convergent validity for the interpretability of research results are examined. Convergent validities above r = .70 are recommended, whereas those below r = .50 should be avoided. Researchers are encouraged to develop and report convergent validity data.
AcquaahM. (2007). Managerial social capital, strategic orientation and organizational performance in an emerging economy. Strategic Management Journal, 28, 1235–1255.
2.
BakkerA. B.DemeroutiE.DollardM. F. (2008). How job demands affect partners' experience of exhaustion: Integrating work–family conflict and cross-over theory. Journal of Applied Psychology, 93, 901–911.
3.
BedeianA. G.SturmanM. C.StreinerD. L. (2009). Decimal dust, significant digits, and the search for stars. Organizational Research Methods, 12, 687–694.
4.
BercovitzJ.MitchellW. (2007). When is more better? The impact of business scale and scope on long-term business survival, while controlling for profitability. Strategic Management Journal, 28, 61–79.
5.
BommerW. H.DierdorffE. C.RubinR. S. (2007). Does prevalence mitigate relevance? The moderating effect of group-level OCB on employee performance. Academy of Management Journal, 50, 1481–1494.
6.
BommerW. H.JohnsonJ. L.RichG. A.PodsakoffP. M.MacKenzieS. B. (1995). One the interchangeability of objective and subjective measures of employee performance: A meta-analysis. Personnel Psychology, 48, 587–605.
7.
BowlingN. A.HendricksE. A.WagnerS. H. (2008). Positive and negative affectivity and facet satisfaction: A meta-analysis. Journal of Business Psychology, 23, 115–125.
8.
BoydB. K.GoveS.HittM. A. (2005). Consequences of measurement problems in strategic management research: The case of Ahmihud and Lev. Strategic Management Journal, 26, 367–375.
CampbellD. T.FiskeD. W. (1959). Convergent and discriminant validation by the multitrait–multimethod matrix. Psychological Bulletin, 56, 81–105.
11.
CarlsonK. D.JiF. X. (2011). Citing and building on meta-analytic findings: A review and recommendations. Organizational Research Methods, 14, 696–717.
12.
ChenW. R.MillerK. D. (2007). Situational and institutional determinants of firms' R&D search intensity. Strategic Management Journal, 28, 369–381.
13.
CombsJ. G.CrookT. R.ShookC. (2005). The dimensionality of organizational performance and its implications for strategic management research. In KetchenD.BerghD. (Eds.), Research methodology in strategy and management (pp. 259–286). San Diego, CA: Elsevier.
14.
EarleJ. S.SpicerA.PeterK. S. (2010). The normalization of deviant organizational practices: Wage arrears in Russia, 1991-1998. Academy of Management Journal, 53, 218–237.
15.
EdwardsJ. R.BagozziR. P. (2000). On the nature and relationship between constructs and measures. Psychological Methods, 5, 155–174.
16.
FeyC. F.FuruP. (2008). Top management incentive compensation and knowledge sharing in multinational corporations. Strategic Management Journal, 29, 1301–1324.
17.
FunderD. C.WestS. G. (1993). Consensus, self-other agreement, and accuracy in personality judgment. Journal of Personality and Social Psychology, 61, 457–476.
18.
GlassG.CollinsJ. (1970). Geometric proof of the restriction on the possible values of rxy when ryz and ryz are fixed. Educational and Psychological Measurement, 30, 37–39.
19.
GongY.LawK. S.ChangS.XinK. R. (2009). Human resource management and firm performance: The differential role of managerial affective and continuance commitment. Journal of Applied Psychology, 94, 263–275.
20.
GulerI.GuillenM. F. (2010). Home country networks and foreign expansion: Evidence from the venture capital industry. Academy of Management Journal, 53, 390–410.
21.
GuthrieJ. P. (2001). High-involvement work practices, turnover and productivity: Evidence from New Zealand. Academy of Management Journal, 44, 180–190.
22.
HunterJ. E.SchmidtF. L. (2004). . Methods of meta-analysis: Correcting error and bias in research findings. Thousand Oaks, CA: Sage.
23.
JacksonD. N. (1976). . Personality inventory manual. Goshen, NY: Research Psychologist Press.
24.
JudgeT. A.JacksonC. L.ShawJ. C.ScottB. A.RichB. L. (2007). Self-efficacy and work-related performance: The integral role of individual differences. Journal of Applied Psychology, 92, 107–127.
25.
KaleP.SinghH. (2007). Building firm capabilities through learning: The role of the alliance learning process in alliance capability and firm-level alliance success. Strategic Management Journal, 28, 981–1000.
26.
Kristof-BrownA. L.ZimmermanR. D.JohnsonE. C. (2005). Consequences of individual’s fit at work: A meta-analysis of person–job, person–organization, person–group, and person–supervisor fit. Personnel Psychology, 58, 281–342.
27.
Larraza-KintanaM.WisemanR. M.Gomez-MejiaL. R.WelbourneT. M. (2007). Disentangling compensation and employment risks using the behavioral agency model. Strategic Management Journal, 28, 1001–1019
28.
LeeG. K. (2007). The significance of network resources in the race to enter emerging product markets: The convergence of telephony communications and computer networking, 1989–2001. Strategic Management Journal, 28, 17–37.
29.
MaurerT. J.AndrewsK. D. (2000). Traditional, Likert, and simplified measures of self-efficacy. Educational and Psychological Measurement, 60, 965–973.
30.
MaurerT. J.PierceH. R. (1998). A comparison of Likert scale and traditional measures of self-efficacy. Journal of Applied Psychology, 83, 324–329.
31.
McDonaldM. L.WestphalJ. D. (2010). A little help here? Board control, CEO identification with the corporate elite, and strategic help provided to CEOs at other firms. Academy of Management Journal, 53, 343–370.
32.
MeehlP. E. (1990). Why summaries of research on psychological theories are often uninterpretable [Monograph]. Psychological Reports, 66, 195–244.
33.
MinerJ. B.RajuN. S. (2004). Risk propensity differences between managers and entrepreneurs and between low and high growth entrepreneurs: A reply in a more conservative vein. Journal of Applied Psychology, 89, 3–13.
34.
MurphyK. R.DeShonR. (2000). Interrater correlations do not estimate the reliability of job performance ratings. Personnel Psychology, 53, 873–900.
35.
NovickM. R. (1966). The axioms and principal results of classical test theory. Journal of Mathematical Psychology, 3, 1–18.
36.
NunnallyJ. C. (1967). Psychometric theory. New York, NY: McGraw-Hill.
37.
PodsakoffN. P.WhitingS. W.PodsakoffP. M.BlumeB. D. (2009). Individual- and organizational-level consequences of organizational citizenship behaviors: A meta-analysis. Journal of Applied Psychology, 94, 122–141.
38.
ScanduraT. A.WilliamsE. A. (2000). Research methodology in management: Current practice, trends and implications for future research. Academy of Management Journal, 43, 1248–1264.
39.
SchwabD. (1980). Construct validity in organizational behavior. Research in Organizational Behavior, 2, 3–43.
40.
ShortJ. C.KetchenD. J.Jr.ShookC. L.Ireland R. E. (2010). The concept of opportunity in entrepreneurship research: Past accomplishments and future challenges. Journal of Management,36, 40–65.
41.
SpanglerW. E. (1992). Validity of questionnaire and TAT measures of need for achievement: Two meta-analyses. Psychological Bulletin, 112, 140–154.
42.
StanleyJ.WangM. (1969). Restrictions on the possible values of r12, given r13 and r23. Educational and Psychological Measurement, 29, 579–581.
43.
StewartW. H.RothP. L. (2001). Risk propensity differences between entrepreneurs and managers: A meta-analytic review. Journal of Applied Psychology, 86, 145–153.
44.
StewartW. H.RothP. L. (2004). Data quality affects meta-analytic conclusions: A response to Miner and Raju (2004) concerning entrepreneurial risk propensity. Journal of Applied Psychology, 89, 14–21.
45.
TohS. M.MorgesonF. P.CampionM. A. (2008). Human resource configurations: Investigating fit with the organizational context. Journal of Applied Psychology, 93, 864–882.
46.
TraubR. E. (1994). Reliability for the social sciences: Theory and applications. (Vol. 3). Thousand Oaks, CA: Sage.
47.
WallT. D.MichieJ.PattersonM.WoodS. J.SheehanM.CleegC. W.WestM. (2004). On the validity of subjective measures of company performance. Personnel Psychology, 57, 95–118.
48.
WatsonD.WieseD.VaidyaJ.TellegenA. (1999). The two general activation systems of affect: Structural findings, evolutionary considerations, and psychobiological evidence. Journal of Personality and Social Psychology, 76, 820–838.
