A significant question in management research is, “What criteria should be used to evaluate the effects of leadership?” In this review, the authors systematically summarize various ways the field of leadership has (and has not) sought to answer questions about whether, when, and how leadership affects outcomes. A total of 1,161 empirical studies over 25 years, spanning micro- and macro-oriented perspectives, were content coded to answer six basic questions that set the scope of leadership science. The authors first descriptively summarize these criterion issues in the empirical literature and draw comparisons across areas (e.g., To what extent have leader-member exchange, transformational, and strategic leadership research differentially examined various outcomes?). Second, the authors explore the implications of criterion selection issues for the further advancement of leadership theory and offer concrete recommendations for future leadership research.
Atwater, L.E., & Yammarino, F.J.1992. Does self-other agreement on leadership perceptions moderate the validity of leadership and performance predictions? Personnel Psychology, 45: 141-164.
2.
Austin, J.T., & Villanova, P.1992. The criterion problem: 1917-1992. Journal of Applied Psychology, 77: 836-874.
3.
Avolio, B.J., Reichard, R.J., Hannah, S.T., Walumbwa, F.O., & Chan, A.2009. A meta-analytic review of leadership impact research: Experimental and quasi-experimental studies. The Leadership Quarterly, 20: 764-784.
4.
Avolio, B.J., Walumbwa, F.O., & Weber, T.J.2009. Leadership: Current theories, research, and future directions . Annual Review of Psychology, 60: 421-449.
5.
Avolio, B.J., Zhu, W., Koh, W., & Bhatia, P.2004. Transformational leadership and organizational commitment: Mediating role of psychological empowerment and moderating role of structural distance. Journal of Organizational Behavior, 25: 951-968.
6.
Barsade, S.G., Ward, A.J., Turner, J.D., & Sonnenfeld, J.A.2000. To your heart’s content: A model of affective diversity in top management teams. Administrative Science Quarterly, 45: 802-836.
7.
Bass, B.M.1996. A new paradigm of leadership: An inquiry into transformational leadership. Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.
Bass, B.M., & Yammarino, F.J.1991. Congruence of self and others’ leadership ratings of Naval officers for understanding successful performance . Applied Psychology: An International Review, 40: 437-454.
10.
Bauer, T.N., Erdogan, B., Liden, R.C., & Wayne, S.J.2006. A longitudinal study of the moderating role of extra-version: Leader-member exchange, performance, and turnover during new executive development . Journal of Applied Psychology, 91: 298-310.
11.
Bommer, W.H., Rich, G.A., & Rubin, R.S.2005. Changing attitudes about change: Longitudinal effects of transformational leader behavior on employee cynicism about organizational change. Journal of Organizational Behavior, 26: 733-753.
12.
Bono, J.E., & Ilies, R.2006. Charisma, positive emotions and mood contagion. The Leadership Quarterly, 17: 317-334.
13.
Bono, J.E., & Judge, T.A.2003. Self-concordance at work: Toward understanding the motivational effects of transformational leaders. Academy of Management Journal, 46: 554-571.
14.
Bono, J.E., & Judge, T.A.2004. Personality and transformational and transactional leadership: A meta-analysis. Journal of Applied Psychology, 89: 901-910.
15.
Bowers, D.G., & Seashore, S.E.1966. Predicting organizational effectiveness with a four-factor theory of leadership. Administrative Science Quarterly, 11: 238-263.
16.
Brown, D.J., & Lord, R.G.1999. The utility of experimental research in the study of transformational/charismatic leadership. The Leadership Quarterly, 10: 531-539.
17.
Brown, M.E., & Treviño, L.K.2006. Ethical leadership: A review and future directions. The Leadership Quarterly, 17: 595-616.
18.
Cable, D.M., & Judge, T.A.2003. Managers’ upward influence tactic strategies: The role of manager personality and supervisor leadership style. Journal of Organizational Behavior, 24: 197-214.
19.
Cameron, K.S., & Whetten, D.A.1983. Organizational effectiveness: One model or several? In K. S. Cameron & D. A. Whetten (Eds.), Organizational effectiveness: A comparison of multiple models: 1-20. Orlando, FL: Academic Press.
20.
Certo, S.T., Daily, C.M., Cannella, A.A., & Dalton, D.R.2003. Giving money to get money: How CEO stock options and CEO equity enhance IPO valuations. Academy of Management Journal , 46: 643-653.
21.
Chen, G., Kirkman, B.L., Kanfer, R., Allen, D., & Rosen, B.2007. A multilevel study of leadership, empowerment, and performance in teams. Journal of Applied Psychology, 92: 331-346.
22.
Chun, J.U., Yammarino, F.J., Dionne, S.D., Sosik, J.J., & Moon, H.K.2009. Leadership across hierarchical levels: Multiple levels of management and multiple levels of analysis. The Leadership Quarterly, 20: 689-707.
23.
Colquitt, J.2008. Publishing laboratory research in AMJ: A question of when, not if. Academy of Management Journal, 51: 616-620.
24.
Conger, J.A., Kanungo, R.N., & Menon, S.T.2000. Charismatic leadership and follower effects. Journal of Organizational Behavior, 21: 747-767.
25.
Daily, C.M., & Dalton, D.R.1995. CEO and director turnover in failing firms: An illusion of change? Strategic Management Journal, 16: 393-400.
26.
Day, D.V.2001. Assessment of leadership outcomes. In S. J. Zaccaro & R. J. Klimoski (Eds.), The nature of organizational leadership: Understanding the performance imperatives confronting today’s leaders: 384-410. San Francisco: Jossey-Bass .
27.
DeChurch, L.A., Hiller, N.J., Murase, T., & Doty, D. in press. Leadership across levels: Levels of leaders and their levels of impact. The Leadership Quarterly.
28.
DeChurch, L.A., & Marks, M.A.2006. Leadership in multiteam systems. Journal of Applied Psychology, 91: 311-329.
29.
De Cremer, D., & Tyler, T.R.2007. The effects of trust in authority and procedural fairness on cooperation. Journal of Applied Psychology, 92: 639-649.
30.
De Cremer, D., van Knippenberg, B., van Knippenberg, D., Mullenders, D., & Stinglhamber, F.2005. Rewarding leadership and fair procedures as determinants of self-esteem. Journal of Applied Psychology, 90: 3-12.
31.
De Hoogh, A.H.B., & Den Hartog, D.N.2009. Neuroticism and locus of control as moderators of the relationships of charismatic and autocratic leadership with burnout. Journal of Applied Psychology, 94: 1058-1067.
32.
Den Hartog, D.N., De Hoogh, A.H.B., & Keegan, A.E.2007. The interactive effects of belongingness and charisma on helping and compliance. Journal of Applied Psychology, 92: 1131-1139.
33.
Dvir, T., Eden, D., Avolio, B.J., & Shamir, B.2002. Impact of transformational leadership on follower development and performance: A field experiment. Academy of Management Journal, 45: 735-744.
34.
Edwards, J.R.2008. To prosper, organizational psychology should . . . overcome methodological barriers to progress. Journal of Organizational Behavior, 29: 469-491.
35.
Engle, E.M., & Lord, R.G.1997. Implicit theories, self-schemas, and leader-member exchange . Academy of Management Journal, 40: 988-1010.
36.
Ensley, M.D., & Pearce, C.L.2001. Shared cognition in top management teams: Implications for new venture performance. Journal of Organizational Behavior , 22: 145-160.
37.
Erez, A., LePine, J.A., & Elms, H.2002. Effects of rotated leadership and peer evaluation on the functioning and effectiveness of self-managed teams: A quasi-experiment. Personnel Psychology, 55: 929-948.
38.
Finkelstein, S., Hambrick, D.C., & Cannella, A.A.2009. Strategic leadership: Theory and research on executives, top management teams, and boards. New York: Oxford University Press.
39.
Foti, R.J., & Hauenstein, N.M.A.2007. Pattern and variable approaches in leadership emergence and effectiveness. Journal of Applied Psychology, 92: 347-355.
40.
Friedrich, T.L., Byrne, C.L., & Mumford, M.D.2009. Methodological and theoretical considerations in survey research. The Leadership Quarterly, 20: 57-60.
41.
George, J.M., & Bettenhausen, K.1990. Understanding prosocial behavior, sales performance, and turnover: A group-level analysis in a service context. Journal of Applied Psychology, 75: 698-709.
42.
Gerstner, C.R., & Day, D.V.1997. Meta-analytic review of leader-member exchange theory: Correlates and construct issues. Journal of Applied Psychology , 82: 827-844.
43.
Graen, G.B., & Uhl-Bien, M.1995. Relationship-based approach to leadership: Development of leader-member exchange (LMX) theory of leadership over 25 years: Applying a multi-level multi-domain perspective. The Leadership Quarterly , 6: 219-247.
44.
Greenberg, J.2007. A plea for methodological diversity. Journal of Organizational Behavior, 28: 929-931.
45.
Hambrick, D.C.2007. Upper echelons theory: An update. Academy of Management Review, 32: 334-343.
46.
Harvey, P., Stoner, J., Hochwarter, W., & Kacmar, C.2007. Coping with abusive supervision: The neutralizing effects of ingratiation and positive affect on negative employee outcomes. The Leadership Quarterly, 18: 264-280.
47.
Henderson, A.D., Miller, D., & Hambrick, D.C.2006. How quickly do CEOs become obsolete? Industry dynamism, CEO tenure, and company performance. Strategic Management Journal, 27: 447-460.
48.
Hiller, N.J., Day, D.V., & Vance, R.J.2006. Collective enactment of leadership roles and team effectiveness: A field study. The Leadership Quarterly, 17: 387-397.
49.
Hooper, D.T., & Martin, R.2008. Beyond personal leader-member exchange (LMX) quality: The effects of perceived LMX variability on employee reactions. The Leadership Quarterly, 19: 20-30.
50.
House, R.J., & Aditya, R.N.1997. The social scientific study of leadership: Quo vadis? Journal of Management, 23: 409-473.
51.
Ilies, R., Nahrgang, J.D., & Morgeson, F.P.2007. Leader-member exchange and citizenship behaviors: A meta-analysis. Journal of Applied Psychology, 92: 269-277.
52.
Jayaraman, N., Khorana, A., Nelling, E., & Covin, J.2000. CEO founder status and firm financial performance. Strategic Management Journal, 21: 1215-1224.
53.
Jenkins, W.O.1947. A review of leadership studies with particular reference to military problems. Psychological Bulletin, 44: 54-79.
54.
Judge, T.A., Bono, J.E., Ilies, R., & Gerhardt, M.W.2002. Personality and leadership: A qualitative and quantitative review. Journal of Applied Psychology, 87: 765-780.
55.
Judge, T.A., & Piccolo, R.F.2004. Transformational and transactional leadership: A meta-analytic test of their relative validity. Journal of Applied Psychology , 89: 755-768.
56.
Judge, T.A., Piccolo, R.F., & Ilies, R.2004. The forgotten ones? The validity of consideration and initiating structure in leadership research. Journal of Applied Psychology, 89: 36-51.
57.
Kahai, S.S., Sosik, J.J., & Avolio, B.J.2003. Effects of leadership style, anonymity, and rewards on creativity-relevant processes and outcomes in an electronic meeting system context. The Leadership Quarterly, 14: 499-524.
58.
Kaiser, R., Hogan, R., & Craig, S.2008. Leadership and the fate of organizations. American Psychologist, 63: 96-110.
59.
Kinicki, A.J., & Vecchio, R.P.1994. Influence on the quality of supervisor-subordinate relations: The role of time-pressure, organizational commitment, and locus of control . Journal of Organizational Behavior, 15: 75-82.
60.
Konovsky, M.A., & Cropanzano, R.1991. Perceived fairness of employee drug testing as a predictor of employee attitudes and job performance. Journal of Applied Psychology, 76: 698-707.
61.
Kor, Y.Y.2006. Direct and interaction effects of top management team and board compositions on R&D investment strategy. Strategic Management Journal, 27: 1081-1099.
62.
Lawrence, B.S.1997. The black box of organizational demography. Organization Science, 8: 1-22.
63.
LePine, J.A.2003. Team adaptation and post-change performance: Effects of team composition in terms of members’ cognitive ability and personality . Journal of Applied Psychology, 88: 27-39.
64.
LePine, J.A., Hollenbeck, J.R., Ilgen, D.R., & Hedlund, J.1997. Effects of individual differences on the performance of hierarchical decision-making teams: Much more than g. Journal of Applied Psychology, 82: 803-811.
65.
Liden, R.C., & Antonakis, J.2009. Considering context in psychological leadership research . Human Relations, 62: 1587-1605.
66.
Locke, E.A., & Latham, G.P.2004. What should we do about motivation theory? Six recommendations for the twenty-first century. Academy of Management Review , 29: 388-403.
67.
Lord, R.G., & Brown, D.J.2004. Leadership processes and follower self-identity. Mahwah, NJ: Lawrence Erlbaum.
68.
Lowe, K.B., Kroeck, K.G., & Sivasubramaniam, N.1996. Effectiveness correlates of transformational and transactional leadership: A meta-analytic review of the MLQ literature. The Leadership Quarterly, 7: 385-425.
69.
Mann, R.D.1959. A review of the relationships between personality and performance in small groups. Psychological Bulletin, 56: 241-270.
70.
Marks, M.A., Mathieu, J.E., & Zaccaro, S.J.2001. A temporally based framework and taxonomy of team processes . Academy of Management Review, 26: 356-376.
71.
Marks, M.A., Zaccaro, S.J., & Mathieu, J.E.2000. Performance implications of leader briefings and team-interaction training for team adaptation to novel environments. Journal of Applied Psychology, 85: 971-986.
72.
Mathieu, J.E., Gilson, L.L., & Ruddy, T.M.2006. Empowerment and team effectiveness: An empirical test of an integrated model. Journal of Applied Psychology, 91: 97-108.
73.
Mathieu, J.E., Marks, M.A., & Zaccaro, S.J.2001. Multiteam systems. In N. Anderson, D. S. Ones, H. K. Sinangil, & C. Viswesvaran (Eds.), Handbook of industrial, work and organizational psychology, volume 2: Organizational psychology: 289-313. Thousand Oaks, CA: Sage.
74.
McGee, J.E., Dowling, M.J., & Megginson, W.L.1995. Cooperative strategy and new venture performance: The role of business strategy and management experience . Strategic Management Journal, 16: 565-580.
75.
Miller, D., & Droge, C.1986. Psychological and traditional determinants of structure . Administrative Science Quarterly, 31: 539-560.
76.
Mitchell, T.R., & James, L.R.2001. Building better theory: Time and the specification of when things happen. Academy of Management Review, 26: 530-547.
77.
Morgeson, F.P.2005. The external leadership of self-managing teams: Intervening in the context of novel and disruptive events. Journal of Applied Psychology, 90: 497-508.
Nishii, L.H., & Mayer, D.M.2009. Do inclusive leaders help to reduce turnover in diverse groups? The moderating role of leader-member exchange in the diversity to turnover relationship. Journal of Applied Psychology, 94: 1412-1426.
80.
O’Driscoll, M.P., & Beehr, T.A.1994. Supervisor behaviors, role stressors and uncertainty as predictors of personal outcomes for subordinates. Journal of Organizational Behavior, 15: 141-155.
81.
Organ, D.W.1988. Organizational citizenship behavior: The good soldier syndrome. Lexington, MA: Lexington Books.
82.
Pearce, C.L., & Conger, J.A.2003. Shared leadership: Reframing the hows and whys of leadership . Thousand Oaks, CA: Sage.
83.
Piccolo, R.F., & Colquitt, J.A.2006. Transformational leadership and job behaviors: The mediating role of core job characteristics. Academy of Management Journal, 49: 327-340.
Ployhart, R.E., & Vandenberg, R.J.2010. Longitudinal research: The theory, design, and analysis of change. Journal of Management , 36: 94-120.
86.
Podsakoff, P.M., Bommer, W.H., Podsakoff, N.P., & MacKenzie, S.B.2006. Relationships between leader reward and punishment behavior and subordinate attitudes, perceptions, and behaviors: A meta-analytic review of existing and new research. Organizational Behavior and Human Decision Processes, 99: 113-142.
87.
Podsakoff, P.M., MacKenzie, S.B., Bachrach, D.G., & Podsakoff, N.P.2005. The influence of management journals in the 1980s and 1990s. Strategic Management Journal , 26: 473-488.
88.
Podsakoff, P.M., MacKenzie, S.B., Moorman, R.H., & Fetter, R.1990. Transformational leader behaviors and their effects on followers’ trust in leader, satisfaction, and organizational citizenship behaviors. The Leadership Quarterly, 1: 107-142.
89.
Rahim, M.A.1989. Relationships of leader power to compliance and satisfaction with supervision: Evidence from a national sample of managers. Journal of Management, 15: 545-556.
90.
Resick, C.J., Whitman, D.S., Weingarden, S.M., & Hiller, N.J.2009. The bright-side and the dark-side of CEO personality: Examining core self-evaluations, narcissism, transformational leadership, and strategic influence. Journal of Applied Psychology, 94: 1365-1381.
91.
Salvaggio, A.N., Schneider, B., Nishii, L.H., Mayer, D.M., Ramesh, A., & Lyon, J.S.2007. Manager personality, manager service quality orientation, and service climate: Test of a model. Journal of Applied Psychology , 92: 1741-1750.
92.
Scandura, T.A., & Williams, E.A.2000. Research methodology in management: Current practices, trends, and implications for future research. Academy of Management Journal, 43: 1248-1264.
93.
Schaubroeck, J., Walumbwa, F.O., Ganster, D.C., & Kepes, S.2007. Destructive leader traits and the neutralizing influence of an "enriched" job. The Leadership Quarterly, 18: 236-251.
94.
Seltzer, J., & Bass, B.M.1990. Transformational leadership: Beyond initiation and consideration . Journal of Management, 16: 693-703.
95.
Settoon, R.P., Bennett, N., & Liden, R.C.1996. Social exchange in organizations: Perceived organizational support, leader-member exchange, and employee reciprocity. Journal of Applied Psychology, 81: 219-227.
96.
Shadish, W.R., Cook, T.D., & Campbell, D.T.2002. Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.
97.
Shamir, B., Zakay, E., Breinin, E., & Popper, M.1998. Correlates of charismatic leader behavior in military units: Subordinates’ attitudes, unit characteristics, and superiors’ appraisals of leader performance. Academy of Management Journal , 41: 387-409.
98.
Sin, H.P., Nahrgang, J.D., & Morgeson, F.D.2009. Understanding why they don’t see eye to eye: An examination of leader-member exchange (LMX) agreement. Journal of Applied Psychology, 94: 1048-1057.
99.
Smith, C.A., Organ, D.W., & Near, J.P.1983. Organizational citizenship behavior: Its nature and antecedents . Journal of Applied Psychology, 68: 653-663.
100.
Smith-Jentsch, K.A., Salas, E., & Brannick, M.T.2001. To transfer or not to transfer? Investigating the combined effects of trainee characteristics, team leader support, and team climate. Journal of Applied Psychology, 86: 279-292.
101.
Sosik, J.J., & Godshalk, V.M.2000. Leadership styles, mentoring functions received, and job-related stress: A conceptual model and preliminary study. Journal of Organizational Behavior, 21: 365-390.
102.
Spector, P.E.2006. Industrial organizational psychology: Research and practice (4th ed.). New York: John Wiley.
103.
Staw, B.M.1995. Repairs on the road to relevance and rigor: Some unexplored issues in publishing organizational research. In L. L. Cummings & P. J. Frost (Eds.), Publishing in the organizational sciences: 85-97. London : Sage.
104.
Stogdill, R.M.1950. Leadership, membership and organization. Psychological Bulletin, 47: 1-14.
105.
Sundstrom, E., McIntyre, M., Halfhill, T., & Richards, H.2000. Work groups: From the Hawthorne studies to work teams of the 1990s and beyond. Group Dynamics: Theory, Research, and Practice, 4: 44-67.
106.
Sy, T., Côté, S., & Saavedra, R.2005. The contagious leader: Impact of the leader’s mood on the mood of group members, group affective tone, and group process. Journal of Applied Psychology, 90: 295-305.
107.
Tahai, A., & Meyer, M.J.1999. A revealed preference study of management journals’ direct influences. Strategic Management Journal, 20: 279-296.
108.
Tichy, N.M., & Sherman, S.1993. Control your destiny: Or someone else will. New York: Doubleday.
109.
Tierney, P., Farmer, S.M., & Graen, G.B.1999. An examination of leadership and employee creativity: The relevance of traits and relationships. Personnel Psychology , 52: 591-620.
110.
Tsui, A.S., & O’Reilly, C.A.1989. Beyond simple demographic effects: The importance of relational demography in superior-subordinate dyads . Academy of Management Journal, 32: 402-423.
111.
Uhl-Bien, M., & Maslyn, J.M.2003. Reciprocity in manager-subordinate relationships: Components, configurations, and outcomes. Journal of Management, 29: 511-532.
112.
Uyterhoeven, H.1989. General managers in the middle. Harvard Business Review, 67: 136-145.
113.
Van Dyne, L., Kamdar, D., & Joireman, J.2008. In-role perceptions buffer the negative impact of low LMX on helping and enhance the positive impact of high LMX on voice. Journal of Applied Psychology, 93: 1195-1207.
114.
van Knippenberg, B., & van Knippenberg, D.2005. Leader self-sacrifice and leadership effectiveness: The moderating role of leader prototypicality. Journal of Applied Psychology, 90: 25-37.
115.
Vermeulen, F.2005. On rigor and relevance: Fostering dialectic progress in management research. Academy of Management Journal, 48: 978-982.
116.
Walumbwa, F.O., Avolio, F.J., Gardner, W.L., Wernsing, T.S., & Peterson, S.J.2008. Authentic leadership: Development and validation of a theory-based measure. Journal of Management , 34: 89-126.
117.
Walumbwa, F.O., Avolio, F.J., & Zhu, W.2008. How transformational leadership weaves its influence on individual job performance: The role of identification and efficacy beliefs . Personnel Psychology, 61: 793-825.
118.
West, C.T., & Schwenk, C.R.1996. Top management team strategic consensus, demographic homogeneity and firm performance: A report of resounding nonfindings. Administrative Science Quarterly, 17: 571-576.
119.
Whittington, J.L., Goodwin, V.L., & Murray, B.2004. Transformational leadership, goal difficulty, and job design: Independent and interactive effects on employee outcomes. The Leadership Quarterly, 15: 593-606.
120.
Wofford, J.C., & Liska, L.Z.1993. Path-goal theories of leadership: A meta-analysis. Journal of Management, 19: 857-876.
121.
Yammarino, F.J., Dionne, S.D., Chun, J.U., & Dansereau, F.2005. Leadership and levels of analysis: A state-of-the-science review. The Leadership Quarterly, 16: 879-919.
122.
Yukl, G.2010. Leadership in organizations (7th ed.). Upper Saddle River, NJ: Pearson.
123.
Zaccaro, S.J., & Klimoski, R.J.2001. The nature of organizational leadership: An introduction. In S. J. Zaccaro & R. J. Klimoski (Eds.), The nature of organizational leadership: Understanding the performance imperatives confronting today’s leaders: 3-41. San Francisco : Jossey-Bass.
