Although interest in multilevel organizational theory, research, and methods has been on the rise in recent years, vigorous debates in the literature regarding appropriate ways to conceptualize and measure multilevel constructs, justify aggregation, and analyze multilevel models have contributed to confusion. New investigators interested in testing multilevel theory are intrigued, but wary. The goal of this article is to cut through the confusion, identifying the critical choices and issues a researcher may confront as he or she shifts from a single level to a multilevel perspective. The authors address four primary choices—construct and measurement issues, model specification, research design and sampling, and data analyses— describing critical steps in conceptualizing and conducting multilevel research.
Get full access to this article
View all access options for this article.
References
1.
Bandura, A. (1997). Self-efficacy: The exercise of control. New York: Freeman.
2.
Barry, B., & Stewart, G. L. (1997). Composition, process, and performance in self-managed groups: The role of personality. Journal of Applied Psychology, 82, 62-78.
3.
Bell, B. S., & Kozlowski, S.W.J. (in press). Virtual teams: Implications for leadership. Groups and Organization Management.
4.
Bliese, P. D. (2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In K. J. Klein & S.W.J. Kozlowski (Eds.), Multilevel Theory, Research, and Methods in Organizations (pp. 349-381). San Francisco: Jossey-Bass.
5.
Bliese, P. D. & Halverson, R. R. (1998). Group consensus and psychological well-being: A large field study. Journal of Applied Social Psychology, 28, 563-580.
6.
Bloom, M. (1999). The performance effects of pay dispersion on individuals and organizations. Academy of Management Journal, 42, 25-40.
7.
Brass, D. J., & Burkhardt, M. E. (1993). Potential power and power use: An investigation of structure and behavior. Academy of Management Journal, 36, 441-470.
8.
Brown, K. G., & Kozlowski, S.W.J. (1999, April). Toward an expanded conceptualization of emergent organizational phenomena: Dispersion theory. In F. P. Morgeson & D. A. Hofmann, (Chairs), New perspectives on higher-level phenomena in industrial/organizational psychology. Symposium conducted at the 14th Annual Conference of the Society for Industrial and Organizational Psychology, Atlanta, GA.
9.
Brown, K. G., Kozlowski, S.W.J., & Hattrup, K. (1996, August). Theory, issues, and recommendations in conceptualizing agreement as a construct in organizational research: The search for consensus regarding consensus. In S. Kozlowski & K. Klein (Chairs), The meaning and measurement of within-group agreement in multi-level research. Symposium conducted at the 56th Annual Convention of the Academy of Management Association, Cincinnati, OH.
10.
Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear models: Applications and data analysis methods. Newbury Park, CA: Sage.
11.
Chan, D. (1998). Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Journal of Applied Psychology, 83, 234-246.
12.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.
13.
Dansereau, F., Alutto, J. A., & Yammarino, F. J. (1984). Theory testing in organizational behavior: The variant approach. Englewood Cliffs, NJ: Prentice-Hall.
14.
Dansereau, F., & Yammarino, F. J. (1998). Introduction and overview. In F. Dansereau & F. J. Yammarino (Eds.), Leadership: The Multiple-Level Approaches (pp. xxv-xliii). Greenwich, CT: JAI Press.
15.
Dansereau, F., & Yammarino, F. J. (2000). Within and between analysis: The variant paradigm as an underlying approach to theory building and testing. In K. J. Klein & S.W.J. Kozlowski (Eds.), Multilevel Theory, Research, and Methods in Organizations (pp. 425-466). San Francisco: Jossey Bass.
16.
DeShon, R. P., Milner, K. R., Kozlowski, S.W.J., Toney, R. J., Schmidt, A., Wiechmann, D., & Davis, C. (1999, April). The effects of team goal orientation on individual and team performance. In D. Steele-Johnson (Chair), New directions in goal orientation research: Extending the construct, the nomological net, and analytic methods. Symposium conducted at the 14th Annual Conference of the Society for Industrial and Organizational Psychology, Atlanta, GA.
17.
Firebaugh, G. (1979). Assessing group effects: A comparison of two methods. Sociological Methods & Research, 7, 384-395.
18.
George, J. M. (1990). Personality, affect, and behavior in groups. Journal of Applied Psychology, 75, 107-116.
19.
George, J. M., & James, L. R. (1993). Personality, affect, and behavior in groups revisited: Comment on appregation, levels of analysis, and a recent application of within and between analysis. Journal of Applied Psychology, 78, 798-804.
20.
Hofmann, D. A., Griffin, M., & Gavin, M. (2000). The application of hierarchical linear modeling to organizational research. In K. J. Klein & S.W.J. Kozlowski (Eds.), Multilevel Theory, Research, and Methods in Organizations (pp. 467-511). San Francisco: Jossey Bass.
21.
Hofmann, D. A., & Stetzer, A. (1996). A cross level investigation of factors influencing unsafe behavior and accidents. Personnel Psychology, 49, 307-339.
22.
House, R., Rousseau, D. M., & Thomas-Hunt, M. (1995). The meso paradigm: A framework for integration of micro and macro organizational. In L. L. Cummings & B. Staw (Eds.), Research in organizational behavior (Vol. 17, pp. 71-114). Greenwich, CT: JAI.
23.
James, L. J., & Williams, L. (2000). The cross-level operator in regression, ANCOVA, and contextual analysis. In K.J. Klein & S.W.J. Kozlowski (Eds.), Multilevel Theory, Research, and Methods in Organizations (pp. 382-424). San Francisco: Jossey Bass.
24.
James, L. R. (1982). Aggregation bias in estimates of perceptual agreement. Journal of Applied Psychology, 67, 2l9-229.
25.
James, L. R., Demaree, R. G., & Wolf, G. (1984). Estimating within group interrater reliability with and without response bias. Journal of Applied Psychology, 69, 85-98.
26.
James, L. R., & Jones, A. P. (1974). Organizational climate: A review of theory and research. Psychological Bulletin, 81, 1096-1112.
27.
Klein, K. J., Bliese, P. D., Kozlowski, S.W.J., Dansereau, F., Gavin, M. B., Griffin, M. A., Hofmann, D. A., James, L. R., Yammarino, F. J., & Bligh, M. C. (2000). Multilevel analytical techniques: Commonalities, differences, and continuing questions. In K. J. Klein & S.W.J. Kozlowski (Eds.), Multilevel theory, research and methods in organizations: Foundations, extensions, and new directions (pp. 512-553). San Francisco, CA: Jossey-Bass.
28.
Klein, K. J., Cannella, A., & Tosi, H. (1999). Multilevel theory: Challenges and contributions. Academy of Management Review, 24, 243-248.
29.
Klein, K. J., Conn, A. B., Smith, D. B., & Sorra, J. S. (in press). Is everyone in agreement? An exploration of within-group agreement in employee perceptions of the work environment. Journal of Applied Psychology.
30.
Klein, K. J., Dansereau, F., & Hall, R. J. (1994). Levels issues in theory development, data collection, and analysis. Academy of Management Review, 19, 195-229.
31.
Kozlowski, S.W.J., & Doherty, M. L. (1989). Integration of climate and leadership: Examination of a neglected issue. Journal of Applied Psychology, 74, 546-553.
32.
Kozlowski, S.W.J., Gully, S. M., Nason, E. R., & Smith, E. M. (1999). Developing adaptive teams: A theory of compilation and performance across levels and time. In D. R. Ilgen & E. D. Pulakos (Eds.), The changing nature of work performance: Implications for staffing, personnel actions, and development (pp. 240-292). San Francisco: Jossey-Bass.
33.
Kozlowski, S.W.J., & Hattrup, K. (1992). A disagreement about within-group agreement: Disentangling issues of consistency versus consensus. Journal of Applied Psychology, 77, 161-167.
34.
Kozlowski, S.W.J., & Hults, B. M. (1987). An exploration of climates for technical updating and performance. Personnel Psychology, 40, 539-563.
35.
Kozlowski, S.W.J., & Klein, K. J. (2000). A multilevel approach to theory and research in organizations: Contextual, temporal, and emergent processes. In K. J. Klein & S.W.J. Kozlowski (Eds.), Multilevel theory, research and methods in organizations: Foundations, extensions, and new directions (pp. 3-90). San Francisco, CA: Jossey-Bass.
36.
Kreft, I., & de Leeuw, J. (1998). Introducing multilevel modeling. Thousand Oaks, CA: Sage.
37.
Lindsley, D. H., Brass, D. J., & Thomas, J. B. (1995). Efficacy-performance spirals: A multilevel perspective. Academy of Management Review, 20, 645-678.
38.
Ostroff, C. (1993). Comparing correlations based on individual level and aggregate data. Journal of Applied Psychology, 78, 569-582.
39.
Rentsch, J. R. (1990). Climate and culture: Interaction and qualitative differences in organizational meanings. Journal of Applied Psychology, 75, 668-681.
40.
Robinson, W. S. (1950). Ecological correlations and the behavior of individuals. American Sociological Review, 15, 351-357.
41.
Schneider, B., & Reichers, A. E. (1983). On the etiology of climates. Personnel Psychology, 36, 19-39.
42.
Staw, B., Sandelands, L. E., & Dutton, J. E. (1981). Threat-rigidity effects in organizational behavior: A multilevel analysis. Administrative Science Quarterly, 26, 501-524.
43.
Thorndike, E. L. (1939). On the fallacy of imputing the correlations found for groups to the individuals or smaller groups composing them. American Journal of Psychology, 52, 122-124.
44.
Tsui, A. S., Egan, T. D., & O’Reilly, C.A. (1992). Being different: Relational demography and organizational attachment. Administrative Science Quarterly, 37, 547-579.
45.
Yammarino, F. J., & Markham, S. E. (1992). On the application of within and between analysis: Are absence and affect really group-based phenomena? Journal of Applied Psychology, 77, 168-176.
46.
Weick, K. E. (1998). Improvisation as a mindset for organizational analysis. Organizational Science, 9, 543-555.
