Organizational data are inherently nested; consequently, lower level data are typically influenced by higher level grouping factors. Stated another way, almost all lower level organizational data have some degree of nonindependence due to work group, geographic membership, and so on. Unaccounted-for nonindependence can be problematic because it affects standard error estimates used to determine statistical significance. Currently, researchers interested in modeling higher level variables routinely use multilevel modeling techniques to avoid well-known problems with Type I error rates. In this article, however, the authors examine how nonindependence affects statistical inferences in cases in which researchers are interested only in relationships among lower level variables. They show that ignoring nonindependence when modeling only lower level variables reduces power (increases Type II errors), and through simulations, the authors show where this loss of power is most pronounced.
Aquinis, H. (1995). Statistical power problems with moderated multiple regression in management research. Journal of Management, 21, 1141-1158.
2.
Barcikowski, R. S. (1981). Statistical power with group mean as the unit of analysis. Journal of Educational Statistics, 6, 267-285.
3.
Bliese, P. D. (1998). Group size, ICC values, and group-level correlations: A simulation. Organizational Research Methods, 1, 355-373.
4.
Bliese, P. D. (2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In K. J. Klein & S. W. Kozlowski (Eds.), Multilevel theory, research, and methods in organizations(pp. 349-381). San Francisco: Jossey-Bass.
5.
Bliese, P. D. (2002). Multilevel random coefficient modeling in organizational research: Examples using SAS and S-PLUS. In F. Drasgow& N. Schmitt (Eds.), Measuring and analyzing behavior in organizations: Advances in measurement and data analysis(pp. 401-445). San Francisco: Jossey-Bass.
6.
Bryk, A. S.,& Raudenbush, S. W. (1992). Hierarchical linear models. Newbury Park, CA: Sage.
7.
Bryk, A. S., Raudenbush, S. W.,& Congdon, R. T. (1994). Hierarchical linear modeling with the HLM/2L and HLM/3L programs. Chicago: Scientific Software, International.
8.
Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences(2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
9.
Davison, M. L., Kwak, N., Seo, Y. S.,& Choi, J. (2002). Using hierarchical linear models to examine moderator effects: Person-by-organization interactions. Organizational Research Methods, 5, 231-254.
10.
Heck, R. H. & Thomas, S. L. (2000). An introduction to multilevel modeling techniques. Mahwah, NJ: Lawrence Erlbaum.
11.
Hofmann, D. A. (1997). An overview of the logic and rationale of hierarchical linear models. Journal of Management, 23, 723-744.
12.
Hofmann, D. A.,& Stetzer, A. (1998). The role of safety climate and communication in accident interpretation: Implications for learning from negative events. Academy of Management Journal, 41, 644-657.
13.
Hox, J. J. (2002). Multilevel analysis: Techniques and applications. Mahwah, NJ: Lawrence Erlbaum.
14.
Jex, S. M.,& Bliese, P. D. (1999). Efficacy beliefs as a moderator of the impact of work-related stressors: A multi-level study. Journal of Applied Psychology, 84, 349-361.
15.
Kenny, D. A.,& Judd, C. M. (1986). Consequences of violating the independence assumption in analysis of variance. Psychological Bulletin, 99, 422-431.
16.
Kenny, D. A., Kashy, D. A., & Bolger, N. (1998). Data analysis in social psychology. In D. Gilbert, S. Fiske, & G. Lindzey (Eds.), Handbook of social psychology (4th ed., Vol. 1, pp. 233-265). Boston: McGraw-Hill.
17.
Kidwell, R. E., Mossholder, K. M.,& Bennett, N. (1997). Cohesiveness and organizational citizenship behavior: A multilevel analysis using groups and individuals. Journal of Management, 23, 775-793.
18.
Kirk, R. E. (1994). Experimental design: Procedures for the behavioral sciences. Pacific Grove, CA: Brooks/Cole.
19.
Klein, K. J., Bliese, P. D., Kozlowski, S. W. J., Dansereau, F., Gavin, M. B., Griffin, M. A., et al. (2000). Multilevel analytical techniques: Commonalities, differences, and continuing questions. In K. J. Klein & S. W. Kozlowski (Eds.), Multilevel theory, research, and methods in organizations(pp. 512-553). San Francisco: Jossey-Bass.
20.
Kozlowski, S. W. J.,& Klein, K. J. (2000). Amultilevel approach to theory and research in organizations: Contextual, temporal and emergent processes. In K. J. Klein & S. W. Kozlowski (Eds.), Multilevel theory, research, and methods in organizations(pp. 3-90). San Francisco: Jossey-Bass.
21.
Kreft, I., & de Leeuw, J. (1998). Introducing multilevel modeling. London: Sage.
22.
Littell, R. C., Milliken, G. A., StroupW. W., & Wolfinger, R. D. (1996). SAS system for mixed models. Cary, NC: SAS Institute.
23.
McClelland, G. H., & Judd, C. M. (1993). Statistical difficulties of detecting interactions and moderator effects. Psychological Bulletin, 114, 376-390.
24.
Pinheiro, J. C., & Bates, D. M. (2000). Mixed-effects models in S and S-PLUS. New York: Springer-Verlag.
25.
Singer, J. D. (1998). Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth models. Journal of Educational and Behavioral Statistics, 24, 323-355.
26.
Snijders, T. A. B.,& Bosker, R. J. (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. London: Sage.
27.
Walsh, J. E. (1947). Concerning the effects of intraclass correlation on certain significance tests. Annals of Mathematical Statistics, 18, 88-96.
