Several issues related to multilevel research and measurement theory are discussed. It was emphasized that questions regarding choice of level of measurement and unit of analysis are less important than the choice of an appropriate statistical model. It was demonstrated that even with an "error-free" environmental measure not all persons should view the setting in the same manner. The split-sample correlation was clearly shown to have undesirable statistical properties, and the use of standard methods was stressed. Investigators were cautioned to carefully choose a coefficient and were shown that this choice is sometimes tricky. Finally, the analysis of multilevel data using multilevel methods was recommended.
Get full access to this article
View all access options for this article.
References
1.
Bartko, J. J. (1976). On various intraclass correlation reliability coefficients. Psychological Bulletin, 83, 762-765.
2.
Brennan, R. L. (1994). Variance components in generalizability theory. In C. R. Reynolds (Ed.), Cognitive assessment: A multidisciplinary perspective (pp. 175-207). New York: Plenum.
3.
Brockett, P. , & Levine, A. (1984). Statistics and probability and their applications. Philadelphia: Saunders.
4.
Bryk, A. S. , & Raudenbush, S. W. (1992). Hierarchical linear models: Applications and data analysis methods. Newbury Park: Sage.
5.
Burstein, L. (1980). The analysis=of multilevel data in educational research and evaluation. In D. C. Berlinger (Ed.), Review of research in education (Vol. 8, pp. 158-233). Washington, DC: American Educational Research Association.
6.
Cramér, H. (1974). Mathematical methods of statistics (13th prnting). Princeton, NJ: Princeton University Press.
7.
Cronbach, L. J. , Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability of scores and profile. New York: John Wiley.
8.
Goldstein, H. (1987). Multilevel models in education and social research. London: Charles Griffin.
9.
Goldstein, H. , & McDonald, R. P. (1988). A general model for the analysis of multilevel data. Psychometrka, 53, 455-467.
10.
Hogg, R. V. , & Craig, A. T. (1978). Introduction to mathematical statistics (4th ed.) New York: Macmillian.
11.
Johnson, N. L. , & Kotz, S. (1970). Distributions in statistics: Continuous univarate distibutions-2. New York: John Wiley.
12.
Kempthome, O. (1989). The fate worse than death and other curiosities and stupidities. American Statffstican, 43, 133-134.
13.
Kraemer, H. C. , Bliwise, N. G., & Bliwise, D. L. (1991). Appropriate statistics for interrater reliability in sleep studies. Sleep, 14, 89-90.
14.
Levine, D. W. (1994). True scores, error, reliability and the unit of analysis in environment and behavior research. Environment and Behavior, 26, 261-293.
15.
O'Bren, R. M. (1990). Estimating the reliability of aggregate-level variables based on individual-level characteristics. Sociological Methods & Research, 18, 473-504.
16.
Raudenbush, S. W. (1988). Educational applications of hierarchical linear models: A review. Journal of Educational Statistics, 13, 85-116.
17.
Raudenbush, S. W. , & Bryk, A. S. (1988). Methodological advances in analyzing the effects of schools and classrooms on student learning. In E. Z. Rothkopf (Ed.), Review of research in education 1988-89 (Vol. 15, pp. 423-475). Washington, DC: American Educational Research Association.
18.
Richards, J. M., Jr. (1990). Units of analysis and the individual differences fallacy in environmental assessment. Environment and Behavior, 22, 307-319.
19.
Richards, J. M., Jr. (1996). Units of analysis, measurement theory, and environmental assessment: A response and clarification. Environment and Behavior, 28(2), 220-236.
20.
Richards, J. M., Jr. , Gottfredson, D. C., & Gottfredson, G. D. (1991a). Units of analysis and the psychometrics of environmental assessment scales. Environment and Behavior, 23, 423-437.
21.
Richards, J. M., Jr. , Gottfredson, D. C., & Gottfredson, G. D. (1991b). Units of analysis and item statistics for environmental assessment scales. Current Psychology: Research & Reviews, 9, 407-413.
22.
Rogosa, D. (1978). Politics, process, and pyramids. Journal of Educational Statistics, 3, 79-86.
23.
Scheuch, E. K. (1969). Social context and individul behavior. In M. Dogan & S. Rokkan (Eds.), Quantitative ecological analysis in the social sciences (pp. 133-155). Cambridge, MA: MIT Press.
24.
Searle, S. R. , Casella, G., & McCulloch, C. E. (1992). Variance components. New York: John Wiley.
25.
Shavelson, R. J. , & Webb, N. M. (1991). Generalizability theory. Newbury Park: Sage.
26.
Shrout, P. E. , & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420-428.
27.
Sutcliffe, J. P. (1965). A probability model for errors of classification. I. General considerations. Psychometrika, 30, 73-96.
28.
Winer, B. J. , Brown, D. R., & Michels, K. M. (1991). Statistical principles in experimental design (3rd ed.). New York: McGraw-Hill.
