Assessing the Unit of Measurement for School Climate through Psychometric and Outcome Analyses of the School Climate Survey

Abstract

This study examines the appropriate unit of theory of school climate while simultaneously conducting a psychometric analysis of a modification of the School Climate Survey for an elementary school population. A conceptualization of school climate as an individual-level property unique to each school participant is compared to the idea that climate is a characteristic of the school with school participants having the role of informants. These two theories are compared by testing their differing implications. The School Climate Survey is found to have a stable and identical factor structure both within and between schools, ratings by different respondent groups are found to be moderately correlated, and average school climate within each school is found to predict a statistically significant amount of the between-school variation in children’s academic achievement and cognitive functioning. Differences between individual raters within each school are not found to have a meaningful relationship to child outcomes. It is concluded that the School Climate Survey is a valid and useful measure of school climate and the school-level theory is found to have the strongest support.

Get full access to this article

View all access options for this article.

References

Anderson, C. S. (1982). The search for school climate: A review of the research. Review of Educational Research, 52(3), 368-420.

Brookover, W. B. (1978). Elementary school social climate and school achievement. American Educational Research Journal, 15, 301-318.

Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear models(Vol. 1). London: Sage.

Dixon, P. N., Johnson, W. L., & Toman, S. (1991). Revisiting the Charles F. Kettering, LTD. school climate profile: Further analysis of the subscale structure. Educational and Psychological Measurement, 51, 135-141.

Esposito, C. (1999). Learning in urban blight: School climate and its effect on the school performance of urban, minority, low-income children. School Psychology Review, 28, 365-377.

Fleis, J. L.,& Cohen, J. (1973). The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and Psychological Measurement, 33, 613-619.

Griffith, J. (1995). An empirical examination of a model of social climate in elementary schools. Basic and Applied Social Psychology, 17(1-2), 97-117.

Howard, E. R.,& Keefe, J. W. (1991). The CASE_IMS school improvement process. Reston, VA: National Association of Secondary School Principals.

Hox, J. J. (1995). Applied multilevel analysis. Amsterdam: TT-Publikaties.

10.

Hox, J. J. (1998). Multilevel modeling: When and why. In I. Balderjahn, R. Mathar, & M. Schader (Eds.), Classification, data analysis, and data highways(Vol. 2000). New York: Springer-Verlag.

11.

James, L. R. (1982). Aggregation bias in estimates of perceptual agreement. Journal of Applied Psychology, 67, 279-229.

12.

Keefe, J. W., Kelley, E. A., & Miller, S. K. (1985, November). School climate: Clear definitions and a model for a larger school setting. NASSP Bulletin, 69, 70-77.

13.

Kelley, E. A., Glover, J. A., Keefe, J. W., Halderson, C., Sorenson, C., & Speth, C. (1986). School Climate Survey (Modified). Reston, VA: NationalAssociation of Secondary School Principals.

14.

Kreft, I. G. G. (1993). Using multilevel analyses to assess school effectiveness: Astudy of Dutch secondary schools. Sociology of Education, 66, 104-129.

15.

Lindell, M. K.,& Brandt, C. J. (2000). Climate quality and climate consensus as mediators of the relationship between organizational antecedents and outcomes. Journal of Applied Psychology, 85, 331-348.

16.

Miller, M. B. (1995). Coefficient alpha: A basic introduction from the perspectives of classical test theory and structural equation modeling. Structural Equation Modeling, 2, 255-273.

17.

Miller, S. I., & Fredericks, J. (1990). The false ontology of school climate effects. Educational Theory, 40(3), 333-342.

18.

Mok, M. M. C.,& McDonald, R. P. (1994). Quality of school life: Ascale to measure student experience or school climate? Educational and Psychological Measurement, 54, 483-495.

19.

Muthén, B. O. (1991). Multilevel factor analysis of class and student achievement components. Journal of Educational Measurement, 28, 338-354.

20.

Nunnally, J. C. (1967). Psychometric theory(2nd ed.). New York: McGraw-Hill.

21.

Purkey, S. C., & Smith, M. S. (1983). Effective schools: A review. The Elementary School Journal, 83, 427-452.

22.

Ramey, C. T., Ramey, S. L., & Phillips, M. M. (1996). Head Start children’s entry into public school: An interim report on the National Head Start-Public School EarlyChildhood Transition Demonstration Study. Washington, DC: Report prepared for the U.S. Department of Health and Human Services, Head Start Bureau.

23.

Raudenbush, S. W., Bryk, A. S., Cheong, Y. F., & Congdon, R. T. (2000). HLM 5: Heirarchiacal linear and nonlinear modeling(Version 5). Lincolnwood, IL: Scientific Software International.

24.

Raudenbush, S. W., Rowan, B.,& Kang, S. J. (1991). Amultilevel, multivariate model for studying school climate with estimation via theEMalgorithm and application to U.S. high-school data. Journal of Educational Statistics, 16, 295-330.

25.

Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 450-428.

26.

Sirotnik, K. A. (1980). Psychometric implications of the unit-of-analysis problem (with examples from the measurement of organizational climate). Journal of Educational Measurement, 17, 245-282.

27.

Woodcock, R., & Johnson, M. (1990). Woodcock-Johnson psycho-educational battery—revised.Allen, TX: DLM Teaching Resources.