Multilevel Factor Structure,Concurrent Validity,and Test–Retest Reliability of the High School Teacher Version of the Authoritative School Climate Survey

Abstract

Although school climate has long been recognized as an important factor in the school improvement process, there are few psychometrically supported measures based on teacher perspectives. The current study replicated and extended the factor structure, concurrent validity, and test–retest reliability of the teacher version of the Authoritative School Climate Survey (ASCS) using a statewide sample of high school teachers. Multilevel confirmatory factor analyses based on surveys completed by 12,808 high school teachers from 302 schools found that factors of disciplinary structure and student support were associated to varying degrees with the teacher reports of the prevalence of student teasing and bullying and student engagement. These findings provide some empirical support for the use of the teacher version of the ASCS in high schools.

Keywords

multilevel CFA school climate teacher perceptions factor structure psychometric properties

Get full access to this article

View all access options for this article.

References

Bandyopadhyay

Cornell

D. G.

Konold

(2009). Internal and external validity of three school climate scales from the School Climate Bullying Survey. School Psychology Review, 38, 338-355.

Baumrind

(1968). Authoritarian vs. authoritative parental control. Adolescence, 3, 255-272.

Bear

G. G.

Yang

Pell

Gaskins

(2014). Validation of a brief measure of teachers’ perceptions of school climate: Relations to student achievement and suspensions. Learning Environments Research, 17, 339-354. doi:10.1007/s10984-014-9162-1

Bliese

(2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In Bollen

Long

(Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions and new directions (pp. 349-381). San Francisco, CA: Jossey-Bass.

Byrne

(2012). Structural equation modeling with Mplus: Basic concepts, applications, and programming. New York, NY: Routledge.

California Healthy Kids Survey. (2015). Available from http://chks.wested.org/

Clifford

Menon

Gangi

Condon

Hornung

(2012). Measuring school climate for gauging principal performance: A review of the validity and reliability of publicly accessible measures. American Institutes for Research. Retrieved from http://eric.ed.gov/?id=ED531401

Cohen

(1992). A power primer. Psychological Bulletin, 112, 155-159. doi:10.1037//0033-2909.112.1.155

Cohen

McCabe

Michelli

Pickeral

(2009). School climate: Research, policy, practice, and teacher education. Teachers College Record, 111, 180-213.

10.

Cornell

Gregory

Huang

Fan

(2013). Perceived prevalence of teasing and bullying predicts high school dropout rates. Journal of Educational Psychology, 105, 138-149. doi:10.1037/a0030416

11.

Cornell

Klein

Konold

Huang

(2012). Effects of validity screening items on adolescent survey data. Psychological Assessment, 24, 21-35. doi:10.1037/a0024824

12.

Cornell

D. G.

Mayer

M. J.

(2010). Why do school order and safety matter? Educational Researcher, 39, 7-15. doi:10.3102/0013189X09357616

13.

Dedrick

R. F.

Greenbaum

P. E.

(2011). Multilevel confirmatory factor analysis of a scale measuring interagency collaboration of children’s mental health agencies. Journal of Emotional and Behavioral Disorders, 19, 27-40. doi:10.1177/1063426610365879

14.

Dong

Peng

C. Y. J.

(2013). Principled missing data methods for researchers. SpringerPlus, 2, 222. doi:10.1186/2193-1801-2-222

15.

Dyer

N. G.

Hanges

P. J.

Hall

R. J.

(2005). Applying multilevel confirmatory factor analysis techniques to the study of leadership. The Leadership Quarterly, 16, 149-167. doi:10.1016/j.leaqua.2004.09.009

16.

Espelage

D. L.

Polanin

J. R.

Low

S. K.

(2014). Teacher and staff perceptions of school environment as predictors of student aggression, victimization, and willingness to intervene in bullying situations. School Psychology Quarterly, 29, 287-305. doi:10.1037/spq0000072

17.

Fabrigar

L. R.

Wegener

D. T.

(2012). Exploratory factor analysis. Oxford, UK: Oxford University.

18.

Gerlinger

J. C.

(2014). Preventing school bullying: Should schools prioritize an authoritative school discipline approach over security measures? Journal of School Violence, 10, 1-25. doi:10.1080/15388220.2014.956321

19.

Gregory

Cornell

(2009). “Tolerating” adolescent needs: Moving beyond zero tolerance policies in high school. Theory Into Practice, 48, 106-113. doi:10.1080/00405840902776327

20.

Gregory

Cornell

Fan

(2012). Teacher safety and authoritative school climate in high schools. American Journal of Education, 118, 401-425. doi:10.1086/666362

21.

Gregory

Cornell

Fan

Sheras

Shih

T.-H.

Huang

(2010). Authoritative school discipline: High school practices associated with lower bullying and victimization. Journal of Educational Psychology, 102, 483-496. doi:10.1037/a0018562

22.

Griffith

(1997). Student and parent perceptions of school social environment: Are they group based? The Elementary School Journal, 98, 135-150. doi:10.1086/461888

23.

Hanges

P. J.

Dickson

M. W.

(2006). Agitation over aggregation: Clarifying the development of and the nature of the GLOBE scales. The Leadership Quarterly, 17, 522-536. doi:10.1016/j.leaqua.2006.06.004

24.

Heck

R. H.

Thomas

S. L.

(2008). An introduction to multilevel modeling techniques (2nd ed.). New York, NY: Routledge.

25.

Hinkle

Wiersma

Jurs

(2003). Applied statistics for the behavioral sciences (4th ed.). Boston, MA: Houghton Mifflin.

26.

Horner

R. H.

Sugai

Anderson

C. M.

(2010). Examining the evidence base for school-wide positive behavior support. Focus on Exceptional Children, 42, 1-14.

27.

Hox

J. J.

(2002). Multilevel analysis: Techniques and applications. Mahwah, NJ: Lawrence Erlbaum.

28.

Bentler

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55. doi:10.1080/10705519909540118

29.

Huang

Cornell

(2015). Using multilevel factor analysis with clustered data: Investigating the factor structure of the Positive Values Scale. Journal of Psychoeducational Assessment. Advance online publication. doi:10.1177/0734282915570278

30.

Huang

Cornell

Konold

Meyer

Lacey

Nekvasil

. . . Shukla

(2015). Multilevel factor structure and concurrent validity of the teacher version of the Authoritative School Climate Survey. Journal of School Health, 85, 843-851.

31.

Johnson

Stevens

J. J.

(2006). Student achievement and elementary teachers’ perceptions of school climate. Learning Environments Research, 8, 111-122.

32.

Johnson

Stevens

J. J.

Zvoch

(2007). Teachers’ perceptions of school climate: A validity study of scores from the Revised School Level Environment Questionnaire. Educational and Psychological Measurement, 67, 833-844. doi:10.1177/0013164406299102

33.

Johnson

J. A.

(2005). Ascertaining the validity of individual protocols from web-based personality inventories. Journal of Research in Personality, 39, 103-129. doi:10.1016/j.jrp.2004.09.009

34.

Julian

M. W.

(2001). The consequences of ignoring multilevel data structures in nonhierarchical covariance modeling. Structural Equation Modeling, 8, 325-352.

35.

Kaplan

Elliott

P. R.

(1997). A didactic example of multilevel structural equation modeling applicable to the study of organizations. Structural Equation Modeling, 4, 1-24. doi:10.1080/10705519709540056

36.

Kennely

Monrad

(2007). Easing the transition to high school: Research on best practices designed to support high school learning. Washington, DC: National High School Center. Retrieved from http://files.eric.ed.gov/fulltext/ED501073.pdf

37.

Kline

(2011). Principles and practice of structural equation modeling (3rd ed.). New York, NY: Guilford Press.

38.

Konold

Cornell

(2015a). Measurement and structural relations of an authoritative school climate model: A multi-level latent variable investigation. Journal of School Psychology, 53, 447-461. doi:10.1016/j.jsp.2015.09.001

39.

Konold

Cornell

(2015b). Multilevel multitrait–multimethod latent analysis of structurally different and interchangeable raters of school climate. Psychological Assessment, 27, 1097-1109. doi:10.1037/pas0000098

40.

Konold

Cornell

Huang

Meyer

Lacey

Nekvasil

. . . Shukla

(2014). Multilevel multi-informant structure of the authoritative school climate survey. School Psychology Quarterly, 29, 238-255. doi:10.1037/spq0000062

41.

Ladd

H. F.

(2011). Teachers’ perceptions of their working conditions: How predictive of planned and actual teacher movement? Educational Evaluation and Policy Analysis, 33, 235-261. doi:10.3102/0162373711398128

42.

LaRusso

Bates

Selman

(2008). Teachers as builders of respectful school climates: Implications for adolescent drug use norms and depressive symptoms in high school. Journal of Youth and Adolescence, 38, 386-398.

43.

Liu

Ding

Berkowitz

M. W.

Bier

M. C.

(2014). A psychometric evaluation of a revised school climate teacher survey. Canadian Journal of School Psychology, 29, 54-67. doi:10.1177/0829573514521777

44.

Meade

A. W.

Bartholomew

(2012). Identifying careless responses in survey data. Psychological Methods, 17, 437-455. doi:10.1037/a0028085

45.

Mikami

A. Y.

Gregory

Allen

J. P.

Pianta

R. C.

Lun

(2011). Effects of a teacher professional development intervention on peer relationships in secondary classrooms. School Psychology Review, 40, 367-385.

46.

Mitchell

M. M.

Bradshaw

C. P.

Leaf

P. J.

(2010). Student and teacher perceptions of school climate: A multilevel exploration of patterns of discrepancy. Journal of School Health, 80, 271-279. doi:10.1111/j.1746-1561.2010.00501.x

47.

Muthén

(1994). Multilevel covariance structure analysis. Sociological Methods & Research, 22, 376-398. doi:10.1177/0049124194022003006

48.

Muthén

(2005, January 20). Negative residual variance. Retrieved from http://www.statmodel.com/discussion/messages/11/555.html?1358188287

49.

Muthén

B. O.

Satorra

(1995). Complex sample data in structural equation modeling. Sociological Methodology, 25, 267-316. doi:10.2307/271070

50.

National Center for Education Statistics. (2005). School Crime Supplement to the National Crime Victimization Survey 2005. Retrieved from http://nces.ed.gov/programs/crime/pdf/student/SCS05.pdf

51.

Pellerin

L. A.

(2005). Applying Baumrind’s parenting typology to high schools: Toward a middle-range theory of authoritative socialization. Social Science Research, 34, 283-303. doi:10.1016/j.ssresearch.2004.02.003

52.

Schweig

(2013). Cross-level measurement invariance in school and classroom environment surveys: Implications for policy and practice. Educational Evaluation and Policy Analysis, 36, 259-280. doi:10.3102/0162373713509880

53.

Thapa

Cohen

Guffey

Higgins-D’Alessandro

(2013). A review of school climate research. Review of Educational Research, 83, 357-385. doi:10.3102/0034654313483907

54.

Thornberry

Lizotte

Krohn

Farnworth

Jang

(1991). Testing interactional theory: An examination of reciprocal causal relationships among family, school, and delinquency. Journal of Criminal Law & Criminology, 82, 3-35.

55.

van Horn

M. L

. (2003). Assessing the unit of measurement for school climate through psychometric and outcome analyses of the school climate survey. Educational and Psychological Measurement, 63, 1002-1019. doi:10.1177/0013164403251317

56.

Voight

Hanson

(2012). Summary of existing school climate instruments for middle school. San Francisco, CA: REL West at WestEd. Retrieved from http://relwest.wested.org/resources/21

57.

Wang

M.-T.

Degol

J. L.

(2015). School climate: A review of the construct, measurement, and impact on student outcomes. Educational Psychology Review. Advance online publication. doi:10.1007/s10648-015-9319-1

58.

Zullig

K. J.

Collins

Ghani

Hunter

A. A.

Patton

J. M.

Huebner

E. S.

Zhang

(2015). Preliminary development of a revised version of the School Climate Measure. Psychological Assessment, 27, 1072-1081. doi:10.1037/pas0000070

59.

Zyphur

M. J.

Kaplan

S. A.

Christian

M. S.

(2008). Assumptions of cross-level measurement and structural invariance in the analysis of multilevel data: Problems and solutions. Group Dynamics: Theory, Research, and Practice, 12, 127-140. doi:10.1037/1089-2699.12.2.127