Academic Culture: Its Meaning,Measure and Contribution to Student Learning

Abstract

This study had two objectives. One objective was to assess the psychometric properties of a survey instrument measuring a new latent variable, Academic Culture (AC), combining three observed variables academic press, disciplinary climate and teachers’ uses of instructional time. The second objective was to replicate the results of an earlier study identifying AC as a significant mediator of school leadership’s influence on student learning. Data for the study were provided from 2068 teachers located in 49 schools in 14 Texas school districts, as well as student achievement data from the State of Texas Assessments of Academic Readiness (STAAR) and student socioeconomic (SES) data available from school websites. Second order Confirmatory Factor Analysis (CFA) and Many-Facet Rasch (MFR) models were used to examine the survey instrument’s construct validity and its measurement invariance. Structural Equation Modeling was used to identify the extent to which AC mediated the effects of school leadership on student achievement controlling for student SES. Rasch analysis and CFA confirmed the measurement invariance and several forms of validity of the survey instrument. Replicating the results of an earlier study, results of structural equation modeling demonstrated significant effects of AC on student achievement and identified AC as a significant mediator of school leadership effects on student achievement. The study contributes to the quality of instruments available to school leaders for their school improvement work and to researchers inquiring about the most promising variables mediating the indirect effects of school leadership on student success.

Keywords

leadership culture replication academic press instructional time disciplinary climate

Get full access to this article

View all access options for this article.

References

Alig-Mielcarek

Hoy

W. K.

(2005). Instructional leadership: Its nature, meaning, and influence. In Hoy

Miskel

(Eds.), Educational leadership and reform (pp. 29–51). Information Age Publishing.

American Educational Research Association . (2014). AERA standards for educational and psychological testing (2014 edition). https://www.aera.net/Publications/Books/Standards-for-Educational-Psychological-Testing-2014-Edition

Andrich

(1978). A rating formulation for ordered response categories. Psychometrika, 43(4), 561–573. https://doi.org/10.1007/BF02293814

Andrich

(2004). Controversy and the Rasch model: A characteristic of incompatible paradigms? Medical Care, 42(1 Suppl), I7–I16. https://doi.org/10.1097/01.mlr.0000103528.48582.7c

Bass

B. M.

Avolio

B. J.

(1994). Improving organizational effectiveness through transformational leadership. Thousand Oaks, CA: Sage Publications.

Benda

S. B.

(2000). The effect of leadership styles on the disciplinary climate and culture of elementary schools. ProQuest Dissertations Publishing.

Bliese

P. D.

(2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In Klein

K. J.

Kozlowski

S. W. J.

(Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 349–381). Jossey-Bass/Wiley.

Boberg

J. E.

Bourgeois

S. J.

(2016). The effects of integrated transformational leadership on achievement. Journal of Educational Administration, 54(3), 357–374. https://doi.org/10.1108/JEA-07-2014-0086

Bond

Yan

Heene

(2020). Applying the Rasch model: Fundamental measurement in the human sciences. Routledge. https://doi.org/10.4324/9780429030499

10.

Borsboom

(2005). Measuring the mind: Conceptual issues in contemporary psychometrics. Cambridge University Press. https://doi.org/10.1017/CBO9780511490026

11.

Carroll

J. B.

(1963). A model of school learning. Teachers College Record: The Voice of Scholarship in Education, 64(8), 1–9. https://doi.org/10.1177/016146816306400801

12.

Chi

M. T.

Adams

Bogusch

E. B.

Bruchok

Kang

Lancaster

Levy

McEldoon

K. L.

Stump

G. S.

Wylie

Yaghmourian

D. L.

Yaghmourian

D. L.

(2018). Translating the ICAP theory of cognitive engagement into practice. Cognitive Science, 42(6), 1777–1832. https://doi.org/10.1111/cogs.12626

13.

Cohen

West

S. G.

Aiken

L. S.

(2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Lawrence Erlbaum Associates Publishers.

14.

Darling-Hammond

Flook

Cook-Harvey

Barron

Osher

(2020). Implications for educational practice of the science of learning and development. Applied Developmental Science, 24(2), 97–140. https://doi.org/10.1080/10888691.2018.1537791

15.

Day

(2018). How successful secondary school principals in england respond to policy reforms: The influence of biography. Leadership and Policy in Schools, 17(3), 332–344. https://doi.org/10.1080/15700763.2018.1496339

16.

DiPaola

M. F.

Tschannen-Moran

(2005). Bridging or buffering? The impact of schools’ adaptive strategies on student achievement. Journal of Educational Administration, 43(1), 60–71. https://doi.org/10.1108/09578230510577290

17.

Domina

Pharris-Ciurej

Penner

A. M.

Penner

E. K.

Brummet

Porter

S. R.

Sanabria

(2018). Is free and reduced-price lunch a valid measure of educational disadvantage? Educational Researcher, 47(9), 539–555. https://doi.org/10.3102/0013189X18797609

18.

Engelhard

Myford

C. M.

(2003). Monitoring faculty consultant performance in the advanced placement English Literature and composition program with a many-faceted Rasch model. ETS Research Report Series, 2003(1), 60. https://doi.org/10.1002/j.2333-8504.2003.tb01893.x

19.

Engelhard

Wind

S. A.

(2018). Invariant measurement with raters and rating scales: Rasch Models for rater-mediated assessments (1st ed.). Routledge. https://doi.org/10.4324/9781315766829

20.

Fauth

Decristan

Rieser

Klieme

Büttner

(2014). Student ratings of teaching quality in primary school: Dimensions and prediction of student outcomes. Learning and Instruction, 29(1), 1–9. https://doi.org/10.1016/j.learninstruc.2013.07.001

21.

Fullan

Hill

Crévola

(2006). Breakthrough. Ontario Principal’s Council.

22.

Goddard

R. D.

Hoy

W. K.

Hoy

A. W.

(2000). Collective teacher efficacy: Its Meaning, measure, and impact on student achievement. American Educational Research Journal, 37(2), 479–507. https://doi.org/10.3102/00028312037002479

23.

Hallinger

(2003). Leading educational change: Reflections on the practice of instructional and transformational leadership. Cambridge Journal of Education, 33(3), 329–352. https://doi.org/10.1080/0305764032000122005

24.

Hallinger

(2005). Instructional leadership and the school principal: A passing fancy that refuses to fade away. Leadership and Policy in Schools, 4(3), 221–239. https://doi.org/10.1080/15700760500244793

25.

Hallinger

Heck

R. H.

(2010). Leadership for learning: Does collaborative leadership make a difference in school improvement? Educational Management Administration & Leadership, 38(6), 654–678. https://doi.org/10.1177/1741143210379060

26.

Harris

Day

Hopkins

Hadfield

Hargreaves

Chapman

(2007). Distributed leadership and school improvement: Leading or misleading? Educational Management Administration & Leadership, 35(1), 11–24. https://doi.org/10.1177/1741143207071387

27.

Holzberger

Schiepe–Tiska

(2021). Is the school context associated with instructional quality? The effects of social composition, leadership, teacher collaboration, and school climate. School Effectiveness and School Improvement, 32(3), 465–485. https://doi.org/10.1080/09243453.2021.1913190

28.

Hossler

C. A.

(1988). The relationship of increased instructional time to student achievement. Policy bulletin No. 1. Consortium on Educational Policy Studies.

29.

Hoy

W. K.

Hannum

Tschannen-Moran

(1998). Organizational climate and student achievement: A parsimonious and longitudinal view. Journal of School Leadership, 8(4), 336–359. https://doi.org/10.1177/105268469800800401

30.

Hoy

W. K.

Sweetland

S. R.

(2000). School bureaucracies that work: Enabling, not coercive. Journal of School Leadership, 10(6), 525–541. https://doi.org/10.1177/105268460001000603

31.

Hoy

W. K.

Tarter

C. J.

(1997). The road to open and healthy schools: A handbook for change (Elementary Edition). Corwin Press.

32.

Hoy

W. K.

Tarter

C. J.

Hoy

A. W.

(2006). Academic optimism of schools: A force for student achievement. American Educational Research Journal, 43(3), 425–446. https://doi.org/10.3102/00028312043003425

33.

Jurewicz

M. M.

(2004). Organizational citizenship behaviors of middle school teachers: A study of their relationship to school climate and student achievement. ProQuest Dissertations Publishing.

34.

Kline

R. B.

(2020). Becoming a behavioral science researcher (2nd ed.). Guilford Press.

35.

Lakshman

L. C.

Ramaswami

(2020). Measurement invariance and nomological validity of the attributional complexity scale: Evidence from Estonia, France, India, United States, and Vietnam. International Journal of Cross Cultural Management, 20(1), 89–111. https://doi.org/10.1177/1470595820913600

36.

Lee

J. S.

(2012). The effects of the teacher–student relationship and academic press on student engagement and academic performance. International Journal of Educational Research, 53(1), 330–340. https://doi.org/10.3350/cmh.2012.18.3.330

37.

Leithwood

(2021). A review of evidence about equitable school leadership. Education Sciences, 11(8), 377. https://doi.org/10.3390/educsci11080377

38.

Leithwood

Day

Sammons

Harris

Hopkins

(2006). Successful schoolleadership: What it is and how it influences pupil learning. DfES Publications.

39.

Leithwood

Harris

Hopkins

(2020). Seven strong claims about successful school leadership revisited. School Leadership & Management, 40(1), 5–22. https://doi.org/10.1080/13632434.2019.1596077

40.

Leithwood

Louis

K. S.

Anderson

Wahlstrom

(2004). How leadershipinfluences student learning. Review of research. Wallace Foundation.

41.

Leithwood

Louise

K. S.

(2012). Linking leadership to student learning. Jossey-Bass.

42.

Leithwood

Sun

Schumacker

Hua

(2023). Psychometric properties of the successful school leadership survey. Journal of Educational Administration, 61(4), 385–404. https://doi.org/10.1108/JEA-08-2022-0115

43.

Leithwood

Sun

J.-P.

(2012). The nature and effects of transformational school leadership: A meta-analytic review of unpublished research. Educational Administration Quarterly, 48(3), 387–423. https://doi.org/10.1177/0013161X11436268

44.

Leithwood

Sun

J.-P.

(2018). Academic culture: A promising mediator of school leaders’ influence onstudent learning. Journal of Educational Administration, 56(3), 350–363. https://doi.org/10.1108/JEA-01-2017-0009

45.

Leithwood

Sun

J.-P.

McCullough

(2019). How school districts influence student achievement. Journal of Educational Administration, 57(5), 519–539. https://doi.org/10.1108/JEA-09-2018-0175

46.

Leithwood

K. A.

Riehl

(2005). What we know about successful school leadership. Laboratory for Student Success, Temple University.

47.

Leonard

L. J.

(1999). Towards maximizing instructional time: The nature and extent of externally-imposed classroom interruptions. Journal of School Leadership, 9(5), 454–474. https://doi.org/10.1177/105268469900900506

48.

Levine

Lezotte

(1990). Unusually effective schools: A review and analysis of research and practice. School Effectiveness and School Improvement, 1(3), 221–224. https://doi.org/10.1080/0924345900010305

49.

Linacre

J. M.

(1989). Many-facet Rasch measurement. MESA Press.

50.

Linacre

J. M.

(2002). What do infit and outfit, mean-square and standardized mean? Rasch Measurement Transactions, 16(2), 878. https://doi.org/10.1093/intimm/dxf013

51.

Linacre

J. M.

(2012). Many-facet Rasch measurement: Facets tutorial. https://winsteps.com/tutorials.htm

52.

Linacre

J. M.

(2014). WINSTEPS Rasch measurement (version 3.81.0). Winsteps.com.

53.

Locke

Latham

Smith

Wood

(1990). A theory of goal setting and task performance. Prentice-Hall.

54.

Willms

J. D.

(2004). School disciplinary climate: Characteristics and effects on eight- grade achievement. Alberta Journal of Educational Research, 50(2), 169–188. https://doi.org/10.2307/4126452

55.

Makel

M. C.

Plucker

J. A.

(2014). Facts are more important than novelty: Replication in the education sciences. Educational Researcher, 43(6), 304–316. https://doi.org/10.3102/0013189X14545513

56.

Marburger

D. R.

(2006). Does mandatory attendance improve student performance? The Journal of Economic Education, 37(2), 148–155. https://doi.org/10.3200/JECE.37.2.148-155

57.

Marks

H. M.

Printy

S. M.

(2003). Principal leadership and school performance: An integration of transformational and instructional leadership. Educational Administration Quarterly, 39(3), 370–397. https://doi.org/10.1177/0013161X03253412

58.

Marzano

R. J.

Marzano

J. S.

Pickering

D. J.

(2003). Classroom management that works. Research-based strategies for every teacher. Pearson Education.

59.

McGraw

K. O.

Wong

S. P.

(1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30–46. https://doi.org/10.1037/1082-989X.1.1.30

60.

McGuigan

(2005). The role of enabling bureaucracy and academic optimism in academic achievement growth. The Ohio State University.

61.

Mortimore

Sammons

Stoll

Lewis

Ecob

(1988). School matters: The junior years. Paul Chapman.

62.

Nosek

B. A.

Errington

T. M.

(2020). What is replication? PLoS Biology, 18(3), Article e3000691. https://doi.org/10.1371/journal.pbio.3000691

63.

Nunnery

J. A.

Bernstein

I. H.

(1994). Psychometric Theory. New York, NY: McGraw-Hill.

64.

OECD . (2013). Lessons from PISA 2012 for the United States, strong performers and successful reformers in education. OECD Publishing. https://doi.org/10.1787/9789264207585-en

65.

Podsakoff

P. M.

MacKenzie

S. B.

Moorman

R. H.

Fetter

(1990). Transformational leader behaviors and their effects on followers’ trust in leader, satisfaction, and organizational citizenship behaviors. The Leadership Quarterly, 1(2), 107–142. https://doi.org/10.1016/1048-9843(90)90009-7

66.

Reynolds

Sammons

Stoll

Barber

Hillman

(1996). School effectiveness and school improvement in the United Kingdom. School Effectiveness and School Improvement, 7(2), 133–158. https://doi.org/10.1080/0924345960070203

67.

Robinson

V. M. J.

Lloyd

C. A.

Rowe

K. J.

(2008). The impact of leadership on student outcomes: An analysis of the differential effects of leadership types. Educational Administration Quarterly, 44(5), 635–674. https://doi.org/10.1177/0013161X08321509

68.

Rosenshine

B. V.

(1978). Academic engaged time, content covered, and direct instruction. Journal of Education, 160(3), 38–66. https://doi.org/10.1177/002205747816000304

69.

Rosenshine

B. V.

(1981). How time is spent in elementary classrooms. Journal of Classroom Interaction, 17(1), 16–25. https://doi.org/10.1086/461325

70.

Sammons

(1994). Findings from school effectiveness research: Some implications for improving the quality of schools. In Ribbins

Burridge

(Eds.), Improving education: Promoting quality in schools (pp. 32–51). Cassell.

71.

Sammons

(1995). Key characteristics of effective schools: A review of school effectiveness research. London University, Institute of Education.

72.

Scherer

(2020). The case for good discipline? Evidence on the interplay between disciplinary climate, socioeconomic status, and science achievement from PISA 2015. In Equity, equality and diversity in the Nordic model of education (pp. 197–224). Springer Nature.

73.

Schumacker

R. E.

Lomax

R. G.

(2016). A beginner’s guide to structural equation modeling (4th. ed.). Routledge.

74.

Louis

Dretzke

Wahlstrom

(2010). How does leadership affect student achievement? Results from a national US survey, School Effectiveness and School Improvement, 21(3), 315–336, https://doi.org/10.1080/09243453.2010.486586

75.

Sebastian

Allensworth

(2012). The influence of principal leadership on classroom instruction and student learning: A study of mediated pathways to learning. Educational Administration Quarterly, 48(4), 626–663. https://doi.org/10.1177/0013161X11436273

76.

Smith

P. A.

Escobedo

Kearney

W. S.

(2020). Principal influence: A catalyst for positive school climate. International Journal of Education Policy and Leadership, 16(5), 1–16. https://doi.org/10.22230/ijepl.2020v16n5a961

77.

Snijders

T. A. B.

Bosker

R. J.

(1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. Sage Publications.

78.

Steiger

J. H.

(2007). Understanding the limitations of global fit assessment in structural equation modeling. Personality and Individual Differences, 42(5), 893–898. https://doi.org/10.1016/j.paid.2006.09.017

79.

Sun

Leithwood

(2015a). Leadership effects on student learning mediated by teacher emotions. Societies, 5(3), 566–582. https://doi.org/10.3390/soc5030566

80.

Sun

J.-P.

Leithwood

(2012). Transformational school leadership effects on student achievement. Leadership and Policy in Schools, 11(4), 418–451. https://doi.org/10.1080/15700763.2012.681001

81.

Sun

J.-P.

Leithwood

(2015b). Direction-setting school leadership practices: A meta-analytical review of evidence about their influence. School Effectiveness and School Improvement, 26(4), 499–523. https://doi.org/10.1080/09243453.2015.1005106

82.

Sun

J.-P.

Leithwood

(2017). Calculating the power of alternative choices by school leaders forimproving student achievement. School Leadership & Management, 37(1-2), 80–93. https://doi.org/10.1080/13632434.2017.1293635

83.

Teddlie

Stringfield

(2007). A history of school effectiveness and school improvement research in the USA focusing on the past quarter century. In Townsend

(Ed.), International handbook of school effectiveness and improvement (pp. 131–166). Springer.

84.

Tornroos

(2005). Mathematics textbooks, opportunity to learn and student achievement. Studies In Educational Evaluation, 31(4), 315–327. https://doi.org/10.1016/j.stueduc.2005.11.005

85.

Mierlo

Vermunt

J. K.

Rutte

C. G.

(2009). Composing group-level constructs from individual-level survey data. Organizational Research Methods, 12(2), 368–392. https://doi.org/10.1177/1094428107309322

86.

Wang

(1998). Opportunity to learn: The impacts and policy implications. Educational Evaluation and Policy Analysis, 20(3), 137–156. https://doi.org/10.2307/1164489

87.

Wang

M. C.

Haertel

G. D.

Walberg

H. J.

(1993). Toward a knowledge base for school learning. Review of Educational Research, 63(3), 249–294. https://doi.org/10.3102/00346543063003249

88.

Wasserstein

Lazar

(2016). The ASA statement on p-values: Context, process, and purpose. The American Statistician, 70(2), 129–133. https://doi.org/10.1080/00031305.2016.1154108

89.

Wasserstein

R. L.

Schirm

A. L.

Lazar

N. A.

(2019). Moving to a world beyond “p < 0.05”. The American Statistician, 73(sup1), 1–19. https://doi.org/10.1080/00031305.2019.1583913

90.

Weiner

J. M.

Higgins

M. C.

(2017). Where the two shall meet: Exploring the relationship between teacher professional culture and student learning culture. Journal of Educational Change, 18(1), 21–48. https://doi.org/10.1007/s10833-016-9292-6

91.

Wineburg

S. S.

(1987). The self-fulfillment of the self-fulfilling prophecy. Educational Researcher, 16(9), 28–37. https://doi.org/10.3102/0013189X016009028

92.

Wright

B. D.

Mead

Draba

R. E.

(1976). Detecting and correcting test item bias with a logistic response model (No.22; Research Memorandum). Statistical Laboratory, Department of Education, University of Chicago.

93.

Wright

B. D.

Panchapakesan

(1969). A procedure for sample-free item analysis. Educational and Psychological Measurement, 29(1), 23–48. https://doi.org/10.1177/001316446902900102

94.

Youou

Yang

Uzzi

(2023). A discipline-wide investigation of the replicability of Psychology papers over the past two decades. Pschological and Cognitive Sciences, 120(6), Article e2208863120. https://doi.org/10.1073/pnas.2208863120