The Equity Implications of School Principal Evaluation for Student Achievement: A Critical Quantitative Policy Analysis

Abstract

This study investigated causal links between states’ principal evaluation policies and academic achievement among Black and Hispanic students, focusing on two policy levers: (a) required inclusion of student growth data (output-based) and (b) mandated observations or site visits (input-based). We used repeated cross-sectional data on policy levers collected at eight time points during 2005 to 2019 and a counterfactual estimation framework for causal inferences. States enacting the output-based policy lever did not typically have higher reading or math achievement scores on the National Assessment of Educational Progress among Black and Hispanic eighth-grade students than states without these policies. In contrast, the input-based policy lever had a significant positive effect on eighth-grade reading achievement scores, particularly among Black students. The importance of focusing on principal behaviors rather than student achievement is discussed for enhancing school accountability with racial equity.

Keywords

principal evaluation principal accountability educational leadership equity Black and Hispanic Students critical quantitative policy analysis

Get full access to this article

View all access options for this article.

References

Alexander

N. A.

Jang

S. T.

(2020). Policy, poverty, and student achievement: An exploration of the impact of state policies. Educational Policy, 34(4), 674–704.

Alexander

Jang

S. T.

Kankane

(2017). The performance cycle: The association between student achievement and state policies tying together teacher performance and accountability. American Journal of Education, 123(3), 413–446.

American College Testing. (2008). The forgotten middle. https://www.act.org/content/dam/act/unsecured/documents/ForgottenMiddle.pdf

Anderson

L. M.

Turnbull

B. J.

(2016). Evaluating and supporting principals. Building a stronger principalship. Policy Studies Associates, Inc.

Atwood

E. D.

Jimerson

J. B.

Holt

(2019). Equity-oriented data use: Identifying and addressing food insecurity at Cooper Springs Middle School. Journal of Cases in Educational Leadership, 22(3), 70–84.

Baker

E. L.

Barton

P. E.

Darling-Hammond

Haertel

Ladd

H. F.

Linn

R. L.

Ravitch

Rothstein

Shavelson

R. J.

Shepard

L. A.

(2010). Problems with the use of student test scores to evaluate teachers. Economic Policy Institute.

Bartanen

Husain

A. N.

Liebowitz

D. D.

(2022). Rethinking principal effects on student outcomes (EdWorkingPaper: 22-621). Annenberg Institute at Brown University. https://doi.org/10.26300/r5sf-3918

Bartanen

Husain

A. N.

Liebowitz

D. D.

Rogers

L. K.

(2024). The returns to experience for school principals (EdWorkingPaper 24-978). Annenberg Institute at Brown University. https://doi.org/10.26300/qq95-3q14

Bell

D. A.

(1995). Who’s afraid of critical race theory? University of Illinois Law Review, 1995(4), 893–910.

10.

Bell

(2008). Ethical ambition: Living a life of meaning and worth. Bloomsbury.

11.

Bleiberg

Brunner

Harbatkin

Kraft

M. A.

Springer

M. G.

(2021). The effect of teacher evaluation on achievement and attainment: Evidence from statewide reforms. Annenberg Institute at Brown University. https://doi.org/10.26300/b1akrr251

12.

Borman

G. D.

Hewes

G. M.

Overman

L. T.

Brown

(2002). Comprehensive school reform and student achievement: A meta-analysis. Center for Research on the Education of Students Placed At Risk.

13.

Callaway

Sant’Anna

P. H.

(2021). Difference-in-differences with multiple time periods. Journal of Econometrics, 225(2), 200–230.

14.

Carey

R. L.

(2014). A cultural analysis of the achievement gap discourse: Challenging the language and labels used in the work of school reform. Urban Education, 49(4), 440–468.

15.

Clifford

Hansen

U. J.

Wraight

(2014). A practical guide to designing comprehensive principal evaluation systems: A tool to assist in the development of principal evaluation systems. American Institutes for Research.

16.

Cohen

(1988). Statistical power analysis for the behavioral sciences. Lawrence Erlbaum.

17.

Danielson

McGreal

T. L.

(2000). Teacher evaluation to enhance professional practice. Association for Supervision and Curriculum Development.

18.

Darling-Hammond

(2007). Race, inequality and educational accountability: The irony of “No Child Left Behind.” Race Ethnicity and Education, 10(3), 245–260.

19.

Darling-Hammond

(2010). The flat world and education: How America’s commitment to equity will determine our future. Teachers College Press.

20.

Darling-Hammond

(2015). Can value added add value to teacher evaluation? Educational Researcher, 44(2), 132–137.

21.

Datnow

Park

(2018). Opening or closing doors for students? Equity and data use in schools. Journal of Educational Change, 19(2), 131–152.

22.

Davis

Kearney

Sanders

Thomas

Leon

(2011). The policies and practices of principal evaluation: A review of the literature. WestEd.

23.

de Chaisemartin

d’Haultfoeuille

. (2020). Two-way fixed effects estimators with heterogeneous treatment effects. American Economic Review, 110(9), 2964–2996.

24.

Delpit

L. D.

(2012). Multiplication is for white people: Raising expectations for other people’s children. The New Press.

25.

DeMatthews

Mawhinney

(2014). Social justice leadership and inclusion: Exploring challenges in an urban district struggling to address inequities. Educational Administration Quarterly, 50(5), 844–881.

26.

Diem

Young

M. D.

Welton

A. D.

Mansfield

K. C.

Lee

(2014) The intellectual landscape of critical policy analysis. International Journal of Qualitative Studies in Education, 27(9), 1068–1090.

27.

Donaldson

Mavrogordato

Dougherty

S. M.

Ghanem

R. A.

Youngs

(2021). Principal evaluation under the Elementary and Secondary Every Student Succeeds Act: A comprehensive policy review. Education Finance and Policy, 16(2), 347–361.

28.

Donaldson

Mavrogordato

Youngs

Daugherty

(2020). Appraising principal evaluation and development: Current research and future directions. In Youngs

Kim

Mavrogordato

(Eds.), Exploring principal development and teacher outcomes (pp. 56–68). Routledge.

29.

Donaldson

M. L.

Firestone

(2021). Rethinking teacher evaluation using human, social, and material capital. Journal of Educational Change, 22(4), 501–534.

30.

Education Commission of the States. (2023). State policy strategies to improve educational equity. https://www.ecs.org/state-policy-strategies-to-improve-educational-equity/

31.

Elmore

R. F.

Fuhrman

S. H.

(1995). Opportunity-to-learn standards and the state role in education. Teachers College Record, 96(3), 1–26.

32.

Firestone

W. A.

(2014). Teacher evaluation policy and conflicting theories of motivation. Educational Researcher, 43(2), 100–107.

33.

Fuller

E. J.

Hollingworth

Liu

(2015). Evaluating state principal evaluation plans across the United States. Journal of Research on Leadership Education, 10(3), 164–192.

34.

Galey-Horn

Woulfin

S. L.

(2021). Muddy waters: The micropolitics of instructional coaches’ work in evaluation. American Journal of Education, 127(3), 441–470.

35.

Gallagher

(2000). A seat at the table: Teachers reclaiming assessment through rethinking accountability. Phi Delta Kappan, 81(7), 502–507.

36.

Gillborn

Warmington

Demack

(2018). QuantCrit: Education, policy, “Big Data” and principles for a critical race theory of statistics. Race Ethnicity and Education, 21(2), 158–179.

37.

Goldhaber

Lavery

Theobald

(2015). Uneven playing field? Assessing the teacher quality gap between advantaged and disadvantaged students. Educational Researcher, 44(5), 293–307.

38.

Goldring

E. B.

Clark

M. A.

Rubin

Rogers

L. K.

Grissom

J. A.

Gill

Kautz

McCullough

Neel

Burnett

(2020). Changing the principal supervisor role to better support principals: Evidence from the Principal Supervisor Initiative. Mathematica.

39.

Goldring

E. B.

Cravens

X. C.

Murphy

Porter

A. C.

Elliott

S. N.

Carson

(2009). The evaluation of principals: What and how do states and urban districts assess leadership?. The Elementary School Journal, 110(1), 19–39.

40.

Goldring

E. B.

Mavrogordato

Haynes

K. T.

(2015). Multisource principal evaluation data: Principals’ orientations and reactions to teacher feedback regarding their leadership effectiveness. Educational Administration Quarterly, 51(4), 572–599.

41.

Goodman-Bacon

(2021). Difference-in-differences with variation in treatment timing. Journal of Econometrics, 225(2), 254–277.

42.

Grissom

J. A.

Kalogrides

Loeb

(2015). Using student test scores to measure principal performance. Educational Evaluation and Policy Analysis, 37(1), 3–28.

43.

Grissom

J. A.

Loeb

(2017). Assessing principals’ assessments: Subjective evaluations of teacher effectiveness in low-and high-stakes environments. Education Finance and Policy, 12(3), 369–395.

44.

Hallinger

Gümüş

Bellibaş

M. Ş

. (2020). “Are principals instructional leaders yet?” A science map of the knowledge base on instructional leadership, 1940–2018. Scientometrics, 122(3), 1629–1650.

45.

Hanushek

E. A.

(2003). The failure of input-based schooling policies. Economic Journal, 113(485), F64–F98.

46.

Hanushek

E. A.

Raymond

M. E.

(2005). Does school accountability lead to improved student performance? Journal of Policy Analysis and Management, 24, 297–327.

47.

Hanushek

E. A.

Rivkin

S. G.

(2007). Pay, working conditions, and teacher quality. The Future of Children, 17(1), 69–86.

48.

Harris

D. N.

Herrington

C. D.

(2006). Accountability, standards, and the growing achievement gap: Lessons from the past half-century. American Journal of Education, 112(2), 209–238.

49.

Illinois Principals Association. (2024). Illinois performance standards for school leaders. https://ilprincipals.org/

50.

Indiana Department of Education. (n.d.). RISE Evaluation model: Evaluator and principal handbook version 3.0. https://www.in.gov/doe/files/rise-handbook-principals.pdf

51.

Jackson

C. K.

Johnson

R. C.

Persico

(2015). The effects of school spending on educational and economic outcomes: Evidence from school finance reforms (No. w20847). National Bureau of Economic Research.

52.

Jacques

Clifford

Hornung

(2012). State policies on principal evaluation: Trends in a changing landscape. National Comprehensive Center for Teacher Quality.

53.

Jang

S. T.

(2018). The implications of intersectionality of race, gender, and socioeconomic status on Southeast Asian female students’ educational outcomes: Critical quantitative intersectionality analysis. American Educational Research Journal, 55(6), 1268–1306.

54.

Jang

S. T.

(2023). Schooling experiences and educational outcomes of Latinx secondary school students living at the intersections of multiple social constructs. Urban Education, 58(4), 708–739.

55.

Jang

S. T.

Alexander

N. A.

(2025). Exploring the link between states’ principal evaluation policies and student achievement. Educational Administration Quarterly. Advance online publication. https://doi.org/10.1177/0013161X251322727

56.

Jones

N. D.

Bell

C. A.

Brownell

Peyton

Pua

Fowler

Holtzman

(2022). Using classroom observations in the evaluation of special education teachers. Educational Evaluation and Policy Analysis, 44(3), 429–457.

57.

Keele

Stevenson

R. T.

(2021). Causal interaction and effect modification: Same model, different concepts. Political Science Research and Methods, 9(3), 641–649.

58.

Khalifa

M. A.

Gooden

M. A.

Davis

J. E.

(2016). Culturally responsive school leadership: A synthesis of the literature. Review of Educational Research, 86(4), 1272–1311.

59.

Kimball

S. M.

Milanowski

(2009). Examining teacher evaluation validity and leadership decision making within a standards-based evaluation system. Educational Administration Quarterly, 45(1), 34–70.

60.

Koretz

(2008). Measuring up: What educational testing really tells us. Harvard University Press.

61.

Kraft

M. A.

Brunner

E. J.

Dougherty

S. M.

Schwegman

D. J.

(2020). Teacher accountability reforms and the supply and quality of new teachers. Journal of Public Economics, 188, 1–24.

62.

Kraft

M. A.

Gilmour

A. F.

(2017). Revisiting the widget effect: Teacher evaluation reforms and the distribution of teacher effectiveness. Educational Researcher, 46(5), 234–249.

63.

Krueger

A. B.

(1999). Experimental estimates of education production functions. The Quarterly Journal of Economics, 114(2), 497–532.

64.

Ladson-Billings

G. J.

(1999). Preparing teachers for diverse student populations: A critical race theory perspective. Review of Research in Education, 24(1), 211–247.

65.

Ladson-Billings

(2006). From the achievement gap to the education debt: Understanding achievement in U.S. Schools. Educational Researchers, 35(7), 3–12.

66.

Lascoumes

Le Gales

(2007). Introduction: Understanding public policy through its instruments—From the nature of instruments to the sociology of public policy instrumentation. Governance: An International Journal of Policy, Administration, and Institutions, 20(1), 1–21.

67.

Lauer

P. A.

Akiba

Wilkerson

S. B.

Apthorp

H. S.

Snow

Martin-Glenn

(2004). The effectiveness of out-of-school-time strategies in assisting low-achieving students in reading and mathematics: A research synthesis. Institute of Education Sciences, Department of Education.

68.

Learning Policy Institute. (2020). The federal role in advancing education equity and excellence. https://learningpolicyinstitute.org/product/advancing-education-2020-brief

69.

Lee

(2016). The anatomy of achievement gaps: Why and how American education is losing (but can still win) the war on underachievement. Oxford University Press.

70.

Lee

(2020). Is “whole child” education obsolete? Public school principals’ educational goal priorities in the era of accountability. Educational Administration Quarterly, 56(5), 856–884.

71.

Lee

Wong

K. K.

(2004). The impact of accountability on racial and socioeconomic equity: Considering both school resources and achievement outcomes. American Educational Research Journal, 41(4), 797–832.

72.

Levinson

B. A. U.

Sutton

Winstead

(2009). Education policy as a practice of power: Theoretical tools, ethnographic methods, democratic options. Educational Policy, 23(6), 767–795.

73.

Liu

Wang

(2022). fect: Fixed effects counterfactuals. R package version 1.0.0. https://yiqingxu.org/packages/fect/articles/tutorial.html

74.

Liu

Wang

(2024). A practical guide to counterfactual estimators for causal inference with time-series cross-sectional data. American Journal of Political Science, 68(1), 160–176.

75.

Mandinach

(2021). Culturally responsive data literacy: Transforming data and information into action. https://ies.ed.gov/ncee/edlabs/regions/northeast/Blog/Post/30

76.

McDonnell

L. M.

Elmore

R. F.

(1987). Getting the job done: Alternative policy instruments. Educational Evaluation and Policy Analysis, 9(2), 133–152.

77.

Milner

H. R.

(2012). Beyond a test score: Explaining opportunity gaps in educational practice. Journal of Black Studies, 43(6), 693–718.

78.

Monk

D. H.

(1989). The education production function: Its evolving role in policy analysis. Educational Evaluation and Policy Analysis, 11(1), 31–45.

79.

National Center for Education Statistics. (n.d.). NAEP technical documentation. https://nces.ed.gov/nationsreportcard/tdw/sample_design/

80.

National Council on Teacher Quality. (2022). State teacher policy database. https://www.nctq.org/yearbook/home

81.

Nielsen

S. R.

Lavigne

A. L.

(2020). Principal evaluation in the United States: A national review of state statutes and regulations. Education Policy Analysis Archives, 28(143), 1–36.

82.

O’Connor

S. D.

& Supreme Court of the United States. (2002) U.S. Reports: Grutter v. Bollinger et al., 539 U.S. 306 [Periodical]. The Library of Congress. https://www.loc.gov/item/usrep539306/

83.

Putman

Walsh

(2021). State of the states 2021: Teacher preparation policy. National Council on Teacher Quality.

84.

Radd

S. I.

Generett

G. G.

Gooden

M. A.

Theoharis

(2021). Five practices for equity-focused school leadership. ASCD.

85.

R Core Team. (2023). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/

86.

Reardon

S. F.

Arshan

Atteberry

Kurlaender

(2010). Effects of failing a high school exit exam on course taking, achievement, persistence, and graduation. Educational Evaluation and Policy Analysis, 32(4), 498–520.

87.

Reardon

S. F.

Valentino

R. A.

Kalogrides

Shores

K. A.

Greenberg

E. H.

(2013). Patterns and trends in racial academic achievement gaps among states, 1999–2011. Stanford University Graduate School of Education.

88.

Reardon

S. F.

Weathers

E.S.

Fahle

E.M.

Jang

Kalogrides

(2022). Is separate still unequal? New evidence on school segregation and racial academic achievement gaps. Center for Education Policy Analysis.

89.

Robinson

V. M.

Lloyd

C. A.

Rowe

K. J.

(2008). The impact of leadership on student outcomes: An analysis of the differential effects of leadership types. Educational Administration Quarterly, 44(5), 635–674.

90.

Ross

Walsh

(2019). State of the states 2019: Teacher and principal evaluation policy. National Council on Teacher Quality.

91.

Scheurich

J. J.

Skrla

Johnson

J. F.

Jr. (2004). Thinking carefully about equity and accountability. In Skrla

Scheurich

J. J.

(Eds.), Educational equity and accountability: Paradigms, policies and politics (pp. 13–28). Psychology Press.

92.

Schmidt

W. H.

McKnight

C. C.

(2012). Inequality for all: The challenge of unequal opportunity in American Schools. Teachers College Press.

93.

Schneider

Ingram

(1990). Behavioral assumptions of policy tools. The Journal of Politics, 52(2), 510–529.

94.

Skrla

Scheurich

J. J.

Johnson

J. F.

Koschoreck

J. W.

(2004). Accountability for equity: Can state policy leverage social justice? In Skrla

Scheurich

J. J.

(Eds.), Educational equity and accountability: Paradigms, policies, and politics (pp. 51–78). RoutledgeFalmer.

95.

Solórzano

D. G.

(1998). Critical race theory, racial and gender microaggressions, and the experiences of Chicana and Chicano scholars. International Journal of Qualitative Studies in Education, 11(1), 121–136.

96.

Sun

Youngs

(2009). How does district principal evaluation affect learning-centered principal leadership? Evidence from Michigan school districts. Leadership and Policy in Schools, 8(4), 411–445.

97.

The Nation’s Report Card. (2024). NAEP Long-term trend assessment results: Reading and Mathematics. https://www.nationsreportcard.gov/highlights/ltt/2023/

98.

Theoharis

(2007). Social justice educational leaders and resistance: Toward a theory of social justice leadership. Educational Administration Quarterly, 43(2), 221–258.

99.

Vernez

Zimmer

(2007). Interpreting the effects of Title I supplemental educational services. RAND.

100.

Wald

J. M.

Losen

D. J.

(2003). Deconstructing the school-to-prison pipeline. Jossey-Bass.

101.

Wang

Goldschmidt

(2003). Importance of middle school mathematics on high school students’ mathematics achievement. The Journal of Educational Research, 97(1), 3–17.

102.

Wang

Cheng

(2020). Cross-sectional studies: Strengths, weaknesses, and recommendations. Chest, 158(1), S65–S71.

103.

Young

M. D.

Diem

(2017). Critical approaches to education policy analysis: Moving beyond tradition. Springer.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.83 MB