This study investigated causal links between states’ principal evaluation policies and academic achievement among Black and Hispanic students, focusing on two policy levers: (a) required inclusion of student growth data (output-based) and (b) mandated observations or site visits (input-based). We used repeated cross-sectional data on policy levers collected at eight time points during 2005 to 2019 and a counterfactual estimation framework for causal inferences. States enacting the output-based policy lever did not typically have higher reading or math achievement scores on the National Assessment of Educational Progress among Black and Hispanic eighth-grade students than states without these policies. In contrast, the input-based policy lever had a significant positive effect on eighth-grade reading achievement scores, particularly among Black students. The importance of focusing on principal behaviors rather than student achievement is discussed for enhancing school accountability with racial equity.
AlexanderN. A.JangS. T. (2020). Policy, poverty, and student achievement: An exploration of the impact of state policies. Educational Policy, 34(4), 674–704.
2.
AlexanderN.JangS. T.KankaneS. (2017). The performance cycle: The association between student achievement and state policies tying together teacher performance and accountability. American Journal of Education, 123(3), 413–446.
AndersonL. M.TurnbullB. J. (2016). Evaluating and supporting principals. Building a stronger principalship. Policy Studies Associates, Inc.
5.
AtwoodE. D.JimersonJ. B.HoltB. (2019). Equity-oriented data use: Identifying and addressing food insecurity at Cooper Springs Middle School. Journal of Cases in Educational Leadership, 22(3), 70–84.
6.
BakerE. L.BartonP. E.Darling-HammondL.HaertelE.LaddH. F.LinnR. L.RavitchD.RothsteinR.ShavelsonR. J.ShepardL. A. (2010). Problems with the use of student test scores to evaluate teachers. Economic Policy Institute.
7.
BartanenB.HusainA. N.LiebowitzD. D. (2022). Rethinking principal effects on student outcomes (EdWorkingPaper: 22-621). Annenberg Institute at Brown University. https://doi.org/10.26300/r5sf-3918
8.
BartanenB.HusainA. N.LiebowitzD. D.RogersL. K. (2024). The returns to experience for school principals (EdWorkingPaper 24-978). Annenberg Institute at Brown University. https://doi.org/10.26300/qq95-3q14
9.
BellD. A. (1995). Who’s afraid of critical race theory?University of Illinois Law Review, 1995(4), 893–910.
10.
BellD. (2008). Ethical ambition: Living a life of meaning and worth. Bloomsbury.
11.
BleibergJ.BrunnerE.HarbatkinE.KraftM. A.SpringerM. G. (2021). The effect of teacher evaluation on achievement and attainment: Evidence from statewide reforms. Annenberg Institute at Brown University. https://doi.org/10.26300/b1akrr251
12.
BormanG. D.HewesG. M.OvermanL. T.BrownS. (2002). Comprehensive school reform and student achievement: A meta-analysis. Center for Research on the Education of Students Placed At Risk.
13.
CallawayB.Sant’AnnaP. H. (2021). Difference-in-differences with multiple time periods. Journal of Econometrics, 225(2), 200–230.
14.
CareyR. L. (2014). A cultural analysis of the achievement gap discourse: Challenging the language and labels used in the work of school reform. Urban Education, 49(4), 440–468.
15.
CliffordM.HansenU. J.WraightS. (2014). A practical guide to designing comprehensive principal evaluation systems: A tool to assist in the development of principal evaluation systems. American Institutes for Research.
16.
CohenJ. (1988). Statistical power analysis for the behavioral sciences. Lawrence Erlbaum.
17.
DanielsonC.McGrealT. L. (2000). Teacher evaluation to enhance professional practice. Association for Supervision and Curriculum Development.
18.
Darling-HammondL. (2007). Race, inequality and educational accountability: The irony of “No Child Left Behind.”Race Ethnicity and Education, 10(3), 245–260.
19.
Darling-HammondL. (2010). The flat world and education: How America’s commitment to equity will determine our future. Teachers College Press.
20.
Darling-HammondL. (2015). Can value added add value to teacher evaluation?Educational Researcher, 44(2), 132–137.
21.
DatnowA.ParkV. (2018). Opening or closing doors for students? Equity and data use in schools. Journal of Educational Change, 19(2), 131–152.
22.
DavisS.KearneyK.SandersN.ThomasC.LeonR. (2011). The policies and practices of principal evaluation: A review of the literature. WestEd.
23.
de ChaisemartinC.d’HaultfoeuilleX. (2020). Two-way fixed effects estimators with heterogeneous treatment effects. American Economic Review, 110(9), 2964–2996.
24.
DelpitL. D. (2012). Multiplication is for white people: Raising expectations for other people’s children. The New Press.
25.
DeMatthewsD.MawhinneyH. (2014). Social justice leadership and inclusion: Exploring challenges in an urban district struggling to address inequities. Educational Administration Quarterly, 50(5), 844–881.
26.
DiemS.YoungM. D.WeltonA. D.MansfieldK. C.LeeP. (2014) The intellectual landscape of critical policy analysis. International Journal of Qualitative Studies in Education, 27(9), 1068–1090.
27.
DonaldsonM.MavrogordatoM.DoughertyS. M.GhanemR. A.YoungsP. (2021). Principal evaluation under the Elementary and Secondary Every Student Succeeds Act: A comprehensive policy review. Education Finance and Policy, 16(2), 347–361.
28.
DonaldsonM.MavrogordatoM.YoungsP.DaughertyS. (2020). Appraising principal evaluation and development: Current research and future directions. In YoungsP.KimJ.MavrogordatoM. (Eds.), Exploring principal development and teacher outcomes (pp. 56–68). Routledge.
29.
DonaldsonM. L.FirestoneW. (2021). Rethinking teacher evaluation using human, social, and material capital. Journal of Educational Change, 22(4), 501–534.
ElmoreR. F.FuhrmanS. H. (1995). Opportunity-to-learn standards and the state role in education. Teachers College Record, 96(3), 1–26.
32.
FirestoneW. A. (2014). Teacher evaluation policy and conflicting theories of motivation. Educational Researcher, 43(2), 100–107.
33.
FullerE. J.HollingworthL.LiuJ. (2015). Evaluating state principal evaluation plans across the United States. Journal of Research on Leadership Education, 10(3), 164–192.
34.
Galey-HornS.WoulfinS. L. (2021). Muddy waters: The micropolitics of instructional coaches’ work in evaluation. American Journal of Education, 127(3), 441–470.
35.
GallagherC. (2000). A seat at the table: Teachers reclaiming assessment through rethinking accountability. Phi Delta Kappan, 81(7), 502–507.
36.
GillbornD.WarmingtonP.DemackS. (2018). QuantCrit: Education, policy, “Big Data” and principles for a critical race theory of statistics. Race Ethnicity and Education, 21(2), 158–179.
37.
GoldhaberD.LaveryL.TheobaldR. (2015). Uneven playing field? Assessing the teacher quality gap between advantaged and disadvantaged students. Educational Researcher, 44(5), 293–307.
38.
GoldringE. B.ClarkM. A.RubinM.RogersL. K.GrissomJ. A.GillB.KautzT.McCulloughM.NeelM.BurnettA. (2020). Changing the principal supervisor role to better support principals: Evidence from the Principal Supervisor Initiative. Mathematica.
39.
GoldringE. B.CravensX. C.MurphyJ.PorterA. C.ElliottS. N.CarsonB. (2009). The evaluation of principals: What and how do states and urban districts assess leadership?. The Elementary School Journal, 110(1), 19–39.
40.
GoldringE. B.MavrogordatoM.HaynesK. T. (2015). Multisource principal evaluation data: Principals’ orientations and reactions to teacher feedback regarding their leadership effectiveness. Educational Administration Quarterly, 51(4), 572–599.
41.
Goodman-BaconA. (2021). Difference-in-differences with variation in treatment timing. Journal of Econometrics, 225(2), 254–277.
42.
GrissomJ. A.KalogridesD.LoebS. (2015). Using student test scores to measure principal performance. Educational Evaluation and Policy Analysis, 37(1), 3–28.
43.
GrissomJ. A.LoebS. (2017). Assessing principals’ assessments: Subjective evaluations of teacher effectiveness in low-and high-stakes environments. Education Finance and Policy, 12(3), 369–395.
44.
HallingerP.GümüşS.BellibaşM. Ş. (2020). “Are principals instructional leaders yet?” A science map of the knowledge base on instructional leadership, 1940–2018. Scientometrics, 122(3), 1629–1650.
45.
HanushekE. A. (2003). The failure of input-based schooling policies. Economic Journal, 113(485), F64–F98.
46.
HanushekE. A.RaymondM. E. (2005). Does school accountability lead to improved student performance?Journal of Policy Analysis and Management, 24, 297–327.
47.
HanushekE. A.RivkinS. G. (2007). Pay, working conditions, and teacher quality. The Future of Children, 17(1), 69–86.
48.
HarrisD. N.HerringtonC. D. (2006). Accountability, standards, and the growing achievement gap: Lessons from the past half-century. American Journal of Education, 112(2), 209–238.
49.
Illinois Principals Association. (2024). Illinois performance standards for school leaders. https://ilprincipals.org/
JacksonC. K.JohnsonR. C.PersicoC. (2015). The effects of school spending on educational and economic outcomes: Evidence from school finance reforms (No. w20847). National Bureau of Economic Research.
52.
JacquesC.CliffordM.HornungK. (2012). State policies on principal evaluation: Trends in a changing landscape. National Comprehensive Center for Teacher Quality.
53.
JangS. T. (2018). The implications of intersectionality of race, gender, and socioeconomic status on Southeast Asian female students’ educational outcomes: Critical quantitative intersectionality analysis. American Educational Research Journal, 55(6), 1268–1306.
54.
JangS. T. (2023). Schooling experiences and educational outcomes of Latinx secondary school students living at the intersections of multiple social constructs. Urban Education, 58(4), 708–739.
55.
JangS. T.AlexanderN. A. (2025). Exploring the link between states’ principal evaluation policies and student achievement. Educational Administration Quarterly. Advance online publication. https://doi.org/10.1177/0013161X251322727
56.
JonesN. D.BellC. A.BrownellM.QiY.PeytonD.PuaD.FowlerM.HoltzmanS. (2022). Using classroom observations in the evaluation of special education teachers. Educational Evaluation and Policy Analysis, 44(3), 429–457.
57.
KeeleL.StevensonR. T. (2021). Causal interaction and effect modification: Same model, different concepts. Political Science Research and Methods, 9(3), 641–649.
58.
KhalifaM. A.GoodenM. A.DavisJ. E. (2016). Culturally responsive school leadership: A synthesis of the literature. Review of Educational Research, 86(4), 1272–1311.
59.
KimballS. M.MilanowskiA. (2009). Examining teacher evaluation validity and leadership decision making within a standards-based evaluation system. Educational Administration Quarterly, 45(1), 34–70.
60.
KoretzD. (2008). Measuring up: What educational testing really tells us. Harvard University Press.
61.
KraftM. A.BrunnerE. J.DoughertyS. M.SchwegmanD. J. (2020). Teacher accountability reforms and the supply and quality of new teachers. Journal of Public Economics, 188, 1–24.
62.
KraftM. A.GilmourA. F. (2017). Revisiting the widget effect: Teacher evaluation reforms and the distribution of teacher effectiveness. Educational Researcher, 46(5), 234–249.
63.
KruegerA. B. (1999). Experimental estimates of education production functions. The Quarterly Journal of Economics, 114(2), 497–532.
64.
Ladson-BillingsG. J. (1999). Preparing teachers for diverse student populations: A critical race theory perspective. Review of Research in Education, 24(1), 211–247.
65.
Ladson-BillingsG. (2006). From the achievement gap to the education debt: Understanding achievement in U.S. Schools. Educational Researchers, 35(7), 3–12.
66.
LascoumesP.Le GalesP. (2007). Introduction: Understanding public policy through its instruments—From the nature of instruments to the sociology of public policy instrumentation. Governance: An International Journal of Policy, Administration, and Institutions, 20(1), 1–21.
67.
LauerP. A.AkibaM.WilkersonS. B.ApthorpH. S.SnowD.Martin-GlennM. (2004). The effectiveness of out-of-school-time strategies in assisting low-achieving students in reading and mathematics: A research synthesis. Institute of Education Sciences, Department of Education.
LeeJ. (2016). The anatomy of achievement gaps: Why and how American education is losing (but can still win) the war on underachievement. Oxford University Press.
70.
LeeJ.LeeM. (2020). Is “whole child” education obsolete? Public school principals’ educational goal priorities in the era of accountability. Educational Administration Quarterly, 56(5), 856–884.
71.
LeeJ.WongK. K. (2004). The impact of accountability on racial and socioeconomic equity: Considering both school resources and achievement outcomes. American Educational Research Journal, 41(4), 797–832.
72.
LevinsonB. A. U.SuttonM.WinsteadT. (2009). Education policy as a practice of power: Theoretical tools, ethnographic methods, democratic options. Educational Policy, 23(6), 767–795.
LiuL.WangY.XuY. (2024). A practical guide to counterfactual estimators for causal inference with time-series cross-sectional data. American Journal of Political Science, 68(1), 160–176.
NielsenS. R.LavigneA. L. (2020). Principal evaluation in the United States: A national review of state statutes and regulations. Education Policy Analysis Archives, 28(143), 1–36.
82.
O’ConnorS. D. & Supreme Court of the United States. (2002) U.S. Reports: Grutter v. Bollinger et al., 539 U.S. 306 [Periodical]. The Library of Congress. https://www.loc.gov/item/usrep539306/
83.
PutmanH.WalshK. (2021). State of the states 2021: Teacher preparation policy. National Council on Teacher Quality.
84.
RaddS. I.GenerettG. G.GoodenM. A.TheoharisG. (2021). Five practices for equity-focused school leadership. ASCD.
85.
R Core Team. (2023). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/
86.
ReardonS. F.ArshanN.AtteberryA.KurlaenderM. (2010). Effects of failing a high school exit exam on course taking, achievement, persistence, and graduation. Educational Evaluation and Policy Analysis, 32(4), 498–520.
87.
ReardonS. F.ValentinoR. A.KalogridesD.ShoresK. A.GreenbergE. H. (2013). Patterns and trends in racial academic achievement gaps among states, 1999–2011. Stanford University Graduate School of Education.
88.
ReardonS. F.WeathersE.S.FahleE.M.JangH.KalogridesD. (2022). Is separate still unequal? New evidence on school segregation and racial academic achievement gaps. Center for Education Policy Analysis.
89.
RobinsonV. M.LloydC. A.RoweK. J. (2008). The impact of leadership on student outcomes: An analysis of the differential effects of leadership types. Educational Administration Quarterly, 44(5), 635–674.
90.
RossE.WalshK. (2019). State of the states 2019: Teacher and principal evaluation policy. National Council on Teacher Quality.
91.
ScheurichJ. J.SkrlaL.JohnsonJ. F.Jr. (2004). Thinking carefully about equity and accountability. In SkrlaL.ScheurichJ. J. (Eds.), Educational equity and accountability: Paradigms, policies and politics (pp. 13–28). Psychology Press.
92.
SchmidtW. H.McKnightC. C. (2012). Inequality for all: The challenge of unequal opportunity in American Schools. Teachers College Press.
93.
SchneiderA.IngramH. (1990). Behavioral assumptions of policy tools. The Journal of Politics, 52(2), 510–529.
94.
SkrlaL.ScheurichJ. J.JohnsonJ. F.KoschoreckJ. W. (2004). Accountability for equity: Can state policy leverage social justice? In SkrlaL.ScheurichJ. J. (Eds.), Educational equity and accountability: Paradigms, policies, and politics (pp. 51–78). RoutledgeFalmer.
95.
SolórzanoD. G. (1998). Critical race theory, racial and gender microaggressions, and the experiences of Chicana and Chicano scholars. International Journal of Qualitative Studies in Education, 11(1), 121–136.
96.
SunM.YoungsP. (2009). How does district principal evaluation affect learning-centered principal leadership? Evidence from Michigan school districts. Leadership and Policy in Schools, 8(4), 411–445.
TheoharisG. (2007). Social justice educational leaders and resistance: Toward a theory of social justice leadership. Educational Administration Quarterly, 43(2), 221–258.
99.
VernezG.ZimmerR. (2007). Interpreting the effects of Title I supplemental educational services. RAND.
100.
WaldJ. M.LosenD. J. (2003). Deconstructing the school-to-prison pipeline. Jossey-Bass.
101.
WangJ.GoldschmidtP. (2003). Importance of middle school mathematics on high school students’ mathematics achievement. The Journal of Educational Research, 97(1), 3–17.
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
