Teacher evaluation systems and their associated incentives have produced fairly mixed results. Our analyses are motivated by theory and descriptive evidence that accountability systems are highly racialized, and that individuals are less likely to respond to incentives when they have low expectations of success (and vice versa). Using a regression discontinuity design, we find that Black novices in the District of Columbia Public Schools faced the most negative consequences (dismissal threats) and the least benefits (salary incentives), without responding to either. White novices, in contrast, exhibited high expectations of success and large behavior changes, particularly in response to dismissal threats (0.6 SD). We also find some evidence of heterogeneity in effects by task difficulty, often used as a proxy for expectancy, though these differences are less stark.
AchinsteinB.OgawaR. T. (2012). New teachers of color and culturally responsive teaching in an era of educational accountability: Caught in a double bind. Journal of Educational Change, 13, 1–39.
2.
AdnotM.DeeT.KatzV.WyckoffJ. (2017). Teacher turnover, teacher quality, and student achievement in DCPS. Educational Evaluation and Policy Analysis, 39(1), 54–76.
3.
AfridiF.LiS. X.RenY. (2015). Social identity and inequality: The impact of China’s hukou system. Journal of Public Economics, 123, 17–29.
4.
AkerlofG. A.KrantonR. E. (2000). Economics and identity. The Quarterly Journal of Economics, 115(3), 715–753.
5.
AkerlofG. A.KrantonR. E. (2005). Identity and the economics of organizations. Journal of Economic Perspectives, 19(1), 9–32.
6.
AtkinsonJ. W. (1957). Motivational determinants of risk-taking behavior. Psychological Review, 64(6p1), 359–372.
7.
AuW. (2016). Meritocracy 2.0: High-stakes, standardized testing as a racial project of neoliberal multiculturalism. Educational Policy, 30(1), 39–62.
8.
BenjaminiY.HochbergY. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1), 289–300.
9.
BettsJ. R.CostrellR. M.WalbergH. J.PhillipsM.ChinT. (2001). Incentives and equity under standards-based reform. Brookings Papers on Education Policy, (4), 9–74.
10.
BlazarD. (2024). Why Black teachers matter. Educational Researcher, 53(8), 450–463.
11.
BlazarD.KraftM. A. (2017). Teacher and teaching effects on students’ attitudes and behaviors. Educational Evaluation and Policy Analysis, 39(1), 146–170.
12.
BleibergJ.BrunnerE.HarbatkinE.KraftM. A.SpringerM. G. (2023). Taking teacher evaluation to scale: The effect of state reforms on achievement and attainment (No. w30995). National Bureau of Economic Research.
CampbellD. T. (1969). Reforms as experiments. American Psychologist, 24(4), 409–429.
16.
CampbellC. M.ParkerC.ShandR.Kelly-MassoundA.FasholaT.BlancJ. (2021). Perspectives on DCPS IMPACT teacher evaluation system: Findings from teachers and school leaders. American University.
17.
CampbellS. L. (2023). Ratings in black and white: A quantcrit examination of race and gender in teacher evaluation reform. Race Ethnicity and Education, 26(7), 815–833.
18.
CampbellS. L.RonfeldtM. (2018). Observational evaluation of teachers: Measuring more than we bargained for?. American Educational Research Journal, 55(6), 1233–1267.
ChambersJ.Brodziak de los ReyesI.O’NeilC. (2013). How much are districts spending to implement teacher evaluation systems?RAND Corporation.
23.
DanielsonC. (2011). Enhancing professional practice: A framework for teaching (2nd ed.). Association for Supervision and Curriculum Development.
24.
Darling-HammondL. (2007). Race, inequality and educational accountability: The irony of “No Child Left Behind.”Race Ethnicity and Education, 10(3), 245–260.
25.
DeeT. S.JamesJ.WyckoffJ. (2021). Is effective teacher evaluation sustainable? Evidence from District of Columbia Public Schools. Education Finance and Policy, 16(2), 313–346.
26.
DeeT. S.WyckoffJ. (2015). Incentives, selection, and teacher performance: Evidence from impact. Journal of Policy Analysis and Management, 34(2), 267–297.
District of Columbia Public Schools. (2010). IMPACT guidebook 2010–2011. Author.
29.
District of Columbia Public Schools. (2021a). Equity review memo. Author.
30.
District of Columbia Public Schools. (2021b). Initial set of evolutions to IMPACT: SY 21-22. Author.
31.
DongN.MaynardR. (2013). PowerUp!: A tool for calculating minimum detectable effect sizes and minimum required sample sizes for experimental and quasi-experimental design studies. Journal of Research on Educational Effectiveness, 6(1), 24–67.
32.
EcclesJ. S.WigfieldA. (2020). From expectancy-value theory to situated expectancy-value theory: A developmental, social cognitive, and sociocultural perspective on motivation. Contemporary Educational Psychology, 61, 101859.
33.
Education Commission of the States. (2019, October21). State information request: Diversifying the teacher workforce. Author.
34.
EhrenbergR. G.SmithR. S. (2016). Modern labor economics: Theory and public policy (12th ed.). Routledge.
35.
FryerR. G.Jr. (2017). The production of human capital in developed countries: Evidence from 196 randomized field experiments. In BanerjeeA. V.DufloE. (Eds.), Handbook of economic field experiments (Vol. 2, pp. 95–322). North-Holland.
36.
GarbersY.KonradtU. (2014). The effect of financial incentives on performance: A quantitative review of individual and team-based financial incentives. Journal of Occupational and Organizational Psychology, 87(1), 102–137.
37.
GershensonS.HartC. M. D.HymanJ.LindsayC.PapageorgeN. W. (2022). The long-run impacts of same-race teachers. American Economic Journal: Economic Policy, 14(4), 300–342.
38.
GneezyU.RustichiniA. (2000). Pay enough or don’t pay at all. The Quarterly Journal of Economics, 115(3), 791–810.
39.
HamreB. K.PiantaR. C.DownerJ. T.DeCosterJ.MashburnA. J.JonesS. M.BrownJ. L.CappellaE.AtkinsM.RiversS. E. (2013). Teaching through interactions: Testing a developmental framework of teacher effectiveness in over 4,000 classrooms. The Elementary School Journal, 113(4), 461–487.
40.
HanushekE. A. (1979). Conceptual and empirical issues in the estimation of educational production functions. Journal of Human Resources, 14(3), 351–388.
41.
HanushekE. A. (2007). The single salary schedule and other issues of teacher pay. Peabody Journal of Education, 82(4), 574–586.
42.
HeiligJ. V.Darling-HammondL. (2008). Accountability Texas-style: The progress and learning of urban minority students in a high-stakes testing context. Educational Evaluation and Policy Analysis, 30(2), 75–110.
43.
HoffK.PandeyP. (2006). Discrimination, social identity, and durable inequalities. American Economic Review, 96(2), 206–211.
44.
HolmströmB. (1979). Moral hazard and observability. The Bell Journal of Economics, 10(1), 74–91.
45.
HolmströmB.MilgromP. (1991). Multitask principal-agent analyses: Incentive contracts, asset ownership, and job design. Journal of Law, Economics, and Organization, 7(Special Issue), 24–52.
46.
HowellW. G.MagazinnikA. (2017). Presidential prescriptions for state policy: Obama’s Race to the Top initiative. Journal of Policy Analysis and Management, 36(3), 502–531.
47.
ImbensG. W.LemieuxT. (2008). Regression discontinuity designs: A guide to practice. Journal of Econometrics, 142(2), 615–635.
48.
JacobB. A.LevittS. D. (2003). Rotten apples: An investigation of the prevalence and predictors of teacher cheating. The Quarterly Journal of Economics, 118(3), 843–877.
49.
KaneT. J.StaigerD. O. (2012). Gathering feedback for teaching: Combining high-quality observations with student surveys and achievement gains. Policy and practice brief (MET Project). Bill & Melinda Gates Foundation.
50.
KaneT. J.TaylorE. S.TylerJ. H.WootenA. L. (2011). Identifying effective classroom practices using student achievement data. Journal of Human Resources, 46(3), 587–613.
51.
KraftM. A.BlazarD.HoganD. (2018). The effect of teacher coaching on instruction and achievement: A meta-analysis of the causal evidence. Review of Educational Research, 88(4), 547–588.
52.
KraftM. A.GilmourA. F. (2017). Revisiting the widget effect: Teacher evaluation reforms and the distribution of teacher effectiveness. Educational Researcher, 46(5), 234–249.
53.
LazearE. P. (2000). The power of incentives. American Economic Review, 90(2), 410–414.
54.
LazearE. P. (2003). Teacher incentives. Swedish Economic Policy Review, 10(2), 179–214.
55.
LeeD. S.LemieuxT. (2010). Regression discontinuity designs in economics. Journal of Economic Literature, 48(2), 281–355.
56.
LindsayC. A.BlomE.TilsleyA. (2017). Diversifying the classroom: Examining the teacher pipeline. Urban Institute.
57.
LloydR.MertensD. (2018). Expecting more out of expectancy theory: History urges inclusion of the social context. International Management Review, 14(1), 28–43.
58.
LockeE. A.LathamG. P. (2002). Building a practically useful theory of goal setting and task motivation: A 35-year odyssey. American Psychologist, 57(9), 705–717.
59.
McCraryJ. (2008). Manipulation of the running variable in the regression discontinuity design: A density test. Journal of Econometrics, 142(2), 698–714.
60.
McGuinnP. (2012). Stimulating reform: Race to the top, competitive grants and the Obama education agenda. Educational Policy, 26(1), 136–159.
61.
Moore JohnsonS. (1984). Merit pay for teachers: A poor prescription for reform. Harvard Educational Review, 54(2), 175–186.
62.
MurnaneR. J.CohenD. (1986). Merit pay and the evaluation problem: Understanding why most merit pay plans fail and a few survive. Harvard Educational Review, 56(1), 1–17.
63.
OmiM.WinantH. (2014). Racial formation in the United States. Routledge.
64.
PapayJ. P.KraftM. A. (2015). Productivity returns to experience in the teacher labor market: Methodological challenges and new evidence on long-term career improvement. Journal of Public Economics, 130, 105–119.
65.
PhamL. D.NguyenT. D.SpringerM. G. (2021). Teacher merit pay: A meta-analysis. American Educational Research Journal, 58(3), 527–566.
66.
PhippsA. R.WisemanE. A. (2021). Enacting the rubric: Teacher improvements in windows of high-stakes observation. Education Finance and Policy, 16(2), 283–312.
67.
PiantaR. C.HamreB. K. (2009). Conceptualization, measurement, and improvement of classroom processes: Standardized observation can leverage capacity. Educational Researcher, 38(2), 109–119.
68.
PrendergastC. (1999). The provision of incentives in firms. Journal of Economic Literature, 37(1), 7–63.
69.
PutnamH.RossE.WalshK. (2018). Making a difference: Six places where teacher evaluation systems are getting results. National Council on Teacher Quality.
70.
ReardonS. F.RobinsonJ. P. (2012). Regression discontinuity designs with multiple rating-score variables. Journal of Research on Educational Effectiveness, 5(1), 83–104.
71.
ReddingC. (2019). A teacher like me: A review of the effect of student–teacher racial/ethnic matching on teacher perceptions of students and student academic and behavioral outcomes. Review of Educational Research, 89(4), 499–535.
72.
RockoffJ. E. (2004). The impact of individual teachers on student achievement: Evidence from panel data. American Economic Review, 94(2), 247–252.
73.
SteeleC. M.AronsonJ. (1995). Stereotype threat and the intellectual test performance of African Americans. Journal of Personality and Social Psychology, 69(5), 797.
74.
SteinbergM. P.SartainL. (2015). Does teacher evaluation improve school performance? Experimental evidence from Chicago's Excellence in Teaching project. Education Finance and Policy, 10(4), 535–572.
75.
SteinbergM. P.SartainL. (2021). What explains the race gap in teacher performance ratings? Evidence from Chicago Public Schools. Educational Evaluation and Policy Analysis, 43(1), 60–82.
76.
TaylorE. S.TylerJ. H. (2012). The effect of evaluation on teacher performance. American Economic Review, 102(7), 3628–3651.
77.
ToddP. E.WolpinK. I. (2003). On the specification and estimation of the production function for cognitive achievement. The Economic Journal, 113(485), F3–F33.
78.
TumaA. P.HamiltonL. S.TsaiT. (2018). How do teachers perceive feedback and evaluation systems?: Findings from the American Teacher Panel. RAND.
79.
WeibelA.RostK.OsterlohM. (2010). Pay for performance in the public sector—Benefits and (hidden) costs. Journal of Public Administration Research and Theory, 20(2), 387–412.
80.
WeisbergD.SextonS.MulhernJ.KeelingD. (2009). The widget effect: Our national failure to acknowledge and act on differences in teacher effectiveness. TNTP.
81.
WhitmireR. (2011). The Bee Eater: Michelle Rhee takes on the nation’s worst school district. John Wiley & Sons.
82.
WigginsG.McTigheJ. (2005). Understanding by design. Association for Supervision and Curriculum Development.
83.
WongV. C.SteinerP. M.CookT. D. (2013). Analyzing regression-discontinuity designs with multiple assignment variables: A comparative study of four estimation methods. Journal of Educational and Behavioral Statistics, 38(2), 107–141.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
