Empirical research investigating the association between teacher pay incentives and student test scores has grown rapidly over the past decade. To integrate the findings from these studies and help inform the debate over teacher merit pay, this meta-analysis synthesizes effect sizes across 37 primary studies, 26 of which were conducted in the United States. Among the U.S. based studies, the results suggest that the effect of teacher merit pay on student test scores is positive and statistically significant (0.043 standard deviation). This summary effect varies by program design and study context, suggesting that teacher merit pay has the potential to improve student test scores in some contexts but researchers and policymakers should pay close attention to program design and implementation.
*Primary studies used in the quantitative meta-analysis.
2.
+Primary studies that met the inclusion criteria but report LATE estimates.
3.
AbramiP. C.BernardR. M.BorokhovskiE.WaddingtonD. I.WadeC. A.PerssonT. (2015). Strategies for teaching students to think critically. Review of Educational Research, 85(2), 275–314. https://doi.org/10.3102/0034654314551063
4.
*AlafitaT. A. (2003). The effects of performance pay for teachers: An analysis of Arizona’s Career Ladder program (Unpublished doctoral dissertation). George Washington University, Washington, DC.
5.
AloeA. M.BeckerB. J. (2009). Teacher verbal ability and school outcomes: Where is the evidence?Educational Researcher, 38(8), 612–624. https://doi.org/10.3102/0013189X09353939
6.
AschB. J. (2005). The economic complexities of incentive reforms. In KlitgaardR.LightP. C. (Eds.), High-performance government: Structure, leadership, incentives (MG-256-PRGS; pp. 309–342). RAND Corporation.
7.
*AtkinsonA.BurgessS.CroxsonB.GreggP.PropperC.SlaterH.WilsonD. (2009). Evaluating the impact of performance-related pay for teachers in England. Labour Economics, 16(3), 251–261. https://doi.org/10.1016/j.labeco.2008.10.003
8.
BajorekZ. M.BevanS. M. (2015). Performance-related-pay in the UK public sector: A review of the recent evidence on effectiveness and value for money. Journal of Organizational Effectiveness: People and Performance, 2(2), 94–109. https://doi.org/10.1108/JOEPP-03-2015-0011
BarnesG.CroweE.SchaeferB. (2007). The cost of teacher turnover in five school districts: A pilot study (ED497176). National Commission on Teaching and America’s Future.
12.
*BarnettJ.WillsK.KirbyP. (2014). Comprehensive educator effectiveness models that work: Impact of the TAP system on student achievement in Louisiana. National Institute for Excellence in Teaching.
13.
BaronR. M.KennyD. A. (1986). The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51(6), 1173–1182. https://doi.org/10.1037/0022-3514.51.6.1173
14.
*Barrera-OsorioF.RajuD. (2017). Teacher performance pay: Experimental evidence from Pakistan. Journal of Public Economics, 148, 75–91. https://doi.org/10.1016/j.jpubeco.2017.02.001
15.
BartolucciA. A.HillegassW. B. (2010). Overview, strengths, and limitations of systematic reviews and meta-analyses. In ChiappelliF.Caldeira BrantX. M.NeagosN.OluwadaraO. O.RamchandaniM. H. (Eds.), Evidence-based practice: Toward optimizing clinical outcomes (pp. 17–33). Springer.
16.
BayonasH. G.BarkerF. (2010). Guilford County Schools Mission Possible program. SERVE Center, University of North Carolina–Greensboro.
17.
*BehrmanJ. R.ParkerS. W.ToddP. E.WolpinK. I. (2015). Aligning learning incentives of students and teachers: Results from a social experiment in Mexican high schools. Journal of Political Economy, 123(2), 325–364. https://doi.org/10.1086/675910
18.
BifulcoR. (2012). Can nonexperimental estimates replicate estimates based on random assignment in evaluations of school choice? A within-study comparison. Journal of Policy Analysis and Management, 31(3), 729–751. https://doi.org/10.1002/pam.20637
19.
BookerK.GlazermanS. (2009). Effects of the Missouri Career Ladder program on teacher mobility. Mathematica Policy Research.
20.
BorensteinM.HedgesL. V.HigginsJ.RothsteinH. R. (2009). Introduction to meta-analysis. Wiley.
21.
+BrehmM.ImbermanS. A.LovenheimM. F. (2015). Achievement effects of individual performance incentives in a teacher merit pay tournament. National Bureau of Economic Research.
22.
*BriggsD.Diaz-BilelloE.MaulA.TurnerM.BibilosC. (2014). Denver ProComp evaluation report: 2010–2012. Center for Assessment, Design, Research and Evaluation/National Center for the Improvement of Educational Assessment.
23.
BurgessS.RattoM. (2003). The role of incentives in the public sector: Issues and evidence. Oxford Review of Economic Policy, 19(2), 285–300. https://doi.org/10.1093/oxrep/19.2.285
24.
ChamberlinR.WraggT.HaynesG.WraggC. (2002). Performance-related pay and the teaching profession: A review of the literature. Research Papers in Education, 17(1), 31–49. https://doi.org/10.1080/02671520110102534
25.
ChoiW. S. (2015). The effect of alternative compensation programs on teacher retention and student achievement: The case of Q Comp in Minnesota (Unpublished doctoral dissertation). University of Minnesota, Minneapolis.
26.
ClabaughG. K. (2010). Teacher merit pay: Is it a good idea?Educational Horizons, 88(1), 16–20.
27.
ClotfelterC.GlennieE.LaddH.VigdorJ. (2008). Would higher salaries keep teachers in high-poverty schools? Evidence from a policy intervention in North Carolina. Journal of Public Economics, 92(5), 1352–1370. https://doi.org/10.1016/j.jpubeco.2007.07.003
28.
CohenB. H. (2013). Explaining psychological statistics (4th ed.). Wiley.
29.
*ContrerasD.RauT. (2012). Tournament incentives for teachers: Evidence from a scaled-up intervention in Chile. Economic Development and Cultural Change, 61(1), 219–246. https://doi.org/10.1086/666955
30.
*CooperS. T.CohnE. (1997). Estimation of a frontier production function for the South Carolina educational process. Economics of Education Review, 16(3), 313–327. https://doi.org/10.1016/S0272-7757(96)00077-5
31.
+CowanJ.GoldhaberD. (2015). Do bonuses affect teacher staffing and student achievement in high-poverty schools? Evidence from an incentive for National Board certified teachers in Washington State. Center for Education Data & Research.
32.
CummingsP. (2004). Meta-analysis based on standardized effects is unreliable. Archives of Pediatrics & Adolescent Medicine, 158(6), 595–597. https://doi.org/10.1001/archpedi.158.6.595
33.
*DeeT. S.KeysB. J. (2004). Does merit pay reward good teachers? Evidence from a randomized experiment. Journal of Policy Analysis and Management, 23(3), 471–488. https://doi.org/10.1002/pam.20022
34.
DeeT. S.WyckoffJ. (2015). Incentives, selection, and teacher performance: Evidence from IMPACT. Journal of Policy Analysis and Management, 34(2), 267–297. https://doi.org/10.1002/pam.21818
35.
DixitA. (2002). Incentives and organizations in the public sector: An interpretative review. Journal of Human Resources, 37(4), 696–727. https://doi.org/10.2307/3069614
36.
DowlingJ.MurphyS.WangB. (2007). The effects of the Career Ladder program on student achievement (Evaluation Rep.). Sheila Murphy.
37.
DuBoisD. L.PortilloN.RhodesJ. E.SilverthornN.ValentineJ. C. (2011). How effective are mentoring programs for youth? A systematic assessment of the evidence. Psychological Science in the Public Interest, 12(2), 57–91. https://doi.org/10.1177/1529100611414806
38.
DuvalS.TweedieR. (2000a). A nonparametric “trim and fill” method of accounting for publication bias in meta-analysis. Journal of the American Statistical Association, 95(449), 89–98. https://doi.org/10.1080/01621459.2000.10473905
39.
DuvalS.TweedieR. (2000b). Trim and fill: A simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455–463. https://doi.org/10.1111/j.0006-341X.2000.00455.x
40.
EggerM.SmithG. D.SchneiderM.MinderC. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ, 315(7109), 629. https://doi.org/10.1136/bmj.315.7109.629
*FryerR. G. (2013). Teacher incentives and student achievement: Evidence from New York City public schools. National Bureau of Economic Research.
44.
*FryerR. G.LevittS. D.ListJ.SadoffS. (2012). Enhancing the efficacy of teacher incentives through loss aversion: A field experiment. National Bureau of Economic Research.
45.
FuR.VandermeerB. W.ShamliyanT. A.O’NeilM. E.YazdiF.FoxS. H.MortonS. C. (2013). Handling continuous outcomes in quantitative synthesis. https://www.ncbi.nlm.nih.gov/books/NBK154408/
46.
FulbeckE. S. (2014). Teacher mobility and financial incentives: A descriptive analysis of Denver’s ProComp. Educational Evaluation and Policy Analysis, 36(1), 67–82. https://doi.org/10.3102/0162373713503185
*GlazermanS.ProtikA.TehB.BruchJ.MaxJ.WarnerE. (2013). Transfer incentives for high-performing teachers: Final results from a multisite randomized experiment (NCEE 2014-4003). Mathematica Policy Research/Institute of Education Sciences. https://files.eric.ed.gov/fulltext/ED544269.pdf
49.
*GlazermanS.SeifullahA. (2012). An evaluation of the Chicago Teacher Advancement Program (Chicago TAP) after four years (Final Rep.). Mathematica Policy Research.
GoldhaberD. (2002). The mystery of good teaching. Education Next, 2(1), 50–55.
52.
*GoldhaberD.WalchJ. (2012). Strategic pay reform: A student outcomes-based evaluation of Denver’s ProComp teacher pay initiative. Economics of Education Review, 31(6), 1067–1083. https://doi.org/10.1016/j.econedurev.2012.06.007
53.
*GoodmanS.TurnerL. (2011). Does whole-school performance pay improve student learning? Evidence from the New York City schools. Education Next, 11(2), 67–71.
54.
GreenJ. L.CamilliG.ElmoreP. B. (2006). Handbook of complementary methods in education research. Lawrence Erlbaum.
55.
GreenlandS.SchlesselmanJ.CriquiM. (1986). The fallacy of employing standardized regression coefficients and correlations as measures of effect. American Journal of Epidemiology, 123(2), 203–308. https://doi.org/10.1093/oxfordjournals.aje.a114229
56.
GreenwaldR.HedgesL. V.LaineR. D. (1996). The effect of school resources on student achievement. Review of Educational Research, 66(3), 361–396. https://doi.org/10.3102/00346543066003361
57.
GuarinoC. M.ReckaseM. D.WooldridgeJ. M. (2014). Can value-added measures of teacher performance be trusted?Education Finance and Policy, 10(1), 117–156. https://doi.org/10.1162/EDFP_a_00153
HedbergE. C. (2014). robumeta: Stata module to perform robust variance estimation in meta-regression with dependent effect size estimates. Statistical Software Components.
61.
HedgesL. V. (1981). Distribution theory for Glass’s estimator of effect size and related estimators. Journal of Educational Statistics, 6(2), 107–128. https://doi.org/10.3102/10769986006002107
62.
HedgesL. V.TiptonE.JohnsonM. C. (2010). Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods, 1(1), 39–65. https://doi.org/10.1002/jrsm.5
HeinrichC. J.MarschkeG. (2010). Incentives and their dynamics in public sector performance management systems. Journal of Policy Analysis and Management, 29(1), 183–208. https://doi.org/10.1002/pam.20484
65.
HigginsJ. P. T.AltmanD. G. (2008). Assessing risk of bias in included studies. In HigginsJ. P. T.GreenS. (Eds.), Cochrane handbook for systematic reviews of interventions (Cochrane Book Series; pp. 187–241). Wiley.
HoughH. J. (2012). Salary incentives and teacher quality: The effect of a district-level salary increase on teacher recruitment. Center for Education Policy Analysis.
69.
*ImbermanS. A.LovenheimM. F. (2015). Incentive strength and teacher productivity: Evidence from a group-based teacher incentive pay system. Review of Economics and Statistics, 97(2), 364–386. https://doi.org/10.1162/REST_a_00486
70.
JacobB. A.LevittS. D. (2003). Rotten apples: An investigation of the prevalence and predictors of teacher cheating. National Bureau of Economic Research.
71.
JinnaiY. (2016). The effects of a teacher performance-pay program on student achievement: A regression discontinuity approach. Economics Bulletin, 36(2), 993–999.
72.
JonesM.HartneyM. T. (2017). Show who the money? Teacher sorting patterns and performance pay across U.S. school districts. Public Administration Review, 77(6), 919–931. https://doi.org/10.1111/puar.12789
73.
KeefS. P.RobertsL. A. (2004). The meta-analysis of partial effect sizes. British Journal of Mathematical and Statistical Psychology, 57(1), 97–129. https://doi.org/10.1348/000711004849303
74.
LaddH. F. (1999). The Dallas school accountability and incentive program: An evaluation of its impacts on student outcomes. Economics of Education Review, 18(1), 1–16. https://doi.org/10.1016/S0272-7757(97)00044-7
75.
*LavyV. (2002). Evaluating the effect of teachers’ group performance incentives on pupil achievement. Journal of Political Economy, 110(6), 1286–1317. https://doi.org/10.1086/342810
76.
*LavyV. (2009). Performance pay and teachers’ effort, productivity, and grading ethics. American Economic Review, 99(5), 1979–2021. https://doi.org/10.1257/aer.99.5.1979
77.
LazearE. P.ShawK. L. (2007). Personnel economics: The economist’s view of human resources. Journal of Economic Perspectives, 21(4), 91–114. https://doi.org/10.1257/jep.21.4.91
78.
LeeB. K.PatallE. A.CawthonS. W.SteingutR. R. (2015). The effect of drama-based pedagogy on PreK–16 outcomes: A meta-analysis of research from 1985 to 2012. Review of Educational Research, 85(1), 3–49. https://doi.org/10.3102/0034654314540477
LipseyM. W.WilsonD. B. (2001). Practical meta-analysis (Vol. 49). Sage.
82.
*MarshJ. A.SpringerM.McCaffreyD.YuanK.EpsteinJ. (2011). A big apple for educators: New York City’s experiment with schoolwide performance bonuses (Final Evaluation Rep.). RAND Corporation.
83.
*MartinsP. S. (2009). Individual teacher incentives, student achievement and grade inflation (IZA DP No. 4051). Institute for the Study of Labor.
84.
*MizalaA.RomagueraP. (2005). Teachers’ salary structure and incentives in Chile. In VegasE. (Ed.), Incentives to improve teaching (pp. 103–150). World Bank.
85.
MoherD.LiberatiA.TetzlaffJ.AltmanD. G., & The PRISMA Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med, 6(7), e1000097. https://doi.org/10.1371/journal.pmed.1000097
86.
*MuralidharanK.SundararamanV. (2011). Teacher performance pay: Experimental evidence from India. Journal of Political Economy, 119(1), 39–77. https://doi.org/10.1086/659655
87.
MurnaneR. J.WillettJ. B. (2010). Methods matter: Improving causal inference in educational and social science research. Oxford University Press.
88.
NealD. (2011). The design of performance pay in education. National Bureau of Economic Research.
89.
PodgurskyM.SpringerM. (2007). Teacher performance pay: A review. Journal of Policy Analysis and Management, 26(4), 909–950. https://doi.org/10.1002/pam.20292
90.
PodgurskyM.SpringerM. (2011). Teacher compensation systems in the United States K–12 public school system. National Tax Journal, 64(1), 165–192. https://doi.org/10.17310/ntj.2011.1.07
*ProctorD.WalterB.ReichardtR.GoldhaberD.WalchJ. (2011). Making a difference in education reform: ProComp external evaluation report 2006–2010. Center for Education Data & Research, University of Colorado–Denver.
93.
RazoA. (2014). Does rewarding performance pay for teachers result in higher student achievement? (Unpublished doctoral dissertation). Arizona State University, Tempe.
94.
*RitterG.HolleyM.JensenN.RiffelB.WintersM.BarnettJ.GreeneJ. (2008). Year Two evaluation of the Achievement Challenge Pilot Project in the Little Rock Public School District. Department of Education Reform, College of Education and Health Professions.
95.
RothsteinR. (2009). The influence of scholarship and experience in other fields on teacher compensation reform. In SpringerM. G. (Ed.), Performance incentives: Their growing impact on American K–12 education (pp. 87–110). Brookings Institution Press.
96.
+SantibañezL.MartinezJ. F.DatarA.McEwanP. J.SetodjiC. M.Basurto-DavilaR. (2007). Breaking ground: Analysis of the assessment system and impact of Mexico’s teacher incentive program. RAND Corporation.
97.
*SchacterJ.ThumY. M. (2005). TAPping into high quality teachers: Preliminary results from the Teacher Advancement Program comprehensive school reform. School Effectiveness and School Improvement, 16(3), 327–353. https://doi.org/10.1080/13803610500146202
98.
*SchacterJ.ThumY. M.ReifsneiderD.SchiffT. (2004). The Teacher Advancement Program Report Two: Year Three results from Arizona and Year One results from South Carolina TAP schools. Milken Family Foundation.
99.
*SlotnikW.SmithM.HelmsB.QiaoZ. (2013). It’s more than money: Teacher incentive fund—leadership for educators’ advanced performance, Charlotte-Mecklenburg Schools. Boston Community Training and Assistance Center.
100.
*SlotnikW. J.SmithM. D.GlassR. J.HelmsB. J.IngwersonD. W. (2004). Catalyst for change: Pay for performance in Denver final report. Boston Community Training and Assistance Center.
101.
*SojournerA. J.MykereziE.WestK. L. (2014). Teacher pay reform and productivity panel data evidence from adoptions of Q-Comp in Minnesota. Journal of Human Resources, 49(4), 945–981. https://doi.org/10.1353/jhr.2014.0029
102.
SpringerM. G. (Ed.). (2009). Performance incentives: Their growing impact on American K–12 education. Brookings Institution Press.
103.
SpringerM. G.BalchR. (2010). Design components of incentive pay programs in the education sector. In SclafaniS. (Ed.), Teacher incentives and stimuli (pp. 65–94). Organisation for Economic Co-Operation and Development.
104.
*SpringerM. G.BallouD.HamiltonL.LeV.-N.LockwoodJ. R.McCaffreyD. F.PepperM.StecherB. M. (2010). Teacher pay for performance: Experimental evidence from the Project on Incentives in Teaching (POINT). National Center on Performance Incentives.
105.
*SpringerM. G.BallouD.PengA. X. (2014). Estimated effect of the teacher advancement program on student test score gains. Education, Finance, and Policy, 9(2), 193–230. https://doi.org/10.1162/EDFP_a_00129
106.
*SpringerM. G.LewisJ. L.EhlertM. W.PodgurskyM. J.CraderG. D.TaylorL. L.GronbergT. J.JansenD. W.LopezO. S.StuitD. A. (2010). District Awards for Teacher Excellence (DATE) program: Final evaluation report (ED538372). National Center on Performance Incentives.
107.
+SpringerM. G.LewisJ. L.PodgurskyM. J.EhlertM. W.TaylorL. L.LopezO. S.PengA. (2009). Governor’s Educator Excellence Grant (GEEG) Program: Year Three evaluation report. National Center on Performance Incentives.
108.
*SpringerM. G.PaneJ. F.LeV.-N.McCaffreyD. F.BurnsS. F.HamiltonL. S.StecherB. (2012). Team pay for performance: Experimental evidence from the Round Rock Pilot Project on team incentives. Educational Evaluation and Policy Analysis, 34(4), 367–390. https://doi.org/10.3102/0162373712439094
SpringerM. G.SwainW. A.RodriguezL. A. (2015). Effective teacher retention bonuses: Evidence from Tennessee. Educational Evaluation and Policy Analysis, 38(2), 199–221. https://doi.org/10.3102/0162373715609687
111.
SpringerM. G.TaylorL. L. (2016). Designing incentives for public school teachers: Evidence from a Texas incentive pay program. Journal of Education Finance, 41(3), 344–381. https://doi.org/10.1353/jef.2016.0001
112.
*SpringerM. G.WintersM. (2009). The NYC teacher pay-for-performance program: Early evidence from a randomized trial (Civic Rep. No. 56). Manhattan Institute for Policy Research.
SteeleJ. L.MurnaneR. J.WillettJ. B. (2010). Do financial incentives help low-performing schools attract and keep academically talented teachers? Evidence from California. Journal of Policy Analysis and Management, 29(3), 451–478. https://doi.org/10.1002/pam.20505
116.
Tanner-SmithE. E.TiptonE. (2014). Robust variance estimation with dependent effect sizes: Practical considerations including a software tutorial in Stata and SPSS. Research Synthesis Methods, 5(1), 13–30. https://doi.org/10.1002/jrsm.1091
117.
ThoonenE. E. J.SleegersP. J. C.OortF. J.PeetsmaT. T. D.GeijselF. P. (2011). How to improve teaching practices: The role of teacher motivation, organizational factors, and leadership practices. Educational Administration Quarterly, 47(3), 496–536. https://doi.org/10.1177/0013161X11400185
118.
TiptonE. (2015). Small sample adjustments for robust variance estimation with meta-regression. Psychological Methods, 20(3), 375–393. https://doi.org/10.1037/met0000011
119.
UmanskyI. (2005). A literature review of teacher quality and incentives. In VegasE. (Ed.), Incentives to improve teaching: Lessons from Latin America (pp. 21–61). World Bank.
120.
VaessenJ.RivasA.DuvendackM.Palmer-JonesR.LeeuwF. L.van GilsG.LukachR.HolvoetN.BastiaensenJ.HombradosJ. G.WaddingtonH. (2014). The effect of microcredit on women’s control over household spending in developing countries: A systematic review. Campbell Systematic Reviews, 10(8). https://doi.org/10.4073/csr.2014.8
121.
ValentineJ. C.KonstantopoulosS.Goldrick-RabS. (2017). What happens to students placed into developmental education? A meta-analysis of regression discontinuity studies. Review of Educational Research, 87(4), 806–833. https://doi.org/10.3102/0034654317709237
122.
VeveaJ. L.HedgesL. V. (1995). A general linear model for estimating effect size in the presence of publication bias. Psychometrika, 60(3), 419–435. https://doi.org/10.1007/BF02294384
123.
ViscardiD. (2014). The teacher pay for performance phenomenon (Unpublished doctoral dissertation). Seton Hall University, South Orange, NJ.
124.
WaddingtonH.WhiteH.SnilstveitB.HombradosJ. G.VojtkovaM.DaviesP.BhavsarA.EyersJ.KoehlmoosT. P.PetticrewM.ValentineJ. C.TugwellP. (2012). How to do a good systematic review of effects in international development: A tool kit. Journal of Development Effectiveness, 4(3), 359–387. https://doi.org/10.1080/19439342.2012.711765
125.
*WellingtonA.ChiangH.HeallgrenK.SperoniC.HerrmannM.BurkanderP. (2016). Evaluation of the Teacher Incentive Fund: Implementation and impacts of pay-for-performance after three years (Final Rep.). Mathematica Policy Research.
*WintersM.GreeneJ. P.RitterG.MarshR. (2008). The effect of performance-pay in Little Rock, Arkansas on student achievement (Working Paper 2008-02). National Center on Performance Incentives.
128.
What Works Clearinghouse. (2017). Standards handbook: Version 4.0. Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance.
YuanK.LeV.-N.MccaffreyD. F.MarshJ. A.HamiltonL. S.StecherB. M.SpringerM. (2013). Incentive pay programs do not affect teacher motivation or reported practices. Educational Evaluation and Policy Analysis, 35(1), 3–22. https://doi.org/10.3102/0162373712462625
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
