We present results from a meta-analysis of 95 experimental and quasi-experimental pre-K–12 science, technology, engineering, and mathematics (STEM) professional development and curriculum programs, seeking to understand what content, activities, and formats relate to stronger student outcomes. Across rigorously conducted studies, we found an average weighted impact estimate of +0.21 standard deviations. Programs saw stronger outcomes when they helped teachers learn to use curriculum materials; focused on improving teachers’ content knowledge, pedagogical content knowledge, and/or understanding of how students learn; incorporated summer workshops; and included teacher meetings to troubleshoot and discuss classroom implementation. We discuss implications for policy and practice.
AgodiniR.HarrisB.RemillardJ.ThomasM. (2013). After two years, three elementary math curricula outperform a fourth. Washington, DC: National Center for Education Evaluation and Regional Assistance.
2.
AlozieN. M.MojeE. B.KrajcikJ. S. (2010). An analysis of the supports and constraints for scientific discussion in high school project-based science. Science Education, 94, 395–427.
3.
ArnoldD. H.FisherP. H.DoctoroffG. L.DobbsJ. (2002). Accelerating math development in Head Start classrooms. Journal of Educational Psychology, 94, 762–770.
4.
BallD. L.CohenD. K. (1996). Reform by the book: What is—or might be—The role of curriculum materials in teacher learning and instructional reform?Educational Researcher, 25(9), 6–14.
5.
BanilowerE.SmithP. S.WeissI. R.PasleyJ. D. (2006). The status of K-12 science teaching in the United States: Results from a national observation survey. In SunalD.WrightE. (Eds.), The impact of the state and national standards on K-12 science teaching (pp. 83–122). Greenwich, CT: Information Age.
6.
BeckerK.ParkK. (2011). Effects of integrative approaches among science, technology, engineering, and mathematics (STEM) subjects on students’ learning: A preliminary meta-analysis. Journal of STEM Education: Innovations and Research, 12(5/6), 23–37.
7.
BlankR. K.de las AlasN. (2010, March). Effects of teacher professional development on gains in student achievement: How meta-analysis provides scientific evidence useful to education leaders. Paper presented at the Society for Research on Educational Effectiveness Spring Conference, Washington, DC.
8.
BormanK. M.CotnerB. A.LeeR. S.BoydstonT. L.LanehartR. (2009, March). Improving elementary science instruction and student achievement: The impact of a professional development program. Paper presented at the Society for Research on Educational Effectiveness Spring Conference, Washington, DC.
9.
BoydD. J.GrossmanP. L.LankfordH.LoebS.WyckoffJ. (2009). Teacher preparation and student achievement. Educational Evaluation and Policy Analysis, 31, 416–440.
10.
BrownH.CardD.DickersinK.GreenhouseJ.KlingJ.LittellJ. (2008). Report of the What Works Clearinghouse expert panel. Washington, DC: National Board for Education Sciences.
CervettiG. N.BarberJ.DorphR.PearsonP. D.GoldschmidtP. G. (2012). The impact of an integrated approach to science and literacy in elementary school classrooms. Journal of Research in Science Teaching, 49, 631–658.
13.
ChambersE. A. (2003). Efficacy of educational technology in elementary and secondary classrooms: A meta-analysis of the research literature from 1992–2002 (Unpublished doctoral dissertation). Southern Illinois University Carbondale.
14.
CheungA.SlavinR. E.KimE.LakeC. (2017). Effective secondary science programs: A best-evidence synthesis. Journal of Research in Science Teaching, 54, 58–81.
15.
ClarkD. B.Tanner-SmithE. E.KillingsworthS. S. (2016). Digital games, design, and learning: A systematic review and meta-analysis. Review of Educational Research, 86, 79–122.
16.
ClementsD. H.SaramaJ. (2007). Effects of a preschool mathematics curriculum: Summative research on the Building Blocks project. Journal for Research in Mathematics Education, 38, 136–163.
17.
ClewellB. C.CampbellP. B.PerlmanL. (2004). Review of evaluation studies mathematics and science curricula and professional development models. Washington, DC: The Urban Institute.
18.
CohenD. K.HillH. C. (1998). Instructional policy and classroom performance: The mathematics reform in California. Philadelphia, PA: Consortium for Policy Research in Education.
19.
CohenD. K.HillH. C. (2001). Learning policy: When state education reform works. New Haven, CT: Yale University Press.
20.
ConfreyJ. (2006). Comparing and contrasting the National Research Council report on evaluating curricular effectiveness with the What Works Clearinghouse approach. Educational Evaluation and Policy Analysis, 28, 195–213.
21.
ConfreyJ.StohlV. (Eds.). (2004). On evaluating curricular effectiveness: Judging the quality of K-12 mathematics evaluations. Washington, DC: National Academies Press.
22.
CorcoranT. B. (1995). Helping teachers teach well: Transforming professional development. Philadelphia, PA: CPRE Policy Briefs. Retrieved from https://files.eric.ed.gov/fulltext/ED388619.pdf
23.
Darling-HammondL.McLaughlinM. W. (1995). Policies that support professional development in an era of reform. Phi Delta Kappan, 76, 597–604.
24.
DedeC.KetelhutD.WhitehouseP.BreitL.McCloskeyE. M. (2009). A research agenda for online teacher professional development. Journal of Teacher Education, 60, 8–19.
25.
DesimoneL. M. (2009). Improving impact studies of teachers’ professional development: Toward better conceptualizations and measures. Educational Researcher, 38, 181–199. doi:10.3102/0013189X08331140
26.
DesimoneL. M. (2011). A primer on effective professional development. Phi Delta Kappan, 92(6), 68–71.
27.
Devlin-SchererW.SpinelliA. M.GiammatteoD.JohnsonC.Mayo-MolinaS.McGinleyP.ZiskL. (1998, February). Action research in professional development schools: Effects on student learning. Paper presented at the Annual Meeting of the American Association of Colleges for Teacher Education, New Orleans, LA.
28.
DietrichsonJ.BøgM.FilgesT.Klint JørgensenA. M. (2017). Academic interventions for elementary and middle school students with low socioeconomic status: A systematic review and meta-analysis. Review of Educational Research, 87, 243–282.
29.
DuvalS.TweedieR. (2000). Trim and fill: A simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56, 455–463.
30.
EggerM.SmithG. D.SchneiderM.MinderC. (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315, 629–634.
31.
FisherZ.TiptonE. (2015). robumeta: An R-package for robust variance estimation in meta-analysis. Retrieved from https://arxiv.org/pdf/1503.02220.pdf
32.
FrechtlingJ.SharpL.CareyN.Vaden-KiernanN. (1995). Professional development programs: A perspective on the last four decades. Washington, DC: National Science Foundation, Division of Research, Evaluation and Communication.
33.
FurtakE. M.SeidelT.IversonH.BriggsD. C. (2012). Experimental and quasi-experimental studies of inquiry-based science teaching: A meta-analysis. Review of Educational Research, 82, 300–329.
34.
GardellaJ. H.FisherB. W.Teurbe-TolonA. R. (2017). A systematic review and meta-analysis of cyber-victimization and educational outcomes for adolescents. Review of Educational Research, 87, 283–308.
35.
GaretM. S.WayneA. J.StancavageF.TaylorJ.WaltersK.SongM.. . . DoolittleF. (2010). Middle school mathematics professional development impact study: Findings after the first year of implementation (NCEE 2010-4009). Washington, DC: National Center for Education Evaluation and Regional Assistance.
36.
GerstenR.TaylorM. J.KeysT. D.RolfhusE.Newman-GoncharR. (2014). Summary of research on the effectiveness of math professional development approaches (REL 2014–010). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southeast. Retrieved from http://ies.ed.gov/ncee/edlabs
37.
GoldenbergC.GallimoreR. (1991). Changing teaching takes more than a one-shot workshop. Educational Leadership, 49(3), 69–72.
38.
HandB.Norton-MeierL. A.GunelM.AkkusR. (2016). Aligning teaching to learning: A 3-year study examining the embedding of language and argumentation into elementary science classrooms. International Journal of Science and Mathematics Education, 14, 847–863.
39.
HedgesL. V.OlkinI. (1980). Vote-counting methods in research synthesis. Psychological Bulletin, 88, 359–369.
40.
HiebertJ.StiglerJ. W.JacobsJ. K.GivvinK. B.GarnierH.SmithM.. . . GallimoreR. (2005). Mathematics teaching in the United States today (and tomorrow): Results from the TIMSS 1999 video study. Educational Evaluation and Policy Analysis, 27, 111–132.
41.
HigginsJ. P. T.DeeksJ. J. (2008). Selecting studies and collecting data. In HigginsJ. P. T.GreenS. (Eds.), Cochrane handbook for systematic reviews of interventions (pp. 151–186). Chichester, UK: John Wiley.
42.
HillC. J.BloomH. S.BlackA. R.LipseyM. W. (2008). Empirical benchmarks for interpreting effect sizes in research. Child Development Perspectives, 2, 172–177.
43.
HillH. C. (2004). Professional development standards and practices in elementary school mathematics. The Elementary School Journal, 104, 215–231.
44.
HillH. C. (2009). Fixing teacher professional development. Phi Delta Kappan, 90, 470–476.
45.
HillH. C.LitkeE.LynchK. (2018). Learning lessons from instruction: Descriptive results from an observational study of urban elementary classrooms. Teachers College Record, 120(12), 1–46.
46.
KaneT.KerrK.PiantaR. (2014). Designing teacher evaluation systems: New guidance from the measures of effective teaching project. San Francisco, CA: John Wiley.
47.
KennedyM. M. (1999). Form and substance in in-service teacher education (Research Monograph No. 13). Arlington, VA: National Science Foundation.
48.
KennedyM. M. (2016). How does professional development improve teaching?Review of Educational Research, 86, 945–980.
49.
KimJ. S.QuinnD. M. (2012, March 2012). A meta-analysis of K-8 summer reading interventions: The role of socioeconomic status in explaining variation in treatment effects. Paper presented at the annual meeting of the Society for Research on Educational Effectiveness, Washington, DC.
50.
KulikJ. A. (2003). Effects of using instructional technology in elementary and secondary schools: What controlled evaluation studies say (P10446.001). Arlington, VA: SRI International.
51.
LipseyM. W.PuzioK.YunC.HebertM. A.Steinka-FryK.ColeM. W.. . . BusickM. D. (2012). Translating the statistical representation of the effects of education interventions into more readily interpretable forms. Washington, DC: National Center for Special Education Research.
52.
LittellJ. H.CorcoranJ.PillaiV. (2008). Systematic reviews and meta-analysis. New York, NY: Oxford University Press.
53.
LlosaL.LeeO.JiangF.HaasA.O’ConnorC.Van BoovenC. D.KiefferM. J. (2016). Impact of a large-scale science intervention focused on English language learners. American Educational Research Journal, 53, 395–424.
54.
MaloufD. B.TaymansJ. M. (2016). Anatomy of an evidence base. Educational Researcher, 45, 454–459.
55.
MarxR. W.BlumenfeldP. C.KrajcikJ. S.FishmanB.SolowayE.GeierR.TalR. T. (2004). Inquiry-based science in the middle grades: Assessment of learning in urban systemic reform. Journal of Research in Science Teaching, 41, 1063–1080.
56.
MilesK. H.OddenA.FermanichM.ArchibaldS. (2004). Inside the black box of school district spending on professional development: Lessons from five urban districts. Journal of Education Finance, 30, 1–26.
57.
MillerB.LordB.DorneyJ. A. (1994). Staff development for teachers: A study of configurations and costs in four districts. Boston, MA: Education Development Center.
58.
MoherD.LiberatiA.TetzlaffJ.AltmanD. G., & The PRISMA Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLOS Medicine, 6(7), e1000097. doi:10.1371/journal.pmed.1000097
59.
MurphyR.PenuelW.MeansB.KorbakC.WhaleyA.AllenJ. (2002). E-DESK: A review of recent evidence on discrete educational software. Menlo Park, CA: SRI International.
60.
National Governors Association Center for Best Practices, Council of Chief State School Officers. (2010). Common Core State Standards. Washington, DC: Author.
61.
National Research Council. (2004). On evaluating curricular effectiveness: Judging the quality of K-12 mathematics evaluations. Washington, DC: The National Academies Press.
62.
National Research Council. (2011). Successful K-12 STEM education: Identifying effective approaches in science, technology, engineering, and mathematics. Washington, DC: The National Academies Press.
63.
National Research Council. (2013). Next Generation Science Standards: For States, by States. Washington, DC: The National Academies Press. doi:10.17226/18290
64.
PaneJ. F.GriffinB. A.McCaffreyD. F.KaramR. (2014). Effectiveness of Cognitive Tutor Algebra I at scale. Educational Evaluation and Policy Analysis, 36, 127–144.
65.
PenuelW. R.SunM.FrankK. A.GallagherH. A. (2012). Using social network analysis to study how collegial interactions can augment teacher learning from external professional development. American Journal of Education, 119, 103–136.
66.
PolaninJ. R.Tanner-SmithE. E.HennessyE. A. (2016). Estimating the difference between published and unpublished effect sizes: A meta-review. Review of Educational Research, 86, 207–236.
67.
PresserA. L.VaheyP.DominguezX. (2015, March). Improving mathematics learning by integrating curricular activities with innovative and developmentally appropriate digital apps: Findings from the next generation preschool math evaluation. Paper presented at the Society for Research on Educational Effectiveness Spring Conference, Washington, DC.
68.
RaudenbushS. W. (2008). Advancing educational policy by advancing research on instruction. American Educational Research Journal, 45, 206–230.
69.
ResendezM.AzinM. (2006). 2005 Prentice Hall Science Explorer randomized control trial. Jackson, WY: PRES Associates.
70.
RoschelleJ.ShechtmanN.TatarD.HegedusS.HopkinsB.EmpsonS.. . . GallagherL. P. (2010). Integration of technology, curriculum, and professional development for advancing middle school mathematics: Three large-scale studies. American Educational Research Journal, 47, 833–878.
71.
RussellS. J.EconomopoulosK.MokrosJ.KlimanM.WrightT.ClementsD. H., et al. (2006). Investigations in Number, Data, and Space. Grade 1. Glenview, IL: Pearson Scott Foresman.
72.
SantagataR.KerstingN.GivvinK. B.StiglerJ. W. (2010). Problem implementation as a lever for change: An experimental study of the effects of a professional development program on students’ mathematics learning. Journal of Research on Educational Effectiveness, 4, 1–24.
73.
SaxeG. B.GearhartM.NasirN. S. (2001). Enhancing students’ understanding of mathematics: A study of three contrasting approaches to professional support. Journal of Mathematics Teacher Education, 4, 55–79.
74.
ScherL.O’ReillyF. (2009). Professional development for K–12 math and science teachers: What do we really know?Journal of Research on Educational Effectiveness, 2, 209–249.
75.
SchneiderM. C.MeyerJ. P. (2012). Investigating the efficacy of a professional development program in formative classroom assessment in middle-school English language arts and mathematics. Journal of Multidisciplinary Evaluation, 8(17), 1–24.
76.
Schwartz-BloomR. D.HalpinM. J. (2003). Integrating pharmacology topics in high school biology and chemistry classes improves performance. Journal of Research in Science Teaching, 40, 922–938.
77.
ShavelsonR. J.TowneL. (Eds.). (2001). Scientific research in education. Washington, DC: The National Academies Press.
78.
ShulmanL. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4–14.
79.
SlavinR. E. (2008). Perspectives on evidence-based research in education—What works? Issues in synthesizing educational program evaluations. Educational Researcher, 37, 5–14.
80.
SlavinR. E.LakeC. (2008). Effective programs in elementary mathematics: A best-evidence synthesis. Review of Educational Research, 78, 427–515.
81.
SlavinR. E.LakeC.GroffC. (2009). Effective programs in middle and high school mathematics: A best-evidence synthesis. Review of Educational Research, 79, 839–911.
82.
SlavinR. E.LakeC.HanleyP.ThurstonA. (2014). Experimental evaluations of elementary science programs: A best-evidence synthesis. Journal of Research in Science Teaching, 51(7), 870–901.
83.
SmithM. S.O’DayJ. (1990). Systemic school reform. Journal of Education Policy, 5, 233-267. doi:10.1080/02680939008549074
84.
SparksD. (2002). Designing powerful professional development for teachers and principals. Oxford, OH: National Staff Development Council.
85.
SteinM. K.RemillardJ.SmithM. S. (2007). How curriculum influences student learning. In Lester Jr.F. K. (Ed.), Second handbook of research on mathematics teaching and learning (pp. 319–369). New York, NY: Macmillan.
86.
SussmanJ.WilsonM. R. (2019). The use and validity of standardized achievement tests for evaluating new curricular interventions in mathematics and science. American Journal of Evaluation, 40(2), 190–213. doi:10.1177/1098214018767313
87.
Tanner-SmithE. E.TiptonE. (2014). Robust variance estimation with dependent effect sizes: Practical considerations and a software tutorial in Stata and SPSS. Research Synthesis Methods, 5, 13–30.
88.
Tanner-SmithE. E.TiptonE.PolaninJ. R. (2016). Handling complex meta-analytic data structures using robust variance estimates: A tutorial in R. Journal of Developmental and Life-Course Criminology, 2, 85–112.
89.
TaylorJ. A.GettyS. R.KowalskiS. M.WilsonC. D.CarlsonJ.Van ScotterP. (2015). An efficacy trial of research-based curriculum materials with curriculum-based professional development. American Educational Research Journal, 52, 984–1017.
90.
TaylorJ. A.KowalskiS. M.PolaninJ. R.AskinasK.StuhlsatzM. A.WilsonC. D.. . . WilsonS. J. (2018). Investigating science education effect sizes: Implications for power analyses and programmatic decisions. AERA Open, 4(3). doi:10.1177/2332858418791991
91.
TimperleyH.WilsonA.BarrarH.FungI. (2008). Teacher professional learning and development. Brussels, Belgium: International Academy of Education.
92.
TiptonE.PustejovskyJ. E. (2015). Small-sample adjustments for tests of moderators and model fit using robust variance estimation in meta-regression. Journal of Educational and Behavioral Statistics, 40, 604–634.
93.
Van EgerenL. A.SchwarzC.GerdeH.MorrisB.PierceS.Brophy-Herb. . . StoddardD. (2014, August). Cluster-randomized trial of the efficacy of early childhood science education with low-income children: Years 1-3. Poster presented at the 2014 Discovery Research K-12 PI Meeting, Arlington, VA.
94.
What Works Clearinghouse. (2008). Middle school math curricula. Washington, DC: U.S. Department of Education.
95.
What Works Clearinghouse. (2010). Procedures and standards handbook (Version 2.1). Washington, DC: U.S. Department of Education.
96.
WilsonS. J.Tanner-SmithE. E.LipseyM. W.Steinka-FryK.MorrisonJ. (2011). Dropout prevention and intervention programs: Effects on school completion and dropout among school aged children and youth. Campbell Systematic Reviews, 8. doi:10.4073/csr.2011.8
97.
WilsonS. M. (2013). Professional development for science teachers. Science, 340, 310–313. doi:10.1126/science.1230725
98.
YoonK. S.DuncanT.LeeW.-Y.ScarlossB.ShapleyK. (2007). Reviewing the evidence on how teacher professional development affects student achievement (Issues & Answers Report, REL 2007–No. 033). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southwest. Retrieved from http://ies.ed.gov/ncee/edlabs
99.
ZaslowM.ToutK.HalleT.WhittakerJ. V.LavelleB. (2010). Toward the identification of features of effective professional development for early childhood educators, literature review. Washington, DC: Office of Planning, Evaluation and Policy Development, U.S. Department of Education.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
