This meta-analysis explores training teachers in the use of data, defined as any quantifiable information that helps teachers know more about their students for instructional decision-making. The questions addressed are as follows: (a) What are the features of data literacy training for kindergarten through 12th-grade teachers? (b) What are the effects of data literacy training on kindergarten through 12th-grade teacher outcomes? (c) Do training characteristics moderate the effects of training? A comprehensive search of research conducted between 1975 and 2019 yielded 33 studies with 163 effect sizes that met inclusion criteria. Using a random effects model, the researchers found significant positive effects on knowledge and skills, g = .67, 95% confidence interval (CI) = [0.40, 0.93], and beliefs, g = .48, 95% CI = [0.17, 0.79]. A collaborative training format significantly and positively moderated effects. Implications for teacher trainings and the design of future research are discussed.
*AlbrittonK.TruscottS. (2014). Professional development to increase problem-solving skills in a response to intervention framework. Contemporary School Psychology, 18, 44–58.
2.
BanduraA. (1997). Self-efficacy: The exercise of control. W.H. Freeman.
3.
*BeesleyA. D.ClarkT. F.DempseyK.TweedA. (2018). Enhancing formative assessment practice and encouraging middle school mathematics engagement and persistence. School Science and Mathematics, 118, 4–16.
4.
BoardmanA. G.ArgüellesM. E.VaughnS.HughesM. T.KlingnerJ. (2005). Special education teachers’ views of research-based practices. The Journal of Special Education, 39, 168–180.
5.
BorensteinM.HigginsJ. P.HedgesL. V.RothsteinH. R. (2017). Basics of meta-analysis: I2 is not an absolute measure of heterogeneity. Research Synthesis Methods, 8(1), 5–18.
6.
BoultonA. (2010). Data-driven learning: Taking the computer out of the equation. Language Learning, 60(3), 534–572.
7.
BreiterA.LightD. (2006). Data for school improvement: Factors for designing effective information systems to support decision-making in schools. Journal of Educational Technology & Society, 9(3), 206–217.
8.
BrockM. E.CarterE. W. (2017). A meta-analysis of educator training to improve implementation of interventions for students with disabilities. Remedial and Special Education, 38(3), 131–144.
9.
*CastilloJ. M.MarchA. L.TanS. Y.StockslagerK. M.BrundageA. (2016). Relationships between ongoing professional development and educators’ beliefs relative to response to intervention. Journal of Applied School Psychology, 32, 287–312.
10.
*ChatterjiM.KohN.ChoiL.IyengarR. (2009). Closing learning gaps proximally with teacher-mediated diagnostic classroom assessment. Research in the Schools, 16, 59–75.
CohenJ. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum.
13.
ColeM. W. (2010). Influence of assessment for learning professional development in rural Georgia public schools [Unpublished dissertation]. Liberty University.
14.
CooperH.HedgesL. V.ValentineJ. C. (Eds.). (2019). The handbook of research synthesis and meta-analysis. Russell Sage Foundation.
15.
Council of Chief State School Officers. (2013). Interstate Teacher Assessment and Support Consortium (InTASC) model core teaching standards and learning progressions for teachers. 1.0.
16.
CuijpersP.WeitzE.CristeaI.TwiskJ. (2017). Pre-post effect sizes should be avoided in meta-analyses. Epidemiology and Psychiatric Sciences, 26(4), 364–368.
17.
Darling-HammondL.WeiR. C.AndreeA.RichardsonN.OrphanosS. (2009). Professional learning in the learning profession. National Staff Development Council.
DatnowA.HubbardL. (2016). Teacher capacity for and beliefs about data-driven decision making: A literature review of international research. Journal of Educational Change, 17, 7–28.
21.
DesimoneL. M. (2009). Improving impact studies of teachers’ professional development: Toward better conceptualizations and measures. Educational Researcher, 38, 181–199
22.
DidionL.TosteJ. R.FildermanM. J. (2020). Teacher professional development and student reading achievement: A meta-analytic review of the effects. Journal of Research on Educational Effectiveness, 13(1), 29–66.
23.
DillonP.ErkensC.SannaD.SavastanoL. F. (2015). Crowdlearning. JSD the Learning Forward Journal, 36(3), 28–31.
24.
EggerM.Davey SmithG.SchneiderM.MinderC. (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315, 629–634.
25.
Endrew F. v. Douglas County School District, 798 F. 3d 1329 (10th Cir. 2015).
26.
EspinC. A.WaymanM. M.DenoS. L.McMasterK. L.de RooijM. (2017). Data-based decision-making: Developing a method for capturing teachers’ understanding of CBM graphs. Learning Disabilities Research & Practice, 32(1), 8–21
27.
Every Student Succeeds Act (2015). Pub. L. No. 114-95. In 114th Congress.
28.
FallonL. M.Collier-MeekM. A.MagginD. M.SanettiL. M.JohnsonA. H. (2015). Is performance feedback for educators an evidence-based practice? A systematic review and evaluation based on single-case research. Exceptional Children, 81(2), 227–246.
29.
*FanY. C.WangT. H.WangK. H. (2011). A web-based model for developing assessment literacy of secondary in-service teachers. Computers & Education, 57(2), 1727–1740.
30.
*FayadhA. (2017). Effectiveness of the proposed training formative assessment programme and its impact on teaching style improvements of Saudi science teachers in Saudi Arabia. Journal of Turkish Science Education, 14, 35–56.
31.
FildermanM. J.TosteJ. R.CoocN. (2020). Does training predict second-grade teachers’ use of student data for decision-making in reading and mathematics?Assessment for Effective Intervention, 46, 247–258.
32.
FildermanM. J.TosteJ. R.DidionL. A.PengP.ClemensN. H. (2018). Data-based decision making in reading interventions: A synthesis and meta-analysis of the effects for struggling readers. The Journal of Special Education, 52(3), 174–187.
33.
*FörsterN.SouvignierE. (2015). Effects of providing teachers with information about their students’ reading progress. School Psychology Review, 44, 60–75.
34.
FuchsD.FuchsL. S.ComptonD. L. (2012). Smart RTI: A next-generation approach to multilevel prevention. Exceptional Children, 78, 263–279.
35.
*FuchsL. S.AllinderR. M.HamlettC. L.FuchsD. (1990). An analysis of spelling curricula and teachers’ skills in identifying error types. Remedial and Special Education, 11, 42–52.
36.
*FuchsL. S.FuchsD. (1993). Effects of systematic observation and feedback on teachers’ implementation of curriculum-based measurement. Teacher Education and Special Education, 16, 178–187.
37.
*FuchsL. S.FuchsD.HamlettC. L.FergusonC. (1992). Effects of expert system consultation within curriculum-based measurement, using a reading maze task. Exceptional Children, 58, 436–450.
38.
*FuchsL. S.FuchsD.HamlettC. L.PhillipsN. B.BentzJ. (1994). Classwide curriculum-based measurement: Helping general educators meet the challenge of student diversity. Exceptional Children, 60, 518–537.
39.
*FuchsL. S.FuchsD.HamlettC. L.SteckerP. M. (1991). Effects of curriculum-based measurement and consultation on teacher planning and student achievement in mathematics operations. American Educational Research Journal, 28, 617–641.
40.
*FuchsL. S.FuchsD.KarnsK.HamlettC. L.KatzaroffM. (1999). Mathematics performance assessment in the classroom: Effects on teacher planning and student problem solving. American Educational Research Journal, 36, 609–646.
41.
*FuchsL. S.FuchsD.SteckerP. M. (1989). Effects of curriculum-based measurement on teachers’ instructional planning. Journal of Learning Disabilities, 22, 51–59.
42.
GallagherL.MeansB.PadillaC. (2008). Teachers’ use of student data systems to improve instruction: 2005 to 2007. U.S. Department of Education.
43.
GraneyS. B. (2008). General education teacher judgments of their low-performing students’ short-term reading progress. Psychology in the Schools, 45(6), 537–549.
44.
GriffinP.MurrayL.CareE.ThomasA.PerriP. (2010). Developmental assessment: Lifting literacy through professional learning teams. Assessment in Education: Principles, Policy & Practice, 17(4), 383–397.
45.
GummerE.MandinachE. (2015). Building a conceptual framework for data literacy. Teachers College Record, 117(4), 1–22.
46.
GuskeyT. R.YoonK. S. (2009). What works in professional development?Phi Delta Kappan, 90, 495–500.
HedgesL. V.HedbergE. (2007). Intraclass correlation values for planning group-randomized trials in education. Educational Evaluation and Policy Analysis, 29, 60–87.
49.
HedgesL. V.TiptonE.JohnsonM. C. (2010). Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods, 1, 39–65.
50.
HigginsJ. P.ThompsonS. G.DeeksJ. J.AltmanD. G. (2003). Measuring inconsistency in meta-analyses. British Medical Journal, 327(7414), 557–560.
51.
HooglandI.SchildkampK.Van der KleijF.HeitinkM.KippersW.VeldkampB.DijkstraA. M. (2016). Prerequisites for data-based decision making in the classroom: Research evidence and practical illustrations. Teaching and Teacher Education, 60, 377–386.
52.
*JimenezB. A.MimsP. J.BakerJ. (2016). The effects of an online data-based decisions professional development for in-service teachers of students with significant disability. Rural Special Education Quarterly, 35(3), 30–40.
53.
JimersonJ. B.WaymanJ. C. (2012, April). Branding educational data use through professional learning: Findings from a study in three school districts. Annual Meeting of the American Educational Research Association Vancouver, BC, Canada.
54.
JungP. G.McMasterK. L.KunkelA. K.ShinJ.SteckerP. M. (2018). Effects of data-based individualization for students with intensive learning needs: A meta-analysis. Learning Disabilities Research & Practice, 33, 144–155.
55.
*KennedyM. J.WagnerD.StegallJ.LembkeE.MiciakJ.AlvesK. D.BrownT.DriverM. K.HirschS. E. (2016). Using content acquisition podcasts to improve teacher candidate knowledge of curriculum-based measurement. Exceptional Children, 82, 303–320.
56.
KennedyM. M. (2016). How does professional development improve teaching?Review of Educational Research, 86(4), 945–980.
57.
*KippersW. B.PoortmanC. L.SchildkampK.VisscherA. J. (2018). Data literacy: What do educators learn and struggle with during a data use intervention?Studies in Educational Evaluation, 56, 21–31.
58.
KlingerD. A.McDivittP. R.HowardB. B.MunozM. A.RogersW. T.WylieE.C. (2015). The classroom assessment standards for preK-12 teachers: Joint committee on standards for educational evaluation. Kindle Direct Press.
59.
KoestersM. (2017). Every effect size has its place: A commentary on the avoidance of pre–post effect sizes. Epidemiology and Psychiatric Sciences, 26(4), 369–370.
60.
KraftM. A.BlazarD.HoganD. (2018). The effect of teacher coaching on instruction and achievement: A meta-analysis of the causal evidence. Review of Educational Research, 88(4), 547–588.
61.
*LembkeE. S.McMasterK. L.SmithR. A.AllenA.BrandesD.WagnerK. (2018). Professional development for data-based instruction in early writing: Tools, learning, and collaborative support. Teacher Education and Special Education, 41(2), 106–120.
62.
LifterK.KrugerL.OkunB.TabolC.PoklopL.ShishmanianE. (2005). Transformation to a web-based preservice training program: A case study. Topics in Early Childhood Special Education, 25(1), 15–24.
63.
LipseyM. W.WilsonD. B. (2001). Practical meta-analysis. SAGE.
64.
LittleJ. W. (2012). Understanding data use practice among teachers: The contribution of micro-process studies. American Journal of Education, 118(2), 143–166.
65.
MandinachE. B. (2012) A perfect time for data use: Using data-driven decision making to inform practice. Educational Psychologist, 47(2), 71–85.
66.
MandinachE. B.GummerE. S. (2016). What does it mean for teachers to be data literate: Laying out the skills, knowledge, and dispositions. Teaching and Teacher Education, 60, 366–376.
67.
MandinachE. B.HoneyM.LightD.BrunnerC. (2008). A conceptual framework for data-driven decision making. In MandinachE.B.HoneyM. (Eds.), Data-driven school improvement: Linking data and learning (pp. 13–31). Teachers College Press.
68.
MarsdenE.TorgersonC. J. (2012). Single group, pre-and post-test research designs: Some methodological concerns. Oxford Review of Education, 38(5), 583–616.
69.
MarshJ. A. (2012). Interventions promoting educators’ use of data: Research insights and gaps. Teachers College Record, 114(11), 1–48.
70.
*MartinC. S.PollyD.WangC.LambertR. G.PugaleeD. K. (2016). Perspectives and practices of elementary teachers using an internet-based formative assessment tool: The case of assessing mathematics concepts. International Journal for Technology in Mathematics Education, 23(1), 3–11.
71.
MeansB.ChenE.DeBargerA.PadillaC. (2011). Teachers’ ability to use data to inform instruction: Challenges and supports. U.S. Department of Education.
72.
MeansB.PadillaC.DeBargerA.BakiaM. (2009). Implementing data-informed decision making in schools: Teacher access, supports and use. U.S. Department of Education.
73.
MorrisS. B.DeShonR. P. (2002). Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods, 7, 105–125.
74.
National Assessment of Educational Progress. (2019). The nation’s report card: 2019 mathematics and reading assessments. National Center for Educational Statistics.
75.
National Center on Intensive Intervention. (2013). Data-based individualization: A framework for intensive intervention. Office of Special Education, U.S. Department of Education.
76.
National Council for the Accreditation of Teacher Education. (2010). Transforming teacher education through clinical practice: A national strategy to prepare effective teachers: A national strategy to prepare effective teachers.
77.
*Newman-ThomasC.SmithC. A.ZhaoX.KethleyC. I.RiethH. J.SwansonE. A.HeoY. (2012). Technology-based practice to teach preservice teachers to assess oral reading fluency. Journal of Special Education Technology, 27, 15–32.
78.
PiroJ. S.DunlapK.ShuttT. (2014). A collaborative Data Chat: Teaching summative assessment data use in pre-service teacher education. Cogent Education, 1(1), 1–24.
79.
*PollyD.WangC.MartinC.LambertR.PugaleeD.MiddletonC. (2018). The influence of mathematics professional development, school-level, and teacher-level variables on primary students’ mathematics achievement. Early Childhood Education Journal, 46, 31–45.
80.
*RandelB.ApthorpH.BeesleyA. D.ClarkT. F.WangX. (2016). Impacts of professional development in classroom assessment on teacher and student outcomes. The Journal of Educational Research, 109, 491–502.
81.
*ReevesT. D.ChiangJ. L. (2017). Building pre-service teacher capacity to use external assessment data: An intervention study. The Teacher Educator, 52, 155–172.
82.
*ReevesT. D.ChiangJ. L. (2019). Effects of an asynchronous online data literacy intervention on pre-service and in-service educators’ beliefs, self-efficacy, and practices. Computers & Education, 136, 13–33
83.
*ReevesT. D.HonigS. L. (2015). A classroom data literacy intervention for pre-service teachers. Teaching and Teacher Education, 50, 90–101.
84.
RemesalA. (2011). Primary and secondary teachers’ conceptions of assessment: A qualitative study. Teaching and Teacher Education, 27, 472–482.
85.
*RiccominiP. J.SteckerP. M. (2005). Effects of technology-enhanced practice on scoring accuracy of oral reading fluency. Journal of Special Education Technology, 20, 5–12.
86.
*RogersM. A. (2015). A developmental study examining the value, effectiveness, and quality of a data literacy intervention [Unpublished dissertation]. University of Iowa.
87.
RuleS.Fodor-DavisJ.MorganR.SalzbergC. L.ChenJ. (1990). An inservice training model to encourage collaborative consultation. Teacher Education and Special Education, 13(3–4), 225–227.
88.
SchildkampK.PoortmanC. (2015). Factors influencing the functioning of data teams. Teachers College Record, 117(4), 1–42.
89.
*SchneiderM. C.MeyerJ. P. (2012). Investigating the efficacy of a professional development program in formative classroom assessment in middle school English language arts and mathematics. Journal of Multi-disciplinary Evaluation, 8(17), 1–24.
90.
*SchützeB.RakoczyK.KliemeE.BesserM.LeissD. (2017). Training effects on teachers’ feedback practice: The mediating function of feedback knowledge and the moderating role of self-efficacy. ZDM Mathematics Education, 49, 475–489.
91.
SolomonB. G.KleinS. A.PolityloB. C. (2012). The effect of performance feedback on teachers’ treatment integrity: A meta-analysis of the single-case literature. School Psychology Review, 41(2), 160–175.
92.
SterneJ. A.SuttonA. J.IoannidisJ. P.TerrinN.JonesD. R.LauJ.CarpenterJ.RückerG.HarbordR. M.SchmidC. H.TetzlaffJ.DeeksJ. J.PetersJ.MacaskillP.SchwarzerG.DuvalS.AltmanD. G.MoherD.HigginsJ. P. (2011). Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. British Medical Journal, 343, d4002.
93.
SteckerP. M.FuchsL. S.FuchsD. (2005). Using curriculum-based measurement to improve student achievement: Review of research. Psychology in the Schools, 42(8), 795–819.
94.
Tschannen-MoranM.McMasterP. (2009). Sources of self-efficacy: Four professional development formats and their relationship to self-efficacy and implementation of a new teaching strategy. The Elementary School Journal, 110, 228–245.
95.
van den BoschR. M.EspinC. A.ChungS.SaabN. (2017). Data-based decision-making: Teachers’ comprehension of curriculum-based measurement progress-monitoring graphs. Learning Disabilities Research & Practice, 32, 46–60.
96.
*van der ScheerE. A.VisscherA. J. (2016). Effects of an intensive data-based decision making intervention on teacher efficacy. Teaching and Teacher Education, 60, 34–43.
97.
VaughnS.ElbaumB.WanzekJ.ScammaccaN.WalkerM. (2014). Code sheet and guide for education-related intervention study syntheses. The Meadows Center for Preventing Educational Risk.
98.
*VendlinskiT. P.PhelanJ. (2011). Using key conceptual ideas to improve teacher use of formative assessment data (CRESST Report 794). University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
99.
VygotskyL. S. (1978). Mind in Society. Harvard University Press.
100.
WagnerD. L.Hammerschmidt-SnidarichS. M.EspinC. A.SeifertK.McMasterK. L. (2017). Pre-service teachers’ interpretation of CBM progress monitoring data. Learning Disabilities Research & Practice, 32, 22–31.
101.
WaymanJ. C.MidgleyS.StringfieldS. (2017). Leadership for data-based decision making: Collaborative educator teams. In DanzigA.BormanK.JonesB.WrightW. (Eds.), Learner-centered leadership (pp. 189–206). Routledge.
102.
WaymanJ. C.JimersonJ. B. (2014). Teacher needs for data-related professional learning. Studies in Educational Evaluation, 42, 25–34.
103.
*WessonC. L. (1991). Curriculum-based measurement and two models of follow-up consultation. Exceptional Children, 57, 246–256.
104.
*WylieE. C.LyonC. J. (2015). The fidelity of formative assessment implementation: Issues of breadth and quality. Assessment in Education: Principles, Policy & Practice, 22, 140–160.
105.
XuY.BrownG. (2016). Teacher assessment literacy in practice: A reconceptualization. Teaching and Teacher Education, 58, 149–162.
106.
YoonK. S.DuncanT.LeeS. W. Y.ScarlossB.ShapleyK. L. (2007). Reviewing the evidence on how teacher professional development affects student achievement. Issues & answers. REL 2007-No. 033. Regional Educational Laboratory Southwest, U.S. Department of Special Education.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
