Despite growing international interest in the use of data to improve education, few studies examining the effects on student achievement are yet available. In the present study, the effects of a two-year data-based decision-making intervention on student achievement growth were investigated. Fifty-three primary schools participated in the project, and student achievement data were collected over the two years before and two years during the intervention. Linear mixed models were used to analyze the differential effect of data use on student achievement. A positive mean intervention effect was estimated, with an average effect of approximately one extra month of schooling. Furthermore, the results suggest that the intervention especially significantly improved the performances of students in low socioeconomic status schools.
AndersonS.LeithwoodK.StraussT. (2010). Leading data use in schools: Organizational conditions and practices at the school and district levels. Leadership and Policy in Schools, 9(3), 292–327. doi:10.1080/157007610 03731492
2.
BatesD.MächlerM.BolkerB.WalkerS. (2014). Fitting linear mixed-effects models using lme4. Retrieved from http://arxiv.org/abs/1406.5823
3.
BennettR. E. (2011). Formative assessment: A critical review. Assessment in Education: Principles, Policy & Practice, 18(1), 5–25. doi:10.1080/096959 4X.2010.513678
4.
BirmanB. F.DesimoneL. M.PorterA. C.GaretM. S. (2000). Designing professional development that works. Educational Leadership, 57(8), 28–33.
5.
BlackB. P.WiliamD. (1998). Inside the black box: Raising standards through classroom assessment. Assessment in Education, 5(1), 7–74.
6.
CampbellC.LevinB. (2008). Using data to support educational improvement. Educational Assessment, Evaluation and Accountability, 21(1), 47–65. doi:10.1007/s11092-008-9063-x
7.
CarlsonD.BormanG. D.RobinsonM. (2011). A multistate district-level cluster randomized trial of the impact of data-driven reform on reading and mathematics achievement. Educational Evaluation and Policy Analysis, 33(3), 378–398. doi:10.3102/0162373711412765
CoburnC. E.TurnerE. O. (2012). The practice of data use: An introduction. American Journal of Education, 118(2), 99–111. doi:10.1086/663272
10.
CoeR. (2002). Evidence on the role and impact of performance feedback in schools. In VisscherA.CoeR. (Eds.), School improvement through performance feedback (pp. 3–26). London: Routledge.
11.
DatnowA.CastellanoM. (2000). Teachers’ responses to success for all: How beliefs, experiences, and adaptations shape implementation. American Educational Research Journal, 37(3), 775–799. doi:10.3102/00028312037003775
12.
DatnowA.HubbardL.MehanH. (1998). Educational reform implementation: A co-constructed process. Berkeley, CA: Center for Research on Education, Diversity & Excellence.
13.
DesimoneL. M. (2002). How can comprehensive school reform models be successfully implemented?Review of Educational Research, 72(3), 433–479. doi:10.3102/00346543072003433
14.
DesimoneL. M. (2009). Improving impact studies of teachers’ professional development: Toward better conceptualizations and measures. Educational Researcher, 38(3), 181–199.
15.
DunnK. E.AirolaD. T.LoW.-J.GarrisonM. (2013). Becoming data driven: The influence of teachers’ sense of efficacy on concerns related to data-driven decision making. The Journal of Experimental Education, 81(2), 222–241. doi:10.1080/00220973.2012.699899
16.
EarlL.KatzS. (2006). Leading schools in a data-rich world: Harnessing data for school improvement. Thousans Oaks, CA: Corwin Press.
FerronJ. M.MoeyaertM.Van den NoortgateW.BeretvasS. N. (2014). Estimating causal effects from multiple-baseline studies: Implications for design and analysis. Psychological Methods, 19(4), 493–510.
19.
FuchsL. S.FuchsD. (1986). Effects of systematic formative evaluation: A meta-analysis. Exceptional Children, 53, 199–208.
20.
GageN. A.LewisT. J. (2014). Hierarchical linear modeling meta-analysis of single-subject design research. The Journal of Special Education, 48, 3–16. doi:10.1177/0022466912443894
21.
GaretM. S.PorterA. C.DesimoneL. M.BirmanB. F.YoonK. S. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38(4), 915–945.
22.
GershensonS.LangbeinL. (2015). the effect of primary school size on academic achievement. Educational Evaluation and Policy Analysis, 37, 135–155. doi:10.3102/0162373715576075
23.
GrayJ. (2002). Jolts and reactions: Two decades of feeding back information on schools’ performance. In VisscherA. J.CoeR. (Eds.), School improvement through performance feedback (pp. 143–162). London: Routledge.
24.
GrissomJ. A.LoebS.MasterB. (2013). Effective instructional time use for school leaders: Longitudinal evidence from observations of principals. Educational Researcher, 42, 433–444. doi:10.3102/0013189X13510020
HanushekE. A. (2011). How much is a good teacher worth?Education Next, 11(3), 40–45.
27.
HattieJ. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. London: Routledge.
28.
HattieJ.TimperleyH. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. doi:10.3102/003465430298487
29.
IkemotoG. S.MarshJ. A. (2007). Cutting through the “data-driven” mantra: Different conceptions of data-driven decision making. In MossP. A. (Ed.), Evidence and decision making: Yearbook of the national society of education (pp. 105–131). London: Wiley-Blackwell.
30.
Inspectie van het Onderwijs. (2012). Beoordeling van opbrengsten in het basisonderwijs. Utrecht, The Netherlands: Author.
31.
Inspectie van het Onderwijs. (2013). De staat van het onderwijs: Onderwijsverslag 2011/2012. Utrecht, The Netherlands: Author.
32.
JensonW. R.ClarkE.KircherJ. C.KristjanssonS. D. (2007). Statistical reform: Evidence-based practice, meta-analyses, and single subject designs. Psychology in the Schools, 44(5), 483–493. doi:10.1002/pits.20240
33.
KamphuisF.MoelandsF. (2000). A student monitoring system. Educational Measurement: Issues and Practice, 19(4), 28–30.
34.
KingstonN.NashB. (2011). Formative assessment: A meta-analysis and a call for research. Educational Measurement: Issues and Practice, 30(4), 28–37. doi:10.1111/j.1745-3992.2011.00220.x
35.
KlugerA. N.DeNisiA. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 199(2), 254–284.
36.
KratochwillT. R.HitchcockJ.HornerR. H.LevinJ. R.OdomS. L.RindskopfD. M.ShadishW. R. (2010). Single-case design technical documentation. Retrieved from http://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf
37.
LaiM. K.McNaughtonS. (2013). Analysis and discussion of classroom and achievement data to raise student achievement. In SchildkampK.LaiM. K.EarlL. (Eds.), Data-based decision making in education: Challenges and opportunities (pp. 23–48). New York, NY: Springer.
38.
LaiM. K.SchildkampK. (2013). Data-based decision making: An overview. In SchildkampK.LaiM. K.EarlL. (Eds.), Data-based decision making in education: Challenges and opportunities (pp. 9–21). NewYork, NY: Springer.
39.
LockeE. A.LathamG. P. (2002). Building a practically useful theory of goal setting and task motivation: A 35-year odyssey. American Psychologist, 57(9), 705–717. doi:10.1037//0003-066X.57.9.705
40.
LumpeA. T. (2007). Research-based professional development: Teachers engaged in professional learning communities. Journal of Science Teacher Education, 18(1), 125–128. doi:10.1007/s10972-006-9018-3
41.
MandinachE. B. (2012). A perfect time for data use: Using data-driven decision making to inform practice. Educational Psychologist, 47(2), 71–85. doi:10.1080/00461520.2012.667064
42.
MandinachE. B.GummerE. S.MullerR. D. (2011). The complexities of integrating data-driven decision making into professional preparation in schools of education: It’s harder than you think. Report from an invitational meeting. Alexandria, VA: CNA Analysis & Solutions.
43.
MarshJ. A. (2012). Interventions promoting educators use of data: Research insights and gaps. Teachers College Record, 114, 1–47.
44.
MayH.HuffJ.GoldringE. (2012). A longitudinal study of principals’ activities and student performance. School Effectiveness and School Improvement, 23, 417–439. doi:10.1080/09243453.2012.678866
45.
MorisanoD.LockeE. A. (2013). Goal setting and academic achievement. In HattieJ.AndermanE. M. (Eds.), International guide to student achievement (pp. 45–48). London: Routledge.
46.
MullisI. V. S.MartinM. O.FoyP.AroraA. (2012). TIMSS 2011 international results in mathematics. Chestnut Hill, MA: TIMSS & PIRL International Study Center, Boston College.
47.
NyeB.KonstantopoulosS.HedgesL. V. (2004). How large are teacher effects?Educational Evaluation and Policy Analysis, 26(3), 237–257.
48.
OláhL. N.LawrenceN. R.RigganM. (2010, April26). Learning to learn from benchmark assessment data: How teachers analyze results. Peabody Journal of Education, 85, 226–245. doi:10.1080/01619561003688688
49.
RCoreTeam. (2013). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from http://www.r-project.org/
50.
SchildkampK.EhrenM.LaiM. K. (2012). Editorial article for the special issue on data-based decision making around the world: From policy to practice to results. School Effectiveness and School Improvement, 23(2), 123–131. doi:10.1080/09243453.2011.652122
51.
SchildkampK.LaiM. K.EarlL. (Eds.). (2013). Data-based decision making in education: Challenges and opportunities. Dordrecht: Springer Netherlands. http://doi.org/10.1007/978-94-007-4816-3
52.
SnijdersT. A. B.BoskerR. J. (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. London: Sage Publishers.
53.
StamanL.VisscherA. J.LuytenH. (2013). The effects of training school staff for utilizing student monitoring system data. In PasseyD.BreiterA.VisscherA. (Eds.), 10th IFIP WG 3.7 Conference, ITEM 2012 (pp. 3–14). Bremen, Germany: Springer.
54.
StamanL.VisscherA. J.LuytenH. (2014). The effects of professional development on the attitudes, knowledge and skills for data-driven decision making. Studies in Educational Evaluation, 42, 79–90. doi:10.1016/j.stueduc.2013.11.002
55.
SupovitzJ. A. (2012). Getting at student understanding—The key to teachers’ use of test data. Teachers College Record, 114(11), 1–29.
TurnerE. O.CoburnC. E. (2012). Interventions to promote data use: An introduction. Teachers College Record, 114, 13.
58.
Van den NoortgateW.OnghenaP. (2003). Combining single-case experimental data using hierarchical linear models. School Psychology Quarterly, 18, 325–346.
59.
Van VeenK.ZwartR.MeirinkJ. (2011). What makes teacher professional development effective? A literature review. In KooyM.Van VeenK. (Eds.), Teacher learning that matters (pp. 3–21). New York, NY: Routledge.
60.
VanhoofJ.VerhaegheG.Van PetegemP.ValckeM. (2013). Improving data literacy in schools: Lessons from the school feedback project. In SchildkampK.LaiM. K.EarlL. (Eds.), Data-based decision making in education: Challenges and opportunities (pp. 113–134). New York, NY: Springer.
61.
VerhaegheG. (2011). School performance feedback systems: Design and implementation issues. Ghent, Belgium: Ghent University.
62.
WaymanJ. C.ChoV. (2008). Preparing educators to effectively use student data systems. In KowalskiT. J.LasleyT. J. (Eds.), Handbook on data-based decision-making in education (pp. 89–104). New York, NY: Routledge.
63.
WaymanJ. C.StringfieldS.YakimowskiM. (2004). Software enabling school improvement through analysis of student data. Baltimore, MD: Johns Hopkins University Center for Research on the Education of Students Placed At Risk.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
