This chapter reviews actionable data use—both as an umbrella term and as a specific concept—developed in three different traditions that data/information can inform and guide P–20 educational practice toward better outcomes. The literatures reviewed are known as data-driven decision making (DDDM), education data mining (EDM), and learning analytics (LA). DDDM is grounded in K–12 settings, has a social orientation, and is shaped by policy. EDM and LA began in higher education using data provided by instructional tools. This review of more than 1,500 publications traced patterns in these communities revealing disciplinary disconnects between DDDM and EDM/LA. Recognizing information’s systemic nature, this review expanded the analysis from teacher practice to educator practice. While methodological progress has been made in all areas, studies of impact were concentrated in DDDM. EDM and LA focus on tools for current/future educational settings and leveraging data harvested for basic research while reconceiving learning practices. The DDDM impact studies did not support a directly beneficial model for data use. Rather, long timescale capacity factors, including cultural and organizational processes that impact data use were revealed. A complementary model of components, infrastructure, and capacity is advanced with recommendations for scholarship in education’s sociotechnical future.
Get full access to this article
View all access options for this article.
References
1.
AndersonS.LeithwoodK.StraussT. (2010). Leading data use in schools: Organizational conditions and practices at the school and district levels. Leadership and Policy in Schools, 9, 292–327.
2.
BehrensJ. T.DiCerboK. E. (2014). Technological implications for assessment ecosystems: Opportunities for digital technology to advance assessment. Teachers College Record, 116(11). Retrieved from https://eric.ed.gov/?id=EJ1040215
3.
BehrensJ. T.PietyP.DiCerboK. E.MislevyR. J. (2018). Inferential foundations for learning analytics in the digital ocean. In NiemiD.PeaR. D.SaxbergB.ClarkR. E. (Eds.), Learning analytics in education (pp. 1–48). Charlotte, NC: Information Age.
4.
BerlandM.MartinT.BentonT.SmithC. P.DavisD. (2013). Using learning analytics to understand the learning pathways of novice programmers. Journal of the Learning Sciences, 22, 564–599.
5.
BertrandM.MarshJ. A. (2015). Teachers’ sensemaking of data and implications for equity. American Educational Research Journal, 52, 861–893.
6.
BliksteinP.WorsleyM. (2016). Multimodal learning analytics and education data mining: Using computational technologies to measure complex learning tasks. Journal of Learning Analytics, 3, 220–238.
7.
Bouwma-GearhartJ.CollinsJ. (2015). What we know about data-driven decision making in higher education: Informing educational policy and practice. In Proceedings of International Academic Conferences (No. 2805154). London, England: International Institute of Social and Economic Sciences.
8.
BowersA. J. (2009). Reconsidering grades as data for decision making: More than just academic knowledge. Journal of Educational Administration, 47, 609–629.
9.
BurchP.SpillaneJ. P. (2003). Elementary school leadership strategies and subject matter: Reforming mathematics and literacy instruction. Elementary School Journal, 103, 519–535.
10.
CetintasS.SiL.XinY. P.ZhangD.ParkJ. Y.TzurR. A. (2014). A joint probabilistic classification model of relevant and irrelevant sentences in mathematical word problems. Retrieved from https://arxiv.org/abs/1411.5732
11.
CoburnC. E.TalbertJ. E. (2006). Conceptions of evidence use in school districts: Mapping the terrain. American Journal of Education, 112, 469–495.
12.
CoburnC.TurnerE. O. (2011). Research on data use: A framework and analysis. Measurement: Interdisciplinary Research and Practice, 9, 173–206. doi:10.1080/15366367.2011.626729
13.
CohenD. K.SpillaneJ. P.PeurachD. J. (2018). The dilemmas of educational reform. Educational Researcher, 47, 204–212.
14.
CrossleyS.AllenL.SnowE.McNamaraD. (2016). Incorporating learning characteristics into automatic essay scoring models: What individual differences and linguistic features tell us about writing quality. Journal of Educational Data Mining, 8(2), 1–19.
15.
Darling-HammondL. (2015). Can value added add value to teacher evaluation?Educational Researcher, 44, 132–137.
16.
DatnowA.HubbardL. (2016). Teacher capacity for and beliefs about data-driven decision making: A literature review of international research. Journal of Educational Change, 17(1), 7–28.
DentonC. A.SwansonE. A.MathesP. G. (2007). Assessment-based instructional coaching provided to reading intervention teachers. Reading and Writing, 20(6), 569–590.
19.
DiamondJ. B. (2007). Where the rubber meets the road: Rethinking the connection between high-stakes testing policy and classroom instruction. Sociology of Education, 80, 285–313.
20.
DiamondJ. B.SpillaneJ. P. (2002). High stakes accountability in urban elementary Schools: Challenging or reproducing inequality? (Working paper). Institute for Policy Research.
21.
DiamondJ. B.SpillaneJ. P. (2016). School leadership and management from a distributed perspective: A 2016 retrospective and prospective. Management Education, 30, 147–154.
22.
Farley-RippleE. N.ButtramJ. L. (2014). Developing collaborative data use through professional learning communities: Early lessons from Delaware. Studies in Educational Evaluation, 42, 41–53.
23.
GallimoreR.ErmelingB. A.SaundersW. M.GoldenbergC. (2009). Moving the learning of teaching closer to practice: Teacher education implications of school-based inquiry teams. Elementary School Journal, 109, 537–553.
24.
GummerE.MandinachE. (2015). Building a conceptual framework for data literacy. Teachers College Record, 117(4), 1–22.
25.
HalversonR.GriggJ.PrichettR.ThomasC. (2007). The new instructional leadership: Creating data-driven instructional systems in school. Journal of School Leadership, 17, 159–194.
HenigJ. R. (2013). The infrastructure of accountability: Data use and the transformation of American education. Cambridge, MA: Harvard Education Press.
28.
HopkinsD.StringfieldS.HarrisA.StollL.MackayT. (2014). School and system improvement: A narrative state-of-the-art review. School Effectiveness and School Improvement, 25, 257–281.
29.
IedemaR. (2003). Multimodality, resemiotization: Extending the analysis of discourse as multi-semiotic practice. Visual Communication, 2(1), 29–57.
30.
KnappM. S.SwinnertonJ. A.CoplandM. A.Monpas-HuberJ. (2006). Data-informed leadership in education. Seattle, WA: Center for the Study of Teaching and Policy.
31.
KonstantopoulosS.MillerS. R.van der PloegA. (2013). The impact of Indiana’s system of interim assessments on reading and math achievement. Educational Evaluation and Policy Analysis, 35(4), 481–484.
32.
KrummA.MeansB.BienkowskiM. (2018). Learning analytics goes to school: A collaborative approach to improving education. New York, NY: Routledge.
33.
LemkeJ. L. (2000). Across the scales of time: Artifacts, activities, and meanings in ecosocial systems. Mind, Culture, and Activity, 7, 273–290.
34.
MandinachE. B.HoneyM.LightD.BrunnerC. (2008). A conceptual framework for data-driven decision making. In MandinachE. B.HoneyM. (Eds.), Data-driven school improvement: Linking data and learning (pp. 13–31). New York, NY: Teachers College Press.
35.
MarshJ. A. (2012). Interventions promoting educators’ use of data: Research insights and gaps. Teachers College Record, 114(11), 1–48.
36.
MarshJ. A.FarrellC. C. (2015). How leaders can support teachers with data-driven decision making: A framework for understanding capacity building. Educational Management Administration & Leadership, 43, 269–289.
37.
MarshJ. A.PaneJ. F.HamiltonL. S. (2006). Making sense of data-driven decision making in education. Santa Monica, CA: RAND.
38.
MeansB.PadillaC.GallagherL. (2010). Use of education data at the local level: From accountability to instructional improvement. Washington, DC: U.S. Department of Education.
39.
MossP. A.PietyP. J. (2007). Introduction: Evidence and decision making. In MossP. A. (Ed.), Yearbook of the National Society for the Study of Education: Vol. 106. Evidence and decision making (pp. 1–14). Hoboken, NJ: Blackwell.
40.
MuldnerK.BurlesonW.Van de SandeB.VanLehnK. (2011). An analysis of students’ gaming behaviors in an intelligent tutoring system: Predictors and impacts. User Modeling and User-Adapted Interaction, 21, 99–135.
41.
No Child Left Behind Act of 2001. (2002). Public Law No. 107–110, 115 Statute 1425.
42.
PapamitsiouZ.EconomidesA. A. (2014). Learning analytics and educational data mining in practice: A systematic literature review of empirical evidence. Journal of Educational Technology & Society, 17(4), 49–64.
43.
ParkV.DatnowA. (2008). Collaborative assistance in a highly prescribed school reform model: The case of success for all. Peabody Journal of Education, 83, 400–422.
44.
Peña-AyalaA. (2014). Educational data mining: A survey and a data mining-based analysis of recent works. Expert Systems With Applications, 41, 1432–1462.
45.
PietyP. J. (2013). Assessing the educational data movement. New York, NY: Teachers College Press.
46.
PietyP. J.BehrensJ.PeaR. (2013, April). Educational data sciences and the need for interpretive skills. Paper presented at American Educational Research Association Annual Meeting, San Francisco, CA.
47.
PietyP. J.HickeyD. T.BishopM. J. (2014). Educational data sciences: Framing emergent practices for analytics of learning, organizations, and systems. In Proceedings of the fourth international conference on learning analytics and knowledge (pp. 193–202). New York, NY: ACM.
48.
PietyP. J.PeaR. D. (2018). Understanding learning analytics across practices. In NiemiD.PeaR. D.SaxbergB.ClarkR. E. (Eds.), Learning analytics in education (pp. 215–232). Charlotte, NC: Information Age.
49.
RibesD.FinholtT. A. (2009). The long now of infrastructure: Articulating tensions in development. Journal for the Association of Information Systems (JAIS): Special Issue on eInfrastructures, 10, 375–398.
50.
RomeroC.ZafraA.LunaJ. M.VenturaS. (2013). Association rule mining using genetic programming to provide feedback to instructors from multiple-choice quiz data. Expert Systems, 30, 162–172.
51.
RowanB.MillerR. J. (2007). Organizational strategies for promoting instructional change: Implementation dynamics in schools working with comprehensive school reform providers. American Educational Research Journal, 44, 252–297.
SiemensG. (2012). Learning analytics: Envisioning a research discipline and a domain of practice. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 4–8). New York, NY: ACM.
54.
SiemensG.BakerR. S. (2012). Learning analytics and educational data mining: Towards communication and collaboration. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 252–254). New York, NY: ACM.
SlavinR. E.CheungA.HolmesG.MaddenN. A.ChamberlainA. (2013). Effects of a data-driven district reform model on state assessment outcomes. American Educational Research Journal, 50, 371–396.
57.
StollL.BolamR.McMahonA.WallaceM.ThomasS. (2006). Professional learning communities: A review of the literature. Journal of Educational Change, 7, 221–258.
58.
StreiferP. A.SchumannJ. A. (2005). Using data mining to identify actionable information: Breaking new ground in data-driven decision making. Journal of Education for Students Placed at Risk, 10, 281–293.
59.
SunJ.PrzybylskiR.JohnsonB. J. (2016). A review of research on teachers’ use of student data: From the perspective of school leadership. Educational Assessment, Evaluation and Accountability, 28(1), 5–33.
60.
TempelaarD. T.RientiesB.GiesbersB. (2015). In search for the most informative data for feedback generation: Learning analytics in a data-rich context. Computers in Human Behavior, 47, 157–167.
61.
ValliL.CroningerR. G.ChamblissM. J.GraeberA. O.BueseD. (2008). Test driven: High-stakes accountability in elementary schools. New York, NY: Teachers College Press.
62.
Van LeeuwenA.JanssenJ.ErkensG.BrekelmansM. (2014). Supporting teachers in guiding collaborating students: Effects of learning analytics in CSCL. Computers & Education, 79, 28–39.
63.
VialardiC.ChueJ.PecheJ. P.AlvaradoG.VinateaB.EstrellaJ.OrtigosaÁ. (2011). A data mining approach to guide students through the enrollment process based on academic performance. User Modeling and User-Adapted Interaction, 21, 217–248.
64.
WaymanJ. C.ShawS.ChoV. (2017). Longitudinal effects of teacher use of a computer data system on student achievement. AERA Open, 3(1), 1–18. doi:10.1177/2332858416685534
65.
WaymanJ. C.StringfieldS. (2006). Data use for school improvement: School practices and research perspectives. American Journal of Education, 112, 463–468.
