Accountability policies assume that educators will use student data to improve student learning, but data use in practice has turned out to be harder than theorized. The purpose of this article was to examine how science teachers in Grades 5 to 8 used data in their classrooms. Utilizing sensemaking theory, we found that teachers decided how to use data based on district and school policies and expectations around assessment and data use, balancing those messages with their own understandings of science education. In practice, this led to the privileging of certain kinds of assessment and data use at the expense of others.
ArmstrongJ.AnthesK. (2001). How data can help. American School Board Journal, 188(11), 38-41.
2.
BeaverJ.WeinbaumE. (2015). State test data and school improvement efforts. Educational Policy, 29, 478-503.
3.
BerlandL. K.McNeillK. L. (2010). A learning progression for scientific argumentation: Understanding student work and designing supportive instructional contexts. Science Education, 94, 765-793.
4.
BerlandL. K.SchwarzC. V.KristC.KenyonL.LoA. S.ReiserB. J. (2015). Epistemologies in practice: Making scientific practices meaningful for students. Journal of Research in Science Teaching, 53, 1082-1112.
5.
BlackP.WiliamD. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21(1), 5-31.
6.
Booher-JenningsJ. (2005). Below the bubble: “Educational triage” and the Texas accountability system. American Educational Research Journal, 42, 231-268.
7.
CarrawayJ. H.YoungT. (2015). Implementation of a districtwide policy to improve principals’ instructional leadership principals’ sensemaking of the skillful observation and coaching laboratory. Educational Policy, 29, 230-256.
8.
ChenE.HeritageM.LeeJ. (2005). Identifying and monitoring students’ learning needs with technology. Journal of Education for Students Placed at Risk, 10, 309-332.
9.
ChoV.WaymanJ. C. (2014). Districts’ efforts for data use and computer data systems: The role of sensemaking in system use and implementation. Teachers College Record, 116(2), 1-45.
10.
CoburnC. E. (2001). Collective sensemaking about reading: How teachers mediate reading policy in their professional communities. Educational Evaluation and Policy Analysis, 23, 145-170.
11.
CoburnC. E.HonigM. I.SteinM. K. (2009). What’s the evidence on districts’ use of evidence. In BransfordJ.GomezL.LamD.VyeN. (Eds.), Research and practice: Towards a reconciliation (pp. 67-87). Cambridge, MA: Harvard Educational Press.
12.
CoburnC. E.TouréJ.YamashitaM. (2009). Evidence, interpretation, and persuasion: Instructional decision making at the district central office. Teachers College Record, 111, 1115-1161.
13.
CrawfordB. A. (2007). Learning to teach science as inquiry in the rough and tumble of practice. Journal of Research in Science Teaching, 44, 613-642.
14.
DalyA. J. (2012). Data, dyads, and dynamics: Exploring data use and social networks in educational improvement. Teachers College Record, 114, 1-38.
15.
DatnowA.ParkV.WohlstetterP. (2007). Achieving with data: How high-performing school systems use data to improve instruction for elementary students. Los Angeles, CA: Center on Educational Governance, USC.
16.
DiamondJ. B. (2007). Where the rubber meets the road: Rethinking the connection between high-stakes testing policy and classroom instruction. Sociology of Education, 80, 285-313.
17.
DiamondJ. B.SpillaneJ. (2004). High-stakes accountability in urban elementary schools: Challenging or reproducing inequality?Teachers College Record, 106, 1145-1176.
18.
DuschlR. A.OsborneJ. (2002). Supporting and promoting argumentation discourse in science education. Studies in Science Education, 38(1), 39-72.
19.
EarlL.KatzS. (2002). Leading schools in a data-rich world. In LeithwoodK. A.HallingerP. (Eds.), Second international handbook of educational leadership and administration (pp. 1003-1022). The Netherlands: Springer.
20.
EdmondsR. (1979). Effective schools for the urban poor. Educational Leadership, 37(1), 15-24.
21.
Every Student Succeeds Act of 2015, Pub. L. No. 114-95, § 114, Stat. 1177 (2015-2016).
22.
FeldmanJ.TungR. (2001). Using data-based inquiry and decision making to improve instruction. ERS Spectrum, 19(3), 10-19.
23.
FuhrmanS.ElmoreR. F. (Eds.). (2004). Redesigning accountability systems for education. New York, NY: Teachers College Press.
24.
FurtakE. M.SeidelT.IversonH.BriggsD. C. (2012). Experimental and quasi-experimental studies of inquiry-based science teaching: A meta-analysis. Review of Educational Research, 82, 300-329.
25.
GallagherL.MeansB.PadillaC. (2008). Teachers’ use of student data systems to improve instruction: 2005 to 2007. Washington, DC: U.S. Department of Education.
26.
HalversonR.GriggJ.PrichettR.ThomasC. (2007). The new instructional leadership: Creating data-driven instructional systems in schools. Journal of School Leadership, 17, 159-193.
27.
HamiltonL. S.BerendsM.StecherB. M. (2005). Teachers’ responses to standards-based accountability. Santa Monica, CA: RAND.
28.
HamiltonL. S.StecherB. (2006). Measuring instructional responses to standards-based accountability. Santa Monica, CA: RAND.
29.
HamiltonL. S.StecherB. M.KleinS. P. (2002). Making sense of test-based accountability in education. Santa Monica, CA: RAND.
30.
HanushekE. A.RaymondM. E. (2005). Does school accountability lead to improved student performance?Journal of Policy Analysis and Management, 24, 297-327.
31.
HofsteinA.LunettaV. N. (2004). The laboratory in science education: Foundations for the twenty-first century. Science Education, 88(1), 28-54.
32.
HonigM. I. (2006). Street-level bureaucracy revisited: Frontline district central-office administrators as boundary spanners in education policy implementation. Educational Evaluation and Policy Analysis, 28, 357-383.
33.
HonigM. I. (2009). No small thing: School district central office bureaucracies and the implementation of new small autonomous schools initiatives. American Educational Research Journal, 46, 387-422.
34.
HonigM. I.CoburnC. (2008). Evidence-based decision making in school district central offices: Toward a policy and research agenda. Educational Policy, 22, 578-608.
35.
HornI. S.KaneB. D.WilsonJ. (2015). Making sense of student performance data: Data use logics and mathematics teachers’ learning opportunities. American Educational Research Journal, 52, 208-242.
36.
IngramD.Seashore LouisK.SchroederR. (2004). Accountability policies and teacher decision making: Barriers to the use of data to improve practice. Teachers College Record, 106, 1258-1287.
37.
JimersonJ. B.WaymanJ. C. (2015). Professional learning for using data: Examining teacher needs and supports. Teachers College Record, 117(4), 1-36.
38.
KhattriN.SweetD. (1996). Assessment reform: Promises and challenges. In KaneM. B.MitchellR. (Eds.), Implementing performance assessment: Promises, problems, and challenges (pp. 1-21). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
39.
KolodnerJ. L.CampP. J.CrismondD.FasseB.GrayJ.HolbrookJ.. . . RyanM. (2003). Problem-based learning meets case-based reasoning in the middle-school science classroom: Putting learning by design (tm) into practice. The Journal of the Learning Sciences, 12, 495-547.
40.
KrajcikJ.McNeillK. L.ReiserB. J. (2008). Learning-goals-driven design model: Developing curriculum materials that align with national standards and incorporate project-based pedagogy. Science Education, 92(1), 1-32.
41.
LachatM. A.SmithS. (2005). Practices that support data use in urban high schools. Journal of Education for Students Placed at Risk, 10, 333-349.
42.
LittleJ. W. (2012). Understanding data use practice among teachers: The contribution of micro-process studies. American Journal of Education, 118, 143-166.
43.
LouisK. S. (2007). Trust and improvement in schools. Journal of Educational Change, 8(1), 1-24.
44.
LoveN. (2000). Using data, getting results: Collaborative inquiry for school-based mathematics and science reform. Cambridge, MA: Regional Alliance at TERC.
45.
MandinachE. B.HoneyM.LightD.BrunnerC. (2008). A conceptual framework for data-driven decision making. In MandinachE. B.HoneyM. (Eds.), Data-driven school improvement: Linking data and learning (pp. 13-31). New York, NY: Teachers College Press.
46.
MarshJ. A.McCombsJ. S.MartorellF. (2010). How instructional coaches support data-driven decision making policy implementation and effects in Florida middle schools. Educational Policy, 24, 872-907.
47.
MarshJ. A.PaneJ. F.HamiltonS. (2006). Making sense of data-driven decision making in education: Evidence from recent RAND research. Santa Monica, CA: RAND.
48.
MarxR. W.HarrisC. J. (2006). No child left behind and science education: Opportunities, challenges, and risks. Elementary School Journal, 106, 455-466.
49.
MassellD. (2001). The theory and practice of using data to build capacity: State and local strategies and their effects. In (National Society for the Study of Education). Yearbook of the national society for the study of education (Vol. 2, pp. 148-169). Chicago, IL: University of Chicago Press.
50.
MatsumuraL. C.WangE. (2014). Principals’ sensemaking of coaching for ambitious reading instruction in a high-stakes accountability policy environment. Education Policy Analysis Archives, 22(51).
51.
McLaughlinM. W. (1990). The Rand change agent study revisited: Macro perspectives and micro realities. Educational Researcher, 19(9), 11-16.
52.
McNeillK. L.KrajcikJ. (2008). Inquiry and scientific explanations: Helping students use evidence and reasoning. In LuftJ.BellR.Gess-NewsomeJ. (Eds.), Science as inquiry in the secondary setting (pp. 121-134). Arlington, VA: National Science Teachers Association Press.
53.
MerriamS. B. (2009). Qualitative research: A guide to design and implementation (Revised and Expanded from Qualitative Research and Case Study Applications in Education). San Francisco, CA: Jossey-Bass.
54.
MilnerA. R.SondergeldT. A.DemirA.JohnsonC. C.CzerniakC. M. (2012). Elementary teachers’ beliefs about teaching science and classroom practice: An examination of pre/post NCLB testing in science. Journal of Science Teacher Education, 23, 111-132.
55.
MinnerD. D.LevyA. J.CenturyJ. (2010). Inquiry-based science instruction—What is it and does it matter? Results from a research synthesis years 1984 to 2002. Journal of Research in Science Teaching, 47, 474-496.
56.
National Research Council. (2007). Taking science to school: Learning and teaching science in grades K-8. Washington, DC: National Academies Press.
57.
National Research Council. (2012). A framework for K-12 science education. Washington, DC: National Academy Press.
58.
NicholsS. L.BerlinerD. C. (2007). Collateral damage: How high-stakes testing corrupts America’s schools. Cambridge, MA: Harvard Education Press.
59.
O’DayJ. A.SmithM. S. (1993). Systemic reform and educational opportunity. In FuhrmanS. H. (Ed.), Designing coherent education policy: Improving the system (pp. 250-312). New Brunswick, NJ: Consortium for Policy Research in Education.
60.
OrfieldM. (2002). American metropolitics: The new suburban reality. Washington, DC: The Brookings Institution.
61.
OsborneJ. (2014). Teaching scientific practices: Meeting the challenge of change. Journal of Science Teacher Education, 25, 177-196.
62.
PalmerD.Snodgrass RangelV. W. (2011, July). High stakes accountability and policy implementation: Teacher decision-making in bilingual classrooms in Texas. Educational Policy, 25(54), 614-647.
63.
PurkeyS. C.SmithM. S. (1983). Effective schools: A review. The Elementary School Journal, 83, 427-452.
64.
RexL.NelsonM. (2004). How teachers’ professional identities position high-stakes test preparation in their classrooms. The Teachers College Record, 106, 1288-1331.
65.
Rustique-ForresterE. (2005). Accountability and the pressures to exclude: A cautionary tale from England. Education Policy Analysis Archives, 13(26).
66.
SmithM. S.O’DayJ. (1990). Systemic school reform. Journal of Education Policy, 5, 233-267.
67.
Snodgrass RangelV.MonroyC.BellE. R. (2016, August). Science teachers’ data use practices: A descriptive analysis. Education Policy Analysis Archives, 24(55).
68.
Snodgrass RangelV.BellE. R.MonroyC. (2016, November). A descriptive analysis of instructional coaches’ data use in science. School Effectiveness and School Improvement, 28(2), 217-241. doi:http://dx.doi.org/10.1080/09243453.2016.1255232
69.
SongerN. B.GotwalsA. W. (2012). Guiding explanation construction by children at the entry points of learning progressions. Journal of Research in Science Teaching, 49, 141-165.
70.
SongerN. B.Ruiz-PrimoM. A. (2012). Assessment and science education: Our essential new priority?Journal of Research in Science Teaching, 49, 683-690.
71.
SpillaneJ. P. (2000). Cognition and policy implementation: District policymakers and the reform of mathematics education. Cognition and Instruction, 18, 141-179.
72.
SpillaneJ. P. (2012). Data in practice: Conceptualizing the data-based decision-making phenomena. American Journal of Education, 118, 113-141.
73.
StecherB.HamiltonL. S.GonzalezG. (2003). Working smarter to leave no child behind. Santa Monica, CA: RAND.
74.
StodolskyS. S.GrossmanP. L. (1995). The impact of subject matter on curricular activity: An analysis of five academic subjects. American Educational Research Journal, 32, 227-249.
75.
StringfieldS.WaymanJ. C.YakimowskiM. E. (2005). Scaling up data use in classrooms, schools, and districts. In DedeC.HonanJ. P.Petersl. C. (Eds.), Scaling up success: Lessons learned from technology-based educational improvement (pp. 133-152). San Francisco, CA: Jossey-Bass.
76.
SupovitzJ. (2009). Can high stakes testing leverage educational improvement? Prospects from the last decade of testing and accountability reform. Journal of Educational Change, 10, 211-227.
77.
SupovitzJ. (2012). Getting at student understanding – The key to teachers’ use of test data. Teachers College Record, 114(11), n11.
WaymanJ. C. (2005). Involving teachers in data-driven decision making: Using computer data systems to support teacher inquiry and reflection. Journal of Education for Students Placed at Risk, 10, 295-308.
80.
WaymanJ. C.BrewerC.StringfieldS. (2009). Leadership for effective data use. In Annual meeting of the American educational research association. San Diego, CA.
81.
WaymanJ. C.ChoV.JimersonJ. B.Snodgrass RangelV. W. (2015). A look into the workings of data use in a mid-sized district. In SutherlandI.SanzoK. S.ScribnerJ. P. (Eds.), Leading small and mid-sized urban school districts. Bingley, UK: Emerald Books.
82.
WaymanJ. C.ChoV.JimersonJ. B.SpikesD. D. (2012). District-wide effects on data use in the classroom. Education Policy Analysis Archives, 20(25), 1-28.
83.
WaymanJ. C.ChoV.JohnstonM. T. (2007). The data-informed district: A district-wide evaluation of data use in the Natrona County School District. Austin: The University of Texas at Austin.
84.
WaymanJ. C.MidgleyS.StringfieldS. (2007). Leadership for data-based decision-making: Collaborative educator teams. In DanzigA. B.BormanK. M.JonesB. A.WrightW. F. (Eds.), Learner-centered leadership: Research, policy, and practice (pp. 189-206). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
85.
WaymanJ. C.Snodgrass RangelV. W.JimersonJ. B.ChoV. (2010). Improving data use in district three: Becoming a data-informed district. Austin: The University of Texas.
86.
WaymanJ. C.StringfieldS. (2006). Technology-supported involvement of entire faculties in examination of student data for instructional improvement. American Journal of Education, 112, 549-571.
87.
WaymanJ. C.StringfieldS.YakimowskiM. (2004). Software enabling school improvement through analysis of student data. Baltimore, MD: Johns Hopkins University.
88.
WertsA.BrewerC. (2015). Reframing the study of policy implementation lived experience as politics. Educational Policy, 29, 206-229.
89.
WigginsG. (1990). The case for authentic assessment. Washington, DC: ERIC Clearinghouse on Tests, Measurement, and Evaluation.
90.
WiliamD.ThompsonM. (2007). Integrating assessment with learning: What will it take to make it work? In DwyerC. A. (Ed.), The future of assessment: Shaping teaching and learning (pp. 53-82). Mahwah, NJ: Lawrence Erlbaum Associates.
91.
WohlstetterP.DatnowA.ParkV. (2008). Creating a system for data-driven decision-making: Applying the principal-agent framework. School Effectiveness and School Improvement, 19, 239-259.
92.
YoungV. (2006). Teachers’ use of data: Loose coupling, agenda setting, and team norms. American Journal of Education, 112, 521-548.
