Research on data and evidence use suggest that productive use depends on interactive processes, including sustained interactions between educators and researchers. Recent research on research-practice partnerships (RPPs) has examined conditions under which these sustained collaborations support evidence use. Findings from these studies can inform research on early warning indicators, helping interpret implementation studies of productive use and creating conditions for use of data from early warning indicator systems.
Purpose
This chapter presents results of a review of studies of data and evidence use within RPPs. It investigates the claim that RPPs can support productive data and evidence use only under certain conditions, conditions that are relevant to studying and supporting the implementation of early warning indicator systems in education.
Research Design
The synthesis focused on identifying studies published between 2013 and 2019 as journal articles, book chapters, and technical reports that focused on data and evidence use in RPPs. To be included, studies had to be empirical and related to the focal topics. A total of 114 studies met criteria for inclusion. For all studies, members of the research team developed summaries, which the team then discussed. Themes emerged from summaries, grouped by RPP, and from team discussions.
Findings
The review found six supportive conditions were needed for productive use of data and evidence to guide decision-making and action. These were (1) valuation of knowledge, experience, and perspectives of partners; (2) processes for identifying sources of evidence needed to answer questions that are priorities for educators and community partners; (3) complementarity of knowledge of partners; (4) adoption of a learning perspective on systems change; (5) routines for sensemaking and collaboration; (5) synchrony with decision-making processes; and (6) a commitment to developing and using evidence among partner organizations.
Conclusions
Developers of early warning indicator systems should consider ways an RPP can support the creation of conditions for productive use of data from systems. Effective systems likely will depend on making room for educator voice and valuing of practitioner perspectives at all stages of design and implementation of systems. They will also require allocation of time and skill for structuring opportunities to make sense of data and developing a culture where evidence plays an important role in decision-making.
Get full access to this article
View all access options for this article.
References
1.
AllensworthE. (2013). The use of ninth-grade early warning indicators to improve Chicago schools. Journal of Education for Students Placed at Risk, 18(1–2), 68–83.
2.
AllensworthE., & EastonJ. Q. (2005). The on-track indicator as a predictor of high school graduation. UChicago Consortium on Chicago School Research.
3.
AllensworthE., & EastonJ. Q. (2007). What matters most for staying on-track and graduating in Chicago public high schools: A close look at course grades, failures, and attendance in the freshman year. UChicago Consortium on Chicago School Research.
4.
AmaraN., OuimetM., & LandryR. (2004). New evidence on instrumental, conceptual, and symbolic utilization of university research in government agencies. Science Communication, 26(1), 75–106.
5.
BevanB., GutwillJ. P., PetrichM., & WilkinsonK. (2015). Learning through STEM-rich tinkering: Findings from a jointly negotiated research project taken up in practice. Science Education, 99(1), 98–120.
6.
BookerA., & GoldmanS. V. (2016). Participatory design research as a practice for systemic repair: Doing hand-in-hand math research with families. Cognition and Instruction, 34(3), 223–235.
7.
BrykA. S., GomezL. M., GrunowA., & LeMahieuP. G. (2015). Learning to improve: How America's schools can get better at getting better. Harvard University Press.
8.
BrykA. S., SebringP. B., AllensworthE., LuppescuS., & EastonJ. Q. (2010). Organizing schools for improvement: Lessons from Chicago. University of Chicago Press.
9.
BulkleyK. E., ChristmanJ. B., GoertzM. E., & LawrenceN. R. (2010). Building with benchmarks: The role of the district in Philadelphia's Benchmark Assessment System. Peabody Journal of Education, 85(2), 186–204.
10.
CobbP. A., & JacksonK. (2012). Analyzing educational policies: A learning design perspective. Journal of the Learning Sciences, 21(4), 487–521.
11.
CobbP. A., JacksonK., HenrickE. C., SmithT. M., & The MIST Team (Ed.). (2018). Systems for instructional improvement: Creating coherence from the classroom to the district office. Harvard Education Press.
12.
CobbP. A., JacksonK., SmithT., SorumM., & HenrickE. C. (2013). Design research with educational systems: Investigating and supporting improvements in the quality of mathematics teaching at scale. In FishmanB.J., PenuelW.R., AllenA.-R., & ChengB.H. (Eds.), Design-based implementation research: Theories, methods, and exemplars. National Society for the Study of Education Yearbook (pp. 320–349). Teachers College.
13.
CoburnC. E., & PenuelW. R. (2016). Research–practice partnerships in education: Outcomes, dynamics, and open questionsEducational Researcher, 45(1), 48–54.
14.
CoburnC. E., PenuelW. R., & GeilK. (2013). Research–practice partnerships at the district level: A new strategy for leveraging research for educational improvement. University of California and University of Colorado.
15.
CohenD. K., MoffittS. L., & GoldinS. (2007). Policy and practice: The dilemma. American Journal of Education, 113(4), 515–548.
16.
ContandriopoulosD., LemireM., DenisJ.-L., & TremblayE. (2010). Knowledge exchange processes in organizations and policy arenas: A narrative systematic review of the literature. The Milbank Quarterly, 88(4), 444–483.
17.
CousinsJ. B., & EarlL. M. (Ed.). (2004). Participatory evaluation in education: Studies in evaluation use and organizational learning. Falmer Press.
DavidsonK. L., & FrohbieterG. (2011). District adoption and implementation of interim and benchmark assessments (CSE Technical Report 806). CRESST.
20.
DennerJ., BeanS., CampeS., MartinezJ., & TorresD. (2019). Negotiating trust, power, and culture in a research–practice partnership. AERA Open, 5(2), 1–11.
21.
DeussenT., HansonH., & BishtB. (2017). Are two commonly used early warning indicators accurate predictors of dropout for English learner students? Evidence from six districts in Washington state. U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Northwest.
22.
DonovanM. S. (2013). Generating improvement through research and development in educational systems. Science, 340, 317–319.
23.
DonovanM. S., SnowC. E., & DaroP. (2013). The SERP approach to problem-solving research, development, and implementation. In FishmanB.J., PenuelW.R., AllenA.-R., & ChengB.H. (Eds.), Design-based implementation research. National Society for the Study of Education Yearbook (pp. 400–425). Teachers College.
24.
DukakisK., & StrobelK. R. (2010, March). University and community partners sharing data to improve systems, services and policy. Paper presented at the Society for Research on Adolescence 2010 Biennial Meeting, Philadelphia, PA.
25.
EarlL. M. (1995). District-wide evaluation of school improvement: A system partners approach. In CousinsJ.B., & EarlL.M. (Eds.), Participatory evaluation in education: Studies in evaluation use and organizational learning (pp. 21–32). Routledge.
26.
FarrellC. C., & CoburnC. E. (2017). Absorptive capacity: A conceptual framework for understanding district central office learning. Journal of Educational Change, 18(2), 135–159.
27.
FarrellC. C., CoburnC. E., & ChongS. (2019). Under what conditions do school districts learn from external partners? The role of absorptive capacity. American Educational Research Journal, 56(3), 955–994.
28.
FarrellC. C., DavidsonK. L., Repko-ErwinM., PenuelW. R., QuantzM., WongH., RiedyR., & BrinkZ. (2018). A descriptive study of the IES Researcher–Practitioner Partnerships in Education Research program. National Center for Research in Policy and Practice.
29.
FarrellC. C., HarrisonC., & CoburnC. E. (2019). “What the hell is this, who the hell are you?” Role and identity negotiation in research–practice partnerships. AERA Open, 5(2), 1–13.
30.
FarrellC. C., & MarshJ. A. (2016). Metrics matter: How data characteristics shape teachers’ responses. Educational Administration Quarterly, 52(3), 423–462.
31.
GallagherH. A., & CottinghamB. (2019). Learning and practicing continuous improvement: Lessons from the CORE districts. Policy Analysis for California Education.
32.
GhisoM. P., CampanoG., ScheabE. R., AsaahD., & RusojaA. (2019). Mentoring in research–practice partnerships: Toward democratizing expertise. AERA Open, 5(4), 1–12.
33.
GoldsteinH., McKennaM. F., BarkerR. M., & BrownT. M. (2019). Research–practice partnership: Application to implementation of multitiered system of supports in early childhood education. Perspectives of the ASHA Special Interest Groups, 4, 38–50.
34.
GriecoJ., VovakH., & LumC. (2014). Examining research–practice partnerships in policing evaluations. Policing: A Journal of Policy and Practice, 8(4), 368–378.
35.
HarrisonC., DavidsonK. L., & FarrellC. C. (2017). Building productive relationships: District leaders’ advice to researchers. International Journal of Education Policy & Leadership, 12(4), 1–16.
36.
HarrisonC., WachenJ., BrownS., & Cohen-VogelL. (2019). A view from within: Lessons learned from doing continuous improvement research. Teachers College Record, 121(9), 1–38.
37.
HascheL. K., LenzeS., BrownT., LawrenceL., NickelM., Morrow-HowellN., & ProctorE. K. (2014). Adapting collaborative depression care for public community long-term care: Using research–practice partnerships. Administration and Policy in Mental Health, 41(5), 687–696. https://doi.org/10.1007/s10488-013-0519-z
38.
HaskinsR., & BaronJ. (2011). Building the connection between policy and evidence: The Obama evidence-based initiatives. NESTA.
HenrickE. C., CobbP., PenuelW. R., JacksonK., & ClarkT. R. (2017). Assessing research–practice partnerships: Five dimensions of effectiveness. William T. Grant Foundation.
41.
HenrickE. C., KlafehnA. B., & CobbP. A. (2018). Assessing the impact of partnership recommendations on district instructional improvement strategies. In CobbP.A., JacksonK., HenrickE., SmithT.M., & The MIST Team (Eds.), Systems for instructional improvement: Creating coherence from the classroom to the district office. Harvard Education Press.
HessF. M., & PetrilliM. J. (2006). No Child Left Behind. Peter Lang.
44.
HonigM. I. (2013). Beyond the policy memo: Designing to strengthen the practice of district central office leadership for instructional improvement at scale. In FishmanB.J., PenuelW.R., AllenA.-R., & ChengB.H. (Eds.), Design-based implementation research. National Society for the Study of Education Yearbook (pp. 256–273). Teachers College.
45.
HonigM. I., VenkateswaranN., & McNeilP. (2017). Research use as learning: The case of fundamental change in school district central offices. American Educational Research Journal, 54(5), 938–971.
46.
HonigM. I., VenkateswaranN., McNeilP., & Myers-TwitchellP. (2014, April). Research use as learning: The case of fundamental change in school district offices. Paper presented at the Annual Meeting of the American Educational Research Association, Philadelphia, PA.
47.
HuguetA., AllenA.-R., CoburnC. E., FarrellC. C., KimD. H., & PenuelW. R. (2017). Locating data use in the microprocesses of district-level deliberations. Nordic Journal of Studies in Educational Policy, 3(1), 21–28.
48.
IkemotoG. S., & HonigM. I. (2010). Tools to deepen practitioners’ engagement with research: The case of the Institute for Learning. In CoburnC.E., & HonigM.I. (Eds.), Research and practice in education: Building alliances, bridging the divide (pp. 93–108). Rowman & Littlefield.
49.
IonG., StînguM., & MarinE. (2019). How can researchers facilitate the utilisation of research by policy-makers and practitioners in education?Research Papers in Education, 34(4), 483–498.
50.
IshimaruA. M., & TakahashiS. (2017). Disrupting racialized institutional scripts: Toward parent-teacher transformative agency for educational justice. Peabody Journal of Education, 92(3), 343–362.
51.
JacksonK., & CobbP. (2013). Coordinating professional development across contexts and role groups. In EvansM. (Ed.), Teacher education and pedagogy: Theory, policy and practice (pp. 80–99). Cambridge University Press.
52.
JohnsonK., GreenseidL. O., ToalS. A., KingJ. A., LawrenzF., & VolkovB. (2009). Research on evaluation use: A review of the empirical literature from 1986 to 2005. American Journal of Evaluation, 30(3), 377–410.
53.
JohnsonR., SeveranceS., PenuelW. R., & LearyH. A. (2016). Teachers, tasks, and tensions: Lessons from a research–practice partnership. Journal of Mathematics Teacher Education, 19(2), 169–185.
54.
JurowA. S., SheaM., & Van SteenisE. (2016). Extending the consequentiality of “invisible work” in the food justice movement. Cognition and Instruction, 34(3), 210–221.
55.
KelleherM. (2018). Progress and promise: Chicago's nation-leading educational gains. Joyce Foundation and Spencer Foundation.
56.
KempleJ. J., SegeritzM. D., & StephensonN. (2013). Building on-track indicators for high school graduation and college readiness: Evidence from New York City. Journal of Education for Students Placed at Risk (JESPAR), 18(1), 7–28.
57.
KochmanskiN. M., HenrickE. C., & CobbP. A. (2015, October). On the development of content-specific practical measures assessing aspects of instruction associated with student learning. Paper presented at Using Continuous Improvement to Integrating Design, Implementation, and Scale Up, Nashville, TN.
58.
Lopez-TurleyR., & StevensC. (2015). Lessons from a school district-university research partnership: The Houston education research consortium. Educational Evaluation and Policy Analysis, 37(1S), 6S–15S.
59.
Mac IverM. A., & MesselM. (2013). The ABCs of keeping on track to graduation: Research findings from Baltimore. Journal of Education for Students Placed at Risk (JESPAR), 18(1), 50–67.
60.
MeansB., PadillaC., DeBargerA., & BakiaM. (2009). Implementing data-informed decision making in schools: Teacher access, supports and use. SRI International.
61.
MoellerE., SeeskinA., & NagaokaJ. (2018). Practice-driven data: Lessons from Chicago's approach to research, data, and practice in education. UChicago Consortium on School Research.
62.
National Research Council. (2002). Scientific research in education. National Academies Press.
63.
National Research Council. (2012). Using science as evidence in public policy. National Academies Press.
64.
NayfackM., ParkV., HoughH., & WillisL. (2017). Building systems knowledge for continuous improvement: Early lessons from the CORE districts. Policy Analysis for California Education.
65.
NelsonI. A., LondonR. A., & StrobelK. R. (2015). Reinventing the role of the university researcher. Educational Researcher, 44(1), 17–26.
66.
OláhL. N., LawrenceN. R., & RigganM. (2010). Learning to learn from benchmark data: How teachers analyze results. Peabody Journal of Education, 85(2), 226–245.
67.
PenuelW. R. (2019). Infrastructuring as a practice of design-based research for supporting and studying equitable implementation and sustainability of innovations. Journal of the Learning Sciences, 28(4–5), 659–677.
68.
PenuelW. R., BriggsD. C., DavidsonK., HerlihyC., ShererD., HillH. C., FarrellC. C., & AllenA.-R. (2016). Findings from a national study on research use among school and district leaders. National Center for Research in Policy and Practice.
69.
PenuelW. R., DeBargerA. H., BoscardinC. K., MoorthyS., BeauvineauY., KennedyC., & AllisonK. (2017). Investigating science curriculum adaptation as a strategy to improve teaching and learning. Science Education, 101(1), 66–98.
70.
PenuelW. R., & FarrellC. C. (2017). Research–practice partnerships and ESSA: A learning agenda for the coming decade. In QuinteroE. (Ed.), Teaching in context: The social side of education reform. Harvard Education Press.
71.
PenuelW. R., FarrellC. C., AllenA.-R., ToyamaY., & CoburnC. E. (2018). What research district leaders find useful. Educational Policy, 32(4), 540–568.
72.
PenuelW. R., & GallagherD. (2017). Creating research–practice partnerships in education. Harvard Education Press.
73.
PenuelW. R., ReiserB. J., NovakM., McGillT., FruminK., Van HorneK., SumnerT., & WatkinsD. A. (2018, April). Using co-design to test and refine a model for three-dimensional science curriculum that connects to students’ interests and experiences. Paper presented at the Annual Meeting of the American Educational Research Association, New York, NY.
74.
PenuelW. R., & ShepardL. A. (2016). Assessment and teaching. In GitomerD.H., & BellC.A. (Eds.), Handbook of research on teaching (pp. 787–851). AERA.
75.
PeurachD. J., PenuelW. R., & RussellJ. L. (2019). Beyond ritualized rationality: Organizational dynamics of instructionally-focused continuous improvement. In ConnollyM., Eddy SpicerD.H., JamesC., & KruseS.D. (Eds.), SAGE international handbook of school organization (pp. 465–488). SAGE.
76.
PotvinA. S., KaplanR. G., BoardmanA. G., & PolmanJ. L. (2017). Configurations in co-design: Participant structures in partnership work. In BevanB., & PenuelW.R. (Eds.), Connecting research and practice for educational improvement (pp. 135–149). Routledge.
77.
RiemerM., KelleyS. D., CaseyS., & HaynesK. T. (2012). Developing effective research–practice partnerships for creating a culture of evidence-based decision making. Administration and Policy in Mental Health and Mental Health Services Research, 39(4), 248–257.
78.
RigbyJ., FormanS., FoxA., & KazemiE. (2018). Leadership development through design and experimentation: Learning in a research–practice partnership. Journal of Research on Leadership Education, 13(3), 316–339.
79.
RoderickM., EastonJ. Q., & SebringP. B. (2009). The Consortium on Chicago School Research: A new model for the role of research in supporting urban school reform. UChicago Consortium on Chicago School Research.
80.
RoderickM., Kelley-KempleT., JohnsonD. W., & BeechumN. O. (2014). Preventable failure: Improvements in long-term outcomes when high schools focused on the ninth grade year. UChicago Consortium on School Research.
81.
RosenquistB. A., HenrickE. C., & SmithT. M. (2015). Research–practice partnerships to support the development of high quality mathematics instruction for all students. Journal of Education for Students Placed at Risk (JESPAR), 20(1–2), 42–57.
82.
RussellJ. L., BrykA. S., DolleJ. R., GomezL. M., LeMahieuP. G., & GrunowA. (2017). A framework for the initiation of networked improvement communities. Teachers College Record, 119, 1–36.
83.
RussellJ. L., CorrentiR. J., SteinM. K., BillV., HannanM., SchwartzN. L., BookerL. N., PrattN. R., & MatthisC. (2020). Learning from adaptation to support instructional improvement at scale: Understanding coach adaptation in the TN Mathematics Coaching Project. American Educational Research Journal, 51(1), 148–187.
84.
RussellJ. L., JacksonK., KrummA. E., & FrankK. A. (2013). Theories and research methodologies for design-based implementation research: Examples from four cases. Yearbook of the National Society for the Study of Education, 112(2), 157–191.
85.
SantoR., ChingD., PepplerK. A., & HoadleyC. (2017). Participatory knowledge building within research–practice partnerships in education. SAGE.
86.
SeidH. H. (2016). The practices of action research and its contribution for school improvement in secondary schools of Borena Zone, Oromia Regional State. Unpublished master's thesis, Harayama University, Harayama, Ethiopia.
87.
ShepardL. A., DavidsonK. L., & BowmanR. (2011). How middle school mathematics teachers use interim and benchmark assessment data (CSE Technical Report 807). CRESST.
88.
StuitD., O'CummingsM., NorburyH., HeppenJ., DhillonS., LindsayJ., & ZhuB. (2016). Identifying early warning indicators in three Ohio school districts. U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Midwest.
89.
ThompsonJ., RichardsJ., ShimS.-Y., LohwasserK., Von EschK. S., ChewC., SjobergB., & MorrisA. (2019). Launching networked PLCs: Footholds into creating and improving knowledge of ambitious and equitable teaching practices in an RPP. AERA Open, 5(3), 1–22.
90.
ThompsonK. D., MartinezM. I., ClintonC., & DiazG. (2017). Considering interest and action: Analyzing types of questions explored by researcher-practitioner partnerships. Educational Researcher, 48(6), 464–473.
91.
VakilS., McKinney de RoystonM., Suad NasirN., & KirshnerB. (2016). Rethinking race and power in design-based research: Reflections from the field. Cognition and Instruction, 34(3), 194–209.
92.
WarrenM. R., ParkS. O., & TiekenM. C. (2016). The formation of community-engaged scholars: A collaborative approach to doctoral training in education research. Harvard Educational Review, 86(2), 233–260. https://doi.org/10.17763/0017-8055.86.2.233
93.
Weidler-LewisJ., PenuelW. R., & Van HorneK. (2017, April). Developing a measure of teachers’ vision for equitable science teaching and learning. Paper presented at the NARST Annual Conference, San Antonio, TX.
