Sage Journals: Discover world-class research

Abstract

This essay offers a framework for broader community involvement as a means of increasing the relevance and usefulness of evidence developed. This essay begins by defining key concepts related to democratizing the development of evidence. The sections that follow outline a logic model that calls for a bidirectional, iterative set of core activities to draw on insights from practitioners, community stakeholders, and researchers to generate a learning agenda, analysis plans, and findings that can be used to develop or refine policies and practices. The essay concludes with implications for advocates of democratizing the development of evidence and for researchers.

Keywords

accountability decision making educational policy educational reform improvement science policy analysis research utilization

With humility, researchers and practitioners can acknowledge the limitations of what they “know,” remain open to the possibility of being wrong, and be willing to learn and improve their own knowledge and work over time.

—Tseng et al. (2018)

Introduction

Over the past two decades, federal education policies have spurred the development of research evidence—evidence derived when systematic methods and analyses are used to address questions (William T. Grant Foundation, n.d.). Since its inception in 2002, the Institute of Education Sciences has awarded research grants, provided independent evaluations of federally funded education programs, and synthesized existing evidence about educational practices, programs, and policies through the What Works Clearinghouse (Institute of Education Sciences, 2018). Under the most recent reauthorization of the Elementary and Secondary Education Act, the 2015 Every Student Succeeds Act (ESSA), educational leaders are required to justify certain spending decisions with evidence (U.S. Department of Education, 2016).

The rationale behind encouraging evidence development and use is straightforward: evidence-informed decisions should yield better policies and practices, ultimately resulting in better educational outcomes. Yet this rationale rests on the assumption that educators and policymakers will be open to evidence that suggests a need for improvement, or that is from different settings. Education leaders may feel pressure to avoid appearing to lack confidence in an established vision or set of reforms (Corcoran et al., 2001), and publicly elected school boards and school board appointed superintendents face disincentives to be transparent about evidence that identifies weaknesses, since voters tend to remember bad news over good (Nielsen & Moynihan, 2017). Additionally, policymakers and practitioners have strong preferences for internal sources of evidence (Finnigan et al., 2013; Massell et al., 2012), and thus may be skeptical of the relevance of evidence developed outside their context.

One potential avenue to counter the tendency to disregard or selectively attend to evidence is democratizing evidence—enabling a variety of stakeholders to participate in the production of evidence to inform educational improvement (Tseng et al., 2018). Providing opportunities for stakeholders to offer input on what questions are asked and answered should result in an evidence base that is more relevant to the concerns of stakeholders, including the communities served by educational institutions. Greater transparency in the evidence development process and responsiveness to stakeholder concerns may increase the community’s desire to act on findings (Nelson et al., 2015), creating incentives for leaders to consider evidence in decision-making processes.

The purpose of this essay is to establish a framework for broader community involvement as a means of increasing the relevance and usefulness of evidence developed. Similar to the set of boundaries Collins (2019) established for “deliberative culture,” this essay offers principles for a democratic approach to evidence building. First, a diverse group of stakeholders, including communities marginalized by systemic inequalities, should have the opportunity to share viewpoints. Incorporating the perspectives of stakeholders can heighten attention to systemic racism and disenfranchisement of minoritized communities in the process of generating evidence, yielding an evidence base that can inform efforts to improve equity (Ming & Goldenberg, 2021). Second, those leading the process agree to pursue a common interest over individual self-interests, taking into consideration the extent to which actions are expected to benefit the public as opposed to being used for private gain (Hutt & Polikoff, 2020).

In the education space, adherence to the democratic principles can be seen in school board meetings that involve both constituent participation and deliberation by policymakers (Collins, 2021) and in school district outreach to gather stakeholder input on specific topics. A recent study found that respondents who viewed a school board meeting in which the board provided both opportunities for people to comment and responses to the commenters led to increased trust in local officials and a stronger willingness to attend school board meetings in the future (Collins, 2021). Deliberative civic engagement processes, such as participatory budgeting, allow community members to learn, listen, debate, and deliberate about the allocation of public resources, and can increase community trust in leadership and democratic institutions (Roza, 2021).

The remainder of this essay begins with definitions of key concepts related to democratizing the development of evidence. The sections that follow outline a logic model (see Figure 1) that calls for a bidirectional, iterative set of core activities to draw on insights from practitioners, community stakeholders, and researchers to generate a learning agenda, analysis plans, and findings that can be used to develop or refine policies and practices. The essay concludes with implications for advocates of democratizing the development of evidence and for researchers.

Figure 1.

Logic model for democratizing evidence use.

Defining Key Concepts Related to Democratizing Evidence

Learning Agendas

Gathering stakeholder input on what evidence is needed, and for what purpose, can inform plans for identifying and addressing priority questions relevant to the programs and policies of an agency. These plans, known as learning agendas, facilitate development of evidence that supports organizational learning and improves decision making (Office of Evaluation Sciences, n.d.). The Evidence Act toolkit encourages frequent internal and external stakeholder engagement throughout the process of setting a learning agenda (Office of Evaluation Sciences, n.d.).

Evidence

Addressing the questions of a learning agenda involves developing research evidence through the systematic collection and analysis of appropriate sources of data. Stakeholders in the education community may view a variety of data sources as evidence. For example, elected school board officials view experience, testimony, examples, law, and policies as types evidence (Asen et al., 2012), and education leaders and teachers view student performance data as an especially credible source of evidence (Finnigan et al., 2013). To meet the definition of research evidence, these sources of data would need to be appropriate for the questions asked, and systematically collected and analyzed.

To illustrate the distinction between data sources and evidence, consider the question of whether schools are improving student achievement. Education stakeholders may look to proficiency rates for answers, but schools where many students are well below proficiency might show little improvement in proficiency rates, even if students’ underlying test scores improve considerably (Shafer, 2016). Given the question, year-to-year changes in test scores is a more appropriate source of data than proficiency rates, with the appropriate method for analyzing test scores depending on the interpretations stakeholders wish to make (Castellano & Ho, 2013).

Defining Evidence in Light of Democratic Principles

What constitutes relevant and convincing evidence (Gordon & Conaway, 2020), and who gets to decide? Adherence to the democratic principles requires considering stakeholders’ concerns. Practitioners may be interested in evidence that can be used to inform and improve program implementation, while federal policy guidance privileges evidence of causal impacts. An agenda informed by the concerns of stakeholders may include a range of methodological approaches that allow for describing conditions, implementation, or outcomes; comparing conditions, implementation, or outcomes to established criteria or to alternative conditions; or estimating the causal impact of a program or policy on a desired outcome (Office of Evaluation Sciences, n.d.).

Developing relevant and convincing evidence also requires consideration to the outcomes that stakeholders value. Education agencies at the federal, state, and local level attend to evidence derived from standardized tests, yet community stakeholders may view students’ sense of safety and belonging at school as valued outcomes. Historically, academic concerns have been privileged over practical concerns (Ming & Goldenberg, 2021), and conventional assumptions tend to center white middle class values, taking a deficit lens to minoritized students (Kirkland, 2019). Navigating differences in what types of evidence stakeholders find relevant and convincing is a challenging but essential aspect of democratizing evidence.

Deliberative democratic structures could add value during the generation of evidence in response to the questions, as stakeholders contribute insights about local context. Democratic structures also offer an opportunity to incorporate a more diverse range of perspectives in evidence interpretation. These three approaches to applying democratic principles to evidence building align to the outputs of the logic model in Figure 1: a learning agenda, an analysis plan, and findings that reflect stakeholder input. In the sections that follow, the key inputs, core activities, and challenges to building evidence in accordance with democratic principles are explored in more detail.

Key Inputs in Democratizing Evidence

Key inputs in the logic model include a leadership council, a team of internal or external researchers and brokers, and time and space for events. In keeping with the principles of democracy, the leadership council should include representation from key stakeholder communities to ensure that the learning agenda reflects the questions of educators seeking to improve their work, as well as parents and community groups invested in improving educational outcomes (Jackson et al., 2021; Tseng et al., 2018). To foster the development of learning agendas that intentionally benefit communities of color (Michener, 2020), minoritized communities that have traditionally had less voice in decision making should be included. While inclusion, by itself, is insufficient to realize democratic ideals, drawing on different perspectives is a step toward an evidence base that is relevant and convincing to a variety of stakeholders.

Developing and enacting a learning agenda is likely to require substantial analytic capacity. While educational agencies often have data-savvy staff, they may have limited time available due to the demands of compliance and accountability reporting. Leaders could reallocate tasks so that these staff have dedicated time to address the learning agenda, or hire additional staff. Advantages to having research partners embedded in an education agency include opportunities to establish relationships and build trust with practitioners, awareness of the intricacies of administrative data, and knowledge of the local context (Conaway et al., 2015). As a result, internal research capacity enables quick-turnaround analyses in a time frame aligned with decision-makers’ needs.

Many education agencies have turned to external research partners or developed research-practice partnerships for additional expertise. External research partners may be better positioned to investigate long-term research questions, conduct resource-intensive data gathering, or when political independence is needed (Conaway et al., 2015). Research-practice partnerships involve long-term collaborations between researchers and practitioners, such as research alliances, design research partners, and networked improvement communities, as well as the Regional Education Laboratories funded by the U.S. Department of Education (Tseng et al., 2017). The National Network of Education Research-Practice Partnerships website (National Network of Education Research-Practice Partnerships, n.d.) contains numerous resources to support research partnerships.

The work of creating norms and routines that allow for connections across different stakeholder groups is often carried out by “brokers” who bridge the research and policy/practice worlds (Conaway, 2019). Brokers may serve a variety of roles, such as developing partnership governance and administrative structures, designing processes and communications routines, sharing knowledge of research and practice within organizations, and translating research to increase accessibility for a general audience (Hopkins et al., 2019; Neal et al., 2015; Wentworth et al., 2021).

Time and space for events are crucial inputs for democratizing evidence, particularly in communities where democratic structures are not the norm. The leadership committee could host meetings to solicit input from community groups and engage in deliberative discourse. Time and space are also critical for planning, negotiation, and developing of trust with research partners. In an example from Detroit, community partners and researchers participate in regular meetings and seasonal retreats for team building, information sharing, and critical reflection (Wilson, 2021). Nelson et al. (2015) found that data managers were often protective of their agency’s data, so research partners established protocols to facilitate data sharing and performed favors such as collecting data or conducting small within-agency analyses to build trust. Involving data managers in leadership council meetings could facilitate enactment of the learning agenda.

Lessons from a partnership in which schools, after-school programs, and public agencies came together to conduct research with university partners suggest that research partners need to demonstrate commitment to prioritizing community agencies’ goals (Nelson et al., 2015). Researchers repositioned themselves as invited collaborators (rather than leaders) in community-driven working groups, taking on the role of coaching agency staff on how to ask questions that were both researchable and actionable. High-quality, relevant research heightened the community’s interest in acting on findings (Nelson et al., 2015).

Codevelopment of Learning Agendas

Including practitioners and community members in the process of developing the learning agenda is key to democratizing evidence (Tseng et al., 2018). The core activities of the logic model depicted in Figure 1 include engagement with practitioners and community stakeholders to generate insights on challenges and potential solutions. While the strategic plans of the state education agency or school district can serve as a starting point for identifying evidence needs, involving stakeholders early in the process is critical, since how problems are framed influences how decision processes unfold, legitimizing some potential responses and not others (Coburn et al., 2009). Furthermore, questions about program and policy effectiveness might be framed differently depending on whether stakeholders’ goals emphasize adequacy, equality, or benefits to the disadvantaged (Brighouse et al., 2018).

Conversations with researchers might include discussion of existing data and research related to the challenges and potential solutions to refine the set of questions for the learning agenda. Review of research syntheses can mitigate the potential for an individual study or researcher to have an outsize influence on the discussion (Ming & Goldenberg, 2021). In addition, school staff appreciate efforts to rule out programs that might prove ineffective (Corcoran et al., 2001), and stakeholders may be especially interested in existing evidence if the options under consideration require large commitments of resources (Weiss, 1980). Programs with a rigorous evidence base may be more likely to successfully replicate. Among programs scaled up under Investing in Innovation grants, two of the four programs with prior experimental evidence of effectiveness had positive impacts on student achievement, while a smaller proportion of the programs with less rigorous prior evidence produced positive results (Boulay et al., 2018).

Sharing data and research should be viewed as a means of sparking dialogue, rather than ending conversations (Kirkland, 2019). The iterative nature of the core activities offers practitioners and community stakeholders an opportunity to discuss the relevance of existing research. The application of existing research may be viewed as a form of “wishful extrapolation” (Manski, 2013), particularly if studies are set in contexts dissimilar from their communities. Practitioners and community members can share insights on the extent to which local contextual factors vary from the study context in ways that could affect a proposed program or policy (Ming & Goldenberg, 2021). Ultimately, evidence from current local data may be more convincing than existing research to address questions (Finnigan et al., 2013; Massell et al., 2012; Yohalem & Tseng, 2015).

To prioritize the questions on the learning agenda, a recently developed framework for public accountability offers two key criteria to consider: actionability of information and the extent to which actions benefit the public as opposed to being used for private gain (Hutt & Polikoff, 2020). Since evidence that is relevant to the current policy agenda and upcoming decisions is more likely to be used (Conaway et al., 2015; Contandriopoulos et al., 2010), it may be worth prioritizing the questions on the learning agenda with these factors in mind. Additionally, by emphasizing benefits to the public, the framework reflects the democratic principle of putting common interests over individual self-interests. Since those with the authority to prioritize items on the learning agenda wield considerable power, granting this authority to the leadership council or an elected school board is more consistent with the democratic principles than leaving these decisions up to education leaders or researchers, as might otherwise be the case.

Analysis Plans

The core activities leading to the analysis plan involve practitioners and community stakeholders providing insight on relevant contextual factors and supports needed for implementation, and researchers identifying the data and methods best suited to answering the research questions. Community stakeholders can contribute to discussions in which the leadership council critically examines the social inequities and racial biases that exist in the context in which evidence is generated (Kirkland, 2019). Practitioners can share insights on the contextual factors that might support or derail implementation of initiatives, such as lack of resources, insufficient time for staff development, or constraints related to existing policies or informal norms. Stakeholder feedback on the analysis plans fosters the production of evidence that reflects “what we want to know, why we want to know it and how we envisage that evidence being used” (Nutley et al., 2013, p. 6).

The choice of what type of data and methods to use should be driven by the questions being asked (Ercikan & Roth, 2006). If practitioners and researchers are collaboratively developing interventions, design-based research would be an appropriate approach to developing knowledge (Penuel & Farrell, 2017). State and district leaders are often interested in understanding the factors that drive variation in the quality of implementation of programs and policies, the resources needed to ensure successful implementation, and drivers of the cost of different options (Conaway et al., 2015; Yohalem & Tseng, 2015). Observations, interviews, and focus groups with community members and practitioners responsible for enacting the programs and policies can be used to address “how” and “why” questions and assess whether programs and policies are working as intended. Interim analyses could support continuous improvement of key initiatives, increasing the likelihood that a new program or policy is implemented successfully.

Research partners could help develop experimental or quasi-experimental analysis plans for questions about causal impact. More complex analysis plans are warranted for causal questions since correlational evidence may be misleading due to selection bias. For example, programs that attract highly motivated students will tend to be positively associated with outcomes, while programs that involve less well-motivated students will appear to performing poorly. Implying a causal relationship where none exists may direct attention away from levers for change that address underlying causes (Ming & Goldenberg, 2021). Correlational relationships also tend to be substantially larger than causal effects, and failing to distinguish between correlational and causal evidence may result in policymakers dismissing programs with “small” effects that are comparatively large relative to alternatives (Kraft, 2020).

The analysis plan should consider potential limitations and be transparent about them. If certain demographics tend to be underrepresented in the data collected, researchers might use purposeful sampling to guide data gathering (Ming & Goldenberg, 2021). For example, for a qualitative analysis of testimony from school board meetings, researchers might wish to consider how to collect supplementary data if individuals testifying are not reflective of the demographics of families served by the district. An analysis plan could specify when further outreach or alternative data sources are warranted to ensure adequate representation from various subgroups. The leadership council might also consider adherence to the Standards for Excellence in Education Research principles of making findings, methods, and data open to improve transparency to the public.

Findings That Address Stakeholder Concerns and Incorporate Insights

The ideal situation for interpreting evidence is one in which community and practitioner expertise on context and implementation is pooled with researcher and analyst expertise (Contandriopoulos et al., 2010). Stakeholders’ feedback on initial findings helps clarify additional steps needed to transform evidence into “knowledge in practice” (Farley-Ripple et al., 2018, p. 239). Practitioners, community members, and students may have insights on the nuances of how data were collected or how programs were implemented that improve interpretation of the findings. Enacting this process requires adequate time for individuals in different roles to interact in substantive ways with the evidence (Coburn et al., 2009). Enabling engagement with evidence by diverse groups of constituents may require substantial investments in brokerage activities that facilitate interpretation of the evidence (Landry et al., 2003). Discussing evidence may help stakeholders understand challenges and design and improve solutions (Booker et al., 2019).

Inviting stakeholders to weigh in on the interpretation of evidence in priority policy areas is risky for education leaders, given that findings may suggest favored policies and practices are less effective than desired (Conaway et al., 2015). In the partnership described by Nelson et al. (2015), community agencies retained control over the release of findings, which helped the research partners build trust with their community partners. While agency control over the release of findings alleviates agency concerns, it may be antithetical to a public accountability framework if other stakeholders’ concerns go unaddressed. For example, people have been silenced when projects “veered too close to unearthing unflattering realities of racialized discipline practices in schools” (Diamond, 2021). Striking a compromise might involve an agreement that requires that answers to questions are publicly provided, but grants the agency involved some amount of time to review findings before they are released.

A key challenge in evidence interpretation is that practitioners may assume that evidence will either validate or help them identify ways to improve; negative findings may be ignored, dismissed, or explained away (Booker et al., 2019). The tendency to explain away findings is consistent with confirmation bias, in which people tend to interpret evidence in ways that align with existing beliefs or expectations (Nickerson, 1998). Additionally, decision-making experiments indicate that individuals disproportionately favor the status quo (Samuelson & Zeckhauser, 1988), which could lead to unduly discounting evidence that contradicts current practices. To counter such biases, education leaders could commit in advance to the criteria that merit reconsidering their previously held positions and communicate how they intend to respond to positive, negative, or inconclusive evidence (Gordon & Conaway, 2020). Prior to conducting the research, working groups might engage in “if . . . then” discussions to specify the actions warranted by different potential findings, following the approach described in Nelson et al. (2015).

Implications for Advocates of Democratizing the Development of Evidence

The democratic principles that underlie the logic model, paired with a focus on actionable items that benefit the public (Hutt & Polikoff, 2020), set the stage for broader stakeholder engagement in establishing and enacting a learning agenda, developing evidence, and interpreting the evidence produced. Diverse stakeholder groups can provide input on what questions should be asked to form a co-constructed learning agenda. Education leaders can foster an inclusive approach at public meetings to encourage stakeholder feedback on challenges and potential solutions. As the evidence base develops, structures and processes that offer opportunities for a variety of stakeholders to engage in interpreting the evidence can shed light on what additional evidence might be needed and what concerns are left to address. The online appendix (available on the journal website) offers questions to consider for each input in the process.

Given the aspirational nature of the democratic principles, and the fact that many education agencies lack the key inputs described in the logic model, it is worth considering what is sufficient to begin working toward the democratization of evidence. It might be most feasible to begin by asking active members of the educational community what questions they would like answered, and examining existing evidence as opposed to building a local evidence base. Publicly available resources, such as Regional Educational Laboratories toolkits and What Works Clearinghouse Practice Guides, may help agencies build capacity to deliberate about evidence. At a minimum, such efforts would require dedicated time for individuals to engage in conversations.

Advocates for democratizing the development of evidence should consider ways of making the development and interpretation of evidence more inclusive. Leaders of state and district agencies could seek input from the teachers, families, students, and communities they serve. Although the process of developing and enacting a community-informed learning agenda likely involves a long time frame and will not be the right approach for every question, the process is intended to build trust and ultimately generate an evidence base that is more relevant and compelling to stakeholders.

A key implication for researchers is to appreciate that methodological expertise alone is insufficient, and that practitioners’ and families’ experiences with the education system and knowledge of local contexts constitute valuable forms of knowledge that can be applied to identifying plausible explanations for findings and interpretations of evidence (Tseng et al., 2018). Researchers also might consider asking practitioners, policymakers, or community stakeholders how they interpret evidence in light of specific political and organizational contexts. Ongoing conversations can encourage depth of evidence use through more robust evidence searches and improved interpretation of evidence (Farley-Ripple et al., 2018).

Supplemental Material

sj-pdf-1-edr-10.3102_0013189X211060357 – Supplemental material for Democratizing the Development of Evidence

Supplemental material, sj-pdf-1-edr-10.3102_0013189X211060357 for Democratizing the Development of Evidence by Cara Jackson in Educational Researcher

Footnotes

ORCID iD

Cara Jackson

Notes

Author

CARA JACKSON, PhD, is a senior associate at Abt Associates, 6130 Executive Boulevard, Rockville, MD 20852; cara_jackson@abtassoc.com . Her research focuses on the implementation and impact of programs intended to improve outcomes for youth, and on evidence use in educational agencies.

References

Asen

Gurke

Conners

Solomon

Gumm

(2012). Research evidence and school board deliberations: Lessons from three Wisconsin school districts. Educational Policy, 27(1), 33–63. https://doi.org/10.1177/0895904811429291

Booker

Conaway

Schwartz

(2019). Five ways RPPs can fail and how to avoid them: Applying conceptual frameworks to improve RPPs. William

Grant Foundation. http://wtgrantfoundation.org/five-ways-rpps-can-fail-and-how-to-avoid-them-applying-conceptual-frameworks-to-improve-rpps

Boulay

Goodson

Olsen

McCormick

Darrow

Frye

Gan

Harvill

Sarna

(2018). The investing in innovation fund: Summary of 67 Evaluations. https://ies.ed.gov/ncee/pubs/20184013/pdf/20184013.pdf

Brighouse

Ladd

H. F.

Loeb

Swift

(2018). Educational goods: Values, evidence, and decision-making. University of Chicago Press. https://press.uchicago.edu/ucp/books/book/chicago/E/bo27256234.html

Castellano

K. E.

A. D.

(2013). A practitioner’s guide to growth models. https://scholar.harvard.edu/files/andrewho/files/a_pracitioners_guide_to_growth_models.pdf

Coburn

C. E.

Toure

Yamashita

(2009). Evidence, interpretation, and persuasion: Instructional decision making at the district central office. Teachers College Record, 111(4), 1115–1161.

Collins

J. E.

(2019). Local democracy matters: How deliberative culture shapes public evaluations of local government and local government performance (EdWorkingPaper No. 19-114). Annenberg Institute at Brown University. https://edworkingpapers.com/ai19-114

Collins

J. E.

(2021). Does the meeting style matter? The effects of exposure to participatory and deliberative school board meetings. American Political Science Review, 115(3), 790–804. https://doi.org/10.1017/S0003055421000320

Conaway

(2019). Maximizing research use in the world we actually live in: Relationships, organizations, and interpretation. Education Finance and Policy, 15(1), 1–10. https://doi.org/10.1162/edfp_a_00299

10.

Conaway

Keesler

Schwartz

(2015). What research do state education agencies really need? The promise and limitations of state longitudinal data systems. Educational Evaluation and Policy Analysis, 37(1 Suppl.), 16S–28S. https://doi.org/10.3102/0162373715576073

11.

Contandriopoulos

Lemire

Denis

J.-L.

Tremblay

. (2010). Knowledge exchange processes in organizations and policy arenas: A narrative systematic review of the literature. Milbank Quarterly, 88(4), 444–483. https://doi.org/10.1111/j.1468-0009.2010.00608.x

12.

Corcoran

Fuhrman

S. H.

Belcher

C. L.

(2001). The district role in instructional improvement. Phi Delta Kappan, 83(1), 78–84. https://doi.org/10.1177/003172170108300116

13.

Diamond

J. B.

(2021). Racial equity and research practice partnerships 2.0: A critical reflection. William

Grant Foundation. http://wtgrantfoundation.org/racial-equity-and-research-practice-partnerships-2-0-a-critical-reflection

14.

Ercikan

Roth

W.-M.

(2006). What good is polarizing research into qualitative and quantitative? Educational Researcher, 35(5), 14–23. https://doi.org/10.3102/0013189X035005014

15.

Farley-Ripple

May

Karpyn

Tilley

McDonough

(2018). Rethinking connections between research and practice in education: A conceptual framework. Educational Researcher, 47(4), 235–245. https://doi.org/10.3102/0013189X18761042

16.

Finnigan

K. S.

Daly

A. J.

Che

(2013). Systemwide reform in districts under pressure: The role of social networks in defining, acquiring, using, and diffusing research evidence. Journal of Educational Administration, 51(4), 476–497. https://doi.org/10.1108/09578231311325668

17.

Gordon

Conaway

(2020). Common-sense evidence: The education leader’s guide to using data and research. Harvard Education Press. https://www.hepg.org/hep-home/books/common-sense-evidence

18.

Hopkins

Weddle

Gluckman

Gautsch

(2019). Boundary crossing in a professional association: The dynamics of research use among state leaders and researchers in a research-practice partnership. AERA Open, 5(4). https://doi.org/10.1177/2332858419891964

19.

Hutt

Polikoff

M. S.

(2020). Toward a framework for public accountability in education reform. Educational Researcher, 49(7), 503–511. https://doi.org/10.3102/0013189X20931246

20.

Institute of Education Sciences. (2018). Institute of Education Sciences Director’s Biennial Report to Congress, Fiscal Years 2017 and 2018. https://eric.ed.gov/?q=Institute+of+Education+Sciences+Director%e2%80%99s+Biennial+Report+to+Congress+Fiscal+Years+2017+and+2018&id=ED598251

21.

Jackson

Harrison

Pullen

K. R.

Lowenstein

(2021, May 24). What education researchers can learn from teachers. Education Week. https://www.edweek.org/teaching-learning/opinion-what-education-researchers-can-learn-from-teachers/2021/05

22.

Kirkland

D. E.

(2019, December 16). No small matters: Reimagining the use of research evidence from a racial justice perspective. The Digest. http://wtgrantfoundation.org/digest/no-small-matters-reimagining-the-use-of-research-evidence-from-a-racial-justice-perspective

23.

Kraft

M. A.

(2020). Interpreting effect sizes of education interventions. Educational Researcher, 49(4), 241–253. https://doi.org/10.3102/0013189X20912798

24.

Landry

Lamari

Amara

(2003). The extent and determinants of the utilization of university research in government agencies. Public Administration Review, 63(2), 192–205. https://doi.org/10.1111/1540-6210.00279

25.

Manski

C. F.

(2013). Public policy in an uncertain world: Analysis and decisions. Harvard University Press. https://www.hup.harvard.edu/catalog.php?isbn=9780674066892

26.

Massell

Goertz

M. E.

Barnes

C. A.

(2012). State education agencies’ acquisition and use of research knowledge for school improvement. Peabody Journal of Education, 87(5), 609–626. https://doi.org/10.1080/0161956X.2012.723506

27.

Michener

(2020, October 14). Panel discussion: Critical race perspectives on the use of research evidence: Research evidence for an equitable democracy (Jamila Michener slides). http://wtgrantfoundation.org/library/uploads/2020/10/Michener.pdf

28.

Ming

N. C.

Goldenberg

L. B.

(2021). Research worth using: (Re)framing research evidence quality for educational policymaking and practice. Review of Research in Education, 45(1), 129–169. https://doi.org/10.3102/0091732X21990620

29.

National Network of Education Research-Practice Partnerships. (n.d.). National Network of Education Research-Practice Partnerships. https://nnerpp.rice.edu/

30.

Neal

J. W.

Neal

Z. P.

Kornbluh

Mills

K. J.

Lawlor

J. A.

(2015). Brokering the research–practice gap: A typology. American Journal of Community Psychology, 56(3–4), 422–435. https://doi.org/10.1007/s10464-015-9745-8

31.

Nelson

I. A.

London

R. A.

Strobel

K. R.

(2015). Reinventing the role of the university researcher. Educational Researcher, 44(1), 17–26. https://doi.org/10.3102/0013189X15570387

32.

Nickerson

R. S.

(1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), 175–220. https://doi.org/10.1037/1089-2680.2.2.175

33.

Nielsen

P. A.

Moynihan

D. P.

(2017). How do politicians attribute bureaucratic responsibility for performance? Negativity bias and interest group advocacy. Journal of Public Administration Research and Theory, 27(2), 269–283. https://doi.org/10.1093/jopart/muw060

34.

Nutley

Powell

Davies

(2013). What counts as good evidence? Provocation paper for the alliance for useful evidence. ALNAP. https://www.alnap.org/help-library/what-counts-as-good-evidence-provocation-paper-for-the-alliance-for-useful-evidence

35.

Office of Evaluation Sciences. (n.d.). Evidence Act toolkit: A guide to developing your agency’s learning agenda. General Services Administration. https://oes.gsa.gov/assets/toolkits/A_Guide_to_Developing_your_Agency's_Learning_Agenda.pdf

36.

Penuel

W. R.

Farrell

C. C.

(2017). Research-practice partnerships and ESSA: A learning agenda for the coming decade. In Quintero

(Ed.), Teaching in context: The social side of education reform (pp. 181–200). Harvard Education Press.

37.

Roza

(2021, July 20). Will the American rescue plan’s “meaningful consultation” requirement usher in community participation in school budgets? Forbes. https://www.forbes.com/sites/margueriteroza/2021/07/20/will-the-american-rescue-plans-meaningful-consultation-requirement-usher-in-community-participation-in-school-budgets/

38.

Samuelson

Zeckhauser

(1988). Status quo bias in decision making. Journal of Risk and Uncertainty, 1(1), 7–59. https://doi.org/10.1007/BF00055564

39.

Tseng

Easton

J. Q.

Supplee

L. H.

(2017). Research-practice partnerships: Building two-way streets of engagement. Social Policy Report: Society for Research in Child Development, 30(4). https://files.eric.ed.gov/fulltext/ED581655.pdf

40.

Shafer

(2016, January 4). When proficient isn’t good: The deceptive nature of proficiency as a measure of student progress–and how to fix the funhouse mirror. Harvard Graduate School of Education. https://www.gse.harvard.edu/news/uk/15/12/when-proficient-isnt-good

41.

Tseng

Fleischman

Quintero

(2018). Democratizing evidence in education. In Bevan

Penuel

W. R.

(Eds.), Connecting research and practice for educational improvement (pp. 3–16). Routledge. http://wtgrantfoundation.org/democratizing-evidence-in-education-from-connecting-research-and-practice-for-educational-improvement

42.

U.S. Department of Education. (2016). Non-regulatory guidance: Using evidence to strengthen education investments. https://www2.ed.gov/policy/elsec/leg/essa/guidanceuseseinvestment.pdf

43.

Weiss

C. H.

(1980). Knowledge creep and decision accretion. Knowledge, 1(3), 381–404. https://doi.org/10.1177/107554708000100303

44.

Wentworth

Conaway

Shewchuk

Arce-Trigatti

(2021). RPP brokers handbook: A guide to brokering in education research-practice partnerships. National Network of Education Research-Practice Partnerships. https://nnerpp.rice.edu/rpp-brokers-handbook/

45.

William

Grant Foundation. (n.d.). Research grants on improving the use of research evidence. http://wtgrantfoundation.org/grants/research-grants-improving-use-research-evidence

46.

Wilson

C. M.

(2021). Research-practice partnerships for racially just school communities. William

Grant Foundation. http://wtgrantfoundation.org/research-practice-partnerships-for-racially-just-school-communities

47.

Yohalem

Tseng

(2015). Commentary: Moving from practice to research, and back. Applied Developmental Science, 19(2), 117–120. https://doi.org/10.1080/10888691.2014.983033

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.16 MB