Research-Practice

Abstract

Recent attention on partnerships between researchers and practitioners highlights the potential of these relationships to provide high-quality usable knowledge for improving schools. But how do we translate guiding partnership principles into specific actionable steps? How do we build and maintain an effective partnership? How do we reconcile and integrate multiple partnership frameworks to establish a coherent set of partnership activities? How do we evaluate partnership progress and outcomes? Building on the recent insightful work on partnerships, we offer a framework for planning, building, implementing, and monitoring partnerships, based on the literature and our experiences in a partnership between a university-based school of education at a major research university and the research office of a big-city school district. Using a theory describing attributes that define a policy’s strength, we propose an organizing framework to transform insights about partnerships into concrete activities and mechanisms to help achieve the potential of these partnerships to use research to improve schooling.

recent attention to partnerships between researchers and practitioners emphasizes their potential to provide high-quality usable knowledge for improving schools (Salmonowicz, 2009). Such partnerships are conceptually appealing because of their promise for addressing typical challenges in translating research to practice—building practitioner knowledge of research and belief in its value and ensuring that the research is relevant, timely, and actionable. Operationally, partnerships can bring systematicity and efficiency to otherwise time-consuming endeavors, such as building relationships, developing data-sharing agreements, and creating mechanisms for discussing, understanding, and using results. Integrating these activities into the infrastructure of a partnership increases the efficiency of both organizations. Furthermore, partnership work can help districts respond to increasing pressure for evidence-based decisions and can increase the immediate influence of research findings on practice, a link usually limited in the traditional paradigm.

Scholarship on partnerships emphasizes that “mutualistic” work (Coburn, Penuel, & Geil, 2013, p. 2) and trusting relationships (Bryk & Schneider, 2002) are essential for building effective sustainable partnerships. This literature also identifies discrete elements important for partnerships, including leveraging district expertise, using district priorities for agenda setting, making data accessible, and allowing sufficient discussion time throughout the work (Coburn et al., 2013; Tseng, 2012). While this burgeoning body of work has elevated the promise of research-practice partnerships, it also points to several critical questions: How do we translate partnership-guiding principles into specific actionable steps? How do we build and maintain an effective partnership? How do we establish and sustain a coherent set of partnership activities? What are the challenges to implementing these partnerships, and what are systematic ways of anticipating and addressing these challenges? How do we evaluate partnership progress and outcomes?

Building on recent work, we seek to bridge conceptual ideas about partnerships with the field’s practical knowledge on building and maintaining them. In doing so, we offer a framework for developing, implementing, and monitoring partnerships, based on the literature and our experiences in our partnership between a school of education at a major research university—the Graduate School of Education at the University of Pennsylvania—and the research office of a big-city school district, the School District of Philadelphia. Using theory about the attributes that describe the strength of a policy, we propose an organizing framework to transform insights about partnerships into concrete activities and mechanisms to help achieve ideals of researcher-practitioner work.

Shared Solutions: A District-University Research Partnership

The School District of Philadelphia Office of Research and Evaluation, hereafter referred to as the SDP, and the Graduate School of Education at the University of Pennsylvania, hereafter referred to as Penn GSE, established a researcher-practitioner partnership in 2014 called Shared Solutions through funding from the Institute of Education Sciences. While the two organizations had collaborated for decades on specific studies, professional development, and program design, Shared Solutions was established to formalize their relationship and bring infrastructure, coherence, and sustainability to their collaborative efforts.

Serving 206,567 preK–12 students, the SDP is the eighth-largest school system in the country. The district enrolls a diverse population of students: 9.4% of students are classified as English-language learners and 13.8% as having disabilities, and 52% are African American, 19% Hispanic/Latino, 14% Caucasian/Euro-Americans, 8% Asian, and 4% identify as multiracial or other (SDP, 2014a, 2014b). While the district has been criticized for its shortcomings, including financial shortfalls and poor student performance (SDP, 2015), it has also been lauded nationally for its several exemplary schools (U.S. News and World Report, 2015).

Penn GSE is a relatively small school, with 39 tenure or tenure-line faculty, 48 other associated faculty, and approximately 1,300 graduate students. Penn GSE faculty are multidisciplinary, and much of their work focuses on topics of direct relevance to the district, such as reform implementation (Desimone, 2002; Supovitz & Weinbaum, 2008), teachers’ professional development (e.g., Desimone & Garet, 2015; Desimone, Porter, Garet, Yoon, & Birman, 2002; Desimone, Smith & Phillips, 2013), school leadership (Porter, Murphy, Goldring, Elliott, & Cravens, 2012), curriculum use (Remillard, Herbel-Eisenmann, & Lloyd, 2011), teacher supply and retention in high need areas (Ingersoll & May, 2012), and college preparation and access for minority students (Perna, Harkavy, & Bowman, 2012).

Roderick, Easton, and Bender Sebring (2009) indicated that models prevalent at the time of their seminal studies of research-practice partnerships did not create sustained relationships that built broader capacity. Taking this finding to heart, Shared Solutions integrates practitioner and researcher involvement in all phases of the research, fostering joint ownership, understanding, and buy-in from conception to conclusion. Thus, the research questions, design, instruments, analyses, and interpretations all benefit from the contributions of practitioners and researchers.

Policy Attributes Theory

Grounded in lessons from the field (e.g., Bryk, Sebring, Allensworth, Easton, & Luppescu, 2010; Coburn & Stein, 2010), Shared Solutions is building the capacity of both organizations to develop a dynamic real-time knowledge base grounded in rigorous principles of research. Our experience in building this partnership has provided us with critical insights into how a university and district might most efficiently and effectively build a partnership, distill lessons from the field, and bridge the research-to-practice divide. We use a theory of policy attributes to offer an integrative framework for building and maintaining district-university partnerships that have the potential to fulfill the vision—outlined by recent thought leaders—of truly mutualistic synergistic partnerships (e.g., Coburn et al., 2013).

The policy attributes theory, created by Porter and his colleagues, is a framework that posits the mechanisms through which policies gain influence over practice (Clune, 1998; Porter, Floden, Freeman, Schmidt, & Schwille, 1988). The theory of policy attributes suggests that policies with high levels of authority, consistency, specificity, power, and stability are more likely to be well implemented and enduring (Desimone, 2002; Polikoff, 2012). Specificity is the extent of detailed guidance and actionable steps embedded in a policy or program. Authority represents the backing of key leadership, support in the form of resources and time, and the understanding and buy-in of critical stakeholders. Consistency reflects the policy’s alignment and coherence with other activities and reforms. Power reflects rewards and incentives, as well as penalties and sanctions, tied to compliance. Stability includes how actors view the longevity of a policy, the mechanisms designed to institutionalize the policy, and the mobility of people within the system. The framework is consistent with the broader literature on systemic reform and policy implementation (e.g., Cohen, 1990; Cohen & Ball, 1990; McLaughlin, 1987, 1990; Smith & O’Day, 1991). Applying the partnership literature and our own experiences, we describe how these policy attributes can serve as a roadmap for building, monitoring, evaluating, and sustaining partnerships.

Specificity

Specificity refers to how extensive and detailed a policy is, the explicitness of the activities and procedures needed to implement the policy (Desimone, 2002; Porter, 1989). Examples of ways to make a policy more explicit include setting clear goals and delineating activities and procedures (e.g., Berends, Bodilly, & Kirby, 2002; Rosen, 2010).

In developing and executing a researcher-practitioner partnership, questions of specificity are often the first to arise. For example, on what will the partnership focus? How will evolving priorities be decided? Through what activities and events will the partnership conduct its work? How will the partnership ensure participation from each organization? We have identified three mechanisms that help translate general partnership goals into specific, clearly articulated actions: (a) establishing a clear vision and path for development, (b) providing mechanisms for regular communication, and (c) defining the scope of work.

Setting a clear vision and path for development

The partnership was initiated by the deputy chief of research and evaluation at the district. She had a vision for a more integrated, coordinated set of research activities emanating from —— Penn GSE, which is responsible for about 60% of research in the district. For the district, the logistic and organizational challenges of approving and executing Penn GSE-initiated research was a major impetus for the partnership. The district also envisioned a time when its action plan and priorities would be thoroughly integrated into every research study that it approved and when it would work collaboratively on all major research initiatives occurring in the district.

This vision of coordinated research that benefits from the expertise of both institutions was shared by Penn GSE and district leadership, but neither organization had preconceived notions about how to achieve this vision. We consider this lack of initial specificity to be one of the key levers of success for our work together. The combination of a shared broad vision with no preconceived agenda for how to get there allowed us to immediately begin to jointly invent the mechanisms to achieve the vision.

Together, we created the building blocks of the partnership—a research agenda and conceptual framework. We jointly articulated a research agenda that focused on one of the district’s centerpiece reforms: its turnaround school efforts. Together, we articulated a set of objectives designed to better understand turnaround schools, what their components were, how they differed from one another, and what seemed to be working and what was a challenge. We wrote an analytic plan that included identifying preliminary achievement trends, studying the quality of implementation, and identifying strengths and weaknesses in the turnaround models, using a combination of interrupted time series, cohort, and implementation analyses. This written detailed research plan served as the foundation and guiding principles for other partnership activities. We then adopted the policy attributes as the conceptual framework for the study. We derived a theory of action based on the work of Bryk and colleagues (2010), who identified five essential supports for successful schools—leadership, parent/guardian-community ties, professional capacity, climate, and instruction. All researcher-practitioner discussions fostered by the partnership were grounded in this jointly adopted framework and theory of action.

Embedding our partnership work in a research-based conceptual framework ensured that studies were grounded in the current literature and so, in addition to being useful to the district, had potential to contribute to broader scholarship. Furthermore, the framework was a powerful mechanism to build shared understanding (e.g., Hubbard, 2010) and to guide our instrument development, analysis, and interpretation.

Mechanisms for communication: Working groups, meetings, and miniconferences

We established several cross-institution working groups to ensure that the core work of the partnership was being accomplished (Research Partnership for New York City Schools, 2007). Each group includes members of Shared Solutions’ leadership team as well as Penn GSE graduate students and district staff. The small group size allows for in-depth discussion and collaboration, and overlapping group membership ensures communication and cross-fertilization. Every group has a specific scope of work, such as survey development or communications, but we allow for evolution and adaptation of roles and responsibilities to capitalize on individual expertise (Engle, 2010).

Another way that we created defined actionable steps was by holding regularly scheduled meetings, essential to establishing a productive working relationship (Stanton & Easton, 2002) and providing a forum for dialogue and debate (Goldring & Sims, 2005). Partnership leadership also met regularly with district and Penn GSE leadership to build strong ties to ensure that the work of the partnership was used in decisions (Coburn & Stein, 2010). These ties also helped the partnership be seen as a resource that engages with educators, rather than as a mechanism for outside evaluation (Roderick & Easton, 2007).

Hosting conferences is another powerful mechanism for shaping the specificity of a partnership. We hosted a series of 3-hour conferences over a 2-year period to work through the process of validating our conceptual framework, developing and validating surveys to study school improvement, sharing findings, and working with practitioners and researchers to use findings for school improvement. We invited principals and teachers from the approximately 40 charter and traditional public schools that were in our school improvement study, as well as researchers who studied the essential supports (e.g., climate, instruction), but events were open to all community members. We involved as many key district decision makers as possible, capitalizing on the important role that they play in shaping how research is used in decision making (Coburn, Toure, & Yamashita, 2009).

Through these miniconferences, Shared Solutions provides ongoing mechanisms to engage with practitioners in interpreting and using research findings for school improvement, opportunities that are typically absent (Tseng, 2012). Consistent with Spillane, Reiser, and Reimer (2002), we do not expect change based on simply providing information; rather, our miniconferences provide opportunities for discussion at various points in time, which has the potential to foster productive use of information (Coburn et al., 2013; Penuel, Fishman, Cheng, & Sabelli, 2011). Additionally, discussions at these forums help to ensure that interpretations are sensitive to variation in school context (e.g., Gomez et al., 1997).

In effect, these activities address Tseng’s (2012) emphasis on researchers’ efforts to “push out” research to practitioners and practitioners’ efforts to “pull in” research to inform their practice (Ikemoto & Honig, 2010). It is in the space of a true partnership that we can enact a new researcher-practitioner push-pull model in which the “push” of researchers and the “pull” of practitioners is given equal attention.

While we had hundreds of attendees in aggregate, this was still only a small percentage of district and school staff. Finding effective ways to reach more practitioners is an ongoing challenge for the partnership.

Defining the scope of work: Is it evaluation or research?

Traditionally, there has been a tension between what academics consider “real” research and the work that districts conduct to evaluate their programs (Perla & Carifio, 2009). This tension reflects a conflict between studies that scholars need in order to prosper in the profession—those that advance the field, produce generalizable principles, or are linked to theoretical frameworks—and the type of inquiry that a district needs: those evaluating programs and identifying areas for improvement to inform resource allocation or policy changes. Furthermore, academia privileges complex data analysis, which is usually not what the district wants or needs.

However, the field seems to be moving toward a new paradigm that integrates process-driven evaluations and so-called academic research. Implementation studies are being reconceptualized as an essential component of causal inquiry. Instead of relying on conclusions focused narrowly on results, a new paradigm is emerging that highlights the importance of understanding how results may differ across students and contexts (Mass Insight Education, 2012; What Works Clearinghouse, 2008), how they are mediated by the quality of implementation and the characteristics of implementers (Hulleman & Cordray, 2009; Imai, Keele, Tingley, & Yamamoto, 2011), and how they are facilitated or diminished by organizational, structural, and cultural factors (Vernez, Karam, Mariano, & DeMartini, 2006).

This increasing recognition of implementation variation, mediation, and moderation as central areas of scholarly inquiry blurs the distinctions between research and evaluation and in turn greatly facilitates partnership work. In this new paradigm, district interest in monitoring and improving programs is no longer necessarily in tension with scholarly inquiry. For example, implementation studies provide data that districts can use to focus their improvement efforts, while producing insights that contribute to a growing body of knowledge about the contextual, mediating, and moderating factors related to program success or failure.

Challenges that remain, however, include the possibility that sharing formative data will interfere with the results of summative evaluations (e.g., schools may change course based on formative results). Furthermore, districts have to contend with the tension between wanting preliminary results and knowing that the results may change after undergoing a thorough review. Another challenge is the perception by some that the work is less objective if done through a partnership; this might be addressed in part by having the work reviewed by stakeholders with different perspectives.

Despite the challenges, we believe that evaluating district-specific programs, a central component of partnership work, can fulfill the goals and needs of both university and district partners.

Integrative research and practice in action

Two examples exemplify the blending of research and practice. In an effort to increase parent/guardian awareness of absenteeism, the district partnered with researchers to conduct a randomized controlled trial to test whether postcards sent to parents alerting them to the number of days that their child was absent from school could help reduce absenteeism. The idea was based on the principles of “nudge” theory, an approach used in the behavioral sciences that involves using unobtrusive interventions to promote desired behaviors (Thaler & Sunstein, 2008). Researchers worked closely with the head of the district attendance office—he participated in every decision about how to execute the study. Because the researchers and attendance office were in constant communication and fully collaborating on the study, the ultimate message on the postcards and decisions about the distribution protocol were shaped to respond to the needs of the randomized controlled trial and the district. When results indicated that the postcards did help to improve attendance (Rogers & Feller, 2016), the district was able to immediately institutionalize the intervention by adding regular mailings of the postcards to its family communication strategy. Thus, the district did not have to adopt an intervention that it had no experience with, which would require learning about it and developing materials, procedures, and buy-in—all of which take considerable time and are usually met with limited success (e.g., Berends et al., 2002; Bryk et al., 2010). Instead, it could seamlessly implement the work that it had already done in collaboration with researchers—work that they knew, understood, believed in, and had the internal capacity to execute. We see this as an example of how collaborative research that is theory based and uses rigorous methodology can lead to real-time immediate changes in practice.

Another example is the district’s adoption of a new framework for turnaround schools, based on work of Shared Solutions. The superintendent of the district recently announced that the district’s new turnaround strategy would be grounded in evidence-based strategies, specifically naming the essential supports that Shared Solutions had been promoting in its work (see https://webapps.philasd.org/news/display/articles/2261). The deputy of research meets with district leadership on a regular basis, ensuring that it has ongoing knowledge of Shared Solutions’ work. Shared Solutions and district improvement initiatives were unfolding at the same time and were able to create a synergy and inform each other because of the collaborative, integrative nature of the work. We see this as a powerful example of blending research and practice: When both are occurring simultaneously by the same people, day-to-day decision making can be informed by and infused with research ideas and findings in a way that just is not possible with externally conducted research. Similarly, the research designs can be more nimbly adapted to the changing district landscape because there is no delay in communication. These examples show how scholarly inquiry and evaluative work can be merged to serve the needs of both research and practice.

Additionally, this new paradigm demonstrates the promise of partnership work that takes on broader, more “academic” questions: How do teacher qualifications relate to classroom instruction? and What teacher and principal behaviors foster greater community involvement? This work may not be linked to specific interventions, but it can inform the creation or refinement of initiatives while contributing to the broader knowledge base. Thus, we believe that partnerships should have the freedom to develop a research agenda that reflects the need to understand if and how a particular program works, as well to ask questions designed to contribute to the broader field of education research.

Authority

Authority operates through persuasion, stakeholder participation, and buy-in (Coburn & Stein, 2010; Datnow & Stringfield, 2000). A policy can establish authority through multiple mechanisms, including law, social norms, resources, support from experts, or promotion by charismatic leaders (Porter, 1994; Porter et al., 1988). Leadership, or institutional authority, is critical for implementation and can be gained through priority setting (McLaughlin, 1987) and providing appropriate time and resources for implementation (e.g., Berends et al., 2002; Datnow & Stringfield, 2000). Community support also plays an important role in establishing a reform’s authority (Berends et al., 2002). We discuss several important paths for developing authority, emphasizing that authority through leadership and practitioner support and resources has been instrumental to the work of Shared Solutions.

Leadership authority

The explicit backing and support of leadership is essential to the success of reform efforts (Leithwood & Jantzi, 2008); it is similarly essential in partnership efforts. Shared Solutions has the support and enthusiasm of the highest levels of each organization—the former and current deans of Penn GSE and the superintendent of the district. The support of these high-level leaders has been essential—signifying the partnership’s value by acknowledging its work at faculty and district meetings, connecting partnership work with other efforts and with internal and external resources, and allowing the partnership directors to prioritize the work.

Furthermore, the authority of the partnership work was enthusiastically and immediately accepted by the Shared Solutions team. Partnership participants recognized the potential benefits of collaboration, and this strong authoritative grounding allowed us to move quickly to develop the specific activities of the scope of work rather than engage in a long process of having to build buy-in and trust.

A continuing challenge, however, is diffusing this trust and understanding to others in the district and Penn GSE. While our individual relationships have been instrumental in the partnership’s initial success, key players at multiple levels need to know, understand, and believe in the partnership work. Fostering widespread buy-in and support among those with different priorities, goals, values, and personal interests is an ongoing challenge. Our efforts to establish diffuse buy-in interact with issues related to power—the incentives and penalties associated with partnership work—which we discuss later.

Practitioner authority

The ever-present top-down/bottom-up debate in education leads us to emphasize not only the importance of university and district leadership authority but also practitioner authority. Since multiple actors at different levels in the education system make decisions about instruction (e.g., Coburn et al., 2009; Spillane, 1998) and often have competing views of best practices (Neufeld, 2007), we include actors from multiple levels in our activities to provide opportunities for them to engage with one another on an ongoing basis (Coburn, 2010). Involving teachers as well as school leaders is critical to ensuring that the partnership work is directly applicable to the realities of schools and classrooms (Coburn, 2001; Engle, 2010).

By assisting the district in developing a culture of research-based practices, Shared Solutions learned that most practitioners would be willing—even enthusiastic—participants in data collection if they perceived the findings to be useful for improving their practice and if they had a say in what type of information was collected.

To address these issues, we held a series of focus groups with teachers and principals throughout our instrument-development process, thereby providing opportunities for practitioners to suggest refinements and additions. For example, principals critiqued our list of common challenges that they face, suggesting that we include the task of finding substitute teachers on short notice. This is a widespread problem in the SDP but is usually not included on surveys of leadership and climate. Including practitioner feedback in shaping the survey not only made it more relevant but also served as a powerful mechanism for fostering participant buy-in. We also include practitioners in discussions about how to present and analyze data, and we host workshops designed to provide tangible strategies for using the data collected by the partnership to make specific improvement decisions.

One major product of our work is a public-facing searchable website (http://schoolsurveys.philasd.org/) that provides districtwide survey data in an accessible format. The website provides school-specific reports that show, numerically and graphically, how each school is doing on the five essential supports (see Figure 1); the database additionally provides item-level data from the survey, allowing practitioners and researchers to explore what specific actions are driving scores in the five essential supports. The database allows users to make comparisons across schools, respondent types, and other key factors. The database contributes to developing practitioner authority through providing readily accessible data based on practitioners’ survey responses and its potential usefulness in school improvement.

Figure 1.

2014–2015 School District of Philadelphia districtwide survey feedback: Example.

We believe that our focus on practitioner authority contributed to a dramatic increase in response rates on the district’s annual districtwide surveys. Response rates skyrocketed, from previous rates hovering <3% to a high of 64% for principals, 53% for teachers, 33% for students, and 7% for parents/guardians. These are districtwide averages; many schools had teacher and student response rates >90% and parent/guardian response rates >60%. The district views the first-year response rates as a strong foundation from which to build. At the same time, participation varied substantially across schools. In response to this variation, the district is asking assistant superintendents to communicate the importance of the survey to principals in their network. Furthermore, the launch of the survey database in 2015 was a powerful indicator that the results are available for practitioners, which addresses a common complaint and reason for nonparticipation. Additionally, the district decided to use the climate and parent/guardian-community ties survey measures on school report cards. This decision is somewhat controversial, given the tension with using the surveys for diagnostics versus evaluation, but it will likely provide an impetus for principals to encourage their constituents to complete the survey.

Resources

Another critical form of authority is having resources to support policies and programs (Kowal, Hassel, & Hassel, 2009). Limited funding is often a motivator for seeking external partners (e.g., Lopez-Turley & Stevens, 2015), and consistent with this idea, a driving impetus for many district-university partnerships is the district’s need for external funding to support internal research. Partnerships that integrate funded research into their portfolio of activities not only bring much-needed resources into a district but also enjoy a certain level of authority that comes with the ability to provide resources for data collection and analysis and the cultivation of professional development to improve the odds that the findings are used.

In our case, the initial influx of a modest amount of grant funding served as powerful leverage, fueling us to organize and build infrastructure for our partnership. At the same time, the two organizations’ budget offices worked in very different ways, and it took time for them to establish ways to accomplish the required joint budgeting. Another resource-related issue is that more external grant funding is available for research activities than for the infrastructure building (e.g., refining databases), which is essential, especially for new partnerships. While the Spencer Foundation stands out as providing a line of support for infrastructure, the status quo in funding continues to prioritize and privilege the academy, often overlooking the real needs of the district partners. This is an area where leadership support can be influential. We had two Penn GSE deans during the first years of our partnership, and both provided institutional resources that would have been difficult to secure from external funding (e.g., technology support). Similarly, district partnership staff were able to contribute to the work beyond the time allotted in the grant, due to the consistency of the partnership’s mission with the district’s ongoing work.

Consistency

Consistency is the degree to which a policy is aligned with other school, district, state, or federal policies (Smith & O’Day, 1991). Such consistency helps to ensure that educators receive coherent policy messages instead of competing demands (Berends et al., 2002; Datnow & Stringfield, 2000; Spillane & Jennings, 1997). One of the potential key contributions of a research-practice partnership is bringing coordination and coherence to both organizations’ work. Partnerships can be an influential tool for building structured, formalized relationships that result in more organized, centralized, and thus more coherent work. To create consistency in our partnership work, we focused our efforts in three areas: aligning goals and benefits, coordinating research efforts, and coordinating roles.

Aligning goals and benefits

The literature suggests that a key to a strong partnership is having clear benefits for each partner (Coburn et al., 2013). Clearly articulating and communicating about these mutual goals and benefits is one way to establish consistency from the onset of the work. For example, the partnership helps fulfill Penn GSE’s scholarly mission by contributing to the literature on school improvement and student learning, and it additionally helps fulfill its mission to connect research and practice by providing opportunities for faculty and graduate students to work directly with district practitioners.

The district benefits from the partnership through building its capacity in terms of additional personnel and research expertise. Other benefits include bringing more streamlining and coordination to research processes, as discussed later. These complementary goals serve as the foundation for the alignment of partnership activities.

Coordinating research efforts

One tangible method for addressing consistency is improving the coherence of the research process in a district. A key challenge in most major urban districts is processing the substantial number of external research requests that they receive. For example, the district receives about 15 to 20 proposals per month.

The district wants to ensure that research conducted in the district addresses their priorities, produces actionable results, will not overburden any particular school, is not redundant with recent or ongoing projects, and is efficiently conducted (e.g., data collections streamlined across projects). A partnership can play an influential role in addressing these issues—in essence, bringing coherence and coordination to research efforts.

Shared Solutions addresses this coherence and coordination in part through the creation of a research project data archive. Such a central data hub can serve multiple purposes (Honig & Coburn, 2008) and is an important component of a research-practice alliance (Coburn et al., 2013). Shared Solutions is creating an archive of approved research projects so that they are available publicly, updated every 6 months, and searchable on key terms. The database will assist the district in providing a coherent mechanism for interfacing with research requests. Specifically, the SDP plans to require that researchers use the database to explain how their work complements ongoing research.

Another way of coordinating research efforts is to bring more research activities under the partnership umbrella. As might be expected, such coordination has been met with resistance when it comes in tension with typical academic norms of autonomy and intellectual freedom. For example, faculty have asked, “Will I have autonomy in deciding the research questions and design?” “Will I be the sole principal investigator on my project?” “Will I have sole control of the data?” “Will others be able to publish from the data?” These questions about ownership and propriety, intellectual and otherwise, are part of the evolving landscape of research-practice partnerships. These issues are related to the questions that we posed earlier about how broad a partnership should be—should it include all the research conducted by the partner organization or only certain projects; in which case, how would those projects be chosen? We do not claim to have answered these questions, but we are optimistic about working toward a more integrated and productive view of partnership research.

Data access is another key point of contention. Some partnerships may choose to have the university play a major role in controlling data access (e.g., Lopez-Turley & Stevens, 2015), but the SDP has a robust data analysis capacity and plans to continue controlling its own data. This is frustrating for some faculty, who see the partnership as a mechanism to increase the ease of their data access to pursue their own research agendas. Bridging these different visions of the role of a partnership requires continuing discussions. Thus, while the research agenda set forth under a partnership is by nature genuinely collaborative, decisions—such as how to integrate additional projects led by others, how data sharing should occur with faculty and students, and how independent or coordinated projects should be—are part of the ongoing challenge of partnership work.

Coordinating roles: That’s my job. No, that’s my job

Several assumptions pervade typical practitioner-researcher collaborations. Academics often assume that district staff do not have the technical knowledge to conduct rigorous research and that the district will have an internal bias in interpreting the results. In contrast, practitioners often assume that external researchers will miss important aspects of a program or policy because they lack internal contextual knowledge; they also question the relevance of the research, assuming that it will take several years before results are shared with the district. As a result, academic researchers often believe that it is their job to conduct studies of district policies and programs, with practitioners relegated to providing access and data. District researchers similarly believe that they themselves are best suited for the task of evaluating their district’s policies and programs and that it is, in fact, their job to do so.

Partnerships address these tensions head on. Jointly developed research projects capitalize on the complementary expertise of scholars and practitioners. Despite stereotypes about district staff having lower levels of expertise, the reality is that academics and practitioners have different types of expertise. Partnership work benefits from the insider and contextual knowledge of district officials, the broader academic knowledge of scholars, as well as the technical knowledge of researchers from both organizations. Furthermore, district officials often have additional expertise related to interpretation, displaying data in accessible ways, and recruitment and data collection methods that work well with their schools. For example, in our partnership, the district knew that the most effective strategy for communicating about the survey was first through regional “network” leaders, who in turn communicated with principals in their networks. Similarly, the Penn GSE researchers were primarily responsible for ensuring that the survey questions mapped to the policy attributes theory and research-based notions of successful schools (i.e., Bryk et al., 2010). We found that truly integrative joint partnership work acknowledged and capitalized on all participants’ expertise, valuing each institution’s expertise equally.

Power

Power includes the rewards and sanctions associated with the implementation of a policy (Calkins Guenther, Belfiore, & Lash, 2007; Clune, 1998). These can include explicit mechanisms, such as monetary incentives or accountability rating systems, or implicit levers, such as public praise from leadership. Pressure from such rewards or sanctions can be effective in eliciting change, particularly in the short run (Desimone, 2002; McLaughlin, 1987). Partnership work comes with its own set of rewards and penalties, which should be thoroughly considered and addressed.

Incentives to build the partnership

As noted, external funding was the catalyst for building the Shared Solutions infrastructure. While at any given time, Penn GSE has >20 individual research projects ongoing with the district, it took a specific funding source to begin building the formal infrastructure and capacity for a relationship that goes beyond a portfolio of unrelated projects. Similarly, the growing interest of funding agencies in partnerships seems to have generated the initiation of many partnerships around the country. These examples suggest that it can be extremely helpful for there to be some type of clear incentive structure for establishing these partnerships—funding through an external agency is one, although we believe that time and resources from each organization’s leaders (authority) could similarly transform informal relationships into more integrated and coherent formal partnerships.

Rewards and incentives for faculty

One reason why district-university partnerships may not thrive is that universities do not typically value practice-related work (Coburn et al., 2013) and the need and desire to publish in scholarly journals is not always consistent with partnership work (Lopez-Turley & Stevens, 2015). The move toward an integrative paradigm of practice-driven evaluation and academic research helps to shape the research so that it is both practical and publishable in scholarly outlets. In terms of university support, university leadership has several powerful levers for providing support and incentives for partnership work. At Penn GSE, the formal yearly faculty evaluation has a section for reporting on activities that involve school districts or the community. Furthermore, partnership activities are featured in Penn GSE newsletters and faculty meetings, providing extrinsic rewards in the form of favorable public acknowledgment. This supports the idea that partnership work is a valued part of a professor’s portfolio.

Rewards and incentives for district administrators

The district uses partnership language in its vision and documents and integrates that paradigm in its thinking about new projects, so partnership activities receive intrinsic rewards in terms of being in line with the mission and goals of the superintendent. One of the major benefits to district staff is the resources that they gain through partnerships, in the form of external grant funds and additional skilled assistance when faculty and students join them in collaborative work. The potential of increasing the quantity and quality of work is a substantial incentive for district administrators. Furthermore, the potential of building the capacity of practitioners to use evidence in decision making serves as a strong incentive for district leadership to participate in partnership work.

Rewards and incentives for principals and teachers

Although a stereotype exists of educators being resistant to participating in research (Lysenko, Abrami, Dagenais, & Janosz, 2014), we have found that principals and teachers are amenable to giving their valuable time to research endeavors if they think that it will benefit practice. Too often educators are asked to be interviewed or fill out lengthy surveys but then never receive any timely information based on that data.

To respond to this concern, Shared Solutions makes it very clear what the incentives are to participate—better information to serve the city’s schools. Throughout the activities of the partnership, we jointly execute the research, communicate findings, work with practitioners on interpreting the data, and consult with them on action plans. We provide accessible summaries of survey results in ways that directly address issues that are important to educators, answering questions such as “What are my school’s successes and challenges?” and “How can we improve?” Sharing results and working together on action plans serves as a strong incentive and reward for participation—incentives and rewards that we believe are much more powerful than monetary incentives.

Rewards for parents and the broader community

Like many struggling urban districts, there is some tension between the community and the school district. While Shared Solutions does not involve itself in local school politics, our goals include sharing ongoing findings with the community and garnering feedback on what we study, how we are studying it, and how we interpret results. We include parents and other community members in invitations to participate in our miniconferences, and we provide mechanisms for feedback through Facebook, Twitter, and letters to the editor. Through our community involvement efforts, we want to establish that rewards for participation include having one’s voice heard and considered in shaping not only the work of the partnership but the consequent district actions. Parents, in turn, have indicated to us that they like the opportunities to offer their opinion and they feel respected as participants in the research process.

Sanctions and penalties

Several tensions have surfaced in the development of Shared Solutions. At the district, not all office staff are open to and enthusiastic about collaborating with university faculty. These relationships are delicate and need to be developed and nurtured (Bryk & Schneider, 2002). We are working on ways to introduce the partnership approach across district offices. It is particularly important to make clear that partnership-type work is designed to be helpful to the district and that the work is jointly created. The traditional view of academic researchers coming into the relationship with their own “unalterable” questions will require time and effort to change.

At the university level, tensions have surfaced concerning competition and who is “in” and who is “out” of the partnership. Many faculty had been working with the district on small- and large-scale projects before the formal partnership came along, raising concerns about who should be involved in partnership work, how such involvement influences access to data and schools, and who should get “credit” for working with the district. Faculty may feel that not being a part of Shared Solutions is a penalty, because of the perceived access to data and district staff that it affords or that partnership activities are getting more resources or attention than their own work with the district, which they view as equally or more important. This is related to a question that is emerging as partnerships become more popular: To be effective, does the partnership need to permeate the entirety of both organizations, or is it more effective if a small cohesive group of people is involved? Surely, different models could be productive, but it is not yet clear when and under what circumstances a broad or focused model is most useful and whether, when, and how faculty-district collaborative research should fit under a broad umbrella effort, such as Shared Solutions, or work within its own unique model of collaboration. We are working toward creating a more unified accessible partnership that is viewed as a strength and resource rather than a source of competition among faculty for data and access to research sites.

Stability

Stability is the extent to which people, circumstances, and policies remain constant over time and it captures the degree to which a policy or program is institutionalized and therefore able to persist over time. Mobility of key actors affects stability when participation in a policy or program is related to particular individuals rather than the positions they hold as part of the organization (Rosenholtz, 1991; Smith & O’Day, 1991). The stability of the policy environment is critical because volatile reform environments send the message that reforms are temporary, thereby weakening commitment to reform efforts (Berends et al., 2002). Stability plays a critical role in facilitating the successful implementation of policies, but it is also the policy attribute that is most difficult to manipulate (Desimone, 2002).

Institutionalizing the partnership

We recognize that research-practice partnerships are fragile, as they often depend on specific personalities and changing priorities (D’Amico, 2010; Grossman, Wineburg, & Woolworth, 2001; Rosen, 2010) and are susceptible to leadership turnover (Hubbard, 2010). Realizing the dangers of relying on current good relationships to sustain the partnership, we put into place several mechanisms designed to institutionalize and sustain the partnership by helping it withstand changes in district and university personnel (Penuel et al., 2011; Stein & Coburn, 2010).

Institutionalized partnership positions, space, and internships

We included partnership activities in the scope of each organization’s work, adding partnership activities to each institution’s organizational chart under a specific job title. This ensures that while particular people may come and go in these positions, overseeing the activities of the partnership is an institutionalized set of requirements. We have also designated space at each organization for partnership staff to use. This formal space serves as a strong symbolic gesture that the partnership is a regular part of the activities of both organizations.

Another method that we adopted for institutionalizing the partnership was establishing a student internship program in which Penn GSE students are placed in partnership working groups and contribute to the research conducted in those groups. This serves as an ongoing and institutionalized source of mutual capacity building. External funding was useful for the internships at the beginning of the partnership; we are now seeking ways to fund the positions internally (e.g., as part of tuition aid at Penn GSE), which is a powerful source of stabilization.

Integrating partnership work into coursework is another way to institutionalize activities. Penn GSE has a practicum course where groups of students work with district clients on problems of practice. We include Shared Solutions projects in this course every year. This practicum provides the district with capable students to help analyze data within a reasonable time period (e.g., similar to the “rapid response work” of the Baltimore Educational Research Collaborative; Coburn et al., 2013, p. 38). At the same time, it affords graduate students valuable opportunities to build their capacity in analyzing administrative data and in working with district officials to interpret results in ways that are meaningful for practice.

Evaluating the Success of Partnerships

Certainly, partnerships can use the attributes as a framework to measure how and in what ways the partnership is specific, consistent, authoritative, powerful, and stable. But how does a partnership know when it has been successful? The lack of objective methods for measuring outcomes of partnerships (Kirschenbaum & Reagan, 2001) has limited the field’s ability to replicate successful endeavors and avoid unsuccessful ones. Establishing evaluation and feedback mechanisms is a powerful way of ensuring that the partnership has data to improve implementation (Cohen, 1990; Cohen & Ball, 1990; Lee, Bryk, & Smith, 1993).

Our partnership is currently exploring ways to evaluate our work. Henrick, Cobb, Penuel, and Clark (2016) developed a preliminary framework for assessing partnerships, which includes building relationships, conducting and using research, enacting local improvements, informing others’ work, and increasing the capacity to engage in partnership work. While they have not yet developed measures for these areas, the work is helpful in shaping thinking. The development of conceptual frameworks and valid and reliable measures of partnership processes and evidence in decision making is an area ripe for contribution.

We suggest taking an implementation-outcome approach to evaluating the success of partnerships, measuring the collaborative process as well as whether the research is used and what its effects are (Kirschenbaum & Reagan, 2001). Implementation indicators reflect how the work is being done and how it is being used. Outcome indicators measure whether changes in practice are occurring and whether those changes are improving school, leader, teacher, and student outcomes.

Implementation Indicators

Primary indicators of progress fall under the domains of participation and use (see Davies & Nutley, 2008). To what extent do district and university researchers participate in research design, instrument development, data collection, analysis, writing, dissemination, and other activities? Some partnerships may “divide and conquer,” while others may collaborate on all aspects of the work. Developing a metric to indicate the level and type of involvement of actors in each phase would provide evidence for how and in what ways collaboration occurs. Interviews or surveys to elicit this information could be conducted by third-party stakeholders, such as advisory panel members.

Intermediate Outcome Indicators

We view partnership products and their use as intermediate outcomes. An intermediate implementation indicator focuses on what products and information have come out of the partnership and what evidence there is that they have been used in decision making at the classroom, school, or district level. We provided two examples of partnership research shaping practice—a collaborative attendance study that resulted in changes in information provided to parents and the district using ideas and findings from partnership work to shape its turnaround efforts. Showing how partnership work has improved the way that work is done—at the district and at the university—is essential. However, tracing decisions about curriculum materials, interventions, and pedagogical approaches to research findings is complicated and complex (Tseng, 2012).

The William T. Grant Foundation is funding a new line of research on how partnerships improve evidence-based decision making, highlighting this area’s value and the field’s nascence. Certainly, a first and important step is to elicit information from practitioners about their use of partnership research in their decision making. One straightforward mechanism is to survey educators, asking if they have heard of the work of the partnership, what they think it is, whether they have participated in its work, how they access it, if and how they have used it in their practice, and what the facilitators and barriers are to their using it for decision making.

If the survey asked which partnership activities and products practitioners used and for what purposes, this could help determine the most effective mechanisms for conducting the work of the partnership. Additional questions could probe practitioners on areas where the partnership could improve. As a field, we have learned valuable lessons about what might work in building partnerships (e.g., Coburn & Stein, 2010), but we have not yet been able to offer any comparative studies that suggest which strategies are more or less effective in certain contexts. Within the context of any given partnership, the evaluative activities that we suggest here may be able to identify particular activities or paths that are more or less effective in building collaborative relationships and sharing information in ways that contribute to its use in decision making.

Outcome Indicators

The most important evaluative question that partnerships must ask is related to effects: Are decisions, priorities, interventions, and procedures that were chosen as a result of the partnership actually improving schools, instruction, and student learning? Integrated monitoring systems can provide suggestive evidence to this effect; randomized controlled trials and similarly rigorous designs can supply the necessary causal evidence about whether interventions are working. But, as we noted, tracing the choice of interventions to the partnership is more complex. Comprehensive evaluation of partnerships remains a challenge for the field.

Conclusion

Our vision of researcher-practitioner partnerships moves beyond collaborative work—it is a unified view of research, policy, and practice that bypasses many of the areas of tension fostered by the traditional research process. The new partnership paradigm has great potential to help bridge the so-called research-to-practice divide and, in doing so, improve the quality of education research and practice. Here we outline a foundational framework for building such a partnership, detailing activities that give specificity, consistency, authority, power, and stability to partnership activities. Research and practice partnerships are burgeoning, highlighting the importance of integrating what we know into a framework that researchers and practitioners can both use to build, monitor, evaluate, and sustain partnership efforts that improve their work.