Role of the Policy Environment in Shaping the Work of Professional Learning Partnerships

Abstract

Building on previous work studying research–practice partnerships, we use the framework of policy attributes to explore success and resilience in organizations tasked with quickly adapting to serving teachers and students in school environments vastly changed by the COVID-19 pandemic. This study leveraged survey and interview data from 12 partnerships located in large urban school districts in the United States to understand how the policy environment of partnerships facilitates success. Although previous literature emphasized the importance of all policy attributes in successful partnership work, we found that only high specificity (having a shared clear understanding of roles, responsibilities, and objectives) and the presence of robust authority (strong buy-in and sufficient resources) are most critical for organizational success.

Introduction

Over the past decade, researcher–practitioner partnerships (RPPs) have become an increasingly popular approach to solving problems of practice (Coburn et al., 2021; Tseng et al., 2017). We have learned that effective partnerships require developing shared understandings (Denner et al., 2019) and building trust through norms and routines that support collaboration (Coburn et al., 2013; Henrick et al., 2017; Penuel et al., 2015). We have also gained insights into barriers to this work, such as unclear roles (Farrell et al., 2019) and challenges around funding and capacity building (Tseng et al., 2017).

Although we have made progress in understanding how to develop, shape, monitor, and improve RPPs, we know little about how policy environments facilitate the work of these partnerships, and the factors that affect their resilience in the face of challenges. Our study addressed these topics.

Our work was part of a national study of 12 professional learning partnerships (PLPs)—collaborations between districts and professional learning providers—working to support the implementation of high-quality curricula through professional learning. The 12 participating districts served 50% or more of students who identify as Black, Latino/a, or English Learners and/or students from low-income families. The PLPs in our study provided district-based curriculum-embedded professional learning in math, English Language Arts, or science, intending to improve outcomes for students of color, students learning English, students with disabilities, and students from low-income families.

Our study was conducted from the fall of 2019 to the spring of 2021, which fell both before and during the COVID-19 pandemic that shut down schools and required a quick shift from in-person to virtual professional learning and instruction. This allowed us to explore success and resilience in organizations tasked with quickly adapting to serving teachers and students in school environments vastly changed by the COVID-19 pandemic.

We applied a conceptual framework (discussed in more detail below) that identifies five dimensions of the policy environment that contribute to building and maintaining strong partnerships: specificity, consistency, authority (buy-in), stability, and power (rewards and sanctions). Grounding our questions and measures in this framework responded to calls for additional cohesive frameworks and robust measures that could be used to empirically evaluate RPPs (Cooper et al., 2021). Furthermore, we extended existing work on identifying partnership challenges (Coburn & Penuel, 2016) by focusing on how to address those challenges. Research on RPPs has identified several features of effective partnerships, such as aligned priorities (Denner et al., 2019), shared vision (Tseng et al., 2017), clear roles (Farrell et al., 2019), strong relationships (Henrick et al., 2017), sustained communication (Penuel et al., 2015), ample resources (López et al., 2015), and a culture of learning (Farrell et al., 2022). Building on work that considers these features individually as beliefs or activities that contribute to a partnership’s success, we integrated them within the context of a broader, cohesive framework of policy attributes that allowed us to systematically consider and explore how the policy environment of partnerships facilitates success. For example, aligned priorities and shared vision are aspects of consistency; clear roles are an aspect of specificity; strong relationships, sustained communication, ample resources, and a culture of learning are aspects of authority. Further, our mixed-methods approach allowed both the systematic identification of key factors from our survey analysis and a deeper understanding of those factors from our interviews.

Our findings documented leaders’ varied perspectives and experiences that have clear implications for shaping partnership work. Analysis of survey data showed that high specificity (having a shared, clear understanding of roles, responsibilities, and objectives) and the presence of robust authority (strong buy-in and sufficient resources) are most critical for organizational success. A unique aspect of our work was the analysis of resilience in times of crises, and we found that in this context, survey data showed that specificity is the most important attribute of partnerships. Qualitative data from interviews supported these findings. PLP leads described specificity and authority in discussions of their organizational success, noting clear designation and understanding of roles and responsibilities, clarity around vision and goals, frequent communication (specificity), and belief in the work of the PLP and prioritization of the work (authority). Further, PLP leads attributed a lack of stability as a major challenge, whereas good relational authority (the quality of relationships and collaboration) contributed to their success as partnerships.

Building on Previous Partnership Work

This analysis built on previous work studying RPPs (Penuel et al., 2015) by using a conceptual model of policy attributes to understand the change efforts of PLPs. We also extended previous work by using this framework to systematically examine key aspects of partnership work and how these aspects relate to success and resiliency in the face of extreme challenges.

The PLP model has commonalities with various models of RPPs—traditional research alliances, networked improvement communities, and regional service centers. Both PLPs and networked improvement communities consist of groups that codesign solutions and seek to understand what is working and why (Coburn et al., 2013; Tseng et al., 2017). PLPs also bear similarities to regional service centers. Just as regional service centers are designed to supplement the capacity of states to enact policy changes (Pak et al., 2021), PLPs are designed to supplement district capacity. Further, PLPs share features of research alliances, including a commitment to long-term collaboration, clear benefits for each partner (mutualism), prioritization of district needs, and an intentional focus on problems of practice (Coburn et al., 2013; Tseng et al., 2017). Additionally, our role in evaluating the PLPs positioned us as partners in research, similar to the connections between researchers and practitioners in a research alliance. We worked with the PLPs to develop the overarching model for our study, including components that reflected their PLPs’ focus; solicited extensive feedback on our instruments; and partnered with each PLP to chart the progress of their work, provide ongoing feedback and a summative study of the success of their reform compared with others in their cohort.

Although PLPs do not fit neatly into a singular model of an RPP, we believe that because of shared features with various types of RPPs, our work is best situated in this body of literature, which seeks to understand the challenges and successes of building and maintaining relationships across different organizations in service of schools.

Establishing a clear definition of a successful partnership has been elusive (Tseng et al., 2017; Welsh, 2021), and the field lacks objective methods for measuring the outcomes of partnerships (Kirschenbaum & Reagan, 2001). Cooper et al. (2021) looked across various frameworks for evaluating RPPs and found that existing frameworks are not widely used, do not explicitly discuss methods, and include inconsistent metrics.

Nevertheless, research in this space has made progress in identifying features of successful RPPs. These include establishing productive working relationships through regularly scheduled meetings (Stanton & Easton, 2002), assigning clear roles and responsibilities that align with individual expertise (Engle, 2010), developing trusting relationships, collaborative decision making (Henrick et al., 2017), and committing to long-term collaboration (Coburn et al., 2013; Henrick et al., 2017). Research also has identified partnerships’ challenges, such as instability in the district due to turnover (Coburn et al., 2013), difficulty maintaining funding, and difficulty navigating complex relationships and power dynamics (Coburn et al., 2013; Tseng et al., 2017).

Notably, research in the RPP space often relies on interviews, observations, and artifacts—focusing on dynamics within a specific partnership. Farrell et al. (2018) built on the work of Henrick et al. (2017), leveraging a mixed-methods approach that included surveying to explore the work of RPPs. Similar to their work, we also explored conditions for maintaining partnerships and examined aspects of RPPs such as goals, communication, and challenges. Farrell et al. (2018) included survey measures of research use, partnership goals, attitudes toward partnerships, and degree of prior and sustained relationships. Whereas prior research has sought to understand attitudes and partnership dynamics and how they contribute to the success of partnerships, in our work, we delved into how to create strong partnerships using an established framework of policy attributes to offer strategic and nuanced insights.

Using the framework of policy attributes offers a systematic way to consider dimensions of RPPs that have been linked to successful RPP outcomes. The features of successful RPPs that researchers have identified fit within the broader context of policy attributes: regularly scheduled meetings foster consistency, clear roles and responsibilities contribute to specificity; trusting relationships and collaborative decision making build authority; and a commitment to long-term collaboration fosters stability. By considering these features as attributes of the policy environment, we were able to offer insights into how to create strong partnerships. We deepened and extended research in this space with a mixed-methods approach that dived deeper into the nuances of relationships, organizational capacity, and processes, leveraging survey and interview data to answer our research questions. We then examined the role that policy attributes play in how partnerships function as an organizational unit—defining partnership success more operationally, in terms of successfully navigating schedules, allocating resources, balancing diverse needs, making decisions quickly, and so on.

Conceptual Framework

We ground our work in Desimone’s (2002) adaptation of the policy attributes theory, which identifies five dimensions through which policies gain influence over practice: specificity, consistency, authority, power, and stability (Porter, 1994; Porter et al., 1988). Specificity (degree of detail and clarity) facilitates implementation by clarifying goals, roles, and actions required to implement policies/programming successfully. Authority (buy-in and resources) facilitates implementation by providing the necessary support—people, time, and money. Consistency (degree of alignment) facilitates implementation by ensuring that everyone is on the same page and that various initiatives work together. Power (rewards and sanctions) facilitates implementation by exerting pressure or offering rewards to implement. Stability (consistency of people, policies, and contextual factors over time) is critical because volatile improvement efforts signal that changes are temporary, which weakens commitment to policies/programming. Although Porter and colleagues conceptualized the policy attributes to be fixed for any given policy, Desimone (2002) suggested that rather than fixed “true” values, perceptions of the policy attributes matter for implementing reforms. The policy attributes can be applied to policies or interventions to capture the mechanisms through which these activities are implemented.

The policy attributes offer a clear organizing framework for making sense of how partnerships function as a unit: Specificity: Are there clear roles and responsibilities? Consistency: Do all partners share the same vision and goals? Authority: Does leadership from member organizations support the work? Power: Is participation in the partnership mandated and/or explicitly rewarded? Stability: Are relationships within the partnership longstanding?

Previous studies of RPPs have identified policy attributes as instrumental in ensuring positive outcomes of partnership work. Desimone et al. (2016) applied the policy attributes theory to the ongoing work of partnerships, suggesting that all five attributes are important to consider in developing, implementing, and monitoring partnerships. In an analysis of regional service centers’ partnerships with states to provide training and resources, Pak et al. (2021) applied the policy attributes framework and found that authority is critical to the successful implementation of interventions and that authority can be built through specificity and consistency.

Our study leveraged the policy attributes to better understand how partnerships can strengthen the aspects of their organizational structure, processes, and relationships that have been shown in previous research to be instrumental in achieving their goals (e.g., increased evidence use). In particular, we asked (1) How do PLPs vary on key policy attributes? (2) What predicts PLPs’ organizational success in terms of leadership, collaboration, and progress toward goals? (3) Are there defining features of PLPs that facilitate resilience in the face of extreme challenges, such as those posed by the COVID-19 pandemic? and (4) How do PLP leads’ descriptions of successes and challenges map on to the policy attributes and help explain organizational success and resilience?

Data and Methods

This study was part of a larger mixed-methods study of the implementation and outcomes of 12 PLPs located in 12 large urban school districts in the United States. These PLPs were funded by a large philanthropic foundation to enact professional learning initiatives from the summer of 2019 through the spring of 2021. Each PLP consisted of at least one school district and one organization external to the district, which was either a curriculum developer, professional learning organization, or university. Together these organizations provided professional learning aligned with high-quality mathematics, English Language Arts, or science curricula to teachers and coaches with the goal of improving instruction and academic outcomes for students. When we reported our findings for individual PLPs, we used pseudonyms to protect their confidentiality.

This analysis leveraged spring 2020 and spring 2021 survey responses from PLP leads across 12 PLPs (n = 69 and n = 59, respectively). We additionally analyzed 106 interviews from 58 leads that took place in the spring of 2020, winter of 2020/2021, and spring of 2021. Although partnership activities began prior to the COVID-19 pandemic, all data were collected after the start of the COVID-19 pandemic. Survey questions in the spring of 2020 asked PLP leads to think back to their work before the pandemic and contrast it with the present day. Those we refer to as PLP leads were first identified using the organization’s funding proposal detailing who was designated as key personnel and who the PLP chose to be the main point of contact for our research team. We then asked each person identified in our initial sampling who they considered to be leaders in their PLP. If those respondents were not already on our list to contact, we added them. This process also was used to identify new PLP leads when turnover occurred. Across PLPs, the leadership team ranged from two to eight people, with most PLPs having a core team of three to five people. PLP leads held positions such as superintendent or assistant superintendent, coordinator, chief program officer, and project director.

In our survey, PLP leads rated agreement with multiple indicators of policy attributes (i.e., specificity, consistency, authority, and power) on a 0–5 agreement scale from 0 = “completely disagree” to 5 = “completely agree”). ¹ Specificity items measured the clarity of objectives, roles, and communication. Consistency items asked about the extent to which the PLP work aligned with other initiatives. Authority focused on the extent of buy-in to the vision and strategies of the PLP, the degree of support from key PLP members, and the availability of resources such as money and expertise. Power items asked about whether participation was required and the extent to which incentives or penalties were tied to participation. For stability—the extent to which people and policies remain constant over time—PLP leads reported the number of years they had worked with partnering organizations and how long they anticipated continuing work. To create our stability variable, we assigned one point for every relationship that had existed for 3 or more years and then divided the sum by the total number of organizations (depth of relationship) to offset the advantage of having more partners. Then we added one point if they anticipated continuing to work with a member of the PLP for 3 or more years (relationship optimism). We provide the complete list of items and scales in the Appendix.

Rather than consider a partnership’s success in terms of outcomes such as research use, producing knowledge, or student achievement, we focused on success in terms of aspects of organizational processes that the previous literature had attributed to successful partnerships, including organizational activities such as scheduling, communication, decision making, and evaluation. Partnerships had different goals and timelines, and this allowed us to measure similar constructs across studies and also builds on previous work that had established key components that lead to success no matter what the goals are. To measure organizational success, we had PLP leads rate the degree of challenge in conducting these activities on a 0–2 scale (“This was a significant challenge” [0], “This could have used some improvement” [1], and “This was going well” [2].) Partnership leads reported changes in these same activities after COVID-19 (“This is more of a challenge” [−1], “This is the same” [0], and “This has improved” [+1]). See the Appendix for a list of all survey items. To create the PLP-level “success“ score, we assigned individual responses one point for each area of work they indicated was going well. We then averaged across all responding members of the PLP to get the PLP’s overall score. The maximum number of points a PLP could receive for organizational success was 15 (mean = 10.5 in 2020, 10.8 in 2021).

The work of the PLPs and our research was conducted before and during the COVID-19 pandemic, allowing us to explore resiliency. To explore resiliency in partnership work during COVID-19, we created a resilience variable that considered whether each area of partnership work stayed the same, improved, or became more of a challenge after COVID-19 disruptions. Resilience was measured as the average change in difficulty of various partnership activities (e.g., scheduling, communication, resources, and collaboration) after COVID-19, with PLPs receiving +1 point for improvement, 0 points for maintenance, and −1 point for increased challenge.

We conducted descriptive analyses of spring 2020 survey data to examine patterns in policy attributes and successes and challenges to implementing professional learning both before and after COVID-19. We repeated these analyses with our spring 2021 data, except for comparisons before and after COVID-19, because all spring 2021 data were treated as after COVID-19.

We additionally used a stepwise regression, with individual PLP leader responses as the unit of analysis, to identify which policy attributes were predictors of success prior to COVID-19 (n = 69, measured on our spring 2020 survey) and 1 year into the COVID-19 pandemic (n = 59, measured in the spring of 2021) as well as which predicted resilience at the start of COVID-19 (measured in that spring of 2020). Two partnerships were excluded from the COVID-19 analyses in the spring of 2020 because their professional learning intervention had not started. Although some argue against the use of stepwise regression (e.g., Thompson, 1995), the approach is widely used to preserve degrees of freedom when faced with a small sample and multiple predictors (Babyak, 2004). Additionally, in instances where one wants to enable a more pragmatic interpretation of a phenomenon, stepwise regression’s identification of the “smallest number of variables which do the job” is sensible (Streiner, 2013, p. 131).

In the spring of 2020, winter of 2020/2021, and spring of 2021, we interviewed PLP leads about their experiences, asking them to reflect on successes and challenges and share details about the development and organizational-level processes of their PLP. To analyze the interview data, our research team coded deductively using a detailed coding scheme that included the policy attributes (Miles & Huberman, 1994). As a second step, we drew on the results from the quantitative analysis of survey data to narrow our focus within the qualitative data, considering only parts of interviews coded for the policy attributes that were identified as significant in our regression models as well as those coded for successes and challenges (Creswell & Clark, 2017). This approach aimed to identify areas of convergence and divergence between the qualitative and quantitative data (Greene, 2007). We then inductively coded within these selections to identify emerging themes in the data that could help to explain the levels of policy attributes and successes and challenges in PLP work. In reporting, we identified key themes in the data based on their prevalence among PLP leads and across PLPs (Ryan & Bernard, 2003). More specifically, when PLP leads from at least half the PLPs in our sample mentioned a particular topic or idea (e.g., clear designation of roles and responsibilities), we considered that a key theme. Once these key themes were identified, we used illustrative quotes to reflect those key themes and help explain our quantitative findings (Creswell & Clark, 2017).

In our study, we analyzed survey data to identify the policy attributes that played a significant role in predicting partnership outcomes before we analyzed our interview data in order to strategically narrow the focus of our qualitative analysis. This approach streamlined the qualitative analysis and enabled us to compare broader trends from the survey with the more nuanced, real-world reflections from PLP leads. By examining similarities and differences between the quantitative and qualitative datasets, we were able to deepen our understanding of the policy attributes and their role in organizational success while also using illustrative quotes to paint a picture of what the attributes look like in real life. Ultimately, this process enriched the study by blending statistical insights with personal experiences, leading to a more comprehensive understanding of the role of policy attributes in partnership work.

Findings

To more comprehensively understand the role of policy attributes in partnership work, we first sought to understand variation within and across PLPs on policy attributes, then to uncover which attributes lead to organizational success, and finally, to identify which of these attributes plays an important role when facing extreme challenges, such as those posed by the COVID-19 pandemic.

How Do PLPs Vary on the Policy Attributes?

We used analysis of survey data to answer our first research question. Analysis showed variation across PLPs in mean leader views of the attributes in years 1 and 2 of data collection (see Figure 1). Additionally, there was considerable variation within PLPs, suggesting that leaders who were members of the same PLP had differing perspectives on their PLP’s attributes. For an example of this variation, see Figures 2 and 3. Due to constraints on space, we present this variation only for the policy attributes we found to be significant in our regression models. Means for PLPs on specificity, consistency, authority, and power ranged from 2.7 (between slightly disagree and slightly agree) to 4.7 (between mostly agree and completely agree), with power having the lowest overall mean. For stability, the low was 0.83, and the high was 1.7 (indicating longer and more sustainable relationships).

Figure 1.

Mean policy attributes by partnership.

Figure 2.

Within-partnership variation for specificity.

Figure 3.

Within-partnership variation for authority.

Generally, there was greater within-PLP variation across all the attributes in 2020 compared with 2021. This may be due to increased challenges during the first year of the pandemic or, alternatively, may reflect growing consensus on partnership vision and goals. Stability had the most within- and across-PLP variation (2020 and 2021), specificity had the least within-PLP variation (2020 and 2021), and power had the least across-PLP variation (2020 and 2021). Some PLPs had less within-organization variation, suggesting a stronger shared vision and similar experiences, whereas others had considerable within-organization variation.

The number of leads participating in the survey per PLP ranged between three and nine (2020) and two and eight (2021). This is important context for interpreting results—this variation suggests that even in a PLP with high specificity overall, members could report low specificity. This finding speaks to the subjective nature of the policy attributes and the role of perception (Desimone, 2002). Averaging across partnerships mutes the importance of individual experiences, and it is important to account for and consider this in interpreting results.

What Predicts a PLP’s Success?

To answer our second research question, we specified two stepwise regression models that used the five policy attributes to predict PLP success, one using the 2020 survey data and one with the 2021 data. Regression analysis revealed that in the spring of 2020, specificity and authority were significant predictors of organizational successes prior to COVID-19 (p = .002 and p = .009, respectively). This means that PLPs that scored higher on specificity and authority were more likely to report successes in a variety of organizational functions such as leadership, scheduling, communication, and evaluation. For a full list of success items, please see the Appendix. This finding is consistent with the year 2 analysis, which also found that PLP specificity and authority were significant predictors of organizational success (p = .027 and p = .005, respectively; see Tables 1–5).

Table 1

Organizational success regression analyses

2020	$\gamma =-12.619+2.281S+3.072A$
2021	$\gamma =-11.99+1.968S+3.250A$

Note. $\gamma$ = organizational success, S = specificity, and A = authority.

Table 2

Organizational success stepwise regression analysis results (2020)

Variable	B (unstandardized coefficients)	SE	Beta (standardized coefficients)	t	Significance	Tolerance	Variance inflation factor
(Constant)	−12.619	3.657	—	−3.451	<.001	1.000	1.000
Specificity	2.281	0.716	0.400	3.188	.002	0.522	1.917
Authority	3.072	1.144	0.337	2.686	.009	0.522	1.918

Table 3

Excluded variables in organizational success stepwise regression (2020)

Excluded variable	Beta in	t	Significance	Partial correlation	Tolerance	Variance inflation factor	Minimum tolerance
Consistency	−0.018	−0.168	.867	−.021	0.713	1.402	0.477
Power	0.019	0.208	.836	.026	0.949	1.053	0.504
Stability	0.110	1.147	.256	.141	0.890	1.123	0.501

Table 4

Organizational success stepwise regression analysis results (2021)

Variable	B (unstandardized coefficients)	SE	Beta (standardized coefficients)	t	Significance	Tolerance	Variance inflation factor
(Constant)	−11.990	3.478	—	−3.447	.001	—	—
Specificity	1.968	0.868	0.318	2.269	.027	0.497	2.013
Authority	3.250	1.113	0.409	2.921	.005	0.497	2.013

Table 5

Excluded variables in organizational success stepwise regression (2021)

Excluded variable	Beta in	t	Significance	Partial correlation	Tolerance	Variance inflation factor	Minimum tolerance
Consistency	0.016	0.110	.913	.015	0.454	2.204	0.378
Power	−0.061	−0.606	.547	−.081	0.959	1.043	0.481
Stability	−0.041	−0.391	.697	−.053	0.917	1.090	0.489

Interview data supported these findings about the value of specificity and authority and provided additional insights into what these attributes look like in practice in the PLPs. Three key themes related to specificity emerged from the interview data: clear designation and understanding of roles and responsibilities, clarity around vision and goals, and frequent communication. For authority, two key themes emerged: belief in the work of the partnership and prioritization of the work.

Specificity Theme 1: Clear Roles and Responsibilities

PLP leaders described clarity around roles and responsibilities as beneficial to their organizational success. For example, partner 2 from Riverview reported:

I think it also really helps that our lanes were really distinct as providers, and so it’s been finding the connections between the lanes versus fighting about whose lane is which lane has been super, super beneficial.

Conversely, partner 1 from Egret described a lack of clarity around roles and responsibilities in their PLP. In particular, when describing the PLP’s potential for making a meaningful difference, partner 1 stated:

It’s tricky to have three organizations try to work with different stakeholders and almost stay in different lanes . . . . Even the why behind that is a little unclear. We could have done it differently. We could have been a little bit more aligned. So in some ways I think it may be coming back to that disorganization and lack of clarity of purpose.

Clear designations of roles and responsibilities can ensure that organizations are not duplicating efforts that, in turn, may help explain organizational success. When the designations of roles and responsibilities are unclear, it can lead to confusion and delay the work of the PLP.

Specificity Theme 2: Clarity Around Vision and Goals

PLP leads discussed the importance of achieving clarity around the vision and goals of the PLP work. For example, when asked what helped their PLP achieve success, partner 1 from Elmwood stated:

I think that we got really clear about what the problem was and we got clear about the solution before we started the partnership, to be honest. I think honestly [that] if we had allowed either [of our partnering organizations to] drive the project or the PLP, then I don’t think we would have gotten as far as we did. We would have been figuring it out. Our own clarity about what we needed was really important.

In contrast, PLPs that lacked a clear understanding of their project’s goals or vision experienced challenges. For example, when asked to describe the PLP’s goals, partner 2 from Egret said: “I was never 100% sure what their goal was.”

Having clarity around the vision and goals of the PLP work can provide a shared understanding of the work. PLPs that are able to build shared understanding may be more adept at organizational functions such as allocating resources and monitoring progress given that they have a clear idea of what they are striving to achieve. This dynamic also may enable them to better align resources toward their vision and goals.

Specificity Theme 3: Communication

PLP leads mentioned regular communication as a means of ensuring that partnering organizations had shared understandings of the PLP work and clarity around roles and responsibilities. In particular, PLP leads mentioned the importance of regular communications (e.g., meetings and newsletters) with partnering organizations. For example, partner 1 from Trinity reported: “I would just say the regular weekly communications just ensured that we were all holding each other accountable and that helped us be effective.” However, lines of communication are not always clear with all organizations, even within the same PLP. For example, partner 1 from Egret explained that their organization “assigned a partner success manager [to the school district], and she scheduled biweekly meetings [with the district].” Yet, their organization “didn’t have a recurring meeting with [the professional learning organization].” For this PLP in particular, lack of communication seemed to hinder their organizational success. Communication in particular seems to be vital to organizational success because mechanisms for communication can facilitate a shared understanding of the vision and goals as well as clarity around roles and responsibilities.

Authority Theme 1: Belief in the PLP Work

PLP leaders were mostly positive in their views of the partnership work, with some leaders expressing interest in continuing the work after the grant ends. They generally viewed the work they were doing as valuable and expressed excitement regarding the potential impact the PLP work could have on students. For example, partner 1 from Trinity explained:

I think it’s been well received. I think that the Trinity leadership values it, and it’s seeing that we’re providing real assistance and support to their teachers. From [my organization’s] perspective, I think that my boss sees value in it and is certainly seeing it as a valuable contribution to the work of the field right now.

Similarly, partner 2 from Edgemont mentioned that “the PLP and the approach to professional development have been overwhelmingly positive. The folks we’re working with are inspired, they’re excited. There’s momentum.” Yet, despite the mostly positive reports, PLP leaders also shared negative reports related to authority. Although PLP leaders reported mostly strong normative authority (e.g., buy-in), their reports surrounding institutional authority (e.g., resources) were generally related to challenges, as we describe in the next section.

PLP leads emphasized buy-in as a critical component of their partnerships’ work. For example, partner 3 from Titan shared:

I also think that this partnership is supported from leadership, senior leadership all the way down to our principal group. We just met with our principal team on Monday, just identifying some additional criteria in terms of selecting schools and some of the options within the contract. People are excited and bought in. I would say that there’s support throughout the organization for it.

Similarly, partner 2 from Ranier spoke of their partnership in terms of everyone “singing from the same sheet of music.” These shared beliefs and values lent themselves to partnership success.

Are There Defining Features of PLPs that Facilitate Resilience in the Face of Extreme Challenges Such as Those Posed by the COVID-19 Pandemic?

We used stepwise regression to answer our third research question. Although the previous literature had talked about the importance of all policy attributes in the ongoing success of partnership work (Desimone et al., 2016), we found that only specificity was a significant predictor of resilience (p < .001; see Tables 6–8). PLPs that reported higher specificity were less likely to report experiencing worsening PLP organizational challenges due to COVID-19. PLPs with high specificity expressed clear distinctions between roles and shared understandings of goals.

Table 6

Resilience regression analysis equation

2020	$\gamma =-7.871+3.415S$

Note. $\gamma$ = resilience and S = specificity.

Table 7

Resilience stepwise regression analysis results

Variable	B (unstandardized coefficients)	SE	Beta (standardized coefficients)	t	Significance	Tolerance	Variance inflation index
(Constant)	−7.871	3.53	—	−2.229	.029	—	—
Specificity	3.415	0.82	0.452	4.143	0	1	1

Table 8

Excluded variables in resilience stepwise regression

Excluded variable	Beta in	t	Significance	Partial correlation	Tolerance	Variance inflation	Minimum tolerance
Consistency	0.172	1.403	.165	.17	0.781	1.281	0.781
Authority	0.248	1.628	.108	.196	0.5	1.999	0.500
Power	0.011	0.098	.922	.012	0.983	1.017	0.983
Stability	0.117	1.012	.315	.124	0.894	1.119	0.894

For example, partner 2 of the Riverview partnership explained:

I think it also really helps that our lanes were really distinct as providers, and so it’s been finding the connections between the lanes versus fighting about whose lane is which lane.

Conversely, PLPs with low specificity described a lack of understanding of the roles and responsibilities within PLPs and the absence of clearly defined objectives for their work. Partner 4 of the Ranier partnership shared, “It’s hard to know whose role is what. Things become very convoluted and confusing. That kind of slows down and makes foggy our work.” Some partnerships recognized the importance of increasing specificity. For example, partner 2 for the Telford partnership spoke to their partnership’s explicit efforts to clarify roles and responsibilities further. This lead explained that while initially they “didn’t do a good job,” as more people joined the initiative, they made a point of having a sit-down and “clearly identifying who is the go-to person for each aspect.” With the shifts necessary to work productively during a pandemic, having a shared clear understanding of roles, responsibilities, and objectives understandably would facilitate more effective partnership work.

How Do PLP Leads’ Descriptions of Successes and Challenges Map onto the Policy Attributes and Help Explain Organizational Success and Resilience?

To answer our fourth research question, we used our interviews to help explore how the enactment of these policy attributes might shape both successes and challenges for PLPs. We additionally allowed other themes to emerge to gain a more complete picture of what factors, beyond the policy attributes, influence partnerships’ successes and challenges.

Reflective of our quantitative findings showing variation across PLPs on survey responses, our interview analysis revealed variation in how leads spoke about the policy attributes. In reviewing data at the PLP lead level, we classified comments about specificity as positive or negative, and in rolling them up to the PLP level, we noted whether interviewees within a PLP were consistent or variable and positive, negative, or neutral.

Leads at six PLPs consistently described strong specificity; for two PLPs, the descriptions were variable but generally described weak specificity, and four PLPs were more neutral in their descriptions of their PLPs’ specificity (three consistently neutral. one variable but generally neutral). Although PLP reports of specificity were largely positive (see our third research question), when specificity was a challenge, it tended to be due to a lack of clarity around roles and responsibilities. For example, partner 4 at Ranier explained:

There have been many points where [the external consultant] and I have just been in the dark—like what should we do? And it’s the blind leading the blind in that. He has great expertise, but he has no clue what’s going on in our district. I don’t know what’s going on in our district. But I don’t really know where we should go because I’m fresh. I’m new at this work, but I’m a fast learner. So I went from swimming to running really fast . . . as a tadpole and had to evolve; that’s how that analogy works. It’s been really weird.

With regard to authority, leaders from nine PLPs described strong authority (seven were consistent within the PLP; two were more variable). Only three PLPs were more neutral in their descriptions of authority (one consistently and two with within-PLP variability). PLP leaders described authority as both a success and a challenge particularly when it came to the districts’ prioritization of the PLP’s work or the resources available for the work. For example, partner 4 from Titan mentioned:

I think another big success, honestly, it’s just [the district’s] prioritizing this work in the way it has. Most districts don’t, but saying like we’re doing K through 12 ELA [English Language Arts] and math implementation and we’re putting our resources against that and prioritizing it for the next three to five years is huge because often there are just so many competing priorities that a lot of this work gets watered down.

However, other PLP leaders expressed feeling like their work was less of a priority in districts due to the content area or described a lack of resources to support the PLP’s work. For example, partner 1 from Elmwood stated:

I think my harping on money right now is the crux of the dilemma which is, there’s a fiscal crisis really here in [the district], so being able to, in a timely fashion, scale high-quality materials and then, in tandem, high-quality professional learning is really a challenge because we can do professional learning, but if it’s not in the presence of high-quality materials, I think it won’t get us where we need to go. . . . We’re looking down the barrel of a 20% deficit we’re trying to make up, it’s very challenging.

Prioritization of the work and having sufficient resources can provide a foundation for accomplishing PLP-related goals. PLPs whose work was prioritized were often also provided with sufficient resources to engage in the PLP work while also encountering fewer competing priorities. Further, having sufficient resources (e.g., time and money) can ensure that PLPs have the tools necessary to facilitate their work.

In exploring the themes that emerged from discussions of successes and challenges by partnership leads, findings across all 106 interviews with PLPs leaders also revealed three core themes related to authority and stability: pivot to a virtual workplace (stability), personnel instability (stability), and relationships/collaboration (authority).

Stability Theme 1: Pivot to a Virtual Workplace

During the COVID-19 pandemic, many personnel within PLPs had to adapt to a major change to their work: the pivot to a virtual workplace. At the peak of the pandemic, PLPs modified all organizational procedures, such as interactions and data monitoring, and professional learning to become fully remote. This posed several difficulties.

About half the PLP leaders we interviewed noted how difficult it became with a virtual workplace to keep track of responsibilities and progress. In addition to putting a stress on logistics, replacing travel by creating a virtual workplace seemed to leave PLP leaders feeling isolated from what partners, such as school districts, were doing and experiencing. For example, partner 2 from Edgemont said: “Doing everything virtually, it just made it a little bit harder to stay on top of what’s happening when and to what extent.”

Partner 2 from Trinity explained how a virtual workplace limited the amount of insights their organization gained from partners:

[If we] don’t have the opportunity to be with our contractors, with our advisors, with our consultants on the ground, it limits the access that we have to their actual work, as well as limits access that we may have . . . to talk about the impact of that work and what they can do differently. So, without having a full picture, it’s difficult for us to provide guidance or support or systemic change.

For PLPs that consisted of teams that worked remotely prior to the pandemic, the pivot to a virtual workplace posed little to no challenges. Partner 1 from Trinity described how their partnering districts’ shift to a virtual environment disrupted their plans to monitor and conduct professional learning. They commented:

We ran into challenges when [the district] went to remote learning; the building schedules got disrupted. And so we didn’t get our whole cohort every time because they had mandatory professional learning at their campus now during the time that we had set for the cohort to meet.

Although this PLPs’ work was remote, the district’s pivot to a virtual workplace resulted in difficulties with scheduling and professional learning attendance.

In contrast, partner 3 from Elmwood expressed how they thought that the pivot to a virtual workplace was not a challenge:

So a lot of our work has been virtual from the beginning. . . . And so it really was just the professional learning for teachers and coaches that had been taking place in person that we have to shift to virtual. But [the partnering organizations] are both remote organizations. And so our whole staff on both sides are very accustomed to working virtually. And [the district] is a really tech-savvy district, so that has been a pretty seamless transition.

Remote work was already embedded in Elmwood’s partnership, but their professional learning was initially in person. At the organizational level, Elmwood was not confronted with issues related to losing progress or momentum with the pivot to a virtual workplace during COVID-19. Indeed, they had procedures and plans in place to accommodate such a shift in professional learning.

Stability Theme 2: Personnel Instability

Aside from the COVID-19 pandemic’s effect on the pivot to a virtual workplace, personnel instability was an ongoing challenge for many PLPs. Turnover, or sudden shifts in personnel, left PLP leaders feeling stagnated at work, with goals and objectives not being met as quickly as they had hoped. In discussing such challenges, partner 2 from Edgemont stated:

So, I think on our side, a bit of it was new staff. . . . our team got bigger. And so, the precision that my team of three focused on in year one and then getting to a team of five very quickly; I think we just, again, lost some of that focus.

PLP leaders, like those from Edgemont, found that expanding or changing employees shifted their attention away from the goals or tasks they initially proposed. Such pauses to their productivity were likely a result of replacing employees with others who may not have understood the culture or work of the PLP in depth. Turnover was not uncommon among partnerships. Titan also faced challenges with instability in their district. Partner 4 explained:

So, I think that those are the constraints, those adaptive issues of the confluence of factors of leadership shift and the ongoing piece of COVID. And I think the other interesting piece, and this might be true, [is that] a lot of districts definitely stepped in as the deal. So geographically situated in an area that has just been hit really, really hard with, with COVID. And it’s also very politically charged space. So they have gone through a new board and a new mayor and it’s a very contentious space. And I think that, that the politics of it also permeates the system a little bit.

Additionally, partner 6 from Evergreen reflected on their experiences with changing personnel and how they could have mitigated this challenge:

You don’t want to have diffusion of responsibility, but you want to probably have duplication of people in the know of what your long-term goals and moves are. So, what happened is we had one key district person who was really invested and did a lot of the work in year one, really two, and both of those people turned over. So, if we had built in some duplication, and instead of assigning just one person as the lead, bring in two folks. Then if one goes down, you still have one of the original people left. I think that would have not tripped us up as much, if we had a little bit of . . . like more people that were in the know on certain parts.

In their quote, partner 6 suggested increasing personnel with similar specialization and/or understanding of the PLP. This could have resulted in an efficient transfer of knowledge and power. With this, their perspective also implies how duplicating roles could have facilitated a support group where multiple individuals could coordinate with each other to duplicate activities related to the work of the PLP. Yet, this was not the case with partner 6’s PLP; their PLP struggled with turnover and losing personnel who were paramount to sustaining their goals.

Authority Theme 2: Cooperation and Openness to Ideas

We found that PLP leaders credited their successes to how responsive they were to each other’s ideas, suggestions, and perspectives. PLP leaders generally discussed how their relationships and collaborative practices involved some level of brainstorming or problem solving, and this was most efficient when members were willing to listen to each other. Partner 2 from Telford, who was a school district lead, provided evidence of this, stating that PLP leaders’ cooperation and openness led to their successful relationship:

I think that has gone exceptionally well in building those relationships and building that bond in the project, because I think they listen. I think they take in and internalize what they’re hearing, and then do what they think they can to support us and say, “Hey, we’re thinking about this.“ They talk more. They plan with us.

Partner 2 from Telford highlighted what was most successful to their partnership, and that was the extent to which their organization’s members cooperated well with each other. For all PLPs, including Partner 2 from Telford, sharing honest thoughts and ideas with each other was necessary to help their projects flourish. Partner 5 from Trinity’s discussion also describes how partnerships focused on the importance of hearing and respecting each other’s perspectives. They explained, “If we see that our partnership is kind of straying off an organizational goal, we have to be open about it. We have to have those conversations, we had them in our weekly meeting, and be responsive to each other’s concern.” Here, Partner 5 from Trinity demonstrated what their PLP did to work towards cooperation and openness to ideas. As evident in their quote, they formally (e.g. in meetings) discussed why the partnership lost focus of a goal and worked with each other to examine what problems or misunderstandings could have contributed to “straying off.” In all, PLP leaders successfully made decisions when they felt that their members were cooperative and open to suggestions without resistance.

Authority Theme 3: Trust

Along with cooperation and openness to ideas, trust also emerged as a crucial component to PLP leaders’ successful relationships. PLP leaders found that they were most successful when they were confident in each other’s contributions and overall having trust. Teams were more likely to perceive trust if they believed in each other and in the impact of their work (i.e., authority). To illustrate this, partner 2 from Ranier said:

I think just this willingness to be experimental and trusting each other that we will not drop the ball, or invest a bunch of time and energy . . . on something that isn’t going to have a life beyond this conversation. . . . I think those are the things that I’ve really appreciated about this team.

Partner 2 from Ranier acknowledged that their team had to rely on each other to make worthwhile choices about proceeding with their work. Having trust also involved respecting boundaries and refraining from excess control over another’s responsibilities or ideas. To evidence this further, partner 2 from Riverview stated:

There’s been a trusting relationship from the early days. I think that we have been willing, [the university partner] hasn’t meddled in what we do with [the district], and we haven’t meddled in what they do with [the district], in part because we trust each other.

For partner 2 from Riverview, they mentioned that their leads maintained boundaries where individual roles and perspectives were respected and trusted. No individual had excessive control over another, which indicated some form of trust in their capacity to develop the work of the partnership. Thus, throughout the life of each PLP, leaders had established trust within their organization, which then allowed for successful relationships and collaboration.

Discussion

Although many have explored the challenges of developing and maintaining RPPs (Coburn et al., 2013) and have developed recommendations for effective partnerships (Penuel & Gallagher, 2017), recent work has called for more rigorous systematic measures for assessing partnerships (Cooper et al., 2021). This analysis contributes to this goal as well as examines the development and effectiveness of RPPs (Coburn & Penuel, 2016; Cooper et al., 2021; Henrick et al. 2017). Although PLPs do not align perfectly with any single model of RPP, they share important characteristics with several types, including traditional research alliances, networked improvement communities, and regional service centers. These parallels suggest that lessons from successful RPPs can be effectively applied to a broader range of partnerships in education. Additionally, the alignment between our findings on policy attributes in PLPs and what is already established about policy attributes in the literature on RPPs further strengthens the argument for classifying partnerships based on key components and organizational features rather than adhering to specific configurations.

Our findings suggested that having a shared clear understanding of roles, responsibilities, and objectives (high specificity) and having buy-in and resources (authority) are most critical for success. This stands in contrast to previous research that has shown that incentives (power), alignment with other initiatives (consistency), and sustained relationships and district context (stability) can be important factors in successful partnerships (Coburn & Penuel, 2016; Desimone et al., 2016; Penuel & Gallagher, 2017) because these attributes did not emerge in our analysis as essential for success. While we found that power, stability, and consistency were not key players in this particular analysis; we do not dismiss their importance to partnership work. Further work is needed to explore how different attributes interact to support partnership work across different contexts.

Qualitative analyses provided insight regarding what specificity and authority looked like within partnerships, emphasizing the role of communication and prioritization of the work. This suggests that partnerships may want to attend to establishing mechanisms for communication and prioritizing partnership work in order to foster effective and resilient partnerships. Analysis of all PLP leaders’ interviews additionally shed light on key challenges, emphasizing shifts in work due to the COVID-19 pandemic and the difficulty of navigating shifting personnel. Nevertheless, PLP leaders underscored the importance of relationships and collaboration, which were facilitated by members’ cooperation and openness to ideas and trust.

We found variation in how leaders of these efforts viewed critical aspects of their policy environment, documenting the varied perspectives and experiences that have clear implications for shaping their work. We relied on self-report data from these leaders, which can come with biases, but we believe that it is what respondents experience in the policy environment that influences their behavior, affirming the importance of self-report in understanding the workings of partnerships. This is consistent with prior research that emphasized the importance of role and identity negotiation in RPPs (Farrell et al., 2019). These different perceptions of the policy environment may reflect real differences in the day-to-day experiences of different members. If this is the case, the implications are that each individual’s particular context and constraints need to be considered in understanding and supporting partnership work. Alternatively, this variation in perceptions within a partnership may reflect variation in values, interpretations, and perspectives. If this is the reason, it suggests that more work needs to be done to share ideas and experiences and to build mutual understanding and shared vision. To this end, emerging methods in sensemaking in complex settings may hold promise for advancing partnership work (see Snowden & Rancati, 2021).

Notably, this study also contributes to an emerging body of literature that seeks to understand how organizations sustained their work and how schools and districts continued to educate amid a pandemic (Onyema et al., 2020). Our findings suggested that specificity may be especially important in times of crisis. This makes intuitive sense. When you are in the middle of a lake and discover that your canoe has a hole, you do not need to spend time building buy-in or creating incentives to fix the problem; instead, clearly defined roles—in terms of who is to patch the hole, who is to scoop out the water, and who is to steer—could be the difference between sinking and making it to shore.

We believe that this study makes a meaningful contribution to the field in applying the policy attributes framework to understanding partnership work, with applications more broadly to many types of educational organizations. Further, we operationalize the policy attributes in ways that are easily replicable and hold promise for improving the systematicity across studies of partnership and alliance work.