Abstract
Although research alliances (RAs) have long been seen as mutualistic and reciprocal, RAs face numerous obstacles navigating stakeholders’ differing goals, incentives, and information. This longitudinal, comparative case study of two RAs uses principal-agent theory to analyze these interdependent challenges and their relationship to RA strategy and design. Findings suggest that while some RAs may be better designed to balance the competing interests of various stakeholders, increasingly contested definitions of RA effectiveness among those stakeholders have muddled RA identities. As a result, RA researchers are now often held to expectations that their organizations were not originally designed to meet. We argue that this has implications for how RAs are funded, designed, and, ultimately, evaluated.
O
The definition of RPPs has evolved since their formation. At their core, RPPs involve long-term collaborative relationships between an educational organization, such as a school or district, and researchers, aimed at investigating problems and solutions around school and district improvement (Coburn et al., 2013). Some suggest that for researchers, RPPs may increase the impact of their research, whereas for practitioners, RPPs may enhance the applicability of research to their needs (Farley-Ripple et al., 2018). More recent definitions propose an expanded role for RPPs, arguing partnerships should be focused on “equitable transformation” and should further be “organized to connect diverse forms of expertise and shift power relations in the research endeavor to ensure that all partners have a say in the joint work” (Farrell et al., 2021, p. 5). Others claim the purpose of RPPs remains contested and subject to the ideological divisions that characterize the education sector (Shirrell et al., in press). This evolving debate about the purpose of RPPs has implications for the expectations placed on RPPs and the resources and capacity necessary to meet those expectations.
Among RPPs, research alliances (RAs) are particularly prolific, comprising the majority of the membership of a national network of RPPs. 1 In most RAs, researchers work with a district or group of education organizations in a given region. Other forms of RPPs, such as those focused on improvement science (Bryk et al., 2015) or design-based implementation research (Fishman & Penuel, 2018), tend to feature more ongoing collaborative engagement and blended roles among researchers and practitioners (Penuel, Farrell, et al., 2020). However, in RAs, researchers and practitioners primarily collaborate at the outset and end of research projects, and largely retain traditional roles (Coburn et al., 2013; Penuel, Farrell, et al., 2020; Penuel, Reidy, et al., 2020).
While RAs are typically intended to promote improvements in practical knowledge, and possibly policy and practice, they face challenges that make improvements far from certain (Coburn & Penuel, 2016; Coburn et al., 2013; Farrell et al., 2021). These challenges include imbalanced authority (Coburn et al., 2008; Denner et al., 2019; Penuel et al., 2015), poorly designed incentives (Gamoran, 2023; Turkey & Stevens, 2015; Snow, 2015), lack of support for boundary-spanning activities (Penuel et al., 2015), and misalignment of partner expectations (Rosen, 2010). RAs are purposefully designed to mediate these challenges (Farrell et al., 2021). However, there has been insufficient attention to whether and how RA design and strategy might exacerbate or ameliorate challenges between contracting parties, leading some to explicitly call for research that investigates the consequences of variability in RPP design (Farrell et al., 2021).
In this article, we take up this charge by presenting results from a comparative case study of two RAs in large urban districts to show how variation in partnership design and strategy mediated partnership challenges over time. To do so, we use principal-agent theory to illuminate variation in partnership dynamics. Principal-agent theorists explore how relationships between contracting parties can foster alignment or misalignment between partners’ objectives, given imbalances of power and information. Here, we frame RAs as long-term principal-agent relationships between three stakeholder groups: researchers, district practitioners, and funders. We use principal-agent theory to surface the complexities of interdependent partnerships intended to overcome historically divergent objectives and incentives. To shed light on how RA designs can exacerbate or mitigate misalignment between contracting parties, this study addresses the following research questions:
How, and in what way, does goal alignment among RA stakeholders vary across partnerships, and how has this alignment or misalignment changed over time?
How does the interaction of stakeholder goals and RA design explain variation in principal-agent dynamics across RAs?
We find that while some RAs may be better designed to balance the competing interests of various stakeholders, increasingly contested definitions of RA effectiveness among those stakeholders have muddled RA identities. As a result, RA researchers are now often held to expectations that their organizations were not originally designed to meet. Overcoming this goal misalignment will require those within RAs and those who study them to reconsider, and perhaps temper, what it means for RAs to be successful so that the organizations’ aims are commensurate with their capacity and design. We argue that these findings have implications for how RAs are funded, designed, and evaluated.
Literature and Theoretical Frame
The Importance of RPP Goals and Design
Although recent scholarship has increased our understanding of the relationships that lie at the center of RAs and other RPPs, this work has raised questions about the implications of RPP goals and designs that remain unexplored. Consider, for example, recent work that has advanced our conceptual understanding of RPPs. Farley-Ripple et al. (2018) devised a framework for conceptualizing depth of research use and production in RPPs, as well as gaps in assumptions and perspectives between researchers and practitioners that may influence that depth. Their frame positions researchers and practitioners as distinct communities with their own goals, norms, and expectations and postulates that greater depth of research production and use are possible when the gaps between these communities are narrowed. Put otherwise, the frame assumes that partnerships promote greater depth of research production and use when there is greater alignment of goals, norms, and expectations among partners.
Research on RPPs points to considerable variation in partnership designs (Coburn & Penuel, 2016; Farrell et al., 2021), which we define as their goals, approaches to research, funding sources, and organizational structure (Farrell et al., 2021; Roderick et al., 2009; Thompson et al., 2017). Penuel et al. (2015) conceptualize RPPs as “joint work,” emphasizing the “boundary crossing” and “boundary practices” that are likely to lead to more productive partnerships. Specifically, they argue that partners in RPPs must move beyond a unidirectional research-to-practice model to one in which both partners intentionally engage in joint work that can bridge boundaries between researchers and practitioners. They show how funders might promote such activity, suggesting that “it is possible to build in structures and routines that provide sustained opportunities for boundary crossing” (Penuel et al., 2015, p. 192). In doing so, the authors imply questions about the structures and routines that promote more joint work within RPPs. RAs also vary in the number of researchers they employ and the kinds of research methods and questions they pursue. In addition, while many RAs have ties to local universities, the nature of these relationships varies, with some RA researchers jointly employed as tenure-track faculty and members of the alliance, and others employed solely by the RA (Shirrell et al., in press). RAs also differ in terms of their levels of funding and the role funders play in determining their agendas, as well as in the theories of action that motivate their vision of research use and organizational change (Tseng, 2017). Such variation in design mediates RAs’ interpretation of and vulnerability to various pressures in their environment and further influences their organizational goals and strategy (Shirrell et al., in press). For example, RAs with closer structural ties to universities may be more vulnerable to the incentive structures that govern academic institutions.
Finally, just as RAs vary across the dimensions mentioned above, so do their district partners differ in size, structure, and capacity. For example, some districts employ district research directors who can increase districts’ ability to learn from research, whereas others lack the means to staff such roles (Conaway, 2020). Another source of variation regards district staff who liaise with researchers, with some districts primarily designating program staff to communicate with the research organization, whereas others rely on the superintendent or other high-level officials to facilitate the relationship and spread of information. Districts, or district teams, with greater capacity to learn from RA research may require less from their research partners—greater absorptive capacity primes districts to learn within research partnerships (Farrell & Coburn, 2017; Glazer et al., 2023).
Scholars have just begun to analyze the tradeoffs associated with variation in RA design and its implications for stakeholders’ abilities to meet mutually beneficial goals (Welsh, 2021), leading to recent calls for research exploring “when, under what conditions, for whom, and with what tradeoffs RPPs make progress toward their goals” (Farrell et al., 2021, p. 30). To address this gap, we consider the alignment between the goals and capacities of research organizations on one hand and the goals, capacities, and incentives of practice partners and funders on the other. Given the complex web of relationships between researchers, practitioners, and funders in RAs (Coburn & Penuel, 2016; Farrell et al., 2021), the question of “alignment” is one that principal-agent theory is particularly well suited to explore.
The Principal-Agent Dilemma
Principal-agent theory arose in economics to describe relationships in which one party, the principal, contracts with another party, the agent, to perform a task or service the principal lacks the time or capacity to perform (Eisenhardt, 1989; Jensen & Meckling, 1976; Perrow, 1986). Canonical principal-agent theory assumes misalignment between the goals of principals and agents. For example, principal-agent theory has been used in education to frame discussions of performance pay for teachers—district and state agencies are positioned as principals leveraging monetary incentives to induce teachers (as agents) to achieve higher student outcomes (Levacic, 2009). In this and other classic principal-agent dilemmas, agents are assumed to be self-interested and are thus not naturally motivated to implement the principal’s goals as intended. Misalignment between principals and agents reduces the likelihood that agents will implement principals’ policy goals (Waterman & Meier, 1998; Wohlstetter et al., 2008) or organizational objectives (Van Puyvelde et al., 2012).
Principal-agent misalignment can be compounded by a host of interrelated challenges. For example, an imbalance of technical expertise, or “information asymmetry,” in the agent’s favor (Eisenhardt, 1989; Miller, 2005; Sobol, 2016; Waterman & Meier, 1998) and “adverse selection,” in which the agent lacks the capacity to meet the principal’s goals (Gailmard, 2014; Wohlstetter et al., 2008), can both exacerbate conflict between principals and agents. Given this assumed conflict, the principal seeks to align the agent’s goals with its own and prevent “agent shirking,” which occurs when the agent pursues its goals at the expense of the principal’s. Thus, the principal incurs the cost of incentivizing the agent “to take those actions that the principal would take if the principal had the same information as the agent” (Miller, 2005, p. 204). At its core, principal-agent theory emphasizes the importance of achieving goal alignment between principals and agents if the relationship is to efficiently and effectively accomplish its aims.
The addition of multiple principals can further exacerbate principal-agent dilemmas (Miller, 2005; Moe, 1987; Sobol, 2016; Waterman & Meier, 1998). In contracting with multiple principals, the agent “must contend with uncoordinated and often conflicting demands, requirements, and incentives” (Moe, 1984, p. 768). Agents might respond to competing demands by aligning their actions with some principals’ objectives at the expense of others or by shirking principals’ demands entirely (Worsham & Gatrell, 2005). Thus, while “pathological delegation” can be present in single-principal cases, it is a more prevalent threat in multiple principal-agent scenarios due to preference heterogeneity or power asymmetries among principals (Miller, 2005; Sobol, 2016). Education is rife with multiple principal dilemmas that result in pathological delegation: Schools (as agents) are accountable to various levels of government, parents, private funders, and other local community members, each of whom (as principals) may possess a distinct set of goals, beliefs, and priorities (Cohen & Spillane, 1992; Labaree, 1997). Goal conflict among principals forces school-level practitioner-agents to contend with multiple, conflicting demands and, in doing so, contributes to public education’s notoriously complex administrative structure.
Principal-agent theorists have increasingly recognized the joint challenges of pathological delegation and agent shirking. Just as principals must clearly communicate their objectives to agents and provide sufficient resources and guidance, agents must have adequate capacity and commitment to meet principals’ objectives. For this reason, mutual agreement around what the “agent should do and what the principal must do in return” (Perrow, 1986, p. 224) is crucial to maximize the outcomes of the principal-agent relationship.
Despite the potential for conflict, partners’ goals are not inherently misaligned (Miller, 2005; Waterman & Meier, 1998). Long-term principal-agent relationships are dynamic, and therefore “conflict varies according to both the level of information that principals and agents possess and the level of goal conflict” over time (Waterman & Meier, 1998, p. 197). For example, while information asymmetry may be considerable during the early stages of a principal-agent relationship, principals and agents can become more knowledgeable about one another’s goals and expertise over time. At the same time, principals and agents whose goals were initially well aligned may experience increasing goal conflict as expectations shift. Thus, issues like power imbalance, adverse selection, and pathological delegation are not fixed features of principal-agent relations, but analytic constructs that highlight ways in which the relational aspect of a partnership may vary. In other words, the power of the principal-agent frame lies not in a priori assumptions about conflict, but in its ability to help researchers analyze the dynamics of multidimensional relations that evolve in complex circumstances.
Applying Principal-Agent Theory to RAs
In education, principal-agent theory has been used to show how variation in organizational strategy and design mediates various principal-agent dilemmas (Bulkley et al., 2021; Wohlstetter et al., 2008). For example, Wohlstetter et al. (2008) found that district central office and charter network staff overcame principal-agent tensions with school-level agents by cocreating goals with school practitioners and ensuring school-level staff had sufficient training and decision rights to make on-the-ground decisions.
In the context of RAs, we conceptualize researchers as agents accountable to two primary principals: their partner district and their funders. These principals have various goals for research but lack the time or capacity to conduct research themselves. Thus, the principals delegate the task of research production to the researcher-agents whose goals may or may not align with those of principals. By applying principal-agent theory, we neither assume a clear demarcation of researcher, practitioner, and funder roles nor do we aim to discount processes that involve the coconstruction and cointerpretation of knowledge. Indeed, principal-agent theory suggests that successful principal-agent relationships may explicitly pursue cooperative approaches (Miller, 2005; Voorn et al., 2019). We do assume, however, that RAs are rooted in a complex web of relations among institutional actors that have historically struggled to work together productively, and that overcoming principal-agent dilemmas may be key to RA success.
Consider, for example, the question of who represents the principal in an RA. On the one hand, one might see districts as the primary principals, because they offer researchers access to district data and resources in exchange for research that will serve their organizational needs. However, as districts are typically nonpaying clients, RAs receive most, if not all, of their funding from outside funding agencies. In this formulation, funders are the second primary principals in a RA and have their own goals, which may or may not align with the district’s goals. This, in turn, raises the question as to whether there are ways to design RAs to be less vulnerable to conflicting agendas pursued by separate principals.
Principal-agent theory enables a conceptualization of the multidimensional nature of the relationship between researchers, practitioners, and funders and, in doing so, highlights five key challenges that can arise in principal-agent relationships (see Figure 1): divergent objectives, pathological delegation, information asymmetry, adverse selection, and weak incentives. The following sections discuss the application of these challenges to RAs.
Hypothesized relationships between principal and agents in research alliances.
Divergent Objectives
While all stakeholders in a RA may ascribe to the broad goal of improving student outcomes within the local district, this is only one of many competing priorities among researchers, practitioners, and funders (Bickel & Hattrup, 1995; Coburn et al., 2008, 2013). Typically, RAs establish formal agreements at the outset of a partnership to specify each stakeholder group’s responsibilities, how the partnership will be governed, and how and when partnership data can be used (Penuel & Gallagher, 2017). While these contracts can support greater alignment between principals and agents (Henrick et al., 2017), there is no guarantee they will obviate the challenges of mutualism and goal alignment.
Pathological Delegation
The existence of multiple principals and the complex nature of educational work suggest that RAs are vulnerable to pathological delegation, which occurs when principals fail to send clear or beneficial signals to agents (Sobol, 2016). In addition to juggling competing goals among practitioners and funders writ large, researchers may receive contradictory signals from different subunits or departments within the district, or from different funding organizations, making it difficult to identify and respond to these demands (Farrell & Coburn, 2017; Penuel et al., 2015; Spillane, 1998). Further, districts, district units, or funders with clearly identified priorities may be unable to communicate how researchers can effectively contribute to those priorities. Thus, while principals and agents may agree on a broad course of action, those actions may not be sufficient to support principals’ goals.
Information Asymmetry
RAs are marked by dual information asymmetries (Henrick, et al., 2017), or imbalances of information between principals and agents. Researchers are trained in the technical elements of research and further hold expertise in substantive areas, which may or may not match the substantive areas that are of highest priority to their local district (Glazer et al., 2023). District practitioners have expertise borne from practical experience and are likely to have better knowledge of district structure and political dynamics. At the same time, information is likely to be asymmetrically distributed among district practitioners and funders who may not be fully aware of each other’s motivations and behavior. This asymmetrical distribution of information among researchers, district practitioners, and funders can constrain their ability to operate collectively and exacerbate power imbalances among principals (Moe, 2005; Voorn et al., 2019).
Adverse Selection
There are many well-known challenges that point to the salience of adverse selection, which occurs when agents lack the capacity or will to meet principals’ goals, within RAs. District practitioners often operate with an urgency that research timelines may be ill-equipped to meet (Coburn et al., 2013; Farley-Ripple et al., 2018). Second, the conclusions that researchers provide may not be immediately actionable to practitioners (Farley-Ripple et al., 2018; Glazer et al., 2023). Finally, RAs, like all RPPs, require both primary partners to take on some responsibilities outside their traditional roles (Penuel & Gallagher, 2017).
Weak Incentives
RAs are characterized by a complex incentive structure. While researchers in RAs may be intrinsically motivated to improve district conditions, academia’s formal incentive structure does little to encourage participation in these partnerships (Gamoran, 2023; Shirrell et al., in press). Further, while district practitioners control access to data and personnel necessary for RA research, the fact that districts are often nonpaying entities means that RAs must seek external funding. If funders’ and districts’ objectives are not aligned, researchers may have little choice but to prioritize the goals of those who hold the purse strings.
Each of these challenges has, to some extent, been addressed individually in the existing literature on RPPs. By reframing the obstacles facing RPPs within principal-agent theory, we cast the above-described challenges as an interdependent set of problems that RPPs must be designed to overcome if all stakeholders are to judge the endeavor “successful” in meeting its goals. In the remainder of this article, we compare the efforts of two RAs struggling to respond to the goals of researchers, practitioners, and funders in organizations that were designed to walk a tightrope between competing demands.
Data and Analytic Methods
Data for this study come from a larger 4-year qualitative cross-case study of whether and how research organizations in RAs build district capacity to use research knowledge. 2 Substantial changes in both research sites necessitated an adaptive approach to data collection and analysis, which we describe below.
Sample
Our sample includes stakeholders in two RAs operating in large, urban districts: university research team (URT) and district research collaborative (DRC; pseudonyms). These RAs were purposefully sampled to maximize variation on numerous dimensions, including organizational design, resources, and local context, while ensuring comparability by limiting the sample to a single type of RPP. 3 Our sampling strategy aimed to gather perspectives from multiple stakeholder groups salient to each partnership, including researchers, funders, district leaders, board members, and other organizations that interacted with each RA. Ultimately, we conducted 107 semi-structured interviews with a purposefully selected sample of district personnel, researchers, funders, and board members in each study site. 4
To examine change over time, we collected data over a 4-year period (see Supplementary Table A1 in the online version of the journal) and had key informants reflect on previous partnership activities that occurred outside of this period. For example, while the foundation of both partnerships predated data collection for the present study, we conducted multiple interviews with current and former leaders in both partnerships focused on the activities and goals during partnership formation. In addition, key informants in both cases were interviewed at multiple points, contributing to our understanding of evolving conditions at both sites. This enabled us to construct a longitudinal account of key stakeholders’ shifting goals and perceptions of the extent to which each partnership was achieving those goals, alongside key aspects of each RA’s design over the duration of both partnerships. Thus, our claims throughout this discussion focus on the evolution of both partnerships over more than a decade, while data collection occurred during a 4-year period. One limitation of this approach is our reliance on respondents’ recall, especially in recounting the goals and conditions during the foundational years of both partnerships.
Data Collection
Data collection began in Summer 2015 and continued through Fall 2019. We worked with key stakeholders in each RA to identify a collection of recently launched research projects to serve as the focus for our data collection efforts. These projects, described in Table 1, represented a variety of topics and methodological approaches, providing a broad range of data through which to analyze each RA’s approach.
Research Projects Sampled Across Cases
Note. These projects were selected with guidance from researchers and district staff as the most promising examples from their broader portfolio of collective work. DRC = district research collaborative; URT = university research team.
Across the above projects, we conducted semistructured interviews, observations of RPP meetings and dissemination events, and document analysis of internal memos and research projects produced by the two RAs. Interview protocols were developed at the outset of the study and then revised in response to interim findings and evolving conditions in each site. Early interviews focused on partnership dynamics, in addition to identifying characteristics of the districts and research organizations. Follow-up interviews utilized RA publications as memory devices and focusing instruments (Saldaña, 2012). This enabled us to collect data on the processes through which research topics were determined, the dynamics of interpretation around those studies, and the ends to which they were used.
In the third year of data collection, we found that dynamics between researchers and district leaders had changed precipitously for both RAs, necessitating a frame that would help us understand these new and more fraught relational dynamics. Responding to these interim findings, we revised later interview protocols to explore sources of conflict and misalignment among RA members, including questions about competing incentives, information flow between partners, and power dynamics during goal setting and organizational strategizing. We were particularly attuned to distinguishing between funders’ and district practitioners’ goals for researchers, which had implications for the present analysis.
In addition, we observed meetings and dissemination events at each RA. Observers focused on issues such as the balance of authority in meetings, how RA goals and agendas were established, structures and opportunities for participants to share information across organizational boundaries, and evidence of efforts to increase alignment between stakeholders or lack thereof. Observation data were primarily used to confirm or disconfirm interim conclusions about partnership dynamics and processes gleaned from interviews. These data were influential in our analysis of principals’ and agents’ goals within DRC. For example, district personnel in DRC described increasing misalignment with their research partners, which we confirmed by observing tensions between researchers and district leaders in RA meetings. Following observations, we conducted interviews with key participants from those meetings to reflect on the conversations and obtain further insight into their takeaways.
Finally, we reviewed more than a dozen research documents produced by each RA. These documents provided data that informed our interview protocols and enabled us to triangulate between research reports, what researchers saw as key takeaways, and what district personnel said they learned from the research. For example, when a major research report for one RA listed a study aim as providing information that would help practitioners “continuously improve the initiative over time,” we questioned both researchers and district practitioners about their perceptions of the centrality of that aim to the overall research, the strategies they had used or observed to reach that aim, and the extent to which they would judge that aim a success of the project. Thus, while the present analysis draws exclusively on interview data, many interviews were directly informed by data gleaned from observations and document analysis.
Analytic Strategy
Analytic procedures included an iterative process of code development, coding, memo writing, and member checking that occurred concurrently with data collection to ensure that our evolving understanding of our cases informed our data collection procedures. In the initial phase, our coding scheme was conceptually tied to key dimensions of the absorptive capacity frame, which informed the early phases of data collection and analysis. 5 However, when our analyses revealed the need for a theoretical frame that explained the interplay of authority and incentives, we engaged in a final round of coding focused specifically on pattern matching for processes and outcomes related to principal-agent theory (Yin, 2018). Specifically, we approached our data with a tentative theoretical explanation—RAs that are designed to overcome the challenges of principal-agent theory are more likely to be perceived as “successful” by principals and agents. For this phase of analysis, we considered all excerpts from codes in our initial coding scheme that were related to partners’ goals, information exchange, partner skills and capacity, and incentives, and reanalyzed the data through the lens of principal-agent theory (see Supplementary Table A2 in the online version of the journal).
We also developed an analytic process to explore variation within and across our two sites over time. Specifically, we constructed a multidimensional spreadsheet for each RA that cross referenced claims, stakeholders, projects, and years. This allowed us to visualize data corresponding to differing role groups (e.g., funders, district staff), and differences in how each stakeholder group perceived the RAs in the context of various projects over time. Once we completed that process for both RAs, we constructed a set of analytic memos that explicitly compared the evolution of empirically corroborated themes among the two cases. For example, one memo focused on how each primary group of stakeholders discussed information flow within each partnership over time, highlighting variation within and across research projects in each case.
Key to our analysis was determining how to evaluate both RAs’ outcomes. Such an evaluation needed to account for various stakeholder groups’ goals for their partnerships and the evolution of those goals over time. Rather than employ our own inherently ideological definition of partnership effectiveness, we instead honored the perspectives of those we interviewed. That is, at various stages in each partnership’s history, we identified principals’ and agents’ goals for their partnerships, as well as the extent to which each group of stakeholders perceived that the RA was meeting those goals. While this approach precludes a singular definition of success, it is aligned with our theoretical frame and the empirical complexity of our data.
Throughout the phases described above, we engaged in multiple meetings to ensure interrater reliability. Early meetings focused on codebook creation, including establishing definitions and sample excerpts for each code, to ensure team members held common understandings of all codes. Subsequent meetings focused on code application. All coders were provided the same 20 initial excerpts within each principal-agent construct (e.g., adverse selection) to code independently. We then met to discuss areas of alignment or misalignment around code application. Once coders were sufficiently aligned, remaining excerpts were divided among team members for further analysis. Each excerpt was tagged with an appropriate subcode (see Supplementary Table A2 in the online version of the journal) and briefly summarized to capture the main idea within the excerpt. The team then met to discuss common patterns and themes across main ideas within each set of subcodes.
Finally, as a team, we regularly met to grapple with disconfirming evidence. For example, the increasing divergence among principals’ and agents’ goals, as described in further detail below, was particularly stark in the case of DRC and largely aligned with broader shifts we had observed within the RPP field (Shirrell et al., in press). However, while we had initially assumed we would find a similar pattern in the case of URT, closer examination of our data revealed a far more subtle change around the emphasis that principals placed on various goals, and an RA more adept, but still not entirely successful, at managing these subtle shifts. Such nuance required us to seek further evidence to explain the variation we observed within each case.
Member checking occurred throughout the data collection and analysis process. Informal member checking occurred during follow-up interviews with key informants in both study sites, during which informants were given opportunities to confirm, disconfirm, or provide further nuance around their own or others’ perceptions of RA activities and outcomes. For example, researchers in one RA described themselves as “radiologists” rather than “surgeons,” perceiving their goals as identifying problems within the partner district rather than solving them. In later interviews, we presented this dichotomy to multiple stakeholders in both RAs to get a sense of which role or roles they expected researchers to play in their respective partnerships. This enabled us to better understand variation in principals’ and agents’ expectations for researchers in both cases. In addition, formal member checking occurred after data collection was completed. Interim conclusions were shared in separate meetings with key members of both RAs and district officials, who largely confirmed our interpretation of the data and provided feedback that further refined our analysis.
Study Context
The research organizations involved in both RAs have existed as formal organizations for over 10 years, have each produced over 30 publi-cations, and maintain longitudinal administrative databases that support ongoing research. Despite these similarities, there were several key differences in the design of both RAs that led to important variation for our study.
Of the two RAs, DRC was a leaner organi-zation, and nearly all of its few employees were accountable to both DRC and to the university by whom they were employed as tenure-track faculty. Further, DRC had a leaner budget, relying primarily on small grants from local foundations. Reports on its website, which were almost exclusively statistical analyses of district administrative data, were simple and organized chronologically with little to connect the information or findings across reports (Freed, 2020). DRC had a strained relationship with the local city and district, in part because of their association with a university system that many local stakeholders viewed as historically oppressive and untrustworthy (Shirrell et al., in press).
By contrast, URT researchers and staff were employed solely by URT and vastly outnumbered those in DRC. The organization had a leadership team and researchers who specialized in a range of methodological approaches and substantive areas in education research. URT’s robust staff were supported by grants from numerous large, national foundations. Again, this additional staffing and capacity were further evident in the organi-zation’s website and reports. URT’s reports were organized thematically suggesting a more coherent research agenda and work that had the potential to build knowledge in key domains. URT was a well-respected institution in the local city and district with a strong reputation for research and engagement.
While both partner districts were large, there were also key differences between them. URT’s district partner experienced relative stability during our study; the district had two superintendents during our investigation and the second was seen as largely furthering the vision of the first (who had led the district for over 4 years). In addition, URT was partnered with a well-resourced district that was further viewed as a national leader in educational improvement and reform. By contrast, DRC’s district partner underwent four leadership transitions over a 6-year period, as well as substantial turnover among mid-level managers. This constant churn resulted in rapidly changing district goals and strategies for improvement—both DRC researchers and our own team felt as though we were studying a moving target. Demographically, while both districts primarily serve low-income students of color, DRC’s district was characterized by more multigenerational poverty and increasing racial tensions during our study. Student performance in DRC’s district was far below that of most other U.S. urban districts. This poor performance combined with a history of political scandal and mismanagement in the city contributed to considerable distrust of education and civic leaders. These contextual differences proved important to understanding the alignment (or misalignment) between the goals of principals and agents, as described in the following section.
Findings
In the sections that follow, we present data from our cases in support of two primary arguments. First, we argue that as multiple principal-agent relationships, both RAs were subject to divergent goals, particularly around knowledge use. Further, in the case of DRC, stakeholders’ goals became increasingly divergent, altering the expectations for DRC’s researcher-agents. Second, we argue that URT was better designed and resourced to respond to its multiple, conflicting goals and overcome many principal-agent challenges, while DRC lacked the information flow, capacity, and resources necessary to address these multiple demands. Ultimately, however, the core challenge of divergent goals was never resolved in either RA, raising questions about the alignment between RAs, as an organizational form, and the constellation of principals that confront them with competing demands.
Multiple Principals, Multiple Demands
Our first research question focused on whether and how stakeholders in both partnerships negotiated goal alignment, and variation in that alignment over time. In this section, we trace the evolution of principals’ and agents’ expectations for their respective RAs during key points over the first 10 to 12 years of each partnership. We further describe the implications of these shifting expectations for stakeholders’ perceptions of each RA’s “success.”
DRC: Initial Conflict Comes Before Increasing Divergence
DRC’s principals and agents maintained competing goals around research use from the time of the RA’s establishment. Researchers, funders, and district practitioners were able to strike an initial détente to secure district buy-in and launch the partnership. However, this goal alignment was not sustained. Over the partnership’s 12-year history, goals among DRC stakeholders continued to diverge, leaving researcher-agents struggling to satisfy increasingly contradictory demands.
During the first 6 years of the partnership, DRC’s primary principals and researcher-agents agreed upon the value of rigorous, objective research, but they maintained divergent goals for research use. As many recounted, funders’ and researchers’ main objective in establishing DRC was to produce research that would increase district accountability and, in the long run, improve student outcomes. For example, DRC’s founding researcher shared that the organization “was founded . . . to be an independent evaluator, or a voice talking about the performance of the district’s schools and kids,” since local foundations “didn’t believe any of the public numbers that were being produced” by the district. Similarly, a funder who had been involved in the partnership from the beginning described their initial belief that DRC would act as “an independent research consortium that would . . . broker honest data for the community.” Both principals and agents, then, valued DRC’s role as a trustworthy arbiter of district data.
Further, while both researchers and funders hoped DRC’s research would lead to improvements in district performance, they agreed that it was not the researchers who were ultimately responsible for those improvements, or even for ensuring that the research was actually put to use. As one funder explicitly summarized, it was not researchers’ “role to tell the district what to do with the numbers, but to clearly indicate what the numbers mean and what they represent.” Similarly, prior to launching the partnership, DRC’s founding researcher believed their team would ultimately “put information in the hands of some angry but strategic change agents, give them some tables and charts; they’ll . . . stir up some righteous indignation, and we’ll get some change.” Funders and researchers both subscribed to the notion that DRC’s researcher-agents’ role was to produce objective research that others could use to catalyze district improvement. Put a little differently, DRC’s job was to provide the grist, but not to operate the mill.
The other principal in DRC’s orbit—the school district—similarly saw objective research as important. District leaders during the partnership’s first 6 years hoped an RA would produce research that would help them “get smarter” about their work. These leaders believed it was researchers’ responsibility to produce research and the district’s responsibility to determine how to use that research. Note the comment of the superintendent at the time of DRC’s establishment:
Whatever solutions emerge, they’re not going to come from research. They’re going to come from [district practitioners] who are wrestling with a problem. But the people wrestling with a problem are going to need to know exactly what the problem is.
District leaders saw value in research and were aligned with researchers and funders in the belief that researchers should only be responsible for research production, not its use.
However, unlike funders and researchers, district staff during this time saw district problem-solving, not external accountability, as the goal of that research. No district staff in our sample expressed a desire that the public should be informed of potential problems within the district, and some were explicitly wary of the scrutiny RA research might bring. One high-level district leader stated that giving a “completely autonomous” research organization access to district data was “really scary” because that research could reflect negatively on the district. The district intended to use DRC’s research for its own purposes and did not wish to be held accountable to external change agents, as researchers and funders envisioned.
The district held considerable power during this phase of principal-agent negotiations as both funders’ and researchers’ goals were dependent on the district’s participation. Thus, funder-principals and researcher-agents were willing to compromise and temporarily prioritize the district’s goals over their own. As DRC’s founder shared, researchers believed they would
start with some respectable, worthwhile, but not especially edgy or dangerous projects. And once we get this thing launched, we’ll get a little more impactful . . . We’ll challenge the status quo through our results or through the research questions we’re posing.
Funders similarly acknowledged that once they secured district involvement, their ultimate goals were “getting the district to use research projects” and “having an impact on student achievement.” Thus, while principals and agents initially aligned their aims around a common goal for the purpose of establishing DRC, it is not surprising that their goals diverged as the partnership evolved.
Throughout DRC’s existence, researcher-agents’ goals remained consistent with their founding vision for the partnership. Twelve years after the partnership was founded, researchers remained primarily focused on producing research that would inform outside stakeholders of district conditions and provide potentially useful, but in many cases unflattering, information to district managers. For example, in the partnership’s 12th year, DRC’s director, who came to the helm 8 years earlier, shared their belief that “if I ever walk into [the district central office] and someone hugs me, I’m not doing my job. My job is to push them to see things they don’t want to see.” This leader struck a similar tone to DRC’s founding researcher, who launched the partnership primarily to “challenge the status quo.”
From DRC leaders’ perspective, the history of poor student outcomes and system-level mismanagement required “radical” change, and “an urban school district was never going to be that radical.” Instead, such change would require considerable external pressure for reform. While this perspective is incommensurate with the collaborative approach championed in much of the RPP literature, the belief in accountability and external pressure as powerful levers on change is not without supporters in the larger education sector. Indeed, it is central to most market-based reforms. From this perspective, working too closely with the district would compromise researchers’ independence and objectivity, undermining the partnership’s integrity.
As tenure-track faculty, DRC’s researchers were also strongly incentivized to tailor their research to wider academic audiences. One DRC researcher suggested that district improvement would be an insufficient goal for RA research, instead noting that DRC’s research would not be considered “successful until it helps me get further funding to continue the research, and until it results in some publications.” While this researcher acknowledged that it was gratifying to see that people in the district were reading and talking about their work, this alone was not the mark of a “successful” research project. Importantly, researchers’ goals as tenure-track academics largely complemented their intention to produce research that would press the district to improve: DRC researchers published numerous articles in well-respected academic journals highlighting problems within their local district.
However, while researchers’ goals remained consistent, by the partnership’s 9th year, district practitioners’ and funders’ goals had changed. Specifically, both principals now believed DRC researchers should be responsible for district leaders’ interpretation and use of partnership research rather than solely be responsible for the production of partnership research. A constellation of factors likely contributed to this evolution in principals’ expectations. In part, shifts within the partnership may have reflected a broader evolution in the larger RPP environment (Shirrell et al., in press). Beginning roughly in 2016, national funders, the federal Department of Education, and education researchers began seeking evidence demonstrating the impact of RPPs on policy, practice, and student outcomes. IES Director Mark Schneider (2018) declared that “RPPs are mostly hope-based. We can’t continue to bet tens of millions of dollars each year on RPPs without a better sense of what they are doing, what they are accomplishing, and what factors are associated with their success.” Around the same time, the W. T. Grant Foundation published a rubric for assessing RPP effectiveness suggesting one dimension of partnership effectiveness was the degree to which researchers supported “the partner practice organization in achieving its goals” (Henrick et al., 2017, p. 11) by helping practitioners identify improvement strategies and directly informing the implementation of those strategies. Academics also increasingly drew attention to partnerships that “. . . engag(ed) in principled efforts to change educational systems, how research is conducted, and researchers’ relationships with educators, families, and community members” (Penuel, Riedy, et al., 2020, p. 631).
Principals’ shifting expectations were also connected to more concrete changes internal to the partnership. Changes in district leadership contributed to a constant evolution in the district’s goals for and relationship with their research partners. Most significantly, the district leadership team that arose 10 years after DRC’s founding dramatically reflected the larger environmental shift in expectations for RPPs. One district leader described how the district now “wanted to make sure we were really getting practical use out of [DRC] research, and that it wasn’t just research for research sake.” Unlike the initial superintendent who believed it was the district’s job to devise solutions to problems uncovered in DRC’s research, district officials in the new regime sought a research partner that could determine a high-leverage research agenda and devise solutions to key problems. As the new district superintendent described, “If you’ve got all this time to look at our system . . . I expect a little bit more around you being able to identify what the key lever [for improvement] is.” This superintendent maintained the same goal as the original superintendent—learning from DRC research—yet they now sought a partner that would directly support district learning, rather than simply present information.
At the same time, driven both by the new district leader’s insistence that DRC was not serving its needs and by the evidence (from DRC’s own research) that district outcomes were not improving, funders increasingly pressed researchers to take a more direct approach. For example, some local funders were present in DRC meetings in which the new district superintendent repeatedly criticized DRC’s effectiveness in supporting district learning. One such funder, who had been integral to DRC’s founding and development, expressed “surprise” at the district’s criticism. However, after hearing the new superintendent’s criticism of DRC’s more traditional approach to research, this funder, who had previously been content with DRC’s role as a “data broker,” began calling for DRC to “really work with the school district to have something happen.” Concurrently, other funders began expressing their own concerns that DRC’s research was not impacting district decision-making. One funder during this stage noted that it was not enough to “just produce research”; there had to be “some advocacy around it.” Still another expressed doubts around DRC’s capacity to support district improvement, asking if other research partners might be “more appropriate” given the district’s needs.
Thus, as a result of changes both internal and external to the partnership (summarized in Table 2), DRC researchers were increasingly expected to demonstrate their direct impact on district outcomes. Researcher-agents faced growing pressure from both principals to, in the words of one funder, “get in the weeds” to support district improvement. This increasing misalignment, or goal divergence, between principals and agents raised new questions about whether researchers had the capacity and willingness to meet evolving demands.
DRC’s Goals and Major Contextual Changes During Key Points in the Partnership’s History
Note. DRC = district research collaborative; RA = research alliance; RPP = research-practice partnership.
URT: Agreeing to Disagree
In a marked contrast to DRC, URT was founded with explicit district cooperation from the outset. A URT leader described the district superintendent’s integral role in forming the partnership, saying his attitude was always to “figure out how to make [URT] work.” In the view of this leader, the district superintendent’s enthusiasm meant that URT did not have to “establish a relationship” with the district because this relationship “was a given.” Unlike the case of DRC, cooperation among principals and agents during URT’s establishment created a balance of power among key stakeholders and their various aims for the partnership. Further, it enabled principals and agents to coconstruct an initial contract and organizational design that attended to each group’s goals despite considerable divergence between those aims.
URT researchers and founding funders sought to influence educational policy and practice in their city and, ultimately, improve student outcomes. Not only was this a formal goal, codified in the organization’s mission, but it was espoused by nearly every URT staff member we interviewed. One high-level URT researcher, for example, described approaching possible projects by asking, “What are the problems [the district is] trying to solve every day . . . and how can we produce something that . . . helps them do their jobs more effectively?” As another URT staff member summarized, “we genuinely hope our work is making the lives of people who are doing this every day easier.” Because URT researchers were solely employed by the research organization, rather than the local university, as was the case for DRC’s researchers, their professional goals and incentives were more directly aligned with those of the district. Thus, from its inception, URT’s organizational goals and identity were more closely tied to supporting district improvement than those of DRC.
However, influencing district policy and practice was not funders’ or researchers’ only goal. One funder, for example, shared that their foundation had invested substantially in district initiatives and saw value in establishing “an evidence base that assessed the impact of those reforms and investments.” Numerous funders that preconditioned monetary support for district initiatives on external evaluations depended on URT to evaluate the impacts of those initiatives regardless of whether district officials were interested in impact evaluations. URT researchers, like their DRC counterparts, also aspired to reach beyond the local context. A URT leader noted that there was “a public informing responsibility of these research practice partnerships,” and that part of URT’s mission was to inform community organizations, local universities, and other researchers and policymakers throughout the country. Similar to DRC, funders and researchers saw informing stakeholders outside the district as a central goal.
Finally, while some district personnel saw URT research as a means of supporting district improvement, others viewed URT as an opportunity to enhance district legitimacy. An individual who was close with the district superintendent noted that the superintendent believed establishing an RA would signal that “the work [the district] had underway was informed by a research base.” Similarly, one of the original funders described how the perceived success of another RA in a large, urban district suggested that “if you were going to be taken seriously as a major urban school district, [a research-practice partnership] was one of the ingredients that was necessary.” Another respondent shared that the district superintendent recognized the benefits of establishing an “evidence base for how successful [their] efforts [to reform the district] had been.”
As was the case in DRC, URT stakeholders articulated divergent goals for research use. Some sought legitimacy, others district improvement, some hoped to inform the public discourse, and others wanted to assess the impacts of their investments. Then there were those who hoped to achieve some combination of these goals. In contrast to DRC, however, URT’s principals and agents worked collaboratively to ensure each of these goals was, to varying degrees, built into the contract launching URT and governing its initial design. In the year before URT was established, key stakeholders, including the district superintendent, the head of the local teacher’s union, several major university presidents, researchers, and representatives for local foundations, regularly met to discuss these goals and devise the contract that would ultimately govern URT’s work. Through these meetings, principals and agents co-designed an organization that would explicitly attend to multiple goals.
Roughly 8 years into the partnership, however, it became clearer that these were not simply theoretical distinctions, but practical tensions that surfaced regularly in URT’s work. For example, researchers and district staff frequently described challenges aligning principals and agents around the scope and methodological approach of URT studies. As one high-level district informant described:
Some people wanted there to be a nice, shiny report that would show what a great job we [the program team] were doing. We [the program team] wanted information that would help us inform the program, but that requires much more on-the-ground, quick tactile, short-term formative evaluation than we really commissioned [URT] to do. Then other people wanted longitudinal data that would inform [the national knowledge base about this topic] . . . there was a triangle among those three things, and we were always not quite meeting any one of those goals because we were pulling [URT] in a bit of a tug-of-war between them.
Other practitioners perceived efforts to produce URT research that bolstered district legitimacy or contributed to public knowledge as limiting the district’s ability to learn from that research. One high-level district official described how competing goals influenced the research questions in URT’s study of computer science. Rather than pursue questions geared toward learning, such as “does instruction around computer science change how students learn or forward their learning in general?,” district leaders, URT, and funders elected instead to pose simpler questions about access that were more likely to shine a positive light on the program (e.g., the number and type of students accessing computer science courses). This leader acknowledged that while these access-oriented questions attenuated the potential for uncovering more impactful issues relating to teaching and learning, the district had “eyes on our funders too, and we want them to be happy and think we’re doing well.” A study of computer science access was one in which “the results were more likely to be positive.”
Thus, while learning and legitimacy are not necessarily mutually exclusive, they can be perceived as conflicting. In the current case, the public-facing nature of URT’s research and the district’s desire to maintain legitimacy ultimately overpowered the goal of district learning and improvement. Speaking more generally, another high-ranking district official noted that the publication of URT’s research changed how district staff responded to that research. They were less focused on learning and more focused on “defense and figuring out how this is going to play in the public and how we respond to it.” Thus, in the view of many district practitioners in particular, the multiple goals among principals and agents in URT were not only different but explicitly contradictory.
While URT’s research was respected by nearly all principals, researcher-agents increasingly described pressure from funder-principals to attend more closely to the goal of improving district outcomes. For example, in the partnership’s 10th year, URT’s leader perceived funders were increasingly interested in whether URT was “making a difference in the world,” further noting that URT’s funding was growing more dependent on their evidence of impact, or the extent to which URT research influenced district decisions and policies. URT researcher-agents struggled with how to balance competing demands as they were increasingly pressed to demonstrate they were effectively meeting the many divergent goals around which they were established. This struggle complicated URT’s work and, at times, strained their organizational identity. As one district informant acknowledged, it “puts [URT] in a tough position.”
As summarized in Table 3, while the dramatic shift in partnership goals and realignment of power dynamics that characterized DRC’s experience did not transpire in URT, it, too, was forced to contend with divergent goals. As we describe in further detail below, the proliferation of competing demands contributed to a perception that URT had, as one funder described in the partnership’s 10th year, “not yet fulfilled its full potential.”
URT’s Goals and Major Contextual Changes During Key Points in the Partnership’s History
Note. URT = university research team; RA = research alliance; RPP = research-practice partnership.
Demands Meet Designs: Implications for Principal-Agent Dynamics
As described above, both RAs were ultimately subject to divergent goals among funders, district personnel, and researcher-agents. The organizational designs of the two RAs, however, shaped their responses to those conflicting demands. In the following section, we describe how URT’s design better enabled the partnership to navigate principal-agent tensions, while DRC’s design provided fewer communication structures and insufficient capacity and incentives to ameliorate principals’ and agents’ divergent goals. However, despite URT’s numerous advantages in organizational design and capacity, the organization struggled to contend with the core challenge of divergent and, ultimately, contradictory demands.
DRC: Limited Capacity Constrained Organizational Evolution
DRC was never intended and, thus, not designed to serve multiple goals; instead, the organization reflected an early agreement among stakeholders on the goal of producing rigorous, objective research. As principals’ and agents’ goals increasingly diverged, shortcomings in DRC’s design that hindered its ability to overcome this divergence (and related principal-agent challenges) grew more evident. As a result, researcher-agents increasingly struggled to manage competing demands and convince both groups of principals of their effectiveness.
One problem involved DRC’s struggle to maintain adequate information flow with its principals, which constrained DRC’s ability to negotiate a mutually satisfactory research agenda. DRC was designed with two primary structures to support information flow with the district. First, quarterly executive committee meetings between DRC staff and top district administrators were intended to ensure a research agenda tied to district interests. During the first 6 years of the partnership, these meetings were seen as effective. The most recent superintendent reflected that the effectiveness of early executive committee meetings may have, in part, hinged on the district superintendent at the time, who was a “directive” leader whose “theory of action was well underway,” allowing them to be “clear what [the district] wanted [DRC] to do.” Second, a series of informal meetings connected researchers with mid-level district staff. While these meetings remained “informal” and district attendance was spotty, they provided an opportunity for researchers to share research findings with district personnel interested in learning from DRC’s research, and to maintain a degree of alignment between DRC researchers and the district.
As funder and district expectations shifted, however, periodic and informal communication structures proved inadequate to enable the more continuous and intense dialogue needed to broker a new shared understanding of DRC’s role. Researchers used these structures to disseminate findings, but they provided limited opportunities to develop shared interpretations of DRC research and collectively grapple with new ideas as principals’ goals for the partnership evolved. Further, these relatively ad hoc communication structures made it difficult for DRC to stay abreast of district priorities as they shifted, and as personnel turned over. As one district stakeholder described, DRC did “not necessarily have clear marching orders on the research agenda for the district” given “changing leadership” and constantly shifting priorities. In other words, lack of consistent information flow left the partnership subject to pathological delegation in the face of tumultuous district conditions.
DRC attempted to improve information flow by creating new structures to increase district researcher communication. For example, a new internal district committee, co-led by the head of DRC and high-level district officials, enabled both groups to coconstruct DRC’s research agenda. As DRC’s leader explained, this structure was intended to ensure that researchers would “better understand what [the district’s] needs are.” Further, meetings of the internal committee focused on both sharing DRC research findings and discussing how the district could apply those findings in practice. These later meetings sought to better balance the power between principals and agents in setting DRC’s research agenda and created additional opportunities for district personnel to grapple with research findings after research was conducted.
However, while structural changes supported stronger information flow, these efforts proved insufficient to fully align principals’ and agents’ goals, in large part because they did not address researchers’ lack of capacity to meet those goals. The research organization included a small team of researchers with advanced methodological training but limited expertise in district improvement. District personnel and funders frequently referred to DRC’s technical strengths when describing the researchers’ capacity, citing researchers’ skills in research design, impact evaluation, data analysis, and fund-raising. DRC researchers similarly saw their strengths as conducting rigorous research instead of applying that research to district policy and practice. One DRC researcher, for example, described DRC researchers as “radiologists” trained in diagnosing problems, as opposed to the “surgeons” who fix those problems—a distinction that resonated with every DRC researcher we interviewed. That is, not only did DRC researchers lack the capacity to meet these new expectations, but they also viewed these goals as incompatible, or at least in tension, with their own goals as researchers. As one researcher shared, DRC researchers felt uncomfortable “thinking of [themselves] as advocates . . . but [are comfortable] just serving in a research role, conducting the types of studies that professor types like to do.”
When both principals and agents focused their goals solely on research production, DRC researchers possessed sufficient capacity to achieve those goals. However, as principals’ goals shifted to focus on research impact within the district and direct support for district improvement, many principals began seeking research that was “actionable.” As one district informant noted, such research required “a very different type of research expertise” and “a different set of skills versus what we’ve been accustomed to having [DRC] do.” Similarly, the superintendent in DRC’s later years perceived that “the actual core of what [DRC researchers] are able to do does not match my most pressing needs.” DRC was designed and staffed to meet one set of goals, but as those goals shifted, the partnership was increasingly subject to “adverse selection,” which constrained principals’ perceptions of the partnership’s effectiveness.
Finally, the new structures to support information flow did little to incentivize DRC researcher-agents to meet principals’ new goals around supporting research use. As district practitioners and local funders became focused on research impact and district improvement, DRC researchers found themselves increasingly torn between their role in the partnership and their primary role as academics, a situation one researcher referred to as trying to “serve two gods.” Another researcher described the strain they faced trying to “function on two tracks”: an academic track focused on “nuanced and sophisticated” analysis that they could “publish in national journals,” and a district reporting track that was less complex but showed “the value we can offer” to “the local folks who we rely on for data.” Despite researchers’ efforts to attend to both goals, DRC’s design made it “a zero-sum game at the end of the day.” Ultimately, DRC researchers’ professional incentives as tenure-track faculty increasingly conflicted with and often outweighed any incentives offered by the partnership’s principals.
In an effort to further buffer themselves from principals’ new demands, DRC researcher-agents pursued broader sources of funding from national foundations that they hoped had less interest in DRC’s local impact than local foundations. While shifts among the priorities of national funders (described previously) cast some doubt on this strategy, DRC researchers nonetheless sought ways to distance themselves from local pressure to more directly support district aims. As one DRC researcher noted, “[national funders] couldn’t care less whether the district superintendent is responsive to us . . . They want to know what the country can learn from your privileged position in [the city].” Said another DRC researcher: “We have to do things that interest foundations or the federal government. That means sexy [research topics that] are easier to sell . . . [They are] not paying for you to be a consultant for an idiosyncratic school district.” Regardless of whether DRC’s approach reflected an accurate assessment of national funders’ priorities, it reflected DRC’s efforts to better align their incentives with their goals as academic researchers, and to some degree, shirk their existing principals’ expectations that they more directly support district research use.
Thus, while DRC successfully redesigned structures to improve information flow between principals and agents, they struggled to overcome the challenges of adverse selection and misaligned incentives that arose as their goals increasingly diverged over the evolution of the partnership. Principals increasingly perceived a “need to re-envision what [DRC] is now versus [when it was founded].” In trying to meet competing principals’ objectives, as well as their own, DRC researchers often seemed to fall short. As DRC’s founding researcher summarized in the partnership’s 10th year,
We’ve used the word trade-offs multiple times. This whole conversation is about trade-offs or walking a tightrope. [DRC] to this day successfully walks a tightrope. A thing to be figured out is if that is what success is. I’m not positive that walking a tightrope is success.
URT: Designed With Multiple Ends in Mind
Like DRC, URT straddled multiple, divergent goals. However, URT’s formal structures and more consistent information-sharing routines, more robust capacity, and better aligned incentives helped it manage the multiple goals of its principals more effectively by allowing for greater ongoing cooperation between principals and agents. Still, while its design was better suited to its multiple goals, the need to satisfy multiple demands strained URT’s organizational identity, raising questions about the persistent challenge of divergent objectives in RAs’ designs.
URT’s design afforded it numerous strengths in attending to multiple demands. For example, from its outset, URT engaged in monthly meetings with leaders of the district’s Research and Strategy Team (RST), a team of researchers within the district. This formal boundary-spanning structure supported consistent information flow between district leaders and researcher-agents and balanced the responsibility for information flow between principals and agents, thereby alleviating some of the demands on URT’s capacity. URT researchers viewed their connection to RST as “hugely advantageous,” as it provided them “with a single point of contact about our research agenda writ large” and “helped around data access and having somebody who speaks our language within the [district] and who can advocate for us within the [district].” Unlike DRC, which largely relied on informal connections and personal relationships to facilitate information flow between researchers and the district, URT’s partnership with RST supported consistent, clear communication between the district and researchers.
The district research team further reduced pathological delegation between stakeholders at multiple levels of the district and URT researchers. For example, RST supported clearer communication between program teams and URT researchers by helping to clarify each team’s objectives and communicate related research needs to URT. URT researchers described meetings in which RST “would bring together relevant people from the department to have a conversation about [a particular study]. They’re providing context that’s really important for us to understand, to ask the right questions, or employ the right methods.” On one level, these meetings served a similar function to the meetings between DRC researchers and district staff, but beneath this surface-level similarity, it was clear that the advanced research capabilities of RST staff facilitated far more substantive communication around research design than was possible for DRC. For example, a URT researcher described how members of the district’s research team would
come to us with concerns that their colleagues have, but put it in research-friendly terms . . . And conversely, I think they can explain the nuances and the limitations of different study methods to colleagues in a way that is probably helpful.
Because RST had considerable knowledge of both research and practice, they supported consistent, effective information flow and better enabled both sides of the partnership to negotiate the fit between principals’ and agents’ goals.
RST further buffered URT researchers from pathological delegation associated with changes in district leadership and priorities, an issue that plagued DRC absent similarly strong communication structures. Through consistent meetings with the district’s research team, URT researchers regularly updated their working knowledge of the district. This proved to be an invaluable asset. One district official noted that “There doesn’t have to be as much background and as much involvement from the [district] when [URT is] involved,” whereas another stated that they “don’t know any other researcher that intimately knows our school structure and our data the way that [URT] does.” While both DRC and URT researchers possessed substantial methodological expertise, URT researchers’ regular meetings with RST made it easier for URT researchers to stay abreast of and aligned with district needs.
As a better-resourced organization, URT was also able to minimize the magnitude of adverse selection by hiring new individuals whose knowledge and skill enhanced organizational capacity to respond to various goals. For example, when launching new research streams, such as one focused on district inequalities, URT hired “some junior or new PhDs” to “build their capacity to do that kind of work.” Similarly, URT invested heavily in communication and dissemination by establishing an internal communications team staffed by communication experts. As one staff member shared:
Finding people who are really strong researchers and also have the capacity to talk about the work in a variety of settings, to a variety of different audiences, and be able to explain why it’s important, and what’s interesting to people, that’s not everybody’s forte. That’s a challenge—finding folks with the right skillset.
Unlike DRC, which relied on the same small group of researchers to attend to both research and communication, URT devoted organizational positions entirely to external communications. This ensured that URT researchers could focus more exclusively on producing research, while the organization as a whole could effectively disseminate that research to multiple audiences that represented their multiple principals. Indeed, URT’s leader described their strategy of investing in communications specialists as a “game changer” and “transformative” within the organization. Overall, the communication team increased the organization’s capacity to meet multiple principals’ demands and minimized adverse selection.
Finally, URT researchers’ incentives were better aligned with their principals’ objectives. As full-time employees of URT, the researchers’ need to pursue scholarly ambitions was far less urgent than their DRC counterparts. In addition, URT’s funding strategy enabled them to better cope with the challenge of multiple principals by tempering funders’ influence on their work. URT maintained more external funding devoted to general operations compared to DRC, which resided in a far less financially robust environment. Unlike project-specific grants in which funders typically exercised more power over the goals and reporting expectations of researchers’ work, general operating expenses allowed URT more discretion to align its work with district goals. Numerous district informants and researchers within URT cited the RA’s work on a school capacity survey—the one project in our sample that was entirely funded through general operating funds—as one example of when their goals were largely in sync. As one high-level district informant shared, the work on school capacity was “definitely driving policy.” And as URT’s director shared, the survey work was “not all research driven, but it’s also not all policy and politics driven. There’s some of each.” When funders’ heavily incentivized objectives were no longer competing with those of other stakeholders, URT researchers and their district partners were able to conduct work that better satisfied their mutual aims.
Despite these design advantages, the school capacity work proved an exception among URT’s portfolio of research, which was otherwise heavily influenced by funders’ goals. In part, this was because general operating funds alone were insufficient to sustain the partnership. The majority of URT’s funding came from large foundations seeking long-term evaluations of district initiatives. These projects positioned funders as the dominant principals, forcing URT researchers to prioritize funders’ goals over those of the district. As one district employee reflected, “the funders care about the outcomes . . . That sometimes takes attention away from the learning part of [the research] and the formative part.” Other district staff noted that “big ticket studies don’t provide actionable information,” and longitudinal evaluations are “not that useful and productive.” An initiative director within the district further noted that while the district had chosen URT to conduct a particular research study because they hoped URT would “do more short-term, flexible iterative feedback sessions,” the research study was eventually “structured as a longitudinal, external evaluation” because that was what funders wanted. Funders’ interest in impact evaluation drove the majority of URT research but did little to meet district officials’ desire for research that supported learning and improvement. URT may have been better resourced and had greater internal capacity than DRC, but the organization still had to juggle multiple principals with divergent objectives rather than focus on a single set of goals.
Thus, while the combination of structures to support consistent information flow, resources to strategically add and train staff to prevent adverse selection, and diverse and robust incentives better supported URT in attending to multiple objectives, few saw URT as a resounding success. One high-level district informant shared, “I haven’t seen a ton of their work really driving policy.” Similarly, a member of URT’s board shared “I can’t think of any major policy changes that have come out of [URT’s] work.” Even the highly touted school capacity work was viewed skeptically by URT’s board and some of their funders. As the head of URT’s board shared, projects like the school capacity work “can’t be all the work that we do” because “it would compromise our independence.”
Just as DRC struggled trying to “walk the tightrope” between divergent objectives, so, too, did URT struggle with what it meant to be a successful RPP. By the end of our data collection, both a key district leader and the head of URT reflected that perhaps URT “wasn’t quite a research practice partnership.” While URT’s work remained respected by all stakeholders, there were, as one founding funder shared, “not wildly ambitious aspirations of what [URT] is going to yield at this point.” Despite URT’s robust resources and numerous design advantages, both principals and agents were in agreement: A single organization would never be able to fully satisfy such diverse purposes.
Discussion
Using principal-agent theory, this article explored the alignment and misalignment of the goals of principals (districts, funders) and agents (researchers) in RAs, and variation in that alignment over both time and context. Our analysis also considered how differences in stakeholder goals and RA design explained variation in principal-agent dynamics. Both cases surfaced the challenges researcher-agents face “walking a tightrope” between multiple, competing goals, a challenge that was built into these partnerships, if not all RAs, from their inception. RAs, it seems, were designed as collective principal-agent relationships in which researchers are accountable to both funders and district practitioners who, at least in the cases presented here, exhibited competing goals.
Multiple principals with multiple goals need not be inherently problematic. After all, numerous organizations, such as hospitals and universities, manage to respond to stakeholder groups with competing practical or ideological interests. One interpretation of our findings, then, is that multiple principals are not a major hindrance so long as RAs have the capacity to manage multiple demands. As we saw in the case of URT, a combination of financial resources, greater staffing capacity, structural independence from the university, and a formal boundary spanner within the district enabled URT to more successfully manage the principal-agent challenges it faced. We found considerable evidence that these design advantages enabled researcher-agents to coconstruct a research agenda with their principals while supporting more consistent communication among partners throughout research projects. These findings echo research on long-term principal-agent relationships that suggests in such cases cooperation and negotiation may support principals and agents to reach more mutually beneficial and efficient aims than mere incentives (Duff & Wohlstetter, 2019; Miller, 2005). We found ample evidence of such cooperation in URT. Principals and agents intentionally co-designed an organization intended to balance the goals of researchers, practitioners, and funders. In short, URT’s design supported many of the practices theorized to be central to RPP success (Penuel & Gallagher, 2017) and improved long-term negotiations between principals and agents (Miller, 2005).
This stands in contrast to DRC, in which the pairing of a low-resourced district with an inadequately staffed research organization contributed to considerable tension. Perhaps if DRC had a team of dedicated researchers under less pressure to publish and teach, a communications director that tailored messages to multiple stakeholders, a dedicated boundary spanner with research expertise within the district, and more robust funding, it could have more adroitly navigated the demands of its multiple principals. It seems safe to assume that the more a district partner wants for capacity and resources, the more the RA must compensate with a robust and well-financed organization. To the extent this interpretation is correct, the implication is that we need better-resourced RAs.
A second interpretation, however, focuses less on what it takes to walk the tightrope and more on the tightrope itself. Specifically, we found the public nature of RA research and the district’s desire to maintain legitimacy explicitly conflicted with and often eclipsed aspirations for district learning and improvement. The presence of funders as a second and powerful principal within research partnerships created conflicting incentives that, despite their design advantages and good intentions, researcher-agents and district-principals were unable to overcome. In short, RAs, and RPPs more broadly, can easily be undermined by the very problems they were designed to remedy, and the fact that principal-agent challenges are central to their design. In both DRC and URT, tensions around goal divergence attenuated the perception of RA success and blurred organizational identity, or, as one respondent summarized, mixed messages and competing incentives force RAs to act as a “jack of all trades and a master of none.” More resources and greater capacity will certainly help RAs, but they do more to mitigate collective principal-agent challenges than they do solve them.
Conclusion
Why do researchers in RAs walk a tightrope that forces them to contend with multiple principals’ competing agendas? One underlying reason regards ambiguity about the actual purpose and mission of RAs. Despite growing consensus in the RPP literature that these organizations exist to help districts make practical improvements that lead to more equitable outcomes, our findings suggest that at the grassroots level the goals and purposes of RAs are far more contested. Just as the purpose of public education is continuously debated, so too are the purposes of those research organizations that have been created to support education’s many aims. It should be of little surprise, then, that RAs are vulnerable to competing goals and incentives as they reside in a sector that sits on the nexus of divergent societal norms and values. Researchers and other commentators would be wise to acknowledge this reality.
Are there ways for RAs to become more coherent, single-mission organizations given they were designed to respond to multiple principals leveraging competing incentives? One productive step would involve greater attention to the match between RAs’ capacity and partners’ expectations. As we noted above, the combination of a struggling district, a thinly resourced RA, and lofty expectations for improvement is a recipe for disappointment. This does not mean that RAs such as DRC cannot provide valuable support for districts with many needs, but expectations for improved outcomes should be tempered.
Similarly, directly supporting improvements in teaching and learning, while admirable, is not the only way for RAs and other forms of RPPs to contribute to public education. DRC, for example, was originally established to provide the public with objective information on its education system without expectations that it would fix the problems highlighted by its research. Not only does informing the public represent a valuable social function in a democratic society, but it also requires a level of capacity and resources more commensurate with relatively lean organizations. In other words, adverse selection, or the mismatch of agents’ capacity and principals’ goals, was not evident in the DRC case until principals’ goals shifted. Increasingly, RAs are expected to meet a set of demands that they were not designed to accomplish.
Our research has also highlighted the critical role that funders play as active stakeholders who bring their own set of goals and concomitant demands to the work of RAs. In the cases discussed here, funders not only required researchers to satisfy parties with multiple, competing interests, but also changed the power dynamics in the principal-agent relationship. Because they pay the bills, funders’ priorities can easily supersede those of the other partners if RAs are not designed to create greater balance between stakeholders. Funders can temper this power imbalance by investing in general operating funds for RPPs, thereby decreasing the pressure on RPPs to chase the next big grant or to tailor their research aims to funders’ interests. That said, investments of this nature would require grant-makers to adopt a long-term perspective that is less tied to the discrete measurable outcomes the philanthropic sector often prioritizes. Whether funders, and their board members, would be willing to pursue such a strategy is far from certain.
Finally, our analysis raises questions about the promise and limitations of RAs as a mechanism for large-scale district improvement. Even well-designed, high-capacity research organizations are an insufficient remedy to the problem of improving districts that are of enormous complexity, highly politicized, and tied to a sprawling web of public and private stakeholders. The notion that these organizations could directly support changes in teaching and learning, or that they should be accountable for such changes, seems unrealistic. This does not mean that RAs cannot play a productive role as part of a larger ecosystem of organizations that collectively support the production, sharing, and use of research knowledge in ways that spur district improvement. It does, however, suggest a role for RAs that is more commensurate with their organizational capacity.
There are undoubtedly additional ways that RAs could be designed to further mitigate the difficulties our analysis has highlighted. Principals’ and agents’ goals reflect shifting social norms, ideologies, and expectations, and future research should consider how the conflicts we documented are further complicated by the complex environments in which these organizations operate. But whether it be the recommendations we have suggested or other ones, changes seem inevitable for an organizational form that ushered in the RPP movement but currently seems adrift. As these changes unfold, researcher-agents would be wise to actively participate in the discussions that will determine the future of RAs while also clarifying their role and likely contributions within education systems. RPPs rest on an assumption of mutualism and reciprocity. However, our findings in these cases coupled with education’s contested social and political environment suggest that fulfilling that vision will require ongoing negotiation and compromise. While our conclusion may be at odds with dominant beliefs about the RPP movement, it is in fact consistent with the democratic underpinnings of public education in a pluralistic society.
Supplemental Material
sj-pdf-1-epa-10.3102_01623737231188366 – Supplemental material for Walking a Tightrope: Navigating Principal-Agent Dilemmas in Research-Practice Partnerships
Supplemental material, sj-pdf-1-epa-10.3102_01623737231188366 for Walking a Tightrope: Navigating Principal-Agent Dilemmas in Research-Practice Partnerships by Megan Duff, Joshua L. Glazer, Matthew Shirrell and Dryw Freed in Educational Evaluation and Policy Analysis
Footnotes
Acknowledgements
The authors are indebted to the remainder of their research team: Sara Gopalkrishna, Anisah Waite, Marian Robinson, and Zach Millington, whose collective intellectual contributions, feedback, and insights were instrumental to the development of this article.
Correction (August 2023):
This article has been updated with the article title in one of the references since its original publication.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This article is part of a larger study that has been generously supported by a research grant from the William T. Grant Foundation (grant no. 183924).
Notes
Authors
MEGAN DUFF, PhD, is a postdoctoral researcher at Peabody College, Vanderbilt University. Her research focuses on policies that support school and system leaders in creating more equitable education systems, in particular, research-practice partnerships, educational improvement networks, and other inter-organizational approaches to educational improvement.
JOSHUA L. GLAZER, PhD, is an associate professor of Education Policy at the George Washington University. His research focuses on large-scale school improvement, school improvement networks, research-practice partnerships, state takeovers, and the education profession.
MATTHEW SHIRRELL, PhD, is an associate professor of educational leadership and administration at the George Washington University. His research focuses on the relationships between policy, the social and organizational characteristics of schools and school systems, and learning, improvement, and teacher retention.
DRYW FREED, EdD, is an education consultant in Washington, D.C. Her research focuses on educational improvement and creating more equitable, learning-focused education systems.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
