Sage Journals: Discover world-class research

Abstract

Nonprofits and public organizations may benefit from using evaluation data to improve programs and managerial decisions. Guided by organizational learning and institutional theory, this study investigates how partners, coalitions, and technical assistance organizations influence these organizations’ professional use of evaluation data within collaborative networks. Data were collected from 449 organizations embedded in an ego-centric network of partnerships and purpose-oriented coalitions on education reform. Results from the hierarchical linear models demonstrated that the only significant source of learning was organizations’ relationships with peer organizations in the ego-centric partnership network. This effect remained significant even after controlling for all the other factors. We discuss our findings and offer implications on integrating learning efforts at the coalition network and ego-centric network levels when motivating nonprofits and public organizations to conduct performance evaluations. (128 words)

Keywords

organizational learning interorganizational partnerships mimetic pressure evaluation capacity coalitions

Introduction

Nonprofits and public organizations often collaborate to address societal issues (Kelman, 2007). They may form coalitions or purpose-oriented networks (Shumate & Cooper, 2022). Due to the nature of their work, these organizations face increasing demand to collect performance data for evaluation, budgeting, promotion, and learning (Behn, 2003). Existing literature shows how these data can be used strategically to make sense of changing environments, create knowledge and innovation, and inform organizational decision-making (Gieske et al., 2019; Lee & Nowell, 2015). This study focuses on what drives nonprofit and public organizations to collect and use performance data, referred to as “evaluation capacity building”—the intentional work to increase an organization’s ability to conduct and use evaluation (Cheng & King, 2017).

Though a common practice in the private sector, evaluation capacity building is more complex for nonprofits and public organizations (Lee & Nowell, 2015), primarily due to their roles in providing public goods and difficulty quantifying effectiveness (Mitchell & Berlan, 2018). Nevertheless, evaluation capacity building could help assess activities supporting their goals and how they may improve (Gieske et al., 2019). Nonprofit evaluation literature and the accountability movement for government organizations were founded on the premise that greater evaluation data use improves both outcomes and sustainability of programs (Bourgeois et al., 2018; Carman & Fredericks, 2010; Despard, 2016).

Scholars note that more research is needed to build evaluation capacity (Cheng & King, 2017; Suarez-Balcazar & Taylor-Ritzler, 2014). In particular, more research is needed to understand evaluation capacity building in collaborative networks where organizations from different sectors work together (Mandell & Keast, 2007). Existing literature has argued that evaluation capacity building is context- and culture-dependent (Taylor-Ritzler et al., 2013). Thus, the culture of a particular organization and its community influence evaluation capacity-building activities.

We use organizational learning theory and institutional theory to examine how institutional pressures influence professional data use by public and nonprofit organizations in collaborative networks and the effectiveness of different organizational learning mechanisms. We conceptualize professional data use as a form of evaluation capacity building toward better performance (Bryan et al., 2021) and interrogate three factors at the network level: roles of partners who engaged in ego-centric collaboration, coalitions (i.e., purpose-oriented networks), and technical assistance organizations through the purpose-oriented networks. Data were collected from 449 nonprofit and public organizations embedded in both ego-centric networks of partnerships and purpose-oriented coalitions focused on education reform.

In this study, ego-centric networks capture structural networks consisting of direct collaborative relationships among organizations. These structural networks are essentially connections with immediate partners and could facilitate organizational learning (Fu, 2024). In contrast, coalitions are purpose-oriented networks that are “comprised of three or more autonomous actors who participate in a joint effort based on a common purpose” (Carboni et al., 2019, p. 210). These networks are purposefully designed and formed to serve a specific goal or function, with the intention to facilitate the collaborative achievement of an objective. To illustrate, structural networks can be compared to pairwise dancing, where couples form connections and move between partners, while purpose-oriented networks resemble a flash mob or line dancing, where all participants engage together in a collective choreography toward a shared goal.

Including both structural and purpose-driven networks in this study allows us to examine both the direct effects of partners and the trickle-down effect at the coalition level on organizational learning. Our results demonstrate that evaluation capacity can be gained through organizations’ relationships with peer organizations in a collaborative network. Moreover, they suggest that mimetic pressure (DiMaggio & Powell, 1983) and lateral collaboration function as significant inducements for organizational learning rather than the presence of technical assistance organizations or lead organizations in a purpose-oriented network (via normative or coercive pressure; DiMaggio & Powell, 1983).

This study makes several contributions. First, it represents a new application of the neo-institutional theory to test multiple learning mechanisms through a network perspective. Specifically, we examined how learning may occur through two types of networks: structural and purpose-oriented. It provides empirical evidence that direct collaboration with peer organizations in ego-centric networks is the primary driver of organizational learning, confirming mimetic pressure. Second, our study challenges the assumption that purpose-oriented network effort in performance evaluation could have spillover effects that benefit member organizations directly due to normative or coercive pressure. We found that the effects of these efforts remain at the purpose-oriented network level, perhaps to demonstrate accountability for external stakeholders. Third, findings offer practical implications for funders and coalition leaders. The results suggest that orchestrating direct collaborative activities between partner organizations may significantly impact evaluation capacity. Providing technical assistance or affiliating with purpose-oriented networks is not sufficient. Organizations need opportunities for direct peer-to-peer learning and knowledge sharing.

Organizational Learning, Institutional Theory, and Evaluation Capacity

An increased emphasis on performance measurement and program evaluation in the nonprofit sector in the United States in the late 2010s was primarily driven by the sector’s rapid growth, professionalization, and competition (Mitchell & Berlan, 2018). Evaluation has thus emerged as an essential tool for managing nonprofit accountability issues (MacIndoe & Barman, 2013; Mitchell, 2014) and social legitimacy (Benjamin et al., 2023). The literature on nonprofit performance measurement highlights the complex and often externally driven nature of evaluation practices in this sector. Scholars have argued that nonprofits may engage in evaluation to establish legitimacy and secure funding from donors, rather than for genuine performance improvement (Coupet & Schehl, 2022). This “warm glow” orientation toward measurement stands in contrast to the more internal, decision-making focus that is often the aim of evaluation capacity-building efforts.

Nevertheless, in keeping up with pressures related to performance measurement and evaluation effort, nonprofits also experience resource challenges around limited funding for data collection and hiring evaluation staff, as well as an unclear understanding of monitoring and evaluation (Carman, 2007; Carman & Fredericks, 2010). Furthermore, they may lack knowledge on how to evaluate effectiveness or apply information collected in the first place. Increasing evaluation demands often leave organizations engaging in multiple strategies to prove they are doing “good work” without improving their evaluation capacity (Carman, 2007). Thus, merely collecting data or information about performance does not guarantee its use for effective decision-making. Some scholars have further suggested that meaningless data collection and reporting can lead to mission drift or goal displacement (Ebrahim, 2005; Lee & Clerkin, 2017).

In the government sector, the need to conduct performance evaluations originated from concerns about corruption (Lee & Liu, 2022). Evaluation focuses on developing government organizations’ intrinsic ability to develop and direct the right resources to the right places at the right time (Ingraham & Donahue, 2000). However, scholars have argued that government organizations are driven by various motivations to collect performance data: stakeholder pressure, power dynamics, leadership support, and innovative culture (Agasisti et al., 2020; de Lancer Julnes & Holzer, 2001; Kroll, 2015). Performance data collected by government organizations can help fight corruption and inform and assist policymaking (Wholey, 2010). However, these activities are often presentational in that formal processes are in place to collect and disseminate data, but data are rarely used. Furthermore, government agencies face numerous challenges in program evaluation, including political and bureaucratic obstacles, power struggles between federal and state governments, tensions between the legislative and executive branches, divisions between major political parties, and conflicting constituencies or interest groups (Wholey, 2010).

Professional Data Use for Evaluation Capacity Building and Organizational Learning

Following Umar and Hassan (2019), we conceptualize evaluation capacity building as an organizational learning behavior that involves detecting and correcting errors. Recent literature has identified several goals regarding how nonprofits and public organizations can use evaluation data: professional or anticipatory use (hereafter, professional use), compliance use, and negotiated use (Lee & Clerkin, 2017). Professional evaluation data use is conducted internally to improve operations and strategic decision-making. Compliance and negotiated uses are externally applied as compliance measures toward external stakeholders (e.g., funders, politicians, and the general public). Research has consistently shown that compliance and negotiated use of evaluation data are the most common ways nonprofits and public organizations show accountability to funders (Kim et al., 2019). These short-term activities hardly contribute to long-term performance assessment (Maxwell et al., 2016). This study focuses on professional data use whose benefits are explicitly related to improving organizational practices and management decisions (de Lancer Julnes & Holzer, 2001; MacIndoe & Barman, 2013; Taylor-Ritzler et al., 2013). This approach thus allows us to focus on an organization’s internal orientation and intention toward data use, rather than just reporting to donors and other stakeholders.

Nonprofits and public organizations can be viewed as learning organizations by formalizing performance evaluation, generating knowledge and understanding about data collection and use, and informing behavior change (Levitt & March, 1988; Newcomer & Brass, 2016). Organizational learning theory is a meta-theory rooted in organizational science that examines how firms “build, supplement, and organize knowledge and routines around their activities and within their cultures, and adapt and develop organizational efficiency by improving the use of the broad skills of their workforces” (Dodgson, 1993, p. 377). From a process-based perspective, it considers multilevel factors that influence organizational learning, including the socio-organizational context of learning, individual-level factors, and macro-environmental influences (Argote & Miron-Spektor, 2011; Argyris & Schön, 1996). Generally, organizational learning is viewed as a social and institutional process where learning takes place via leadership, communication, and partnerships (Choi & Woo, 2022; Perkins et al., 2007). Organizations can learn from each other by recognizing the value of new knowledge and practices and applying them to improve performance, known as improving absorptive capacity (Lichtenthaler, 2009).

Neo-institutional Theory and Isomorphism in Organizational Learning

Although much of the research on organizational learning has been conducted in the private sector (Basten & Haamann, 2018), organizational scholars have argued that it can be applied to nonprofits and public organizations (Greiling & Halachmi, 2013; Perkins et al., 2007). Several factors induce organizational learning. For a review of those factors, we refer to concepts from the neo-institutional theory of organizational behaviors and the different forms of isomorphism, or processes that prompt “one unit in a population to resemble other units that face the same set of environmental conditions” (DiMaggio & Powell, 1983, p. 149). Neo-institutionalists emphasize the need for organizations to conform to externally determined expectations concerning their structure and functioning to obtain legitimacy and support (Minkoff & Powell, 2006). DiMaggio and Powell (1983) suggest that institutions may experience three types of isomorphism. Coercive isomorphism describes pressures to resemble other units that result from political influence or a need for legitimacy. In contrast, mimetic isomorphism is an enacted response to uncertainty experienced by the unit, which organizations resolve by enacting practices they observe in peers. Finally, normative isomorphism is associated with professionalization and enacting the practices of highly esteemed organizations (Hwang & Powell, 2009).

Isomorphism has previously been used to examine organizational learning in the for-profit setting (Powell et al., 1996). More recent literature has identified that evaluation practices may also result from isomorphic pressures external to the nonprofit. For example, a funding agency that expects evaluation or a common evaluation practice enacted by peers can motivate an organization to build evaluation capacity (Eckerd & Moulton, 2011). Although isomorphism is a long-standing idea, it has become a dominant frame for nonprofit scholars over the past 20 years. Researchers argue that nonprofits do not collect evaluation data to measure their performance but rather because it is necessary for establishing legitimacy in their environments (Benjamin et al., 2023). However, the literature still needs to address the network aspects of organizational learning via neo-institutional theory regarding learning from whom. The following section shows examples of each form of isomorphism in the context of purpose-oriented networks and performance evaluation.

Mechanisms Underlying Organizational Learning: Toward Evaluation Capacity in Collaborative Networks

Interorganizational networks are often formed in response to a problem, and they can take on different forms, ranging from collaborative networks to purpose-oriented networks or coalition networks (Carboni et al., 2019; Cooper et al., 2024; Shumate & Cooper, 2022). These networks can be important for organizational learning by offering opportunities for knowledge exchange, collaboration, and acquisition of new skills (Du & Wang, 2019; Perkins et al., 2007). In particular, direct partnerships and coalition networks driven by collaboration play critical roles in organizational learning as partner organizations are bound within an institutional field to share resources and risks (Powell et al., 1996; Uzzi, 1997).

Studies suggest that institutional pressures from external stakeholders (funders, accrediting bodies, or national associations) often motivate nonprofit evaluation activity (Carman, 2011) and that government organizations may benefit from agreements on approaches to evaluation and the creation of performance partnerships involving joint accountability (Moynihan & Kroll, 2016; Wholey, 2010). These findings indicate that interorganizational partnerships may function as a conduit for organizational learning toward evaluation capacity building. However, key questions still need to be explored, including the available mechanisms and from which organizations learn.

Learning From Organizational Partners Within Ego-Centric Networks

Organizational learning related to evaluation capacity can occur through partner organizations within a collaborative initiative. Organizations form alliances and coalitions with other organizations based on a common purpose (Bryson et al., 2015). We refer to these interorganizational collaborative relationships as networks to capture the shared affiliations and the interdependence among organizational activities (Isett et al., 2011). Partners in these networks may come from various sectors, including nonprofits, private companies, and government agencies (Boyer, 2016; Wang et al., 2020). They are thus embedded in an institutional environment where organizations’ decisions are subject to the norms and social expectations deemed appropriate (AbouAssi et al., 2021).

Interorganizational collaboration is a communicative process in which interdependent organizations or stakeholders work together to solve problems (Keyton et al., 2008). However, the nature of these communication practices may depend on the structures enacted. Existing research has suggested multiple forms of interorganizational relationships in response to local social issues, including community collaboration (Heath & Frey, 2004), bona fide networks (Cooper & Shumate, 2012), and social impact networks (Shumate & Cooper, 2022), and recommended that nonprofits maintain different types of relationships (Fu & Cooper, 2021).

There are long-standing arguments that engaging in collaborative or network activity can help organizational learning (Knight & Pye, 2005; Kraatz, 1998; Powell, 1998). More recent research affirms this argument. Moreover, varied network structures are seen as contributing to learning, inviting further exploration of different network types and methodologies (Anand et al., 2021). In this study, we examine two types of collaborative networks. The first type is ego-centric partnership networks that describe sets of collaborating organizations that engage in data-sharing, joint delivery of service or programs, joint budgeting, staff cross-training, or policy work. In ego-centric networks, partners are only connected if they directly engage in these collaborative practices with the other organization (Scott, 2000). The second type is coalitions, which, in contrast, are purpose-oriented networks of organizations working toward (a) common goal(s). Organizations join coalitions through membership, regardless of participation or activities. Considering these two types of networks enables us to examine both the direct effect of learning from collaborating partners and the trickle-down effect of network affiliations.

In ego-centric networks, working alongside other agencies may allow mimetic isomorphism in nonprofit evaluation (Lee & Clerkin, 2017), which refers to organizations’ tendency to imitate each other when facing uncertainty or unclear goals (DiMaggio & Powell, 1983). For government organizations, existing literature has documented their mimetic behaviors in program evaluation through collaborative relationships with their peers (Piña & Avellaneda, 2018). Some even argue that government organizations are more vulnerable to this institutional process than other sectors (Frumkin & Galaskiewicz, 2004). Based on these studies, we argue that ego-centric partnerships help to create a culture that motivates nonprofits and public organizations to share knowledge and insights related to data use in program evaluation (Milway & Saxton, 2011). Therefore, we propose the following:

Hypothesis 1 (H1): Nonprofits and public organizations that have direct partnerships with other organizations with higher professional data use will engage in more professional data use themselves.

Learning From Coalitions

Besides the influence of partner organizations in ego-centric networks, coalitions that function as collaborative networks could also improve evaluation capacity building in nonprofits and public organizations. Coalitions are purpose-oriented networks and can operate with various governance models, which influence how organizational members provide input toward collective goals, how coordination among partners occurs, and how to assess network effectiveness (Nowell & Albrecht, 2024; Provan & Kenis, 2008; Shumate & Cooper, 2022). Due to their collaborative nature, coalitions often set rules and practices regarding what metrics to track and how member organizations share data and results from the evaluation (Wolff et al., 2016). Coalition networks with clearly defined metrics to track, a shared database among partners, a robust data team, and frequent communication about data-driven reports have good data use practices (Kania & Kramer, 2011). In contrast, coalition networks that need to track data to support their theory of change, need more data systems, or report data haphazardly in response to stakeholder demands, have poor data use practices (Shumate & Cooper, 2022).

In coalitions with strong data practices, member organizations may face coercive pressure in learning, which refers to external forces compelling an organization to conform to certain practices due to funding requirements, influences of specific stakeholder groups, or fidelity of a collaboration model (Wang et al., 2020). As organizations face increasing scrutiny from coalition lead agencies, they might be more motivated to adopt specific data use practices to maintain their legitimacy in the coalition.

Similarly, coalitions may also generate normative pressure, which captures the tendency for organizational activities in a specific field to homogenize over time and demonstrate professionalization (DiMaggio & Powell, 1983). Evaluation goals for coalitions move beyond individual organizations, encompassing information sharing and service coordination. Evaluation data aid coalitions in assessing resource allocation, enhancing community outreach, and influencing policy change (Fyall & McGuire, 2015). Organizations that actively participate in a coalition with good data use practices could learn through observation. Normative pressure has been found to influence how nonprofits and government organizations adopt performance data practices (George et al., 2020; Lee & Clerkin, 2017). Member organizations can formally or informally conform to normative practices and organizational protocols about evaluation capacity through training, meetings, and workshops in the network (Ofek, 2015).

To summarize, organizational learning can occur through affiliations with coalition networks that engage in strong data practices due to either coercive or normative pressures. We propose:

Hypothesis 2 (H2): Nonprofits and public organizations in coalitions with good data use practices will engage in more professional data use than organizations in coalitions with poor data use practices.

Learning From Technical Assistance Organizations

Organizational scholars have long recognized the role of technical assistance organizations in promoting effective evaluation (Katz & Wandersman, 2016; Olson et al., 2020; Umar & Hassan, 2019). These organizations function as consultants, facilitating conventions and coordinating coalitions (Reisman, 2024). Normative pressure originates primarily from the need for professional standardization (Zorn et al., 2011). From the lens of organizational learning, technical assistance organizations can generate normative pressure by demonstrating good practices in relevant industries and encouraging the adoption of best practices (Gage et al., 2024; Gibbs et al., 2009; Olson et al., 2020; Umar & Hassan, 2019).

Previous research has primarily examined the role of technical assistance organizations in supporting organizational evaluation (Gibbs et al., 2009). We extend this line of research to examine whether evaluation technical assistance in purpose-driven networks or coalitions in which organizations are embedded has spillover effects (Wolff, 2001). Nonprofit and government staff can benefit from technical assistance at the network level. Coalitions may suffer from inconsistent member attendance and data collection (Jarosewich et al., 2013). This inconsistency in participation can be attributed to a lack of communication to achieve consensus on critical metrics, a lack of user-friendly formats to follow, or simply a lack of enforcement. Technical assistance can help coalitions create evaluation systems that serve the network and its organizational members.

However, the effective role of technical assistance organizations on organizational learning is contingent upon other factors. For example, organizations typically require technical assistance to be provided both intentionally and over time for it to be effective (Meehan et al., 2013) as well as widely applied across participating organizations (Gage et al., 2024). Nevertheless, we expect that technical assistance organizations could foster a culture of accountability and continuous improvement, encouraging member organizations in a purpose-oriented network to adopt and align with good data practices. In other words, normative pressure driven by a collective commitment to enhance performance could be in place. Therefore, we propose the following:

Hypothesis 3 (H3): Nonprofits and public organizations in coalitions that receive evaluation technical assistance will use more professional data than their counterparts without technical assistance.

Method

This study uses interview and survey data from 26 cross-sector coalitions to improve educational outcomes. These initiatives were sampled through the following procedures: a). We compiled a database of collective impact coalitions working on education by identifying member affiliations with national networks (e.g., StriveTogether, Collective Impact Forum, Ready by 21, and the Campaign for Grade-Level Reading). Collective impact coalitions follow a structured partnership model where partners are formed under a common agenda, share measurement data, engage in mutually reinforcing activities, and communicate regularly under the leadership of a backbone organization (Kania & Kramer, 2011). We successfully recruited 13 collective impact coalitions that provided consent to participate. b). We used the following matching criteria to recruit another 13 coalitions that were less structured compared to the first half of the sample: geographic (e.g., population density, coverage area, and the number of school districts or municipalities), demographic (e.g., race, and poverty rate), and labor market (e.g., unemployment rate, and median income). All the sampled coalitions were in 13 states.

The matching process was conducted to minimize institutional differences across educational contexts and to enable the research to compare the outcomes of networks of different design features while controlling for potential confounding factors. This matching technique helps strengthen the internal validity of the study by reducing the likelihood that observed differences in outcomes are attributable to underlying differences in the coalition environments. The use of matched pairs from the same state also ensured consistency in the policy and regulatory context affecting the coalitions, further enhancing the ability to isolate the impact of the coalition activities on educational improvements. This paper is part of a larger study in which the matching strategy was incorporated into the design, but it is not directly pertinent to the current investigation.

Coalitions in our sample varied on several factors: the model of collaboration (collective impact or not), year of establishment (2006 to 2015), location, size (M = 45.964, S.D. = 27.594), composition (what types of organizations are in the network), and social network structure. Notably, nonprofits, education organizations (e.g., K-12 public schools, early childhood schools), and government agencies (e.g., the Department of Public Health, Department of Children and Families, Department of Labor, and Department of Social Services) made up the majority of each of these coalitions, with less representation from the business sector. There is no co-membership across coalitions.

Network-Interviews

Semi-structured interviews were conducted with the coalition leads of all 26 communities, lasting about an hour. Questions included their history, mission statement, funding sources, strategies for aligning partners, community engagement activities, and data collected. Each interviewee also provided a roster of their member organizations. In addition, we also requested archival data from all these communities, including their founding documents such as Memorandum of Understanding (MOU), meeting notes, and partner rosters. The interviews were recorded and transcribed with their permission. The interview data and archival data were used to measure coalition-level attributes.

Surveys of Organizations in Coalitions

A total of 920 organizations were identified from these coalition networks, using rosters collected from network leads. We collected data from these sampled organizations through a phone-based survey. We used emails and phone calls to recruit participants. Our final sample included 449 surveys (51% response rate) from 273 nonprofits and 176 government agencies. Respondents who filled out the survey occupied a leadership position at their organization (e.g., executive director, CEO, community engagement director of a nonprofit, or department director at a government agency). Likewise, organizations in these coalitions also varied in size, scope, budget, and other organizational factors.

Questions in the survey included items about the organizational sector, ego-centric partnerships, and data use activities. When reporting ego-centric partnership activities, each organization was given the roster of all member organizations in their affiliated coalition and was asked to identify as many partners as possible for specific collaboration activities. The collaboration activities were having positive and close relationships with senior leaders, competing for public and private funds, staff, and clientele, sharing data, delivering joint services or programs, joint budgeting, staff cross-training, and policy work. The data reported were used to construct ego-centric networks for each organization sampled. We then used ego-centric data to calculate the partner organization’s average data use score; thus, our measure was less sensitive to missing data.

Measures

Professional use of outcome data in a coalition is the dependent variable of this study and is measured at the organizational level. We adopted the 6-item scale about professional use of evaluation data by Lee and Clerkin (2017). It draws on multiple theoretical frameworks to capture the implementation of performance measurement and inform better operational and management practices. We asked the respondents to describe the extent to which their organization engaged in data use activities on a 5-point frequency scale, with 1 marked as “not at all” and 5 marked as “to a very great extent.” Example statements include: My organization uses data to improve management decisions, improve strategic planning, improve services for clients, appraise employee performance, support budget decisions, and for outreach and public relations. A confirmatory factor analysis showed that this six-item scale demonstrated strong evidence of construct validity and unidimensionality, χ²(9) = 51.325, p < .001; M = 3.98, S.D. = 0.63, Cronbach’s α = .77. See more details in Appendix C.

Coalition-Level Variables

Two coalition-level variables are measured using interview data with coalition leaders. They were corroborated with archival documents requested from the coalitions.

First, coalitions that use data to support theories of change and have developed the technical and human resources needed to track and report their data have good data use practices. To determine the quality of coalition practice, we examined two sources of data: transcripts of interviews with coalition leads (specifically related to questions about data collection, data sharing, metrics used, and dissemination of reports to stakeholders; see Appendix A for the list of questions) and archival data. The first step in identifying the coding involved four undergraduate research assistants writing a detailed memo on each network based on the two data sources, following a template to address the following question: What things are they doing well? Where do they fall short, according to our criteria? Given examples and quotes to support your claims. Next, two of the co-authors evaluated all the memos, coded each network individually as ordinal with four magnitudes, discussed the coding, and developed schemes to collectively review and confirm the results (See Table 1 for the codebook and frequency in each category).

Table 1.

Coding of Network Data Use Practice.

Magnitude	Description	Frequency
4 = Comprehensive data use practices	This includes all or nearly all of the following: using common metrics, a shared database system, a strong data team, a network and/or organization using data-driven goal setting, and regular dissemination of reports to members and the community.	6
3 = Good data practices but with gaps	These might include not having a database, but using a reporting system instead. Or the network might not have common metrics developed yet.	7
2 = Poor data practices	There are significant gaps in data practices. Several of the top elements are missing.	5
1 = Very few data practices	The network uses summary reports derived from public sources. They may disseminate a community report.	8

Note. In the sampled networks, it is only sometimes the case that all organizational members within a network collect the same performance evaluation data.

Second, some coalitions received technical evaluation assistance. We operationalized this variable by examining whether a coalition is affiliated with a national organization that promotes strong coalition data use (n = 14). These umbrella networks also provide technical assistance in adopting a particular model and data use practices. The following were identified: Campaign for Grade-Level Reading, Collective Impact Forum, Communities that Care, My Brother’s Keeper, Ready by 21, StriveTogether, Ready by 21, and United Way.

We also included several control variables at the coalition level. Coalition age was measured as the years since the coalition was founded (M = 8.85, S.D. = 6.60). Coalition size was measured as the number of organizational members (M = 34.42, S.D. = 24.12). The coalition model was measured as a categorical variable where 1 = collective impact and 0 = non-collective impact.

Organizational-Level Variables

First, the sector is measured to indicate whether an organization is a nonprofit (n = 283) or a public organization (n = 188). This variable was created to capture the potential differences between public organizations and nonprofits. Second, ego-centric network partner professional use of outcome data is measured by calculating the average score of all the nominated partners from one organization (M = 4.044, S.D. = 0.298, N = 449). Thus, each organization was given a score based on its portfolio of partners. Each organization in the network completed a survey that recorded its partnerships and level of professional data use. Ego-centric network questions asked the respondents to report with which organizations in the coalition network they collaborated with in the past three years on 7 activities. The ego-centric network data were used to identify which organizations were partners. Then, we averaged the professional data use of partner organizations to predict organizational data use.

Analysis

A summary of key variables at both organizational and coalition levels and data collection methods is shown in Appendix B. To test the hypotheses, we used hierarchical linear modeling (HLM; Raudenbush & Bryk, 2002). We used the Linear and Nonlinear Mixed Effects Model package nlme in R (Pinheiro et al., 2021). This allowed us to examine both fixed (i.e., population-level effects) and random effects (i.e., subject-specific) in a single model. Following the guidance of Raudenbush and Bryk (2002), we created several models to assess the incremental contribution of specific variables to the overall model fit and the amount of variance explained. The first model was the null model, where there was a random coefficient linear mixed-effect model by maximum likelihood. The second model contained all the covariates at both the organizational and network levels. And the final model contained all the factors from model 2 and all the independent variables that were intended to test our proposed hypotheses.

Results

We presented descriptives of all the variables and bivariate correlations among them in Table 2. The findings show that significant correlations occurred between network size and age, network age and collective impact coalitions, network age and umbrella networks, network size and magnitude of data use practices, and professional data use practices and partners’ average score of data use.

Table 2.

Descriptives and Correlations (n = 449).

Variable	1	2	3	4	5	6	7	8
1. Sector	1
2. Network age	.03	1
3. Network size	.05	0.59***	1
4. Network model	.03	0.37***	–.02	1
5. Partner average	–.07	–.03	.09*	–.06	1
6. Magnitude	0.11*	.08	0.37***	–.09	.08	1
7. Umbrella	.09*	0.39***	.004	0.47***	–.09	–.20***	1
8. Professional data use	–.01	.03	.07	–.03	.51***	.05	–.02	1
Min	0	2	8	0	1.17	1	0	1.17
Max	1	36	102	1	4.92	4	1	5
Mean	0.39	10.65	45.96	0.50	4.04	2.88	0.50	3.99
SD	0.49	8.12	27.59	0.50	0.30	1.13	0.50	0.65

p ≤ .05. ** p ≤ .01. *** p ≤ .001.

Table 3 shows findings from the three models tested with HLM. As shown in Model 2, none of the network-level (level 2) factors were significant. In model 3, which included all the level 1 and level 3 factors, the only significant effect came from partners’ professional data use (1.16, S.E. = 0.11, p < .001). H1 was thus supported. H2 and H3 were not supported. The results indicate that coalition-level factors (i.e., coalition data use practices and technical assistance membership) did not influence organizations’ professional data use, evidenced by a low intraclass correlation coefficient (ICC). It suggests that an extremely small proportion of the total variance in professional data use at the organizational level is attributable to the network-level influence.

Table 3.

Results From the Hierarchical Linear Regression Models (n = 449).

	1		2		3
Independent Variables	B	SE (p-value)	B	SE (p-value)	B	SE (p-value)
Intercept	3.99	.03 (<.001)	43.94	.07 (<.001)	–.70	0.45 (.13)
Public organization			–.02	.06 (.73)	0.14	0.79 (.86)
Network age			–.001	.01 (0.81)	.01	.01 (0.34)
Network size			.002	.001 (0.19)	–.001	.002 (0.66)
Network type/CI			–.02	.07 (0.74)	–.04	.07 (0.55)
Partner professional use of outcome data					1.16	0.11 (<.001)
Partner * org type					–.03	0.20 (0.88)
Magnitude2					–.08	0.10 (0.42)
Magnitude3					–.04	.09 (0.66)
Magnitude4					–.001	.08 (0.99)
Technical Assistance Membership					.02	.97 (0.75)
R ²	0.58%		0.64%		26.64%
AIC	893.72		899.03		776.31
BIC	906.01		927.72		829.59
ICC	.01		2.67 × 10^–6		8.07 × 10^–10

Note. SE = standard error; AIC = Akaike information criterion; BIC = Bayesian information criterion; ICC = intraclass correlation coefficient.

We conducted multiple robustness checks to verify the findings. First, we ran a multiple linear regression using all the IVs and covariates in one model (including the interaction effect of organizational type and professional data use by partners) without considering the multilevel or cluster effects that may occur within networks. The model was significant, F(10, 434) = 15.72, p < .001, adjusted R² = 0.25. The results showed that the only significant effect remained from the average partner’s professional data use activity, confirming the findings from the HLM (B = 1.16, p < .001).

Second, we ran two separate multiple linear regression models: one with only organizational-level variables and the other with only network-level variables. The organizational model was significant, F(3, 441) = 52.25, p < .001, adjusted R² = 0.26, with the only significant factor being partner’s professional data use activity (B = 1.15, p < .001). The network model was not significant, F(7, 437) = 0.76, p = 0.62, adjusted R² = -.004. The negative R² suggests that the model does not explain any of the variation in the DV better than simply using the mean of the DV as the prediction, and thus, other factors should be accounted for beyond the network level variances. So far, the two steps confirmed the findings revealed in the HLM reported in the manuscript.

Finally, we recoded the DV into a binary variable and reran the HLM. The three new models tested did not reveal higher R². Findings remained consistent that the only significant effect was from the partner’s average score in professional data use. See Table 4 for the summary. The findings holding up across various robustness checks suggest that the results are robust and not overly sensitive to the specific modeling choices or method assumptions. This provides greater confidence in the reliability and validity of the conclusions drawn from the original analysis. It also helps to strengthen the evidence and supports the generalizability of the study's conclusions.

Table 4.

Results From the Hierarchical Linear Regression Models Using a Binary DV (n = 449).

	1		2		3
Independent Variables	B	SE (p-value)	B	SE (p-value)	B	SE (p-value)
Intercept	1.62	.03 (<. 001)	1.63	.06 (< .001)	–.36	.37 (0.33)
Public organization			–.01	.05 (0.92)	–.57	0.64 (0.38)
Network age			–.01	.004 (0.24)	–.001	.01 (0.84)
Network size			.001	.001 (0.19)	.0003	.001 (0.79)
Network type/CI			–.04	.05 (0.46)	–.07	.05 (0.19)
Partner professional use of outcome data					0.50	.09 (< .001)
Partner * org type					0.15	0.16 (0.37)
Magnitude2					–0.06	.08 (0.44)
Magnitude3					–0.08	.08 (0.28)
Magnitude4					–0.02	.07 (0.79)
Technical Assistance Membership					.07	.05 (0.24)
R²	0.83%		0.97%		11.83%
AIC	626.15		630.14		590.56
BIC	638.45		658.82		643.84

Note. SE = standard error; AIC = Akaike information criterion; BIC = Bayesian information criterion.

Discussion

Guided by organizational learning and neo-institutional theories, this study examines how partners, coalitions, and technical assistance memberships influence evaluation capacity in nonprofits and public organizations which was measured as the use of evaluation data to improve professional practices. Results from HLM showed that partner organizations’ professional data use was the only significant factor, indicating that the only driver underlying these organizations’ evaluation capacity building was the influence of their peer organizations within a purpose-oriented network. We discuss our main findings through the lens of organizational learning and offer implications on what interorganizational partnerships can function as effective pathways for organizational learning toward evaluation capacity building.

Organizational Learning and the Lack of Influence of Coalitions

In response to increasing demands of demonstrating accountability and social legitimacy (e.g., Benjamin et al., 2023), nonprofits and public organizations often collect evaluation data without possessing the information and knowledge to benefit from it. As a result, evaluating data to inform strategic decision-making and improve organizational practices is also limited. In other words, organizational learning related to evaluation capacity often only captures the single loop within the existing goals organizations have set (Ebrahim, 2005). This study hypothesized that from ego-centric networks and purpose-oriented networks, nonprofits and public organizations would be motivated by mimetic, normative, and coercive pressures (e.g., DiMaggio & Powell, 1983).

In networks with good data use practices, nonprofits and public organizations showed no greater professional use of evaluation data than those in networks with poor practices. Literature indicates that effective network governance is crucial for managing the interdependence of member organizations when conducting evaluations (Ofek, 2015). Many education reform coalitions lack established norms and rules for tracking metrics, sharing data, and utilizing data to achieve network goals (Wang et al., 2020). Variation in member involvement, such as differing attendance at coalition meetings, may also affect engagement in agenda-setting related to data use (Jarosewich et al., 2013). This suggests that neither coercive nor normative pressure was activated.

Given the prevalence of technical assistance organizations offering support in evaluation capacity to coalitions, we expected that member organizations might face more normative pressure to learn about how to use evaluation data to inform decision-making (Wolff, 2001). However, we found that consulting roles of technical assistance organizations did not necessarily result in strong professional data use. This challenges the assumption that technical assistance organizations effectively promote professional data use or facilitate follow-up assessments on implementation plans (Olson et al., 2020). This finding suggests that technical assistance to coalitions does not create spillover effects for its organizational members.

The lack of effect at the coalition level suggests that membership provides insufficient incentives or observation opportunities to improve evaluation capacity. The lack of coalition-level trickle-down effects may indicate that the potential costs of collaborative efforts surpass the benefits (Nowell & Albrecht, 2024). Network governance and the varying levels of organizational participation may explain why training, technical assistance, and accountability requirements at the network level do not result in the formation of community ways of building evaluation capacity.

Mimetic Pressure and Organizational Learning Through Ego-Centric Partnerships

Although prior research on neo-institutional theory has suggested that, through interorganizational collaboration (Powell et al., 1996), more knowledge is needed regarding learning mechanisms. Some studies have demonstrated the importance of mimetic isomorphic pressures in adopting nonprofit evaluation methods (Eckerd & Moulton, 2011). The network context of this study offers insights into the peer dynamics that may shape evaluation practices. The only significant factor from our analysis is peer organizations’ professional data use. In other words, ego-centric partnerships within a coalition signal more than shared affiliation and could help build a culture to motivate nonprofits and public organizations in evaluation capacity building (Milway & Saxton, 2011).

A classroom analogy provides an illustration. Classrooms may provide opportunities for active learning (e.g., students participate in and reflect on activities) and passive learning (e.g., students consume material, typically through lectures; Bonwell & Eison, 1991). Likewise, coalitions may encourage organizations in ego-centric partnerships to participate in activities such as joint programming or staff training. In contrast, coalitions may also feature more passive learning strategies in which organizations do little together outside of attending meetings. Ego-centric partnerships may provide a better context for active learning and professional data use.

Such a distinction makes sense in professional data use because of its tacit knowledge instead of explicit knowledge. Tacit knowledge describes “know-how,” or the processes and techniques one uses to accomplish a goal. In contrast, explicit knowledge refers to “know what” or codified knowledge. Research on interorganizational learning has long realized that partnerships are ideal for gaining tacit knowledge (Mariotti, 2012). The trust and close proximity that partnerships between organizations provide is an ideal opportunities for observing and trying out tacit knowledge.

Similarly, professional data use requires know-how. Although many evaluation guides are available to leaders to learn how to collect the correct data and the appropriate analysis techniques (e.g., Fitzpatrick et al., 2017), they do not necessarily demonstrate how an organization might use the data in everyday management decisions or refine its practice. Knowing how to embed these techniques into organizational routines is a tactic and may best be learned in the context of active partnerships.

Prior research (e.g., Mandell & Keast, 2007) suggests that sectors are rarely explored alongside one another despite the potential insights that may come from this approach. Findings indicate that no sector-level characteristics would generate additional challenges when motivating organizations from either sector to engage in organizational learning toward evaluation capacity building. Regardless of their sector affiliation, public organizations and nonprofits can equally practice mimetic learning via their peer partners in the same coalition.

Implications for Organizational Learning

Since nonprofits and public organizations have limited time for collaboration, this study offers insights into where leaders who want to improve their evaluation capacity should invest their time. Participating in coalitions is time-consuming; consequently, partner organizations may participate to varying degrees. If the goal is to increase evaluation capacity, their time is better spent learning partnerships with peers than joining coalitions with greater evaluation capacity.

The effect of lateral-level knowledge sharing among partner organizations also demonstrates the need for partnerships to emulate know-how. It suggests significant challenges exist when networks aim to integrate organizational learning through the traditional consulting model (Boyer, 2016). Therefore, another implication of this research is that coalitions may leverage direct partnerships among member organizations to formalize expectations about evaluation capacity and select leading organizations to champion performance evaluation efforts. These champion organizations can be local schools that use performance data to serve students better or nonprofits with the knowledge or experience of using performance data for community engagement.

Furthermore, technical assistance organizations can also consider incentivizing peer organizations to implement evaluation programs (Brown & Duguid, 1991). Regardless of their engagement as a partner in a coalition, funders may consider simplifying data reporting formats and advocate evaluation use beyond accountability purposes. Funders can also invite beneficiary organizations to lead the effort in evaluation capacity building to showcase that evaluation data can serve organizations’ strategic planning.

Limitations and Future Research

Several limitations of the current study warrant future research. First, the outcome variable was measured using self-reported behavioral data. Future research could collect other data sources, such as reports and action plans generated with evaluation data tied to actual behavioral outcomes, to examine if the effect of partner organizations remains significant. In addition, future research can directly measure if an organization has an evaluation team or how much money they have for data use activities as a proxy of their evaluation capacity.

Second, we did not collect data on organizations’ engagement with coalitions or technical assistance organizations. Coalitions or technical assistance organizations may already have influenced their data use practices. With the affiliation with a coalition or support from a technical assistance organization, we assumed that resources regarding data use were available to organizations. In other words, normative pressure was measured indirectly by comparing the level of performance evaluation to their partners and to the network as a whole. We call for future research to explore other direct measures (e.g., rating of certain norms in their own sector, behavioral variables such as participation in technical training or meetings hosted by coalition backbone organizations or organizations that provide consulting services) and verify if they affect learning experience.

Third, cross-sectional data and HLM do not reveal causality. The significant effect revealed from direct partners’ data use practices could only be inferred as a correlation. Future research should incorporate a longitudinal study design to examine changes in professional data use over time. It should also apply relevant methods to test causal relationships among factors.

Last, this current study focused solely on isomorphism. We acknowledge that other theoretical frameworks, such as resource dependence theory, contingency theory, or stakeholder theory, could have informed the research questions and study design. We call for future research to examine other frameworks together with longitudinal data to verify our findings further.

Conclusion

Nonprofits and public organizations face increasing demands for evaluation, yet simply collecting data does not equal improving organizational performance. We draw on institutional and organizational learning theory (DiMaggio & Powell, 1983; Dodgson, 1993) to unpack what drives these organizations to use performance data to inform decision-making and modify organizational norms and practices within collaborative networks. Findings demonstrate challenges and the importance of integrating knowledge sharing at the organizational and coalition levels (Greiling & Halachmi, 2013; Perkins et al., 2007). We highlight the critical role that peer organizations in the ego-centric network play in facilitating evaluation capacity building.

This study makes several contributions. First, it applies the existing neo-institutional theory to test multiple learning mechanisms in networks (i.e., effects of structural networks and purpose-oriented networks). Empirical evidence demonstrates that despite multiple venues of organizational learning to build evaluation capacity, direct collaboration with partner organizations is the only driving force. When partner organizations have good data use practices, mimetic pressure through structural ego-centric relationships results in a focal organization’s willingness to engage in evaluation capacity building.

Second, our study challenges the assumption that the network-level effort in performance evaluation could have spillover effects to benefit member organizations directly due to normative or coercive pressure. Nonprofits and public organizations sampled in this study were not subject to normative or coercive pressure to build evaluation capacity in coalition networks. The spillover effect may be associated with surface-level evaluation to demonstrate accountability for external stakeholders. Evaluation data collected to please funders and network leaders do not necessarily serve organizational goals or stakeholders (e.g., employees and clients).

Third, our findings offer practical implications that funders and coalition leaders should focus on fostering strong relationships and collaborative activities between partner organizations. Simply providing technical assistance or having a network affiliation may not be sufficient. Organizations need opportunities for direct peer-to-peer learning and knowledge sharing within ego-centric networks. Funders and coalitions can better support public and nonprofit organizations in developing the evaluation skills and practices needed to maximize performance and impact.

This study demonstrates the importance of evaluation capacity building for improving the offerings of public organizations and nonprofits in the context of education reform. Through the lenses of organizational learning theory and institutional theory, our findings offer critical inquiries into how to govern coalition networks and motivate network members to engage in good data practices, what it means to be an active learner in a coalition, and how to integrate network and organizational-level learning toward performance evaluation.

Footnotes

Appendix A

Appendix B

Data Structure and Source.

Level	Variable	Method
Coalition	Network age	Network interviews
	Network size	Network interviews
	Umbrella network (technical assistance)	Network interviews
	Coalition model (collective impact or not)	Network interviews
	Magnitude of professional data use	Coding of interview scripts and archival data
Organization	Sector	Survey
	Professional data use	Survey
	Partner’s average score of professional data use	Identifying all direct partners in a purpose-oriented network via survey, then aggregating partners’ scores

Appendix C

ORCID iDs

Rong Wang

Michelle Shumate

Katherine R. Cooper

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The Army Research Office Grant W911NF1610464 funded the work.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data Availability Statement

The datasets analyzed during the current study are available in the IPCSR repository, which can be accessed at . All relevant data and analytic scripts that support the findings of this study are included. Access is unrestricted, and the files are made available under a CC BY license to permit reuse, subject only to proper citation of the dataset and associated publication.

Author Biographies

Rong Wang (Ph.D., University of Southern California) is an Assistant Professor at the Department of Human and Organizational Development at Peabody College of Education and Human Development, Vanderbilt University. Dr. Wang researches collective action, open collaboration, and inter-organizational alliances that are designed to achieve collective goals.

Michelle Shumate (Ph.D., University of Southern California) is the founding director of the Network for Nonprofit and Social Impact (NNSI), the Delaney Family University Research Professor at the School of Communication, and Associate Faculty at the Institute for Policy Research at Northwestern University. Dr. Shumate’s research focuses on how to design interorganizational networks to make the most social impact.

Katherine R. Cooper (Ph.D., University of Illinois at Urbana–Champaign) is an Associate Professor of communication studies in the College of Communication at DePaul University. As a researcher, Dr. Cooper is primarily interested in nonprofit organizations and interorganizational collaboration in response to social problems.

References

AbouAssi

Wang

Huang

(2021). Snuggling together or exploring options? A multilevel analysis of nonprofit partnership formation and evolution in an unstable institutional context. Nonprofit and Voluntary Sector Quarterly, 50(1), 143–164. https://doi.org/10.1177/0899764020945800

Agasisti

Agostino

Soncin

(2020). Implementing performance measurement systems in local governments: Moving from the “how” to the “why.” Public Performance & Management Review, 43(5), 1100–1128. https://doi.org/10.1080/15309576.2019.1700806

Anand

Brøns Kringelum

Øland Madsen

Selivanovskikh

(2021). Interorganizational learning: A bibliometric review and research agenda. T. He Learningorganization, 28(2), 111–136. https://doi.org/10.1108/TLO-02-2020-0023

Argote

Miron-Spektor

(2011). Organizational learning: From experience to knowledge. Organization Science, 22(5), 1123–1137. https://doi.org/10.1287/orsc.1100.0621

Argyris

Schön

D. A.

(1996). Organizational learning II: Theory, method, and practice. Addison-Wesley. https://doi.org/10.1177/103841119803600112

Basten

Haamann

(2018). Approaches for organizational learning: A literature review. Sage Open, 8(3), 1–20. https://doi.org/10.1177/2158244018794224

Behn

R. D.

(2003). Why measure performance? Different purposes require different measures. Public Administration Review, 63(5), 586–606. https://doi.org/10.1111/1540-6210.00322

Benjamin

L. M.

Ebrahim

Gugerty

M. K.

(2023). Nonprofit organizations and the evaluation of social impact: A research program to advance theory and practice. Nonprofit and Voluntary Sector Quarterly, 52(1S), 313S–352S. https://doi.org/10.1177/08997640221123590

Bonwell

C. C. J. A.

Eison ., (1991). Active learning: Creating excitement in the classroom (ASHEERIC higher education report 1). George Washington University.

10.

Bourgeois

Simmons

Buetti

(2018). Building evaluation capacity in Ontario’s public health units: Promising practices and strategies. Public Health, 159, 89–94. https://doi.org/10.1016/j.puhe.2018.01.031

11.

Boyer

E. J.

(2016). Identifying a knowledge management approach for public-private partnerships. Public Performance & Management Review, 40(1), 158–180. https://doi.org/10.1080/15309576.2016.1204928

12.

Brown

J. S.

Duguid

(1991). Organizational learning and communities-of-practice: Toward a unified view of working, learning, and innovation. Organization Science, 2(1), 40–57. https://doi.org/10.1287/orsc.2.1.40

13.

Bryan

T. K.

Robichau

R. W.

L’Esperance

G. E

. (2021). Conducting and utilizing evaluation for multiple accountabilities: A study of nonprofit evaluation capacities. Nonprofit Management and Leadership, 31(3), 547–569. https://doi.org/10.1002/nml.21437

14.

Bryson

J. M.

Crosby

B. C.

Stone

M. M.

(2015). Designing and implementing cross-sector collaborations: Needed and challenging. Public Administration Review, 75(5), 647–663. https://doi.org/10.1111/puar.12432

15.

Carboni

J. L.

Saz-Carranza

Raab

Isett

K. R.

(2019). Taking dimensions of purpose-oriented networks seriously. Perspectives on Public Management and Governance, 2(3), 187–201. https://doi.org/10.1093/ppmgov/gvz011

16.

Carman

J. G.

(2007). Evaluation practice among community-based organizations: Research into the reality. American Journal of Evaluation, 28(1), 60–75. https://doi.org/10.1177/1098214006296245

17.

Carman

J. G.

(2011). Understanding evaluation in nonprofit organizations. Public Performance & Management Review, 34(3), 350–377. https://doi.org/10.2753/PMR1530-9576340302

18.

Carman

J. G.

Fredericks

K. A.

(2010). Evaluation capacity and nonprofit organizations: Is the glass half-empty or half-full? American Journal of Evaluation, 31(1), 84–104. https://doi.org/10.1177/1098214009352361

19.

Cheng

S. H.

King

J. A.

(2017). Exploring organizational evaluation capacity and evaluation capacity building: A Delphi study of Taiwanese elementary and junior high schools. American Journal of Evaluation, 38(4), 521–539. https://doi.org/10.1177/1098214016672344

20.

Choi

Woo

(2022). Understanding diverse types of performance information use: Evidence from an institutional isomorphism perspective. Public Management Review, 24, 2033–2052. https://doi.org/10.1080/14719037.2021.1955953

21.

Cooper

K. R.

Shumate

(2012). Interorganizational collaboration explored through the bona fide network perspective. Management Communication Quarterly, 26(4), 623–654. https://doi.org/10.1177/0893318912462014

22.

Cooper

K. R.

Wang

Harris

J. L.

Miles

J. P.

Shumate

(2024). Leading resilient, purpose-oriented networks through change. Management Communication Quarterly, 38(4), 978–1005. https://doi.org/10.1177/08933189241241226

23.

Coupet

Schehl

(2022). Government grants, donors, and nonprofit performance. Journal of Public Administration Research and Theory, 32(1), 97–110. https://doi.org/10.1093/jopart/muab022

24.

de Lancer Julnes

Holzer

. (2001). Promoting the utilization of performance measures in public organizations: An empirical study of factors affecting adoption and implementation. Public Administration Review, 61(6), 693–708. https://doi.org/10.1111/0033-3352.00140

25.

Despard

M. R.

(2016). Strengthening evaluation in nonprofit human service organizations: Results of a capacity-building experiment. Human Service Organizations: Management, Leadership & Governance, 40(4), 352–368. https://doi.org/10.1080/23303131.2016.1140101

26.

DiMaggio

P. J.

Powell

W. W.

(1983). The iron cage revisited: Institutional isomorphism and collective rationality in organizational fields. American Sociological Review, 48(2), 147–160. https://doi.org/10.2307/2095101

27.

Dodgson

(1993). Organizational learning: A review of some literatures. Organization Studies, 14(3), 375–394. https://doi.org/10.1177/017084069301400303

28.

Wang

(2019). Knowledge transfer and boundary conditions: A study of SMEs in business incubation centers in China. New England Journal of Entrepreneurship, 22(1), 31–57. https://doi.org/10.1108/NEJE-04-2019-0021

29.

Ebrahim

(2005). Accountability myopia: Losing sight of organizational learning. Nonprofit and Voluntary Sector Quarterly, 34(1), 56–87. https://doi.org/10.1177/0899764004269430

30.

Eckerd

Moulton

(2011). Heterogeneous roles and heterogeneous practices: Understanding the adoption and uses of nonprofit performance evaluations. American Journal of Evaluation, 32(1), 98–117.

31.

Fitzpatrick

J. L.

Sanders

J. R.

Worthen

B. R.

(2017). Program evaluation: Alternative approaches and practical guidelines. Pearson Education.

32.

Frumkin

Galaskiewicz

(2004). Institutional isomorphism and public sector organizations. Journal of Public Administration Research and Theory, 14(3), 283–307. https://doi.org/10.1093/jopart/muh028

33.

Fyall

McGuire

(2015). Advocating for policy change in non-profit coalitions. Nonprofit and Voluntary Sector Quarterly, 44(6), 1274–1291. https://doi.org/10.1177/0899764014558931

34.

J. S.

(2024). Network portfolio diversity and social innovation: An egocentric approach to cross-sector partnerships. Social Networks, 78, 238–252. https://doi.org/10.1016/j.socnet.2024.03.002

35.

J. S.

Cooper

K. R.

(2021). Interorganizational network portfolios of nonprofit organizations: Implications for collaboration management. Nonprofit Management and Leadership, 31(3), 437–459. https://doi.org/10.1002/nml.21438

36.

Gage

N. A.

Lewis

T. J.

Johnson

N. L.

Way

G. T.

(2024). The impact of the Missouri school-wide positive behavior support (MO SW-PBS) on disciplinary exclusions. Journal of Positive Behavior Interventions, 27(2), 82–93. https://doi.org/10.1177/10983007241275691

37.

George

Baekgaard

Decramer

Audenaert

Goeminne

(2020). Institutional isomorphism, negativity bias and performance information use by politicians: A survey experiment. Public Administration, 98(1), 14–28. https://doi.org/10.1111/padm.12390

38.

Gibbs

D. A.

Hawkins

S. R.

Clinton-Sherrod

A. M.

Noonan

R. K.

(2009). Empowering programs with evaluation technical assistance. Health Promotion Practice, 10(1_Suppl.), 38S–44S. https://doi.org/10.1177/1524839908316517

39.

Gieske

Van Meerkerk

Van Buuren

(2019). The impact of innovation and optimization on public sector performance: Testing the contribution of connective, ambidextrous, and learning capabilities. Public Performance & Management Review, 42(2), 432–460. https://doi.org/10.1080/15309576.2018.1470014

40.

Greiling

Halachmi

(2013). Accountability and organizational learning in the public sector. Public Performance & Management Review, 36(3), 380–406. https://doi.org/10.2753/PMR1530-9576360301

41.

Heath

R. G.

Frey

L. R.

(2004). Ideal Collaboration: A Conceptual Framework of Community Collaboration. Annals of the International Communication Association, 28(1), 189–231. https://doi.org/10.1080/23808985.2004.11679036

42.

Hwang

Powell

W. W.

(2009). The rationalization of charity: The influences of professionalism in the nonprofit sector. Administrative Science Quarterly, 54(2), 268–298. https://doi.org/10.2189/asqu.2009.54.2.268

43.

Ingraham

P. W.

Donahue

A. K.

(2000). Dissecting the black box revisited: Characterizing government management capacity. In Heinrich

C. J.

Lynn

L. E.

(Eds.), Governance and performance: New perspectives (pp. 292–318). Georgetown University Press.

44.

Isett

K. R.

Mergel

I. A.

LeRoux

Mischen

P. A.

Rethemeyer

R. K.

(2011). Networks inpublic administration scholarship: Understanding where we are and where we need to go. Journalof Public Administration Research and Theory, 21(Suppl. 1), i157–i173. https://doi.org/10.1093/jopart/muq061

45.

Jarosewich

Mir

Simkin

(2013). Measuring dimensions of network functioning: the KnowHow2GO Network Survey. The Foundation Review, 5(2), 14–35. https://doi.org/10.9707/1944-5660.1154

46.

Kania

Kramer

(2011). Collective impact. Stanford Social Innovation Review. https://ssir.org/articles/entry/collective_impact. https://doi.org/10.48558/5900-kn19

47.

Katz

Wandersman

(2016). Technical assistance to enhance prevention capacity: A research synthesis of the evidence base. Prevention Science, 17(4), 417–428. https://doi.org/10.1007/s11121-016-0636-5

48.

Kelman

(2007). Public administration and organization studies. Academy of Management Annals, 1, 225–267. https://doi.org/10.5465/078559810

49.

Keyton

Ford

D. J.

Smith

F. L.

(2008). A mesolevel communicative model of collaboration. Communication Theory, 18(3), 376–406. https://doi.org/10.1111/j.1468-2885.2008.00327.x

50.

Kim

Charles

Pettijohn

S. L.

(2019). Challenges in the use of performance data in management: Results of a national survey of human service nonprofit organizations. Public Performance & Management Review, 42(5), 1085–1111. https://doi.org/10.1080/15309576.2018.1523107

51.

Knight

Pye

(2005). “Network learning: An empirically derived model of learning by groups of organizations..” Human Relations, 58(3), 369–392. https://doi.org/10.1177/0018726705053427

52.

Kraatz

M. S.

(1998). Learning by association: Interorganizational networks and adaptation to environmental change. Academy of Management Journal, 41(6), 621–643. https://doi.org/10.5465/256961

53.

Kroll

(2015). Drivers of performance information use: Systematic literature review and directions for future research. Public Performance & Management Review, 38(3), 459–486. https://doi.org/10.1080/15309576.2015.1006469

54.

Lee

Clerkin

R. M.

(2017). Exploring the use of outcome measures in human service nonprofits: Combining agency, institutional, and organizational capacity perspectives. Public Performance & Management Review, 40(3), 601–624. https://doi.org/10.1080/15309576.2017.1295872

55.

Lee

Nowell

(2015). A framework for assessing the performance of nonprofit organizations. American Journal of Evaluation, 36(3), 299–319. https://doi.org/10.1177/1098214014545828

56.

Lee

Liu

(2022). Public corruption and government management capacity. Public Performance & Management Review, 45(2), 397–427.

57.

Levitt

March

J. G

. (1988). Organizational learning. Annual Review of Sociology, 14, 319–340. https://doi.org/10.1146/annurev.so.14.080188.001535

58.

Lichtenthaler

(2009). Absorptive capacity, environmental turbulence, and the complementarity of organizational learning processes. Academy of Management Journal, 52(4), 822–846. https://doi.org/10.1080/00207543.2010.543175

59.

MacIndoe

Barman

(2013). How organizational stakeholders shape performance measurement in nonprofits: Exploring a multidimensional measure. Nonprofit and Voluntary Sector Quarterly, 42(4), 716–738. https://doi.org/10.1177/0899764012444351

60.

Mandell

Keast

(2007). Evaluating network arrangements: Toward revised performance measures. Public Performance & Management Review, 30(4), 574–597. https://doi.org/10.2753/PMR1530–9576300406

61.

Mariotti

(2012). Exploring interorganizational learning: A review of the literature and future directions. Knowledge and Process Management, 19(4), 215–221. https://doi.org/10.1002/kpm.1395

62.

Maxwell

N. L.

Rotz

Garcia

(2016). Data and decision making: Same organization, different perceptions; different organizations, different perceptions. American Journal of Evaluation, 37(4), 463–485. https://doi.org/10.1177/1098214015623634

63.

Meehan

T. P.

Sr Meehan

T. P.

Jr Kelvey-Albert

Van Hoof

T. J.

Ruth

Petrillo

M. K.

(2013). The path to quality in outpatient practice: Meaningful use, patient-centered medical homes, financial incentives, and technical assistance. American Journal of Medical Quality, 29(4), 284–291. https://doi.org/10.1177/1062860613500334

64.

Minkoff

D. C.

Powell

W. W.

(2006). Nonprofit mission: Constancy, responsiveness, or deflection. The nonprofit sector: A research handbook, 2, 591–611.

65.

Milway

K. S.

Saxton

(2011). The challenge of organizational learning. Stanford Social Innovation Review, 9(3), 44–49. https://doi.org/10.48558/dge6-xz93

66.

Mitchell

G. E.

(2014). Why will we ever learn? Measurement and evaluation in international development NGOs. Public Performance & Management Review, 37(4), 605–631. https://doi.org/10.2753/PMR1530-9576370404

67.

Mitchell

G. E.

Berlan

(2018). Evaluation in nonprofit organizations: An empirical analysis. Public Performance & Management Review, 41(2), 415–437. https://doi.org/10.1080/15309576.2017.1400985

68.

Moynihan

D. P.

Kroll

(2016). Performance management routines that work? An early assessment of the GPRA Modernization Act. Public Administration Review, 76(2), 314–323. https://doi.org/10.1111/puar.12434

69.

Newcomer

Brass

C. T.

(2016). Forging a strategic and comprehensive approach to evaluation within public and nonprofit organizations: Integrating measurement and analytics within evaluation. American Journal of Evaluation, 37(1), 80–99. https://doi.org/10.1177/1098214014567144

70.

Nowell

Albrecht

(2024). A population ecology of network domains. Public Management Review, 26(8), 2218–2241. https://doi.org/10.1080/14719037.2023.2182903

71.

Ofek

(2015). The missing linkage in evaluating networks: A model for matching evaluation approaches to system dynamics and complexity. Public Performance & Management Review, 38(4), 607–631. https://doi.org/10.1080/15309576.2015.1031005

72.

Olson

J. R.

Coldiron

J. S.

Parigoris

R. M.

Zabel

M. D.

Matarese

Bruns

E. J.

(2020). Developing an evidence-based technical assistance model: A process evaluation of the National Training and Technical Assistance Center for Child, Youth, and Family Mental Health. The Journal of Behavioral Health Services & Research, 47(3), 312–330. https://doi.org/10.1007/s11414-020-09686-5

73.

Perkins

D. D.

Bess

K. D.

Cooper

D. G.

Jones

D. L.

Armstead

Speer

P. W.

(2007). Community organizational learning: Case studies illustrating a three-dimensional model of levels and orders of change. Journal of Community Psychology, 35(3), 303–328. https://doi.org/10.1002/jcop.20150

74.

Piña

Avellaneda

C. N.

(2018). Municipal isomorphism: Testing the effects of vertical and horizontal collaboration. Public Management Review, 20(4), 445–468. https://doi.org/10.1080/14719037.2017.1412116

75.

Pinheiro

Bates

DebRoy

Sarkar

(2021). R Core Team (2021) nlme: Linear and nonlinear mixed effects models (R package version 31-152). R Core Team.

76.

Powell

W. W.

(1998). Learning from collaboration: Knowledge and networks in the biotechnology and pharmaceutical industries. California Management Review, 40(3), 228–240. https://doi.org/10.2307/41165952

77.

Powell

W. W.

Koput

K. W.

Smith-Doerr

(1996). Interorganizational collaboration and the locus of innovation: Networks of learning in biotechnology. Administrative science quarterly, 116–145.

78.

Provan

K. G.

Kenis

(2008). Modes of network governance: Structure, management, and effectiveness. Journal of Public Administration Research and Theory, 18(2), 229–252. https://doi.org/10.1093/jopart/mum015

79.

Raudenbush

S. W.

Bryk

A. S.

(2002). Hierarchical linear models: Applications and data analysis methods (Vol. 1). Sage.

80.

Reisman

L. M. G.

(2024). How consultants shape nonprofits: Shared values, unintended consequences. Stanford University Press.

81.

Scott

(2000) Social network analysis (2nd ed.). Sage.

82.

Shumate

Cooper

K. R.

(2022). Networks for Social Impact. Oxford University Press.

83.

Suarez-Balcazar

Taylor-Ritzler

(2014). Moving from science to practice in evaluation capacity building. American Journal of Evaluation, 35(1), 95–99. https://doi.org/10.1177/109821401349944

84.

Taylor-Ritzler

Suarez-Balcazar

Garcia-Iriarte

Henry

D. B.

Balcazar

F. E.

(2013). Understanding and measuring evaluation capacity: A model and instrument validation study. American Journal of Evaluation, 34(2), 190–206. https://doi.org/10.1177/1098214012471421

85.

Umar

Hassan

(2019). Encouraging the collection of performance data in nonprofit organizations: The importance of organizational support for learning. Public Performance & Management Review, 42(5), 1062–1084. https://doi.org/10.1080/15309576.2018.1481118

86.

Uzzi

(1997). Social structure and competition in interfirm networks: The paradox of embeddedness. Administrative Science Quarterly, 42(1), 37–69. https://doi.org/10.2307/2393808

87.

Wang

Cooper

Shumate

(2020). The community system solutions framework. Stanford Social Innovation Review. https://ssir.org/articles/entry/community_system_solutions_framework_offers_an_alternative_to_collective_impact_model#

88.

Wholey

J. S.

(2010). Use of evaluation in government: The politics of evaluation. In Wholey

J. S.

Hatry

H. P.

Newcomer

K. E.

(Eds.), Handbook of practical program evaluation (3rd ed., pp. 651–667). John Wiley & Sons; Jossey-Bass.

89.

Wolff

(2001). A practitioner’s guide to successful coalitions. American Journal of Community Psychology, 29(2), 173–191. https://doi.org/0091-0562/01/0400-0173$19.50/0

90.

Wolff

Minkler

Wolfe

S. M.

Berkowitz

Bowen

Butterfoss

F. D.

Lee

K. S.

(2016). Collaborating for equity and justice: Moving beyond collective impact. Nonprofit Quarterly, 9, 42–53. https://nonprofitquarterly.org/collaborating-equity-justice-moving-beyond-collective-impact/

91.

Zorn

T. E.

Flanagin

A. J.

Shoham

M. D.

(2011). Institutional and noninstitutional influences on information and communication technology adoption and use among nonprofit organizations. Human Communication Research, 37(1), 1–33. https://doi.org/10.1111/j.1468-2958.2010.01387.x

Professional Data Use Among Nonprofits and Public Organizations in Collaborative Networks: The Influence of Partners,Coalitions,and Technical Assistance Organizations

Abstract

Keywords

Introduction

Organizational Learning, Institutional Theory, and Evaluation Capacity

Professional Data Use for Evaluation Capacity Building and Organizational Learning

Neo-institutional Theory and Isomorphism in Organizational Learning

Mechanisms Underlying Organizational Learning: Toward Evaluation Capacity in Collaborative Networks

Learning From Organizational Partners Within Ego-Centric Networks

Learning From Coalitions

Learning From Technical Assistance Organizations

Method

Network-Interviews

Surveys of Organizations in Coalitions

Measures

Coalition-Level Variables

Organizational-Level Variables

Analysis

Results

Discussion

Organizational Learning and the Lack of Influence of Coalitions

Mimetic Pressure and Organizational Learning Through Ego-Centric Partnerships

Implications for Organizational Learning

Limitations and Future Research

Conclusion

Footnotes

Appendix A

Appendix B

Appendix C

ORCID iDs

Funding

Declaration of Conflicting Interests

Data Availability Statement

Author Biographies

References