Sage Journals: Discover world-class research

Abstract

This paper responds to challenges around how to generate robust evidence in keeping with the principles of an asset-based approach based on mobilization of community strengths. The design of a collaborative evaluation of a multi-site Asset Based Community Development program is described and emergent learning discussed. A qualitative mixed method design was used to capture changes at community and program level drawing on diverse sources of evidence. Shared principles on the conduct of the evaluation were developed with program leads and community practitioners and opportunities for shared learning were built in. The paper distils learning on evaluation into six design features including the asset-based model as a framework, understandings of evidence and outcomes, ethical conduct, and the centrality of a collaborative and developmental approach. The paper concludes that these features form a coherent approach to asset-based evaluation which can link the theory and practice of Asset Based Community Development.

Plain Language Summary

Designing a collaborative evaluation of an Asset Based Community Development program

Asset Based Community Development (ABCD) is a way of working with communities based on understanding the strengths within communities. It is difficult to evaluate whether ABCD works as each community is different and there is very little agreement about the best way to research ABCD. This paper responds to these challenges and describes in detail how we went about designing a collaborative evaluation of an ABCD program across multiple neighborhoods in one city. The methods used to capture changes at community and program level are discussed. We developed shared principles with program leads and community practitioners about how the evaluation should run and built in lots of opportunities for shared learning. The paper distils this learning on evaluation into six design features which are all in keeping with an asset-based approach. The paper concludes that these features can be used to guide other asset-based evaluations as they link the theory and practice of Asset Based Community Development.

Keywords

Asset Based Community Development asset-based strengths-based case study design evaluation experiential knowledge

Introduction

The rise of asset-based approaches in health and care has been swift since the notion of an asset model for public health evidence was first proposed by Morgan and Ziglio (2007). Frustration with the limitations of a deficit model of research based on investigating risk and health need has not yet been replaced with shared understandings of how knowledge on health assets should be built (Alvarez-Dardet et al., 2015; de Andrade & Angelova, 2020; Rippon & South, 2017). There is broad consensus that the current evidence base for asset-based approaches is limited and there is scope for more robust research (Alvarez-Dardet et al., 2015; Cassetti et al., 2020; Hills et al., 2010; Morgan, 2014). This includes a distinct research gap around Asset-Based Community Development (ABCD) (Agdal et al., 2019; Blickem et al., 2018; Harrison et al., 2019), which is a major community-based intervention model (Asset-Based Community Development Institute, 2020). This is not a simple case of doing more research, as there is considerable debate about the most appropriate methodologies to use when evaluating asset-based approaches (Alvarez-Dardet et al., 2015; de Andrade & Angelova, 2020; Rippon & South, 2017). Morgan (2014) argues that the asset model requires researchers, policy makers and practitioners “to think and act differently” (p. 3). Those evaluating asset-based approaches need innovative research strategies to understand how assets can be mobilized and then be able to capture any resulting outcomes (Hills et al., 2010). A key research question concerns how asset-based principles can be used to co-develop evaluation designs for asset-based interventions such as ABCD. This paper responds to this methodological challenge by presenting the design of an asset-oriented evaluation of a multi-site ABCD program. Based on emergent learning from the evaluation, six key features of evaluation design are critically discussed. The purpose of the paper is to offer insight into ways of generating robust evidence that are aligned with an asset orientation and fit with the realities of community practice. We first provide a brief overview of ABCD and outline some of the methodological challenges associated with evaluation of asset-based approaches.

ABCD is an established model of community organizing that involves identification and mobilization of individual and community strengths (or assets) to effect positive change in disadvantaged neighborhoods and other settings (Kretzmann & McKnight, 1993; Mathie & Cunningham, 2005). Assets can be understood as any individual, community or organizational resource or protective factor that supports health or wellbeing (Morgan & Ziglio, 2007). At a community-level, assets can include aspects such as supportive networks or community cohesion as well as more tangible aspects such as land or schools (Van Bortel et al., 2019). ABCD as an approach to community building using community assets was originally developed in the US more than three decades ago (Kretzmann & McKnight, 1993). As interest has grown in the UK and Europe about the application of asset-based approaches in health and care, ABCD has emerged as one of the most influential traditions of thinking and practice (Agdal et al., 2019; Cassetti et al., 2020; Harrison et al., 2019; Hopkins & Rippon, 2015; Mathie & Cunningham, 2005). ABCD is also a recognized model in sustainable development (Mathie & Cunningham, 2005), neighborhood regeneration (Torbay Community Development Trust, 2020) and education (Forrester et al., 2020). Despite a wealth of experiential evidence derived from ABCD practice in community settings (Asset-Based Community Development Institute, 2020), there are relatively few robust studies that comprehensively capture the processes and outcomes associated with ABCD (Blickem et al., 2018). The challenge now is to develop a stronger evidence base for this important model using valid and appropriate methods.

Meeting this methodological challenge requires consideration of the difficulties and potential solutions. As others have argued, traditional approaches to public health evaluation offer a poor fit with the developmental nature of community-based-interventions and how they work within a community ecology (Durie & Wyatt, 2013; George et al., 2018; Trickett et al., 2011). ABCD is inherently relational, aimed at surfacing and strengthening social resources within a given community (Mathie & Cunningham, 2005), rather than delivering standard intervention components for specific population groups. ABCD is described as being citizen-led and relationship-oriented (Agdal et al., 2019). Community ownership is likely to evolve over time (Andajani-Sutjahjo et al., 2018; de Andrade & Angelova, 2020), suggesting the need for longitudinal research. Activities and assets will vary between communities and this lack of uniformity makes evaluation challenging. Furthermore, the logic of a social constructivist epistemology where communities identify meaningful assets in context (McKnight, 2010) may be at odds with the information needs of decision makers (Mitchell, 2017).

Paradoxically, the relational features, and mechanisms of action, that make ABCD a conceptually coherent model for community practice pose challenges for researchers who wish to evaluate asset-based approaches. Yet it remains important to “grasp the nettle”. This requires developing evaluation designs that fit with the ethos of an asset orientation and can generate robust evidence at a community-level (Hills et al., 2010). There is little consensus on the best measures and methods. A previous narrative literature review undertaken by two of the authors (Rippon & South, 2017) highlighted the diversity of approaches to measurement, ranging from practical strategies used in community organizing, such as asset mapping (Sharpe et al., 2000), through to development of validated scales (Becker et al., 2015). The conclusions of the review were that evaluation should aim to capture the multi-dimensional nature of health assets (Ickovics et al., 2014; van de Venter & Redwood, 2016), use a theory of change or logical framework to improve rigor (Sigerson & Gruer, 2011), and have community involvement in the research process (Mathie & Cunningham, 2005; Rippon & South, 2017).

ABCD works through strengthening connections and social action within communities at different levels. Using a theory of change approach to surface the links between context, activities, mechanisms of change and outcomes in ABCD has been recommended (Blickem et al., 2018; Sigerson & Gruer, 2011). This is supported by the original rationale for using theory of change in evaluating neighborhood-based comprehensive community initiatives (Connell & Kubisch, 1988). Stakeholder engagement is a critical feature of the theory of change approach in terms of articulating assumptions and valued outcomes (Blamey & Mackenzie, 2007). This aligns with the participatory ethos of asset-based practice (de Andrade & Angelova, 2020).

The developmental nature of ABCD means that there is scope to learn from doing. In relation to the challenge of evaluating asset-based interventions, Hills et al. (2010) argue for greater collaboration and that “it is time to stop fretting about how difficult it is and time to start developing innovative solutions” (Hills et al., 2010: 97). A small number of studies have provided critical commentaries on the choice of research strategies in specific asset-based evaluations (de Andrade & Angelova, 2020; Durie & Wyatt, 2013). This paper builds on this tradition by reporting and reflecting on a collaborative evaluation of a multi-site ABCD program in a large UK city.

Study Context and Aims

The evaluation setting was a large city in the North of England, with significant social and health inequalities. The goals of building more resilient and inclusive communities led to the local authority establishing a pilot ABCD program with three community-based organizations in 2017, and then in 2019 expanding this to 12 community-based organizations working at neighborhood level, known as “pathfinder sites.” The ABCD program had fidelity with the original model of neighborhood-based community organizing developed in the US (Kretzmann, 1998; Kretzmann & McKnight, 1993). Each community-based organization employed a community development worker, known as a “community builder,” to map local assets and connect with residents wanting to be actively involved. Using community organizing methods, community builders facilitated community-led action to improve the social and physical environment of the neighborhood, such as setting up new community groups or activities. A small local authority team commissioned and oversaw the program, providing coordination and ongoing support to the network of community builders, and community-based organizations where required.

The implementation of this major asset-based program was accompanied by a two-year collaborative evaluation to gather learning and evidence impact on ABCD to inform local decision making. This created a significant opportunity to develop an asset-oriented evaluation capable of measuring progress whilst maintaining alignment with the principles of ABCD. An academic research team was commissioned to evaluate the program, working closely with the local authority ABCD team and the network of community builders. There was a shared ambition to address evidence gaps while respecting community knowledge. The main evaluation objectives were to articulate the ABCD model and how it worked, and to gather data on processes and impacts at individual, community and program levels. A third objective was to facilitate shared learning on ABCD including implementation, leadership and evaluation.

Study Design and Methods

Evaluation Design

The evaluation was designed to fit with the strengths-based ethos and principles of ABCD (Mathie & Cunningham, 2005) and to incorporate research methods appropriate for capturing change at different levels (individual, community and organizational) (Public Health England, & NHS England, 2015). We attempted to apply key recommendations from the earlier narrative review (Rippon & South, 2017) to the ABCD evaluation including taking a collaborative approach that respected community knowledge and developing a theory of change (Connell & Kubisch, 1988) to explore mechanisms and expected outcomes with stakeholders (Blamey & Mackenzie, 2007). Prior to the evaluation, an initial ABCD program theory of change had been developed by the local authority ABCD leads with community and academic input (involving one of the authors). This theory of change provided indicative evaluation priorities (Mackenzie & Blamey, 2005) grouped around three primary program outcomes articulated as: (i) individuals and communities are better connected; (ii) communities identify and work to bring about the changes they want to see (iii) people have good friends.

A mixed methods qualitative design was used, which incorporated various elements of comparison. The evaluation was multi-site, aimed to track the development of the program over 2 years, and drew on different stakeholder perspectives at community and program level. A thematic synthesis brought findings together from different components (see Figure 1). Informed by case study design (Yin, 2009), the focus was on understanding whether and how the ABCD program (as the primary case) worked, with a sample of pathfinder sites (as sub-units of analysis) offering varied perspectives on ABCD practice.

Figure 1.

ABCD evaluation design flow chart.

Methods

This was a collaborative evaluation working with the local authority ABCD team and community builders throughout design, data collection and synthesis. A formative evaluation workshop was held with the research team, local authority team and some experienced community builders, where we explored the theory of change, identified suitable methods for community fieldwork and agreed how we would work together (Table 1).

Table 1.

Guiding Principles for the ABCD Evaluation.

Taking an asset-based approach to the evaluation means that:
•we will work collaboratively with stakeholders to gather evidence
•we will respect the wisdom and experience of those working and living in communities
•we will seek to identify strengths as well as needs
•we will commit to sharing learning so that those involved in community building can benefit from the evaluation.

The evaluation was informed by an epistemological stance in public health that seeks understanding of community ecologies (Trickett et al., 2011) and values experiential knowledge (Raphael & Bryant, 2002). The design therefore drew on various sources of evidence, mostly gathered through qualitative methods (Table 2). Primary data collection was undertaken by the research team using interviews, focus groups and community walks. Secondary data sources were the program monitoring reports and self-evaluation evidence collected by community builders.

Table 2.

Evidence Sources & Methods.

Evidence source	Methods	Type of data	Scope of evidence[links to theory of change]
Case studies -ABCD pathfinder sites	Community walks Interviews with community builders (face-to-face and phone) Focus group interviews with volunteers and community members Researcher visits with notes	Primary Qualitative	Perspectives on a pathfinder site: •local context & assets [context] •development of ABCD [activities] •community builder/connector roles [mechanisms] •influencing factors •outcomes [outcomes]
Stakeholder interviews—program level	Individual interviews with stakeholders from the local authority staff and from the voluntary and community sector (online and phone)	Primary Qualitative	Perspectives from those supporting ABCD program: •program model [mechanisms] •commissioning •program delivery; training & support [activities] •influencing factors •links to health & care agendas[context]
Self-evaluation	Evaluation training workshops for community builders × 2 Community builders select from range of data collection methods: survey, individual case studies, community stories, photo voice.	Secondary Qualitative (potential for quantitative)	1 × piece of evidence demonstrating community-level change for one of the three program outcomes: social connectedness; community-led change; friendship. [outcomes] Evidence selected & gathered by community builders.
Program Monitoring data	Secondary analysis of anonymised quarterly monitoring reports Program-level data & documentation	Secondary Qualitative Quantitative	Detailed accounts of activities, outputs & outcomes from individual pathfinders including: •Numbers of activities, events etc [activities] •Numbers of volunteers recruited •Progress & learning summarized from diaries [outcomes] •Program inputs & costs [activities]
Summative learning workshops	Online workshops with participatory exercises × 2	Primary Qualitative	Presentation & validation of emerging findings Critical discussion of outliers, additional evidence, attribution to ABCD and external factors. [context; mechanisms; outcomes]

Qualitative primary research was undertaken with a sample of six pathfinder sites. In order to tap into the relational way that the intervention worked (Mathie & Cunningham, 2005), a naturalistic approach was taken to data collection (Silverman, 2006). Community walks, a method developed in US for asset mapping (Sharpe et al., 2000), were used alongside interviews with community builders to elicit how assets were mobilized and what if any outcomes resulted. Two focus group discussions with community volunteers and residents associated with ABCD activities were also held at familiar community venues. These discussions focused on the local “asset map,” using creative techniques to stimulate discussion (Sharpe et al., 2000). Community fieldwork was later supplemented with interviews with stakeholders at a program level working in the local authority or voluntary sector organizations. All interviews and focus groups were recorded and transcribed verbatim. Researcher notes were written up following community walks.

Secondary analysis of program monitoring data was carried out. This evidence came primarily from quarterly reports submitted by pathfinder sites. Community builders documented activities, outputs and outcomes qualitatively through diaries and case studies, and quantitatively by collating numeric data on community volunteers, grants, activities and groups associated with ABCD. Summary reports from each site were then submitted to the local authority ABCD team. The research team were able to analyze quarterly reports from 11 of 12 pathfinders between October 2019 and September 2020, supplemented by some program data.

Secondary data were also generated through ABCD pathfinder sites. Following the formative workshop, the research team held two further evaluation training workshops with the aim of supporting community builders and local leads from community-based organizations across all sites to be able to evidence their journey of change. Community builders committed to capturing one piece of evidence on outcomes that could be shared through the evaluation. This could be visual representation of change, such as a photo or diagram, or text-based, such as an individual’s story about how they become involved. Later in the evaluation, the community builders and local leads were involved in analysis through their participation in summative learning workshops. This generated some further primary data from group discussions which was used to help refine initial analysis.

The evaluation was conducted according to the guidelines of the Social Research Association (2003) and received ethical approval through Leeds Beckett University Research Ethics - Ref 91128. Appropriate ethical safeguards were put in place for all interviews, focus groups and walks to ensure confidentiality, anonymity and informed consent. A process was agreed for handling anonymised monitoring data, which reported community activities, outputs and outcomes using both quantitative and qualitative data. Community builders sending in monitoring reports to the local authority ABCD team were notified in advance, verbally and in writing, that these would be used in the evaluation. It was decided not to use any case studies of individuals collated by the local authority ABCD team because anonymity could not be preserved in processing the data. The evaluation training workshops developed awareness of ethical practice in community research; however, community builders independently decided on what evidence they collected in agreement with community members.

Sampling

In line with case study design where there are multiple sites (Yin, 2009), a purposive sampling strategy aimed for maximum variation in terms of diverse areas and a range of stakeholder perspectives (Patton, 2002). ABCD pathfinders were selected based on their stage of development, with three “embedded” sites where ABCD had been successfully implemented in the first phase, and three “emerging” sites where ABCD had been recently established. There was potential to undertake further interviews with other pathfinders to explore difficulties in implementation. The sample of participants and the selection of community walks was developed in collaboration with community builders within individual pathfinder sites and with local authority leads for stakeholder interviews. In the second year of the evaluation, dealing with the COVID-19 pandemic curtailed data collection in the pathfinder sites, as most were heavily involved in the community response to the pandemic. Where possible face-to-face data collection was shifted to online and phone interviews.

Analysis

The evaluation generated a significant amount of data, despite the impact of the pandemic. Data analysis was based on a thematic approach (Braun & Clarke, 2006) and qualitative findings were analyzed using NVIVO software to help with systematic coding and organization of data. A thematic chart was developed using the initial theory of change as a framework supplemented by inductive themes developed through open coding of data from pathfinder sites. The framework was expanded and refined as qualitative results from the secondary analysis of monitoring reports were charted. An additional thematic chart was created for the program-level stakeholder interviews. Cross-case analysis was undertaken to map major themes, common change mechanisms and outcomes (Lee & Chavis, 2012). Vignettes of the six sampled pathfinder sites were also written up to preserve the context-bound narratives of community change.

An additional element was a pilot Social Return on Investment (SROI) analysis of two pathfinder sites, one established in the first phase and one in the second phase. This element was introduced late in the evaluation at the request of local authority leads to aid decision-making around future investment in asset work. This used standard SROI methods (Nicholls et al., 2009) to interrogate the monitoring data around key aspects of social value (Global Value Exchange, undated), such as increases in social interactions with friends, relatives and neighbors (Powdthavee, 2008).

Synthesis and Validation

The final stage was a synthesis of findings and respondent validation of the main conclusions (Figure 1). Interim findings were presented at two summative learning workshops, held online in November 2020. Participants included the local authority ABCD team, community builders, local leads from community-based organizations and the research team. The aim of the workshops was to allow those directly involved in the evaluation to sense check the findings and discuss the implications for ABCD practice and research through a mix of interactive plenary and group discussions.

Synthesis involved summarizing and mapping all findings, cross referenced to evidence sources, against the program theory of change and refining it to achieve the best fit with the data. Using a matrix, this allowed us to see where triangulation occurred, and where contexts differed (Lee & Chavis, 2012). In order to inform local authority decision making around future commissioning of the ABCD program, evidence statements were rated as Strong, Promising or Initial, according to the terminology around strength of evidence adopted by What Works Wellbeing (Snape et al., 2019).

Results

In this methodology paper, we now consider the evaluation results in terms of two aspects. Firstly, how the qualitative findings arose from the logic of the enquiry and the application of an asset-oriented design. Secondly, the methodological contribution represented by the development of six evaluation design features aligned to ABCD principles.

The application of the evaluation design described above resulted in a synthesized set of qualitative findings, which were mapped to the program theory of change framework. Figure 2 displays the final qualitative framework as an abstracted summary of results across this theory of change. We used the ABCD program theory, as developed by stakeholders (Mason & Barnes, 2007), together with data collected (Table 2), to draw conclusions about whether, how and why the ABCD program was working. The asset-oriented evaluation design meant that we were able to provide an account of underpinning relational mechanisms for ABCD and the links to community-level outcomes. Evaluation results were used to inform program and pathfinder development. The detailed thematic findings from the evaluation are reported elsewhere (South et al., 2021).

Figure 2.

Summary of final ABCD theory of change.

An important outcome from the evaluation was the emergent learning from the development and application of an asset-oriented design. Throughout the evaluation, design choices and evaluation practice were documented and reflected on by the research team in dialog with local authority leads, ABCD team and community builders. Much discussion centered on balancing requirements to gather robust evidence with the use of a naturalistic research approach consistent with asset-based working and acceptable to community participants. Additionally, post evaluation we conducted a short email survey to gather feedback from those with direct involvement in data collection, including the community builders. Based on learning from the implementation of the evaluation, we were able to identify six evaluation design features that are consistent with the principles and practice of ABCD and have yielded robust results.

Key Design Features for Evaluating an ABCD Programme

The six design features for ABCD evaluation are now explained, with reference to underpinning literature. Figure 3 represents how the features could form a coherent approach to evaluating ABCD in community settings.

Figure 3.

Key design features for evaluating ABCD.

A Coherent Asset-Based Model Linked to an Evaluation Framework

Conceptual clarity was critical in this evaluation. This started with the foundational work led by the local authority to develop a coherent asset-based program aligned to the relational principles of ABCD (Keenan & Ward, 2020). The program theory of change was considered in all evaluation workshops (Figures 1 and 2) and participants confirmed that it was consistent with how the program worked and the outcomes that mattered to those involved. This then formed the framework for data collection and was built on in the analysis. The program theory of change created challenges in the evaluation; for example, “people have good friends” was consistently endorsed as a valued primary outcome but measuring this was very difficult. The local authority team and community builders acknowledged the subjectivity of the concept of “a good friend,” nevertheless they strongly supported the inclusion of this outcome because it encapsulated a relational asset-based approach. Overall, the articulation of the ABCD model and the local program theory of change together formed an important frame for conducting the evaluation.

Valuing Diverse Sources of Evidence

Learning confirmed the importance of a mixed method design that acknowledged the value of experiential evidence (Figure 1 & Table 1). Questions on what counted as evidence and what was an asset were discussed and documented (de Andrade & Angelova, 2020). This places asset-based evaluation firmly in a research paradigm that seeks a context-specific account of what matters in communities (de Andrade & Angelova, 2020; McKnight, 2010). Drawing out cross-cutting themes was possible because of triangulation of data sources, including evidence gathered by community builders and monitoring data capturing narratives of pathfinder development (Table 2). Multiple data sources meant that a robust analysis process was required to synthesize data and develop an explanatory account of program mechanisms and outcomes (Lee & Chavis, 2012; Miles et al., 2014).

Use of Naturalistic Methods

Context is key in community-based research (south et al., 2020) and the evaluation needed to be sensitive to diverse community contexts represented by the pathfinder sites. Community walks as a method had value in exploring assets from a community perspective (Sharpe et al., 2000), despite practical challenges with the weather and recording. In general, we found that a naturalistic approach to data collection garnered important insights, but this could be at odds with the technical language of asset-based approaches. An asset map appeared to be a fluid notion in community practice. It was rarely the expected inventory of assets (Sharpe et al., 2000) that could be used in the evaluation.

Capturing Outcomes at Community-Level and Organisational-Level

Asset-based approaches work at different levels—engaging individuals, building community capacity and reorienting organizations (Scottish Community Development Centre, 2013). These levels need to be reflected in assessing outcomes (Cassetti et al., 2020; Hills et al., 2010). Community-level changes are difficult to capture in the short term and our qualitative primary research explored the influence of context on community-led activity and what outcomes emerged. The self-evaluation element was a small part of the evaluation, nonetheless it was helpful in capturing some community changes and in building relationships with researchers and community builders. Growing understanding of the ABCD program led to additional stakeholder interviews to explore mechanisms at an organizational level and what type of infrastructure was required.

Navigating Ethical Community Research

Community fieldwork requires skilled researchers who develop respectful relationships with participants and community organizations (Warwick-Booth, 2021). In the formative evaluation workshop, participants wanted a commitment that researchers would share findings with them and not disappear once data had been extracted. This was written into the agreed principles (Table 1). In terms of methods, community builders welcomed the research team doing orientation visits and walks. While this may avoid the research being intrusive, especially where informal activities are taking place, researchers still need to be transparent about what is happening and obtain consent when collecting data. We spent considerable time developing protocols and accessible information about the study, such as postcards that with different scenarios to explain why researchers were present. A further ethical issue was how secondary monitoring data, particularly community-derived evidence, was handled as there were potential risks to anonymity and confidentiality when context-specific stories were documented by community builders. Again, this took time working with all stakeholders to agree ethical protocols that allowed some secondary monitoring data on community activities and outcomes to be used with appropriate safeguards. Individual case studies were not used; however, it was recognized that valuable data on outcomes could be reported through monitoring reports and interviews, without the risks to anonymity.

A Developmental and Collaborative Approach

Early on we co-created a set of guiding principles for the evaluation that reflected the importance of collaboration (Table 1). In discussions on the evaluation process at the summative workshops and in the final feedback survey, community builders confirmed the ongoing relevance of those principles. Working together allowed shared interpretations to emerge and some respondent validation of conclusions. Taking a developmental stance meant that the evaluation could evolve alongside the ABCD program. This allowed us to respond to changing priorities, and to opportunities and challenges as they arose. A major disruptive event being the COVID-19 pandemic, which had a significant impact on how community-based research was conducted (Shand & Jarvis, 2021). Maintaining contact through this difficult time helped retain engagement in the evaluation. Working closely with the local authority ABCD team and community builders meant we were able to adapt planned data collection to fit with community priorities and to use alternatives such as online and phone interviews when social restrictions prevented fieldwork.

Discussion

An improved evidence base on asset-based approaches is needed to fulfill the agenda originally set out by Morgan and Ziglio (2007). This paper contributes to those wider debates in setting out an original methodological approach for evaluating a neighborhood-based ABCD program and discussing how asset-based principles were incorporated into the study design. This paper addresses the recommendation of Hills et al. (2010) to improve the evaluation of asset-based interventions by engaging in, and learning from, evaluative studies. It is one of the first systematic program-wide evaluations of ABCD, thereby adding to small number of papers that critically discuss research strategies in this area. Transparency about key evaluation features and critical reflection on what worked and what did not will help other researchers in undertaking community-based evaluation, whether in the field of public health, sustainable development or community organizing. The set of design features presented here reflect core themes in the theory and practice of asset-based approaches in relation to the nature of assets (Van Bortel et al., 2019), collaborative endeavor, and relational mechanisms of change (Harrison et al., 2019; Mathie & Cunningham, 2005). In setting out a coherent approach to evaluating ABCD, the paper contributes to wider methodological debates on how best to gather evidence on developmental community work (Durie & Wyatt, 2013; George et al., 2018; Trickett et al., 2011).

Our learning confirms that evaluation approaches need to be consistent with asset-based theory and practice and therefore to acknowledge relational and contextual elements (de Andrade & Angelova, 2020; Rippon & South, 2017). In this evaluation, incorporating relational elements from the beginning, such as the formative evaluation workshop, and critically recognizing diverse sources and types of evidence strengthened the design. This is in keeping with theoretical understandings of how asset-based approaches draw on community knowledge to develop collective activity (Agdal et al., 2019). The design builds on the conceptual work of Blickem et al. (2018) and Cassetti et al. (2020) in examining ABCD mechanisms and outcomes, and in using a theory of change framework to guide analysis (Blamey & Mackenzie, 2007). The paper adds a further example of research practice that navigates the complexity of asset-based working in and with communities (de Andrade & Angelova, 2020). The six design features presented in this paper could offer some guidelines for developing future ABCD evaluations (Figure 3). We agree with George et al. (2018) that some humility is needed over the complexity of evaluating participatory approaches, and therefore this papers seeks to be a foundation for further discussion, not a fixed solution.

The challenges of undertaking research that adheres to asset-based principles, while attempting to produce valid evidence for decision makers, should not be underestimated. We concur with Morgan (2014) that the asset model is genuinely a radical approach that challenges thinking and practice. Community-level outcomes are difficult to measure, especially with developmental, participatory interventions (George et al., 2018; Rifkin, 2014). From our experience, the formal language of measurement does not always fit easily with articulation of meaningful outcomes that are understood by participants engaged in community practice. This suggests that in-depth qualitative research teasing out interpretations and community priorities is warranted (de Andrade & Angelova, 2020). A theory of change approach was also a useful framework to help synthesize results. This supports wider arguments about its value as an approach in community-based evaluation (Breuer et al., 2015; Connell & Kubisch, 1988). A constant theme of discussion was the tension between fidelity to an evaluation approach aligned to ABCD and producing credible evidence to inform local authority decisions. Our experience confirms the importance of a collaborative and iterative approach to evidence generation, which has potential to support scaling and adaptation as a program grows (Leask et al., 2019). An earlier local authority working group had helped foster links between the university and ABCD leads, which were then strengthened through the evaluation. The local authority leads, ABCD team and community builders were willing to engage in nuanced discussions about evidence and those discussions informed the features presented in this paper. Later feedback suggested that a collaborative approach to evaluation was highly valued. An acknowledged gap with quantitative data has led to further work with the local public health department to explore how community insights on assets can be combined with analysis of routinely collected data.

Limitations

Having multiple sources of evidence helped develop a layered analysis; however, there were limitations with using secondary sources as there was no guarantee of rigor or consistency in how project monitoring data were collected. Although the evaluation aimed to build community capacity to self-evaluate, there was a risk that this placed an additional burden on projects compared to a fully researcher-led study (Salway et al., 2015). The pandemic further reduced the capacity that community-based organizations and staff had to engage with research (Shand & Jarvis, 2021). This resulted in less community-generated evidence than anticipated, despite the earlier workshops providing training on evaluation skills.

Implementing the research design highlighted the difficulty of attributing change to the intervention. The longitudinal element and retrospective community stories help evidence the theory of change; however, narratives are dependent on who is at the table to tell them. The perspectives of community builders and some active community members were included, but there was no primary data collection with the wider communities. This is a gap and should be addressed in future research.

Conclusions

There is broad agreement that asset-based approaches, including ABCD as a neighborhood model, require a reorientation of research practice. There is much less consensus on how that might be achieved and what methods are most appropriate. Transparent accounts of design choices and evaluation priorities are needed to advance the science and art of asset-based research in communities. The methodological approach explored in this paper aligns with the theory and principles of asset-based approaches and attempts to translate these to pragmatic research strategies. Key design features offer a coherent approach to evaluation, underpinned by asset theory and sensitive to the practice of ABCD. Core arguments concern the radical shift which needs to occur in how we do research and the necessity for pragmatic approaches to evidence gathering that fit with the developmental activity taking place in and by communities. Looking forward, there is scope for identifying alternative approaches for the evaluation of ABCD and asset-based approaches more generally, and for methods to enhance rigor in community-based evaluations. Better evaluation will be the foundation for an improved evidence base for asset-based approaches in health and care.

Footnotes

Acknowledgements

The authors would like to thank all those individuals from Leeds City Council and from community-based organizations who collaborated on the evaluation and committed to share their learning. In particular, Carmel Langstaff, Leeds City Council, who provided strategic oversight of the evaluation from 2020 and Verity O’Hara, Leeds City Council ABCD team, who coordinated monitoring and links to the community builders’ network. Also, thanks to Lisa Keenan and Mick Ward, formerly of Leeds City Council, whose ideas shaped the early design and initial theory of change. Further information about Leeds ABCD program can be found at:

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by funding from Leeds City Council, Adults & Health Directorate, UK, to inform the development of their ABCD program.

ORCID iDs

Jane South

Susan Coan

Anne-Marie Bagnall

Data Availability

Thematic findings, case study summaries and the full theory of change can be found in the project report which is available from:

References

Agdal

Midtgård

I. H.

Meidell

(2019). Can asset-based community development with children and youth enhance the level of participation in health promotion projects? A qualitative meta-synthesis. International Journal of Environmental Research and Public Health, 16(19), 3778. https://doi.org/10.3390/ijerph16193778

Alvarez-Dardet

Morgan

Cantero

M. T. R.

Hernán

(2015). Improving the evidence base on public health assets—the way ahead: A proposed research agenda. Journal of Epidemiology and Community Health, 69(8), 721–723. https://doi.org/10.1136/jech-2014-205096

Andajani-Sutjahjo

Liew

T. C. H.

Smith

J. F.

Esekielu

Mason

Tariu

(2018). Engaging community volunteers in participatory action research in Tāmaki community of Auckland, New Zealand. Health Promotion International, 33(2), 219–228. https://doi.org/10.1093/heapro/daw057

Asset-Based Community Development Institute. (2020). Asset-Based Community Development Institute. De Paul. Retrieved 14th May from https://resources.depaul.edu/abcd-institute/Pages/default.aspx

Becker

C. M.

Chaney

B. H.

Shores

Glascoff

(2015). The salutogenic wellness promotion scale for older adults. American Journal of Health Education, 46(5), 293–300. https://doi.org/10.1080/19325037.2015.1055019

Blamey

Mackenzie

(2007). Theories of change and realistic evaluation: Peas in a pod or apples and oranges? Evaluation, 13(4), 439–455.

Blickem

Dawson

Kirk

Vassilev

Mathieson

Harrison

Bower

Lamb

(2018). What is asset-based community development and how might it improve the health of people with long-term conditions? A realist synthesis. Sage Open, 8, 1–13. https://doi.org/10.1177/2158244018787223

Braun

Clarke

(2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77–101.

Breuer

Lee

De Silva

Lund

(2015). Using theory of change to design and evaluate public health interventions: A systematic review. Implementation Science, 11(1), 63. https://doi.org/10.1186/s13012-016-0422-6

10.

Cassetti

Powell

Barnes

Sanders

(2020). A systematic scoping review of asset-based approaches to promote health in communities: Development of a framework. Global Health Promotion, 27, 15–23. https://doi.org/10.1177/1757975919848925

11.

Connell

J. P.

Kubisch

A. C.

(1988). Applying a theory of change approach to the evaluation of comprehensive community initiatives: Progress, prospects and problems. In Fulbright-Anderson

Kubisch

Connell

(Eds.), New approaches to evaluating community initiatives (Vol. 2). Theory, measurement and analysis, (pp. 1–13). The Aspen Institute.

12.

de Andrade

Angelova

. (2020). Evaluating and evidencing asset-based approaches and co-production in health inequalities: measuring the unmeasurable? Critical Public Health, 30(2), 232–244. https://doi.org/10.1080/09581596.2018.1541229

13.

Durie

Wyatt

(2013). Connecting communities and complexity: A case study in creating the conditions for transformational change. Critical Public Health, 23(2), 174–187.

14.

Forrester

Kurth

Vincent

Oliver

(2020). Schools as community assets: An exploration of the merits of an asset-based community development (ABCD) approach. Educational Review, 72(4), 443–458. https://doi.org/10.1080/00131911.2018.1529655

15.

George

A. S.

LeFevre

A. E.

Schleiff

Mancuso

Sacks

Sarriot

(2018). Hubris, humility and humanity: Expanding evidence approaches for improving and sustaining community health programmes. BMJ Global Health, 3, e000811. https://doi.org/10.1136/bmjgh-2018-000811

16.

Global Value Exchange (undated). Global Vaue exchange tool. Global Value Exchange. Retrieved 20th June, 2022. http://www.globalvalueexchange.org

17.

Harrison

Blickem

Lamb

Kirk

Vassilev

(2019). Asset-based community development: Narratives, practice, and conditions of Possibility—A qualitative study with community practitioners. Sage Open, 9(1), https://doi.org/10.1177/2158244018823081

18.

Hills

Carroll

Desjardins

(2010). Assets based interventions: evaluating and synthesizing evidence of the effectiveness of the assets based approach to health promotion. In Morgan

Davies

Ziglio

(Eds.), Health assets in a global context: Theory, methods, action (pp. 77–98). Springer.

19.

Hopkins

Rippon

(2015). Head, hands and heart: asset-based approaches in health care: A review of the conceptual evidence and case studies of asset-based approaches in health, care and wellbeing.

20.

Ickovics

J. R.

Carroll-Scott

Peters

S. M.

Schwartz

Gilstad-Hayden

McCaslin

(2014). Health and academic achievement: cumulative effects of health assets on standardized test scores among urban youth in the United States. Journal of School Health, 84(1), 40–48. https://doi.org/10.1111/josh.12117

21.

Keenan

Ward

(2020). Asset-based approaches in local authorities: the Leeds experience. https://www.local.gov.uk/asset-based-approaches-local-authorities-leeds-experience

22.

Kretzmann

(1998). Building the Bridge from Client to Citizen: A Community Toolbox for Welfare Reform. www.abcdinstitute.org/docs/ClienttoCitizen.pdf

23.

Kretzmann

McKnight

(1993). Building communities from the Inside Out: A path toward finding and mobilizing a community’s assets. Institute for Policy Research.

24.

Leask

C. F.

Sandlund

Skelton

D. A.

Altenburg

T. M.

Cardon

Chinapaw

M. J. M.

De Bourdeaudhuij

Verloigne

Chastin

S. F. M.

, ; GrandStand, Safe Step and Teenage Girls on the Move Research Groups. (2019). Framework, principles and recommendations for utilising participatory methodologies in the co-creation and evaluation of public health interventions. Research Involvement and Engagement, 5(1), 2. https://doi.org/10.1186/s40900-018-0136-9

25.

Lee

K. S.

Chavis

D. M.

(2012). Cross-case methodology: Bringing rigour to community and systems change research and evaluation. Journal of Community & Applied Social Psychology, 22(5), 428–438.

26.

Mackenzie

Blamey

(2005). The practice and the theory:Lessons from the application of a theories of change approach. Evaluation, 11(2), 151–168. https://doi.org/10.1177/1356389005055538

27.

Mason

Barnes

(2007). Constructing theories of change: Methods and sources. Evaluation, 13, 151–170.

28.

Mathie

Cunningham

(2005). Who is driving development? Reflections on the transformative potential of asset-based community development. Canadian Journal of Development, 26(1), 175–186.

29.

McKnight

(2010). Asset mapping in communities. In Morgan

Davies

Ziglio

(Eds.), Health assets in a global context: Theory, methods, action (pp. 59–76). Springer.

30.

Miles

M. B.

Huberman

A. M.

Saldaña

(Eds.). (2014). Qualitative data analysis. A methods sourcebook (3rd ed.). Sage.

31.

Mitchell

(2017). Owning the process - taking a ’Thriving Places’ approach to asset mapping. A case study in the Thriving Places series. Practitioner Research Report (What Works Scotland, Ed.). What Works Scotland. https://whatworksscotland.ac.uk/publications/thriving-places-case-studies/

32.

Morgan

(2014). Revisiting the asset model: A clarification of ideas and terms. Global Health Promotion, 21(2), 3–6.

33.

Morgan

Ziglio

(2007). Revitalising the evidence base for public health: An assets model. Promotion & Education, Suppl 2(Supplement 2), 17–22.

34.

Nicholls

Lawlor

Neitzert

Goodspeed

(2009). A guide to social return on investment. Cabinet Office.

35.

Patton

M. Q.

(2002). Qualitative research and evaluation methods. Sage.

36.

Powdthavee

(2008). Putting a price tag on friends, relatives, and neighbours: Using surveys of life satisfaction to value social relationships. Journal of Socio-Economics, 37, 1459–1480.

37.

Public Health England, & NHS England. (2015). A guide to community-centred approaches for health and wellbeing. https://www.gov.uk/government/publications/health-and-wellbeing-a-guide-to-community-centred-approaches

38.

Raphael

Bryant

(2002). The limitations of population health as a model for the new public health. Health Promotion International, 17(2), 189–199

39.

Rifkin

S. B.

(2014). Examining the links between community participation and health outcomes: A review of the literature. Health Policy and Planning, 29Suppl 2, ii98–106. https://doi.org/10.1093/heapol/czu076

40.

Rippon

South

(2017). Promoting Asset Based Approaches for Health and Wellbeing: Exploring a Theory of Change and Challenges in Evaluation. Leeds Beckett University. http://eprints.leedsbeckett.ac.uk/4497/

41.

Salway

Chowbey

Such

Ferguson

(2015). Researching health inequalities with community researchers: practical, methodological and ethical challenges of an ‘inclusive’ research approach. Research Involvement and Engagement, 1(1), 9. https://doi.org/10.1186/s40900-015-0009-4

42.

Scottish Community Development Centre. (2013). Assets based approaches to health improvement - creating a culture of thoughtfulness, briefing paper. SCDC.

43.

Shand

Jarvis

(2021). Responding to COVID-19 in the Liverpool City Region. The Impact of COVID-19 on Methods and Approaches to Community-Based Participatory Research. https://www.liverpool.ac.uk/media/livacuk/publicpolicyamppractice/covid-19/PB039.pdf

44.

Sharpe

P. A.

Greaney

M. L.

Lee

P. R.

Royce

S. W.

(2000). Assets-oriented community assessment. Public Health Reports, 115(2-3), 205–211.

45.

Sigerson

Gruer

(2011). Asset-based approaches to health improvement. www.healthscotland.com/uploads/documents/17101-assetBasedApproachestoHealthImprovementBriefing.pdf

46.

Silverman

(Ed.). (2006). Intrepreting qualitative data. Methods for analyzing talk, text and interaction (3rd ed.). Sage.

47.

Snape

Meads

Bagnall

A. M.

Tregaskis

Mansfield

MacLennan

Brunetti

(2019). A guide to our Evidence Review Methods. Revised 2019. What Works Wellbeing. https://whatworkswellbeing.org/resources/updated-a-guide-to-our-evidence-review-methods/

48.

Social Research Association. (2003). Ethical guidelines. Social Research Association. https://the-sra.org.uk/common/Uploaded%20files/ethical%20guidelines%202003.pdf

49.

South

Button

Quick

Bagnall

Trigwell

Woodward

Coan

Southby

(2020). Complexity and community context: Learning from the evaluation design of a national community empowerment programme. International Journal of Environmental Research and Public Health, 17(1), 91. https://doi.org/10.3390/ijerph17010091

50.

South

Woodward

Coan

Bagnall

Rippon

(2021) Asset based community development: evaluation of Leeds ABCD. Project Report. Leeds Beckett University. http://eprints.leedsbeckett.ac.uk/id/eprint/7640/

51.

Torbay Community Development Trust. (2020). Community Building: Connecting people and place to build community and reduce social isolation. 2020 Learning Report.

52.

Trickett

E. J.

Beehler

Deutsch

Green

L. W.

Hawe

McLeroy

Miller

R. L.

Rapkin

B. D.

Schensul

J. J.

Schulz

A. J.

Trimble

J. E.

(2011). Advancing the science of community-level interventions. American Journal of Public Health, 101(8), 1410–1419.

53.

Van Bortel

Wickramasinghe

N. D.

Morgan

Martin

. (2019). Health assets in a global context: A systematic review of the literature. BMJ Open, 9(2), https://doi.org/10.1136/bmjopen-2018-023810

54.

van de Venter

Redwood

(2016). Does an asset-based community development project promote health and wellbeing? Lancet, 388, S108. https://doi.org/10.1136/bmjopen-2018-023810

55.

Warwick-Booth

Bagnall

Coan

. (2021). Ethics of participatory research. In Creating participatory research: Principles, practice and reality (pp. 65–80) Policy Press.

56.

Yin

Y. K.

(Ed.). (2009). Case study research: Design and methods (4th ed.). Sage.

Asset Based Community Development: Co-Designing an Asset-Based Evaluation Study for Community Research

Abstract

Plain Language Summary

Keywords

Introduction

Study Context and Aims

Study Design and Methods

Evaluation Design

Methods

Sampling

Analysis

Synthesis and Validation

Results

Key Design Features for Evaluating an ABCD Programme

A Coherent Asset-Based Model Linked to an Evaluation Framework

Valuing Diverse Sources of Evidence

Use of Naturalistic Methods

Capturing Outcomes at Community-Level and Organisational-Level

Navigating Ethical Community Research

A Developmental and Collaborative Approach

Discussion

Limitations

Conclusions

Footnotes

Acknowledgements

Declaration of Conflicting Interests

Funding

ORCID iDs

Data Availability

References