Exploring the Use of Multiple Participatory Tools to Engage Community Health Workers in Program Evaluation and Implementation: A Case Study From the Philippines

Abstract

This study explored the use of three participatory tools within a Philippines-based case study with community health workers (CHWs) by comparing and contrasting the process and data generated across the tools, and critically reflecting on adaptations and facilitation considerations that affected the tools’ use. Facilitator notes and audio-recordings of discussions were integrated and analyzed thematically. Tools differed by the type of data generated: program-specific data related to CHWs’ roles and responsibilities or data on broader structural factors. A stepwise approach within each tool facilitated focused, in-depth sharing, as did initial paired discussions that allowed exchange of knowledge and experiences among CHWs. Facilitators required topic- and context-specific knowledge to guide discussion effectively. CHWs discussed challenges and successes in their roles; program recommendations; and broader challenges related to healthcare delivery in their communities. This study contributes critical insights on the use of participatory tools to promote the inclusion of implementer perspectives in health program co-design, implementation, and evaluation.

Keywords

non-governmental organizations (NGOs)community health workers (CHWs)program evaluation implementation participation Philippines

Background

Community health worker (CHW) programs are increasingly implemented across low- and middle-income countries to extend the reach of formal healthcare systems to communities. CHWs are often frontline care providers embedded within their communities and with knowledge of local healthcare needs, particularly of those individuals requiring additional supports outside of the formal healthcare system.^1,2 These individuals also have ‘on-the-ground’ program implementation experience critical for evaluating and strengthening CHW programs³; however, CHW programs are often implemented with a top-down approach, potentially limiting the bottom-up integration of CHWs’ knowledge, experiences, and insights.^3,4

To address this gap, participatory approaches have been viewed as means to involve CHWs more deeply in the design, implementation, and evaluation of CHW programs.^4,5 Participatory approaches centre the perspectives of populations and individuals that have often been marginalized in the decision-making process of implementing health programs through creating an equitable and trusting relationship between participants and researchers.⁶ By recognizing that communities hold expertise through their experiences and insider knowledge, participatory approaches aim to build on the strengths and resources within the community and create knowledge and action for the mutual benefit of all partners.⁷ Previous research has indicated that strong organizational support for incorporating community perspectives within health programming has a positive impact on health improvements and community empowerment.^8,9

More broadly, there exists widespread acceptance that participation is crucial to program design and implementation in health research.^10–13 By adapting methods to align with a particular cultural context or by designing studies that prioritize and centre participants’ voices, more fulsome participation can be promoted.¹⁴ Despite this recognition, fewer studies have compared and contrasted the strengths and limitations of different participatory approaches, or critically examined the use of specific methods in practice, with an eye to promoting the inclusion and centring of participants’ voices while also achieving broader study goals and objectives. Importantly, participation is not an end in and of itself or solely a normative value to be espoused. Rather, participation can also be a means to generate high-quality, useful, contextually-informed programmatic data that is crucial to the development and evaluation of complex health interventions.^15,16

The objective of this study was to explore the utility of three different participatory tools in effectively engaging CHWs in the co-design, implementation, and evaluation of a CHW program in the Philippines. More specifically, the objectives of this study were: (1) to compare and contrast the facilitation process and data generated across the three tools; and (2) to critically reflect on adaptation of the three tools and other facilitation considerations that may affect tool use and implementation. While this study is focused on the evaluation of a CHW program in the Philippines, these insights may have broader relevance to the design and implementation of participatory approaches within other community health programs.

Methodology

Study Context

This project was anchored by a longer-term collaboration between the University of Waterloo, Canada and Philippines-based non-governmental organization (NGO), International Care Ministries (ICM). ICM implements a CHW program (called Flourish) across the Visayas and Mindanao regions of the Philippines that engages more than 1300 volunteers in community-based maternal and child health support (e.g., provision of antenatal care; screening for maternal and child health issues; distribution of nutritional supplements and other basic treatments). This study was part of a broader evaluation of Flourish, located specifically within the province of Negros Oriental, Philippines. In this province, the CHW program operates across three branches. Each branch involves a group of approximately 30 CHWs who are supervised by a CHW program coordinator and a paid ICM staff member. ICM identified a need to evaluate their CHW program to strengthen program implementation and improve intervention outcomes. In doing so, ICM was interested in using participatory approaches to gather insights from CHWs to help inform future revisions and updates to the program. Moreover, ICM was interested in developing a sustainable process of gathering feedback from CHWs, to ensure that these insights are centred in program changes in the long-term.

The research team consisted of six individuals from the University of Waterloo, Canada (n = 3) and contracted by ICM (n = 3), four of whom are Filipino. All team members (Canadian and Filipino) shared facilitation responsibilities. Team members worked closely in partnership with ICM, but as academic researchers and research associates who were not embedded within ICM’s programs. Team members had prior program evaluation and facilitation experience, including experience conducting participatory, qualitative research in cross-cultural contexts. This study received ethics approval from the University of Waterloo (Certificate Number: 44828). Participants provided verbal informed consent to participate in the study. All participants received an honorarium of 200 Philippine Pesos (PHP; approximately 3.62 USD) for their participation.

Recruitment and Data Collection

In April 2023, 11 participatory group activity sessions were conducted with CHWs (n = 75 total) across six locations in Negros Oriental, Philippines (Figure 1). Most sessions were held in the common area of a local church or community centre and in conjunction with a regularly-scheduled monthly CHW meeting to reduce the burden of time and added transportation costs for participants. Study participants were all female CHWs that were recruited by their CHW coordinator to participate in the group activity sessions. All CHWs across the six locations were invited to participate (n = 78 total); only three declined participation.

Figure 1.

Map of Negros Oriental and the six regional locations where participatory group activity sessions were conducted by the research team: Guihulngan, Pamplona, Dumaguete, Manjuyod, La Libertad, Ayungon.

Three different participatory tools were used to generate and enrich discussion related to CHWs’ delivery of care for maternal and child health: a balloons and stones activity (n = 4 groups); patient pathway analysis (n = 3); and problem tree analysis (n = 4). These tools were selected based on the feasibility of implementing the activity, the different data that could be generated using the tool, and the tool’s relevance for informing ICM’s CHW program. Approximately half of the activity sessions focused on maternal health and half on child health (Table 1). Each activity was co-facilitated by a University of Waterloo team member and an ICM-contracted team member who was fluent in the local language of Bisaya and could interpret between Bisaya and English. Sessions were primarily facilitated in English (translated to participants in Bisaya, with participant discussion translated from Bisaya back into English for the other facilitator). For tools that prompted participants to write responses, most participants wrote in Bisaya, which was translated into English. Participants were asked to provide verbal feedback on their experience of and perspectives on the activity at the end of each session. Sessions ranged from 59 to 90 min in duration (average duration = 75.3 min). Sessions were audio-recorded and transcribed, and facilitators took detailed notes on process in a team debrief after each session. Following these team debriefs, each activity was iteratively adapted in an effort to enhance data quality, process, and participant experience based on insights from team debrief meetings.

Table 1.

Location, Topic, and Number of Participants of Each Participatory Activity Session.

	Location^a	Activity	Session topic	# Participants
Sessions 1-2	Guihulngan (n = 15)	Balloons and stones	Child health	7
Sessions 1-2	Guihulngan (n = 15)	Balloons and stones	Maternal health	8
Session 3	Pamplona (n = 5)	Patient pathway	Child health^b	5
Sessions 4-5	Dumaguete (n = 13)	Balloons and stones	Child health	7
Sessions 4-5	Dumaguete (n = 13)	Balloons and stones	Maternal health	6
Sessions 6-7	Manjuyod (n = 18)	Patient pathway	Child health	9
Sessions 6-7	Manjuyod (n = 18)	Patient pathway	Maternal health	9
Sessions 8-9	La Libertad (n = 12)	Problem tree	Child health	6
Sessions 8-9	La Libertad (n = 12)	Problem tree	Maternal health	6
Sessions 10-11	Ayungon (n = 12)	Problem tree	Child health	6
Sessions 10-11	Ayungon (n = 12)	Problem tree	Maternal health	6
			TOTAL	75

^an represents the total number of individuals who participated in a given participatory group activity.

^bOnly one group, focused on child health, was conducted in this location due to the small participant group.

Description of Activities

Each activity began with a general introduction to the overarching purpose for the participatory group activity session, followed by a round of personal introductions from facilitators and participants. Subsequently, the facilitator(s) explained the activity to participants. Each activity below involved a visual aid to facilitate the discussion, presented on a piece of white chart paper that was displayed in a convenient location for all participants to see (Figure 2). All activities involved a mix of paired and whole-group discussion of facilitators and barriers associated with implementer (CHW) work and potential solutions (see Appendix 1 for complete facilitation guides for each activity). Activities varied by their specific objectives, individual steps involved, and associated visual aids.

Figure 2.

Visual output from each of the three activities, consisting of a piece of white chart paper and a configuration of sticky notes. Balloons and Stones: The ‘end goal’ is indicated on the far-right side of the paper and balloon- and stone-shaped papers denoted with participant ideas are positioned above and below a separating line, leading to the ‘end goal’. Patient Pathway: Sticky notes denote enablers (yellow) or barriers (pink) that participants experience in their work, positioned along a line (symbolizing a ‘pathway’) of caring for patients from patient identification through to treatment or support. Problem Tree: A piece of white chart paper has the outline of a tree drawn on it and sticky notes positioned at various heights on each tree (i.e., the trunk symbolizes the problem; below are ‘root causes’ to that problem; above are solutions or responses to addressing that problem).

Activity #1: Balloons and Stones Activity

This activity involved the use of balloon- and stone-shaped ‘sticky notes’ to represent enablers and barriers, respectively, to achieving a pre-determined ‘end goal’ as CHWs within ICM’s program. Groups focused on child health used an ‘end goal’ of curing child malnutrition, while groups focused on maternal health oriented the activity around an ‘end goal’ of a healthy mother and baby. The specific objectives were (1) to identify enablers (‘balloons’) and barriers (‘stones’) to supporting child nutrition and maternal health in their communities from the perspectives of CHWs; and (2) to collaboratively generate potential solutions with CHWs to remove barriers and strengthen enablers to inform the iterative design and implementation of ICM’s CHW program¹.

Activity #2: Patient Pathway Analysis

This activity involved mapping the enablers and barriers that CHWs experienced when providing maternal or child health supports within ICM’s program. To do this, ‘sticky notes’ were positioned along a line symbolizing a hypothetical patient’s ‘pathway’ to care. The specific objectives were (1) to understand the role of CHWs in supporting pregnant women and children in their communities across the care cascade, from identifying individuals to screen to providing a referral to health services or basic treatment, and the enablers/barriers at each step; and (2) to understand how CHWs perceive of their roles within the broader health and social care systems and collaborate with public sector health care workers to deliver maternal and child health care.²

Activity #3: Problem Tree

This activity involved using parts of a tree to reflect on the ‘root causes’ of maternal-child health challenges, resultant consequences, and potential solutions. The tree trunk represented a problem that CHWs aim to address in their work (i.e., malnutrition; a maternal health challenge). Sticky notes were placed along the bottom of the tree to symbolize and denote the ‘roots’ of the problem, and also placed as ‘leaves’ on the tree to symbolize and discuss solutions to the root causes or ways of supporting CHWs in addressing root causes. Specific objectives were (1) to explore the perspectives of CHWs on structural factors influencing child nutrition and maternal health in their communities; and (2) to identify and discuss child and maternal health challenges in CHWs’ communities and identify the root causes, impacts, and potential responses to these challenges.³

Data Analysis

Qualitative data for this study (transcripts; notes) were integrated and analyzed thematically, with a constant comparative method.¹⁷ In addition, data collection materials (i.e., facilitation guides) provided additional context to examine the intended objectives of each activity and initial plans for facilitation prior to adaptation and ongoing iteration.

Results

Comparing and Contrasting Participatory Tools by Process and Type of Data Generated

All activities were observed to be effective in facilitating a space for participants to reflect on challenges they experienced in their roles, share critical perspectives on the program, and identify resources that would further support them. This outcome may have been due to the collective format of the activities, whereby participants may have felt more comfortable sharing negative program feedback or reflect honestly on challenges they experienced than in a one-to-one interview format. The generative nature of group discussion, in which participants could build from others’ ideas, may also have facilitated the fulsome discussion we observed. All activities were very similar in requiring minimal costs to implement, in terms of material costs, time, energy, and overall effort.

Strengths and Limitations of Each Visual Aid and Associated Process

All activities were largely facilitated similarly to a focus group discussion; however, the visual aid and function of having an ‘activity’ seemed to create an initial frame or focus to guide the discussion, which was observed to be helpful in generating participant insights. Each activity’s visual aid and associated process was useful in generating high-quality data for different reasons, but also had limitations (Table 2).

Table 2.

Types of Data Generated in Each Activity (Balloons and Stones; Patient Pathway Analysis; Problem Tree), Adaptations Made to the Activity Within the Study Context (Both Before Sessions Began and Within or Between Sessions), and Strengths and Limitations From the Research Team’s Experience of Implementing Each Activity.

	Balloons and stones	Patient pathway	Problem tree
Types of data generated	Operational, program-specific	Both program-specific and structural data	Broader, community-level and structural factors
Initial adaptations	• Adapted initially from Kumar (1999)• Alternated between written content and verbal sharing and how much time was given to each (balance between detailed written data versus verbal data vis-à-vis participants elaborating on written notes)	• Adapted initially from COUNTDOWN (2021) toolkit for participatory health research methods• Conceptualized the activity as more ‘vertically’ than ‘horizontally’ oriented (i.e., rather than asking, “how do you get a patient from A → B?”, we asked, “who or what supports you in doing your work more effectively at point A? Point B?”)	• Adapted initially from Williams et al. (2019) principles of CBPR.• Adapted the activity to also include appraising and choosing action/solutions, in addition to problematization, so the activity had potential on its own to more readily inform the NGO’s iteration of the program
Adaptations within/between sessions	• Experimented with integrating ‘solutions’ with ‘enablers’ and ‘barriers’, rather than a linear, step-wise approach (i.e., enablers, then barriers, then solutions to both)• Reframed questions/prompts to move beyond program challenges/barriers to what strategies or approaches are enabling the work to continue (i.e., “It sounds like there are many challenges in the CHW role; yet, you are still finding ways to do the work. What are strategies you use to do X?”)	• Adapted discussion probes based on program details (i.e., “You have two routes of action based on what the mHealth app instructs you to do (app used by CHWs to direct their provision of care): treat and/or refer. If treat, how do you ensure treatment is taken? If refer, how do you ensure the referral loop is closed?”)• Adapted the components of the patient pathway based on facilitator understanding of the CHW program (i.e., the final stage of the pathway was not ‘treatment’ but the choice of ‘treatment’ or ‘referral’ and associated follow-up)	• Iterated the activity’s structure based on participant insights (i.e., overall, less structured and stepwise approach from ‘root cause’ to problems; consequences; and solutions). Less clear distinctions between these steps
Strengths^a	• Easy to understand concept; an engaging visual aid• More open-ended nature led to sharing of diverse operational experiences among CHWs	• Stepwise approach of the activity mirrors the CHW workflow and facilitates more focused, in-depth sharing on enablers/barriers• Multilayered data generated related to both program operations and structural factors (collaboration between CHWs and broader health system actors)	• Created new lines of inquiry on broader, structural challenges affecting health for use in other evaluation components
Limitations	• Fairly unstructured by design: requires significant prompting from the facilitator, and some structuring of conversation associated with ‘balloons’ and ‘stones’ insights	• Varied positionality of participants may be more likely to present facilitation challenges or constrain conversation	• More challenging to identify potential solutions• Facilitation more challenging and training required: Stepwise approach less clear than other activities (i.e., differentiating a ‘root cause’ from a ‘consequence’ or ‘impact’). Important to have knowledge of historical/political context from which to prompt or probe

^aInformation provided on the strengths and limitations of each activity is based on reflection by research team members, not on feedback from participants.

Balloons and Stones

The visual of this tool was easy to comprehend. A Filipina facilitator likened this activity to the notion of systematically developing a list of ‘pros’ and ‘cons’ which she felt was familiar and frequently utilized. For future use, ‘balloons’ and ‘stones’ could be replaced with visual symbols more tailored to study context and culture, which may make the activity even more engaging and easily understood. Regardless, many participants engaged with the visual aid, utilizing the symbols in the program perspectives they shared, which enhanced the quality and tone of discussion. For example, participants remarked about wanting to focus discussion on the ‘stones’ (i.e., challenges in their roles) and referred to challenges in their work as “big stones”. They also shared that they appreciated the experience of providing program feedback, as it enabled them to “[get] the stones out of our heads”. A limitation of this activity was that it was more unstructured compared to the other two activities, as the enablers and barriers discussed by participants could pertain to any element of the CHW role and the broader CHW program. Discussion had potential to be diffuse and lacking of depth, scattered, or redundant without skilled facilitation. Facilitators had to prompt effectively and frequently to generate high-quality data. On the other hand, the more open-ended nature of this activity also led to sharing of diverse operational experiences among CHWs.

Patient Pathway

This visual aid was more conducive to identifying and exposing broader gaps in the program and drawing out specific implementation challenges that were useful for program evaluation. Given that the aid focused the discussion on the various CHW’s roles across the pathway of support for mothers or children, it prompted reflections pertaining to both operational and structural factors along this continuum.⁴ A strength of this activity was the ability to generate multilayered data on program mechanics from CHW perspectives (i.e., what they are doing to support maternal and child health and how). This data built a foundation for discussion on enablers and barriers and CHW responses to these, including at both program and structural levels (i.e., how they interface with public sector health workers in their role and how that affects their work within the CHW program). Overall, the activity was more structured in comparison to the other activities. Facilitators took a stepwise approach, prompting in-depth and focused sharing on enablers and barriers at each step in the CHW workflow, versus the more diffuse structure of ‘balloons and stones’, for instance. One limitation of this activity was that facilitation was challenging when discussing how CHWs interface with the public sector health system, given the complexities of how health care is accessed and delivered.

Problem Tree

In their verbal feedback at the end of the session, multiple participants provided positive comments on this specific activity. The visual aid of a tree may have been more memorable than in the other two activities. Discussion of ‘root causes’, with the accompanying visual aid, led to nuanced data on the role of structural, systems-level factors in shaping maternal and child health outcomes, and the broader implementation context within which the CHW program is embedded. A strength of this activity was, therefore, in creating new lines of inquiry around these broader, structural factors that were useful for other components of the CHW program evaluation (i.e., one-on-one interviews). A limitation was that it appeared more challenging for CHWs to identify potential solutions to the barriers being discussed in this activity compared to others. This challenge is likely due to the barriers being structural and at a systems-level – thereby more complex, particularly in a resource-constrained study context – as compared to the specific programmatic or operational barriers discussed in the other two activities that may have simpler solutions. Additionally, facilitation was challenging with this activity (i.e., differentiating a ‘root cause’ from a ‘consequence’ or ‘impact’ of a given cause was not always clear). Facilitators had to be flexible and iterate the planned activity’s stepwise approach, based on participant insights – in essence, allowing participants to ‘drive’ the structure in order to generate high-quality data.

Type of Data Generated

Overall, all activities aimed to generate discussion about enablers and barriers to CHWs’ work supporting maternal and child health, though varied in how program-oriented and operational or structural those factors were. Additionally, all activities aimed to integrate discussion of potential solutions or recommendations for how to address barriers. As such, activities could be placed along a continuum, in terms of how useful they were in generating applied, operational data for program co-design, evaluation, and implementation (i.e., balloons and stones, patient pathway) versus understanding the broader implementation context (i.e., problem tree) (Figure 3).

Figure 3.

Utility of each participatory activity in terms of the type of data generated.

Promoting Participation and Achieving Objectives: Adaptations and Critical Considerations for Activity Implementation

Adaptations to Activities to Enhance Activity Facilitation and Data Quality

It was necessary to iterate and further contextualize each activity as subsequent sessions were implemented, in order to enhance the process, quality of data generated, and participant experience. Ongoing and detailed debrief among facilitators, after each session, involved identifying what was and was not working well and discussing potential adaptations.

In terms of activity format, each activity was structured to involve varied opportunities for participation, whether writing ideas down on a sticky note in English or Bisaya, or discussing ideas verbally. Each activity also combined paired (in partners) and full-group discussion, allowing for multilayered discussions among participants. Some partners also worked on their own and silently, exchanging written sticky notes with a partner and working independently to add ideas to what their partner had initially written. Moreover, there appeared to be value to a ‘presentation’ format after paired discussions, with stepwise ‘sharing back’ around the circle rather than an open ‘free for all', to promote more inclusive participation. Similarly, facilitators would intermittently pose an ‘entry point’ question that was easier to respond to and ask one-by-one around the circle, so that diverse perspectives and voices were heard.

In terms of activity environment and the physical configuration of groups, most sessions were in a community space, whereby two groups (one for each of maternal and child health) were held concurrently in the same space. At times, this presented logistical challenges with hearing everyone speak and not being distracted by the other group’s conversation. The research team experimented with the physical configuration of the group in response, to enhance the process. Most sessions had a similar configuration of facilitators and participants sitting in a circle together, which also conveyed a more conversational and participatory tone. Sometimes this involved sitting around a table with the visual on a piece of paper in front of the group.

Adaptations were also made to frame guiding questions in order to generate the intended ideas or insights. In doing so, it was helpful to have both the broader research objectives and specific activity objectives in mind (see Table 2 for further detail regarding adaptations). For example, especially in the more program/operationally-oriented activities (i.e., ‘balloons and stones’, patient pathway), facilitators found they needed to reframe the instructions to prompt participants to go beyond the mechanics of their CHW role (i.e., the tasks they do that support or constrain positive maternal-child health outcomes) to what supports that work being done more effectively or what barriers exist.⁵ Extensive debriefing among research team members following sessions was helpful for discussing the ‘ebb’ and ‘flow’ of participant discussion in a given group and brainstorming specific prompts or rephrasing of questions to address facilitation challenges in subsequent sessions.

Knowledge of Program and Implementation Context

Facilitators’ knowledge of the program and broader implementation context enhanced facilitation (types of questions asked; informed by contextual knowledge) and likely generated higher quality data overall. Across activities, later sessions were enhanced in process and data quality, as the research team experimented with each tool and adapted accordingly (as above) to promote CHW participation. Moreover, facilitators had more knowledge of the program itself and the implementation context after initial sessions, based on the CHWs’ insights shared. As part of the broader program evaluation, the research team was concurrently conducting one-to-one CHW interviews with activity sessions in each location – which further triangulated data from sessions and enhanced facilitator knowledge. As such, for subsequent groups, facilitators could even probe specifically based on accumulated program knowledge and shared participant perspectives.

Across activities, the same facilitator(s) conducted groups on the topic of maternal health and the same on child health. This enabled facilitators to build familiarity with their given topic, including the details of the CHW role associated with supporting either maternal or child health (operational context) and structural (i.e., broader, historical/political context) associated with a given topic. For the patient pathway activity, it was particularly helpful that facilitators could build understanding of CHWs’ roles along the care pathway for either maternal or child health, and thus probe more effectively in the discussion. For the problem tree activity, it was especially useful for the facilitator to have knowledge of the broader health system and political context, to be able to initiate discussion around the potential influence of these broader structural factors on maternal-child health outcomes or delivery of care. Overall, this facilitation approach across activities was conducive to generating higher-quality data as facilitators were better able to iterate and probe effectively as data collection progressed.

Positionality of Participants and Facilitators

The positionality of both participants and facilitators (with respect to gender, age, roles/occupations, ethnicity) likely influenced group dynamics and the degree of sharing that occurred. These positionalities were also observed to shape the type of information being discussed, as CHWs spoke from their perspectives and shared experiences along lines of gender and occupation, especially.

All participants (and all but one facilitator) were women, which may have increased the overall level of comfort in the discussion, based on this study’s socio-cultural context. That all participants were women may have also created new opportunities for discussion due to the gendered implications of some aspects of the CHW role. For example, in some sessions, participants shared their concerns about safety in their CHW roles, particularly in the context of travelling long distances on their own – as women – to more remote communities and/or those perceived to be unsafe. Across activities, many participants also discussed their household caregiving responsibilities for both children and aging parents, and the challenges of navigating those responsibilities in addition to being a CHW. These perspectives on navigating the caregiving landscape as CHWs, amid their households and communities, were more likely experienced among women than in a more gender-diverse group.

Moreover, participants were of diverse ages, ranging from young to older adults. One Filipina facilitator reflected on her positionality as a younger person addressing an elder in this cultural context and whether that may have generally influenced the tone of the group. Younger individuals may have participated differently or given additional space in conversation for elders to speak. Participants also varied with respect to other roles or occupations they held in communities – beyond the CHW role – which may have facilitated or hindered the participation of others. For instance, some groups involved CHWs who were also barangay health workers (i.e., community health workers connected to local government units) and may have been perceived by others as holding more specialized knowledge.

A shared challenge across tools was also the positionality of facilitators. Tension existed between how the research team framed themselves as not working directly for the NGO (i.e., ICM) and that participants were welcome to provide critical feedback; however, it was evident that the team would be sharing synthesized findings back to the NGO to inform programming. The team also embedded the research process within ICM operations, as they held activity sessions during regular monthly CHW meetings, seemingly making their relationship as researchers to the NGO more complex. This dialectic between partnering with, but still being a neutral ‘third-party’ to, the work of the NGO was observed to require clarification among participants and may have limited the level of critical feedback provided. In response, the research team reiterated participant confidentiality; that honest discussion of challenges within the program evaluation would not have broader repercussions for CHWs; and that the sharing of diverse perspectives – both positive and negative feedback – could contribute to strengthening the program. As three members of the research team were from Canada and formally affiliated with the University of Waterloo, their positionality as ‘outsiders’ conducting the evaluation may have both constrained and facilitated discussion at times. Potential constraints related to their positionality may have been mediated, in part, by Filipina team members local to Negros Oriental who co-facilitated and probed conversation when needed.

Summary of Findings

Overall, these findings present critical insights from a research team’s experience of utilizing three participatory tools for NGO program evaluation and implementation. From this experience, this study outlined strengths and limitations of using these tools to promote participant engagement and generate relevant data – attributed to the design/nature of a given tool itself, but also associated with adaptation and facilitation considerations (i.e., facilitator knowledge of context; positionality) that affected the tool’s utility and implementation.

Discussion

This study provided applied, methodological insights on the utility of multiple participatory tools in generating relevant, useful program evaluation and implementation data within a CHW program and NGO context. These findings include examples of specific tools that NGOs and other community-based organizations might employ to promote the inclusion and prioritization of CHWs’ insights in the design, evaluation, and implementation of large health and poverty alleviation initiatives, and critical reflection from our experience with their use. Ultimately, these findings highlight that when aiming to promote participation and inclusion of implementer perspectives in program co-design, implementation, and evaluation, it is necessary to thoughtfully consider the combination of tools, driven by broader study or process objectives, as well as other factors such as facilitator and participant positionalities and context. Given the research team’s evaluation objectives (i.e., interested in both operational data and data on implementation context), the three tools were considered effective in generating useful data and promoting CHW participation. As reflected in the findings, use of these tools created spaces where individuals could collectively reflect on and corroborate their experiences of program implementation, sharing both positive and negative program feedback that had not previously been communicated back to the NGO. The experiential, collective nature of the activity sessions was itself reportedly enjoyed by participants.

That said, the three specific participatory tools in this study varied with respect to their utility in either generating program evaluation/operational data (‘balloons and stones'; ‘patient pathway') or for deepening understanding of the program implementation context (‘patient pathway'; ‘problem tree'). Depending on the type of insights one aims to generate, the tool matters in terms of the ways in which participation is enabled through the tool’s design or process and the resultant data that is generated. As such, when aiming to promote participation in program design and evaluation, critical and careful thought as to what participatory tools to select is necessary and may – in part – be dictated by specific aims and objectives. Although we used these tools separately, if time and resources permit, opportunity may exist to combine them to build a more comprehensive process. Given the different types of data they generate, sequencing these tools could enable both deep contextual understanding (i.e., beginning with ‘problem tree’) followed by more solutions/action-oriented reflection (i.e., with ‘balloons and stones’ or ‘patient pathway’). This cycle of problematization, action, and reflection on action, is a crucial foundation to participatory approaches^18,19 and helps to move beyond a siloed view of participatory tools to one that is more iterative, integrated, and – arguably – effective in achieving objectives. Other studies have proposed specific frameworks to aid in systematizing the selection and application of participatory tools for co-creation of health interventions, such as the 2019 ‘PRODUCES’ framework of Leask and colleagues (6:5). In this framework, as reflected in this study’s findings, the specific aim/objective of the participatory process is considered alongside the positionality and interests of participants (i.e., end-users, co-creators).

Relatedly, even if a tool is selected to align with the type of data desired and how participation is intended to be fostered, this study’s findings indicate the significance of further considerations (positionality; knowledge of study or program context) in ultimately shaping experience with a given tool. That is, it is necessary to reflect on and be responsive to these broader factors that shape the quality of participant experience, quality of participation overall, and quality of data generated. Indeed, these considerations have been highlighted via efforts to enhance increased patient and public involvement and participation in healthcare delivery, particularly across primary healthcare contexts.^20–23 This study contributes to the practical considerations raised in this literature by highlighting how the use of participatory tools outside of primary care settings must be responsive to program (e.g., geography, resource constraints) and implementation (e.g., central role of CHWs rather than other healthcare providers) context. Further, given a substantial literature on the time required for CHW tasks, particularly in low- and middle-income countries,²⁴ as well as a fulsome debate on the merits of different compensation models,^25,26 it is important for researchers to critically reflect on the time required to implement a given participatory tool in these contexts and appropriate compensation for CHW participation. For the research team, ongoing debrief was crucial to continually iterate the methods to ensure congruence with the overall study objectives and to promote the quality of participation. Others utilizing these specific methods or participatory tools may benefit from a similar non-rigid approach. An ability to be flexible and iterate as needed can ensure high-quality and useful data is generated alongside – and, in part, due to – fulsome participation.

Moreover, critical reflection in this study around the research team’s positionality relates to other studies exploring the complexity of ‘insider-outsider’ affiliations in relation to participation and empowerment.^27,28 The research team was positioned as ‘insider’ to the NGO operations and structure at a high level, given that the researcher-NGO affiliation was evident to participants. The team attempted to “articulate strategically” their knowledge as ‘outside’ of the program expertise of CHWs, who were ‘insiders’ to the program operations as implemented in practice, and also held expert knowledge of their communities (i.e., the implementation context) (24:109). Nevertheless, the nuances of this ‘insider-outsider’ status, and the power differential inherent to team members’ respective positionalities, shaped activity facilitation. Thus, it is necessary to carefully consider how activity processes can be structured, and tools adapted and utilized, such that participants are empowered as active agents in program co-design, implementation, and evaluation with the particular ‘insider’ knowledge they hold.²⁷ Moreover, given the research team’s positionality, and the nature of the tools themselves, our findings emphasize the critical need for skilled and sensitive facilitation to create an environment where participants feel comfortable sharing and high-quality data is generated. Particularly in some cultural contexts, age and gender of both participants and facilitators may also be important considerations when selecting tools, structuring activities, and planning facilitation approaches.²⁹

This study presents a theoretical contribution as well. While participatory approaches are broadly valued in health research,³⁰ there is a need to move beyond promotion of these approaches to a more critical assessment of what tools might be appropriate for specific objectives and how to most effectively adapt tools based on the participant population and broader implementation context. By critically and reflexively examining tools in practice within this case study, we engage the concept of participation – how it might be promoted through specific tools, and what some of the careful considerations might be when upholding the value of participation in practice, within a particular context. Others have reflected on how participatory tools might be used to contribute to broader systems transformation and the empowerment of individuals to enact change within their communities.³¹ While promoting active participation and inclusion of CHW perspectives, this study can be contrasted by these more transformative and emancipatory forms of participation. Instead, we contribute a critical reflection on the utility of different participatory approaches to generate in-depth implementer feedback that may inform program design and delivery within an NGO or other organizational context. Leveraging these tools for systems-level change requires sustained participant engagement and substantive resources and effort.³²

This study had several limitations. These findings are based primarily on researcher observations of the activity sessions and insights derived from triangulating sources of data (notes; audio recordings), rather than explicit feedback on the sessions from participants themselves. While participants were invited to share verbal feedback at the end of each session, these data were limited, potentially due to the 'insider-outsider' tensions the research team experienced and hesitation among participants to share critical feedback on the sessions. Future research should aim for more in-depth and creative integration of participant insights into this type of critical, methodological reflection, to enhance depth and validity of findings through a more participatory evaluation of the process. In addition, data that were gathered from participants may have been subject to selection bias, whereby participants with positive feedback on the sessions may have been more likely to share feedback. We have aimed to address this limitation through ensuring critical, reflexive, and authentic engagement with other data sources that highlighted both strengths and challenges of the activity sessions. A key strength of this study is its embeddedness within a sustained NGO-researcher partnership and broader CHW program evaluation, which facilitates direct engagement with the NGO and potential application of findings to their iteration of the program. Overall, this study contributes a critical reflection on an application of three participatory tools, which may be considered a suite of options for other organizations implementing community-based health programs. By including the types of data generated with the tools, our experience of iterating the tools, and strengths and limitations of each tool – alongside important considerations regarding positionality and implementation context – we offer an applied methodological toolkit to support the inclusion of implementer perspectives in health program co-design, implementation, and evaluation (Box 1).

Box 1.

Specific Practice-Based Lessons from Our Application of Three Participatory Tools.

• Select tools thoughtfully according to (a) broader process aims and objectives; as well as (b) facilitator and participant positionalities and context

• Consider combining tools and sequencing to build a comprehensive understanding of both program or operational facilitators/barriers as well as program implementation context

• Critically reflect on the time required to implement a tool and consider appropriate compensation for participants

• Carefully structure and adapt activities with an eye to power differentials (or ‘insider-outsider’ dynamics) between facilitators and participants

• Recruit and invest in training of skilled and sensitive facilitators

Conclusion

The use of participatory tools can create opportunities for CHWs to share their experiences and knowledge and for these experiences and knowledge to be valued and ultimately integrated into NGO program implementation. Indeed, the use of participatory tools may have a role in contributing to ‘bottom-up’ institutional change in the delivery of CHW programs as well as other programming led by NGOs or other community-based organizations. However, as illustrated in this study, it is also necessary to critically understand and consider how knowledge of program and implementation contexts as well as participant and facilitator positionality may influence the selection and adaptation of various participatory tools. Addressing these considerations may ultimately shape the ability of participatory approaches to promote inclusion of participant voices and to generate high-quality data that meets study and program objectives.

Supplemental Material

Supplemental Material - Exploring the Use of Multiple Participatory Tools to Engage Community Health Workers in Program Evaluation and Implementation: A Case Study From the Philippines

Supplemental Material for Exploring the Use of Multiple Participatory Tools to Engage Community Health Workers in Program Evaluation and Implementation: A Case Study From the Philippines by Laura J. Brubacher, Lincoln L. Lau, Monica Bustos, Melinda K. Mijares, Krisha L. Mar, and Warren Dodd in Community Health Equity Research and Policy

Footnotes

Acknowledgments

We would like to express gratitude to the community health workers who participated in this study and shared their insights and experiences with our research team. Thank you to Ruth Laagan, Alona Pestanas, and Jes Hernani, as well as ICM staff in Dumaguete, who supported data collection through translation, co-facilitation, and administration. Thank you also to Bridget Beggs who created for this manuscript.

Declaration of Conflicting Interests

Authors (LLL, MKM, KLM) receive remuneration from International Care Ministries (ICM). The authors have been provided academic freedom by ICM to publish both negative and positive results. Authors LJB, MB, and WD have no competing interests to declare.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the New Frontiers in Research Fund (NFRFR-2021-00227).

ORCID iDs

Laura J. Brubacher

Warren Dodd

Supplemental Material

Supplemental material for this article is available online.

Notes

Author Biographies

Laura J. Brubacher is a Postdoctoral Research Fellow in the School of Public Health Sciences, University of Waterloo, Canada.

Lincoln L. Lau is the Director of Research at International Care Ministries, Philippines.

Monica Bustos is a Research Coordinator in the School of Public Health Sciences, University of Waterloo, Canada.

Melinda Kelly Mijares is the Chief Health Officer of Global Health Strategy at International Care Ministries, Philippines.

Krisha Lim Mar is the Director of Impact at International Care Ministries, Philippines.

Warren Dodd is an Associate Professor in the School of Public Health Sciences, University of Waterloo, Canada.

References

Kane

Radkar

Gadgil

, et al.

Community health workers as influential health system actors and not “just another pair of hands.”

Int J Health Pol Manag 2021; 10(8): 465–474.

Dodd

Kipp

Nicholson

, et al. Governance of community health worker programs in a decentralized health system: a qualitative study in the Philippines. BMC Health Serv Res 2021; 21(1): 1–14.

Kane

Kok

Ormel

, et al. Limits and opportunities to community health worker empowerment: a multi-country comparative study. Soc Sci Med 2016; 164: 27–34.

O’Donovan

Thompson

Onyilofor

, et al. The use of participatory visual methods with community health workers: a systematic scoping review of the literature. Global Publ Health 2019; 14(5): 722–736.

Peacock

Issel

Townsell

, et al. An innovative method to involve community health workers as partners in evaluation research. Am J Publ Health 2011; 101(12): 2275–2280.

Leask

Sandlund

Skelton

, et al. Framework, principles and recommendations for utilising participatory methodologies in the co-creation and evaluation of public health interventions. Res Involv Engagem 2019; 5(2): 1–16.

Israel

Schulz

Parker

, et al. Review of community-based research: assessing partnership approaches to improve public health. Annu Rev Publ Health 1998; 19: 173–202.

Haldane

Chuah

Srivastava

, et al. Community participation in health services development: a systematic review of empowerment, health, community, and process outcomes. PLoS One 2019; 14(5): e0216112.

Haldane

Dodd

Kipp

, et al. Extending health systems resilience into communities: a qualitative study with community-based actors providing health services during the COVID-19 pandemic in the Philippines. BMC Health Serv Res 2022; 22(1385): 1–12.

10.

Cook

Boote

Buckley

, et al. Accessing participatory research impact and legacy: developing the evidence base for participatory approaches in health research. Educ Action Res 2017; 25(4): 473–488.

11.

Eyre

Farrelly

Marshall

. What can a participatory approach to evaluation contribute to the field of integrated care? BMJ Qual Saf 2017; 26(7): 588–594.

12.

Bourke

. Reflections on doing participatory research in health: participation, method and power. Int J Soc Res Methodol 2009; 12(5): 457–474.

13.

Springett

. Impact in participatory health research: what can we learn from research on participatory evaluation? Educ Action Res 2017; 25(4): 560–574.

14.

Brubacher

Dewey

Tatty

, et al. “Sewing is part of our tradition”: a case study of sewing as a strategy for arts-based inquiry in health research with Inuit women. Qual Health Res 2021; 31(14): 2602–2616.

15.

Hudon

Chouinard

M-C

Bisson

, et al. Case study with a participatory approach: rethinking pragmatics of stakeholder engagement for implementation research. Ann Fam Med 2021; 19(6): 540–546.

16.

Hamilton

Yano

. The importance of symbolic and engaged participation in evidence-based quality improvement in a complex integrated healthcare system: response to “The science of stakeholder engagement in research”. Transl Behav Med 2017; 7(3): 492–494.

17.

Braun

Clarke

. Using thematic analysis in psychology. Qual Res Psychol 2006; 3: 77–101.

18.

Loewenson

Laurell

Hogstedt

, et al. Part Two: methods. In: Participatory Action Research in Health Systems: A Methods Reader [Internet]. Harare, Zimbabwe: TARSC, AHPSR,WHO, IDRC Canada. Harare: EQUINET, 2014, pp. 37–69. Available from: https://equinetafrica.org/sites/default/files/uploads/documents/PAR_Methods_Reader2014_for_web.pdf.

19.

D’Ambruoso

Mabetha

Twine

, et al. Voice needs teeth to have bite’! Expanding community-led multisectoral action-learning to address alcohol and drug abuse in rural South Africa. PLOS Glob Public Heal 2022; 2(10): e0000323.

20.

Hertel

Cheadle

Matthys

, et al. Engaging patients in primary care design: an evaluation of a novel approach to codesigning care. Health Expect 2019; 22(4): 609–616.

21.

Modigh

Sampaio

Moberg

, et al. The impact of patient and public involvement in health research versus healthcare: a scoping review of reviews. Health Pol 2021; 125(9): 1208–1221.

22.

Biddle

MSY

Gibson

Evans

. Attitudes and approaches to patient and public involvement across Europe: a systematic review. Health Soc Care Community 2021; 29(1): 18–27.

23.

de Graaff

Kleinhout-Vliek

Van de Bovenkamp

. In the works: patient and public involvement and engagement in healthcare decision-making. Health Expect 2021; 24(6): 1903–1907.

24.

Astale

Abebe

Mitike

. Workload and emerging challenges of community health workers in low- and middle-income countries: a mixed-methods systematic review. PLoS One 2023; 18(3): e0282717.

25.

Colvin

Hodgins

Perry

. Community health workers at the dawn of a new era: 8. Incentives and remuneration. Health Res Pol Syst 2021; 19(Suppl 3): 106.

26.

Ballard

Westgate

Alban

, et al. Compensation models for community health workers: comparison of legal frameworks across five countries. J Glob Health 2021; 11: 04010.

27.

Dodd

Wyngaarden

Humphries

, et al. How long-term emancipatory programming facilitates participatory evaluation: building a methodology of participation through research with youth in Honduras. Action Res 2023; 0(0): 1–19.

28.

Wood

Kallestrup

. Benefits and challenges of using a participatory approach with community-based mental health and psychosocial support interventions in displaced populations. Transcult Psychiatr 2021; 58(2): 283–292.

29.

Frediani

Peris

Boni

. Notions of empowerment and participation: contributions from and to the capability approach. In: The capability approach, empowerment and participation: Concepts, methods, and applications. Springer Nature Limited, 2019, pp. 101–124.

30.

Greenhalgh

Hinton

Finlay

, et al. Frameworks for supporting patient and public involvement in research: systematic review and co-design pilot. Health Expect 2019; 22(4): 785–801.

31.

Kothari

. Power, knowledge and social control in participatory development. In: Cooke

Kothari

(eds). Participation: The New Tyranny? London, UK: ZED Books, 2001, pp. 139–152.

32.

Loewenson

Beznec

Coelho

, et al. Building social power and participation in local health systems: learning from practice [Internet]. Training and Research Support Centre. 2017. Available from: https://www.tarsc.org/publications/documents/Shapinghealthsynthesisreport2017.pdf

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.74 MB