Addressing Failing Water Systems: A Qualitative Protocol for Intervention Research on Water Insecurity

Abstract

It is increasingly acknowledged that high income countries do not enjoy universal drinking water coverage, with pockets of water insecurity persisting close to areas that have had high-quality, centralized water provision for half a century or more. The reasons behind this are complex, and effective ways to improve water security in these areas defy simple prescriptions. Nuanced measures that disaggregate the different facets of water insecurity are especially important as we attempt to understand and, crucially, improve water security in high-income countries.

In this paper, we describe a qualitative protocol for assessing water insecurity at the household level in a study conducted in rural and unincorporated settlements in the Southwestern United States. The study focuses on households connected to small, piped water systems; households that rely on hauled water, individual wells, and other sources; and those who are withdrawing from centralized systems. The goals of this study are: 1) to measure a baseline for individual households’ water security; 2) identify mechanisms for coping with poor water insecurity; 3) evaluate changes in water insecurity following co-designed interventions. We include in this protocol three codebooks for the coding and analysis of interview data, to make them available for use by other water insecurity researchers.

The design embeds the interventions into the study without pre-supposing the nature of the interventions. The socio-technical interventions will be co-designed by the study team, project partners, and the communities themselves. By identifying and working with communities experiencing different water insecurity challenges the study aims to directly increase their water security and develop a deeper understanding of the drivers of and responses to water insecurity in the Southwestern United States.

Background

Lack of safe drinking water is a longstanding global health and development challenge (Bartram et al., 2014; Davis & Evans, 2006; Esrey & Habicht, 1986; United Nations, 1990). Yet, globally 2.2 billion people still do not have safely managed water, and the burden of disease from water-related illnesses remains high (UNICEF & WHO, 2023; Wolf et al., 2023). In recent years, research has highlighted issues of water security in high-income countries (Jepson, 2014; Meehan et al., 2020; Underhill et al., 2025). This scholarship has shown that while aggregate levels of water security are high, water insecurity remains a stubborn problem in high-income countries, often concentrated in certain geographies and within certain communities (Brown et al., 2023; Deitz & Meehan, 2019; Mattos et al., 2021).

Much of the work to improve water security in lower- and middle-income countries has focused on improving public health outcomes, and it focuses on the prevalence of diarrhea or nutritional status indicators like stunting and wasting in infants (Pickering et al., 2019; Wolf et al., 2022). While improving and protecting public health is extremely important, new methods and metrics for measuring water security go beyond infrastructural and biomedical indicators and incorporate measures of access, adequacy, and reliability (Jepson, 2014; Jepson et al., 2021; Wutich et al., 2021; Young et al., 2019). Water insecurity in the USA reaches beyond the estimated 2.2 million people in the USA lacking basic running water and indoor plumbing in their homes (EPA, 2025). More nuanced measures that disaggregate the different facets of water insecurity are especially important as we attempt to understand and improve water security in high-income countries.

In this paper, we describe a qualitative protocol for assessing water insecurity at the household level in a study conducted in rural and unincorporated settlements in the southwestern United States. This focuses on households connected to small piped water systems; households that rely on hauled water, individual wells, and other sources; and those who are withdrawing from centralized systems. We have previously argued that reliance on non-networked water systems is likely to increase under the growing environmental and political uncertainty we are currently witnessing. Without active consideration and management of this transition, maladaptation and undesirable outcomes are likely (Stoler et al., 2022; Thomson et al., 2024; Wutich et al., 2023).

Households may withdraw from using piped water systems for a variety of reasons. There may be specific characteristics of their water that leads them to reject it. Water that is designated safe to drink according to official guidelines can have an undesirable taste or odor. Others may have concerns about specific contaminants (Jakus et al., 2009) or a more generalized mistrust or skepticism of municipal water, found to be higher among Black and Hispanic households (Javidi and Pierce, 2018; O’Brien et al., 2025). Choices may also be driven by prestige, with certain brands of bottled waters may be viewed as premium products and preferred over other bottled waters, or because tap water has been devalued (Brewis et al., 2021). An individual or household decision not to drink their tap water may be a combination of all these factors, which can be mutually reinforcing.

While taste is a personal preference, there are aesthetic guidelines, set by the EPA and WHO for example, that suggest maximum levels of certain minerals for pleasant tasting water. The decision of an individual to drink one’s tap water may be a binary one that takes place at a discrete moment in time or may be gradual (e.g. initially providing bottled water only for guests). In aggregate we describe this as retreat from centralized provision and distinguish between managed retreat and unmanaged retreat: We define the former as being an informed choice with decisions made in response to specific measurable factors, e.g. boiling water to kill bacteria or avoiding tap water due to high levels of metals or minerals that affect taste such as sulphate or iron; we define the latter being choices made for less concrete reasons, such as an unspecified mistrust, dissatisfaction or fear.

It is not our role to judge people’s preferences for the taste, smell, and color of their water, but we can intervene ethically in a number of ways, with the distinction between managed and unmanaged retreat informing how we might do this:

1. It is appropriate for us to inform households about why their water may taste as it does, for example explain how—depending on the chemical content of their water—that the poor taste may not be linked to any health risk.

2. If people have an unspecified mistrust of their water which is leading them to retreat from centralized provision, we should share water test results so that they can make informed decision, making no judgement if they continue to reject municipal water when it is shown to meet all relevant guidelines

3. Finally, in cases where waters do fall outside guidelines, be that health or aesthetic, we should use our knowledge to guide households toward the most appropriate treatment that might make their water safe and acceptable to drink.

Study Goals and Research Questions

The goals of this study are: 1) to measure a baseline for individual households’ water security; 2) identify mechanisms for coping with poor water insecurity due to the lack of a reliable connection to a centralized water system; 3) evaluate changes in water insecurity following interventions designed to improve households’ situations. This study design embeds the interventions into the study, with the study acting as an evaluation of the particular interventions themselves and enable us to design and implement more effective responses for comparable communities in the future. By identifying and working with communities that typify different challenges with centralized water systems in three different ways, we aim to answer three respective research questions:

1. Can unmanaged retreat from a functioning water system be reversed, and does this improve water security?

2. Can managed retreat from a failing water system be facilitated in a way that maintains or improves water security?

3. Can existing modular, adaptive, and decentralized (MAD) water systems be strengthened to improve water security with coupled engineered and social infrastructures interventions?

This study has a multi-method intervention research design, conducted using a participatory convergence approach. Participatory convergence is an approach in which researchers from multiple disciplines co-develop with community members and other stakeholders a shared understanding of complex problems and solutions (Castro-Díaz et al., 2024; A. Roque et al., 2021). This shared understanding will inform the interventions that are an integral part of the study design.

Summary of Pilot Work

Pilot work was conducted under the Action for Water Equity project supported by the U.S. National Science Foundation Growing Convergence Research program. An interdisciplinary team consisting of engineers, hydrologists, ecologists, economists, lawyers and social scientists conducted a review of the literature (Wutich et al., 2022; Zheng et al., 2022) and analysis of secondary data. The latter includes deploying machine learning to cluster more than 2,000 colonias (informal marginalized settlements in the north of the US-Mexico border) to identify communities that need priority intervention (Gu et al., 2023).

To conduct qualitative pilot research in selected colonias derived from the quantitative cluster analysis, we also developed a novel approach, community-based participant observation (CBPO) (A. Roque et al., 2024), which merges participant observation and community-based participatory research (CBPR). With community members empowered as co-researchers to generate culturally grounded insights from the participants, the CBPO methods has been used to collect data about the lived experiences of water insecurity, co-producing knowledge with partners, including the research team, colonia residents and NGOs (Castro-Díaz et al., 2024).

These participatory approaches were central in each step of this work, from data collection to designing MAD solutions for communities (Stoler et al., 2022; Thomson et al., 2024; Wutich et al., 2023). Our research has shown that MAD water can contribute to water justice, particularly when driven by the participatory approaches and embracing a moral economy framework (Beresford, Brewis, et al., 2024; Roque et al., 2023). All together, these pilot works set the foundation for refining and scaling MAD interventions for a more inclusive and equitable water-secure society for the most vulnerable and marginalized groups.

Site Selection

All research sites are Arizona towns and communities with established water problems, selected to represent the three different water insecurity scenarios in which communities’ experience of centralized water systems are unsatisfactory.

Scenario 1: Reverse Unmanaged Retreat From Functioning Water Systems

In a small town in central Arizona we have observed unmanaged retreat from a functioning water system. The town has a municipal water system that has repeatedly passed water safety tests and is well-performing. However, the smell and taste of water from the municipal system is considered unpleasant by many, raising suspicions about its safety among residents. As a result, many residents refuse to drink tap water, use unmaintained water filters, and shift their consumption to bottled water and other packaged beverages. These strategies of unmanaged retreat lead to economic and labor burdens, and may create health risks. The purpose of our research is to identify interventions that reverse this unmanaged retreat and test for improvements in water security.

Scenario 2: Facilitate Managed Retreat From Failing Water Systems

In southern Arizona, we conducted interviews in a colonia with a failing water system. Colonias are communities within 200 miles of the U.S.-Mexico border that are federally designated as having insufficient public infrastructure, including water and sewage. This colonia has a small municipal water system that is not meeting the water needs of all residents. This is in an agricultural area of the state. It utilizes groundwater and the system has historically been viewed by community members as unreliable due to complaints over poor water quality (taste and color). The water situation in this community is causing ongoing labor burdens, economic costs, and possible health risks to residents. The purpose of our research is to facilitate managed retreat and test for improvements in water security.

Scenario 3: Strengthen MAD Water Systems

In the third research site, residents are living off-grid, without a municipal or centralized water system. The reasons this community lacks sufficient public infrastructure, including water and sewage, are complex. Some residents are unable to access and afford public water systems, while others prefer an independent off-grid sustainable/survivalist lifestyle. For this study, we selected an off-grid community of approximately 100 households in southern Arizona. The nearby town center has a small, well-functioning municipal water system, but many residents live beyond the reach of this water system and so rely on private wells. Residents here get their water through a patchwork of hybrid MAD water systems including rainwater collection, water hauling, bottled water purchases, and private or shared wells, a situation associated with health risks, labor burdens, and economic costs to residents. Pilot interviews found that residents are uncertain about the quality of their well water due to concern about contamination and limited capacity for water testing, and they are unsure how to implement or manage in-house treatment options. The purpose of our research here is to understand the issues faced by these households and then strengthen their capacity to use these MAD water systems and test for improvements in water security.

Data Collection

This study will use data from key informants and community participants, both selected through purposive samples. Inclusion is determined based on clear selection criteria, which are articulated below. Sample size will be determined based on guidance from the literature on theme saturation (Guest et al., 2006; Hennink & Kaiser, 2022; Wutich et al., 2024). We will select key informants based on the research team’s prior knowledge of the sites, partner organizations, and local organizations working in this area. Community participants interviewees will be recruited using chain-referral sampling, where participants help researchers identify other potential participants and with the study team determining if possible new participants meet the study criteria. Key informants represent the starting point of the chain-referral. We will supplement key informant and community participant interviews with participant observation by the study team and by community-based participant-observers (Roque et al., 2024). This triangulation will lead to richer and more robust findings.

During the pre-test and post-test for each phase, we conduct participant observation, informal and walk-along interviews with key informants, and semi-structured interviews with a purposive sample of community members. The purpose of this data collection is to: (1) capture a baseline qualitative measure of community water needs and concerns before the intervention, including information to co-design the intervention, and (2) determine if community water needs and concerns change following the intervention. Interview transcription will be conducted using AI-powered software and then reviewed and corrected by project assistants. We will analyze meeting and interview transcripts in the initial language of data collection, Spanish or English. Data are not translated to English until they are selected for inclusion as exemplars in final presentations or publications. The study phases are outlined in Figure 1.

Figure 1.

Data collection process

Key Informant Interviews

The key informant interviews aim to collect data on the water insecurity issues in the study areas and identify key themes to be examined in the research. We will conduct in-depth interviews in English, Spanish, or both (depending on the participants’ preference), as all lead project personnel are bilingual. These interviews are essential to help the researchers better understand the community context and their water-related issues to tailor other research instruments (i.e.,the semi-structured interview described below) and the interventions to be designed based on an initial understanding of root causes of water insecurity and potential technological and social solutions.

Key informant interviews have been used, together with other ethnographic and participatory methods, to identify and co-analyze water-related problems and possible measures to address them (A. Roque et al., 2021). We will use multiple probing interview techniques to ensure that we collect extensive data from informants (Bernard & Wutich, 2026). In this study, we select key informants based on one or more of the following purposive criteria:

1. Nomination by one of our local partner organizations

2. Referral by someone from the research team who previously worked in the area

3. Participation in previous workshops or trainings of Arizona Water for All.

4. Contact established during exploratory fieldwork.

Key informant selection is informed by the methodological literature, which suggests appropriate a priori criteria for ethnographic studies (Johnson, 1990). Our target sample size is set at six interviews per site, during pre-tests and post-tests, yielding a total n=36 (6x3x2). Sample size is selected to meet the minimum number of interviews needed to identify 94% of high-frequency codes during each pre-test and post-test in each phase (Guest et al., 2006).

Semi-structured Interviews with Community Participants

The purpose of the semi-structured interviews is to collect data on each community participant’s views and experiences of water insecurity before and after the intervention. Semi-structured interviews are conducted in English, Spanish, or both (depending on the participants’ preference), as all lead project personnel are bilingual. We use a semi-structured elicitation approach because it allows us to collect a mix of deductive and inductive information from participants (Bernard & Wutich, 2026). Because water insecurity is a well-established theoretical domain (Jepson, Budds, et al., 2017; Jepson, Wutich, et al., 2017), we are able to identify key topics which must be addressed in a systematic study of water security interventions.

Our semi-structured protocol addresses all key aspects of the water insecurity domain: water access, water use, water quantity, water quality, water concern, water sharing and participation in moral economies for water. As well, we collect broadly relevant data to facilitate comparisons, including participant demographics, data on social context, and assessments of social networks and social capital. We also ask open-ended exploratory questions on water and social context to allow participants to volunteer any additional information they feel is important. We use a range of probing techniques to ensure that we have exhaustively collected data from participants (Bernard & Wutich, 2026).

In each site, community participants are selected based on the following purposive criteria (all requirements must be met):

1. Living within the city, colonia or community limits.

2. Experiencing concerns over the quality of their water.

3. Willingness to participate in an interview and provide informed consent.

4. Participant being at least 18 years of age.

Our target sample size is set at 12 interviews per site, during pre-tests and post-tests, yielded a total n=72 (12x3x2). Sample size is selected to meet the minimum number of interviews needed to identify 92% of all themes present during each pre-test and post-test in each phase (Guest et al., 2006); the overall community participant sample size exceeds the number of interviews recommended for theme analysis (Hennink & Kaiser, 2022; Wutich et al., 2024).

Participant Observation

The purpose of the participant observation is to supplement the key informant community interviews, and understand each research site’s history, context, setting, and broader social, economic, and ecological issues. We conduct intensive participant observation at each site, which includes walking observations, unstructured interviews, and archival research. Field notes are recorded after each participant observation by at least one social scientist and one engineer, and whenever possible by a community-based participant-observer who is a member of a local partner organization (Roque et al., 2024). Observations address interpersonal interactions, social structure and dynamics, the built environment, local ecology, and all dimensions of water (in)security, as addressed below in our in-depth discussion of the interview protocol. Where feasible and ethical, we supplement written field notes with photographs, maps, and drawings.

Participant observation is a time-intensive method that involves many hours of engagement with the social, physical, and ecological environment in our field sites. There is no agreed-upon method to determine the minimum number of participant observations needed to detect key themes (Wutich et al., 2024), although recent ethnographic research suggests that supplementing participant observation with interviews can facilitate thematic saturation (DiStefano & Yang, 2024). Participant observation will be conducted at all three sites by two core study team members, supplemented by at least one community-based participant-observer and other team members whenever possible.

Consent and Compensation

Participants will be consented following IRB procedures established under STUDY00022435. They will be provided a clear consent form and an explanation from the researcher about the study’s purpose, potential risks and benefits, confidentiality measures, and how data would be used. They will be allowed to ask any questions and raise concerns. After that, if they agree to participate and allow the audio recording of the interview, they must give either verbal or written consent, acknowledging their voluntary participation and the right to withdraw at any time. In gratitude for their participation, they received monetary compensation in the form of gift cards of USD 100, divided between pre-test and post-test interviews and participation in community meetings.

Interventions

Community Meetings

In each site, a community meeting will be organized by the research team. The purpose is to create integrated engineered and social infrastructures that are tailor-designed to address community and resident water needs. The meeting itself is conducted using participatory convergence methods (Castro-Díaz et al., 2024; Roque et al., 2021) in which community members co-produce knowledge about the water systems and how they can be better run. Community meetings can elicit more information about people’s concerns and the issues underlying water insecurity than other mechanisms (Castro-Díaz et al., 2024; Wutich et al., 2024). While one of the purposes of the meeting is to co-design the most appropriate intervention for the community, the meeting itself is an intervention as it is part of the process of developing the social infrastructure that will complement the engineering infrastructure.

Prior to the intervention, during the pre-test phase, we establish a baseline understanding of individual residents’ water security needs as well as overall community concerns through participant observation, key informant interviews, and semi-structured interviews. All members of the community participant sample (see Table 1) will be invited to attend the meeting and key informants are invited to attend as observers. The intervention is facilitated by a PhD-level environmental social scientist. Technical demonstrations of engineered technologies are demonstrated by a PhD-level engineer, using protocols approved by a team of experts. Community discussions are facilitated by a PhD-level community engagement specialist.

Table 1.

Data Sources and Samples Sizes

Data source	Data type	Sample size (pre-intervention per site)	Sample size (post-intervention per site)	Sites	Total
Key Informants	Open-ended interviews	6	6	3	36
Community Participants	Semi-structured interviews	12	12	3	72
Research Team	Participant Observation	3	3	3	18

Data collection during community meetings takes three forms. Community meetings and interventions are audio recorded. Facilitators are also Participant Observers who record extensive field notes following each intervention, primarily capturing social interactions that may not be audio recorded (e.g., a community member flinches when reading the results of her home’s water quality report). Other members of the wider research team, including trained social scientists, engineers, and community partners, may provide further practical support. They will also undertake direct observation (e.g., make qualitative observations and take photographs of community meetings and interventions, capturing the setting, context, and non-verbal interactions.)

Data Analysis: Qualitative Longitudinal Assessment

We use a combination of inductive and deductive methods for our qualitative data analysis. Inductive theme identification will be used to identify major themes at each site pertaining to water issues and problems faced by the community. Inductive theme identification allows us to remain open to identifying new themes and/or theoretical domains that we may not have captured otherwise through our deductive coding. Our deductive coding deploys three different codebooks for the coding of our qualitative data. This approach allows us to quickly identify themes and theoretical domains that are expected based on the existing water insecurity literature. Combining an inductive and deductive analytical approach enables us to triangulate common and intersecting themes which serves as a cross-check on our thematic findings (Morse, 2015; Tracy, 2010).

Theme Identification

The purpose of the theme identification analysis is to inductively identify key concepts that the participants view as important. For example, participants may view the smell and taste of water as central to their experiences of water insecurity, while scientists generally do not consider organoleptic data to be reliable indicators of water quality. To inductively identify themes, we follow the techniques of Bernard et al. (2016). These include theme identification techniques like repetition, linguistic transitions, and metaphor/analogy. Multiple analysts review the data line-by-line looking for commonalities in meanings. Each analyst independently yields a list of candidate themes. The themes are then commonly discussed by each analyst, using a consensus-based technique to yield a final list of key themes for further analysis. The entire dataset is then coded for key themes, with comparisons made between themes identified pre-intervention and post-intervention.

The purpose of the thematic comparisons is to determine if the content, context, or prevalence of water insecurity themes changes from the pre-intervention to the post-intervention stage of the study. These comparisons focus on tracking changes in inductive themes suggested by the participants’ own perspectives. Following Boeije’s (2002) method for constant comparison, we will systematically compare text segments coded with the same inductive theme during the pre-test and post-test phase. We will make comparisons across participant characteristics, themes, within sites, and across sites. This analysis yields rich nuanced data on changes in inductive themes over time and space.

Systematic Coding

We develop, test, and code with three codebooks: (1) water insecurity, (2) moral economies for water, and (3) coping with water insecurity and adaptive management. The purpose of the systematic coding is to deductively identify changes in established concepts from the water insecurity literature from pre-test to post-test. For each codebook, we begin by defining the theoretical domain—all key indicators in each domain become systematic codes. We define each code using the approach of MacQueen et al. (1998) as modified by Bernard et al. (2016) and Wutich et al. (2024).

Water Insecurity Codebook

The water insecurity domain is operationalized based on the 19 indicators established in the HWISE-USA scale protocol (Pearson et al., 2025). These indicators characterize household water insecurity across six domains: (1) Affordability; (2) Quality; (3) Emotions; (4) Quantity; (5) Reliability; (6) Coping. The 19 indicators include: purchasing water, experiencing poor water quality, feeling angry over water, being unable to bathe due to insufficient water, interruptions in water access, changing schedule due to water concerns. Using this codebook, qualitative data can be coded by the six domains, by all or of the 19 sub-codes, or by a subset, e.g., by the seven codes within the water quantity domain.

Moral Economies for Water Codebook

A moral economy is a system of social norms within a community for the distribution and exchange of key resources (Beresford et al., 2025). Moral economies for the exchange and distribution of drinking water are prevalent in communities facing water insecurity (Beresford et al., 2023). When moral economies for water function well (i.e., norms are well-established and consistently upheld), they buffer households from the most severe impacts of water insecurity (Beresford, Adams, et al., 2024; Beresford, Brewis, et al., 2024). The moral economies codebook is derived from the moral economy framework developed by Beresford et al., (2023) and contains three domains: (1) shared understandings of water justice, (2) normative economic practices, and (3) social pressure mechanisms.

Coping & Adaptive Management Codebook

This codebook is developed from key theoretical and review literature summarizing coping with water insecurity (Achore et al., 2020; Collins et al., 2024; Majuru et al., 2016; Sen, 1982; Venkataramanan et al., 2020; Wutich & Slade, 2014) and adaptive management (Pahl-Wostl, 2007; Scott et al., 2013; Tompkins & Adger, 2004). The coping codes include five domains with 2-4 codes each: intensification (own labor, production, trade, transfer entitlements), modified consumption (rationing, modification/substitution, stigma), migration (fostering, migration, resettlement), reprioritization and withdrawal (withdraw, reprioritize), psychosocial and spiritual responses (psychosocial coping, spiritual coping). The adaptive management code covers responses including planning flexibly, sharing knowledge, and building enhanced response capacity.

Codebook Development

By using a team-based approach we ensure the rigor of our qualitative data analysis (Beresford et al., 2022; Bernard et al., 2016; Campbell et al., 2013; Hruschka et al., 2004; Krippendorff, 2018; MacQueen et al., 1998). Two lead coders develop a codebook with full instruction for coding units (by speaker turn) and coding level (nominal). The lead coder then draws a sample of text segments to further test and develop the codebook. This process is repeated until all text segments are easily and correctly coded for all codes, and all necessary exemplars are included in the codebook. To validate the robustness of the codes the lead coder then draws another sample of 30-40 text segments. The two lead coders separately code all segments. Inter-rater reliability between the two lead coders is calculated for each code. When Cohen’s kappa (Cohen, 1960) meets or exceeds 0.8, inter-rater reliability is “almost perfect” (Landis & Koch, 1977) and no further action is needed. If Cohen’s kappa is less than 0.6, then this process (sample test segments, test codes, modify the codebook, test inter-rater reliability) is repeated until every code reaches a kappa of 0.6, interpreted as “substantial agreement” (Landis & Koch, 1977). The three codebooks can be found in the appendices in the supplementary material.

Analysis

Following Krippendorff’s (2018) methods for classical content analysis, we will systematically compare text segments coded with the same deductive code during the pre-interventions and post-intervention phase to test for changes in pre-defined theoretical categories from the pre-intervention to the post-intervention stage of the study. These comparisons track changes in pre-defined theoretical categories in the three developed codebooks. We use statistical tests (e.g., McNemar test or a Wilcoxon signed-rank test), as well as descriptive comparisons, to test for changes.

Ethics

Intervention research in water insecurity presents ethical and moral dilemmas, highlighting the tension between supporting households’ and communities’ efforts to maintain connections to centralized systems and managing transitions to MAD systems when necessary. Along these lines, researchers and policymakers have a moral responsibility to protect participants from harm and support safe, just, and sustainable transitions in water service delivery (Wutich et al., 2025). Within this context, intervention research often blurs the lines between data collection and community engagement, adding a layer of ethical concerns (Garcia-Iglesias et al., 2024).

Researchers have a responsibility to minimize harm and to recognize and navigate power asymmetries, participation fatigue, and the uneven burdens placed on marginalized communities. However, the conventional biomedical or behavioral research frameworks of IRBs might not be well suited for assessing participatory community-based interventions, often failing to recognize the complex, structural, and layered challenges associated with water insecurity research (Brown et al., 2010). We remain attentive to issues of power, inequality, disconnection, and long-term sustainability as we navigate the challenges of either reinforcing connections to centralized systems or supporting retreat to alternative solutions.

To address these issues, we supplement formal IRB review with context-sensitive tools from participatory research frameworks such as participatory convergence (Castro-Díaz et al., 2024). We approach our ethical commitments to research interventions as an ongoing process of engagement, reflection, and shared responsibility rather than simply a necessary administrative step at the start of the research process.

Rigor

By using multiple data sources and different ways of generating data, we aim to strengthen the rigor, consistency, and trustworthiness of our work. Following Leech & Onwuegbuzie (2007), we use three strategies for triangulation of qualitative data collection and analysis. In addition to combining inductive and deductive approaches and using data from three sources (participant observations, key informant interviews, and community participant interviews), we triangulate through the different positionalities and disciplines of the research team. For participant observation, interviews, and community meetings, multiple observers are capturing field notes and other key data. At every data collection event, we have a minimum of two observers; for major events (e.g., community meetings) we have up to six observers present.

To ensure rigorous and representative qualitative data collection, we ensure that observers represent multiple genders, socioeconomic backgrounds, and places of origin. Similarly—as part of our participatory convergence approach—we have a team trained variously in anthropology, environmental social science, environmental engineering, and other interdisciplinary fields (Patton, 1999). Having observers with different social positionalities ensures that we capture data in ways that are sensitive to gender roles, cultural backgrounds, and other forms of social structure; having different disciplinary training ensures that they can identify topic-specific issues (e.g., human health risks posed by specific metals and ions, institutional barriers to water system improvements). This observation is supplemented with the perspectives of community-based participant-observers, who provide a locally grounded perspective.

Taking a structured and disciplined approach to data analysis also ensured rigor. The three code books for the systematic analysis of qualitative data were developed using established approaches (Bernard et al., 2016; Campbell et al., 2013; Krippendorff, 2018; MacQueen et al., 1998) and based on key theoretical literature on household water insecurity (Jepson, Budds, et al., 2017; Jepson, Wutich, et al., 2017; Pearson et al., 2025; Young et al., 2019), moral economies (Beresford et al., 2023; Beresford, Adams, et al., 2024; Beresford, Brewis, et al., 2024), coping (Achore et al., 2020; Collins et al., 2024; Majuru et al., 2016; Sen, 1982; Venkataramanan et al., 2020; Wutich & Slade, 2014), and adaptive management (Pahl-Wostl, 2007; Scott et al., 2013; Tompkins & Adger, 2004).

To ensure consistent coding between multiple coders on the research team, we adopt the following protocol to assess inter-rater reliability. The lead researcher selects 30-40 sub-sentence level text segments from those already correctly coded by the two leads. This constitutes the “test” that each new coder must pass in order to analyze the data (Beresford et al., 2022). The new coder codes all segments for all codes, and the results are compared against those of the lead coders. The Cohen’s kappa score (Cohen, 1960) is calculated for each code and, if above 0.6, the coder is approved to use that code. If the coder scores less than 0.6 for any code(s), they repeat the process until they reach an acceptable level of inter-rater reliability for each code. Testing for inter-rater reliability via Cohen’s kappa ensures not only that coders are applying codes to the data set in a consistent way but also demonstrates the validity of the codebook; that concepts are not perceived by only one analyst but are consistently identified by multiple researchers. Additionally, using multiple coders enables us to more rigorously identify key exemplars in the text, as key exemplars identified by multiple coders indicate the most salient examples of key themes in the data set (Ryan, 1999).

Supplemental Material

Appendix 1: Household Water Insecurity Experiences codebook

Supplemental Material for Addressing Failing Water Systems: A Qualitative Protocol for Intervention Research on Water Insecurity by Patrick Thomson, Melissa Beresford, Jobayer Hossain, Cara Jacob, Carolina Jordão, Oswaldo Medina-Ramírez, Sophie Neems, Daniel Salcedo Serrano, Dave White, Amber Wutich, and the Arizona Water for All Consortium in International Journal of Qualitative Methods.

Supplemental Material

Appendix 2: Moral Economies for Water codebook

Supplemental Material for Addressing Failing Water Systems: A Qualitative Protocol for Intervention Research on Water Insecurity by Patrick Thomson, Melissa Beresford, Jobayer Hossain, Cara Jacob, Carolina Jordão, Oswaldo Medina-Ramírez, Sophie Neems, Daniel Salcedo Serrano, Dave White, Amber Wutich, and the Arizona Water for All Consortium in International Journal of Qualitative Methods.

Supplemental Material

Appendix 3: Coping and Adaptive Management codebook

Supplemental Material for Addressing Failing Water Systems: A Qualitative Protocol for Intervention Research on Water Insecurity by Patrick Thomson, Melissa Beresford, Jobayer Hossain, Cara Jacob, Carolina Jordão, Oswaldo Medina-Ramírez, Sophie Neems, Daniel Salcedo Serrano, Dave White, Amber Wutich, and the Arizona Water for All Consortium in International Journal of Qualitative Methods.

Supplemental Material

Arizona Water for All Consortium co-authors

Supplemental Material for Addressing Failing Water Systems: A Qualitative Protocol for Intervention Research on Water Insecurity by Patrick Thomson, Melissa Beresford, Jobayer Hossain, Cara Jacob, Carolina Jordão, Oswaldo Medina-Ramírez, Sophie Neems, Daniel Salcedo Serrano, Dave White, Amber Wutich, and the Arizona Water for All Consortium in International Journal of Qualitative Methods.