Failure of the “Gold Standard”: The Role of a Mixed Methods Research Toolkit and Human-Centered Design in Transformative WASH

Abstract

From preventing cholera and diarrhea by reducing exposure to human waste, to reducing transmission of COVID-19 through handwashing, water, sanitation and hygiene (WASH) can save lives. Numerous global health initiatives have been created to combat the spread of infectious diseases. However, according to the Sanitation and Hygiene Fund, “decades of under investment in sanitation and hygiene have made this sector the weakest link in our efforts to achieve the [Sustainable Development Goals (SDGs)].” There appear to be various reasons for the lag in global attention to, funding toward, and innovation around WASH-related diseases. Firstly, WASH is comprised of three interrelated components, water, sanitation, and hygiene, each of which has its own subset of indicators, priorities, and infrastructure, thus making streamlined communications and impact measurement within the sector incredibly complex. Secondly, WASH is a field that bridges many sectors, and there has historically been a lack of understanding of where responsibility lies to consistently fund and execute WASH interventions, programming, and policymaking. Additionally, public health research and funding tend to favor evaluations using randomized controlled trials (RCTs), which are often referred to as the “gold standard.” RCTs, like all single evaluative methods, have limitations which may not effectively capture the complexity of WASH interventions and their subsequent multi-sectoral outcomes. In some cases “it may be infeasible (or unethical) to randomize communities to a [WASH] intervention” which would prohibit the research from reaching the current “gold standard” threshold for academic rigor and subsequent funding. A new concept called “Transformative WASH” has recently emerged in the WASH sector as a result of three RCTs and calls for a “comprehensive package of WASH interventions” to effectively improve health and social outcomes. We believe that the current definition of the “gold standard” in academic research is failing the WASH sector and does not align with “Transformative WASH.” Rather, the “gold standard” should instead be a mixed methods research toolkit that utilizes Human-Centered Design (HCD) practices, and proxy methods such as “participatory design” or “Behavior Centered Design theory” to better design and evaluate WASH interventions.

The Need for Iterative and Human-Centered Approaches in WASH Research

Effective water, sanitation, and hygiene (WASH) systems are crucial for preventing and mitigating infectious disease outbreaks. Over 2.3 billion people lack safely managed drinking water and 4.5 billion lack safely managed sanitation, which contribute to widespread outbreaks. ¹ From preventing cholera and diarrhea by reducing exposure to human waste to reducing transmission of COVID-19 through handwashing, WASH can save lives. It is widely documented that lack of effective WASH service provision and infrastructure leads to negative health and social outcomes; ² despite this, the United Nations (UN) has reported that the “world is off track when it comes to reaching the sixth Sustainable Development Goal (SDG),” which seeks to “ensure the availability and sustainable management of water and sanitation for all by 2030.” ³

Numerous global health initiatives have been created to combat the spread of infectious diseases. However, according to the Sanitation and Hygiene Fund, “decades of under investment in sanitation and hygiene have made this sector the weakest link in our efforts to achieve the [SDGs].” ⁴

Various reasons exist for the lag in global attention to, accelerated funding toward, and innovation around WASH-related diseases. Firstly, WASH is comprised of three interrelated components, water, sanitation and hygiene, each of which has its own subset of indicators, priorities, and infrastructure, thus making streamlined communications and impact measurement within the sector incredibly complex. Secondly, WASH is a field that bridges many sectors, and there has historically been a lack of understanding of where responsibility lies to consistently fund and execute WASH interventions, programming, and policymaking. ⁵ WASH projects and programs also produce multi-sectoral outcomes which include, but are not limited to, reducing enteric diseases as well as improving nutrition, social well-being, and economic productivity. ³ From our perspective, the necessary multidisciplinary inputs and inevitable multi-sectoral outcomes of WASH interventions add unique challenges that should be addressed in research study designs and evaluations. Since global health research and funding priorities are typically disease specific rather than interdisciplinary and reactionary rather than preventive, it can be challenging to attribute indirect and multi-sectoral health and social outcomes to a WASH intervention; thus, it becomes difficult to measure and showcase the full impact of WASH work. Additionally, public health research and funding tend to favor evaluations using randomized controlled trials (RCTs), which are often referred to as the “gold standard.” ⁶ In the context of some research, RCTs are used effectively as an evaluation method to “reduce bias and provide a rigorous tool to examine cause-effect relationships between an intervention and outcome” through randomization. ⁶ However, this is not always possible, effective, nor appropriate in WASH. RCTs have limitations which may not effectively capture the complexity of WASH interventions and their outcomes. ⁶ In some cases “it may be infeasible (or unethical) to randomize communities to a [WASH] intervention” which would prohibit the research from reaching the current “gold standard” threshold for academic rigor and subsequent funding. ⁷ Angus Deaton states that “RCTs can play a role in building scientific knowledge and [are] useful predictions, but they can only do so as part of a cumulative program, combining with other methods, including conceptual and theoretical development, to discover not “what works,” but “why things work.” ⁸

Some limitations of RCTs in WASH can be observed from the findings of the WASH-Benefit trial and the Sanitation, Hygiene, Infant Nutrition Efficacy (SHINE) trials, as well as observing learnings from the Effect of an Urban Sanitation Intervention on Child Health (MapSan) non-randomized trial. These were global WASH trials “of unprecedented scale and cost [and that] included novel factorial designs.” ⁹ Despite all of the resources devoted to this work, the trials “found no effect of basic [WASH] interventions on childhood stunting, and only mixed effects on childhood diarrhea.” ⁹ These outcomes were largely attributed to fecal contamination in the surrounding environment at the community-level that was not addressed by the WASH intervention nor included in the RCT evaluations. In many cases, the trials were unable to evaluate a key route of fecal exposure and thus researchers were left with an incomplete picture. Cumming et al ⁹ concluded that a package of WASH interventions is needed and should be “tailored to address the local exposure landscape and enteric disease burden.” Additionally, it has been noted that “the relationships between sanitation, environment, behavior, and health are complex” and requires vast changes across sectors, domains and behavior to reduce environmental fecal contamination and improve health outcomes. ¹⁰ Amy Pickering and her colleagues stated in an observation of these trials that “RCTs often do not fully capture heterogeneity within a community or represent the target population of interest” and that “community-scale interventions are more challenging to study using the RCT methodology.” ¹¹ These findings echo international consensus that safely managed sanitation, as measured by the Joint Monitoring Programme, must include the entire sanitation value chain, not just a single measure of access to basic facilities. ¹² The subsequent conversations surrounding these trial and WASH indicators have led to the emergence of a new concept called “Transformative WASH,” ¹¹ rooted in the argument that WASH provision alone does not reduce environmental fecal contamination and calls for a “comprehensive package of WASH interventions” ¹⁰ to effectively improve health and social outcomes.

Transformative WASH and Human-Centered Design (HCD)

We believe that the current definition of the “gold standard” in academic research is failing the WASH sector and does not align with “Transformative WASH.” Rather, the “gold standard” should instead be a mixed methods research toolkit that utilizes Human-Centered Design (HCD) ¹³ practices and proxy methods such as “participatory design” or “Behavior Centered Design theory” to better design and evaluate WASH interventions.

HCD is a creative, generative, and human-centered problem-solving framework that has been growing in popularity in the business and academic worlds over the past two decades. ¹³ HCD was developed to complement the scientific method, a rigorous and linear problem-solving methodology, in order to promote “inclusion, starting with the people you are designing for” using Design Thinking, “the iterative method used to apply this approach towards innovation”. ¹³ While business innovation methods typically consider the criteria of desirability, feasibility, and viability when evaluating if a concept will succeed, design processes expand on this by considering these criteria throughout the entire development process rather than as metric at the end. The HCD process begins with listening to stakeholders to build empathy (hear), developing solutions with end user input by iterating based on feedback (create), and implementing open-ended concepts to further develop based on ongoing feedback (deliver). ¹⁴ In addition to IDEO’s “Hear-Create- Deliver” approach, ¹⁴ there are several Design Thinking and HCD frameworks that exist such as the DEEP (Discover, Empathize, Experiment, Produce) Design Thinking methodology ¹⁵ and the Double Diamond process. ¹⁶

The value of these frameworks is that they offer a cyclical and qualitative opportunity for continued alignment, engagement, and iteration throughout the research process based on feedback from all stakeholders—from the multi-sectoral project planners and implementers to the end users. As depicted in Figure 1, we call for the use of HCD methods across WASH interventions and throughout program development. Rather than relying on needs assessments alone, which often are the only point when stakeholders are consulted, integrating HCD into the full research and program process gives the team time to realign and pivot hypotheses and methodologies prior to the end of the project scope or budget. Since WASH infrastructure is known for being a large capital investment with evidence suggesting that 30% to 50% of WASH projects fail within two to five years because of human-centered issues such as maintenance, ⁵ we believe that HCD’s iterative and user-focused approach could greatly benefit the WASH sector and allow for more effective use of limited available resources.

Figure 1.

Human-centered design (HCD) and a mix methods research toolkit concept for transformative WASH.^13,19

Siloed approaches to WASH research are limiting and inefficient. Incorporating multi-sectoral collaboration from the onset of WASH interventions and consistently revisiting and iterating through HCD methods could help ensure that the common goals of WASH researchers, the public sector, and the private sector are aligned; that resources are effectively utilized; and that efforts are not unnecessarily duplicated. ¹⁷ According to the Acumen Academy, “the sooner you embed a culture of smart failure into your workflows and teams, the faster you will design useful solutions for the people you serve.” ¹⁸ This mix-method, HCD research toolkit could better enable the WASH sector to have smart failures that are calculated for at the pilot level before projects are scaled up.

Our hope is to help shift WASH research, practice, and funding to be more interdisciplinary, inclusive, and iterative. We also believe that the historic, neocolonial norms that have been codified in the WASH and public health sectors should be eradicated in order for the world to reach the sixth SDG. If used intentionally, HCD can provide an equity-centered and sustainable lens that can combat the top-down approach to research and pave the way towards the future of “Transformative WASH.”

Human-Centered Design (HCD) in Action

In the literature, there has been limited use of HCD and proxy methods in the WASH sector, though several organizations such as iDE ²⁰ and Population Services International (PSI) have been utilizing HCD frameworks in their WASH work over the past few years. Ben Cole and his coauthors sought to examine the application of “participatory design methods” in the sanitation sector by critically reflecting on three case studies that applied this HCD proxy method to sanitation innovation in rural Malawi. ²¹ The authors note that the participatory design methods “led to a number of [community-based] innovations including corbelling structures and trapezium shaped bricks” to address the lack of effective sanitation technologies in the communities. ²¹

Another example of HCD in WASH was the Wish for WASH (W4W) SafiChoo toilet pilot in Zambia; the W4W product design team utilized an abandoned bathroom facility and upgraded it with diverse members of the community to ensure that both the toilet seat and waste management systems were sustainable and usable based on their local needs. ²² The W4W team utilized the Hear-Create-Deliver method where the “Hear” steps involved a baseline needs assessment and monitoring of the pilot household; the “Create” stage was characterized by rapid toilet product iteration; and the “Deliver” stage was the upgraded toilet facility. ²² While this approach often takes more time, the pilot showcased the power of rapid HCD prototyping and iteration. Integrating HCD could ultimately strengthen WASH interventions before investing the time, human capital, and funding for a WASH project to scale up.

Practitioners who utilize HCD processes must avoid asserting their own biases and assumptions to allow long-term stakeholders such as community members and municipalities to drive the process. Both the design and global health sectors are currently working to “decolonize” their methodologies to enable truly sustainable and inclusive product, service, program, and intervention development that includes diverse leadership, representation and skill-sets to address health and research inequities. ²³ We believe that HCD can be a generative tool that can be used to build societal equity by building pipelines for diverse innovators and project leaders in WASH and design to take the lead.

Conclusion

Overall, HCD has not yet been effectively integrated into the WASH sector. Codifying RCTs as the single “gold standard” evaluation methodology and continuing siloed WASH research and practice is limiting. Relying largely on the results of RCTs to inform policymaking and strategic funding in WASH has resulted in underfunding and poor sustainability. We believe that further integrating multi-sectoral collaborations and HCD in WASH interventions could lead to more sustainable and equitable innovation and funding in WASH research.

While HCD is a promising framework to achieve “Transformative WASH” and could help the sector move away from its top-down, neocolonial norms which have consistently proven to fail, integrating HCD into traditional WASH research methods will require additional funding and structural changes in order to meet the UN’s global targets for SDG six by 2030. Expanding the definition of the “gold standard” to include this methodological toolkit could help shift the culture of science to ultimately pave the way for more innovative, sustainable, and inclusive projects, programs, and products in WASH and in comparable sectors such as global health and development.