Acknowledging and Learning from Different Types of Failure

Abstract

The challenges faced in sanitation and hygiene programmes are numerous and complex. Failures are inevitable. From our experience of working on rapid action learning and research in this sector we have found that when mistakes are shared they are usually those which were uncontrollable and unanticipated i.e. somebody else’s fault. In this perspectives piece we propose a typology of failure alongside criteria for research and learning processes that prioritises timeliness, relevance and actionability. We argue that these can be used together to identify and reflect on failures (and successes) quickly. We provide some practical suggestion for different stakeholders to support a shift towards a more open and reflexive sector, where all types of failures can be shared broadly.

Introduction

The barriers to improving sanitation and hygiene (S&H) services for all are complex and numerous, encompassing social, institutional, technical and environmental challenges. ¹ Tackling these at scale using a single, static approach has shown not to be effective everywhere.^2,3 Consequently, to leave no one behind and ensure sustainable, safely managed S&H services, nuanced, tailored and adaptive interventions are essential. At the same time, reaching people in the most challenging contexts will be harder to do, be more expensive and have higher risks. Failures are inevitable – it is important that we learn from these mistakes, take corrective actions, reflect and share these experiences with others to build the evidence base on what does and what does not work rapidly.

S&H programmes and implementers (both government and non-government stakeholders) face many challenges when trying to adapt and learn from failures. Programmes are often rigidly mapped out at the beginning, with little space for reflection, learning and adaptation along the way. Waiting for mid or end-term programme evaluations is unlikely to be sufficient to take timely corrective actions.

Conventional academic research faces limits with identifying and responding to challenges rapidly. Randomised Control Trials, for example, often considered the gold standard for reliable and rigorous research, are narrow in scope, costly, long and rigid once launched. Traditional scientific rigour often means it takes months or years before findings are released, meaning operational challenges are not picked-up rapidly enough, making it too late to respond and adapt accordingly. ⁴ A common academic view is that rigour generally requires more rather than less time, and that less time means less rigour. However, the quality and depth of insights that result from more rapid approaches can have their own rigour, through quick triangulation, being in touch and up to date and reflection, deliberation and cross-learning. ⁵

To support programmes to become adaptive and meet these complex challenges, new, innovative learning approaches are needed which uncover ground realities, identify mistakes and challenges quickly and encourage action and innovative practice. At the Sanitation Learning Hub (SLH) we have been using the term Rapid Action Learning (RAL) and developing a range of methods that can support this learning to take place. ⁴ Key to success is embracing both the successes and failures, learning from and sharing these experiences, being transparent and open and adapting and pivoting constantly to keep up with the evolving context.

Typology of Failure

Failures occur for a wide-ranging and intersecting number of reasons, some are easier to anticipate and control than others. Figure 1 outlines a typology of different types of failures. The typologies cross-over, interlink and can change depending on the level you are operating at and over time. For example, a failure can move from being unanticipated to anticipated if it occurs multiple times. While systems can be put into place to make something more controllable. However, the typologies are useful when reviewing the types of failures people are willing (and unwilling) to share and help actors think through ways these can learnt from rapidly.

Figure 1.

Typology of failure.

Below a brief description of each failure is given and in the spirit of transparency failures of our own from the last 6 years provided.

• Type 1: Avoidable Failure can be anticipated and directly controlled. These are failures that programmes face where the challenge was foreseen and preventative actions could have been taken. Other types of failure can move into this space if repeated or mitigation strategies are not put in place. These failures are hardest to acknowledge and share.

SLH Example: Not allocating time to present the findings from research back to communities. Failure to speak truth to power, due to fear of rupturing close relationships with key stakeholders and losing what ‘influence’ we may have with governments, has sometimes led us to hold back on speaking publicly where we see problems.

• Type 2: Calculated (Risk) Failure are due to factors than can be anticipated and mitigated against, but cannot be directly controlled. Unlike preventable failures, these can be attributed to others, making them slightly easier to acknowledge and share.

SLH Example: Organising field trips for S&H stakeholders who once in communities lecture and criticise households, take photographs without asking or visit toilet facilities and enter houses without requesting permission from the owners. This should have been mitigated against by having more in-depth preparatory sessions covering attitudes, behaviours and ‘do no harm principles’.

• Type 3: Manageable failure come as a surprise but could be better controlled. These can include unintended consequences of programming that may cause harm to wellbeing, something both market-based sanitation ⁶ and Community-Led Total Sanitation ⁷ (CLTS) has faced criticism for. Under these conditions mitigating strategies can be built into programmes, even if the precise timing and nature of the failure is unforeseeable. Similar to calculated failures, these are easier to share than avoidable failures but harder to share than unpreventable failures.

SLH Example: In our previous incarnation as the CLTS Knowledge Hub we were slow to realise and openly acknowledge the challenges of implementing the CLTS approach at scale, the potential for people to be left behind, the need for external support mechanisms in some circumstances, and the risk of human rights abuses being committed in the drive to open defecation free communities.

• Type 4: Unpreventable failures are due to unpredictable events and shocks which are out of your sphere of influence. These failures are easiest to acknowledge and share.

SLH Example: Having to cancel and delay events and activities due to snap elections or natural disasters making locations either unsafe or collaborating organisations being directed towards more immediate needs. For many programmes (including ours), the COVID-19 pandemic is the most current external shock.

The type of ‘failure’ impacts the willingness to accept and openly share them with others. Mistakes are often framed as external, unanticipated, or uncontrollable. We experienced this ourselves when brainstorming our own failures for this piece. At first we came up failures which put us in a good light, or that could be blamed on others. For example, we tried to do something but were let down by someone else or we were overambitious and overstretched ourselves. Being externally facilitated through the process of identifying failure and seeking ways forward can be a useful and positive way of identifying and exploring more preventable failures.

Rapid Action Learning (RAL) to help address failure

We’ve been trialling and advocating for RAL, which prioritises timeliness, relevance and actionability when designing research and learning activities. These methods can help identify challenges and failures (alongside successes) at pace. Key criteria for RAL are:

• • Timely – rapid learning, analysis, triangulation and feedback enabling problems and challenges to be identified and rectified sooner.

• • Relevant – being in-touch and up to date with ground realities, enabling issues to be identified as they emerge. Encouraging the capture of negative information, such as unintended consequences. Sharing what has not worked, lessons learnt and the adaptations, adding to credibility, authority and trust.

• • Actionable – innovations and recommendations given are practical and adapted to the context and enable stakeholders to utilise and plan accordingly (This criteria follows in the tradition of other action-orientated methods including Rapid Rural Appraisal, Participatory Learning and Action, Rapid Epidemiological Assessments and Participatory Action Research.^8,9).

RAL covers methods that focus on data collection and uncovering field realities and those that focus on sharing knowledge, analysis, reflections and generate action. These include debiasing visits, rapid topic explorations, RAL Workshops and immersive research ⁴ which can help rapidly identify and overcome avoidable and manageable failures (1 and 3) and build capacity and resilience to respond and adapt to calculated and unpreventable failures (2 and 4). RAL Workshops ¹⁰ for example have been found to enable the identification and sharing of challenges and good practices between government staff implementing India’s Swachh Bharat Mission, ¹¹ developed the capacity of participants and led to the development of achievable action plans. ¹²

Incentives for researchers can either support or undermine willingness to engage in RAL and more applied research methods. In some fields it can be more challenging to publish work using these types of methods in high-ranking journals.

Recommendations

A shift in how we view failure and learning is needed by all stakeholders, especially as we work in more challenging contexts which will involve more experimentation, innovation and in some cases failure. This will require (amongst other things):

Implementers who. . .

• ○ take risks, but embed learning, reflection and course correction processes into their programmes.

• ○ share failures (and successes) publicly to avoid similar mistakes being repeated by others.

Donors who. . .

• ○ encourage and enable implementers to innovate, reflect and adapt.

• ○ have a two-way relationship of transparency and trust with their grantees,

• ○ acknowledge that unforeseen blockages and challenges will exist.

• ○ allow flexibility within donor reporting frameworks and budgets ¹³ ; enable adjustments to targets and related payments, when provided with appropriate evidence to justify the change.

• ○ find new ways to build in incentives to report on failures as well as successes.

• ○ invest in the set-up of internal programme management structures that facilitate reflection, learning and course correction ¹³ and double-loop learning.

Researchers who. . .

• ○ are open to unexpected results and share when research goes wrong and theories are not proven.

• ○ are open to expand the criteria for rigour to factor in timeliness, cost-effectiveness, relevance and actionability of findings.

• ○ will talk freely about failures as a vital aspect of research transparency. ¹⁴

Publishers who. . .

• ○ understand that failures are worthy of reporting on and provide incentives to do so. ¹⁴

• ○ value more applied research methods with direct real-world outcomes.

Policy makers who. . .

• ○ are open to learning about realities on the ground, realistic about targets and open to acknowledging when they are missed.

A broader sector that. . .

• • fosters neutral spaces where reflection and sharing of mistakes is encouraged

• • acknowledges that working in more challenging operational environments and insuring inclusivity is likely to cost more with increased risk of failure. ¹⁵

• • incentivises and rewards the sharing of failures as well as success and redefines failure as a learning opportunity that leads to better practice.

We hope these typologies and RAL approaches can help different stakeholders think through failures (ideally before but definitely after they happen) and share all types more broadly. In the current COVID-19 context, mistakes, failures and barriers seem to be becoming more openly discussed. We hope this will continue in the post-pandemic work with all stakeholders working to foster an open learning environment and failing forwards collectively.