Realist Lens and the Theory-of-change Approaches to Evaluate the Impact of Digitally Driven Capacity-building Initiatives for Mental Health in India: A Conceptual Framework

India’s growing burden of mental health disorders is a pressing public health challenge, with significant gaps in the availability, accessibility, and affordability of mental health care services. ¹ Currently, the country has 0.75 psychiatrists available for every 1 lakh (100,000) population. ² The government is making efforts to increase the number of psychiatrists toward the WHO norm of 3 per lakh population. However, this would take longer to achieve. Similarly, the annual report of the Rehabilitation Council of India (RCI) 2023–2024 reports that there are 3,793 Clinical Psychologists registered with the RCI, ³ which would be about 0.29 per lakh population. Though similar data on psychiatric social workers and psychiatric nurses is not readily available, the norms throughout the country reflect an immediate need to empower the existing resources in healthcare to deliver mental health care to the community at large. In this context, capacity-building initiatives targeting frontline healthcare providers have emerged as a crucial strategy to bridge this gap, equipping them with the skills and knowledge necessary to deliver mental health services at the community level. ⁴

Discussion

Capacity building in primary mental health care is based on the concept of task shifting and task sharing. ⁵ While this process has usually involved an in-person/offline mode of delivery for skill transfer and enriching the knowledge, the introduction of digitally driven capacity-building initiatives—such as e-learning platforms, telepsychiatry services, and mobile-based applications—has brought in a significant shift in capacity building. These approaches hold immense potential for scaling up training, reducing geographic barriers, and providing continuous learning opportunities to healthcare providers. However, implementing such initiatives is complex, requiring consideration of local sociocultural contexts, digital literacy levels, and existing health system capacities. We believe that effective implementation depends on collaboration among multiple stakeholders, both vertically and horizontally, including government agencies, academic institutions, technology providers, community-based organizations, and public-private partnerships.

While these efforts signify a positive shift in primary mental health care delivery, the actual test lies in evaluating their impact and sustainability. Evaluation would not just be about measuring outputs—such as the number of providers trained—but about understanding the mechanisms of change that underpin the success or failure of these initiatives. The mechanisms of change here refer to the processes involved in implementation, the stakeholders involved and their knowledge, attitudes, and practices, the health systems and policies at large, and the barriers and facilitators to implementation that would bring about the desired change or outcome. It is essential to assess how digital tools and training frameworks influence the knowledge, attitudes, and practices of healthcare providers, the implementation process, and, in turn, how these changes translate into improved patient outcomes. This also provides critical insights into the scalability and replicability of these models, ensuring that investments in digital health solutions are cost-effective, evidence-based, and aligned with community needs.

In this regard, the evaluation needs to expand from what works to what works where, to whom, how, why, and in what circumstances. This brings us to the philosophy of realism ⁶ or looking through a realist lens to understand the impact of digital capacity building.

Why Realist Lens?

The health system in India has both public and private sectors delivering care through a tiered approach at three levels: primary, secondary, and tertiary. ⁷ There also exist alternative medicine approaches (AYUSH) and various cultural/religious practices and norms among different communities. In addition to the trained specialists, there are Medical Officers at Primary Health Centers, Community Health Officers at Ayushman Arogya Mandirs, and Accredited Social Health Activists (ASHAs) at the grassroots level for healthcare delivery. There are also motivated community volunteers, Anganwadi workers, and others who help in health care delivery. ⁸ Governing them at the district level is the district level administration. Apart from health care, law enforcement agencies (police and the judiciary) also play a role. Complicating this, there are quacks and malpractices in the community. There are national and state-level programs, policies, and laws that govern and provide directives for healthcare delivery. A cursory look at these reveals that the health system and its interactions are incredibly complex and not linear.

The non-linearity of health systems, their interactions, and the need to consider them during the implementation of initiatives/programs have been acknowledged in the scientific literature for a long time. ⁹ That is the reason why programs/interventions might work somewhere for someone, while they may fail somewhere else.^10–12

This highlights the need to look at systems, implementations, and interactions in health care not just through systematic reviews, but also through another lens. The realist approach to evaluation aims to answer context-sensitive, policy-relevant questions. Developed by Pawson and Tilly (1997), ¹³ the philosophy of realist evaluation is rooted in the idea that interventions work differently across various environments. Realist evaluation attempts to answer that.

This theory-driven approach has the following major concepts: ¹³

Context: The unique environment in which the intervention operates.

Mechanisms: The processes and resources activated by the intervention.

Outcomes: The observed changes resulting from the interaction between context and mechanisms.

Actors

Actors refer to the individuals, groups, or organizations whose decisions, actions, and responses influence how an intervention works. In this case, it relates to frontline healthcare workers, trainers, supervisors, community members, patients, program managers, funders, policy implementers, and administrators at the community, district, state, and national levels.

Initial Program Theory

An explicit set of propositions about how an intervention is supposed to generate desired outcomes by triggering specific mechanisms within particular contexts, which can later be tested, refined, or refuted through realist evaluation.

Testing the Hypothesis and Refining Program Theory

This refers to systematically examining whether the assumptions and causal pathways proposed in the initial program theory are observed in real-world implementation, refining the program theory, and generating the context-mechanism-outcome (C-M-O) configurations.

Another theory-driven approach that facilitates insight into it is the theory-of-change (ToC) approach. A brief overview of this approach is given below:

ToC approach ¹⁴ : This approach follows the linear pattern of if …, then …. This provides a structured, visual, and narrative representation of how a program is expected to bring about the desired outcome. The components of this approach are described in Figure 1. It is important to note that, unlike realist evaluation, ToC does not answer the why and in what context, and how.

Figure 1.

Connell and Kubisch. ¹⁴

Realist lens and ToC approach to evaluate digitally driven capacity-building: The process of realist evaluation of digitally driven capacity-building in mental health care involves generating context, mechanisms, and outcomes. These could be evaluators’ or assessors’ hypotheses or assumptions about why digital capacity building in mental health care will lead to the desired outcome.

In this regard, the contextual factors could include digital literacy and technology accessibility, sociocultural attitudes toward mental health, variability in mental health needs across regions, felt need among the frontline healthcare workers, willingness of the local health administration to accept the training as a necessary step, importance given toward integrating mental healthcare into primary care health systems, workload of primary care providers, intrinsic motivation of care providers toward these training initiatives, whether incentives are given or not etc. These contextual factors play a significant role in determining the mechanisms and outcomes. For example, if the digital literacy among the targeted group were low, the mechanisms used would have to be altered to compensate. If not, achieving the desirable outcome would be difficult. There can be various contexts in this regard. Context provides valuable insights into whether the results can be generalized. The mechanisms can include knowledge transfer through digital modules, supportive hand-holding through engagement strategies (collaborative video consultations), motivation of participants to apply acquired skills, and administrative support to facilitate the transfer of skills to clinical practice, to name a few. This would lead to outcomes including enhanced mental health literacy, improved mental healthcare delivery among frontline healthcare workers, and greater accessibility to mental health services in underserved areas. This is only an example of how the context, mechanisms, and outcomes could look. These are derived through the collection and analysis of existing data. Based on this, an initial program theory could be proposed, like:

If a structured, context-sensitive digitally driven capacity-building program in primary mental health care (including supportive hand-holding through collaborative video consultations) is implemented for Frontline Healthcare Workers, then their knowledge, attitudes, and practices regarding mental health will improve, leading to increased identification, referral, and care of persons with mental illness. This is because the program activates mechanisms such as increased confidence, improved mental health literacy, perceived self-efficacy, particularly in enabling environments with administrative support.

This theory will then be tested and refined, and a C-M-O configuration will be arrived at.

To better understand this, we highlight two realist studies. The first one evaluated a capacity-building program for district health managers in a southern district of India using realist evaluation. ¹⁵ This study noted the potential mechanisms through which a capacity-building program in health may drive organizational change that can operate across multiple levels: at the individual level, through enhanced self-efficacy and organizational commitment; within teams through improved workplace dynamics; at the organizational or district level through strengthened structural and managerial characteristics and clearer reporting relationships across hierarchies; and through interactions with other talukas or healthcare organizations by fostering collaboration and integration within the broader local health system.

The second study utilized a realist process evaluation within a randomized controlled trial to integrate Diabetes and Depression treatment. This study used a case-control design across two clinics. It was noted that differences in organizational structures and implementation processes across the two clinics offered valuable insights into how various contextual factors facilitate or impede implementation. ¹⁶

Unlike Realist evaluation, ToC maps how program inputs (e.g., training modules, digital platforms, mentoring), activities (e.g., online training sessions, skill-based workshops, support through collaborative video consultations, feed-backs through focused group discussions), and immediate outputs (e.g., number of participants trained) are logically expected to produce short-term outcomes (e.g., increased knowledge), intermediate outcomes (e.g., improved detection, referral and management of mental health conditions), and long-term impacts (e.g., reduced treatment gap, strengthened primary mental health care eco-system).

ToC is helpful at the program design and planning stage, as it clarifies the assumptions underpinning digital training interventions. For instance, it assumes that FHWs have reliable internet connectivity, are willing to acquire skills and expand their knowledge, and that leadership at the district or state-level supports the integration of mental health into primary care. It also identifies risks such as poor digital literacy, parallel clinical and administrative responsibilities, or inadequate administrative support, which may hinder uptake and impact. By explicitly articulating these assumptions, a ToC helps stakeholders—program designers, implementers, and funders—align on expected pathways of change and develop measurable indicators to monitor progress.

However, as mentioned earlier, while a ToC describes how change is intended to occur, it does not fully explain why outcomes vary across contexts or among actors. In diverse healthcare settings and given that health is a state subject in India, variations in infrastructure, sociocultural factors, and organizational readiness may influence how frontline workers interpret and respond to digital training. This is where realist evaluation complements ToC, by empirically testing its assumptions and unpacking the C-M-O relationships that drive program success or failure. Together, ToC and realist evaluation provide a comprehensive approach: ToC offers the roadmap for change, while realist evaluation provides the assessment tool to understand what works, for whom, under what circumstances, and why.

Benefits and Limitations

Benefits

The Realist Evaluation approach, combined with ToC, offers a detailed understanding of how digital training interventions can be tailored to meet diverse regional needs. This dual approach enables iterative improvement, allowing the intervention to be adjusted based on ongoing findings.

Limitations and Challenges

We feel that implementing a Realist Evaluation in the Indian mental health context might pose logistical challenges, including difficulties tracking outcomes across diverse contexts and the need for significant resources to tailor interventions regionally. Additionally, digital access remains uneven, affecting the reach and efficacy of digital programs in remote areas. Evaluation also incurs a separate budget and extended timelines that a program might not have. Also, the time taken for realist evaluation is longer. Due to the dynamic nature of contexts, this can be a disadvantage, as contexts might change when the evaluation results are arrived at. Another challenge is the lack of expertise. The realist approach is an emerging evaluation approach; more knowledge and capacity are needed to conduct realist evaluations of mental health capacity-building programs.

Conclusion

With diverse landscapes, practices, and healthcare delivery systems across the country, there is a gap between policy and practice in communities. Realist evaluation and ToC approaches help understand how this time gap can be reduced and how digital capacity-building initiatives can be better implemented across cadres for primary mental healthcare delivery.