An Effectiveness-implementation Hybrid Cluster Randomized Controlled Trial to Evaluate Add-on Online Mental Health Training for Primary Care Doctors in Influencing Their Management of Commonly Prevalent Psychiatric Disorders: Description of the Methodology

Abstract

Background:

Building capacity in primary mental healthcare supports effective task-shifting and sharing. Using technology in this context offers convenience and scalability.

Methods:

We describe the methodology of a novel effectiveness-implementation hybrid randomized controlled trial to evaluate the effect of an add-on online mental health training (OMHT) program for primary care doctors (PCDs) on their management of commonly prevalent psychiatric disorders. The comparator was training as usual (TAU).

Nine (9) taluks of Tumkur District in Karnataka state were divided into study (SG) and control group (CG) clusters through randomization (SG-6; CG-3). TAU: an annual, in-person training program for 2 days on identifying and treating commonly prevalent psychiatric disorders in the community. was received by all PCDs across the district. An add-on OMHT (12 hours of digital training followed by supportive handholding through Collaborative Video Consultations for 3 months) was provided to the PCDs in SG. Primary outcomes included comparison of diagnostic concordance, treatment concordance, and combined concordance (diagnosis and treatment) between the assessors (research psychiatrists) and PCDs. Secondary outcomes were comparison of (a) short-term clinical outcomes of patients with psychiatric disorders treated by PCDs using standardized scales, (b) implementation questionnaire having a bearing on integrating mental health into primary healthcare delivery systems and (c) procurement pattern of psychiatric medications at Primary Health Centers (PHC) across three phases: 4 months pre-training, training (4 months) and 4 months post training.

Results:

The results of this trial are described separately in other articles.

Conclusions:

The trial’s results shed light on the utility of online training methods for PCDs to hone their skills in primary care psychiatry. It would also provide insights into the implementation barriers and facilitators for capacity-building programs, as well as their translation into clinical practice.

Registration:

Clinical Trials Registry of India (CTRI/2024/02/062906).

Key Messages:

• This article describes the methodology of an effectiveness-implementation hybrid cluster randomized controlled trial to evaluate the effect of add-on online mental health training for primary care doctors in mental health care delivery.

• Results from this trial may provide insights into the utility and implementational aspects for the integration of mental health into general healthcare in primary care settings of India.

Primary care doctors (PCDs) can correctly diagnose and initiate treatment for commonly prevalent psychiatric disorders, such as depression. ¹ However, in our experience, this is not getting implemented at the desired rates, as evidenced by the fact that about 50%–75% of mental health disorders go undiagnosed (and consequently untreated) at Primary Health Centers (PHC) in India. In India, PCDs are trained medical doctors with a bachelor’s degree. Unfortunately, most undergraduate psychiatry teaching in South Asian countries, including India, is rendered inadequate to tackle problems prevalent in primary care settings. ² To overcome this hurdle, task-shifting and task-sharing approaches have been in vogue to equip and empower PCDs with skills to manage commonly prevalent mental health problems in primary care settings. Indeed, the National Mental Health Program and its operational arm, the District Mental Health Program (DMHP), consider integrating mental healthcare into general primary health care as one of the foundational objectives to achieve the ambitious aim of reducing the burgeoning treatment gap for psychiatric disorders in India.

In collaboration with state governments, several large-scale programs are underway in India to scale up mental health capacities of the primary healthcare workforce, including non-specialist medical officers (MO), Community Health Officers (CHOs) of the Ayushman Arogya Mandirs, and Accredited Social Health Activists (ASHAs) through digital technology.^3–5 However, their uptake has not been uniform. For example, in Karnataka, as part of a multistate, digitally driven capacity-building program, only 50% of the 15,000 or so frontline healthcare workers attended 50% of the training sessions. Insights from the focused group discussions with the participants revealed challenges related to digital literacy, network connectivity, concurrent parallel programs, their intrinsic motivation, and administrative support.

Though some outcome indicators are documented at the programmatic level,^6–8 deep dives into the implementation aspects of these initiatives have not occurred, particularly in a prospective comparative fashion. ⁹ The critical issues are (a) factors affecting implementation, (b) process of implementation, and (c) results of implementation. Finally, these issues should lead us to introduce potential solutions that feed into health systems and to promote their large-scale use and sustainability.

Rationale

From the perspective of community mental health research, there is a time lag between the scientific evidence of an intervention’s effectiveness and its implementation at the community level. This involves various implementation issues. To better understand and bridge this gap at the community level, the study was designed as a hybrid randomized controlled trial combining effectiveness and implementation.

This effectiveness-implementation hy-brid randomized controlled trial aimed to compare two implementation strategies for upskilling PCDs (i.e., two training methods) to deliver primary mental health care, namely identifying and treating common psychiatric disorders presenting to their outpatient clinics. That is, to test whether the add-on Online Mental Health Training (OMHT; with extended hours of training engagement, on-the-job support, and a bottom-up approach) is better than “Training As Usual” (TAU) in carrying out the evidence-based intervention (i.e., managing commonly prevalent psychiatric disorders that present to PHC). Primary outcome variables studied were (a) Diagnostic Concordance, (b) Treatment Concordance, and (c) combined concordance (diagnosis and treatment put together). Secondary outcomes were comparison of (a) short-term clinical outcomes of patients with psychiatric disorders treated by PCDs using standardized scales, (b) implementation questionnaire covering these domains: (i) acceptability, (ii) adoption, (iii) appropriateness, (iv) feasibility, and (v) fidelity.

We have described the study’s methodology in the following paragraphs. CONSORT (Consolidated Standards of Reporting Trials) guidelines have been followed for this description (Ref: Supplementary Online Material). This research was designed as part of a larger multistate capacity-building initiative in primary mental healthcare for frontline healthcare workers. Details of this program are mentioned. ¹⁰

Methods

Ethical Considerations and Approval

The study was approved by the Institutional Ethics Committee and registered with the Clinical Trial Registry of India.

Operational Definitions

Primary Care Doctors

Doctors serving as MO at PHC, with a basic qualification of a Bachelor of Medicine, Bachelor of Surgery (MBBS) degree, with or without additional qualifications

Commonly Prevalent Psychiatric Disorders

Substance Use Disorder (SUD), comprising tobacco (nicotine) addiction and alcohol use disorder, common mental disorder (CMD) comprising depression, generalized anxiety disorder, panic disorder, somatization disorder and Severe Mental Disorder (SMD) comprising psychosis and bipolar affective disorder as per Clinical Schedules in Primary Care Psychiatry. ¹¹ These were akin to disorders due to use of nicotine (6C4A), disorders due to use of alcohol (6C40), depressive disorder (single episode (6A70) or recurrent (6A71), generalized anxiety disorder (6B00), panic disorder (6B01), bodily distress disorder (6C20), schizophrenia and other primary psychotic disorders (6A20-6A2Y), bipolar or related disorders (6A60-6A6Z) as per International Classification of Diseases 11th Revision (ICD-11). ¹²

Interventions

TAU given by the District Mental Health Program team, and an add-on OMHT by the research team (details in the manuscript).

Study Arm/Study Group (SG)

PCDs received both TAU and add-on OMHT during the study.

Control Arm/Control Group (CG)

PCDs received only TAU during the study.

Setting

This study was conducted in the Tumkur District of Karnataka, India, which is located around 70 kilometers away from the state capital, Bengaluru (Figure 1). It has the following health care infrastructure: 10 taluks, 151 PHCs. Out of 10 taluks, one taluk (Turavekere) was excluded as it has an ongoing mental health program in the community, which would lead to bias and affect the results of the study. The remaining nine taluks, comprising 140 PHCs, were included in the study (Figure 2).

Figure 1.

Geographical Location of the Study.

Figure 2.

Study and Control Group Details.

Sample Size Calculation

Assuming an intra-cluster correlation coefficient of 0.04 based on available scientific literature ¹³ and a 2:1 allocation ratio, a trial size of at least 300 patients (including the SG and CGs) visiting the PHC was established. This would have 80% power for the study to reject the null hypothesis that add-on OMHT is not more effective than TAU.

Process of Randomization and Blinding

Cluster randomization was followed for the current study. Each taluk, with all its PHCs, was considered a single cluster. The clusters were randomized using the lottery method, a simple randomization procedure. With an allocation ratio of 2:1, the nine taluks were divided into the SG (6 taluks-91 PHCs) and CG (3 taluks-49 PHCs). This ratio was selected to ensure an adequate number of participants meeting the inclusion criteria in the SG, taking into account the research team’s previous experience. Within the SG and CG clusters, a 1:1 ratio was used to evaluate and compare the primary and secondary outcomes of the study. The principal investigator and the trainers remained unblinded to the randomization process. A separate team of assessors, comprising psychiatrists, psychiatric social workers, and psychologists (two at a time), who were blinded entirely about the training and randomization process, conducted outcome evaluations at both baseline and follow-up (8 weeks after baseline). Due to time and resource constraints, follow-up assessments were conducted by telephone. Diagnosis and treatment decisions were made solely by psychiatrists.

Informed Consent

Written informed consent was obtained from all participants (SG and CG), including PCDs and patients/caregivers, who were included in this study.

Selection of PCDs in the SG and the CG

PCDs of both the SG and CG would receive TAU, provided by the District Mental Health Program (DMHP) Team. With respect to OMHT as an adjunct to TAU, following administrative approvals at the state and district levels, an online training link was shared with the PCDs of the 91 PHCs in SG. Among the 91 PCDs from SG who were provided OMHT, 13 participants fulfilled the inclusion criteria and were chosen for the SG. The intervention (training) was supplied as per the details provided below. Similarly, among the 49 PHCs in the selected taluks, 13 participants who met the inclusion criteria were randomly selected for the CG. CG participants received OMHT after the data collection was completed. Each selected participant in both SG and CG was requested to screen/diagnose/start treatment for the subsequent 25 patients visiting their respective PHCs (on the given day) and enter details in a prescribed proforma for possible mental health conditions. Each of these patients was independently evaluated by psychiatrists, blinded to the training status, for diagnosis and treatment.

Training as Usual

The 2-day onsite training conducted by the DMHP Team was considered as TAU. The objective of this annual training is to equip PCDs with the knowledge and skills needed to deliver mental health care. ¹⁴ This was conducted at the District Centers. TAU involved didactic lectures by the DMHP Psychiatrist, including role play demonstrations on commonly prevalent mental health disorders prevalent in the community, including SMDs, CMDs, SUD, and mental health issues in vulnerable populations. Each day included 6 hours of training, for a total of 12 hours. Through official communications, all district PCDs were directed to attend this training. Even PCDs (of both SG and CG) of this study had undergone TAU at the beginning of the study period.

Add-on OMHT

As both SG and CG were already exposed to TAU, OMHT could serve only as an add-on. Two trainers conducted OMHT for SG. Six main sessions (Ref: Supplementary Document-1) with a total training of 12 hours were conducted over a period of 6 weeks. However, 12 revision sessions were subsequently held over two additional months due to poor attendance by the PCDs, with only seven meeting the attendance criteria (>80%) to be eligible for the SG. Revision training lasted 6.5 hours in total, with each session averaging 30–40 minutes because participants had to attend to regular OPD duties and lacked a set training time. The total duration of training, including revision classes, was 18.5 hours, with a frequency of one to two sessions per week. Supportive handholding was done through Collaborative Video Consultations (CVCs) for 3 months. CVCs were participant-initiated for immediate assistance/clarifications regarding the patient’s diagnosis and management, and were handled in real time (i.e., as and when PCDs consult patients) by the tele psychiatrist through video consultations. Following data collection of the trial, the CG was also provided with an add-on OMHT and supportive handholding through CVCs.

Training Curriculum for Add-on OMHT

The training curriculum for the SG was based on Clinical Schedules of Primary Care Psychiatry (CSP), ¹¹ which is specifically designed for training PCDs. It is a validated schedule that provides guidelines for screening, diagnosis, first-line treatment, follow-up care, and basic counseling. It covers common psychiatric disorders, including tobacco addiction, alcohol use disorders, psychotic disorders, depressive disorders, anxiety disorders, and somatization disorder.

The details regarding TAU and add-on OMHT are provided in Table 1.

Table 1.

Details of TAU and Add-on OMHT.

Details	Training as Usual (TAU)	Add-on Online Mental Health Training (OMHT)
Mode	In-person	Online
Sessions	Four sessions (over 2 days)	Six main sessions and 12 revision sessions
Total duration	12 hours (over 2 days)	18.5 hours (over 14 weeks)
Module	Severe mental disorders, common mental disorders, substance use disorders, and mental health issues in vulnerable populations	Tobacco addiction, alcohol use disorders, psychotic disorders, depressive disorder, anxiety disorders, and somatization disorder, as per Clinical Schedules for Primary Care Psychiatry
Conducted by	DMHP team	Research Team
Participants	Primary care doctors of the study and control group before the baseline and follow-up assessments	Primary Care Doctors of study group, before baseline and follow-up assessment, and control group, after the assessment
Supportive handholding	–	Provided through Collaborative Video Consultations (CVCs)

Ministry of Health and Family Welfare, Government of India. National Mental Health Programme [Internet]. New Delhi: MoHFW; [cited 8 July 2025]. Available from: https://www.mohfw.gov.in/sites/default/files/56464578341436263710_0_0.pdf.

Inclusion criteria for Intervention group [SG; (TAU + Add-on OMHT group)]:

Participants, including PCDs, pati-ents, and caregivers, who have provided written informed consent before participating in the study.

Patients and caregivers linked to PCDs participating in the study: Individuals who are receiving mental health care services from PCDs trained by the research team through add-on OMHT.

PCDs who may or may not have undergone DMHP training (TAU).

Inclusion criteria for comparator group [CG; TAU only group]:

Participants, including PCDs, patients, and caregivers, who have provided written informed consent before participating in the study.

Patients and caregivers linked to PCDs participating in the study: Individuals who have received mental health care services from PCDs in the control PHCs.

PCDs in the CG who may or may not have undergone DMHP training.

Exclusion criteria for Intervention group [SG; (TAU + Add-on OMHT group)]:

Participants, including PCDs, pati-ents, and caregivers who refused to provide written informed consent before participating in the study.

Patients and caregivers not linked to PCDs participating in the study: - Individuals who have not received mental health care services from PHPs trained by the research team through add-on OMHT.

Exclusion criteria for comparator group [CG; TAU only group]:

Participants, including PCDs, pati-ents, and caregivers who refused to provide written informed consent before participating in the study.

Patients and caregivers not linked to PCDs participating in the study.

Data Collection

A hybrid data-collection method was used. The SG and CG were allocated randomly by the principal investigator. A group of blinded assessors visited the SG and CG PHCs to collect data. Assessors were present during PCDs’ evaluation of patients visiting the PHC. After the PCDs completed their assessments, the assessor reassessed the same patient and documented the findings (baseline). After these in-person baseline assessments, a telephonic follow-up was conducted 8 weeks later. To assess patient-reported outcome measures. Most PCDs in both SG and CG assessed up to 25 consecutive patients who consulted PHCs on the day of assessment. Some, however, received fewer patients on the day of the evaluation.

Ethical Aspects

Data collection was conducted without disrupting the routine workflow at the PHC. Written informed consent was obtained from all participants in the study. No discrimination was made in the training of PCDs or in the clinical care of patients who did not consent to the study. Following data collection, the CG was also provided with an add-on OMHT.

Outcome Measures

Primary outcomes:

Diagnostic concordance between PCD and the assessor.

Treatment concordance between PCD and the assessor.

Combined concordance (of both diagnosis and treatment put together).

Secondary outcomes:

Clinical outcomes measured through validated scales, which include:

Kessler Psychological Distress Scale (K10).

Changes in the levels of disability: Indian Disability Evaluation and Assessment Scale (IDEAS).

Improvement in work functioning and Quality of life: EuroQol.

Reduction in symptoms: Clinical Global Impression-severity scale (CGI-S) and Brief Addiction Rating Scale (BARS).

Implementation questionnaire.

Assessment Tools

Kessler Psychological Distress Scale (K10): A 10-item self-report questionnaire designed to measure non-specific psychological distress. It is widely used in population surveys and clinical settings to assess the level of emotional distress individuals have experienced over the past 30 days. The K10 demonstrates strong psychometric properties, including high reliability and validity, and is sensitive to the detection of mental health disorders such as anxiety and depression. Its brevity and ease of administration make it a valuable tool for both epidemiological research and clinical practice. ¹⁵

IDEAS was developed to assess and certify the disabilities of patients with mental health conditions. IDEAS exhibits strong internal consistency, criterion validity, and face validity. It was initially designed to certify the disability of people with mental diseases. Still, it has also been used for research and has been sensitive in identifying milder forms of disability. ¹⁶

EuroQol (EQ-5D-5L): This widely used tool measures quality of life across five dimensions: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. EQ-5D-5L health states are summarized using a 5-digit code and expressed as a single index value, which indicates the overall quality of a health state based on the preferences of the general population in a specific country or region. Additionally, it includes a Visual Analog Scale (EQ-VAS) that allows respondents to rate their perceived health on a 0–100 scale. ¹⁷

The CGI scale: This is a 3-item observer-rated scale. In this study, we focused on the CGI-S component, which assesses illness severity on a 7-point Likert scale. ¹⁸

BARS: This is a concise, multidimensional tool designed to assess the severity of addiction-related dysfunction in individuals with alcohol dependence. It evaluates 10 key areas of impairment, including substance use dyscontrol, family and occupational dysfunction, social and legal problems, financial issues, general ill-health, psychiatric and neuropsychiatric symptoms, and sexual functioning. Each domain is rated on a seven-point scale based on the number of days affected within the past month. BARS is easy to administer and reliable, with high interrater reliability (0.9), making it suitable for both clinical and research purposes. Its brief format makes it feasible for use in follow-ups and treatment monitoring. ¹⁹

In addition to IDEAS, EQ-5D-5L, and K10, we considered two specific scales to assess the secondary outcomes for patients identified with SMDs and Alcohol Use Disorders. CGI-S was used for patients with SMD, and BARS was used for patients with alcohol use disorder.

Implementation Variables

The Implementational component of this hybrid trial considered the following domains as outcome variables. ²⁰

Acceptability

The perceived agreeability of PCDs on integrating mental healthcare into primary care and their willingness to engage with it. It refers to how agreeable and comfortable PCDs are to diagnose and treat uncomplicated psychiatric conditions, as well as to initiate first-line treatment (after undergoing training).

Adoption

The intention or initial decision to implement the task of integrating mental healthcare into primary care, as well as the actual action of doing so. It examines whether (shortly after the training) PCDs made an initial decision and attempted to identify and treat individuals with uncomplicated psychiatric disorders in the PHCs.

Appropriateness

The perceived relevance of the task of integrating mental healthcare into primary care within PHCs or for patients. It relates to whether the identification and treatment of uncomplicated psychiatric disorders are considered by PCDs to be both appropriate and relevant for the patients who routinely visit PHCs.

Feasibility

The perceived practicality of implementing mental healthcare delivery within PHCs. It addresses how feasible it is for the PCDs to carry out the task of identifying and treating uncomplicated psychiatric disorders within PHCs.

Fidelity

The degree to which mental healthcare is integrated into primary care, as initially recommended or as designed and planned. This includes aspects of adherence, integrity, and the quality of mental health delivery. It pertains to the extent to which PCDs have been able to identify and treat uncomplicated psychiatric conditions as part of their routine practice within PHCs. In the current context, the subjective aspect of fidelity is considered, as no single objective measure has been identified, and the study was not designed to incorporate such an approach.

This data was collected using an implementation questionnaire (Supplementary Document 2) comprising 12 questions.

In addition, answers to key questions assessing research designs or implementation research reports have been provided in Supplementary Document 3.

Data Handling

The research team handled data storage and ensured data privacy and protection. All the stored data was filed in a locked cabinet in a secure office (paper forms), and encrypted files were located on a secure, non-public, password-protected computer (digital forms) in a locked room. Paper forms were digitized to facilitate data analysis. All data collected at the participating PHCs were obtained without hampering patients’ clinical care. Data analysis was carried out under the supervision of the principal investigator and the biostatistician.

Statistical Analysis

A Licensed Version of IBM SPSS Statistics 29.0 was used for Statistical Analysis of the data. ²¹ Relevant statistical tests were used based on the analysis of data for primary and secondary outcomes.

Expected Results

The CONSORT Diagram regarding the flow of the trial is shown in Figure 3. The TAU for the SG and CG was conducted in January 2024. The add-on OMHT for the SG was conducted between January 2024 and May 2024. The baseline assessments were conducted in May-June 2024, and the follow-up assessments in July-August 2024. A total of 578 patients, equally divided between the SG and CGs, were assessed by the assessor team across 26 PHCs (13 PHCs in the SG and 13 PHCs in the CG). Among these 578 patients evaluated for primary outcomes, 235 were identified as having a psychiatric diagnosis. These 235 patients were approached for follow-up evaluation. However, 95 were lost to follow-up. Follow-up assessments were completed for 140 patients (61 in the SG and 79 in the CG). The expected results on the primary and secondary outcomes of the trial are beyond the scope of the current article and are described elsewhere.^22,23

Figure 3.

*The PHCs were excluded based on the exclusion criteria (details mentioned under the heading-exclusion criteria).

Discussion

This manuscript describes the design and methodology of an effectiveness-implementation hybrid cluster randomized controlled trial (Type II), conducted to assess the impact of an add-on OMHT for PCDs on their ability to manage common psychiatric disorders in primary care settings. The hybrid design enables simultaneous evaluation of effectiveness and key implementation outcomes, providing a comprehensive understanding of the clinical and contextual determinants of capacity-building initiative success from a public mental health perspective.

Through technological developments, innovative methods have been utilized for training and capacity-building activities. However, prospective hybrid trials to evaluate both effectiveness and implementational aspects of such innovative training methods in the Indian context are limited, to the best of the authors’ knowledge. As this trial is designed for a real-time study, we believe it provides insights not only into improved understanding and skills among PCDs, but also into improved clinical outcomes through effective primary mental health care delivery, and into implementation aspects for scalability and sustainability.

This mode of add-on OMHT, if demonstrated to be effective, can be generalized and widely merged into the public health care system to aid in the integration of mental health care into general health care, as advocated by the National Mental Health Program. Digital Academies can be established in every state and UT to continue supporting the primary healthcare workforce.

Expected Limitations

This trial was conducted in one of the districts in Karnataka. As multiple administrative and implementation aspects are instrumental to successful training delivery, skill transfer, and skill reflection in clinical practice, there is a need for more such studies across different districts and states to enhance the generalizability of results. The authors also acknowledge that other cadres of frontline healthcare workers involved in the delivery of primary mental healthcare include ASHAs and CHOs. A similar trial can be conducted for ASHAs and CHOs, which helps in better understanding implementation and effectiveness aspects for the integration of mental health into primary care.

Although all the PCDs in the SG clusters were considered, the training completion rate was low. The baseline assessments were in-person, but the follow-up assessments were conducted telephonically. The authors would also like to note that allocation concealment is challenging to achieve in community health cluster RCTs. The risk of unblinding for PCDs and assessors would always be present, which is a limitation. Because of the smaller sample sizes for primary and secondary outcomes, a larger sample would be necessary to improve the generalizability of the results.

Conclusions

This trial highlights the potential utility of online training for enhancing psychiatric skills among primary care doctors. Furthermore, the study provides valuable insights into the specific barriers and facilitators involved in implementing capacity-building programs. Ultimately, these findings are crucial for understanding how digital training initiatives can be effectively translated into clinical practice to improve primary mental healthcare

Supplementary Material

Supplementary material for this article is available online.

Supplementary Material

Supplementary material for this article is available online.

Supplementary Material

Supplementary material for this article is available online.

Supplementary Material

Supplementary material for this article is available online.