Effectiveness of a stepped-care programme of internet-based psychological interventions for healthcare workers with psychological distress: Study protocol for the RESPOND healthcare workers randomised controlled trial

Abstract

Background and aims

The coronavirus disease 2019 pandemic has challenged health services worldwide, with a worsening of healthcare workers’ mental health within initial pandemic hotspots. In early 2022, the Omicron variant is spreading rapidly around the world. This study explores the effectiveness and cost-effectiveness of a stepped-care programme of scalable, internet-based psychological interventions for distressed health workers on self-reported anxiety and depression symptoms.

Methods

We present the study protocol for a multicentre (two sites), parallel-group (1:1 allocation ratio), analyst-blinded, superiority, randomised controlled trial. Healthcare workers with psychological distress will be allocated either to care as usual only or to care as usual plus a stepped-care programme that includes two scalable psychological interventions developed by the World Health Organization: A guided self-help stress management guide (Doing What Matters in Times of Stress) and a five-session cognitive behavioural intervention (Problem Management Plus). All participants will receive a single-session emotional support intervention, namely psychological first aid. We will include 212 participants. An intention-to-treat analysis using linear mixed models will be conducted to explore the programme's effect on anxiety and depression symptoms, as measured by the Patient Health Questionnaire – Anxiety and Depression Scale summary score at 21 weeks from baseline. Secondary outcomes include post-traumatic stress disorder symptoms, resilience, quality of life, cost impact and cost-effectiveness.

Conclusions

This study is the first randomised trial that combines two World Health Organization psychological interventions tailored for health workers into one stepped-care programme. Results will inform occupational and mental health prevention, treatment, and recovery strategies.

Registration details

ClinicalTrials.gov Identifier: NCT04980326.

Keywords

MeSH terms coronavirus disease 2019 anxiety depression adjustment disorders psychological distress resilience psychological psychosocial intervention internet-based intervention healthcare facilities workforce and services analysis cost

The coronavirus disease 2019 (COVID-19) pandemic has challenged healthcare systems worldwide. Healthcare workers (HCWs) in some of the early pandemic hotspots, such as Spain, experienced major restructuring at work. For instance, they were deployed from their usual functions, sometimes to COVID-19-specific activities, working long shifts, with limited access to adequate protective equipment, as well as being forced to make decisions on patient prioritisation without proper guidelines (1 –3). At the same time, increasing levels of discrimination and violence were being reported (4 –6), which resulted in transnational organisations such as the World Health Organization (WHO) or the International Committee of the Red Cross (ICRC) calling for protection for HCWs. Many cross-sectional studies showed that both poor working conditions and self-perceived stigma were associated with poor self-reported anxiety and depression symptoms, sleep problems, post-traumatic stress disorder (PTSD) symptoms or suicidal ideation (7 –11) – mental health problems already reported by HCWs before the COVID-19 pandemic (12).

During the first half of 2020, HCWs in Spain were massively exposed to potentially traumatic stressors, while lockdown measures restricted regular social activities and hampered access to existing mental health services, including psychotherapy. By late 2020, the prevalence of major depressive disorder, generalised anxiety disorder (GAD) and PTSD amongst Spanish HCWs was high (24%, 19,4%, and 21%, respectively (13). As the pandemic drags on into 2022, countries worldwide are imposing once again severe restrictive measures – including lockdowns – to contain the spread of the Omicron variant of the virus – Spain shows the highest incidence rates since the pandemic began, and primary care services are completely overburdened. Since poor mental health outcomes seem to persist over time among HCWs (14,15), evidence-based mental health interventions need to be culturally and locally adapted to COVID-19's rapidly changing environment. However, the adaptation process presents several challenges. First, mental health programmes must be feasible. They should consider the economic impact of the COVID-19 pandemic, which requires intervention programmes that are not only efficacious but also cost-effective. In this regard, internet-based psychological interventions can help overcome these issues by reducing costs (16), and increasing access to mental health care services, besides allowing HCWs and mental health service providers to comply with social distancing measures (17). Second, the prevalence of mental health problems and the persistence of the COVID-19 pandemic require that programmes be scaled-up at early stages to rapidly reach as many HCWs in need as possible and at a lower cost. The implementation of stepped-care programmes, which are based on principles such as doing more with less (i.e., providing care to more people with less amount of effort devoted to each one) (18) or what works for whom (19) has been shown to be cost-effective for common mental health problems. These programmes are becoming increasingly popular in mental health (20 –22) and offer help that gains intensity only if the participant does not reach a particular milestone – e.g., if they do not lose weight in a weight-loss intervention (23). Finally, intervention protocols should be standardised to prove their effect across different settings, such as general hospitals and primary healthcare settings.

Taking these factors into account, we designed a stepped-care programme including two scalable WHO internet-based psychological interventions that were locally adapted and tailored for HCWs in Spain. In the adaptation process, we included HCWs from a wide variety of care facilities and used standardised protocols for training care providers and implementing the interventions to improve transferability across settings. We chose two interventions developed by the WHO for communities affected by adversity (24). These brief, evidence-based interventions include self-help materials and guided self-help programmes that can be easily adapted to different contexts; they are affordable and can be delivered online. The first step consists of access to a guided self-help intervention delivered through a mobile-supported website adapted from a stress management guide called ‘Doing What Matters in Times of Stress’ (DWM), which is part of WHO's evidence-based self help plus (SH+) stress management course (25). DWM uses a model adapted from another scalable WHO intervention (i.e., Step-by-Step, a guided self-help online intervention) which has shown to be effective in treating depression in communities exposed to adversity in Lebanon (26). The second step is problem management plus (PM+) (27), an intervention based on cognitive behavioural therapy (CBT) techniques delivered individually through video calls and offered only to participants who show no reduction in psychological distress after step 1. Both interventions have proved effective in humanitarian settings (28 –30). For instance, SH+ has been implemented as a preventive intervention for asylum seekers and refugees with psychological distress resettled in Europe and Turkey (31,32). It has also been used to reduce psychological distress in South Sudanese female refugees in Uganda (28). Other studies have shown the effectiveness of PM+ in reducing depression and anxiety in communities affected by violence in Kenya (33) and Pakistan (29). In addition, PM+ has been used to help reduce symptoms of anxiety and depression in cancer patients (30).

Although these scalable interventions have been adapted to the COVID-19 pandemic in previous studies for long-term care workers and distressed people (34,35), to our knowledge, this is the first time they have been integrated into an online stepped-care programme for HCWs. This parallel-group clinical trial explores this programme's effect on self-reported anxiety and depression symptoms among HCWs with psychological distress based on the hypothesis that reductions will be larger in the experimental arm than in the control arm (care as usual [CAU]).

Methods and analysis

Study design and participants

This study is a multi-centre (two sites), parallel-group (1:1 allocation ratio), analyst-blinded, superiority, randomised (stratified by centre), controlled (versus CAU) trial that explores the effect of a stepped-care psychological intervention on anxiety and depression symptoms among HCWs with psychological distress at 21 weeks from the baseline assessment. Our main aim is to test the effectiveness of the stepped-care intervention on anxiety and depressive symptoms, based on the hypothesis that the improvement will be larger among participants in the intervention arm compared to CAU. Our secondary aims are to test the effectiveness of the intervention on PTSD symptoms, quality of life, and resilience, based on the hypothesis that the improvement will be larger among the participants in the intervention arm compared to CAU. We prospectively published the trial protocol on ClinicalTrials.gov on 28 July 2021. The record log does not show any significant modification after the first participant entered the study on 1 November 2021. The study is part of an European Union (EU)-funded project named ‘Improving the Preparedness of Health Systems to Reduce Mental health and Psychosocial Concerns resulting from the COVID-19 Pandemic’ (RESPOND) (www.respond-project.eu), and it is sponsored and coordinated by the Hospital La Paz Institute for Health Research (Instituto de Investigación del Hospital Universitario La Paz).

RESPOND is conducting trials focused on different population groups, including our trial on HCWs. RESPOND-HCWs is done in Spain, where 17 Autonomous Communities are responsible for healthcare provision and policy. Participants will be recruited from the Community of Madrid and Catalonia. In the Community of Madrid, with a registered population of 6,745,591 as of January 2021, eligible participants are HCWs employed by the Department of Health (88,717 workers as of October 2021). In Catalonia, with a registered population of 7,716,760 as of January 2021, eligible participants are HCWs funded by the Department of Health (109,346 workers as of December 2020). On the day the first participant was enrolled in the study (1 November 2021), 364 and 517 confirmed COVID-19 cases were reported by the Community of Madrid and Catalonia, respectively.

The research team will contact all participants interested in the study by phone. After confirming their interest and signing the informed consent form (see Supplemental File 1), the assessor will conduct a brief interview to explore whether they can be enrolled. We interview participants for approximately 15 min and ask a series of pre-specified screening questions (e.g., have you got any acute medical conditions? Have you ever been diagnosed with a mental disorder?) and items (e.g., Does the person understand the questions? Does the person find it hard to follow the interview?) to check whether they meet inclusion and exclusion criteria.

We set the following inclusion criteria:

HCW from primary, specialised, or emergency care facilities, including doctors, psychologists, nurses, nursing technicians, orderly, and administrative staff.

Psychologically distressed, as measured by the Kessler Psychological Distress Scale (K10) above cut-off score of 15.9 (36).

18 or older.

Able to read and speak Spanish, Catalan, or both.

We also set the following exclusion criteria:

Acute medical conditions that require immediate hospitalisation.

Imminent risk of suicide or self-harm or risk of harming others.

Severe mental disorder (e.g., psychotic disorder, delirium).

Severe cognitive impairment (e.g., intellectual disability, dementia).

Initiated, stopped, or significantly modified pharmacotherapy in the last eight weeks.

Initiated or stopped standardised psychological treatment (e.g., CBT, psychoanalytic therapy) in the last eight weeks.

We did not specify any study withdrawal criterion.

The scalable interventions used in this trial can be delivered by non-professional helpers, such as a trained peer, a workplace helper, or a psychosocial worker. These interventions have also been designed to be widely applicable to various mental health problems, such as anxiety and depression, and are easily adaptable to different populations, cultures, and languages. Care providers in RESPOND-HCWs are junior psychiatrists, psychologists, and mental health nurses in training (e.g., residents) who have undergone specific preparation shortly before the trial (6 days of training in Psychological First Aid [PFA] and online delivery of DWM and PM+). Training includes presentations, group activities, active discussion, case studies and role-plays. Care providers will also attend weekly supervision sessions during the trial, consisting of 60-min online group sessions with trained supervisors. In these sessions, care providers can ask for guidance for specific participants or enquire more generally about the intervention protocol. The trainers/supervisors are psychiatrists and clinical psychologists who have received 9 days of master training from senior mental health professionals who were involved in the development of the intervention programmes or were trained directly by intervention developers. Supervisors will also ensure protocol adherence and carry out informal weekly competency and fidelity checks, which will inform individualised feedback for care providers. Supervisors will also conduct a formal fidelity check at the end of the trial, which will consist of structured checklists to be completed based on audios of the intervention and video recordings. Local project managers will supervise trainers/supervisors, providing training in supervision skills.

Interventions

We use intervention programmes developed by the WHO. Following the Programme Design, Implementation, Monitoring, and Evaluation (DIME) protocol (37), we used a two-step qualitative research design to interview frontline HCWs, mental health experts, administrators, and service planners in Spain, and we analysed their responses to locally adapt and tailor the interventions for HCWs in Spain, following similar studies (38,39). Participants in the intervention arm are offered a stepped-care programme consisting of two scalable psychological interventions: DWM in Times of Stress (DWM) – part of the Self-Help Plus (SH+) course, and PM+. All participants are also offered a short counselling session, namely PFA, and can maintain their usual care, which might include non-structured psychological support or stabilised psychopharmacological treatment.

We did not establish any intervention withdrawal criterion, but care providers will report adverse events to the principal investigator, who will decide whether a participant must discontinue the intervention.

PFA. WHO defines PFA as a ‘humane, supportive and practical help to fellow human beings suffering serious crisis events’ (40). PFA providers are trained in three basic helping skills: looking, listening, and linking. RESPOND participants are HCWs that have not necessarily been exposed to serious adversities but have been exposed to a stressful and potentially traumatic event such as the COVID-19 pandemic. We include PFA in a single 15-min phone session conducted 2–5 days after enrolment. Before starting PFA, the helper tells participants whether they have been allocated to the intervention arm or the control arm (see Figure 1) and answer any questions regarding the intervention process, including the allocation. Next, the helper informs participants about the aim of the call and its duration, explaining that they have 15 min to talk about anything they need. The helper will then support the participant with basic skills and strategies and offer specific resources that might be helpful (e.g., hotlines for people in distress or experiencing loneliness, support for women who might be suffering gender-based violence, etc.). If participants do not answer the phone, helpers will try to contact them twice, after which they will label the intervention as ‘not provided’.

Figure 1.

Participants’ flow diagram since they show interest in the study until they complete the follow-up assessment. DWM: Doing What Matters; K10: Kessler Psychological Distress Scale; PFA: Psychological First Aid; PM+: Problem Management Plus.

Stepped-care programme. We designed a two-step programme for the RESPOND trial comprising two scalable psychological interventions. Firstly, a guided stress-management course based on the SH+booklet called Doing What Matters in Times of Stress (DWM) (25) and, secondly, PM+, an individual intervention based on CBT (27). The criterion for stepping up is reporting significant levels of psychological distress after step 1, as measured by a K10 score higher than 15.9. We will offer PM+ to participants who do not reach that milestone.

Step #1 is DWM, a booklet divided into five monographic chapters covering psychoeducation on stress and its causes, as well as five strategies from acceptance and commitment therapy (ACT) for managing stress. It was designed to support learning during SH+, a 5-week group-based course but is available for use as a standalone stress management guide. The chapters contain the same techniques and skills provided in the longer SH+ course. Chapters include information on the ACT techniques, along with audio recordings to support practice. As a result of the local adaptation process undertaken in RESPOND, we transformed DWM into a mobile-friendly website, re-recorded audios and adapted some content to reflect barriers or stress triggers that might affect HCWs in Spain. This included adding additional exercises to help support motivation to use the guide.

DWM is provided as guided self-help. It uses a model adapted from another WHO intervention Step-by-Step, a guided self-help intervention for depression, which was tested in randomised controlled trials in Lebanon (41). After allocation, DWM users are assigned to a helper who offers ongoing support with practices and key concepts over the phone. An initial call is arranged 2–5 days after entering the study. After that call, the participant receives a message with login details. The course is spread over 5 weeks, and new modules are released every week. Helpers also schedule weekly ongoing support calls. Participants who do not want to receive phone calls can also contact their helpers using the messaging system included on the website. We keep track of every helper-participant contact made by phone or DWM website.

Step #2 is PM+, a brief psychological intervention based on CBT techniques. Helpers or facilitators schedule five weekly interventions covering each strategy. As a result of the local adaptation process, we adapted PM+ to be delivered online (videoconference) and shortened sessions from 90 to 60 min. We also tailored case examples to HCWs (e.g., job-related triggers of stress, barriers for practice because of working shifts, etc.). Helpers will record calls for adherence purposes and go through identified barriers during practice over the week.

We present an overview of the stepped-care programme in Figure 2.

Figure 2.

Overview of the RESPOND stepped-care programme.

Outcomes

Participants self-report all outcomes at baseline (t₁) and the three endpoint assessments (t₂, t₃, and t₄). We selected three relevant mental health outcomes for HCWs from COVID-19 pandemic hotspots, namely anxiety, depression, and PTSD symptoms (7,8,14). We also included quality of life and resilience as secondary outcomes and the cost of implementing the intervention programme to determine cost-effectiveness. We did not include any harm outcomes, although we will report harm indicators, such as an increase in symptoms or suicidal thoughts, as well as adverse events (see Monitoring for a detailed description of adverse event monitoring). The primary outcome is an aggregated measure of anxiety and depression symptoms at t₄, that is, 21 weeks after baseline assessment or 2 months after PM+. We are unaware of any validation studies conducted among HCWs using our instruments. In a previous study, we used the PHQ-9 in a large sample of HCWs, and Cronbach's alpha was 0.88 (95%CI: 0.87, 0.89) (8).

Self-reported anxiety and depression symptoms, as measured by the Patient Health Questionnaire – Anxiety and Depression Scale (PHQ-ADS) summary score at t₄ (primary outcome). The PHQ-ADS (42) is a 16-item self-reported instrument that combines the nine-item PHQ depression scale (PHQ-9) (43) and seven-item GAD scale (44) into a composite measure of depression and anxiety. Respondents are asked how much each symptom has bothered them over the past 2 weeks, with response options of ‘not at all’, ‘several days’, ‘more than half the days’, and ‘nearly every day’, scored as 0, 1, 2, and 3. The scale can range from 0 to 48, with higher scores indicating higher levels of depression and anxiety symptoms. Spanish versions of both the PHQ-9 (45) and the GAD-7 (46) are validated and will be combined into the PHQ-ADS.

Self-reported anxiety and depression symptoms, as measured by the PHQ-ADS summary score at t₂ and t₃ (secondary outcome). The PHQ-ADS summary score will also be collected as a secondary outcome at 7 and 13 weeks from the baseline assessment.

Self-reported anxiety and depression symptoms, as measured by the PHQ-ADS anxiety and depression domain scores, at t₂, t₃, and t₄ (secondary outcome). PHQ-ADS domain scores can range from 0 to 27 and from 0 to 21 for the depression (i.e., PHQ-9) and anxiety (i.e., GAD-7) domains, respectively, with higher scores indicating higher levels of anxiety and depression. The Spanish version of both instruments includes a cut-off score of ≥ 10 to detect people with probable depression (47,48) and anxiety (46), and these cut-offs have been used in large samples of Spanish HCWs after the COVID-19 outbreak (7,8).

Self-reported symptoms of PTSD, as measured by the 8-item version of the PTSD Checklist for DSM-5 (PCL-5) summary score, at t₂, t₃, and t₄ (secondary outcome). The 8-item PCL-5 (49) is a self-reported instrument that measures PTSD symptoms according to DSM-5 criteria. Respondents are asked how much each symptom has bothered them over the past 4 weeks, with response options of ‘not at all’, ‘a little bit’, ‘moderately’, ‘quite a bit’, and ‘extremely’. Items are rated on a 0–4 scale. The scale can range from 0 to 32 for the 8-item version, with higher scores indicating higher levels of PTSD symptoms. The instrument is based on the PCL-C, a DSM-IV-based checklist validated in the Spanish language (50).

Self-reported health-related quality of life, as measured by the EuroQol 5-dimensional descriptive system – 5-level version (EQ-5D-5L) domains at t₂, t₃, and t₄ (secondary outcome). The EQ-5D-5L (51) consists of the EQ-5D and the EQ-VAS. Part 1, the EQ-5D, rates the level of impairment across five dimensions: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. Each dimension has five levels: none, slight, moderate, severe, and extreme problems. The labels for the 5L followed the format ‘no problems’, ‘slight problems’, ‘moderate problems’, ‘severe problems’, and ‘unable to’/‘extreme problems’ for all dimensions. Part 2, the EQ-VAS, is a visual analogue scale. The endpoints of the scale are called ‘The best health you can imagine’ and ‘The worst health you can imagine’, and the current health status of that day needs to be indicated, after which the number checked on the scale also needs to be written down. Higher scores indicate poor quality of life. A Spanish version with population-based reference norms is available (52).

Cost of programme implementation, as measured by the domain scores of the Client Service Receipt Inventory (CSRI) – RESPOND adaptation, at t₂, t₃, and t₄ (secondary outcome). A bespoke version of the CSRI (53) has been developed to collate information on changes in health care service utilisation and changes in usual activities that will inform cost-effectiveness analysis. The RESPOND-bespoke version consists of a 13-item self-reported instrument that asks about the number and duration of contacts with healthcare professionals (physicians, mental health specialists, and nurses) in the past 2 months. It collects data on service utilisation (e.g., use of health system, other services, time out of employment and other usual activities, need for informal care) and related characteristics of people with mental disorders. In addition to collecting data using the CSRI, the trial separately collates information on the resources and costs of implementing the intervention, including resources required for initial and ongoing training/supervision.

Resilience, as measured by the PHQ-ADS summary score in relation to stressor exposure, at t₂, t₃, and t₄ (secondary outcome). Resilience can be defined as having good mental health when facing adversity, which requires collecting information on mental health status and exposure to stressors (54). The PHQ-ADS measures mental health status, while the RESPOND adapted version of the Mainz Inventory of Microstressors (MIMIS) measures the exposure to objective microstressors or daily hassles (55). After the COVID-19 outbreak, a shorter version, including pandemic-related stressors, was developed (56). In RESPOND, we use an 18-item adaptation that includes: three general life events (e.g., recent break-up); six everyday stressors (e.g., excessive workload, financial problems); five COVID-19-specific stressors (e.g., being forced to quarantine); and four HCW-specific stressors (e.g., COVID-19 patients died under your care). The first three items (general life events) are rated on a 5-point Likert scale, ranging from 0 (never happened) to 4 (it had a major impact on me). The remaining 15 items are rated on a 4-point Likert scale, ranging from 0 (did not happen/almost never) to 3 (every day or nearly every day). All items ask about the last 14 days.

Other measures. We will conduct in-depth interviews with key informants to assess the feasibility of programme implementation (e.g., adherence, penetration, acceptability) and to conduct a process evaluation following widely accepted procedures (57,58). We will select informants among completers and non-completers of DWM and PM+ interventions. We will also use the Positive Appraisal Style Scale, content-focused (PASSc; in preparation), a 12-item self-report measure, to assess typical appraisal of stressors. Respondents are asked to rate the frequency of each item with the options ‘never’, ‘sometimes’, ‘often’, ‘almost always’, scored as 1, 2, 3, 4.

Participant timeline. After being included in the study, participants will complete baseline assessment within the subsequent 5 days (i.e., before treatment allocation so this will not bias self-reported baseline outcomes). Follow-up assessments are scheduled at weeks 7 (t₂), 13 (t₃), and 21 (t₄), from baseline assessment, with a 14-day window period (e.g., endpoint t₁ may take place 7 or even 8 weeks after baseline). Questionnaires are always presented in the same order, namely (a) sociodemographic characteristics (baseline only), (b) PHQ-ADS (PHQ-9 and GAD-7), (c) stressor exposure (RESPOND-adapted MIMIS), (d) PCL-5, (e) RESPOND-bespoke CSRI, and (f) EQ-5D-5L (7) PASSc. Figure 3 shows an overview of the participants’ timeline.

Figure 3.

Participants’ timeline. PFA: Psychological First Aid; DWM: Doing What Matters; PM+: Problem Management Plus; PHQ-ADS: Patient Health Questionnaire – Anxiety and Depression Scale; PCL-5: PTSD Checklist for DSM-5 (PCL-5) ; EQ-5D-5L: the EuroQol 5-dimensional descriptive system – 5-level version; CSRI: Client Service Receipt Inventory; PASSc: Positive Appraisal Style Scale – content focused.

Sample size

We estimated sample size to detect a small-to-moderate effect size (defined as the square root of the ratio of the variance of the tested effect to the comparison error variance, Cohen's d = 0.3) on the PHQ-ADS summary score at t₄ based on previous studies using PM+ (33,59) and on recent studies using similar online mental health interventions during the COVID-19 pandemic (34,35). A power calculation for an analysis of variance (ANOVA) repeated measurement design with two time periods to identify the effect of treatment at the last endpoint with a two-sided 5% significance level, a power of 95%, and an estimated attrition of 30%, a sample size of 106 participants per group is required (n = 212), for which we anticipate a 12-month inclusion period.

Recruitment

The trial target population includes HCWs from the Community of Madrid and Catalonia. We will recruit participants using word-of-mouth strategies, primarily via social media (e.g., WhatsApp groups, Facebook, LinkedIn posts, etc.). We will contact key stakeholders, including hospital managers and communication departments, scientific societies, labour unions, and other associations. They will be asked to announce the research project through formal (i.e., emails, interviews on press and radio) and informal methods (i.e., Whatsapp messages, SMSs). We expect potential participants to enter the study in ‘waves’, e.g., the days following a certain event in which the trial is advertised. We will enrol new participants as they approach us, taking into account helpers’ availability.

Assignment of interventions

A research assistant generated the allocation sequence using the electronic data capture (EDC) software Castor (www.castoredc.com). We stratified randomisation by ‘centre’ with a 1:1 allocation ratio using random blocks of unequal sizes. Local project managers will enrol participants (i.e., discuss the trial, assess eligibility, and obtain informed consent) and assign them to each arm based on the allocation sequence. However, they will not be aware of the randomisation sequence nor administer any intervention. The Principal Investigator will restrict the access of the data analyst (i.e., the statistician) to the electronic dataset to ensure blindness after the intervention assignment. The study does not include any outcome assessor who could be kept blinded because there are no observer-reported outcomes. As it often happens with behavioural interventions, neither participants nor care providers (helpers) are blinded to allocation.

Data collection

We collect baseline and outcome data exclusively through electronic case report forms (eCRFs) using Castor EDC and Qualtrics. Participants receive an email with a link to the baseline assessment right after enrolling in the study. We schedule the remaining follow-up assessments at that moment, and they are automatically sent on due time. Participants can complete the assessments using any electronic device. We placed the primary outcome (PHQ-ADS) at the beginning of the form, except for baseline assessments, where we collect sociodemographic variables first. The PHQ-ADS is the only mandatory outcome at all time points, meaning that participants are not allowed to go further on the questionnaire until they have filled in all the items. Results from a pilot testing conducted in both study locations estimate the average completion time of the assessments in 10–15 min. Data collection forms are available from the corresponding author upon request.

We use reminders to maximise responsiveness even among non-adherent participants who discontinue or deviate from the assigned intervention protocol. Three reminders are sent 2, 5 and 10 days after each assessment to those participants without a complete primary outcome assessment. These messages acknowledge the effort made by the participants and emphasise the importance of data collection in clinical trials to increase retention rates. Reminders were locally adapted and translated into Catalan. Participants will not be reimbursed for participating in the trial or performing the assessments.

The same eCRF was used in both study locations.

Data management

Four types of data are generated in this study: participants’ outcomes, participants’ contact details, DWM metadata, and PM+ recordings. Local teams can only access data generated on their sites at both study locations. All servers comply with the General Data Protection Regulation of the European Union (EU).

Outcomes. Participants enter their outcome data in the eCRF. Data is stored on Castor EDC and Qualtrics servers and is accessible only for local project managers. All variables have a restricted range of valid values, and field-text variables are only used for specifying the option ‘Other’ to minimise errors.

Contact details. Local project managers note down the contact details of all study participants during the screening call. These details include name and surname, phone number, postal address, and email address. These data are stored in local servers at the Universidad Autónoma de Madrid (Madrid) or the Parc Sanitari Sant Joan de Déu (Barcelona). Local helpers access these data to retrieve participants’ phone numbers and manage risks of harm when needed (e.g., knowing the postal address is important if a serious adverse event occurs during a phone call).

Metadata. The DWM website automatically generates and stores metadata on EU servers. These metadata include dates and times of logins and logouts, whether the participants have clicked on a specific audio recording, or whether they have completed a module. This information is available both for helpers (to inform DWM of ongoing support calls) and local project managers (to monitor the app's functioning). We will report some of this data as indicators of feasibility (i.e., adherence) to the DWM intervention.

Recordings. In Madrid, PM+ sessions are video recorded if the participant gives verbal consent at the beginning of the session. Only helpers and trainers/supervisors have access to these recordings. Helpers use corporate accounts of the Department of Health, where recordings are securely stored. In Barcelona, only the audio of the helper (not the participant) is recorded with an external recorder. These recordings are securely stored in servers at Parc Sanitari Sant Joan de Déu, which only helpers/supervisors can access.

Data analysis

We will answer the main research question based on the intention-to-treat (ITT) analysis of the primary endpoint: self-reported anxiety and depression symptoms as measured by the PHQ-ADS summary score measured at t₄. Firstly, we will look for baseline differences between the two groups of participants, using appropriate statistical tests based on variables’ types and distributions. Secondly, we will estimate the treatment effect at t₂, t₃, and t₄. We will use a linear mixed model, where treatment (i.e., group) will be entered as a fixed effect and participant (i.e., subject) as a random effect while controlling for the PHQ-ADS summary score measured at baseline (t₁). The model will constrain the treatment fixed effect to be null and look for the time by treatment interaction at all endpoints, although t₄ will be our primary time point of interest. We will analyse the stepped-care programme as a single intervention (regardless of whether the participant steps up to PM+ or finishes the intervention after DWM), and we will report the proportion of participants at each step, following previous studies (23). Finally, we will report the treatment effect estimators, i.e., the model parameters and the mean difference between treatment arms at each time point plus 95% confidence intervals obtained from robust standard errors. Additional ITT analyses will use the same model on secondary outcomes, namely PHQ-ADS domain scores (PHQ-9 and GAD-7 summary scores, respectively), PCL-5 summary score, and EQ-5D-5L summary and domain scores, as well as on outcome-based resilience scores, as measured by the PHQ-ADS summary score against a stressor reactivity score. These resilience scores will be calculated based on the stressor exposure and the PHQ-ADS to assess individual deviation from the normative stressor reactivity (see (60,61)). Stressor reactivity will be computed based relationship between stressor exposure and mental health problems within the sample.

We will also conduct per-protocol analyses on all outcomes as confirmatory robustness analyses. We will include participants with at least 3 DWM contacts (phone calls or messages) and, if applicable, 5 PM+ sessions. Other additional analyses include exploratory sensitivity analyses clustering participants based on relevant variables (e.g., gender, symptom severity, involvement in the treatment of COVID-19 patients) to explore the effect of the intervention across strata of interest and mediation analyses using appraisal style as measured by the PASSc or treatment adherence proxies as potential mediators.

We will also conduct health economic analyses to determine the cost-effectiveness of stepped care. The total costs of delivering interventions will be estimated and described and combined with data on changes in health service utilisation and time out of usual activity over 21 weeks (from t₁ to t₄) obtained using our bespoke CSRI. The economic analysis will focus on incremental cost per quality adjusted life year gained (using data from the EQ-5D-5L) as well as the incremental cost per change in PHQ-ADS summary score, both at 20 weeks follow up. The analysis will be conducted from both the health care system and societal perspectives. Between-group comparison of mean costs will be completed using appropriate statistical tests depending on the type and distribution of data. Univariate sensitivity analyses and non-parametric bootstrapping will be used to account for uncertainty in trial parameters; cost-effectiveness planes and cost-effectiveness acceptability curves will be constructed.

We will not impute missing data because linear mixed models use all available information to calculate effect estimators. We will report significant deviations from the assumption that data are missing at random or completely at random (e.g., sensitivity analysis including strong predictors of missingness as covariates). To deal with problems associated with multiple testing, at each time point, the global statistical significance of the secondary outcomes will be assessed through the seemingly unrelated regressions equations model, controlling for baseline values. All analyses will be done using R Studio (62) and Stata (63).

Monitoring

Local project managers independent from the study sponsor will act as data monitors. They will access data forms every day and oversee that data is being collected and that no adverse events are reported. Participants who register any serious death thoughts or plans to end their lives as part of follow-up assessments (see ‘Outcomes’ section) will receive an automatic warning message. That alert reminds them that the research team cannot monitor real-time responses and that they should seek help if they are at imminent risk. One of the trainers/supervisors will reach out to them over the next 48 h to follow up and offer support if necessary. Helpers can also detect adverse events while delivering the interventions. Suppose there is an immediate risk of harm. In that case, helpers will contact one of the trainers/supervisors to evaluate the situation and make rapid decisions (e.g., ask the person to go to the nearest emergency department or send an ambulance if required). If there is no imminent risk, helpers will handle the situation and report to the trainers/supervisors after finishing the intervention. The helpers on Castor EDC will record adverse events. Local project teams will report to the RESPOND Ethics and Data Advisory Board, chaired by Dr Sonja Rutten, which will act as Data Monitoring Committee. We do not plan any interim analyses or any external trial audit.

Ethics and dissemination

The study was approved by institutional review boards (IRBs) at Hospital La Paz in Madrid (ID: PI-4857) and Parc Sanitari San Joan de Deu in Barcelona (ID: PIC-129–21). All participants enrolled in the trial must sign the informed consent form through Docusign (Madrid) or via Qualtrics’ digital signature functionality (participants sign using their mouse or their finger on a mobile device). The harm management protocol described above will be implemented if any participant experiences harm due to trial participation. We will not offer any ancillary or post-trial care or compensation. Any necessary protocol modification will be updated on the clinical trial public register and reported to the local IRBs and to the RESPOND Ethics Advisory Board.

Participants’ data will be kept confidential unless there is an ethical or legal reason for disclosing it. We need contact details to provide the interventions and monitor harms but they will be stored separately from endpoints data. Endpoints will only include anonymised data linked to a unique record identifier automatically generated by the EDC software. The key linking contact details and record identifier will be securely stored in Castor EDC and Qualtrics servers and will only be accessible to local project managers.

The final trial dataset will be accessible to local project managers and the data analyst. We will use it to write scientific publications and disseminate outstanding findings to the general audience. All publications, including the trial protocol, will be open access and include the statistical code. The Authors will consist of members of the RESPOND consortium who make significant contributions to the study design, data collection and analysis, and manuscript writing.

RESPOND partners will sign data-sharing agreements. If possible, pooled analyses will be done remotely so that primary custodians can keep complete control of it. We do not plan to share participant-level data outside the RESPOND consortium.

Conclusions

This study is the first randomised trial that combines two scalable psychological interventions developed by the WHO into one stepped-care, internet-based programme tailored for HCWs. Participants will be enrolled amidst a global spread of the highly contagious Omicron variant of severe acute respiratory syndrome coronavirus 2, which puts a lot of pressure on healthcare systems. HCWs shall face this new challenge while dealing with the mid-and long-term psychological impact of early pandemic outbreaks, which were particularly virulent in Spain. Importantly, the pandemic has revealed that most HCWs, and not only those in direct acute care of COVID-19 patients, are exposed to highly distressing working conditions, such as having to work long shifts or experiencing burnout syndromes, that may easily affect not only their working environment but also their personal lives. Therefore, exploring the effectiveness and cost-effectiveness of this programme among such a vulnerable and essential population is undoubtedly pertinent, and it could rapidly inform occupational and mental health prevention, treatment, and recovery strategies that shall persist beyond the COVID-19 pandemic.

Supplemental Material

sj-doc-1-dhj-10.1177_20552076221129084 - Supplemental material for Effectiveness of a stepped-care programme of internet-based psychological interventions for healthcare workers with psychological distress: Study protocol for the RESPOND healthcare workers randomised controlled trial

Supplemental material, sj-doc-1-dhj-10.1177_20552076221129084 for Effectiveness of a stepped-care programme of internet-based psychological interventions for healthcare workers with psychological distress: Study protocol for the RESPOND healthcare workers randomised controlled trial by Roberto Mediavilla, Kerry R McGreevy, Mireia Felez-Nobrega, Anna Monistrol-Mula, María-Fe Bravo-Ortiz, Carmen Bayón, Beatriz Rodríguez-Vega, Pablo Nicaise, Audrey Delaire, Marit Sijbrandij, Anke B. Witteveen, Marianna Purgato, Corrado Barbui, Federico Tedeschi, Maria Melchior, Judith van der Waerden, David McDaid, A-La Park, Raffael Kalisch, Papoula Petri-Romão, James Underhill, Richard A. Bryant, Josep Maria Haro, José Luis Ayuso-Mateos and in Digital Health

Footnotes

Authors’ contribution

Conceptualisation, methodology, data curation and writing – review and editing were handled by all authors. Investigation was carried out by RM, KRM, MF-N, AM-M, MFB-O, CB, BR-V, JMH, JLA-M. . Writing – original draft was done by RM, KRM.Supervision was done by JLA-M. Project administration was done by RM, KRM, MF-N, AM-M, MFB-O, CB, BR-V, JMH, JLA-M. Funding acquisition was handled by PN, MS, MP, MM, DM, RK, RB, JMH, JLA-M.

Authors note

The RESPOND Consortium includes the following collaborators: Henrik Walter, Ellenor Mittendorfer-Rutz, Antonio Lora, Vincent Lorant, Giovanni Corrao, Brenda Penninx, Ainoa Muñoz-Sanjosé, Eduardo Fernández-Jiménez, Javier Curto-Ramos, Iker Louzao, Fernando Arias-Vicente, Laura Castilla-Rodríguez, Álvaro de-Vicente-Blanco, Andrea Fernández-López, Blanca García-Vázquez, Luis Heredia-Castro, Paula Ibáñez-Mendoza, Cristina Martín-Madrigal, Beatriz Orgaz-Álvarez, Irene Pérez-de-Ciriza-Galarza, Miguel Velasco-Santos, Salvatore Aguilar, Rut Villaescusa, Paula Cristóbal-Narváez, Santiago Palomo, Irene Martinez Chaves, Laura Sanchez Rodriguez, Paula Arin Gonzalez, Alba Jimenez Lafuente, Mercedes Lo Monaco, Aida Fernandez Sanz, Elisabet Salomon Mallat, Maria Roura Adserias. The content of this article reflects only the authors’ views and the European Community is not liable for any use that may be made of the information contained therein

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

The study was approved by institutional review boards (IRBs) at Hospital La Paz in Madrid (ID: PI-4857) and Parc Sanitari San Joan de Deu in Barcelona (ID: PIC-129-21).

Funding

The RESPOND project was funded under Horizon 2020 -the Framework Programme for Research and Innovation (2014–2020) (grant number: 101016127), and the work of MF-N was supported by a postdoctoral fellowship of the ISCIII (CD20/00036).

Guarantor: JLA-M

Informed consent

All participants enrolled in the trial have signed the informed consent form through Docusign (Madrid) or via Qualtrics’ digital signature functionality (participants sign using their mouse or their finger on a mobile device).

ORCID iDs

Roberto Mediavilla

Kerry R McGreevy

María-Fe Bravo-Ortiz

David McDaid

Supplemental material

Supplemental material for this article is available online.

References

Armocida

Formenti

Ussai

, et al. The Italian health system and the COVID-19 challenge. The Lancet Public Health 2020 May 1; 5: e253.

The Lancet. COVID-19: protecting health-care workers. Lancet 2020; 395: 922.

García-Basteiro

Alvarez-Dardet

Arenas

, et al. The need for an independent evaluation of the COVID-19 response in Spain. The Lancet 2020 Aug 22; 396: 529–530.

Mediavilla

Fernández-Jiménez

Andreo

, et al. Association between perceived discrimination and mental health outcomes among health workers during the initial COVID-19 outbreak. Revista de Psiquiatría y Salud Mental [Internet]. 2021 Jun 18 [cited 2021 Jul 13]. Available from: https://www.sciencedirect.com/science/article/pii/S1888989121000628?via%3Dihub

Mostafa

Sabry

Mostafa

. COVID-19-related stigmatization among a sample of Egyptian healthcare workers. PLOS ONE 2020 Dec 18; 15: e0244172.

Taylor

Landry

Rachor

, et al. Fear and avoidance of healthcare workers: an important, under-recognized form of stigmatization during the COVID-19 pandemic. J Anxiety Disord 2020 Oct 1; 75: 102289.

Alonso

Vilagut

Mortier

, et al. Mental health impact of the first wave of COVID-19 pandemic on Spanish healthcare workers: a large cross-sectional survey. Revista de Psiquiatría y Salud Mental 2021 Apr 1; 14: 90–105.

Mediavilla

Fernández-Jiménez

Martínez-Alés

, et al. Role of access to personal protective equipment, treatment prioritization decisions, and changes in job functions on health workers’ mental health outcomes during the initial outbreak of the COVID-19 pandemic. J Affect Disord 2021 Dec 1; 295: 405–409.

Lasalvia

Bonetto

Porru

, et al. Psychological impact of COVID-19 pandemic on healthcare workers in a highly burdened area of north-east Italy. Epidemiol Psychiatr Sci 2021 [cited 2021 Apr 15]; 31. Available from: https://www.cambridge.org/core/journals/epidemiology-and-psychiatric-sciences/article/psychological-impact-of-covid19-pandemic-on-healthcare-workers-in-a-highly-burdened-area-of-northeast-italy/E112BA22EEFDD73599534AC313CA531F.

10.

Pappa

Ntella

Giannakas

, et al. Prevalence of depression, anxiety, and insomnia among healthcare workers during the COVID-19 pandemic: a systematic review and meta-analysis. Brain Behav Immun 2020 Aug 1; 88: 901–907.

11.

Santabárbara

Bueno-Notivol

Lipnicki

, et al. Prevalence of anxiety in health care professionals during the COVID-19 pandemic: a rapid systematic review (on published articles in Medline) with meta-analysis. Prog Neuro-Psychopharmacol Biol Psychiatr 2021 Apr 20; 107: 110244.

12.

Kunzler

Röthke

Günthner

, et al. Mental burden and its risk and protective factors during the early phase of the SARS-CoV-2 pandemic: systematic review and meta-analyses. Global Health 2021 Mar 29; 17: 34.

13.

Alonso

Vilagut

Alayo

, et al. Mental impact of COVID-19 among Spanish healthcare workers. A large longitudinal survey. Epidemiol Psychiatr Sci 2022 [cited 2022 May 9]; 31. Available from: https://www.cambridge.org/core/journals/epidemiology-and-psychiatric-sciences/article/mental-impact-of-covid19-among-spanish-healthcare-workers-a-large-longitudinal-survey/B5ABB7A7982BB9A02C7E4CC3DA781019.

14.

Mediavilla

Fernández-Jiménez

Martinez-Morata

, et al. Sustained negative mental health outcomes among healthcare workers over the first year of the COVID-19 pandemic: a prospective cohort study. Int J Public Health 2022 Jun 17; 67: 1604553.

15.

Sasaki

Asaoka

Kuroda

, et al. Sustained poor mental health among healthcare workers in COVID-19 pandemic: a longitudinal analysis of the four-wave panel survey over 8 months in Japan. J Occup Health 2021 Jan; 63: e12227.

16.

Gomes

Murray

Raftery

. Economic evaluation of digital health interventions: methodological issues and recommendations for practice. Pharm Econ 2022 Apr 1; 40: 367–378.

17.

Whaibeh

Mahmoud

Naal

. Telemental health in the context of a pandemic: the COVID-19 experience. Curr Treat Options Psych 2020 Jun 1; 7: 198–202.

18.

Davison

. Stepped care: doing more with less? J Consult Clin Psychol 2000 Aug; 68: 580–585.

19.

Haaga

. Introduction to the special section on stepped care models in psychotherapy. J Consult Clin Psychol 2000 Aug; 68: 547–548.

20.

van Straten

Hill

Richards

, et al. Stepped care treatment delivery for depression: a systematic review and meta-analysis. Psychol Med 2015 Jan; 45: 231–246.

21.

Kampman

van Balkom

AJLM

Broekman

, et al. Stepped-care versus treatment as usual in panic disorder: a randomized controlled trial. PLoS One 2020; 15: e0237061.

22.

Salomonsson

Santoft

Lindsäter

, et al. Stepped care in primary care - guided self-help and face-to-face cognitive behavioural therapy for common mental disorders: a randomized controlled trial. Psychol Med 2018 Jul; 48: 1644–1654.

23.

Jakicic

Tate

Lang

, et al. Effect of a stepped-care intervention approach on weight loss in adults: a randomized clinical trial. JAMA 2012 Jun 27; 307: 2617–2626.

24.

World Health Organization. Scalable psychological interventions for people in communities affected by adversity: a new area of mental health and psychosocial work at WHO [Internet]. World Health Organization 2017. Available from: https://apps.who.int/iris/handle/10665/254581.

25.

Epping–Jordan

Harris

Brown

, et al. Self-Help Plus (SH+): a new WHO stress management package. World Psychiatry 2016; 15: 295–296.

26.

Cuijpers

Heim

Ramia

, et al. Guided digital health intervention for depression in Lebanon: randomised trial. Evidence-Based Mental Health [Internet]. 2022 May 16 [cited 2022 Jun 24]; 0: 1–7. Available from: https://ebmh.bmj.com/content/early/2022/05/15/ebmental-2021-300416.

27.

Dawson

Bryant

Harper

, et al. Problem Management Plus (PM+): a WHO transdiagnostic psychological intervention for common mental health problems. World Psychiatry 2015 Oct; 14: 354–357.

28.

Tol

Leku

Lakin

, et al. Guided self-help to reduce psychological distress in South Sudanese female refugees in Uganda: a cluster randomised trial. Lancet Global Health 2020 Feb 1; 8: e254–e263.

29.

Rahman

Khan

Hamdani

, et al. Effectiveness of a brief group psychological intervention for women in a post-conflict setting in Pakistan: a single-blind, cluster, randomised controlled trial. The Lancet 2019 Apr 27; 393: 1733–1744.

30.

Zhang

Lin

, et al. Problem Management Plus in the treatment of mental disorders in patients with multiple myeloma. Support Care Cancer [Internet]. 2020 Jan 22 [cited 2020 Aug 17]; 28: 4721–4727. Available from: http://link.springer.com/10.1007/s00520-019-05289-9.

31.

Acarturk

Uygun

Ilkkursun

, et al. Effectiveness of a WHO self–help psychological intervention for preventing mental disorders among Syrian refugees in Turkey: a randomized controlled trial. World Psychiatry 2022 Feb; 21: 88–95.

32.

Purgato

Carswell

Tedeschi

, et al. Effectiveness of self-help Plus in preventing mental disorders in refugees and asylum seekers in Western Europe: a multinational randomized controlled trial. Psychother Psychosom 2021 Oct; 90: 403–414.

33.

Bryant

Schafer

Dawson

, et al. Effectiveness of a brief behavioural intervention on psychological distress among women with a history of gender-based violence in urban Kenya: a randomised clinical trial. PLoS Med 2017 Aug 15; 14: e1002371.

34.

Riello

Purgato

Bove

, et al. Effectiveness of self-help plus (SH+) in reducing anxiety and post-traumatic symptomatology among care home workers during the COVID-19 pandemic: a randomized controlled trial. R Soc Open Sci 2021; 8: 210219.

35.

Bryant

Dawson

Keyan

, et al. Effectiveness of a videoconferencing-delivered psychological intervention for mental health problems during COVID-19: a proof-of-concept randomized clinical trial. Psychother Psychosom 2021 Dec 7; 91: 63–72.

36.

Kessler

Mroczek

. Final versions of our non-specific psychological distress scale. Ann Arbor, MI: University of Michigan, 1994.

37.

Applied Mental Health Research Group. Design, implementation, monitoring, and evaluation of mental health and psychosocial assistance programs for trauma survivors in low resource countries: a user’s manual for researchers and program implementers (adult version) [Internet]. Baltimore, USA: John Hopkins University, 2013 [cited 2021 Nov 12], 66 p. Available from: https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&ved=2ahUKEwidlvGivZL0AhUM2BQKHQl4AQsQFnoECAMQAQ&url=http%3A%2F%2Fhopkinshumanitarianhealth.org%2Fassets%2Fdocuments%2FVOT_DIME_MODULE1_FINAL.PDF&usg=AOvVaw2rdDuUrX6QOgDIvtFs80hX.

38.

Brown

Aoun

Taha

, et al. The cultural and contextual adaptation process of an intervention to reduce psychological distress in young adolescents living in Lebanon. Front Psychiatry 2020 Mar 23; 11: 212.

39.

Burchert

Alkneme

Bird

, et al. User-centered app adaptation of a low-intensity E-mental health intervention for Syrian refugees. Front Psychiatry 2019 Jan 25; 9: 663.

40.

World Health Organization, War Trauma Foundation, World Vision International. Psychological first aid: facilitator’s manual for orienting field workers [Internet]. Geneva: World Health Organization, 2013 [cited 2021 Nov 29], Available from: https://apps.who.int/iris/handle/10665/102380.

41.

Hof

Heim

Ramia

, et al. Evaluating the effectiveness of an E-mental health intervention for people living in Lebanon: protocol for two randomized controlled trials. JMIR Res Protoc 2021 Jan 28; 10: e21585.

42.

Kroenke

, et al. The patient health questionnaire anxiety and depression scale (PHQ-ADS): initial validation in three clinical trials. Psychosom Med 2016; 78: 716–727.

43.

Kroenke

Spitzer

Williams

JBW

. The PHQ-9. J Gen Intern Med 2001; 16: 606–613.

44.

Spitzer

Kroenke

Williams

JBW

, et al. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006 May 22; 166: 1092–1097.

45.

Diez-Quevedo

Rangil

Sanchez-Planell

, et al. Validation and utility of the patient health questionnaire in diagnosing mental disorders in 1003 general hospital Spanish inpatients. Psychosom Med 2001 Aug; 63: 679–686.

46.

García-Campayo

Zamorano

Ruiz

, et al. Cultural adaptation into Spanish of the generalized anxiety disorder-7 (GAD-7) scale as a screening tool. Health Qual Life Outcomes 2010 Jan 20; 8: 8.

47.

Muñoz-Navarro

Cano-Vindel

Medrano

, et al. Utility of the PHQ-9 to identify major depressive disorder in adult patients in Spanish primary care centres. BMC Psychiatry 2017 Aug 9; 17: 291.

48.

Manea

Gilbody

McMillan

. Optimal cut-off score for diagnosing depression with the Patient Health Questionnaire (PHQ-9): a meta-analysis. CMAJ 2012 Feb 21; 184: E191–E196.

49.

Price

Szafranski

van Stolk-Cooke

, et al. Investigation of abbreviated 4 and 8 item versions of the PTSD Checklist 5. Psychiatry Res 2016 May 30; 239: 124–130.

50.

Miles

JNV

Marshall

Schell

. Spanish and English versions of the PTSD Checklist–Civilian version (PCL-C): testing for differential item functioning. J Trauma Stress 2008; 21: 369–376.

51.

Herdman

Gudex

Lloyd

, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res 2011 Dec 1; 20: 1727–1736.

52.

Hernandez

Garin

Pardo

, et al. Validity of the EQ–5D–5L and reference norms for the Spanish population. Qual Life Res 2018 Sep 1; 27: 2337–2348.

53.

Vázquez-Barquero

Gaite

Cuesta

, et al. Versión española del CSRI: una entrevista para la evaluación de costes en salud mental. Archivos de Neurobiología 1997; 60: 171–184.

54.

Kalisch

Baker

Basten

, et al. The resilience framework as a strategy to combat stress-related disorders. Nat Hum Behav 2017 Nov; 1: 784–790.

55.

Chmitorz

Kurth

Mey

, et al. Assessment of microstressors in adults: questionnaire development and ecological validation of the mainz inventory of microstressors. JMIR Ment Health 2020 Feb 24; 7: e14566.

56.

Veer

Riepenhausen

Zerban

, et al. Psycho-social factors associated with mental resilience in the Corona lockdown. Transl Psychiatry 2021 Jan 21; 11: 1–11.

57.

Proctor

Silmere

Raghavan

, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 2011 Mar 1; 38: 65–76.

58.

Moore

Audrey

Barker

, et al. Process evaluation of complex interventions: medical research council guidance. Br Med J 2015 Mar 19; 350: h1258.

59.

Rahman

Riaz

Dawson

, et al. Problem Management Plus (PM+): pilot trial of a WHO transdiagnostic psychological intervention in conflict–affected Pakistan. World Psychiatry 2016 Jun; 15: 182–183.

60.

Kalisch

Köber

Binder

, et al. The frequent stressor and mental health monitoring-paradigm: a proposal for the operationalization and measurement of resilience and the identification of resilience processes in longitudinal observational studies. Front Psychol 2021 Aug 26; 12: 710493.

61.

Kalisch

Müller

Tüscher

. A conceptual framework for the neurobiological study of resilience. Behav Brain Sci 2015; 38: e92.

62.

R Core Team. R: a language and environment for statistical computing [Internet]. Vienna, Austria: R Foundation for Statistical Computing, 2020, Available from: https://www.R-project.org/.

63.

StataCorp. Stata Statistical Software: Release 17. Texas: StataCorp LLC; 2021.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.29 MB