Sage Journals: Discover world-class research

Abstract

French

Objectives:

First-responder mental health, especially in Canada, has been a topic of increasing interest given the high incidence of poor mental health, mental illness, and suicide among this cohort. Although research generally suggests that resiliency and stigma reduction programs can directly and indirectly affect mental health, little research has examined this type of training in first responders. The current paper examines the efficacy of the Road to Mental Readiness for First Responders program (R2MR), a resiliency and anti-stigma program.

Methods:

The program was tested using a pre-post design with a 3-month follow-up in 5 first-responder groups across 16 sites.

Results:

A meta-analytic approach was used to estimate the overall effects of the program on resiliency and stigma reduction. Our results indicate that R2MR was effective at increasing participants’ perceptions of resiliency and decreasing stigmatizing attitudes at the pre-post review, which was mostly maintained at the 3-month follow-up.

Conclusions:

Both quantitative and qualitative data suggest that the program helped to shift workplace culture and increase support for others.

Keywords

absenteeism meta-analysis stigma evaluation

Introduction

There has been a recent interest in the mental health of first responders, highlighted by media reports of poor mental health and high rates of suicide in these groups. In addition to media reports of stress, trauma, and self-injury in these groups, there is evidence that first responders have a higher incidence of mental illness than the general population. For example, a recent study¹ reported that 44.5% of a sample of 5,813 Canadian first responders had a positive screen for at least one mental illness on a battery of self-reported diagnostic measures. A recent meta-regression of 29 studies suggested that 10.0% of first responders currently experience post- traumatic stress disorder (PTSD).² Similarly, Carleton and colleagues found that 23.2% of their sample screened positively for PTSD. Comparatively, these are much higher prevalence rates than those found in the general population, where, for example, the 1-year prevalence of mental illness is about 20%.^3,4 It should be noted, however, that many of the first-responder studies cited above used convenience samples and self-report measures rather than representative population-based samples and diagnostic interviews to generate their estimates of disorders.

Other research suggests that first responders are exposed to high levels of stress in their work environments. For example, Johnson et al.⁵ examined the level of psychological symptoms indicative of stress in 26 occupations and found that Fire Services personnel, Paramedics, and Police Services personnel ranked third, fourth, and eleventh, respectively. Paramedics had the worst physical symptoms associated with stress, whereas prison officers, paramedics, and police workers were ranked first, second, and third, respectively, in terms of lower job satisfaction. In a large survey of about 4,500 police officers, Duxbury and Higgins⁶ found that 50% of their sample rated their perceived stress as high, with 46% in the moderate range. These authors also found that 30% of their sample had a high depressed mood and 40% a moderate depressed mood. In the workplace of paramedics, both operational and organizational stressors, such as excessive demand/workload, a lack of time for rest, a lack of control over job situation, a lack of decision making ability (e.g., “hierarchical bureaucracy”), and a lack of support from the employer, exacerbate the physical and psychological demands.⁷ Taken together, the literature suggests that first responders have high-stress occupations that are related to worse mental health outcomes and a higher incidence of mental illness.

A logical extension of these conclusions is that the implementation of programming to improve mental health and increase the individuals’ ability to manage stress or deal with challenges would be beneficial for this population. Similarly, a program that reduces the stigma associated with mental illness may help to increase early help-seeking and create a supportive mental health culture, and therefore potentially benefit organizational productivity and financial well-being.^8,9 For example, a recent meta-analysis in first responders found that stigma is a barrier to care, with particular concerns about confidentiality and its impact on one’s career.¹⁰

Research on mental health promotion and prevention in the workplace has generally demonstrated positive outcomes.¹¹ For example, one review showed that 80% of the primary interventions aimed at reducing burnout were successful, with the effect sustained in the long-term.¹² In a systematic review of 24 available workplace mental health prevention studies from 2001 to 2006, Corbière et al.¹³ found that almost 70% of the studies had positive effects on psychological outcomes (e.g., stress, mental illness). Further, 10 of 17 studies that included workplace measures (e.g., job satisfaction) showed positive effects in that domain. Workplace mental illness anti-stigma programs also generally lead to reduced stigma and increased knowledge and support; albeit, methodological and design issues are concerns in the extant literature.¹⁴

Despite evidence that prevention-focused mental health programs increase resilience and improve mental health, along with the need to reduce mental illness stigma in first responders, there is scant research in this domain. Andersen and colleagues¹⁵ have argued that, although resilience training is relatively new, evidence does suggest that police organizations should implement resilience training more widely. Gayton and Lovell¹⁶ similarly noted that little attention has been given to resilience training for paramedics despite the potentially positive impact of this type of training both financially and in terms of individual wellbeing. Finally, as indicated before, Haugen et al.¹⁰ found that stigma is an issue for first responders and that it has negative consequences in terms of reduced help-seeking. Given these lines of research, it seems evident that implementation of a program that addresses resilience and stigma reduction in first responders is warranted and may offer positive impact (e.g., improved mental health).

The Road to Mental Readiness for First Responders and the Current Study

The current study is an evaluation of the efficacy of the Road to Mental Readiness (R2MR) for First Responders program. This program addresses the need for increased resiliency training in first responders as well as providing ways to reduce the stigma of mental illness. The R2MR for First Responders was developed by the Canadian Department of National Defence (DND) in conjunction with the Calgary Police Service (CPS) and the Opening Minds Initiative of the Mental Health Commission of Canada. It was adapted from the R2MR program developed by the DND (see Szeto and Adair¹⁷; Szeto et al.⁸). The development process started in early 2013 with a focus on police personnel, and was spearheaded by the CPS. The pilot R2MR for First Responders program was finalized between September and December, 2013, with training for the program trainers initiated in December 2013. The program was implemented at several police organizations in 2014 with a larger roll out of the program in 2015. During this time, the program was also adapted for Paramedics, Fire Services personnel, 911 call centres, and Corrections Officers. The current study reports on this wider dissemination of the R2MR program to various first-responder groups.

The program has been described extensively in other publications (e.g., Szeto & Adair¹⁷; Szeto et al.⁸; as well, for DND’s version of the program see http://www.forces.gc.ca/en/caf-community-health-services-r2mr/index.page). The R2MR Program for First Responders is a 4-h program intended for frontline staff. The program contains 3 main components: stigma reduction through video contact-based education, the Mental Health Continuum Model, and the “Big 4” coping and resilience skills. Two of the main goals of the program are to decrease mental illness stigma and increase resiliency. An 8-h version of the program also exists for supervisors and leaders with the same core components as well as additional discussions and skill building tools for supervisors and leaders to take care of their staff at various stages of the Mental Health Continuum Model. The extended version also incorporates ways to create a supportive working environment and describes how to follow-up after stressful or critical incidences.

This paper reports on the results of a pooled analysis from 16 replications of the R2MR for First Responders program for numerous Canadian first-responder groups who undertook the program between February 2015 and June 2016. It was expected that program completion would decrease mental illness stigma and increase resiliency when compared with assessments conducted before the program. These hypotheses were tested with study-level meta-analysis methods, with data assessed both immediately after the program and at the 3-month follow-up.

Methods

Data Sources, Participants, and Procedures

Details about the various program implementations are provided in Table 1, including the number of participants and survey completions across the 16 sites. All implementations were evaluated using a non-randomized quasi-experimental pre-post follow-up design. Further, except for sites 9 and 16, all sites included a 3-month follow-up survey. Surveys were linked across time points while preserving confidentiality through a process whereby participants answered 4 questions at each time point (e.g., the sixth digit of home phone number) to generate a pseudorandom code.

Table 1.

R2MR Individual Evaluation Details: Setting, Target Audience, Total Participants and Pre-Post Completed Surveys.

Study/Site	Province	First-Responder Type	Audience	N	# Matched Pre-Post
1	British Columbia and Ontario	Corrections	Supervisors and Frontline staff	590	418
2	Alberta	Emergency Services	Supervisors and Frontline staff	195	182
3	Alberta	Fire Services	Supervisors and Frontline staff	184	139
4	Ontario	Fire Services	Supervisors and Frontline staff	893	682
5	Ontario	Police Services	Frontline staff (trainees)	369	347
6	New Brunswick	Police Services	Frontline staff	65	57
7	Alberta	Police Services	Supervisors and Frontline staff	793	714
8	Alberta	Police Services	Supervisors and Frontline staff	210	210
9	Alberta	Police Services	Supervisors and Frontline staff	130	108
10	British Columbia	Police Services	Supervisors and Frontline staff	395	289
11	Quebec	Police	Frontline staff	134	100
12	Ontario	Police Services	Supervisors and Frontline staff	819	665
13	Ontario	Police Services	Supervisors and Frontline staff	89	81
14	Ontario	Paramedics	Supervisors and Frontline staff	500	456
15	Ontario	Paramedics	Supervisors and Frontline staff	173	149
16	British Columbia	Police Services	Frontline staff	59	52

Participants completed the R2MR for First Responders program (either the 4 or 8-h version) as a part of their organizational training. All participants completed the pre-intervention questionnaire package, including consent forms before undertaking the program. The post-intervention questionnaire package was completed at the end of the program. At this time point, participants provided their email address for the 3-month follow-up questionnaire. At about 3 months, participants were sent a link via email to access the follow-up questionnaire package online. All participants were sent 2 reminder emails. Ethics approval for this evaluation was granted by the University of Calgary Conjoint Faculties Research Ethics Board (ID: REB14-1611).

Primary Outcomes

Two primary outcomes were identified for the program: reduction in mental illness stigma and improvement in resiliency skills. Stigma was measured using the Opening Minds Scale for Workplace Attitudes (OMS-WA).¹⁸ The OMS-WA is a 22-item scale designed specifically for workplace environments to assess attitudes, stereotypes, and behavioural intentions toward persons with mental illness. Examples of scale items included: “Most employees with a mental illness are too disabled to work,” “Employees with a mental illness often become violent if not treated,” “I would help a co-worker who got behind in their work because of a mental illness,” “You can’t rely on an employee with a mental illness,” and “I would try to avoid an employee with a mental illness.” There were 5 subscales on the OMS-WA: desire for avoidance, perceptions of dangerousness and unpredictability, attitudes about mental illness in the workplace, attitudes towards helping people with a mental illness, and beliefs about responsibility for having a mental illness. All items were scored on a Likert-like scale from 1 to 5, where lower scores indicated less stigma. Mean scores were used for the full scale and each of the 5 subscales. Previous studies have shown that this measure has good internal consistency (i.e., α > 0.70) (e.g., Szeto et al.¹⁸; Szeto et al.¹⁹).

The outcome of improvement in resiliency skills was assessed with a 5-item scale, developed specifically for the current evaluations. The scale captured participants’ perceptions of their level of skill and ability to recover from adverse or traumatic situations. Scale items included “I have the skills to cope with traumatic events or adverse situations,” and “I believe I can recover quickly if I am negatively affected by traumatic events or adverse situations.” Responses were scored on a Likert-like scale from 1 to 5, with higher scores indicating greater perceived resiliency skills.

Additional Outcomes

Three additional outcomes were explored. First, stigma and resiliency skills were evaluated at the 3-month follow-up. Second, 4 questions explored the extent to which participants’ understanding of workplace mental health improved after the program, and their willingness to discuss mental health in general, to seek help, and support colleagues regarding mental health in the workplace. These questions related to workplace mental health were rated on a 5-point Likert-like scale from strongly agree to strongly disagree, and participants were asked these questions at pre-test and at the 3-month follow-up. The statements were: “I understand how mental health problems present in the workplace;” “I plan to seek help for my mental health problems, when needed;” “When I am concerned, I ask my colleagues how they are doing;” and “I talk about mental health issues as freely as physical health issues.” Finally, at the 3-month assessment, participants were asked about the extent to which they were using the skills and knowledge learned through the R2MR program. Participants were asked if they had used any of what they had learned in the R2MR program at home or at work (yes/no response), and also to describe their response in more detail. Open-ended responses were coded for themes and analysed by producing frequency tables.

Although the primary outcomes may be viewed as a direct assessment of program impact, the additional outcomes were identified to assess changes over the longer term, which may be one way to glimpse whether broader organizational or cultural shifts regarding workplace mental health might be taking place. It should be noted that the evaluation surveys contained numerous additional questions that were not analysed for the purposes of this report. The full set of measures used for these evaluations can be found on The Working Mind website under site reports (http://theworkingmind.ca/working-mind#Research).

Results

Approach to Data Analysis

The analysis approach was 2-fold and conducted using STATA v.12.²⁰ First, using an effect measure of pre- to post-test change, the “metan” command was used to show outcomes by study, using a forest plot to visually display program outcomes.²¹ A random effects model was chosen, as such models account for both random variability and the variability in effects among data sets. Studies were weighted based on the inverse of the variance of the study’s estimated effect. An assessment of the consistency of the effect is a key advantage of the meta-analysis technique, and this approach permits a consideration of program fidelity. Both the Q statistic and the I² test were used to examine heterogeneity of the results across the studies.^21,22 I² describes the percentage of variation due to heterogeneity rather than chance, where a value of 0% indicates no observed heterogeneity, values between 25% and 50% may be considered “low,” those between 50% and 75% as moderate, and those between 75% and 100% as large.²²

Second, a pooled dataset was produced to explore participant-level fixed effects on program outcomes. A random intercept linear mixed model approach was used for analysis, with study also being modelled as a random effect. This approach supported the modelling of participant characteristics as independent variables, with the random intercept used to account for random variability across different studies. Participant characteristics were entered separately. Tests included pre-post change by participant type (frontline staff or supervisor), first-responder type (police, firefighters, etc.), gender, age, education, marital status, and self-rated mental health at baseline. These participant factors were captured as part of the pre-test questionnaire. The assessment of change from post to follow-up for both the OMS-WA and the resiliency skills scale was performed using the random intercept linear mixed model analysis. This same method was also employed for the 4 measures related to understanding and intentions regarding mental health in the workplace.

Preliminary Analysis

The dataset for the pooled analysis included 5,598 participants across 16 sites, with a total of 4,649 completed and matched pre-post surveys. Attrition was mainly due to difficulties in matching some of the pre and post surveys (i.e., inconsistencies in responses across the identifier questions). The number of matched surveys at follow-up was 845, with a total of 1,154 follow-up surveys completed. The main reason for attrition at follow-up was non-response; although, again, there were some difficulties in survey matching. We analysed the non-response for the follow-up sample, and found that participants who completed all 3 surveys had lower (i.e., more positive) baseline stigma scores than did participants who did not complete all 3 surveys (1.92 and 1.99 respectively, p < 0.001). Those who completed all 3 surveys were also on average slightly older (41.5 and 40.0 years, respectively, p < 0.001). Women were also more likely than men to complete all 3 surveys (non-completers: male, 68.3%, female, 31.7%; completers: male, 55.9%, female, 44.1%, p < 0.001). No significant differences in marital status, education level, or baseline resiliency scores were observed between those who completed all 3 surveys and those who did not. Two sites (study 9 and 16, see Table 1) did not issue a follow-up survey.

Cronbach’s alphas for the OMS-WA were 0.91 at pre-test, 0.93 at post-test, and 0.92 at follow-up. For the OMS-WA subscales, Cronbach’s alphas were acceptable or near acceptable at all time points (social avoidance: 0.85, 0.89, 0.89; danger/unpredictability: 0.74, 0.80, 0.80; work-related beliefs: 0.79, 0.80, 0.81; helping behaviour: 0.61, 0.69, 0.69; responsibility for one’s illness: 0.68, 0.76, 0.77). The subscales for helping behaviour and responsibility for one’s illness contain few items (4 and 3, respectively), which likely contributed to lower internal reliability for these subscales. Values for Cronbach’s alpha for the resiliency skills scale were 0.84 at pre-test, 0.87 at post-test, and 0.85 at follow-up, indicating a high level of internal consistency at all 3 time points. An examination of a histogram and the Q–Q plot showed a normal distribution of change scores for both primary outcome measures. Table 2 shows the participant characteristics for the pooled sample.

Table 2.

Summary of Participant Characteristics (n = 4649).

Variable	% (n)
First-Responder Group
Corrections	9.0% (418)
Emergency Services (9-1-1)	3.9% (192)
Fire Services	17.7% (821)
Police Services	56.5% (2,623)
Paramedics	13.0% (605)
Participant type
Frontline staff	75.8% (3,449)
Supervisory staff	26.4% (1,210)
Gender
Male	65.2% (3,029)
Female	33.2% (1,542)
No response	1.7% (78)
Education
Less than high school	0.2% (10)
High school	13.4% (624)
Some post-secondary/Non-university certificate	41.4% (1,926)
Bachelor’s degree	34.7% (1,611)
Graduate degree	7.9% (367)
No response	2.3% (111)
Age group (years)
<30	17.9% (831)
30-39	31.9% (1,485)
40-49	28.5% (1,323)
50-59	16.5% (767)
60+	2.6% (123)
No response	2.3% (120)
Marital Status
Single	16.7% (778)
Married	59.6% (2,770)
Divorced or Separated	7.6% (355)
Common Law	13.6% (634)
Widowed	0.6% (27)
No response	1.9% (85)
Self-rated mental health
Poor	0.8% (38)
Fair	6.2% (288)
Good	28.9% (1,343)
Very good	41.0% (1,905)
Excellent	20.1% (936)
No response	3.0% (139)

Primary Outcomes

Figure 1 shows the forest plot of the individual program effects for the OMS-WA measure. Program effect sizes or standardized mean differences (SMDs; i.e. Cohen’s d) ranged from 0.12 to 0.45, with an overall combined effect size of 0.26. The test of SMD = 0 revealed a z-score of 12.43, which was significant at the 95% confidence interval (CI; p < 0.001). Heterogeneity across studies was not observed (Heterogeneity χ2 (Q) =11.99, df = 15, p = 0.680; I² = 0.0%).

Figure 1.

Forest Plot of R2MR Program Effects by Study/Site: Change in OMS-WA.

For the pooled sample, the OMS-WA mean score at pre-test was 1.97 (SD = 0.47). At post-test, the score was 1.85 (SD = 0.49), representing an overall mean stigma reduction of 0.12 scale points. Table 3 provides the results of the mixed model analysis for the pre to post change on the OMS-WA. Statistically significant reductions in stigma were observed for the total scale and all subscales. The analysis of participant factors found no difference in the outcomes by participant type (frontline vs. supervisor), age, education, marital status, or self-rated mental health. Significant differences were found for gender (more improvement for men) and first-responder type (more improvement for police). A repeat of the mixed model analysis including a baseline score as a control variable resulted in non-significant results for both gender (p = 0.97) and first-responder type (p = 0.37).

Table 3.

Random Intercept Mixed Model Regression: OMS-WA Pre to Post-Change and Effect of Participant Characteristics.

	Coeff	SE	z	p
Total Scale	0.123	0.008	15.87	<0.001
Social distance /avoidance	0.128	0.010	12.94	<0.001
Dangerousness/unpredictably	0.207	0.013	16.18	<0.001
Work Beliefs	0.091	0.012	7.47	<0.001
Helping Behaviour	0.108	0.010	10.61	<0.001
Responsibility for Illness	0.055	0.007	7.55	<0.001
Supervisor vs Frontline
Frontline staff	−0.009	0.011	−0.78	0.438
Supervisor (constant)	0.131
First Responder Type
Police	0.030	0.014	2.19	0.028
Other (constant)	0.106
Gender
Male	0.014	0.006	2.54	0.011
Female (constant)	0.114
Age
Yrs from 20	−0.001	0.001	−1.61	0.107
20 yrs old (constant)	0.136
Education
Bachelor Degree or higher	0.011	0.009	1.22	0.223
Less than Bachelor (constant)	0.118
Marital Status
Married/common law	−0.001	0.0010	−0.09	0.925
Other (constant)	0.124
Self-rated mental health (1 to 5)
Rating from Poor	0.005	0.005	1.01	0.313
Poor (constant)	0.104

Table 4 shows the results of the mixed model analysis for the pre to post change on the resiliency skills scale. The mean score for resiliency skills was 3.65 (SD = 0.64) at pre-test and 3.84 (SD = 0.60) at post-test, representing an overall mean improvement of 0.19 scale points, which was statistically significant at the 95% CI. No differences in outcome was found for age, education, marital status, gender, or first-responder type. A significant difference was found for participant type, with a greater improvement observed among supervisors. A significant difference was observed for self-rated mental health, where a lower self-rated mental health correlated with a larger improvement in resiliency skills. Differences persisted when the mixed model analysis was repeated with the baseline score included as a control variable. Figure 2 shows the forest plot of the individual program effects for resiliency skills outcomes. Program effects ranged from 0.15 to 0.49 (SMD), with an overall combined effect size of 0.32 (z = 12.95, p < 0.001). The Q test for heterogeneity across the studies was not significant (Heterogeneity χ2 (Q) = 18.73, df = 15, p = 0.23), and shows an I² in the very low range (19.9%).

Table 4.

Random Intercept Mixed Model Regression: Resiliency Skills Pre to Post-Change and Effect of Participant Characteristics.

	Coeff	SE	z	p
Total Scale (5 items)	−0.190	0.015	−12.90	<0.001
Supervisor vs Frontline
Frontline staff	0.041	0.020	2.06	0.040
Supervisor (constant)	−0.220
First Responder Type
Police	0.050	0.026	1.91	0.057
Other (constant)	−0.218
Gender
Male	0.001	0.010	0.09	0.926
Female (constant)	−0.191
Age
Years from 20	−0.001	0.001	−0.81	0.070
20 years (constant)	−0.161
Education
Bachelor degree or higher	−0.003	0.016	−0.20	0.845
Less than Bachelor (constant)	−0.189
Marital Status
Married/common law	−0.006	0.018	−0.37	0.714
Other (constant)	−0.187
Self-rated mental health (1 to 5)
Rating from Poor	0.063	0.009	7.10	<0.001
Poor (constant)	−0.425

Figure 2.

Forest Plot of R2MR Program Effects by Study/Site: Change in Resiliency Skills.

Additional Outcomes

A total of 1,179 follow-up surveys were completed, 845 (72%) of which could be matched to corresponding pre and post surveys. Table 5 shows the results of the post to follow-up analysis for the OMS-WA total scale and subscales, the resiliency skills scale, and the pre to follow-up change for the 4 statements pertaining to mental health knowledge and intentions in the workplace. Reductions in stigma were maintained until the final follow-up for the total scale and for the 2 subscales of avoidance and danger/unpredictability. Additional significant improvements were observed for the subscales of work-related beliefs and responsibility for one’s illness. The improvement in the helping behaviour subscale was not retained at the final follow-up. An analysis of participant factors showed the loss in gain on the helping subscale was greater for supervisors than for frontline staff (coefficient = 0.109; SE = 0.053; z = 2.04; p = 0.041; constant = −0.215).

Table 5.

Random Intercept Mixed Model Regression: OMS-WA Post to Follow-up-Change, Post to Follow-up Resiliency Skills Change, Pre to Follow-up Change for Mental Health Knowledge/Intentions Statements.

	Coeff	Sd. Error	z	p
OMS-WA Total Scale	−0.002	0.012	−0.13	0.899
Social distance /avoidance	0.012	0.016	0.73	0.462
Dangerousness/unpredictably	−0.018	0.·020	−0.88	0.380
Work-related beliefs	0.081	0.019	4.29	<0.001
Helping behaviour	−0.135	0.024	−5.68	<0.001
Responsibility for illness	0.042	0.019	2.22	0.026
Resiliency Skills Scale	0.076	0.022	3.54	<0.001
Q1: I understand how mental health problems present in the workplace	−0.409	0.047	−8.61	<0.001
Q2: I plan to seek help for my mental health problems, when needed	−0.232	0.047	−4.93	<0.001
Q3: When I am concerned, I ask my colleagues how they are doing	−0.094	0.032	−2.94	0.003
Q4: I talk about mental health issues as freely as physical health issues	−0.195	0.037	−5.23	<0.001

A reduction in reported resiliency skills was observed from post to follow-up; although, the follow-up scores were still significantly improved over those at baseline (coefficient = −0.134; SE = 0.24; z = −5.63; p < 0.001). Scores for the resiliency skills at the 3 time points were pre (baseline) = 3.61 (SD = 0.64), post-test = 3.83 (SD = 0.60), follow-up = 3.76 (SD = 0.61) (n = 793 matched). An analysis of participant groups showed that Fire Service participants had better retention of resiliency skills scores as compared with the other first-responder groups (coefficient = −0.088; SE = 0.042; z = −2.11; p = 0.035; constant = 0.093). There were no other post- to follow-up differences.

The items pertaining to participants’ mental health in the workplace, intentions towards seeking help, and behaviours related to openness and supporting fellow colleagues were all significantly improved from baseline to follow-up (see Table 5). Scores were: Q1: baseline = 3.35 (SD = 0.80), follow-up = 3.77 (SD = 0.60); Q2: baseline = 3.78 (SD = 0.76), follow-up = 3.99 (SD = 0.89); Q3: baseline = 4.09 (SD = 0.64), follow-up = 4.18 (SD = 0.55); Q4: baseline = 3.25 (SD = 1.04), follow-up = 3.45 (SD = 0.97). Greater improvements for the question, “I understand how mental health problems present in the workplace,” were observed among Fire Services as compared with other first-responder groups (coefficient = −0.234; SE = 0.098; z = −2.39; p = 0.017; constant = −0.362). Somewhat lower levels of improvement were also observed among frontline staff as compared with supervisors (coefficient = 0.144; SE = 0.072; z = 2.00; p = 0.045; constant = −0.522). For the item, “I plan to seek help for my mental health problems, when needed,” improvements were greater among Police Services participants (coefficient = -0.201; SE = 0.078; z = −2.60; p = 0.009; constant = −0.140) and lower among Paramedic Services participants (coefficient = 0.63 SE = 0.102; z = −2.58; p = 0.010; constant = −0.261). For improvements on the measure, “I talk about mental health issues as freely as physical health issues,” Paramedic Services showed less improvement than other first-responder groups (coefficient = 0.203, SE = 0.102; z = −1.99; p = 0.045; constant = −0.227). No other participant differences were observed on these measures.

For the follow-up question, “Have you used any of what you learned at R2MR at work or at home?” 59.2% of respondents indicated positively. The coded descriptions for participants who responded “Yes” are provided in Table 6. Many participants mentioned using the tools taught in the program, particularly the “Big 4” skills (62.8%) and, to a lesser extent, the Mental Health Continuum Model (12.5%). Talking more about mental health in the workplace and providing support to another person were also commonly mentioned (12.5% and 14.0%, respectively). Additionally, 6.3% of participants indicated that the program led them to seek professional help for their own mental health and/or enabled them to get a friend or colleague to seek professional help for their mental health. Most respondents who replied “No” did not provide further details (80.0%; n = 375). Several reasons were provided, however, for why the skills had not been used: there was a lack of need or opportunity (10.9%), they had forgotten what they had learned (2.6%), the course had not taught them anything new (3.4%), they were unsure how to use the skills (1.5%), and the course was not sufficient to improve support for workplace mental health in their specific organization (1.7%).

Table 6.

Have You Used Any of What You Learned at the R2MR at Work or at Home – Specify: Respondents Who Answered “Yes” (n = 524 Cases; n = 661 Responses).

	n	% of Responses	% of Cases
One or more of Big 4 skills (self and/or other))	328	49.6%	62.8%
Provided support to another / asked about colleagues’ MH / reached out to someone concerned about	73	11.0%	14.0%
Mental Health continuum (self and/or others)	65	9.8%	12.5%
More openness / talking about mental health in workplace/with others	65	9.8%	12.5%
More self care (made changes/focus on wellness/more balance/exercise more/meditating	44	6.7%	8.4%
More awareness / understanding / appreciation of mental health in the workplace	36	5.4%	6.9%
Sought professional help (self and/or others)	33	5.0%	6.3%
Less stigma / less judgmental / more accepting of people with mental health problems	14	2.1%	2.7%
Other	3	0.5%	0.6%
Total	661	100%	126.6%

Note: non-response set to missing (n = 159).

Discussion

The results indicate that R2MR for First Responders was effective at reducing the stigma of mental illness and increasing resiliency skills after program implementation in participants across 16 different sites and in 5 different first-responder groups. As such, our results indicate that the program was successful in achieving its main course objectives. In general, after taking the R2MR for First Responders program, participants reported fewer stigmatizing attitudes towards those with mental health illnesses and felt more prepared to handle stressful and traumatic events in their workplace.

It is important to note that all sites achieved positive outcomes for both main outcome measures, with low variability of the program effects across sites and first-responder groups (see Figures 1 and 2). The lack of variability suggests that the program has wide applicability and utility across diverse sites and first-responder audiences. The result also likely reflects the standardized training given to program trainers. Specifically, trainers attended a 5-day “train-the-trainer” session, in which they were taught the core components of the course and provided with additional mental health background information. Throughout the “train-the-trainer” program, participants were given feedback during “microteaching” sessions (e.g., Boman²³; Levinson-Rose and Menges²⁴) and the training stressed the importance of program fidelity and the use of the course manuals during course delivery. At the end of the 5-day “train-the-trainer” program, trainers were evaluated on a pass/fail basis, and trainers were not certified to teach the program if they did not pass. Overall, these results speak to the success of the standardized training and to program fidelity across sites. Beshai and Carleton²⁵ have also recently recommended that “training for and application of peer support or crisis-focused psychological intervention programs involves systematic and comprehensive adherence to program protocols” (p. 8).

At face value, the mean effect sizes for both stigma reduction and resiliency improvement across the sites were small,²⁶ which implies a modest program impact. That said, the program is 4-h or 8-h in length, and it is unlikely that such a short intervention will create a huge impact when compared with longer or more embedded interventions. In addition, the size of the stigma reduction outcome is consistent with the extant literature. For example, Knaak et al.²⁷ found a mean Cohen’s d of 0.30 in the “Understanding Stigma” intervention for health care providers across 6 sites. Similarly, the meta-analysis by Corrigan et al.²⁸ found a mean Cohen’s d of ∼0.28 for both knowledge and contact-based interventions aiming to reduce the stigma of mental illness. Finally, Pettigrew and Troop’s²⁹ meta-analysis of contact-based interventions to reduce prejudice in various groups (e.g., minorities, sexual orientation, etc.) also found a small effect (mean r of −0.21) of the interventions, with an effect size of r = −0.18 for interventions specific for mental illness stigma.

The current analyses revealed that the R2MR program for First Responders yielded significant increases in self-reported resiliency, with effect sizes in the same range as for stigma reduction. This effect size was comparable with other workplace resiliency trainings^30,31 as well as our own evaluation of The Working Mind (see Dobson et al.³²). Despite the modest global impact, the open-ended questions suggested that the program had substantial impact for many program participants. For example, well over half (59.2%) of the respondents reported at follow-up that they had actively used the skills learned in the program. Of those, 62.4% said they had used at least one of the “Big 4” coping skills before the 3-month follow-up. Furthermore, 14% of the follow-up respondents had used what they learned in the program to support someone else’s mental health, and 6.3% had sought professional help or helped a co-worker seek help because of the program. Although it is beyond the scope of this paper to attach financial numbers to these outcomes, there is an enormous benefit from even one person at an organization seeking help early and not having to go on disability. Dewa et al.³³ found that in one Canadian resource sector organization, disability claims due to mental illnesses were double that of the average claim in both episode length and cost, at a total of 65 days and total cost of about $18,000.

The gains observed at the post-intervention assessment for stigma reduction were retained at the 3-month follow-up, and those for resiliency skills were partially maintained. These results suggest that participants retained the content of the program relatively well over time and did not return to baseline levels. The resiliency result suggests the need to augment program content and use over time, as coping skills may require repeated use. For instance, refresher or “booster” sessions may help maintain program skills, especially if these sessions reinforce the “Big 4” skills learned in the original training.

Two of the 5 OMS-WA subscales reduced (i.e., demonstrated enhanced stigma reduction) at the 3-month follow-up as compared with the post-training values. It is possible that the additional gains seen in the work-related beliefs and responsibility subscale could be connected to on-going changes in workplace culture within these organizations and that these were increasingly supportive of mental health (see also Knaak et al.³⁴). The responses to the 4 workplace mental health and support questions also suggest that the program affected workplace culture to some degree, making it more open and supportive of mental health and help-seeking. There has been a call to examine organizational-level and structural-level factors and to assess how they affect mental health and stigma in the workplace; this topic has emerged as one of the next frontiers for researchers.^11,35,36

Strengths and Limitations

There are both strengths and limitations of the current work. In addition to the positive results, some of the strengths are based on the methodological approach of the study. First, the current study had a large sample size of over 5,000 participants, with over 4,500 matched pre- and post-test questionnaires. Similarly, the current sample comprised a group of diverse first responders, with participants from various geographic regions and occupations across Canada. Another strength was the use of both quantitative scales and qualitative open-ended questions. The open-ended questions helped provide elaboration to the quantitative findings. Finally, the inclusion of a follow-up time point sheds light on the medium-term sustainability of the effects found immediately post-intervention.

Despite the above strengths, it is also important to mention the limitations. First, the study design was a pre-post test with a follow-up open trial. A preferred design would have been a randomized control trial. However, it was difficult to have organizations agree to this type of design for multiple reasons (see discussion of this topic in Szeto et al.⁸). Future evaluations of such programs should ideally use randomized control trials to ascertain that the effects are not due to confounders or sample bias.¹⁵ Second, despite the large pre-post sample, there was substantial attrition at the follow-up time point. Some of this attrition was due to an inability to match participants using the coding system. Therefore, the follow-up data, although positive, should be interpreted with caution. Future research may use a “back-loaded” strategy for participant incentives to decrease the attrition rate at follow-up time points. An alternative matching system or a dedicated site coordinator (see discussion of this point in Szeto et al.⁸) would also increase participant matches at follow-up and reduce attrition while maintaining anonymity. Finally, it is worth noting that the current study conducted follow-up assessment at a 3-month period. It is unclear how long these effects might persist beyond 3 months. Researchers have called for more longitudinal research in this domain,^36,37 as interventions like the current one that focus on promotion and prevention may take longer to filter through an organization, as compared with a more targeted or clinical intervention.

Conclusions

The current results indicate that R2MR for First Responders is an effective program to reduce the stigma of mental illness and increase resiliency. Our results also indicate that these effects persist over the medium term. Further, the consistency of the results across sites, regions, and groups speaks favourably to the foundations of the current program. The wide-spread adoption of the program in the first-responder community in Canada is a testament to the desire for first-responder organizations to improve their members’ mental health and provide opportunities for first responders to better equip themselves to face the stressors and traumas in their daily working lives.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Carleton

Afifi

Turner

. Mental disorder symptoms among public safety personnel in Canada. Can J Psychiatry. 2017;63(1):54−64.

Berger

Coutinho

ESF

Figueira

. Rescuers at risk: a systematic review and meta-regression analysis of the worldwide current prevalence and correlates of PTSD in rescue workers. Soc Psychiatry Psychiatr Epidemiol. 2012;47(6):1001−1011.

Kessler

Berglund

Demler

Jin

Merikangas

Walters

. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the national comorbidity survey replication. Arch Gen Psychiatry. 2005;62(6):593–602. doi:10.1001/archpsyc.62.6.593.

Smetanin

Briante

Stiff

. The life and economic impact of major mental illnesses in Canada. RiskAnalytica, on behalf of the Mental Health Commission of Canada, 2011. Retrieved February 23, 2019. https://www.mentalhealthcommission.ca/sites/default/files/MHCC_Report_Base_Case_FINAL_ENG_0_0.pdf.

Johnson

Cooper

Cartwright

. The experience of work-related stress across occupations. J Managerial Psychol. 2005;20(2):178−187.

Duxbury

Higgins

. Caring for and about those who serve: Work-life conflict and employee well-being within Canada’s police departments. Carleton University, Sprott School of Business; 2012. Available From: https://sprott.carleton.ca/wp-content/uploads/Duxbury-Higgins-Police2012_fullreport.pdf. Retrieved March 3, 2019.

Regehr

Millar

. Situation critical: High demand, low control, and low support in paramedic organizations. Traumatology. 2007;13(1):49−59.

Szeto

Dobson

Luong

Krupa

Kirsh

. Workplace antistigma programs at the Mental Health Commission of Canada: Part 2. Lessons learned. Can J Psychiatry. 2019;64(6S):S13–17.

Dewa

Hoch

. Barriers to mental health service use among workers with depression and work productivity. Occup Environ Med. 2015;57(7):726.

10.

Haugen

McCrillis

Smid

. Mental health stigma and barriers to mental health care for first responders: a systematic review and meta-analysis. J Psychiatr Res. 2017;94:218−229.

11.

Bhui

Dinos

Stansfeld

. A synthesis of the evidence for managing stress at work: a review of the reviews reporting on anxiety, depression, and absenteeism. J Environ Public Health. 2012;2012:515874. doi:10.1155/2012/515874.

12.

Awa

Plaumann

Walter

. Burnout prevention: a review of intervention programs. Patient Educ Couns. 2010;78(2):184−190.

13.

Corbiere

Shen

Rouleau

. A systematic review of preventive interventions regarding mental health issues in organizations. Work. 2009;33(1):81−116.

14.

Hanisch

Twomey

Szeto

ACH

. The effectiveness of interventions targeting mental illness stigma in the workplace. BMC Psychiatry. 2016;16:1. doi:10.1186/s12888-015-0706-4.

15.

Andersen

Papazoglou

Arnetz

. Mental preparedness as a pathway to police resilience and optimal functioning in the line of duty. Int J Emerg Ment Health. 2015;17:624−627.

16.

Gayton

Lovell

. Resilience in ambulance service paramedics and its relationships with well-being and general health. Traumatol. 2012;18(1):58−64.

17.

Szeto

ACH

Adair

. Paramedicine and The R2MR - road to mental readiness. Can Paramedicine. 2016;39:18−21.

18.

Szeto

ACH

Luong

Dobson

. Does labelling matter? An examination of attitudes and perceptions of labels for mental disorders. Soc Psychiatry Psychiatr Epidemiol. 2013;48(4):659−671.

19.

Szeto

ACH

O’Neill

Dobson

. The association between personality and stigmatizing attitudes towards people with mental disorders. Am J Psychiatric Rehab. 2015;18:303−332.

20.

StataCorp. Stata Statistical Software: Release 12. College Station, Texas: StataCorp LP; 2011.

21.

Harris

Bradburn

Deeks

. Metan: fixed- and random-effects meta-analysis. Stata J. 2008;8(1):3−28.

22.

Higgins

JPT

Thomson

Deeks

. Measuring inconsistency in meta-analyses. BMJ. 2003;327:557−560.

23.

Boman

. Graduate student teaching development: evaluating the effectiveness of training in relation to graduate student characteristics. Can J Higher Educ. 2013;43(1):100.

24.

Levinson-Rose

Menges

. Improving college teaching: a critical review of research. Rev Educ Res. 1981;51(3):403−434.

25.

Beshai

Carleton

. Peer support and crisis-focused psychological intervention programs in Canadian first responders: Blue Paper. Regina, SK: University of Regina Collaborative Centre for Justice and Safety; 2016. Available From: http://www.justiceandsafety.ca/resources/cipsrt. Retrieved February 23, 2019.

26.

Cohen

. A power primer. Psychol Bull. 1992;112(1):155−159.

27.

Knaak

Szeto

Kassam

. Understanding stigma: a pooled analysis of a national program aimed at healthcare providers to reduce stigma towards patients with a mental illness. J Ment Health Addict Nurs. 2017;1(1):e19−e29.

28.

Corrigan

Morris

Michaels

. Challenging the public stigma of mental illness: a meta-analysis of outcome studies. Psychiatr Serv. 2012;63(10):963–973.

29.

Pettigrew

Tropp

. A meta-analytic test of intergroup contact theory. J Pers Soc Psychol. 2006;90(5):751−783. doi:10.1037=0022-3514.90.5.751.

30.

Leppin

Bora

Tilburt

. The efficacy of resiliency training programs: a systematic review and meta-analysis of randomized trials. Plos One. 2014;9(10):e111420.

31.

Robertson

Cooper

Sarkar

. Resilience training in the workplace from 2003 to 2014: a systematic review. J Occup Organ Psychol. 2015;88(3):533−562.

32.

Dobson

Szeto

Knaak

. The Working Mind: A Meta-Analysis of a Workplace Mental Health and Stigma Reduction Program. Can J Psychiatry. 2019;64(6S):S39−47.

33.

Dewa

Chau

Dermer

. Examining the comparative incidence and cost of physical and mental health-related disabilities in an employed population. Occup Environ Med. 2010;52(7):758−762.

34.

Knaak

Luong

McLean

Dobson

. Implementation, Uptake, and Culture Change: Results of a Key Informant Study of a Workplace Mental Health Training Program in Police Organizations in Canada. Can J Psychiatry. 2019;64(6S):S30–38.

35.

Hatzenbuehler

ML.

Structural stigma: Research evidence and implications for psychological science. Am Psychol. 2016;71(8):742−751.

36.

Szeto

ACH

Dobson

. Reducing the stigma of mental disorders at work: A review of current workplace anti-stigma intervention programs. Appl Preventive Psychol. 2010;14:41−56.

37.

Pomaki

Franche

Murray

. Workplace-based work disability prevention interventions for workers with common mental health conditions: a review of the literature. J Occup Rehabil. 2012;22(2):182−195.

The Road to Mental Readiness for First Responders: A Meta-Analysis of Program Outcomes

Abstract

Objectives:

Methods:

Results:

Conclusions:

Keywords

Introduction

The Road to Mental Readiness for First Responders and the Current Study

Methods

Data Sources, Participants, and Procedures

Primary Outcomes

Additional Outcomes

Results

Approach to Data Analysis

Preliminary Analysis

Primary Outcomes

Additional Outcomes

Discussion

Strengths and Limitations

Conclusions

Footnotes

Declaration of Conflicting Interests

Funding

References