Abstract
Objectives:
Despite growing awareness of the importance of workplace mental health training and an increasing number of educational resources, there is a gap in knowledge regarding what shapes training effectiveness. The purpose of this study was to compare and describe the active ingredients of 2 workplace mental health education programs for health care workers.
Methods:
Within the context of a randomized clinical trial, a multimethod process evaluation was conducted to explore key process elements shaping implementation outcomes: the innovation, service recipients, service providers, and the organizational context. Data collection included descriptive statistics regarding program participation, postprogram interviews with a purposive sample of 18 service recipients, 182 responses to open-ended questions on postgroup and follow-up surveys, and field journal reflections on the process of implementation. Data analysis was informed by an interpretive description approach, using a process evaluation framework to categorize responses from all data sources, followed by within and cross-case comparison of data from both programs.
Results:
Five key forces shaped the implementation and perceived outcomes of both programs: a contact-based education approach, information tailored to the workplace context, varied stakeholder perspectives, sufficient time to integrate and apply learning, and organizational support. The Beyond Silence program provided more opportunity for contact-based education, health care–specific content, and in-depth discussion of diverse perspectives.
Conclusions:
To increase mental health literacy and reduce stigma, workplace training should be based on best practice principles of contact-based education, with contextually relevant examples and support from all levels of the organization.
Background
The business case for addressing the mental health of health care workers is well established, and there is a growing demand for programs and tools to promote psychological health and safety. 1 Despite recent increases in the number of available resources, very few have been systematically evaluated. 2 In addition, many are generic and do not consider the unique educational needs of health care workers who may have higher levels of mental health literacy yet higher levels of stigma and silence surrounding mental ill health at work. 3,4
“Beyond Silence” is a new workplace mental health education program customized for health care organizations. The curriculum was informed by the findings of several qualitative research studies exploring the mental health experiences of health care workers. 4,5 It incorporates best-practice principles of contact-based education to address the stigma associated with mental illnesses and help seeking. 6 The program is designed to be led by 2 peer educators: employees within the organization who have personal experience with mental health recovery and are trained to facilitate a respectful dialogue regarding workplace mental illness, substance use, and psychological health and safety. The program includes 12 hours of in-person training plus online discussions to complement each in-person session. Topics include understanding how common mental disorders present in the workplace, recognizing when help is needed, reaching out to colleagues, knowing what works, and building on success. There is an emphasis not only on information sharing but also on discussion, problem solving, and skill development, including examples that are specific to health care workplaces. The program is designed for a closed group of 8 to 12 participants.
To evaluate the impact of the Beyond Silence program, a randomized, parallel-group trial was conducted comparing Beyond Silence and Mental Health First Aid training (MHFA) on mental health literacy, stigmatized beliefs, and help-seeking/help-outreach behaviors. 7 MHFA was selected as a “gold standard” comparator as it also involves 12 hours of mental health literacy training and has a good base of evidence regarding its effectiveness. 8 MHFA is a standardized, module-based program that focuses on the signs and symptoms of a range of mental disorders, and a standard approach to providing “mental health first aid.” 8 It was developed for the general public and has been implemented in many settings, including workplaces, 8,9 but its relevance and impact within health care has not been established.
There is growing recognition of the importance of process evaluation within the context of a clinical trial, particularly with complex health service interventions that include many active ingredients. 10 Process evaluations “explore the implementation, receipt, and setting of an intervention and help in the interpretation of the outcome results.” 11(p413) Exploring the nuances of program implementation can help to expand, extend, and complement the outcome evaluation. Process evaluations include an exploration of the social context, components of the intervention itself, and how it is received by various subgroups of participants. 11,12 This study is a process evaluation of mental health literacy training with health care workers, documenting and comparing active ingredients of the Beyond Silence and the MHFA programs. Two key questions guided the research are (1) What is the perceived impact of the program, from the perspective of participants? and (2) How is the program impact influenced by (a) the content and format of the program itself, (b) actions of program facilitators, (c) characteristics of the program participants, and (d) the organizational context?
Methods
A multilevel implementation science framework informed the study design using a multimethods approach to data collection. The implementation science framework proposed by Chaudoir et al. 13 focuses on 4 implementation variables: the innovation, the service recipients, the service providers, and the organizational context. Initial comparisons were based on quantitative descriptive data, followed by a focus on qualitative data from the participant interviews and surveys. Details regarding the methodology have been noted in the study protocol. 7 Ethics approval was obtained from both participating hospitals.
Participants
Participants in the overall clinical trial were hospital employees in 2 multisite hospitals in a mid-sized urban city in southern Ontario. Employees could participate if they were (1) full, part-time, or casual employees; (2) agreeable to being randomly assigned to either of the 2 programs; (3) committed to attending 12 hours of mental health education outside of paid work time; (4) able to communicate in English; and (5) had no prior training in either program. Approximately 59% of the study participants were in clinical positions, 20% were support staff, and 21% were in a management role.
Data were collected from 182 employees who completed postgroup and/or 3-month follow-up surveys. A subgroup of 18 participants was also recruited to participate in in-depth individual interviews about their experience. We used purposive sampling to recruit key informant volunteers who were participants in the first 3 intake periods. To include participants who varied in in gender, age, organizational role, and program experience, the project coordinator identified 3 participants from programs in each of the 3 cohorts (representing approximately one-third of the total number of participants at the time) and invited them to participate in a follow-up interview about their experience. All who were invited agreed to participate. The final sample of interviewees included 9 participants from each of the 2 programs. As noted in Table 1, the majority of participants were female and older than 45 years, with a mix of clinical and nonclinical roles.
Interview Participants.
Intervention Approach
There were 8 intakes over the course of 1 year (September 2014–October 2015). In each intake, employees were randomly assigned to one of two 12-hour educational interventions (Beyond Silence or MHFA). Table 2 outlines the key characteristics of each program.
Key Program Characteristics.
Data Collection
An online survey was distributed to all participants immediately following completion of the program and again at a 3 months. The survey included demographic data (age, gender, occupational role), as well as 3 open-ended questions asking for general feedback about the program, identification of what they gained from attending, and suggestions for improvement. Survey completion rates postgroup and at the 3-month follow-up were 87% and 78%, respectively, with comments from more than 92% of the survey respondents. There were 73 postgroup plus 68 follow-up comments from Beyond Silence participants and 83 postgroup plus 79 follow-up comments from the MHFA participants.
In-depth semistructured interviews were conducted with 18 participants approximately 3 months after they completed the program. Interviews explored their perceptions of the program, including how they became involved, their impressions of the experience, what they gained from participating, and suggestions for improvement. The interviewers were students who were not involved in development or implementation of the program. A field journal was also maintained to track the implementation process, from recruitment through data collection and analysis.
Data Analysis
An interpretive description approach was used to analyze qualitative survey responses, interview transcripts, and data regarding program implementation. Interpretive description involves identifying themes and patterns within subjective data to generate findings that inform clinical understanding and application. 14 Dedoose software 15 was used to manage the data. Data were initially coded in an iterative process of categorizing comments that reflected each of the 4 process factors. Double coding of initial interviews was completed by members of the research team to ensure consistency in coding practices. Comments within each category were then reviewed to identify key experiences for participants in each program and the similarities and differences across programs.
Findings
Findings from all data sources were compiled to explore the impact of 4 key dimensions of program implementation: (1) the innovation itself (i.e., content and format), (2) the program participants, (3) the program leaders, and (4) the organizational context. A summary of quotes from participants in each program is provided in Table 3.
Comparing Perspectives on Program Characteristics.
Innovation Content and Format
Both programs provided 12 hours of education related to mental health and mental illnesses; however, as noted on Table 2, there were differences in content, format, and scheduling. Many MHFA survey respondents indicated that the course content met their expectations of a basic introduction to mental health and illness. Some liked the structure of reviewing each type of illness or “disease process”; others, particularly those with a clinical background, indicated that the information was a “refresher” about various mental health conditions. A few clinicians complained that the information was too basic with limited opportunity for new learning. Another consistent comment from the MHFA participants was that they would like more examples and case scenarios that were relevant to their workplace. In contrast, the Beyond Silence participants talked about how much they valued the opportunity to reflect on and discuss issues and scenarios that were directly relevant to their day-to-day work.
Sharing personal stories was emphasized by all participants as having the greatest impact on their learning. Even though it was not a formal part of the MHFA program, it was something that participants sought out. Many of the Beyond Silence participants referred to the personal stories they heard (this was a core element of the program), including opportunities for discussion on a weekly basis. The personal stories reportedly increased their insight into what colleagues may be experiencing and also helped to combat the stigma associated with the condition. This process challenged their preconceived ideas about mental illness and had a lasting emotional impact on them. Although the opportunity for sharing stories was typically identified as a valued part of the program, it is important to note that there were a few exceptions. For example, one male survey respondent noted, “For the introverts, [it was] a little too much required participation.” Since the information was deeply personal at times, it was not necessarily easy for them to hear about or share experiences.
In addition to the content, there were comments about the program schedules, including both benefits and challenges of the 2-day format (MHFA) versus the 6 biweekly sessions (Beyond Silence). Dropout rates were higher for the Beyond Silence program (34% as opposed to 20% in MHFA), since it was difficult to commit to evening meetings over a 3-month period. On the other hand, some reported that they valued the time to reflect upon the information and apply it in their work life. There were many requests for additional sessions once the training program was complete. They felt that “refresher” sessions could provide an opportunity to “try out” different strategies and maintain their motivation to apply the principles in their day-to-day work.
Program Participants
Participants in both programs came from all areas of the organization, including clinical and nonclinical positions, as well as both managers and front-line workers. The heterogeneity of participants, including age, position in the organization, work and personal experiences, was noted by all of the interviewees. Many reported an appreciation of this diversity; it led to enhanced learning from multiple perspectives. The different roles and experiences reportedly facilitated a rich conversation and exchange of ideas, and they valued the opportunity to connect with staff from across the organization. The Beyond Silence participants, in particular, had more opportunity for dialogue about workplace mental health issues, noting the value of exchanging ideas between front-line workers and managers. One of the managers indicated that it was valuable to hear the viewpoint of front-line staff as well as the union/nonunion points of view. In addition, front-line workers found it valuable to hear the perspective of managers.
Service Providers
Facilitators in both programs were employees with specialized training to deliver the program. One key difference is that the Beyond Silence program was co-led by 2 peer educators, whereas MHFA was led by 1 trained facilitator who did not share personal experiences. Crossover in leadership and small numbers in each group prevented quantitative comparison of outcomes based on program facilitators; however, comments from the survey and interview data highlighted important similarities and differences.
Facilitators in both programs reportedly played an important role in sharing information, as well as creating a positive, engaging learning environment. One notable difference was that the Beyond Silence participants talked about the significant impact of leaders who were able to share their personal experiences in an honest and authentic way. They noted the leaders’ ability to use these personal experiences to create an open dialogue about mental health issues in a supportive, nonjudgmental way and that their advice had more impact since “they’ve been there.” In contrast, several MHFA participants suggested that the program could be enriched by including leaders or speakers with personal experiences, noting it “makes a difference to have some personal knowledge…‘cause then you’re more passionate about it.”
The Beyond Silence co-leader model was set up so they had complementary areas of expertise and personal characteristics (e.g., male/female, manager/front-line worker). We trained 5 co-leaders, so they could fill in for each other in the case of vacation or illness. We also held monthly meetings to proactively support them in their role.
Social/Organizational Context
The programs were offered in 2 different health care organizations. Quantitative analysis noted no differences in outcomes between the 2 organizations (publication forthcoming). From a qualitative perspective, however, there were important differences in implementation. In the first organization, there was a history of collaboration with leaders in Health Safety and Wellness, support from senior leadership, and an advisory team of union representatives, managers, and front-line workers from both clinical and nonclinical areas. This level of engagement facilitated communication for recruitment, implementation and evaluation. In the second organization, time was needed to establish new relationships, channels of communication, awareness, and trust, as well as a positive reputation for the educational initiatives. Information was shared at senior leadership meetings to promote awareness and managerial support (e.g., flexible scheduling) for their employees to attend.
Although participants in this study volunteered to attend sessions outside of working hours, many participants suggested mandatory attendance, particularly for employees in formal or informal leadership positions. They emphasized the importance of training to address the challenges of reaching out to provide support and suggested inclusion in employee orientation. Although some participants were proactive in sharing resources and information with their team, others found it more difficult and discussed the importance of leadership “buy-in” for real change to occur. One participant, for example, noted that she had changed but that broader system change is still needed. This is a stigma that needs to be broken by our corporation when it comes to staff. We are quick to help patients who may be suffering for these issues but are not as compassionate with our own coworkers when they are suffering from the same sort of issues.
Discussion
The ultimate goal of both training programs was to promote early intervention and support for health care employees who may be struggling with mental health issues. Based on the study findings, 5 key design principles appeared to shape the perceived impact of the programs: (1) contact-based education, (2) contextually relevant information, (3) an opportunity to explore varied perspectives, (4) sufficient time to integrate and apply learning, and (5) organizational readiness/support.
Our findings are congruent with current research on the value of contact-based education as a best practice approach in changing attitudes about mental illness. 16 Many studies have documented the value of positive, voluntary, prolonged, personal contact with a respected peer who has a mental illness on reducing stigma. 6,16,17 In workplace training, the nature of “contact” could range from videos to in-person stories about mental illness and recovery. In our study, the Beyond Silence peer educators were intentionally hired to facilitate opportunities for contact-based education, and there was clearly a positive response to their role. Participants valued the leaders’ ability to share authentic, relevant experiences in a way that created an open, safe space for discussion and learning. The other, more unexpected response was the personal sharing that happened between participants; many shared stories about themselves, family members, and colleagues. By creating time and space for these conversations to happen, the boundaries between “us and them” were challenged, and a deeper understanding emerged regarding workplace mental health as a universal, important issue for all employees to understand and address. The impact of this approach led not only to a change in knowledge but also to significant changes in attitudes and beliefs, as noted in both quantitative and qualitative findings.
Another key finding relates to the value of contextually-relevant content. This finding is consistent with principles of adult education, which stipulates that learners are interested in learning about subjects that have immediate relevance and impact to their job and that experience provides that basis for learning. 18 Many current training approaches such as MHFA are designed to be implemented across many contexts, including the workplace. 2 Although some issues may be relevant across a range of sectors and settings, our findings suggest there is value in identifying unique issues in particular workplace settings. Beyond Silence was designed for health care; a publicly funded, service-oriented sector with a female-dominated workforce. It may not be relevant in the manufacturing sector, which has a private, for-profit orientation and predominantly male workforce. In designing and implementing training, it is important to consider how the terminology, examples, leaders, and even the amount of time spent on specific issues should be tailored to the specific learning needs of employees. The workplace culture, as well as cultural and educational backgrounds of employees, can affect their training needs.
An opportunity to explore varied perspectives was another active ingredient of both training programs. Many training programs are designed exclusively for managers since they are responsible for managing performance when employees are struggling. 2 There is increasing attention, however, on the important role of colleagues in identifying and responding to mental ill health at work. 19,20 Workplace are social spaces with many different personalities and relationships that can shape responses to employees who may be struggling. In our study, diversity in roles and personal mental health experience provided an opportunity to explore the diversity of perspectives regarding the nature of the issues and perceived responsibilities for action. As noted by our Beyond Silence participants, the opportunity to exchange diverse perspectives enhanced understanding and empathy.
Sufficient time to integrate and apply learning was another key finding. Many training programs are typically a few hours up to 2 days in length 2 ; this format is efficient but may not be sufficient to sustain learning over time. Knowledge can be transmitted fairly quickly, but attitude and behavior change takes time. The Beyond Silence participants brought real workplace examples to the session for discussion and appreciated time between sessions to integrate and apply the information they were learning. Unfortunately, busy work and personal lives can interfere with a long-term training commitment, and missing sessions detracts from optimal learning. Additional research is needed regarding the value of periodic “refresher” sessions or technology-based support for continued application of the ideas. In addition, further study is needed to track behavior change over time.
The final critical ingredient for successful implementation is organizational readiness and support. In both programs, support at all levels of the organization, including high-level executives, front-line managers, union representatives, and health and safety providers, was important in promoting participation and integration of ideas into day-to-day work practice. This finding is congruent with findings of a national case study project on implementation of standards of psychological health and safety. 21 Their keys to successful implementation included ongoing leadership support and involvement, adequate structure and resources, and commitment throughout the organization. It is not only the program itself but also how information is shared with and interpreted by employees. Support and/or incentives may be needed for coworkers to reach out to others, particularly since it is not a required part of their role. “Organizational citizenship” literature highlights strategies to engage employees in voluntary, positive action that supports coworkers and benefits the overall organization. 22
Overall, the active ingredients for workplace mental health training are relevant to design and implementation beyond the 2 programs in this study. In comparing the 2 programs, however, the Beyond Silence participants received more contact-based, contextually relevant information and had more time to integrate and apply learning. As such, the Beyond Silence program appeared to provide added value to participants.
Study Limitations
Generalizability is limited by the small sample of participants who were likely the early adopters and motivated to participate in workplace mental health training. Most participants were female clinicians, which is not unusual in health care; however, learning needs might be different for male respondents and those in nonclinical positions. The study was conducted in 1 medium and 1 large hospital in the same urban center. Since the organizational context can affect implementation and outcomes, further research is needed to explore application in other health care settings. Strengths of the study include high survey completion rates, as well as triangulation of data sources and methods, which enhanced depth of understanding and credibility of the findings.
Conclusion
The “active ingredients” of implementation noted in this study complement the randomized controlled trial outcome data. General principles for implementation have been identified, including the importance of participant composition, scheduling, and organizational commitment. In addition, specific elements of Beyond Silence, including contact-based education, contextually-relevant content, and prolonged contact, reportedly enhanced integration and application of information. Additional research is needed to explore application in other settings and how this training should be embedded within broader organizational initiatives to address psychological health and safety.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: this research study was funded by the Ontario Mental Health Foundation.
