Abstract
Background:
The National Institute of Environmental Health Sciences Worker Training Program developed instructor curriculum, a training tool, and materials to prepare trainees to conduct opioids workplace awareness training. Opioids and the Workplace Prevention and Response (OWPR) Train-the-Trainer (TTT) courses were held during three instructor-led online programs.
Methods:
The OWPR TTT was evaluated using an online pre- and posttest among registered participants, and an assessment of discussion among trainees during the training program.
Results:
A total of 31 pretests and 24 posttests were completed for the three TTT courses conducted. Quantitative and qualitative data complemented each other in supporting the conclusion that the training achieved intended objectives.
Conclusion/Applications to Practice:
The evaluation illustrated that the training was effective in increasing knowledge and confidence in conducting awareness training on opioid use and addiction prevention.
Background
The National Institute of Environmental Health Sciences (NIEHS) Worker Training Program (WTP) recognized the need for training workers, their representatives, and employers on the prevention of opioid use and addiction (NIEHS, 2018). NIEHS WTP developed an awareness level Opioids and the Workplace Prevention and Response (OWPR) training tool and materials, which were piloted in the summer of 2019 (NIEHS, 2019). According to participant evaluations, the OWPR piloted trainings were found to be delivered effectively and increased knowledge related to opioids use and addiction prevention (NIEHS, 2019).
A voluntary survey and interview were conducted with the OWPR pilot trainees 6 months posttraining (Persaud, Geer, et. al., under peer review). Results from the 6-month follow-up found individual-level actions, such as shared fact sheets and information from opioid training with co-workers, to be reported more frequently than organizational-level actions, for example, evaluated use of employee assistance programs. An OWPR TTT program was then developed and piloted in the summer of 2020, which produced trainers who could deliver the OWPR awareness training to workers (Persaud, Geer, et. al., under peer review). The TTT was delivered online, as virtual instructor-led training, to maximize trainer and trainee safety due to the COVID-19 pandemic. Persaud, Geer, et. al., under peer review indicated that the OWPR TTT was effective in improving participants’ ability to conduct opioid awareness training (Persaud, Geer, et. al., under peer review).
The WTP continues to evaluate the OWPR TTT program on an ongoing basis. This study reports on additional OWPR TTT programs after the summer 2020 pilots. Based on initial evaluations, we hypothesize that the TTT programs continued to increase knowledge and confidence in delivering OWPR training among participants.
Methods
The TTT programs were held online using the Zoom platform from November 16–19, 2020, for the United Association of Journeymen and Apprentices of the Plumbing, Pipefitting and Sprinkler Fitting Industry of the United States, Canada (UA); on February 24 and 26, 2021, for the Midwest Consortium for Hazardous Waste Worker Training (Midwest); and on July 7, 2021, for the International Association of Machinists and Aerospace Workers (IAM).
The length of the training varied by training provider. The UA training was 20 hours total. The Midwest training was 14 hours total. The IAM training was 5 hours total. All courses had time for breaks and a 1-hour lunch. The length of training was based on the predetermined allotted time by the training providers and not impacted by the results of the evaluation.
At registration, trainees were emailed an online survey link through Qualtrics by the evaluator. The site organizer followed up on the evaluator email by clarifying the evaluator’s involvement in the training and to ask trainees to participate in the voluntary survey. The survey had 14 items for awareness that were yes–no and 7 items for TTT objectives that were on a Likert-type scale of “strongly agree” to “strongly disagree.” The opening page of the survey link contained a consent form with detailed descriptions of the evaluation study. If the participant consented by clicking “I agree to participate in this study,” they were taken to the quiz items or “pretest.” After the training, the participants who attended were emailed the same quiz items as a posttest by the evaluator. The posttest contained additional items addressing the process of the delivery of the training. Demographic questions were included at the end of the survey, including gender, age, highest education level, and race/ethnicity. Previous validation of the training tool and evaluation items is discussed elsewhere (Persaud et al., 2021).
The evaluator collected qualitative data on what trainees said during the training, the discussion of small group activities, the discussion of group activities as a class, and what trainees typed into the group chat or typed onto the screen during participatory exercises. The collected data were categorized into common themes as it related to awareness and instructing the course. No identifying information was collected during the notetaking, and any qualitative data that may identify a trainee had been removed or paraphrased.
The quantitative data were analyzed by calculating the changes in percentage agreement between the pre- and posttest responses for each training and overall. Changes in percentage of 15% or greater in agreement with the correct response were highlighted in bold. Chi-square analysis was used to compare the pre- and posttest proportions by item. The Fisher exact test was used for items with 100% agreement in posttests to calculate p values. As the pre- and posttest scores could not be matched by individual, only group proportion statistics were available and were analyzed by the chi-square test or Fisher exact test.
The results of the qualitative and quantitative analyses were integrated together into a joint table for assessment of consistency between these results, and to help generate lessons learned. All quantitative analyses were performed using IBM SPSS Statistics Version 27.
The study was reviewed by the State University of New York (SUNY) Downstate Health Sciences University Institutional Review Board and granted an exemption under study number 1605399-1.
Results
There were 33 persons who registered and attended, 31 completed pretests, and 24 completed posttests. Based on self-reported demographics, most trainees were male (70.8%), between the ages of 36–50 (36.4%) and 51–64 (50.0%), White (81.8%), and had at least some college (45.5%) or graduate school (45.5%) as highest level of education.
Percent agreement with the correct response during pretest and posttest and change in percent for all three trainings combined are shown in Table 1. Questions pertaining to instructor confidence in teaching course material, items 15 to 21, were all statistically significant in change from pre- to posttest.
Quiz Percent Agreement With Correct Response During Pretest and Posttest and Change in Percent, n = 31 for Pretest, n = 24 for Posttest Training Participants
Note. Changes pre–post greater than 15% or statistically significant changes are in bold.
Fisher exact test was used for posttests having 100% agreement, with the p value included in the table.
Trainees described how stressors related to COVID-19 and stigma related to substance use disorder and addiction are impacting opioid awareness education and training, for example, “stigma was coming down, then COVID hit. . .all those guardrails to help are gone.”
This is a workshop with tremendous information. Unfortunately, we only have an hour and a half and we have to boil it down. That’s about the limit. With the pandemic and all those elements we have to keep that in mind.
Participants mentioned how the online learning platform can be both a barrier and helpful, for example, “so much of this we are going to adapt to remote learning” and “If we have zoom meetings, we can use this technology to reach people who wouldn’t attend an in-person meeting.”
Train-the-Trainer quantitative and qualitative data were compared using a joint display in Table 2.
Joint Display Table of Train-the-Trainer Qualitative and Quantitative Results
Discussion
The NIEHS WTP OWPR Train-the-Trainer training tool and materials were developed to increase trainees’ knowledge and confidence in conducting training and education on opioid use and addiction prevention in the workplace. Quantitative and qualitative data complemented each other in supporting the hypothesis that training effectively improved intended objectives. With the Zoom platform being used, some trainees expressed challenges in not having previous or having limited experience in performing online training. Orientation before training starts or pretraining to review the platform with technology tips may help improve trainees’ confidence and comfort in using virtual education. Some participants described how using an online platform can help reach a wider audience and bring together more trainees who would not have previously been able to attend an in-person-only meeting.
Participants discussed actions they can take following the training. Most trainees, being instructors, described how they can use the TTT training tool and materials to deliver opioid prevention training. However, the training could not be delivered as is and would generally require some revisions to shorten it to allow for available training time.
Opioids and the Workplace Prevention and Response Train-the-Trainer Course Description
The OWPR TTT focused on preparing trainees to conduct opioid awareness training back at their organizations across various industries and occupations. The OWPR TTT content included sections dedicated to reviewing the OWPR awareness training topics on: the background of the epidemic, fentanyl and synthetic opioids, understanding opioid use disorder, stigma, prescription opioids, related infectious disease, occupational exposures, opioids and work, prevention: identifying program gaps and risk factors, employee assistance and peer assistance programs, and workplace substance use prevention programs. Accompanying the TTT courses was an instructor manual that reviewed the training content and suggestions for delivery of course materials and how the resources can be customized to suit specific training needs and target populations. The OWPR training content is available online for free download to the public, including the awareness OWPR training content and OWPR TTT resources (National Institute of Environmental Health Sciences, 2021). The TTT course objectives were for participants to be able to:
Discuss the scope and severity of the opioid crisis.
Summarize the relationship between workplace injuries and illnesses, working conditions, and opioid use disorder.
Identify occupational exposure, prevention, and response.
List actions that might be taken at the workplace to prevent and respond to opioid use and misuse.
All of the evaluation questions showed a positive change in agreement with the correct response from pre- to posttest. There were 14 of 21 items that either had a significant or substantial increase in agreement with the correct response from pre- to posttest. In six of the items that did not have a substantial or significant increase from pre- to posttest, there was a high pretest agreement of 80% or greater. Therefore, there was little room for change in those evaluation items. Trainees generally had high pretest agreement with topics of stigma, naloxone, and peer advocacy. However, the pretest agreement was low for most other evaluation questions, including items related to course objectives. This may suggest most trainees had a good knowledge of the background of the opioid epidemic but were limited in knowledge on prevention and response strategies in the workplace as well as confidence in teaching opioid awareness. The OWPR TTT may help improve knowledge of opioid use prevention and response workplace policies and programs and help improve instructor skills among trainees.
A limitation to this study included an inability to pair pretest with posttest responses due to concerns of confidentiality, therefore we could not measure changes at an individual level. A strength of the study included the ability to collect and compare both qualitative and quantitative data. Future evaluations should include intermediate and long-term outcome measures, such as ongoing confidence with teaching the material, worker-level awareness improvements, reduced incidence of opioid use and addiction in workplaces where training took place, as well as overall reductions in injuries.
Implications for Occupational Health Practice
The workplace continues to provide an important venue for intervention in addressing the opioids crisis. This Train-the-Trainer was a tool for increasing capacity to deliver awareness level trainings in the workplace.
Applying Research to Occupational Health Practice
The NIEHS WTP OWPR Train-the-Trainer training tool and materials were developed to increase trainees’ knowledge and confidence in conducting training and education on opioids and the workplace. The training was delivered in a manner that was designed to increase participants’ knowledge and confidence in conducting OWPR training. Evidence gathered through posttraining evaluation support the hypothesis that the objectives of the training tool and subject matter were met.
Footnotes
Acknowledgements
The authors acknowledge and thank the National Institute of Environmental Health Sciences Worker Training Program for their review and feedback throughout the study and program evaluation. This work was made possible by contract number 47QRAA20D0028 from the National Institute of Environmental Health Sciences.
Author Contributions
All authors substantially contributed to the concept of the work, interpretation of the data, and drafting of the manuscript.
Conflict of Interest
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Dr. Persaud received financial compensation to perform “Opioids and the Workplace Training Evaluation” as a consultant to MDB, Inc. The original source of the funds was the National Institute of Environmental Health Sciences. The time period of the funding was September 20, 2020 to September 19, 2021. Dr. Persaud is continuing to evaluate the program for MDB, Inc. at the time of submission. Ms. Weinstock is a senior staff member of MDB, Inc. and the program manager for the contract under which the work was performed at the time of submission. All other authors have no conflict of interest to declare.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was made possible by contract number 47QRAA20D0028 from the National Institute of Environmental Health Sciences, NIH.
