Abstract
Background: Due to the high prevalence of prescription opioid misuse, the US Food and Drug Administration (FDA) mandated a Risk Evaluation and Mitigation Strategy (REMS) requiring manufacturers of extended-release/long-acting (ER/LA) opioids to fund continuing education based on an FDA curricular Blueprint. This paper describes the Safe and Competent Opioid Prescribing Education (SCOPE of Pain) train-the-trainer program and its impact on (1) disseminating the SCOPE of Pain curriculum and (2) knowledge, confidence, attitudes, and performance of the participants of trainer-led compared with expert-led meetings. Methods: SCOPE of Pain is a 3-hour ER/LA opioid REMS education. In addition to expert-led live statewide meetings, a 2-hour train-the-trainer (TTT) workshop was developed to increase dissemination nationally. The trainers were expected to conduct SCOPE of Pain meetings at their institutions. Participants of both the trainer-led and expert-led SCOPE of Pain programs were surveyed immediately post and 2 months post meetings to assess improvements in knowledge, confidence, attitudes, and self-reported safe opioid prescribing practices. Results: During 9 months (May 2013 to February 2014), 89 trainers were trained during 9 TTT workshops in 9 states. Over 24 months (May 2013 to April 2015), 33% of the trainers conducted at least 1 SCOPE of Pain training, with a total of 79 meetings that educated 1419 participants. The average number of meetings of those who conducted at least 1 meeting was 2.8 (range: 1–19). The participants of the trainer-led programs were significantly more likely to be practicing in rural settings than those who participated in the expert-led meetings (39% vs. 26%, P < .001). At 2 months post training, there were no significant differences in improvements in participant knowledge, confidence, attitudes, and performance between expert-led and trainer-led meetings. Conclusions: The SCOPE of Pain TTT program holds promise as an effective dissemination strategy to increase guideline-based safe opioid prescribing knowledge, confidence, attitudes, and self-reported practices.
Introduction
In the United States, there has been a dramatic increase in opioid prescribing for chronic pain, with an associated rise in unintentional opioid overdoses. 1 Prescriber education has the potential to both improve safe opioid prescribing practices and maintain access to opioid analgesics for those patients that may benefit from them. 2 In July 2012, the US Food and Drug Administration (FDA) approved a single shared Risk Evaluation and Mitigation Strategy (REMS) requiring manufacturers of extended-release/long-acting (ER/LA) opioid analgesics to jointly fund Continuing Education (CE) based on an FDA curricular Blueprint for Prescriber Education. 3 The FDA developed core messages to be communicated to prescribers in the Blueprint, published a draft for public comment, and considered the public comments when finalizing the Blueprint. It was approved as part of the ER/LA opioid analgesic REMS and is used by CE providers to develop the actual educational activity, which must be produced independent of commercial influence. The goal of this REMS was to educate through certified medical education at least 192,000 ER/LA opioid prescribers (60% of the 320,000 total active prescribers) within 4 years from the release date of the first REMS-compliant training.3
After the launch of the first REMS activity developed by Boston University School of Medicine (BUSM) in February 2013, the manufacturers of ER/LA opioids awarded 31 educational grants to accredited medical education providers in order to meet this goal. Each organization that received a grant was tasked with developing education to disseminate the ER/LA opioid REMS content to as many clinicians as possible. Although this REMS program has trained tens of thousands of prescribers, it has not yet achieved its target numbers of prescribers trained.4
BUSM's ER/LA opioid REMS program entitled Safe and Competent Opioid Prescribing Education (SCOPE of Pain), the first funded program, has been found to be effective in improving knowledge, attitudes, confidence, and self-reported clinical practice in safe opioid prescribing. 5 The program was disseminated through a series of online modules, as well as live expert-led meetings. Although expert-led education is the traditional model of continuing medical education, 6 for this REMS initiative, where the content is predetermined by the FDA and there is a need for quick and widespread national dissemination, alternative modalities beyond expert-led live meetings are necessary. Online education was an obvious choice, but it is not optimal for the learner who prefers to learn in live settings and does not provide the same level of personal interaction with colleagues and faculty that live meetings can offer. 7 – 10 The question still remained how to disseminate this content efficiently and yet preserve the benefits of live meetings. With limited numbers of experts who are available to lead trainings at multiple small venues, a train-the-trainer (TTT)-type model could be optimal for expanding the reach of the programs beyond urban centers. 11 , 12 This study was designed to compare 2 different dissemination strategies of the ER/LA opioid REMS (SCOPE of Pain) education. We evaluated whether a trained trainer (trainer-led) would enhance program dissemination to a more rural audience and achieve similar educational outcomes as the expert-led education.
Methods
SCOPE of Pain train-the-trainer (TTT) description
SCOPE of Pain is a 3-hour live or online education available at www.scopeofpain.org and is based on the FDA curricular Blueprint. 3 In addition to the 20 expert-led live meetings and an online program that have been described in a previous publication, 5 a TTT model was developed to increase the reach of the program beyond the urban areas in which they are held. In this TTT model, all registrants of the expert-led sessions were sent e-mail invitations to apply to be a trainer for SCOPE of Pain. The TTT Web-based application included open-ended questions about their teaching experience, why they were interested in becoming a trainer, and who their proposed target audience was. In addition, each applicant uploaded their curriculum vitae and completed a disclosure form to capture any possible conflicts of interest. Appropriate candidates were selected based on their level of teaching experience, motivation to become trainers, and their lack of any relationships with commercial interests. Their motivation was measured by evaluating their answers to the question about how they would implement the training in their local area. Enthusiastic and well-thought-out plans were criteria used to gauge motivation to become a trainer.
The selected trainers attended a 2-hour TTT workshop that was held following the expert-led live SCOPE of Pain meeting. The TTT workshops focused on strategies to teach the SCOPE of Pain curriculum at their home institutions with the use of an Action Plan process to define their target audience, setting, timing, resources, and potential challenges to leading SCOPE of Pain meetings. The workshop was led by the SCOPE of Pain course director (D.P.A.), who is experienced in safe opioid prescribing education. Trainers were offered logistical support from the SCOPE of Pain staff to plan and run the live meetings along with a $300 honorarium per meeting. The support included a dedicated Web site to download educational materials (e.g., PowerPoint slides), a project coordinator available to help with scheduling, educational material printing, and participant outreach after the training was completed. In addition, SCOPE of Pain experts were available if trainers had any questions about the SCOPE of Pain content or needed assistance answering questions from their participants.
Outcomes
Measuring impact of expert-led and trainer-led meetings
Comparative evaluation of the expert-led and trainer-led meetings included 2 data collection time points: (1) immediate post program (IMMED) and (2) 2 months post program (2MO) (Figure 1). It was not possible to deploy a pre-post study design because participants of the trainer-led group did not preregister and logistically were unable to complete the pre-test on the day of the meeting. Multiple steps, described in a previous publication, have been taken to ensure the evaluation had face and content validity, was REMS compliant, and had clinical relevance to respondents. 5 The evaluation was designed to ensure assessment of changes in the participants’ knowledge, attitudes, confidence, and clinical practice to demonstrate alignment with the best practices described in the FDA Blueprint. For the trainer-led meetings, since there was no pre-test, this was accomplished by asking participants to self-report their own changes in attitudes, confidence, and practice. For knowledge, a minimum standard of receiving a score of 70% or higher enabled the assessment of basic knowledge and comparison of IMMED and 2MO determined maintenance.
Comparative evaluation of expert-led and trainer-led SCOPE of Pain: structure, data collection points, and outcome metrics.
For both expert-led and trainer-led meetings, 4 key metrics (knowledge, confidence, attitudes, and clinical practice) were measured. These metrics are explicitly linked to the ER/LA opioid REMS FDA Blueprint, each including multiple quantitative items to fully cover the SCOPE of Pain content.
Participant knowledge (Key Metric 1) was assessed through 20 IMMED knowledge-testing items, 10 of which were repeated at 2MO. The 10 that were repeated at the 2MO facilitated assessment of participant clinical performance change, which can only be assessed effectively after participants have time to go back to their practices and implement changes. Knowledge-testing questions were a combination of dichotomous true/false questions, item-matching questions, and multiple nominal choice questions. Participant confidence to manage patients with chronic pain (Key Metric 2) was assessed with 6 self-report items with Likert-type response formats at 2MO. Items were based on common confidence issues faced by clinicians in the use of opioids when managing chronic pain. Participant attitudes when treating patients with chronic pain and using guideline-based care (Key Metric 3) were assessed using 13 self-report items with Likert-type response formats, including questions of motivation and willingness to treat patients with pain at 2MO. Finally, changes in participant self-reported clinical practice using guideline-based care (Key Metric 4) was assessed with 7 Likert-type response format self-report items about “commitment to change” 13 clinical performance (IMMED) and self-reported change (2MO), as well as 2 multiple nominal choice items about practice changes implemented and barriers to implementation (2MO). The BUSM institutional review board (IRB) determined this evaluation to be exempt from further IRB review.
Participant recruitment for expert-led and trainer-led meetings
Health care providers who manage patients with chronic pain and are licensed to prescribe opioids analgesics were the primary target group for the SCOPE of Pain program, and the audience of interest for the overall REMS initiative. This group included physicians, advanced practice nurses, and physician assistants licensed to prescribe ER/LA opioids practicing in primary care and specialties identified as managing patients with chronic pain. (See full list in Table 1.) Participants who were excluded from the study were either clinicians who do not manage chronic pain as part of their specialty practice (e.g., emergency medicine, psychiatry) or are not prescribers but worked with prescribers and were often responsible for creating and/or maintaining the systems needed for safe opioid prescribing (e.g., registered nurses, pharmacists). So although we felt that it was very important for these other health professionals to participate in SCOPE of Pain, generally it is the clinicians who prescribe opioids who would be the key respondents in determining the effect of the program. Given the focus of the REMS program, only the outcomes assessment completed by the aforementioned target group are included in the analyses for this paper.
Expert- and trainer-led SCOPE of Pain participant characteristics.
Distribution is significantly different (chi-square: P < .001 at IMMED, P < .001 at 2MO).
For both the expert-led and the trainer-led meetings, completion of the IMMED was a requirement to receiving continuing education credit. For the 2MO assessment, an automatic e-mail was sent to all participants at 60 days using the e-mail address they had provided to the Web site either at registration or when claiming credit. Reminders were sent to noncompleters at 63 and 66 days. An incentive (entering a drawing for an e-book reader) was provided to encourage completion of the 2MO.
Analyses
Frequencies and cross-tabulations were calculated for each item using IBM SPSS 22.0 software (IBM Corporation, Armonk, NY). Comparison of trainer-led versus expert-led participants’ responses was done using t tests for the continuous variables (percentage of correct answers to the knowledge questions, and number of changes intended/reported) and chi-squares for categorical variables (all other variables examined). In order to have sufficient statistical power to detect a difference between the trainer-led and expert-led groups, considering a medium effect size w = .3, an error α = .05, and a statistical power 1 − β = .8, a minimum of 133 respondents are required for the chi-square on the variable using highest number of potential values (5). Considering the same parameters, 82 respondents are required for the t test on the percentage of correct knowledge answers.
Results
TTT workshops and trainer-led meetings
During 9 months (May 2013 to February 2014), 89 trainers were trained during 9 TTT workshops in 9 states. Over 24 months (May 2013 to April 2015), 33% of the trainers conducted at least 1 training, with a total of 79 trainings training 1419 participants from the target group. The average number of trainings of those that conducted at least 1 training was 2.8, with a range of 1 to 19. At the same time, BUSM conducted 20 live expert-led meetings in 17 states educating 1742 participants from the target group.
Expert and trainer faculty description
The SCOPE of Pain expert faculty were all physicians who are experienced and accomplished educators in safe opioid prescribing for chronic pain, including experience presenting safe opioid prescribing education at national meetings and/or publishing on safe opioid prescribing and chronic pain in peer-reviewed journals. By contrast, the trainer faculty came from a variety of professions, including 50% physicians, 20% nurse practitioners, 12% pharmacists, and 10% physician assistants and with different levels of clinical pain expertise, ranging from none to fellowships in pain medicine. In addition, the trainers’ level of experience as educators varied from none to extensive, with all having minimal to no experience training others in safe opioid prescribing for chronic pain. All the trainers had a strong commitment and a sense of responsibility to train others to simultaneously address both chronic pain and the prescription opioid misuse problems at their institutions and in their communities. Although we were unable to assess the trainers’ competency, we did attempt to ensure content fidelity of their trainings by asking all trainers to sign a form committing to presenting all the Blueprint content. In addition, we were able to audit a random selection of trainer-led programs to assess that the entire SCOPE of Pain content was delivered.
Participants of trainer-led and expert-led trainings (Figure 2, Table 1)
A total of 3161 participants completed either the expert-led or trainer-led SCOPE of Pain meetings between February 28, 2013, and June 16, 2015. Twenty-nine percent (913/3161) were considered the primary target group for SCOPE of Pain (defined as being a physician, an advanced practice nurse, or a physician assistant from 1 of the 13 disciplines and specialties listed in Table 1 and being licensed to prescribe ER/LA opioid analgesics). Completion was defined as participants who completed the IMMED assessment after attending the SCOPE of Pain meeting.
Expert- and trainer-led SCOPE of Pain: primary target group participants and evaluation sample.
The IMMED assessments were completed by 489 participants of the expert-led meetings and by 424 participants of the trainer-led meetings. Of those, 26% (128/489) of the expert-led and 17% (70/424) of the trainer-led groups completed the 2MO assessment. Table 1 presents the sociodemographics for the final samples of the comparative evaluation (primary target participants who completed SCOPE of Pain either through expert-led or trainer-led meetings). Demographic distribution between the 2 groups was significantly different with regards to their practice setting with 39% of participants (163) who attended the trainer-led meetings were from rural areas compared with only 26% (125) who attended the expert-led meetings.
The following section presents the findings divided into 3 sections: (1) Assessment of knowledge comparing expert-led and trainer-led IMMED and 2 MO assessments; (2) Intention to change comparing expert-led and trainer-led IMMED assessments; and (3) Assessments of attitude, confidence, and practice (patient communication, guideline-based care, and barriers to change) comparing expert-led and trainer-led in 2MO assessments.
Knowledge (Table 2)
The average proportions of correct responses in the IMMED were 92% for the expert-led group and 91% for the trainer-led group, whereas in the 2MO they were 77% for the expert-led group and 78% for the trainer-led group. The differences between the 2 groups were not significant (IMMED, P = .091; 2MO, P = .538).
Knowledge assessment of expert- and trainer-led SCOPE of Pain participants immediate and 2 months post training.
n.s. = nonsignificant (t test).
Intention to change
Immediate post program, 93% of expert-led participants and 91% of trainer-led participants stated they were planning to make at least 1 change to align their practice with guideline-based care. There were significant differences between the 2 groups in the number of changes selected in the “check all that apply” list provided, with members of the expert-led group agreeing to 4.1 changes versus 3.5 changes for the trainer-led group (P < .001).
Confidence (Table 3)
For all 6 confidence items, approximately two thirds of the participants in each group reported increased confidence in guideline-based opioid prescribing practices. There were no significant differences between the 2 groups.
Changes in participant confidence of expert- and trainer-led SCOPE of Pain at 2 months.
Actual n for each item might vary slightly due to nonresponse. Percentages were calculated excluding nonrespondents for each item.
n.s. = nonsignificant.
Attitudes (Table 4)
A majority of items showed no significant differences in the attitudes between the 2 groups, with 2 exceptions, the item “It is my responsibility and role to discuss with my patients not to give away their medications to relatives or friends” (97% agreement [expert-led] and 91% [trainer-led]; P = .028) and the item “I cannot get my patients to be truthful about illicit drug use” (13% agreement [expert-led] and 21% [trainer-led]; P = .025).
Attitudes of participants of expert- and trainer-led SCOPE of Pain at 2 months.
Scale: 1 = Strongly disagree to 5 = Completely agree.
Actual n for each item might vary slightly due to nonresponse. Percentages were calculated excluding nonrespondents for each item.
n.s. = nonsignificant.
Clinical practice (patient communication, guideline-based care, and barriers to change)
Patient communication (Figure 3)
For the 7 clinical practice behaviors related to patient communication, although trainer-led groups reported slightly lower endorsements of guideline-based behaviors than the expert-led groups, no significant differences were observed between the groups. The item “Give my patients a patient counseling document and tools as part of the discussions with them when prescribing opioid analgesics” was the least reported behavior, as 59% of expert-led participants and 54% of trainer-led participants reported performing the behavior at all, and only 28% of expert-led and 29% of trainer-led performing the behavior with “all pain patients.” At the other end of the spectrum, the item “Inform my patients about taking medication exactly as prescribed” was reported to be performed by 99% of expert-led and 94% of trainer-led and performed with “all pain patients” by 81% of expert-led and 73% of trainer-led of 2MO respondents.
Self-reported patient communication strategies of participants of expert- and trainer-led SCOPE of Pain at 2 months
Guideline-based care (Figure 4)
Between one third and two thirds of respondents for both the expert-led and trainer-led groups reported having either partially or fully implemented or improved the implementation of the 9 specific clinical practice changes at 2MO. The highest implementation/improvement rates were observed in regards to documentation in patient medical records relating to opioid prescribing: 70% of the expert-led group and 73% of the trainer-led group reported having fully or partially implemented a change following the program, whereas 20% of expert-led and 13% of trainer-led reported having already fully implemented before their SCOPE of Pain attendance. The lowest implementation/improvement rates were for patient prescriber pain agreements with patients (41% and 39% of the expert-led and trainer-led groups, respectively); however, this item was the item most often reported as already fully implemented before the program (45% and 49% of the expert-led and trainer-led groups, respectively). At least 4 partially/fully implemented changes were reported by 59% of expert-led and 60% of trainer-led respondents, and 28% of expert-led and 29% of trainer-led reported implementing 7 to 9 changes. The proportions reporting implementing none of the 9 changes were 13% (expert-led) and 7% (trainer-led). No significant difference was observed between the 2 groups for any of the 9 practice changes.
Changes in guideline-based practices at 2 months.
Barriers to change
All participants reported at least 1 barrier to making practice change. The most significant barriers reported were lack of support staff to help make these changes (48% expert-led and 44% trainer-led) and patients’ resistance to change (37% expert-led and 37% trainer-led). No significant difference was observed between the 2 groups.
Discussion
This comparative evaluation assessed the educational outcomes for participants trained using the SCOPE of Pain curriculum taught by either experts in safe opioid prescribing education or by trainers who participated in a 2-hour TTT workshop. This study demonstrates that selected trained trainers are able to achieve a similar level of learner impact as expert faculty while broadening the reach to more rural audiences. The ambitious goal set by the FDA for reaching 192,000 clinicians nationally in 4 years has spurned the need to develop scalable models for knowledge dissemination. The fact that the trainer-led meetings resulted in similar outcomes to expert-led meetings makes the TTT model appealing for public health initiatives, such as safe opioid prescribing, where widespread dissemination of information is needed.
Originally the TTT program was designed to enable the trainers to go back to their home institution and complete a single meeting for their colleagues. However, the fact that many trainers went on to complete more than 1 meeting (with one trainer completing 19 meetings) suggests that trainers were being identified as a resource in their regional community beyond their own institutions. In addition, once a trainer completed a meeting, they likely felt more confident conducting additional meetings.
The trainer-led education was more successful than the expert-led in reaching rural-based clinicians. This finding provides further evidence of the usefulness of the TTT model in reaching clinicians in less accessible locations. 14 Although this study had similar results to a study by Martino et al. by showing that the effects of the trainer-led education were the same as the expert-led education, more investigation is needed to determine if the TTT model could lead to more sustained changes due to the creation of local leaders. Since most of our trainers were not previously experts in safe opioid prescribing education, the effects of having newly trained clinicians in these locations is unknown. The need to activate more members of the community in the fight against the opioid morbidity and mortality crisis is clear, and the TTT model has the potential to create local expert educators.
The education delivered to both groups was live and therefore sustained all the benefits of face-to-face education. It allowed the participants to ask questions of the trainers in real time, talk to colleagues about issues surrounding the content, identify local resources, and be removed from distractions as well as other commonly described disadvantages of online education, such as social isolation, difficulty of meaningful interaction, and de-individualized instruction. 8 , 9 , 10
Two of the 13 attitude items had statistically significant differences between the groups (i.e., “It is my responsibility and role to discuss with my patients not to give away their medications to relatives or friends,” and “I cannot get my patients to be truthful about illicit drug use”). These 2 items focused on patient trust. It is possible that expert trainers were more skilled in teaching these more nuanced approaches to safe opioid prescribing that could be perceived by the patient as being judged. Further research is needed to test this hypothesis.
It is unclear why only a third of the trainers conducted their own trainings. There are likely numerous barriers to conducting trainings upon return to their home institution. It is possible that the external (BUSM) and/or internal (trainers’ home institution) support provided to trainers was not adequate to conduct meetings. A 2-hour TTT workshop may be insufficient for some to feel that they have mastered of the content in order to train others. Further study is needed to determine how to best facilitate trained trainers as they plan their own educational meetings, as well as to determine what barriers they may face that prevent completing trainings.
The study has a few limitations to consider. This was a post-assessment-only design that did not allow for the assessment of baseline knowledge of both the expert-led and trainer-led groups. Although a pre-test/post-test design would have been preferable, it would have been logistically challenging to implement. As opposed to the expert-led sessions, participants of the trainer-led sessions did not preregister and therefore could not receive the pre-test before the activity. Due to the time constraints of typical trainer-led activities, the pre-test would have been too time-consuming to administer at the start of the activity. The changes in attitudes, confidence, and performance were self-reported and therefore open the risk for self-assessment bias and social desirability bias. The participants of the IMMED and 2MO post-assessments were self-selected and not randomly chosen, so it is possible that participants who found the education the most useful or had some changes to report were more likely to complete the assessments. However, this self-selection bias should apply equally to both groups. Without a control group, we cannot attribute changes exclusively to SCOPE of Pain. The high attrition in our study is consistent with other educational evaluation studies. 15 Our follow-up response rates were similar between our 2 comparison groups and likely limited by the same biases. In addition, the sample size was still above the minimum value calculated to be required a priori.
This study provides additional evidence of the effectiveness of education to promote guideline-based safe opioid prescribing and in particular how the TTT model can further disseminate this education to rural areas. With the ever-growing need to educate clinicians on the safe opioid prescribing practices for chronic pain, a model that does not rely on experts, who have limited time to train, is crucial. With the continued high prevalence of prescription opioid misuse, nontraditional models can help accelerate knowledge dissemination and possibly activate local leaders to help stem this growing crisis.
Footnotes
Funding
The SCOPE of Pain program was funded by an independent educational grant awarded by the manufacturers of extended-release (ER) and long-acting (LA) opioid analgesics, collectively known as the Risk Evaluation and Mitigation Strategy (REMS) Program Companies or RPC. The funding organization had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. Boston University School of Medicine partnered with the Federation of State Medical Boards and the Council of Medical Specialty Societies in the development, execution, and promotion of the SCOPE of Pain program. The authors declare they have no conflicts of interest.
Author contributions
All coauthors were involved in the writing or revisions of the manuscript. Dr. Zisblatt was involved in research conception and design, collection of data, analysis, and interpretation of the results. Dr. Alford was involved in research conception and design and interpretation of the results. Dr. Hayes was involved in research conception and design, collection of data, analysis, and interpretation of the results. Ms. Lazure was involved in data collection and analysis. Ms. White and Ms. Hardesty were involved in research conception and design and interpretation of results.