Abstract
Objective:
Screening, brief intervention, and referral to treatment (SBIRT) for adolescent alcohol and drug (AOD) use is recommended to occur with adolescents admitted to pediatric trauma centers. Most metrics on SBIRT service delivery only reference medical record documentation. In this analysis we examined changes in adolescents’ perception of SBIRT services and concordance of adolescent-report and medical record data, among a sample of adolescents admitted before and after institutional SBIRT implementation.
Methods:
We implemented SBIRT for adolescent AOD use using the Science to Service Laboratory implementation strategy and enrolled adolescents at 9 pediatric trauma centers. The recommended clinical workflow was for nursing to screen, social work to provide adolescents screening positive with brief intervention and referral to their PCP for continued AOD discussions with those. Adolescents screening as high-risk also referred to specialty services. Adolescents were enrolled and contacted 30 days after discharge and asked about their perception of any SBIRT services received. Data were also extracted from enrolled patient’s medical record.
Results:
There were 430 adolescents enrolled, with 424 that were matched to their EHR data and 329 completed the 30-day survey. In this sample, EHR documented screening increased from pre-implementation to post-implementation (16.3%-65.7%) and brief interventions increased (27.1%-40.7%). Adolescents self-reported higher rates of being asked about alcohol or drug use than in EHR data both pre- and post-implementation (80.7%-81%). Both EHR data and adolescent self-reported data demonstrated low referral back to PCP for continued AOD discussions.
Conclusions:
Implementation of SBIRT at pediatric trauma centers was not associated with change in adolescent perceptions of SBIRT, despite improved documentation of delivery of AOD screening and interventions. Adolescents perceived being asked about AOD use more often than was documented. Referral to PCP or specialty care for continued AOD discussion remains an area of needed attention.
Trial registration:
Clinicaltrials.gov NCT03297060
Keywords
Introduction
Injury is a leading cause of death for adolescents 1 and alcohol plays a significant role in death and disability across many injury types. 2 The pediatric trauma center represents an important setting for identifying and addressing alcohol use and misuse with injured adolescents. Studies have reported current alcohol use rates in pediatric trauma patients ranging from 14% to as high as 34%.3 -7 These alcohol use rates are elevated compared to patients seen in other medical settings 7 and the overall 12 to 17 year old US population. 8
The American College of Surgeons Committee on Trauma verification requirements include universal alcohol screening for all trauma patients 12 years or older. For Level 1 and Level 2 trauma centers, a brief intervention must be provided to all individuals screening positive. 9 The model of universal screening, brief intervention, and referral to treatment (SBIRT) for alcohol or drug (AOD) use has demonstrated effectiveness in a variety of healthcare settings, including adult trauma centers, pediatric primary care, and schools.10,11 While this verification requirement has been in place for more than a decade, data evaluating SBIRT among pediatric trauma centers is still emerging and implementation of SBIRT in this setting has been inconsistent. A study examining the ACS Trauma Quality Program database on AOD screening in 2017 found that biochemical screening for AOD with adolescent trauma patients increased with age; however, only 15.2% of 12 year olds were screened and 50.9% of 17 year olds screened. 3 Another study reported that only 39% of adolescent trauma patients were referred to secondary services after a positive AOD screen. 12 Less is known about how often adolescents with AOD use who are referred for AOD follow-up care after a pediatric trauma admission actually followed through with the referral.
Prior SBIRT effectiveness and implementation trials have historically relied on electronic health record (EHR) data to examine changes in SBIRT uptake over time. Such trials have generally found that providing training and implementation support to front-line clinical staff is associated with increased rates of documented screening and brief intervention.13,14 Although EHR documentation is important for subsequent clinical care and the data used by trauma centers to monitor compliance with ACS requirements, it is also important to examine SBIRT delivery from the perspective of admitted adolescents and consider whether adolescents’ perspectives are concordant with services documented within the EHR. In the current analysis embedded within a multi-site trial, we examined changes in admitted injured adolescents’ perception of SBIRT services (including linkage to care in the 30 days following admission) as well as the concordance of adolescents’ perceptions with medical record data among adolescents admitted before and after institutional SBIRT implementation.
Methods
Study design
This was a prospective study examining hospitalized adolescent trauma patients’ self-report of receiving SBIRT for AOD use and the concordance of self-report with EHR data. The evaluation was nested within an implementation trial that utilized a stepped wedge, cluster randomized control design in which 10 US pediatric trauma centers implemented SBIRT at different time points using an empirically supported multi-level implementation strategy called the Science to Service Laboratory (SSL) to make SBIRT part of standard clinical care for admitted adolescent trauma patients. 15 The SSL implementation strategy was first developed by the New England Addiction Technology Transfer Center in 2008 16 and consists of 3 core elements: didactic training, performance, feedback, and external coaching. 15 These 3 core elements were specifically designed to address well-documented determinants of implementation at the provider- and organizational-levels: specifically, didactic training addresses provider knowledge performance feedback addresses provider skill, and external coaching addresses organizational infrastructure and leadership support. Nurses received didactic training on screening in the form of a 30-minute webinar. Social work received didactic training and performance feedback on brief intervention paired with referral to treatment in the form of a 1-hour webinar and a 2-hour live training consisting of practice administration. Pediatric trauma center administrators received external coaching in the form of 1-hour monthly sessions. All SSL elements were co-led by a licensed emergency medicine physician and 2 licensed clinical psychologists. Prior trials have shown that receipt of the SSL has been associated with significantly higher rates and speed of adoption of evidence-based interventions than didactic training only17,18 and that the SSL is acceptable to front-line staff. 16 In addition, the SSL is still routinely offered by the New England ATTC, a federally funded implementation support center, making it a credible real-world implementation strategy.
The study had 6 wedges, each 9 months in length, for a total study period of 54 months (3/1/18-8/31/22). During the first wedge, none of the sites received SSL implementation support and then with the start of the second wedge, 2 randomly selected trauma centers transitioned into receipt of active SSL implementation support while the other 8 sites remained in the pre-implementation control phase. At the start of the third wedge, the first cohort moved into a sustainment phase (a period when they were expected to continue SBIRT delivery without active SSL support) while a second cohort of 2 additional randomly selected sites received active SSL support, and the remaining cohorts stayed in the pre-implementation control phase. This study design continued until all sites received the SSL. A single central IRB approval was obtained from Lifespan Institutional Review Board for the research protocol at the study’s coordinating center (Approval #1092046) on 09/15/2017. Participants provided written consent/assent prior to their participation.
SBIRT intervention
The SBIRT intervention and recommended workflow involved inpatient nurses conducting AOD screening with the Screening to Brief Intervention (S2BI) tool, a validated screening tool that asks 3 questions about past year use of AOD, including alcohol, tobacco, and marijuana (cannabis). 19 Adolescents who reported past year use on one or more of the questions were asked 4 additional questions about past year use of prescription drug misuse, inhalants, illegal drugs, and synthetic drugs. If the S2BI was positive, social workers were then consulted to further assess adolescents’ level of risk using the CRAFFT tool. The CRAFFT is a validated brief assessment tool that includes questions regarding risks associated with substance use (e.g. driving a car while under the influence of AOD), and provides social workers with key information for discussion during brief intervention.19,20 Social workers were trained by a psychologist member of the research team to deliver a brief intervention using a motivational interview approach to all adolescents who screened positive, as well as to refer all adolescents with past year AOD use for follow-up conversations with their primary care provider (PCP) and if indicated, referral for specific AOD treatment.
Study participants
A sample of adolescents meeting inclusion criteria were enrolled across the entire study period to measure their perception of SBIRT services and assess concordance with EHR data. Each center was asked to enroll 1 adolescent per month for the duration of the stepped-wedge trial, with a target goal of 50 adolescents per center or 500 adolescents across the 10 centers. Research staff performed an initial screening of adolescents admitted to the trauma center for the following inclusion criteria: adolescents aged 12 to 17 years old; medically stable; accompanied by a parent or legal authorized representative to provide study consent, able to speak English or Spanish, and with telephone or email access. We specifically wanted to oversample for adolescents who were AOD positive to allow for assessment of the brief intervention and referral components of SBIRT, so for the first 7 months of each wedge we only recruited adolescents who were AOD positive on biologic testing or self-report with an AOD screening tool. Adolescents were excluded from the study enrollment if they were incarcerated, a ward of the state, admitted due to suicide attempt or acute conditions that would preclude provision of informed consent and assent, and if they had already been previously enrolled in this study. Recruitment was limited to when research staff were available for enrolling eligible adolescents. Enrolled adolescents provided written assent (and their parents written consent) and agreed to receive a survey (see Supplemental Data for survey instrument) after discharge and to allow the study team to access their EHR for additional study data. Adolescents also completed a brief demographic data survey at enrollment.
Sources of data
Enrolled participants were contacted 30 days after hospital discharge by the study coordinating center to complete a survey created by the research team to collect data on the adolescent patient’s experience of SBIRT services (see online supplemental material). Adolescents were surveyed about their experiences while they were a trauma center patient; specifically, the survey included whether they were asked about their AOD use (using questions adopted from S2BI), referred to a counselor or PCP for their AOD use, visited a counselor or PCP for AOD use, or discussed their AOD use with a counselor or PCP. The post hospital discharge surveys were administered during both the pre and post implementation phases of the study and were created for this study.
Research staff at each study site were trained by the study coordinating center on EHR data collection and submitted EHR data 6 times through the study. EHR data collected included: whether a validated adolescent AOD screening tool (S2BI or CRAFFT) was administered and its results, if a brief intervention for AOD was delivered after a positive screen, and if a referral for specialized AOD care or continued AOD discussions with their PCP was provided.
Analysis plan
Data were exported into SAS (Version 9.4, Carey, NC) for analysis. Analyses of differences between pre and post implementation data were conducted using t-tests for means (with standard deviation, SD), and Pearson Chi-Square tests for frequencies. Statistical significance was set a priori as P < .05. Analysis was conducted on the EHR and adolescent patient self-report survey data independently. The agreement between patient responses to the 30-day post discharge survey and provider EHR documentation of AOD screening was assessed using Cohen’s Kappa and the Prevalence Adjusted Bias Adjusted Kappa (PABAK) statistic, with 95% confidence intervals. PABAK controls for bias due to higher prevalence in some responses and provides the proportion of positive and negative agreement between different sources of data. For this analysis we included the adolescent patient responses and the EHR data that documented receipt of a validated screen (CRAFFT and/or S2BI). Biological screening was excluded because the results, especially if negative, may not have been shared with the patient.
Sample size
The sample size for this study was calculated using the proc power application (SAS). We conservatively estimated that there would be a minimum of a 20% increase in objectively reported AOD screening recorded in the EHR data. This would require a minimum sample of 87 EHR pre and post implementation study time points to achieve an acceptable statistical power (β = .80) for an α < .05 to demonstrate significant difference in this outcome between the 2 study timepoints.
Results
Sample characteristics
During the study period (2018-2022), a total of 6189 adolescents (12-17 years old) were identified as admitted to the trauma services across the 9 study sites that enrolled adolescent patients and provided linkage to the patients EHR for analysis. Of the admitted patients, 1416 were assessed by research staff and met eligibility criteria, and 691 patients (48.8% of eligible patients) were approached for study enrollment during the research staff’s availability. Adolescent assent and parent consent for study enrollment were obtained for 430 patients (62.6% of approached patients, 86% of the recruitment target) with 6 later withdrawing from study participation.
The flow of participant recruitment throughout the study is displayed in Figure 1. In total, 424 patients were matched with EHR data, completed the brief patient survey at enrollment, and comprised the final analytical sample. Of these patients, 329 (77.6% of the analytical sample) also completed the 30-day post discharge patient survey. There were no significant differences between those who completed the follow-up survey by age (t (418) = −1.39, P = .16) and gender, [χ2 (1) = 0.23, P = .63], but those who completed the survey were more likely identified in the EHR as Non-Hispanic White (82.4%) or Hispanic (73.3%) compared to patients identified as Non-Hispanic Black (62.2%), [χ2 (3) = 10.94, P = .01]. Reflecting the design of the study, more adolescent patients were recruited during the pre-implementation study phase than the active implementation or sustainment phases (n = 252, versus n = 172).There were no significant differences in recruited patients’ age (15.0versus 15.2 years, P = .27), gender (males = 70.2%versus 72.9%, P = .53) or race/ethnicity distribution (Non-Hispanic White: 54.4%versus 59.4%, P = .18), across the study phases (Table 1).
Flowchart of study participants.
Characteristics of Adolescent Patients by Implementation Phase (N = 424).
Analysis of SBIRT services
EHR data on the delivery of SBIRT services of the 424 recruited patients across all study sites demonstrated a significant increase in EHR documented validated screening between pre and post implementation, with a 4fold increase in administered screens (16.3% (n = 41) versus 65.7% (n = 113), P < .001). Brief intervention for those with any positive AOD screen showed a 1.5-fold increase (27.1%versus 40.7%, P = .003). There was a statistically significant decrease, between the pre and post implementation periods, in EHR documentation of referral to treatment following the BI with a 2.5-fold decrease in such referrals (11.1% (n = 28) pre versus 4.1% (n = 7) post, P = .01; not shown in tables).
Adolescent survey data from pre and post implementation is presented in Table 2. There was no significant difference in patient report of having been asked by a trauma staff member about their AOD pre to post implementation, with most patients reporting having this discussion at both study timepoints (80.7%versus 81%, P = .95). Among those patients who were documented in the EHR as having screened positive for AOD, the majority self-reported as having been asked about their AOD use (80.9% versus 81.7%). Additionally, pre to post implementation self-reports of trauma staff advice to discuss AOD use at their PCP visit (P = .53) or self-report of being referred by a trauma staff member to an AOD counselor (P = .80) were comparable between pre and post implementation.
Adolescent Patient Survey Responses (N = 329).
Abbreviations: AOD, alcohol or other drugs; PCP, Primary care practitioner.
Among those who screened positive as documented in electronic health record.
Analyzes for agreement between EHR validated screening (CRAFFT and/or S2BI) documentation and adolescent survey responses had an overall Kappa value of 0.09 (95% CI: 0.03, 0.15), indicating poor agreement between the patient self-report and the EHR data. However, further review of these data using PABAK, showed that the proportion of positive agreement (EHR and teen survey agreed screening occurred) was 53.1% (95% CI: 47.0, 59.2%) and the proportion of negative agreement (EHR and teen survey agreed screening had not occurred) was 34.6% (95% CI: 29.3, 39.9%). At both pre and post implementation, the teen self- report of being asked about AOD use was significantly greater than the documentation of validated screening (pre = 73.4% patient versus 16.3% EHR, P < .001: post = 77.4% patient versus 65.7% EHR, P = .004).
Discussion
The current study was embedded within an implementation trial at pediatric trauma centers examining the SSL implementation strategy, in the current analysis we assessed change in SBIRT delivery from the adolescents’ perspective, comparing EHR documentation to adolescent perceptions of SBIRT service delivery. On a positive note, in this sample of adolescents who consented to a follow-up survey, we found a 4-fold increase in EHR documented validated screening between pre- and post-implementation as well as 1.5-fold increase in documented brief interventions for those with any positive AOD screen across all study sites. In contrast to these significant increases in EHR documented screening and intervention rates, adolescent self-report data did not indicate a difference between how often adolescents were asked about AOD use before and after SBIRT implementation. Whereas the EHR data demonstrated a clear increase in documentation of screening for AOD using validated screens after implementation (16.3%-65.7%), most adolescents replied that someone asked or discussed AOD use with them both before and after implementation (73.4%versus 77.4%). Other investigators have similarly reported differences in EHR recorded data and self-reported data among adolescents receiving treatment in 6 emergency departments. 21
We offer several hypotheses for the discrepancy between the adolescents’ EHR and adolescent self-reported data in our sample. First, some clinicians may as part of their medical history and symptoms interview ask adolescents about AOD use without documenting findings in the EHR. Second, in our EHR review, we only code for use of validated adolescent screening tool (CRAFFT, S2BI); anecdotal data from an environmental scan prior to the start of the study indicated that some centers were using non-validated AOD question(s) prior to SBIRT implementation that were not aligned with best practices for AOD screening in healthcare settings. Third, we stressed the importance of universal screening for all adolescents admitted to the trauma service as a key part of SBIRT implementation. Yet our study sample was purposefully constructed to have a majority of AOD positive adolescents, which limits our ability to evaluate whether we increased screening rates across those adolescents who did not screen positive. It is possible that during the pre-implementation period, results of AOD positive biologic screening data may have prompted clinicians to discuss AOD use with adolescents. The significantly higher concordance when adolescents’ EHR data were positive for AOD relative to when adolescents had a negative response may support this observation.
Although the RT component of SBIRT has frequently meant referral to specialty treatment, in our implementation model we defined RT as: (a) referral back to the adolescent’s primary care provider for continued AOD discussions for those with lower risk adolescents or (b) referral to both specialty treatment and primary care provider for those at high risk. We found no change in referrals for additional AOD counseling from the pre-implementation to post-implementation phase including no change in prompts to discuss AOD with the primary care physician. Both EHR and adolescent survey data confirmed that referrals were infrequent both pre- and post-implementation of SBIRT. These data indicate that there is a missed opportunity for in-hospital clinicians to connect the adolescent with continuing AOD care after discharge. Almost half of the adolescents reported seeing their primary care provider within 30 days of their hospital discharge which would provide a timely occasion to continue discussions of AOD use that had occurred in the hospital. Such discussion might also provide an opportunity for primary care providers to monitor the trajectory of AOD use into future healthcare visits and engage adolescents in appropriate specialty treatment if needed. Such care, in turn, may reduce the likelihood of the severity of adolescent substance use increasing.22,23 Future implementation efforts will need to have a more comprehensive referral component that engages both adolescents and caregivers in continuing AOD discussions with primary care providers and, if needed, specialty care. 24
Our study had several limitations that may have impacted our findings. First, our recruitment and enrollment of patients was contingent on the limited availability of research staff at each site and did not reach our intended recruitment target, limiting power to detect effects. The sample was also affected by those declining participation (37.4%) and those enrolled in the study but who did not complete the 30-day follow-up survey (22.4%). Institutional infection protocols during the initial phase of the COVID-19 pandemic also necessitated a complete pause in recruitment being done from March 2020 to September 2020 and pandemic-related research staff reductions resulted in more adolescents recruited during pre-implementation than the post-implementation period. Second, our analysis examined whether the proportion of adolescents that reported receiving an AOD discussion increased from the pre-implementation to the post-implementation phase but did not examine whether the discussions occurred with fidelity to our SBIRT protocol. Adolescents simply reported whether an AOD discussion occurred and not on the caliber of the discussion. The data from this self-report questionnaire are also potentially subject to recall bias acquiescence bias, or respondent bias from the adolescent. It may be that the proportion of adolescents who reported receiving SBIRT did not change following receipt of the SSL, even though the quality of SBIRT delivery and compliance with best practices both increased. Furthermore, our survey instrument though brief and straightforward has not been previously validated or pilot tested. Finally, our analysis of EHR data is limited by the quality of documentation at the sites as the absence of documentation does not necessarily indicate the absence of service delivery.
Conclusions
This study found in a subset of admitted injured adolescents an increase in EHR documented, delivery of validated screening and brief interventions after the SSL implementation strategy was implemented in pediatric trauma centers but found no difference in the rates of adolescents’ self-reported in-hospital discussions of AOD use before and after SBIRT implementation. Importantly, a paucity of adolescents reported being directed to continue AOD discussion with their primary care provider after discharge and referral back to the primary care provider was rarely documented in the EHR. Specific strategies to promote the RT component of SBIRT, referral to a primary care provider or specialty program, represents a key area for additional research and clinical efforts.
Supplemental Material
sj-docx-1-sat-10.1177_29768357241272356 – Supplemental material for Adolescents’ Perceptions of Screening, Brief Intervention, and Referral to Treatment Service at Pediatric Trauma Centers
Supplemental material, sj-docx-1-sat-10.1177_29768357241272356 for Adolescents’ Perceptions of Screening, Brief Intervention, and Referral to Treatment Service at Pediatric Trauma Centers by Michael J Mello, Janette Baird, Anthony Spirito, Kelli Scott, Mark R Zonfrillo, Lois K Lee, Andrew Kiragu, Emily Christison-Lagay, Julie Bromberg, Stephanie Ruest, Charles Pruitt, Karla A Lawson, Isam W Nasr, Jeremy T Aidlen, Robert Todd Maxson and Sara Becker in Substance Abuse: Research and Treatment
Footnotes
Acknowledgements
The authors gratefully acknowledge Timmy Lin, MPH for assistance in data analysis and data visualization. The authors acknowledge contribution to the conduct of the study by Beth E. Ebel MD, MSc, MPH.
Authorship Contributions
The study was conceptualized and designed by MJM, JBa, AS, MRZ, JBr, SB. Funding obtained by MJM. JBa led data analysis. Initial manuscript drafting by MJM, JBa, AS, MRZ, JBr, SB. All authors contributed to the acquisition of data, conduct of the study, and critically reviewed manuscript drafts.
Funding:
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Research reported in this publication was supported by the National Institute on Alcohol Abuse and Alcoholism of the National Institutes of Health under Award Number R01AA025914. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Declaration of conflicting interests:
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethical Approval and Informed Consent Statements
A single central IRB approval was obtained from Lifespan Institutional Review Board for the research protocol (Approval #1092046) on 09/15/2017. Participants provided written consent/assent prior to their participation.
Data Availability Statement
Data available from corresponding author upon request.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
