Sage Journals: Discover world-class research

Abstract

Objectives

Tracking nonfatal overdoses in the escalating opioid overdose epidemic is important but challenging. The objective of this study was to create an innovative case definition of opioid overdose in North Carolina emergency medical services (EMS) data, with flexible methodology for application to other states’ data.

Methods

This study used de-identified North Carolina EMS encounter data from 2010-2015 for patients aged >12 years to develop a case definition of opioid overdose using an expert knowledge, rule-based algorithm reflecting whether key variables identified drug use/poisoning or overdose or whether the patient received naloxone. We text mined EMS narratives and applied a machine-learning classification tree model to the text to predict cases of opioid overdose. We trained models on the basis of whether the chief concern identified opioid overdose.

Results

Using a random sample from the data, we found the positive predictive value of this case definition to be 90.0%, as compared with 82.7% using a previously published case definition. Using our case definition, the number of unresponsive opioid overdoses increased from 3412 in 2010 to 7194 in 2015. The corresponding monthly rate increased by a factor of 1.7 from January 2010 (3.0 per 1000 encounters; n = 261 encounters) to December 2015 (5.1 per 1000 encounters; n = 622 encounters). Among EMS responses for unresponsive opioid overdose, the prevalence of naloxone use was 83%.

Conclusions

This study demonstrates the potential for using machine learning in combination with a more traditional substantive knowledge algorithm-based approach to create a case definition for opioid overdose in EMS data.

Keywords

opioid overdose surveillance machine learning natural language processing case definition

The drug overdose epidemic claimed 67 367 lives nationwide in 2018, with 70% of deaths involving an opioid.¹ In 2018 in North Carolina, about 5 deaths per day were attributed to unintentional opioid overdoses, equaling 1718 in total.² These fatalities constitute the most extreme outcome of this epidemic and are accompanied by far greater numbers of nonfatal opioid overdoses requiring hospital care, rehabilitation services, police management, and government resources, and resulting in shared social trauma.³

Our ability to address this crisis may be hampered by inadequate surveillance of nonfatal opioid overdoses. Research suggests that nonfatal opioid overdoses are frequent but difficult to estimate because of inadequate data sources and the stigmatized nature of opioid use.^4

-8 One proposed solution is to track these events through naloxone administrations and other more nuanced case definitions in emergency medical services (EMS) data.^9,10 Although naloxone is used solely to reverse opioid overdose, it may be administered in cases where opioid overdose is possible but not confirmed. For example, it may be given in concert with advanced cardiovascular life support (ACLS), when EMS personnel encounter a patient who is in cardiac or respiratory arrest of unknown etiology.¹¹ Describing this misclassification potential, a 2018 study in North Carolina found that naloxone administration was only 57% sensitive to opioid overdoses.⁵

In the community, conscious individuals who may have heavily misused opioids are often said to have “overdosed,” but diagnostically, opioid overdose is characterized by unresponsiveness, constricted or pinpoint pupils, and respiratory distress.¹² Because of the complicated and varied nature of opioid overdoses (illicit/legal, intentional/unintentional), identification using expert medical record review is considered the gold standard.⁵ However, recent endeavors have successfully used natural language processing (NLP)—a method of engaging computer assistance for language interpretation—combined with machine learning (ML) to develop case definitions for opioid overdose and related conditions using health data, including EMS.^13
-15 We attempted to create an improved case definition by combining NLP–ML with a rule-based algorithm that was informed by expert personnel, including a nurse (J.S.) and a paramedic (D.E.) on the study team. This hybrid approach was meant to optimally balance transparency and performance.

To explore the value of this approach, we compared it with a purely rule-based case definition published previously using data from Rhode Island.¹⁶ Although the rules applied in the North Carolina and Rhode Island case definitions differed (ie, the North Carolina algorithm is less complex because it relies on NLP–ML in several branches), this comparison is meant to illustrate the strengths and limitations of taking a rule-based approach versus a rule-based plus NLP–ML approach.

In ML, data are separated into “training” and “test” sets of labeled data to predict classification labels for unclassified data.¹⁷ Our model helped to identify patients whose chief concerns did not label them as having an opioid overdose but whose narratives looked similar to patients with a chief concern of opioid overdose. Although the distinction between ML and traditional predictive modeling can be blurry, ML typically involves many predictors, large stores of data, and a flexible, agnostic approach to variable selection.¹⁸ ML works well if it can be trained and applied to a well-defined patient population. Thus, we used expert opinion to refine the general EMS patient population to cases of possible opioid overdose, and we incorporated the ML model to eliminate false positives in patient subpopulations that were similar to the training set. The experts then provided rules to restrict those identified cases of opioid overdose to unresponsive opioid overdose, which included fatalities.

Methods

Data

This study used data on all North Carolina EMS encounters with narratives from January 1, 2010, through December 31, 2015, for patients aged >12 years (n = 6 210 572 encounters; Figure 1); individual patients may have had multiple encounters. Variables of interest included the chief concern, dispatch concern, narrative, medication administration, primary impression, and secondary impression of first responders. We obtained data from the National Emergency Medical Services Information System version 2 (NEMSIS V2), which is the universal standard for EMS data nationwide. These data do not include diagnosis codes. Thus, the case definition that is discussed herein refers to “suspected” opioid overdoses. This analysis used SAS version 9.4 (SAS Institute, Inc) and R version 4.0.2 (R Core Team). Analytic code is available online.²⁰ The University of North Carolina at Chapel Hill Institutional Review Board approved this project.

Figure 1

Case definition for opioid overdose using emergency medical services (EMS) dispatch data on patients aged >12 years, North Carolina, 2010-2015. In 2015, the American Heart Association released updated guidelines for naloxone administration in the presence of cardiac arrest, in recognition that opioid overdose can be difficult to distinguish from other causes of cardiac and respiratory arrest.¹¹ Even before 2015, EMS personnel were sometimes administering naloxone during cardiac emergencies of unknown origins. Therefore, this case definition for opioid overdoses required that patients who improved after receiving naloxone in conjunction with advanced cardiovascular life support (ACLS) met additional criteria for inclusion in this case definition. One possible additional criteria for inclusion is age <40, based on recent findings that patients with overdose-related cardiac arrest had a mean age of 38 (vs 66 for non–overdose-related cardiac arrest). Dispatch concerns were “breathing problem” or “sick person” or “unknown problem man down” or “ingestion/poisoning” or “unconscious/fainting.” Unresponsive opioid overdoses were defined as opioid overdose patients with level of consciousness as “unresponsive” or “painful”; a Glasgow Coma Scale score <8¹⁹; or a narrative or chief concern that contained the terms “unresponsive,” “unconscious,” “responsive to pain,” or “seizure.” Categories 1-5 are not mutually exclusive. Abbreviation: ML, machine learning.

Case Definition Algorithm

The case definition algorithm defined an opioid overdose as any encounter that met the qualifications for any 1 or more of 5 categories (Figure 1). The categories were not mutually exclusive. Encounters in any of the 5 categories that met additional criteria (described hereinafter) were considered unresponsive opioid overdoses. The case definition algorithm was mostly rule based but did include 2 categories that incorporated ML modeling to eliminate false positives. We first describe the broader, rule-based case definition algorithm. As has been done previously,⁹ this algorithm began by searching within patients who received naloxone. The algorithm automatically included patients whose response to naloxone was “improved” according to the EMS variable for medication response, as long as that patient did not receive naloxone as one of a cluster of resuscitation drugs that are generally considered part of the ACLS protocol.¹¹ Patients who received ACLS were excluded from category 1 but could be included in categories 2-5.

The algorithm also automatically included patients aged <40 who received naloxone and had cardiac arrest (category 2), because cardiac events among people aged <40 are more likely to be opioid related than cardiac events among older people.²¹ The algorithm also included any people who received naloxone and for whom the narrative included the word “pinpoint”—a reference to pupil size (category 3). Pinpoint pupils are commonly associated with opioid overdose, and naloxone is not used in the treatment of other precipitating factors (eg, stroke, head, or eye trauma). Other precipitating factors for pinpoint pupil also do not typically resemble opioid overdose in the nonsurgical context.²² For patients who received naloxone but did not have cardiac arrest and who had a dispatch concern consistent with—although not specific to—opioid overdose (“breathing problem,” “sick person,” “unknown problem/man down,” “ingestion/poisoning,” or “unconscious/fainting”),¹² we used an ML model (described in the following section) to designate suspected opioid overdoses (category 4).

Patients who did not receive naloxone may have also had an opioid overdose. Instances in which naloxone may not have been administered included patients who were dead at the scene, who were deemed stable enough by EMS that they did not require naloxone, or who were not identified as having an opioid overdose. To include these events in our case definition, we ran the ML model (described in the following section) on all encounters in which the dispatch concern, primary impression, or secondary impression were “ingestion/poisoning” and on all encounters in which the narrative or the chief concern contained the word “overdose” (category 5).

We chose to also describe cases of unresponsive opioid overdose because the algorithm appeared to capture many cases of opioid misuse and suspected misuse that did not put patients at risk of fatality. To designate cases of unresponsive opioid overdose, we divided the opioid overdose group into patients with a documented level of consciousness of “unresponsive” or “painful,” a Glasgow Coma Scale score <8,¹⁹ or a narrative or chief concern that contained the terms “unresponsive,” “unconscious,” “responsive to pain,” or “seizure” (Figure 1).

ML Model

To create the ML model, we created a training set from the 6 210 572 encounters with narratives for patients aged >12 years (Figure 1). We extracted data on all encounters with a chief concern consistent with the word “overdose” (n = 52 408). Within this sample, we identified likely opioid overdoses by again searching the chief concern, this time for known opioids. This designation intended to identify encounters that were reasonably representative of opioid overdoses. We then attempted to identify known, non-opioid overdoses because we needed an explicit set of non-opioid incidents to train the model (in many cases, the type of overdose was not specified—these were the incidents we wanted the model to help us identify). We did this by again searching the chief concerns for strings of words of non-opioid substances that are commonly used in other types of overdoses. We considered chief concerns that did not contain an opioid string and contained a non-opioid string to be non-opioid overdoses. Again, this list of non-opioid substances was meant to be representative for model-training purposes. The dataset of opioid and non-opioid overdoses contained 3968 observations: 1508 (38.0%) were labeled opioid overdoses and 2460 (62.0%) were labeled non-opioid overdoses. We performed this initial data cleaning process in SAS version 9.4.

We then used the “tm” package in R to create a corpus (a term in NLP referring to a collection of texts [ie, EMS narratives] with metadata) of patient encounter–level data consisting of the unique identifier and the narrative text. According to established methods, we cleaned the text by converting every letter to lowercase, removing all punctuation, replacing all instances of the word “naloxone” with “narcan,” and replacing our list of known opioids with the word “opioid.”²³ We combined multiple blanks into single blanks. At this stage, we did not remove stop words (ie, “the,” “in,” or “and”) or stem the terms (reducing words to the root form; ie, “injected” becomes “inject”) because we found that doing so made the text senseless. We then converted the cleaned text into a document-term matrix, which provided the frequencies that each term occurred in each narrative. We then used the “tidytext” package to make a data frame with a 1-word-per-narrative-per-row structure to eliminate sparsity and make the data more user friendly.

We weighted individual words on the basis of how likely they were to be associated with opioid overdose narratives. We divided the number of opioid narratives containing each word by the total number of opioid narratives to determine each word’s “opioid prevalence.” For each word, we used the same procedure to determine the “non-opioid prevalence.” Then, we calculated the natural log of the ratio of the opioid prevalence to the non-opioid prevalence. Doing so yielded a single weight for each word that was higher (ie, “more positive”) if the word was more likely to be associated with opioid narratives and lower (ie, “more negative”) if it was more likely to be associated with non-opioid overdoses.²⁴ The data frame was then edited to remove common and rare words and words with weights close to zero. Nonsensical words or words that appeared to have been included by chance were manually removed in an iterative fashion to improve the model performance. We developed these weights in contrast to the more common technique of Term Frequency Inverse Document Frequency,²³ because we wanted a system that would both prioritize terms strongly associated with opioid overdose events and deprioritize terms strongly associated with non-opioid overdose events.

To develop the ML model, each EMS encounter’s weights were ranked from the highest positive weight to the lowest positive weight. Essentially, this weighting created a series of ranked variables that each contained a number—the first variable reflected the weight of the narrative’s word that was most strongly associated with opioid overdose (for example, the word “narcan” had a high weight), and the second variable reflected the weight of the next most strongly associated word. A separate set of ranked variables was created to reflect how each encounter’s weights were ranked from the lowest negative weight to the highest negative weight. These variables represented terms that were strongly associated with non-opioid overdoses, with more negative weights reflecting stronger associations. We then randomly split our data set into 80:20 training and test sets and modeled the ranked variables against the labeled outcomes for all 3968 encounters using a classification tree algorithm.²⁵ Using 10-fold cross-validation, the model produced the following performance metric averages: accuracy, 93.7%; sensitivity, 87.4%; specificity, 97.6%; positive predictive value (PPV), 95.8%; and negative predictive value, 92.7%.

Testing the Case Definition

To further test the performance of the case definition algorithm and to understand how it would generalize to the full, unlabeled dataset, we pulled a random sample of 50 000 EMS encounters with narratives. Excluding people aged ≤12 years yielded 48 267 narratives, of which the algorithm identified 397 cases of opioid overdose (including unresponsive opioid overdoses). The 397 cases were hand reviewed by a nurse (J.S.) for evidence of probable or possible opioid overdose. We report the PPV of our case definition. We also ran this sample against a case definition proposed in Rhode Island and compare the overall number of opioid overdoses identified by both definitions, their degree of overlap, and their PPV.¹⁶ The Rhode Island case definition did not distinguish between responsive and unresponsive opioid overdoses.

Descriptive Results

Using this case definition, we describe demographic characteristics and trends in any opioid overdoses, unresponsive opioid overdoses, and naloxone administrations, regardless of case definition designation, for all EMS encounters from January 1, 2010, through December 31, 2015. We used prevalence differences with 95% CIs to designate significant increases or decreases in prevalence. We similarly explored trends in patient disposition.

Results

The case definition algorithm identified 52 048 opioid overdoses seen by EMS during the study period, of which 30 483 met the definition of an unresponsive opioid overdose (Table). In total, 83.3% (n = 25 391) of patients who met the definition of unresponsive opioid overdose received naloxone. Conversely, of the 48 273 patients who met study criteria and received naloxone, 52.6% (n = 25 381) met the definition for unresponsive opioid overdose.

Table

Characteristics and disposition of patients seen by emergency medical services (EMS) and who received naloxone, or who met the case definition for any opioid overdose, or the case definition for an unresponsive opioid overdose, North Carolina, 2010-2015^a

Characteristic	Naloxone administration (n = 48 273)	Opioid overdose (n = 52 048)	Unresponsive opioid overdose (n = 30 483)
Sex
Male	24 850 (51.5)	25 933 (49.8)	15 784 (51.8)
Female	23 183 (48.0)	25 293 (48.6)	14 461 (47.4)
Unknown	240 (0.5)	822 (1.6)	238 (0.8)
Race
White	35 692 (73.9)	40 003 (76.9)	23 503 (77.1)
Black	9042 (18.7)	7686 (14.8)	4683 (15.4)
American Indian/Alaska Native	1151 (2.4)	1081 (2.1)	639 (2.1)
Native Hawaiian/Other Pacific Islander	28 (0.1)	38 (0.1)	23 (0.1)
Asian	112 (0.2)	90 (0.2)	55 (0.2)
Other	691 (1.4)	742 (1.4)	424 (1.4)
Unknown	1557 (3.2)	2408 (4.6)	1156 (3.8)
Residence
Urban	29 297 (60.7)	33 003 (63.4)	19 110 (62.7)
Rural	16 563 (34.3)	16 200 (31.1)	9706 (31.8)
Unknown	2413 (5.0)	2845 (5.5)	1667 (5.5)
Age, median (IQR), y	51 (35-64)	45 (30-58)	45 (30-59)
Disposition
Received naloxone	48 273 (100.0)	31 775 (61.0)	25 391 (83.3)
Treated and transported by EMS	43 824 (90.8)	44 959 (86.4)	27 341 (89.7)
Treated and released	1860 (3.9)	1654 (3.2)	1071 (3.5)
Treated, transferred care	521 (1.1)	778 (1.5)	470 (1.5)
Patient refused care	151 (0.3)	1621 (3.1)	333 (1.1)
Dead at scene	1725 (3.6)	890 (1.7)	840 (2.8)
No treatment required	33 (0.1)	1656 (3.2)	284 (0.9)
Other	159 (0.3)	490 (0.9)	144 (0.5)

Abbreviation: IQR, interquartile range.

^aData source: North Carolina Emergency Medical Services, captured using the National Emergency Medical Services Information System version 2. All values are number (percentage) unless otherwise indicated.

Comparison With Rhode Island Case Definition

Using a new random sample of 48 332 narratives for EMS encounters with patients aged >12 years, we applied this case definition and a recently published case definition from Rhode Island. The Rhode Island definition identified 248 cases, whereas our definition identified 379 cases as opioid overdose (responsive or unresponsive) and 208 cases as unresponsive opioid overdoses. The PPV for suspected true opioid overdose was 90.0% using the North Carolina case definition and 82.7% using the Rhode Island case definition (Figure 2). Using the North Carolina case definition, 59% of responsive and unresponsive opioid overdoses combined received naloxone, and 81% of unresponsive opioid overdoses received naloxone. Using the Rhode Island definition, 84% of encounters designated as opioid overdoses received naloxone.

Figure 2

Comparison of suspected opioid overdose case definitions from Rhode Island and North Carolina in a sample of emergency medical services encounters among patients aged >12 years, 2010-2015. Abbreviation: PPV, positive predictive value.

Trends in Opioid Overdose and Naloxone Administration

According to the case definition, the number of EMS encounters for unresponsive opioid overdoses increased from 3412 encounters in 2010 to 7194 encounters in 2015. The corresponding monthly rate per 1000 encounters increased by a factor of 1.7 from January 2010 (3.0; n = 261 encounters) to December 2015 (5.1; n = 622 encounters). This trend was closely followed by the case definitions’ estimation of any opioid overdose encounters (Figure 3). The rate of EMS encounters in which patients received naloxone also initially followed the same trend but leveled off in 2013. This rate increased by a factor of 1.3 from January 2010 (5.4; n = 465) to December 2015 (7.1; n = 870).

Figure 3

Trends in emergency medical services (EMS) naloxone administrations and case definition–derived opioid overdose encounters and unresponsive opioid overdose encounters for patients aged >12 years, North Carolina, 2010-2015. Jagged lines represent monthly rates, and smooth lines represent locally weighted smoothing rate averages.

Generally, the demographic characteristics of patients receiving naloxone were similar to the characteristics of patients meeting the case definitions for opioid overdose and unresponsive opioid overdose (Table), although patients receiving naloxone had a higher median age (51 vs 45). In aggregate, patient dispositions were generally similar among these 3 groups. However, trends in patient disposition indicated that the proportion of patients who met the case definition for unresponsive opioid overdose and were then treated and transported to a hospital by EMS declined from 93.3% to 86.1% (prevalence difference [PD] = –7.2; 95% CI, –11.0 to –3.3) during the study period; the proportion refusing care rose slightly, from 0.8% to 1.2% (PD = 0.4; 95% CI, 0.01-0.8; Figure 4); and the proportion who were treated and released increased from 1.6% to 5.9% (PD = 4.3; 95% CI, 3.6-5.0). The proportion of patients who were considered unresponsive opioid overdoses and determined to be dead at the scene increased from 1.8% to 3.5% (PD = 1.6; 95% CI, 1.0-2.3).

Figure 4

Trends in emergency medical services (EMS) percentages of patients who refused care among patients receiving naloxone and among case definition–derived opioid overdose encounters and unresponsive opioid overdose encounters for patients aged >12 years, North Carolina, 2010-2015. Jagged lines represent monthly rates, and smooth lines represent locally weighted smoothing rate averages.

Discussion

Using a case definition algorithm enhanced by NLP, this study identified 52 048 opioid overdoses from 2010 through 2015. In addition, we found that the rate of unresponsive opioid overdoses nearly doubled during the study period. Although 83% of unresponsive opioid overdoses received naloxone, only about half of patients who received naloxone met the definition of unresponsive opioid overdose, providing more evidence that naloxone administration is not a precise indicator of opioid overdose.⁵ As NLP becomes increasingly common in health care research^{13,17,18,24
-26} it is important to understand how this method can be used in combination with expert-informed/rule-based case definition algorithms. The North Carolina (rule-based plus NLP–ML) and Rhode Island (purely rule-based) case definitions obtained strong and comparable PPVs (90.0% and 82.7%, respectively) when applied to a sample of North Carolina EMS encounters, especially considering that the Rhode Island case definition was developed using that state's EMS data. However, the North Carolina case definition identified 131 more possible or probable cases of opioid overdose than the Rhode Island case definition, a 52.8% increase. This finding suggests that rule-based case definitions that incorporate NLP–ML may result in improved specificity, although additional research is warranted.

This study also identified an increase in the proportion of patients who experienced opioid overdose who were treated and released by EMS and, therefore, would not be captured in emergency department data or death records. EMS provider are on the frontlines of fighting the opioid overdose epidemic, providing a critical data source for tracking opioid overdose trends. Because nonfatal opioid overdoses are more frequent than fatal opioid overdoses, timely surveillance that includes data on nonfatal opioid overdoses will help public health rapidly identify and respond to new clusters, including emerging demographic or social subgroups at increased risk of opioid overdose.²⁷ Timely surveillance can improve local prevention and response resources and assist in the identification and investigation of novel exposure pathways and scenarios.²⁷

Limitations

This study had several limitations. First, systematic misclassification in our case definition may have biased our estimate of the true prevalence of opioid overdose. Second, given the use of EMS data, our case definition did not include patients who went directly to the emergency department or who received naloxone in the community. Third, related to data quality, the proportion of missing narratives in the EMS data was substantially higher at the beginning of the study period than in later years. However, this missingness was not common and more stable among chief concerns of opioid overdose. Fourth, related to the case definition itself, as the opioid overdose epidemic gained momentum during the study period, EMS providers may have changed their narrative language to more precisely describe what they witnessed, leading to an algorithm that may perform better and capture more cases in later years. Fifth, it is also possible that the ML model performed differently for different demographic groups. Sixth, given the number of narratives in the data subset used to compare the Rhode Island and North Carolina case definitions (n = 48 332), the study team did not have the resources to evaluate specificity and similar measures directly, warranting further investigation. Lastly, because this study used the NEMSIS V2 standard, the study might perform differently using NEMSIS V3 (the current standard). We recommend reapplying the model to NEMSIS V3 for future exploration.

Conclusions

This study demonstrates the potential for combining traditional algorithm-based case definition approaches with the selective use of more nuanced ML techniques. This methodology built on previous practices by combining rule-based and NLP–ML case definition approaches. When we compared this study’s case definition for opioid overdose with the rate of naloxone administrations during the study period, we saw rates initially track in a parallel nature and subsequent leveling of naloxone administrations, while opioid overdoses continued to increase. It is possible that this increase represents a change in how EMS providers chose to use naloxone as the opioid overdose epidemic progressed and they became more familiar with the drug and the condition it is used to treat. This comparison suggests that the common practice of using naloxone administrations as a proxy for opioid overdose not only overestimates the rate of unresponsive opioid overdoses but also may not reliably represent the trend. Because of the recent transition to NEMSIS version 3, our study does not contain the most recent trend information, but the flexibility of the method will allow the algorithm to be modified once sufficient version 3 data are available. Indeed, a strength of our study’s methodology was that it can be easily replicated and modified to fit the unique profiles of EMS data in other states. The NLP modeling helps to select and rank elements of the text that are highly predictive of opioid overdose, rather than supplying a universal list of terms and their rankings, which may not fit all contexts. By adopting and extending this approach, states may improve our understanding of nonfatal opioid overdose trends and improve prevention practices.

Footnotes

Acknowledgments

The authors acknowledge Mike Dolan Fliss for his consultation on this project.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by an award from the National Center for Injury Prevention and Control at the Centers for Disease Control and Prevention to the North Carolina Division of Public Health (grant 5NU17CE002728). This study was funded through a subcontract to the University of North Carolina at Chapel Hill under the Prevention for States grant.

ORCID iD

Josie J. Sivaraman, PhD

References

Wilson

Kariisa

Seth

Smith

H IV.

Davis

. Drug and opioid-involved overdose deaths—United States, 2017-2018. MMWR Morb Mortal Wkly Rep. 2020;69(11):290-297.doi:10.15585/mmwr.mm6911a4

32191688

North Carolina Department of Health and Human Services . Opioid action plan data dashboard. Updated August 4, 2020. Accessed March 20, 2021. https://www.ncdhhs.gov/about/department-initiatives/opioid-epidemic/opioid-action-plan-data-dashboard

Hagemeier

. Introduction to the opioid epidemic: the economic burden on the healthcare system and impact on quality of life. Am J Manag Care. 2018;24(10 suppl):S200-S206.29851449

Chatterjee

Larochelle

MR.

Xuan

et al. Non-fatal opioid-related overdoses among adolescents in Massachusetts 2012-2014. Drug Alcohol Depend. 2019;194:28-31.doi:10.1016/j.drugalcdep.2018.09.020

30391835

Grover

JM.

Alabdrabalnabi

Patel

et al. Measuring a crisis: questioning the use of naloxone administrations as a marker for opioid overdoses in a large U.S. EMS system. Prehosp Emerg Care. 2018;22(3):281-289.doi:10.1080/10903127.2017.1387628

29297739

Latkin

CA.

Gicquelais

RE.

Clyde

et al. Stigma and drug use settings as correlates of self-reported, non-fatal overdose among people who use drugs in Baltimore, Maryland. Int J Drug Policy. 2019;68:86-92.doi:10.1016/j.drugpo.2019.03.012

31026734

Ray

BR.

Lowder

EM.

Kivisto

AJ.

Phalen

Gil

. EMS naloxone administration as non-fatal opioid overdose surveillance: 6-year outcomes in Marion County, Indiana. Addiction. 2018;113(12):2271-2279.doi:10.1111/add.14426

30255531

Slavova

Costich

JF.

Bunn

et al. Heroin and fentanyl overdoses in Kentucky: epidemiology and surveillance. Int J Drug Policy. 2017;46:120-129.doi:10.1016/j.drugpo.2017.05.051

28735777

Cash

RE.

Kinsman

Crowe

RP.

Rivard

MK.

Faul

Panchal

. Naloxone administration frequency during emergency medical service events—United States, 2012-2016. MMWR Morb Mortal Wkly Rep. 2018;67(31):850-853.doi:10.15585/mmwr.mm6731a2

30091966

10.

Singleton

MD.

Rock

. Assessment of a surveillance case definition for heroin overdose in emergency medical services data. Online J Public Health Inform. 2018;10(1):e31. doi:10.5210/ojphi.v10i1.8353

11.

Lavonas

EJ.

Drennan

IR.

Gabrielli

et al. Part 10: special circumstances of resuscitation: 2015 American Heart Association guidelines update for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation. 2015;132(18 suppl 2):S501-S518.doi:10.1161/CIR.0000000000000264

26472998

12.

Boyer

. Management of opioid analgesic overdose. N Engl J Med. 2012;367(2):146-155.doi:10.1056/NEJMra1202561

22784117

13.

Prieto

JT.

Scott

McEwen

et al. The detection of opioid misuse and heroin use from paramedic response documentation: machine learning for improved surveillance. J Med Internet Res. 2020;22(1):e15645. doi:10.2196/15645

31899451

14.

Badger

LaRose

Mayer

Bashiri

Page

Peissig

. Machine learning for phenotyping opioid overdose events. J Biomed Inform. 2019;94:103185. doi:10.1016/j.jbi.2019.103185

31028874

15.

Hazlehurst

Green

CA.

Perrin

et al. Using natural language processing of clinical text to enhance identification of opioid-related overdoses in electronic health records data. Pharmacoepidemiol Drug Saf. 2019;28(8):1143-1151.doi:10.1002/pds.4810

31218780

16.

Lasher

Rhodes

Viner-Brown

. Identification and description of non-fatal opioid overdoses using Rhode Island EMS data, 2016-2018. R I Med J. 2019;102(2):41-45.30823701

17.

Deo

. Machine learning in medicine. Circulation. 2015;132(20):1920-1930.doi:10.1161/CIRCULATIONAHA.115.001593

26572668

18.

Beam

AL.

Kohane

. Big data and machine learning in health care. JAMA. 2018;319(13):1317-1318.doi:10.1001/jama.2017.18391

29532063

19.

Cervantes-Arslanian

AM.

Mercado

Greer

. Coma and brain death. In: Nelson

SE.

Nyquist

PA.

, eds. Neurointensive Care Unit. Springer; 2020:87-105.

20.

Center for Open Science . EMS opioid overdose project. Accessed March 29, 2021. https://osf.io/djp3z/?view_only=914a108355794cf9acb1d9ef748cc0e6

21.

Smith

Beger

Vadeboncoeur

et al. Trends in overdose-related out-of-hospital cardiac arrest in Arizona. Resuscitation. 2019;134:122-126.doi:10.1016/j.resuscitation.2018.10.019

30352247

22.

Dhingra

Kaur

Ram

. Illicit drugs: effects on eye. Indian J Med Res. 2015;150(3):228-238.doi:10.4103/ijmr.IJMR_1210_17

23.

Silge

Robinson

. Text Mining With R: A Tidy Approach. O’Reilly; 2017.

24.

Barak-Corren

Castro

VM.

Javitt

et al. Predicting suicidal behavior from longitudinal electronic health records. Am J Psychiatry. 2017;174(2):154-162.doi:10.1176/appi.ajp.2016.16010077

27609239

25.

Uddin

Khan

Hossain

ME.

Moni

. Comparing different supervised machine learning algorithms for disease prediction. BMC Med Inform Decis Mak. 2019;19(1):281. doi:10.1186/s12911-019-1004-8

31864346

26.

Boggs

JM.

Beck

Ritzwoller

DP.

Battaglia

Anderson

HD.

Lindrooth

. A quasi-experimental analysis of lethal means assessment and risk for subsequent suicide attempts and deaths. J Gen Intern Med. 2020;35(6):1709-1714.doi:10.1007/s11606-020-05641-4

32040838

27.

Landen

. Nonfatal opioid overdose standardized surveillance case definition. Paper presented at: Council of State and Territorial Epidemiologists; 2019. Accessed March 20, 2021. https://cdn.ymaws.com/www.cste.org/resource/resmgr/2019ps/final/19-CC-01_final_7.31.19.pdf

Characterizing Opioid Overdoses Using Emergency Medical Services Data

Abstract

Objectives

Methods

Results

Conclusions

Keywords

Methods

Data

Case Definition Algorithm

ML Model

Testing the Case Definition

Descriptive Results

Results

Comparison With Rhode Island Case Definition

Trends in Opioid Overdose and Naloxone Administration

Discussion

Limitations

Conclusions

Footnotes

Acknowledgments

Declaration of Conflicting Interests

Funding

ORCID iD

References