The Prevalence of Healthcare Effectiveness Data and Information Set (Hedis) Initiation and Engagement in Treatment among Patients with Cannabis Use Disorders in 7 Us Health Systems

Abstract

Background

Cannabis use disorders (CUDs) have increased with more individuals using cannabis, yet few receive treatment. Health systems have adopted the Healthcare Effectiveness Data and Information Set (HEDIS) quality measures of initiation and engagement in alcohol and other drug (AOD) dependence treatment, but little is known about the performance of these among patients with CUDs.

Methods

This cohort study utilized electronic health records and claims data from 7 health care systems to identify patients with documentation of a new index CUD diagnosis (no AOD diagnosis ≤60 days prior) from International Classification of Diseases, Ninth revision, codes (October 1, 2014, to August 31, 2015). The adjusted prevalence of each outcome (initiation, engagement, and a composite of both) was estimated from generalized linear regression models, across index identification settings (inpatient, emergency department, primary care, addiction treatment, and mental health/psychiatry), AOD comorbidity (patients with CUD only and CUD plus other AOD diagnoses), and patient characteristics.

Results

Among 15,202 patients with an index CUD diagnosis, 30.0% (95% confidence interval [CI]: 29.2–30.7%) initiated, 6.9% (95% CI: 6.2–7.7%) engaged among initiated, and 2.1% (95% CI: 1.9–2.3%) overall both initiated and engaged in treatment. The adjusted prevalence of outcomes varied across index identification settings and was highest among patients diagnosed in addiction treatment, with 25.0% (95% CI: 22.5–27.6%) initiated, 40.9% (95% CI: 34.8–47.0%) engaged, and 12.5% (95% CI: 10.0–15.1%) initiated and engaged. The adjusted prevalence of each outcome was generally highest among patients with CUD plus other AOD diagnosis at index diagnosis compared with those with CUD only, overall and across index identification settings, and was lowest among uninsured and older patients.

Conclusion

Among patients with a new CUD diagnosis, the proportion meeting HEDIS criteria for initiation and/or engagement in AOD treatment was low and demonstrated variation across index diagnosis settings, AOD comorbidity, and patient characteristics, pointing to opportunities for improvement.

Keywords

Cannabis comorbidity health services research quality indicators substance use disorder treatment

Introduction

Cannabis is the third most widely used substance in the United States after alcohol and tobacco, with 8.9% of the population aged 12 and older reporting current use in 2016.¹ The prevalence of cannabis use disorders (CUDs) has increased in the United States since 2002, with 3.5% of men and 1.5% of women having a current CUD, and is expected to increase as legal access expands.² –5 Although nearly 3 in 10 adults who use cannabis have a CUD, only 7.6% of individuals with a current diagnosis receive treatment.¹–3 ,6

The National Committee for Quality Assurance's (NCQA) Healthcare Effectiveness Data and Information Set (HEDIS) is the most widely used set of health care quality measures in the United States.⁷ HEDIS includes measures to assess Initiation and Engagement of Alcohol and Other Drug Dependence Treatment (IET).⁷,⁸ These health care system–level performance measures are intended to assess and promote system-wide efforts to assure patient referral and engagement in alcohol and other drug (AOD) treatment through electronic health record (EHR) documentation of early involvement in treatment following a new AOD diagnosis.⁹ Historically, most health care systems have underperformed on these measures⁹,¹⁰ and little is known about patient- and system-level factors associated with higher performance, aside from health care setting and specialty. Namely, patients who initiate AOD treatment as outpatients, compared with as inpatients, are more likely to engage, whereas patients in contact with specialty addiction treatment are more likely to initiate and engage compared with patients in other settings.11

To our knowledge, no study has evaluated the performance of HEDIS initiation and engagement in AOD treatment measures among patients with CUD. As a result, little is known about the association between characteristics of patients with CUD and these performance metrics. Using EHR and claims data from 7 health care systems within the Health Systems Node of the National Institute on Drug Abuse (NIDA) Clinical Trials Network (CTN), we sought to evaluate initiation and engagement in AOD treatment, among patients with a new CUD diagnosis, and assess variation across health care settings and patient characteristics. As other comorbid AOD disorders strongly predict treatment utilization among patients with CUD,¹²,¹³ we also evaluated outcomes stratified by patients based on AOD comorbidity: those with CUD only, and those with CUD plus other AOD disorder diagnoses.

Methods

Setting and study sample

This observational cohort study utilized EHR and claims data from 7 geographically diverse health care systems within the Health Systems Node of NIDA CTN. These health systems serve patients in 9 states, 3 of which had legalized nonmedical and medical cannabis use, 4 had legalized medical use, and 2 had no legalized cannabis use at the time of this study. Each health system utilizes a system-wide EHR and provides both health care and insurance coverage. Each health system also employs a common data model with harmonized EHR and medical insurance claims data, including diagnoses, procedures, and utilization, contained in a standardized data repository, the Virtual Data Warehouse (VDW), designed to support multisite research. The VDW was accessed locally by programmers at each site, and data were aggregated by the lead site into a single data set for analysis. Data included patient demographics, procedures, health care utilization, mental health and AOD diagnoses, and medical comorbidity diagnoses.

The study sample included adult patients (>18 years old) from each site who met criteria for enrollment and for the HEDIS AOD IET denominator between October 1, 2014, and August 31, 2015. Specifically, patients were eligible if (1) they had a health care encounter, including inpatient, outpatient, detoxification, or emergency department visit, with documentation of an index AOD diagnosis, defined as a primary or secondary AOD diagnosis without documentation of any AOD diagnosis in the prior 60 days; and (2) were continuously enrolled for the 2 months prior through 44 days post index identification date. Patients with an index AOD diagnosis that included a CUD diagnosis were included in the present analyses. Because patients could have had more than 1 AOD diagnosis documented at the index identification date (an indicator of substance use severity), 2 mutually exclusive subsamples of CUD index diagnosis groups were categorized: (1) patients with CUD only and (2) patients with CUD plus other AOD diagnoses. This study received approval and waivers of consent and HIPAA (Health Insurance Portability and Accountability Act of 1996) authorization from the institutional review boards at each site.

Measures

Measures at index CUD diagnosis

International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM), abuse and dependence diagnosis codes for an index AOD diagnosis were specified by HEDIS, including ICD-9-CM codes for cannabis use disorders (i.e., index CUD diagnosis; 305.2–22; 304.3–304.32). Indicators for other AOD disorder diagnoses at index included alcohol, opioid, stimulant (i.e., cocaine and amphetamine), and other/unspecified drug (i.e., hallucinogen, sedative, antidepressant, unspecified) abuse and dependence. These indicators were used to derive a count of other AOD diagnoses at index CUD diagnosis (0, 1, ≥2). Because the HEDIS AOD IET measures consider index identification setting,⁸,¹⁴ this measure was categorized into 5 health care settings: (1) inpatient; (2) emergency department; (3) primary care, including internal medicine, family practice, urgent care, and obstetrics/gynecology; (4) addiction treatment; and (5) mental health and psychiatry, with the latter 3 defined by outpatient visits.

Measures prior to index CUD diagnosis

ICD-9-CM codes were used to identify AOD and mental health disorders diagnosed in the year prior to the index encounter. Specifically, a indicator for any AOD diagnosis in the year prior to index included indicators for cannabis, alcohol, opioid, stimulant, and other drug use (e.g., barbiturate, sedative or hypnotic, hallucinogen) disorders, as well as ICD-9-CM codes for nicotine dependence. An indicator for any mental health disorder in the year prior to index included indicators for anxiety, depression, and serious mental illness. Medical comorbidity in the past year was characterized by the Charlson comorbidity index (score 0–2 vs. ≥3).¹⁵,¹⁶ Utilization included counts (≥1 visit) of emergency department, primary care, mental health/psychiatry, and other specialty care visits in the 60 days prior to index.

Other patient characteristics

Other patient characteristics measured at the time of index identification date include gender (male/female), age (18–29, 30–49, 50–64, ≥65 years), race/ethnicity (black/African American, Hispanic/Latino, white, other/unknown), and health insurance (commercial/private pay, Medicare, state subsidized, including Medicaid, unknown).

Outcome measures

The HEDIS AOD IET treatment indicators (i.e., 0/1) included (1) initiation, defined as an encounter with a documented AOD diagnosis within 14 days of index AOD diagnosis; and (2) engagement, among those who initiated treatment, defined as 2 or more similar encounters within 30 days after initiation. Consistent with HEDIS, patients diagnosed with an index AOD diagnosis while inpatient were assumed to have initiated. To evaluate the extent of initiation and engagement among patients with an index CUD diagnosis and account for bias as a result of engagement based on only those who initiate,¹⁷ a composite indicator of initiation & engagement was also evaluated.

Analyses

Patient-level analyses described sample characteristics overall and across CUD index diagnosis groups, with chi-square tests of independence used to test for significant differences between the 2 groups. The unadjusted probability and 95% confidence intervals for each of the 3 outcomes (initiation, engagement, and initiation & engagement) was calculated for comparison with NCQA published performance estimates.¹⁰ For these analyses, patients diagnosed during an inpatient stay were included in initiation estimates.

To estimate the adjusted probability of each outcome, overall and stratified by index CUD diagnosis group, generalized linear models with a logit link were used. Adjusted estimates for initiation excluded inpatients, as index identification during inpatient was equated with initiation consistent with HEDIS, whereas engagement included all patients who initiated, including inpatients. Robust standard errors were calculated using the sandwich estimator to account for correlation between patients from the same health system. Models were adjusted for covariates available in the data set, with known associations between CUD and treatment¹²,¹⁸,¹⁹ and significant bivariate associations between outcomes (Appendix A). These included gender, age, race/ethnicity, health insurance, Charlson comorbidity, other AOD diagnoses at index, any AOD diagnosis and/or any mental health diagnosis in the year prior to index, utilization in the 60 days prior to index, and index identification setting. Results are presented as the average adjusted predicted probability of each outcome based on recycled predictions.²⁰,²¹ Along with stratification by CUD index diagnosis group, results are presented across index identification settings and patient characteristics. Analyses were completed using Stata version 15.0 MP edition.22

Results

Sample characteristics

Among the 86,565 patients with an index AOD diagnosis during the study period, 15,202 (17.6%) had an index diagnosis of CUD and were included. The sample was mostly male (63.1%), younger (47.8% 18–29 years old), and white (52.2%) (Table 1). The index CUD diagnosis most frequently occurred in primary care (33.5%), followed by emergency departments (29.4%), inpatient stays (23.6%), mental health and psychiatry visits (10.2%), and addiction treatment settings (3.7%), with significant variation between patients with and without other AOD diagnosis at index diagnosis. Among all patients with an index CUD, 74.6% had no other AOD diagnosis at index, whereas the remaining 25.4% had other AOD diagnoses at index diagnosis: 19.1% with 1 and 6.4% with 2 or more additional AOD diagnoses at index, with 58.3% of patients with other AOD diagnoses at index having an alcohol use disorder. Patients with and without other AOD diagnoses at index differed across all patient characteristics (Table 1). For example, in the year prior to index diagnosis, patients with other AOD diagnoses experienced a greater burden of AOD (50.1% vs. 37.1%; p < .001) and mental health disorders (56.6% vs. 47.2%; p < .001) compared with patients with CUD only.

Table 1

Patients with an index cannabis use disorder (CUD) diagnosis.

	CUD only(n = 11,338)		CUD plus other AOD(n = 3,864)		All patients with CUD(n = 15,202)
Characteristic	n	%	n	%	n	%
Female	4,372	(38.6)	1,241	(32.1)*	5,613	(36.9)
Age categories*
18–29	5,403	(47.7)	1,859	(48.1)	7,262	(47.8)
30–49	2,898	(25.6)	1,136	(29.4)	4,034	(26.5)
50–64	2,210	(19.5)	664	(17.2)	2,874	(18.9)
≥65	827	(7.3)	205	(5.3)	1,032	(6.8)
Race/ethnicity*
Black/African American	2,023	(17.8)	546	(14.1)	2,569	(16.9)
Hispanic	2,257	(19.9)	797	(20.6)	3,054	(20.1)
White	5,854	(51.6)	2,080	(53.8)	7,934	(52.2)
Other/unknown	1,204	(10.6)	442	(11.4)	1,645	(10.8)
Insurance*
Commercial/private pay	7,861	(69.3)	2,825	(73.1)	10,686	(70.3)
Medicare	1,436	(12.7)	414	(10.7)	1,850	(12.2)
State subsidized	1,672	(14.7)	489	(12.7)	2,161	(14.2)
Unknown	369	(3.3)	136	(3.5)	505	(3.3)
Other AOD disorder diagnoses at index
Alcohol	—	—	2,254	(58.3)*	2,254	(14.8)
Opioid	—	—	530	(13.7)*	530	(3.5)
Stimulant	—	—	1,075	(27.8)*	1,075	(7.1)
Other drug	—	—	1,141	(29.5)*	1,141	(7.5)
Carlson comorbidity score ≥3 in prior year	970	(8.6)	267	(6.9)*	1,237	(8.1)
Any AOD disorder in year prior to index	4,212	(37.1)	1,936	(50.1)*	6,148	(40.4)
Cannabis	3,929	(34.7)	1,630	(42.2)*	5,559	(36.6)
Nicotine	2,492	(22.0)	1,201	(31.1)*	3,693	(24.3)
Alcohol	601	(5.3)	1,139	(29.5)*	1,740	(11.4)
Opioid	155	(1.4)	358	(9.3)*	513	(3.4)
Stimulant	240	(2.1)	618	(16.0)*	858	(5.6)
Other drug	57	(0.5)	116	(3.0)*	173	(1.1)
Any mental health disorder diagnosis**	5,352	(47.2)	2,188	(56.6)*	7,540	(49.6)
Anxiety	3,101	(27.4)	1,247	(32.3)*	4,348	(28.6)
Depression	3,179	(28.0)	1,313	(34.0)*	4,492	(29.5)
Serious mental illness^†	1,799	(15.9)	937	(24.2)*	2,736	(18.0)
Utilization in 60 days prior to index episode
Emergency department ≥1 visit	2,093	(18.5)	914	(23.7)*	3,007	(19.8)
Primary care ≥1 visit	4,466	(39.4)	1,257	(32.5)*	5,723	(37.6)
Mental Health & psychiatry ≥1 visit	734	(6.5)	338	(8.7)*	1,072	(7.1)
Other specialty ≥1 visit	2,332	(20.6)	599	(15.5)*	2,931	(19.3)
Index identification setting*
Inpatient	2,436	(21.5)	1,152	(29.8)	3,588	(23.6)
Emergency department	3,316	(29.3)	1,100	(28.5)	4,416	(29.1)
Primary care	4,349	(38.4)	748	(19.4)	5,097	(33.5)
Addiction treatment (outpatient)	112	(1.0)	444	(11.5)	556	(3.7)
Mental Health & psychiatry	1,125	(9.9)	420	(10.9)	1,545	(10.2)

Note. CUD = cannabis use disorder; AOD = alcohol and other drugs.

Chi-square test of independence significant at p < .000.

In year prior to index diagnosis.

Unadjusted prevalence of initiation and engagement

For all patients with CUD, the unadjusted prevalence of initiation was 30.0% (95% confidence interval [CI]: 29.2–30.7%), the majority (78.0%) of which was accounted for by inpatient encounters, which, by definition, met HEDIS guidelines for initiation. Among patients who initiated treatment, 6.9% (95% CI: 6.2–7.7%) engaged in treatment and 2.1% (95% CI: 1.9–2.3%) of all patients initiated & engaged in treatment.

Adjusted prevalence of initiation and engagement

The adjusted prevalence of initiation, excluding patients with inpatient index diagnoses, varied across index identification settings and was highest for patients diagnosed in addiction treatment settings (25.0% [95% CI: 22.5–27.6%]; Table 2). Initiation was also highest for patients with other AOD diagnoses at index, which ranged between 10.4% (95% CI: 4.5–16.5%) and 42.9% (95% CI: 37.4–48.4%), depending on index identification setting, compared with patients with CUD only at index, which ranged between 3.8% (95% CI: 1.9–5.9%) and 21.6% (95% CI: 19.7–23.6%) (Table 2).

Table 2

Among patients with an index diagnosis of CUD, the adjusted* prevalence of initiation and engagement in AOD treatment by index identification setting.

	Inpatient(n = 3,588)		Emergency department(n = 4,416)		Primary care(n = 5,097)		Addiction treatment(n = 556)		Mental health & psychiatry(n = 1,545)		Total(n = 15,202)
	Inpatient(n = 3,588)		Emergency department(n = 4,416)		Primary care(n = 5,097)		Addiction treatment(n = 556)		Mental health & psychiatry(n = 1,545)		Total(n = 15,202)
Initiation/engagement	%	95% CI	%	95% CI	%	95% CI	%	95% CI	%	95% CI	%	95% CI
Initiation**
All patients with a CUD	na	na	9.2	(6.4–12.0)	6.2	(4.7–7.6)	25.0	(22.5–27.6)	5.5	(2.5–8.5)	na	na
CUD only	na	na	6.8	(4.1–9.4)	4.4	(3.6–5.1)	21.6	(19.7–23.6)	3.8	(1.9–5.9)	na	na
CUD plus other AOD	na	na	17.1	(14.0–20.2)	11.7	(8.0–15.4)	42.9	(37.4–48.4)	10.4	(4.3–16.5)	na	na
	Inpatient(n = 3,547)		Emergency department(n = 449)		Primary care(n = 231)		Addiction treatment(n = 227)		Mental health & psychiatry(n = 100)		Total(n = 4,554)
Engagement (among those who initiated)^†
All patients with a CUD	2.9	(2.0–3.9)	7.3	(5.1–9.6)	15.8	(11.9–19.7)	40.9	(34.8–47.0)	31.3	(12.7–49.9)	6.9	(5.6–8.2)
CUD only	2.3	(1.8–2.7)	5.7	(4.6–6.8)	12.7	(9.5–15.9)	35.4	(26.8–44.1)	26.4	(8.1–44.8)	5.6	(4.7–6.4)
CUD plus other AOD	4.2	(2.1–6.2)	10.2	(5.9–14.5)	21.4	(14.9–27.9)	50.6	(45.4–55.8)	40.1	(20.8–59.4)	8.4	(6.1–10.7)
	Inpatient(n = 3,588)		Emergency department(n = 4,416)		Primary care(n = 5,097)		Addiction treatment(n = 556)		Mental health & psychiatry(n = 1,545)		Total(n = 15,202)
Initiation & engagement (among all patients)^‡
All patients with a CUD	2.0	(1.2–2.8)	1.0	(0.5–1.4)	0.9	(0.7–1.1)	12.5	(10.0–15.1)	2.2	(−0.2–4.7)	2.1	(1.7–2.4)
CUD only	1.4	(0.8–2.0)	0.7	(0.4–0.9)	0.6	(0.5–0.8)	9.8	(7.3–12.4)	1.5	(−0.2–3.3)	1.4	(1.1–1.6)
CUD plus other AOD	3.8	(2.1–5.5)	1.8	(0.9–2.7)	1.7	(1.4–2.0)	22.5	(19.8–25.2)	4.1	(−0.2–8.4)	3.1	(2.5–3.7)

Note. CUD = cannabis use disorder; AOD = alcohol and other drugs.

Adjusted for age, gender, race/ethnicity, health insurance, Charlson comorbidity, any mental health and any AOD disorder diagnoses in year prior to index, and utilization in 60 days prior to index.

Initiation estimates excluded inpatient index identification setting.

†

Engagement was estimated among all patients who initiated, including patients with an index inpatient encounter (n = 4,554).

‡

Initiation & engagement was estimated among all patients.

The adjusted prevalence of engagement, among all who initiated, also varied across index identification settings, ranging from 2.9% (95% CI: 2.0–3.9%) for inpatient to 40.9% (95% CI: 34.8–47.0%) for addiction treatment settings, and was generally highest among patients with other AOD diagnoses at index (8.4% [95% CI: 6.1–10.7%] compared with patients with CUD only at index (5.6% [95% CI: 6.1–10.7%]).

Finally, the adjusted prevalence of initiation & engagement, among all patients, varied across index identification settings, which ranged from 0.9% (95% CI: 0.7–1.1%) for primary care to 12.5% (95% CI: 10.0–15.1%) for addiction treatment settings and was also highest among patients with other AOD diagnoses at index (3.1% [95% CI: 2.5–3.7%]), compared with patients with CUD only at index (1.4% [95% CI: 1.1–1.6%]). For all 3 outcomes, prevalence estimates were generally lowest among uninsured and older patients (≥65 years) and highest among patients with other AOD diagnoses at index diagnosis (Table 3; Appendix B).

Table 3

Adjusted* prevalence of initiation & engagement** across CUD index groups.

	All patients with CUD(n = 15,202)		CUD only(n = 11,338)		CUD plus other AOD(n = 3,864)
Characteristic	%	95% CI	%	95% CI	%	95% CI
Gender
Female	1.7	(1.4–2.1)	0.9	(0.6–1.1)	4.5	(3.5–5.6)
Male	2.2	(1.8–2.7)	1.2	(0.9–1.5)	5.3	(4.3–6.4)
Age categories
18–29	2.2	(1.6–2.8)	1.3	(1.0–1.7)	4.9	(3.5–6.2)
30–49	2.4	(2.1–2.6)	1.1	(0.8–1.4)	6.2	(5.8–6.5)
50–64	1.6	(1.3–2.0)	0.7	(0.5–1.0)	4.4	(3.5–5.2)
≥65	1.2	(0.4–1.9)	0.6	(0.2–1.0)	1.8	(0.8–2.9)
Race/ethnicity
Black/African American	1.7	(1.5–1.9)	0.9	(0.7–1.1)	4.0	(2.8–5.2)
Hispanic	2.2	(1.6–2.7)	1.0	(0.8–1.2)	5.4	(3.7–7.1)
White	2.1	(1.8–2.4)	1.1	(0.8–1.4)	5.2	(4.6–5.8)
Other/unknown	2.2	(1.7–2.7)	1.0	(0.4–1.7)	5.3	(4.6–6.0)
Insurance
Commercial/private pay	2.2	(1.8–2.6)	1.0	(0.8–1.3)	5.5	(4.7–6.2)
Medicare	2.1	(1.5–2.8)	1.4	(1.0–1.7)	4.1	(2.2–6.0)
State subsidized	1.6	(1.0–2.2)	0.9	(0.2–1.7)	3.4	(2.2–4.6)
Unknown	0.8	(0.2–1.3)	0.3	(0.4–0.9)	2.2	(1.5–2.9)
Other AOD disorder diagnoses at index
1	2.8	(2.2–3.5)	na	na	4.6	(3.7–5.5)
≥2	3.6	(2.9–4.4)	na	na	6.1	(5.2–7.0)
Charlson comorbidity score ≥3 in prior year	3.0	(2.2–3.8)	2.2	(1.3–3.1)	4.9	(3.3–6.5)
Any AOD use disorder in year prior to index	2.3	(1.9–2.7)	1.3	(0.8–1.7)	5.1	(4.4–5.8)
Any mental health disorder in prior year	2.3	(1.8–2.7)	1.2	(0.9–1.5)	5.3	(4.4–6.2)
Utilization in 60 days prior to index episode
Emergency department ≥1 visit	2.9	(2.4–3.4)	1.6	(1.0–2.1)	6.7	(5.8–7.6)
Primary care ≥1 visit	1.9	(1.4–2.4)	1.0	(0.6–1.3)	4.6	(3.7–5.4)
Psychiatry ≥1 visit	2.7	(0.9–4.6)	1.9	(0.5–3.4)	5.2	(2.2–8.2)
Other specialty ≥1 visit	2.2	(1.9–2.5)	0.9	(0.7–1.1)	6.3	(4.1–8.5)

Note. CUD = cannabis use disorder; AOD = alcohol and other drugs.

Adjusted for age, gender, race/ethnicity, health insurance, Charlson comorbidity, any mental health and any AOD disorder diagnoses in year prior to index, and utilization in 60 days prior to index.

Initiation & engagement was estimated among all patients in each index CUD group.

Discussion

This study evaluated rates of initiation and engagement in AOD treatment among patients with a new CUD diagnosis in 7 health care systems across the United States, as measured by HEDIS performance measures. About one third initiated treatment, and among those who initiated, less than 7% engaged in treatment. Overall, 2% of patients with a CUD initiated and engaged in treatment for alcohol and other drug use disorders, based on HEDIS criteria. However, rates varied considerably depending on patient and clinical characteristics. For each outcome, rates were highest among patients diagnosed in addiction treatment settings, despite contributing the fewest patients among those with CUD, compared with other settings, and among those with other AOD diagnoses at the time of diagnosis compared with those with CUD only. Rates were typically lowest among uninsured and older patients.

This is the first study to assess HEDIS initiation and engagement performance measures among patients with a new episode of CUD, and how these results compare with patients with CUD in other health systems is not known. Among all patients newly diagnosed with AOD disorders, the HEDIS-defined initiation rate has ranged between 17% and 40.8% depending on health care system, region, and insurance type.¹⁰,²³–25 The 2016 HEDIS IET performance rates reported by NCQA for health maintenance organizations ranged between 32.7% and 40.8%, among patients with Medicare, commercial, and Medicaid coverage.¹⁰ Similarly, the engagement rate among those who initiated ranged between 3.6% and 12.5%, with initiation & engagement among all patients ranging between 1.2% and 5.1%.¹⁰ The comparable rates of these measures found among patients with a new episode of CUD in this study suggest that HEDIS measures may have similar results for specific substance use disorders, such as CUD, as for AOD disorders overall.

Consistent with previous findings, this study found that the health care setting of a newly diagnosed episode of CUD was associated with significant variation in initiation and engagement in treatment, with patients diagnosed during an addiction treatment visit demonstrating the highest rates compared with other settings.¹¹,²⁴,²⁶ Notably, less than 4% of patients with CUD were identified in addiction treatment settings, and the majority (80%) had multiple AOD disorders when diagnosed. Patients with CUD presenting to addiction treatment may have higher motivation to self-refer or act upon referral to treatment and to engage in treatment. Although 65% of patients with CUD who initiated and engaged in AOD treatment had been identified in settings other than addiction treatment, nearly half had been diagnosed while inpatient (where identification is equated with initiation), limiting comparisons with other settings. Factors that may influence differences in initiation and engagement in other settings include the underlying prevalence of CUD among patients, which can impact provider awareness and incentive to facilitate treatment, and the influence of CUD and other AOD symptom severity on patient selection of visit setting.²⁷,²⁸ Despite these potential differences, rates of initiation and engagement, although variable across settings, were low, indicating that efforts to improve health system rates of treatment initiation and engagement for patients with CUD are needed.

This study also found that a new encounter of CUD often occurred with other AOD diagnoses at the time of diagnosis. Individuals with CUDs frequently have comorbid alcohol, illicit drug, and prescription medication use disorders,²⁹–31 and a quarter of patients with a new episode of CUD in this study had at least 1 other AOD diagnosis. Consistent with previous findings,¹² the results from this study highlight that AOD severity among patients with CUD is associated with a higher likelihood of initiating and engaging in AOD treatment compared with patients with CUD only. However, this study assessed initiation and engagement documented for any AOD diagnosis, not just CUD (e.g., initiation could be for an alcohol use disorder, not CUD), consistent with HEDIS, and other factors may explain these differences. For example, patients with other AOD use disorder may have had more initiation opportunities resulting from multiple AOD use disorders. Other AOD use disorder comorbidity could have also contributed to patient willingness to seek treatment and triggered enhanced treatment linkage and retention, whereas patients with CUD only may have been less likely to be encouraged to initiate treatment, despite the potential to benefit from treatment.

Recognition and assessment of patients for CUD and other AOD diagnoses is a key prerequisite of HEDIS AOD treatment measures, and several factors could have contributed to a whether a patients was diagnosed with CUD in this study.⁹ Patients in states with medical and nonmedical cannabis use laws may have been more willing to acknowledge cannabis use, with providers more willing to ask about use, and AOD comorbidity at presentation and in the prior year likely influenced recognition and diagnosis. Conversely, providers in busy medical settings, including primary care and emergency departments, may not feel they have the time or tools necessary to diagnosis patients, and some patients may have only received an AOD diagnosis at the time they were willing to enter treatment (e.g., indication bias). Moreover, routine cannabis and drug screening is not currently recommended, and underrecognition of AOD use disorder in medical settings remains a concern.³²–34 For example, the prevalence of CUD among primary care patients in one health system included here was found to be considerably lower than population estimates.¹,²,³⁵ Yet recent efforts to integrate screening and assessment in the same system improved recognition and diagnosis of AOD use disorder.36

Whether adherence to HEDIS treatment measures impacts cannabis use or CUD symptoms is not known. Assessed at the health system level, AOD treatment initiation and engagement have been associated with meaningful but clinically modest substance use symptom improvement²³,³⁷,³⁸ and may be most meaningful for specific subgroups, such as those mandated to and/or attending specialty AOD treatment.³⁹–41 However, the potential influence of HEDIS measures on outcomes among patients with CUD is unknown, and research is needed address this gap.

This study has several limitations. HEDIS measures of initiation and engagement are based on EHR documentation of a visit-based qualifying AOD diagnosis. The quality and type of AOD treatment based on these measures is not known and likely varies by health setting and site.²⁶ Moreover, this study specifically examined HEDIS measures and did not consider broader definitions (e.g., longer initiation period to account for potential delays in treatment access; brief treatment interventions) that may have captured other AOD treatment initiation and engagement.²⁴ The treatment options available to patients with CUD in this study were not captured, and it is unclear what evidence-based treatments for CUD would have been available across systems and settings or what treatment may have been missed outside the system (e.g., self-help groups, out-of-pocket treatment). No medications have been approved or demonstrated to be broadly effective for the treatment of CUD,⁴²,⁴³ and psychosocial therapies, including cognitive behavioral and motivational enhancement therapies, remain the first line of treatment.⁴²,⁴⁴ The potential influence of legalized cannabis use on patient initiation and engagement in AOD treatment is unknown. Providers may be uncertain how to diagnose and advise patients who use cannabis for medical reasons who may also have a CUD, and patients may not view their medical or recreational cannabis use as problematic, leading to underdiagnosis and lower treatment acceptance. The study sample was composed of mostly insured patients enrolled in large health systems, and the generalizability to other systems, such as federally qualified health centers, is unclear. For the minority with unknown insurance, the prevalence of initiation and engagement could be low if patients were uninsured, restricting available treatment data to encounters within the health system. HEDIS measures are meant as health system–level quality measures, and results found here were aggregated across health systems to explore associations among patients with CUD.

This study also has important strengths. The 7 integrated health systems included had access to common EHR data elements based on reliable measures. Patients with CUDs in this study represent those in several states with legal access to cannabis, with legalization in the United States expected to continue to expand. Although the rates of initiation and treatment among CUD patients in this study were low, several health systems included have initiated efforts to improve access to appropriate AOD treatment.36

In summary, among patients with a new episode of CUD, rates of HEDIS-defined AOD treatment initiation and engagement were generally low, demonstrating missed opportunities to initiate and retain patients in treatment, yet the findings highlighted important variation in these measures across health care settings and AOD comorbidity. Greater adherence to these system-level treatment measures will likely require strong leader and provider partnerships to improve recognition of as well as linkage to and retention in evidence-based treatment for CUD, particularly for patients with less AOD severity who could still benefit from treatment.

Footnotes

Acknowledgments

We thank Malia Oliver, BS, at Kaiser Permanente Washington Health Research Institute and Andrea Altschuler, PhD, at Kaiser Permanente Northern California for their contributions.

Author contributions

G.T.L. conceived of the analysis, collected data, conducted the analysis, and wrote the manuscript. B.A., C.C., R.C.H., I.V.H., and B.J.Y. contributed data and critically revised the manuscript, and D.D.S. and A.L. critically revised the manuscript. A.H.K.-S. coordinated data collection across sites, oversaw analysis, and drafted portions of the Methods section. C.W. oversaw research conception and design and critically revised the manuscript. I.A.B. contributed data, contributed to the conception of the analyses, and critically revised the manuscript.

Funding

This study was supported by a grant from the National Institute of Drug Abuse (NIDA) (UG1DA040314, CTN Protocol 0072-OT). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The NIDA Clinical Trials Network reviewed the study protocol, and the NIDA Clinical Trials Network publications committee reviewed and approved the manuscript for publication. The funding organization had no role in the collection, management, analysis, and interpretation of the data or decision to submit the manuscript for publication.

Appendix A. Patient characteristics associated with initiation & engagement in treatment

	All patients with CUD					CUD only					CUD plus other AOD
	No I&E(n = 14,887)		I&E(n = 315)			No I&E(n = 11,220)		I&E(n = 118)			No I&E(n = 3,667)		I&E(n = 197)
	No I&E(n = 14,887)		I&E(n = 315)			No I&E(n = 11,220)		I&E(n = 118)			No I&E(n = 3,667)		I&E(n = 197)
Characteristic	n	%	n	%	p value	n	%	n	%	p value	n	%	n	%	p value
Gender					.002					.154					.078
Female	5,523	(37.1)	90	(28.6)		4,334	(38.6)	38	(32.2)		1,189	(32.4)	52	(26.4)
Male	9,364	(62.9)	225	(71.4)		6,886	(61.4)	80	(67.8)		2,478	(67.6)	145	(73.6)
Age categories					.002					.960					.003
18–29	7,107	(47.7)	155	(49.2)		5,345	(47.6)	58	(49.2)		1,762	(48.1)	97	(49.2)
30–49	3,930	(26.4)	104	(33.0)		2,867	(25.6)	31	(26.3)		1,063	(29.0)	73	(37.1)
50–64	2,828	(19.0)	46	(14.6)		2,189	(19.5)	21	(17.8)		639	(17.4)	25	(12.7)
≥65	1,022	(6.9)	10	(3.2)		819	(7.3)	8	(6.8)		203	(5.5)	2	(1.0)
Race/ethnicity					.188					.889					.469
Black/African American	2,530	(17.0)	39	(12.4)		2,005	(17.9)	18	(15.3)		525	(14.3)	21	(10.7)
Hispanic	2,985	(20.1)	69	(21.9)		2,234	(19.9)	23	(19.5)		751	(20.5)	46	(23.4)
White	7,763	(52.1)	171	(54.3)		5,790	(51.6)	64	(54.2)		1,973	(53.8)	107	(54.3)
Other/unknown	1,609	(10.8)	36	(11.4)		1,191	(10.6)	13	(11.0)		418	(11.4)	23	(11.7)
Insurance					.000					.369					.000
Commercial/private pay	10,429	(70.1)	257	(81.6)		7,777	(69.3)	84	(71.2)		2,652	(72.3)	173	(87.8)
Medicare	1,822	(12.2)	28	(8.9)		1,418	(12.6)	18	(15.3)		404	(11.0)	10	(5.1)
State subsidized	2,134	(14.3)	27	(8.6)		1,657	(14.8)	15	(12.7)		477	(13.0)	12	(6.1)
Unknown	502	(3.4)	3	(1.0)		368	(3.3)	1	(0.8)		134	(3.7)	2	(1.0)
Other AOD disorder diagnoses at index															.000
1	2782	18.7	114	36.2		na	na	na	na		2,782	(75.9)	114	(57.9)
≥2	885	5.9	83	26.3		na	na	na	na		885	(24.1)	83	(42.1)
Charlson comorbidity score ≥3 in prior year	1,207	(8.1)	30	(9.5)	.363	950	(8.5)	20	(16.9)	.001	257	(7.0)	10	(5.1)	.298
Any AOD use disorder in year prior to index	5,964	(40.1)	184	(58.4)	.000	4,135	(36.9)	77	(65.3)	.000	1,829	(49.9)	107	(54.3)	.225
Any mental health disorder in prior year	7,338	(49.3)	202	(64.1)	.000	5,267	(46.9)	85	(72.0)	.000	2,071	(56.5)	117	(59.4)	.421
Utilization in 60 days prior to index episode
Emergency department ≥1 visit	2,912	(19.6)	95	(30.2)	.000	2,053	(18.3)	40	(33.9)	.000	859	(23.4)	55	(27.9)	.148
Primary care ≥1 visit	5,616	(37.7)	107	(34.0)	.173	4,418	(39.4)	48	(40.7)	.773	1,198	(32.7)	59	(29.9)	.427
Mental health/psychiatry ≥1 visit	1,034	(6.9)	38	(12.1)	.000	715	(6.4)	19	(16.1)	.000	319	(8.7)	19	(9.6)	.647
Other specialty ≥1 visit	2,873	(19.3)	58	(18.4)	.693	2,306	(20.6)	26	(22.0)	.692	567	(15.5)	32	(16.2)	.768
Index identification setting					.000					.000					.000
Inpatient	3,487	(23.4)	101	(32.1)		2,392	(21.3)	44	(37.3)		1,095	(29.9)	57	(28.9)
Emergency department	4,377	(29.4)	39	(12.4)		3,302	(29.4)	14	(11.9)		1,075	(29.3)	25	(12.7)
Primary care	5,065	(34.0)	32	(10.2)		4,327	(38.6)	22	(18.6)		738	(20.1)	10	(5.1)
Addiction treatment (outpatient)	447	(3.0)	109	(34.6)		93	(0.8)	19	(16.1)		354	(9.7)	90	(45.7)
Mental health & psychiatry	1,511	(10.1)	34	(10.8)		1,106	(9.9)	19	(16.1)		405	(11.0)	15	(7.6)

Note. CUD = cannabis use disorder; AOD = alcohol and other drugs.

* Initiation & engagement was estimated among all patients in each CUD group.

Appendix B. Adjusted* prevalence of initiation and engagement across CUD index groups

	Initiation**						Engagement^†
	All patients with CUD(n = 11,607)		CUD only(n = 8,896)		CUD plus other AOD(n = 2,711)		All patients with CUD(n = 4,554)		CUD only(n = 2,921)		CUD plus other AOD(n = 1,633)
Characteristic	%	95% CI	%	95% CI	%	95% CI	%	95% CI	%	95% CI	%	95% CI
Gender
Female	8.0	(7.4–8.5)	4.9	(3.9–5.8)	18.2	(16.1–20.2)	6.2	(5.2–7.1)	3.6	(2.8–4.3)	11.2	(9.4–13.0)
Male	9.1	(8.4–9.8)	5.9	(5.2–6.5)	19.6	(18.2–21.0)	7.3	(5.5–9.1)	4.3	(3.6–5.1)	12.4	(8.7–16.2)
Age categories
18–29	9.1	(8.4–9.9)	6.5	(5.6–7.5)	18.1	(16.6–19.5)	7.5	(5.3–9.7)	4.7	(3.6–5.8)	12.5	(8.3–16.6)
30–49	8.8	(7.9–9.7)	4.9	(3.8–6.0)	21.6	(19.7–23.5)	7.6	(6.9–8.3)	4.3	(3.5–5.0)	13.4	(11.5–15.3)
50–64	7.5	(6.3–8.7)	4.2	(3.2–5.1)	18.2	(15.3–21.2)	5.7	(4.4–6.9)	3.3	(2.5–4.0)	10.1	(6.8–13.4)
≥65	7.4	(4.9–10.0)	4.3	(2.7–5.8)	17.7	(10.6–24.8)	3.8	(2.1–5.4)	2.5	(1.0–3.9)	4.2	(2.4–6.1)
Race/ethnicity
Black/AA	8.5	(7.9–9.0)	5.2	(4.4–6.0)	19.4	(16.4–22.4)	5.6	(4.9–6.3)	3.7	(3.2–4.2)	9.0	(7.5–10.4)
Hispanic	8.2	(7.0–9.3)	5.1	(3.6–6.6)	18.1	(16.5–19.8)	7.7	(5.8–9.6)	4.4	(3.8–5.1)	13.3	(8.6–18.0)
White	9.0	(8.4–9.5)	5.7	(4.7–6.7)	20.0	(18.5–21.6)	6.9	(5.7–8.2)	4.0	(3.6–4.5)	12.2	(9.7–14.7)
Other/unknown	8.5	(6.1–11.0)	5.9	(3.7–8.1)	16.6	(11.7–21.6)	7.4	(5.3–9.4)	4.0	(2.6–5.5)	13.2	(9.2–17.3)
Insurance
Commercial/private pay	9.2	(8.6–9.8)	5.8	(5.3–6.4)	20.2	(18.3–22.1)	7.0	(5.6–8.4)	3.9	(3.3–4.6)	12.6	(10.0–15.2)
Medicare	8.0	(6.9–9.2)	5.3	(4.0–6.6)	17.1	(14.8–19.3)	7.7	(5.4–10.0)	5.3	(3.5–7.2)	11.1	(5.5–16.6)
State subsidized	7.3	(5.5–9.1)	4.5	(2.8–6.2)	16.2	(13.3–19.2)	6.0	(4.5–7.6)	3.8	(2.0–5.6)	9.2	(4.4–14.0)
Unknown	5.7	(4.1–7.4)	4.2	(2.2–6.1)	10.1	(8.6–11.5)	4.1	(1.0–7.3)	1.7	–(2.8–6.1)	8.3	(6.6–10.0)
Other AOD disorder diagnoses at index
1	12.1	(10.4–13.8)	na	na	16.5	(14.7–18.4)	8.5	(5.8–11.2)	na	na	11.9	(8.6–15.2)
≥2	18.7	(17.0–20.4)	na	na	25.6	(23.7–27.5)	8.3	(6.3–10.3)	na	na	12.3	(9.6–14.9)
Charlson comorbidity score ≥3 in prior year	9.7	(5.3–14.1)	7.4	(2.4–12.5)	15.1	(9.5–20.7)	9.5	(7.8–11.1)	6.8	(5.4–8.2)	13.1	(9.9–16.2)
Any AOD use disorder in year prior to index	12.6	(11.1–14.1)	8.9	(7.9–9.9)	24.9	(20.4–29.3)	6.9	(5.8–8.0)	4.7	(3.6–5.7)	11.1	(8.3–13.8)
Any mental health disorder in prior year	9.8	(9.0–10.5)	6.6	(5.7–7.6)	20.3	(19.7–20.8)	7.5	(5.7–9.2)	4.5	(4.0–5.0)	12.6	(9.4–15.7)
Utilization in 60 days prior to index episode
Emergency department ≥1 visit	18.2	(13.1–23.3)	12.8	(8.8–16.7)	35.9	(27.2–44.6)	7.5	(6.2–8.8)	4.4	(3.7–5.2)	12.7	(9.9–15.6)
Primary care ≥1 visit	8.6	(7.2–10.0)	5.7	(4.6–6.8)	17.7	(15.3–20.2)	6.2	(4.8–7.5)	3.6	(2.7–4.5)	10.7	(8.9–12.5)
Mental health/psychiatry ≥1 visit	12.1	(9.8–14.4)	8.3	(6.0–10.6)	24.6	(21.5–27.7)	7.8	(3.2–12.5)	6.1	(3.0–9.2)	10.6	(4.3–16.9)
Other specialty ≥1 visit	7.7	(7.1–8.3)	4.7	(3.4–6.0)	17.6	(13.3–21.8)	7.1	(5.9–8.4)	3.5	(2.5–4.5)	14.7	(8.2–21.2)

Note. CUD = cannabis use disorder; AOD = alcohol and other drugs.

* Adjusted for age, gender, race/ethnicity, health insurance, Charlson comorbidity, any mental health and any AOD disorder diagnoses in year prior to index, and utilization in 60 days prior to index.

Initiation estimates excluded patients diagnosed inpatient.

†

Engagement was estimated among all patients who initiated, including patients with an index inpatient identification encounter (n = 4,554).

References

National Survey on Drug Use and Health. Key Substance Use and Mental Health Indicators in the United States: Results from the 2016 National Survey on Drug Use and Health. https://www.samhsa.gov/data/sites/default/files/NSDUH-FFR1-2016/NSDUH-FFR1-2016.pdf. Published 2017.

Hasin

, Kerridge

, Saha

, . Prevalence and correlates of DSM-5 cannabis use disorder, 2012–2013: findings from the national epidemiologic survey on alcohol and related conditions-III. Ajp. 2016;173(6):588–599.

Hasin

, Saha

, Kerridge

, . Prevalence of marijuana use disorders in the United States between 2001-2002 and 2012-2013. JAMA Psychiatry. 2015;72(12):1235–1242.

Hall

, Weier

. Assessing the public health impacts of legalizing recreational cannabis use in the USA. Clin Pharmacol Ther. 2015;97(6):607–615.

Kerridge

, Pickering

, Chou

, Saha

, Hasin

. DSM-5 cannabis use disorder in the National Epidemiologic Survey on Alcohol and Related Conditions-III: Gender-specific profiles. Addict Behav. 2018;76:52–60.

Hasin

, O'Brien

, Auriacombe

, . DSM-5 criteria for substance use disorders: recommendations and rationale. Am J Psychiatry. 2013;170(8):834–851.

National Committee for Quality Assurance. HEDIS 2015 Technical Specifications. Vol 2. Washington, D.C.: National Committee for Quality Assurance; 2015.

Garnick

, Lee

, Horgan

, . Lessons from five states: public sector use of the Washington Circle performance measures. J Subst Abuse Treat. 2011;40(3):241–254.

Garnick

, Horgan

, Chalk

. Performance measures for alcohol and other drug services. Alcohol Res Health. 2006;29(1):19–26.

10.

National Committee for Quality Assurance. The state of health care quality. Available from: http://www.ncqa.org/report-cards/health-plans/state-of-health-care-quality/2017-table-of-contents/alcohol-treatment. Published 2017.

11.

Harris

, Bowe

, Finney

, Humphreys

. HEDIS initiation and engagement quality measures of substance use disorder care: impact of setting and health care specialty. Popul Health Manag. 2009;12(4):191–196.

12.

Kerridge

, Mauro

, Chou

, . Predictors of treatment utilization and barriers to treatment utilization among individuals with lifetime cannabis use disorder in the United States. Drug Alcohol Depend. 2017;181:223–228.

13.

Weisner

, Mertens

, Tam

, Moore

. Factors affecting the initiation of substance abuse treatment in managed care. Addiction. 2001;96(5):705–716.

14.

Garnick

, Lee

, Horgan

, Acevedo

. Adapting Washington Circle performance measures for public sector substance abuse treatment systems. J Subst Abuse Treat. 2009;36(3):265–277.

15.

Charlson

, Szatrowski

, Peterson

, Gold

. Validation of a combined comorbidity index. J Clin Epidemiol. 1994;47(11):1245–1251.

16.

D'Hoore

, Bouckaert

, Tilquin

. Practical considerations on the use of the Charlson comorbidity index with administrative data bases. J Clin Epidemiol. 1996;49(12):1429–1433.

17.

Bradley

, Chavez

, Lapham

, . When quality indicators undermine quality: bias in a quality indicator of follow-up for alcohol misuse. Psychiatr Serv. 2013;64(10):1018–1025.

18.

van der Pol

, Liebregts

, de Graaf

, Korf

, van den Brink

, van Laar

. Facilitators and barriers in treatment seeking for cannabis dependence. Drug Alcohol Depend. 2013;133(2):776–780.

19.

Gates

, Copeland

, Swift

, Martin

. Barriers and facilitators to cannabis treatment. Drug Alcohol Rev. 2012;31(3):311–319.

20.

Kleinman

, Norton

. What's the Risk? A simple approach for estimating adjusted risk measures from nonlinear models including logistic regression. Health Serv Res. 2009;44(1):288–302.

21.

Basu

, Rathouz

. Estimating marginal and incremental effects on health outcomes using flexible link and variance function models. Biostatistics 2005;6(1):93–109.

22.

StataCorp. Stata Statistical Software: Release 15.0 [Computer Program]. College Station, TX: StataCorp LP; 2017.

23.

Harris

, Humphreys

, Finney

. Veterans Affairs facility performance on Washington Circle indicators and casemix-adjusted effectiveness. J Subst Abuse Treat 2007;33(4):333–339.

24.

Bensley

, Harris

, Gupta

, . Racial/ethnic differences in initiation of and engagement with addictions treatment among patients with alcohol use disorders in the veterans health administration. J Subst Abuse Treat. 2017;73:27–34.

25.

MacLean

, Marnoch

, Sun

, . Quality varies across health insurance marketplace pricing regions. Med Care. 2015;53(7):607–618.

26.

Harris

, Ellerbe

, Phelps

, . Examining the specification validity of the HEDIS quality measures for substance use disorders. J Subst Abuse Treat. 2015;53:16–21.

27.

Campbell

, Bahorik

, Kline-Simon

, Satre

. The role of marijuana use disorder in predicting emergency department and inpatient encounters: a retrospective cohort study. Drug Alcohol Depend. 2017;178:170–175.

28.

Charilaou

, Agnihotri

, Garcia

, Badheka

, Frenia

, Yegneswaran

. Trends of cannabis use disorder in the inpatient: 2002 to 2011. Am J Med. 2017;130(6):678–687.e677.

29.

Hayley

, Stough

, Downey

. DSM-5 cannabis use disorder, substance use and DSM-5 specific substance-use disorders: Evaluating comorbidity in a population-based sample. Eur Neuropsychopharmacol. 2017;27(8):732–743.

30.

John

, Wu

. Problem alcohol use and healthcare utilization among persons with cannabis use disorder in the United States. Drug Alcohol Depend. 2017;178:477–484.

31.

Bahorik

, Satre

, Kline-Simon

, Weisner

, Campbell

. Alcohol, cannabis, and opioid use disorders, and disease burden in an integrated health care system. J Addict Med. 2017;11(1):3–9.

32.

Polen MR , Whitlock

, Wisdom

, Nygren

, Bougatsos

. Screening in Primary Care Settings for Illicit Drug Use: Staged Systematic Review for the U.S. Preventive Services Task Force. Rockville, MD: AHRQ; January 2008.

33.

Institute of Medicine. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press; 2006.

34.

Demyttenaere

, Bruffaerts

, Posada-Villa

, . Prevalence, severity, and unmet need for treatment of mental disorders in the World Health Organization World Mental Health Surveys. JAMA 2004;291(21):2581–2590.

35.

Lapham

, Lee

, Caldeiro

, . Frequency of cannabis use among primary care patients in Washington State. J Am Board Fam Med. 2017;30(6):795–805.

36.

Bobb

, Lee

, Lapham

, . Evaluation of a pilot implementation to integrate alcohol-related care within primary care. Int J Environ Res Public Health 2017;14(9):E1030.

37.

Harris

, Humphreys

, Bowe

, Tiet

, Finney

. Does meeting the HEDIS substance abuse treatment engagement criterion predict patient outcomes? J Behav Health Serv Res. 2010;37(1):25–39.

38.

Harris

, Humphreys

, Bowe

, Kivlahan DR , Finney

. Measuring the quality of substance use disorder treatment: evaluating the validity of the Department of Veterans Affairs continuity of care performance measure. J Subst Abuse Treat. 2009;36(3):294–305.

39.

Dunigan

, Acevedo

, Campbell

, . Engagement in outpatient substance abuse treatment and employment outcomes. J Behav Health Serv Res. 2014;41(1):20–36.

40.

Garnick

, Horgan

, Acevedo

, . Criminal justice outcomes after engagement in outpatient substance abuse treatment. J Subst Abuse Treat. 2014;46(3):295–305.

41.

Acevedo

, Garnick

, Dunigan

, . Performance measures and racial/ethnic disparities in the treatment of substance use disorders. J Stud Alcohol Drugs. 2015;76(1):57–67.

42.

Danovitch

, Gorelick

. State of the art treatments for cannabis dependence. Psychiatr Clin North Am. 2012;35(2):309–326.

43.

Marshall

, Gowing

, Ali

, Le Foll

. Pharmacotherapies for cannabis dependence. Cochrane Database Syst Rev 2014;12(12):CD008940.

44.

Gates

, Sabioni

, Copeland

, Le Foll

, Gowing

. Psychosocial interventions for cannabis use disorder. Cochrane Database Syst Rev 2016;5(5):CD005336.