Sage Journals: Discover world-class research

Abstract

Background: Chronic insomnia disorder affects 10–15% of adults, causing significant individual and societal burden. Despite Cognitive Behavioral Therapy for Insomnia (CBT-I) being the recommended first-line, sleep medications remain more common due to limited access to trained providers. Digital CBT-I offers a scalable solution, but evidence of its real-world impact in U.S. clinical settings is lacking. Method: This study evaluates real-world implementation and impact of digital CBT-I in U.S. clinical settings, using Normalization Process Theory (NPT) to guide integration at Henry Ford Health, Detroit, Michigan. Implementation success was assessed through order rates, patient sign-ups and workflow acceptability. We assess the effect on healthcare utilization through a propensity-matched observational treatment-control design. Results: Implementation was successful, with 1,162 patients offered digital CBT-I. From this cohort, we analyzed a sample of 340 patients with sufficient chart data and established care (120 days) who utilized digital CBT-I, comparing them to 340 matched standard care controls. Patients who used digital CBT-I had a 64% reduction in the odds of any medication fill during the postwindow period (p < .001) and were 53% less likely to fill insomnia medication prescriptions compared with the preperiod (p = .013). Controls did not have any significant reductions in medication fill rates. Time-varied analysis showed digital CBT-I patients had transiently higher outpatient visit odds at 30–60 days, followed by sustained reductions of 28% (120–150 days) and 31% (150–180 days). After covariate adjustment, early differences were nonsignificant while later reductions remained significant. Conclusions: NPT facilitated integration of digital CBT-I into existing workflows, allowing immediate access while minimizing disruption to routine practice. Provider training sessions and reminders effectively promoted suitable patient uptake. Digital CBT-I was associated with reduced medication fills pre-to-post with an initial rise and then sustained reduction in outpatient service utilization patterns over time. A key limitation is the use of individuals who declined digital CBT-I as comparators, which may introduce selection bias. Generalizability may be limited as the study was conducted within a single healthcare system. Trial Registration: Not applicable—the assignment of the medical intervention to patients was not at the discretion of the investigators.

Plain Language Summary

Helping People With Insomnia Sleep Better: A New Digital Treatment Delivered in U.S. Clinical Settings

Why was this study done?

Many adults struggle with chronic sleep problems, which can seriously affect their health and daily life. While therapy for insomnia (called CBT-I) is recommended as the first treatment, most people end up taking sleep medications because it's hard to find trained therapists to deliver it. This study looked at whether a digital version of CBT-I could be successfully used in clinical settings by patients.

What did the researchers do?

The research team introduced a digital CBT-I treatment at Henry Ford Health in Detroit, Michigan. They trained healthcare staff on how to offer this new treatment to patients with sleep problems. They then compared 340 patients who used the digital treatment to 340 similar patients who received usual care, looking at how often they filled prescriptions for sleep medications and used other healthcare services.

What did the researchers find?

Digital CBT-I was successfully integrated into normal clinical routines, with 1,162 patients being offered the treatment. Patients who used the digital treatment were 64% less likely to fill any medication prescriptions and 53% less likely to fill sleep medication prescriptions compared to before they started the treatment. In the first 2 months, these patients used slightly more outpatient services, but after 4–6 months, they were using significantly fewer services compared to those who didn't receive the digital treatment.

What do these findings mean?

This study shows that digital CBT-I can be successfully implemented in U.S. healthcare settings, providing an effective alternative to sleep medications. It demonstrates that this digital treatment can help reduce both medication use and healthcare service use over time. This approach could offer a scalable solution to help more people access effective sleep treatment without relying on medications.

Keywords

implementation science digital health cognitive behavioral therapy for insomnia healthcare utilization normalization process theory real-world evidence

Background

Sleep disturbances are among the most common health concerns in clinical practice, with up to 50% of U.S. adults reporting symptoms of either acute or chronic insomnia to their physician during their lifetime (Healy et al., 2024). At any given time, chronic insomnia disorder, characterized by persistent difficulties with falling or staying asleep along with significant daytime impairments, affects approximately 10–15% of adults and is associated with substantial individual and societal burden (Morin & Buysse, 2024). Cognitive Behavioral Therapy for Insomnia (CBT-I) is the recommended first-line treatment (Qaseem et al., 2016) due to its sustained efficacy and more favorable side-effect profile (Healy et al., 2024; Matheson et al., 2024). Despite this, prescription sleep medications remain the most commonly utilized treatment option in U.S. clinical practice (Grandner & Chakravorty, 2017) due to a lack of providers trained in delivering CBT-I (Haycock et al., 2021).

This discrepancy between guidelines and real-world clinical practice may be attributed to barriers in healthcare delivery. Clinicians who manage the majority of insomnia patients may lack familiarity with CBT-I or face challenges referring patients to trained and specialized CBT-I providers (Dyas et al., 2010). Consequently, many clinicians resort to prescribing sleep medications or offering more basic sleep hygiene advice, despite expressing dissatisfaction with these approaches (Ulmer et al., 2017; Vargas et al., 2023). The integration of digital CBT-I as a treatment option into clinical workflows presents a promising solution to these challenges. Fully automated digital CBT-I platforms offer immediate, standardized, and convenient access to evidence-based treatment, potentially overcoming barriers related to provider availability and geographical limitations (Thomas et al., 2016). Digital CBT-I has been found to be cost-effective (Darden et al., 2021), and successfully integrated into existing U.K. and Australian healthcare settings, allowing for scalable delivery of CBT-I without increasing the burden on clinical staff and resources (Miller et al., 2018; Sampson et al., 2022; Stott et al., 2021).

While digital CBT-I has shown promise in U.S. trials for improving sleep outcomes across diverse patient populations (Cheng et al., 2019), a significant knowledge gap persists regarding its uptake and impact in routine clinical settings. This gap limits our understanding of digital CBT-I's effects on patient healthcare utilization outside of clinical trials, which often lack external validity and generalizability. Our study addresses this by evaluating digital CBT-I in a truly representative real-world clinical environment, capturing the heterogeneity of patients typically seen in clinics. Unlike previous studies focusing on specific populations such as veterans (Reed et al., 2024) or imposing strict inclusion criteria (Thorndike et al., 2021), our approach assesses digital CBT-I's impact when integrated into routine clinical practice across a diverse patient population. We aim to employ a cohort analysis of patient chart data with a treatment-control design, comparing patients who received digital CBT-I at their healthcare provider's discretion with contemporaneous patients who declined and continued with standard care (e.g., second-line medications for sleep), thereby mirroring routine clinical choice and enabling a pragmatic assessment of digital CBT's implementation and impact in diverse healthcare settings.

To effectively integrate digital CBT-I into existing clinical workflows, it is also crucial to consider the complex dynamics of healthcare systems and the processes involved in implementing new practices. In this context, the Normalization Process Theory (NPT) framework offers a valuable approach for understanding and facilitating the implementation of novel interventions in healthcare settings (May et al., 2018). NPT provides a structured way to examine how new practices are operationalized in healthcare and become routinely embedded in their social contexts. By applying NPT to the implementation of digital CBT-I, we systematically address potential barriers to adoption across key domains. This structured approach allows for targeted, low-lift changes to adapt clinical workflows, maximizing the potential for successful and sustainable implementation of digital CBT-I in real-world clinical settings. We selected NPT as a pragmatic heuristic device to support early planning upfront due to its broad focus on implementation, providing a structured way of considering how individuals and collectives adopt, embed and sustain innovations pragmatically in real world practice, leading to routine delivery in healthcare settings, a long-term aim for digital CBT-I delivery (De Brün et al., 2016; Ross et al., 2018).

This current evaluation aims to conduct a comprehensive, real-world implementation and assessment of the impact of digital CBT-I in U.S. clinical settings. For objectives, NPT first provides a framework to guide integration of an FDA-cleared digital CBT-I treatment for patients aged 18 and older as part of current clinical workflows. We assess implementation outcomes such as acceptability and feasibility of providing a digital CBT-I treatment for patients and their providers in a structured approach. We also evaluate the impact of digital CBT-I using patient chart information in a propensity-matched treatment-control design. By employing propensity score matching across a broad, diverse patient population, we assess the impact of digital CBT-I on healthcare utilization, including visits, psychology referrals, and medication fill rates. Our goal is to understand the real-world impact of digital CBT-I and its potential to improve access to first-line recommended treatment options through adapted clinical workflows. Findings may provide important implications for healthcare policy and clinical practice treatment decisions.

Method

Participants

Recruitment for this study cohort took place over a 12-month period starting in July 2023 and ending in July 2024. Patients presenting with insomnia or sleep difficulties at Henry Ford Health (HFH) were consecutively offered digital CBT-I by their healthcare provider in two clinical settings: the Academic Internal Medicine (AIM) clinic within primary care and seven sleep clinics across Detroit, Michigan, US.

Staff Training, Implementation Preparation, and Support

NPT (May et al., 2018) characterizes and explains key mechanisms that promote and inhibit implementation, embedding and integration of new health techniques, other technologies and complex interventions. NPT was used as a framework to help embed digital CBT-I access across four main domains: coherence (making sense of the intervention), cognitive participation (engaging stakeholders), collective action (operationalizing the intervention through workflow integration), and reflexive monitoring (appraising the intervention's effects through ongoing support and feedback). The evaluation of our implementation approach was conducted through a retrospective team review, focusing on what worked well and areas for improvement. Guided by the NPT subcomponents outlined by Bracher and May (2019), the assessment combined subjective impressions, qualitative feedback, and systematic analysis to identify key strengths and weaknesses. Data collection for implementation evaluation included verbal feedback captured during provider training sessions, documented email correspondence regarding implementation barriers, and meeting notes from departmental check-ins. Implementation metrics were systematically tracked, including provider-specific order rates across clinic locations.

Provider training sessions were led by clinical representatives from Big Health Inc, the developers of the digital CBT-I treatment, as well as internal clinical champions, and expert sleep specialists who are board-certified in Sleep Medicine. Additionally, providers from the AIM clinic, including the Clinical Division Head and the Associate Division Head of General Internal Medicine, were involved. Around 50 healthcare providers from both primary care and sleep clinics took part in a thorough training session to learn how to offer and order digital CBT-I for their patients. The sessions covered various treatment options for insomnia, methods for identifying suitable patients, and the steps to introduce and order digital CBT-I through the electronic medical record (EMR) system. Each session, lasting 20–30 min, also allowed for clinical feedback and discussion.

To facilitate implementation over time, several key strategies were employed. Email reminders accompanied by digital flyers were sent out, and Best Practice Alerts for insomnia management were integrated into the EMR (Epic) system. Printed referral cards with sign-up links and QR codes were provided to healthcare providers for patient distribution, and additional referral cards were placed in the clinics for patient access. Periodic refresher training meetings were held to maintain providers’ familiarity with the process. Automated after-visit summaries were used to streamline the order and patient sign-up process. Patients could access the program either through their MyChart application when ordered by a provider via the EMR, or by using a paper-based flier with a link and QR code for immediate registration and digital CBT-I initiation.

Ongoing support was provided by a part-time trained staff member (research assistant) within the healthcare system, who conducted regular check-ins to address implementation challenges and offer updates. Patients at both clinics also had access to embedded psychologists who may provide CBT, more generally, in addition to this digital CBT-I treatment. These combined efforts, particularly the refresher training meetings and the automated after-visit summaries, helped facilitate implementation.

Outcomes

This article is focused on data available from patient EMRs within the healthcare system. Outcome measures were chosen based on their clinical relevance and availability within the EMR system. This included medication fills, for any medication and for insomnia-specific medication fills, including benzodiazepines, Z-drugs, and sedating antidepressants (Bazell et al., 2023), assessed 90–0 days before and 30–120 days after the offer of digital CBT-I. The 30–120 day period following the index event (when digital CBT-I was offered by a healthcare provider) was chosen for outcome assessment based on the number of established patients with sufficient follow-up data within this timeframe. This limited the number of patients who could be included in the analysis. This timeframe, however, allowed for a sufficient follow-up period to observe potential changes in medication use and healthcare utilization patterns in eligible patients. Secondary outcomes included healthcare utilization (e.g., outpatient visits, including outpatient clinics, same-day surgeries, observation beds, urgent care visits, and same-day ambulatory hospital encounters) documented in the medical records of patients. Demographic data and relevant medical history were also collected at baseline from the patient's EMR. The number of referrals to psychology or similar providers 1 year before and after the index date were evaluated.

Intervention

SleepioRx, developed by Big Health, Inc. is a digital therapeutic intended for the treatment of chronic insomnia/insomnia disorder as an adjunct to usual care in patients aged 18 and older. SleepioRx delivers CBT-I and can be made available on the order of a licensed healthcare provider. Utilizing audio and video narration, SleepioRx delivers evidence-based CBT-I techniques such as sleep restriction, stimulus control, sleep hygiene, and cognitive therapy, enabling patients to access these methods conveniently via smartphone. The intervention is personalized according to individual lifestyle, progress, and goals, reflecting the flexibility typically found in therapist-delivered CBT. In addition to the core educational sessions, SleepioRx provides supplementary tools, including daily sleep diaries and a virtual library of educational resources aimed at enhancing learning and practice. SleepioRx has been clinically validated in multiple randomized controlled trials (Cheng et al., 2019; Espie et al., 2019), received FDA-clearance via the 510 (k) process in August 2024¹, and was also recommended by the National Institute for Health and Care Excellence as a first-line cost-saving treatment for insomnia (Wise, 2022). For this implementation evaluation, SleepioRx (now referred to as digital CBT-I) was offered at no cost to patients.

Study Design

This study employs a propensity-matched treatment-control observational cohort design. Those treated were patients who were offered and used digital CBT-I, while controls were patients who were offered digital CBT-I by their provider but did not use it, opting for standard care instead. This approach allows for a comparison of digital CBT-I users with contemporaneous patient controls managed for insomnia. Propensity scores were used to form a clinically matched control group in a 1:1 ratio. Propensity score matching was based on demographics including age, race, clinic location, and comorbidities, with comorbidities assessed using the Charlson Comorbidity Index (CCI) as it was routinely captured in patient charts. This matching process ensures that the groups were comparable in terms of key demographic and clinical characteristics, reducing potential confounding factors in the analysis. Clinical or demographic characteristics identified as imbalanced after propensity matching were subjected to additional sensitivity analyses, and matching success was assessed using density plots and Love plots.

Statistical Analysis

All patient data were deidentified and statistical analyses were performed by the HFH Biostatistician team. Analyses were conducted using R Statistical Software v4.4.0 (R Core Team, 2023). This study was exploratory and the required cohort sample size was determined based on previous real-world reports that observed statistically significant between-group effects (Stott et al., 2021).

In the matched cohort, McNemar's test was first used to determine if there was a change over time in the number of patients with medication fills (for any medication and for insomnia-specific medications) and any referrals to psychology (including behavioral health and neuropsychology) before and after the index date. As patients were offered digital CBT-I at different times across the year, we standardized the timeframe to account for these differences at the patient level. We evaluated medication fills 90 days prior to the index date to those in the 30–120 days postindex date for both cohorts. This approach enabled us to evaluate patients who had been established for similar periods within the rolling cohort, ensuring a consistent comparison.

For binary outcomes in the postwindow, multiple logistic regression analysis was used for outcome variables with a sufficient event rate, such as visit data, which was prevalent in most participants. The outcomes analyzed included any psychologist referral, any outpatient visit, any inpatient visit, and any emergency department (ED) visit in the postwindow period. In all models, an indicator was included to control for whether the patient had the outcome in the specified lead-in period prior to the index date, adjusting for the patient's prior history of the outcome in question. Logistic regression analyses were conducted to explore and isolate when any significant effects occurred in the postwindow period (30–120 days). To explore longer-term effects for visits, the analysis window was extended to 180 days by including two additional 30-day periods: 120–150 days and 150–180 days. This extension was performed because the impact on healthcare utilization may become more pronounced with time (Sampson et al., 2022).

Ethical Oversight and Reporting

This study was reviewed and approved by the HFH Institutional Review Board (IRB approval number: HFH-2023-16305) as minimal risk. The IRB waived the requirement for individual informed consent for this retrospective chart review. All digital CBT-I patients included here consented to their data being collected within the digital CBT-I program and shared securely with HFH. The study was conducted in accordance with the Declaration of Helsinki and all relevant guidelines and regulations. We aligned this report with the StaRI (Standards for Reporting Implementation Studies; Pinnock et al., 2017) reporting checklist (see Supplemental File 1).

Results

Demographic and Clinical Characteristics of Patients

A total of 1,162 patients were initially offered digital CBT-I. Of these, 340 patients agreed to data sharing and had sufficient chart data from both before and after starting digital CBT-I to be included in the statistical analysis dataset. Among the participants, 47 patients had multiple registration attempts for the digital CBT-I program. For these cases, only the patient's first registration was used in the analysis. Two patients out of the 47 with multiple registrations had duplicate entries with identical index dates. These patients were both included in the digital CBT-I cohort, with one duplicate record retained to define the index date.

Implementation Approach

The implementation of digital CBT-I was guided by the four key domains of NPT: coherence, cognitive participation, collective action, and reflexive monitoring. Within the coherence domain, we focused on communal and individual specification, differentiation, and internalization. Communal and individual specification was fostered through targeted initiatives that helped stakeholders understand digital CBT-I, achieved through a series of discussions with clinical champions, wider departmental meetings (where complex cases with comorbidities and other challenges were discussed), and group training activities aimed at clarifying its purpose, function, and value. The implementation team differentiated digital CBT-I by highlighting its digital nature, accessibility, and evidence-based approach, emphasizing how it complemented traditional face-to-face CBT-I in terms of reach and scalability and how patients may continue with further standard of care treatments. To facilitate internalization, success stories, challenges (including complex patient cases and comorbidities discussed by HFH providers), and randomized controlled efficacy and effectiveness data were shared. HFH also conducted a system-wide grand rounds on generic digital CBT-I (nonbranded). Implementation challenges encountered included initial EMR integration difficulties requiring workflow updates by the IT team for functionality to include generation to the after-visit summary. We also addressed what seemed like early provider cognitive overload for the new order flow during trainings by highlighting a simplified ordering process in further training sessions to encourage familiarity (Asgari et al., 2024; Sweller, 1988). Together these NPT elements pairing provider education sessions communicating a simple and clear order flow process were deemed most crucial to engagement and sustained adoption.

Moving to the cognitive participation domain, we enhanced legitimation by engaging key stakeholders (clinical champions at the Sleep and AIM clinics) to drive implementation forward. We also organized teams to contribute collectively, fostering an understanding of how they would contribute to the implementation process. To support these efforts, we established interactive department meetings, with approximately 4–5 scheduled sessions spread across the year.

In the collective action domain, we focused on supporting interactional workability. This was achieved by establishing clear workflow protocols and integrating digital CBT-I into the EMR. To improve patient access and engagement, we implemented several features: automatic generation of after-visit summaries following a CBT-I order, provision of QR codes for easy access, and enabling patient access through the MyChart application. These measures aimed to streamline the process for both healthcare providers and patients, facilitating the adoption of digital CBT-I in clinical practice.

Finally, NPT created a structured way for the implementation team to capture feedback from on-the-ground clinicians and teams during the study as part of reflexive monitoring. This ongoing feedback cycle further informed adaptation and improvement to domains considered across NPT, focused on training and engagement actives. These included tracking uptake numbers across locations and clinics, praising top referrers, encouraging clinicians who may not have referred, and continuously gathering feedback to inform improvements (from AIM and sleep clinic teams, EMR teams, integration teams). Ongoing feedback mechanisms, such as departmental meetings, allowed for continuous refinement of the implementation strategy.

Propensity Score Matching

Propensity score matching was used to ensure that the digital CBT-I cohort and the matched controls were similar with respect to the covariates in the propensity model. Propensity scores were calculated using a logistic regression model, incorporating covariates that differed significantly between cohorts. These included age, race, clinic location and diagnosis of renal disease (this was statistically significant between cases and controls from the CCI and further common comorbid conditions were not different between groups). Optimal propensity score matching was employed to reduce the overall difference between the propensity scores in the two cohorts (see Figures S1 and S2 in Supplemental File 2). One-to-one matching was performed, and all 340 treated patients were matched successfully to controls. Figures and tables were evaluated to represent the balance of the propensity scores before and after matching, demonstrating that the propensity score matching accurately balanced the cohorts in terms of their probability of receiving treatment conditional on the covariates that were different at baseline and included in the propensity score model. We reviewed a Love plot for between group covariates which helped confirm balance in the cohorts after adjustment (Love, 2002). Balance, as defined as a Standardized Mean Difference less than 0.1, was observed in all variables considered in the propensity score model as well as covariates not included in the model. Table 1 presents baseline demographic and clinical characteristics of the study population before (overall) and after propensity score matching by group. Supplemental Table S1 provides further clinical characteristic background information prior to the encounter where digital CBT-I was offered (any time prior). Most patients had a diagnosis of insomnia (≥82%), and approximately 34% of patients previously received sleep-promoting medication. Supplemental Table S2 gives an overview of the specific insomnia diagnosis and 39% of patients received their insomnia diagnosis at the encounter where digital CBT-I was offered. These patients may be new patients seeking their first management for insomnia.

Table 1.

Clinical demographic characteristics by group.

Characteristic	Overall, N = 1,162	Control, N = 340	Digital CBT-I, N = 340	p
Age at encounter	54.2 (16.1)	51.8 (15.8)	52.6 (16.3)	.6
Race				.4
Black	337 (29%)	67 (20%)	73 (21%)
Other	147 (13%)	44 (13%)	33 (9.7%)
White	678 (58%)	229 (67%)	234 (69%)
Sex				.039
F	754 (65%)	200 (59%)	226 (66%)
M	408 (35%)	140 (41%)	114 (34%)
Chart with insomnia term	1,108 (95%)	326 (96%)	327 (96%)	.8
Sleep clinic	1,018 (88%)	313 (92%)	311 (91%)	.8

Note. n (%); mean (SD), p-value determined by Chi-squared test, or t-test depending on variable type.

Pre-to-post analysis

Only the digital CBT-I group had a significant reduction for both any medication fills and for insomnia-specific medication fills in the postwindow. The analysis revealed that the odds of any medication fills among patients who engaged with digital CBT-I decreased significantly by 64% during the post-window period (OR = 0.36, Χ² = 16.41, p < .001). In contrast, the control group exhibited a nonsignificant reduction in the odds of a medication fill of 37% (OR = 0.63, Χ² = 3.41, p = .06). For insomnia-specific medications, patients using digital CBT-I had 53% lower odds of filling a prescription compared to before (OR = 0.47, Χ² = 6.11, p = .013), while controls showed a 24% decrease that was not statistically significant (OR = 0.76, Χ² = 0.68, p = .41). Results of the tests indicate that both control and treatment groups had significantly lower odds of psychologist referrals in the postwindow digital CBT-I OR = 0.26; Control OR = 0.41; p < .01; (see Figure 1).

Figure 1.

Number of Patients With Medication Fills (Any Type and Those for Insomnia), and Psychologist Referrals by Group and by Time Period.

Logistic Regression Analysis for Binary Outcomes

The time-varied logistic regression analysis found a significant effect of time and group for outpatient visits. The odds of an outpatient visit were significantly higher 30–60 days postindex date (OR = 1.37, p = .048), indicating that digital CBT-I patients’ odds are 37% higher than the odds of a patient in the control cohort to use outpatient services 30–60 days after the index date (see Figure 2). However, digital CBT-I patients were significantly less likely to use outpatient services in both the 120–150 (OR = 0.72, p = .041) and 150–180 (OR = 0.69, p = .039) window days, 28% and 31% lower, respectively. As we observed a statistically significant imbalance in sex after matching, we performed sensitivity analysis including sex in the logistic regression model. In the sex-adjusted model, the early 30–60 day increase was no longer significant (p = .066), while the decreases at 120–150 days (p = .037) and 150–180 days (p = .033) remained significant.

Figure 2.

Time Varied Logistic Regression for Outpatient Visits Between Digital CBT-I and Standard of Care Control Patients.

Digital CBT-I patients may, therefore, experience a higher initial usage of outpatient services compared to control patients but then see significantly lower odds of outpatient usage sustained with time compared with controls. For the purposes of this analysis, we assume both groups had a similar number of patients who had at least 180 days of follow-up. The latest encounter date for the control group was April 22nd, 2024, and the latest encounter date for the treatment group was April 19th, 2024. The 75th percentiles were January 30th, 2024, and February 28th, 2024, indicating that this assumption is reasonable.

The analysis of inpatient and ED visits was excluded due to the low event rate and insufficient time for patients to exhibit effects in less frequent outcomes, given the sample size. Similarly, we performed the logistic regression analysis comparing medication fill rates between groups, and those results indicate that, over the initial 120 days, digital CBT-I did not significantly alter the tendency of patients in the treatment group to fill insomnia medications or medications in general compared with control. However, these findings are contingent on the availability of complete follow-up data and may evolve as more data become available from patients with shorter follow-up periods.

Discussion

This study aimed to evaluate the implementation of digital CBT-I in U.S. clinical healthcare settings and assess its impact on healthcare utilization using outcomes from patient chart records. We utilized NPT to provide a robust framework for implementing digital CBT-I, enabling a structured and theoretically grounded approach to addressing key implementation challenges. By systematically targeting coherence, cognitive participation, collective action, and reflexive monitoring, we were able to foster stakeholder engagement, streamline operational processes, and establish mechanisms for continuous improvement, ultimately facilitating the successful integration of digital CBT-I into existing clinical workflows. These findings demonstrate that the integration of digital CBT-I into existing workflows was successful, leading to significant patient uptake and measurable reductions in healthcare utilization. The impact on healthcare utilization was assessed through an observational design, comparing patients who used digital CBT-I to those who were offered, but did not use it and experienced standard of care. Specifically, digital CBT-I was associated with reductions in the odds of both overall and insomnia-specific medication fills from pre- to postdigital CBT-I, along with an initial increase followed by a sustained reduction in the odds of outpatient visits over time compared to the standard of care control group.

The successful implementation of digital CBT-I at HFH was guided by NPT, facilitating its seamless embedding into routine clinical practice. Unlike previous real-world evaluations of insomnia interventions, which often lack or did not report structured implementation frameworks, our use of NPT enabled low-lift changes that integrated smoothly with existing clinical workflows. By adapting the EMR system to include a digital CBT-I order, we created a streamlined process that minimized disruption to clinicians’ routines while maximizing accessibility for patients. This approach, coupled with targeted clinician training, resulted in significant uptake with 1,162 patient orders across clinics. Notably, clinicians provided positive feedback on treatment integration, reporting that digital CBT-I enhanced their ability to offer evidence-based insomnia treatment without significantly increasing their workload. This positive reception from healthcare providers further underscores the usefulness of digital CBT-I as a treatment. Our approach addresses a key gap in previous evaluations, which often struggled with integration of CBT-I into real-world clinical settings (Koffel et al., 2020), leading to poor uptake when offered at scale (Bramoweth et al., 2021). Our findings demonstrate that with appropriate support and theoretical grounding, digital CBT-I solutions can be effectively and positively incorporated into usual practice, overcoming implementation challenges that have hindered previous large-scale and more top-down efforts.

It is important to note that HFH was chosen as the location for this evaluation as it is centrally located in Metro Detroit, MI and is one of the most racially, culturally, and socioeconomically diverse regions in the United States. The main hospital in Detroit is an 800+ bed facility with every major medical specialty and nationally recognized departments and is part of a state-wide collection of six large not-for-profit hospitals. HFH is one of the nation's leading comprehensive, integrated health systems; it provides acute, specialty, primary, and preventive care. Within its ambulatory clinics and hospitals, there are >3.4 million outpatient visits and 77,000 surgical procedures performed each year. HFH sees over 70,000 patients with insomnia per year and has a diverse payer distribution: Medicare and Medicare HMO, 43%; Blue Cross, 22%; Medicaid and Medicaid HMO, 18%; Other, 17%. The diverse patient population makes HFH an ideal environment for this implementation project. We used SleepioRx, an FDA-cleared, standardized digital treatment for insomnia, which has been utilized in clinical research at HFH for more than a decade (Pillai et al., 2015). SleepioRx has undergone extensive evaluation in clinical trials, and health economic analyses indicate it is more cost-effective than both medication and therapist-delivered CBT-I (Darden et al., 2021). Results may not be generalizable to other digital CBT-I programs or wellness applications that are not indicated for the treatment of insomnia disorder.

Our impact results reveal a complex pattern of healthcare utilization changes following early digital CBT-I implementation. While between-group analyses for any and insomnia-specific medication fill rates post digital CBT-I implementation were nonsignificant, likely due to insufficient time to establish comprehensive patient chart histories as only the first 30–120 days postimplementation were evaluated, within-subject analyses showed promising results. Patients who engaged with digital CBT-I demonstrated a 64% reduction in the odds of any medication fill and a 53% decrease in the odds of insomnia-specific medication fills during the postwindow period. Those in the control group who were offered and did not uptake digital CBT-I, did not have statistically significant reductions in medication fill rates. These preliminary findings, though requiring further investigation with larger sample sizes and extended timeframes, align with previous research (Luik et al., 2020; Sampson et al., 2022), which reported reductions in both any medication and insomnia-specific medications following digital CBT-I. Further work may also result in additional support packages that specifically target medication reduction in patients when appropriate (Gardner et al., 2024). This is because digital CBT-I does not directly target reduction of insomnia medication use with patients. It should also be highlighted that only about 34% of those who used digital CBT-I had received medication for insomnia previously, and 39% of patients were managed for insomnia for the first time when digital CBT-I was offered.

Outpatient visits revealed a nuanced pattern in healthcare utilization over time. Patients were found to experience an initial increase in the odds of an outpatient visit (37% higher 30–60 days after the index date relative to matched controls; but this was not statistically significant once the model was adjusted for sex), likely due to the need for early management of insomnia and related comorbid conditions (Anderson et al., 2014; Wickwire, 2024), aligning with the typical care trajectory when patients first present with insomnia symptoms. This initial increase often involves more frequent clinical interactions for assessment and treatment of associated psychological distress, and may include anxiety and depression management (Hayward et al., 2010). As patients progress through time, they may require fewer visits (Sampson et al., 2022) consistent with symptom resolution and improved outcomes. This temporal pattern underscores the potential long-term benefits of effective insomnia management. Consistent with this hypothesis, digital CBT-I users had lower odds of an outpatient visit in later time windows (28% lower at 120–150 days and 31% lower at 150–180 days), with both reductions remaining statistically significant after adjustment for sex. This pattern suggests that while patients may require more clinical support during the early stages of digital CBT-I implementation, they experience a sustained decrease in healthcare utilization as the treatment takes effect with time.

Together, these findings align with previous research on the health economic impacts of CBT for insomnia. For instance, a study by Sampson et al. (2022) observed a reduction in both medications for insomnia and utilization of Primary Care visits with digital CBT-I in England. Forma et al., (2022) found a reduction in healthcare utilization visits over time. These studies collectively support the potential of CBT-I, including digital formats, to not only improve clinical outcomes but also reduce healthcare utilization. This may help reduce associated healthcare costs over time (Darden et al., 2021). The mechanism for this reduction may be rooted in improved sleep quality leading to better daytime functioning and overall health with reduced need for further outpatient visits for ongoing medication management. Future research should now explore longer-term trends associated with digital CBT-I use and examine how CBT-I can be offered by other clinician specialties beyond primary care and behavioral sleep medicine. Further research is also needed to explore how similar frameworks can be applied to other digital behavioral health interventions for common conditions like anxiety and depression.

Limitations include a lack of a formal implementation process evaluation in part due resource availability, and a more formal evaluation process may have been established to provide more information to inform future work. A relatively short follow-up period for patients to establish comprehensive chart histories and detect long-term trends in medication fill rates. Second, the generalizability of our findings may be limited by the specific context of our study. The implementation was conducted within a single healthcare system (HFH) and may not fully represent the diverse range of healthcare settings and patient populations in the U.S. factors such as the health system's infrastructure, patient demographics, and local healthcare practices could influence the success of digital CBT-I implementation and its impact on healthcare utilization. The AIM clinic, with its established residency training program, presents a valuable opportunity for enhancing digital CBT-I awareness and implementation. Its educational environment provides an ideal platform for integrating digital CBT-I into clinical training, potentially improving knowledge and patient utilization of digital treatments. Future efforts should focus on leveraging the AIM clinic's educational structure with dedicated and more frequent training to further promote awareness of digital CBT-I among rotating residents. Additionally, the study focused on patients who chose to uptake digital CBT-I compared with those who did not, and further qualitative evaluations are also now needed to better understand patient perspectives. Including patients who declined digital CBT-I as the comparator group faithfully reflects routine clinical decision-making; however, it also may introduce a preference bias because individuals who accept digital CBT-I may differ systematically in motivation and treatment expectations. Unmeasured confounding may influence our regression results; we account for this through propensity score matching and adjustment for prior healthcare utilization in the analysis. Regarding generalizability of the study sample, results from the CCI (Supplemental Table S2) show that most patients here had low scores, indicating low rates of more severe comorbid conditions (e.g., myocardial infarction, dementia). This may reflect an insomnia patient population that is at risk of developing further conditions, if their insomnia is not addressed effectively (Gibson et al., 2023).

Conclusions

This study provides emerging evidence for the successful implementation and early impact of digital CBT-I in real-world U.S. clinical settings. Our findings demonstrate that integrating digital treatments into existing workflows can enhance access to effective insomnia treatment, leading to significant reductions in medication use and healthcare utilization over time. While the results are promising, further research is needed to explore longer-term outcomes and generalizability across diverse healthcare settings. Ultimately, digital CBT-I has the potential to improve insomnia patient treatment options and outcomes and help optimize healthcare utilization.

Supplemental Material

sj-doc-1-irp-10.1177_26334895251386306 - Supplemental material for Real-World Implementation and Impact of Digital CBT for Insomnia on Healthcare Utilization: A Propensity-Matched Controlled Study

Supplemental material, sj-doc-1-irp-10.1177_26334895251386306 for Real-World Implementation and Impact of Digital CBT for Insomnia on Healthcare Utilization: A Propensity-Matched Controlled Study by Christopher B Miller, Danielle Bradley, Ian Wood, David Willens, Anupama Nair, Benjamin Brennan, Shane Bole, Laila Poisson, Shana Hall, Greig Thomson, Mika Hirata, David A Kalmbach and Christopher L Drake in Implementation Research and Practice

Supplemental Material

sj-docx-2-irp-10.1177_26334895251386306 - Supplemental material for Real-World Implementation and Impact of Digital CBT for Insomnia on Healthcare Utilization: A Propensity-Matched Controlled Study

Supplemental material, sj-docx-2-irp-10.1177_26334895251386306 for Real-World Implementation and Impact of Digital CBT for Insomnia on Healthcare Utilization: A Propensity-Matched Controlled Study by Christopher B Miller, Danielle Bradley, Ian Wood, David Willens, Anupama Nair, Benjamin Brennan, Shane Bole, Laila Poisson, Shana Hall, Greig Thomson, Mika Hirata, David A Kalmbach and Christopher L Drake in Implementation Research and Practice

Footnotes

Abbreviations

Acknowledgements

We wish to express our sincere gratitude to all staff members at HFH who contributed to the success of this implementation evaluation project. We particularly appreciate the collaborative efforts of the clinical teams, administrative staff, and IT personnel who facilitated patient engagement, data outcomes, and the seamless integration of digital CBT-I into existing healthcare processes. This project's success is a testament to the collective commitment of the HFH community to advancing patient care and medical education.

ORCID iDs

Christopher B. Miller

Danielle Bradley

David Willens

Anupama Nair

Benjamin Brennan

Shane Bole

Laila Poisson

Shana Hall

David A Kalmbach

Christopher L Drake

Ethical Considerations

This study was prospectively reviewed and approved by the HFH Institutional Review Board (IRB Approval No. HFH-2023-16305) as minimal risk.

Consent to Participate

The IRB waived the requirement for individual informed consent for the retrospective chart review. All patients whose data were used in this analysis had prospectively consented to their SleepioRx data being securely shared from Big Health to HFH. This study was conducted in accordance with the Declaration of Helsinki and all relevant guidelines and regulations.

Consent for Publication

Not applicable.

Authors’ Contributions

CBM, DB, and CD conceived the study question, designed the evaluation and obtained IRB approval. BB and LP performed the statistical analysis, and SB was responsible for accessing Henry Ford patient chart data. IW, DW, AN, SH, GT, MH, and DK contributed to the implementation methodology and designed, trained, and provided clinical integration access to the digital CBT-I intervention at their clinics. All authors performed statistical interpretation. CBM wrote the paper with input from authors. All authors reviewed and edited the paper and agreed to submission.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Declaration of Conflicting Interest

CBM, DB, IW, SH, and GT are all employed by Big Health Inc. and are salaried by the company with stock options. For the remaining authors, none were declared.

Data Availability

The data that support the findings of this study are derived from patient charts at HFH, Detroit, Michigan. Due to the sensitive nature of patient health information and privacy regulations, these data are not publicly available in a repository. Data may, however, be available from the authors upon reasonable request and with permission of HFH, subject to appropriate data sharing agreements and ethical approvals.

Supplemental Material

Supplemental material for this article is available online.

Notes

References

Anderson

L. H.

Whitebird

R. R.

Schultz

McEvoy

C. E.

Kreitzer

M. J.

Gross

C. R.

(2014). Healthcare utilization and costs in persons with insomnia in a managed care population. The American Journal of Managed Care, 20(5), e157–e165. PMID: 25326930

Asgari

Kaur

Nuredini

Balloch

Taylor

A. M.

Sebire

Robinson

Peters

Sridharan

Pimenta

(2024). Impact of electronic health record use on cognitive load and burnout among clinicians. Narrative Review. JMIR Medical Informatics, 12, Article e55499. https://doi.org/10.2196/55499

Bazell

Mirchandani

Pelizzari

Brinkers

(2023). Treatment patterns for generalized anxiety disorder and insomnia in Medicare Fee-For-Service. https://www.milliman.com/-/media/milliman/pdfs/2023-articles/3-28-23_treatment-patterns-generalized-anxiety-disorder-insomnia-medicare-ffs.ashx

Bracher

May

C. R.

(2019). Implementing and embedding health informatics systems—Understanding organizational behaviour change using normalization process theory (NPT). Studies in Health Technology and Informatics, 263, 171–190. https://doi.org/10.3233/SHTI190121

Bramoweth

A. D.

Tighe

C. A.

Berlin

G. S.

(2021). Insomnia and insomnia-related care in the department of veterans affairs: An electronic health record analysis. International Journal of Environmental Research and Public Health, 18(16), Article 8573. https://doi.org/10.3390/ijerph18168573

Cheng

Luik

A. I.

Fellman-Couture

Peterson

Joseph

C. L. M.

Tallent

Tran

K. M.

Ahmedani

B. K.

Roehrs

Roth

Drake

C. L.

(2019). Efficacy of digital CBT for insomnia to reduce depression across demographic groups: A randomized trial. Psychological Medicine, 49(3), 491–500. https://doi.org/10.1017/S0033291718001113

Darden

Espie

C. A.

Carl

J. R.

Henry

A. L.

Kanady

J. C.

Krystal

A. D.

Miller

C. B.

(2021). Cost-effectiveness of digital cognitive behavioral therapy (sleepio) for insomnia: A Markov simulation model in the United States. Sleep, 44(4), Article zsaa223. https://doi.org/10.1093/sleep/zsaa223

De Brün

O’Reilly-de Brún

O’Donnell

C. A.

MacFarlane

(2016). Learning from doing: The case for combining normalisation process theory and participatory learning and action research methodology for primary healthcare implementation research. BMC Health Services Research, 16(1), Article 346. https://doi.org/10.1186/s12913-016-1587-z

Dyas

J. V.

Apekey

T. A.

Tilling

Ørner

Middleton

Siriwardena

A. N.

(2010). Patients’ and clinicians’ experiences of consultations in primary care for sleep problems and insomnia: A focus group study. The British Journal of General Practice: The Journal of the Royal College of General Practitioners, 60(574), e180–e200. https://doi.org/10.3399/bjgp10X484183

10.

Espie

C. A.

Emsley

Kyle

S. D.

Gordon

Drake

C. L.

Siriwardena

A. N.

Cape

Ong

J. C.

Sheaves

Foster

Freeman

Costa-Font

Marsden

Luik

A. I.

(2019). Effect of digital cognitive behavioral therapy for insomnia on health, psychological well-being, and sleep-related quality of life: A randomized clinical trial. JAMA Psychiatry, 76(1), Article 21. https://doi.org/10.1001/jamapsychiatry.2018.2745

11.

Forma

Knight

T. G.

Thorndike

F. P.

Xiong

Baik

Velez

F. F.

Maricich

Y. A.

Malone

D. C.

(2022). Real-world evaluation of clinical response and long-term healthcare resource utilization patterns following treatment with a digital therapeutic for chronic insomnia. ClinicoEconomics and Outcomes Research: CEOR, 14, 537–546. https://doi.org/10.2147/CEOR.S368780

12.

Gardner

D. M.

Turner

J. P.

Magalhaes

Rajda

Murphy

A. L.

(2024). Patient self-guided interventions to reduce sedative use and improve sleep: The YAWNS NB randomized clinical trial. JAMA Psychiatry, 81(12), 1187–1197. https://doi.org/10.1001/jamapsychiatry.2024.2731

13.

Gibson

M. J.

Lawlor

D. A.

Millard

L. A. C.

(2023). Identifying the potential causal role of insomnia symptoms on 11,409 health-related outcomes: A phenome-wide Mendelian randomisation analysis in UK biobank. BMC Medicine, 21(1), Article 128. https://doi.org/10.1186/s12916-023-02832-8

14.

Grandner

M. A.

Chakravorty

(2017). Insomnia in primary care: Misreported, mishandled, and just plain missed. Journal of Clinical Sleep Medicine: JCSM: Official Publication of the American Academy of Sleep Medicine, 13(8), 937–939. https://doi.org/10.5664/jcsm.6688

15.

Haycock

Grivell

Redman

Saini

Vakulin

Lack

Lovato

Sweetman

Zwar

Stocks

Frank

Mukherjee

Adams

McEvoy

R. D.

Hoon

(2021). Primary care management of chronic insomnia: A qualitative analysis of the attitudes and experiences of Australian general practitioners. BMC Family Practice, 22(1), Article 158. https://doi.org/10.1186/s12875-021-01510-z

16.

Hayward

Jordan

K. P.

Croft

(2010). Healthcare use in adults with insomnia: A longitudinal study. The British Journal of General Practice: The Journal of the Royal College of General Practitioners, 60(574), 334–340. https://doi.org/10.3399/bjgp10X501822

17.

Healy

W. J.

Khayat

R. N.

Kwon

(2024). Insomnia: Advancements and limitations of current management strategies. American Family Physician, 109(2), 107–108. PMID: 38393789

18.

Koffel

Amundson

Polusny

Wisdom

J. P.

(2020). You’re missing out on something great: Patient and provider perspectives on increasing the use of cognitive behavioral therapy for insomnia. Behavioral Sleep Medicine, 18(3), 358–371. https://doi.org/10.1080/15402002.2019.1591958

19.

Love

(2002, August 11). Displaying covariate balance after adjustment for selection bias. Joint Statistical Meetings 2002, New York City. https://ww2.amstat.org/meetings/jsm/2002/onlineprogram/index.cfm?fuseaction=abstract_details&abstractid=301213

20.

Luik

A. I.

Marsden

Emsley

Henry

A. L.

Stott

Miller

C. B.

Espie

C. A.

(2020). Long-term benefits of digital cognitive behavioural therapy for insomnia: Follow-up report from a randomized clinical trial. Journal of Sleep Research, 29(4). https://doi.org/10.1111/jsr.13018

21.

Matheson

E. M.

Brown

B. D.

DeCastro

A. O.

(2024). Treatment of chronic insomnia in adults. American Family Physician, 109(2), 154–160. PMID: 38393799

22.

May

C. R.

Cummings

Girling

Bracher

Mair

F. S.

May

C. M.

Murray

Myall

Rapley

Finch

(2018). Using normalization process theory in feasibility studies and process evaluations of complex healthcare interventions: A systematic review. Implementation Science: IS, 13(1), Article 80. https://doi.org/10.1186/s13012-018-0758-1

23.

Miller

C. B.

Espie

C. A.

Bartlett

D. J.

Marshall

N. S.

Gordon

C. J.

Grunstein

R. R.

(2018). Acceptability, tolerability, and potential efficacy of cognitive behavioural therapy for insomnia disorder subtypes defined by polysomnography: A retrospective cohort study. Scientific Reports, 8(1), Article 6664. https://doi.org/10.1038/s41598-018-25033-3

24.

Morin

C. M.

Buysse

D. J.

(2024). Management of insomnia. New England Journal of Medicine, 391(3), 247–258. https://doi.org/10.1056/NEJMcp2305655

25.

Pillai

Anderson

J. R.

Cheng

Bazan

Bostock

Espie

C. A.

Roth

Drake

C. L.

(2015). The anxiolytic effects of cognitive behavior therapy for insomnia: Preliminary results from a web-delivered protocol. Journal of Sleep Medicine and Disorders, 2(2), 1017. PMID: 32195356.

26.

Pinnock

Barwick

Carpenter

C. R.

Eldridge

Grandes

Griffiths

C. J.

Rycroft-Malone

Meissner

Murray

Patel

Sheikh

Taylor

S. J. C.

, & StaRI Group . (2017). Standards for reporting implementation studies (StaRI): Explanation and elaboration document. BMJ Open, 7(4), Article e013318. https://doi.org/10.1136/bmjopen-2016-013318

27.

Qaseem

Kansagara

Forciea

M. A.

Cooke

Denberg

T. D.

, & Clinical Guidelines Committee of the American College of Physicians . (2016). Management of chronic insomnia disorder in adults: A clinical practice guideline from the American college of physicians. Annals of Internal Medicine, 165(2), 125–133. https://doi.org/10.7326/M15-2175

28.

R Core Team . (2023). R: A Language and environment for statistical computing. https://cran.r-project.org/

29.

Reed

A. C.

Rogers

D. G.

Berlin

G. S.

Burrone

Dante

DeViva

McCarthy

Niculete

M. E.

Santoro

Hermes

E. D. A.

(2024). Does format matter? A naturalistic study of digital and provider-led cognitive behavioral therapy for insomnia implemented in a healthcare system. Behavioral Sleep Medicine, 22(6), 883–893. https://doi.org/10.1080/15402002.2024.2385822

30.

Ross

Stevenson

Dack

Pal

May

Michie

Barnard

Murray

(2018). Developing an implementation strategy for a digital health intervention: An example in routine healthcare. BMC Health Services Research, 18(1), Article 794. https://doi.org/10.1186/s12913-018-3615-7

31.

Sampson

Bell

Cole

Miller

C. B.

Marriott

Williams

Rose

(2022). Digital cognitive behavioural therapy for insomnia and primary care costs in England: An interrupted time series analysis. BJGP Open, 6(2), BJGPO.2021.0146. https://doi.org/10.3399/BJGPO.2021.0146

32.

Stott

Pimm

Emsley

Miller

C. B.

Espie

C. A.

(2021). Does adjunctive digital CBT for insomnia improve clinical outcomes in an improving access to psychological therapies service? Behaviour Research and Therapy, 144, Article 103922. https://doi.org/10.1016/j.brat.2021.103922

33.

Sweller

(1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285. https://doi.org/10.1207/s15516709cog1202_4

34.

Thomas

Grandner

Nowakowski

Nesom

Corbitt

Perlis

M. L.

(2016). Where are the behavioral sleep medicine providers and where are they needed? A geographic assessment. Behavioral Sleep Medicine, 14(6), 687–698. https://doi.org/10.1080/15402002.2016.1173551

35.

Thorndike

F. P.

Berry

R. B.

Gerwien

Braun

Maricich

Y. A.

(2021). Protocol for digital real-world evidence trial for adults with insomnia treated via mobile (DREAM): An open-label trial of a prescription digital therapeutic for treating patients with chronic insomnia. Journal of Comparative Effectiveness Research, 10(7), 569–581. https://doi.org/10.2217/cer-2021-0004

36.

Ulmer

C. S.

Bosworth

H. B.

Beckham

J. C.

Germain

Jeffreys

A. S.

Edelman

Macy

Kirby

Voils

C. I.

(2017). Veterans affairs primary care provider perceptions of insomnia treatment. Journal of Clinical Sleep Medicine: JCSM: Official Publication of the American Academy of Sleep Medicine, 13(8), 991–999. https://doi.org/10.5664/jcsm.6702

37.

Vargas

Egeler

Walker

Benitez

D. D.

(2023). Examining the barriers and recommendations for integrating more equitable insomnia treatment options in primary care. Frontiers in Sleep, 2, Article 1279903. https://doi.org/10.3389/frsle.2023.1279903

38.

Wickwire

E. M.

(2024). Looking back and looking ahead: Advancing understanding of insomnia economics. Sleep, 47(8), Article zsae140. https://doi.org/10.1093/sleep/zsae140

39.

Wise

(2022). Insomnia: NICE recommends digital app as treatment option. BMJ, 377, o1268. https://doi.org/10.1136/bmj.o1268

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.12 MB

0.41 MB