Abstract
Background
Telemedicine outreach for posttraumatic stress disorder (TOP) is a virtual evidence-based practice (EBP) involving telephone care management and telepsychology that engages rural patients in trauma-focused psychotherapy. This evaluation examined implementation and intervention costs attributable to deploying TOP from a health system perspective.
Methods
Costs were ascertained as part of a stepped wedge cluster randomized trial at five sites within the Veterans Affairs (VA) Healthcare System. All sites initially received a standard implementation strategy, which included internal facilitation, dissemination of an internal facilitators operational guide, funded care manager, care managing training, and technical support. A subset of clinics that failed to meet performance metrics were subsequently randomized to enhanced implementation, which added external facilitation that focused on incorporating TOP clinical processes into existing clinic workflow. We measured site-level implementation activities using project records and structured activity logs tracking personnel-level time devoted to all implementation activities. We monetized time devoted to implementation activities by applying an opportunity cost approach. Intervention costs were measured as accounting-based costs for telepsychiatry/telepsychology and care manager visits, ascertained using VA administrative data. We conducted descriptive analyses of strategy-specific implementation costs across five sites. Descriptive analyses were conducted instead of population-level cost-effectiveness analysis because previous research found enhanced implementation was not more successful than the standard implementation in improving uptake of TOP.
Results
Over the 40-month study period, four of five sites received enhanced implementation. Mean site-level implementation cost per month was $919 (SD = $238) during standard implementation and increased to $1,651 (SD = $460) during enhanced implementation. Mean site-level intervention cost per patient-month was $46 (SD = $28) during standard implementation and $31 (SD = $21) during enhanced implementation.
Conclusions
Project findings inform the expected cost of implementing TOP, which represents one factor health systems should consider in the decision to broadly adopt this EBP.
Keywords
Introduction
Posttraumatic stress disorder (PTSD) is characterized by persistent symptoms including intrusive trauma memories, increased arousal, avoidance, and negative thinking that are associated with significant problems in social situations and relationships. Rates of PTSD are disproportionately high within the Veterans Affairs (VA) Health Care System, which provides comprehensive health care to eligible veterans. About 10% of VA enrollees have been diagnosed with PTSD (Trivedi et al., 2015), including 23.6% of enrollees returning from recent conflicts in Iraq and Afghanistan (Office of Mental Health Operations VA Central Office, 2012).
The first line treatments for PTSD are trauma-focused psychotherapies including cognitive processing therapy (CPT) and prolonged exposure therapy (PE) (Department of Veterans Affairs, 2020). These psychotherapies have been proven efficacious in randomized clinical trials (RCTs) and are disseminated widely by VA (Brooks et al., 2012). Prior research has also demonstrated that CPT and PE can be feasibly and effectively delivered through interactive video (Gros et al., 2011; Morland et al., 2014; Tuerk et al., 2010).
Despite broad dissemination of CPT and PE within VA, several studies have shown rurality is the strongest barrier of not engaging in trauma-focused psychotherapy for PTSD (Dufort et al., 2021; Hundt et al., 2018; Rosen et al., 2019). Theory suggests aspects of rurality including longer travel distances and less broadband access for interactive video encounters are key barriers preventing engagement (Fortney et al., 2011). Travel is a barrier because trauma-focused psychotherapies are mostly provided in specialty mental health or PTSD clinics at larger VA medical centers (VAMCs) located in urban areas. Less than half of veterans diagnosed with PTSD in primary care are successfully referred to specialty mental health (Magruder et al., 2005; Spoont et al., 2010). Moreover, treatment drop-out from trauma-focused psychotherapies delivered in VA specialty mental health care settings is extremely high, ranging from 28.3%-47.8% (Belleau et al., 2017; Goetter et al., 2015; Gros et al., 2011; Gutner et al., 2016; Kaysen et al., 2014; Keefe et al., 2018; Kehle-Forbes et al., 2016; Rauch et al., 2019; Reger et al., 2016). Engagement in trauma-focused psychotherapy is a critical issue to address in the VA.
Virtual delivery of trauma-focused psychotherapy represents one promising approach to address barriers to care for PTSD patients. A prior pragmatic RCT examined the effectiveness of Telemedicine Outreach for PTSD (TOP) among rural patients using a virtual care team to coordinate and to provide care at local VA Community Based Outpatient Clinics (CBOCs) (Fortney et al., 2015). This care team consisted of an on-site primary care team and off-site telephone care managers, telepsychologists, and telepsychiatrists at affiliated VAMCs in urban areas. Care managers encouraged medication initiation and adherence and engagement in CPT. Furthermore, prior research indicates care managers can also be beneficial by assisting in the adaptation of evidence-based practices normally delivered in-person to telehealth (Gros et al., 2013). This includes coordinating the use telehealth technologies by patients and provider, fostering adjustments to communication style and adjusting in-person assessments to be compatible with telehealth. In TOP, telepsychologists were responsible for delivering CPT through interactive video and telepsychiatrists made medication recommendations to the on-site prescriber. Compared to patients randomized to usual care, patients randomized to TOP were more likely to initiate CPT, complete at least eight CPT sessions and experienced larger reductions on PTSD symptom severity. Improved outcomes were mediated by attending eight or more CPT sessions.
Few studies have examined the cost of implementing Evidence Based Practices (EBPs) for mental health like TOP, and no studies to our knowledge have examined the cost of implementing practices to improve outcomes for patients with PTSD. Such information is critical to inform the business case for systematic efforts to increase engagement for trauma-focused psychotherapy for PTSD. To address the gaps in policy knowledge, this evaluation seeks to describe the implementation and intervention costs resulting from targeted strategies to increase CPT and PE uptake among rural patients with PTSD.
Methods
Study Design and Setting
This descriptive economic analysis examined implementation and intervention costs within a Type III Hybrid Effectiveness Implementation trial (Clinical Trials Identifier: NCT02737098) conducted within six VAMCs located nationally (Curran et al., 2012). The trial took place from April 2016 to July 2019. Per VHA Handbook 1200.21 (Department of Veterans Affairs, 2020), all VA authors of this manuscript attest that the activities that resulted in producing this manuscript were not conducted as part of a research project, but as part of the non-research evaluation conducted under the authority of the VA Office of Rural Health.
Implementation Trial Overview
VAMCs and affiliated CBOCs (henceforth denoted as sites) were randomized using a stepped wedge design with an adaptive implementation strategy (Figure 1) (Kilbourne et al., 2014). This project originally targeted patients receiving primary care at 12 identified CBOCs located in rural catchment areas, who had a primary or secondary diagnosis of PTSD at any outpatient encounter in fiscal year 2015 through 2016, had a Primary Care PTSD Screen for DSM-5 (PC-PTSD-5) score ≥ 3 and were not engaged in specialty mental healthcare for PTSD at a VAMC. A sample of patients meeting these eligibility criteria were included in the evaluation.
Overview of implementation trial design.
One site never implemented TOP and was excluded from the analysis. The other five sites all initially received standard implementation for a nine-month period. Over the next two months, there was a gap period where sites were evaluated based on whether they enrolled at least 75 patients with a care manager and at least 35% of enrolled patients received a trauma-focused psychotherapy. The choice of 75 patients enrolled with a care manager was based on our experience that this is the maximum number of patients a care manager is able oversee. Correspondingly, the choice of 35% as a threshold for the percentage of patients receiving trauma-focused psychotherapy was based on the prior finding that 55% of eligible patients in a randomized control trial received trauma-focused psychotherapy (Fortney et al., 2015). A lower percentage was chosen because 55% was thought not to be feasible in an implementation trial. At month 11, sites meeting this threshold remained in the standard implementation group. Sites not meeting the threshold were then randomized with half receiving enhanced implementation and the others remaining in the standard implementation group. Sites were then reevaluated over a two-month period at month 20 (nine months after randomization). At month 22, sites not deemed successful were assigned to enhanced implementation over the next 9 months. The goal was to have all sites either achieve successful implementation by meeting the defined threshold or receive enhanced implementation. Further details on the implementation trial are presented in a prior study (Fortney et al., 2022).
Standard Implementation Strategy
The standard implementation strategy focused on internal facilitation to actively encourage adoption of TOP. This was accomplished through the dissemination of an internal facilitation manual for use by the internal facilitator who also served as the local clinical champion. Internal facilitators were either physicians or PhD level clinical psychologists. The manual was adapted from prior research and provided sections identifying the role of the internal facilitator, a team meeting planner and implementation tools such as a site self-assessment checklist. The facilitation manual also identified specific activities the internal facilitator could adopt to promote TOP adoption including stakeholder engagement, site assessment, and program design. In addition to internal facilitation, a full-time care manager, funded by the VA Office of Rural Health was assigned to each site. The care managers’ time was devoted to managing CBOC patients with PTSD and they received care manager training from the VA’s national Primary Care Mental Health Integration initiative. Care managers were registered nurses or master’s level social workers. Each site was provided with a list of all patients (sampled and not sampled) at their facility’s targeted CBOCs who had a primary or secondary diagnosis of PTSD at any outpatient encounter in fiscal year 2015 through 2016, had a Primary Care PTSD Screen for DSM-5 (PC-PTSD-5) score ≥ 3 and were not engaged in specialty mental healthcare for PTSD at a VAMC.
Enhanced Implementation Strategy
The enhanced implementation strategy added external facilitation to internal facilitation and focused on rapid ethnographic assessment (REA) combined with systems redesign (Scrimshaw et al., 1991; Stetler et al., 2006). Enhanced implementation was conducted by a multidisciplinary team comprised of a trained external facilitator (PhD-level implementation researcher), PhD-level ethnographers and a veteran consultant with lived experience seeking VA mental health care. The key component of enhanced implementation, REA, seeks to understand how an intervention fits into the existing clinical workflow at the local practice setting. To achieve this goal, ethnographers conducted first-person observations and open-ended interviewing at clinic sites. Insights from the REA process were then used to inform systems redesign by the internal facilitator, external facilitator, ethnographers, and a veteran consultant. This specifically included mapping of existing clinical workflow, the development of a revised workflow map and the adaptation of TOP to better fit into the revised clinical workflow. The veteran consultant specifically focused on ensuring that the clinical workflow optimized patient engagement. Other enhanced implementation activities included (1) implementation team members collaborating with clinical supervisors, (2) mitigating the impact of negative stakeholders, (3) audit and feedback, (4) technical assistance, (5) design and adaptation of TOP, and (6) identifying and solving problems.
Data Sources
Data on implementation costs were collected through detailed project records. This included structured activity logs that tracked attendance and time involved in meetings and other activities related to implementation (Appendix, Figure 1) (Liu et al., 2009). Activity data were recorded on a routine basis by internal facilitators, care managers (implementation activities only) and external facilitators. Activity data included implementation stage (e.g., standard or enhanced implementation), type of implementation activity (pre-defined categories capturing how activity relates to implementation such as staff training), date, duration of activity, preparation time and a list of participants. A hierarchy for tracking was defined when multiple staff (e.g., internal facilitator, ethnographer, external facilitator, veteran consultant) were involved in activities to minimize reporting burden. For example, the external facilitator was at the top of the hierarchy. Thus, other participants involved in implementation activities with the external facilitator did not need to record time because data was tracked by the external facilitator. Project records also included time spent traveling to sites, travel costs (e.g., flights, hotels) and composing emails.
Salaries of implementation participants were derived from VA Personnel and Accounting Integrated Data (PAID). PAID data were augmented with data from the Bureau of Labor Statistics, which includes average wages by occupation and geographical region. Information on patient characteristics, outpatient encounters including mental health related telehealth visits was collected from the national VA Corporate Data Warehouse. Visits with TOP care managers were identified from text integration utility (TIU) notes. The cost of the veteran consultant was based on his annual consulting fee.
Uptake of Evidence-Based Psychotherapy
To provide additional context for the economic analysis, we present the proportion of sampled patients receiving at least one trauma-focused psychotherapy visit during standard implementation. In addition, we present the effect of enhanced implementation, compared to standard implementation, in the odds of having at least one trauma-focused psychotherapy visit. Having at least one trauma-focused psychotherapy encounter was modeled using mixed-effects logistic regression with fixed effects for time (months), implementation (standard or enhanced), and site, as well as interactions between implementation and site, and random patient-level intercepts. This model included one patient-month observation for each month during each implementation stage, resulting in 31 observations per patient over the three stages.
Costing Methodology Overview
Analysis of costs was conducted from the perspective of the health care system. Implementation costs represent the opportunity cost of time devoted to implementation activities, and the cost of travel (Shaw, 1992). For each staff member, we monetized time by multiplying the number of hours devoted to implementation by their estimated wage rate. The wage rate has been used to capture an individual’s opportunity cost, or the value of time performing an alternative activity, in prior research (Shaw, 1992). Travel costs were measured as reimbursed amounts from travel records. We summed participant-level time costs and travel costs by site and implementation stage.
We measured intervention costs for a random sample of approximately 100 patients at each site that met the clinical eligibility criteria for TOP at baseline. Intervention costs for these patients were measured over the duration of the trial and calculated as costs of 1) patient visits with care managers and 2) mental health telemedicine visits (both trauma-focused psychotherapy and other telemental health encounter types). Care manager visits related to TOP were identified through chart review and had to have a documented core element of care management (e.g., measurement-based care). These visits were linked to VA administrative data to identify the accounting-based cost of the visit to the VA site (Barnett, 2003). Mental health telemedicine visits were identified using a previously developed algorithm that captures pairs of records for encounters that occurred on the same day for a given patient (Adams et al., 2019). Record pairs were required to have qualifying stop codes, or identifiers capturing the clinical service type rendered at a designated clinic location (Hebert et al., 2014). We required record pairs to each have a mental health related primary stop code and secondary stop codes denoting telehealth at an origination and destination site, respectively.
Because the enhanced implementation was not more successful than the standard implementation, we did not conduct a population-level cost-effectiveness analysis (Fortney et al., 2014). Instead, we conducted descriptive analyses for implementation and intervention costs. Because sites received standard and enhanced implementation for different lengths of time, we calculated site-level implementation costs per month. Correspondingly, we calculated intervention costs per patient-month. We report means, standard deviations and ranges of site-level implementation and intervention costs for standard and enhanced implementation stages, respectively. Estimates were inclusive of costs that accumulated during the two-month gap evaluation periods and attributed to the stage that preceded the two-month gap. All costs were inflation adjusted to 2019 constant dollars using the Consumer Price Index (Dunn et al., 2018).
Results
Descriptive Statistics of Patients
Across the five sites, mean age of randomly sampled patients ranged from 47.7 to 54.9 years (Table 1). Most patients were male (85.5% to 95.9%) and married (54.3% to 66.3%). Non-Hispanic white veterans comprised over 70% of the sample at three sites, but less than 40% at two sites. We observed wide variation in the period of service of veterans and branch of service. Army represented the most frequent branch of service at all sites, ranging from 41.5% to 95.5% of veterans. Between 10.5% and 52.1% of veterans were identified as having combat exposure. Correspondingly, between 65.2% and 82.3% of veterans had a PTSD related service-connected disability. The percentage of veterans with active medications for PTSD (antidepressants and prazosin) ranged from 19.1% to 31.5%.
Selected statistics of target patient population.
NH = non-Hispanic, OEF = Operation Enduring Freedom, OIF = Operation Iraqi Freedom, PTSD = post-traumatic stress disorder.
Evidence-Based Psychotherapy
There was little variation in the proportion of patients with a trauma-focused psychotherapy encounter in the first (standard implementation) stage with no sites reaching a meaningful number of sampled veterans: site 1–2.1%, site 2–3.2%, site 3–5.5%, and site 5–3.2%. Site 4 met the implementation benchmark (75 patients enrolled in care management and 35% of care manager patients with an evidence-based psychotherapy encounter), but only 3.4% of veterans sampled from the target population had a trauma-focused psychotherapy encounter. Note that patients who were ultimately enrolled with a care manager over the course of the implementation were not necessarily those in the target population defined in the Implementation Trial Overview subsection.
Overall, we found the odds that a sampled veteran received an evidence-based psychotherapy encounter declined the more months they were in the study (OR = 0.938; 95% CI = 0.893 to 0.986) (Appendix, Table 1). Compared to standard implementation, odds for having an evidence-based psychotherapy encounter were not significantly higher for enhanced implementation at three of the four sites. For site 5, the odds an evidence-based psychotherapy encounter were 5.62 times greater (95% CI = 1.51 to 20.92, p = 0.010) during enhanced compared to standard implementation.
Implementation Costs
Site 4 met the threshold for successful implementation and was assigned to standard implementation in all three phases. The remaining four sites were randomized to enhanced implementation, sites 1 and 3 after the first stage and sites 2 and 5 after the second stage.
Across the five sites, mean site-level standard implementation costs were $919 (SD = $238) per month and ranged from $594 to $1,172 (Table 2). For the four sites that were randomized to enhanced implementation, mean site-level costs were $1,651 (SD = $460) per month, and ranged from $1,042 to $2,151. Markedly lower cost of enhanced implementation at site 3 were due to lower per-month cost of field-level activities and external facilitation. Specifically, at site 3, the per-month cost of field activities was $694 compared to an average of $1,020 for the remaining three sites. Also, the per-month cost of external facilitation was $185 compared to an average of $556 for the remaining three sites. The higher per-month cost of enhanced implementation at site 2 largely reflected travel costs ($488 per month, compared to an average of $170 for remaining sites).
Summary of implementation costs.
Cost estimates in 2019 constant dollars.
Includes activities conducted by personnel at sites during the standard and enhanced implementation phases, respectively.
Table 3 presents implementation costs by stage. For site 4, which did not receive enhanced implementation, costs decreased from $1,117 per month in stage 1 to $412 and $162 per month in stages 2 and 3, respectively. For sites randomized to enhanced implementation, costs increased in the subsequent stage. Costs stemming from enhanced implementation decreased from $1,565 per month in stage 2 to $373 per month in stage 3 for site 3 but increased from $1,398 in stage 2 to $1,953 per month in stage 3 for site 1.
Summary of implementation costs per month by stage.
Cost estimates in 2019 constant dollars.
Shaded boxes represent enhanced implementation.
Intervention Costs
Across the five sites receiving standard implementation, mean site-level intervention cost was $46 (SD = $28) per patient-month and ranged from $6 to $85 per patient-month (Table 4). For three of the five sites, the cost of tele-mental health (TMH) visits represented most of the intervention costs. For the four sites randomized to enhanced implementation, mean site-level intervention cost was $31 (SD = $21) per patient-month and ranged from $14 to $58 per patient-month.
Summary of intervention costs.
* CM-Care Manager.
* Costs are in 2019 constant dollars.
Discussion
Trauma focused psychotherapies such as CPT or PE are effective in treating PTSD, but not readily accessible to many patients that live in rural areas. New approaches to deliver these treatments virtually by a telepsychologist supported by a telephone care manager represent a promising approach for VA; however, uptake is low. Targeted implementation strategies have the potential to facilitate uptake programs like TOP. However, stakeholders have little information on the economic impacts of investing in such implementation strategies. To our knowledge, this study is the first to report the economic costs of strategies seeking to rollout evidence-based health programs for PTSD.
While the implementation trial was successful in connecting patients with care managers (Fortney et al., 2022), it was less successful in engaging those patients in trauma-focused psychotherapy. The difficulty of engaging veterans in trauma-focused psychotherapy observed in this implementation trial is consistent with prior research indicating the marked challenge in reaching rural veterans with PTSD (Dufort et al., 2021; Hundt et al., 2018; Rosen et al., 2019). Despite uniformly low levels of engaging rural veterans in trauma-focused psychotherapy, our findings indicate substantial variation in the monthly implementation cost incurred by sites during standard implementation. Monthly costs were around $1,000 per month for three sites, and markedly lower for the remaining two sites. At one of these two sites (site 4) that met the performance benchmark, implementation costs were relatively high over the first 11-month standard implementation stage, and then decreased substantially in later stages. This suggests that the lower costs over the duration of the evaluation were due to the implementation threshold for success being met early on followed by a reduction of implementation in later stages of the evaluation as sites switched focus to other goals.
Variation also existed in site-level costs of enhanced implementation. This variation was partly due to the difference in cost of travel for external facilitators to visit sites, particularly for site 2. Lower costs per month at sites 1 and 3 also reflect a longer implementation period in which to absorb fixed costs such as those related to travel. Another contributor of lower per-month costs at site 3 was turnover in the care manager (without replacement) early in stage three that ended enhanced implementation activities prematurely.
Implementation cost estimates from this evaluation are similar to two prior studies that examined strategies to implement evidence-based practices for mental health in VA. Fortney et al. (2014) examined strategies to implement telemedicine-based collaborative care for depression in rural VA clinics and found that implementation costs per site per month (inflation adjusted to 2019 dollars) were $1,490 (Fortney et al., 2013; Fortney et al., 2014). Using activity data collection methods similar to those in this study, Liu et al. (2009) found the clinic-level cost of implementing collaborative care for depression in primary care settings, after inflation adjusting to 2019 dollars, was $1,747 per month. Despite using disparate methods, our estimates are in line with these previous findings.
Differences in intervention costs across sites were reflective of TMH utilization rates during the standard implementation phase. Notably, sites 3 and 4 incurred the highest costs per patient-month for TMH costs. Site 1, which experienced challenges with implementation, also exhibited markedly low intervention costs, reflecting both few care manager and TMH visits.
Intervention costs per patient-month decreased for sampled patients in three of four sites receiving enhanced implementation, including a substantial decline at Site 2. One explanation is that intervention costs were ascertained for a sample of patients in the target population determined to be eligible for TOP at baseline. The reduction in intervention costs (and underlying CM and TMH visits) likely reflects the fact that midway through the trial, the local internal and external facilitators at the five sites jointly decided to focus efforts on patients referred for a TMH encounter instead of patients in the target population as described in the Implementation Trial Overview subsection. As a result, patients in the target population were much less likely to seek TMH visits from VA during the latter half of stage 2 and all of stage 3. Prior research has shown that VA enrollees with PTSD have been challenging to persistently engage in treatment (Mott et al., 2014) and focusing on those willing to be scheduled for an appointment was determined to be a more efficient use of resources.
Collectively, results from this evaluation provide information to help health systems and other stakeholders understand the expected costs of potential implementation strategies to broadly disseminate EBPs for the treatment of PTSD in rural patients. The enhanced implementation strategy has been successfully applied to the deployment of care management of depression (Kirchner et al., 2014). Cost estimates from this study can inform the expected financial burden to health systems of employing these two strategies (internal and external facilitation). It can be concluded that neither implementation strategy was intensive enough to achieve marked increases in trauma-focused psychotherapy. In this scenario, cost estimates would represent a lower bound to achieve success. The economic implications of implementation should be one of several considerations for health systems in deciding whether to engage in targeted efforts to promote the uptake of this EBP. While our evaluation measured the economic impacts of implementing an EBP for PTSD, findings and methods are also likely applicable to similar implementation strategies promoting uptake of practices for other mental health conditions.
This evaluation has several limitations. First, measurement of implementation costs was dependent on primary data collection from staff. Although steps were taken to minimize respondent burden including the use of structured activity logs and hierarchical delegation of activity tracking, collected data is subject to recall bias. Second, tracking of activities did not capture intensity or quality of effort. Finally, generalizability of findings from this evaluation should consider relative differences between non-VA and VA salaries, the latter of which were used to calculate the opportunity cost of time.
In summary, this evaluation reports implementation and intervention costs resulting from systematic efforts to increase the uptake of an evidence-based virtual care approach for treating PTSD. This study provides the first implementation cost estimate for a PTSD implementation trial and can provide a reference for future implementation initiatives for PTSD. In addition, cost estimates can be used by health systems to inform the relative value of employing the implementation strategies used in this study to increase adoption of EBPs for PTSD or other mental health conditions.
Supplemental Material
sj-docx-1-irp-10.1177_26334895221116771 - Supplemental material for Economic costs of implementing evidence-based telemedicine outreach for posttraumatic stress disorder in VA
Supplemental material, sj-docx-1-irp-10.1177_26334895221116771 for Economic costs of implementing evidence-based telemedicine outreach for posttraumatic stress disorder in VA by Edwin S. Wong, Suparna Rajan, Chuan-Fen Liu and Leslie A. Morland, Jeffrey M. Pyne, Fatma Simsek-Duran, Heather S. Reisinger, Jane Moeckli, John C. Fortney in Implementation Research and Practice
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
This project was funded by the VA Quality Enhancement Research Initiative (QUE 15–282) and the VA Office of Rural Health. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the authors' affiliated academic institutions.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
