Abstract
Background
Previously, consensus MS care standards were defined by MS specialist neurologists from 19 countries. We developed, piloted and refined an Excel-based quality improvement tool to enable MS services to benchmark against these standards. Here, we examine the refined tool.
Objective
To determine the applicability of the quality improvement tool in different healthcare settings.
Methods
MS centres across the globe were invited to pilot the quality improvement tool by coding the medical records of 36 adults with MS. We invited feedback on user friendliness, quality improvement tool usefulness and relevance of data collected.
Results
Seventeen centres from 14 countries participated; 14 completed the post-service evaluation survey. Over 50% of responders rated the tool ‘very easy’ or ‘easy’ to use and ‘very relevant’ to their service. Almost 85% of responders (11/13) planned to introduce changes to their service, including improvements in documentation, communication, interactions with colleagues and referrals; 85% would use a future shorter version of the tool.
Conclusions
The quality improvement tool can enable MS centres globally to benchmark their services. Widespread uptake of a shorter tool may help MS centres to work towards achieving consensus standards for brain health-focused care. Incorporation into routine clinical practice would drive adoption.
Keywords
Introduction
Time matters in MS. Delayed diagnosis and treatment, as well as suboptimal treatment, can result in irreversible accumulation of disability, transition from relapsing‒remitting to secondary progressive MS and reduced life quality.1–3 This understanding of MS underpins the MS Brain Health initiative, which aims to preserve neurological reserve – the ‘brain health’ – of people living with MS.
The initiative's first output was the report Brain health: time matters in multiple sclerosis. This evidence-based, now widely endorsed, policy report describes a strategy for preserving brain health. A central theme is timely action at every stage of the MS care pathway. 1
The initiative's second step was to articulate this strategy formally as consensus care standards. MS specialist neurologists from 19 countries participated in a modified Delphi consensus process to define international standards for timely MS care.
4
Using an iterative, online process, the participants reached consensus on a set of time frames for each variable that was included. A three-level framework of standards was agreed, based on the following definitions:
These MS Brain Health consensus standards for core, achievable and aspirational care provide MS teams across the globe with a shared framework for service evaluation, benchmarking and improvement.
4
If applied widely, these benchmarks should help individual MS centres strive for the highest level of care as well as reduce global disparities in service provision.
The initiative's third goal was to produce a quality improvement (QI) tool – a mechanism by which the consensus care standards could be measured in routine clinical practice, anywhere. Over the last few years, the value of consensus standards and a QI tool have become increasingly clear. Moreover, growing evidence supports the value of early MS treatment in reducing relapse rates5,6 and disability progression,7–11 as well as the importance of diligent, proactive disease monitoring and attention to comorbidities and lifestyle issues such as smoking and alcohol use.12–19 A recent audit has exposed a UK MS care crisis and the need for significant service improvements.20,21 In other disease areas, quality standards and improvement programmes have been associated with improved patient outcomes and experiences.22,23 MS services desperately need practical, widely applicable methods that support their case for improved care delivery.
Our aim was to develop, and test in diverse settings, a QI tool based on the global MS Brain Health consensus standards. Our focus was to test the QI tool's clinical usability and international applicability for service evaluation and improvement. A secondary objective was to collect preliminary data from services across a broad geographic area.
Materials and methods
QI tool development
MS healthcare professionals (members of the MS Brain Health Steering Committee) from MS clinics in Germany, Australia and the UK previously collaborated with a clinical trials specialist to incorporate the MS Brain Health standards for timely MS care into an Excel-based QI tool. They then piloted the tool in their MS centres (Dresden, Melbourne and Plymouth, respectively); local analysis of results from that initial pilot study led to changes in clinical practice in those centres. 24 Feedback from the three centres was incorporated to update and refine the QI tool, creating prototype 2 (described as ‘QI tool’ hereafter).
Pilot study design
MS healthcare professionals who had endorsed the MS Brain Health approach to MS care (MS Brain Health ‘champions’) and authors of the consensus standards paper 4 were invited, via email in June 2020, to pilot the QI tool in their centres. Each participating MS centre obtained approval from their local ethics committee; patient consent was not required. The period for data entry completion (service evaluation) was from 31 August 2020 to 10 May 2021 (Figure 1). Each centre reviewed the medical records of 36 adults with MS who attended during the study period (1 September 2017 to 29 February 2020; Figure 1).
Patient populations, study period and service evaluation period. PMS included patients with primary or secondary PMS and relapsing secondary PMS.
To obtain data relevant to each stage of the MS care pathway, 12 patient records were extracted from each of three different patient populations (Table 1: RMS-newly diagnosed; RMS-established; PMS). To allow for potential differences in prescribing practice, in MS centres where some patients with RMS were not receiving a DMT, nine patient records (rather than 12) were extracted from each of the patient populations described in Table 1; a further nine records were included from patients with established RMS, diagnosed before 1 September 2016, but not receiving a DMT. Those receiving and not receiving a DMT were included in the same group for data collection and analysis.
Definitions used for patient populations.
DMT: disease-modifying therapy; PMS: progressive MS; RMS: relapsing MS.
Included patients with relapsing secondary PMS.
Study inclusion criteria were:
aged 18 years or older on 1 September 2017; confirmed diagnosis of MS using contemporary diagnostic criteria25,26: diagnosis of RMS before 1 September 2016 (established RMS sample) or between 1 September 2017 and 28 February 2019 (newly diagnosed RMS sample); or diagnosis of primary or secondary progressive MS (PMS sample) before 1 September 2016); attendance at the MS centre during the study period; under regular follow-up at the centre on 29 February 2020.
To minimize selection bias, eligible participants were selected chronologically from people with MS attending the centre during the study period.
At each participating MS centre, medical records were systematically examined to identify the information required (Supplemental Table). Extracted data were entered into the QI tool. Results from all centres were combined for analysis.
Survey development and analysis
We developed an online survey (SurveyMonkey) to assess the QI tool's usability globally (Supplemental Text). Questions included the tool's ease of use and value for facilitating local change, the relevance of data captured and key data for repeated use. Data were analysed using descriptive statistics.
Results
QI tool: structure
The QI tool is an Excel-based workbook. Multiple tabs (Figure 2(a)) incorporate: detailed instructions for use; the MS Brain Health consensus standards for core, achievable and aspirational MS care 4 ; and three worksheets to input data from patient records (one for each patient sample; new RMS; established RMS, PMS) (Figure 2(b)). The data input worksheets contain fields for demographics, dates of events in the care pathway, duration of first post-diagnosis consultation and answers to binary (yes/no) questions about patient care (Supplemental Table). There are options for recording ‘missing’, ‘not applicable’ and ‘in future’.
Schematic of the MS brain health quality improvement tool, showing: (a) the content of the seven worksheets within the Excel-based workbook; (b) the stages of the MS care pathway assessed by the tool (full details in Hobart et al.)4 and examples of data entry fields for each of the three patient populations; (c) sample visual results summary for clinic-level data, showing how a clinic performed compared with each of the MS Brain Health standards; and (d) sample visual results summary of patient-level data from 10 patients with PMS in one clinic. Lines show core, achievable and aspirational MS Brain Health standards and circles show individual patient results.
Data validation prevents entry of invalid information. Hidden worksheets compute time intervals between dates of key events in the care pathway (Supplemental Table), automatically comparing the results with the MS Brain Health consensus standards.
Visual results summaries for clinic-level data (Figure 2(c)) and patient-level data (Figure 2(d)) are displayed within the QI tool. These auto-populate when the required data fields are completed to show the proportion of patient records that met or did not meet core, achievable and aspirational standards.
Participating MS centres and survey responders
Table 2 shows that between 31 August 2020 and 10 May 2021 the QI tool was trialled in 17 leading MS centres in 14 countries. Although widely spread geographically, and involving countries in different socioeconomic brackets, the triallists were representing MS services affiliated to university hospitals (12/17), national institutes (3/17) or located in major cities (2/17).
MS centres that participated in the pilot study.
Investigators at 14 of 17 centres completed the post-service evaluation survey. All but one was a healthcare professional or researcher (Table 3). Nine survey responders (64.3%) worked in centres routinely measuring performance against locally agreed MS care standards. Six centres (42.9%) benchmarked their practice against international or national MS care standards. Three of these six centres reported using the MS Brain Health standards for benchmarking. Table 3 shows reasons for participating in the pilot study. A common factor was the desire to identify areas for improvement. Most responders (9/14, 64.3%) required 7 days or fewer to complete the service evaluation.
Information about the survey responders and MS centres that participated.
Other included MS fellow, physiotherapist, clinical academic, research assistant and non-clinical staff (all n = 1).
Centres could select all options that applied.
Other was: ‘Identify where doing well, where not doing well and how we can improve’ (free-text response).
Defined as the time needed to input the required data from 36 patient records.
QI tool: ease of use
Figure 3 shows that over half (8/14, 57.1%) of the survey responders reported the QI tool ‘very easy’ or ‘easy’ to use. The remainder found it ‘somewhat easy’ to use. No responders reported the tool ‘not easy’ to use. Over half (8/14; 57.1%) of responders reported no difficulties using the tool or completing the service evaluation. Moreover, 6/14 responders (42.9%) reported that their centres routinely collect all the information required to complete the service evaluation questions.
Ease of use of the quality improvement tool reported by survey responders (n = 14).
Of the 14 survey responders, eight indicated that their centre does not routinely collect all the information requested in the tool. Two of the eight specified they do not routinely collect cognitive screening data and two do not record appointments with additional support services. Further, Table 4 shows that 9/14 (64.3%) noted specific data that most centres in their country would find difficult to collect but that is currently included in the QI tool.
Data requested within the tool that some respondents considered might be difficult for centres in their country to collect (n = 9).
DMT: disease-modifying therapy; MRI: magnetic resonance imaging.
QI tool: relevance of data captured
Figure 4(a) shows that 92.9% of responders (13/14) regarded the data collected as ‘very relevant’ or ‘relevant’ to their centre. All the data points were applicable to nearly all centres and countries (13/14; 92.9%). Most responders (9/13; 69.2%) thought their results reflected the care that people with MS receive at their centre. Figure 4(b) shows that all responders (13/13) considered it ‘very important’ or ‘important’ to regularly review timeframes relating to referral and diagnosis, treatment decisions, brain-healthy lifestyle, disease monitoring and managing new symptoms. Table 5 shows that over half the responders (8/14; 57.1%) made suggestions for key steps in the MS care pathway or other important considerations relating to MS care that the QI tool does not currently capture.
Number of responders reporting: (a) the relevance of the data captured in the quality improvement tool (n = 14); and (b) the importance of reviewing key aspects of the MS care pathway (n = 13).
Key steps in the MS care pathway or other important considerations relating to MS care that were reported as missing from the QI tool (n = 8).
DMT: disease-modifying therapy; QI: quality improvement.
QI tool: value for promoting service improvement
Over half (7/13; 53.8%) of responders reported that one or more results from their service evaluation surprised them. For example, one centre obtained a higher score than expected for cognitive screening and, conversely, two centres noted the absence of discussion about brain-healthy lifestyle in recently diagnosed patients. Table 6 shows the changes or strategies that 11 of 13 responders (84.6%) planned to introduce to their service because of the assessment. Further, 11 of 13 responders (84.6%) would use a short/refined version of the QI tool routinely in the future to help assess their clinical practice, and the same individuals were willing to participate in future discussions to develop a short version for regular use.
Reported planned changes/strategies to local MS services as a result of the service evaluation (n = 13).
With regard to potential future simplification of the tool – to a shorter instrument assessing fewer standards – survey responders were asked to list their top five priorities for annual assessment. Figure 5 shows the standards most commonly selected: frequency of follow-up clinical evaluations; time to referral to a neurologist following reporting to a healthcare professional of symptoms that might be related to MS; and time to appointment with a member of the MS team following an acute deterioration of symptoms.
MS brain health standards selected by the greatest number of responders as their top five priorities to reassess every year (n = 13). aOnly standards that were rated in the top five priorities by at least three responders were included in this graph.
Evaluation of the consensus standards
Figure 6 shows the pooled service evaluation results split by sample: Figure 6(a) shows the recently diagnosed RMS sample (16 standards, 198 records assessed); Figure 6(b) shows the established RMS sample (16 standards, 216 records); Figure 6(c) shows the PMS sample (10 standards, 198 records).
Pooled service evaluation results from records of patients with: (a) recently diagnosed RMS (diagnosed between 1 September 2017 and 28 February 2019); (b) RMS with or without treatment (diagnosed before 1 September 2016); and (c) PMS (diagnosed before 2016). The column to the right of each graph shows the total percentage and number of patients who achieved the core MS Brain Health standard or above. aThese standards were binary questions and they were not time limited; the level of the standard (core, achievable or aspirational) was assigned by the participants in the Delphi process.4 For all patient groups, the clinics entered only birth year from the records; day and month of birth were subsequently assigned to calculate mean age. Figures for eligible patient records that met the core standards or above for all patient records, across the three populations, were calculated excluding ‘na [not applicable] for this patient’ from the denominator of the %. Data were recorded as ‘missing’ when an event did not occur, or if there was no evidence in the patient's medical records that it had occurred. Insufficient information at data cut-off shows when it was considered not possible for an event to have occurred by the end of the study period, but that it may happen in the future. Discussion on brain-healthy lifestyle included a record of a discussion on any of the following topics: keeping weight under control; limiting the use of alcohol; avoidance of smoking; exercise for improving cardiovascular fitness; and intellectually enriching activities. Additional support services were defined as healthcare services that provide support to patients to make lifestyle modifications, including smoking cessation and bariatric clinics, appointments with nutritional specialists and physiotherapy.
The total sample mean age was 45 (SD 12.9 years). Recently diagnosed RMS and established RMS samples were younger than the PMS sample. The majority of patient records (69.3%; 424/612) were from women. These results are consistent with expectations.
At the level of the standards, a total of 31 standards were evaluated. Usable data, where we could determine if standards were met (at core, achievable or aspirational level) or not met, were present for 95.4% of ‘items’ (i.e. answers for all questions across all participants).
Not surprisingly, none of the standards were met by all patients. The proportion (number) of standards met, at the core level or above, for ≥ 75% of eligible patients was low across all three MS samples: recently diagnosed RMS, 31%; established RMS, 25%; PMS 30%. This implies that base-level standards of care recommended by healthcare professionals and endorsed by patients are not being achieved for people with MS. However, if this cut off is lowered to ≥ 50% of eligible patients, these sample percentages were higher (88%, 81% and 60%, respectively). This is consistent with clinical experience that people with PMS receive worse care than people with RMS, probably largely due to the emphasis on DMT-related aspects of care.
At the level of patients, a total of 612 people with MS were included. Eligible patient records that met or exceeded core standards, for all patient records across the three populations, varied substantially: 19‒98% in the newly diagnosed RMS sample; 39‒93% in the established RMS sample and 41‒92% in the PMS sample. Missing data across the total sample was 4.6% of ‘items’. The offer of cognitive screening was the standard achieved least often across all samples (range 20‒41%). These results indicate huge variability in care quality for all three samples. Interestingly, and contrary to the above, variability was greatest for people with newly diagnosed RMS and least for people with PMS.
The information not routinely documented by some MS centres included: discussions about brain-healthy lifestyle; referral for lifestyle modification support; screening for comorbidities; the response of the MS team to a patient with MS reporting an acute deterioration of symptoms; a record of follow-up appointments with a relevant member of the MS team.
Discussion
The MS Brain Health consensus standards for timely, brain health-focused MS care 4 provided an important step towards formalizing recommendations for MS management globally. We have developed an Excel-based QI tool to help MS centres audit and benchmark their own service against these international standards. MS centres across different healthcare settings participated in this pilot study. Survey feedback from 14 of the 17 centres show that the data requested in the QI tool are widely applicable to MS services. Moreover, the service assessment can be carried out by non-clinical as well as clinical staff (Table 3).
The QI tool enabled centres to identify gaps in their service, prompting discussions about how to implement appropriate changes in clinical practice. Changes already introduced, or planned, at participating MS centres include improving documentation, discussing brain-healthy lifestyle and enhanced MS monitoring. Not surprisingly, these planned changes (Table 6) overlap with the gaps in records. Follow-up discussions with the pilot centres should provide further information about the long-term effect of the resulting changes on clinical practice and patient care.
None of the 17 MS centres received monies for their participation. Their enthusiasm and degree of engagement was particularly encouraging, considering the added challenges posed by the COVID-19 pandemic during the study. This is promising for future uptake of a refined version of the QI tool.
Important limitations of our study are that the sample size is small and participating services were highly motivated centres already advocating timely MS care, thus resulting in a sample that is probably not representative of MS services worldwide. These two limitations have implications. First, they affect the generalizability and representativeness of our findings. Second, they prevent us making meaningful inferences about the causes of variability in standards of care among patients or centres such as, for example, type of MS centre, country, location, levels of expertise and presence of supportive staff. However, the demonstration of variability in standards alone is important because it provides evidence for the value of a QI tool to assist in service evaluation and development. Also, the variability we have demonstrated, in volunteer centres from motivated, leading MS Brain Health ‘champions’, raises very significant concerns about the state of MS care globally and the need to address service development on a global scale. There is little doubt that country-based health care provision, staffing, facility levels, data maturity and funding will be highly relevant. These questions can be answered once the QI tool is being widely used.
The very low overall missing data level (4.6%) implies that in its current format the QI tool can be incorporated into a clinical service. Nevertheless, we recognize that retrospective review of patient medical records may have resulted in inaccuracies with data interpretation and input. Locating some of the information requested was challenging for some centres, particularly those without electronic documentation. This implies that if the QI tool can be used in real-time it can drive service evaluation and development.
Access and extraction of data from historical patient records is labour intensive and additional to normal clinical care. Data collection needs to be incorporated into routine clinical practice to enable evaluation in real time. Adaptation (probably to a web-based format) to allow prospective access to records would facilitate use of the tool to gather data, for example, for national audits. Increased useability resulting in wider adoption of the consensus standards could help to transform routine practice globally.
Researchers in Germany have recently proposed the application of ‘digital twins’ in the MS care setting. 27 Using big data and artificial intelligence, this revolutionary approach enables visualization of a virtual copy (twin) of the patient at different stages of the disease and supports further therapeutic decisions. Digital twins have the potential to improve precision medicine for people with MS and provide more personalized and effective care. 27 The combination of digital twins with a quality improvement tool may facilitate a greater degree of personalized medicine and quality management.
Further work is required to refine the QI tool for widespread use. Feedback and recommendations from participating MS centres will be assessed to achieve a consensus on how best to enhance the tool's clinical usability and applicability. The survey responses have already helped to identify which standards might have the greatest effect on service improvement. We are aiming to develop a user-friendly, flexible instrument that can be incorporated into routine daily care documentation. In essence, our long-term aim is to make the tool indispensable to clinical teams. Once we have incorporated the feedback from this pilot, we will be better placed to make the next prototype more widely available, not just in additional countries but also in a greater variety of centres within each country. A modified, potentially shorter or customizable version of the QI tool could support external benchmarking and decisions about health policy. The current version of the QI tool (comprising all 26 consensus standards) is useful if healthcare professionals wish to identify gaps across the full-service offering. The results may help them demonstrate the need for improvement 28 and secure support for changes from the decision-makers and budget holders in their centres.
Patient perspectives on the consensus standards would be beneficial and would complement the information collected by MS centres. Additional details collected from the lived experience of people with MS could provide accurate data on benchmarks such as times to referral and diagnosis, and length of the initial appointment, to discuss the implications of their diagnosis. The QI tool could also support longitudinal studies into the impact of the standards on patient outcomes.
Widespread dissemination of the QI tool is only one part of the overall effort needed to achieve further endorsement and adoption of the consensus standards. The MS community should continue to debate and refine their quality standards for MS care, and work towards the international adoption of benchmarks that are clinically meaningful in most healthcare settings. A survey by the MS International Federation among healthcare professionals revealed that MS experts from 67 countries were personally aware of the MS Brain Health standards, and that the standards are endorsed and are being followed to some degree in 20 countries. Furthermore, experts from 16 countries within the MS International Federation reported future intentions to develop national standards based on the MS Brain Health initiative). 29 The establishment of MS Care Units 30 could provide a valuable opportunity for collaboration and a vehicle for the widespread dissemination of quality standards.
Our vision is for MS centres worldwide to use the QI tool in real time to compare their current practice with the MS Brain Health consensus standards and embark on a quality improvement cycle, ideally reporting annually for combined data analysis. Despite its limitations, this study demonstrates the potential power of the MS Brain Health QI tool to change clinical processes. Widespread future uptake of a potential modified version of the tool may help MS centres implement the MS Brain Health consensus standards, 4 and the evidence-based policy recommendations from which they evolved, 1 which are aimed at improving outcomes for people with MS.
Supplemental Material
sj-docx-1-mso-10.1177_20552173221124023 - Supplemental material for Timely intervention, monitoring and education MATTERS in MS (TIME MATTERS in MS): Development of a globally applicable quality improvement tool
Supplemental material, sj-docx-1-mso-10.1177_20552173221124023 for Timely intervention, monitoring and education MATTERS in MS (TIME MATTERS in MS): Development of a globally applicable quality improvement tool by Jeremy Hobart, Helmut Butzkueven, Jodi Haartsen, Tjalf Ziemssen and Thirusha Lane, Gavin Giovannoni in Multiple Sclerosis Journal – Experimental, Translational and Clinical
Supplemental Material
sj-docx-2-mso-10.1177_20552173221124023 - Supplemental material for Timely intervention, monitoring and education MATTERS in MS (TIME MATTERS in MS): Development of a globally applicable quality improvement tool
Supplemental material, sj-docx-2-mso-10.1177_20552173221124023 for Timely intervention, monitoring and education MATTERS in MS (TIME MATTERS in MS): Development of a globally applicable quality improvement tool by Jeremy Hobart, Helmut Butzkueven, Jodi Haartsen, Tjalf Ziemssen and Thirusha Lane, Gavin Giovannoni in Multiple Sclerosis Journal – Experimental, Translational and Clinical
Footnotes
Acknowledgements
The authors would like to thank the MS centres that participated in this pilot study. Dr Heather Lang of Oxford PharmaGenesis, Oxford, UK, developed the Excel-based workbook; Dr Emily Stock of PharmaGenesis London, UK, and Ruth Bentley, Independent Consultant, UK, provided medical writing and editorial support. All three were funded by Oxford Health Policy Forum, on behalf of the MS Brain Health initiative.
Declaration of conflicting interests
The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article:
J Hobart has received consulting fees, honoraria, support to attend meetings or research support from Acorda, Asubio, Bayer Schering, Biogen Idec, F. Hoffmann-La Roche, Genzyme, Merck Serono, Novartis, Oxford PharmaGenesis and Teva.
H Butzkueven's institution receives compensation for advisory board, steering committee and educational activities from Biogen, Merck, Novartis and Roche. His institution receives research support from Biogen, Novartis, Medical Research Future Fund and National Health and Medical Research Council Australia, MS Research Australia and Roche. He receives personal compensation from Oxford Health Policy Forum for serving in the steering group of MS Brain Health.
J Haartsen has received consulting fees from Biogen, Merck and Roche.
T Ziemssen has received grants from Biogen, Novartis, Sanofi and Teva, and personal fees from Almirall, Alexion, Celgene, Novartis, Sanofi, Teva, Merck, Roche and Viatris.
T Lane is the Director of Clinical Research London Ltd and has received fees from Oxford Health Policy Forum.
G Giovannoni has received consulting fees from AbbVie, Atara Bio, Bayer HealthCare, Biogen, Canbex Therapeutics, Five Prime Therapeutics, GlaxoSmithKline, GW Pharma, Japanese Tobacco, Merck, Merck Serono, Novartis, Oxford PharmaGenesis, Protein Discovery Laboratories, Roche, Sanofi Genzyme, Synthon, Teva Neuroscience and UCB, and grant/research support from Bayer HealthCare, Biogen, Merck, Merck Serono, Novartis and Sanofi Genzyme.
Funding
The authors disclosed reimbursement to Oxford PharmaGenesis, which provided research, writing and editing support in the development of this article: This research was commissioned by Oxford Health Policy Forum, on behalf of the MS Brain Health initiative. Grants to support MS Brain Health were provided by Actelion Pharmaceuticals, Biogen, Bristol Myers Squibb, F. Hoffmann-La Roche and Merck KGaA, all of whom had no role in study design, data collection, data analysis, data interpretation, writing of the report or in the decision to submit the paper for publication. All authors had full access to the data in the study and the corresponding author had the final responsibility for the decision to submit for publication.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
