Abstract
Background
Blood-based biomarkers have emerged as promising tools to optimize treatment decisions in multiple sclerosis (MS) including initiation, switch, or cessation of disease modifying therapies.
Objectives
The clinically validated MS disease activity (MSDA) test measures 18 proteins to derive a disease activity score. This study tests the clinical utility of MSDA in real-world practice.
Methods
Twenty clinicians from 14 clinics conducted a chart review utilizing a retrospective, longitudinal design, with a pre-post component. Chart reviews captured clinician decision-making before and after receipt of each MSDA result, while separate clinician assessments also captured the perceived impact of MSDA on MS management.
Results
A total of 352 charts were reviewed. The overall rate of clinical decision changes after MSDA testing (19.4%) exceeded predefined benchmarks. The proportion of patient time points where clinicians “strongly agreed” or “agreed” that MSDA results influenced their decision-making was greater when multiple longitudinal MSDA results were available compared to a single result: 69.2% (95%CI: [60.2%, 78.3%) vs. 59.8% (95%CI: [43.7%, 76.0%]), respectively.
Conclusion
When used in addition to standard of care, MSDA demonstrates clinical utility for real-world decision-making in MS management, based on objective changes in treatment plan and clinician
Keywords
Introduction
Multiple sclerosis (MS) is a complex and heterogeneous neurological disorder characterized by immune-mediated demyelination and neurodegeneration. Early and effective disease modifying therapy (DMT) use is associated with better long-term outcomes by limiting damage to the central nervous system, and subsequently preventing or slowing the accumulation of disability.1,2 However, optimal outcomes are dependent on choosing suitable therapies for patients at the right point in the individual patient journey.3,4 There are multiple challenges in MS care which collectively demand an individualized approach: variability in initial presentation and disease course, heterogeneity in underlying mechanisms of pathology, differences in DMT efficacy, patient-specific safety risks, tolerability issues, adherence barriers, and access to healthcare resources. These factors also highlight the significant difficulties for clinicians in deciding when to initiate, continue, switch, de-escalate, or discontinue DMT for individual patients.
Blood-based biomarkers have emerged as promising tools to guide treatment decisions in MS.5,6 The MS disease activity (MSDA) test is a commercially available, analytically and clinically validated blood-based biomarker test that measures 18 proteins to determine four (4) disease pathway scores (i.e. immunomodulation, neuroinflammation, myelin biology, and neuroaxonal integrity) and an overall disease activity (DA) score scaled from 1.0 to 10.0 with thresholds corresponding to low (1–4), moderate (4.5–7), and high (7.5–10) levels of disease activity.7–9 Figure S1 shows a portion of the MSDA clinician report with mock data.
Analytical 7 and clinical validation 9 studies demonstrated the accuracy, precision, sensitivity, and robustness of the MSDA panel and showed that MSDA score is associated with disease activity endpoints. High MSDA scores have a 21-fold greater odds of having 2 + gadolinium-enhancing lesions. 9
Understanding the real-world clinical utility of a disease-specific, biology-based test like MSDA, in a complex disease such as MS, is a necessary step for ushering precision medicine into routine neurological practice. Thus, to determine the value and impact of the MSDA on clinician decision-making as an adjunct to the clinical exam and MRI findings, the present study examined whether and how clinicians’ management plans change after reviewing MSDA results. Specifically, we assessed whether clinicians alter existing therapies, initiate new therapies, or implement other interventions.
Methods
Study design and chart selection
A retrospective chart review with a pre-post test design component (Figure 1) was conducted to assess the impact of MSDA on clinician decision-making for people with MS (PwMS).
Study design. A multicenter, retrospective chart review study with a pre-post test component was conducted in 14 clinics with MSDA in routine clinical use. Charts were reviewed to ascertain clinician decision-making before and after receipt of MSDA results after the first MSDA test and then second and third longitudinal MSDA tests. Purple dotted text boxes indicate time points for chart review by study site staff. Brown dashed text boxes indicate time points for the treating clinician.
Clinician assessment and staff chart review procedures
Out of the 16 sites and 22 clinicians invited, 14 sites and 20 clinicians participated in the study. All participating sites had utilized the MSDA as part of their routine clinical care following its commercial launch in March 2022, and none of the sites or data points were previously included in the prior clinical and analytical validation studies. The study included various types of clinicians, including neurologists and MS specialists. The clinical sites represent a diverse range of geographical locations and practice types, including academic medical centers, large hospital systems, and community-based practices (Table S1).
Charts eligible for review were identified based on longitudinal samples between March 1st, 2022 and January 8th, 2024. The clinician assessments on MSDA utility and the chart reviews were completed between January 2024 and May 2024. The treating clinician provided their subjective interpretation of the impact of the MSDA results on their clinical decision. To avoid recall bias, the clinical recommendations made by the treating clinician were collected by study site staff (for all but 1 site) through chart review of treating clinician notes to ascertain MS management before and after receipt of MSDA results. All data collected was de-identified.
Four types of data were collected from the chart review: (a) baseline demographics, (b) clinical recommendations before and after MSDA results were received, (c) clinician subjective assessment of the impact of MSDA results on subsequent clinical recommendations, and (d) clinicians were also asked about disease activity events within 60 days before or after the MSDA results were delivered (e.g. clinical or radiographic evidence of disease activity and/or infections), but, as they are not directly related to decision impact, they have been reserved for future analysis.
Baseline demographic characteristics and eligibility included age, sex, height, weight, race, ethnicity, smoking status, and other comorbidities. Additionally, MS disease history was collected and included year of diagnosis, MS type, DMTs currently prescribed, and the historical and present MS management plan.
The impact of MSDA was assessed via Likert scale (Strongly Agree, Agree, Neutral, Disagree, Strongly Disagree) for each time point:
Specifically, the Likert scaled responses to Prompt A and Prompt B were completed by the clinicians and consolidated into two binary outcomes: where “strongly agree” and “agree” responses (1; positive) were grouped and “neutral,” “disagree,” and “strongly disagree” (0; negative) were grouped for each statement separately.
To quantify the change in MS management decisions before and after the receipt of the MSDA results, binary responses (“continue the treatment plan” or “change the treatment plan”) to the following paired chart review prompts for analysis were considered for analysis and completed by the study site staff:
At each time point, the clinician decision was stratified into: continue current plan or change current plan. A change in current MS management plan was further subdivided into: initiate DMT, switch DMT, discontinue DMT, prescribe steroids, hospitalization or emergency room visit, request additional MRI or other imaging (i.e. imaging ordered outside of routine monitoring or standard of care), additional testing or referral (not including MRI or other imaging), and other with a prompt to describe the change in MS management.
Statistical analyses
Binary outcomes were derived from Prompts A, B, C and D. For each binary outcome we fit linear probability models. 10 To account for correlation among responses from the same treating clinician, we use cluster-robust standard errors 11 in reported p-values and 95% confidence intervals (CIs). To test whether single/multiple MSDA test results influenced decision-making based on responses to Prompts A and B, we consider a null hypothesis of a positive response rate expected from a fair coin flip. To test the statistical significance of the pre/post-decision change based on responses to Prompts C and D, we consider null hypotheses of 7% and 13% positive response rates. 12 Additional details are described in the Supplementary Materials.
Standard protocol approvals, registrations, and patient consents
The study received an IRB exemption determination from WIRB-Copernicus Group (WCG) IRB.
Results
Chart review demographics
Out of 412 charts screened for eligibility a total of 352 charts were included in final analysis. Table 1 summarizes the demographics of the chart review population which was consistent with the reported demographics of MS in the United States.13,14 Sixty charts total were excluded due to incomplete or missing data or screen fail (Table S2).
Characteristics of the chart review study population (N = 352 patients).
Example cases
Six cases from the chart review, exemplifying different MSDA use cases, can be found in Figures 2 and S3. For each case, the trajectories of clinical decisions are shown before and after MSDA results were received, along with the clinician-reported impact of the MSDA results at each decision point.
MSDA use cases for longitudinal clinical management. Cases include stable low MSDA patient with three time points (a), stable score post-high efficacy DMT therapy (b), and pre- and post- discontinuation of DMT (c). Purple dotted text boxes indicate time points for chart review by study site staff. Brown dashed text boxes indicate time points for the treating clinician.
Clinician-reported impact of MSDA on MS management at decision time points
Twenty treating clinicians completed assessments on 347 patient charts (including 722 MSDA results) to assess the impact of MSDA results in MS management of their patients. The results are summarized in Figure 3(a). For 59.8% (432/722) of all patient-time points, the treating clinician “strongly agreed” or “agreed” that a single MSDA result had an impact on their decision-making for the MS management of the patient (p = 0.217; 95% CI: [43.7%, 76.0%]). When restricting to longitudinal (second and third) patient-time points, the positive (“strongly agree” or “agree”) response rate occurs more frequently than a fair coin flip (estimate = 66.1%; 95% CI: [54.1%, 78.2%]; p = 0.011).
Counts of clinician assessment responses for up to 3 MSDA tests per patient chart reviewed. (a) The Single MSDA Influence summary shows results from the question: The MSDA test results influenced the decision to continue or change the MS management of this patient at this time. Data points are grouped by time point: baseline and otherwise (longitudinal). The dashed line highlights that 59.8% of clinicians “agreed” or “strongly agreed” that the single MSDA test result influenced their MS management decision across baseline and longitudinal time points. (b) The multiple MSDA Influence summary show results from the question: The change (or stability) across the [first and second/three] MSDA test results influenced the decision to continue or change the MS management of this patient. This question was only asked at longitudinal time points. The dashed line highlights that 69.2% of clinicians “agreed” or “strongly agreed” that the multiple MSDA test results influenced their MS management decisions. Note: The data shown comes from assessments completed by the treating clinician.
For 69.2% (261/377) of follow-up patient-time points, the treating clinician “strongly agreed” or “agreed” that multiple, longitudinal MSDA results influenced decision-making for the MS management of the patient (p < 0.001; 95%CI: [60.2%, 78.3%], Figure 3(b)).
Study staff chart review of MS management decisions
A total of 350 patients’ charts which included 723 MSDAs tests were reviewed to understand MS management decisions before and after MSDA results were received. In 19.4% of the 723 paired patient-time points, clinicians changed their clinical decision among the 5 options: continue, change: initiate new DMT, change: discontinue DMT, change: switch DMT, and change: other (no change to DMT), see Figures 4 and S4. The reported pre-post-MSDA decision change rate (19.4%; 95% CI: [13.3%, 25.4%]) is statistically significantly different from the benchmark null rates (p < 0.001 and 0.039 for benchmark rates of 7% and 13%, respectively 12 ), indicating a significant impact of MSDA on clinician decision-making.
Counts of pre/post-decision impact responses for up to 3 MSDA tests per patient chart reviewed by study site staff. Study site staff responses for the pre- (a) and post-MSDA results questionnaires (b) and paired pre/post-responses (c) are shown across baseline and longitudinal time points combined. The MSDA decision was a binary choice (continue or change) to the following prompts: “What was the clinician’s recommended treatment plan before receipt of the MSDA test result?” for the pre-decision, and “after the date of the [first/second/third] MSDA report delivery, was there a clinician recommendation to continue or change the MS management?” for the post-decision. Note: The data shown comes from study site staff chart review.
The impact of MSDA in clinician decision-making was more prominent for the first MSDA compared to the longitudinal MSDA(s). For the first MSDA, 23.4% of the 346 paired pre- and post-MSDA decisions changed. For the second MSDA, 16.1% of the 348 paired decisions changed. For the third MSDA, 10.3% of the 29 paired decisions changed. The proportions of changed clinician decisions for single (first MSDA) versus longitudinal (second/third MSDA) were significantly different (p-value from binomial test of proportions = 0.023).
Single and longitudinal MSDA results and MS management decision patterns
The majority of individual (single) time points examined had MSDA scores in the low range. For the pre-post chart review data, 53.5% of scores were low (1–4), 37.2% were moderate (4.5–7.0), and 9.3% were high (7.5–10). This pattern indicates overall low disease activity in the chart population sampled, consistent with the observed high proportion of patients on high efficacy DMTs.
Mean MSDA score was significantly different (p-value < 0.001) across groups defined by the five post-MSDA decision response categories. The distribution of MSDA scores by post-MSDA decision response is shown in Figure 5. Mean MSDA score was not significantly different (p-value = 0.42) across groups defined by the five Single MSDA Influence response categories. See Figures 6(a) and S5(B) for the distribution of MSDA scores and score categories, respectively, by the 5 single MSDA influence response categories. However, the positive response rates for the binary representation of single MSDA influence (i.e. indicator of “strongly agree” or “agree”) was significantly different across low, moderate and high MSDA categories (64.5% of low, 59.7% of high and 53.0% of moderate MSDA scores had positive responses; p-value from the Pearson's Chi-squared test = 0.013).
Relationships between MSDA scores and post-MSDA decisions from study site staff chart review. (a) MSDA score distribution by post-MSDA decision response. Low (1–4), Moderate (4.5–7.0), and High (7.5–10) MSDA score categories thresholds are shown on the right-hand y-axis. (b, c) Post-MSDA decision distribution by MSDA disease activity score category. Note: The data shown comes from study site staff chart review.
Relationship between MSDA scores and MSDA impact from clinician assessments. (a) MSDA score by single MSDA Influence decision impact category. Low (1–4), moderate (4.5–7.0), and high (7.5–10) MSDA score categories are shown on the right-hand y-axis. (b) MSDA Score change by Multiple MSDA Influence decision impact category for longitudinal MSDA tests. The MSDA score change contrasts two consecutive, longitudinal MSDA tests for a patient. Note: The data shown comes from assessments completed by the treating clinician.
There were 375 paired (baseline, longitudinal 2nd and 3rd) data points available for analysis of change in MSDA over time in the pre-post chart review data. Most data points correspond to no change in MSDA Score categorization (67.5% of paired score categories were unchanged, 20.3% increased, 12.3% decreased). The number of datapoints and proportions were similar in the clinician chart review data.
For paired, longitudinal time point data, there were too few observations to perform statistical analyses of the MSDA score change distributions and MSDA category changes by post-MSDA decision variables. The average MSDA score change was significantly different by multiple MSDA Influence category (p < 0.001, Figure 6(b)). The positive response rates for multiple MSDA influence (i.e. “strongly agree” or “agree”) was significantly different across unchanged, increased, and decreased MSDA change categories (87.0% of decreased, 73.7% of increased, and 65.2% of unchanged MSDA had positive responses; p = 0.009, Figure 7).
Relationship between multiple, longitudinal MSDA scores and MSDA impact from clinician assessments. MSDA score category change (low-low, moderate-moderate, high-high, increased, decreased) by multiple MSDA Influence decision impact category for longitudinal MSDA tests. The MSDA category change contrasts two consecutive, longitudinal MSDA tests for a patient. Note: The data shown comes from assessments completed by the treating clinician.
Discussion
The study's pre-post analysis emphasizes the decision impact and clinical utility of MSDA. In 19.4% of paired pre- and post-MSDA patient time points, clinicians change their MS management after viewing MSDA results; a change rate greater than that reported in an MRI clinical utility study (4.4%) even in the presence of new, enlarged, or active lesions. 15 Considering a conservative benchmark for human error in the chart review process at intra-rater variability estimates observed in previous studies, the observed rate of 19.4% indicates a significant difference from the benchmarks of 7% and 13%. 12
Based on the clinicians’ assessment of the impact of the MSDA on their decision-making, the MSDA had a meaningful impact on MS management at decision time points, particularly after multiple, longitudinal MSDA tests are available (i.e. clinicians were more likely to have “agreed” or “strongly agreed” that MSDA influenced decision-making than “neutral,” “disagreed,” or “strongly disagreed” for 2 out of 3 patient time points, on average; p < 0.011).
The MSDA score distribution by decision impact category demonstrated patterns in clinician MS management following receipt of the MSDA results (Figure 5). The median MSDA score was highest when the Post-MSDA decision was to change the treatment plan by initiating DMT and lowest where the decision was to continue the treatment plan or to change the treatment plan by discontinuing the DMT. The results may indicate that a high MSDA score among DMT naive patients help clinicians determine when to start DMT and that a low MSDA score among DMT discontinuation eligible patients help clinicians determine when to discontinue DMT. Further study is needed to confirm these observed trends.
Additionally, low MSDA scores may have confirmatory value for clinicians to continue their patient's MS management plan. Positive response rates (i.e. “strongly agree,” “agree”) to the single and multiple MSDA impact clinician assessments were highest when MSDA was low and decreased, respectively (Figure 3(a) case is an example). This may indicate the MSDA is most useful to clinicians when disease activity appears dormant and may confirm their current treatment plans are efficacious. Further study is needed to confirm these observed trends.
This study is novel in its investigation of a multivariate biomarker panel (MSDA) as a decision-making tool in real-world neurological practice. The closest parallel is with a single protein, neurofilament light chain (NfL) which is also one of the 18 proteins on the MSDA. Elevated serum NfL levels are indicative of axonal damage from myriad causes. 16 Guidelines for clinical use and interpretation of NfL as a single analyte, as well as consensus statements, have been published.17,18 The 18 protein multianalyte algorithm used in MSDA showed superior performance over NfL alone for three endpoints of disease activity 9 and, in concert, may allow for deeper—and potentially more actionable—insights into patient-specific drivers of disease activity.
Based on clinical validation, no single protein included in the panel or MSDA test algorithm is entirely specific for MS-related disease activity. Furthermore, within our clinical validation, approximately 80% of individuals in the low range did not have Gd + lesions and approximately 80% of individuals in the high range did have Gd + lesions. MSDA assesses the risk of a patient having new Gd + lesions and is not an absolute guarantee that a patient will/will not have Gd + lesions. No biomarker (fluid, imaging, or clinical) including MSDA is 100% specific and 100% sensitive. The MSDA test is intended to aid in the assessment of disease activity in patients with MS when used in conjunction with standard clinical and radiographic assessments as stated on our test report.
The study has several limitations inherent to real-world utility studies. The study was completed at sites that were already utilizing the MSDA test in routine clinical practice. This may limit generalizability of the results as this population of sites may already be inclined to use biomarker results compared to practices not currently using this information. We opened participation in the study to all sites with 5 or more longitudinal MSDA results to mitigate selection bias to the extent possible in this type of retrospective, real-world study.
The second limitation is the variability in how different sites and clinicians utilized the test in routine clinical practice. The test's application may have differed based on individual clinician experience, site protocols, and patient demographics. Charts were excluded for certain factors that significantly and independently impacted or limited MS management decisions, such as pregnancy and major comorbidity, and these special circumstances will be valuable to further assess in future studies. Given the retrospective design of this study, the subjective interpretation of the impact of the MSDA on clinical decision-making may be impacted by recall bias. However, to avoid recall bias by design, clinical recommendations were collected by study site staff through chart review of treating clinician notes to ascertain MS management before and after receipt of MSDA results. Given that clinicians were asked to evaluate the impact of the MSDA up to 2 years after its use, the delay in collecting this information has the potential to introduce recall bias. This potential bias may have impacted their assessment of the MSDAs true impact. Finally, due to the relatively short time frame, the long-term impact on patient outcomes was not captured. These factors highlight the value of future prospective studies to characterize the use of MSDA on patient, population, and societal outcomes.
The findings from this retrospective chart review study demonstrate the impact of MSDA on clinician decision-making in MS and provide evidence for the clinical utility of MSDA, particularly when used longitudinally approximately every 6 months to initiate, or switch DMT. Future studies would disentangle the relative contributions of different types of disease activity assessments (i.e. MRI, clinical exam, and blood-based biomarkers). Based on published criteria, this study provides evidence to support MSDA as a monitoring biomarker with a B3 level of evidence.19,20 When used in addition to standard of care, MSDA demonstrates clinical utility for real-world decision-making in MS management, based on the observed objective changes in treatment plan and clinician-reported impact, which increases with longitudinal use.
Supplemental Material
sj-docx-1-mso-10.1177_20552173251331030 - Supplemental material for Real-world clinical utility of a multi-protein, blood-based biomarker assay for disease activity assessments in multiple sclerosis
Supplemental material, sj-docx-1-mso-10.1177_20552173251331030 for Real-world clinical utility of a multi-protein, blood-based biomarker assay for disease activity assessments in multiple sclerosis by Angela Sanchez, Elisa Sheng, Sarah Eagleman, James L Eubanks, Patricia Izbicki, Shannon McCurdy, Matt Burrill, Ferhan Qureshi, Ati Ghoreyshi, Mitzi Joi Williams, Megan Weigel, William Kilgo, Jacqueline Nicholas, Annette Okai, Martin Belkin, Julie Burnham, Yasir Jassam, Michael Sy and Taylor Gonyou in Multiple Sclerosis Journal – Experimental, Translational and Clinical
Footnotes
Declaration of Conflicting Interests
The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Elisa Sheng, Sarah Eagleman, Shannon McCurdy, Matt Burrill, and Ferhan Qureshi, are employees of Octave Bioscience. Angela Sanchez, Patricia Izbicki, James L Eubanks, and Ati Ghoreyshi were previously employed at Octave Bioscience.
Taylor Gonyou, DO has acted as a consultant for Horizon Therapeutics, Sanofi, Genentech and EMD Serono.
Michael Sy, MD, PhD has received consulting fees from Roche, AstraZeneca, Octave Biosciences, and Pliant Therapeutics. He is named as inventor on a patent owned by University of California, Irvine describing disease therapy targeting N-glycosylation.
Yasir Jassam, MD has provided Research Support for Novartis Pharmaceuticals, SANOFI-GENZYME pharmaceuticals, Genentech Pharmaceuticals, TG Therapeutics, Octave Bioscience. Speaker Bureau of Amgen Pharmaceuticals, Alexion-AstraZeneca, Pharmaceuticals and TG pharmaceuticals. Advisory board participation: Octave Bioscience
Julie Burnham, DO Consultant/Ad board for Octave Bioscience.
Martin Belkin, DO has received research support from Biogen, EMD Serono, Sanofi, and Genentech. He received speaker's honoraria and acted as a consultant for Biogen, EMD Serono, Sanofi, Genentech, Alexion, Bristol Myers Squibb, TG Therapeutics, and Octave Bioscience.
Annette Okai, MD has received speaking and consulting fees from Alexion, Biogen, EMD Serono, Novartis, Roche Genentech, Sanofi Genzyme, TG Therapeutics, and research support from Alexion, EMD Serono, Novartis, Octave Bioscience, Roche Genentech, Sanofi Genzyme, and TG Therapeutics.
Mitzi Joi Williams, MD has provided Research Support for EMD Serono, Genentech, Novartis. Advisory Committee/ Consultant for Biogen Idec, Bristol Myers Squibb, EMD Serono, Genentech, Novartis, Sanofi Genzyme, Janssen, Alexion, TG Therapeutics. Speaker Bureau for Biogen Idec, Bristol Myers Squibb, EMD Serono, Genentech, Novartis, Sanofi Genzyme, Janssen.
Megan R. Weigel, DNP, ARNP-c, MSCN Consultant/Ad board for Sanofi, Horizon/Amgen, Cycle Pharma, Octave Bioscience. Speakers Bureau for Sanofi, Horizon/Amgen, BMS, Biogen, TG Therapeutics.
William Kilgo, MD serves on the speaker bureaus of Biogen, Genentech, and TG Therapeutics.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Octave Bioscience.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
