Abstract
Background
Age and sex were shown to influence multiple sclerosis (MS) relapse activity in the 1990s. Whether relapse risk factors are the same with new treatment paradigms is unclear. We evaluate predictors of clinical relapse following the first clinic visit (FV) across different treatment eras in a large, retrospective cohort.
Methods
Adults with clinically isolated syndrome or relapsing-onset MS were divided into cohorts with FV at the Brigham Multiple Sclerosis Center (Boston, MA) from 1997 to 2010 (“early”) and 2010 to 2020 (“recent”). Risk factors for relapse in 3 years after the FV were assessed for each cohort using multivariable logistic regression, and interaction terms were evaluated.
Results
2192 patients were included (early: 1536; recent: 656). Younger age, female sex, relapsing-remitting disease, more prior relapses, and the use of platform therapy were associated with a future relapse in the early cohort. Age, family history of MS, and platform therapy were predictive in the recent cohort. Interaction terms for all variables were not significant. Model accuracy was similar across treatment eras.
Conclusions
Predictors of future relapse did not differ substantially across treatment eras. Younger age and the use of less effective therapies were strong risk factors at FV. However, significant heterogeneity exists in individuals’ relapse risk.
Introduction
Annualized relapse rates (ARRs) vary among individuals with multiple sclerosis (MS). 1 Age, sex, and disease duration were shown to be associated with greater relapse activity based on data from the 1990s and early 2000s.2,3 Studies identifying other risk factors, such as the number of gadolinium-enhancing lesions on baseline MRI, also primarily included patients treated with lower efficacy therapies like interferon and glatiramer acetate (platform therapy).4–7 In 2010, the United States Food and Drug Administration approved the use of fingolimod, an oral disease-modifying therapy (DMT). Since then, there has been a proliferation of new moderate- and high-efficacy DMTs. 8 With increasing evidence supporting the use of high-efficacy therapy early in the disease course, DMT choice has changed substantially over the past 10 to 20 years, and relapses are now less common.9,10 It is not clear whether the risk factors for clinical relapse that were identified in the 1990s and early 2000s have the same significance today.
Potent immunosuppressive therapies have revolutionized the treatment of MS, but present risks such as infection, a small potential for malignancy, and reduced vaccine efficacy.11–13 Although high-efficacy therapy is beneficial at the population level, some patients do relatively well with minimal or no DMT. 14 However, it is difficult to identify these individuals when first making treatment decisions. Models of relapse risk and breakthrough disease activity may help with personalized treatment selection. Previous work has demonstrated the potential to predict the rate of disability accumulation in MS using patient characteristics and MRI data.15,16 Few studies have focused on predicting individuals’ relapse risk.17,18
In this study, we investigate clinical and demographic predictors of future relapse in the 3 years following the first clinic visit (FV) at our center in a large population of patients with clinically isolated syndrome (CIS) and relapsing-onset MS. We evaluate whether variables associated with subsequent relapse have changed over the past decades and how well basic clinical and demographic factors can predict MS relapse. We hypothesize that the salience of traditional relapse risk factors may have changed over time, potentially as a result of new diagnostic criteria, an increase in referrals for incidental MRI findings (possibly leading to a higher percentage of early or milder cases), and because more widespread use of high-efficacy DMTs might itself change the relative importance of traditional risk factors.
Methods
Approval
This study was approved by the Mass General Brigham Institutional Review Board. The need for individual patient consent was waived as part of the secondary use of data for which patients previously provided informed consent.
Inclusion criteria
Age 18 years or older; diagnosis of relapsing-onset MS, CIS, or suspected MS at the FV.
Exclusion criteria
Fewer than 3 years of follow-up available; diagnosis other than CIS or relapsing-onset MS at the most recent clinic visit; treatment with high-efficacy therapy (cyclophosphamide, natalizumab, alemtuzumab, S1P receptor modulators, anti-CD20 therapies, and mitoxantrone) prior to the FV.
Data source and cohorts
Records were queried from the Brigham and Women's Hospital MS Center Research Database, which includes all baseline and follow-up visits. Participants were dichotomized into “early” or “recent” treatment cohorts. Early cohort patients had the FV at the Brigham MS Center between January 1, 1997 (the start of record-keeping), and December 31, 2009. The recent cohort had the FV between January 1, 2010 and December 31, 2019 (to allow for at least 3 years of follow-up). 2010 was chosen to separate the cohorts to align with the approval of fingolimod and subsequent oral therapies, reflecting the timepoint after which DMT selection changed for the preponderance of patients.
Variable definitions and selection
We defined the FV as the first clinic visit at our center, not necessarily the first presentation to care. It is therefore possible for patients to have been started on DMT prior to the FV (although the use of high-efficacy or long-lasting therapies such as cyclophosphamide and alemtuzumab prior to the FV was an exclusion criterion). Variables were selected for inclusion based on data availability and literature review of established relapse risk factors.2–7,18 Age, disease duration, Expanded Disability Status Scale (EDSS), and MS subtype were calculated at the FV. Patients with “suspected MS” did not have CIS or meet contemporaneous McDonald criteria (or Poser criteria prior to 2001) at the FV, but had presentations thought to be consistent with MS; those who went on to be diagnosed with other conditions, such as neuromyelitis optica spectrum disorder, were excluded from the study, as described above. Smoking history included current and former smokers. Family history is recorded as positive or negative by the treating clinician, but the specific family member affected cannot be easily determined from the database. Clinical relapses were adjudicated and reported by the treating MS specialist, as were new T2 and gadolinium-enhancing lesions. Patients were marked as using platform therapy if they were treated with interferon or glatiramer acetate at any point and for any duration in the 3 years after the FV.
Additional imaging features, such as the total number of enhancing lesions, were not available in this dataset. Granular information on treatment was limited by several factors. It was difficult to confirm the exact dates that treatments were started or stopped, and we had no way to ascertain adherence. Patients could, for example, be started on glatiramer acetate, relapse, start anti-CD20 therapy, and be listed as receiving both platform therapy and high-efficacy therapy during the study period. Platform therapy was therefore a more interpretable predictor, since few patients relapsed on a higher-efficacy agent and then switched to low-efficacy therapy.
Missing data
EDSS at the FV was excluded as an independent variable due to a greater-than-expected number of missing data. EDSS was absent more often in the early cohort (6.0%) than in the recent cohort (3.2%), and was missing primarily for patients with “suspected MS” at the first visit. There were no notable issues with missing data for other continuous variables. For categorical variables such as new T2 lesions, patients were listed as yes, no, or unknown. Unlike continuous variables, unknown categorical data did not cause patients to be excluded from logistic regression.
Outcomes
The outcome of the prediction models was a binary classification of whether a clinical relapse would occur in 30 days to 3 years after the FV. This time window was chosen to balance a larger sample size with medium-term follow-up; our assumption was that a risk assessment algorithm at the FV might guide initial DMT decisions, but that relapse risk and treatment strategies would change over the long-term disease course.
Statistics
Logistic regression was performed using baseline demographic and clinical features as predictors and at least one future relapse over the 3 years after the FV as the outcome. This model was fit separately in the early and recent treatment cohorts, as well as in the overall dataset. A full multivariable model was fit including all predictors (listed in Table 1) for each of the cohorts. Univariable logistic regression was also performed for each predictor as supplementary analysis. To assess if the effect of the predictors was the same in the two cohorts, interaction terms for each variable were added to the multivariable model. Model fit was assessed by calculating performance statistics, including the area under the ROC curve (AUC) for discrimination and measures of classification accuracy. In order to assess model performance on the out-of-sample data, 10-fold cross-validation was used, with performance statistics calculated in the left-out sample and averaged over the folds. Classification accuracy was assessed at the approximate probability where sensitivity and specificity intersected for each cohort across all folds, in order to balance the prediction performance for those with and without future relapse (Figure 1). Statistical tests were run in Stata/SE 18.0 (StataCorp, LLC., College Station, TX, USA). An alpha level of 0.05 was used.
Patient characteristics.
Characteristics of the treatment cohorts and overall patient population. MS subtype was included as a categorical variable with SPMS as the reference group. All variables were included in the logistic regression models, with the exception of EDSS, which was omitted due to incomplete data in ∼5% of patients. EDSS is shown here for the remaining 95% of patients. Abbreviations: FV: first clinic visit; SD: standard deviation; IQR: interquartile range.
Results
Patient characteristics
The early treatment era cohort included 1536 patients with a mean age of 40.5 and a median disease duration of 4.5 years at the FV (Table 1). The recent treatment cohort had 656 patients (mean age: 41.6; median disease duration: 2.3 years). The difference in sample sizes was primarily due to early expansion of the Brigham MS Center, with a shift to a greater percentage of follow-up visits over time. The early treatment group had higher rates of current or former smokers, those with progressive disease, and those with a family history of MS. They similarly had slightly fewer clinical attacks in the year prior to the FV. Notably, 82.1% of the early patients were treated with platform therapy in the 3 years following the FV, while 47.1% of the recent cohort received platform therapy (Supplementary Materials: Table A).
Relapse rates have decreased over time
As expected, the early treatment cohort had substantially more relapses following the FV. The 3-year ARR of the early cohort was double that of the recent treatment cohort (0.24 vs. 0.12). Overall, 41.0% of patients in the early treatment group experienced a relapse in the 3 years following the FV, compared to 24.7% in the recent treatment group (V), yielding an odds ratio (OR) of 2.11 (p < 0.001; 95% confidence interval [CI]: 1.72–2.60). Comparing specifically those patients treated with platform therapy, 44.7% of early cohort patients experienced a relapse in the 3 years following the FV versus 32.0% of the recent cohort (OR: 1.72; p < 0.001; 95% CI: 1.32–2.23) (Supplementary Materials: Table B).
Predictors of relapses have remained similar over time
In a multivariable logistic regression model, younger age, female sex, RRMS or suspected MS, a greater number of attacks prior to the FV, and use of platform therapy were associated with future relapse in the early cohort (Table 2). Younger age and treatment with platform therapy were also predictive of relapse in the recent treatment cohort, along with a family history of MS. Sex, MS subtype, and total relapses prior to the FV were not significantly associated in the recent cohort. However, the interaction term for each variable across different treatment eras was not significant (Table 3).
Odds ratios of relapse across treatment eras in a multivariable logistic regression model.
Odds ratio of relapse in the three years after the first clinic visit (FV), 95% confidence intervals (brackets), and p-values (parentheses) for the multivariable logistic regression model in each treatment era and in the combined cohort. For the MS subtype, SPMS was used as the reference group. Variables listed in italics were found to be statistically significant predictors of relapse.
Interaction terms for predictors across treatment eras.
Results of interaction analyses for variables in the multivariable logistic regression model across treatment eras. No interaction terms were significant, highlighting that these variables did not have different relationships to future relapse in the early and recent treatment cohorts.
As predictors did not behave differently with respect to relapse risk across treatment eras, we evaluated the overall cohort to see if more accurate relapse predictions could be reached with a greater sample size (n = 2192). The predictors of future relapse identified in different treatment eras (age, sex, MS subtype, the family history of MS, a greater number of attacks prior to the FV, and the use of platform therapy) were all found to be significant in the combined cohort (Table 2). Additionally, the presence of at least one relapse in the 3 years before the FV was found to be significantly associated with future relapse. Several clinical features that might be assumed to impact relapse risk, such as shorter disease duration and the presence of at least one relapse in the year prior to the FV, were associated with future relapse in univariable analyses, but did not remain significant predictors in the multivariable models (Supplementary Materials: Table C).
For all patients, regardless of the treatment era, younger age was one of the strongest factors associated with relapse in the multivariable model, with an OR of 0.97 (p < 0.001; 95% CI: 0.96–0.98) for each additional year of age. Treatment with platform therapy was similarly one of the strongest predictors of relapse in the early, recent, and combined cohorts. As DMT selection is informed by the perceived risk of disease activity, we further evaluated the relevance of platform therapy use in the multivariable model under different conditions. Among 804 patients without a relapse in the 3 years before the FV, platform therapy was still a significant predictor of future relapse (OR: 1.77; p = 0.005; 95% CI: 1.18–2.64); in comparison, the OR for the 1388 patients with recent disease activity was 2.79 (p < 0.001; 95% CI: 2.10–3.70). Among the 448 patients over age 50, platform therapy remained associated with future relapse risk in multivariable analysis (OR: 1.73; p = 0.048; 95% CI: 1.01–2.96). For patients younger than 50, platform therapy carried an OR of 2.63 for subsequent relapse in the 3 years after the FV (p < 0.001; 95% CI: 2.04–3.39).
Relapse prediction accuracy is modest
The logistic regression model of the pooled early and recent treatment cohorts had a similar AUC of 0.66 compared to the models for the early (AUC: 0.65) and recent (AUC: 0.62) treatment cohorts. The sensitivity and specificity for future relapse differed between cohorts, but the accuracy was similar at 60–64% (Supplementary Materials: Table D).
Discussion
In this study, we evaluate whether clinical and demographic risk factors of MS relapse differ for patients who established care between 1997–2010 and 2010–2020. We find that the predictors of future relapse in the 3 years following the FV have not changed over the decades, despite changes in MS diagnostic criteria and disease-modifying therapies. We also find that substantial heterogeneity exists in MS relapse risk, even after accounting for significant clinical and demographic predictors of future relapse.
While relapse risk factors were similar between 1997–2010 and 2010–2020, the ARR in the recent cohort was half of that seen in the early cohort. This is consistent with the increased availability and use of higher efficacy treatment among patients with the FV between 2010 and 2020. Looking specifically at those who received platform therapies, the relapse rate remained lower in the recent cohort compared with the early cohort, potentially reflecting treatment selection bias. Nevertheless, even in the recent treatment cohort, platform therapy was associated with a substantially increased likelihood of relapse.
Among early treatment patients, younger age, female sex, RRMS or suspected MS, and the total number of relapses before the FV were also statistically significant predictors of future relapse. The results from the early treatment cohort are consistent with previous population studies of patients who were predominantly untreated or treated with platform therapies in which younger age, female sex, and more active disease were predictors of greater relapse risk.2,3 Younger age, along with the use of platform therapies, remained associated with a greater relapse risk in those with the FV between 2010 and 2020 in our dataset. Several variables were found to be significantly associated with relapses in only one of the early or recent treatment cohorts (e.g., family history). However, interaction terms were not significant, meaning that these variables did not have a fundamentally different impact on relapse risk across treatment eras. Consequently, differences between relapse predictors were likely the result of the different cohort sizes and the fact that many variables were either substitutable, conveying similar information about relapse risk, or had a marginal predictive value. This is supported by the fact that the accuracy and AUC of the early, recent, and combined cohort models were all similar. Sensitivity, specificity, and positive predictive value differed slightly between the models, reflecting in part the different relapse rates between the cohorts.
Despite the strong association of variables such as younger age with future relapse risk, significant heterogeneity exists among patients. This is highlighted by the AUC of the models for the early, recent, and overall cohorts, which ranged from 0.62 to 0.66. It may be that clinical and demographic variables do not capture enough information to adequately predict relapse risk for a given individual, and that incorporating additional data sources (e.g., fluid and imaging biomarkers) could help in modeling relapse risk.19–21 Another consideration is that the patient population may be too heterogeneous to be accurately represented by a single model. Notably, the importance of platform therapy as a risk factor for relapse was less pronounced in older patients and in those without recent disease activity at the FV, reflecting that the salience of risk factors is likely dynamic, varying between subsets of patients and at different points in the disease course. Consequently, developing different models of disease activity for more homogenous groups of patients could potentially improve prediction accuracy.
Although additional variables might be assumed to increase model performance, some that were associated with future relapse in univariable analysis did not contribute significantly to a multivariable model. A history of at least one relapse in the year before the FV is one such example. Shorter disease duration is widely viewed as a risk factor for relapse, which was supported by univariable analysis in this study, but not seen in the multivariable model. 2 New T2 and gadolinium-enhancing MRI lesions among early cohort patients were similarly significant predictors in univariable, but not multivariable, analysis. These results highlight that many risk factors, such as age and disease duration, may convey similar or overlapping information with respect to relapse risk. Incorporating additional clinical and demographic variables, therefore, may not necessarily improve model performance. Instead, predictors that provide unique information about relapse likelihood are needed; polygenic risk scores and serum biomarkers such as neurofilament light chain are potential examples.20,22,23
Multiple limitations to this study should be considered. Most importantly, DMT selection was a potential source of bias. Furthermore, we did not have granular information on adherence or exact start and stop times with respect to treatment. Consequently, specific DMT use in the 3 years after the FV was essentially binary in our dataset. This limited our ability to investigate associations with high-efficacy therapy. Platform therapy was a more interpretable predictor, since few patients relapsed on an alternate agent and then switched to lower-efficacy therapy. As many visits occurred prior to 2017, anti-CD20 use was less prevalent in our sample than it is now. While we show that there is no significant difference in clinical and demographic relapse predictors between treatment eras, it is possible that the results would be different for a cohort treated primarily with anti-CD20 therapy. The current prevalence of anti-CD20 use and the emergence of novel therapies may therefore limit the generalizability of the results. However, as the ARR of patients on anti-CD20s is very low, relapse risk factors may be difficult to identify in a cohort primarily treated with these therapies, potentially increasing selection bias.24,25 Another potential limitation is the lack of a standardized definition and the inability to validate clinical relapses, which were recorded at the discretion of the treating MS specialist. This could potentially introduce error that limits the accuracy of the predictive models. Changes in MS diagnostic criteria over time could add further heterogeneity. Additionally, the lack of granularity for variables such as family and smoking history (there was no differentiation of current from former smokers, for example) may affect the salience of these features in the model. 1 As this work was done at a single center with a predominantly Caucasian cohort, generalizability to other patient populations should also be considered.
A strength of this study is the large sample size. Furthermore, we believe that the focus on predicting relapses from the first visit is clinically relevant. Little has been published so far on modeling relapse risk at DMT initiation. Additional time points make prognostication easier, but models that predict relapse risk 3–5 years into patient care are less helpful in informing DMT selection, given the importance of early DMT initiation in most patients.
26
While some patients in this study were seen elsewhere prior to their first visit at our center, 61% had never been on DMT at the FV. We believe that our results are helpful in identifying factors relevant to relapse risk when seeing new patients in the clinic, while highlighting the variability that exists among individuals.
Sensitivity and specificity trade-off of the logistic regression model. Example plot of sensitivity and specificity versus estimated relapse probability for a representative cross-fold validation of the logistic regression relapse model for the total cohort.
Supplemental Material
sj-docx-1-mso-10.1177_20552173251408619 - Supplemental material for Multiple sclerosis relapse risk factors across treatment eras
Supplemental material, sj-docx-1-mso-10.1177_20552173251408619 for Multiple sclerosis relapse risk factors across treatment eras by Evan Madill, Brian Healy, Mariann Polgar-Turcsanyi and Tanuja Chitnis in Multiple Sclerosis Journal – Experimental, Translational and Clinical
Footnotes
The study was conducted in accordance with the Declaration of Helsinki and was approved by the Mass General Brigham Institutional Review Board, with the need for individual written informed consent waived.
Not applicable.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by a Clinical Research Training Scholarship from the American Academy of Neurology to the institution of E.M.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data availability
De-identified data described in this article may be made available upon reasonable request to the corresponding author (subject to IRB review).
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
