Predicting disability progression in multiple sclerosis: Insights from advanced statistical modeling

Abstract

Background:

There is an unmet need for precise methods estimating disease prognosis in multiple sclerosis (MS).

Objective:

Using advanced statistical modeling, we assessed the prognostic value of various clinical measures for disability progression.

Methods:

Advanced models to assess baseline prognostic factors for disability progression over 2 years were applied to a pooled sample of patients from placebo arms in four different phase III clinical trials. least absolute shrinkage and selection operator (LASSO) and ridge regression, elastic nets, support vector machines, and unconditional and conditional random forests were applied to model time to clinical disability progression confirmed at 24 weeks. Sensitivity analyses for different definitions of a combined endpoint were carried out, and bootstrap was used to assess prediction model performance.

Results:

A total of 1582 patients were included, of which 434 (27.4%) had disability progression in a combined endpoint over 2 years. Overall model discrimination performance was relatively poor (all C-indices ⩽ 0.65) across all models and across different definitions of progression.

Conclusion:

Inconsistency of prognostic factor importance ranking confirmed the relatively poor prediction ability of baseline factors in modeling disease progression in MS. Our findings underline the importance to explore alternative predictors as well as alternative definitions of commonly used endpoints.

Keywords

Prognostic factor ranking pooled placebo arms MS disease progression advanced methods random forests model performance

Get full access to this article

View all access options for this article.

References

Pittock

Mayr

McClelland

, et al. Change in MS-related disability in a population-based cohort: A 10-year follow-up study. Neurology 2004; 62(1): 51–59.

Jokubaitis

Spelman

Kalincik

, et al. Predictors of long-term disability accrual in relapse-onset multiple sclerosis. Ann Neurol 2016; 80(1): 89–100.

Steyerberg

Uno

Ioannidis

JPA

, et al. Poor performance of clinical prediction models: The harm of commonly applied methods. J Clin Epidemiol 2018; 98: 133–143.

Moons

KGM

Altman

Reitsma

, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): Explanation and elaboration. Ann Intern Med 2015; 162(1): W1–W73.

Bagherzadeh-Khiabani

Ramezankhani

Azizi

, et al. A tutorial on variable selection for clinical prediction models: Feature selection methods in data mining could improve the results. J Clin Epidemiol 2016; 71: 76–85.

Zhao

Healy

Rotstein

, et al. Exploration of machine learning techniques in predicting multiple sclerosis disease course. PLoS ONE 2017; 12(4): e0174866.

Van der Ploeg

Austin

Steyerberg

. Modern modelling techniques are data hungry: A simulation study for predicting dichotomous endpoints. BMC Med Res Methodol 2014; 14: 137.

Steyerberg

Harrell

Jr.

Prediction models need appropriate internal, internal-external, and external validation. J Clin Epidemiol 2016; 69: 245–247.

Calabresi

Kieseier

Arnold

, et al. Pegylated interferon β-1a for relapsing-remitting multiple sclerosis (ADVANCE): A randomised, phase 3, double-blind study. Lancet Neurol 2014; 13(7): 657–665.

10.

Fox

Miller

Phillips

, et al. Placebo-controlled phase 3 study of oral BG-12 or glatiramer in multiple sclerosis. N Engl J Med 2012; 367(12): 1087–1097.

11.

Gold

Kappos

Arnold

, et al. Placebo-controlled phase 3 study of oral BG-12 for relapsing multiple sclerosis. N Engl J Med 2012; 367(12): 1098–1107.

12.

Polman

O’Connor

Havrdova

, et al. A randomized, placebo-controlled trial of natalizumab for relapsing multiple sclerosis. N Engl J Med 2006; 354(9): 899–910.

13.

Altman

Royston

What do we mean by validating a prognostic model. Stat Med 2000; 19(4): 453–473.

14.

Van Klaveren

Gonen

Steyerberg

, et al. A new concordance measure for risk prediction models in external validation settings. Stat Med 2016; 35(23): 4136–4152.

15.

Raghupathi

James

Pace

, et al. Creating an integrated multiple sclerosis clinical trial analysis database. Neurology 2015; 84(14 Suppl.): P3.211.

16.

Debray

Riley

Rovers

, et al. Individual participant data (IPD) meta-analyses of diagnostic and prognostic modeling studies: Guidance on their use. PLoS Med 2015; 12(10): e1001886.

17.

Debray

Vergouwe

Koffijberg

, et al. A new framework to enhance the interpretation of external validation studies of clinical prediction models. J Clin Epidemiol 2015; 68(3): 279–289.

18.

Hastie

Tibshirani

Friedman

JH.

The elements of statistical learning: Data mining, inference, and prediction. New York: Springer, 2009.

19.

Strobl

Boulesteix

Kneib

, et al. Conditional variable importance for random forests. BMC Bioinformatics 2008; 9: 307.

20.

Pellegrini

Xiao

Castrillo

, et al. Baseline cognitive function is predictive of clinical disability progression in relapsing-remitting multiple sclerosis: A statistical approach. J Neurol Sci 2015; 357(Suppl. 1): e315.

21.

Steyerberg

EW.

Clinical prediction models: A practical approach to development, validation, and updating. New York: Springer, 2009.

22.

Uno

Cai

Pencina

, et al. On the C-statistics for evaluating overall adequacy of risk prediction procedures with censored survival data. Stat Med 2011; 30(10): 1105–1117.

23.

Marshall

Altman

Holder

, et al. Combining estimates of interest in prognostic modelling studies after multiple imputation: Current practice and guidelines. BMC Med Res Methodol 2009; 9: 57.

24.

Zeileis

Hothorn

Hornik

Model-based recursive partitioning. J Comput Graph Stat 2008; 17(2): 492–514.

25.

Sormani

De Stefano

Defining and scoring response to IFN-β in multiple sclerosis. Nat Rev Neurol 2013; 9(9): 504–512.

26.

Nordon

Karcher

Groenwold

RHH

, et al. The “efficacy-effectiveness gap”: Historical background and current conceptualization. Value Health 2016; 19(1): 75–81.

27.

Van Klaveren

Steyerberg

Serruys

, et al. The proposed “concordance-statistic for benefit” provided a useful metric when modeling heterogeneous treatment effects. J Clin Epidemiol 2018; 94: 59–68.

28.

Bovis

Signori

Carmisciano

, et al. Expanded disability status scale progression assessment heterogeneity in multiple sclerosis according to geographical areas. Ann Neurol 2018; 84(4): 621–625.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.68 MB