Abstract
Background. Cognitive impairments are common in people with multiple sclerosis (MS). Systematic reviews reported promising evidence for various cognitive interventions in this population. Computerized cognitive training (CCT) has strong evidence for safety and efficacy in several populations, but its effects in MS have yet to be specified. Objective. We aimed to synthesize the evidence from randomized controlled trials (RCTs) investigating the effects of CCT on cognitive, psychosocial, and functional outcomes in adults with MS. Method. We searched MEDLINE, EMBASE, PsycINFO, CINAHL, and CENTRAL from inception to March 2019. We calculated standardized mean difference (Hedges’ g) of change from baseline in untrained measures of cognition, individual domains, psychosocial functioning, and daily function between CCT and control groups using a random-effects model. Results. A total of 20 RCTs encompassing 982 participants (78% with relapsing-remitting MS) were included. The overall cognitive effect size was moderate (g = 0.30; 95% CI = 0.18-0.43), with no evidence of small-study effect or between-study heterogeneity (prediction interval = 0.17-0.44). Small to moderate effect sizes were found for attention/processing speed, executive functions, and verbal and visuospatial memory. Evidence for working memory, fatigue, and psychosocial and daily functioning were inconclusive. Cognitive effects waned without further training. Conclusions. CCT is efficacious for overall and key cognitive domains in adults with MS, but efficacy on other outcomes and in progressive subtypes remains unclear. Long-term and well-powered trials with diverse cohorts are needed to optimize and maintain the efficacy of CCT, investigate transfer to daily living, and determine who can benefit and whether CCT is a cost-effective strategy to attenuate cognitive decline in MS.
Introduction
Cognitive impairment is a common feature affecting 40% to 70% of multiple sclerosis (MS) patients. 1 The most prevalent impairments appear to be in the domains of memory, attention, and processing speed,1-3 but prevalence and severity estimates vary across populations, neuropsychological measures, and the underlying cognitive domains. 4 In general, however, cognitive impairments are more prevalent and severe in primary-progressive MS (PPMS) and secondary-progressive MS (SPMS) compared with relapsing-remitting MS (RRMS), 5 and risk of cognitive impairment is arguably related to the burden of neuro-axonal damage and cognitive reserve. 4 Overall, cognitive impairments affect daily functioning, quality of life, fatigue, and mood in patients with MS.2,6
Developing interventions to effectively target cognitive impairment in MS is a priority in the field. 7 Recent systematic reviews reported mixed and generally low-quality evidence for pharmacotherapy, 8 physical exercise,9,10 and complementary medicine. 11 Systematic reviews of cognitive interventions reached similar conclusions but recommended further research using several approaches that show promising evidence for efficacy.12-17 Computerized cognitive training (CCT) is one particular form of cognitive intervention, which differs from other approaches by focusing on repeated practice on controlled learning events over structured sessions, targeting specific cognitive processes rather than explicit learning. 18 CCT is typically based on game-like computerized exercises, adaptive to individual performance, and able to target multiple cognitive domains, and it can be administered inexpensively as a standalone intervention or as part of more complex rehabilitation programs. It was shown to be efficacious on cognition and behavior in a range of clinical populations, but effects vary across populations, cognitive domains, and specific intervention design factors.19-22
A recent systematic review and meta-analysis reported a small effect size in favor of CCT on memory. 17 However, this review was small (9 studies) and was not limited to randomized controlled trials (RCTs). There is, therefore, a dearth of systematic evidence for interventions that can enhance or maintain cognition in people with MS, and the quality of evidence is unclear. 7 Given its inherent safety and preliminary evidence for efficacy,12,17 we aimed to synthesize the randomized evidence for the effects of narrowly defined CCT on cognitive, functional, and psychosocial outcomes in people with MS.
Methods
Study Design and Registration
This systematic review and meta-analysis adheres to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA), 23 was prospectively registered with PROSPERO (CRD42016036565), and follows our published methods for meta-analyses of CCT trials.19-21 Key differences between the registered protocol and the final version are separation of fatigue outcomes from the functional category, assessment of participant and assessor blinding, and omission of subgroup analyses. The first 2 changes were carried out following reviewer suggestions, whereas the latter was omitted because of lack of analyzable between-study heterogeneity in the final data set. The PRISMA checklist is provided in Supplemental Table 1.
Eligibility Criteria
Studies and Participants
We included RCTs investigating the effects of CCT on one or more eligible cognitive, functional, psychosocial, or neuroimaging end points in adult patients with MS (of any etiology). Multiple reports from the same trial were combined into a single unit of analysis.
Interventions and Comparisons
At least 4 hours of drill-and-practice was given on computerized tasks with a clear cognitive rationale aiming to improve cognition. A minimum intervention time of 4 hours was set to reduce the likely bias in studies of acute interventions. 24 Studies that combined CCT with other interventions were included if at least 50% of training time was CCT or the adjacent intervention was equivalent between the experiment and control groups. Studies were excluded if the computerized intervention targeted primarily motor functions (eg, Wii Balance Board) or did not involve interaction with a computer (eg, passive observation of stimuli on a screen). Eligible comparisons included either passive (wait-list or no contact) or active (eg, psychoeducation, low-level CCT) control conditions.
Outcomes
Change from baseline to posttraining follow-up in one or more of nontrained measures of cognition (attention/processing speed, executive functions, verbal memory, visuospatial memory, working memory, visuospatial skills), functional measures (disability, instrumental activities of daily living), fatigue, psychosocial functioning (mood, anxiety, quality of life, self-efficacy, subjective cognitive performance), total brain volume, and T2 lesion load were determined. A full list of eligible outcome measures per domain is provided in Supplemental Table 3.
Search Strategy and Study Selection
We searched MEDLINE, EMBASE, PsycINFO, Cumulative Index to Nursing and Allied Health Literature (CINAHL), and Cochrane Central Register of Controlled Trials (CENTRAL) from inception to August 1, 2017. A MEDLINE update was conducted on March 9, 2019. The search strategy included keywords related to cognitive training and MS as a keyword or term (see Supplemental Table 2 for the full search strategy). We did not apply any search filters, and non-English articles were translated. In addition, we manually searched for articles in the reference lists of previous reviews and trial reports. Two independent reviewers screened search results based on title and abstract. Assessment of full-text articles was once more done by 2 independent reviewers (NTMH and IHKL). When eligibility was unclear or reports seemed to overlap, we consulted the authors of the original reports. Finally, a senior reviewer (AL) determined the final list of included studies.
Data Extraction and Coding
Two independent reviewers (NTMH and IHKL) extracted all relevant data from published reports and determined effect direction. Missing or incomplete outcome data were requested from authors of original reports. A neuropsychologist (JH) coded measures into outcomes (cognitive or functional domains) based on commonly used categorization conventions 25 or by consulting with another reviewer (AL). The final categorization of measures into outcome domains is provided in Supplemental Table 3. Data were extracted in the form of means and SDs for each group and outcome measure at baseline and follow-up, with the exception of 3 trials that reported means and SDs of change from baseline.26-28 In addition, data from 1 trial 29 were extracted from figures using WebPlotDigitizer V4.1. Data from multiple reports of the same study were combined into a single study after consulting with the authors. The complete data set is available from the corresponding author.
Risk of Bias in Individual Studies
Two independent reviewers (NTMH and IHKL) performed individual study appraisal using the Cochrane Risk of Bias tool for RCTs. 30 Studies with high or unclear risk of bias for assessor blinding or incomplete outcome data (ie, possible reporting bias) were coded as high risk of bias. In addition, we used the Physiotherapy Evidence Database Rating Scale (PEDro-P) 31 to assess methodological and reporting quality. PEDro scores range between 0 and 11, with higher scores representing better quality. Of note, the maximal score in this review was 9 because 2 PEDro items—blinding of therapists and patients—are difficult to establish in assessments of CCT trials.32,33 A senior reviewer (AL) resolved disagreements between assessors and established consensus scores.
Statistical Analysis
We calculated effect sizes as standardized mean difference (Hedges’ g) of change in each measure from baseline to follow-up between the CCT and control arm in each study. Positive effect sizes represent greater benefit in CCT over control, regardless of the direction of the original scale. Precision of effect sizes was assessed using 95% CIs.
Because CCT trials often report more than one measure per outcome, we applied a 3-tier procedure consistent with our previous meta-analyses in the field.19-21 First, all measures per outcome and study were combined into a single effect size and 95% CI. Second, we performed category-level meta-analyses of cognitive, psychosocial, and cognitive outcomes across studies to examine overall effects of CCT. Third, we performed a series of outcome-specific meta-analyses for each of the 5 cognitive domains. When studies reported more than 1 measure for analysis, all relevant outcomes were combined to produce a single effect size per study. Pooling of outcomes was done using the random-effects model. We considered Hedges’ g estimates of <0.30, <0.60, and ≥0.60 as small, moderate, and large effect sizes, respectively.
Between-study heterogeneity was estimated using τ2, which quantifies the variance of true effects across studies. The proportion of variation in true effects of total observed variance was assessed using the I2 statistic. 34 By convention, low values of I2 indicate that most of the observed variability is a result of random error, whereas increasing values of I2 indicate a higher proportion of true heterogeneity between studies. To account for between-study heterogeneity in the main analyses, we calculated a prediction interval, which estimates the possible range of treatment effect in at least 95% of individual settings. 35 Planned investigations of between-study heterogeneity were not performed because true heterogeneity was not found (ie, τ2 < 0.001 and, thus, the I2 = 0) in the analyses of overall cognitive, functional, and psychosocial outcomes.
Bias across studies because of the small-study effect was investigated by visually inspecting funnel plots of effect size versus standard error. 36 When at least 10 studies were available for analysis, we used Egger’s Test of the Intercepts 37 to formally test for small-study effect. If statistically significant asymmetry was found (1-sided P < .1), we used Duval and Tweedie’s Trim and Fill method 38 to quantify the magnitude of bias. If fewer than 10 studies were available, we investigated bias across studies by locating outliers in the funnel plot and recalculating effect sizes after their removal. All analyses were performed using Comprehensive Meta-Analysis version 3.
Results
Search Results
Our systematic search yielded 2642 entries across all databases, in addition to 2 studies obtained from manual searches. After removal of duplicates, we screened 2642 entries based on title and abstract and assessed 284 full-text articles for eligibility; 20 RCTs were eligible for this review (Figure 1).
Flowchart of study selection process.
Characteristics of Included Studies
Overall, the 20 RCTs included 982 patients (CCT: n = 523, mean group size n = 26; control: n = 459, mean group size n = 23; Table 1). Mean age ranged between 33 and 59 years; 77.9% of participants were female; 78.7% had a diagnosis of RRMS at baseline (ie, at commencement of CCT); 13.6% had SPMS; and 7.1% had PPMS. The weighted mean Expanded Disability Status Scale score across studies was 3.4 (median = 3.5). In all, 7 studies were from Italy,28,29,39-43 7 from the United States,27,44-49 2 from Austria,50,51 and the others from the United Kingdom, 26 Switzerland, 52 Greece, 53 and Spain. 54 Among the studies, 13 used multidomain training, 4 provided attention training, and active control was confirmed in 11/20 studies. The mean PEDro-P score was 7.8/11 (SD = 1.3), and 15/20 studies had a high risk of bias (Supplemental Table 4).
Characteristics of Included Studies.
Abbreviations: CCT, computerized cognitive training; CP, chronic progressive; CTL, control; EDSS, Expanded Disability Status Scale; ITT, intention-to-treat analysis; MS, multiple sclerosis; PP, primary progressive; RR, relapsing-remitting; SP, secondary progressive.
Group size at posttraining analysis.
At baseline.
Studies were considered as high risk of bias if they had high risk for assessor blinding or incomplete outcome data (ie, did not perform intention-to-treat analyses).
The maximum score is 9.
Median.
Efficacy of CCT on Cognitive Outcome
Overall Efficacy on Cognitive Outcomes
The overall effect size across 20 RCTs was moderate and statistically significant (g = 0.30, 95% CI = 0.18-0.43, P < .001, τ2 < 0.001, I2 = 0%; Figure 2). The prediction interval did not find evidence for inefficacy in individual studies (0.17-0.44). The funnel plot did not reveal significant asymmetry (Egger’s intercept = 0.656, 1-sided P = .10; Supplemental Figure 1). The effect size across active-controlled trials (k = 11; g = 0.27; 95% CI = 0.12-0.42; I2 = 0%) was lower but not significantly different from that of trials with passive control groups (k = 9; g = 0.36; 95% CI = 0.15-0.58; I2 = 0%; Q-statistic for between-subgroup heterogeneity = 0.460; df = 1; P = .50). Similarly, pooled effect sizes were comparable across studies with high (k = 15; g = 0.30; 95% CI = 0.15-0.44; I2 = 0%) or low risk of bias (k = 5; g = 0.32; 95% CI = 0.07-0.57; I2 = 0%; Q = 0.031; df = 1; P = .86).
Meta-analyses of overall cognitive outcomes.
Attention/Processing Speed
The pooled effect size across attention and processing speed outcomes was moderate and statistically significant (k = 20, g = 0.32, 95% CI = 0.19-0.44, P < .001, τ2 = 0, I2 = 0%; Figure 3). The funnel plot did not reveal significant asymmetry (Egger’s intercept = 0.736, P = .13; Supplemental Figure 1).
Meta-analysis of processing speeds and executive function outcomes.
Executive Functions
The pooled effect size across executive function outcomes (fluency, set shifting, inhibition, and reasoning) was small and statistically significant (k = 14, g = 0.29, 95% CI = 0.14-0.44, P < .001, τ2 = 0, I2 = 0%; Figure 3). The funnel plot did not reveal significant asymmetry (Egger’s intercept = 0.851, P = .16; Supplemental Figure 1).
Verbal Memory
The pooled effect size across verbal learning and memory outcomes was small and statistically significant (k = 17, g = 0.28, 95% CI = 0.14-0.42, P < .001, τ2 = 0.008, I2 = 8.5%; Figure 3). The funnel plot did not reveal significant asymmetry (Egger’s intercept = 0.649, P = .21; Supplemental Figure 1).
Visuospatial Memory
The pooled effect size across nonverbal learning and memory outcomes was small and statistically significant (k = 15, g = 0.21, 95% CI = 0.04-0.38, P = .02, τ2 = 0.03, I2 = 27.4%; Figure 4). The funnel plot did not reveal significant asymmetry (Egger’s intercept = 0.961, P = .15; Supplemental Figure 1).
Meta-analyses of verbal, visuospatial, and working memory outcomes.
Working memory was assessed in only 4 studies.27,44,45,52 The pooled effect size was small and statistically nonsignificant (g = 0.21, 95% CI = −0.07 to 0.49, P = .13, τ2 = 0, I2 = 0%; Figure 4). The funnel plot did not reveal significant asymmetry (Supplemental Figure 1), but this was not formally tested, because of the small number of studies.
Efficacy of CCT on Other Outcomes
Psychosocial Functioning
The pooled effect size across 9 RCTs was small and statistically nonsignificant (g = 0.18, 95% CI = −0.01 to 0.37, P = .06, τ2 = 0, I2 = 0%; Figure 5). The prediction interval indicated that CCT may be inefficacious for psychosocial functioning in some settings (−0.04 to 0.40). We detected evidence of unusual funnel plot asymmetry, whereby larger studies reported larger effect sizes (Supplemental Figure 1), driven by the smallest study in the analysis (n = 10) that reported an effect size of g =−0.82 (ie, in favor of control) on the Beck’s Depression Inventory. 52 A sensitivity analysis after removal of this study revealed a small but statistically significant effect size (g = 0.20, 95% CI = 0.02-0.39, τ2 = 0, I2 = 0%; Figure 5), but the prediction interval did not exclude negative findings in some settings (−0.21 to 0.25). None of the subcategories of psychosocial function revealed statistically significant effects (Supplemental Table 4).
Meta-analyses of psychosoci, visuospatial, and working memory outcomes.
Functional Outcomes
The pooled effect size across 3 RCTs was small and statistically nonsignificant (g = 0.13, 95% CI = −0.14 to 0.39, P = .35, τ2 = 0, I2 = 0%; Figure 5). The prediction interval corroborated possible negative findings in some settings (−1.58 to 1.84). The funnel plot did not reveal potential asymmetry (Supplemental Figure 1), but asymmetry was not formally tested because of the small number of studies. None of the subcategories of function revealed statistically significant effects (Supplemental Table 4).
Fatigue
The effect size across 6 RCTs was small and statistically nonsignificant (k = 6; g = 0.20; 95% CI = −0.09 to 0.49; P = .17; τ2 = 0; I2 = 0%). The prediction interval corroborated possible negative findings in some settings (−0.21 to 0.62). The funnel plot did not reveal potential asymmetry, but asymmetry was not formally tested because of the small number of studies.
Long-term Effects
Pooling of outcomes from longitudinal follow-ups revealed small and statistically nonsignificant effect sizes on overall cognitive (k = 7; g = 0.16; 95% CI = −0.02 to 0.40; I2 = 0%), psychosocial (k = 5; g = 0.11; 95% CI = −0.11 to 0.33; I2 = 0%), fatigue (k = 2; g = −0.05; 95% CI = −0.57 to 0.47; I2 = 0%), and functional outcomes (k = 2; g = 0.11; 95% CI = −0.17 to 0.39; I2 = 0%).
Planned analyses of visuospatial skills, disability, total brain volume, and T2 lesion load were not performed because of insufficient number of studies available for analysis (all k < 3).
Discussion
The evidence base for CCT in MS has grown rapidly, with 13 studies published only in the past 4 years. Based on data from 20 RCTs of moderate quality, we report that CCT is efficacious for objective cognitive performance in people with MS. Effect sizes on overall cognitive performance and specific cognitive domains were small to moderate and do not seem to be materially biased by heterogeneity or small-study effect (publication bias). Moderate effect sizes on attention/processing speed and memory are particularly encouraging because these were reported to be the key area of cognitive impairment in MS.1-3 The overall cognitive effect did not depend on study quality and type of control.
Our analysis is a major update to the state of the evidence in the field, which has been so far inconclusive7,17 and the first to focus specifically on randomized trials of CCT. Whereas posttraining effects are encouraging, loss of cognitive gains after training cessation is a known limitation of CCT 21 and implies a need to design booster training in order to maintain CCT effects. 55 Overall, these results provide a strong rationale for large RCTs in the field aiming to optimize CCT for clinical practice, investigate its mechanisms of effect, and examine whether it can be used as a long-term secondary prevention strategy for cognitive decline across the spectrum of MS disease phenotypes. Equally important is continued efforts to improve clinical trial standards in the field, particularly in relation to blinding, randomization, concealed allocation, transparency, preregistration, and adherence to intention-to-treat using robust adjustments for missing data.
Conversely, we are still not able to determine whether CCT effects on psychosocial functions, which have been demonstrated in other populations,21,22 can be expected in MS. Pooled analyses of subjective cognition and quality-of-life outcomes suggest that such effects are possible, but these were clearly underpowered. Analyses of depression and anxiety outcomes revealed substantial heterogeneity and no clear efficacy signal. Similarly, we did not find evidence that CCT programs generalize to fatigue or other functional outcomes. Given findings from recent systematic reviews supporting the efficacy of physical exercise on fatigue 56 and cognitive behavioral therapy on depression, anxiety, and quality of life, 57 intervention plans that combine CCT with exercise and psychosocial approaches might prove a low risk and effective rehabilitation strategy. Similarly, combining CCT with other promising approaches to cognitive rehabilitation12-17 might reveal larger cognitive and behavioral effects than those found for CCT alone. Further development of ecologically valid outcome measures for this population might help assess whether cognitive enhancement translates into functional improvement, especially in the long run. 16
Because of insufficient between-study heterogeneity in our main analyses, we could not investigate the moderating effects of methodological differences across studies. This was partially a result of the relatively small sample sizes in included studies; a median sample size of n =34 meant that effect sizes of individual studies were often imprecise and overlapping, thus obstructing potential heterogeneity in real effects between studies. Moreover, combining the results into single effect sizes did not distinguish between primary and secondary outcome measures within studies. Our results provide the rationale for designing well-powered studies that compare CCT approach or dose in a head-to-head fashion instead of randomizing participants to inert control conditions, using sensitive outcome measures in MS. 7
Conclusions/Implications
CCT improves overall cognition as well as untrained outcome measures of attention/processing speed, executive functions, and memory immediately after training in MS. Efficacy data for working memory and psychosocial functioning are inconclusive, and there is currently no evidence that the benefits of CCT can be generalized to fatigue or everyday function. The bulk of participants in published clinical trials are patients with RRMS. Future studies should include larger sample sizes and patients with progressive forms of MS and investigate long-term interventions, optimal CCT design, and combination with other interventions.
Supplemental Material
Supplemental_Material_CLEAN – Supplemental material for Computerized Cognitive Training in Multiple Sclerosis: A Systematic Review and Meta-analysis
Supplemental material, Supplemental_Material_CLEAN for Computerized Cognitive Training in Multiple Sclerosis: A Systematic Review and Meta-analysis by Amit Lampit, Josephine Heine, Carsten Finke, Michael H. Barnett, Michael Valenzuela, Anna Wolf, Isabella H. K. Leung and Nicole T. M. Hill in Neurorehabilitation and Neural Repair
Footnotes
Acknowledgements
We thank the authors of primary studies for providing data and answering our enquiries.
Supplementary material for this article is available on the Neurorehabilitation & Neural Repair website along with the online version of the article.
Authors’ Note
Nicole T. M. Hill is also affiliated with The National Centre of Excellence in Youth Mental Health, University of Melbourne, Parkville, Victoria, Australia.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: AL and MV are funded by research fellowships from the National Health and Medical Research Council of Australia (NHMRC Grant IDs 1108520 and 1112813). IHKL was funded by a HHMRC scholarship (ID 1133682). NTMH is funded by Australian Rotary Health scholarship. CF and MV receive research support in the form of cognitive training software free of charge from Synaptikon (Berlin) for projects unrelated to this work. CF is a lead investigator and AL is partially funded by a grant from German Federal Ministry of Education and Research (BMBF, ID 13GW0212A) to develop a cognitive assessment and intervention platform in collaboration with Synaptikon.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
