Novel computer-based testing shows multi-domain cognitive dysfunction in patients with multiple sclerosis

Abstract

Background

Although cognitive dysfunction is a leading cause of disability and poor quality of life in patients with multiple sclerosis (MS), it is infrequently tested in routine clinical evaluation. Development of a cognitive testing paradigm that captured MS-related cognitive dysfunction and could be obtained in a routine clinical setting may increase surveillance and recognition of cognitive dysfunction.

Objectives

This was a pilot study to determine if Cognivue could find cognitive performance differences between patients with MS and healthy controls (HC).

Methods: A total of 24 patients with MS and 12 HCs between 18 and 50 years old were enrolled. Baseline testing included an Expanded Disability Scale (EDSS), paced auditory serial additions test (PASAT), symbol digit modalities test (SDMT) and Cognivue. Subjects then had repeat testing every 1–2 months for a maximum of three tests.

Results

Significant differences were found between MS and HC on SDMT, PASAT, and Cognivue Total score. Most Cognivue subtests showed significant differences between MS and HC. Cognivue scores correlated with both SDMT and PASAT and had high test-retest reliability in HCs.

Conclusion

Cognivue was able to detect multi-domain cognitive dysfunction in MS. Further studies to determine validity of Cognivue in MS with comparison with neuropsychological testing and sensitivity to clinical change are still needed.

Keywords

Multiple sclerosis (MS)cognition cognitive testing computer-based cognitive testing

Introduction

Cognitive impairment is a leading cause of diminished quality of life in patients with multiple sclerosis (MS). It is associated with loss of employment and inability to perform activities of daily living (ADL) and instrumental activities of daily living (IADLs).^1–5 Despite the substantial impact of cognitive impairment on quality of life, routine clinical assessment of cognition is rarely performed.⁶ Identification of cognitive dysfunction with testing that could be implemented in the routine clinical setting, thereby increasing cognition dysfunction surveillance, may marshal social supports, guide cognitive rehabilitation, target drug therapies, and serve as a biomarker for evaluation of disease modifying therapies.

MS-related cognitive dysfunction is heterogeneous. Cognitive impairment is seen early in the disease course prior to a clinical relapse in a disease status known as radiologically isolated syndrome (RIS). In RIS, patients have been found to have common deficits in information processing speed and verbal memory, in common with patients with clinically definite relapsing MS.⁷ A study by Rao et al. in 1991 in 100 patients with MS showed cognitive deficits in recent memory, abstract reasoning, attention/concentration, and visuospatial skills.⁸ A study by Benedict et al. further supported the heterogeneity of cognitive dysfunction in validating the Minimal Assessment of Cognitive Function in MS (MACFIMS) by showing impairments in verbal fluency, visuospatial perception, verbal memory, visual memory, working memory, processing speed and concept formulation.⁹ Cognitive dysfunction is related to both patient demographics, education, as well as disease-related factors including disease duration, disease type, disease-related disability and disease burden (lesion volume, brain atrophy, etc.).^7,10–16 Therefore, evidence of cognitive impairment is seen early in the disease course and worsens with clinical and pathological features of the disease.

Formal neuropsychological testing (NPT) in patients with MS shows a broad spectrum of cognitive dysfunction.^17,18 However, NPT may take several hours to perform and is infrequently used in the clinical setting. Instead, the Symbol Digit Modalities Test (SDMT) and Paced Auditory Serial Addition Test (PASAT) have been developed for use as tools for evaluation of cognitive dysfunction for use in clinics and clinical trials.^19–21 Both the SDMT and PASAT focus on the known common impairment of processing speed and take fewer than 5 min to administer. While the SDMT and PASAT evaluate other cognitive domains in addition to processing speed, the single outcome score does not allow for specific domain assessment; thus, they may not capture the heterogeneity of MS-related cognitive dysfunction.

The ideal cognitive test would be easy to use, of brief duration, and control for motor and visual impairment. It would be desirable for such tests to be autonomously administered, test multiple domains, and provide performance scores that facilitate tracking of cognitive dysfunction in MS. In 2015, the Food and Drug Administration (FDA) gave Cognivue, a computer-based cognitive test of perception and memory, de novo approval for the assessment of cognitive dysfunction in individuals over the age of 55 years. Cognivue has many features of the ideal cognitive screening test for patients with MS, including autonomous administration, short duration (10 min), calibration for dominant hand and vision impairment and perception abilities for memory testing. In this study, we explored the ability of Cognivue to find differences in cognition in patients with MS and healthy controls (HCs), and thus evaluate preliminary validity of Cognivue as a screening measure for cognition in MS in comparison to validated measures of the SDMT and PASAT.

Materials and methods

Cognivue

Cognivue is a computer-based cognitive test that received de novo FDA approval for the assessment of early signs of cognitive impairment in individuals over the age of 55 years in 2015 based on a validation study with the St. Louis University Mental Status (SLUMS) test (FDA De Novo Number 130033). Cognivue testing is based on findings of perceptual deficits in patients with Alzheimer’s disease, including letter, word, and motion perception.^22–24

The Cognivue system is a computer-on-wheels with rotatory mandipulum (joystick) for item selection. First, the patient engages in visuomotor testing for calibration of dominant hand motor and visual function (Figure 1). The first visuomotor test is the “Adaptive Motor” test. During the adaptive motor test, dots on the screen move in a circular pattern with random changes in direction, speed. The subject is cued to keep the dots selected by moving the rotatory mandipulum. During testing the sensitivity of rotatory mandipulum also changes requiring the subject to modifying the extent of mandipulum movement. This test is used to calibrate for dominant hand motor impairment. Next, the subject performs the “Visual Salience” test in which the patient again tracks the dots as they move in a circular pattern and change direction and also change contrast – starting with higher contrast and then becoming dim and then becoming bright (Figure 1). The overall time that the patient can keep the dots selected is reflected in a higher score on a scale of 0–100.

Figure 1. Cognivue testing. Samples of each Cognivue test.

Following visuomotor testing, perception and memory tests are calibrated such that patients who perform well on adaptive motor testing will be given less time to make selections and those who perform well on visual salience testing will have reduced visual presentation of words, letters, and symbols in both perception and memory testing (Figure 1). Next, the patient is tested in perception of letters, words, symbols, and direction of movement. In each task the subject is prompted to identify the correct letter, word, symbol, and direction of movement (Figure 1). Perception tasks then calibrate for memory testing in letter, word, symbol, and movement (Figure 2). In memory testing, the subject is prompted to remember the first item and shown several items in sequence dependent on task, e.g. letters, words, symbols, or direction of movement dependent on task. The subject is then presented several items, only one of which is the correct response.

Figure 2.

Cognivue testing algorithm. Sequential intra-testing calibration schematic for motor (dominant hand), visual disability, and perceptual testing prior to memory testing. MD/CAS Access: treating clinician and Cerebral Assessment Systems (CAS) access to Cognivue testing.

Lastly, a total score is calculated based on the mean of the perception and memory tests. Subtest scores are also calculated. All scores are percentage of correct responses on a scale of 0–100.

Subjects

We enrolled 24 patients with clinically-definite MS, by the 2010 revised McDonald criteria with relapsing phenotype, and 12 HCs s were enrolled. Patients with MS were recruited from the Rochester MS Center at the University of Rochester Medical Center, Rochester, NY. HCs were frequently spouses or acquaintances of the MS patients enrolled. Inclusion criteria for both groups were age between 18 and 50 years. Exclusion criteria for both groups included a history of cognitive impairment related to another cause, Hamilton Depression Rating Scale (HAM-D) score of > 20, and inability to complete the informed consent process. The HAM-D is a 17-item questionnaire to assess symptoms and severity of depression. The upper age limit of the study was used to reduce the influence cognitive changes that could be due to age-related neurodegenerative diseases. The study was approved by the University of Rochester Research Subjects Review Board. All subjects provided written informed consent. All MS patients were being treated with natalizumab to reduce the risk of clinical or MRI relapse and promote availability for monthly testing around the time of their infusions. Additional exclusion criteria for MS patients included a clinical or radiological relapse in the prior 3 months.

Testing

Subjects were scheduled for a total of three study visits scheduled at an interval of 1–2 months. At each study visit the subjects completed the Multiple Sclerosis Functional Composite (MSFC; timed 25-foot walk T25W, 9-hole peg test, and PASAT), SDMT, and Cognivue testing. At the first visit all subjects were evaluated and scored on the Expanded Disability Severity Scale (EDSS).

Statistical analysis

Due to the small number of subjects and concern for non-normal distributions, nonparametric analyses were performed for comparisons between subjects with MS and HCs. Mann–Whitney tests were used for comparisons of age, education (years), HAM-D, and EDSS. A Chi-squared test was used to compare gender proportions between groups. The first trials of MSFC and SDMT were compared between MS and HC again using Mann–Whitney tests. The first trial results of Cognivue total and subtask scores were compared using Mann–Whitney tests with conservative correction for multiple comparisons by multiplying the p value for each analysis by the total number of comparisons (p′ = p * # comparisons, while keeping alpha = 0.05.). A Bland–Altman plot, which plots the mean score of trial 1 and 2 (x-axis) and the difference between trials 1 and 2, was used to assess reliability. Finally, correlation analysis between first trials of Cognivue total score, SDMT, PASAT, and 9-hole peg test and Cognivue Adaptive Motor test were made with combined MS + HC data as well as MS only using nonparametric Spearman correlation analysis. The p value was adjusted for multiple comparisons (p′ = p * # comparisons), while keeping alpha = 0.05.

Results

Demographics

All subjects (MS, n = 24; HC, n = 12) completed a total of two study visits, 9 MS subjects and 5 HCs completed all three study visits. Demographic data, including age, sex, and level of education, were similar between HC and MS (Table 1). Depression severity, as measured by the HAM-D was not different between HC and MS (median interquartile range (IQR), MS = 3.5 IQR 1–7, HC = 4 IQR 2.75–5.75, p = 0.7). The mean disease duration in the MS patients was 9.5 years (SD 6.19).

Table 1.

Demographics and baseline characteristics.

	HC median (SD, IQR)	MS median (SD, IQR)	p value
Age (SD)	39 (8.19, 28.75–42.0)	35.5 (6.16, 32.75–38.25)	0.52
Gender – % women	50%	62.5%	0.47a
Education (SD)	16 (2.61, 13.5–17.25)	14 (2.59, 12–16)	0.23
Hamilton depression (SD)	4.33 (3.23, 2.75–5.75)	4.88 (5.63, 1.0–7.0)	0.7
EDSS (SD)	0 (0.89, 0–0)	2.25 (1.38, 1.5–3.0)	<0.00001

HC: Healthy control; IQR: interquartile range; MS: multiple sclerosis; EDSS: expanded disability scale.

^aChi square.

Baseline disability

MS had a median EDSS of 2.25 (SD 1.38, IQR 1.5–3.0, Table 1). The T25W was, as expected, higher in MS than HC (Table 2). Additional MSFC components of 9-hole peg test and PASAT and SDMT scores were also different between MS and HC (Table 2).

Table 2.

Multiple sclerosis functional composite and SDMT.

	Multiple sclerosis	Healthy controls
	Mean (SD, IQR)	Mean (SD, IQR)	p
Timed 25-foot walk (s)	4.62 (1.56, 3.75–5.02)	3.50 (0.60, 3.09–3.84)	.01
9-hole peg test (s)	26.47 (18.64, 19.71–24.60)	18.17 (2.22, 17.12–18.63)	.001
PASAT	38.75 (12.18, 31.00–50.25)	50.42 (6.44, 45.00–55.25)	.005
SDMT	45.67 (12.55, 39.75–50.50)	61.67 (8.32, 56.50–65.75)	.0005

IQR: interquartile range; PASAT: paced auditory serial additions test; SDMT: Symbol digit modalities test.

Cognivue testing

MS patients performed significantly worse on Cognivue testing than HCs (Figure 3). Total scores, reflecting average scores across perception/discrimination and memory testing, were 80.71 and 93.00 for MS and HC, respectively (p = 0.001). Both visuomotor calibration tests, adaptive motor and visual salience, were significantly different between MS and HC. Of the cognitive battery components, motion discrimination and letter memory tasks were significantly different between MS and HC (p < 0.05). The test with largest magnitude difference between MS and HC was motion discrimination (74.50 SD 13.61 4.22 vs 97.92 SD 4.95).

Figure 3.

Cognivue scores. Mean first Cognivue testing scores for both healthy controls (blue) and patients with multiple sclerosis (orange). Perception and memory tasks are identified with labelled brackets. Error bars = SD. Analysis with nonparametric Mann–Whitney tests with correction for multiple comparisons. p < 0.05. HC: Healthy controls; MS: multiple sclerosis.

Cognivue reliability

We evaluated intra-subject Cognivue reliability in subjects who had completed two or more study visits. Test-retest stability in both HC and MS subjects were analyzed with a Bland–Altman plot (Figure 4), which plots the mean Cognivue Total score vs the difference between the scores on Test 1 and Test 2 for each subject.²⁵ HCs had lower test-retest variability compared to MS.

Figure 4.

Bland–Altman plot. Total Cognivue scores from the first and second tests are plotted for healthy controls (blue) and patients with multiple sclerosis (orange). 95% confidence interval for all scores is noted by red lines. SE: standard error.

Convergent validity

Cognivue testing was compared to both SDMT and PASAT by calculating Spearman correlation coefficients (Figure 5, Table 3). There were significant positive correlations between Cognivue Total scores and both SDMT and PASAT scores for all subjects (MS + HC) and MS alone (Table 3). The Cognivue Total score was more strongly with SDMT than PASAT. Additionally, there was a significant positive correlation between Cognivue Adaptive Motor and the 9-hole Peg test, tests of dominant hand function (Table 3).

Figure 5.

Spearman correlation of cognitive testing. Correlation analysis of (a) SDMT and PASAT, (b) total Cognivue score and PASAT, and (c) total Cognivue score and SDMT for healthy controls (blue) and patients with multiple sclerosis (orange). R and p values are in Table 3. Correction for multiple comparisons. SDMT: symbol digit modalities test; PASAT: paced auditory serial additions test.

Table 3.

Spearman correlation analysis of cognition testing.

Combined MS + HC	r	p
SDMT–PASAT	0.63	0.0002
Cognivue–SDMT	0.82	1 × 10⁻⁸
Cognivue–PASAT	0.55	0.004
9-hole peg – Adaptive Motor	0.57	0.002
MS only	r	p
SDMT–PASAT	0.48	0.1
Cognivue–SDMT	0.79	<0.0001
Cognivue–PASAT	0.61	0.1
9-hole peg – Adaptive Motor	−0.15	3.9

MS: multiple sclerosis; HC: Healthy control; SDMT: Symbol digit modalities test; PASAT: paced auditory serial additions test.

We also compared MS patient’s scores on SDMT and Cognivue. A total of 12 MS subjects had a SDMT Z score of less than –1.0, and all 12 of these subjects had a Cognivue Total score that was greater than 1 SD below the HC mean. No HC subject had a SDMT Z score <1.0 SD or a Cognivue Total score < 1.0 SD below the HC mean.

Discussion

This is the first study to evaluate cognitive dysfunction in patients with MS using the computer-based testing system Cognivue. Based on our baseline data and demographics, MS subjects in this study reflect a population that would be typical of a MS population with stable, relapsing disease of significant disease duration, and moderate disease-related disability EDSS. Not unexpectedly, subjects with MS did significantly worse on all validated motor and cognition tests, which again confirms that these patients have signs of MS-related disability. With this established, we explored the cognitive phenotype of this sample population with respect to Cognivue testing.

Cognivue total scores were significantly worse in subjects with MS (Figure 3). Additionally, subcomponent tasks of visuomotor performance, motion discrimination and letter memory were significantly different between HC and MS. The fact that visuomotor tasks were significantly different between the groups highlights the importance of calibrating cognitive testing for dominant hand and visual impairment, which is something that Cognivue attempts to perform. The exact implication of motion discrimination and letter memory impairment in patients with MS is uncertain. However, a prior study has shown that motion perception is impaired with patients with optic neuritis following recovery of visual acuity.²⁶ We did not obtain clinical data with regard to history of optic neuritis as a part of this study. Letter memory testing may reflect previously reported verbal memory impairment in patients with MS, but it is unclear why Cognivue letter but not word memory is impaired.^7,9 This study was not designed to answer these questions, but to determine whether Cognivue has the potential to be a screening tool for cognitive dysfunction in patents with MS. Larger studies that include neuropsychological testing and brain imaging (quantitative MRI analyses) are needed to further explore links between Cognivue, cognitive impairment, and MS-related pathology.

The primary limitations of this study are the small sample size and study completion failure (14 of 24 subjects completed all three study visits). However, even with that small sample size, significant differences between HC and MS were seen. A major strength of this study was our matched age, sex, and education matched HC group. Findings of correlation between SDMT and PASAT and differences between MS and HC groups in these validated tests further strengthen this study.

Evaluation of test–retest variability showed greater variability in MS patients compared to HCs. This finding may reflect MS patients’ greater variability across a wide range of functional assessments.^27–30 Subjects with MS tended to have improved scores on the second test, which may reflect a learning or practice effect in combination with a ceiling effect seen in HC. Though not yet reported, similar practice/learning effects with Cognivue have been seen in individuals over the age of 55 years.

Further evaluation of Cognivue in MS patients is needed. First, obtaining a large Cognivue dataset in HC will allow for creation of total Cognivue and Cognivue subtest score normative quantiles based on age and education, similar to the SDMT. This will allow clinicians to consider both the total score and the subtest scores with respect to established normative data in this population. Second, Cognivue may provide a useful clinical biomarker of disease progression. In this regard, it will be important to understand the sources of variability in performance in routine testing to optimize sensitivity to changes in individual patient’s cognition. Morrow et al. showed that the SDMT detected a worsening in cognitive function during a MS relapse with near recovery to baseline 2 months later.³¹ We will attempt a similar study using Cognivue to determine if it is sensitive to acute cognition changes seen during MS relapses. Additionally, a longitudinal study is needed to evaluate for progressive cognitive changes in patients with relapsing and progressive forms of MS.

In conclusion, this was first study of cognition in patients with MS using Cognivue, an FDA approved computer-based cognitive test. We were able to detect significant differences in cognition between age, gender and education-matched MS and HC subjects. Cognivue may present a practical alternative to traditional cognitive screening tests. Strengths of Cognivue testing include its automated administration, short duration, multiple domain assessments, and adaptive approach. These technical features and results from this pilot study suggest that Cognivue may capture MS-related cognitive dysfunction and could be implemented in the clinical setting. This preliminary study lays the groundwork for future studies in Cognivue reliability, validity, and sensitivity to disease-related change in MS.

Supplemental Material

Supplemental material for Novel computer-based testing shows multi-domain cognitive dysfunction in patients with multiple sclerosis

Supplemental material for Novel computer-based testing shows multi-domain cognitive dysfunction in patients with multiple sclerosis by Andrew D Smith III, Charles Duffy and Andrew D Goodman in Multiple Sclerosis Journal – Experimental, Translational and Clinical

Footnotes

Conflicts of interest

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: ADS received fellowship training support from the National MS Society Sylvia Lawry Physician Grant and personal compensation for consulting from Sanofi Genzyme. CD is the CEO and founder of Cognivue, Inc., formerly Cerebral Assessment Systems, manufacturer of Cognivue. ADG received personal compensation for consulting from the following commercial entities: Abbvie, Acorda Therapeutics, Adamas, Sanofi Genzyme, Mylan, Novartis. ADG's employer, the University of Rochester, received research support for conducting clinical trials from the following commercial entities: Acorda Therapeutics, Avanir, Biogen-Idec, Novartis, Genetech Roche, Sanofi Genzyme, Sun Pharma, Teva.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Rao

Leo

Ellington

et al . Cognitive dysfunction in multiple sclerosis. II. Impact on employment and social functioning. Neurology 1991; 41: 692–696.

Amato

Ponziani

Pracucci

et al . Cognitive impairment in early-onset multiple sclerosis: Pattern, predictors, and impact on everyday life in a 4-year follow-up. Arch Neurol 1995; 52: 168–172.

Benedict

RHB

Wahlig

Bakshi

et al . Predicting quality of life in multiple sclerosis: Accounting for physical disability, fatigue, cognition, mood disorder, personality, and behavior change. J Neurol Sci 2005; 231: 29–34.

Cadden

and Arnett

Factors associated with employment status in individuals with multiple sclerosis.

Int J MS Care 2015; 17: 284–291.

Cutajar

Ferriani

Scandellari

et al.

Cognitive function and quality of life in multiple sclerosis patients. J Neurovirol 2000; 6(Suppl 2): 186–190.

Foley

Benedict

RHB

Gromisch

et al . The need for screening, assessment, and treatment for cognitive dysfunction in multiple sclerosis: Results of a multidisciplinary CMSC consensus conference, September 24, 2010. Int J MS Care 2012; 14: 58–64.

Benedict

RHB

Cookfair

Gavett

et al . Validity of the minimal assessment of cognitive function in multiple sclerosis (MACFIMS). J Int Neuropsychol Soc 2006; 12: 549–558

Amato

Hakiki

Goretti

et al . Association of MRI metrics and cognitive impairment in radiologically isolated syndromes. Neurology 2012; 78: 309–314.

Rao

Leo

Bernardin

et al . Cognitive dysfunction in multiple sclerosis. I. Frequency, patterns, and prediction. Neurology 1991; 41: 685–691

10.

Beatty

Goodkin

Hertsgaard

et al . Clinical and demographic predictors of cognitive performance in multiple sclerosis: Do diagnostic type, disease duration, and disability matter? Arch Neurol 1990; 47: 305–308.

11.

Papadopoulou

Müller-Lenke

Naegelin

et al . Contribution of cortical and white matter lesions to cognitive impairment in multiple sclerosis. Mult Scler 2013; 19: 1290–1296.

12.

Harrison

Roy

et al . Association of cortical lesion burden on 7-T magnetic resonance imaging with cognition and disability in multiple sclerosis. JAMA Neurol 2015; 72: 1004–1012.

13.

Calabrese

Agosta

Rinaldi

et al . Cortical lesions and atrophy associated with cognitive impairment in relapsing–remitting multiple sclerosis. Arch Neurol 2009; 66:1144–1150.

14.

Sacco

Bisecco

Corbo

et al . Cognitive impairment and memory disorders in relapsing–remitting multiple sclerosis: The role of white matter, gray matter and hippocampus. J Neurol 2015; 262: 1691–1697

15.

Huijbregts

SCJ

Kalkers

de Sonneville

LMJ

et al . Differences in cognitive impairment of relapsing remitting, secondary, and primary progressive MS. Neurology 2004; 63: 335–339.

16.

Roosendaal

Moraal

Pouwels

et al . Accumulation of cortical lesions in MS: Relation with cognitive impairment. Mult Scler J 2009; 15: 708–714.

17.

Rao

Leo

Bernardin

et al . Cognitive dysfunction in multiple sclerosis. I. Frequency, patterns, and prediction. Neurology 1991; 41: 685–691

18.

Wishart

Sharpe

Neuropsychological aspects of multiple sclerosis: A quantitative review.

J Clin Exp Neuropsychol 1997; 19: 810–824.

19.

Parmenter

Weinstock-Guttman

Garg

et al . Screening for cognitive impairment in multiple sclerosis using the symbol digit modalities test. Mult Scler J 2007; 13: 52–57.

20.

Cutter

Baier

Rudick

et al . Development of a multiple sclerosis functional composite as a clinical trial outcome measure. Brain 1999; 122: 871–882.

21.

Drake

Weinstock-Guttman

Morrow

et al . Psychometrics and normative data for the multiple sclerosis functional composite: Replacing the PASAT with the symbol digit modalities test. Mult Scler J 2010; 16: 228–237.

22.

Velarde

Perelstein

Ressmann

et al . Independent deficits of visual word and motion processing in aging and early Alzheimer's disease. J Alzheimers Dis 2012 31:613–621.

23.

Kavcic

Vaughn

and Duffy

CJ.

Distinct visual motion processing impairments in aging and Alzheimer's disease.

Vision Res 2011 Feb 9; 51: 386–395.

24.

Fernandez

Monacelli

and Duffy

CJ.

Visual motion event related potentials distinguish aging and Alzheimer's disease.

J Alzheimers Dis 2013; 36: 177–183.

25.

Bland

and Altman

DG.

Statistical methods for assessing agreement between two methods of clinical measurement.

Lancet 1986; 327: 307–310.

26.

Raz

Dotan

Benoliel

et al . Sustained motion perception deficit following optic neuritis: Behavioral and cortical evidence. Neurology 2011; 76: 2103–2111

27.

Leocani

Comi

Annovazzi

et al . Impaired short-term motor learning in multiple sclerosis: Evidence from virtual reality. Neurorehabil Neural Repair 2007; 21: 273–278.

28.

Tomassini

Johansen-Berg

Leonardi

et al . Preservation of motor skill learning in patients with multiple sclerosis. Mult Scler J 2011; 17: 103–115.

29.

Schwid

Goodman

McDermott

et al . Quantitative functional measures in MS: What is a reliable change? Neurology. 2002; 58: 1294–1296.

30.

Wojtowicz

Omisade

and Fisk

JD.

Indices of cognitive dysfunction in relapsing–remitting multiple sclerosis: Intra-individual variability, processing speed, and attention network efficiency.

J Int Neuropsychol Soc 2013; 19: 551–558.

31.

Morrow

Jurgensen

Forrestal

et al . Effects of acute relapses on neuropsychological status in multiple sclerosis patients. J Neurol 2011; 258: 1603–1608.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.17 MB