Abstract
Background
Cognitive monitoring that can detect short-term change in multiple sclerosis is challenging. Computerized cognitive batteries such as the CogState Brief Battery can rapidly assess commonly affected cognitive domains.
Objectives
The purpose of this study was to establish the acceptability and sensitivity of the CogState Brief Battery in multiple sclerosis patients compared to controls. We compared the sensitivity of the CogState Brief Battery to that of the Paced Auditory Serial Addition Test over 12 months.
Methods
Demographics, Expanded Disability Status Scale scores, depression and anxiety scores were compared with CogState Brief Battery and Paced Auditory Serial Addition Test performances of 51 patients with relapsing–remitting multiple sclerosis, 19 with secondary progressive multiple sclerosis and 40 healthy controls. Longitudinal data in 37 relapsing–remitting multiple sclerosis patients were evaluated using linear mixed models.
Results
Both the CogState Brief Battery and the Paced Auditory Serial Addition Test discriminated between multiple sclerosis and healthy controls at baseline (
Conclusions
The CogState Brief Battery is highly acceptable and better able to detect cognitive change than the Paced Auditory Serial Addition Test. The CogState Brief Battery could potentially be used as a practical cognitive monitoring tool in the multiple sclerosis clinic setting.
Keywords
Introduction
Cognition deficits in multiple sclerosis (MS) reflect the widespread damage to the central nervous system and commonly manifest as difficulty in information processing, concentration and attention, working memory and new learning. 1 It is possible that, long before these deficits are obvious, subtle dysfunction may be present that can be difficult to detect using current clinical screening techniques.2,3 The gold standard for diagnosis of cognitive impairment in MS patients is a formal neuropsychological assessment, however, this is time and resource intensive. Abbreviated MS-specific batteries such as the Brief International Cognitive Assessment for MS (BICAMS) battery have been developed to shorten and simplify assessments, but still require skilled and trained operators. In practice, a BICAMS test can take 15–20 min to fully complete.4,5 Furthermore, fatigue and learning effects (despite alternate forms), 6 restrict the use of most batteries as monitoring tools in the MS clinic. The sensitivity of current tests to detect slow subclinical decline in ostensibly cognitively normal patients beyond that expected for age alone, is limited.7,8 Single tests that potentially measure cognitive changes under treatment conditions are often used in research trials. Tools such as the Paced Auditory Serial Addition Test (PASAT) 9 and Symbol Digit Modalities Test (SDMT) 10 have been validated against neuropsychological testing in people with MS (pwMS). While the PASAT utility is limited by its relatively low acceptability, the SDMT could be used to monitor longitudinal cognitive changes. 11 However, it also has significant practice effects even with multiple different versions, 12 which are likely even with emerging computerized versions. 13
Effective cognitive monitoring requires rapid cognitive assessments that can maintain motivation, minimize process factors (such as fatigue or anxiety), and avoid significant learning effects with repeated use. Computerized, brief and repeatable cognitive batteries such as the CogState Brief Battery (CBB) 14 have been extensively validated (see http://www.cogstate.com). The test can be Web-based and potentially represents an opportunity for self-administered cognitive screening. Psychometric properties have been optimized to minimize practice effects after an initial familiarization, making it useful for repeated testing 15 such as before and after clinical intervention. 12 In this study, we compared performance on the CBB and the PASAT in pwMS and healthy controls (HCs) cross-sectionally. In addition, we provide a 12-month, prospective comparison between the two tests in patients with relapsing–remitting multiple sclerosis (RRMS).
Materials and methods
Participants
Seventy MS patients with RRMS or secondary progressive MS (SPMS) were recruited from a tertiary hospital clinic in Melbourne Australia. HCs included partners, friends of patients, hospital staff and students. Participants needed to be able to use a computer (adequate vision, upper limb dexterity) and English speaking. Existing cognitive impairment due to other neurological, psychiatric or medical disease was an exclusion. We recorded relevant demographic characteristics, and medical, psychiatric and neurological history. The Expanded Disability Status Scale (EDSS) score was completed by a trained investigator. All participants were screened and enrolled for cross-sectional comparison of the CBB and the PASAT. To study longitudinal changes in early MS, only patients with RRMS were invited to repeat CBB testing at one, two, three, six and 12 months. The PASAT was repeated at the six- and 12-month visits. The study was approved by the local Human Research Ethics Committee and all participants provided informed consent.
Cognitive testing
Cognitive testing was performed on a desktop computer in a quiet room. All participants were tested twice at screening to ensure familiarity and minimize practice effects before the baseline test (within 28 days after screening). The total duration of a study visit was approximately 30 min.
The CBB. 14
The CBB (CogState Ltd) version consisted of six different tasks, using playing card visuals, in a game-like interface. The cognitive domains measured include psychomotor function, information processing speed, attention, visual learning, spatial problem solving, executive function and (working) memory. Each task displays an instruction screen, followed by a short familiarization, and then the real test. Single random playing cards appear face-down in the center of the screen, then turn face-up, and the participant is required to press either a ‘yes’ or ‘no’ button as quickly as they can. The first task (‘Detection’ (DET)) requires pressing the ‘yes’ button ‘as soon as the card turned face-up. Subsequent tasks used either button to respond ‘if the card is red’ (‘Identification’ (IDN)), ‘if you have seen the card before in the same task’ (‘One card learning’ (OCL)) or ‘if the card is the same as the previous card’ (‘One card back’ (ONB)). An executive function (‘Groton Maze learning) test that involved discovering a hidden pathway in a 10x10 tile maze was performed. Finally, a spatial memory task that required learning the screen locations of simple colored shapes (Continuous paired associated learning task (CPAL)) was completed. After a single practice, most participants were familiar with the test and showed no/minimal practice effects subsequently. The total test battery duration was 15 min.
The PASAT. 16
The PASAT is an audio-presented test that assesses information processing speed, attention, working memory and calculation ability. Single digits are presented every three seconds and the subject is required to add each new digit to the one immediately prior to it. To minimize familiarity with stimulus items, two alternate forms were used. Total test duration was 13 min.
Acceptability
Acceptability (e.g. ‘Did you enjoy the tasks?’) was scored on 10-point Likert scales, ranging from a maximum negative response (=0) to a maximum positive response (=10). Questionnaires were delivered electronically after completion of the cognitive tasks at baseline and each subsequent visit.
Anxiety, depression and fatigue
MS participants completed the Patient Health Questionnaire (PHQ-9) 17 to assess depression and the Penn State Worry Questionnaire (PSWQ) 18 to assess anxiety at all visits. The Participant Fatigue Questionnaire was completed prior to each test. The participant was asked: How sleepy are you feeling right now? Responses were rated from 0–10 with 0='I am fully awake'; and 9='I am asleep'; with an associated cartoon depiction of a face in various stages of drowsiness.
Statistical analysis
Non-parametric statistics were used to compare (a) acceptability and mental health scores and (b) baseline disease characteristics and cognitive scores. For each CBB task that included playing cards, accuracy was defined by the number of correct responses expressed as a proportion of the total responses made (true positive, true negative, false positive, false negative, trials in which they failed to respond (maxout) and premature responses (anticipations)). An arcsine transformation was used to normalize the distribution. Performance speed was defined as the average reaction time (RT; (ms)) for correct responses made; mean RT was transformed using a base 10 logarithmic transformation. The GML and CPAL results were recorded as the total errors made. For the PASAT, the total correct answers out of 60 was recorded. Student
Results
Baseline characteristics
Demographic and disease characteristics are shown in Table 1. Of the 70 patients enrolled, 51 (73%) had RRMS. Two RRMS and two SPMS patients withdrew consent before baseline assessment. The mean disease duration was 14 (SD=10.8) years. The median EDSS was 2.5 (interquartile range (IQR) 1.5–4.0), the cerebellar Kurtzke functional score (KFS) was 0 (0–2), the pyramidal KFS was 1 (0–2) and the visual KFS was 0 (0.0). The PASAT was performed in 56 patients (10 patients declined doing the test) at baseline. Thirty-nine RRMS patients agreed to longitudinal follow-up, 34 completed the six-month assessment and 30 completed the 12-month assessment. Patients who completed baseline and longitudinal follow-up, were similar in sex (80.5% and 81.6% female), age (
Demographics at baseline.
CBB: CogState Brief Battery; EDSS: Expanded Disability Status Scale; IQR: interquartile range; MS: multiple sclerosis; PASAT: Paced Auditory Serial Addition Test; RRMS: relapsing–remitting multiple sclerosis; SD: standard deviation.
aValues shown after base 10 logarithmic transformation.
Acceptability
Acceptability data were available for 37 MS participants. The duration of the CBB test was rated as ‘not too long, not too short’ to ‘just right’ by 86% of participants compared to 50% for the PASAT (
Anxiety, depression and fatigue
The mean baseline PSWQ score was 40.5 (SD=14.8) with a score suggestive of significant worry (>45) recorded in 11 (31%) participants. The mean PHQ-9 score was 7.3 (SD 5.7) with 14 participants (52%) reporting no depression and three (8%) reporting moderate to severe depression. The median baseline fatigue score was two (IQR 0–4) indicating mild levels of fatigue in this cohort. There was no correlation between baseline PSWQ-15, PHQ-9 or fatigue scores and performance scores on the PASAT or CBB.
Disease severity markers and cognitive outcomes
There was no correlation between PASAT scores at baseline and the EDSS (
Comparison of cognitive performance between people with MS and HCs
The mean performance values (with SD) are reported in Table 1. A comparison between MS participants and HCs is shown in Table 2 with the mean difference and 95% CIs between the groups reported. The cognitive performance of MS patients on both CBB and PASAT was significantly worse (
Baseline comparison between multiple sclerosis (MS) patients and healthy controls.
CBB: CogState Brief Battery; CI: confidence interval; PASAT: Paced Auditory Serial Addition Test.
Lower values correspond to a slower reaction time or decreased accuracy.
aValues shown after base 10 logarithmic transformation.
Correlation between CBB tasks and PASAT
Moderate correlations between baseline performance on PASAT and CBB tasks were found (Table 3).
Comparison a between Paced Auditory Serial Addition Test (PASAT) and CogState Brief Battery (CBB) tasks.
CPAL: Continuous paired associate learning; DET: Detection; GML: Groton Maze learning; IDN: Identification; OCL: One card learning; ONB: One card back.
aPearson correlation.
Longitudinal change in CBB and PASAT scores in RRMS patients
Linear mixed models to analyze the change of task performance over time indicated that RT slowed significantly over 12 months for the DET (ß=0.001,

Adjusted mean change in detection (DET) and identification (IDN) speed over time with time modeled as a factor (co-varying for baseline). Reaction speed during the CogState Brief Battery (CBB) Detection and Identification tasks slowed by a significant amount over 12 months (
The same models revealed no change over 12 months in PASAT scores (standardized slope ß=0.044,
Individual patient trajectories for DET and IDN RT over 12 months are shown in Figure 2 and indicate variability in RT during the first six months with a general trend for RT slowing emerging between 6–12 months. IDN RT over 12 months slowed by a mean of 6.1% (3.1); 29.4% participants worsened by more than one SD of the mean one-month DET RT. Reaction time for the IDN task slowed by 7.5% (4.3) with 32.4% slowing more than one SD from the mean IDN RT at one month; four participants (11.8%) improved over 12 months.

Individual trajectories for detection (DET) and identification (IDN) reaction time (RT) over 12 months. Compared to mean DET and IDN RT at one month, we identified 29.4% participants with slower DET RT and 32.4% participants with slower IDN RT at 12 months.
Discussion
This study demonstrates higher acceptability of computerized cognitive testing using CBB in pwMS compared with the established PASAT test. 19 This is important, as patient acceptance for a cognitive monitoring test is paramount for feasibility of repeat administration. The CBB tests were rated as enjoyable and engaging and caused less patient anxiety than the PASAT. Both the PASAT and the CBB enabled detection of slower processing speed in pwMS compared to HCs tested in the same environment at baseline. In addition, the CBB task RTs were correlated with disease severity (EDSS) but not disease duration, whereas PASAT scores did not correlate with either measure. Although cognitive impairment in MS can occur irrespective of disease duration20,21 and physical disability,22–24 it is well known that the presence of cognitive decline is associated with higher EDSS scores and a more progressive disease course. 25
Importantly, in an RRMS population, our data showed subtle slowing (in the absence of any change in visual, motor or sensory functional scores) of CBB RT and demonstrated the feasibility of detecting RT change over 12 months. During each CBB task, performance speed was more impaired than accuracy, demonstrating the sensitivity of RT as a measure of processing speed in MS. This has also recently been reported in pediatric MS using the DET and IDN speed measures of the CBB at a single time point, showing similar sensitivity to the BICAMS speed measure. 26 In addition, our longitudinal analyses identified two CBB tasks, DET and IDN speed, for which performance slowed over the 12-month study period. The PASAT measures failed to identify such change, suggesting that the sensitivity of the CBB measures to detect change was greater and measured real decline, a conclusion supported by the correlation of the change in CBB speed measures over 12 months with the change in EDSS scores (also not seen with the PASAT measures). These findings are of particular importance given that a decline or slowing in information processing speed is one of the principal cognitive domains affected in MS. 27 Both processing speed and working memory are core cognitive skills that are essential to performance in domains such as attention and learning. 25 However, the change was subtle and determined in group analyses rather than being observable in individual trajectories. When individual trajectories were interrogated (Figure 2), patients with slowing in RT for DET and IDN could be identified despite significant variability in this small cohort. Further experience with these speed measures is required to determine whether more accurate measurement of this commonly affected domain presents an opportunity to develop a valid monitoring tool for use in individual pwMS in the clinic. Longitudinal measurements in HCs were not done and should be included in future studies to better understand the significance of these changes.
The therapeutic landscape for MS is rapidly evolving,28,29 with more than 12 disease-modifying treatments available. Monitoring to detect suboptimal therapeutic response is a key aim of modern MS services, with regular magnetic resonance imaging (MRI) scans supplementing relapse assessments and clinical evaluations. Unfortunately, cognitive function is assessed rarely during patients’ clinic visits due to the resource requirements and subtle nature 2 of these symptoms. Cognitive decline is, however, a major determinant of disease outcome25,30,31 and quality of life 32 in pwMS. Our results suggest that automated computerized cognitive testing could potentially fill this monitoring gap, allowing for earlier identification and more comprehensive assessment. This would ideally include evaluation of potential confounders such as emergent depression or increased fatigue, but also, if no medical reason other than potential MS-associated decline emerges, therapeutic escalation of immunotherapy33,34 and rehabilitation strategies that can improve outcomes. 35
Our preliminary results need validation in larger cohorts of MS patients. In addition, it is possible that the educational status of the HC group was higher than the MS cohort due to the inclusion of hospital staff and students. Although repeat testing was performed to ensure familiarization with the tests at baseline, learning effects between the pwMS and HC were not formally explored. This study used the PASAT 3 9 as a comparative test – this is often poorly tolerated by patients. Whilst we selected the PASAT 3 based on the recommendations from the Multiple Sclerosis Functional Composite (MSFC) developers, 19 it should be noted that it is a quick index of overall function and does not provide a clinical cognitive assessment. Notably, recent work has recommended that the PASAT be replaced by the SDMT in the MSFC 8 and future studies need to include the SMDT as a comparative test of processing speed. 10 The specificity of using relatively simple tasks of a CBB as a long-term, serial, cognitive monitoring tool in MS, also needs to be established. PwMS with a decline in detection and identification speed will need to be tested more comprehensively in comparison to abbreviated batteries such as BICAMS 4 or Auditory Recorded Cognitive Screen (ARCS) 36 to better understand the implications of CBB decline in comparison to impairment in a multi-domain cognitive assessment. Although the CBB has been extensively longitudinally validated in HCs in other studies,15,37–39 a comparison in parallel with a MS cohort is needed. Nine of the 39 RRMS patients who agreed to longitudinal follow-up, did not complete the 12-month visit, and our results need to be viewed as preliminary. If, however, future studies confirm our results, then computerized monitoring of reaction speed in three tasks, with a brief total test-time of three minutes, potentially entirely self-administered, could prove to be sufficient to monitor cognitive change in processing speed over time in pwMS. This would allow for efficient testing and improve the feasibility of rapid, large-scale implementation in clinical practice, and hopefully in the future, Web-based home testing.
Supplemental Material
Supplemental material for Monitoring cognitive change in multiple sclerosis using a computerized cognitive battery
Supplemental Material for Monitoring cognitive change in multiple sclerosis using a computerized cognitive battery by L De Meijer, D Merlo, O Skibina, EJ Grobbee, J Gale, J Haartsen, P Maruff, D Darby, H Butzkueven* and Van der A Walt* in Multiple Sclerosis Journal-Experimental, Translational and Clinical
Footnotes
Acknowledgements
The authors wish to dedicate this article to the memory of Louise Kurczyski, their study coordinator, who died before completion of the analyses.
Conflicts of Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: L De Meijer: nothing to declare. D Merlo: nothing to declare. O Skibina has received travel support and honoraria from Biogen, Roche and Merck. EJ Grobbee: nothing to declare. J Gale: nothing to declare. J Haartsen: nothing to declare. D Darby DD is a consultant to Neurability, former founder and shareholder of CogState, CEO of Cerescape, and received honoraria for lectures from Biogen, Novartis and other pharma. Paul Maruff is the Chief Science Officer at CogState Ltd. H Butzkueven has received research support, travel support and speaker's honoraria from Biogen, Novartis, Merck, Teva, Roche and Bayer.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by the MS Research Fund, Box Hill Hospital.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
