Detection of subtle changes in cognition in multiple sclerosis (MS) may be enabled by ambulatory smartphone-based cognitive tests.
Objective:
To examine the feasibility, reliability, and validity of ambulatory cognitive tests in people with MS.
Method:
Adults with MS (N = 255) completed the NIH Toolbox Cognitive Battery, Symbol Digit Modalities Test, Paced Auditory Serial Addition Test-3, and 14 days of 4X/day ambulatory cognitive tests of working memory (dot memory) and processing speed (symbol search). Feasibility (response rates), measurement reliability (overall and by number of days of tests), and construct validity (convergent validity and known-groups analyses) were calculated.
Results:
Participants completed 80.1% of available testing sessions. Reliability was 0.98 for dot memory and 0.99 for symbol search. Reliability attained ⩾0.90 within 3 days for dot memory and 1 day for symbol search. Symbol search scores correlated significantly with neuropsychological tests of processing speed (rs range = |0.53–0.69|) and differed significantly by disability level (t = −3.95, p < 0.001) and MS subtype (t = −2.81, p = 0.005). Dot memory scores correlated with neuropsychological tests of working memory (rs range = |0.45–0.58|); dot memory scores did not differ by disability level or MS subtype.
Conclusion:
Smartphone-administered tests of working memory and processing speed demonstrate good feasibility, reliability, and construct validity in MS.
RaoSMLeoGJBernardinL, et al. Cognitive dysfunction in multiple sclerosis. I. Frequency, patterns, and prediction. Neurology1991; 41(5): 685–691.
2.
GrigsbyJAyarbeSDKravcisinN, et al. Working memory impairment among persons with chronic progressive multiple sclerosis. J Neurol1994; 241(3): 125–131.
3.
FoongJRozewiczLQuaghebeurG, et al. Executive function in multiple sclerosis. The role of frontal lobe pathology. Brain1997; 120(Pt. 1): 15–26.
4.
FinlaysonMPetersonEMatsudaPN. Participation as an outcome in multiple sclerosis falls-prevention research: Consensus recommendation from the international MS Falls prevention research network. Int J MS Care2014; 16(4): 171–177.
5.
ChiaravallotiNDDeLucaJ. Cognitive impairment in multiple sclerosis. Lancet Neurol2008; 7(12): 1139–1151.
6.
GunnHJNewellPHaasB, et al. Identification of risk factors for falls in multiple sclerosis: A systematic review and meta-analysis. Phys Ther2013; 93(4): 504–513.
7.
BenedictRHCookfairDGavettR, et al. Validity of the minimal assessment of cognitive function in multiple sclerosis (MACFIMS). J Int Neuropsychol Soc2006; 12(4): 549–558.
8.
LangdonDWAmatoMPBoringaJ, et al. Recommendations for a brief international cognitive assessment for multiple sclerosis (BICAMS). Mult Scler2012; 18(6): 891–898.
9.
BeverCTJrGrattanLPanitchHS, et al. The brief repeatable battery of neuropsychological tests for multiple sclerosis: A preliminary serial study. Mult Scler1995; 1(3): 165–169.
10.
SpoonerDMPachanaNA. Ecological validity in neuropsychological assessment: A case for greater consideration in research with neurologically intact populations. Arch Clin Neuropsychol2006; 21(4): 327–337.
11.
AllardMHuskyMCathelineG, et al. Mobile technologies in the early detection of cognitive decline. PLoS ONE2014; 9(12): e112197.
12.
TimmersCMaeghsAVestjensM, et al. Ambulant cognitive assessment using a smartphone. Appl Neuropsychol Adult2014; 21(2): 136–142.
13.
AschenbrennerAJHassenstabJMorrisJC, et al. Relationships between hourly cognitive variability and risk of Alzheimer’s disease revealed with mixed-effects location scale models. Neuropsychology2024; 38(1): 69–80.
14.
CerinoESKatzMJWangC, et al. Variability in cognitive performance on mobile devices is sensitive to mild cognitive impairment: Results from the Einstein aging study. Front Digit Health2021; 3: 758031.
15.
SliwinskiMJMogleJAHyunJ, et al. Reliability and validity of ambulatory cognitive assessments. Assessment2018; 25(1): 14–30.
16.
BielakAAHultschDFStraussE, et al. Intraindividual variability is related to cognitive change in older adults: Evidence for within-person coupling. Psychol Aging2010; 25(3): 575–586.
17.
BielakAAHultschDFStraussE, et al. Intraindividual variability in reaction time predicts cognitive outcomes 5 years later. Neuropsychology2010; 24(6): 731–741.
18.
FoongYCBridgeFMerloD, et al. Smartphone monitoring of cognition in people with multiple sclerosis: A systematic review. Mult Scler Relat Disord2023; 73: 104674.
19.
HarrisCTangYBirnbaumE, et al. Digital neuropsychology beyond computerized cognitive assessment: Applications of novel digital technologies. Arch Clin Neuropsychol2024; 39(3): 290–304.
20.
ValentineTRKratzAL. Feasibility, reliability, and validity of ambulatory cognitive tests in fibromyalgia and matched controls. J Int Neuropsychol Soc2023; 29(9): 893–901.
21.
KatzMJGarzaAGDLRoqueNA, et al. Feasibility, reliability and predictive validity of digital cognitive assessments in a diverse cohort of community-residing older adults. Alzheimers Dement2023; 19(suppl. 22): 1.
22.
KratzALEhdeDMAlschulerKN, et al. Optimizing detection and prediction of cognitive function in multiple sclerosis with ambulatory cognitive tests: Protocol for the longitudinal observational CogDetect-MS study. JMIR Res Protoc2024; 13: e59876.
23.
HoholMJOravEJWeinerHL. Disease steps in multiple sclerosis: A longitudinal study comparing disease steps and EDSS to evaluate disease progression. Mult Scler1999; 5(5): 349–354.
24.
GershonRCWagsterMVHendrieHC, et al. NIH toolbox for assessment of neurological and behavioral function. Neurology2013; 80(11, suppl. 3): S2–S6.
25.
WeintraubSDikmenSSHeatonRK, et al. Cognition assessment using the NIH Toolbox. Neurology2013; 80(11, suppl. 3): S54–S64.
26.
SmithA. Symbol Digit Modalities Test (Revised). Torrance, CA: Western Psychological Services, 1982.
27.
GronwallDSampsonH. The psychological effects of concussion. Auckland, New Zealand: Auckland University Press, 1974.
28.
BrickTRMundieJWeaverJ, et al. Low-burden mobile monitoring, intervention, and real-time analysis using the wear-IT framework: Example and usability study. JMIR Form Res2020; 4(6): e16072.
29.
ScaramozzaMChiesaPAZajacL, et al. Konectom™ cognitive processing speed test enables reliable remote, unsupervised cognitive assessment in people with multiple sclerosis: Exploring the use of substitution time as a novel digital outcome measure. Mult Scler2024; 30(9): 1193–1204.
30.
Van OirschotPHeeringsMWendrichK, et al. Symbol digit modalities test variant in a smartphone app for persons with multiple sclerosis: Validation study. JMIR Mhealth Uhealth2020; 8(10): e18160.
31.
PoddaJTacchinoAPonzioM, et al. Mobile Health App (DIGICOG-MS) for Self-assessment of cognitive impairment in people with multiple sclerosis: Instrument validation and usability study. JMIR Form Res2024; 8: e56074.
32.
LamKHvan OirschotPden TeulingB, et al. Reliability, construct and concurrent validity of a smartphone-based cognition test in multiple sclerosis. Mult Scler2022; 28(2): 300–308.
33.
LamKHBucurIGVan OirschotP, et al. Towards individualized monitoring of cognition in multiple sclerosis in the digital era: A one-year cohort study. Mult Scler Relat Disord2022; 60: 103692.
34.
CostaSLGenovaHMDeLucaJ, et al. Information processing speed in multiple sclerosis: Past, present, and future. Mult Scler2017; 23(6): 772–789.
35.
Navarro-LopezVCano-de-la-CuerdaRFernandez-GonzalezP, et al. Reliability and construct validity of a mobile application for the finger tapping test evaluation in people with multiple sclerosis. Brain Sci2024; 14(4):407.
