Sage Journals: Discover world-class research

Abstract

Background:

Existing metrics of patient-reported cognitive difficulties in multiple sclerosis (MS) are lengthy, lack psychometric rigor, and/or fail to query prevalent expressive language deficits.

Objective:

Develop a brief psychometrically robust metric of patient-reported cognitive deficits that includes language items; the Multiple Sclerosis Cognitive Scale (MSCS).

Method:

Exploratory factor analysis (EFA) was conducted on 20 Perceived Deficits Questionnaire (PDQ) items plus five newly developed language questions in a large MS sample and matched respondents without neurologic disease. Independent confirmatory principal components analysis (PCA) assessed EFA factor structure. Reliability of the new scale and subscales, and relationships with objective cognitive impairment and cognitive change, were assessed.

Results:

EFA in patients (n = 502) and controls (n = 350), item analyses, and confirmatory PCA in an independent sample (n = 361 patients; 150 controls) supported construction of an eight-item scale with four two-item subscales: Executive/Speed, Working Memory, Expressive Language, and Episodic Memory. Internal consistency and test–retest reliability were excellent for the total MSCS (α = 0.93, ICC = 0.95) and good for each subscale (α’s:0.83–0.87; ICCs: 0.86–0.92). MSCS showed medium-size links to cross-sectional objective cognitive impairment (η² = .06) and cognitive change over time (η² = .07); the traditional PDQ did not (η²s = 0.01 and 0.02).

Conclusion:

The brief MSCS is a psychometrically robust, reliable, and valid metric of patient-reported cognitive deficits in MS that holds promise for improving assessment of MS cognitive dysfunction.

Keywords

Multiple sclerosis cognition patient-reported outcomes MSCS psychometrics self-report

Introduction

Cognitive decline is common in multiple sclerosis (MS);¹ it is therefore important to have psychometrically validated, clinically feasible patient-report tools to screen for cognitive deficits. One option is the Perceived Deficits Questionnaire (PDQ),² which is a 20-item scale developed in 1990 with four a priori subscales (attention/concentration, planning/organization, retrospective memory, prospective memory). The PDQ has limitations: subscales were not validated by factor analytic studies, there are no questions about expressive language deficits although word-finding difficulty is frequently reported by patients,³ and the relatively long length of the PDQ reduces clinical feasibility due to patient burden, especially when included within a wider collection of questionnaires (e.g. fatigue, mood, and physical disability). Another option is the MS Neuropsychological Questionnaire (MSNQ),⁴ which is a 15-item scale developed in 2003 without cognitive subscales; items focus heavily on attention/executive function and memory without assessment of word-finding difficulty. It is also notable that the MSNQ and PDQ were developed over two and three decades ago, respectively. Others have more recently recommended Neuro-QoL short forms including a brief questionnaire assessing communication difficulties;⁵ however, even these lack questions on word-finding difficulty. This study aims to develop a brief, reliable, clinically useful measure of patient-reported cognitive difficulties with cognitive subscales validated with factor analytic techniques.

Methods

Sample

The Corinne Goldsmith Dickinson Center for Multiple Sclerosis at Mount Sinai Hospital is a tertiary care center with a catchment area encompassing the racially/ethnically diverse New York Metropolitan Area. In 2018, we established a clinic aiming to perform cognitive screenings as standard of care for all patients at our center; as such, we have shown that demographic and disease characteristics do not differ between the patients completing these cognitive screenings and a random sample of patients cared for at our MS Center.⁶ The 20-item PDQ plus five language questions were completed by patients from August 2018 through August 2021. We performed an Institutional Review Board (IRB) approved retrospective chart review of clinical and patient-reported cognitive data from all patients aged 18 to 65 years and diagnosed with relapse-onset MS from 1995 onward who completed cognitive screenings. Self-reported cognitive difficulty was also collected from demographically matched persons without neurologic conditions through an IRB-exempt anonymous electronic capture questionnaire; links to this survey were sent to patients at the MS Center who were encouraged to share the link with friends and/or post on social media at their discretion.

PDQ plus language questions

The PDQ is a 20-item self-report inventory assessing frequency of cognitive difficulties as never (0), rarely (1), sometimes (2), fairly often (3), or very often (4) over the last 4 weeks. The PDQ has four a priori subscales (with five items each) assessing attention/concentration, planning/organization, retrospective memory, and prospective memory, but there is no assessment of language difficulty. Our development of five language questions to add to the PDQ was informed by clinical experience with patients and confirmed by retrospective chart review of self-report remote electronic capture (REDCap) questionnaire in consecutive patients (meeting aforementioned inclusion criteria) during a 12-month period. Patients were asked “Do you have any concerns about your current cognition?” If yes, an open-field question displayed: “In a few words, please describe the cognitive problem(s) that you experience.” These questions were completed prior to completion of the PDQ to avoid biasing results. Three independent raters (see acknowledgements) reviewed all free-text responses to identify the presence or absence of self-reported language difficulty and then to categorize difficulties into like categories.

Statistical approach

Exploratory factor analysis (EFA) was performed using R version 4.2.1 and the psych package.⁷ Goodness of fit was evaluated using root mean square error (RMSE: ⩽.05 good, ⩽.08 acceptable, ⩽.10 marginal) and Tucker–Lewis Index (TLI: ⩾.95 good, ⩾.90 acceptable, ⩾.85 marginal). The goal was to develop an empirically derived brief cognitive screener; as such, we identified the two items with the highest factor loadings and highest internal consistency (Cronbach’s alpha: ⩾.90 excellent, ⩾.80 good, ⩾.70 acceptable) for each factor, which were selected for the new brief screener. Next, confirmatory PCA was performed for selected items in an independent sample of patients and controls.

Results

Sample

Data on the 25-item self-report cognitive questionnaire were captured from 502 persons with MS and 350 demographically matched control respondents for the EFA (Table 1).

Table 1.

Sample characteristics.

	EFA sample (n = 852)		Confirmatory PCA samples (n = 511)
	MS (n = 502)	Controls (n = 350)	Research MS (n = 167)	Clinical MS (n = 194)	Controls (n = 150)
Age (years), mean [SD]	43.6 [11.5]	44.3 [12.5]	36.2 [7.5]	42.9 [10.8]	43.5 [14.0]
Sex, N [%]
Women	357 [71.1]	252 [72.0]	111 [66.5]	147 [75.8]	108 [72.0]
Men	145 [28.9]	98 [28.0]	56[33.5]	47 [24.2]	42 [28.0]
Bachelor’s degree, N [%]	373 [74.3]	275 [78.6]	130 [77.8]	128 [66.0]	85 [56.7]
MHI-5, mean [SD]	70.5 [15.8]	76.5 [13.0]	72.4 [15.8]	69.9 [15.1]	73.5 [13.2]
Race and ethnicity, N [%]
Black, (not Hispanic or Latino/a)	82 [16.3]		24 [14.4]	53 [27.3]
Hispanic or Latino/a	87 [17.3]		38 [22.8]	45 [23.2]
White (not Hispanic or Latino/a)	318 [63.3]		98 [58.7]	92 [47.4]
Other (not Hispanic or Latino/a)	15 [3.0]		7 [4.2]	4 [2.1]
Years since MS dx, median [IQR]	7 [2, 13]		4 [3, 6]	5.5 [1, 12]
Disease course, N [%]
RRMS	419 [83.5]		167 [100]	175 [90.2]
SPMS	83 [16.5]		0 [0]	19 [9.8]
Disease-modifying therapy, N [%]
None	73 [14.5]		13 [7.8]	23 [11.9]
Platform injectables, teriflunomide	97 [19.3]		24 [14.4]	30 [15.5]
S1p modulators, fumarates	142 [28.3]		62 [37.1]	48 [24.7]
Monoclonal antibodies	190 [37.8]		68 [40.7]	93 [47.9]

Language questions

Data were captured from a subset of 319 consecutive patients with MS for validation of language questions. Of 319 patients, about half (51%, n = 163) endorsed “yes” to having concerns about cognition; free-text responses were reviewed by three independent raters to identify the presence or absence of self-reported language difficulty (interrater agreement was excellent, Kappa [95% confidence interval (CI)] of 0.85 [0.76, 0.93]). Language deficits were considered present if all three raters (n = 54) or two of three raters (n = 10) coded it present; language difficulty was reported by 39% of patients endorsing concerns about cognition (n = 64 of 163), and 20% of all patients regardless of endorsement of cognitive concerns (n = 64 of 319). Analysis of the 64 responses identified (a) word-finding difficulty (n = 46) with descriptions of trouble retrieving known words (i.e. “tip of the tongue” phenomenon), (b) difficulty clearly expressing thoughts (n = 13), (c) using the wrong word or misspeaking (n = 10), (d) difficulties comprehending text or discourse (n = 6), and (e) other and/or vague responses (n = 6; e.g. “speech problems,” misspelling). Analyses support use of the five language questions (last five items in Table 2).

Table 2.

Results of exploratory factor analysis of 25 self-reported cognitive deficits.

Perceived deficits questionnaire item plus language questions	Factors
	A	B	C	D	E
Losing your train of thought	0.13	0.01	0.29	0.18	0.46
Trouble remembering people’s names, even familiar people	−0.09	0.25	0.28	0.11	0.35
Forgetting what you came into the room for	0.07	0.17	0.30	0.06	0.47
Trouble getting things organized	0.62	0.10	−0.03	−0.01	0.28
Trouble concentrating on what people are saying, or on what you are reading or watching on TV	0.41	−0.05	0.05	0.41	0.25
Forgetting if you had already done something	0.15	0.30	0.15	0.13	0.33
Forgetting appointments or meetings you had planned	0.16	0.58	0.00	0.04	0.22
Difficulty planning what to do in the day	0.66	0.23	−0.05	0.03	0.03
Difficulty doing more than one thing at a time	0.54	0.23	0.07	0.06	0.01
Trouble remembering where you put something, like your keys or where you parked your car	0.14	0.54	0.06	−0.06	0.25
Forgetting what day of the week it is	0.10	0.59	0.05	0.05	−0.02
Trouble getting started, even if you had lots to do	0.84	−0.01	0.06	−0.03	−0.03
Taking a long time to finish things	0.81	0.01	0.11	−0.03	−0.01
Forgetting details of a recent conversation	0.13	0.18	0.23	0.37	0.15
Forgetting to do things like turning on your alarm clock, or charging your smartphone or computer	0.04	0.79	0.03	−0.04	−0.06
Finding your mind drifting, or your mind going blank	0.37	0.09	0.11	0.32	0.09
Trouble holding a string of numbers in your head, even for a few seconds	0.04	0.34	0.09	0.31	0.18
Trouble recalling what happened during the last week	−0.07	0.43	0.14	0.44	0.07
Forgetting to do routine things without reminders, like taking your medication, or picking someone up	0.06	0.72	0.07	0.05	−0.07
Trouble making decisions	0.58	0.10	0.05	0.23	−0.17
Having a word “on the tip of your tongue” but with difficulty getting it out	0.03	0.00	0.81	−0.05	0.14
Having to read something several times to understand it	0.26	−0.02	0.35	0.32	0.04
Having a sense of what you want to say, but you have trouble clearly expressing your thoughts	0.07	0.00	0.78	0.04	0.01
Accidentally saying the wrong word / misspeaking	−0.01	0.07	0.80	−0.01	−0.09
Having difficulty following a conversation, especially a conversation with multiple people or parts	0.15	0.15	0.39	0.36	−0.16

The 25 items are from the 20-item PDQ plus five new language items (in italics).

EFA

Responses to the 25 questions by 852 consecutive respondents (502 patients, 350 controls) were analyzed with EFA. The Kaiser–Meyer–Olkin measure of sampling adequacy was .98 suggesting excellent factorability. Results from the parallel analysis, in concordance with the scree plot, suggested that a five-factor solution with oblimin rotation has excellent fit (root mean square error of approximation [RMSEA] = .05, TLI = .962; Table 2, Figure 1). Of the four items with highest loadings for each factor, we identified the two with the highest internal consistency (Cronbach’s alpha). For factor A, internal consistency was good for “get started when lots to do” and “take long time to finish things” (α = 0.88). For factor B, internal consistency was acceptable for “forget to take medication, etc.” and “forget meetings, appointments” (α = 0.79). For factor C, internal consistency was good for “word on ‘tip of tongue’” and “clearly expressing thoughts” (α = 0.85). For factor D, internal consistency was good for “forgetting details of a recent conversation” and “trouble recalling what happened during the last week” (α = 0.85). For factor E, internal consistency was good for “forget why entered room” and “losing train of thought” (α = 0.85). Internal consistency was good for all two-item pairs across factors except factor B. Further inspection revealed possible floor effects for all four items with highest loadings for factor B, with <10% of all respondents endorsing “fairly often” or “very often.” These items are four of the five items of the PDQ prospective memory subscale; means for this scale were much lower than all other PDQ subscales in the original publication.² Only one of the other 21 questions showed a possible floor effect (“follow complex conversations”). To derive a brief scale with the best reliability and clinical relevance, we excluded factor B. EFA with the remaining eight items yielded four factors (Table 3, Figure 2), which are best characterized as Executive/Speed, Episodic Memory, Working Memory, and Expressive Language.

Figure 1.

Exploratory factor analysis of 25 self-reported cognitive deficits.

Table 3.

Results of exploratory factor analysis of eight MS Cognitive Scale (MSCS) items.

MSCS item	Current sample w/PDQ (n = 852; 502 MS, 350 HC)
MSCS item	Executive/Speed (ES)	Working Memory (WM)	Expressive Language (EL)	Episodic Memory (EM)
Trouble getting started	0.88	0.04	−0.02	−0.04
Take a long time to finish things	0.88	−0.04	0.03	0.05
Losing your train of thought	0.01	0.79	0.02	0.05
Forgetting why you entered a room	0.01	0.84	0.01	0.00
Word on the tip of your tongue	0.00	0.13	0.79	−0.05
Trouble clearly expressing your thoughts	0.02	−0.08	0.85	0.06
Forgetting details of conversations	0.04	0.02	0.00	0.84
Trouble recalling what happened during last week	−0.03	0.01	0.01	0.83

Figure 2.

Exploratory factor analysis of the eight MSCS items.

Confirmatory Principal Components Analyses

To replicate and verify the robustness of the scale, confirmatory principal components analyses (PCAs; four components, oblimin rotation) with the eight selected items were performed in an independent sample combining a research sample of 167 persons with relapsing-remitting MS who completed the full 25-item survey, a clinical sample of 194 patients with MS who completed the brief scale, and a sample of 150 respondents without neurologic conditions who completed the brief scale. As shown (Table 4), confirmatory PCA in this independent sample supported the factor structure of the eight-item MSCS, which was also shown when performing separate PCAs for all patients (n = 863) and all controls (n = 500; Table 5).

Table 4.

Results of confirmatory principal component analyses in independent sample.

MSCS item	Confirmatory PCA sample (n = 511; 361 MS, 150 HC)
MSCS item	Executive/Speed (ES)	Working Memory (WM)	Expressive Language (EL)	Episodic Memory (EM)
Trouble getting started	0.93	0.07	−0.02	0.04
Take a long time to finish things	0.86	−0.03	0.08	−0.06
Losing your train of thought	0.06	0.95	−0.06	−0.01
Forgetting why you entered a room	−0.02	0.74	0.18	−0.08
Word on the tip of your tongue	0.00	0.12	0.89	0.05
Trouble clearly expressing your thoughts	0.15	−0.05	0.78	−0.15
Forgetting details of conversations	−0.07	0.07	0.16	−0.83
Trouble recalling what happened during last week	0.08	0.01	−0.09	−0.94

Table 5.

Results of confirmatory principal component analyses separately for all patients and all controls.

MSCS item	MS (n = 863)				Controls (n = 500)
MSCS item	E/S	WM	EL	EM	E/S	WM	EL	EM
Trouble getting started	0.97	0.07	−0.03	0.05	0.93	0.01	0.00	−0.02
Take a long time to finish things	0.87	−0.05	0.07	−0.08	0.92	0.00	0.01	0.02
Losing your train of thought	0.02	0.88	0.02	−0.04	0.04	0.90	0.08	0.06
Forgetting why you entered a room	0.01	0.92	0.01	0.00	−0.01	0.79	−0.15	0.01
Word on the tip of your tongue	−0.04	0.09	0.92	0.04	0.02	0.22	−0.80	−0.07
Trouble clearly expressing your thoughts	0.07	−0.07	0.90	−0.06	0.04	−0.10	−0.87	0.14
Forgetting details of conversations	−0.02	0.11	0.10	−0.79	0.09	0.02	−0.18	0.72
Trouble recalling what happened during last week	0.03	−0.03	−0.04	−0.99	−0.03	0.04	0.06	0.96

Reliability

Internal consistency (Cronbach’s alpha) of the eight-item scale among patients who completed the brief form (n = 194) was excellent for the total MSCS (α = 0.93) and good for each subscale (Executive/Speed, α = 0.85; Episodic Memory, α = 0.85; Working Memory, α = 0.83, Expressive Language, α = 0.87). Test–retest reliability was assessed with intraclass correlation coefficients (ICC; two-way mixed analysis of variance [ANOVA]; absolute agreement between single scores)⁸ for the brief form in an independent sample of 40 consecutive patients with MS (mean [SD] age: 45.7 [12.3] years; 31 women, 19 men; 62.5% White non-Latino; inter-test interval median [interquartile range (IQR)]: 2 [1, 3] days). Reliability (ICC [95% CI]) was excellent for the total MSCS (0.95 [0.90, 0.97]) and good to excellent for subscales (Executive/Speed: 0.92 [0.85, 0.96]; Episodic Memory: 0.91 [0.83, 0.95]; Working Memory: 0.86 [0.75, 0.92]; and Expressive Language: 0.88 [0.70, 0.94]).

Construct validity: link to cross-sectional objective cognitive performance

A brief screening battery adopted for MS (BICAMS)⁹ consists of a high-sensitivity information processing task (Symbol Digit Modalities Test, SDMT)¹⁰ and measures of word-list learning and object-location memory. Our modified version of this battery includes SDMT, word-list total learning on the Hopkins Verbal Learning Test, Revised (HVLT-R),¹¹ and object-location memory on CANTAB Paired Associate Learning (PAL; tablet-based task not affected by sensorimotor ability, see online Supplement).¹² Task performance data were available for 502 consecutive patients who completed the full 20-item PDQ (August 2018 through August 2021) and 194 consecutive patients who completed the MSCS (which replaced the PDQ after August 2021). Raw scores were converted to age-adjusted norm-referenced z-scores relative to each test’s healthy standardization sample, which was then used to characterize impairment for each task as performance ⩽1.5 standard deviations below normal (z score ⩽ −1.5). Patients were categorized as having impairment on 0, 1, or 2+ tests. We then matched patients from the larger PDQ sample to the smaller MSCS sample for age, sex, race/ethnicity, education, MS phenotype, time since diagnosis, and mood (MHI-5), resulting in extremely well-matched samples of (a) 194 patients who completed the full 20-item PDQ and (b) 194 patients who completed the MSCS (Table 6).

Table 6.

Matched samples of patients with MS who completed the PDQ or the MSCS.

	PDQ sample	MSCS sample
Age (years), mean [SD]	42.3 [11.6]	42.9 [10.8]
Sex, N [%]
Women	147 [75.8]	147 [75.8]
Men	47 [24.2]	47 [24.2]
Bachelor’s degree, N [%]	128 [66.0]	128 [66.0]
MHI-5, mean [SD]	68.3 [16.8]	69.9 [15.1]
Race and ethnicity, N [%]
Black, (not Hispanic or Latino/a)	53 [27.3]	53 [27.3]
Hispanic or Latino/a	45 [23.2]	45 [23.2]
White (not Hispanic or Latino/a)	92 [47.4]	92 [47.4]
Other (not Hispanic or Latino/a)	4 [2.1]	4 [2.1]
Years since MS dx, mean [SD]	8.3 [6.3]	7.3 [6.6]
Disease course, N [%]
RRMS	175 [90.2]	175 [90.2]
SPMS	19 [9.8]	19 [9.8]
Disease-modifying therapy, N [%]
None	32 [16.5]	23 [11.9]
Platform injectables, teriflunomide	31 [16.0]	30 [15.5]
S1p modulators, fumarates	54 [27.8]	48 [24.7]
Monoclonal antibodies	77 [39.7]	93 [47.9]

One-way ANOVAs tested differences in the PDQ (mean of 20 items) and the MSCS (mean of eight items) across patients with impairment on 0, 1, or 2+ tests. As shown (Figure 3), MSCS differed across levels of cognitive impairment (F[2, 193] = 5.85, p = 0.003; η² [95% CI] = 0.06 [0.01, 0.13]) whereby patient-reported difficulty was worse among patients with 2+ impaired tests than those with ⩽1 impaired test. In contrast, there was no difference in PDQ across levels of cognitive impairment (F[2, 193] = 1.34, p = 0.265; η² [95% CI] = 0.01 [0.00, 0.06]). One-way ANOVAs were repeated after adjusting MSCS and PDQ for mood (MHI-5, using GLM). Again, as shown (Figure 3), there were differences in MSCS across levels of cognitive impairment F[2, 193] = 3.83, p = 0.023; η² [95% CI] = 0.04 [0.00, 0.10]), but PDQ did not differ across levels of impairment (F[2, 193] = 0.15, p = 0.864; η² [95% CI] = 0.00 [0.00, 0.02]).

Figure 3.

Differences in patient-reported cognitive difficulty across levels of cognitive impairment in matched samples completing the PDQ or MSCS.

Construct validity: representativeness of scale to cognitive difficulties in MS

To evaluate whether the scale overlooked any prevalent cognitive difficulties, we reviewed open-ended descriptions of cognitive difficulties by 464 patients (aforementioned 163 responses used to develop language items, plus an additional 301 responses from subsequent patients). As detailed within the online Supplement, no other prevalent cognitive difficulties were identified beyond attention/executive function, working memory, expressive language, and episodic memory.

Responsiveness to change in objective cognitive performance

To examine responsiveness of the MSCS to change in cognition over time (relative to the PDQ), retrospective chart review identified a consecutive sample of 120 patients with annual follow-ups who completed the full PDQ at their first visit (V1) and second visit (V2), and completed the MSCS at their third visit (V3; characteristics at V1: mean [SD] age 45.5 [11.7] years; 72.5% female; 65.0% non-Latino White; 23.3% progressive course; median [IQR] 6.5 [1.5, 13.0] years since diagnosis). Patients completed aforementioned cognitive tasks (SDMT, HVLT, CANTAB PAL) at each visit (alternate forms used as appropriate); performance was converted to normative z-scores and averaged into composite z-scores for each time point. Cognitive change scores were derived as V2 minus V1, and V3 minus V1. Changes in PDQ and MSCS raw scores were derived as V2 minus V1, and V3 minus V1, respectively (V1 MSCS derived from its eight items within larger item pool). All values were winsorized (1.5*IQR) to avoid undue impact of outliers; there was no skewness (all <±0.45) or kurtosis (all <±0.31) for any values. Dependent t-tests showed no difference in cognitive change (mean [SD]) between V2-V1 (0.07 [0.45]) versus V3-V1 (0.10 [0.47]; t[119] = 0.56, p = 0.575, d = 0.05), and no difference in patient-reported cognitive change on PDQ V2-V1 (0.06 [0.45]) versus MSCS V3-V1 (0.07 [0.54]; t[119] = 0.16, p = 0.874, d = 0.01). Pearson correlations examined associations between (a) V2-V1 PDQ change and cognitive change, and V3-V1 MSCS change and cognitive change. Cognitive change was not related to PDQ change (r = −0.14, p = 0.115), but it was related to MSCS change (r = −0.23, p = 0.011); findings were maintained when re-analyzed as partial correlations adjusting for changes in mood (MHI-5; PDQ: r = −0.18, p = 0.057; MSCS: r = −0.21, p = 0.025).

Finally, ANOVAs tested differences in PDQ change and MSCS change across patients classified as cognitively worsened, stable, or improved on tasks (SDMT, HVLT, PAL) for V2-V1 and V3-V1, respectively (worsened: ⩾1 SD lower on ⩾1 task; improved: ⩾1 SD higher on ⩾1 task; stable: <1 SD change on all tasks). Eight patients (four in each comparison) were excluded due to mixed cognitive change (i.e. worsened on one task, improved on another). As shown (Figure 4, Table 7), MSCS change (F[2, 109] = 3.97, p = 0.022, η_p² = 0.07) but not PDQ change (F[2, 109] = 0.39, p = 0.387, η_p² = 0.02) differed across cognitive change groups; this pattern remained even when re-analyzed with analyses of covariance (ANCOVAs) adjusting for changes in mood (MHI-5; MSCS: F[2, 108] = 4.10, p = 0.019, η_p² = 0.07; PDQ: F[2, 108]= 1.53, p = 0.221, η_p² = 0.03).

Figure 4.

Differences in patient-reported cognitive change on MSCS versus PDQ across levels of objective cognitive change.

Table 7.

MSCS and PDQ change scores by cognitive performance change.

	Mean [95% CI]
	Unadjusted for mood change	Adjusted for mood change
MSCS
Worsened	0.38 [00.14, 0.62]	0.38 [0.14, 0.62]
Stable	0.01 [−0.12, 0.13]	0.00 [−0.13, 0.12]
Improved	0.00 [−0.19, 0.20]	0.03 [−0.17, 0.23]
PDQ
Worsened	0.18 [−0.02, 0.39]	0.22 [0.01, 0.42]
Stable	0.02 [−0.10, 0.14]	0.02 [−0.10, 0.14]
Improved	0.04 [−0.11, 0.19]	0.02 [−0.13, 0.16]

Discussion

We report good reliability and validity for the Multiple Sclerosis Cognitive Scale (MSCS), a new eight-item patient-report cognitive questionnaire with four factor analytically derived subscales (executive/speed, working memory, expressive language, episodic memory). The MSCS is provided in Appendix 1. MSCS showed medium-sized cross-sectional and longitudinal relationships to objective general cognitive impairment and cognitive changes, which remained statistically significant even when adjusting for mood. In contrast, the traditional PDQ was unrelated to objective cross-sectional impairment or longitudinal change with and without adjusting for mood, despite having 2.5 times as many items as the MSCS. It may be that patients respond more thoughtfully when there are fewer items and that MSCS items better represent the cognitive problems experienced by persons living with MS, especially given assessment of expressive language difficulty (which is missing from existing patient-report cognitive scales).

Development of the MSCS aligns with the Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN Taxonomy of Measurement Properties).¹³ Herein, we have demonstrated the MSCS is reliable as indicated by good test–retest reliability and good internal consistency, established using EFA and verified independently using PCA. The full-scale MSCS score is also a construct valid instrument as evidenced by its link with objective cognitive performance and responsiveness to change in cognitive performance over time. The next step is to evaluate additional measurement properties of the scale. For example, cross-cultural validity and content validity of the subscales (i.e. relationships with objective estimates of cognitive performance in these domains). Published norms were used to quantify cognitive impairment and the design was retrospective, introducing limitations that could be overcome with a matched healthy control group and prospective design in future work.

The MSCS holds promise as a brief, reliable, psychometrically robust self-report scale with good links to objective cognitive impairment and responsiveness to cognitive change. Widespread adoption of valid and reliable patient-reported outcomes in research and clinical settings will help advance the field toward meaningful interventions.

Supplemental Material

sj-docx-1-msj-10.1177_13524585241309805 – Supplemental material for Multiple Sclerosis Cognitive Scale (MSCS): A brief psychometrically robust metric of patient-reported cognitive difficulty

Supplemental material, sj-docx-1-msj-10.1177_13524585241309805 for Multiple Sclerosis Cognitive Scale (MSCS): A brief psychometrically robust metric of patient-reported cognitive difficulty by James F Sumowski and Joshua Sandry in Multiple Sclerosis Journal

Footnotes

Appendix 1: Multiple Sclerosis Cognitive Scale (MSCS)

Please check the box to indicate how frequently during the past month you experienced:

Acknowledgements

The authors thank the patients, faculty, and staff of the Corinne Goldsmith Dickinson Center for Multiple Sclerosis. Special thanks to Jordyn Anderson, PsyD, Hanaan Bing-Canar, PhD, and Emily Dvorak, PhD for their work as independent raters of responses.

Data Availability

Data are available from the corresponding author upon request.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was funded in part by the National Institute of Child Health and Development (NICHD) within the National Institutes of Health (NIH; R01 HD082176 to JFS).

ORCID iDs

James F Sumowski

Joshua Sandry

Supplemental Material

Supplemental material for this article is available online.

References

Sumowski

Benedict

Enzinger

, et al. Cognition in multiple sclerosis: State of the field and priorities for the future. Neurology 2018; 90(6): 278–288.

Sullivan

Edgley

Dehoux

. A survey of multiple sclerosis: I. Perceived cognitive problems and compensatory strategy use. Can J Rehabil 1990; 4: 99–105.

Brandstadter

Fabian

Leavitt

, et al. Word-finding difficulty is a prevalent disease-related deficit in early multiple sclerosis. Mult Scler 2020; 26(13): 1752–1764.

Benedict

Munschauer

Linn

, et al. Screening for multiple sclerosis cognitive impairment using a self-administered 15-item questionnaire. Mult Scler 2003; 9(1): 95–101.

Medina

Torres

Alvarez

, et al. Patient-reported outcomes in multiple sclerosis: Validation of the Quality of Life in Neurological Disorders (Neuro-QoLTM) short forms. Mult Scler J Exp Transl Clin 2019; 5(4): 2055217319885986.

Katz Sand

Levy

Fitzgerald

, et al. Mediterranean diet is linked to less objective disability in multiple sclerosis. Mult Scler 2023; 29(2): 248–260.

Revelle

. Package “psych.” The Comprehensive R Archive Network, 2015. https://CRAN.R-project.org/package=psych

Qin

Nelson

McLeod

, et al. Assessing test-retest reliability of patient-reported outcome measures using intraclass correlation coefficients: Recommendations for selecting and documenting the analytical formula. Qual Life Res 2019; 28(4): 1029–1033.

Langdon

Amato

Boringa

, et al. Recommendations for a Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS). Mult Scler 2012; 18(6): 891–898.

10.

Smith

. Symbol digit modalities test (SDMT) manual (revised). Los Angeles, CA: Western Psychological Services, 1982.

11.

Brandt

Benedict

. Hopkins verbal learning test, Revised. Lutz, FL: Psychological Assessment Resources, 2001.

12.

Barnett

Blackwell

Sahakian

, et al. The Paired Associates Learning (PAL) test: 30 years of CANTAB translational neuroscience from laboratory to bedside in dementia research. Curr Top Behav Neurosci 2016; 28: 449–474.

13.

Mokkink

Terwee

Patrick

, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol 2010; 63(7): 737–745.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB