Obtaining Patient Priorities in a Multiple Sclerosis Comprehensive Care Center: Beyond Patient-Reported Outcomes

Abstract

Background:

In order to provide patient center care, our multiple sclerosis (MS) clinic assesses patient concerns before clinical encounters, first by asking the optional qualitative question “What is the most important thing you what your health-care provider to know today” (most important concern of the patient [MIPC]) and then completing quantitative patient-reported outcome measures (PROMs) including Quality of Life in Neurological Disorders (Neuro-QoL). Both sets of questions are designed to facilitate encounters that address patients’ values and preferences.

Objective:

Determine whether the qualitative MIPC responses provided unique information not included in PROMs or clinical assessments.

Methods:

We randomly selected 400 first-time MIPC responders and 400 first-time MIPC nonresponders from 2788 participants in our database. We categorized MIPC responses by content and number of unique concerns and appended them to the Neuro-QoL framework. Nonresponders were compared to those who provided 1 and 2 or more responses.

Results:

Several MIPCs MS symptoms categories were added to the Neuro-QoL Physical domain. Most important concern of the patients work and cost-of-care categories were added to the Social Domain. Domains regarding treatment satisfaction and disease management were added. Two hundred thirty (58%) MIPC respondents reported 1 concern, 140 (35%) expressed 2 to 6 concerns, and 30 (7%) reported MS-unrelated concerns and not analyzed. Physical symptoms were the most common MIPC (69.9%). Respondents with more concerns were more likely African American, lacked private insurance, and worse disability.

Conclusions:

Importantly, MIPC responders described idiosyncratic symptoms, disease management, and social concerns not included in the PROMS, suggesting the MIPC question offered patients a unique opportunity to share specific concerns with their providers.

Keywords

quantitative methods relationships in health care patient perspectives/narratives communication clinician–patient relationship learning health system

Background

Patient-centered care, as is advocated for by the Institute of Medicine (1) and chronic disease management (2), both of which emphasize patient engagement (3) in their care and, more specifically, patient activation (4) to manage their own health care, is central to our treatment approach to managing multiple sclerosis (MS). Central to that engagement is patients’ direct input about their concerns. While there are relatively high correlations between patient-reported outcome measures (PROMs) and clinical assessment of walking speed and manual dexterity (5), research shows that patients and clinicians differ in what they consider to be the most important MS symptoms and how they prioritize treatments (6,7). Moreover, the relevance of using PROMs in MS clinical practice is well recognized because many of the most disabling symptoms including fatigue and pain do not have objective clinical measures that assess them (8,9).

Our MS Center has established a learning health system (LHS) model to advance our ongoing care and research goals. The platform we developed to support this LHS is called the Multiple Sclerosis Performance Test (MSPT). This platform allows clinicians and researchers to measure and address the spectrum of clinical questions across diverse MS patients. The goal of the LHS is to accumulate large-scale clinical and objective data to develop more effective patient-specific treatment approaches. This model will transform treatment approaches from one in which the “typical patient with MS” is considered to one in which a precise phenotype of the patient will be treated based on effectiveness observed in patients of a similar phenotype. This approach will assure that unique groups of patients get the best care to meet their needs. The benefit of an LHS can have an immediate impact on patient engagement and health-care delivery when they provide self-reported data that are incorporated into the clinical encounter. The basis of an LHS is the structured and systematic collection of consistent data elements that are typically determined by members of the health-care team (10). While constructing this system, the development team, with input from our Voice of the Patient Advisory Council, and in response to recommendations by the MS in the 21st Century Steering Group (11), wanted to ensure that we offered patients an opportunity to communicate their goals for any given clinical encounter.

The MSPT is an iPad application that consists of 6 assessment modules. Two modules deliver health-related questionnaires related to QoL and self-reported disease history. The MyHealth module contains the only free-text question: “What is the most important thing you want your health care provider to know today.” This question was included to better understand the most important concern of the patient (MIPC). A second MSPT module includes PROMs including the Quality of Life in Neurological Disorders Measurement System (Neuro-QoL) (12,13), which includes 12 domains and is administered using computer adaptive testing (14). The 4 remaining modules are iPad-based neuro-performance tests (NPTs) that are self-administered analogs of common clinician-assessed measures of MS functional domains: walking speed, manual dexterity, cognition, and vision, that are included in the Multiple Sclerosis Functional Composite (15).

We conducted this qualitative study to determine whether there was unique information to be gleaned from the MIPC response (MIPC-R) compared to the information gathered from the Neuro-QoL other PROMs and clinical data. A secondary aim was to evaluate whether the number of concerns included MIPC-R was linked to certain patient characteristics.

Methods

Design, Setting, and Study Population

We retrospectively analyzed data from the first patient visit for all patients included in the MSPT database between September 2015 and July 2017. As indicated, all patients had the opportunity to enter text in the MIPC field before completing the rest of the MSPT which included a maximum of 48 questions. There were 2788 patients included in the initial visit data set, and 922 (33%) patients provided a MIPC-R. We randomly selected 400 responder patients and analyzed their MIPC-Rs as well as their personal and disease characteristics. Next, 400 patients who did not provide any response to the MIPC were randomly selected as a comparator group. The responders were subsequently grouped according to the number of concerns they voiced for the MIPC question. This study was conducted after Cleveland Clinic Institutional Review Board approval.

Creating MIPC Domains, Groups, Subgroups, and Variables

A coding system was developed to categorize and subcategorize the MIPC responses into discrete variables. This involved developing a theoretical framework of potential MIPC responses based on the Neuro-QoL domains (16), see Figure 1.

Figure 1.

Constructed Categories of Verbatim Responses to the Most Important Thing Question.

The study team used the Neuro-QoL framework as our starting point, as it was designed to meet the concerns of patients with a variety of neurological conditions, including domains generally of concern to the MS patient group. We divided the MIPC responder group into 2 halves. One of the authors (B.M.) conducted a detailed review of the first half of the sample to identify groups and subgroups of the higher order domains and additional categories not previously captured by the framework. With the second half of the sample, we assessed whether there was a comprehensive set of groups and subgroups based on saturation of themes. Finally, we mapped each MIPC response to the final list of subgroups. A single response could be mapped to multiple subgroups. For example, the following MIPC-R, “I’m having a lot of difficulty sleeping because of pain, and it is deeply effecting my ability to think and perform duties at my job and in daily activities,” was mapped to the pain and sleep subgroups under physical symptoms, the confusion subgroup under cognitive concerns, and the community resources and social support subgroup under social concerns.

Study Measures

Demographics and disease characteristics

Demographics (age, sex, race, educational level, and employment status) and patient-reported disease characteristics (disease course and disease duration) were collected using a questionnaire included in the first module of the MSPT (My Health) that patients were required to complete. Disease course refers to the pattern of disease progression an individual experiences. There are 3 categories of MS progression. A relapsing-remitting (RR) disease course is characterized by clearly defined attacks of new or increasing neurologic symptoms then followed by periods of partial or complete recovery. A secondary progressive (SP) disease course initially follows a RR course that eventually transitions to a SP course, in which there is a progressive worsening of neurologic function (accumulation of disability) over time. A primary progressive (PP) disease course is characterized by worsening neurologic function from the onset of symptoms, without early relapses or remissions. For this analysis, individuals with SP and PP were pooled.

Patient-Reported Outcome Measures

Quality of Life in Neurological Disorders (12)

Neuro-QoL is a patient-reported outcome assessment system developed and tested specifically for use in populations with neurological conditions. It includes scales that assess unique domains of physical, emotional, and social well-being (17). It has been validated for use in the population with MS (13). Scores are converted to a T-score metric with a mean score of 50, with each 10-point increase or decrease in score, considered a standard deviation of change. Higher scores indicate better functioning for fine motor control, mobility, cognitive function, social participation, social satisfaction, and emotional well-being. Higher scores indicate worse symptom severity for fatigue, sleep, depression, anxiety, and emotional dyscontrol. The item banks were administered using computer-adapted testing.

Patient Determined Disease Steps(18,19)

(PDDS): The PDDS (19) is a PROM used to assess walking and general levels of disability for individuals with MS. It is based on the Expanded Disability Status Scale which is the most commonly used measures to clinically rate MS disability (20). Scores range from “0” (normal functioning at 1-point intervals) to “8” (bedridden). It was administered as a component of the computerized MyHealth Assessment.

Patient Health Questionnaire-9(21)

(PHQ-9); The PHQ-9 is a self-administered screening measure of depression that assess the 9 screening questions included in the Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition). Scores of 5, 10, 15, and 20 represented mild, moderate, moderately severe, and severe depression, respectively (21). It was administered as a component of the computerized MyHealth Assessment.

Patient-Reported Outcomes Measurement Information System

(PROMIS) Global Health (22) (PROMIS-10): The PROMIS-10 is a short form component of the Patient Reported Outcomes System that assesses general domains of health and functioning including overall physical health, mental health, social health, pain, fatigue, and overall perceived QoL (23). Items may be examined individually or as two 4-item summary scores of Global Physical Health and Global Mental Health. Scores are converted to a T-score metric with a mean score of 50. It was administered as a component of the MyHealth Assessment.

Neuro-performance tests

Four NPTs were included in the MSPT, each forming a separate module:

– Walking Speed Test, an iPad-adapted version of the Timed-25 Foot Walk (15).

– Manual Dexterity Test (MDT), an iPad-adapted version of the 9-Hole Peg Test (15).

– Processing Speed Test, an iPad-adapted version of the Symbol Digit Modalities Test (24).

– Contrast Sensitivity Test (CST), an iPad-adapted version of the Sloan Low Contrast Letter Acuity (25).

Data Analysis Plan

Patients were categorized by the number of responses they provided to the MIPC. These categories included no response, 1 response, or 2 or more responses. Descriptive statistics, including mean and standard deviations, and number and column percentage, were used to compare the patient demographics, clinical characteristics, structured patient-reported outcomes, and MSPT performance measures among respondent groups. Values of P for differences across the groups for each of these characteristics were calculated for categorical variables using the χ² test, while differences for continuous, quantitative variables used the Cochran-Armitage test. All analyses were performed using SAS version 9.4. Two-sided P values are presented, and P < .05 is considered as statistically significant.

Results

There were 2788 patients included in the initial visit data set. The majority 1866 (67%) of the patients did not respond to the MIPC question, while 922 (33%) did respond. The CONSORT chart is reported in Figure 2.

Figure 2.

CONSORT Enrollment.

Categories of Reported Concerns

Based on the MIPC analysis, we added additional categories of responses Neuro-QoL framework. In addition to the Physical, Mental, and Social domains, we added 2 additional domains, “Other” and “Non-Clinical.” The Other domain included responses that were related to treatment concerns including use of disease-modifying therapies and positive statements about satisfaction with care. The “Non-Clinical” domain included responses unrelated to MS care and were excluded from subsequent analysis. At the group level, Wellness was added to the Physical domain groups. Additional groups that were added to the Social domain included financial concerns, care needs, and community resources. The majority of MIPC-Rs at the subgroup level addressed specific symptoms that were not included in the Neuro-QoL framework, including bowel and bladder symptoms (both have had Neuro-QoL item banks developed but not calibrated), disturbing sensations, strength, walking, and wellness. There were a number of concerns that emerged in the “Social” category that were not included in the Neuro-QoL framework, including “Insurance Concerns,” “Cost of Care,” and “Social Support.”

Of the 400 randomly selected responders, the number of MIPC responses ranged from 1 to 6 per individual respondent; 230 (58%) individuals reported 1 MIPC concern, and 140 (35%) individuals reported 2 or more MIPCs. The vast majority of the MIPCs related to physical issues, particularly pain and disturbing sensations, followed, in order, by concerns about disease-modifying therapies, mental, and social concerns. As demonstrated in Figure 3, symptoms were the most commonly reported concern in the physical domain as were cognition in the mental domain, financial issues in the social domain, and disease management in the “other” domain.

Figure 3.

Percentage of Patients who Endorsed a Specific Verbatim Response Domain or Group.

Examples of Most Important Patient Concern Responses

The majority of the MIPC-Rs were straightforward and included easily interpreted, discrete, issues: “Pain management,” “Terrible Tremors,” “I am very emotional,” “I think my current medicine is working,” “I need a new placard for parking.” Many included positive statements about successful disease management: “I feel great and have not had any problems with my MS for the past year!” and “I am doing well.” In other cases, the MIPC-R indicated concern about more than one aspect of a given function: “my gait and balance are getting worse” or include an example of the consequence of a symptom change. and “I have been having severe dizziness that is affecting my daily life. I have not been able to go to work or class since Monday.” Some MIPC-Rs were very complex and included a range of issues reflecting the consequences of life with MS. Two such examples are “I feel like things have been getting worse. I have a hard time being the mother that I want to be to my children and that is very hard to admit. I want to start on my medication that we discussed at my last visit and get my MS under control” and “I would like to qualify for a wheelchair. I am tired of falling when I go out in public I want my independence back”. Occasionally, as in the following example, the MIPC-R was marginally MS-related: “My roommate is a schizophrenic and I do not have the strength to be his caretaker. It seems to be making my MS worse.”

Characteristics Of individuals Who Reported “No,” “1,” or “2 or more” MIPC Responses

Table 1 presents demographic and disease characteristics varied among individuals who reported no, 1, or 2 or more MIPC-Rs at a single visit. Characteristics that were shown to statistically trend with reporting 1 or more MIPC-R included being African American or of mixed race, lacking private insurance, having a PP disease course, being disabled, and living with assistance. Consistent with those associations, having worse upper extremity functioning as measured by the MDT, higher PDDS scores (higher scores indicate worse functioning), and worse depression, as measured by the PHQ-9, were also statistically trended to be associated with responding to the MIPC question (Table 1).

Table 1.

Comparison of Groups Who Provided No MIPC Responses to Those Who Provided One Concern and Multiple Concerns.

Factor^a	No Concern, N = 400		One Concern, N = 230		Two or More Concerns, N = 140		P Value
Factor^a	n	Summary	n	Summary	n	Summary	P Value
Age, years	400	46.5 ± 10.8	212	46.2 ± 10.1	140	47.1 ± 10.7	.52^b
Race	400		211		140		<.001^c
White		350 (87.5)		153 (72.5)		110 (78.6)
Black		32 (8.0)		48 (22.7)		24 (17.1)
Other/Mixed		18 (4.5)		10 (4.7)		6 (4.3)
Gender, female	400	274 (68.5)	212	140 (66.0)	140	112 (80.0)	.04^b
Education	372		189		133		.10^c
High school or lower		123 (33.1)		53 (28.0)		41 (30.8)
Associate/college		172 (46.2)		89 (47.1)		74 (55.6)
Master or higher		77 (20.7)		47 (24.9)		18 (13.5)
Private insurance	394	259 (65.7)	211	119 (56.4)	140	70 (50.0)	.005^b
Duration from diagnosis, years	383	12.1 ± 9.9	209	11.8 ± 9.1	138	11.6 ± 9.8	.67^b
MS course, PMS	348	99 (28.4)	193	71 (36.8)	135	65 (48.1)	<.0001^b
Disabled	396	94 (23.7)	210	73 (34.8)	140	51 (36.4)	.001^b
Living with assistance	398	61 (15.3)	211	43 (20.4)	140	34 (24.3)	.01^b
Neuro performance tests
CST, No. correct	196	32.4 ± 13.3	81	31.7 ± 12.5	57	29.4 ± 12.3	.13^b
PST, No. correct	339	48.2 ± 12.7	160	46.7 ± 12.9	116	44.3 ± 12.8	.005^b
WST, sec	333	7.7 ± 4.8	160	7.6 ± 2.7	111	8.8 ± 4.2	.06^b
MDT, dominant hand, sec	350	28.2 ± 8.9	174	29.0 ± 9.0	122	31.8 ± 10.7	.0002^b
Patient-reported outcome measures
PDDS	368	2.8 ± 2.1	188	3.2 ± 2.1	133	3.6 ± 2.1	.0003^b
PHQ-9	226	5.8 ± 5.6	212	7.3 ± 6.0	140	8.7 ± 6.4	<.0001^b
PROMIS-10
Mental T-score	226	43.9 ± 8.6	212	43.8 ± 9.2	140	43.4 ± 7.4	.59^b
Physical T-score	225	39.9 ± 4.9	206	40.1 ± 4.6	135	39.4 ± 5.2	.42^b

Abbreviations: CST, contrast sensitivity test; MDT, manual dexterity test; MIPC, most important concern of the patient; MS, multiple sclerosis; PDDS, patient-determined disease steps; PHQ-9, patient health questionaire-9; PROMIS-10, Patient-Reported Outcomes Measurement Information System; PMS, progressive MS; PST, processing speed test; SD, standard deviation; MIPC, most important concern of the patient; WST, walking speed test.

^aValues presented as Mean ± SD or n (%).

^bP value = χ² test.

^cP value, t = Cochran-Armitage test for trend.

We also assessed the differences in Neuro-QoL scores for those who reported the different numbers of MIPC-Rs (Figure 4). Respondents who reported no MIPC-Rs had Neuro-QoL scores that indicated better functioning and fewer symptoms than those reporting any MIPC-Rs, while those who reported 2 or more MIPC-R concerns had scores indicating poorer functioning and greater symptoms than the other groups (P < .05) with 2 exceptions. Scores for positive affect and well-being followed the same pattern but did not reach significance. Those who scored the highest on social satisfaction were more likely to report 1 MIPC-R concern but were closely followed by those who reported no MIPC-R concern. Conversely, and as expected, individuals who reported the highest symptom severity were most likely to report more than 1 MIPC-R at a single visit (P <.05).

Figure 4.

Neuro-QoL T-Scores by Number of MIPC-R Issues per Patient.

Discussion

In order to enhance patient-centered care, we implemented a routinized data collection system to collect standardized NPTs and PROMs. With this study, we evaluated if additional information about patients concerns can be obtained prior to the clinical encounter when patients are asked the specific open-ended question “What is the most important thing you want your clinician to know today (Most Important Patient Concern [MIPC]).”

Thirty-three percent of the patients responded to the MIPC question and expressed a wide range of issues. The responses generally fit into the constructs of physical, mental, and social concerns and were often specific concerns that were not captured by the structured PROM. This is not offered as a critique of the Neuro-QoL measures, which were developed to address the most common aspects of QoL that are affected by people with chronic neurological diseases. Rather, it demonstrates that in clinical settings, open-ended questions such as the MIPC can provide a valuable complement to structured PROMs. It is important to ask patients to name their priorities for a given visit in addition to collecting structured data that allow us to monitor patients both individually and as groups over time. By far, the majority of concerns expressed were of a physical nature, several of which are included in Neuro-QoL, as there are calibrated item banks for lower extremity, upper extremity, sleep, and fatigue. Nonetheless, these structured measures do not “call out” issues the patient wants to address at any given encounter. It was of interest to note that although social concerns were infrequently offered, issues of insurance, cost of care, employment, and the need for social support, topics not often addressed in neurological encounters, were foremost on the minds of some patients. The number of responses include in the “Other” domain is an indication of how unique those responses were. These analyses suggest that there are no indicators to help practitioners to anticipate what concerns an individual will present as most important. However, the data do indicate that those individuals who present with complex, multifaceted concerns are among the more physically disabled and depressed and generally have worse Neuro-QoL scores so that beyond what the patient presents as their important concern, these aspects of a person’s functioning should be explored more closely.

Limitations to this study include that given the large number of individuals in our database, we randomly selected for analysis an equal number of responders and nonresponders to the MIPC. Although this does not reflect the proportions of the 2 groups, we believe that the sample sizes allow us to generalize these results to the overall sample.

Incorporating the MIPC question was an early step in using our LHS to better understand our patients’ concerns and help them develop strategies to manage those concerns as a means of actively participating in their care. In order to extend the level of patient engagement in our center, a next step we are planning is to better gauge an individual’s level of “patient activation” (26), using a measure such as The Patient Activations Measure (PAM). Understanding patients’ level of activation for engagement in their care would allow our clinicians introduce interventions that would increase patients’ appreciation of their role in their care, increase knowledge and confidence in managing their health, engage them in taking action to improve health, and to help them sustain that empowerment. Just as we have implemented the MIPC and well-validated PROMs measures to obtain structured and consistent patient self-report of their functioning and well-being, the inclusion of a validated patient activation measure such as the PAM would be an important assessment measure to include and use in our LHS.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Committee on Quality of Health Care in America and Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001.

Wagner

Bennett

Austin

Greene

Schaefer

Vonkorff

. Finding common ground: patient-centeredness and evidence-based chronic illness care. J Altern Complement Med. 2005;11:S7–15.

Boissy

. Patient engagement versus patient experience. In: Catalyst

, ed. Patient Engagement II. https://catalyst.nejm.org/patient-engagement-vs-patient-experience/ (2017, accessed 8 July 2019).

Roundtable on Value and Science-Driven Health Care, Institute of Medicine. Partnering with Patients to Drive Shared Decisions, Better Value, and Care Improvement: Workshop Proceedings. Washington (DC): National Academies Press (US) Copyright 2014 by the National Academy of Sciences. All rights reserved., 2014.

Miller

Baier

Cutter

. Reducing the multiple sclerosis quality of life inventory using longitudinal data. Mult Scler. 2004;10:S128–S128.

Rothwell

McDowell

Wong

Dorman

. Doctors and patients don’t agree: cross sectional study of patients’ and doctors’ perceptions and assessments of disability in multiple sclerosis. BMJ (Clinical Research Ed). 1997;314:1580–3.

Khan

McPhail

Brand

Turner-Stokes

Kilpatrick

. Multiple sclerosis: disability profile and quality of life in an Australian community cohort. Int J Rehabil Res. 2006;29:87–96.

Nowinski

Miller

Cella

. Evolution of patient-reported outcomes and their role in multiple sclerosis clinical trials. Neurotherapeutics. 2017;14:934–44.

Gerhardt

Mara

Kudel

Morgan

Schoettker

Napora

, et al. Systemwide implementation of patient-reported outcomes in routine clinical care at a children’s hospital. Jt Comm J Qual Patient Saf. 2018;44:441–53.

10.

Lessard

Michalowski

Fung-Kee-Fung

Jones

Grudniewicz

. Architectural frameworks: defining the structures for implementing learning health systems. Implement Sci. 2017;12:78.

11.

Members of the MS in the 21st Century Steering Group; Rieckmann

Centonze

Elovaara

Giovannoni

Havrdová

Kesselring

, et al. Unmet needs, burden of treatment, and patient engagement in multiple sclerosis: a combined perspective from the MS in the 21st Century Steering Group. Mult Scler Relat Disord. 2018;19:153–60.

12.

Cella

Lai

Nowinski

Victorson

Peterman

Miller

, et al. Neuro-QOL: brief measures of health-related quality of life for clinical research in neurology. Neurology. 2012;78:1860–7.

13.

Miller

Bethoux

Victorson

Nowinski

Buono

Lai

, et al. Validating Neuro-QoL short forms and targeted scales with people who have multiple sclerosis. Mult Scler. 2016;22:830–41.

14.

Gershon

. Computer adaptive testing. J Appl Meas. 2005;6:109–27.

15.

Fischer

Jak

Knicker

Rudick

. Administration and Scoring Manual for the Multiple Sclerosis Functional Composite Measure (MSFC). New York, NY: National Multiple Sclerosis Society, Demos; 1999.

16.

Neuro QoL Reports and Manuals [online]. 2015. http://www.neuroqol.org/Resources/Neuro-QoLReports-Manuals/Pages/default.aspx.

17.

Gershon

Lai

Bode

Choi

Moy

Bleck

, et al. Neuro-QOL: quality of life item banks for adults with neurological disorders: item development and calibrations based upon clinical and general population testing. Qual Life Res. 2012;21:475–86.

18.

Hohol

Orav

Weiner

. Disease steps in multiple sclerosis: a simple approach to evaluate disease progression. Neurology. 1995;45:251–5.

19.

Hohol

Orav

Weiner

. Disease steps in multiple sclerosis: a longitudinal study comparing disease steps and EDSS to evaluate disease progression. Mult Scler. 1999;5:349–54.

20.

Kurtzke

. Rating neurologic impairment in multiple sclerosis: an expanded disability status scale (EDSS). Neurology. 1983;33:1444–52.

21.

Kroenke

Spitzer

Williams

. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16:606–13.

22.

Cella

Riley

Stone

Rothrock

Reeve

Yount

, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. J Clin Epidemiol. 2010;63:1179–94.

23.

Hays

Bjorner

Revicki

Spritzer

Cella

. Development of Physical and Mental Health Summary Scores from the Patient Reported Outcomes Measurement Information System (PROMIS) Global Items. Unpublished 2008.

24.

Rao, S.M. and the Cognitive Function Study Group of the National Multiple Sclerosis Society. (1990) A manual for brief repeatable battery of the neuropsychological tests in multiple sclerosis. Milwaukee, WI: Medical College of Wisconsin.

25.

Balcer

Raynowska

Nolan

Galetta

Kapoor

Benedict

, et al. Validity of low-contrast letter acuity as a visual performance outcome measure for multiple sclerosis. Mult Scler. 2017;23:734–47.

26.

Green

Perrin

Polen

Leo

Hibbard

Tusler

. Development of the patient activation measure for mental health. Adm Policy Ment Health. 2010;37:327–33.