Development and usability testing of your MS questionnaire: A patient-based digital tool to monitor symptoms of multiple sclerosis

Abstract

Objectives

We describe the development of Your Multiple Sclerosis Questionnaire and present the real-world usability testing results of Your Multiple Sclerosis Questionnaire.

Methods

The Your Multiple Sclerosis Questionnaire tool was developed in four stages to collect feedback from people living with MS (plwMS), patient organizations, and clinicians on content, format, and applicability. To assess its usability, 13 clinicians across 7 countries completed an online survey after using the tool with plwMS in a total of 261 consultations from September, 2020 to July, 2021.

Results

The initial Your Multiple Sclerosis Questionnaire version was based on findings from previous research developing MSProDiscuss™, a clinician-completed tool. Subsequently, insights from plwMS obtained during cognitive debriefing, patient councils and advisory boards led to changes including the addition of mood and sexual problems and the definition of relapse. All 13 clinicians completed the individual survey, whereas 10 clinicians completed the final survey. Clinicians “strongly agreed” or “agreed” that Your Multiple Sclerosis Questionnaire was easy to use and understand (98.5%; 257/261 patient consultations). The clinicians were willing to use the tool again with the same patient (98.1%; 256/261 patient consultations). All clinicians who completed the final survey (100%; 10/10) reported the tool to have a positive influence on their clinical practice, helped patients engage with their MS, facilitated discussion with patients, and complemented neurological assessment.

Conclusion

Your Multiple Sclerosis Questionnaire benefits both plwMS and clinicians by facilitating a structured discussion and engaging the plwMS to self-monitor and self-manage. Your Multiple Sclerosis Questionnaire is compatible with telemedicine practice and integration of the tool into electronic health records would enable tracking of the disease evolution and individual monitoring of MS symptoms over time.

Keywords

Patient-completed tool multiple sclerosis monitoring people living with MS telemedicine < General

Introduction

Multiple sclerosis (MS) is a chronic neurological condition that mostly affects young adults. The mean age at diagnosis is approximately 30 years.¹ As MS is a progressive illness, most people living with MS (plwMS) gradually transition from relapsing-remitting MS (RRMS) to secondary progressive MS (SPMS) within a median time of 10–23 years.^2,3 MS is a heterogeneous disease and can manifest differently for each person, with diverse clinical features such as physical disability, cognitive impairment, visual loss, sensory loss, bladder, bowel and sexual dysfunction, fatigue, and mood disorders.^4,5 Given both the heterogeneous and progressive nature of MS, clinical measures to monitor changes including worsening and progression are important tools for clinicians and plwMS alike.⁶

The Expanded Disability Status Scale (EDSS) is a widely accepted clinical tool to assess disease progression but has low sensitivity at higher values: EDSS values from 4.5 to 6 mainly reflect walking impairment, while values >6 focus on general quality of life (QoL) and self-care.⁷ Thus, important functional measures including cognitive skills, visual function, arm function, and bowel and bladder function are less well captured.⁸ To assess these functions, additional instruments have been developed, such as the Symbol Digit Modality Test (SDMT) for cognitive processing speed, and the Nine-Hole Peg Test for arm and hand dexterity.^9,10 Composite measures are being used to capture a more complete picture of the illness in plwMS. However, they are often limited to three clinical measures and may miss subtle changes and especially those of importance to plwMS.^11,12 Thus, more sensitive outcome measures are needed, that can capture individual perceptions of plwMS.⁷

The inclusion of patient-reported outcomes (PRO) for MS disease monitoring is gaining acceptance as it allows patients to provide crucial information to clinicians on how they experience the impact of disease, the impact of treatment and what issues most concern them.^13–15 There are many available PRO instruments designed to measure factors impacting plwMS, including symptoms, activity limitations, fatigue, QoL, and health-related QoL (HRQoL).¹⁵ However, not all instruments were developed with input from plwMS, which is however regarded as essential to the PRO development process.¹⁶ Taking this important aspect of PRO instrument development into consideration, the Your Multiple Sclerosis Questionnaire (YMSQ; www.yourms.com) digital tool was co-developed with plwMS, patient advocacy groups, and clinicians.¹⁷

Digitalization in healthcare facilitates improved monitoring and treatment of chronic diseases like MS, which require continuous long-term monitoring and individualized treatment.¹⁸ Individual use of social eHealth interventions supports patients in tracking their daily activities and disease progression and helps to disseminate unbiased and scientifically accurate information to patients. By linking patients to health management systems, advanced integrated eHealth solutions not only improve patient-clinician communication, but are also useful in collecting and interpreting patient data.¹⁹

As a 20-question digital tool, YMSQ was developed to facilitate and standardize the discussion between clinicians and plwMS and to collect the plwMS's perspective on changes in MS symptoms, relapses, and impact experienced by them on daily living activities in the previous 6 months. This study describes the stages involved in the development of YMSQ followed by the results of real-world usability testing of YMSQ conducted across 7 countries.

Methods

YMSQ development

Overview of stages of YMSQ development

YMSQ was developed in four stages to collect feedback from plwMS, patient organizations, and clinicians on the content, format, and applicability as described below: (Figure 1).

Figure 1.

YMSQ development stages. Note: Stage 4: Participants in international MS councils and patient advisory board were from Australia, Belgium, Canada, France, Germany, Italy, Ireland, Portugal, Spain, and the UK. MS: multiple sclerosis; PDF: portable document format; UK: United Kingdom; US: United States; YMSQ: your MS questionnaire.

Stage 1

The initial draft of the YMSQ was prepared based on findings from qualitative and quantitative research which had been conducted to develop MSProDiscuss™.^20,21

Stage 2

A Portable Document Format (PDF) version and an online version of the YMSQ were designed.

Stage 3 : Final cognitive debriefing

Study overview

Cognitive debriefing interviews were conducted to ensure that the patient-completed communication support tool is patient-friendly, well-understood, and appropriate for self-completion. This non-interventional, cross-sectional, qualitative interview study included 20 adults (10 United States [US]; 10 Germany) with a clinician-confirmed RRMS or SPMS diagnosis (Figure 2). Two rounds of telephone interviews were conducted in the local language of the participant. The tool was updated based on the feedback obtained from participants in round 1 (US, n = 5; Germany, n = 5) and finalized after receiving feedback from participants in round 2 (US, n = 5; Germany, n = 5). Participants were recruited into the study by a partner agency (MedQuest Global in the US and Rapa Research in Germany) from clinicians/general practitioners or specialist neurologists. Eligible participants were aged ≥18 years, had a clinician-confirmed diagnosis of RRMS or SPMS, had provided written informed consent, and were fluent in US English or German. Participants were excluded if they had a life-threatening health condition other than MS, were diagnosed with primary progressive MS (PPMS) or clinically isolated syndrome, or had a severe neurological/cognitive deficit or psychological disorder that might affect their ability to participate in an interview. All interviews were conducted via telephone (approximately 30 min) by Adelphi values-trained qualitative interviewers. During the interview, participants completed the communication support tool using a think-aloud approach where they were asked to speak their thoughts aloud as they read aloud and completed each item.

Figure 2.

Overview of cognitive briefing study methodology. PDF: portable document format; plwMS: people living with multiple sclerosis; US: United States.

Stage 4 : Patient councils/advisory board meetings

Two international MS council meetings were held on May 13, 2019 and September 16, 2019. Each meeting was attended by eight participants. A patient advisory board meeting was held on May 13, 2019 and included six plwMS and two neurologists. The participants in the council meeting and advisory board were from Australia, Belgium, Canada, France, Germany, Italy, Ireland, Portugal, Spain, and the UK. Participants provided general feedback on the draft tool, potential improvements, and suggestions for changes to the YMSQ.

YMSQ usability testing

Survey methodology

The survey was carried out by IQVIA from September, 2020 to July 23, 2021. An online survey link was sent to the clinicians who agreed to participate in the study. The survey consisted of 2 questionnaires (15 questions in each survey). The testing was completed by clinicians from 7 countries (Australia, China, Germany, Italy, Spain, the UK, and the US). The majority of the clinicians associated with the usability testing were from the YMSQ development steering committee. All clinicians were required to fill both surveys (Appendix 1). Clinicians indicated their level of agreement to each statement using a 4-point Likert scale: “strongly agree,” “agree,” “disagree,” or “strongly disagree.” Clinicians filled out the individual survey after each patient consultation. This individual survey aimed to collect feedback on the comprehensibility of the questions in the tool, overall usability, usefulness of YMSQ, and patient/clinician satisfaction. Clinicians asked their patients to fill out the paper version of the YMSQ prior to consultation in the waiting room or the link to the online version was shared via email/text message/telehealth platform. During the consultation, clinicians obtained feedback on usability and satisfaction with YMSQ and filled the individual surveys based on patient responses and their own experiences. Each clinician was required to fill out 10–40 individual questionnaires. As the final step, clinicians completed a second survey to provide in-depth feedback on YMSQ's usefulness, integration into clinical routine, and recommendations for improvement areas (Figure 3).

Figure 3.

Steps of the Your Multiple Sclerosis Questionnaire (YMSQ) usability test survey. plwMS: people living with multiple sclerosis.

Statistical analysis

The responses to both the individual and final usability surveys were analyzed separately and descriptively on a question-by-question basis. The responses to the questions were reported as a percentage of the total responses.

Results

The YMSQ draft was developed in a PDF printable format for manual completion and as an online interactive version. The development of the initial version was based on findings from previous research to develop MSProDiscuss™.^20,22 The insights from plwMS obtained during cognitive debriefing were incorporated. The questionnaire was further refined with input from key stakeholders, including patient advocates, key opinion leaders, and a patient council.

Cognitive debriefing

Demographics and clinical characteristics

More than half of the participants (n = 11, 55.0%) reported their MS to be moderate in severity. Most plwMS categorized their general health, at the time of the interview, as “fair” (n = 10, 50.0%) or “good” (n = 8, 40.0%), whereas only one reported it to be “poor” (n = 1, 5.0%). According to clinician reports, most plwMS had a history of RRMS for more than 16 years (n = 8, 40.0%) or had been diagnosed with SPMS 0–5 years ago (n = 8, 40.0%). At the time of completion of the case report form, clinicians reported that most plwMS had a mild EDSS score (EDSS 0–3.5; n = 10, 50.0%), followed by moderate/severe EDSS score (EDSS ≥4; n = 8, 40.0%).

Outcomes of cognitive debriefing

In cognitive debriefing, feedback was obtained on the paper version of YMSQ. Overall, the YMSQ items and instructions were well understood by the plwMS. The plwMS gave some key suggestions to improve the comprehension of the tool. Formatting and wording were modified to present the item flow more clearly. Following round 1, an additional instruction was added above items 2b and 2c to clarify that these 2 items should be completed only if plwMS responded “Yes” to item 2a. “Yes”/”No” response options were reversed in section two to ‘No/‘Yes’ and dashed arrows were added to guide the plwMS to the next item. After round 2, additional minor modifications were made, in particular involving symptom and impact items that contained more than one concept. Here, a clarification was added to select “Yes” if at least one aspect was present. The initial and revised versions of the YMSQ with modifications after 2 rounds of interviews are provided in Appendix 2 and 3. Additional general feedback was requested from 11 plwMS. PlwMS provided feedback on the name of the tool in addition to its usability and relevance. Overall, plwMS perceived that the tool included concepts that were highly relevant to their experiences with MS and highlighted that they would feel comfortable completing the tool either on paper (n = 6/8) or electronically (n = 7/8). There was an equal split of plwMS regarding the perceived usefulness of assistance from a family member or friend when completing the tool. In some cases of upper limb mobility issues or fatigue, a support person may record the responses of the plwMS. Most plwMS (n = 9/11), however, reported that they felt the tool would be valuable in helping them communicate their MS experience to their doctors. Owing to time constraints during the interview, not all plwMS were asked to provide general feedback on the communication support tool. Supplemental Table 1 provides details of the general feedback provided by plwMS for YMSQ.

Outcomes of patient councils and advisory board

Key recommendations from the international MS council and patient advisory board are presented in Figure 4. The key improvements to the tool included the addition of mood problems and sexual dysfunction to the impacts and the addition of the definition of relapse. It was suggested to change the name of the tool from “My MS” to “Your MS”. Further, there was a recommendation to change the time period in the introduction of the tool from “this tool asks about your MS in the past 12 months’ to 6 months. Patient councils also noted that in some instances, plwMS might not be aware that certain symptoms were caused by their MS, and they pointed out that YMSQ presents an opportunity to inform plwMS about the diversity of symptoms and fully capture their experience.

Figure 4.

Outcomes from patient councils and advisory board meetings. MRI: magnetic resonance imaging; plwMS: people living with multiple sclerosis.

Feedback on the usability and usefulness of YMSQ

The usability and usefulness of YMSQ were assessed using both individual and final surveys. All 13 clinicians completed the individual survey after every consultation (N = 261) whereas 10 clinicians completed the final survey. Clinicians from different countries including Australia, China, Germany, Italy, Spain, the UK, and the US participated (Table 1). Most of the patients (n = 219; 83.9%) had RRMS diagnosis prior to using the questionnaire.

Table 1.

Completion status for YMSQ usability testing per country.

Country	Clinicians participated	No. of individual surveys completed	No. of final surveys completed
Australia	1	20	1
China	3	34	2
Germany	2	40	2
Italy	1	40	1
Spain	2	80	2
UK	1	20	1
US	3	27	1
Total	13	261	10

No: Number; UK: United Kingdom; US: United States; YMSQ: Your Multiple Sclerosis Questionnaire.

Distribution channels and completion status for YMSQ

More than 40% of the patients filled the paper version of the questionnaire in the waiting room, whereas 35.6% received a link to the online version via email (Figure 5(a)). Overall, 65.1% of the patients received the questionnaire on the day of their consultation and 86.2% of the questionnaires were completely filled (Figure 5(b)). Most questionnaires (n = 223; 85.4%) were completed alone by plwMS.

Figure 5.

Distribution channels for Your Multiple Sclerosis Questionnaire (YMSQ) and completion status.

Individual survey

A total of 261 consultations were conducted by the clinicians. Most of the clinicians strongly agreed (138/261; 52.9%) or agreed (120/261; 46.0%) that YMSQ was useful for both remote and face-to-face consultations with their patients. YMSQ was easy to use and understand in 98.5% of cases (257/261). The time taken by patients to complete the tool was deemed as satisfactory (97%; 254/261). Most of the clinicians (98.1%) were willing to use the tool again with the same patient (Figure 6).

Figure 6.

Summary findings from the usability testing of the YMSQ individual survey (N = 261). The numbers in the bars reflect response of clinicians as agree and strongly agree to each item after patient consultations. MS: multiple sclerosis; YMSQ: Your Multiple Sclerosis Questionnaire.

Final survey

Ten clinicians completed the final survey. All clinicians (100%; 10/10, 261 patient consultations) reported that the use of YMSQ positively influenced their clinical practice, helped them in engaging patients with their MS, facilitated discussion on progression with patients, and complemented neurological assessment. The current version of the YMSQ was ideal for patients and the current format was easy to use and generally satisfactory (9/10; 90% of the clinicians). Clinicians agreed that the tool covers all aspects of general consultation (9/10; 90% of the clinicians) (Figure 7). Overall, the YMSQ tool was reported to have excellent usability and usefulness in clinical practice.

Figure 7.

The summary findings from the usability testing of the YMSQ final survey (N = 10). The numbers in the bars reflect the proportion of clinicians who responded agree and strongly agree for each item. MS: multiple sclerosis; YMSQ: Your Multiple Sclerosis Questionnaire.

Discussion

Principal findings

YMSQ is designed to be completed ahead of consultation by plwMS to support a focused discussion with the clinician during the consultation, and to highlight any changes they have noticed. PlwMS can fill in the questionnaire with the help of their caregivers. The YMSQ tool was developed in 4 stages. Development of the draft content followed by its designing as a PDF for the print version and as an online interactive version, then, the draft was revised as per suggestions received from plwMS during cognitive debriefing interviews and feedback from patient councils and patient advisory board. The development process followed the scientific method, included all stakeholders, and made YMSQ a relevant, comprehensible, quick, well-accepted, and useful tool.

Impact of implementation of YMSQ

In the last few years, the importance of including the perspective of plwMS in the assessment of MS outcomes has been increasingly recognized. PRO measures, which collect information on the functioning health and well-being from patients’ perspective, lead to patients reporting symptoms more often and earlier than usual.²³ YMSQ is a patient-completed tool that helps collect patient-driven information on their symptoms and any changes including disease worsening or progression to assist in the patient-clinician discussion. It would support optimal use of the consultation time as plwMS are prepared ahead of the consultation and primed to discuss the issues of most concern. YMSQ could complement clinical measurements including MRI and relapses, and thus aid in treatment decisions and symptom management. The importance of self-management is recognized in many chronic conditions, including cancer and diabetes.²⁴ The need for appropriate self-management techniques for plwMS has been discussed and investigated for more than a decade. In particular, given the need for continuous monitoring and symptom tracking, MS is appropriate for digital self-management approaches. Studies have reported on diverse mobile applications for plwMS, but often they fail to meet patients’ and clinicians’ requirements.²⁵ Usability is particularly limited when applications focus on only one or few symptoms of MS.^26,27 In contrast, YMSQ is a 20-questionnaire tool that captures a broad range of symptoms and impacts of MS, addressing the heterogeneous nature of MS. If used by plwMS, e.g., every 6 months, YMSQ enables longitudinal monitoring of symptoms. By highlighting changes, symptom management and treatment decisions can be made in a timely fashion. The downloadable feature of YMSQ reports may further help plwMS and their clinicians to compare different reports, monitor changes, and standardize follow-up appointments. Health literacy enhances self-care and treatment adherence. Increased engagement and experience using digital tools can aid patients, enhancing their health literacy as they access information from digital sources and health personnel including clinicians and nurses.²⁸ YMSQ aims to improve the plwMS-clinician interaction; thus, it could contribute to the development of a trusting partnership between plwMS and their clinician.

YMSQ and MSProDiscuss™ in the management of MS telemedicine for patients with MS

Telemedicine use in the management of chronic diseases is increasing as it ensures accessibility of healthcare to people living in rural areas with limited healthcare facilities.^29–31 The coronavirus disease 2019 pandemic has further shown an increase in the trend of teleconsultations.³² Both patients and clinicians tend to be satisfied with the use of telehealth because it saves time, reduces costs for patients, and improves adherence to disease-modifying therapy.^32,33 Inclusion of PROs into teleconsultation for plwMS is important for long-distance examinations, particularly for those with limited mobility.⁷ YMSQ usability has been tested in this pandemic, and clinicians found it useful in both remote and face-to-face consultations (98.9% of the clinicians). Indeed, the need for a patient-focused and patient-completed tool was highlighted as an unmet need during the testing of MSProDiscuss™, ultimately leading to the creation of YMSQ. The MSProDiscuss™ digital tool is validated for use by clinicians during patient consultations to facilitate structured interactions that aid in the timely evaluation of subtle signs of MS disease progression. The tool differentiated between patients with RRMS (with EDSS: sensitivity = 0.83; specificity = 0.82 and without EDSS: Sensitivity = 0.76; specificity = 0.86) and SPMS (with EDSS: sensitivity = 0.82; specificity = 0.84 and without EDSS: Sensitivity = 0.79; specificity = 0.74) with high sensitivity and specificity.²⁰ The usability of MSProDiscuss™ has been tested among 301 HCPs from 34 countries who used it in 6974 people with MS. In the final questionnaire filled by HCPs (N = 274), a high proportion agreed that the tool helped them to understand the impact of MS symptoms on daily activities (91%) and cognitive function (80%). The tool assisted with deep clinical phenotyping of signs of progression in remote visits.²² These two tools are complementary and could be used independently, as both achieve the same purpose of identifying changes in MS and improving the patient-physician dialogue. However, there are some advantages of using the YMSQ and MSProDiscuss™ tools together: Firstly, plwMS would come to consultations having reflected on the symptoms in the last 6 months and prepared to raise the issues of most concern to them; consequently, input into the MSProDiscuss™ tool would be faster and more comprehensive. Thus, dual integration of YMSQ and MSProDiscuss™ with teleconsultation has distinct advantages.

Integration of YMSQ into electronic health records

The integration of YMSQ into electronic health records (EHR) software such as Epic and Cerner that have a patient portal could help both neurologists and patients. Patients would be able to view their reports at any time on the EHR portal, along with reports of other clinical measures. The availability of all the data in a single place on the EHR tool would reduce the clinician burden of curating different reports in paper and digital format. Thus, more consultation time could be used for the clinician and plwMS to interact, discuss, and plan treatment or complementary approaches accordingly. Clinicians could also easily track plwMS's disease evolution and monitor them individually over time. In the long term, this would allow more confident descriptions of relative benefits and safety of disease-modifying treatments.³⁴ In a recent study, a patient pre-visit questionnaire was integrated into the EHR tool Epic. The study concluded that the integration of patient engagement tools into EHR helped in the inclusion of patient priorities into agenda-setting and documentation in real-world primary care practices. Thus, EHR tools can support patient engagement during in-person as well as telehealth consultations.³⁵ User-centric design of eHealth interventions drives acceptance and engagement.¹⁹ YMSQ has been developed with input from the users, and its usability in engaging plwMS more with their disease was determined in this study. Further, the tool is being integrated into EHR tools by some clinicians across the UK and is being positively accepted by patients and clinicians. In some countries or patient groups the paper version of the tool might be preferable due to relatively less sophisticated information technology systems, use of firewalls in some hospitals, concerns about data security (in light of cyber-attacks on hospitals) and some patient groups being less confident using electronic devices.^36,37

Adoption and dissemination of the YMSQ tool

Examples exist of apparently useful digital tools that nevertheless had poor adoption and uptake either by plwMS or clinicians.^19,38,39 This may be due to them not truly being useful (especially if the primary users of the tools were not included in creation), poor usability, or a lack of awareness and dissemination of the tool.^38,39 By including plwMS in the creation of YMSQ and surveying clinicians to confirm ease of use, the first two pitfalls should have been avoided. However, to achieve broad clinical adoption of YMSQ, successful dissemination is needed. Currently, YMSQ is freely available on a public website (www.yourms.com). Rather than disseminating the YMSQ tool through a single channel or single user type, the hope is for all participants in MS care to be included. For example, stakeholders at the academic, clinical, and pharmaceutical levels can disseminate this tool within health care organizations. Patient associations and plwMS can share the tool within patient communities. Social media platforms and social media influencers from all stakeholder groups can play a significant role in adoption as social media usage by clinicians and patients is increasingly contributing to medical decision-making and patient-clinician interactions.⁴⁰

Future directions

Presently, the tool has only been tested for its usability on adults living with MS. In the future, YMSQ could be adapted for pediatric MS forms. Modifications would be required as some symptoms/impacts are specific to children or adolescents, while others are more relevant to adults. Prospective longitudinal studies of YMSQ in comparison to other validated clinical instruments such as SDMT, Timed 25-Foot Walk, and EDSS could test whether YMSQ is more sensitive to change than conventional measurements. The advantage of using YMSQ is that it has a physician version of the questionnaire (MSProDiscuss^TM), hence, both patient and HCP could fill in their versions of the questionnaire and the differences in their responses could be discussed in depth, resulting in a better understanding and management of the disease. Studies comparing MSProDiscuss™ and YMSQ in terms of sensitivity and specificity could also be conducted. Although the current usability results indicated that the present version of the questionnaire is ideal and the current format is easy to use, with an increase in the tool's use in clinical trials and clinical practice, it will evolve based on further feedback.

Study limitations

In the cognitive debriefing stage, the sample size was smaller (n = 20) and plwMS from only 2 countries were included (the US and Germany); therefore, the findings should be generalized to other countries with caution. Future research could assess the content validity of the patient-completed tool in other countries. However, there were no clear differences in findings between Germany and the US, suggesting that it is likely the tool will have strong cross-cultural validity and perform well in other countries and languages too, assuming best practice methods are used for translation and linguistic validation. Further the diverse number of plwMS, in the real-world usability testing, from 7 countries including Australia, China, Germany, Italy, Spain, the UK, and the US also confirms the cross-cultural applicability of the tool. The limitation of the usability testing survey was that the majority of the clinicians were part of the steering committee; this may have led to some bias in responses to some items.

Conclusion

YMSQ, a patient-completed tool, was co-developed with plwMS, patient advocacy groups, and clinicians to capture MS symptom changes and facilitate discussion between plwMS and their physicians on relapses, changes in MS symptoms, and their impact on daily activities within the past 6 months, enabling treatment planning and symptom management. The initial version of YMSQ was based on findings from previous research to develop MSProDiscuss™, a physician-completed tool. In the real-world usability testing of YMSQ, clinicians found the tool useful for consultation with their patients and were willing to use it again with the same patients. Filling the YMSQ tool ahead of the consultations would benefit both plwMS and clinicians by allowing a focused and better-structured consultation. It would encourage self-management by enhancing the engagement of plwMS in the management of their disease. Further, integration of the tool into EHRs would help in easy tracking of the disease evolution and individual monitoring of symptoms over time.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076231173531 - Supplemental material for Development and usability testing of your MS questionnaire: A patient-based digital tool to monitor symptoms of multiple sclerosis

Supplemental material, sj-docx-1-dhj-10.1177_20552076231173531 for Development and usability testing of your MS questionnaire: A patient-based digital tool to monitor symptoms of multiple sclerosis by Gavin Giovannoni, Enrique Alvarez, Ellen Tutton, Olaf Hoffmann, Yan Xu, Patrick Vermersch, Celia Oreja-Guevara, Maria Trojano, Ralf Gold, René Robles-Cedeño, Mudeer Khwaja, Bianca Stadler, Jo Vandercappellen and Tjalf Ziemssen in DIGITAL HEALTH

Supplemental Material

sj-docx-2-dhj-10.1177_20552076231173531 - Supplemental material for Development and usability testing of your MS questionnaire: A patient-based digital tool to monitor symptoms of multiple sclerosis

Supplemental material, sj-docx-2-dhj-10.1177_20552076231173531 for Development and usability testing of your MS questionnaire: A patient-based digital tool to monitor symptoms of multiple sclerosis by Gavin Giovannoni, Enrique Alvarez, Ellen Tutton, Olaf Hoffmann, Yan Xu, Patrick Vermersch, Celia Oreja-Guevara, Maria Trojano, Ralf Gold, René Robles-Cedeño, Mudeer Khwaja, Bianca Stadler, Jo Vandercappellen and Tjalf Ziemssen in DIGITAL HEALTH

Supplemental Material

sj-png-3-dhj-10.1177_20552076231173531 - Supplemental material for Development and usability testing of your MS questionnaire: A patient-based digital tool to monitor symptoms of multiple sclerosis

Supplemental material, sj-png-3-dhj-10.1177_20552076231173531 for Development and usability testing of your MS questionnaire: A patient-based digital tool to monitor symptoms of multiple sclerosis by Gavin Giovannoni, Enrique Alvarez, Ellen Tutton, Olaf Hoffmann, Yan Xu, Patrick Vermersch, Celia Oreja-Guevara, Maria Trojano, Ralf Gold, René Robles-Cedeño, Mudeer Khwaja, Bianca Stadler, Jo Vandercappellen and Tjalf Ziemssen in DIGITAL HEALTH

Supplemental Material

sj-png-4-dhj-10.1177_20552076231173531 - Supplemental material for Development and usability testing of your MS questionnaire: A patient-based digital tool to monitor symptoms of multiple sclerosis

Supplemental material, sj-png-4-dhj-10.1177_20552076231173531 for Development and usability testing of your MS questionnaire: A patient-based digital tool to monitor symptoms of multiple sclerosis by Gavin Giovannoni, Enrique Alvarez, Ellen Tutton, Olaf Hoffmann, Yan Xu, Patrick Vermersch, Celia Oreja-Guevara, Maria Trojano, Ralf Gold, René Robles-Cedeño, Mudeer Khwaja, Bianca Stadler, Jo Vandercappellen and Tjalf Ziemssen in DIGITAL HEALTH

Supplemental Material

sj-png-5-dhj-10.1177_20552076231173531 - Supplemental material for Development and usability testing of your MS questionnaire: A patient-based digital tool to monitor symptoms of multiple sclerosis

Supplemental material, sj-png-5-dhj-10.1177_20552076231173531 for Development and usability testing of your MS questionnaire: A patient-based digital tool to monitor symptoms of multiple sclerosis by Gavin Giovannoni, Enrique Alvarez, Ellen Tutton, Olaf Hoffmann, Yan Xu, Patrick Vermersch, Celia Oreja-Guevara, Maria Trojano, Ralf Gold, René Robles-Cedeño, Mudeer Khwaja, Bianca Stadler, Jo Vandercappellen and Tjalf Ziemssen in DIGITAL HEALTH

Supplemental Material

sj-docx-6-dhj-10.1177_20552076231173531 - Supplemental material for Development and usability testing of your MS questionnaire: A patient-based digital tool to monitor symptoms of multiple sclerosis

Supplemental material, sj-docx-6-dhj-10.1177_20552076231173531 for Development and usability testing of your MS questionnaire: A patient-based digital tool to monitor symptoms of multiple sclerosis by Gavin Giovannoni, Enrique Alvarez, Ellen Tutton, Olaf Hoffmann, Yan Xu, Patrick Vermersch, Celia Oreja-Guevara, Maria Trojano, Ralf Gold, René Robles-Cedeño, Mudeer Khwaja, Bianca Stadler, Jo Vandercappellen and Tjalf Ziemssen in DIGITAL HEALTH

Footnotes

Acknowledgements

The authors wish to acknowledge the contribution of Virender Bhan, Jason Burton, Salvatore Napoli, Augusto Miravalle, and Thomas Hach for their contribution in the usability testing of the Your MS Questionnaire. The authors also acknowledge Anupam Dwesar and Anuradha Nalli, both of Novartis Healthcare Pvt. Ltd, Hyderabad, India, for medical writing and editorial assistance in preparing the manuscript. The final responsibility for the content lies with the authors.

Contributorship

T.Z., E.T., B.S., J.V., P.V., and M.K., were involved in the design and development of the tool. E.T. is part of the international multiple sclerosis patient council. B.S., M.K., and J.V. contributed to the concept and design of the tool's usability testing survey. G.G., E.A., O.H, C.O.G., R.R.C., M.T., R.G., and Y.X. contributed to data collection. G.G., E.A., O.H., C.O.G., R.R.C., M.T., Y.X., and P.V. were members of the steering committee. All named authors meet the International Committee of Medical Journal Editors (ICMJE) criteria for authorship for this article, take responsibility for the integrity of the work as a whole, and have given their approval for this version to be published.

Declaration of conflicting interests

The author(s)declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article:M.K. and B.S. are employees of Novartis. J.V. was an employee of Novartis Pharma AG during the development of the tool and the usability testing until the final version of the manuscript development. G.G. received consulting fees from AbbVie, Actelion, Atara Bio, Biogen, Celgene, Sanofi-Genzyme, Genentech, GlaxoSmithKline, Merck-Serono, Novartis, Roche and Teva, for research from Biogen, Roche, Merck, Merck-Serono, Novartis, Sanofi-Genzyme, and Takeda. E.A. received compensation for consulting from Actelion/Janssen, Alexion, Bayer, Biogen, Celgene/BMS, EMD Serono/Merck, Genentech/Roche, Genzyme, Novartis, Sanofi, and TG Therapeutics, and for research from Biogen, Genentech/Roche, Novartis, TG Therapeutics, Patient-Centered Outcomes Research Initiative, National Multiple Sclerosis Society, National Institutes of Health, and Rocky Mountain MS Center. O.H. received consulting fees from Biogen, Celgene, Janssen, Merck, Novartis, Roche, and Sanofi; funding for research from Biogen, Novartis, and Sanofi; and as a speaker from Merck, Novartis, Roche, and Sanofi. C.O.G. received consulting fees from Novartis, Alexion, and Roche; compensation for research from Alexion; and as a speaker from Novartis and Roche. P.V. has received compensation for consulting and/or research and registration, travel, and accommodation for meetings from Biogen, Roche, Novartis, Sanofi, Teva, Merck, AB Science, Imcyse and Celgene. T.Z. has received compensation for consulting and lecturing from Alexion, Biogen, Celgene, Novartis, Roche, Sanofi, and Teva and for research from Biogen, Novartis, Roche, Teva, and Sanofi. Y.X. has nothing to disclose. R.R.C. has received compensation for consulting services and speaking fees from Biogen, Roche, Novartis, Bayer, Merck, Sanofi, Genzyme, Teva Pharmaceutical Industries Ltd, and Almirall. M.T. has received compensation for consulting from Novartis, Biogen, Merck, Roche, and Sanofi, and her institution received support for research from Biogen, Merck, Novartis, and Roche. R.G. has received compensation for serving as a consultant or speaker from Bayer HealthCare, Biogen Idec, Merck Serono, Novartis and Teva Neuroscience. He, or the institution he works for, has received research support from Bayer HealthCare, Biogen Idec, Merck Serono, Novartis and Teva Neuroscience. He has also received honoraria as a Journal Editor from SAGE and Thieme Verlag. E.T. has received compensation for consulting from Novartis, Biogen, and Sanofi/Genzyme Europe B.V.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by Novartis Pharma AG, Basel, Switzerland.

Ethical approval

No ethics approval was required because this was a survey. Nevertheless, all respondents were offered detailed information about the survey method, and they provided written consent before participating.

Guarantor

T.Z.

ORCID iD

Tjalf Ziemssen

Supplemental material

Supplemental material for this article is available online.

References

Kamm

Uitdehaa

Polman

, et al. Multiple sclerosis: current knowledge and future outlook. Eur Neurol 2014; 72: 132–141.

Inojosa

Proschmann

Akgun

, et al. A focus on secondary progressive multiple sclerosis (SPMS): challenges in diagnosis and definition. J Neurol 2021; 268: 1210–1221.

Ziemssen

Tolley

Bennett

, et al. A mixed methods approach towards understanding key disease characteristics associated with the progression from RRMS to SPMS: physicians’ and patients’ views. Mult Scler Relat Disord 2020; 38: 101861.

Rahn

Slusher

Kaplin

. Cognitive impairment in multiple sclerosis: a forgotten disability remembered. Cerebrum 2012; 2012: 14.

Lakin

Davis

Binns

, et al. Comprehensive approach to management of multiple sclerosis: addressing invisible symptoms-a narrative review. Neurol Ther 2021; 10: 75–98.

Ziemssen

Kern

Thomas

. Multiple sclerosis: clinical profiling and data collection as prerequisite for personalized medicine approach. BMC Neurol 2016; 16: 124.

Inojosa

Schriefer

Ziemssen

. Clinical outcome measures in multiple sclerosis: a review. Autoimmun Rev 2020; 19: 102512.

Inojosa

Proschmann

Akgun

, et al.

Should we use clinical tools to identify disease progression?

Front Neurol 2021; 11: 628542.

Cutter

Baier

Rudick

, et al. Development of a multiple sclerosis functional composite as a clinical trial outcome measure. Brain 1999; 122: 871–882.

10.

Kalb

Beier

Benedict

, et al. Recommendations for cognitive screening and management in multiple sclerosis care. Mult Scler 2018; 24: 1665–1680.

11.

Hegen

Bsteh

Berger

. ‘No evidence of disease activity’ - is it an appropriate surrogate in multiple sclerosis? Eur J Neurol 2018; 25: 1107–e101.

12.

Meyer-Moock

Feng

Maeurer

, et al. Systematic literature review and validity evaluation of the Expanded Disability Status Scale (EDSS) and the Multiple Sclerosis Functional Composite (MSFC) in patients with multiple sclerosis. BMC Neurol 2014; 14: 58.

13.

Talib

DeChant

Kean

. A qualitative study of patients’ perceptions of the utility of patient-reported outcome measures of symptoms in primary care clinics. Qual Life Res 2018; 27: 3157–3166.

14.

Raffel

Wallace

Gveric

, et al. Patient-reported outcomes and survival in multiple sclerosis: a 10-year retrospective cohort study using the multiple sclerosis impact scale-29. PLoS Med 2017; 14: e1002346.

15.

D'Amico

Haase

Ziemssen

. Review: patient-reported outcomes in multiple sclerosis care. Mult Scler Relat Disord 2019; 33: 61–66.

16.

Food and Drug Administration. Guidance for Industry. Patient-reported outcome measures: use in medical product development to support labeling claims, https://www.fda.gov/media/77832/download (2009, accessed 8 July 2022).

17.

Your MS Questionnaire. https://www.yourms.com/#/welcome (accessed 8 July 2022).

18.

Ziemssen

Haase

. Digital innovation in multiple sclerosis management. Brain Sci 2021; 12: 40.

19.

Scholz

Haase

Schriefer

, et al. Electronic health interventions in the case of multiple sclerosis: from theory to practice. Brain Sci 2021; 11: 180.

20.

Ziemssen

Piani-Meier

Bennett

, et al. A physician-completed digital tool for evaluating disease progression (multiple sclerosis progression discussion tool): validation study. J Med Internet Res 2020; 22: e16932.

21.

Tolley

Piani-Meier

Bentley

, et al. A novel, integrative approach for evaluating progression in multiple sclerosis: development of a scoring algorithm. JMIR Med Inform 2020; 8: e17592.

22.

Ziemssen

Giovannoni

Alvarez

, et al. Multiple sclerosis progression discussion tool usability and usefulness in clinical practice: cross-sectional, web-based survey. J Med Internet Res 2021; 23: e29558.

23.

Rieckmann

Boyko

Centonze

, et al. Achieving patient engagement in multiple sclerosis: a perspective from the multiple sclerosis in the 21st Century Steering Group. Mult Scler Relat Disord 2015; 4: 202–218.

24.

Fraser

Ehde

Amtmann

, et al. Self-management for people with multiple sclerosis: report from the first international consensus conference, November 15, 2010. Int J MS Care 2013; 15: 99–106.

25.

Mokhberdezfuli

Ayatollahi

Naser Moghadasi

. A smartphone-based application for self-management in multiple sclerosis. J Healthc Eng 2021; 2021: 6749951.

26.

Palotai

Wallack

Kujbus

, et al. Usability of a mobile app for real-time assessment of fatigue and related symptoms in patients with multiple sclerosis: observational study. JMIR Mhealth Uhealth 2021; 9: e19564.

27.

Van Kessel

Babbage

Reay

, et al. Mobile technology use by people experiencing multiple sclerosis fatigue: survey methodology. JMIR Mhealth Uhealth 2017; 5: e6.

28.

Dehghani

. Health literacy in multiple sclerosis patients: a concept analysis using the evolutionary method. J Caring Sci 2021; 10: 49–56.

29.

Yeroushalmi

Maloni

Costello

, et al. Telemedicine and multiple sclerosis: a comprehensive literature review. J Telemed Telecare 2020; 26: 400–413.

30.

Corbett

Opladen

Bisognano

. Telemedicine can revolutionize the treatment of chronic disease. Int J Cardiol Hypertens 2020; 7: 100051.

31.

Butzner

Cuffee

. Telehealth interventions and outcomes across rural communities in the United States: narrative review. J Med Internet Res 2021; 23: e29575.

32.

Xiang

Bernard

. Telehealth in multiple sclerosis clinical care and research. Curr Neurol Neurosci Rep 2021; 21: 14.

33.

Haase

Voigt

Scholz

, et al. Profiles of eHealth adoption in persons with multiple sclerosis and their caregivers. Brain Sci 2021; 11: 1087.

34.

Matthews

Block

Leocani

. E-health and multiple sclerosis. Curr Opin Neurol 2020; 33: 271–276.

35.

Tai-Seale

Rosen

Ruo

, et al. Implementation of patient engagement tools in electronic health records to enhance patient-centered communication: protocol for feasibility evaluation and preliminary results. JMIR Res Protoc 2021; 10: e30431.

36.

Jalali

Kaiser

. Cybersecurity in hospitals: a systematic, organizational perspective. J Med Internet Res 2018; 20: e10059.

37.

Muthuppalaniappan

Stevenson

. Healthcare cyber-attacks and the COVID-19 pandemic: an urgent threat to global health. Int J Qual Health Care 2021; 33: mzaa117.

38.

Gordon

Landman

Zhang

, et al. Beyond validation: getting health apps into clinical practice. NPJ Digit Med 2020; 3: 14.

39.

Gromisch

Turner

Haselkorn

, et al. Mobile health (mHealth) usage, barriers, and technological considerations in persons with multiple sclerosis: a literature review. JAMIA Open 2021; 4: ooaa067.

40.

Patrick

Venkatesh

Stukus

. Social media and its impact on health care. Ann Allergy Asthma Immunol 2022; 128: 139–145.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

0.31 MB

0.47 MB

0.65 MB

0.61 MB