Abstract
Background:
We developed a 29-item Questionnaire, Long-term Unmet Needs in MS (LUN-MS) to identify the unmet needs of people with multiple sclerosis (pwMS).
Objective:
To assess acceptability, test-retest reliability, internal consistency, and validity of the LUN-MS.
Methods:
Participants completed the LUN-MS and MSIS-29 twice, four weeks apart. Acceptability was assessed by looking at the response rate in each time point. Reliability was calculated by comparing the response during the two time points using Cohen’s weighted kappa. Using principal component analysis, the dimensionality of the questionnaire’s items was reduced, to five domains and the internal consistency of each domain was assessed using Cronbach’s alpha. Concurrent validity was tested by comparing the total LUN-MS score against MSIS-29 and EQ-5D-3L using Pearson’s product-moment correlation coefficient.
Results:
Among 88 participants, rate of completion at time points-1 and 2 was 96 and 80% respectively. Test-retest reliability for individual items was between fair to near-perfect (weighted Cohen’s kappa 0.39-0.81). The unmet needs could be divided into five internally consistent domains (Cronbach’s alpha 0.83-0.74): neuropsychological, ambulation, physical, interpersonal relationship and informational. Concurrent validity with MSIS-29 (r = 0.705, P < .001) and EQ-5D-3L (r = 0.617, P < .001) were good.
Conclusion:
LUN-MS is a reliable, valid, and acceptable tool to identify the unmet needs of pwMS.
Keywords
Background
Health, defined as a state of complete physical, psychological and social wellbeing and not merely the absence of disease or infirmity, is the amalgamation of three core domains – biological, psychological and social needs. 1 Need denotes the capacity to benefit from a specific health intervention in pursuance of an acceptable quality of life 2 and unmet needs are health needs that remain unattainable within the current service provisions. 3
Disease modifying drugs (DMDs) are effective in reducing the activity of Multiple Sclerosis ( MS).The medical team’s focus is currently on the modification of course of MS using DMDs and management of objective symptoms like spasticity and weakness. People with MS (pwMS) experience needs which are not associated with objective neurological signs or symptoms. Many of these biopsychosocial needs are inadequately addressed by the services.4 -6 These unmet needs, may perpetuate a steady decline of their overall quality of life (QoL). With the advent of expensive disease-modifying therapies for MS, the focus of many service providers has also shifted to pharmacological needs, at the expense of rehabilitation and psychosocial needs. 7 A study from South of England on unmet needs of 93 people with disability (28 of participants had MS), showed that 83 had at least one unmet need. The unmet needs spanned a variety of domains like information, finance, care, social and intellectual fulfilment. These are not usually captured by the current assessments for MS. 8 A study from five European countries, including UK showed that pwMS reported an average of 2.9 unmet needs. 9 A study from Ireland reported at least one, non-pharmacological unmet need in 170 of 325 people with MS. 7
The pwMS consistently reported that the ‘invisible symptoms’ of MS, such as fatigue, pain, sexual and psychological disorders have all been associated with health related distress.10,11 Current monitoring tools used in MS such as the Expanded Disability Status Scale (EDSS) 12 addresses mostly the physical aspects of MS and do not address these invisible symptoms. Patient Reported Outcome Measures (PROM) measure a patient’s health status or health related quality of life at a single point in time through self-completed questionnaires. Most of the PROMs in MS focus on specific domains such the 12-Item MS Walking Scale (MSWS-12). 13 Multiple Sclerosis Impact Scale 29 (MSIS-29), a multi-domain PROM, measures the physical and psychological impact of pwMS. 14 It is not designed to identify the unmet needs and does not cover issues like information about MS, psychological issues and relationships. In its consensus statement, the steering group of the MS in 21st century initiative recommended that, to deliver optimum disease management, it is essential to identify the unmet needs of PwMS. 15 There is a need to develop a PROM to identify the unmet needs of pwMS.
Longer-term Unmet Needs after Stroke (LUNS) is a valid, reliable, and acceptable 22-item tool used for identifying the long-term unmet needs in post-stroke patients. 16 We obtained permission from the authors of LUNS to modify it to suit the needs of pwMS. We developed a 29-item questionnaire called the Long-term Unmet Needs in MS (LUN-MS) through a process of review of relevant literature, consultations with health care professionals including neurologists, neurorehabilitation specialists and MS nurse specialists and group consultations with pwMS and their family members. 4 The LUN-MS is a 29-item self-reported questionnaire with each item scoring in 3-point Likert scale ranging from (1) ‘No, I don’t have this problem’, (2) ‘Yes, I have this problem, but I don’t need any help with it’, and (3) ‘Yes, this is a problem, and I would like help’ (Appendix). The Flesch-Kincaid reading level is 5.5 and Flesch reading ease is 60.8%.
This article outlines the psychometric evaluation of LUN-MS in terms of its acceptability, test-retest reliability, component structure and validity in line with the COSMIN guidelines for PROMs. 17 This study received ethical approval from the Health Research Authority (HRA), North of Scotland Research Ethics Service (IRAS Project ID: 241772) and the Research Ethics Committee of The University of Sheffield.
Methods
Participants
This study was conducted at a regional MS centre of a tertiary teaching hospital in United Kingdom. The participants were identified by MS nurses and doctors from outpatient clinics and day care services delivering DMDs. Those attending remote consultations (telephone or video) were also included in the study. The inclusion criteria were (a) patients diagnosed with MS (by a consultant neurologist) attending the hospital MS clinic, or Neuro Day-Case Unit (NDCU) (b) ability to provide informed consent (c) ability to complete the questionnaires (d) sufficient communication skills in English. Consequently, participants who lacked sufficient knowledge of English to read and understand the questionnaires were excluded. Those associated with neurological, cardiac, musculoskeletal and pulmonary diseases which can affect their unmet needs were excluded.
Measures
Participant’s clinical information were collected from their hospital records. Participants were asked to complete a paper questionnaire pack at each of the two separate time points (Time point-1 and Time point-2), four weeks apart (Table 1). Content of the questionnaire packs specific to each time point is further detailed in Table 1. The participants were asked to complete the questionnaires by themselves, if possible. The participants who cannot complete the questionnaire due to their impairments, could seek the assistance of the family members and\or carers of the participants to transcribe the participant’s responses on to the paper questionnaire. The participants were given the options of either completing the questionnaire during the hospital visit or to take the questionnaire packs home and return the completed questionnaires by post.
Questionnaire pack content.
Questionnaire pack content at Time point-1 and Time point-2.
Other PROM in the questionnaire packs include MSIS-29 14 and EuroQoL EQ-5D-3L. 18 The MSIS-29 is a 29-item self-rated outcome measure which identifies the impact of MS in the physical (20 items) and psychological (9 items) domains. 13 The EQ-5D-3L consist of the EQ-5D descriptive system and the EQ visual analogue scale (VAS). 19 The EQ-5D-3L descriptive system measures patients’ health state in five dimensions (mobility, self-care, usual activities, pain/discomfort and anxiety/depression), each of which has three levels of response: no problems, some problems and extreme problems. The EQ VAS is a vertical visual analogue scale measuring patient’s self-rated health state and ranges from 0 (worst imaginable health state) to 100 (best imaginable health state). 18 The patient-reported Kurtzke Expanded Disability Status Scale (EDSS) 12 ambulation score was also administered during the study, which was a modification of the EDSS mobility component, allowing participants to self-rate their motor function through a 5-point ordinal response scale.
Data on participant’s demography (age, gender) and health condition information (duration since MS diagnosis, type of MS, use of DMDs, EDSS) were collected from their electronic patient records (EPR), clinical notes and DMD treatment/funding records.
Analysis
The data was analysed using SPSS version 28 and JASP version 0.16.3.20,21,22 Prior to analysis, the datasets were cross-examined by four different investigators for their face validity, presence of univariate outliers and any missing data. Descriptive statistics (measures of central tendency and dispersion) were used to summarise participants’ demographics and clinical information. Missing values were analysed using Little’s missing completely at random (MCAR) test to establish their likely missing mechanism. 19 The degree of skewness and kurtosis of LUN-MS response was also measured at individual item level to determine if each item meet the normality assumption. 23
Acceptability
The acceptability of LUNS-MS was determined by: (1) the percentage of participants completing the questionnaire and (2) the responses to the satisfaction questionnaire. The satisfaction questionnaire comprised of five close-ended questions and two open-ended questions (Table 2). This is a bespoke questionnaire developed by the study team. We assessed the 2 core components defining a tool’s acceptability: usability and accessibility. 23 The first question required the participants to comment on any needs not covered by the questionnaire. The second question allowed participants to comment any aspects of the questionnaire. Responses to the open-ended satisfaction indicators were examined qualitatively through thematic analysis. In this context, the usability of this questionnaire was effectively captured by the satisfaction questions whilst accessibility of the questionnaire was measured by both the overall participant’s response rate and the open-ended satisfaction questions.
Satisfaction questionnaire close-ended questions.
Satisfaction questionnaire for the Long-term Unmet Needs in multiple sclerosis (LUN-MS) questionnaire.
Test-retest reliability
Participants responded to a second pack of questionnaires at Time point-2, four weeks after Time point-1, to assess the test-retest reliability of the LUN-MS and MSIS-29. The clinical status monitoring questions were used to identify patients experiencing any worsening of existing symptoms or new MS symptoms. Those that reported any such changes were excluded from the reliability analysis. The test-retest reliability was evaluated using item-level values instead of the total score values as this facilitates identification of problematic items that are prone to inconsistencies and allowed the authors to further investigate on them. Use of the total score values could potentially obscure any variabilities introduced at item-level and create a misleading impression of consistency, as it may overlook individual items that demonstrate inconsistent responses across multiple administrations of the test. The intra-rater reliability was initially assessed by percent agreement, measuring the proportion of participants who consistently gave the same response at both time points. Item-level responses from both time points were cross-tabulated and coded according to their level of need. As this method may disregard agreement due to chance, weighted Cohen’s Kappa (κ) statistic was selected as an auxiliary measure of intra-rater reliability between the two-time points. 24 Weighted κ considers the severity of misclassifications in an ordinal scale unlike unweighted kappa which treats all disagreements equally. Weighted κ values across all 29 items of both LUN-MS v2 and MSIS-29 were calculated. Weighted κ values were interpreted as follows: no agreement (⩽0), slight agreement (0.01-0.20), fair agreement (0.21-0.40), moderate agreement (0.41-0.60), substantial agreement (0.61-0.80), near-perfect agreement (0.81-0.99) and perfect agreement (1.00). 25 κ values across all 29 items of both LUN-MS v2 and MSIS-29 were juxtaposed with their corresponding percent agreements to identify any significant numerical discordance. We also included the results for percent agreement as stipulated by the COSMIN framework. 17
Component analysis
Principal component analysis (PCA) was integrated into the questionnaire’s validation framework to identify different components within the questionnaire. 26 An oblique (oblimin) rotation method was adopted to allow a certain degree of laxity towards potential collinearity between the extracted components. Eigen decomposition was based on the variance-covariance matrix, justified by the lack of need for variable normalisation in this sample. The number of underlying components was tentatively inferred from the generated pattern matrix and component eigenvalues (λ). Kaiser criterion (λ greater than 1 rule) was applied to establish the absolute lower bound of eigen values tolerated for component retention. 27 Comparative dimensionality assessment by the more robust parallel analysis (Monte Carlo method) was deemed appropriate given the propensity of the former method to overinflate λ at the sample level. 28 Outcomes from both methods later formed the basis of the underpinning theoretical justification for the final number of principal components retained.
Internal consistency of the proposed unidimensional constructs was assessed using the intraclass correlation coefficients (ICC) function in SPSS. The data were analysed using a ‘two-way mixed effects, consistency, multiple measurement’ model, ICC (3, k) to generate the Cronbach’s alpha, α, reliability coefficients for each component.29,30,31 The calculation of Cronbach's alpha was done for establishing component internal consistency and not as an assessment of component unidimensionality. Alpha value of 0.6 to 0.7 indicates acceptable internal consistency.32,33
Validity
Concurrent validity 35 was assessed by measuring the correlation between LUN-MS v2 response with other validated concurrent measures (MSIS-29, EuroQoL EQ-5D-3L and the EQ VAS). The bivariate correlation was determined using the Pearson product-moment correlation coefficient.
Construct validity was assessed through convergent and discriminant validity. Convergent validity looked at how closely two measures of the same construct were related and conversely, discriminant validity was established if two measures, each measuring an entirely different construct, were in fact unrelated to each other. 34 To establish convergent validity, the correlation of specific component with their corresponding surrogate measures were analysed (monotrait-heteromethod correlation). The comparator instruments used include MSIS-29 Physical, MSIS-29 Psychological, EQ-5D-3L Mobility, EQ-5D-3L Self-Care, EQ-5D-3L Usual Activities, EQ-5D-3L Pain/Discomfort, EQ-5D-3L Anxiety/Depression, EQ vertical visual analogues scale (EQ VAS) and patient-reported EDSS ambulation score. Priori hypotheses were made for each latent constructs and their corresponding surrogate measures. As per the COSMIN, we proposed the following expected relationship between each latent construct and their respective comparator instruments:
Neuropsychological: MSIS-29 Psychological and EQ-5D-3L Anxiety/Depression.
Ambulation: MSIS-29 Physical, EQ-5D-3L Mobility and Patient-reported EDSS ambulation score.
Physical: MSIS-29 Physical, EQ-5D-3L Mobility, EQ-5D-3L Usual Activities and EQ-5D-3L Pain/Discomfort.
Interpersonal relationship: MSIS-29 Psychological and EQ-5D-3L Anxiety/ Depression.
Informational: MSIS-29 Physical, MSIS-29 Psychological, EQ-5D-3L Self-Care, EQ-5D-3L Usual Activities and EQ VAS.
A correlation value of 0.50 and above signified acceptable convergent validity.
Discriminant validity was analysed by looking at the heterotrait-heteromethod correlations and inter-component correlation coefficients. Sufficient distinction was achieved when the correlation coefficient does not exceed the upper limit of 0.80 to 0.85 when assessing the discriminant validity. 36
Results
Study population, missing data and normality testing
146 pwMS were invited to take part in the study (Figure 1), of which 128 (88%) gave consent. Among them, 101 participants completed the questionnaire at both time points. We excluded 13 participants who experienced either relapse or disease progression between time points 1 and 2. Data from the 88 participants who completed the questionnaire packs at both time points were included in the analysis. The demographics and clinical information of the study cohort is shown in Table 3.
Descriptive statistics for LUN-MS patient cohorts.
Descriptive statistics comparing Long-term Unmet Needs in multiple sclerosis (LUN-MS) patient cohorts. T1-Time point-1, T2-Time point-2, MS-multiple sclerosis, RRMS-relapsing-remitting multiple sclerosis, SPMS-secondary progressive multiple sclerosis, PPMS-primary progressive multiple sclerosis, EDSS-Expanded Disability Status Scale, DMT-disease modifying therapy.
The flow of participants through the study with reasons for exclusion at each stage.
The number of missing item responses was 14 (0.4%) at Time point-1 and 27 (1.1%) at Time point-2. The maximum number of missing responses in any single questionnaire item was two. Little’s MCAR test (Time point-1 P = 0537; Time point-2 P = .400) failed to reject the null hypothesis that the data was MCAR. Normality analysis of LUN-MS individual item responses revealed twelve items (4, 7-10, 17-20, 26-28) with right-skewness (skew level exceeding +1), indicating that participants were more likely to be unaffected by these health issues. No item had kurtosis value exceeding six, though two items (items 4 and 8) were relatively leptokurtic, suggesting moderate departure from normality.
Acceptability
Among the 128 participants, 123 (96%) responded to the questionnaire at Time point-1 and 101 (80%) at Time point-2 (Figure 1). Eleven participants (9%) required prompting to return the questionnaire. Among the 88 participants, 83% (n = 73) were happy to use the LUN-MS questionnaire as part of a research study and 78% (n = 69) of respondents were keen on using this questionnaire as part of their routine MS clinic appointment. While 72% (n = 63) of respondents agreed that this questionnaire addressed all their MS-related concerns, 16% (n = 14) neither agreed nor disagreed with this notion and 12% (n = 11) expressed disagreement with it. 97% (n = 85) of respondents agreed that the length of this questionnaire was about right.
The open-ended satisfaction questions received 68 responses, 30% (n = 20) of which were on the general perception of the questionnaire including remarks on its comprehensiveness and user experience. Thirteen percent (n = 9) of the responses focused on logistical issues with the study. Meanwhile, 57% (n = 39) of the responses were specific suggestions and feedbacks on different items in the questionnaire. Table 4 summarise the results of thematic analysis conducted on the open-ended questions. The comments supported the need for the questionnaire and complemented the structure of the questionnaire. It also reflected some of the patient’s experiences with unmet needs such as, ‘I’m left in limbo and find it hard to contact services’ and ‘as a young female, it is the pregnancy and planning for children that I am most concerned about’.
Thematic analysis of the open-ended satisfaction questions.
Thematic analysis of the open-ended satisfaction questions. Common themes extracted: General perception (n = 20), Technicality & participant’s intrinsic factor (n = 9), Item-related (n = 39).
Test-retest reliability
Eighty-seven percent (n = 88) of respondents reported no fluctuations in their MS symptoms between the two time points. We assessed the reliability using responses from these 88 participants. The frequency of responses to each item for these88 participants is shown in Table 5. Table 6 shows the frequency of responses for the 35 participants who were excluded from the analysis (excluded for not returning their questionnaire packs at Time point-2 (n = 22), excluded due to a change in medical status between the two time points (n = 13). Item-specific percent agreement (%), weighted Cohen’s Kappa (κ) and agreement level (based on weighted κ) for LUN-MS and MSIS-29 are shown in Tables 7 and 8, respectively. Percent agreement of LUN-MS ranged from 58-95%. Weighted κ ranged from 0.39 to 0.81; fair-2 items, moderate-15 items, substantial-10 items, and near-perfect-2 items. Percent agreement for MSIS-29 ranged from 39-65% and weighted κ ranged from 0.05 to 0.51. The test-retest reliability of LUN-MS questionnaire was better than that of MSIS-29. These questionnaires assessed a range of needs (LUNSMS) and impact of different impairments on patients physical and psychological well being (MSIS-29). Therefore the research team choose to analyse the reliability of individual questions rather than the total score.
Frequencies of individual item responses at Time point-1 and Time point-2.
Frequencies of individual item responses at Time point-1 and Time point-2 (n = 88). Response: (1) ‘No, I don’t have this problem’, (2) ‘Yes, I have this problem, but I don’t need any help with it’ and (3) ‘Yes, this is a problem, and I would like help’. Columns 6 and 10 illustrated the number of missing responses at each Time point.
Frequencies of individual item responses at Time point-1 from 35 participants who completed their questionnaires at Time point-1 but were later excluded in analysis.
Frequencies of individual item responses at Time point-1 from 35 participants who completed their questionnaires at Time point-1 but were later excluded in analysis. Thirty-five respondents were excluded from the total of 123 responses received in Time point-1. Response: (1) ‘No, I don’t have this problem’, (2) ‘Yes, I have this problem, but I don’t need any help with it’ and (3) ‘Yes, this is a problem, and I would like help’. Column 6 illustrated the number of missing responses at Time point-1.
Test-retest reliability of LUN-MS.
Test-retest reliability of the 29 items in the Long-term Unmet Needs in multiple sclerosis (LUN-MS) questionnaire. Individual item percent agreement (%), weighted Cohen’s Kappa (κ) and agreement level (based on weighted Cohen’s Kappa value) are shown in columns 3, 4 and 5 respectively.
Test-retest reliability of MSIS-29.
Test-retest reliability of the 29 items in the Multiple Sclerosis Impact Scale (MSIS-29) questionnaire. Individual item percent agreement (%), weighted Cohen’s Kappa (κ) and agreement level (based on weighted Cohen’s Kappa value) are shown in columns 3, 4 and 5 respectively.
Principal component analysis
Eigen value decomposition of the variance matrix resulted in eight principal components with λ over 1 (accounted for 71% of the cumulative multivariate variability) (Table 9). Results from the comparative parallel analysis suggested a two-component model but this only account for 37% of the overall variance, as opposed to 71% from the eight-component model derived by Kaiser criterion. For this reason, we have decided to opt for the eight-component solution as the initial baseline model.
Component loadings of LUN-MS.
Abbreviations: The eight components identified were: NP, Neuropsychological; AMB, Ambulation; PA, Physical A; IR, Interpersonal Relationship; GI, General Information; HRI, Health-related Information; PB, Physical B; PES, Pregnancy and Emotional Support.
Component loadings for the Long-term Unmet Needs in multiple sclerosis (LUN-MS) questionnaire. A lower threshold of 0.4 was chosen for the loadings.
Of the eight components extracted, seven components had a minimum of three variables each. The eighth component, Pregnancy and Emotional Support (PES) however, was only explained by two variables (items 4 and 29), both of which accounted for a cumulative 5% of the total variance. Despite satisfactory compliance to Kaiser criterion, retention of PES was poorly substantiated solely from its λ due to the tendency of such method to over-extract principal components. We decided to abandon the PES component as this was only explained by two of the measured variables instead of the recommended lower threshold of three, and the total common variance explained by this component was insignificant when compared to other components. We included item 4 in the informational component and item 29 into the interpersonal relation component based on their next largest cross-loading values (Table 10).
Cronbach’s alpha (α) reliability coefficient for each LUN-MS domain.
Abbreviations: The five domains used in validity analysis include: NP, Neuropsychological; AMB, Ambulation; P, Physical; IR, Interpersonal Relationship; I, Informational.
Cronbach’s alpha (α) reliability coefficient for each of the Long-term Unmet Needs in multiple sclerosis (LUN-MS) questionnaire’s domain (native and joint component).
We unified the domain 3 (Physical A) and domain 7 (Physical B), which were only marginally distinct under a single common domain ‘Physical Needs’. Due to the homogenous nature of the fifth (General Information) and the sixth (Health-related Information) domains, we merged them into a single domain labelled as ‘Informational Needs’. The final five domains were neuropsychological needs, ambulation needs, physical needs, interpersonal relationship needs and informational needs.
Table 10 displayed the internal consistency (Cronbach’s alpha reliability coefficient) for each LUN-MS domain used in validity analysis. α values were high, ranging from 0.736 to 0.834 across all five domains.
Validity
Bivariate correlation between the total LUN-MS v2 score and other validated concurrent measures were significant (MSIS-29: r = 0.705, P < .001; EQ-5D-3L: r = 0.617, P < .001) (Figure 2). Negative correlation was noted between the total LUN-MS v2 score and EQ VAS, r = −0.429, P < .001 (Figure 2). Convergent and discriminant validity for each hypothetical construct was described in Table 11. A heuristic approach was taken when determining the threshold value for each correlation coefficient, given the nature of each surrogate measure not exactly evaluating the targeted latent construct as intended. Table 12 outlined the inter-domain correlation matrix, used as part of the discriminant validity assessment. None of the correlation coefficients, r, exceeded the 0.80 to 0.85 upper threshold limit.
Scatterplot matrix of long-term Unmet Needs in multiple sclerosis (LUN-MS) with Multiple Sclerosis Impact Scale (MSIS-29), EuroQoL EQ-5D-3L and EQ vertical visual analogue scale (EQ VAS).
Correlation coefficient between each LUN-MS domains and their concurrent measures.
The correlation coefficient between each domains of Long-term Unmet Needs in multiple sclerosis (LUN-MS) hypothetical construct (component) and their corresponding concurrent measures. The concurrent measures used include the Multiple Sclerosis Impact Scale (MSIS-29) Physical component, Multiple Sclerosis Impact Scale (MSIS-29) Psychological component, EQ-5D-3L 5 health dimensions (Mobility, Self-Care, Usual Activities, Pain/Discomfort, Anxiety/Depression), EQ vertical visual analogue scale (EQ VAS) and patient-reported Kurtzke Expanded Disability Status Scale (EDSS) Ambulation score. The Pearson product-moment correlation coefficient (Pearson’s r) for each concurrent measure is listed in the fourth column, alongside its 2-tailed significance value (P-value) in the next column. Concurrent measures with statistically significant bivariate correlation were written in bold and labelled with ‘*’.
LUN-MS inter-domain correlation matrix.
Abbreviations: The hypothetical constructs measured were: NP, Neuropsychological; AMB, Ambulation; P, Physical; IR, Interpersonal Relationship; I, Informational.
Inter-domain correlation matrix of Long-term Unmet Needs in multiple sclerosis (LUN-MS). Two-tailed significance value (P-value) were P < .001 across all correlation coefficients except for NP with AMB, which was at P < .01.
Discussion
The PROMs used in MS research, such as MSIS-29 and MSWS-12, identify limitations and functional impairments and do not directly assess the unmet needs pwMS. The previous studies on unmet needs of pwMS did not use PROMS. They used questionnaires and semi structured interviews administered by trained researchers. These questionnaires were neither designed as PROM nor for administration in a clinic setting.7,8 Our study showed that the LUN-MS is a valid and reliable tool to identify the unmet needs of pwMS. The tool has good concurrent and construct validity. The test-retest reliability was satisfactory for all 29 individual items, with most items having moderate to substantial level of agreement. The MSIS-29 is an established research tool in MS. We wanted to compare the reliability of our tool against the reliability of an existing PROM in MS. The comparison of test-retest reliability between the LUN-MS and MSIS-29 showed that the reliability of individual items of LUNS-MS is equivalent or better than that of the MSIS-29.
Principal component analysis served as a guide to identify patterns and associations between the questionnaire items. We used clinical rationale to group the thematically linked items into five domains (neuropsychological, ambulation, physical, interpersonal relationship and informational). The domains demonstrated significant unidimensional reliability, indicating robust inter-correlations among the variables within each respective domain.
In line with the COSMIN, as there exists no ‘gold standard’ PROM, we decided to not assess the criterion validity of LUN-MS. 17 The strong correlation between the total LUN-MS score with the total MSIS-29 and EQ-5D-3L scores signalled good concurrent validity. Significant monotrait-heteromethod correlations with chosen surrogate measures evaluating specific domain implied good convergent validity for all hypothetical constructs (Table 11). The results for the discriminant validity were mixed, with neuropsychological and ambulation domains showing good discriminant validity while physical and interpersonal relationship showed ambiguous discriminant validity. Discriminant validity was not done for informational needs due to lack of an established PROM to measure against it.
The physical needs domain of LUN-MS showed significant correlation with the psychological component of MSIS-29 (r = 0.58), suggesting codependency between the three elements (Table 11). This was partly explained by the classification difference of certain symptoms, with the MSIS-29 labelling sleep disturbances and fatigue as psychological elements, whereas we have grouped them with the physical symptoms. The initial study on MSIS-29 also demonstrated a significant inter-construct correlation between its physical and psychological scales (r = 0.62). 14
The LUN-MS enabled the participants to identify whether the needs were met or unmet rather than simply acknowledging the presence or absence of needs. This allows to distinguish between those whose needs were already met and those with unmet health needs. The strong correlation between the total LUN-MS score and MSIS-29 indicates that unmet needs correlate with the impact of MS in person’s physical and psychological wellbeing. The items 4, 7, 8 and 10 showed floor effect on the normality testing due to the lack of dispersion between the responses. 35 This could either be due to these needs being infrequent or adequate service provisions were already in place to meet the specific needs. Nevertheless, this did not affect the core purpose of LUN-MS, which is to identify the unmet needs of pwMS.
The participants found the questionnaire to be acceptable and accessible. Minimal missing values were evident from the missing data analysis and were regarded as MCAR, eliminating potential intrinsic components to the missing mechanism. This questionnaire has the potential to be used as a pre-clinic appointment checklist by patients, allowing them to prepare for their upcoming visits and prioritise what they need to ask their clinicians during the brief clinical encounter. A completed LUN-MS questionnaire could draw the attention of clinicians to the unmet needs of pwMS thus the limited appointment time could be used efficiently to focus on these needs.
UK MS Register, a national digital platform for MS collects data from directly from the NHS clinical record and from pwMS via an online ‘portal’. 10 Our next aim is to explore the option of integrating LUN-MS into UK MS Register. In our opinion, the data on unmet needs could identify gaps in service provisions for pwMS in different geographical areas. It also could help to identify the unmet needs of different social and ethnic groups. It could also be used as a powerful tool to monitor the effect of service or policy changes. LUN-MS could also provide valuable data for clinical audits which would ultimately facilitate service planning and delivery of care to pwMS.
Limitations
This is a single centre study done in a tertiary MS centre in NHS England with 93% (n = 82) of participants on DMDs. The results were derived from pwMS that are mostly already on DMDs. The median EDSS for the study cohort was 5.5 (disability level that affects full daily activities but still able to ambulate for about 100 m without walking aids 12 ), with only 15% (n = 13) of the participants having an EDSS score of 6.0 or above (requiring assistance to walk), indicating a lack of representation for those on the higher burden of disability. In UK, recommended stopping criteria for DMD is EDSS of 7.0 (inability to walk). Even though 93% of our participants were receiving DMD, the median EDSS was unexpectedly high (5.5), the reason behind this anomaly remains unexplained. We did not calculate the sample size based on number of questions before the study.
We used proxy tools (MSIS-29 and EQoL EQ-5D-3L) to validate LUNS-MS. The LUN-MS questionnaire alone does not provide insights into the underlying reasons behind the unmet needs. As we have eliminated individuals who experienced disease-related events during the study period, we were unable to comment whether the needs were dependent on activity of MS. The exclusion of individuals experiencing disease-related events during the study period was necessary as the evaluation of test-retest reliability required patients with stable MS between the administrations of the two questionnaire packs. However, we understand that including the patients experiencing a relapse to be important as unmet needs are likely to arise when patients experience such events during their disease. Further work is necessary to include pwMS who are currently experiencing disease-related events, as this can provide valuable insights into their specific unmet needs.
The COSMIN systematic review for PROMs stated that correlations with instruments measuring similar latent construct should be 0.50 or greater and correlations with instruments measuring related but dissimilar constructs should be 0.30 to 0.50. 17 As the PROMs we used for comparisons were not measuring the same latent constructs, we evaluated the convergent validity for each construct on an individual basis in relation to their respective discriminant correlation values.
A major part of this study took place while the restrictions related to COVID-19 pandemic were in place. It is possible that some of the responses were influenced by the restrictions associated with the pandemic and.
Conclusion
This study showed that LUN-MS is a valid and reliable measure to assess the unmet needs of pwMS. The pwMS found LUN-MS questionnaire acceptable. A nationwide study using LUN-MS on platforms such as the UK MS register could facilitate identification of unmet needs in pwMS in the UK.
Footnotes
Appendix
Please read each statement and answer it as follows
Tick ‘YES This is a problem and I would like help with it’ if you currently have this problem and would like help.
Tick ‘YES I have this problem but I don’t need help with it’ if you have had this problem but it currently isn’t something you would like help with
Tick ‘NO I don’t currently have this problem’ if this is something not bothering you at the moment
Acknowledgements
We acknowledge Professor Anne Forster, the lead of the LoTS Care LUNS Study Team and Head of the Academic Unit for Ageing and Stroke Rehabilitation, University of Leeds, and Bradford Teaching Hospitals NHS Foundation Trust, for permission to modify the LUNS tool.
Funding:
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research received funding from The University of Sheffield in the form of a student grant. It did not receive any specific grant from funding agencies in the public, commercial or not-for-profit sectors. This study was supported by the NIHR Sheffield Biomedical Research Centre (BRC)/NIHR Sheffield Clinical Research Facility (CRF). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care (DHSC).
Declaration of Conflicting Interests:
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Amin Mohamed Abu Baker has no disclosures to make. Harriet Moore has no disclosures to make. Dr Kathleen Baster is employed by the Statistical Services Unit, The University of Sheffield who was paid to provide statistical support and advice on the analysis for this study. She has no disclosures to make. Dr Esther Hobson has no disclosures to make. Dr David Paling reports travel support, honoraria from speaking and ad boards from Novartis, Celgene, Biogen, Sanofi Genzyme, MedDay, Merck and Roche. Prof. Basil Sharrack had led clinical trials of disease modifying drugs for Novartis, Celgene, Biogen, Sanofi Genzyme, Merck and Roche. Prof. Krishnan Padmakumari Sivaraman Nair has led clinical trials on spasticity in MS for Celgene and GWS pharma.
