Abstract
Background:
Few studies have addressed the results of educational efforts concerning proper use of McDonald criteria (MC) revisions outside multiple sclerosis (MS) subspecialty centers. Neurology residents and MS subspecialist neurologists demonstrated knowledge gaps for core elements of the MC in a recent prior study.
Objective:
To assess comprehension and application of MC core elements by non-MS specialist neurologists in the United States who routinely diagnose MS.
Methods:
Through a cross-sectional study design, a previously developed survey instrument was distributed online.
Results:
A total of 222 neurologists completed the study survey. Syndromes atypical for MS were frequently incorrectly considered “typical” MS presentations. Fourteen percent correctly identified definitions of both “periventricular” and “juxtacortical” lesions and 2% correctly applied these terms to 9/9 images. Twenty-four percent correctly identified all four central nervous system (CNS) regions for satisfaction of magnetic resonance imaging (MRI) dissemination in space. In two presented cases, 61% and 71% correctly identified dissemination in time (DIT) was not fulfilled, and 85% and 86% subsequently accepted nonspecific historical symptoms without objective evidence for DIT fulfillment.
Conclusion:
The high rate of knowledge deficiencies and application errors of core elements of the MC demonstrated by participants in this study raise pressing questions concerning adequacy of dissemination and educational efforts upon publication of revisions to MC.
Introduction
Over the last 20 years, diagnostic criteria for multiple sclerosis (MS)—the McDonald criteria (MC)—have been revised approximately every 5 years,1–4 facilitating earlier diagnosis.5–7 MS remains a clinical diagnosis. Accurate application of MC rely on iterative clinical and radiological assessments requiring awareness of syndromes “typical” of demyelinating attacks and criteria-defined “objective evidence” and “dissemination in time and space” as well as attention to atypical presentations that suggest alternative diagnoses. Few studies have addressed proper use of the MC after published revisions, or adoption in practice, particularly outside MS subspecialty centers. Surveys of neurologists in the United Kingdom concerning 2005 and 2010 MC suggested difficulty with comprehension and application, and a low level of adoption in practice.8,9
Recent studies10–12 suggest that misunderstanding and misapplication of core elements of the MC in patients with common diagnoses led to misdiagnosis of MS. MS misdiagnosis is associated with morbidity and considerable unnecessary healthcare cost10,13 and has been identified in 7%–19% of patients referred to MS subspecialty centers in contemporary studies within the United States 11 and Europe. 12 These studies were limited by their retrospective nature. In a recent study, 13 we surveyed neurology residents as well as MS subspecialist neurologists in the United States and Canada to assess application of the MC. Although the number of participants was small, the results demonstrated misunderstanding and misapplication of core elements of the MC, including knowledge gaps for dissemination in space (DIS) criteria even by MS subspecialists. Although many neurology residents graduate without pursuing MS subspecialty training, residents who participated in the study may have not yet completed their education in MS, limiting its conclusions in these trainees.
Many patients in the United States are diagnosed with MS by neurologists without subspecialty training or focus in MS and neuroimmunology. In the present study, we assessed comprehension and application of key elements of the MC in non-MS specialist neurologists in the United States who reported that they routinely diagnose MS. The aim of the study was to identify common knowledge gaps and resulting opportunities for further education that would ensure early and accurate diagnosis of MS.
Methods
A previously developed survey instrument 14 (see supplementary material) was distributed online to practicing neurologists in the United States with the assistance of MedSurvey, a medical market research company that routinely surveys neurologists in practice. This study conforms to Consensus-Based Checklist for Reporting of Survey Studies (CROSS) 15 guidelines for survey studies. Previous demographic, practice characteristic, and diagnostic approach questions were adapted for the non-specialist neurologist population. This survey was validated by testing in MS experts before its previous implementation. 14
Participation was limited to neurologists trained in adult neurology reporting they “sometimes diagnose patients with multiple sclerosis” as part of their clinical practice. Pediatric neurologists and neurologists with “subspecialty training or clinical focus in MS” were excluded. Participants reviewed information about the study before consenting to proceed. Response choices were randomly re-ordered for each participant except where responses were continuous. Data received for analysis were de-identified. MedSurvey maintains identifiable information, including national provider identifiers, to verify the identity of physicians it solicits for survey participation and to prevent repeat participation in surveys. Study participants received an honorarium. The survey was available for 5 weeks during March and April 2021. Reminders were regularly sent to potential participants.
Survey responses are primarily analyzed as descriptive data. Participants who responded correctly versus incorrectly on select MC questions were compared on responses to demographic and practice characteristics and diagnostic approaches. Questions with responses on a continuous scale were evaluated using a two-sided, two-sample t-test, and those with categorical responses were compared using a chi-square test; p-values less than 0.05 from either a t-test or chi-square test were considered statistically significant. Two questions requested participants to rate a statement on a scale of 1 (strongly disagree) to 10 (strongly agree). Since the current study is exploratory, we did not correct for multiple comparisons.
This study was approved by the University of Vermont Institutional Review Board.
Results
Demographic and clinical practice characteristics of participants
A total of 222 neurologists participated and completed the entire survey. Response rate could not be calculated as the number of invited participants by MedSurvey was proprietary information. Sex ratio was 3:1 male to female. The median age was 45.0 (range, 25–78 years) and median years since residency completion was 14 (range, 0–45 years). Participants practiced neurology in one or more of 41 different states and in a variety of practice settings. 85.1% spend greater than 75% or more of their time in direct patient care. Participants estimated diagnosing a median of 10 patients with MS per year (range, 1–200); 90% had a practice with 25% or fewer patients receiving ongoing care for MS (Table 1).
Demographic and clinical practice characteristics.
SD: standard deviation; IQR: interquartile range; Years since graduation: number of years since graduating neurology residency; Subspecialty: non-MS subspecialty fellowship training or practice focus; Academ: Academic institution and associated teaching hospital; AcAffil: Affiliate hospital or practice of an academic institution; HospSys: Hospital System; HOrg: Healthcare organization, or HMO, Private or group practice; PhysO: physician-owned practice; VA: Veterans Affairs Hospital System; CommH: Community public health clinic; % clinical effort: percentage of position or FTE involving the clinical care of patients (vs research or administrative); new MS diagnoses: estimate of the number of new patients diagnosed by participants with MS per year; % patients w/MS: approximate percentage of total patients participant provides ongoing care for who have MS.
In total, 126 of 222 (56.8%) participants had one or more area of subspecialty training or a practice focus. The most frequent were: neurophysiology—electroencephalography (EEG) 38 (30.2%), neurophysiology—electromyography (EMG) 35 (27.8%), neuromuscular 23 (18.3%), vascular neurology (stroke) 21 (16.7%), epilepsy 21 (16.7%), and sleep medicine 17 (13.5%). The remainder reported were autonomic disorders, behavioral or cognitive neurology, headache, movement disorders, neuro-oncology, neuro-ophthalmology, neuro-hospitalist, pain medicine, neurocritical care, neurorehabilitation, or brain injury.
MS typical syndromes
Participants reviewed a case (see survey instrument) of a 49-year-old patient with multiple vascular comorbidities associated with brain magnetic resonance imaging (MRI) white matter abnormalities and paroxysmal sensory symptoms atypical for demyelination and non-localizing to the central nervous system (CNS) by history and neurological examination. Brain MRI fulfilled DIS but not dissemination in time (DIT), spinal cord imaging was normal, and cerebrospinal fluid (CSF) was not obtained. A total of 127 (57.2%) participants correctly identified that although the MRI potentially fulfilled DIS, this patient presented with a syndrome atypical for demyelination and as a result diagnosis of MS by 2017 MC could not be fulfilled. 1
Participants were asked to identify typical clinical presentations of MS from a list referenced by the 2017 MC 1 containing both “typical” and “atypical or red flag” presentations. 16 Results are presented in Table 2.
Correct and incorrect identification of typical clinical presentations of MS.
MS: multiple sclerosis; CNS: central nervous system.
MRI dissemination in space
Identification of periventricular and juxtacortical MRI lesions
Identification of T2 hyperintense periventricular lesions was assessed on five single fluid-attenuated inversion recovery (FLAIR) brain MRI slices (3 axial and 2 sagittal views), and T2 hyperintense juxtacortical lesions on four single FLAIR brain MRI slices (2 axial and 2 sagittal views) (Figure 1). Correct responses averaged 25.2% for periventricular lesion questions and 21.3% for juxtacortical lesion questions. Four (1.8%) participants correctly answered all 9/9 questions, four (1.8%) correctly answered 8/9, and three (1.4%) correctly answered 7/9.
Six examples of study survey images that evaluated participant knowledge for periventricular and juxtacortical lesion location.
Participant knowledge for MC definitions 1 of “periventricular” and “juxtacortical” MRI lesion location was assessed after application of these terms in prior questions. Sixty-six (29.7%) participants correctly chose “a lesion touching the ventricle” and 57 (25.7%) correctly chose “a lesion touching the cortex.” Out of 222 participants, 39 (14.4%) correctly identified both definitions.
Identification of regions for MRI dissemination in space
Participants were asked to identify CNS regions satisfying MRI DIS criteria from an available list. Correctly selected regions included infratentorial by 169 (76.1%), juxtacortical by 207 (93.2%), periventricular by 211 (95.1%), and spinal cord by 207 (93.2%). Incorrectly selected regions included optic nerve by 135 (60.8%), subcortical by 96 (43.2%), and deep white matter by 78 (35.1%). A total of 53 (23.9%) participants selected all 4 correct regions without selecting any incorrect regions.
Lesion size criteria
The minimum MRI lesion diameter of 3 mm specified by the MC 1 was correctly chosen by 88 (39.6%) participants.
Clinical dissemination in time
Participants were presented with three cases of clinically isolated syndrome (CIS) not yet fulfilling DIT (see survey instrument). The first case was CIS in the form of partial myelitis. In total, 136 (61.3%) participants correctly responded that DIT criteria were not satisfied. When provided with additional history of painless unilateral blurred vision of 2–3 days duration occurring 2 years prior, 115 of 136 (84.6%) responded that DIT was fulfilled based on this history alone. Out of the 21 remaining participants, 13 (61.9%) participants responded that objective evidence—neurological examination findings of an afferent pupillary defect and a prolonged visual evoked potential—would have satisfied DIT in this patient.
The second case was CIS in the form of unilateral internuclear ophthalmoplegia. A total of 157 (70.7%) participants correctly responded that DIT criteria were not satisfied. When provided with an additional history of a past history of painless left leg numbness and tingling and coordination difficulty 4–5 years prior that resolved after 1 week, 135 of 157 (86.0%) responded that the presentation now fulfilled DIT based on this history alone. Out of the 22 (54.6%) remaining participants, 12 participants responded that objective evidence—neurological examination findings of left leg hyperreflexia, clonus, and T2 lesion on MRI corresponding to historical symptoms—would have satisfied DIT in this patient.
The third case was CIS also in the form of partial myelitis. When presented with details of a recurrent identical syndrome of myelitis 11 months later, 86 (38.7%) indicated that the patient satisfied DIT. The remaining 136 (61.3%) indicated that MRI objective evidence of a new lesion was necessary to fulfill DIT.
MS diagnosis and education in practice
A total of 126 (56.8%) participants reported that they had reviewed the 2017 MC manuscript since its publication. Table 3 provides responses to questions concerning practice approaches to a new diagnosis of MS. Participants were asked how often they made a new diagnosis of MS in a patient who had both a normal spinal cord MRI and a normal spinal fluid examination and 34 (15.3%) responded “never,” 72 (32.4%) “rarely,” 92 (41.4%) “occasionally,” 24 (10.8%) “often,” and 0% “always.”
Practice approaches to new MS diagnoses.
CSF: cerebrospinal fluid; MRI: magnetic resonance imaging; DIS: dissemination in space; MS: multiple sclerosis.
The statement that the MC was “easy to understand and apply for diagnosis of MS in routine practice” was rated with a mean of 6.2 on the Likert-type scale (range 1 to 10 with 10 “strongly agree”). “Occasionally misdiagnosing MS in patients who do not have MS is necessary in order to prevent under-diagnosing MS, and to start treatment, in patients who do have MS” received a mean rating of 4.3.
The educational venues where participants reported they typically received updates to the MC or MS diagnosis prior to the Covid-19 pandemic were as follows: 175 (78.8%) educational or continuing medical education (CME) sessions at national meetings, 172 (77.5%) “academic journal articles,” 101 (45.5%) online CME programs, 70 (31.5%) CME programs sponsored by local universities, and 43 (19.4%) regional industry-sponsored CME programs. The 175 who indicated national meetings as an important source of education were asked to choose specific meetings they attended, and the American Academy of Neurology Annual Meeting was chosen 169 times, and other meetings were cumulatively chosen 22 times.
Responses compared to demographic and practice characteristics
Participants providing correct (125/222 or 56.3%) and incorrect (54/222 or 24.3%) responses for both of the first two CIS cases regarding initial fulfillment of DIT were compared on demographic and practice responses. Participants who answered correctly were younger (mean ± standard deviation: 45.7 ± 13.0 vs 50.2 ± 12.8 years; p = 0.035) and completed neurology residency more recently (14.6 ± 12.8 vs 19.2 ± 13.0 years since 2021; p = 0.032). Compared to other practice types reported, participants in physician-owned practices were more likely to choose incorrect responses for DIT for both questions (23/54 (42.6%) vs 29/125 (23.2%), p = 0.015). No additional differences in demographic or practice characteristics, including number of diagnoses of MS per year or percentage of practice devoted to clinical effort, were observed based on these two questions.
Participants who answered both MRI DIS definition questions concerning the terms “periventricular” and juxtacortical” lesion location either correctly (39/222 or 14.4%) or incorrectly (138/222 or 62.2%) were compared on demographic and practice responses and there were no significant differences between the groups, including responses regarding reliance on radiologist interpretation of MRI.
Discussion
In this study, 222 neurologists who routinely diagnose MS as part of their clinical practice demonstrated a high rate of knowledge deficiencies and application errors of core elements of the MC. These findings support prior studies suggesting misunderstanding and misapplication of the MC is common and may frequently contribute to MS misdiagnosis.10,14 These data raise pressing questions concerning adequacy of dissemination and educational efforts upon publication of revisions to MC, as well as the design of future revisions to promote ease of use and adoption in practice.
Since the Schumacher criteria, 17 diagnostic criteria for relapsing-remitting MS have been designed for and validated in patients with a restricted set of clinical presentations or syndromes “typical” for an MS attack or relapse. Application in nonspecific or atypical syndromes is expected to diminish specificity and increase the risk of misdiagnosis. 18 However, almost half of the participants in this study (43%) misidentified a case with numerous clinical red flags atypical for MS, as a syndrome appropriate for MC application and typical for MS. When presented with specific typical and atypical syndromes referenced by the MC, 11 almost half (38%, 45%, and 47%) misjudged three “red flag” atypical presentations as typical for MS that should have raised concern for alternative disorders. Misunderstanding of the term “typical presentation” may have influenced survey responses and may impact MC application in practice. While fatigue and cognitive impairment are common symptoms of MS, application of MC in patients with relapse-onset MS is not intended for presentations consisting of these non-specific symptoms alone; yet they were frequently chosen by participants.
Knowledge gaps for the assessment of DIS by MRI were evident. The optic nerve (61%), subcortical white matter (43%), and deep white matter (35%) were incorrectly identified as regions for DIS fulfillment. Although periventricular and juxtacortical location was correctly chosen as a DIS region from a list by almost all participants, application of these terms to MRI images resulted in average correct responses of only 21% and 25%, respectively. Only 11 (5%) participants correctly answered 7 or more of 9 questions applying these terms. Similarly, a third or less of participants chose the correct definitions for periventricular (30%) and juxtacortical (26%) lesions, and only 14% identified both definitions correctly. Yet the majority of neurologists participating in the survey (80.1%) reviewed MRI images themselves when making a new diagnosis of MS, with 19.9% reporting that they either often or always instead rely on radiologists for MRI determination of DIS.
Neurologists participating in the study demonstrated difficulty assessing DIT and “objective evidence” of a CNS lesion as defined by the MC. In two hypothetical cases of a typical CIS, 30% and 39% of participants incorrectly indicated DIT was fulfilled in patients with a monophasic syndrome in the absence of a new clinical relapse, new or enhancing MRI T2 lesions, or reported the presence of intrathecal IgG CSF markers. However, the majority of participants who initially correctly noted that DIT was not fulfilled were subsequently willing to accept non-specific prior neurological symptoms as fulfillment of DIT without objective evidence. By contrast, the 2017 MC emphasize that “caution should be taken in accepting historical events as an attack in the absence of contemporaneous or current objective evidence providing corroboration.” 1 In a case of monophasic myelitis, recurrence of previous clinical symptoms and deficits, frequently a sign of a MS pseudo-relapse, 19 was accepted as DIT by 39% without imaging to confirm a new lesion.
These data suggest that knowledge gaps and misapplication of the MC are frequent and may be a common cause of MS misdiagnosis.10–12,20 Application of MC in patients with nonspecific or atypical syndromes using incorrect regions and lesion size for MRI DIS 21 without corroborating objective evidence of a prior CNS lesion for DIT greatly reduces their specificity for MS. 18 Uncertainty surrounding the correct diagnosis in such patients combined with perceived urgency to treat MS early may contribute to misdiagnosis in these situations. 22 Indeed, many neurologists participating in the study agreed that they would accept occasional misdiagnosis a necessary consequence of diagnosing MS early. Eleven percent of participants also indicated they “often” diagnosed MS in patients with both normal CSF and normal spinal cord imaging. Although data in the literature are insufficient to indicate that this is never justified, this presentation is uncommon in patients correctly diagnosed with MS and potentially poses a higher risk of misdiagnosis. This study highlights that in addition to key elements of the MC, specific education defining an appropriate balance between early diagnosis and risk of misdiagnosis when using the MC is also needed.
This study has limitations. Inability to capture the response rate and limited characterization of participating neurologists makes it impossible to address issues of selection bias that might have influenced results and impacted the generalizability of the data. The survey was not previously validated in practice; therefore, it is uncertain how well responses mirror application of the MC in routine care. Reproducibility was also not tested. CSF results were not incorporated into questions assessing fulfillment of DIT in the previous survey that was contemporaneous to publication of 2017 MC—this was maintained to allow comparisons between the studies. Assessment of progression and diagnosis of primary progressive MS were not specifically evaluated. The study was limited to the United States. The proportion of women neurologists in practice is difficult to determine, and although women comprised only 25% of our participants, a recent study found that as of 2016 31.5% of American Academy of Neurology members were women. 23 Finally, neurologists reported diagnosing a median of 10 new MS diagnoses per year, likely accounting for a small proportion of incident cases in the United States. 24 Although these limitations may influence the representativeness of the data and should prompt further study utilizing more rigorous methodology, neurologists who participated in the study estimated collectively diagnosing approximately 2000 new cases of MS per year, and in this light these data remain concerning and interventions to improve knowledge gaps identified potentially impactful.
This study suggests that current dissemination and educational efforts focused on the MC are inadequate. Approximately half (44%) of this cohort had not reviewed the most recent MC revision published over 3 years ago, and approximately one fifth (21.6%) expressed that the criteria were neither easy to understand nor to apply. This study and our prior data 13 that reveal a high rate of misunderstanding and misapplication of the MC suggest that further educational activities are needed. Neurology residents and MS specialists performed better on questions focused on DIT in our prior study. 14 Yet many knowledge deficiencies were consistent across both surveys, particularly regarding MRI DIS, and even in MS specialists. Knowledge assessment for MRI DIS and DIT in radiologists whom neurologists may rely upon for these determinations should also be considered.
Neurologists in this study indicated online continuing medical education, academic journals, and sessions at the American Academy of Neurology annual meeting as their predominant source of education, and future efforts should take this into account. Targeted educational efforts for residency and MS fellowship curricula as well as MS subspecialty meetings are also needed. Consensus guidelines on how to optimally design, disseminate, and encourage adoption of clinical diagnostic criteria are lacking but the literature on implementation of practice guidelines may be informative.25–28 Recent research suggests that active learning approaches,29–33 rather than traditional didactic content, may prove more successful. Implementation of specific interventions such as “virtual patient simulation” may improve diagnostic performance further. 34 To ensure early and accurate diagnosis of MS, this study justifies further investigation utilizing rigorous methodologies to characterize knowledge gaps for key elements of the MC in practice in order to optimally design and implement appropriate concerted educational interventions.
Supplemental Material
sj-pdf-1-msj-10.1177_13524585211048401 – Supplemental material for Multiple sclerosis diagnosis: Knowledge gaps and opportunities for educational intervention in neurologists in the United States
Supplemental material, sj-pdf-1-msj-10.1177_13524585211048401 for Multiple sclerosis diagnosis: Knowledge gaps and opportunities for educational intervention in neurologists in the United States by Andrew J Solomon, Marwa Kaisey, Stephen C Krieger, Salim Chahin, Robert T Naismith, Sarah M Weinstein, Russell T Shinohara and Brian G Weinshenker in Multiple Sclerosis Journal
Supplemental Material
sj-pdf-2-msj-10.1177_13524585211048401 – Supplemental material for Multiple sclerosis diagnosis: Knowledge gaps and opportunities for educational intervention in neurologists in the United States
Supplemental material, sj-pdf-2-msj-10.1177_13524585211048401 for Multiple sclerosis diagnosis: Knowledge gaps and opportunities for educational intervention in neurologists in the United States by Andrew J Solomon, Marwa Kaisey, Stephen C Krieger, Salim Chahin, Robert T Naismith, Sarah M Weinstein, Russell T Shinohara and Brian G Weinshenker in Multiple Sclerosis Journal
Footnotes
Author Contributions
A.J.S., M.K., S.C.K., S.C., R.T.N., and B.G.W. contributed to drafting/revision of the manuscript for content, including medical writing for content; major role in the acquisition of data; study concept or design; and analysis or interpretation of data. S.M.W. and R.T.S. contributed to drafting/revision of the manuscript for content, including medical writing for content; and analysis or interpretation of data.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: A.J.S. has consulted for EMS Serono, Genentech, Biogen, Alexion, Celgene, and Greenwich Biosciences. S.C. has received consulting and/or speaking fees from Biogen, Genentech, Genzyme, and Novartis. M.K. has received honoraria for speaking engagements from Alexion, Biogen, Genentech, and Viela Bio. S.C.K. reports consulting or advisory work with Biogen, EMD Serono, Genentech, Genzyme, Mallinckrodt, MedDay, Novartis, Octave, Teva, and TG Therapeutics, and non-promotional speaking with Biogen, EMD Serono, Genentech, and Novartis. Grant and research support from Biogen and Novartis. R.T.N. has consulted for Biogen, Bristol Myers Squibb, Genentech, Genzyme, GW Therapeutics, Janssen, Lundbeck, Nervgen, Third Rock Ventures, TG Therapeutics, and Viela Bio. S.M.W. reports no conflicts. R.T.S. reports consulting income from Octave Bioscience. I also receive compensation for reviewing scientific articles from the American Medical Association and for reviewing grants for the Emerson Collective, National Institutes of Health, and the Department of Defense. B.G.W. receives royalties from RSR Ltd, Oxford University, Hospices Civil de Lyon, and MVZ Labor PD Dr. Volkmann und Kollegen GbR for a patent of NMO-IgG as a diagnostic test for neuromyelitis optica spectrum disorders, served on adjudication committee for clinical trials in neuromyelitis optica spectrum disorders being conducted by MedImmune/VielaBio, Alexion, UCB Biosciences and consulted for Chugai/Roche/Genentech and Mitsubishi-Tanabe regarding a clinical trial for neuromyelitis optica spectrum disorders. He has received honoraria for speaking at internal meetings of Genentech, Novartis, and external meetings for Roche.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: We wish to acknowledge funding from Biogen that supported development of the study survey instrument for a prior study.
Supplemental Material
Supplemental material for this article is available online.
Statistical Analysis
Statistical analysis is performed by Sarah M Weinstein, MS, PhD candidate at the Penn Statistics in Imaging and Visualization Center, Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
