Sage Journals: Discover world-class research

Abstract

Randomized controlled clinical trials and real-world observational studies provide complementary information but with different validity. Some clinical questions (disease behavior, prognosis, validation of outcome measures, comparative effectiveness, and long-term safety of therapies) are often better addressed using real-world data reflecting larger, more representative populations. Integration of disease history, clinician-reported outcomes, performance tests, and patient-reported outcome measures during patient encounters; imaging and biospecimen analyses; and data from wearable devices increase dataset utility. However, observational studies utilizing these data are susceptible to many potential sources of bias, creating barriers to acceptance by regulatory agencies and the medical community. Therefore, data standardization and validation within datasets, harmonization across datasets, and application of appropriate analysis methods are important considerations. We review approaches to improve the scope, quality, and analyses of real-world data to advance understanding of multiple sclerosis and its treatment, as an example of opportunities to better support patient care and research.

Keywords

Multiple sclerosis real-world data real-world evidence observational studies pragmatic clinical trials

Introduction

Disease course and severity of multiple sclerosis (MS) vary between individuals and over time, and factors influencing these outcomes are poorly understood. The availability of an increasing number of disease-modifying therapies (DMTs) has made treatment decisions more complicated.^1,2 Randomized controlled clinical trials (RCTs; Table 1 provides a glossary of key terms) have been invaluable in evaluating DMTs, since their design reduces the risk of alternative explanations for results that can affect non-randomized studies.^6,7 However, MS treatment has not advanced to evidence-informed personalized medicine. Major questions remain concerning risk stratification, treatment sequencing, outcome definition and monitoring, and biology-based selection of treatment for individual patients, for which RCTs are not feasible.⁸ Moreover, results from RCTs might not generalize to clinical populations due to trial restrictions on age, disease type, comorbidities, treatment environment, and other factors.⁸ Finally, typical, relatively short RCTs fail to provide information about the long-term risks and benefits of therapies.⁸

Table 1.

Glossary of terms.

Term	Definition
Big data	Extremely large or complex collections of data, which can be mined using non-traditional data processing approaches
Causal inference	The process of reaching a conclusion about whether a causal relationship exists among variables
Comorbidity	Total burden of illness other than the specific (index) disease of interest
Data	Set of values for one or more variables, either quantitative or qualitative
Database	An organized collection of information which is stored electronically for ease of storage, management, and retrieval
Dataset	Data stored in tabular form, for example, rows and columns, often to enable analysis
Disease behavior	Clinical, imaging, and non-biomarker manifestations of MS; their interrelatedness; and how they evolve over time
Effectiveness	Therapeutic benefit under usual use in clinical practice
Efficacy	Therapeutic benefit under optimal circumstances in restricted populations, for example, in a traditional randomized clinical trial
Electronic medical record	An electronic version of a patient’s medical history maintained by the provider over time. May include all of the key administrative clinical data relevant to that person’s care under a particular provider, including demographics, progress notes, problems, medications, vital signs, past medical history, immunizations, laboratory data, and radiology reports³
Explanatory clinical trial	A clinical trial primarily designed to determine the safety and efficacy of an intervention under optimal circumstances
Harmonization	The process of combining data from different sources to make measurements, methods, data format, and terminologies compatible. Prospective harmonization is conducted a priori, that is, before data collection. Retrospective harmonization is conducted using data that are already collected. Harmonization can be stringent, the most stringent form being standardization (e.g. using identical methods), or flexible (e.g. using differing data collection methods)
Observational study	A study that draws inferences from an identified sample (such as clinical records or administrative data) to a larger population, where the investigator does not assign treatment but simply notes and analyzes outcomes
Patient-reported outcome measure	A report of a patient’s health status that comes directly from the patient without any interpretation of the patient’s response by a clinician or anyone else (US Food and Drug Administration); an outcome evaluated directly by the patient himself or herself and based on the patient’s perception of the disease and its treatment (European Medicines Agency)
Pragmatic clinical trial	Clinical trial embedded in clinical practice primarily designed to determine the effects of an intervention under the usual conditions of use
Randomized controlled trial	A formal experiment testing the efficacy and safety of an intervention on human subjects, in which the investigator attempts to control variability, confounding factors, and bias by defining the population studied, randomized assignment and application of the treatments, ascertainment and measurement of the outcomes, and analysis of the results;⁴ clinical trials generally exist on the spectrum of explanatory or pragmatic design
Real-world data	Data generated in routine clinical practice that capture an individual’s health status or health services use
Real-world evidence	Evidence concerning the risks and benefits of an intervention (or interventions) from analysis of real-world data
Registry	An organized system that uses observational study methods to collect uniform data to evaluate specified outcomes for a particular disease, condition, or exposure and that serves one or more predetermined scientific, clinical, or policy purposes⁵
Standardization	Creation and adoption of uniform technical specifications, criteria, methods, and processes to measure an item, which can be regarded as the most stringent form of (prospective) harmonization

Real-world data (RWD) are used increasingly to address clinical questions related to MS disease behavior, prognosis, and treatment (Table 2). MS registries including RWD have existed for 70 years,⁹ but have proliferated recently. Over 500,000 persons with MS have participated in registries and related efforts worldwide,¹⁰ leading to treatment recommendations that guide daily practice.⁸ Experience with RWD emphasizes the need for collaboration to address data quality and harmonization, sharing of data across jurisdictions with differing legal and ethical regulations, and sustainability of data sources.^8,11 This review identifies key clinical questions best addressed using RWD; critically evaluates the current uses of available registries, databases, and other MS RWD sources; discusses challenges and barriers to gathering, using, sharing, and pooling such data; and provides conceptual, structural, and analytical solutions to address them.

Table 2.

Multiple sclerosis–related questions addressed using real-world data.

Disease behavior

• Defining disease course, phenotype, and prognosis, including in specific populations

• Predictors of long-term disability

• Impact of comorbidities, health behaviors, genetics and epigenetics, microbiome, and environment

• Pre-diagnosis manifestations

Diagnostic criteria

• Evaluation of the performance of updated criteria compared to previous versions

• Evaluation of the performance of diagnostic criteria in diverse populations

Disease outcomes

• Measurement properties, validity, and usefulness of outcome measures, including clinician-assessed outcomes, patient-reported outcomes, imaging outcome measures, non-imaging paraclinical markers, and health-care utilization

• Interrelationships between outcome measures

• Definition of clinically meaningful change (within an individual) of outcomes

Therapeutic effectiveness of disease therapies

• Long-term effectiveness

• Definitions and predictors of treatment response/non-response

• Comparative effectiveness of treatments in different scenarios, for example, relapsing-remitting disease, progressive disease, as initial therapy, or upon treatment switching

• Comparative effectiveness of treatment strategies, for example, treat-to-target, escalation versus early high-efficacy treatment

• Economics and cost–benefit evaluation

• Utilization in clinical practice

Treatment safety and tolerability of disease therapies

• Safety and risk/benefit assessments, including less common adverse events, delayed adverse events, and cumulative risk of adverse events

• Comparative safety and tolerability between treatments

• Comparative safety in treated subjects versus untreated patients or healthy individuals

• Factors that determine or predict safety and tolerability, including comorbidities, health behaviors, concomitant medications, genetics, race and ethnicity, and therapy adherence and persistence

• Reproductive safety, including to the mother (e.g. ovulation, fertility, pregnancy, breastfeeding), to the father (e.g. sperm production and quality, fertility), to the fetus (e.g. developmental abnormalities), newborn (e.g. short-term impact on developmental milestones), and child (e.g. longterm impact)

Therapeutic effectiveness of non-medication approaches and effects on disease course of other factors

• Benefit of rehabilitation strategies

• Benefit of management of comorbidities or health behaviors

• Effects of pregnancy and breast feeding

RWD

RWD related to an individual’s health status or health services use are collected from various sources—product and disease registries, electronic medical records (EMRs), and administrative (health claims) databases. Other data types include clinician-assessed and performance measures, patient-reported outcome measures (PROMs), imaging studies and biospecimens, and data from wearable devices. The sources of data included in current registries were recently summarized.¹¹ No single data source can address all study questions. While each source presents specific opportunities and challenges (Supplementary Material, Supplementary Table 1), certain issues are common to all RWD sources. Data quality control is critical to the quality of related research.¹² Data quality control begins with developing a data quality framework which includes a list of data to be collected, groups the data according to data sources, lists the common sources of error for each data source, and identifies potential strategies to prevent or mitigate them. The framework should identify the data quality dimensions (e.g. accuracy, completeness, internal consistency) which are important given the intended uses of the data and list the metrics and operational definitions for each data quality dimension. Finally, the framework should describe data monitoring frequency and how data quality will be reported.¹³

Registries

Registries use observational study methods to collect uniform health status data and are defined by their focus on a particular disease or exposure.⁵ Their purpose is defined a priori, and some aspects of data collection are active (solely for registry purposes). Data collection is focused on the registry’s goals, since stakeholder participation may depend on data collection burden. In multicenter disease registries, ensuring high data quality is a major challenge.¹⁴ All participating centers must follow a plan to collect the same minimum dataset using consistent tools, standardized definitions, and trained assessors. Data quality should be routinely evaluated using systematic quality control procedures.^15,16

EMR data

EMRs are increasingly used in MS care,¹⁷ facilitating clinical information capture, though such data are typically unstructured. Available natural language processing or other techniques that extract information from unstructured data are of limited use,¹⁸ but utilization of structured data fields makes the information more useful for research. Systematic capture of health status data can inform the care of individual patients contributing the data and improve disease management generally, a so-called “learning health system.”¹⁹ This model may be more scalable than passive data collection or active data collection added to clinical care, increasing the range of data elements collected and improving the representativeness of the patients (Table 3 presents an example).

Table 3.

Example of a learning health system in multiple sclerosis.

What is Multiple Sclerosis Partners Advancing Technology and Health Solutions (MS-PATHS)?

• MS-PATHS is a multi-institutional observational study sponsored by Biogen that uses technology to create and capture standardized data from people with MS during routine office visits.²⁰ By reducing heterogeneity in the way clinical information is captured and reported, the goal is to enable and accelerate research in a large sample of people with MS while providing information to monitor outcomes and support clinical decision-making. The project focuses on data collected as part of clinical care that are comparable, sharable, and usable for research studies. Biogen received feedback from the FDA, so these data potentially could be used for regulatory decision-making.

How are the data collected?

• Patients arrive before clinical visits to complete data gathering, using a device (MSPT).²⁰ The MSPT includes a battery of neuroperformance tests modeled after the MSFC²¹ and PROMs (disease history, patient-reported relapses, PDDS, Neuro-QoL, PHQ-9, and PROMIS physical health).

• Routine clinical MRIs are acquired according to a standardized protocol and shared with the research platform.

• Structured EMR data are also shared with the research platform.

• Data are uploaded immediately to the EMR to be available at the clinical encounter, and anonymized data are simultaneously uploaded to a cloud-based multi-institutional pooled database.

What are the implementation issues?

• Clinicians need to change workflow.

• Some information relevant to clinical care and research, such as neurologist-determined relapse occurrence and EDSS, is not currently captured.

• Some standardized information captured (e.g. symptoms of depression) is more familiar to clinicians than others (e.g. quantification of emotional/behavioral dyscontrol).

• Acquisition and reporting of imaging studies need to be standardized, and radiologists need to change the workflow.

• Pooling of data across institutions and secondary use of clinical data for research require oversight by institutional review boards.

• Information technology teams are needed to design the approach to push data to the EMR and outside of the institutional firewall to the pooled database.

MS: multiple sclerosis; FDA: US Food and Drug Administration; MSPT: Multiple Sclerosis Performance Test; MSFC: multiple sclerosis functional composite; PROM: patient-reported outcome measure; PDDS: Patient-Determined Disease Steps; Neuro-QoL: Quality of Life in Neurological Disorders; PHQ-9: Patient Health Questionnaire–9; PROMIS: Patient-Reported Outcomes Measurement Information System; MRI: magnetic resonance imaging; EMR: electronic medical record; EDSS: Expanded Disability Status Scale.

Administrative (health claims) databases

Administrative data result from the delivery of health care services, reimbursement for health care services, or enrollment into health insurance plans.²² They may be generated by government or private (commercial insurance) organizations. Information captured may include demographics, diagnoses, procedures, medications, and health services used.

Clinician-reported data and performance measures

In MS studies, the most common clinician-reported data comprise DMT utilization, reasons for changing therapy, relapses, and disability measures (e.g. rating scales based on the neurological examination such as the Expanded Disability Status Scale (EDSS)).²³ Performance-based measures, such as the MS Functional Composite,²³ quantify patients’ ability to perform functional tasks. They assess what the person can do, rather than what the clinician reports (e.g. EDSS) or what the person perceives he or she can do (e.g. PROMs).

Patient-reported data sources

While some domains relevant to MS can be assessed by clinicians (e.g. disability), others can only be assessed using PROMs (e.g. subjective symptoms or quality of life). PROMs are collected from the patient without interpretation or influence of a clinician or other party and are intended to capture a comprehensive picture of the impact of MS and its treatments.²⁴ Regulatory agencies strongly encourage their use in RCTs.²⁵ PROMs are increasingly incorporated in patient care and registries.²⁶

Several factors should be considered when selecting PROMs, such as validity, reliability, and responsiveness; relation to the research question; relevance to patient experience; and feasibility of use. PROMs are either generic, allowing comparisons across diseases, or symptom- or disease-specific, which might be more sensitive to changes in a particular domain or population. Challenges exist in harmonizing the results of different PROMs assessing the same domain (e.g. fatigue) and accounting for cross-cultural and environmental differences when using a PROM in different populations.²⁷

Imaging

Harmonization of magnetic resonance imaging (MRI) data acquired through patient care to allow secondary use for research presents challenges related to standardizing image acquisition, analysis, and reporting.^28,29 Structured radiology report templates that quantify active lesions, lesion burden, and brain volume facilitate capture of MRI data into registries.³⁰ However, differences in image interpretation, scanners (field strength, vendors), acquisition protocols, and image quality introduce variability, making longitudinal assessment difficult. Importantly, MS-related lesion burden and brain atrophy require segmentation-based image analysis to be accurate and reliable, but use of different methods further increases variability. A partial solution is linkage of clinical registries to MR image repositories, allowing centralized analysis using standardized image-processing software.

Optical coherence tomography (OCT) provides non-invasive quantitative imaging of the retinal structures and is increasingly used as an MS outcome measure.³¹ Obstacles to incorporation of OCT data into registries are analogous to those of MRI: optimization and standardization of image acquisition, analysis, quality control, and reporting.^32,33

Biospecimens

Numerous non-imaging biomarkers for diagnosis, monitoring disease status, and measuring treatment response are under investigation.³⁴ Analogous to imaging data repositories, biobanks can be linked to registries to augment the information available by including the results of biospecimen analyses. Biospecimens can include blood components (e.g. serum or peripheral blood mononuclear cells), other body fluids, genetic material, microbiota samples, and others. Reuse of specimens for multiple studies reduces costs and time. Standardized operating procedures for sample collection, processing, transportation, storage, and assays reduce variability. Recording meta-data, which may account for variation in the samples collected (e.g. demographics, health behaviors, and time of day of collection), is critical.

Data from wearable biosensors

Traditionally, MS disease status is assessed during office visits. There is substantial interest in utilizing wearable biosensors to measure physiologic and kinetic parameters to assess activity, ambulation, falls, sleep, vital signs, and other parameters during routine activities of daily living, thus reflecting functional status more accurately.³⁵ How best to analyze the large quantity of data provided by continuous monitoring and integrate it with clinical, patient-reported, imaging, and biomarker data are active areas of research.

RWD challenges common to all data sources

Important differences exist in completeness and quality between data captured through prospectively defined research (whether RCT or observational studies) and those routinely collected. RWD offer the advantage of large samples but typically provide less detailed information compared with the smaller numbers of highly selected and intensely studied participants in RCTs. Information quality from RWD is a multidimensional concept, encompassing the degree to which data are appropriate for the desired purpose (i.e. the research question), syntactic (structure) and semantic (meaning) data accuracy, completeness, depth of information available, consistency, and timeliness. For example, the MSBase database contains information on >60,000 unique individuals, but only approximately 80% have an MRI time point captured and approximately 40% have routine MRI follow-up.¹⁶ Also, information about some data elements such as relapses (e.g. symptoms, severity) are often incomplete, and data quality varies across centers.¹⁶ Increased emphasis on capturing MRI data and implementation of data quality processes will help mitigate these limitations.¹⁶

Quality of RWD is also influenced by the clinical setting in which such data are gathered and reasons for collection. Important information regarding exposures, outcomes, and confounders such as health behaviors may be lacking, although linkage of different data sources for identified subjects can address this concern. For example, administrative databases include large cohorts but lack information about MS-related clinical characteristics. Linkage to clinical databases and registries can address this gap, allowing more complex research questions, such as the impact of comorbidities on disability progression³⁶ and identifying biases in tertiary care populations.³⁷ Similarly, linkage to non-clinical databases can support analyses of the effects of environmental exposures.³⁸

Real-world evidence

The US Food and Drug Administration (FDA)³⁹ defines real-world evidence (RWE) as “clinical evidence regarding the usage and potential benefits or risks of a medical product derived from analysis of RWD.” Observational and interventional studies relying on routinely collected health data from clinical practice constitute forms of RWE. High-quality RWE can support treatment decisions and provide information about quality of care.

Regulatory agencies sometimes use RWE to inform post-approval evaluation of drug utilization, particularly long-term safety and generalizability of safety results from registration RCTs to a clinical setting where patients may differ in age or comorbidity burden from trial populations.⁴⁰ Other uses might include identification of drug–drug interactions, assessment of long-term effectiveness, evaluation of alternative dosing regimens, and support of product label revisions or extensions for already-approved agents.⁴¹ While approval of new therapeutics by regulatory agencies usually requires robust RCTs to demonstrate safety and efficacy,⁴² alternative study designs might accelerate approval of treatments for rare and serious diseases where placebo-controlled trials are not feasible or ethical, or when sufficient sample sizes or active treatment controls are not available.⁴³ In such situations, RWD might include historical controls against which a new therapeutic can be compared. The use of RWE generated from RWD in these ways requires a priori engagement of and guidance from regulators, who will usually require information about data structure, content, and standardization; quality assurance measures; and data quality to determine the acceptability of the proposed study design. Regulatory agencies are actively exploring the value of disease registries and other RWD sources for evaluating drugs and devices,^39,44 including in MS.⁴⁵

Considerations in study designs to generate RWE

Use of RWD to generate high-quality RWE requires a clearly articulated research question, high data quality, and a sound study design and analysis plan. A spectrum of study designs can be used to generate RWE, including observational studies (e.g. cross-sectional studies, case–control studies, cohort studies), intervention studies (e.g. registry-based pragmatic RCTs), and meta-analyses synthesizing reported data.

Observational studies

Associations identified in observational studies might be altered due to random error, bias (systematic error), or confounders. These issues and related mitigation strategies are reviewed in the MS context elsewhere.⁸ An important obstacle to causal inference is time-varying confounding (i.e. affected by previous exposure or treatment and affecting future exposures). For example, in observational studies assessing the effect of DMT on MS-related disability, individuals with more frequent relapses may be more likely to initiate DMT and, because of this relapse activity, are also more likely to experience short-term disability worsening. Conversely, those who initiate DMT may experience a reduction in relapse rate, which reduces risk of short-term disability worsening. Marginal structural modeling is a common approach that accounts for this issue if the relevant confounding factors are known.^46,47

Analogous to the Cochrane Collaboration tool to assess risk of bias in randomized trials,⁴⁸ several tools have been proposed to assess the risk of bias in observational studies of exposures (Risk of Bias in Observational Studies of Exposures (ROBINS-E)),⁴⁹ interventions (Risk of Bias in Non-randomized Studies of Interventions (ROBINS-I)),^50,51 and diagnostic tests (Quality Assessment of Studies of Diagnostic Accuracy included in Systemic reviews (QUADAS and QUADAS-II)).^52–54 The Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework was developed to assess risk of bias, quality of evidence, and recommendations based on observational studies.^55,56

Pragmatic registry-based randomized controlled trials

Clinical trial designs can be categorized generally as explanatory or pragmatic,^57,58 but exist on a continuum when all of the trials’ design features are considered.⁵⁹ Pivotal trials of DMTs in MS have used explanatory designs, evaluating DMT efficacy under optimized conditions. These trials seek to enroll relatively homogeneous populations based on rigorous inclusion and exclusion criteria, and employ randomization and tight control on how the interventions are delivered and outcomes are assessed. However, such efforts to increase internal validity reduce generalizability to clinical practice.

Pragmatic trials can bridge the gap between observational studies and RCTs by evaluating the effectiveness of an intervention in real-world clinical settings. The design and setting can enhance generalizability and lower costs compared to explanatory RCTs, since many costs may be covered through the health system. Randomization in a pragmatic design addresses the major weaknesses of observational studies⁶⁰—the potential lack of comparability of the treatment groups and other biases—and minimizes the influence of known and unknown confounders. Pragmatic trials sometimes require larger sample sizes and longer follow-ups than explanatory trials, as increased heterogeneity may lead to reduced effect sizes.

A pragmatic registry-based RCT uses a RWD source (e.g. a registry) as the platform for recruitment, recording intervention delivery, and collecting follow-up data.⁵⁷ This approach can facilitate enrollment by using existing data to determine eligibility and provide information regarding the population not enrolled.⁶¹ Registry-based trials face potential challenges related to data quality, ethical issues, and methodological considerations.^57,60 The data collected may be of lower quality than from an RCT due to missing data; failure to capture relevant prognostic factors; and uncertainty regarding variable definitions, accuracy, and collection methods. Ethical issues may arise related to screening participants for inclusion if they have not consented to the use of their data for this purpose. Consenting participants to receive treatment(s) routinely used in clinical practice, participant withdrawal from a trial using a registry platform, data management, and privacy concerns are also issues. Methodologically, the information a registry captures determines the research questions it can address and study designs it can accommodate.

Data storage, sharing, harmonization, and analysis

Data storage and sharing

Increasingly, funders require data sharing, and scientists seek opportunities for secondary analyses of previously collected data and for joint analyses. Data sharing is a complex topic, which includes research data management plans, open access, licensing, and so on.⁶² Timely and effective use of existing data is impeded when existence of the data is unknown, the data are not accessible, or structured documentation about the data is lacking. The FAIR Guiding Principles (findability, accessibility, interoperability, and reusability) propose ways to improve discoverability, access, and reuse of data.⁶³ Data should be findable with a globally unique and persistent identifier and be registered in a searchable resource. Similarly, meta-data should be registered in a searchable resource and be explicitly linked to the data source. To be interoperable, data should use a formal, shared language (syntax and vocabulary), include references to other (meta-)data, and use vocabularies that follow FAIR principles. Meta-data catalogs are emerging to improve findability, accessibility, and description of data.⁶⁴ The European Medical Information Framework, for example, developed a data catalog in Alzheimer’s disease and information technology to support data reuse.⁶⁵

Effective data sharing requires a community of trust and transparency between those who provide and who use the data, and clarity about credit for data creation and ownership, particularly when data derive from clinical practice. Data scientists using information technology solutions are critical to address these concerns.^66,67 Regulations concerning confidentiality and consent are additional barriers, depending on the data source and jurisdiction (see below).⁶⁶

Harmonization

The analysis of combined data from different sources allows comparison and replication of findings, supporting generalizability, and enhancing statistical power in a collaborative and cost-effective manner.⁶⁸ Data harmonization can improve comparability of similar measures from different studies to facilitate joint analysis. The analysis of individual-level data following data harmonization offers multiple advantages compared with meta-analysis of published studies, including enhanced comparability across studies; minimization of biases; and the ability to study rarer outcomes, explore sources of heterogeneity, and conduct sensitivity analyses.⁶⁸

Data harmonization can be achieved prospectively (Table 4) or retrospectively. Standardization, the most stringent form of prospective harmonization and the best way to ensure that data from multiple sources are comparable, involves a priori adoption of uniform technical specifications of measurement procedures and of data structure, format, and terminology.⁷³ This approach improves data quality and synthesis, but agreement between multiple parties and implementation can be difficult. Standardization limits the studies that can be included and takes time to develop given its prospective nature. Flexible harmonization of data across studies using different data collection tools and methods allows inclusion of more studies, but data synthesis is more difficult, and the amount of data that can be harmonized is influenced by the degree of heterogeneity across studies. Common data models (CDMs) are a mechanism by which raw data are standardized to a common structure, format, and terminology independent from any particular study.

Table 4.

Selected examples of prospective data harmonization initiatives in multiple sclerosis.

Pediatric MS Tool-Kit^69,70

• Measurement framework designed to improve exposure measurement in future pediatric MS etiologic studies and facilitate harmonization

• Includes six core variables each to measure environmental tobacco exposure, sun exposure, and vitamin D intake

The National Institute of Neurological Disorders and Stroke Common Data Elements (CDE) project: MS⁷¹

• Aimed at facilitating future data sharing including comparisons across studies and data aggregation for meta-analysis, improving data standardization and quality, and reducing the time and costs involved in developing data collection tools

• Provides data standards for clinical research in neurology, including common definitions, core variables (essential for all studies), supplemental variables and exploratory variables, and case report forms

• CDE for MS include demographics, general health history, disease characteristics, paraclinical tests, and outcome measures (patient-reported, clinician-assessed, and performance-based). Some elements are stringent (e.g. core variables), and some are flexible (e.g. multiple choices of outcome measures proposed for fatigue)

Clinical Data Interchange Standards Consortium (CDISC)⁷²

• CDISC is a common data model for clinical trials, which is flexible and scalable. It is an organization that develops data standards that “enable information system interoperability to improve medical research and related areas of healthcare”

• Therapeutic area data standards specify how to structure data, but not how to collect the data or what should be collected. Data structure is standardized using a tabulation model that organizes domains and variables within those domains

• Data content is structured using controlled terminology (vocabulary) because it is difficult to combine data when different terms or codes are used for similar or identical concepts

• CDISC standards are required by the US Food and Drug Administration and Japanese Pharmaceuticals and Medical Devices Agency for regulatory submissions

Prospective data harmonization might not always be desirable for technical or scientific reasons. Future data comparability requirements cannot always be foreseen at the time of study design. Moreover, the strength of epidemiologic evidence is enhanced by demonstrating similar findings across studies conducted at varied times with different designs and measurement approaches. Therefore, retrospective data harmonization remains important. A high degree of variability exists in the methods, terminology, and technology used to support retrospective data harmonization initiatives.⁷⁴ Potential challenges include timeliness of data access; variable restrictions and procedures surrounding access to individual-level participant data; recognizing contributions of study-specific investigators; diverse authorship rules; time and resources required to conduct the project; the need to gather comprehensive information regarding the aspects of the studies such as design, standard operating procedures, data collection tools, data format, and data content and quality; defining acceptable levels of heterogeneity across databases; data processing and documentation thereof; and sustainability.⁷⁴ The Maelstrom Research guidelines proposed a stepwise approach to achieving retrospective data harmonization.⁷⁴ The Big MS Data Network has undertaken an examination of the feasibility of retrospective harmonization preparatory to sharing data across five MS registries.⁷⁵

Analysis of data from multiple sources

Several approaches exist for analyzing data from multiple studies. The choice depends on ethical and regulatory limitations on data sharing. These approaches include study-specific analysis followed by meta-analysis, federated analysis, and pooled analysis. In a study-specific analysis, each study is analyzed separately using common variable definitions and protocol⁷⁶ to produce an effect estimate. Meta-analysis then assigns a weight to each study using a fixed-effects or random-effects models, producing a summary estimate.

In a federated information governance model,⁷⁷ data custodianship remains at the level of the individual contributing data sources (e.g. registries). Person-level data remain on local servers under the direction of local data custodians. Analysis is centralized such that the datasets undergo simultaneous analysis in parallel, and then the results are combined.⁷⁸ This approach enables a high level of data security and protection without needing to address international laws and ethics approaches. To succeed, standard operating procedures must be clearly defined because local queries require local harmonization procedures to allow maximal interoperability and efficient exchange of results. Information technology solutions,⁷⁹ such as DataSHIELD,⁸⁰ have been developed to support such analyses, producing findings similar to individual-level meta-analysis. Several initiatives in MS have used this approach, including the Big MS Data Network.⁷⁵

In a pooled analysis, all data are gathered centrally under a single custodian and analyzed as a whole. This approach requires the greatest data sharing and is the most complex from a governance perspective. It also ignores any heterogeneity in the characteristics of the individual studies being combined, which may lead to paradoxical findings.

Pragmatic issues related to maintaining and using RWD

In addition to methodological challenges related to collection, pooling, and analyzing RWD, several other pragmatic challenges threaten the ability to optimally share data and to maintain data sources.

Data governance and ethics

Legal and ethical frameworks, and data governance structures differ substantially across jurisdictions.⁸¹ In 2018, the European Commission implemented the General Data Protection Regulation (GDPR), which aims to harmonize the rules for all the European Union (EU) member states to reduce the legal fragmentation, complexities, and uncertainties that existed between member states and to strengthen the data subject’s rights.⁸² Essential steps needed for data transfer outside the EU include consent adequacy, standard contractual clauses, binding corporate rules, and codes of conduct.⁸³ Importantly, the GDPR specifies consent language that allows maintenance of personal sensitive data for unanticipated future scientific uses. In 1996, the Health Insurance Portability and Accountability Act (HIPAA) created a legal framework to protect health data derived from US sources.⁸⁴ Further regulations are mandated on a state-by-state basis.^85,86 Canadian regulations differ from those in the EU and US and may differ between Canadian provinces.⁸⁷ This complexity creates additional barriers to data transfer and sharing between countries and across continents.

Registry governance and sustainability

The feasibility and sustainability of a registry require that issues related to privacy and confidentiality, human and financial resources, governance, and data quality be addressed.⁸⁷ Important initial steps in registry development include a precise definition of its goals that reflect the needs of its users and early identification of key stakeholders, which influence whether the registry development can proceed, how the registry will be conducted, and the type and scope of data collected.^5,87

Most registries have a governance model relying on principles and constraints based on their mandate, operating procedures, legal environment, and funding sources.⁸⁸ The governance structure usually includes an Executive or Steering Committee with financial, administrative, legal/ethical, and organizational responsibilities to ensure that objectives are clear and that the interests of the general community are met; a Scientific Committee composed by clinicians, methodologists, and representatives of participating centers which oversees scientific initiatives, promotes specific strategic projects, and approves requests for access to centralized data for research projects; and a variable number of stakeholder representatives.¹⁰ All aspects of governance should be clearly codified in a format that can be revised over time according to the legal or regulatory context and requirements of different stakeholders.

Sustainable funding is a major challenge for maintaining registry infrastructure and for adapting the registry to changes in the legal environment, technology, and key scientific questions over time. Different sources of funding, sometimes temporary or transitional, often coexist¹⁰ and may include patient advocacy organizations, public health or regulatory authorities, product manufacturers, health care service providers, payer or commissioning authorities, academic institutions or consortia, and professional societies. A registry’s funding often evolves over time.

The future of real-world studies

This review focuses on approaches to improve the scope, quality, and analyses of RWD to advance understanding of MS and its treatment. Although substantial progress has occurred in understanding the natural history of MS and developing therapies, many questions persist concerning disease behavior and therapeutics. RCTs and real-world studies, including observational studies and pragmatic trials, are complementary approaches to inform clinical practice, representing a continuum that provides different information with different validity and insights. Some questions are better answered using RWD (Table 2) than data from traditional RCTs. However, RWD present methodological challenges, and care must be taken to address data quality and ensure that study design accounts for potential sources of bias. Despite these challenges, observational studies and registries have informed our understanding and treatment of the disease.^8,11

Several approaches could improve the quality of RWE generated from existing RWD sources and facilitate its application in MS (Table 5). Creation of a meta-data catalog of existing cohorts and registries would help improve use of existing RWD sources. This meta-data catalog could include a tool kit with standard procedures for exchanging data and guidelines to facilitate data harmonization across datasets, as well as resources and guidance to help investigators navigate increasingly complex privacy and data protection regulations, their evolution over time, and their heterogeneity across jurisdictions.

Table 5.

Approaches to improve the quality of real-world evidence to advance understanding of the MS disease process and treatment.

• Develop a detailed meta-data catalog of existing cohorts and registries

• Develop standard procedures to facilitate exchanging data

• Develop guidelines to facilitate harmonization of data in the MS field

• Develop a tool kit, which would be housed with the meta-data catalog and include information on common ethical and privacy issues (e.g. HIPAA, GPDR), standard informed consent language and process, reporting guidelines, and standard operating procedures for imaging and biospecimen repositories

• Further develop methods to facilitate efficient, standardized, and comprehensive data capture as part of patient care

• Develop the methodology and infrastructure to incorporate data from PROMs, imaging and non-imaging biomarkers, and environmental monitoring into existing and future registries

• Further research into methods of combining and synthesizing real-world data, and methods for handling bias in the context of large linked, longitudinal observational datasets

• Standardize and improve reporting of studies describing real-world data. Although several relevant guidelines already exist, these guidelines need to be implemented with MS-specific journals

• Funding and sustainability of real-world data sources is a key issue, which needs to be addressed

• Develop/retain the workforce of biostatisticians, data scientists, and epidemiologists to support this work

MS: multiple sclerosis; HIPAA: Health Insurance Portability and Accountability Act; GPDR: General Data Protection Regulation; PROM: patient-reported outcome measure.

Currently, patient care and clinical investigation are largely separate activities in most institutions.⁸⁹ Data acquired during patient care are sometimes entered into registries and utilized for research, but the process often is inefficient, and the data may not directly inform the care of individual patients from whom the data are generated. The widespread implementation of EMRs offers the opportunity for more efficient collection of standardized comprehensive data during patient encounters for use in both patient management and clinical research.

Many current RWD sources have the advantage of scale but are largely limited to disease and treatment history and clinician-reported measures of disease activity and disability. The utility of RWD would be improved by inclusion of data from performance measures, PROMs, imaging studies, biomarkers, and wearable biosensors—by adding these existing databases or by linkage to other data sources.

Reporting of RWD studies needs to be improved to promote their acceptance by regulatory agencies and the scientific community. MS-specific journals need to implement existing reporting guidelines.⁹⁰ Funding and sustainability of RWD sources is also critical. Technical and personnel costs require support from project inception onward, and the need for ongoing support of infrastructure remains a major obstacle for many RWD projects. Finally, the development and retention of the necessary workforce of biostatisticians, data scientists, and epidemiologists with interest and expertise in MS is critical to the field.

Supplemental Material

MSJ892555_supplemental_material – Supplemental material for Leveraging real-world data to investigate multiple sclerosis disease behavior, prognosis, and treatment

Supplemental material, MSJ892555_supplemental_material for Leveraging real-world data to investigate multiple sclerosis disease behavior, prognosis, and treatment by Jeffrey A Cohen, Maria Trojano, Ellen M Mowry, Bernard MJ Uitdehaag, Stephen C Reingold and Ruth Ann Marrie in Multiple Sclerosis Journal

Footnotes

Acknowledgements

The International Conference on Data Leveraging to Answer Key Clinical Questions in Multiple Sclerosis (MS) convened 65 experts in real-world data efforts in MS and other diseases, including clinicians, imaging experts, epidemiologists, statisticians, data managers, representatives of patient advocacy organizations, and regulators (Supplementary Material, ) on 12–15 September 2018 in Washington, DC under the auspices of the International Advisory Committee on Clinical Trials in Multiple Sclerosis. We thank all Conference participants, all of whom were given the opportunity to review a draft of the manuscript and suggest revisions prior to finalization.

Authors’ Contribution

J.A.C., M.T., E.M.M., B.M.J.U., S.C.R., and R.A.M. designed the program agenda for the conference that served as the basis for this review, prepared the initial drafts of this manuscript, edited the manuscript, and approved the final version for submission.

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Dr Cohen reports consulting fees from Alkermes, Biogen, Convelo, EMD Serono, ERT, Gossamer Bio, Novartis, and ProValuate; speaking fees from Mylan and Synthon; and serves as an Editor of Multiple Sclerosis Journal outside the submitted work. Dr Trojano reports grants and personal fees from Biogen, Merck, Novartis, and Roche and personal fees from Sanofi Genzyme, and Teva outside the submitted work. Dr Mowry reports grants from Biogen and Genzyme; other support from Teva; and personal fees from UpToDate outside the submitted work. Dr Uitdehaag reports personal fees from Genzyme, Biogen Idec, Teva, Merck Serono, and Roche outside the submitted work. Dr Reingold reports personal fees and non-financial support from National Multiple Sclerosis Society; personal fees and non-financial support from European Committee for Treatment and Research in Multiple Sclerosis during the conduct of the study; personal fees from BrainStorm Cell Therapeutics, Teva Pharmaceutical Industries, and TG Therapeutics; personal fees and non-financial support from MedDay Pharmaceuticals SA, MedImmune Inc/AstraZeneca, Novartis, and Scientific and Clinical Review Associates LLC; and non-financial support from Observatoire Français pour la Sclérose en Plaques outside the submitted work. Dr Marrie has nothing to disclose.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The International Advisory Committee on Clinical Trials in Multiple Sclerosis and the International Conference on Data Leveraging to Answer Key Clinical Questions in Multiple Sclerosis were supported by the National Multiple Sclerosis Society and the European Committee for Treatment and Research in Multiple Sclerosis.

ORCID iDs

Jeffrey A Cohen

Ruth Ann Marrie

Supplemental material

Supplemental material for this article is available online.

References

Montalban

Gold

Thompson

, et al. ECTRIMS/EAN guideline on the pharmacological treatment of people with multiple sclerosis. Mult Scler J 2018; 24(2): 96–120.

Rae-Grant

Day

Marrie

, et al. Practice guideline recommendations summary: Disease-modifying therapies for adults with multiple sclerosis: Report of the guideline development, dissemination, and implementation subcommittee of the American Academy of Neurology. Neurology 2018; 90(17): 777–788.

Centers for Medicare Medicaid Services. Electronic health records, https://www.cms.gov/Medicare/E-Health/EHealthRecords/index.html (accessed 22 February 2019).

Piantadosi

Clinical trials: A methodologic perspective. 3rd ed. Hoboken, NJ: John Wiley & Sons, 2017.

Gliklich

Dreyer

Leavy

Registries for evaluating patient outcomes: A user’s guide (Report No. 13(14)-EHC111). 3rd ed. Rockville, MD: Agency of Healthcare Research and Quality, 2014.

Green

Byar

DP.

Using observational data from registries to compare treatments: The fallacy of omnimetrics. Stat Med 1984; 3(4): 361–373.

Bothwell

Podolsky

SH.

The emergence of the randomized, controlled trial. N Engl J Med 2016; 375(6): 501–504.

Trojano

Tintore

Montalban

, et al. Treatment decisions in multiple sclerosis—Insights from real-world observational studies. Nat Rev Neurol 2017; 13(2): 105–118.

Koch-Henriksen

Rasmussen

Stenager

, et al. The Danish multiple sclerosis registry. History, data collection and validity. Dan Med Bull 2001; 48(2): 91–94.

10.

Glaser

Stahmann

Meissner

, et al. Multiple sclerosis registries in Europe—An updated mapping survey. Mult Scler Relat Disord 2019; 27: 171–178.

11.

Bebo

Jr Fox

Lee

, et al. Landscape of MS patient cohorts and registries: Recommendations for maximizing impact. Mult Scler 2018; 24(5): 579–586.

12.

Posada

del Otero

Villaverde

, et al. EpiRare Deliverale D4. Guidelines for data sources and quality for RD registries in Europe, February 2014, https://ec.europa.eu/health/sites/health/files/ehealth/docs/ev_20150512_co013_en.pdf (accessed 24 November 2019).

13.

Arts

De Keizer

Scheffer

GJ.

Defining and improving data quality in medical registries: A literature review, case study, and generic framework. J Am Med Inform Assoc 2002; 9(6): 600–611.

14.

Cook

Collins

GS.

The rise of big clinical databases. Br J Surg 2015; 102(2): e93–e101.

15.

Weiskopf

Weng

Methods and dimensions of electronic health record data quality assessment: Enabling reuse for clinical research. J Am Med Inform Assoc 2013; 20(1): 144–151.

16.

Kalincik

Kuhle

Pucci

, et al. Data quality evaluation for observational multiple sclerosis registries. Mult Scler 2017; 23(5): 647–655.

17.

Damotte

Lizee

Tremblay

, et al. Harnessing electronic medical records to advance research on multiple sclerosis. Mult Scler 2019; 25(3): 408–418.

18.

Ahlbrandt

Lablans

Glocker

, et al. Modern information technology for cancer research. Oncology 2018; 15: 1–7.

19.

Scott

Dunscombe

Evans

, et al. Learning health systems need to bridge the “two cultures” of clinical informatics and data science. J Innov Health Inform 2018; 25(2): 126–131.

20.

Rhodes

Schindler

Rao

, et al. Multiple Sclerosis Performance Test: Technical development and usability. Adv Ther 2019; 36: 1741–1755.

21.

Rudick

Miller

Bethoux

, et al. The Multiple Sclerosis Performance Test (MSPT): An iPad-based disability assessment tool. J Vis Exp 2014; 30: e51318.

22.

Smith

Lix

Azimaee

, et al. Assessing the quality of administrative data for research: A framework from the Manitoba Centre for Health Policy. J Am Med Inform Assoc. Epub ahead of print 17 September 2019. DOI: 10.1093/jamia/ocx078.

23.

Cohen

Reingold

Polman

, et al. Disability outcome measures in multiple sclerosis trials: Current status and future prospects. Lancet Neurol 2012; 11(5): 467–476.

24.

Ysrraelit

Fiol

Gaitan

, et al. Quality of life assessment in multiple sclerosis: Different perception between patients and neurologists. Front Neurol 2017; 8: 729.

25.

US Food and Drug Administration. Guidance for industry—Patient-reported outcome measures: Use in medical product development to support labeling claims, December 2009, https://www.fda.gov/downloads/drugs/guidances/ucm193282.pdf (accessed 6 December 2011).

26.

Vollmer

Stanton

, et al. The NARCOMS patient registry: A resource for investigators. Int J MS Care 1999; 1: 28–34.

27.

Bottomley

Sloan

, et al. Analysing data from patient-reported outcome and quality of life endpoints for cancer clinical trials: A start in setting international standards. Lancet Oncol 2016; 17(11): e510–e514.

28.

Wattjes

Rovira

Miller

, et al. Evidence-based guidelines: MAGNIMS consensus guidelines on the use of MRI in multiple sclerosis—Establishing disease prognosis and monitoring patients. Nat Rev Neurol 2015; 11(10): 597–606.

29.

Traboulsee

Simon

Stone

, et al. Revised recommendations of the Consortium of MS Centers task force for a standardized MRI protocol and clinical guidelines for the diagnosis and follow-up of multiple sclerosis. AJNR 2016; 37(3): 394–401.

30.

Dickerson

Davenport

Syed

, et al. Effect of template reporting of brain MRIs for multiple sclerosis on report thoroughness and neurologist-rated quality: Results of a prospective quality improvement project. J Am Coll Radiol 2017; 14(3): 371–379.

31.

Petzold

de Boer

Schippling

, et al. Optical coherence tomography in multiple sclerosis: A systematic review and meta-analysis. Lancet Neurol 2010; 9(9): 921–932.

32.

Schippling

Balk

Costello

, et al. Quality control for retinal OCT in multiple sclerosis: Validation of the OSCAR-IB criteria. Mult Scler 2015; 21(2): 163–170.

33.

Cruz-Herranz

Balk

Oberwahrenbrock

, et al. The APOSTEL recommendations for reporting quantitative optical coherence tomography studies. Neurology 2016; 86(24): 2303–2309.

34.

Khalil

Teunissen

Otto

, et al. Neurofilaments as biomarkers in neurological disorders. Nat Rev Neurol 2018; 14(10): 577–589.

35.

Bradshaw

Farrow

Motl

, et al. Wearable biosensors to monitor disability in multiple sclerosis. Neurol Clin Pract 2017; 7(4): 354–362.

36.

McKay

Tremlett

Fisk

, et al. Psychiatric comorbidity is associated with disability progression in multiple sclerosis. Neurology 2018; 90(15): e1316–e1323.

37.

McKay

Tremlett

Zhu

, et al. A population-based study comparing multiple sclerosis clinic users and non-users in British Columbia, Canada. Eur J Neurol 2016; 23(6): 1093–1100.

38.

Bai

Burnett

Kwong

, et al. Long-term exposure to air pollution and the incidence of multiple sclerosis: A population-based cohort study. Environ Res 2018; 166: 437–443.

39.

US Food and Drug Administration. Real world evidence, 2019, https://www.fda.gov/scienceresearch/specialtopics/realworldevidence/default.htm (accessed 20 January 2019).

40.

Jarow

LaVange

Woodcock

Multidimensional evidence generation and FDA regulatory decision making: Defining and using “real-world” data. JAMA 2017; 318(8): 703–704.

41.

US Food and Drug Administration. Postmarketing studies and clinical trials—Implementation of Section 505(o)(3) of the Federal Food, Drug, and Cosmetic Act, April 2011, https://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM172001.pdf (accessed 20 January 2019).

42.

Van Norman

. Drugs and devices: Comparison of European and U.S. approval processes. JACC Basic Transl Sci 2016; 1(5): 399–412.

43.

Beaver

Howie

Pelosof

, et al. A 25-year experience of US Food and Drug Administration accelerated approval of malignant hematology and oncology drugs and biologics: A review. JAMA Oncol 2018; 4(6): 849–856.

44.

European Medicines Agency. Patient registries workshop, 28 October 2016: Observations and recommendations arising from the workshop, https://www.ema.europa.eu/documents/report/report-patient-registries-workshop_en.pdf (accessed 20 January 2019).

45.

European Medicines Agency. Report on multiple sclerosis registries—Workshop 7 July 2017. Patient registry initiative, https://www.ema.europa.eu/documents/report/report-multiple-sclerosis-registries_en.pdf (accessed 20 January 2019).

46.

Daniel

Cousens

De Stavola

, et al. Methods for dealing with time-dependent confounding. Stat Med 2013; 32(9): 1584–1618.

47.

Karim

Gustafson

Petkau

, et al. Marginal structural Cox models for estimating the association between β-interferon exposure and disease progression in a multiple sclerosis cohort. Am J Epidemiol 2014; 180(2): 160–171.

48.

Higgins

Altman

Gotzsche

, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 2011; 343: d5928.

49.

Bero

Chartres

Diong

, et al. The risk of bias in observational studies of exposures (ROBINS-E) tool: Concerns arising from application to observational studies of exposures. Syst Rev 2018; 7(1): 242.

50.

Sterne

Hernan

Reeves

, et al. ROBINS-I: A tool for assessing risk of bias in non-randomised studies of interventions. BMJ 2016; 355: i4919.

51.

Morgan

Thayer

Santesso

, et al. Evaluation of the risk of bias in non-randomized studies of interventions (ROBINS-I) and the “target experiment” concept in studies of exposures: Rationale and preliminary instrument development. Environ Int 2018; 120: 382–387.

52.

Whiting

Rutjes

Reitsma

, et al. The development of QUADAS: A tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol 2003; 3: 25.

53.

Whiting

Weswood

Rutjes

, et al. Evaluation of QUADAS, a tool for the quality assessment of diagnostic accuracy studies. BMC Med Res Methodol 2006; 6: 9.

54.

Whiting

Rutjes

Westwood

, et al. QUADAS-2: A revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011; 155(8): 529–536.

55.

Morgan

Thayer

Bero

, et al. GRADE: Assessing the quality of evidence in environmental and occupational health. Environ Int 2016; 92–93: 611–616.

56.

Morgan

Thayer

Santesso

, et al. A risk of bias instrument for non-randomized studies of exposures: A users’ guide to its application in the context of GRADE. Environ Int 2019; 122: 168–184.

57.

Sajobi

Menon

, et al. Registry-based randomized controlled trials—What are the advantages, challenges, and areas for future research. J Clin Epidemiol 2016; 80: 16–24.

58.

Patsopoulos

NA.

A pragmatic view on pragmatic trials. Dialogues Clin Neurosci 2011; 13(2): 217–224.

59.

Thorpe

Zwarenstein

Oxman

, et al. A pragmatic-explanatory continuum summary (PRECIS): A tool to help trial designers. J Clin Epidemiol 2009; 62(5): 464–475.

60.

Lauer

D’Agostino

Sr . The randomized registry trial—The next disruptive technology in clinical research. N Engl J Med 2013; 369(17): 1579–1581.

61.

van Staa

Dyson

McCann

, et al. The opportunities and challenges of pragmatic point-of-care randomised trials using routinely collected electronic records: Evaluations of two exemplar trials. Health Technol Assess 2014; 18(43): 1–146.

62.

Johnston

Campbell

Hayward

, et al. Registry data storage and curation. Can J Neurol Sci 2013; 40(4 Suppl. 2): S35–S40.

63.

Wilkinson

Dumontier

Aalbersberg

, et al. The FAIR guiding principles for scientific data management and stewardship. Sci Data 2016; 3: 160018.

64.

Bergeron

Doiron

Marcon

, et al. Fostering population-based cohort data discovery: The Maelstrom Research cataloguing toolkit. PLoS ONE 2018; 13(7): e0200926.

65.

Perera

Pedersen

Ansel

, et al. Dementia prevalence and incidence in a federation of European Electronic Health Record databases: The European Medical Informatics Framework resource. Alzheimers Dement 2018; 14(2): 130–139.

66.

Pisani

AbouZahr

Sharing health data: Good intentions are not enough. Bull World Health Organ 2010; 88(6): 462–466.

67.

Peeters

LM.

Fair data for next-generation management of multiple sclerosis. Mult Scler 2018; 24(9): 1151–1156.

68.

Thompson

Thinking big: Large-scale collaborative research in observational epidemiology. Eur J Epidemiol 2009; 24(12): 727–731.

69.

Magalhaes

Banwell

Bar-Or

, et al. A framework for measurement and harmonization of pediatric multiple sclerosis etiologic research studies: The Pediatric MS Toolkit. Mult Scler J 2019; 25(8): 1170–1177.

70.

Maelstrom. Pediatric MS tool-kit, https://www.maelstrom-research.org/mica/network/tool-kit#/ (accessed 20 January 2019).

71.

National Institutes of Neurological Disorders and Stroke. NINDS common data elements: Multiple sclerosis, https://www.commondataelements.ninds.nih.gov/MS.aspx#tab=Data_Standards (accessed 22 February 2019).

72.

Multiple Sclerosis Outcome Assessments Consortium and the CFAST Multiple Sclerosis Development Team. Therapeutic area data standards user guide for multiple sclerosis (Version 1.0), 2 May 2014, https://www.cdisc.org/standards/therapeutic-areas/multiple-sclerosis (accessed 22 February 2019).

73.

Fortier

Doiron

Burton

, et al. Invited commentary: Consolidating data harmonization—How to obtain quality and applicability. Am J Epidemiol 2011; 174(3): 261–264, author reply 265.

74.

Fortier

Raina

Van den Heuvel

, et al. Maelstrom research guidelines for rigorous retrospective data harmonization. Int J Epidemiol 2017; 46(1): 103–105.

75.

Iaffaldano

Butzkueven

Hillert

, et al. Big multiple sclerosis data network: Data sharing among five large MS registries (P738), https://onlinelibrary.ectrims-congress.eu/ectrims/2017/ACTRIMS-ECTRIMS2017/200393/pietro.iaffaldano.big.multiple.sclerosis.data.network.data.sharing.among.five.html (accessed 24 November 2019).

76.

Suissa

Henry

Caetano

, et al. CNODES: The Canadian network for observational drug effect studies. Open Med 2012; 6(4): e134–e140.

77.

Floridi

Luetge

Pagallo

, et al. Key ethical challenges in the European medical information framework. Mind Mach 2018; 29: 355–371.

78.

Ardeshirdavani

Souche

Dehaspe

, et al. NGS-Logistics: Federated analysis of NGS sequence variants across multiple locations. Genome Med 2014; 6(9): 71.

79.

Doiron

Burton

Marcon

, et al. Data harmonization and federated analysis of population-based studies: The BioSHaRE project. Emerg Themes Epidemiol 2013; 10(1): 12.

80.

Budin-Ljosne

Burton

Isaeva

, et al. DataSHIELD: An ethically robust solution to multiple-site individual-level data analysis. Public Health Genomics 2015; 18(2): 87–96.

81.

Rumbold

Pierscionek

BK.

A critique of the regulation of data science in healthcare research in the European Union. BMC Med Ethics 2017; 18(1): 27.

82.

The European Parliament and the Council of the European Union. REGULATION (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation), http://eurlex.europa.eu/eli/reg/2016/679/oj (accessed 20 January 2019).

83.

Phillips

International data-sharing norms: From the OECD to the General Data Protection Regulation (GDPR). Hum Genet 2018; 137(8): 575–582.

84.

US Department of Health and Human Services. OCR privacy brief: Summary of the HIPAA Privacy Rule, May 2003, https://www.hhs.gov/sites/default/files/privacysummary.pdf (accessed 14 April 2019).

85.

Reynolds A; International Association of Privacy Professionals. GDPR matchup: U.S. state data breach laws, May 2018, https://iapp.org/news/a/gdpr-match-up-u-s-state-data-breach-laws (accessed 20 January 2019).

86.

US Department of Health and Human Services. HIPAA Breach Notification Rule, 45 CFR §§ 164.400-414, https://www.hhs.gov/hipaa/for-professionals/breach-notification/index.html (accessed 20 January 2019).

87.

Pringsheim

Marrie

Donner

, et al. Neurological registry feasibility and sustainability. Can J Neurol Sci 2013; 40(4 Suppl. 2): S55–S59.

88.

Viviani

Zolin

Mehta

, et al. The European cystic fibrosis society patient registry: Valuable lessons learned on how to sustain a disease registry. Orphanet J Rare Dis 2014; 9: 81.

89.

Rudick

Cosgrove

DM.

A radical proposal: Integrate clinical investigation into the U.S. health care system. Sci Transl Med 2009; 1(4): 4cm4.

90.

Vandenbroucke

von Elm

Altman

, et al. Strengthening the reporting of observational studies in epidemiology (STROBE): Explanation and elaboration. PLoS Med 2007; 4: e297.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.28 MB

Leveraging real-world data to investigate multiple sclerosis disease behavior,prognosis,and treatment

Abstract

Keywords

Introduction

RWD

Registries

EMR data

Administrative (health claims) databases

Clinician-reported data and performance measures

Patient-reported data sources

Imaging

Biospecimens

Data from wearable biosensors

RWD challenges common to all data sources

Real-world evidence

Considerations in study designs to generate RWE

Observational studies

Pragmatic registry-based randomized controlled trials

Data storage, sharing, harmonization, and analysis

Data storage and sharing

Harmonization

Analysis of data from multiple sources

Pragmatic issues related to maintaining and using RWD

Data governance and ethics

Registry governance and sustainability

The future of real-world studies

Supplemental Material

MSJ892555_supplemental_material – Supplemental material for Leveraging real-world data to investigate multiple sclerosis disease behavior, prognosis, and treatment

Footnotes

Acknowledgements

Authors’ Contribution

Declaration of Conflicting Interests

Funding

ORCID iDs

Supplemental material

References

Supplementary Material