Translating evidence into practice during the COVID-19 pandemic: pitfalls and mileages

The ongoing coronavirus disease 2019 (COVID-19) pandemic has posed unprecedented challenges to national healthcare systems and economies because of its unpredictable progress, the failure to successfully protect vulnerable populations, the paucity of proven therapies pending the development of effective vaccines, and the emerging evidence of long-term sequelae affecting different organs and systems.^1–3 Despite these uncertainties, tens of thousands of peer-reviewed papers have been published over the last few months on all aspects of the disease, including the effects of different pharmacological treatments. The rapid understanding of the mechanisms involved in the host interaction and replication of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the agent responsible for COVID-19, has led to the early identification of specific agents that could be repurposed for the management of the disease. ⁴ The massive financial support provided by governmental agencies and philanthropic organizations has been instrumental in the conduct of intervention trials of antivirals (e.g. remdesivir, lopinavir, and ritonavir) and immunomodulating drugs (e.g. hydroxychloroquine, tocilizumab, and dexamethasone) in thousands of patients worldwide. The generation of much needed evidence has undoubtedly led to significant improvements in patient risk stratification and management over time. However, healthcare professionals, policy-makers, and the general public have also been overwhelmed by the vast amount of information disseminated within a relatively short time frame. This has not been without unintended consequences. In the early stages of the pandemic, the results of small single-center studies and case series have negatively influenced clinical decisions, in stark contrast with the fundamental principles of evidence-based medicine. ⁵ For example, the reduced viral load reported in patients with COVID-19 with the disease-modifying, antirheumatic drug hydroxychloroquine in a small open-label nonrandomized trial conducted in France has facilitated its unrestricted use, even as a prophylactic agent, in several countries. ⁶ This practice has led to significant shortages in the availability and supply of hydroxychloroquine, with negative repercussions for the management of other conditions, particularly autoimmune diseases. ⁷ While the exact benefits and risks of the compassionate use of hydroxychloroquine in patients with COVID-19 may never be fully determined, being outside the realm of a rigorously conducted intervention study, several randomized controlled trials (RCTs) have since failed to report any tangible benefits of hydroxychloroquine on ‘hard’ clinical endpoints in this group.^8,9 Such RCTs were not free from design flaws either. For example, in 51 RCTs of hydroxychloroquine registered during the first quarter of 2020, only 34 (67%) reported a clinical outcome, whereas 24 (47%) did not present strategies to investigate safety outcomes. ¹⁰ The comprehensive assessment of toxicity during treatment with hydroxychloroquine is particularly important given the potential pro-arrhythmic effects associated with the relatively high doses tested in these trials, with or without concomitant treatment with azithromycin, and the systemic pro-inflammatory state commonly observed in patients at high risk with COVID-19.^11,12 Systematic reviews and meta-analyses have also reported an excess risk of other adverse events, particularly those affecting the gastrointestinal system, the skin, and the subcutaneous tissue, associated with high-dose hydroxychloroquine in patients with COVID-19 (risk difference 0.19, 95% CI 0.14–0.24, p < 0.00001). ¹³ Another key issue arising from the evidence generated from RCTs of hydroxychloroquine, and other agents, in patients with COVID-19 is its questionable translation into the routine care of relevant patient populations. For example, the much higher rates of hospitalization and death observed in patients with COVID-19 over 75–80 years of age, when compared with younger cohorts, should have prompted a substantial representation of older patients in clinical trials. ² Yet, the mean or median age of participants in RCTs of hydroxychloroquine, remdesivir, and tocilizumab was considerably lower, between 40 years and 50 years.^14–16 Furthermore, the effects of common conditions in the older population, such as heart failure, chronic obstructive pulmonary disease, and chronic kidney disease, on treatment efficacy and safety were largely unaccounted for in these studies.^14,15 Consequently, the benefits and risks of prescribing these agents in the typical frail older patient with multiple comorbidities and polypharmacy suffering from COVID-19 are largely unknown. ¹⁷ While the issues described above need to be adequately addressed in preparation for future global health emergencies, the COVID-19 pandemic has also tested the potential value of specific approaches to capture and interpretation of observational data (machine learning) and results of RCTs (living systematic reviews and meta-analyses), as well as hibernating trials and using platform adaptive designs. Machine learning, an application of artificial intelligence based on the study of specific computer algorithms that improve automatically through experience, has been successfully used to identify patterns of risk and therapeutic response in patients with COVID-19 using a wide array of clinical, demographic, biochemical, and genomic data from electronic repositories, for example, hospital databases.^18,19 In this context, the 4CE, an international consortium of 96 hospitals across North America, Europe, and Asia, was established to capture temporal changes in laboratory parameters from electronic COVID-19 patient data primarily through the open source i2b2 software platform. This software supports query and analysis pertaining to clinical and genomics information. ²⁰ This, and similar initiatives, is likely to prove instrumental in identifying specific trajectories in patients hospitalized with COVID-19 and assessing their response to various treatments. Importantly, such technologies can also assist with the early development of clinical decision trees, while traditional study methodologies, for example, RCTs, are being conducted, as well as with targeted drug discovery and drug repurposing strategies.^21–23 An important issue with the widespread use of electronic health-related information for machine learning and other purposes is the potential risk of accessing identifiable data. Therefore, stringent, yet transparent, policies and procedures are essential to ensure that the type of data collected is appropriate, securely stored, properly managed and analyzed, and that the rights of individual patients are protected. ²⁴ The capacity to conduct and disseminate living systematic reviews and network meta-analyses that are regularly updated following the publication of new RCTs has also allowed healthcare professionals and policy-makers to critically assess, on an ongoing basis, the benefits and harms of specific therapies in patients with COVID-19.^25,26 Finally, the practice of trial hibernation and adaptive study design has proven successful in facilitating the rapid completion of intervention trials of dexamethasone. ²⁷ The practice of trial hibernation, developed by the National Institute for Health Research in the UK, initially consisted of funding a number of studies investigating different pharmacological therapies that were ready to commence at the onset of the next influenza pandemic. These trials were later adapted to include new infectious diseases, which led to their rapid implementation in the early phases of the COVID-19 pandemic. ²⁸ It is hoped that government-funded research agencies in other countries, that is, the National Institutes of Health (USA) and the National Health and Medical Research Council (Australia), will also give consideration to creating funding streams for similar strategies involving trial hibernation and adaptive study design.

In conclusion, the global response to the current pandemic has revealed a number of pitfalls in relation to the generation and interpretation of an enormous amount of scientific and clinical information in a time of significant pressure to ensure the viability of healthcare systems. These issues represent important lessons to be learned in preparation for future pandemics. At the same time, the significant progress made with capturing and interpreting data using machine learning algorithms and the ‘live’ update of the results of RCTs offer important opportunities for the provision of robust, yet rapid, clinical information, in conjunction with the traditional tenets of evidence-based medicine and the use of innovative strategies, such as trial hibernation, in the event of future public health emergencies.