Abstract
The current COVID-19 pandemic situation has stimulated an unplanned clinical research paradigm which is evident from the surge of clinical trial registrations and the increasing number of COVID-related publications. We aimed to explore the standards for research conduction, publications and retraction of articles related to COVID-19 pharmacotherapy research during the pandemic. We analysed data from the contemporary literatures on studies reporting pharmacological agents for COVID-19 using MEDLINE, PubMed, WHO database and Google Scholar between January 01, 2020 and March 20, 2021. The initial search revealed a total of 61,801 articles. Based on the inclusion criteria, a total of 124 studies related to various pharmacological agents were included in the final analysis. Most of the studies were reported from the United States (n = 30, 24%). Of the 124 studies, 50 (40%) were randomized controlled trials (RCTs). Immunomodulatory drugs-related (n = 17, 34%) and COVID-19 vaccine-related studies (n = 14, 28%) were the main topics in the relevant RCTs. The median days for dissemination of findings in journals were 114 days (IQR 61–189). A comparative analysis revealed that RCTs were disseminated earlier (median 79 days; IQR 52–131) when compared to observational studies (median = 144 days; IQR 69–206) (p = 0.003). Six papers were retracted from high impact journals; in which the average period till publication was 33 days. Retraction of papers occurred within 10–48 days. Expedited reviews, research approval and early publications of COVID-19 related pharmaceutical studies could have an impact on the quality of publications. However, the huge number of publications in short time creates confusion for readers during the early phases of the pandemic. Retraction of papers is alarming but ensures research integrity and correctness of scientific information. These abbreviated processes could affect patient care and public awareness. It is imperative to follow rapid but rigours ethical standards for research approval and peer-review process for publications during health pandemics.
Introduction
The global spread of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), which called COVID-19, is unprecedented and taking a devastating human toll. As of 21 March 2021, over 122 million people have been infected (44% Americans and 35% Europeans) and 2.7 million deaths (48% Americans and 34% Europeans) have been reported. 1 In response to COVID-19, the scientific community is in a race to acquire and share knowledge and experience, mainly based on other disease outbreaks that occurred in the past few decades. Obviously, this pandemic situation also stimulated new clinical research paradigm which is evident from the surge of clinical trial registrations on various websites and the increasing number of COVID-related publications. 2 This rapid increase in publications is in line with the steps taken by academic journals that accelerated the flow of peer-reviewed information by expediting editorial and peer-reviewed for COVID 19-related manuscripts and granting them open-access status upon publication.
Credible data and reliable conclusions are the key in policy decisions; however, urgency of finding a cure in a pandemic situation should not preclude responsible research practices. All research activities are subjected to scrutiny by competent ethics committees that must continue to function uninterrupted. Ethical principles should not be transgressed but may be adjusted to accelerate review and approval of novel approaches. Ethics and good practice of clinical research include autonomy where respected, and beneficence, non-maleficence and justice for research participants are ensured. Several international initiatives are dedicated on developing guidance for ethical conduct of research conducted during public health emergencies following the 2002–2004 SARS and 2014–2016 Ebola outbreaks. 3 Research ethics should not be a barrier to conduct research during an emergency setting. Therefore, it is very crucial to encourage stakeholders such as clinical researchers, institutional review bodies, sponsors, publishers and drug manufacturers to play their own parts in a neutral, transparent, on-time and scientific way. Rapid but rigorous ethical revisions of publications, especially during the early phases of pandemic, in which a large number of people receive new experimental or off-label treatments when involved in clinical trials.
In a circumstance of large volumes of research conducted on COVID-19, with expedited peer reviews for ethical approvals and the rapid pace in publications in an unprecedently short time, the main question is whether these studies comply with ethical standards set for this pandemic situation. There is a major concern especially when novel or alternate use of some pharmacological agents is being tested in volunteers or patient population. COVID-19 situation has stimulated high volumes of research being conducted and there is a concomitant surge in publications. However, focused analyses of compliance with ethical standards of COVID-19 publications that involves pharmacological agents are lacking.2,3 We aim to explore the standards for research conduction, publications and articles retraction in the therapeutic-related research during the COVID-19 pandemic.
Methods
The present report analysed data from the contemporary literatures on Covid-19 therapeutic-related research. The PubMed database of the National Centre for Biotechnology Information (NCBI), MEDLINE, WHO database and Google Scholar were used to identify research publications related to COVID-19. The PubMed search builder used following terms (‘Coronavirus’ [Mesh]) OR (‘COVID-19’ [Mesh] OR ‘SARS-CoV-2’ [Mesh]) to identify all publications between January 01, 2020 and March 20, 2021. Filters such as ‘humans’ species, ‘full text’ availability and ‘English’ language in addition to the study duration were performed. Only original studies related to COVID-19 treatment using pharmacological agents were selected. Meta analyses, systematic reviews, literature reviews, books and documents, letter to the editor that not containing any original data, opinions, guidelines, protocols and retracted or withdrawn publications were excluded. Articles that did not describe the duration of data collection were also excluded. Observational or interventional studies not related to COVID-19 treatment using pharmaceutical agents, for instance, diagnostic, molecular modelling and imaging studies were excluded. All related articles unavailable on PubMed were identified using Google Scholar, WHO database, other bibliographic databases, table of content of relevant journals and from reference list of other related scientific publications. Articles were identified and duplicates were removed from our final database by two independent reviewers.
A separate search to identify retraction of publications was conducted on PubMed using same search strategies adopted for identifying publications. The ‘retracted publication’ filter was used in the category of article types in PubMed search. In addition to this, newspaper articles, other relevant database on retraction of papers, reference list of other journal articles were explored. Only retracted original research articles on potential pharmaceutical agents in COVID-19 treatment were included. Temporarily retracted papers were excluded. Reasons for retraction were identified from journal information and other related articles.
From the extracted data of publications, we explored the chronological distance between the final date of data collection in a study and its first publication in a journal (mostly online) to reflect the delay or expedition in dissemination of study findings. Median days and interquartile range (IQR) were reported. Mann–Whitney U Test was used to compare the median days took for dissemination for various types of studies. A 2-tailed p-value <0.05 was considered as significant.
In the case of retracted paper, in addition to the number of days needed for dissemination of findings in journals, number of days for retraction after publication was also calculated. Finally, ethical approval for each study included in the analysis was examined to assess the standards for research conduction.
Results
Initial search following the search strategies resulted in a total of 61,801 journal articles related to COVID-19. Further application of restrictions on article types to original studies such as clinical study, clinical trials (phases I, II and III), observational study, controlled clinical trial or RCTs resulted in 2180 articles. Abstracts of these studies were reviewed for the inclusion criteria which finally yielded 124 original articles (Figure 1). Of these, the majority were related to immunomodulation (n = 48, 39%) followed by hydroxychloroquine/chloroquine (n = 28, 23%), use of antiviral agents (n = 16, 13%) and safety and efficacy of COVID-19 vaccines (n = 14, 11%). Most of the studies were reported from the United States (n = 30, 24%) followed by China (n = 23, 19%), Italy (n = 16, 13%) and France (n = 12, 10%).
Flow diagram of study selection process.
Of the 124 original studies, 50 (40%) were RCTs,4–53 10 (8%) were non-randomized clinical trials and 64 (52%) were retrospective or prospective observational studies (Table 1). Immunomodulation was the main topic in RCTs along with safety and efficacy of COVID-19 vaccines (both n = 14; 28%). The majority of RCTs were reported from China (n = 13; 26%) followed by US, United Kingdom and Brazil (n = 8, 16% each).
Publications of randomized controlled trials (RCTs) on use of different drugs in COVID-19 treatment.
Overall, the median days for dissemination of study findings in a journal were 114 days (IQR 61–189). The comparative analysis revealed that RCTs were disseminated earlier in median 79 days (IQR 52–131) when compared to observational studies (median = 144 days (IQR 69–206) (p = 0.003).
All the studies stated obtaining ethical approval from the relevant Institutional Review Boards (IRB). As few medical journals such as BMJ, required the authors to include a statement of ‘patient and public involvement’ in the design, or conduct, or reporting, or dissemination plans of the research; this statement was given in only two publications.9,37,54 However, these two publications stated that the authors did not involve patients or public at any stage before publication.
The PubMed search on retracted articles resulted in retrieval of eight articles which were not relevant to our topic. However, further search using the reference list of related articles and other databases yielded six articles.55–60 (Table 2). The average period till publication was 33 days. Retraction of papers occurred within 10–48 days.
Retracted research publications related to the use of different drugs in COVID-19 treatment.
Discussion
Covid-19 pandemic is an unprecedented crisis; most of the world was not prepared for it and responded with a mix of natural apprehension and intentional confusion generation. The waffling science gave further credit to fears and confusion. Hasty publications that were soon retracted were interpreted as attempts to manipulate the truth; to serve the interest of a few and cover ups more than science correcting its own imperfections. There was some misinformation and mistakes that could be made while trying to expedite the publications. Misinformation and confusion during the pandemic not only impact patient care but may also affect public awareness and perception towards novel therapies and vaccines. The challenge will be deeper than just to develop vaccines that work and are safe, but also how many will trust and use them. Similar processes are being used to hastily publish manuscript and to develop vaccines. We do believe that the quality, but not the quantity of publications is much needed in such pandemic situation.
Research governance and ethics
COVID-19 pandemic has stimulated research activities across the world with expedited research approval and conduct process which includes quick peer-review process and publication. This is in-line with the exceptional circumstances; however, academic institutions are duty bound to ensure research ethics and good clinical practice to maintain highest quality and integrity. During the current situation, the surge of COVID-19 research proposals submitted for IRBs approval has exerted enormous administrative pressure on the governing system. 61 There are certain administrative challenges such as involving researchers from other institutions, conducting multicentre trials and preparation of research protocols for low-risk studies could impede the research during a pandemic. Therefore, implementation of reasonable solutions to these administrative processes through fast-track channel for research approval without compromising research ethics could be necessarily under these circumstances. 62 Despite that, it is the prime responsibility of IRB to look for the ethical aspects of any fast-track COVID-19 research through stringent review and oversight of the proposed research protocols submitted for approval. Due to the paucity of information regarding the severity and mechanism of COVID-19 in its early stage, particularly if subsequent waves of the disease are expected or occurred, it is difficult to propose a gold standard or universal treatment. Therefore, there is a need for a fast-track review and approval of research studies followed by active governance oversight by the IRB. However, strengthening the institutional research governance and infrastructure is almost needed for an independent ethical review with special considerations during these exceptional circumstances. 63 The World Health Organization (WHO) has provided guidelines for conducting ethical research in emergency contexts during the COVID-19 pandemic. 64 These ethical standards should be followed by the clinical researchers, IRBs, sponsors, journal editors and manufacturers during the emergency situations (Figure 2). 64 In addition, various concerns regarding the rapid development of research design, ethical review, conduction and dissemination of research within the context of COVID-19 pandemic should be taken into consideration. However, the lack of guidelines, consensus and standard of care for treatment of symptomatic COVID-19 patients worldwide make the situation worse especially with emergent of new virus strains and new waves of the pandemic. Nevertheless, in addition to the feasibility and dissemination of research projects, we should not ignore the impact of ethnicity, economic status, public awareness, social media and politics on the pandemic constrain and management.
Ethics standards during public health emergencies.
The FDA has developed a special emergency program, the Coronavirus Treatment Acceleration Program (CTAP), for future coronavirus therapies (https://acrpnet.org/2020/08/10/fda-unveils-coronavirus-treatment-acceleration-program/). 65 The system uses every available tool to transfer experimental therapies to patients as quickly as possible and at the same time ascertaining whether they are beneficial or harmful. Prior to publication in a journal, the peer review process by field experts remains crucial to guide the journal editor whether it should be approved for publication, refused or reconsidered after modification. The journal’s credibility relies on high-quality peer review, and once a paper is written, it is in the public domain where it can be reviewed and judged by other scientists afterwards.
Studies published on pharmaceutical agents during COVID-19
For better understanding the nature, transmission, prevention, pathogenesis, treatment and outcomes of the new pandemic, surge of articles has been published in a short time. The average daily publications on COVID-19 are around 137 PubMed papers; this number of papers never happened before with any disease in the world. 66 However, this could be risky and associated with inaccuracy, errors and misinformation with various degrees. Raynaud et al., reviewed 10,516 articles on COVID-19 and concluded that the majority was published without original data and even those with data showed a higher risk of bias and limited sample size. 67
All the current pharmaceutical agents during COVID-19 are still non-specific or off-labelled. Some of studies related to safety and effectiveness of COVID-19 vaccines reported quickly, in less than 2 months from the final day of data collection from healthy volunteers.43,46,50 The production of new vaccines against COVID-19 in short period, with emergency approval of its use, in addition to the novelty of the type of vaccine such as mRNA raised many questions regarding its efficacy and safety on the short and long-term run that cannot be answered with certainty at the moment. This short period from discovery of the disease, studying vaccination in human and publication needs further evaluation. Although the timely dissemination of research is required to honour the commitment of study participants, improve clinical care and advance the research enterprise, previous experience has demonstrated that about a quarter to half of all clinical trials remain unpublished, sometimes years after completion. 68 In addition, only 29% of completed clinical trials conducted by the faculty at major academic centres were published within two years of completion and only 13% reported results on clinical trial databases. 68 However, in the pandemic situation, knowledge dissemination through peer-review process of publication is under enormous pressure to minimize the time from article submission to acceptance and online publication. 61 To understand the publishing trend during pandemic situation, a recent publication has analysed six groups of journal articles. 2 It is evident that COVID-19 has stimulated the speedy publication of results from numerous clinical research trials, as evidenced by the dramatically shorter median time of 6 days between submission of an article and acceptance for publication. 2 Moreover, higher proportion of research articles on COVID has been accepted within 7 days in journals under as expedited fast-track publications. Contrarily, the median acceptance time for non-COVID-19 articles was found to be 84 days in the same journals during the same period and only 3% of these articles got accepted within 7 days post submission. These hasty publications pace can be further demonstrated based on more than 15 bibliometric analyses published to date. Odone et al demonstrated that 2.3% (around 10,000) of the total scientific publications worldwide during the initial 5 months in 2020 were on COVID-19. 68 Such large volume of research focused on a single topic is for unprecedented in the history of academic publications. Nearly 33% of papers were on the clinical management of COVID-19. However, more than 60% of articles were expert opinions that lack original data. Papers were published in 1881 journals, but half of these were featured in only 8% of journals. The US accounted for one-fourth of overall scientific production, followed by China and Italy. 68 Kambhampati et al. extracted PubMed data for 6831 papers till April 25, 2020 and found that 6415 papers (93.9%) were published in English language. 69 The ‘British Medical Journal’ (BMJ) published the highest number of articles (n = 252), followed by the ‘Journal of Medical Virology’ (n = 186). Darsono et al. conducted bibliometric analysis of 1475 publications from the Scopus database published between December 2019 and March 2020. 70 The authors reported 11 different types of research publications; of which 66.1% were original articles followed by reviews (11.3%) and notes (7.6%). The authors revealed that the scientific journal ‘Viruses’ was leading with 74 published articles, followed by ‘Lancet’ (n = 50) and the ‘Journal of Virology’ (n = 39). Another, bibliometric analysis by Tao et al. 71 using a larger dataset for over 20 years (2000–2020) from Web of Science database extracted 9760 publications. The Journal of Virology was leading in publications with 885 papers and the USA was the leading country with 959 publications, followed by China with 469 articles. The University of Hong Kong was the leading institution with 411 publications. Lou et al. 72 demonstrated that nearly 43% of publications were from hospitals, 35% from universities and 21% from research institutions. Nearly 33% were original articles, 16% were reviews and 11% were short communications. English was the dominant language, followed by Chinese. The authors also showed that 38% of publications were in epidemiology, 27% in virology and 14% in clinical features. Al-Zaman et al. 73 analysed 16,384 COVID-19-related papers from the Web of Science database, published between December 2019 and June 2020 and found 15 types of publications of which 40% were original articles.
Nevertheless, in order to maintain a balance for the anticipated relevance and significance of the fast-track COVID publications, it is the prime responsibility of the journal editors to maintain scientific peer-review based on the expertise of the reviewers in a particular field. The journal editors must adhere to the regulations of research ethics, with special consideration on the consenting process under the public health emergency. 74 Unfortunately, this process in some cases, could be impaired by the unprecedented increase in publishing by various free-access domains that has either suboptimal or no peer review process for assessing the quality and authenticity of data. 75 The preprint server such as bioRxiv, arXiv and medRxiv which allow online posting of manuscripts prior to peer review have published 2355, 801 and 587 COVID-related articles by May 2020. 76 Notably, publishing of scientific study without peer-review may be inaccurate and substandard. Therefore, the publication surge of COVID-19 articles with fast-track editorial process often undermines the peer-review process that may lead to the retraction of published studies. 77 It has been reported that a quarter of COVID-19 papers were retracted early during its presence in the preprint repository. 78
Retraction of COVID-19 publications
Up to date, according to Retraction Watch website (https://retractionwatch.com/retracted-coronavirus-covid-19-papers/), there are 102 retracted papers related to COVID-19, 11papers retracted due to journal error, four papers retracted and reinstated and six have expressions of concern. However, this website does not distinguish retraction and withdrawal of papers. 79 Based on our search criteria, we pointed to six retracted papers related to COVID-19 treatment as shown in Table 2.55–60,80 Authors have raised the concern that the results of some retracted papers could be used and cited in another publications and quickly spread through the social media before or without knowing the retraction of the original paper. 80
In contrast to the high number of COVID-19 publications which were believed to be based on high quality data, the retractions of two articles from prestigious journals in 2020 such as The New England Journal of Medicine (NEJM) and The Lancet, raised concerns about the ethical standards applied in these studies as well as the expedited peer-review process by these journals. 81 In 2020, as of 8 June, the retraction rate for COVID-19 papers was 0.097 in comparison to 0.023 for HIV publications. 66 The average retraction time was less than 2 weeks. 66 Retraction of papers was reported to occur regardless of the journal impact factor, author h-index and being open access or not. 78
Soltani and Patini showed that the database of retraction revealed a total of 26 articles were retracted as on June 18, 2020. 82 According to the authors, China and the US were the leading countries with papers retracted and the most common reasons for retraction were concerns, issues, errors in data, results and/or conclusion. 82 In these retracted papers, the authors were not directly involved in data collection and data sources and these studies were on use of antihypertensive and antimalarial drugs in COVID-19 patients. Unfortunately, the company that compiled and analysed the data, did not cooperate with the independent reviewers as they found inconsistencies in their data. Such retractions often pose serious concerns about the state of scientific research, as thousands of articles were submitted to online databases and journals with little to no peer review, and many critics believe long-held norms, for even the most prestigious journals, are eroding as they face mounting pressure to quickly review and publish new research articles.
The retracted Lancet and the NEJM papers failed at the peer review level; however, scientific community attention captured the errors in record time. The Lancet retracted the paper in question just 13 days after publications. 55 Cortegiani et al., reported 45 retracted papers of which 39 articles definitively retracted (20 were clinical studies and 5 were preclinical studies and none of the studies was RCT). 78 The median time from publication to retraction was 14(IQR 3.5–52.5 days) and this was due to results, data, ethical violation and lack of IRB approval.
In short, publication and retraction trends in COVID-19 reflect the quick response from the scientific community which, while not completely free from flaws, can still be credited with paying close attention and following every scientific development in the COVID-19 sphere of scientific activity. A collective and external peer-review process is still intact. However, relying on this system to ensure the accuracy and quality of time-sensitive scientific publications seems difficult for academia to adapt.
Conclusions
Expedited reviews, research approval and publications of COVID-19 related pharmaceutical studies could have an impact on the quality of publications. However, the huge number of publications in short time creates confusion for readers during the early phases of the pandemic. Retraction of papers is alarming but also assuring sign. We still need to explore whether these abbreviated processes could affect patient care and public awareness in this regard and how to measure this impact. It is imperative to follow rapid but rigours ethical standards for research approval and peer-review process for publications during health pandemics based on international guidelines that should ensure compliance with scientific and temporal requirements of research ethics during the emergency situations.
Footnotes
Authors’ contributions
all authors have a substantial contribution in the study design, data interpretation and writing, reviewing and approving the final manuscript
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
