Sage Journals: Discover world-class research

Abstract

The current COVID-19 pandemic situation has stimulated an unplanned clinical research paradigm which is evident from the surge of clinical trial registrations and the increasing number of COVID-related publications. We aimed to explore the standards for research conduction, publications and retraction of articles related to COVID-19 pharmacotherapy research during the pandemic. We analysed data from the contemporary literatures on studies reporting pharmacological agents for COVID-19 using MEDLINE, PubMed, WHO database and Google Scholar between January 01, 2020 and March 20, 2021. The initial search revealed a total of 61,801 articles. Based on the inclusion criteria, a total of 124 studies related to various pharmacological agents were included in the final analysis. Most of the studies were reported from the United States (n = 30, 24%). Of the 124 studies, 50 (40%) were randomized controlled trials (RCTs). Immunomodulatory drugs-related (n = 17, 34%) and COVID-19 vaccine-related studies (n = 14, 28%) were the main topics in the relevant RCTs. The median days for dissemination of findings in journals were 114 days (IQR 61–189). A comparative analysis revealed that RCTs were disseminated earlier (median 79 days; IQR 52–131) when compared to observational studies (median = 144 days; IQR 69–206) (p = 0.003). Six papers were retracted from high impact journals; in which the average period till publication was 33 days. Retraction of papers occurred within 10–48 days. Expedited reviews, research approval and early publications of COVID-19 related pharmaceutical studies could have an impact on the quality of publications. However, the huge number of publications in short time creates confusion for readers during the early phases of the pandemic. Retraction of papers is alarming but ensures research integrity and correctness of scientific information. These abbreviated processes could affect patient care and public awareness. It is imperative to follow rapid but rigours ethical standards for research approval and peer-review process for publications during health pandemics.

Keywords

COVID-19 pandemic publications research ethics retraction pharmaceutical studies

Introduction

The global spread of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), which called COVID-19, is unprecedented and taking a devastating human toll. As of 21 March 2021, over 122 million people have been infected (44% Americans and 35% Europeans) and 2.7 million deaths (48% Americans and 34% Europeans) have been reported.¹ In response to COVID-19, the scientific community is in a race to acquire and share knowledge and experience, mainly based on other disease outbreaks that occurred in the past few decades. Obviously, this pandemic situation also stimulated new clinical research paradigm which is evident from the surge of clinical trial registrations on various websites and the increasing number of COVID-related publications.² This rapid increase in publications is in line with the steps taken by academic journals that accelerated the flow of peer-reviewed information by expediting editorial and peer-reviewed for COVID 19-related manuscripts and granting them open-access status upon publication.

Credible data and reliable conclusions are the key in policy decisions; however, urgency of finding a cure in a pandemic situation should not preclude responsible research practices. All research activities are subjected to scrutiny by competent ethics committees that must continue to function uninterrupted. Ethical principles should not be transgressed but may be adjusted to accelerate review and approval of novel approaches. Ethics and good practice of clinical research include autonomy where respected, and beneficence, non-maleficence and justice for research participants are ensured. Several international initiatives are dedicated on developing guidance for ethical conduct of research conducted during public health emergencies following the 2002–2004 SARS and 2014–2016 Ebola outbreaks.³ Research ethics should not be a barrier to conduct research during an emergency setting. Therefore, it is very crucial to encourage stakeholders such as clinical researchers, institutional review bodies, sponsors, publishers and drug manufacturers to play their own parts in a neutral, transparent, on-time and scientific way. Rapid but rigorous ethical revisions of publications, especially during the early phases of pandemic, in which a large number of people receive new experimental or off-label treatments when involved in clinical trials.

In a circumstance of large volumes of research conducted on COVID-19, with expedited peer reviews for ethical approvals and the rapid pace in publications in an unprecedently short time, the main question is whether these studies comply with ethical standards set for this pandemic situation. There is a major concern especially when novel or alternate use of some pharmacological agents is being tested in volunteers or patient population. COVID-19 situation has stimulated high volumes of research being conducted and there is a concomitant surge in publications. However, focused analyses of compliance with ethical standards of COVID-19 publications that involves pharmacological agents are lacking.^2,3 We aim to explore the standards for research conduction, publications and articles retraction in the therapeutic-related research during the COVID-19 pandemic.

Methods

The present report analysed data from the contemporary literatures on Covid-19 therapeutic-related research. The PubMed database of the National Centre for Biotechnology Information (NCBI), MEDLINE, WHO database and Google Scholar were used to identify research publications related to COVID-19. The PubMed search builder used following terms (‘Coronavirus’ [Mesh]) OR (‘COVID-19’ [Mesh] OR ‘SARS-CoV-2’ [Mesh]) to identify all publications between January 01, 2020 and March 20, 2021. Filters such as ‘humans’ species, ‘full text’ availability and ‘English’ language in addition to the study duration were performed. Only original studies related to COVID-19 treatment using pharmacological agents were selected. Meta analyses, systematic reviews, literature reviews, books and documents, letter to the editor that not containing any original data, opinions, guidelines, protocols and retracted or withdrawn publications were excluded. Articles that did not describe the duration of data collection were also excluded. Observational or interventional studies not related to COVID-19 treatment using pharmaceutical agents, for instance, diagnostic, molecular modelling and imaging studies were excluded. All related articles unavailable on PubMed were identified using Google Scholar, WHO database, other bibliographic databases, table of content of relevant journals and from reference list of other related scientific publications. Articles were identified and duplicates were removed from our final database by two independent reviewers.

A separate search to identify retraction of publications was conducted on PubMed using same search strategies adopted for identifying publications. The ‘retracted publication’ filter was used in the category of article types in PubMed search. In addition to this, newspaper articles, other relevant database on retraction of papers, reference list of other journal articles were explored. Only retracted original research articles on potential pharmaceutical agents in COVID-19 treatment were included. Temporarily retracted papers were excluded. Reasons for retraction were identified from journal information and other related articles.

From the extracted data of publications, we explored the chronological distance between the final date of data collection in a study and its first publication in a journal (mostly online) to reflect the delay or expedition in dissemination of study findings. Median days and interquartile range (IQR) were reported. Mann–Whitney U Test was used to compare the median days took for dissemination for various types of studies. A 2-tailed p-value <0.05 was considered as significant.

In the case of retracted paper, in addition to the number of days needed for dissemination of findings in journals, number of days for retraction after publication was also calculated. Finally, ethical approval for each study included in the analysis was examined to assess the standards for research conduction.

Results

Initial search following the search strategies resulted in a total of 61,801 journal articles related to COVID-19. Further application of restrictions on article types to original studies such as clinical study, clinical trials (phases I, II and III), observational study, controlled clinical trial or RCTs resulted in 2180 articles. Abstracts of these studies were reviewed for the inclusion criteria which finally yielded 124 original articles (Figure 1). Of these, the majority were related to immunomodulation (n = 48, 39%) followed by hydroxychloroquine/chloroquine (n = 28, 23%), use of antiviral agents (n = 16, 13%) and safety and efficacy of COVID-19 vaccines (n = 14, 11%). Most of the studies were reported from the United States (n = 30, 24%) followed by China (n = 23, 19%), Italy (n = 16, 13%) and France (n = 12, 10%).

Figure 1.

Flow diagram of study selection process.

Of the 124 original studies, 50 (40%) were RCTs,^4–53 10 (8%) were non-randomized clinical trials and 64 (52%) were retrospective or prospective observational studies (Table 1). Immunomodulation was the main topic in RCTs along with safety and efficacy of COVID-19 vaccines (both n = 14; 28%). The majority of RCTs were reported from China (n = 13; 26%) followed by US, United Kingdom and Brazil (n = 8, 16% each).

Table 1.

Publications of randomized controlled trials (RCTs) on use of different drugs in COVID-19 treatment.

Author	Country	Data collection period	Days took for dissemination in journals	Drug under study
Huang et al.⁴	China	27 Jan to 15 Feb 2020	48	Chloroquine
Cao et al.⁵	China	18 Jan 3 Feb 2020	45	Lopinavir, ritonavir
Hung et al.⁶	China	10 Feb to 20 Mar 2020	55	Lopinavir, ritonavir, ribavirin and interferon
Reynolds et al.⁷	China	6 Feb to 3 Apr 2020	53	Remdesivir
Wang et al.⁸	China	Dec 2019 to 25 Feb 2020	14	Methylprednisolone
Qin et al.⁹	China	11 to 29 Feb 2020	77	Hydroxychloroquine
Tang et al.¹⁰	Iran	20 Apr to 20 May 2020	97	Interferon beta 1b
Rahmani et al.¹¹	Iran	20 Mar to 30 May 2020	8	Colchicine, lopinavir/ritonavir (Kaletra)
Ten Doesschate et al.¹²	Netherlands	25 Mar to 23 Apr 2020	45	Bacillus Calmette-Guérin Vaccination
Garcaia et al.¹³	Spain	21 Apr to 31 May 2020	5	Melatonin
Beigel et al.¹⁴	UK	21 February to 19 April 2020	33	Remdesivir
Boulware et al.¹⁵	US and Canada	17 March to 6 May 2020	92	Hydroxychloroquine
Borba et al.¹⁶	Brazil	23 Mar to 5 Apr 2020	20	Chloroquine
Recovery Collaborative Group et al.¹⁷	UK	19 Mar to 8 Jun 2020	40	Dexamethasone
Abdelalim et al.¹⁸	Egypt	Aug to Nov 2020	35	Corticosteroid
Jamaati et al.¹⁹	Iran	Mar 20	322	Corticosteroid
Tang et al.²⁰	China	14 Feb to 31 Mar 2020	298	Corticosteroids
Dequin et al.²¹	France	7 Mar to 29 Jun 2020	66	Hydrocortisone
Angus et al.²²	Australia, Canada, France, Ireland, Netherlands, New Zealand, UK, US	9 Mar to 17 Jun 2020	78	Hydrocortisone
Tomazini et al.²³	Brazil	17 Apr to 23 Jun 2020	72	Dexamethasone
Lopes et al.²⁴	Brazil	9 Apr to 26 Jul 2020	178	ACEI/ARB
Recovery Collaborative Group²⁵	UK	7 Apr to 27 Nov 2020	68	Lopinavir, ritonavir
Cavalcanti et al.²⁶	Brazil	29 Mar to 17 May 2020	68	Hydroxychloroquine, azithromycin
Sekhavati et al.²⁷	Iran	24 Apr to 8 May 2020	110	Azithromycin
Furtado et al.²⁸	Brazil	28 Mar to 19 May 2020	110	Azithromycin
Mitjà et al.²⁹	Spain	17 Mar to 28 Apr 2020	180	Hydroxychloroquine
Kalil et al.³⁰	US, Singapore, South Korea, Mexico, Japan, Spain, UK, Denmark	8 May to 1 Jul 2020	164	Baricitinib, remdesivir
Goldman et al.³¹	US, Italy, Spain, Germany, Hong Kong, Singapore, South Korea and Taiwan	6 Mar to 26 Mar 2020	63	Remdesivir
Spinner et al.³²	US, Europe, Asia	15 Mar to 18 Apr 2020	126	Remdesivir
Zhao et al.³³	China	Feb 2 to 15 Mar 2020	200	Tocilizumab, favipiravir
Salama et al.³⁴	US	30 Sep 2020	79	Tocilizumab
Salvarani et al.³⁵	Italy	31 Mar to 11 Jun 2020	132	Tocilizumab
Hermine et al.³⁶	France	31 Mar to 18 April 2020	290	Tocilizumab
Veiga et al.³⁷	Brazil	8 May to 17 Jul 2020	188	Tocilizumab
Stone et al.³⁸	US	20 Apr to 15 Jun 2020	129	Tocilizumab
Lopes et al.³⁹	Brazil	11 Apr to 30 Aug 2020	159	Colchicine
Folegatti et al.⁴⁰	UK	23 Apr to 21 May 2020	61	Adenovirus recombinant vector vaccine (ChAdOx1)
Keech et al.⁴¹	Australia	27 May to 6 Jun 2020	89	Recombinant spiroprotein nanoparticle vaccine (NVX-CoV2373)
Logunov et al.⁴²	Russia	7 Sep to 4 Nov 2020	91	rAd26 and rAd5 vector-based vaccine
Mulligan et al.⁴³	US	4 May to 19 June 2020	54	RNA vaccine (BNT162b1)
Ramasamy et al.⁴⁴	UK	30 May to 8 Aug 2020	133	Adenovirus recombinant vector vaccine (ChAdOx1)
Walsh et al.⁴⁵	US	4 May to 22 Jun 2020	114	RNA vaccine (BNT162b1/BNT162b2)
Xia et al.⁴⁶	China	16 Jun to 27 Jul 2020	43	Inactivated vaccine
Xia et al.⁴⁷	China	29 Apr to 28 Jun 2020	109	Inactivated vaccine (BBIBP-CorV)
Zhang et al.⁴⁸	China	16 Apr to 25 Apr 2020	283	Inactivated vaccine
Zhu et al.⁴⁹	China	11 April to 16 April 2020	95	Adenovirus type-5-vectored vaccine
Pu et al.⁵⁰	China	Jan to Aug 2020	37	Inactivated vaccine
Voysey et al.⁵¹	UK, Brazil and South Africa	23 Apr to 4 Nov 2020	67	Adenovirus recombinant vector vaccine (ChAdOx1nCoV-19/AZD1222)
Ella et al.⁵²	India	7 Sep to 13 Sep 2020	101	Inactivated vaccine (BBV152)
Richmond et al.⁵³	Australia	19 Jun to 23 Sep 2020	129	Recombinant spiroprotein vaccine (SCB-2019)

Overall, the median days for dissemination of study findings in a journal were 114 days (IQR 61–189). The comparative analysis revealed that RCTs were disseminated earlier in median 79 days (IQR 52–131) when compared to observational studies (median = 144 days (IQR 69–206) (p = 0.003).

All the studies stated obtaining ethical approval from the relevant Institutional Review Boards (IRB). As few medical journals such as BMJ, required the authors to include a statement of ‘patient and public involvement’ in the design, or conduct, or reporting, or dissemination plans of the research; this statement was given in only two publications.^9,37,54 However, these two publications stated that the authors did not involve patients or public at any stage before publication.

The PubMed search on retracted articles resulted in retrieval of eight articles which were not relevant to our topic. However, further search using the reference list of related articles and other databases yielded six articles.^55–60 (Table 2). The average period till publication was 33 days. Retraction of papers occurred within 10–48 days.

Table 2.

Retracted research publications related to the use of different drugs in COVID-19 treatment.

Author	Country	Type	Data collection period	Days for dissemination in journal	Days for retraction after dissemination	Drug under study	Reason for retraction
Mehra et al.⁵⁵	North and South America, Europe, Africa, Asia, Australia	Retrospective	20 Dec 2019 to 14 Apr 2020	39	14	Hydroxychloroquine, chloroquine, macrolide	Third-party investigation, unreliable data
Davido et al.⁵⁶	France	Retrospective	2 Mar to 17 Apr 2020	25	10	Hydroxychloroquine, Azithromycin	Issues in data and results
Patel et al.⁵⁷	North and South America, Europe, Africa, Asia	Retrospective	1 Jan to 1 Mar 2020	32	NA	Ivermectin	Unknown
Raharusuna et al.⁵⁸	Indonesia	Retrospective	2 Mar to 24 Apr 2020	7	NA	Vitamin D	Unknown
Kim et al.⁵⁹	South Korea	Retrospective	28 Feb to 28 Apr 2020	61	48	Lopinavir, Ritonavir	Issues in results
Nogués et al.⁶⁰	Spain	NA	NA	NA	29	Vitamin D	Concerns about the description

Discussion

Covid-19 pandemic is an unprecedented crisis; most of the world was not prepared for it and responded with a mix of natural apprehension and intentional confusion generation. The waffling science gave further credit to fears and confusion. Hasty publications that were soon retracted were interpreted as attempts to manipulate the truth; to serve the interest of a few and cover ups more than science correcting its own imperfections. There was some misinformation and mistakes that could be made while trying to expedite the publications. Misinformation and confusion during the pandemic not only impact patient care but may also affect public awareness and perception towards novel therapies and vaccines. The challenge will be deeper than just to develop vaccines that work and are safe, but also how many will trust and use them. Similar processes are being used to hastily publish manuscript and to develop vaccines. We do believe that the quality, but not the quantity of publications is much needed in such pandemic situation.

Research governance and ethics

COVID-19 pandemic has stimulated research activities across the world with expedited research approval and conduct process which includes quick peer-review process and publication. This is in-line with the exceptional circumstances; however, academic institutions are duty bound to ensure research ethics and good clinical practice to maintain highest quality and integrity. During the current situation, the surge of COVID-19 research proposals submitted for IRBs approval has exerted enormous administrative pressure on the governing system.⁶¹ There are certain administrative challenges such as involving researchers from other institutions, conducting multicentre trials and preparation of research protocols for low-risk studies could impede the research during a pandemic. Therefore, implementation of reasonable solutions to these administrative processes through fast-track channel for research approval without compromising research ethics could be necessarily under these circumstances.⁶² Despite that, it is the prime responsibility of IRB to look for the ethical aspects of any fast-track COVID-19 research through stringent review and oversight of the proposed research protocols submitted for approval. Due to the paucity of information regarding the severity and mechanism of COVID-19 in its early stage, particularly if subsequent waves of the disease are expected or occurred, it is difficult to propose a gold standard or universal treatment. Therefore, there is a need for a fast-track review and approval of research studies followed by active governance oversight by the IRB. However, strengthening the institutional research governance and infrastructure is almost needed for an independent ethical review with special considerations during these exceptional circumstances.⁶³ The World Health Organization (WHO) has provided guidelines for conducting ethical research in emergency contexts during the COVID-19 pandemic.⁶⁴ These ethical standards should be followed by the clinical researchers, IRBs, sponsors, journal editors and manufacturers during the emergency situations (Figure 2).⁶⁴ In addition, various concerns regarding the rapid development of research design, ethical review, conduction and dissemination of research within the context of COVID-19 pandemic should be taken into consideration. However, the lack of guidelines, consensus and standard of care for treatment of symptomatic COVID-19 patients worldwide make the situation worse especially with emergent of new virus strains and new waves of the pandemic. Nevertheless, in addition to the feasibility and dissemination of research projects, we should not ignore the impact of ethnicity, economic status, public awareness, social media and politics on the pandemic constrain and management.

Figure 2.

Ethics standards during public health emergencies.

The FDA has developed a special emergency program, the Coronavirus Treatment Acceleration Program (CTAP), for future coronavirus therapies (https://acrpnet.org/2020/08/10/fda-unveils-coronavirus-treatment-acceleration-program/).⁶⁵ The system uses every available tool to transfer experimental therapies to patients as quickly as possible and at the same time ascertaining whether they are beneficial or harmful. Prior to publication in a journal, the peer review process by field experts remains crucial to guide the journal editor whether it should be approved for publication, refused or reconsidered after modification. The journal’s credibility relies on high-quality peer review, and once a paper is written, it is in the public domain where it can be reviewed and judged by other scientists afterwards.

Studies published on pharmaceutical agents during COVID-19

For better understanding the nature, transmission, prevention, pathogenesis, treatment and outcomes of the new pandemic, surge of articles has been published in a short time. The average daily publications on COVID-19 are around 137 PubMed papers; this number of papers never happened before with any disease in the world.⁶⁶ However, this could be risky and associated with inaccuracy, errors and misinformation with various degrees. Raynaud et al., reviewed 10,516 articles on COVID-19 and concluded that the majority was published without original data and even those with data showed a higher risk of bias and limited sample size.⁶⁷

All the current pharmaceutical agents during COVID-19 are still non-specific or off-labelled. Some of studies related to safety and effectiveness of COVID-19 vaccines reported quickly, in less than 2 months from the final day of data collection from healthy volunteers.^43,46,50 The production of new vaccines against COVID-19 in short period, with emergency approval of its use, in addition to the novelty of the type of vaccine such as mRNA raised many questions regarding its efficacy and safety on the short and long-term run that cannot be answered with certainty at the moment. This short period from discovery of the disease, studying vaccination in human and publication needs further evaluation. Although the timely dissemination of research is required to honour the commitment of study participants, improve clinical care and advance the research enterprise, previous experience has demonstrated that about a quarter to half of all clinical trials remain unpublished, sometimes years after completion.⁶⁸ In addition, only 29% of completed clinical trials conducted by the faculty at major academic centres were published within two years of completion and only 13% reported results on clinical trial databases.⁶⁸ However, in the pandemic situation, knowledge dissemination through peer-review process of publication is under enormous pressure to minimize the time from article submission to acceptance and online publication.⁶¹ To understand the publishing trend during pandemic situation, a recent publication has analysed six groups of journal articles.² It is evident that COVID-19 has stimulated the speedy publication of results from numerous clinical research trials, as evidenced by the dramatically shorter median time of 6 days between submission of an article and acceptance for publication.² Moreover, higher proportion of research articles on COVID has been accepted within 7 days in journals under as expedited fast-track publications. Contrarily, the median acceptance time for non-COVID-19 articles was found to be 84 days in the same journals during the same period and only 3% of these articles got accepted within 7 days post submission. These hasty publications pace can be further demonstrated based on more than 15 bibliometric analyses published to date. Odone et al demonstrated that 2.3% (around 10,000) of the total scientific publications worldwide during the initial 5 months in 2020 were on COVID-19.⁶⁸ Such large volume of research focused on a single topic is for unprecedented in the history of academic publications. Nearly 33% of papers were on the clinical management of COVID-19. However, more than 60% of articles were expert opinions that lack original data. Papers were published in 1881 journals, but half of these were featured in only 8% of journals. The US accounted for one-fourth of overall scientific production, followed by China and Italy.⁶⁸ Kambhampati et al. extracted PubMed data for 6831 papers till April 25, 2020 and found that 6415 papers (93.9%) were published in English language.⁶⁹ The ‘British Medical Journal’ (BMJ) published the highest number of articles (n = 252), followed by the ‘Journal of Medical Virology’ (n = 186). Darsono et al. conducted bibliometric analysis of 1475 publications from the Scopus database published between December 2019 and March 2020.⁷⁰ The authors reported 11 different types of research publications; of which 66.1% were original articles followed by reviews (11.3%) and notes (7.6%). The authors revealed that the scientific journal ‘Viruses’ was leading with 74 published articles, followed by ‘Lancet’ (n = 50) and the ‘Journal of Virology’ (n = 39). Another, bibliometric analysis by Tao et al.⁷¹ using a larger dataset for over 20 years (2000–2020) from Web of Science database extracted 9760 publications. The Journal of Virology was leading in publications with 885 papers and the USA was the leading country with 959 publications, followed by China with 469 articles. The University of Hong Kong was the leading institution with 411 publications. Lou et al.⁷² demonstrated that nearly 43% of publications were from hospitals, 35% from universities and 21% from research institutions. Nearly 33% were original articles, 16% were reviews and 11% were short communications. English was the dominant language, followed by Chinese. The authors also showed that 38% of publications were in epidemiology, 27% in virology and 14% in clinical features. Al-Zaman et al.⁷³ analysed 16,384 COVID-19-related papers from the Web of Science database, published between December 2019 and June 2020 and found 15 types of publications of which 40% were original articles.

Nevertheless, in order to maintain a balance for the anticipated relevance and significance of the fast-track COVID publications, it is the prime responsibility of the journal editors to maintain scientific peer-review based on the expertise of the reviewers in a particular field. The journal editors must adhere to the regulations of research ethics, with special consideration on the consenting process under the public health emergency.⁷⁴ Unfortunately, this process in some cases, could be impaired by the unprecedented increase in publishing by various free-access domains that has either suboptimal or no peer review process for assessing the quality and authenticity of data.⁷⁵ The preprint server such as bioRxiv, arXiv and medRxiv which allow online posting of manuscripts prior to peer review have published 2355, 801 and 587 COVID-related articles by May 2020.⁷⁶ Notably, publishing of scientific study without peer-review may be inaccurate and substandard. Therefore, the publication surge of COVID-19 articles with fast-track editorial process often undermines the peer-review process that may lead to the retraction of published studies.⁷⁷ It has been reported that a quarter of COVID-19 papers were retracted early during its presence in the preprint repository.⁷⁸

Retraction of COVID-19 publications

Up to date, according to Retraction Watch website (https://retractionwatch.com/retracted-coronavirus-covid-19-papers/), there are 102 retracted papers related to COVID-19, 11papers retracted due to journal error, four papers retracted and reinstated and six have expressions of concern. However, this website does not distinguish retraction and withdrawal of papers.⁷⁹ Based on our search criteria, we pointed to six retracted papers related to COVID-19 treatment as shown in Table 2.^55–60,80 Authors have raised the concern that the results of some retracted papers could be used and cited in another publications and quickly spread through the social media before or without knowing the retraction of the original paper.⁸⁰

In contrast to the high number of COVID-19 publications which were believed to be based on high quality data, the retractions of two articles from prestigious journals in 2020 such as The New England Journal of Medicine (NEJM) and The Lancet, raised concerns about the ethical standards applied in these studies as well as the expedited peer-review process by these journals.⁸¹ In 2020, as of 8 June, the retraction rate for COVID-19 papers was 0.097 in comparison to 0.023 for HIV publications.⁶⁶ The average retraction time was less than 2 weeks.⁶⁶ Retraction of papers was reported to occur regardless of the journal impact factor, author h-index and being open access or not.⁷⁸

Soltani and Patini showed that the database of retraction revealed a total of 26 articles were retracted as on June 18, 2020.⁸² According to the authors, China and the US were the leading countries with papers retracted and the most common reasons for retraction were concerns, issues, errors in data, results and/or conclusion.⁸² In these retracted papers, the authors were not directly involved in data collection and data sources and these studies were on use of antihypertensive and antimalarial drugs in COVID-19 patients. Unfortunately, the company that compiled and analysed the data, did not cooperate with the independent reviewers as they found inconsistencies in their data. Such retractions often pose serious concerns about the state of scientific research, as thousands of articles were submitted to online databases and journals with little to no peer review, and many critics believe long-held norms, for even the most prestigious journals, are eroding as they face mounting pressure to quickly review and publish new research articles.

The retracted Lancet and the NEJM papers failed at the peer review level; however, scientific community attention captured the errors in record time. The Lancet retracted the paper in question just 13 days after publications.⁵⁵ Cortegiani et al., reported 45 retracted papers of which 39 articles definitively retracted (20 were clinical studies and 5 were preclinical studies and none of the studies was RCT).⁷⁸ The median time from publication to retraction was 14(IQR 3.5–52.5 days) and this was due to results, data, ethical violation and lack of IRB approval.

In short, publication and retraction trends in COVID-19 reflect the quick response from the scientific community which, while not completely free from flaws, can still be credited with paying close attention and following every scientific development in the COVID-19 sphere of scientific activity. A collective and external peer-review process is still intact. However, relying on this system to ensure the accuracy and quality of time-sensitive scientific publications seems difficult for academia to adapt.

Conclusions

Expedited reviews, research approval and publications of COVID-19 related pharmaceutical studies could have an impact on the quality of publications. However, the huge number of publications in short time creates confusion for readers during the early phases of the pandemic. Retraction of papers is alarming but also assuring sign. We still need to explore whether these abbreviated processes could affect patient care and public awareness in this regard and how to measure this impact. It is imperative to follow rapid but rigours ethical standards for research approval and peer-review process for publications during health pandemics based on international guidelines that should ensure compliance with scientific and temporal requirements of research ethics during the emergency situations.

Footnotes

Authors’ contributions

all authors have a substantial contribution in the study design, data interpretation and writing, reviewing and approving the final manuscript

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Ayman El-Menyar

Author biographies

Ayman El-Menyar, MS Cardiology, FRCP (Glasgow), FESC, FACC, is a Director of Clinical Research in Trauma and Vascular Surgery at Hamad Medical Corporation (HMC) and Associate Professor, Clinical Medicine at Weill Cornell Medical College, Qatar.

Ahammed Mekkodathil, MPH, is Research Specialist, Trauma Surgery, Hamad Medical Corporation.

Mohammad Asim, MSc, PhD, is a Research Specialist at Trauma Surgery, Hamad General Hospital, Hamad Medical Corporation (HMC), Qatar.

Rafael Consunji, MD, MPH, is a General Surgeon with additional training in public health, surgical critical care and trauma surgery. He is Director, Injury Prevention at Hamad Trauma Center, Hamad Medical Corporation.

Sandro Rizoli, MD, PhD, FRCSC, FACS, is a Medical Director Trauma Surgery Section, HMC, Editor-in-Chief of Panamerican Journal of Trauma and JEMTAC.

Ahmed Abdel-Aziz Bahey, Bsc, Msc, BCPS, is a Senior Clinical Pharmacist in Trauma Critical Care, Hamad Medical Corporation.

Hassan Al-Thani, MD, FACS, MBA, MBBch, CABS, FRCSI/C, is Head of Trauma and Vascular Surgery Sections of Hamad Medical Corporation. Also, President of the Gulf Vascular Society and Chair of ATLS Chapter.

References

WHO status report: Coronavirus disease (COVID-19) weekly epidemiological update, https://www.who.int/publications/m/item/weekly-epidemiological-update-on-covid-19 (2021, accessed 8 April 2021).

Palayew

Norgaard

Safreed-Harmon

, et al. Pandemic publishing poses a new COVID-19 challenge. Nat Hum Behav 2020; 4: 666–669.

World Health Organization. Ethical standards for research during public health emergencies: distilling existing guidance to support COVID-19 R&D. Geneva, Switzerland: WHO, https://apps.who.int/iris/handle/10665/331507 (2020, accessed 20 March 2021).

Huang

Tang

Pang

, et al. Treating COVID-19 with chloroquine. J Mol Cell Biol 2020; 12(4): 322–325.

Cao

Wang

Wen

, et al. A trial of lopinavir-ritonavir in adults hospitalized with severe Covid-19. N Engl J Med 2020; 382(19): 1787–1799.

Hung

Lung

Tso

, et al. Triple combination of interferon beta-1b, lopinavir-ritonavir, and ribavirin in the treatment of patients admitted to hospital with COVID-19: an open-label, randomised, phase 2 trial. Lancet 2020; 395(10238): 1695–1704.

Reynolds

Adhikari

Pulgarin

, et al. Renin-angiotensin-aldosterone system inhibitors and risk of Covid-19. N Engl J Med 2020; 382(25): 2441–2448.

Wang

Zhang

, et al. Remdesivir in adults with severe COVID-19: a randomised, double-blind, placebo-controlled, multicentre trial. Lancet 2020; 395(10236): 1569–1578.

Qin

Zhou

, et al. Effectiveness of glucocorticoid therapy in patients with severe coronavirus disease 2019: protocol of a randomized controlled trial. Chin Med J 2020; 133(9): 1080–1086.

10.

Tang

Cao

Han

, et al. Hydroxychloroquine in patients with mainly mild to moderate coronavirus disease 2019: open label, randomised controlled trial. BMJ 2020; 369: 1849.

11.

Rahmani

Davoudi-Monfared

Nourian

, et al. Interferon β-1b in treatment of severe COVID-19: a randomized clinical trial. Int Immunopharmacol 2020; 88: 106903.

12.

Ten Doesschate

Moorlag

SJCFM

van der Vaart

, et al. Two randomized controlled trials of bacillus Calmette-Guérin vaccination to reduce absenteeism among health care workers and hospital admission by elderly persons during the COVID-19 pandemic: a structured summary of the study protocols for two randomised controlled trials. Trials 2020; 21(1): 481.

13.

García

Rodriguez-Rubio

Mariblanca

, et al. A randomized multicenter clinical trial to evaluate the efficacy of melatonin in the prophylaxis of SARS-CoV-2 infection in high-risk contacts (MeCOVID Trial): a structured summary of a study protocol for a randomised controlled trial. Trials 2020; 21(1): 466.

14.

Beigel

Tomashek

Dodd

, et al. Remdesivir for the treatment of Covid-19 - preliminary report. N Engl J Med 2020; 383(19): 1813–1826.

15.

Boulware

Pullen

Bangdiwala

, et al. A randomized trial of hydroxychloroquine as postexposure prophylaxis for Covid-19. N Engl J Med 2020; 383(6): 517–525.

16.

Borba

MGS

Val

FFA

Sampaio

, et al. Effect of high vs low doses of chloroquine diphosphate as adjunctive therapy for patients hospitalized with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection: a randomized clinical trial. JAMA Netw Open 2020; 3(4): e208857.

17.

Recovery Collaborative Group, Horby

Lim

, et al. Dexamethasone in hospitalized patients with Covid-19. N Engl J Med 2021; 384(8): 693–704.

18.

Abdelalim

Mohamady

Elsayed

, et al. Corticosteroid nasal spray for recovery of smell sensation in COVID-19 patients: a randomized controlled trial. Am J Otolaryngol 2021; 42(2): 102884.

19.

Jamaati

Hashemian

Farzanegan

, et al. No clinical benefit of high dose corticosteroid administration in patients with COVID-19: a preliminary report of a randomized clinical trial. Eur J Pharmacol 2021; 897: 173947.

20.

Tang

Feng

, et al. Early use of corticosteroid may prolong SARS-CoV-2 shedding in non-intensive care unit patients with COVID-19 pneumonia: a multicenter, single-blind, randomized control trial. Respiration 2021; 100(2): 116–126.

21.

Dequin

Heming

Meziani

, et al. Effect of hydrocortisone on 21-day mortality or respiratory support among critically ill patients with COVID-19: a randomized clinical trial. JAMA 2020; 324(13): 1298–1306.

22.

Angus

Derde

Al-Beidh

, et al. Effect of hydrocortisone on mortality and organ support in patients with severe COVID-19: the REMAP-CAP COVID-19 corticosteroid domain randomized clinical trial. JAMA 2020; 324(13): 1317–1329.

23.

Tomazini

Maia

Cavalcanti

, et al. Effect of dexamethasone on days alive and ventilator-free in patients with moderate or severe acute respiratory distress syndrome and COVID-19: the CoDEX Randomized Clinical Trial. JAMA 2020; 324(13): 1307–1316.

24.

Lopes

Macedo

AVS

de Barros

Silva

PGM

, et al. Continuing versus suspending angiotensin-converting enzyme inhibitors and angiotensin receptor blockers: impact on adverse outcomes in hospitalized patients with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)-the BRACE CORONA Trial. Am Heart J 2020; 226: 49–59.

25.

Recovery Collaborative Group. Lopinavir-ritonavir in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial. Lancet 2020; 396(10259): 1345–1352.

26.

Cavalcanti

Zampieri

Rosa

, et al. Hydroxychloroquine with or without azithromycin in mild-to-moderate Covid-19. N Engl J Med 2020; 383(21): 2041–2052.

27.

Sekhavati

Jafari

Seyed Alinaghi

, et al. Safety and effectiveness of azithromycin in patients with COVID-19: an open-label randomised trial. Int J Antimicrob Agents 2020; 56(4): 106143.

28.

Furtado

RHM

Berwanger

Fonseca

, et al. Azithromycin in addition to standard of care versus standard of care alone in the treatment of patients admitted to the hospital with severe COVID-19 in Brazil (COALITION II): a randomised clinical trial. Lancet 2020; 396(10256): 959–967.

29.

Mitjà

Corbacho-Monné

Ubals

, et al. Hydroxychloroquine for early treatment of adults with mild Covid-19: a randomized-controlled trial. Clin Infect Dis 2020; 2020: ciaa1009.

30.

Kalil

Patterson

Mehta

, et al. Baricitinib plus Remdesivir for hospitalized adults with Covid-19. N Engl J Med 2021; 384(9): 795–807.

31.

Goldman

Lye

DCB

Hui

, et al. Remdesivir for 5 or 10 days in patients with severe Covid-19. N Engl J Med 2020; 383(19): 1827–1837.

32.

Spinner

Gottlieb

Criner

, et al. Effect of Remdesivir vs standard care on clinical status at 11 days in patients with moderate COVID-19: a randomized clinical trial. JAMA 2020; 324(11): 1048–1057.

33.

Zhao

Zhu

Zhang

, et al. Tocilizumab combined with favipiravir in the treatment of COVID-19: a multicenter trial in a small sample size. Biomed Pharmacother 2021; 133: 110825.

34.

Salama

Han

Yau

, et al. Tocilizumab in patients hospitalized with Covid-19 pneumonia. N Engl J Med 2021; 384(1): 20–30.

35.

Salvarani

Dolci

Massari

, et al. Effect of tocilizumab vs standard care on clinical worsening in patients hospitalized with COVID-19 pneumonia: a randomized clinical trial. JAMA Intern Med 2021; 181(1): 24–31.

36.

Hermine

Mariette

Tharaux

, et al. Effect of tocilizumab vs usual care in adults hospitalized with COVID-19 and moderate or severe pneumonia: a randomized clinical trial. JAMA Intern Med 2021; 181(1): 32–40.

37.

Veiga

Prats

JAGG

Farias

DLC

, et al. Effect of tocilizumab on clinical outcomes at 15 days in patients with severe or critical coronavirus disease 2019: randomised controlled trial. BMJ 2021; 372: n84.

38.

Stone

Frigault

Serling-Boyd

, et al. Efficacy of tocilizumab in patients hospitalized with Covid-19. N Engl J Med 2020; 383(24): 2333–2344.

39.

Lopes

Bonjorno

Giannini

, et al. Beneficial effects of colchicine for moderate to severe COVID-19: a randomised, double-blinded, placebo-controlled clinical trial. RMD Open 2021; 7(1): e001455.

40.

Folegatti

Ewer

Aley

, et al. Safety and immunogenicity of the ChAdOx1 nCoV-19 vaccine against SARS-CoV-2: a preliminary report of a phase 1/2, single-blind, randomised controlled trial. Lancet 2020; 396(10249): 467–478.

41.

Keech

Albert

Cho

, et al. Phase 1-2 trial of a SARS-CoV-2 recombinant spike protein nanoparticle vaccine. N Engl J Med 2020; 383(24): 2320–2332.

42.

Logunov

Dolzhikova

Shcheblyakov

, et al. Safety and efficacy of an rAd26 and rAd5 vector-based heterologous prime-boost COVID-19 vaccine: an interim analysis of a randomised controlled phase 3 trial in Russia. Lancet 2021; 397(10275): 671–681.

43.

Mulligan

Lyke

Kitchin

, et al. Phase I/II study of COVID-19 RNA vaccine BNT162b1 in adults. Nature 2020; 586(7830): 589–593.

44.

Ramasamy

Minassian

Ewer

, et al. Safety and immunogenicity of ChAdOx1 nCoV-19 vaccine administered in a prime-boost regimen in young and old adults (COV002): a single-blind, randomised, controlled, phase 2/3 trial. Lancet 2021; 396(10267): 1979–1993.

45.

Walsh

Frenck

Jr Falsey

, et al. Safety and immunogenicity of two RNA-based Covid-19 vaccine candidates. N Engl J Med 2020; 383(25): 2439–2450.

46.

Xia

Duan

Zhang

, et al. Effect of an inactivated vaccine against SARS-CoV-2 on safety and immunogenicity outcomes: interim analysis of 2 randomized clinical trials. JAMA 2020; 324(10): 951–960.

47.

Xia

Zhang

Wang

, et al. Safety and immunogenicity of an inactivated SARS-CoV-2 vaccine, BBIBP-CorV: a randomised, double-blind, placebo-controlled, phase 1/2 trial. Lancet Infect Dis 2021; 21(1): 39–51.

48.

Zhang

Zeng

Pan

, et al. Safety, tolerability, and immunogenicity of an inactivated SARS-CoV-2 vaccine in healthy adults aged 18-59 years: a randomised, double-blind, placebo-controlled, phase 1/2 clinical trial. Lancet Infect Dis 2021; 21(2): 181–192.

49.

Zhu

Guan

, et al. Safety, tolerability, and immunogenicity of a recombinant adenovirus type-5 vectored COVID-19 vaccine: a dose-escalation, open-label, non-randomised, first-in-human trial. Lancet 2020; 395(10240): 1845–1854.

50.

Yin

, et al. An in-depth investigation of the safety and immunogenicity of an inactivated SARS-CoV-2 vaccine. medRxiv. 2020. DOI: 10.1101/2020.09.27.20189548.

51.

Voysey

Clemens

SAC

Madhi

, et al. Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK. Lancet 2021; 397(10269): 99–111.

52.

Ella

Vadrevu

Jogdand

, et al. A phase 1: safety and immunogenicity trial of an inactivated SARS-CoV-2 vaccine-BBV152. medRxiv. 2020. DOI: 10.1101/2020.12.11.20210419.

53.

Richmond

Hatchuel

Dong

, et al. Safety and immunogenicity of S-Trimer (SCB-2019), a protein subunit vaccine candidate for COVID-19 in healthy adults: a phase 1, randomised, double-blind, placebo-controlled trial. Lancet 2021; 397(10275): 682–694.

54.

Price

Schroter

Snow

, et al. Frequency of reporting on patient and public involvement (PPI) in research studies published in a general medical journal: a descriptive study. BMJ Open 2018; 8(3): e020452.

55.

Mehra

Desai

Ruschitzka

, et al. Retracted: hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis. Lancet 2020; 395(10240): 1820.

56.

Davido

Lansaman

Lawrence

, et al. Withdrawn. Hydroxychloroquine plus azithromycin: a potential interest in reducing in-hospital morbidity due to COVID-19 pneumonia (HI-ZY-COVID)? MedRxiv. 2020. DOI: 10.1101/2020.05.05.20088757.

57.

Patel

Desai

Grainger

, et al. Retracted. Ivermectin in COVID-19 related critical illness. ISGlobal, https://www.isglobal.org/documents/10179/6022921/Patel+et+al.+2020+version+1.pdf (2020, accessed 20 March 2021).

58.

Raharusuna

Priambada

Budiarti

, et al. Patterns of COVID-19 mortality and vitamin D: an Indonesian study. SSRN Electron J 2020. Available at: https://www.readcube.com/articles/10.2139%2Fssrn.3585561

59.

Kim

Jang

Park

, et al. Treatment response to hydroxychloroquine, lopinavir–ritonavir, and antibiotics for moderate COVID-19: a first report on the pharmacological outcomes from South Korea. MedRxiv. 2020. DOI: 10.1101/2020.05.13.20094193.

60.

Nogués

Ovejero

Quesada-Gomez

, et al. Calcifediol treatment and COVID-19-related outcomes. https://ssrn.com/abstract=3771318 (2021).

61.

Citerio

Bakker

Brochard

, et al. Critical care journals during the COVID-19 pandemic: challenges and responsibilities. Intensive Care Med 2020; 46: 1521–1523.

62.

Sisk

DuBois

. Research ethics during a pandemic: A call for normative and empirical analysis. Am J Bioethics 2020; 20: 82–84.

63.

Wang

Gao

, et al. Challenges and strategies to research ethics in conducting COVID-19 research. J Evid Based Med 2020; 13: 173–177.

64.

World Health Organization (WHO reference number: WHO/RFH/20.1.) Ethical standards and their application to research during public health emergencies, https://www.who.int/blueprint/priority-diseases/key-action/liverecovery-save-of-ethical-standards-for-research-during-public-health-emergencies.pdf (2020, accessed 20 March 2021).

65.

https://acrpnet.org/2020/08/10/fda-unveils-coronavirus-treatment-acceleration-program/ (2021, accessed 8 April 2021).

66.

Yeo-Teh

NSL

Tang

. An alarming retraction rate for scientific publications on coronavirus disease 2019 (COVID-19). Account Res 2021; 28(1): 47–53.

67.

Raynaud

Zhang

Louis

, et al. COVID-19-related medical research: a meta-research and critical appraisal. BMC Med Res Methodol 2021; 21(1): 1.

68.

Odone

Salvati

Bellini

, et al. The runaway science: a bibliometric analysis of the COVID-19 scientific literature. Acta Biomed 2020; 91(9-S): 34–39.

69.

Kambhampati

SBS

Vaishya

Vaish

. Unprecedented surge in publications related to COVID-19 in the first three months of pandemic: a bibliometric analytic report. J Clin Orthop Trauma 2020; 11(Suppl 3): S304–S306.

70.

Darsono

Rohmana

Busro

. Against COVID-19 pandemic: bibliometric assessment of world scholars’ international publications related to COVID-19. Jurnal Komunikasi Ikatan Sarjana Komunikasi Indonesia 2020; 5(1): 75–89.

71.

Tao

Zhou

Yao

, et al. COVID-19 will stimulate a new coronavirus research breakthrough: a 20-year bibliometric analysis. Ann Transl Med 2020; 8(8): 528.

72.

Lou

Tian

Niu

, et al. Coronavirus disease 2019: a bibliometric analysis and review. Eur Rev Med Pharmacol Sci 2020; 24(6): 3411–3421.

73.

Al-Zaman

. Bibliometric analysis of COVID-19 literature. medRxiv. 2020. DOI: 10.1101/2020.07.15.20154989.

74.

Shaw

Elger

. Publication ethics in public health emergencies. J Public Health 2017; 39(3): 640–643.

75.

Memon

Rathore

. Publishing research during pandemics: are you vulnerable to the COVID-19 or predatory publishers? J Pak Med Assoc 2020; 70(Suppl 3): S166–S168.

76.

Kwon

. How preprint servers are blocking bad coronavirus research. Nature 2020; 581: 130–131.

77.

Teixeira da Silva

. An alert to COVID-19 literature in predatory publishing venues. J Acad Libr 2020; 46(5): 102187.

78.

Cortegiani

Catalisano

Ippolito

, et al. Retracted papers on SARS-CoV-2 and COVID-19. Br J Anaesth 2021; 126(4): e155–e156.

79.

https://retractionwatch.com/retracted-coronavirus-covid-19-papers/ (2021, accessed 8 April 2021).

80.

Henrina

Lim

Pranata

. COVID-19 and misinformation: how an infodemic fuelled the prominence of vitamin D. Br J Nutr 2021;125(3): 359–360.

81.

Piller

Servick

. Two elite medical journals retract coronavirus papers over data integrity questions. Science Magazine, https://www.sciencemag.org/news/2020/06/two-elite-medical-journals-retract-coronavirus-papers-over-data-integrity-questions# (2020, accessed 20 March 2021).

82.

Soltani

Patini

. Retracted COVID-19 articles: a side-effect of the hot race to publication. Scientometrics 2020; 125: 819–822.

Publications and retracted articles of COVID-19 pharmacotherapy-related research: A systematic review

Abstract

Keywords

Introduction

Methods

Results

Discussion

Research governance and ethics

Studies published on pharmaceutical agents during COVID-19

Retraction of COVID-19 publications

Conclusions

Footnotes

Authors’ contributions

Declaration of conflicting interests

Funding

ORCID iD

Author biographies

References