Comparison of COVID-19 Preprint and Peer-Reviewed Versions of Studies on Therapies for Critically Ill Patients

Abstract

Purpose

Significant increases in the volume of preprint articles due to the COVID-19 pandemic, we examined the reliability of preprint articles compared to their peer-reviewed publications.

Materials and Methods

Preprint articles evaluating experimental studies of select treatment options (anticoagulation, dexamethasone, hydroxychloroquine, remdesivir, and tocilizumab) for COVID-19 in the critically ill, available in a peer-reviewed publication were screened for inclusion within Altmetric (n = 2040). A total of 40 articles met inclusion criteria, with 21 being randomly selected for evaluation. The primary outcome of this evaluation was a change in a study's reported primary outcome or statistical significance between preprint and peer-reviewed articles. Secondary outcomes included changes in primary/secondary outcome effect size and change in study conclusion.

Results

One article (4.8%, 95% CI 0.12%-23.8%) had a change in the primary outcome. Seven articles (33.3%, 95% CI 14.6%-57.0%) had a change in the primary outcome's effect measure. Five studies (23.8%, 95% CI 8.2%-47.2%) had changes in statistical significance of at least one secondary outcome. Four studies (19.0%, 95% CI 5.4%-41.9%) had a change in study conclusion.

Conclusions

In preprint articles of COVID-19 treatments, the provided primary outcome is generally reliable, while interpretation of secondary outcomes should be made with caution, while awaiting completion of the peer-review process.