Rigor and Transparency in Two Neurotrauma-Publishing Journals: Editorial Policies Improve Transparent Reporting

Abstract

Preclinical research in traumatic brain injury (TBI) continues to significantly increase knowledge and yield a large number of peer-reviewed studies, but translation of these results to the clinical setting has been minimal. Rigor and transparency factors such as concealment of group allocation (e.g., “blinding”) or ensuring that reagents are identifiable are critical in ensuring that scientific studies are replicable and translatable. Yet, nearly all efforts aimed at measuring these factors have concluded that reporting practices are problematic and incomplete. One way to improve transparency of reporting practices is to require that authors address a set of transparency-related items in some way, such as a checklist or an article section. Recently, Journal of Neurotrauma, a leading publisher of preclinical TBI research, instituted a required rigor-related section, which is explained to authors via a set of transparency, rigor, and reproducibility (TRR) instructions (one example for each article type). These documents include specific transparency sections explaining blinding, power calculations, protocols, code, and data deposition. Experimental Neurology is a journal that is similar in size, impact, and topic, but the journal does not have explicit instructions to authors about transparency items. The purpose of this study was to assess the degree to which transparency reporting items were included in published articles comparing reporting practices in the Journal of Neurotrauma and Experimental Neurology. We used a commercial software, SciScore, which is an AI tool tuned to detect rigor/transparency sentences in published articles and count the number found (roughly dividing by the number expected) to obtain a score. Overall, SciScore found that in six of eight items that were explicitly asked for, such as power calculations, investigator blinding, inclusion criteria, attrition, and data, there were significant differences (more than 10%) compared to Experimental Neurology. However, in Journal of Neurotrauma articles with the extra rigor section, three of four rigor items that were not explicitly asked for in the template rigor documents, such as subject demographics or transparent antibody reporting, were not different from Experimental Neurology. One item, reporting of the sex of subjects, was significantly better in Experimental Neurology. This shows that the Journal of Neurotrauma’s required rigor section is effective in improving reporting, but it would be far better if sex as a biological variable and transparent reporting of reagents (items present on major checklists, including NIH rigor criteria) would be included.