Preliminary Evidence for an Association between Journal Submission Requirements and Reproducibility of Published Findings: A Pilot Study

Abstract

Systemic efforts have been employed to improve the reproducibility of published findings in psychology. To date, little research has been conducted evaluating how well these efforts work. In an effort to bridge this gap, the current study looked at journal submission requirements intended to encourage authors to engage in best practices for facilitating reproducible science and offers preliminary evidence for their potential efficacy. We calculated reproducibility indices (p-curves) for three randomly selected empirical studies published in each of 23 psychology journals in 2019 and correlated quantitative results from those analyzes with the number of submission requirements for each journal that intended to ensure compliance with best reporting practices. Results indicated a greater number of submission requirements at a given outlet was associated with indices indicating greater likelihood of reproducibility of findings. We frame findings as impetus for future, more extensive, research to identify causal links between submission requirements and reproducibility.