Demystifying cited open peer review reports: Bringing the behind-the-scenes academic genre to the stage

Abstract

Despite the controversies surrounding the open peer review system, an open peer review report (OPRR) is a potentially valuable academic genre. However, the state of OPRRs being used by researchers and the general public remains unknown. In this study, OPRRs provided by Crossref (n = 548,307) were used as the data sample. Altmetrics of OPRRs, as well as bibliometric indicators of their mothers’ articles, were also included in this study. Descriptive statistics, chi-square tests, Spearman correlation analyses and content analysis were used to examine the data. The major findings were as follows: (1) The cited rate of OPRRs was 1.15%. (2) Referee-reports with authors’ comments had the highest cited ratio and editor-reports had the lowest cited ratio. (3) The difference in the cited ratio between anonymous and signed referee-reports was inconclusive and required further confirmation. (4) 1.41% of the cited OPRRs earned an Altmetric score. (5) There was no significant correlation between cited OPRRs’ citation counts and their Altmetrics. (6) There was a significant positive correlation between the number of citations to OPRRs and the page views, downloads and citations of their mothers’ articles. (7) OPRRs were most often cited in the section Introduction to support the authors’ arguments, followed by the sections Acknowledgements and Methods to acknowledge the reviewers’ contributions and to guide readers on the details of the methodology, respectively. (8) OPRRs on topics that were relevant and controversial to people’s interests were more likely to be disseminated and discussed in social media.