Introduction to the Special Series on Results-Blind Peer Review: An Experimental Analysis on Editorial Recommendations and Manuscript Evaluations

Abstract

Publication bias occurs when studies with statistically significant results and large effects are more likely to be published than similarly rigorous studies with null and mixed findings. Results-blind peer review requires peer reviewers to consider only the “Introduction” and “Method” sections of submitted manuscripts prior to making editorial recommendations. This process ensures recommendations for publication focus on methodological rigor and not the direction, significance, or magnitude of the reported effects. The current investigation experimentally tested whether reviewers’ editorial recommendations and perceptions of manuscript importance, quality, and rigor varied as a function of type of review (i.e., results-blind or results-included) among 44 reviewers. Results indicated reviewer recommendations did not vary as a function of review type. However, reviewers found results-blind manuscripts less rigorous than results-included and reported less confidence in their recommendations on result-blinded manuscripts. Descriptive findings of results-blind reviewing were mixed with some support for the method, but a lack of confidence in its overall effectiveness. We discuss findings in relation to the conceptual benefits of results-blind reviewing and the increased focus on open and transparent science within special education and preview the papers included in the special section.