Analytic Review as a Solution to the Misreporting of Statistical Results in Psychological Science

Abstract

In this article, we propose analytic review (AR) as a solution to the problem of misreporting statistical results in psychological science. AR requires authors submitting manuscripts for publication to also submit the data file and syntax used during analyses. Regular reviewers or statistical experts then review reported analyses in order to verify that the analyses reported were actually conducted and that the statistical values are accurately reported. We begin by describing the problem of misreporting in psychology and introduce the basic AR process. We then highlight both primary and secondary benefits of adopting AR and describe different permutations of the AR system, each of which has its own strengths and limitations. We conclude by attempting to dispel three anticipated concerns about AR: that it will increase the workload placed on scholars, that it will infringe on the traditional peer-review process, and that it will hurt the image of the discipline of psychology. Although implementing AR will add one more step to the bureaucratic publication process, we believe it can be implemented in an efficient manner that would greatly assist in decreasing the frequency and impact of misreporting while also providing secondary benefits in other domains of scientific integrity.