‘Just think positive’: We can all work to address the publication bias issue

To the Editor

The recent editorial by Porter et al. (2017) raises important questions for psychiatric research. As the authors highlighted, distortions in the published literature reflect a waste of resources and increase the risk of clinical harm.

The preponderance of positive findings is not simply a consequence of editorial favoritism for positive and ‘newsworthy’ results. Publication bias also exists at the level of individual researchers and the decisions we make about what we submit for publication and how we conduct and report our studies. These individual biases contribute to the publication landscape in a number of different disciplines.

In psychology, much of the discussion of these issues has been focused at the level of the individual researcher. When conducting a study, a researcher must make a series of seemingly innocuous decisions about when to stop recruitment, which observations to exclude, which moderating variables to consider and the most appropriate way of analyzing the results. These ‘researcher degrees of freedom’ (Simmons et al., 2011) are thought to have a cumulative effect in tilting the likelihood of study findings toward the positive.

Pre-registration of research studies, as advocated by Porter et al., may help to restrain researchers from indulging in many of these potentially distorting ‘freedoms’. However, pre-registration may not be sufficient in its own right (Goldacre, 2016). A further step would be to encourage researchers to submit their proposed research methodologies to peer review before commencing a study, as already occurs when researchers publish study protocols for clinical trials. This provides peer-reviewed legitimacy to one’s research, and protects against negative findings being dismissed on purely methodological grounds. Researchers would then feel emboldened to submit their methodologically robust research for publication even when the findings are negative. When combined with other initiatives, such as efforts to ensure the open accessibility of data, these approaches hold promise for improving the integrity of the published literature.

There are of course increased administrative demands on researchers in attending to all these recommendations. On the other hand, there are also important potential benefits. Peer review of study protocols offers researchers increased recognition for their work, such that careers may become defined as much by the development and application of rigorous methodologies as by the chance significance or newsworthiness of one’s results.