Response to Scientific journals are ‘faith based’: is there a science behind peer review?

Linkov et al. ¹ criticize the lack of scientific rigour in peer review research by writing a non-evidence-based piece themselves. A literature search would have identified an increasing body of scientific research on evaluating peer review and the publication process.^2,3 Since 1989 there have been five international congresses on peer review and biomedical publication, organized by Drummond Rennie at JAMA, where some research in this area is presented. ⁴ Admittedly, it is surprising that progress has been slow in this important research area, but these authors should not be so quick to dismiss this valuable body of work. They misquote the review by Jefferson et al. ⁵ by saying that they only found 19 scientifically sound studies on peer review. That review did focus on 19 papers, but it only included studies looking at the effectiveness of peer review. There are many more scientifically sound studies on the subject of peer review. So the use of scientific method is not ‘almost non-existent’ in the publication process, but I agree more rigorous research is needed. Randomized controlled trials have been done and I hope they will continue. However, peer review research should not be limited to randomized controlled trials. More extensive rigorous qualitative research is needed to unpack some of the more complex issues which are not suitable for study by randomized controlled trials. We also need to agree on the objectives of peer review and develop appropriate validated tools that can measure its effects. The BMJ Publishing Group now has an extensive programme of research into evaluating the publishing process both in-house and in collaboration with external researchers (www.bmjresearch.com). ⁶ More research funds are needed to help support researchers and journals wanting to conduct research with the aim of improving the publishing process.