Finding reviewers: A crisis for journals and their authors

In the ANZJP, and other similar scientific journals of merit, submitted manuscripts are routinely sent to two or more respected individuals for their anonymous assessment. Being invited in this manner to act as a peer reviewer for the journal is a badge of honour. It means that you have been selected by ‘those in the know’ to offer your opinion because they recognise you as an expert. In other words, your view matters and carries weight. Although the process is anonymous, it is normative for reviewers to include in their track records and CVs their role as a reviewer for particular journals, with kudos associated with being invited to review papers especially for the most prestigious journals. All in all, being asked to review papers means that one is held in high regard by one’s peers. However, in recent years in particular, it has become increasingly difficult to persuade qualified people to undertake reviews.

In the ANZJP, reviewers for submitted manuscripts are sought from Australia, New Zealand and overseas. These reviewers are chosen by the Editors because they are considered experts in the subject matter or, alternatively, because they work in an overlapping field, have no special investment in the topic but have general expertise, good sense and a critical eye. Authors can suggest who might be approached or avoided – however, these recommendations are not binding and editors will often make selections of their own. In some journals, reviewers are blind to the identity of the authors. However, in many other journals, including our own, the reviewer is shown the identity of the authors. Realistically, reviewers can never be wholly free of bias in either direction, but there is an expectation that they conduct themselves ethically. For example, upon receipt of an invitation to assess a paper, the reviewer must alert the journal if they feel they have a conflict of interest and cannot conduct the review because of this concern. Often there is no reasonable way for the editor to know whether, for instance, the reviewer and author have a joint research grant, are personally related or were best friends at school. Furthermore, the reviewer must not contact the author regarding their manuscript without the Editor’s permission, nor should they inform any third party about the content of the manuscript and its authorship, or subsequently reveal that they were asked to assess it. This is accepted etiquette.

To carefully read and critique a manuscript can take anywhere from 20 to 30 minutes to 2–3 hours, depending on the kind of article that is being assessed (e.g. letter, commentary, original research, systematic review). For a research article, a good reviewer will comment on the importance of the research question being addressed, the originality of the study, the adequacy of the methodology including statistical analysis, whether the author’s interpretation is justified by the data, clarity of writing and any relevant ethical issues (Garmel, 2010). In the ANZJP, there is an expectation that the report be sent to the Editor within 4 weeks. What the reviewer writes is in two parts: text that is forwarded verbatim to the author and text ‘For the attention of the Editors only’. The process also involves dialogue between the reviewer and the Editorial team and then, subsequently, via the Journal, communication with the authors. This to-and-fro has generally been a cordial exchange in which everyone understands that the overarching, ultimate aim is to advance science and knowledge and to produce the best possible publication. Authors have commonly thanked Editors for timely feedback and specifically thanked reviewers for constructive comments that improved the quality and clarity of their manuscript. Reviewers, in turn, appreciated the fact that they had been selected by the Editors and would often appreciate the opportunity to review new research and emerging ideas. Such exchanges allowed reviewers to feel that they were part of the scientific process – especially that their suggestions were acted upon and improved the final manuscript. Editors were grateful to all parties and were particularly appreciative when the matter concluded amicably, resulting in a good quality publication for the journal. Generally, the process was professional and congenial and seen as educational for all concerned – indeed it was an essential part of being an academic, and an important aspect of one’s roles and responsibilities that was commonly noted in reviews of one’s performance. Occasionally, authors interpret the reviews as being unduly critical and a robust exchange, via the review process, may then take place. These conversations may still be productive for all involved. Peer review helped communicate ideas among different groups and meant that the field developed and progressed. It was a means of exchanging emergent knowledge and drawing on the insights of one’s peers and colleagues.

Not long ago, the time it took to undergo peer review was far more sedate. This aided the transmission of thoughtful exchange. Opinions were generally more measured and more considered because communication was reliant on conventional mail and comments were either handwritten or typed – another personal aspect that reminded everyone that people were at the centre of the process. Indeed, the whole machinery of publication was labour intensive and hence, every step took time. At the same time, a clear understanding of the mechanisms of publication and the contract between Editors, authors and reviewers meant that everyone took their role seriously. If reviewers, for example, were unable to review or would not be able to do so in a timely manner, then they would point this out at the outset and enter into an agreement only if they could guarantee the provision of a useful opinion within a reasonable period of time.

Critically, in the past across the whole review and publication process, no money changed hands. Meanwhile, at an international level, the scientific quality of academic work is in aggregate raised globally, the Journal prospers, its Impact Factor rises and the publishing house accepts substantial profits. In the case of the ANZJP, our College acquires added prestige in our two countries and, importantly, internationally. For the Journal and the Editors, the review process was a key and attractive means of promoting the Journal. A fair, open and constructive process that was conducted in a timely fashion translated to good quality submissions. Reviewers themselves enjoyed the process, felt engaged and achieved a sense of accomplishment. These were their rewards.

But, alas, those halcyon days have passed. Those who are experts in a field now find themselves inundated with invitations from a vastly increased number of journals. Across the scientific world, Journal Editors are finding their invitations to reviewers are increasingly ignored or declined (Bakker and Traniello, 2019). The most commonly noted reason is work pressure and reviewer fatigue (Tite and Schroter, 2007). The most disappointing scenario is when a reviewer agrees to undertake the task but never completes it, while the journal and author remain waiting for a decision. For the present writers, it is not uncommon for invited reviewers to immediately decline assessing a paper, probably in many cases without even reading the abstract or only reviewing it cursorily. It seems this pattern is now happening across all branches of medicine and other areas of science (Willis, 2016). A likely contributory factor is the rise of Open Access publication. This has broken the unwritten agreement that you publish for free and you review for free. The basic principles of behavioural economics predict that people will push back against this disruption and consciously or subconsciously begin to refuse to review for free while having to pay to publish.

This difficulty is not trivial because peer review is central to scientific progress (Laine and Mulrow, 2003). To have no independent evaluation of manuscripts is unthinkable. We therefore need to find ways to preserve high-quality peer review. A good start would be to understand more clearly why people take on doing reviews in the first place. The reasons are usually reciprocal and altruistic. Some may be flattered to be asked their opinion; they may have a long-standing interest in, or attachment to, the topic; they may acquire new information or a new view of the field from reading the emerging, unpublished work of others; they may have some sense of obligation to the Editors or they may out of a sense of duty simply, and quite correctly, see themselves as contributing meaningfully to the advancement of science. They may recognise the mutuality of the process as they expect their submissions to be promptly reviewed and so want to give something back to the Journal. This is to be encouraged and should be recognised as a worthwhile exercise. One means by which this is occurring is through the commercial website Publons, which provides a free service to academics to have their work as a reviewer captured and publicly acknowledged. It is also useful for their career progression and allows them to demonstrate that they are known for their expertise in a certain field. It certainly adds to their track record for competitive grant applications. However, whatever the many motives, participation by reviewers seems to be declining. It is possible that the explosion in the number of papers being published has meant reviewers can manage only so many. If only Editors and the Associate Editors themselves critically examined all submissions, with no external referee, the scientific standard of a Journal would collapse and its reputation would plummet. So, what is to be done?

At present, some publishers, in our case SAGE, offer free access to all its Journals for 60 days and a 25% book discount on all SAGE books ordered online. But few reviewers take this up. Everyone knows that money is one of the most powerful motivators. It is worrying that payment of fees for reviewing by a publishing house has already begun. The College itself could offer more generous CME points and publicise this option. Many of the reviewers we invite are overseas experts or science graduates. A fanciful possibility is that the Australian and New Zealand Governments might offer a modest tax concession for each review, as is done for donations to a charity. Perhaps, some sense of mutual obligation could be considered; authors of publications in the ANZJP could be expected to undertake reviews in return for peer review and publication of their papers.

However, we think such inducements have little chance of being realised. Instead, in this Editorial, we suggest that altruistic behaviour nevertheless persists and a vigorous appeal to the entire scientific community has some chance of being heeded. In the present context, every time a review is undertaken, it is in aggregate making some contribution to science. Doing reviews is a professional activity deserving allocation of appropriate time in our working lives. Most importantly, our College, Universities and Health Authorities need to accord it greater recognition. We cannot allow this service to our profession to become extinct.