So…….You Want to be a Peer Reviewer?

Abstract

Peer review is essential to preserving the integrity of the scientific publication process. Peer reviewers must adhere to the norms of the peer review process, including confidentiality, avoiding actual or apparent conflicts of interest, timeliness, constructiveness, and thoroughness. This mini review will discuss some of the different formats in which peer review might occur, as well as advantages and disadvantages of each. The topics then shift to providing advice for prospective reviewers, as well as a suggested format for use in writing a review.

The Need for Peer Reviewers and Some Peer Review Models

Effective peer review is essential to the integrity of scientific publishing, and quality peer reviews not only help authors to write better papers but also help to ensure that scientific misconduct is not perpetuated. Investigators want fair and constructive peer review of their work, so it is imperative that peer reviewers are aware of some of the foundations of effective peer review.

Peer review comes in many “flavors.” ¹ The format that is most familiar to many who publish in the biomedical literature is commonly referred to as “single anonymized” (formerly “single blinded”). In this peer review format, the reviewer knows the identity of the authors and their institutions, but ideally, the author will never learn the identity of the reviewer (note previous comment about the importance of confidentiality!). Advantages of single anonymized peer review include the fact that reviewers can be critical and honest without fearing retaliation. A couple of disadvantages of single anonymized review include the fact that reviewers can be unconstructive, unfair, or impolite in their comments and that there may be risk of bias, in that reviewer may give an extremely critical or unfair review to someone perceived as a rival. Peer review may also be undertaken under the double anonymized format. In this case, the authors and reviewers are deidentified from each other, potentially removing conscious or unconscious bias. A disadvantage of the double anonymized model is that the review process may not actually be anonymized from the perspective of the reviewer. For example, if authors cite some of their previous work and provide a reference, then the reviewer has little trouble in identifying one or more of the authors. Another issue that is shared in common with single anonymized reviews is that the (anonymous) reviewer may be unconstructive in their comments. Finally, let’s briefly outline the open peer review model. In this model, the authors and reviewers are known to each other, and the reviewers’ names may be published along with the manuscript. An obvious advantage of this model is that authors might receive more polite and constructive reviews, as they will eventually know the identity of the reviewer. Disadvantages include the possibility that reviewers may decline review invitations because they don’t want their identity and/or review comments made public. Another possible disadvantage is that reviewers may give an overly favorable review if they don’t want to appear critical and/or do not want to offend the authors. More details and other peer review models are available online. ¹

Ten Pieces of Advice for Prospective Peer Reviewers

1. Confirm that you have the subject matter knowledge to review the paper. Most journals include the abstract of the paper in their invitation letter to help a prospective reviewer make this decision.

2. Do some soul searching and ask yourself whether you have a conflict of interest (COI) in reviewing the paper. This could include being asked to review the work of a collaborator, a peer, a competitor, or having some other vested interest in the publication (or rejection) of the paper. If you are in doubt as to whether you have a COI, either consult with the editor or decline the peer review invitation.

3. If your answers to questions 1 and 2 above were “yes” and “no,” respectively, then determine whether you have the time to review the manuscript in the expected time frame. Do not accept the invitation if you will not be able to do a thorough and fair review in a timely fashion. Most reviews typically take about 4 hours, but this can vary considerably, depending on the complexity of the manuscript.

4. Be constructive in your comments. Do not demean the authors if the work contains numerous problems with experimental design, data interpretation, etc. Address your comments such that the flaws in the manuscript are the topic of the review, not the perceived ineptness of the authors. Read your review from the perspective of being on the receiving end of the review, and gauge whether you would be offended by the language in the review.

5. Make your comments “actionable.” To state in another way, give the authors some direction to help them address issues that you might uncover in their work. Statements such as “The manuscript is too long.” do not really help an author. Instead, perhaps state, “The authors have included much of the same material in the Introduction and the Discussion sections of the manuscript. Removal of overlapping material is suggested.”

6. If you choose to include your comments for the authors as track changes or comments directly on the manuscript, PLEASE DEIDENTIFY YOURSELF. If a reviewed document is submitted with the reviewer’s name attached to the comments or suggested changes, most editors do not have the time or the software to deidentify that peer reviewer. It is suggested that reviewers show their comments as coming from “Anonymous” or “Reviewer,” instead of having their name associated with the comments.

7. If you, as a peer reviewer, suspect scientific misconduct (defined as the falsification or fabrication of data, or plagiarism), communicate these concerns IMMEDIATELY to the editor. Vigilance on the part of peer reviewers to avoid duplicate publications, “paper mill” publications, etc., is essential to the integrity of biomedical literature.^2,3

8. Peer reviewers ARE NOT expected to rewrite papers to improve grammar, spelling, and so on. If there are deficiencies in these areas, please point them out as confidential comments to the editor.

9. It is practically unheard of for a manuscript to be accepted for publication without some revisions. Think about the peer review process as an iterative process, wherein authors receive advice, attempt to incorporate that advice, and resubmit their work for re-evaluation (Figure 1). If authors cannot (or will not) satisfactorily address your comments, explain this confidentially to the editor, and communicate to the authors in a constructive manner that their responses did not adequately address the reviewer comments. Ultimately, it is the editor’s responsibility to determine the adequacy of the authors’ responses to peer review comments and to evaluate the overall suitability of the manuscript for publication.

10. Spare the authors your own personal experience. It is seldom helpful to authors to learn of how a reviewer handled a certain situation/criticism in the past.

Figure 1.

Flow chart for manuscript submission and navigation through the peer review and publication processes. Courtesy of Elisa Turner, Deputy Director, American College of Toxicology. Used with permission.

Writing Your Review

While different journals may have specified formats for manuscript reviews, the outline presented below may serve as a good starting point.

• Begin by reiterating the title of the paper, and state in your own words what you believe to be the major findings of the paper.

• Determine the appropriateness of the background information provided in the introduction. A comprehensive review of the literature is generally not needed; instead, evaluate whether the background materials provide justification for the studies presented in the manuscript.

• Assess the methods section. Are the methods appropriate to answer the specified questions, and are they presented in enough detail that the experiments could be independently replicated? Assure that appropriate statistical analyses have been conducted.

• Evaluate the results section. Are results sufficiently presented? If tables and figures are used to present data, ensure that the data are not presented in both formats. Are tables and figures labeled in enough detail for interpretation without referring to the text of the manuscript (i.e., can they “stand alone”)? Are photomicrographs crisp, in focus, color balanced, and clearly labeled with arrows/arrowheads/lettering to show important features?

• The discussion session should put the current findings in context with what is already known on the topic. The discussion should not merely re-present the results; rather, the results section should explain how the new findings advance the field. Do you share the authors’ interpretations in the context of prior findings?

• Your review should NOT contain your recommendation as to the suitability of the manuscript for publication. Instead, this recommendation should be communicated only to the editor.

You’ve completed your evaluation of the manuscript. Now what? Reviewers may be asked to suggest the following actions:

• Minor revisions: Some rewriting and/or data reanalysis is needed, but no further experiments are needed.

• Major revisions: Additional experiments may be needed; major rewriting is needed to improve English presentation or remove or add literature citations; perhaps reanalysis or reinterpretation of data is required.

• Reject: Manuscript is so flawed that even substantial rewriting or additional experiments cannot rescue the studies presented.

In the case of the decisions of “Minor Revision” and “Major Revision,” the editor will very likely want to return the revised manuscript to you, as the initial peer reviewer, as you will best be able to gauge whether the authors have made changes consistent with your suggestions to improve the manuscript. If you are given the opportunity to check a box (Yes/No) as to whether you would be willing to review a revised manuscript, please say “Yes.” Please. Editors are greatly inconvenienced when a reviewer does not agree to review a revised manuscript. If you recommend the manuscript to be rejected, it is unlikely that the editor would ask you to review the revised manuscript. However, if you suggest minor or major revisions, your critique of the changes made by the authors is highly valued, and it saves the editor the time and effort of finding someone else to review a revised manuscript (and critique how the authors responded to YOUR comments).

Final Comment

Editors are immensely grateful for the time, efforts, and expertise of peer reviewers. If you enjoy the subject matter of any particular journal, an editor will likely welcome you as a volunteer reviewer, and potentially eventually as an Editorial Board member. Reach out to be part of the peer review process, and thank you to all of the peer reviewers out there!