Abstract
In response to the increasing difficulty of obtaining high quality peer reviews, our invited paper describes the concept of review avoidance and why this phenomenon occurs. In reaffirming the professional responsibilities and potential benefits of reviewing, we also emphasize the interdependent nature of the ideal peer review process. We suggest that the review process is a three-way street where the respective roles and responsibilities of authors, editors and editorial teams, and reviewers are inextricably linked. We present thematic illustrations of undesirable reviewer comments, and a brief synthesis of broad themes in the literature on high-quality reviewing. The synthesis is complemented by a master reviewer’s fine-grained perspective on crafting high quality reviews. A final Appendix presents additional sources that may be informative for prospective reviewers, submitting authors, and those mentors and colleagues who may wish to provide guidance and training to them.
Introduction and An Ideal Peer Review Process
When you receive an invitation from a journal to perform a peer review, how do you react?
Is your first thought, “Great, here is another opportunity to read an interesting and relevant article related to my areas of content and method expertise that I can learn from? or,
Do you think to yourself, “What a great opportunity to help a colleague!”; or, Do you cringe and say, “Oh no!!! Not another review request!!! When will I ever find the time to complete this review?” or,
Do you simply ignore the invitation as if it never entered into your email inbox? And, then ignore all of the subsequent reminders to kindly accept, decline, or indicate that you are unavailable?
Unfortunately, recent editorials suggest that review avoidance is escalating. Journal editors have acknowledged that “we appear to be undergoing a ‘crisis’ related to an increasing difficulty in securing peer reviewers to guide us in our decision-making about submissions” (Kerig, 2021, p. 5). Hazen et al. (2016) highlight unwillingness to review, reviewers performing non-constructive reviews, and authors either submitting poor-quality manuscripts or slicing data – and story – too thinly as causes for a “broken review process” (p. 623). More specifically, ignoring review invitations, declining review invitations, and perceiving review invitations as invasions of one’s professional and personal time tend to be the more commonplace reactions from potential reviewers. And many of us would likely concede that, at some time, we have been guilty of such reactions and may contribute to this broken review process.
Many reasons have been advanced for why reviewers decline to participate in the peer reviewing process. For example, Kerig (2021) identified the following four reasons for review avoidance: thankless task, absence of fairness and consistency, lack of training, and shortage of diverse voices. Kerig argues that peer reviewing is often perceived as a thankless task because most journals do not have budgets for paying reviewers: [T]here is no compensation offered that comes close to matching the time and intellectual effort involved in crafting a thoughtful review. Revised-and-resubmitted articles require even more work from reviewers, as the best reviews will entail reading not only the revised manuscript but calibrating it against the previous reversion as well as the other reviewers’ comments and the authors’ responses to each of those reviews (p. 5).
Kerig also suggests that reviewers may question the “fairness, validity, and efficacy of the peer review process” (p. 6). Thus, poor articles are often seen to be published, while unorthodox, innovative, ground-breaking, and game-changing articles sometimes have difficulty getting published or are rejected. Rather surprisingly, given the essential role that peer reviewing plays in academe, Kerig also highlights lack of training in the peer review process as a reason for review avoidance. With the significant growth of global submissions, it is also possible that many of these submitting authors may not have been afforded opportunities to be schooled to perform proficient reviews and may therefore be uncomfortable when asked to do so. Lastly, Kerig also notes “we do not all see our peers in the peer review process” (p. 6). Thus, scholars who are from “diverse geographic locations and those from historically minoritized groups based upon race and ethnicity, gender, gender identity, or ability, as well as representation across stages in the profession” (p. 6) may not see themselves and their diversity reflected in visible editorial appointments or in scholarly publishing.
Given the apparent escalation in review avoidance, editors frequently stress the need to reinforce and embrace the notion that performing high-quality, thoughtful, and constructive reviews is a professional obligation and a responsibility for scholars in the knowledge creation and dissemination enterprise (e.g., Ellinger et al., 2013; Hazen et al., 2016; Treviño, 2008; Wang, 2018). To reinforce these notions, we may also need to revisit and reacquaint ourselves with the numerous benefits that can be realized from constructively participating in the peer review process. These benefits include taking pride in knowing that we have been generous in assisting our peers in evolving and progressing their work and in helping editors to have “checks and balances” in place to reduce the potential for bias and inconsistency. Moreover, becoming and being a good reviewer often leads to opportunities for other leadership roles in the scholarly publishing endeavor, such as editorial board appointments and roles as assistant, associate, or editors-in-chief. Finally, performing reviews can help us become better authors and reviewers. The fond hope is that, as we are exposed to new ideas, current, interesting, and relevant literature, robust theories and models, new and advanced data collection and analyses techniques, we become better writers, thinkers, conceptualizers, and communicators.
The purpose of this invited paper is to advance a perspective that performing high-quality reviews is a three-way street that reflects interdependence among submitting authors, editors and editorial teams, and peer reviewers. As authors, we submit our research expecting that it will undergo a fair, timely, and efficient review process with thoughtful, and constructive feedback that helps us improve our manuscripts for publication. Accordingly, in order to keep the peer review process functioning, we must “pay it forward” by serving as high-quality, fair, and timely reviewers ourselves. However, as former journal editors and award-winning reviewers, we maintain that reviewers are not the only parties responsible for ensuring a positive, efficient, and effective review process. We subscribe to Kerig’s (2021) perspective that “we represent an interdependent community of scholars who are reliant on one another for the peer review process to succeed” (p. 8). To this end, we believe that authors and editors and their editorial teams, as well as reviewers, all play crucial, interdependent roles in the peer review process.
In the following sections, we begin by describing the stages in an ideal peer review process. Next, we highlight the interdependent responsibilities of authors, editors and their editorial teams, as well as reviewers. We then share some examples of poor reviewer comments that serve as benchmarks for the types of comments that should be avoided when performing reviews. The next sections identify general themes from the literature on reviewing that are complemented by the presentation of a master reviewer’s fine-grained perspective on crafting high quality reviews.
An Ideal Peer Review Process
Most journals provide prescriptions for performing proficient reviews of submitted manuscripts. However, performing reviews is only one aspect of the review process. An ideal peer review process that includes five steps was articulated by Hazen et al. (2016): Step 1: authors submit their best research to a journal; Step 2: editors promptly assess fit and initial quality of submissions and send the best work to reviewers; Step 3: since reviewing is an implicit duty of the profession, reviewers accept the invitation without delay and then follow through, providing a timely, thorough, and constructive review; Step 4: editors then compile reviews, write up their own assessment and summary, make a decision, and inform the author of the decisions, providing constructive feedback to help improve the research; and Step 5: authors respond thoroughly and expeditiously to review requests and submit a revised manuscript which moves rapidly through the review process (p. 623).
Hazen et al. contend that, if authors and reviewers respectively submitted great research and performed great reviews, “the review process should take between 30 and 60 days. And the endless cycle of monster revisions that is becoming the norm would disappear” (p. 623). Building on Hazen et al.'s call for action, we strongly suggest that authors, editors, and their editorial teams, as well as reviewers must all fulfill their respective responsibilities on each interdependent element of the three-way street that is needed to establish an ideal peer review process.
The Three-Way Review Process: Responsibilities of Submitting Authors, Responsibilities of Editors, Responsibilities of Reviewers
Responsibilities of Submitting Authors
The first interdependent element of the three-way street that is needed to establish an ideal peer review process is that authors ensure their manuscripts are appropriate for the targeted journal. The peer review process begins when a prospective author or multi-author research team identifies a target journal and then submits their manuscript to the journal for consideration. When the manuscript is received in the online submission system, it may be desk rejected for lack of fit with the aims and scope of the journal, a fatal flaw, or lack of contribution, among other reasons. A desk rejection is not unusual and a large percentage of manuscripts receive this fate. Thus, many leading journals advise prospective authors that less than ten percent of submitted manuscripts are entered into the review process. The manuscript may also be un-submitted and returned to the author(s) because it did not conform to the manuscript submission guidelines; it may be too lengthy and exceed maximum stated word and page counts for the journal. Other reasons for un-submitting manuscripts are excessive grammatical, spelling and linguistic errors, or too high a match with previously published content if plagiarism checking software is utilized in the incoming check process. Alternatively, the manuscript may be placed under review and assigned a manuscript identification number.
As can be seen, there are numerous reasons why a manuscript may never be placed under review or why manuscripts must be amended prior to further consideration before being sent out for peer review. To alleviate some of these issues, we maintain that authors have several responsibilities they should fulfill, even though we fully appreciate the pressure to “publish or perish.” One of the first and most important responsibilities is for authors to ensure that their manuscript is suitable for the targeted journal. Most, if not all, journals and their publishers have online websites that provide detailed insights about their missions, their aims and scopes, and their formatting and submission guidelines, among many other resources for authors.
Many journals have also published editorials about how authors can better navigate the publication process at their journals or increase their chances of publication for a specific journal. For example, in the HRD field, Lunn (2014) developed a guideline to help authors better strengthen their submissions for Human Resource Development Quarterly (HRDQ). Similarly, Ghosh (2019) and Li (2019) have written editorials to help authors better position their research to meet the aims and scope of Human Resource Development International (HRDI), and Wang’s (2019a, 2019b) editorials in Human Resource Development Review (HRDR) have focused on becoming a responsible writer and considering the specific issues that can enhance authors’ articulation of the significance of their research. The editors of Academy of Management Learning and Education (AMLE) (2019) also published an editorial on “Being an AMLE Reviewer” and also compiled “AMLE Reviewer Resource Library: A Collection of Recommended Pieces on Developmental Reviewing”. Most recently, Cho and Ghosh (2022) described a jointly-sponsored reviewer development workshop by HRDR and HRDI intended to build and improve upon reviewer skills. Further, an insightful recent editorial for increasing prospective authors’ chances of publication in the Journal of Managerial Psychology by Meuser et al. (2020) provides a thorough appendix that contains references to assist authors in their publishing endeavors based upon their articulation of issues that have resulted in manuscripts being rejected at the journal. In addition to the resources noted in the Meuser et al. (2020) editorial, we have compiled Appendix A that consolidates additional HRD-specific and discipline-related resources that have been developed to guide authors as they strive to create high-quality submissions and reviewers as they assess submissions during the review process.
Ensuring that their manuscripts are appropriate for the targeted journal is achieved by author(s) taking time to familiarize themselves with the journal, its past issues, and its most recent submissions on Early Cite. Doing so provides a better understanding of what has been published and what has not been published, the types of articles that are published, the designs of studies, the data collection and analytical approaches that have and are being used, and the typically incorporated or preferred structures for the organization and flow of manuscripts. Also recommended is becoming aware and knowledgeable about how the author(s)’ manuscript has the potential to foster additional or new dialogue on a research topic so that its contribution to the journal can be positioned and made apparent.
Authors may be tempted to submit poorly-written, sloppily-edited, under-developed and/or ill-prepared submissions – perhaps in anticipation that reviewers will fix the problems for them? However, the role of a reviewer is not to be a copy editor, formatter, or translator! Moreover, compelling reviewers to fulfill these roles by submitting papers that require such attention will only exacerbate review avoidance! Therefore, ensuring that any manuscript submitted to a journal represents the author(s)’ best effort is a responsibility that must be fulfilled in order for reviewers to be able to expend their energies appropriately by providing constructive feedback. Accordingly, consistent with the prescriptions of Hazen et al. (2016), Meuser et al. (2020) and others, author(s) are strongly encouraged to seek feedback from peers and colleagues prior to submission and to employ competent copy and style editors to polish their submissions so that content and intent are clearly communicated.
Responsibilities of Editors
The next interdependent element of the three-way street that is needed to establish an ideal peer review process is for editors to ensure that manuscripts are suitable for the targeted prospective reviewers. Of course, doing so begins by ensuring that poorly-written, sloppily-edited, under-developed and/or ill-prepared submissions that may show promise are promptly returned to their authors with instructions about what needs to be fixed before the manuscript can be sent out for review. Making sure that editorial teams do not send such papers out to reviewers is critical, given the consistent laments at many editorial board meetings about the increasing difficulty associated with identifying reviewers who are willing to perform reviews and then obtaining good reviews from them. Another essential editorial responsibility that, in our collective experience, is all too frequently overlooked relates to the selection of reviewers whose subject matter expertise and methodological skills appropriately align with the submission.
The notion that journals’ reviewer selection processes often leave a lot to be desired is not new. Past studies have suggested that editors have limited knowledge about reviewers’ areas of expertise. For example, a study by Bedeian (2003) reported that more than half of the reviewers surveyed indicated they had been asked to review a manuscript that they did not feel competent to review. Somewhat alarmingly, more than one-third of the respondents also reported that they had still accepted the invitation and had submitted a review report. However, an initial mismatch between the reviewers’ expertise and the expertise required for the submitted manuscript invariably influences the helpfulness of any review that is submitted, and may indeed negatively impact the development of a submission. Even worse, initial misalignment may result in a lengthy revision process that might have been avoided if appropriate reviewer selection and alignment had initially been achieved.
We therefore suggest that ensuring a good match between manuscripts and prospective reviewers is arguably the most important task for editors. Many editors exercise the diligence required to match expertise by personally corresponding with potential reviewers about their need for their specific expertise or by reaching out to convey their rationale for why they are asking that reviewer to perform a particular review for their journal. Conversely, others seem to rely on some sort of quota system whereby reviewers are selected on the basis that they should perform a specific number of reviews or because the reviewer’s name comes up in a particular rotation of reviewer names. Another ineffective tactic is to delegate reviewer selection responsibilities to administrative staff who may not know the reviewers’ skills and expertise well enough to ensure that an appropriate match is made and may not even understand the importance of retaining the same reviewers throughout the manuscript’s journey in the review and revision process.
In our experience, an editor’s up-front efforts to appropriately align reviewers with manuscripts tend to pay off by greatly reducing the subsequent exertions inherent in trying to make decisions based on lackluster reviews submitted by reviewers who have not been effectively matched with the manuscripts that they have been invited to review. Additionally, in such situations, reviewers face the difficult choice of whether to decline the invitation and risk falling out of favor with the journal or to go ahead and submit a review on a topic that may be outside their comfort zones. This is often a very hard decision for prospective reviewers that can be avoided by editorial diligence in the reviewer selection process.
A journal editor’s responsibilities also include sharing blinded decision letters and consolidated feedback as well as other reviewers’ assessments with their reviewers. This generally accepted practice is necessary in order for reviewers to feel that they are involved and have contributed to the decision process by learning the fate of the manuscripts they have reviewed. Doing so also facilitates learning by allowing reviewers to compare their feedback with that provided by the other reviewers on the manuscript. Finally, rather than relying on a generic system-generated acknowledgement or on offers of free access or discounts on the publisher’s products, a sincere personal thank-you from the editor-in-chief may go a long way towards motivating reviewers to accept the next request from that journal. Reviewer awards and recognition listings are becoming commonplace. Yet, when reviewers are publicly acknowledged for their excellence, other journals often leverage such listings to expand their own reviewer databases! Thus, publicly recognizing reviewer excellence represents something of a double-edged sword, since it may serve to intensify review avoidance by providing the wherewithal for journal editors from other fields to extend additional invitations to acclaimed reviewers in other already overburdened disciplines.
Responsibilities of Reviewers, Poor Reviewer Comments, and Themes Associated with Performing High Quality Reviews
The final interdependent element of the three-way street that is needed to establish an ideal peer review process is that, when presented with an appropriate manuscript, reviewers perform great reviews. Given the proliferating trend of review avoidance, many of us may need to consider being more responsive by immediately accepting, declining or indicating our unavailability so that our delays do not impact the overall review process. Submitting manuscripts for review to selected journals in our respective fields also entails an obligation to support these journals by proactively performing reviews for them. From a numerical perspective, if an author submits between three and five manuscripts per year to journals in their field and two or three reviewers are needed for each submission, then between 6 and 15 reviewers will be asked to review that author’s submissions. In an ideal review process, that author should therefore also be willing to perform between 6 and 15 reviews per year for those journals.
Ideally, when we patronize specific journals, we should accept manuscripts when we are invited to review an appropriate submission rather than delay our response. However, when there is a clear mismatch, we should contact the editor to see if reviewers with more suitable subject matter or methodological expertise can be identified. And we should also clearly communicate deficiencies in our expertise as it relates to the manuscript in the review invitation so that blind spots can be taken into consideration by the editor in the decision-making process. We should also be timely in completing our reviews. Typically, reviewers are asked to complete reviews within a specific time frame. Should the designated timeline for completing the review not be feasible, we recommend that reviewers reach out to the editors to request an extension. We have all likely sought extensions, and this does not seem unreasonable as the volume of review requests increases. Yet we must also be mindful that extensions lengthen the time to render a decision which can be detrimental to authors who may need decisions for tenure and promotion purposes.
In addition to fulfilling obligations to review, the nature and spirit of the reviews that are sent back to authors is of paramount importance. Accordingly, the next section presents examples of the types of comments gleaned from poor reviews that are not helpful to authors or editors.
Poor Reviewer Comments
Thematic Examples of Poor Reviewer Comments.
Note. The contents of this table are based on a presentation made by Alexander E. Ellinger and Patrik Jonsson: “Some Prescriptions for Navigating the Publication Process in General and IJPDLM in Particular”, 2015 Nordlog Doctoral Symposium, Molde, Norway. Some of the comments in this table also appear in Hazen et al. (2016).
The above comments are rather disrespectful and do not add value for authors. Therefore, in an ideal world, diligent editors should endeavor to humanely rescind them before sending the reviews back. For the purposes of this submission, we would urge reviewers to avoid generating such comments in their reviews by considering the fairness, tone and utility of all feedback to authors. The next section offers a brief synthesis of broad themes stressed in the literature on high-quality reviewing.
Themes Associated with Performing High Quality Reviews
As noted earlier, numerous editors have shared their expectations for reviewers through editorials in their journals; in other cases, experienced reviewers have written about successful approaches for preparing reviews. Analyzing these for common themes provides insight into the broader question of what constitutes a high-quality review.
To begin, many recommend reading the entire manuscript through before starting the review process (Bagchi et al., 2017; Ballinger & Johnson, 2015; Harrison, 2002); multiple readings will likely be needed for a thorough review (Wang, 2018). Reviewers may also need to consult the supporting literature, especially if there is a particular prior study that is a key reference (Bagchi et al., 2017; Ballinger & Johnson, 2015). Harrison (2002) reminds us that authors have the right to expect reviewers to do a thorough, careful job with their review.
There is a strong consensus among editors that reviewers should present their findings in a respectful, collegial tone (Caligiuri & Thomas, 2013; Ellinger et al., 2013; Harrison, 2002; Ragins, 2015; Wang, 2018). The reviewer should remember that it is a “peer” review (Ragins, 2015). Wang (2018) reminds the reviewer to critique the content, not the person.
In a similar vein, reviewers should not just identify strengths and weaknesses of the manuscript, but should also provide clear, specific recommendations on how to address problem areas (Bagchi et al., 2017; Bergh, 2002; Brown, 2012; Caligiuri & Thomas, 2013; Ellinger et al., 2013; Lepak, 2009; Ragins, 2015). Such recommendations should be included even when the reviewer ultimately recommends rejecting the manuscript (Brown, 2012; Caligiuri & Thomas, 2013). Clear, specific recommendations will assist the author(s) in further developing the manuscript and are of greater use to the author(s) than simple evaluative statements (Lepak, 2009). The role of the reviewer is not just to help the editor make a decision, but also to help the author(s) improve (Bagchi et al., 2017).
Reviewer comments should be presented in a clear, organized manner that prioritizes the most important areas of concern (Ballinger & Johnson, 2015; Caligiuri & Thomas, 2013; Harrison, 2002; Lepak, 2009; Wang, 2018). A laundry list of points is not as helpful to the author(s) as discussion of clearly-identified major and minor areas of concern (Ballinger & Johnson, 2015; Lepak, 2009). There should be enough detail to be useful but not to the point of overwhelming the author(s) (Brown, 2012); a balance should be struck between breadth and depth (Bergh, 2002).
Recommended Sources on Preparing a High-Quality Review.
Reflections on Reviewing and Reviewer Roles
Next, we reinforce the broad themes synthesized in the previous section by presenting a master reviewer’s fine-grained perspective on crafting high quality reviews. These insights have been well-received in multiple seminars designed to guide and inspire prospective reviewers of journal manuscripts.
Crafting a Good Review Report – The Essential Ingredients
A fundamental premise of the peer review process is to develop a fruitful dialogue between reviewers and authors. As the entire reviewer-author discourse is conducted through the written review and reviewer response reports, the reviewer should be cognizant of how influential their comments may be to the authors of the manuscript. Authors may have difficulties accepting change suggestions, and we as reviewers may have misunderstood something when reading the manuscript. A review report should include the following parts: 1. Brief summary: The brief summary at the beginning of the report is normally a paragraph describing your perspective when reading and reviewing (you may have read the paper with a specific methodology or perspective on a particular topic), and your overall understanding of the manuscript and its contribution. This summary should identify the strength of the manuscript (even if you believe it will be rejected by the editor – all manuscripts have strengths). 2. Specific comments/major concerns: The number of specific comments in the review report should not be too many. Focus should be on the most fundamental issues. Comments could be clustered and synthesized into larger areas. In doing so it is normally possible to end up with no more than five (and often fewer) areas of specific comments. This list of specific comments should identify fatal flaws, as well as make suggestions for improvement. 3. Minor problems (which could be few or many) can be listed after the specific concerns.
Positively versus Negatively Influencing Throughput Time
Reviewers should be aware of the influence that their actions have on the length of time that a manuscript is in the review process (from initial submission to final decision). I have had a couple of my own papers in review processes that lasted a couple of years, where my total revision time was no longer than a few calendar months. Reviewers can unnecessarily lengthen a manuscript’s time from initial submission to final decision by rejecting a review invitation any later than the same day that it is received and by submitting review reports after the requested due date. Such delays may add days, weeks, or even months to the process.
Writing the review report may create even longer delays. However, succumbing to the pressure of delivering the review by the due date may result in the submission of a “not-so-good” report that sells the author(s) short due to not enough time and effort being spent on the review. To avoid selling author(s) short, it is necessary to understand what is expected in a review report and to ensure good review quality. As outlined above, the reviewer is expected to develop an understanding of the manuscript and its contribution, identify any fatal flaw or major problem, suggest directions for improvements, list minor issues that need to be addressed, and summarize all this in a written review report using a positive tone that encourages a dialogue with the author(s). Sometimes a review report requires several days of work to develop, but there are other situations where it is also possible to submit the report after a couple of hours of work. So, what is reasonable amount of time to spend on a review? In my experience, I have encountered reviewers that spend too little and others that spend too much time on their reviews.
Reviewing a paper requires at least two reads of the manuscript. During the first read, I make note of all different types of details and issues as I read. This normally ends up in a manuscript full of comments and marks, and gives me an overall idea of how the report will be written. From the first read, I am aware of potential strengths and weaknesses, and I have ideas about what to write and suggest in the report. I normally put the paper aside after the first read and do the second read at a later time. It is seldom possible to get a full day free for reviewing, and it is good to have a fresh mind when reading a paper for the second time. Some reviews are easier and are faster to do, while some are more difficult and take longer time. It is good to do the first read as soon as possible after accepting the invitation. Then you can determine whether it is a straightforward or a more complicated reviewing task. If you believe that the review is straightforward, then you may be able to finish and submit after a few additional hours. Alternatively, more complicated reviews may require considerably more time and effort.
During the second read, I try to take a broader perspective and explore the identified areas further. I also think about directions for improvement. Quite often I realize that I have missed something fundamental in the paper during the first read. During and after the second read I start to cluster and synthesize comments, prioritize them in order of importance and think about directions for improvement to suggest. Thereafter, I write the report and at the same time looking back into the manuscript (and sometimes read it in full for a third time) as I generate my written comments. This process normally takes me about a day. Some articles are harder to review and therefore take longer. Some take shorter time, but it is hard to spend much less than a day in order to read, think, write, and develop a report with good enough quality feedback to guide the author(s). In some instances, it would be easy to spend far more than a day on a review, but very few reviewers are able to devote more than a day to performing a review.
In addition to understanding what is expected from a review and being aware that reviews require varied allocations of time and effort, one should remember that a quick review is never better than a delayed review – especially if the quick review is thrown together to meet a due date without devoting sufficient time and effort towards understanding the manuscript and shaping the report. So, if you realize you cannot meet the required due date, then tell the editor that you will be delayed and suggest a date that you can meet – and then prioritize to ensure that you meet that date. As a colleague noted, “I may be late but I am worth the wait.” Always inform editors about delays but do not make it a practice to always ask for extensions!
The average time for a review-revision-decision cycle can become considerably longer if authors have to deal with reviewer comments that require substantial revision. It is, of course, quite okay to ask for such revisions, and reviewers’ insightful observations and suggestions are often the input that makes a difference and helps to develop a paper from being good to being excellent. Moreover, while the time to reach an accept/reject decision may be longer, an additional round of reviews provides the author(s) with a further opportunity to improve the paper.
All manuscripts can be improved – even the very good ones. Hence, two rounds of revision will almost always be needed, but sometimes more rounds are justified. To this end, as reviewers, we should consider how we can contribute towards minimizing the number of review rounds to help the editor reach as early a decision as possible. The average decision cycle can be dramatically reduced if manuscripts are rejected no later than after the first round of reviewing, and are accepted, or are close to being accepted, after two rounds of reviewing-revising. So, how could a manuscript that seems far from publishable when submitted be accepted after only two review-revise rounds? Most of them cannot, so they are rejected. The role of a reviewer is to focus on identifying problems and articulating the issues that need to be improved, rather than passing judgment on the severity of flaws and opining on whether a revision opportunity should be afforded. Recommended courses of action should be shared separately with the editor rather than in the review sent to the author(s). Thus, reviewers must communicate with the both the editor and the author(s).
Performing the Gatekeeper Role
Remember: bad research can never be turned into a good paper, but good research can be turned into a bad paper. The first reviewer task is to assess whether the research behind the manuscript merits publication or has the potential to be accepted for publication after revision. The initial identification of potential problems or flaws and evaluation of whether the problems detected are correctable/uncorrectable and minor/major has been referred to as the “gatekeeping role” (Carter & Ellram, 2010). You are not a good gatekeeper if you fail to identify or incorrectly point out a fundamental problem. When such mistakes occur, your review may incorrectly generate a reject decision or significantly delay the review process.
When identifying an issue that is perceived as a major uncorrectable flaw, it is likely that you will recommend to the editor that the manuscript be rejected. However, most problems are correctable – it is just a question of what it takes to correct them. Should a reviewer suggest re-analyzing data using another method, collecting new data, adding and deleting research questions (RQs) and hypotheses, adopting a different theoretical framework? Well, it depends. Some of the best published papers were close to being rejected after the first round of reviews, or were previously rejected by another journal. Erroneous editorial decisions are often due to the potential or contribution of a manuscript not being recognized in the first round of reviews. Identifying a manuscript’s potential can be very difficult when the submission is not written in a traditional way, not using established theory or methodology, or simply because the good research is hidden behind bad writing and presentation. If the topic addressed and empirical data look promising or there are other indications of unrealized potential, then we normally want to give the authors a chance to clarify and develop their work.
Finally, uncorrectable problems may not necessarily eliminate enough of the study’s contribution to motivate a reject decision. Explicitly discussing the problem as a limitation may be sufficient. Remember: all research, methods, and papers have flaws and problems (Summers, 2001). So, the question often boils down to “What flaws and problems must be eliminated?” and “Is a major revision really merited?”
Performing the Gardener Role
You are not really helping the author by merely taking on the gatekeeping role to identify what you believe to be flaws and problems. A second reviewing task is to help develop the paper by suggesting how to revise in order to improve the paper. This task has been referred to as the “gardener role” (Carter & Ellram, 2010). Taking on this developmental role is challenging, because determining the extent of the guidance/solutions that should be provided to the author(s) can be difficult. A good developmental review identifies and explains problems, and gives guidance without delivering solutions (Ragins, 2017). The authors are afforded flexibility to solve the problems and write the article they want to write. Authors are more likely to return a revised manuscript that has solved the problems and makes a strong contribution if they are afforded the freedom to address the issues identified by the reviewer, rather than being compelled to slavishly adhere to the reviewer’s prescribed remedies.
To more clearly appreciate the nuances of the developmental aspect of reviewing, we can start by thinking about our own experiences of receiving review comments. I am sure most of us have encountered review comments where a reviewer has misunderstood something in the paper and asked for a revision that is either incorrect or does not obviously improve the paper. I am afraid most of us, at least sometimes, have also revised our manuscripts in order to comply with such questionable reviewer comments. Erroneous comments inevitably create potentially unnecessary work on our end that may extend the revision period, and sometimes even reduce the quality of the final paper. I am certain that, on occasion, my own review comments have unintentionally created similar predicaments for authors! Constructive communication can greatly help to ensure that our review comments really do develop rather than hinder the manuscript.
Constructively Communicating To Authors
The identified issues and suggested remedies are the content of the review, but the way the review is communicated with the author(s) and the editor is equally key to a good review. Identifying and communicating critical issues, even when the manuscript is being rejected, is very beneficial for the authors if they are able to understand them and use the feedback to develop their manuscript. In contrast, harsh critique is hard to accept – especially if it is not fully understood. For example, just stating that a clear contribution is missing is not sufficient. The reviewer must clearly explain why an issue is a problem.
Even explanations are not always enough. A critical message is seldom fully understood and absorbed by the author if it is communicated in a negative and impolite way. Unfortunately, many potentially good reviews are not well received because of comments and wording with negative connotations. Such reviews invariably upset authors. Hopefully, after the first frustrating read, authors will revisit the review and objectively try to understand the message behind the impolite presentation. However, it is more likely that the majority of such reviews are never really taken seriously by the authors. Thus, when communicating very critical concerns, it is especially important to be as polite as possible. For some people, this is an obvious and easy task, but for others it is not. Just because you are a good researcher, or trained to spot flaws, does not mean you are a good reviewer. You may still be unsuccessful at the review task if you are not able to understand and/or communicate with authors.
Try not to give false hope by being overly optimistic about a manuscript you believe will have difficulties being published, and try not to use a negative tone about manuscripts you do not like. The tone should be positive and give credit, but at the same time be critical when necessary. This is not easy. Authors have most likely put lots of effort into their manuscripts. Sometimes their tenure or promotion may be dependent on successfully progressing this work. If the work is being rejected this time, then a good review should give the authors the motivation and guidance to go back to their material and develop it. As a reviewer, you must understand that you are writing the review to the author(s) – not to the editor or to yourself. Both reviewers and authors should anticipate a dialogue in the review-revise-process rather than two monologues. For the author this means that they should not slavishly accept all the reviewer comments. I recently suggested accepting a paper after authors had revised a resubmitted paper; the authors had revised according to some of my suggestions, but in their response letter they convinced me that I had misunderstood several issues. Hence, the way a review report is written opens up the potential for a dialogue.
Formulating your critique in an objective way without recommending an editorial decision or suggesting other journal outlets to the authors is easier. Write such thoughts and decision suggestions in a confidential message to the editor whose role it is to communicate the likelihood of a successful revision. Doing so also makes it easier and faster for the editor to understand your review and render a decision. Finally, think twice before submitting your review by asking yourself: “Have I understood the authors and evaluated the manuscript without making biased judgments?”…“Am I forcing the author to incorporate my own thoughts?”…“Might the review upset the authors?”, and perhaps most importantly, “Would I have liked to receive this review?”
Efficient AND Effective Reviewing
A final observation is that, reviewing takes time – sometimes a long time. On the supposition that at least 10, and perhaps as many as 20, reviews may be expected each year and counting how many days a year can be devoted to performing them, it is evident that reviewing should be done in a structured and efficient way. How much time and energy should really be put into reviewing? My target is to complete a review in a day, but sometimes crafting the assessment takes more time and sometimes less. However, I simply cannot spend much more time on reviewing than I do. So, in order to perform the requested reviews, I need to be more efficient without compromising quality. Relatedly, Romanelli’s (1996) editorial reflected on how she developed as a reviewer over the years. During her early years in academia, she spent considerable time on each review because she wanted to impress the editors. After several years she more clearly understood how to create effective reviews for authors in as efficient a manner as possible. Although some universities have reviewing classes, most of us have learned and developed our reviewing style and skills by ourselves. Yet anyone can learn and improve as a reviewer. Such improvement is facilitated when journals share all reviewer comments, authors’ response letters, and editors’ comments with each reviewer to enable them to better determine what type of review comments are really helping.
Implications for the HRD Field
High-quality research published in peer-reviewed outlets reflects favorably on the journals, the field, and on the broader research community. However, given the concerns about the broken review process and review avoidance, our invited paper endeavors to convey the interdependent nature of the ideal review process. More specifically, to promote the high-quality reviews that stimulate the publication of high-quality research, authors, editors and their editorial teams, as well as reviewers, must each participate effectively in the review process. To this end, we outline the specific responsibilities of authors, editors and their editorial teams, and reviewers. In addition, we provide illustrations of undesirable reviewer comments and present an overview of the broad themes on high-quality reviewing that have been identified in the literature. We also offer an in-depth reflection on reviewing from a master reviewer that readers may draw upon to guide their approach to reviewing. We are hopeful that this article will encourage and promote dialogue on more effective implementations of the review process to arrest the alarming escalation in review avoidance reported by editors of HRD journals. We urge publishers, journal editors, their editorial teams, and scholarly institutions to more expeditiously disseminate the fundamental components of an ideal review process to ensure that prospective and potential reviewers from all realms and domains can be provided with opportunities to better understand their roles and responsibilities and how to perform proficient reviews. Offering reviewer training sessions in academic programs and by professional associations along with mentoring opportunities to cultivate reviewer expertise is particularly important in the HRD field where journal editors have frequently highlighted the challenges of obtaining high quality reviews as submissions continue to rise. Finally, we envision that the perspectives shared and the resources provided within this article will be an informative guide that can be used for reviewer training sessions and by mentors to improve scholarly publication and peer review processes.
Footnotes
Acknowledgments
The authors would like to thank the Editor and Editorial Team for their helpful feedback during the review process. We also thank Dr. George Franke, Professor Emeritus, The University of Alabama, for allowing us to include one of his expressions about reviewing.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
