Peer Review and Medical Journals

Abstract

Journal editors depend on peer reviewers to make decisions about submitted manuscripts. These reviewers help evaluate the methods, the results, the discussion of the results, and the overall organization and presentation of the manuscript. In addition, reviewers can help identify important mistakes and possible misconduct. Editors frequently have difficulty obtaining enough peer reviews which are submitted in a timely manner. This increases the workload of editors and journal managers and potentially delays the publication of clinical and research studies. This commentary discusses of the importance of peer reviews and make suggestions which potentially can increase the participation of academic faculty and researchers in this important activity.

Editors depend on reviewers to help evaluate manuscripts submitted for possible publication. These reviewers can help analyze the methods, the results, the interpretation of the results, and the discussion of the results in the context of other medical literature. In addition, they can comment on the clarity of the text and any figures and tables. In the best circumstances, the reviewer has knowledge and experience working with the topic in the manuscript. However, experienced reviewers can make comments about the manuscript which can potentially improve its presentation without having in-depth expertise about the topic.

There has been a significant increase in the number of electronic journals and in the number of published articles over the last decade. This has increased the workload for editors, editorial boards, journal managers, and reviewers. An additional challenge is finding reviewers who agree to review the manuscript in a timely manner. The editor for the Journal of Primary Care and Community Health recently updated its editorial board regarding journal activity. He stated that he often needed to contact 20 to 25 potential reviewers to obtain an adequate number of reviews for an article, and on occasion, he has had to contact up to 45 potential reviewers. In addition, many reviewers do not complete the review in a satisfactory timeframe. This situation clearly increases his workload and delays decisions regarding manuscripts. Do other journals have similar problems?

Hillard and Baber ¹ published an editorial in the journal Climacteric in 2020 in which they state that the search for talented and willing peer reviewers is the number 1 problem for journal editors worldwide. They note that peer review supports the quality of medical research and that the work involved in undertaking these reviews can enhance the academic career and credentials of the reviewers. Fuster and Turco ² wrote an editorial entitled “Protecting Peer Review” in the Journal of the American College of Cardiology in 2020. They indicate that peer review should improve the quality of submitted manuscripts and can help identify fraudulent submissions, and they describe a demanding peer review process used by their journal. Keating and Mohile ³ in a commentary in the Journal of Geriatric Oncology suggested that authors who publish manuscripts have created an obligation that they should in turn review other manuscripts with an expected ratio of 3 reviews for each publication. In addition, they thought that any reviewer who declines a review must identify other possible qualified reviewers. They suggested several methods to increase participation in this process which could include payments for reviews and professional incentives for tenure and promotion, and they encouraged more training in this process which might improve participation and the quality of reviews.

Chloros et al ⁴ have published a detailed review of peer reviews in surgical journals published in the Annals of Surgery. They discussed the causes of the peer review crisis, the incentives associated with being a peer reviewer, and current peer review practices to reward reviewers and then made recommendations to improve this process. One important suggestion involves the establishment of peer review activity criteria for academic appointments that should be included in the tenure and promotion reviews and in job descriptions. In addition, they suggested that the use of artificial intelligence software could potentially identify more qualified reviewers and reduce the frequency of reviewer declines based on lack of expertise. The scientific publications committee of the American College of Cardiology developed a task force to review and propose new methods for obtaining high-quality peer reviews. ⁵ The most important issues identified in this analysis were the effect of preprint servers, reviewer blinding, reviewer selection, reviewer incentives, and publication of peer review comments. They concluded that the review process was essential for maintaining scientific integrity, facilitating the distribution of information, and improving clinical care based on clinical research. There are no obvious simple solutions to these controversies and challenges which largely focus on the peer review process and not on the low participation level by academic faculty.

Publons ⁶ conducted a global reviewer survey between May and July 2018 that involved over 11 800 researchers identified in the Publons database and from articles indexed in the Web of Science. Some of the numerical results developed from responses from “established” regions included 1.95 reviews per submission, a median review time of 16.4 days, a 49% reviewer invitation acceptance rate, and a review lengths of 527 words. Forty-one percent of the respondents thought that peer reviewing was part of their job as researchers, 35% thought they should do their fair share, 33% thought it would allow them to keep up to date, and 33% thought it would help ensure the quality and integrity of research published. The majority of respondents (85%) thought that institutions should require and recognize peer review contributions. This survey took place prior to the COVID-19 pandemic, and at present the reviewer acceptance rate is likely lower, and the time to review completion is almost certainly longer. However, there is no solid information about current reviewer statistics publicly available, and a new survey should be undertaken by academic publishers with a large number of medical journals.

The authors of this editorial suggest the following possible solutions: (1) Academic institutions should expect faculty members to provide periodic peer reviews. Faculty members should understand that this an expectation for working in the academic organizations, but with the recognition that that not all review invitations can be accepted given time constraints and limitations in expertise. (2) These institutions should add workshops to explain the process to students, post-graduate trainees, and faculty and include participation in the peer review as a graduation requirement for students and trainees. (3) Current active peer reviewers should identify at least 2 additional faculty members or research associates at their institutions who can serve as peer reviewers. (4) Journals should offer discounts on article processing fees for reviewers. (5) Journals should offer streamlined, formal recognition for reviews to support reviewers’ annual academic reports. (6) Academic institutions should recognize peer review as a form of academic work and productivity and consider this in evaluations for promotion and tenure. (7) Journals should use a uniform format for peer reviews and reduce bureaucratic tasks for the reviewer when possible.

Suggestion number 7 encourages journals and editors to consider the mechanics involved in peer review for their journals. Most journals have their own propriety review portals which include multiple choice questions about the basic features of the manuscript, such as data integrity and statistical soundness, and a free text portion for comments. In addition, some portals may require reviewers to answer a list of questions about the manuscript, which do not apply to certain types of studies and manuscripts. While these questions may ensure that reviewers address crucial portions of the manuscript (methodology and statistics), they can be cumbersome to sort through and may provide little meaningful commentary on the manuscript itself. These logistical challenges, though seemingly small, can nonetheless combine to make the review process cumbersome, which may increase the frequency of decisions to decline the review invitation, especially when there are time constraints and conflicts with other professional commitments. Consequently, journal should periodically review their website to make this process as simple as possible and get feedback from current reviewers.^7,8

Ultimately, solutions for this problem will require substantial efforts by academic institutions and will not have widespread, uniform support, or interest. This concern probably does not represent a priority for institutions with complex responsibilities. The easiest first step would involve regular communication by journal editors with their editorial boards to update them on the current situation with their particular journals and make requests for more support based on ideas they think are most relevant to their scientific disciplines and journals. However, until academic institutions and faculty recognize the importance of this particular problem, peer review will continue to depend on the efforts of a small percentage of academic and research faculty. Unfortunately, this unsatisfactory situation is unlikely to change much in the near future.