A modest proposal to the peer review process: a collaborative and interdisciplinary approach in the assessment of scholarly communication

Abstract

The purpose of the traditional peer review process (TPR) is to provide a more constructive and scientifically rigorous critical review of scholarly research that builds scientific rigor and validity within diverse academic disciplines. Peer review has received criticism as the demand for publications in a variety of competitive journals has significantly increased while the number of individuals who are both willing and qualified to conduct thorough reviews is significantly declining. The purpose of this topic piece is to examine the overall efficacy of the peer review process and provide recommendations toward a more collaborative, transparent (i.e. “open”), and interdisciplinary communication process.

Current challenges of the peer review process

A primary goal in the development of the scientific method lies in our ability to test hypotheses and communicate research findings in both a scientifically rigorous and replicable process, one that helps us to better understand human behaviors and promotes the exchanges of innovative ideas among scholars (Parks, 2020). The traditional peer review process (TPR) has been described as a type of “quality filter” that maintains the highest levels of academic publications in terms of integrity and ethics (Panda, 2019). Similarly, the peer review process can be thought of as a necessary (but insufficient) component of the scientific method in that it remains both a traditional and fundamental tool in communicating interdisciplinary information while also maintaining the highest scientific standards in publishing scientific literature to the public (Ahmed and Gasparyan, 2013). However, despite the fundamental and scientifically important purposes that the peer review process has historically served (i.e. reliability and replicability), it has become the focus of increasing criticism for several reasons, including delayed reviewing time for authors to receive results, averaging over 168 days for accepted manuscripts (Okuzaki et al., 2019) or that the overall process of peer review has become increasingly biased and unreliable, thereby reducing overall predictive validity (Meruane et al., 2016).

More recently, attention has focused on the issues of personal and ad hominem attacks from reviewers directed toward authors regarding the overall quality of their work (Barnes et al., 2018; Burnett, 2017; COPE, 2020). Herber et al. (2020) have noted that while many reviewer comments have remained respectful, a growing number of critical peer reviews were “unhelpful and destructive” to authors (p. 12). This issue was most recently addressed by the Committee on Publication Ethics (COPE) Forum in March 2020, where comments from those attending indicated that a distinction should exist between “personal attacks from principled discussion” and “civil, courteous and respectful discourse” should always prevail in the dialogs between reviewers and authors. The purpose of this topic piece is to provide a more comprehensive, accountable, and interdisciplinary approach to the peer review process that both emphasizes and underscores the utilitarian value of transparent collaboration among reviewers and authors.

The evolutionary history of peer review

The history of the peer review process is credited as beginning in the early to mid-17th century within formal higher educational learning institutions and academies that were generally located in western Europe, what Tennant et al. (2017) refer to as “the primordial time of peer review” (p. 4). In 1660 The Royal Society of London was established where one individual (i.e. typically the editor) provided an open literature source (i.e. The Philosophical Transactions) that was made available to a broad range of scholars and academics who would contribute their work for critical review. (Oddli et al., 2020; Panda, 2019). Interestingly, Tennant et al. (2017) note that the actual term “peer review” was not generally used among academics in reference to general publication practices until the 1960s (Tennant et al., 2017). Traditional peer review continued for centuries without much change until relatively recently (i.e. early 21st century) when the publication of manuscripts became essentially synonymous with the tenure process in higher education within the United States. While some journals continue to use a double-blind procedure (i.e. the identities of the author and reviewer remain anonymous to each other), this approach has come under criticism in that the identity of the author(s) is less secure given the increased capacity of internet technology. The problem of unmasking the identity of the author(s) becomes even more apparent in smaller (i.e. more specific) areas of scientific study and research (Panda, 2019). An additional problem in the traditional review process is the practice of confirming what we believe to be true while ignoring or deemphasizing other existing information that may challenge our intuitive beliefs. Mahoney (1977) describes these phenomena in mental health practice as “confirmatory bias” (p. 161) and notes that individuals suffering from depression may often confirm maladaptive belief systems while ignoring positive events that exist in their lives. Additionally, Miller (2006) describes “confirmation bias” (p. 426) as an increasing problem within the peer review process where a manuscript may become rejected simply because it challenges conventional thinking and does not support traditional publications in the scientific world.

Suggestions for creating a more collaborative and constructive peer review process

Constructive and effective peer reviews advance scientific knowledge through respectful and civil critiques that identify both the strengths and weaknesses of the manuscript presented to them. This is perhaps even more true with specific types of research that address the topics of community-based learning, service work activities and participatory research in that researchers often work more interpersonally and with fewer community participants (i.e. typical of qualitative or participatory research). In this way, the creation and development of innovative scientific knowledge hinges on our capacity of building and sustaining collaborative and professional working relationships with each other. Current trends within the peer review process recommend the use of “article-commenting functionality” (p. 82), a process that provides a venue for sharing public comments, although larger publications remain reluctant to incorporate this practice (Wakeling et al., 2020). An important quality in peer review is what Panda (2019) refers to as “trust” (p. 240). Trust in that reviewers and authors are both working collaboratively and in good faith in a process that examines both the merits and challenges of each submission in a fair and impartial manner. A more transparent and interdisciplinary peer review process provides opportunities for experts within different disciplines and fields to work in a more supportive, transparent, and collaborative manner in their evaluation of research articles. Successful interdisciplinary team work requires a combination of skills, such as trust, professional competency, and a commitment among all people involved in the general evaluation of research (Nancarrow et al., 2013). After reviewing dozens of recent articles and publications addressing the problems that are associated with the traditional peer review process (e.g. Houry et al., 2012; Tennant et al., 2017; Wakeling et al., 2020), I have identified four consistent intervention practices that I believe will provide a more accountable, collaborative, and respectful experience in the peer review process.

Streamlined process

Given the increased volumes of submitted works for publication, a faster and more efficient streamlined process should be developed among editors that allows for an accurate screening process to emerge identifying ideal and relevant manuscripts that match the mission and goals of the journal. One approach (i.e. “portable peer review”) has been introduced by Panda (2019) that provides rival publication sources with copies of the peer reviews of previously rejected papers. The peer reviews are then able to be reused by other journals thereby significantly reducing the overall time of the peer review process. Additionally, some recent research (Sciullo and Duncan, 2019) has identified compensation of reviewers as an effective approach to attract a broader range of reviewers as well as providing a 30-day maximum period of time for manuscript reviews.

Accountable transparency

Accountable transparency within the peer review process refers to general comments made by both practitioners and academics who agree to disclose their identity and facilitate a culture of trust that helps reduce (or eliminate) reviewer bias and what Bali (2015) refers to as “unethical criticism”. Peer reviews that embrace and utilize an open and transparent relationship between reviewer and author can remove the problems of confirmation bias and harsh criticisms through an improved (i.e. more accountable and educational relationship) process that helps to identify the potential and merits of the work itself (Moylan et al., 2014). Additionally, OPR can be developed in providing varying levels of transparency and openness to the public that can also facilitate interdisciplinary research and activities. For example, Tennant et al. (2017) has described how recent innovations of the internet and Web resources can help promote a greater process of “open, collaborative and social communication” (p. 10) and how OPR functions primarily from three primary criteria:

• (a) Referee names are identified to the authors that may or may not include readership;

• (b) Referee reports are made publicly available; and

• (c) Peer review is not limited to invited referees only.

Growth through interdisciplinary collaboration

The OPR process has been described as a collaborative, transparent, and comprehensive process that gathers the unique experiences and knowledge of several actors (i.e. authors, reviewers, and editors) in the development a document that represents scientifically accurate information across all disciplines within higher education. An open and interdisciplinary approach to peer review provides greater opportunities and collaboration for experts from different disciplines (i.e. social and applied sciences, humanities, etc.) to share their individual skills in the overall critical review of scientific research. Despite the reported general advantages of an interdisciplinary and collaborative approach in the peer review process (Litano, 2019), challenges still exist based on the specific pedagogical content of some courses (e.g. humanities, chemistry, or calculus) that often lack a multidisciplinary approach.

Educating reviewers

As part of the OPR movement, many editors and authors are calling for a universal process that helps to both educate and inform potential reviewers about their responsibilities and the criteria that should be used in the review process. Recent research addressing the benefits of educating or training reviewers has been mixed depending on the nature, discipline, and specific areas of expertise of the publication journal. For example, efforts in training and educating reviewers in the medical sciences have remained relatively unsuccessful, whereas pairing inexperienced with experienced reviewers has been more successful (Houry et al., 2012). Reviewers can also be trained or certified from a variety of professional organizations or institutions of higher education depending on the level of their skills and expertise within specific disciplines (Sciullo and Duncan, 2019). The Committee on Publication Ethics (COPE) provides direct support and information to reviewers, authors, and editors addressing multiple issues that may involve publication ethics and general responsibilities. In a recent (July 6, 2021) COPE publication, several recommendations addressed fundamental techniques in how to be a good peer reviewer, citing the need for shorter review periods, being intentional in helping authors to revise and ultimately improve their work, as well as being familiar with journal guidelines in peer review practices (Wallace, 2021). Additionally, a recent survey addressing some of the common problems that have been associated with peer review, over 56% of the researchers interviewed indicated that there “lacked coordination and guidance in how reviews were to be conducted” (Sense About Science, 2009). An important recommendation that can help address some of the problems that researchers are experiencing relative to a more coordinated review process would include listing COPE reviewer guidelines in the home page for journals.

Future trends in peer review

The purpose of this topic piece has been to identify the benefits of an interdisciplinary, open, and collaborative peer review process. Given the history, purpose, and development of the peer review process, there has been a shift in the amount of available and qualified peer reviewers who are capable of providing their expertise to a growing number of manuscripts. While it is true that not all submitted manuscript proposals can ever be published, I propose a few recommendations that can create a more respectful and productive environment within the peer review process:

• A more respectful, supportive and collaborative process is currently needed in the peer review process. Publication and review processes should not be an ivory tower “rite of passage” event that includes weaponized reviewer jargon that de facto only serves as a form of modernized hazing;

• Simple guidelines (when followed) can facilitate the transition from a traditional peer review process into a more interdisciplinary, collaborative, accountable, and supportive process that can actually build and strengthen the relationship between reviewers, editors, and authors. This would include reviewers who remain mindful of deadlines and follow manuscript guidelines while evaluating each manuscript; and

• Recognizing that the current peer review practices should be recognized as a shared responsibility and collaborative (i.e. not demeaning) process that can improve the quality of scholarly work while also fostering high ethical standards and integrity in the development of professional relationships among reviewers, editors, and authors.