Abstract
The standing and progress of science depends upon confidence in the evaluation of knowledge claims. This essay affirms the value of peer review as a ‘gold standard’ but argues that its efficacy for scientific progress is, on balance, diminished by blinding. It reflects critically upon the anomaly between an ethos of openness that is widely held to define scientific work, and the opacity institutionalized in reviewing and editorial processes, with specific reference to the field of management and organization studies. The anomaly is attributed to the operation of asymmetrical relations of power in the establishment and reproduction of evaluation processes. The proposed means of mitigating the anomaly, and thereby improving manuscript evaluation, is movement in the direction of more open peer review.
Keywords
Dissents speak to a future age. (Ruth Bader Ginsburg)
The focus of this essay, it is worth stressing at the outset, is not peer review but its blinding. Despite its well-rehearsed shortcomings (Atkinson, 2001; Macdonald, 2015; Souder, 2011; Weiskittel, 2015), my purpose is not to discredit or reject the principle and value of peer review. Instead, it is to stimulate critical reflection on how and why editorial and review activities are shrouded in secrecy, and to foster debate about the benign necessity attributed to blinding manuscript evaluation (for an overview, see Shatz, 2004: esp. ch. 2). My conjecture is that, to quote one of the reviewers of this essay, many management and organization studies (MOS) scholars, myself included, have become immersed in a world where blinding of reviews is taken for granted. Perhaps it is time to surface from this comfortable immersion by subjecting blinding to critical scrutiny. Adopting the view of a ‘devil’s advocate’,1 I argue that greater openness, and less secrecy, is more congruent with the distinctive, open status ascribed to science and is more supportive of its development.
Peer review is, in my experience, invaluable when it provides informed, challenging and constructively critical feedback. Peer review is generally more fruitful, and is certainly more defensible, than ‘pal review, boss review, or no review at all’ (Alvesson and Gabriel, 2013: 258), even if these categories are sometimes difficult to distinguish. My concern is that secrecy, and lack of accountability, subverts and diminishes the credibility of peer review and its capacity to assist authors to discern and realize the potential of their submissions; and to resist some other purpose or agenda (e.g. of the reviewer or the editor) being pressed upon them. Openness, not opacity, makes it possible to detect, expose and thereby deter instances of ‘poor review’, as exemplified by reports that overflow with opinionated prescriptions that are largely irrelevant, or that are obstructive for cultivating more diverse, creative and relevant forms of scientific knowledge, including contributions that are directed at ‘grand challenges’ (Harley and Fleming, 2021).
I do not advocate replacement of peer review by fully open review. Instead, I identify some imperfections of (blinded) peer review; and I suggest how they may be mitigated by reducing opacity in processes of manuscript evaluation. Other advocates of greater openness (e.g. Osterloh and Kieser, 2015) have argued convincingly, in my view, that malpractice and abuses of power in manuscript evaluation processes are inhibited and reduced by increased ‘sunlight’. Greater visibility of reviewer and editorial practice can encourage the preparation of reviews and editorial decisions that are more considered and constructive. Conversely, by veiling abuses, secrecy contributes, I conjecture, to the parlous state of management and organization studies (e.g. Alvesson and Gabriel, 2013; Corbett et al., 2014; DeNisi, 2010; Grey, 2010; Honig et al., 2018), including its failure to meet the challenge of addressing contemporary, real-world problems (e.g. Barley, 2007; Davis, 2015; George, 2014).
Much MOS scholarship has been lampooned as isomorphic and formulaic (Greenwood, 2016) – which is perhaps to be expected when editors issue authors with ‘helpful guidance’ on ‘managing topic choice and the research and writing process to effectively convey a novel and empirical contribution’ (George, 2012: 1023). The unspoken threat of many, doubtless well-intended, MOS editorials is to remind authors of the perils of failing to heed ‘guidance’ – which does little to discourage the preparation and publication of imitative, ‘dry, impersonal and overcautious texts’ (Alvesson and Gabriel, 2013: 247) drained of meaningful novelty or critical insight. Too rarely is the flow of such scholarship disturbed by something more lively, personal, thought-provoking and/or distressing (e.g. De Rond and Lok, 2016). By opening up journal evaluation processes to inspection, insights can be gained into the justification for desk rejection of manuscripts deemed, for example, to lack ‘novel’ contribution; and it would shed light on how reviewers and editors ‘guide’ authors in identifying and conveying such novelty.
When addressing malaise in MOS scholarship, remedies tend to be framed in relation to the suppliers of research rather than to processes of evaluation – a partial exception being Hillman and Rynes (2007)2 who attend directly to the blinding of peer review.3 Yet, the preparation and publication of innovative, distinctively ‘voiced’ manuscripts, including those that aspire to address ‘grand challenges’ and ‘real-world problems’, is contingent upon their anticipated or actual reception, initially by the editor and reviewers. Accordingly, the focus of this essay is (re)directed from deficits ascribed to authors’ submissions to the processes of evaluation that condition the supply, as well as determine the fate, of novel, diverse and/or problem-relevant submissions (Dean and Forray, 2019). With specific reference to blinding peer review in MOS (but this point applies across the social sciences), the absence of discussion is striking. The silence is deafening not least because, in the natural and medical sciences – whose (idealized) practices many (neopositivist) contributions to social science, including MOS, strive to emulate – blinding is increasingly seen to shelter malpractice and to impede or discredit peer review. For example, secrecy makes it easier to conceal data fabrication, coercive citation, falsification including photo manipulation, self-plagiarism and so on. These malpractices have their equivalents in MOS (Karabag and Berggren, 2016; Tourish, 2019; Tourish and Craig, 2020). Conversely, unblinding offers a means of exposing, and thereby deterring and counteracting such abuses.
Keeping, or making, science more open and honest – by questioning the norm of secrecy – is congruent with a broader movement of intellectual activism that aspires to develop ‘more just, democratic and equal relations’ (Contu, 2020: 741). Confidence in science is relevant and critical for addressing every contemporary ‘grand challenge’. Openness is essential to ‘“epistemic” justice . . . of what counts (and how) as legitimate, recognized knowledge and what is excluded or silenced and with what effects’ (Contu, 2020: 745). It involves ‘“speaking truth to power” . . . “challenging power structures from the inside, working the cracks within the system” . . . (Collins, 2012: xiii)’ (Contu, 2020: 745, emphasis added). Such ‘cracks within the system’ include the abuses, contradictions and pathologies that are disregarded, or simply normalized, by the benign necessity ascribed to blinding.
Blinding as benign necessity?
The process of manuscript evaluation typically includes: the decision to review a submission rather than ‘desk reject’ it; allocation of the manuscript to a senior editor who selects the reviewers and handles it through the review process; editorial assessment of the review reports; and guidance provided to authors when a resubmission decision is made. The wider significance and impact of manuscript evaluation lies in encouraging or impeding current and novel lines, or agendas, of research (Davis, 2015); and, relatedly, in developing or narrowing disciplines and sub-disciplines. Because evaluation work is also consequential for the careers of authors with regard to job applications and tenure decisions, it also assists or curtails authors’ prospects for making future contributions to the body of scientific knowledge.
Returning to my earlier reference to Hillman and Rynes (2007), they urge ‘journal editors and scholars [to] conduct additional research on the effects of non-blind, single-blind and double-blind review processes’ (2007: 626, emphasis added). Of the handful of authors who have cited this article (see note 2), only Clark and Wright (2007), then editors of the Journal of Management Studies, engage directly with its central concerns. After endorsing a withering assessment of the shortcomings of (blinded) peer review provided by Macdonald and Kam (2007), Clark and Wright (2007: 614) comment that ‘the situation is possibly more serious’. They then propose a number of remedies that include more careful selection of reviewers and the introduction of ‘greater transparency in the operation’ of journals (2007: 618–619). Notably, their reflections and proposals are restricted to improving ‘transparency’ within blinding. The possibility of unblinding editorial processes is excluded. No consideration is given to how open review may offer an antidote to some of the shortcoming ascribed by Clark and Wright to peer review or to other practices that bring it into disrepute, such as predatory publishing by ‘junk’ and ‘fake’ academic journals (e.g. Dobusch and Heimstädt, 2019). It is curious and disappointing that no research on the effects of blinding and non-blinding has, to my knowledge, been undertaken by MOS researchers or appeared in MOS journals. A possible explanation of this absence will emerge as the argument of this essay unfolds, and to which it proposes a corrective response.
As an author, reviewer and editor, I had devoted little reflection to the blinding of peer review prior to encountering a series of instances of editorial conduct that proved difficult to reconcile with declared editorial policy. Cumulatively, these experiences provoked me into reflecting on the justification of secrecy, in the form of blinding, that has become so strongly institutionalized. Earlier, some degree of reflection had been stimulated when sharing privately and informally cases of deficient or abusive editorial conduct. Such failings may be procedural (delays, administrative errors, misidentification of peer reviewers) or they may be more substantive (e.g. receipt of unprofessional, ill-informed, hostile or abusive reviews or editorial decisions). These shortcomings can, of course, have substantive impacts, especially for junior scholars whose appointment or tenure may hang on receipt of timely and reliable evaluations of their manuscripts. Very rarely, however, are these concerns publicly aired as a stimulus for reflection and potential remedy.
One instructive example of ‘going public’ is Honig’s post to the Academy of Management Connect Blog where he describes an experience of submitting a paper to a FT50 journal (see https://connect.aom.org/blogs/benson-honig/2016/10/24/journal-editors-unregulated-and-unmonitored [accessed 6 June 2020]). Reflecting on this episode, Honig is led ‘to question the authority, transparency, and lack of professionalism sometimes exhibited by editors of even top journals’. I have likewise been shocked and dismayed to find, for example, ‘ethical’ principles of anonymity and confidentiality, and even informed consent and intellectual property rights, being invoked, also by FT50 journal editors, to frustrate and close down questioning and scrutiny of editorial practice (Willmott, 2021). Indeed, only yesterday, I received an email from a well-known senior MOS scholar (with whom I have had no previous contact and who had no knowledge of this essay), in which s/he refers to a manuscript that ‘was accepted and then suddenly unaccepted at Academy of Management Learning and Education (AMLE), another that an editor wished to re-write (I refused) at Academy of Management Review (AMR) and a third that was for all intents and purposes re-written’ (see also Clark et al., 2016: 5; O’Doherty and Willmott, 2009). Perhaps such experiences are exceptional, or maybe they are exaggerated, if not fabricated. It is impossible to know when editorial practices are shrouded in secrecy.
Escaping ‘bias’ and vendettas?
Defenders of opacity contend that the blinding of reviews significantly diminishes partisanship and ‘biases’ arising from reviewers’ knowledge of the identity and/ or affiliation of the author (e.g. ‘Matthew’ and ‘Matilda’ effects; Merton, 1968);4 and, it is also argued that blinding reduces the risks of retribution by authors who, after receiving unflattering assessments of their manuscripts from (named) reviewers, may retaliate by blocking the latter’s grant proposals or promotion cases (e.g. Broder, 1993; Merton, 1968). Yet, even those studies that confirm that, for example, single blinded (access to author information) compared with single blinded review (no access to author information) tends to advantage well-established authors and/or well-regarded institutions (Tomkins et al., 2017: 12712), acknowledge that the availability of author information can enable reviewers to ‘make better overall judgements’, with the exception of articles that are judged to be ‘of roughly equal merit’. With regard to the issue of retaliation by senior authors, Morrison (2006: 831) wonders how much ‘bias’ is removed, rather than created, by blinding (see discussion below), and how many senior people have ‘no more pressing matters on their minds than carrying out vendettas against junior faculty who criticise a paper they have written’. That acknowledged, it is of course possible that some colleagues do indeed devote time to engaging in ‘vendettas’, or at least believe that others are so engaged.
To illustrate the point: one commentator on this essay expressed concern that unblinding peer reviewing would, for example, result in a scenario where a senior male author might mount a robust defence of his work in response to a review prepared by a junior female referee that, in turn, could result in the junior female referee refusing any further service to peer review. That outcome would, indeed, be horrible, personally and collectively. It is therefore an issue that should not be ducked. Noting, but leaving aside the gender and age stereotyping in this comment that can have their own performative effects, the ‘senior author–junior referee’ scenario omits reference to the key role of editors in their duty of care towards reviewers (as well as authors) when detecting that some (albeit ‘grey’ and contested) line of ‘professionalism’ may have been crossed. In open review, reviewers have the opportunity, with editors’ support, to respond directly and publicly to ‘robust’ responses received from authors when the latter are believed to have ‘crossed a line’. As well as potentially facilitating debate on contentious issues, the publication of referee reports and authors’ responses can enable readers to assess their merits. One downside of open review, it should be acknowledged, is that reviewers may withdraw from their initial agreement to the publication of their reports and responses for non-scholarly reasons – for example, if either party believes that honouring the agreement could adversely affect their reputation or self-esteem. In general, I wish to underscore my argument that if it is assumed that unblinding will increase or extend forms of malpractice (e.g. ‘crossing a line’), then unblinding will at least render such malpractices accessible to scrutiny and potential correction, rather than endorsing their concealment and perpetuation.
Few cheers for blinding
Research has indicated how, as a consequence of poor evaluative processes, a large number of poorly cited articles have appeared in ‘top-tier’ journals (with a few articles being very highly cited); and many highly cited articles are published in lower-tier journals, many of them having been previously rejected from ‘top-flight’ journals (Baum, 2011). In order to understand how this can happen, and introduce reforms that might improve processes of evaluation, it is necessary to make them accessible to scrutiny. In another study, Peters and Ceci (1982) found that eight out of 12 blinded articles were rejected for having methodological flaws when resubmitted to the same journals in which they had already appeared! It is probable that a number of these flaws would have been deterred or corrected if the review process had been accessible for inspection. Giving visibility to evaluation processes by unblinding peer review offers a first step in diagnosing shortcomings and facilitating their remedy.
Blinding is, in any event, often something of a charade when, in practice, it is more difficult to achieve or enforce than it may seem; and it can also be perverse in its effects. It is difficult to remove all identifiable traces of articles from the web, including previous versions and conference presentations; and/or fully anonymize manuscripts. Perversely, moreover, the very removal of reference to the work of the author(s) from a submitted manuscript, especially when they are well established in a specialist field, can make their identification easier. Additionally, perhaps of greatest importance, removing identifiers (e.g. self-citation) from submissions is counterproductive for the development of scientific knowledge when it precludes authors positioning the relevance and contribution of their manuscripts within an evolving body of work. The removal of identifiers also deprives reviewers (and editors) of this highly relevant information when assessing the potential added-value of a manuscript, and so it makes author malpractices, such as ‘salami slicing’, easier to conceal from reviewers and ‘mail carrier’ editors (Clark et al., 2016: 4). When the anonymization of manuscripts is successful, there is then a risk of reviewers mis-attributing authorship when lacking the saintly capacity to refrain from speculating, mistakenly, on their authors’ identity. It is also paradoxical that authors who ‘do the right thing’ – by soliciting feedback on their article at conferences or seminars – risk losing the most appropriate, well-informed reviewers as, in conformity with the protocols of blinding, they are obliged to decline invitations to review submissions whose authorship is known to them. Finally, unless submissions are triple-blinded, the impersonality and impartiality celebrated by subscribers to blinding is compromised by editors’ knowledge of the identity of the author(s). Few editors, it seems, are inclined to extend their support for blinding to themselves.
Most, if not all, of the misgivings about unblinding aired above are apparently swept away by (de)personalizing the issue of un/blinding as one in which ‘person’ and ‘article’ can become ‘condensed’, as one reviewer of this essay puts it, with detrimental consequences. Such condensation may occur although, interestingly, another reviewer expressed a diametrically opposed view. As an editor, he had been ‘humbled’ by reviewers who, in effect, embrace ‘condensation’ when they ‘ask to be put in direct contact with the author so that they may offer more assistance. They unblind themselves because they figure they can be more useful unblinded than blinded.’ The antipathy to unblinding is attributed by this reviewer to publishers, as well as editors, who fear, not without justification, that its revelations may damage the carefully guarded prestige and value of their position and assets (Harvie et al., 2012, 2013; see also Macdonald, 2015).
Repeatedly voiced concerns, or fantasies, about how unblinding will result in ‘condensation’ or lead to retaliation by authors may be exaggerated, and not least by elites who, more or less consciously, appreciate the portrayal of horrific scenarios in which the revelation of reviewer reports would risk humiliation or worse; and who, more justifiably, fear that removing the fig leaf of blinding will weaken or compromise editors’ position of autocratic invulnerability as well as reduce the rents generated by journals. With regard to the risks of retaliation from displeased authors, it is relevant to recall that several referees are normally called upon to provide evaluations of manuscripts (or grant applications and appointments). It is therefore likely that retaliatory actions directed at a junior reviewer by an aggrieved senior author will be buffered by such checks and balances. I return to this issue later in the essay.
Ensuring the opacity of evaluation processes, Lipworth and Kerridge (2011: 98, emphasis added) suggest, is the ‘means by which powerful members of the scientific community (i.e. reviewers and editors) maintain their power . . . at others’ expense’ (see also Rennie, 2003) – the expense incurred being the invisible receipt of incompetent, high-handed or even abusive editorial conduct. On this point, I would question Lipworth and Kerridge’s (2011) claim that editors possess the power ascribed to them when, arguably, such ‘power’ is temporarily loaned to editors by publishing houses or learned societies (Parker, 2013). The central point being made by Lipworth and Kerridge (2011), nonetheless, is that routine editorial exercises of power, including those that established and now perpetuate blinded peer review, operate to exclude, discredit and suppress more transparent (e.g. unblinded) alternatives.
In sum, when elevated and venerated as a ‘gold standard’, blinded peer review becomes widely ‘apprehended as an invisible fate, for which the individual [e.g. the editor or reviewer] may disclaim responsibility’ (Berger and Luckmann, 1967: 108). Maintained through editorial ‘control mechanisms’ (Berger and Luckmann, 1967: 78) often imposed by publishers (e.g. non-disclosure agreements; Parker, 2013; Willmott, 2021), opacity in evaluation processes is legitimized not only by its normalization but also by ostensibly ethical principles, such as those of anonymity and confidentiality.
Mea culpa
Prior to being provoked to reflect upon blinding, I had naively accepted it as a ‘self-evident and compelling facticity’ of academic life (Berger and Luckmann, 1967: 37). My unquestioning faith in blinding’s benign necessity was maintained despite, or perhaps because of, my privileged experience of evaluation processes as an editor as well as an author and reviewer. It might have been expected that my experience as an author, my spats with editors over the selection of referees and/or the justification of their decisions would have shaken this faith. As a reviewer, as well as an associate editor of Organization and AMR, I had seen many reviews and decision letters, often well executed but also, and too frequently, leaving much to be desired. Despite this, I had failed to connect my disquieting ‘private troubles’ (Mills, 1959) to the broader ‘public issue’ of how manuscript evaluation processes are shrouded in secrecy. In effect, I had become resigned to, and had normalized, instances of what I regard as malpractice or incompetence – ‘poor review’ by referees and/or editors – as an irremediable fact of academic life. As a consequence, I had given almost no thought to why secrecy surrounds the evaluation of manuscripts or if, on balance, such opacity is defensible. Instead, I had simply internalized (that is to say, swallowed) a key element of an occupational ideology in which, the blinding of reviews self-evidently signals scientific respectability and authority. Faith in blinding had impaired my capacity to apply even the basics of Sociology 101 to reflect upon how, in common with other institutions, blinding is established and defended within ‘[s]hifting power relations and the ethics of journal peer review’ (Lipworth and Kerridge, 2011; see also Dean and Forray, 2019) that their opacity serves to protect and reproduce. Naively, I had not contemplated how ostensibly benign ethical principles may be engaged to protect and legitimize asymmetrical relations of power/knowledge.
A mundane example of how asymmetrical relations of power/knowledge in editorial evaluation processes operate and are reproduced is provided by Starbuck (2003: 344), himself a highly experienced editor, when he notes how ‘editors typically act as if reviewers have more competence and more valid opinions than authors’ (see also Weiskittel, 2015), even though authors have typically spent many months and often years preparing and revising a manuscript prior to its submission, while reviewers and editors may, at best, devote a few hours to its evaluation. A presumption of ‘more competence and more valid opinions’, combined with the protection afforded by opacity, operates, it seems, to empower some editors and reviewers to ‘lord’ it over authors in a manner that, because it goes undetected courtesy of blinding, invites and accommodates instances of coercion and abuse. Whereas signed reviews ‘attach the reviewer’s reputation to the judgment’, anonymous reviews ‘enable reviewers to criticize without fear of negative consequence’ (Osterloh and Kieser, 2015: 310). Other studies have suggested that blinding manuscripts has an unintended consequence of ‘reviewers fabricat(ing) images about authors as they read anonymized manuscripts (Tardy and Matsuda, 2009)’, so that ‘biases’ may be retained or even engendered by ‘double-blind review’ (Starbuck, 2016: 169), rather than diminished by it. Fabrications, Starbuck (2016: 169) continues, ‘include guesses about authors’ employers, genders, intellectual conformity, nationalities, races, and names. Editors and reviewers believe these fabricated images are more accurate than they actually are (Yankauer, 1991).’ This speaks to the risk of false ‘condensation’ (see above) for which unblinding presents a remedy.
Of course, forms of open, unblinded peer review are not without their shortcomings, as noted earlier when acknowledging the existence of Matthew and Matilda effects, and the prospects of retaliation by disgruntled authors. When assessing the pros and cons of open peer review, Wicherts et al. (2012: 8) acknowledge that ‘there is no system without drawbacks’ but they conclude that increased transparency assists in mitigating problems endemic to blinded peer review while bringing benefits (e.g. access to insights contained in reviewer reports) that blinding denies. Given all the ‘drawbacks’ associated with blinding, which I have sought to bring together here, how is it possible that so much ‘good faith’ is invested in opaque review and editorial processes? To answer this question, it is instructive to begin by delving briefly into the history of scientific manuscript evaluation.
A brief history of peer review or how did we get here?
The activity of peer review can be traced in the UK to the work of members of royal academies who were licensed by the state to censor controversial material. Positioned within a particular relation of power to knowledge, the task of ‘peer’ reviewers was to protect the Monarch, and the wider Establishment, by ‘peering’ closely into forms of knowledge that might contest their absolutism (Biagioli, 2002: 32). Today, the equivalent of the Monarch is the editor and ultimately the publishing house who, I contend, like the Sovereign, enjoy protection of their position and assets from unwelcome scrutiny by championing the ethics of opacity (see Atkinson, 1999).
It is significant, but also ironic, that when, in 1665, the Royal Society pioneered the modern form of peer review with its inauguration of the journal Philosophical Transactions, ‘men of science’ were invited ‘to abandon their attachment to secrecy and to submit their work to the judgment of its fellows’ (Eamon, 1985: 344, emphasis added). In this context, secrecy meant lack of openness to consideration and examination by other scientists. Abandonment of such secrecy required that reviews were undertaken by small teams of journal editors who were readily identifiable by authors of submissions and so, in effect, were open (Burnham, 1990). Reviewer reports were treated as confidential only from the early 1830s (Moxham and Fyfe, 2018), but ‘open’ review – in the sense that the small circle of reviewers was known to authors and vice-versa – continued until the mid-20th century.
After the Second World War, there was a rapid expansion and specialization in scientific knowledge, including the social scientific study of management and organization. The resulting increase in the volume of manuscripts placed reviewing by comparatively small editorial teams under mounting pressure. At about the same time, advances in reprographic technologies made it comparatively cheap and easy to distribute submissions (mimeos) to external evaluators (ad hoc reviewers). Securing the time and effort of ad hoc reviewers, who crucially did not enjoy the profile and influence of members of the small editorial teams, was obtained through the quid pro quo of anonymizing, or blinding, their reports. For reviewers, anonymized access to manuscripts presents an opportunity to frustrate or block, as well as support and advance, particular lines of inquiry without being identifiable, or accountable, for their evaluations. For editors, outsourcing reviews contributes to the discombobulation of editorial responsibility as it enables decisions to be obscured through a judicious selection of ad hoc ‘expert’ reviewers, and by making selective use of their reports.
For members of editorial teams, the outsourcing of reviews also brings the substantial benefit of reducing their workload, with minimal loss of editorial status or prerogative. Commenting on this interpretation, a reviewer of this essay expressed the view that it would take an editor less time to evaluate a manuscript than to handle the process of its outsourcing to a reviewer. This view, I suggest, presumes that the editor (or associate editor) is genuinely a ‘peer’ for all the manuscripts that s/he handles – in the sense of already being sufficiently familiar with the domain and contents of submissions to provide an informed, developmental reviewer report. Given the level of specialization within MOS, I believe that, in most cases, editors would have to undertake a considerable amount of additional, specialist reading in order to assess a manuscript’s originality and potential contribution. When acting as associate editor at AMR, I can say that it would have taken me many hours, and perhaps days, to prepare anything approaching a peer review of the heterogenous manuscripts allocated to me.
A key role of editors is to decide whether a manuscript should be reviewed and, ultimately, published. Editors may therefore decide to reject manuscripts that receive positive reviews or vice-versa (see, for example, Haier et al., 2014a, 2014b); they may desk reject manuscripts without subjecting them to review by specialist peers; and they may also seek further reviewers in support of a desired (negative or positive) decision and so on. All such actions may be defensible, but this conjecture cannot be assessed in the absence of visibility. As noted above, publishers have a vested interest in defending opacity in order to retain control of the highly lucrative academic journal market. As if in anticipation of challenges to the dominant system of blinded manuscript evaluation, major publishers subscribe to the Committee of Publication Ethics (COPE), funded by them, to which disgruntled authors are invited to submit their grievances (see MacEachern, 2018). COPE, it has been suggested, provides ‘an ethical mask’ for ‘powerful editing and publishing corporations’ (Teixeira da Silva, 2017a: 11). Moreover, it lacks ‘rules of engagement’ that, in principle, would enable authors to ‘challenge editors or publishers’ (Teixeira da Silva, 2017a: 12) about possible editorial misconduct that breaches its (imprecise) guidelines (see also Teixeira da Silva, 2017b).5
It is possible to acknowledge, as noted earlier when considering the Matthew and Matilda effects, that blinding may, on occasion, provide individual authors with some measure of protection from forms of discrimination and retribution associated with the revelation of affiliation, gender, reputation and so on. Of greater systemic significance, however, is the contribution of blinded peer review to the concealment of possible malpractices; and its frustration of accountability in ways that are beneficial to the powerful – publishers, editors, reviewers – but detrimental to the development and publication of more diverse, creative and relevant forms of research and scholarship.
Power and ethics in knowledge warranty
A mutually beneficial settlement between publishers, editors and reviewers has cemented blinding as a central plank of ‘good’ editorial practice, with its primary justification being the anonymization and protection of the identity of the reviewer (single blinding) and/or the author (double blinding), and very exceptionally the editor (triple blinding). Peer review that is blinded, as Fuller (2000: 87, emphasis added) notes, ‘miraculously discriminates important from unimportant research as the overall result of a set of privately taken decisions in referee’s reports’ whose opacity publishers as well as editors and reviewers are more than content to maintain. This entrenched arrangement has, arguably, enabled, and perhaps encouraged, editors to present and conceive of themselves as benevolent guardians, rather than unaccountable overlords, of scientific fields (Wellington and Nixon, 2005) – as displayed in self-congratulatory ‘Meet the Editors’ and ‘Manuscript Development’ events hosted by journals and/or their publishers (see Parker, 2013). Reflecting upon the role of reviewers in relation to editors, Zuckerman and Merton (1971: 96, emphasis added) ruefully comment: . . . the corps of typically anonymous referees sometimes serves the incidental and not altogether latent function of protecting the highly visible editor from the wrath of disappointed authors. But what is helpful for the editor can of course be injurious to the author.
When voiced by Merton, a bastion of academic conservativism (except, it seems, when writing with his spouse), this assessment conveys a sceptical understanding of the application of ethical principles of anonymity and confidentiality in the blinding of peer review. It suggests how, through the operation of asymmetrical relations of power, anonymity is lent to reviewers in ways that are facilitative and beneficial for editors, but can be ‘injurious’ for ‘disappointed’ authors – an assessment echoed more recently by Dallyn et al. (2015: 1033) in their reflections on how academic journal publishing ‘is embedded in a complex power relation between author, peer-reviewer and editors’.
As an instrument of power, blinding spares the conduct of reviewers and editors – whether conscientious, instructive and collegial, or inept, corrupt and useless – from scrutiny. That such asymmetries of power are institutionalized should not be surprising (Boden et al., 2009), but neither should their normalization go unquestioned or unchecked. Consider then how, in a research context, the operation of ethical principles of anonymity and confidentiality can protect private interest. Van Maanen (1983) ‘confesses’ how, when undertaking fieldwork on police work, he felt an ethical obligation to anonymize his research subjects’ identity. This, paradoxically, led him to shield two white policemen from due legal process by declining to testify as a witness to their beating up of a black homosexual man. I applaud Van Maanen’s openness, courage and self-effacement in sharing this experience; and I suggest that it could be equally instructive for editors and reviewers to report comparable episodes. What I question is whether Van Maanen’s pledge of secrecy was justified or defensible when it is probable, given the nature of his ethnographic study, that some other, ‘higher’ ethical demand (e.g. of justice) might compel him to suspend or override such a pledge (see Bok, 1983: 25–26; see also Calvey, 2008). Remaining silent in pursuit of his ‘private interest’ (of preserving access to the field) gave protection to persons (e.g. racist, homophobic policemen) occupying positions of comparative privilege. No less worryingly, it allowed violence against a minority group to be concealed and perpetuated. As this example shows, prioritizing the protection of individuals (e.g. researchers, informants) can be problematical for the wider community when its effect is to hide issues of public interest under a cloak of secrecy (Baez, 2002b). This logic is no less applicable to the blinding of peer review where ethical principles are invoked to deny accountability, and thereby conceal malpractices, in the name of protecting authors and reviewers from possible instances of ‘bias’ and retribution.
With specific respect to editorial practice, I have been asked to respond to reviewer comments that the previous draft of this essay was (and may, in their eyes, remain), ‘overly critical of current journal editors’ when, for example, I describe them as ‘autocratic’ (see above) and draw attention to the ‘negative ways that editors can skew the review process’. I will address the ‘autocratic’ claim shortly. With regard to ‘skewing’, one of the reviewers of this essay acknowledges that it may happen, but then writes that: ‘I really think that editors today are very attentive to selecting appropriate reviewers for manuscripts (not people who they believe are going to give a particular evaluation)’ and that ‘the majority of editors see that they have a responsibility to ensure that the author not only gets a fair review of their work, but also gets helpful developmental comments and suggestions for improvement’. In my recent (past two to three years’) experience – on which I am necessarily obliged to rely because, of course, ‘closed’ processes of manuscript evaluation make all other reviews and editorial decisions inaccessible to me – I would say that I have received informed and ‘fair review[s]’ with ‘developmental comments and suggestions’ in about 50–60% of cases, with perhaps another 20% being broadly adequate. Feldman (2005: 654) reports similar figures. Shockingly often, however, I have received reviews that are casual, floundering, superficial, tunnel-visioned and even outright abusive. I call this shocking because, in these cases, editors did not hesitate in sending out obviously inadequate or insulting reviews bereft of ‘developmental’ content. Perhaps reviewers and editors lack the time or the inclination to read papers sufficiently closely before confirming their agreement to provide a review, or when they prepare decision letters.6 I hope that my experience is exceptional, even if it is not unique (Feldman, 2005), but unless evaluation processes are opened up for scrutiny, it is impossible to know.
Turning now to the questioning of my characterization of editors as ‘autocratic’, my response is that currently editorial practice is systemically autocratic: by and large, and however personally well intentioned editors may be, they exercise absolute authority; their decision making is opaque; and they are unaccountable for their decisions. In my role as an associate editor at Organization and AMR, I too was inescapably autocratic because ‘absolute authority’ comes with the editorial territory.
That said, let me underscore how an attentive, well-informed editor who selects reviewers carefully (and not on the basis of a belief that they will ‘give a particular evaluation’, Reviewer comment on this essay) is as invaluable as a well-informed, diligent, developmental reviewer. On many occasions, editors and reviewers have saved me from my own ignorance and stupidity, including ill-judged claims made in earlier versions of this essay. Nonetheless, beyond that, great editors can make a significant difference to our field by actively seeking out and encouraging authors or manuscripts that are ‘rough diamonds’; and when they shepherd authors through as many rounds of review as may be necessary, rather than subjecting them to the widget-producing discipline of ‘two strikes and you are out’. I therefore gratefully acknowledge the immense benefit that I have personally gained from editorial and reviewer attentiveness. We do not do enough, I believe, to recognize and celebrate editors and reviewers who make an outstanding difference to the development and our field. I would therefore like to pay tribute to the work and inspiration of three exemplary editors: the late Anthony Hopwood, founding editor of Accounting, Organizations and Society; David Cooper, founding editor of Critical Perspectives in Accounting; and Karen Legge, co-editor of Journal of Management Studies. I doubt, for example, that ‘Strength is ignorance’ (Willmott, 1993), a highly cited article, would have seen the light of day without Karen Legge’s close interest, encouragement and guidance.
Let me state unequivocally, then, that I consider editorial work to be both crucial and challenging. It is crucial for ensuring confidence among readers of the journal, as well as authors who intend to submit manuscripts, and who hope to receive reviews that enable them better to articulate, clarify and amplify their distinctive contribution. Editorial work is also challenging. That is not least as a consequence of commercial and peer pressures (e.g. raising the impact factor of the journal; facilitating or frustrating the career progression of prospective contributors) to comply with established norms (e.g. deference to established patterns of publication; see Harley and Fleming, 2021). The job of editors is particularly demanding when such ‘challenges’ are imperfectly aligned with ‘crucial’ elements of the editorial process.7 I hope, therefore, that these brief reflections are sufficient to convey to readers of this essay, including its reviewers, that I am deeply respectful and admiring of editors who, for example, seek out and nurture ‘rough diamonds’; and who give careful thought to the selection of reviewers and the interpretation of their reports. I hold in particularly high regard editors who willingly discuss their decisions and/or establish and support meaningfully independent processes of appeal.
Power in manuscript evaluation
I have argued that manuscript evaluation, like many other social practices, is organized within (asymmetrical) relations of power through which conventions, such as blinding, are institutionalized. I now seek to elaborate this claim by engaging Lukes’ (1974, 2005) analysis of power, comprising his identification of three dimensions that encompass: the most readily identified expressions of power (e.g. overt decision making); exercises of power that are mainly hidden (e.g. background agenda setting); and, finally, institutionalized power that is least visible and most insidious (e.g. the mundane reproduction of dominant frames of reference).
An example of the first of Lukes’ dimensions is editorial effort to ensure the anonymity of reviewers and/or authors, ostensibly in order to protect them. Evaluation processes are thereby shielded from examination in a way that might be construed as broadly equivalent to maintaining the confidentiality of patients’ medical records. In the case of medical records, however, confidentiality is justified in terms of reducing the risk of their criminal use (e.g. for purposes of blackmail). Nonetheless, and equally, as Costas and Grey (2014: 1425) have observed, confidentiality of records may be institutionalized for the self-serving purpose of escaping accountability. Turning to the second, agenda-setting, dimension of blinded peer review processes, this is exemplified in the contents of editorials where the necessity and benefits of anonymity and confidentiality are taken for granted, thereby downplaying, if not entirely excluding, recognition and discussion of other, more open processes of reviewing. Finally, Lukes’ third dimension of power is found in, for example, the entrenchment of the (background) assumption that blinding is necessary if the enterprise of science is to operate productively and flourish. When blinding is institutionalized, those ‘subject to it are led to acquire beliefs and form desires that result in their consenting or adapting to being dominated’ (Lukes, 2005: 13). Authors, reviewers and editors learn to normalize and cherish processes of blinded peer review when we, as a community, focus primarily upon its promised protection of us as individuals, rather than upon its systemic effects for the ‘republic of science’ (Polanyi, 1962). It is notable how discussions of (blinded) peer review rarely consider the second, let alone the third, of Lukes’ dimensions of power. This silence on the problematical, systemic effects of opacity in manuscript evaluation processes is interpreted, from a Lukesian perspective, as expressing and sustaining hierarchies of dis/advantage: ‘Hidden power arrangements are maintained by secrets – the secrets of those who might benefit from those arrangements and the secrets of those who might be victimized by them (because they fear retaliation)’ (Baez, 2002b: 52, emphases added; see also Baez, 2002a). The secrecy, exemplified by blinding, that shields members of the scientific elite from scrutiny may, however, also provoke suspicion and dissent. While the relative dependence of authors on editors and reviewers tends to deter probing of the necessity and/or the ethics of blinded peer review, it may also stimulate questioning – as illustrated by Zuckerman and Merton’s (1971: 96) call to ‘enlarge the accountability of referees by removing their cloak of anonymity’ and by Honig’s (2016) blog cited earlier (see also Bernard, 1991).
To defenders of blinding, the infrequency of public challenges may be construed as an indicator of its benign necessity and efficacy. Alternatively, and when guided by Lukes’ analysis, their rarity may be symptomatic of how occupants of editorial roles, supported by publishers, define agendas and dominate the air space around questions of journal evaluation processes. Editors (and publishers) are heard, routinely and insistently, to repeat the mantra that the greatest risk facing individual authors is the ‘unintended bias of reviewers who know the seniority, gender, or nationality of a paper’s author’ (see https://authorservices.taylorandfrancis.com/peer-review/ (accessed 25 March 2020)); and they repeatedly reinforce the received wisdom that blinding is key to mitigating this risk. Rarely is reference made to the systemic risks for the development of disciplines that result from blinding, or to the pathologies associated with its institutionalization, as detailed earlier. Scant consideration is given to how asymmetries of power relations institutionalized in blinded manuscript evaluation processes simultaneously foster and obfuscate dysfunctionalities. These include editors’ and publishers’ dismissive or ‘stonewall’ handling of author grievances that Atkinson (2001: 196) has characterized as ‘power play[s]’ expressed as ‘impatience, inattention, bluster and self-assertion’. Dysfunctionality is also reflected in, and perpetuated by, the rarity or invisibility of independent appeal or arbitration procedures. In 2013, only two out of 20 ‘top-tier’ journals in our field had grievance process statements and only one had contact information other than an editor for raising grievances (Clair, 2015b: 127).
Opening the black box of manuscript evaluation
The discipline-forming, or deforming, aspects of editorial work become apparent when, for example, manuscripts prepared by heterodox/orthodox scientists are sent for review and evaluation by orthodox/heterodox reviewers (see Alvesson and Sandberg, 2014; see also Bedeian, 2008; Willmott, 2021). Such (mis)matching can hardly be counted as peer review. It occurs when, for example, editors select reviewers who are believed to share authors’ interest in a particular topic, or because the software registers a match of interest based upon the submission’s keywords. In each case, little account is taken of the divergence, and perhaps incommensurability, of theoretical and/or methodological orientations between the authors and reviewers (Bristow, 2012; Özkazanç-Pan, 2012; see also Frey, 2003). Occasionally, the automation of reviewer selection can work in an author’s favour, as when the software selects the author to act as their own reviewer!8 In general, however, deficits of meaningful peer review are concealed by blinding; and this dysfunctionality is compounded when editors or reviewers have been trained to believe that all scientific knowledge develops along a single path and/or that it shares a single format in order to demonstrate its (incremental) contribution. The invisible, but painfully experienced, outcome for the orthodox/heterodox author is receipt of reviewer reports that are a poor fit for the submitted manuscript, accompanied by a correspondingly informed editorial decision.
In such instances, ‘peer’ review is a risible misnomer: the orthodox/heterodox reviewer is not, in any meaningfully scientific sense, a ‘peer’ of the heterodox/orthodox author. Rather, s/he is a potential antagonist. At best, mismatched reviewer reports are a tiresome distraction – a mismatch that is, of course rendered invisible by blinding, except to the author, of course! This experience of being ‘boxed in’ (see Alvesson and Sandberg, 2014) is also a case of being ‘mis-recognized’ – in the sense of being ‘prevented from participating as a peer in social life . . . as a consequence of institutionalized patterns of interpretation and evaluation that constitute one as comparatively unworthy of respect or esteem’ (Fraser, 1997: 280). Some commentators have gone so far as to claim that the institutionalization of mis-recognition is the ‘secret purpose’, or the purposive secrecy, of editorial gatekeepers, arguing that opacity enables them, whether intentionally or not, to ‘exercise “power as discipline” to control the distribution of acceptable knowledge’ (Bedeian et al., 2008: 213, emphasis added). Without necessarily reaching that dystopian conclusion, I have conjectured that the operation and effects of such exercises of ‘control’, with regard to the development and diversity of ‘acceptable knowledge’, is concealed. The comparative power and privilege of scientific elites is thereby protected in the (patronizing, self-serving) cause of offering protection to the vulnerable. Conversely, a more open approach to manuscript evaluation is consistent with the avowed commitment of science to ‘open communication and deliberation’ (The Royal Society, 2012: 13). In sum, shielding editorial practices from scrutiny is difficult to reconcile with valorizing ‘open communication’.
Opening (peer) review
So far, I have alluded repeatedly, if tangentially, to open (peer) review, which I now address directly. ‘Open review’ is an ill-defined umbrella term that encompasses numerous variants (Ford, 2013; Ross-Hellauer et al., 2017): open identity when the identity of authors, reviewers and editors is disclosed; open inspection when the review process and the reviews are available for public scrutiny; and open participation when reviewers may volunteer to provide comments, rather than be selected by editors or recommended by authors (see Schmidt et al., 2016; also Mirowski, 2018; Pöschl, 2004). My preferred version of open review conceives of it as at least partially inclusive of the first and second of these elements, with the third element (open participation) excluded. Specifically, in this version, there is no blinding of the identity of the author, reviewer or editor, or their unblinding is voluntary; the review process is made transparent following the editorial decision concerning publication of the manuscripts; and editors select the reviewers, rather than relying upon the crowdsourcing of reviews. In short, this version of open review is modest, and by no means radical.
Enter open review
Increased interest in open review during the 1990s (Tennant et al., 2017) coincided with the availability of digital technologies and web platforms that enabled scientists to publish their research rapidly. Almost exclusively the preserve of the natural sciences, open review has provided authors with a means of declaring themselves to be the ‘first movers’; and it has enabled them to obtain feedback from numerous sources without relying upon the comparatively slow, secretive and uneven process of blinded peer review. Notably, ArXiv, launched in 1991 but now discontinued, was a public server repository hosting e-prints of research in the fields of physics, mathematics, computer science, quantitative biology, quantitative finance, statistics, electrical engineering and systems science, and economics. Deposited manuscripts were typically pre-publication versions of research papers. Submissions received a brief editorial check to ensure their fit with one of ArXiv’s fields before being posted on-line for anyone to view and provide reviews. Thirty years later, there are numerous open access platforms and journals that allow comment both before and after publication (e.g. PeerJ – https://peerj.com/) and/or that make peer review processes fully available (ScienceOpen – https://www.scienceopen.com/). Others include BioMed Central and eLife as well as Nature, which is considered in more detail below. A fairly typical example of the kind of ‘Instructions to Authors’ found in such journals is that given by Royal Society Open Science (published by the Royal Society): Royal Society Open Science operates open peer review on all manuscripts. At all points in the peer review process we encourage referees to waive their traditional right to anonymity and sign their reports, thereby disclosing their name to the author. However, this will remain voluntary and anonymity will be strictly maintained if requested. To increase the transparency of the peer review process, we publish referee reports, the substantive part of the decision letter after review, and the associated author responses alongside published articles. By agreeing to become a reviewer, you are agreeing to the publication of the report alongside the article should the article be accepted. (https://royalsocietypublishing.org/rsos/for-reviewers, emphases added [accessed 6 June 2020])
In a study of four open peer review journals (PLoS One, Atmospheric Chemistry & Physics (ACP), PeerJ and F1000Research) – selected to reflect differences in age, perceived stature and the information about open peer review presented on their websites – Ford (2015) highlights their diverse policies towards greater ‘openness’. She notes that, in the case of ACP and PeerJ, reviewers are strongly encouraged to disclose themselves, but they have the option of being anonymous (as is also the case for Royal Society Open Science), unlike the other two journals where disclosure is mandatory. F1000Research (in which Ford’s article appears) is, she suggests, the ‘gold standard of transparent open peer review’ because the review process is editor-mediated (rather than crowdsourced) and because, for example, F1000Research suppresses from search results manuscripts receiving unanimously negative reviews. Of particular relevance to this essay is Ford’s (2015) call for further research into processes of open access reviewing within non-open access journals, including those outside the science, technology, engineering and mathematics (STEM) disciplines. In MOS, there are a number of innovative, well-established, high-quality (in my opinion!) open access journals (e.g. Ephemera, M@n@gement) but few if any, as far as I am aware, have yet incorporated some version of open peer review.9
Examples of open review: Nature and the British Medical Journal
A significant, comparatively recent development (February 2012) has been the piloting of open peer review at Nature (which, as one of the reviewers of this essay reminded me, only introduced peer review in 1976). First published in 1869, Nature is a multidisciplinary scientific journal whose primary audience is research scientists. It was the most cited scientific journal in the 2018 Journal Citation Reports with an impact factor of 43.070. By such measures, that figure makes Nature one of the world’s top academic journals.10 Once a manuscript is accepted and ready for publication prospective authors are offered the option of anonymous referee reports being published, along with their own responses and rebuttals. Referees may also opt to be named, which many of them do. The justification for trialling open peer review at Nature is that ‘[r]esearch communities are unanimous in acknowledging the value of peer review’ and that ‘there’s a growing desire for more transparency in this process’ (Nature, 2020). ‘Researchers’, the Editorial continues, ‘want to see how publishing decisions are made, and they want greater assurance that referees and editors act with integrity and without bias’ (emphasis added). This desire reflects a concern that blinding may undermine, rather than safeguard, ‘integrity’; and also, that blinding may accommodate rather than reduce ‘bias’ – a view that inverts the established mantra, or eulogizing, of blinded peer review.
The decision to trial open peer review at Nature followed a positive experience at Nature Research journals.11 It is, nonetheless, acknowledged that some researchers believe that, without blinding, reviewers will be deliberately less critical or softer as they may make the calculation, correctly or not, that, as authors, they will receive reciprocal treatment when their own submissions are evaluated. The Nature Editorial also acknowledges reviewers’ concern that the transparency of open review places pressure on reviewers to provide outstandingly good reports. These and other apprehensions are addressed by referring to the experience of open peer review gained at the sister journal, Nature Communications. At that journal, ‘[t]he vast majority of our reports are already written in a professional and constructive manner – greatly enhancing the integrity of our research papers – so we do not want our reviewers to change what they do’ (https://www.nature.com/articles/d41586-020-00309-9 [accessed 2 April 2020]). It is also appreciated in the Nature Editorial that the introduction of open peer review may, initially, be personally troublesome as it exposes review(er)s to scrutiny beyond the tiny, closed circle of author, editor and, sometimes, other reviewers, of a manuscript. Indeed, there seems to have been some suggestion of this ‘trouble’ in the evaluation of this essay as two of the reviewers withdrew their agreement to publish their reports. Nonetheless, set against the argument for retaining blinding in order to avoid such ‘trouble’, there is, as I have argued, the wider public issue of its concealment of forms of malpractice, abuse and fraud that drains public confidence in how scientific knowledge is warranted.
The British Medical Journal (BMJ) provides another high-profile example of open review. BMJ now makes available on the web the full pre-publication history of its research articles. This history includes all previous versions of the manuscript, the report from the manuscript committee meeting, the reviewers’ signed comments and the author’s/authors’ responses to all the comments from reviewers and editors. This policy was introduced with awareness of its accompanying challenges, including an anticipation of it presenting difficulties for early career reviewers (defined as under 40) who, it is noted, often provide the ‘best opinions’ as reviewers but who may be ‘reluctant to criticize the work of senior researchers for fear of reprisals’ (Smith, 1999: 5). Notwithstanding such concerns, open peer review at BMJ is, on balance, assessed to have more (ethically) positive effects as it ‘reminds the reviewer that with power comes responsibility’; it ‘links accountability with credit’; and it ‘should eliminate some of the worst abuses of peer review, where reviewers – under the cloak of anonymity – steal ideas or procrastinate’ (Smith, 1999: 4; see also Smith, 2006). The considerable challenges associated with greater openness – including loss of protection afforded by anonymity and confidentiality (e.g. in relation to junior researchers acting as reviewers of senior authors’ manuscripts, and biases preserved or occasioned by disclosure of the names of authors and/or reviewers) – are, it is argued by the editor of BMJ, outweighed by the systemic benefits of more open peer review.
This positive assessment of open peer review does not, in my view, deny that, in some circumstances, concealing identity under a cloak of anonymity may be justified. For example, anonymity is more readily defensible when a problem – such as malpractice or lack of accountability by specific editors and/or specific reviewers – is framed, as far as is practically possible, namelessly as a public issue, rather than personalized by presenting it as the individual failure of named editors or reviewers (see Mills, 1959). The focus of attention is then primarily on the systemic dysfunctionality of blinding and the broader impact of secrecy in the adjudication of scientific knowledge claims, rather than upon the possible malfeasance of individuals who, for example, are suspected of abusing, or gaming, an institution whose designed-in opacity accommodates and protects incompetence, misconduct and wrongdoing (Eamon, 1985; Rosenzweig, 1985).
By opening peer review, malpractices and violations are more readily detected, challenged and deterred. Openness makes it more apparent when, to give a couple of examples, suggestions provided by reviewers are appropriated by authors without due (if anonymous) acknowledgement; and when articles become ‘ghostwritten’ by editors and/or reviewers (see Bedeian, 2008; Schaffalitzky de Muckadell and Petersen, 2017; Wicherts et al., 2012). More positively, open review provides recognition of the time and effort devoted to preparing diligent and developmental reviews, and also of the receipt of editorial guidance consistent with the declared intent of the manuscript. By providing this recognition, open review is also responsive to a deepening and pressing crisis of recruiting careful and committed reviewers. By appreciating, and thereby easing, the burden shouldered by public-spirited, conscientious reviewers, more open peer review can enhance our collective capacity to perform the critical and exacting task of evaluating and warranting scientific knowledge claims. Finally, opening up editorial processes to inspection enables authors to make more informed decisions about where to submit their work.
Discussion
The premium placed on secrecy in peer review, and associated resistance to greater transparency, is somewhat perplexing when it is recalled that editors sign desk rejection letters as well as the ‘Dear John’ (or Joanna) letters communicating the unwelcome news of a manuscript’s rejection, often after authors have engaged in numerous arduous rounds of revision, and in response to encouraging reviews (Cederström and Spicer, 2017). As Hull (1985: 4) comments: ‘if scientists feel free to sign even the most negative book reviews, then why are [reviewers] reluctant to sign their referees’ reports?’ Perhaps some reviewers are not necessarily so ‘reluctant’ to acknowledge authorship, as demonstrated by the positive responses received by the editor to the invitation to referee this essay (but see my reflections in Supplemental Material B). Yet, currently, ‘top-tier’ MOS journals do not require, or even invite, reviewers to disclose their identity. Indeed, reviewers are actively discouraged, or prevented, from doing so. As a consequence, and like me, reviewers may not even contemplate any alternative to accepting the norm of blinding – whether passively and unreflexively, willingly and thankfully, or grudgingly and compliantly.
When the blinding of peer review is sanctified as a ‘gold standard’, authors/reviewers/editors are inclined to presume that it is self-evidently ethical, with the consequence that we immunize our evaluation practices from critical self-reflection. Conversely, when evaluation processes are opened up to collective self-examination, it becomes possible to appreciate and respect the perspectival nature of reviews, and to place in question the inadequacies attributed to manuscripts by ‘peer’ reviewers and editors who do not comprehend, let alone share, the author’s perspective. In such ways, editors as well as reviewers can become more ‘accountable and sensitive to their own forms of partiality’ (Lee et al., 2013: 11). In this process, the disclosure of reviewer identities can, I suggest, be helpful for removing the guesswork from determining ‘the particular theoretical, methodological, disciplinary, and cultural perspective from which the review is written’ (Lee et al., 2013: 11). In such ways, greater openness can, I submit, contribute to the development of a more informed and mature understanding of processes of manuscript evaluation that, in turn, can foster the development of more ‘refined critiques and responses’ (Lee et al., 2013: 11).
When de-normalized and debunked, blinding is reframed as expressive of a kind of spurious virtue-signalling that, more or less intentionally, distracts attention from how opacity cannot, for example, ‘remove biases against unconventional methodology, radical new ideas . . . or results that contradict a reviewer’s viewpoints’ (Benos et al., 2006: 149). Nor, in the absence of a triple bypass to include editors, can blinding remove the ‘halo effect’ or ‘dunno effect’ that plays into the process of allocating reviewers. Blinding nurtures complacency and casualness as it presumes a capacity to eliminate, or very substantially reduce, forms of bias, while disregarding how it has the effect of concealing it. In this regard, the observation of a past editor of Administrative Science Quarterly on organizational ‘wrongdoing’ is instructive. Where engagement in questionable practices becomes normal or institutionalized, he writes, persons ‘often embark on organizational wrongdoing under the influence of their immediate social context’ (Palmer, 2013: 12). In the context of manuscript evaluation – encompassing reviewer selection, report writing, processes of manuscript revision, and editorial guidance and decision making – academics may become drawn into forms of ‘wrongdoing’ whose legitimacy and defensibility is provided by its taken-for-granted normalization. In conditions where the desire to secure an appointment, gain tenure or be promoted is compounded by commercial pressures to raise journal impact factors, there are ‘incentives for cutting corners and, worse, misconduct’ (Jubb, 2016: 20). Then, in this context, the entrenchment of blinding leads its defenders to deem formal, procedural compliance with blinding as sufficient to ensure the minimizing of possible wrongdoing and/or bias. From the standpoint of supporters of more open review, in contrast, the application of the ostensibly ethical principles of anonymity and confidentiality is seen to harbour and hide institutionalized wrongdoing and normalized ‘bias’ inasmuch that it endorses the shielding of elite decision-makers from scrutiny and accountability, as illustrated by the example of Van Maanen’s (1983) ‘confession’ considered earlier.
When any form of dysfunctionality in an institution is encountered, it is not uncommon to seek a way of ‘integrating it into what is already unproblematic’ (Berger and Luckmann, 1967: 38). As an author, I doubt that I am alone in responding to pathologies of manuscript evaluation by treating editorial decision making as something of a lottery (Cole et al., 1981) – with the obvious difference that drawing lots is more transparently random. As I observed earlier, I have, on occasion., been sufficiently incensed, aggrieved and uncalculating (that is to say, naive) to challenge reviewers’ comments and question editorial decisions only to be stonewalled in the absence of independent appeal procedures (Clair, 2015b). My individualistic response has been to fall back on a cynical stratagem: place a bet on some other journal in the hope that my manuscript’s ‘number’ will be called. As Clair (2015a: 167) comments, such responses express rather ‘negative attitudes’ that ‘increase the likelihood of scientific misconduct’ (see also Glynn, 2018). Other consequences, I submit, are a displacement of reflection on the necessity, and consequences, of the opacity of blinded peer review processes; and, relatedly, an impairment of the critical, kynical capacity (Sloterdijk, 1988) to contemplate the possibility, and then to press for the development, of alternative (e.g. more open) forms of evaluation.12
Drawing attention to the disconnect between the principle of openness and the practice of opacity does not necessarily imply a denial or dismissal of ethical principles that may, for example, ‘decrease the violation of participants’ [i.e. research subjects’] rights and increase our accountability and true obligation to them, to self, and to the professional community’ (Karnieli-Miller et al., 2009: 287; see also Resnick, 2006). However, simply demonstrating (procedural) compliance with ethical principles risks narrowing and impairing the sensibility of researchers who might otherwise give priority to a wider range of stakeholders and their concerns, as noted in the Van Maanen example given earlier. Researchers might, for example, be more inclined to protect ‘participants’ rights’ as citizens on the basis that citizens potentially derive benefit from the broader public interest protections afforded by increased openness in all aspects of scientific knowledge production, including processes of manuscript evaluation.
Given the focus of this essay upon the pathologies and paradoxes of blinded peer review, it is relevant to acknowledge and address head-on the inconvenient fact (for me) that many authors, as well as reviewers and editors are supporters of opacity (Bornmann and Mungra, 2011; Ware, 2008). Empirical evidence on the strength of such support is thin, generally dated, and somewhat mixed. For example, a survey undertaken by Snell and Spencer (2005) reported that 74% of reviewers for Medical Education indicated a willingness to sign their reviews. However, most studies indicate a significantly lower figure closer to 50% or less in favour of signing (e.g. Molero and López-Santoveña, 2001; Regehr and Bordage, 2006; Ross-Hellauer et al., 2017). Considering the number and extensiveness of the shortcomings ascribed to blinding in this essay, the strength of support for blinding may seem surprising, and perhaps irrational. What explanation for this anomaly can I offer?
Recall the earlier consideration of Lukes’ three dimensions of power. When the evidence of support for blinding is situated within a context of asymmetrical relations of power between authors, reviewers and editors, the preference for opacity is heard to express and affirm a dominant discourse. In this discourse, anonymity and confidentiality are (commonsensically) associated with protecting the vulnerable (e.g. authors), and not with securing the domination of the institutionally privileged (editors and reviewers). This entrenched sensemaking is plausible and appealing from the standpoint of academics as individuals who, when submitting a manuscript, wish to receive ‘unbiased’ reviewer evaluations and editorial decisions as well as to avoid retribution. However, when reframed within Lukes’ three dimensions of power, the anticipated (or fantasized) protection of individual authors from anticipated malpractice and wrongdoing is seen to reflect and reinforce systemic power asymmetries by ensuring that editors and reviewers are shielded from scrutiny and accountability. When situated in this context, and taking into account that blinding is widely regarded an integral element of peer review, it is not so surprising that surveys show a majority of academics being supportive of its retention.
More open forms of peer review cannot, of course, entirely eliminate editorial deficiencies and malpractices (e.g. failings of editorial judgement when selecting reviewers and/or interpreting their reports). Nonetheless, open review makes such shortcomings (e.g. sloppiness, hostility and ignorance as well as fabrication and fraud) more accessible to detection and thereby operates as a deterrent as well as lending impetus to their remedy. More open peer review can, for example, show how authors can feel pressurized to satisfy, or appease, editorial demands (Atkinson, 2001) by responding ‘positively’ (that is to say, deferentially) to all manner of casual or ill-conceived ‘advice’. Openness can let in some ‘sunshine’ to reveal when processes of manuscript evaluation are systemically ‘stupid’ with regard to advancing the development of science, yet they are also accommodated and perpetuated as they are ‘functional’ for preserving the status quo and safeguarding the publishing machine (Alvesson and Spicer, 2012). Greater openness can, for example, bring visibility to how, in the process of responding to reviewers and editors, the distinctive voice and contribution of the author can be stifled, distorted and ventriloquized, rather than better articulated, clarified and amplified.
Of course, when open review allows authors to respond publicly to reviewer reports, there is a risk that some prospective reviewers will shy away from such unwelcome symmetry by declining to serve as reviewers, or by withholding or withdrawing their permission to make their reviews public (see Supplemental Material B). The unsettling prospect of author responses to referee reports becoming accessible to scrutiny may also be resisted more surreptitiously. For example, it may be claimed that an open process of manuscript evaluation will render reviewer reports anodyne, without contemplating that greater openness might also encourage (even) more careful and considered preparation of developmental reports. If reviewers believe in their reports, why should there be any problem with addressing and, if necessary, challenging author responses? It is, of course, likely that increased openness will, on occasion, be uncomfortable, or perhaps embarrassing. The publication of exchanges between authors, reviewers and editors may, sometimes and sadly, sour relations when colleagues struggle with unwelcome dialogue and accountability. Nonetheless, the recurrent and monotonous emphasis upon possible troubles of open review may, I suggest, also be symptomatic of a particular and problematical culture of scholarship in which the ego-protection provided by secrecy is prioritized over the openness that, arguably, defines scientific knowledge, and that is key to its development. The challenge, then, is to press for reforms of scientific practice, including evaluation practices, that increase openness, and thereby strengthen institutional and individual capacities to receive and learn from criticism, rather than defending a culture of scholarship that shrinks from its expression. Finally, the editor has asked me to offer some reflections on the present ‘experiment’ in open review. These have been placed in Supplemental Material B.
Conclusion
It is paradoxical and perverse that, on the one hand, science is identified as a uniquely open institution; and, on the other hand, that manuscript evaluation processes are opaque and unavailable for inspection. Blinding is regarded as integral to the ‘gold standard’ of peer review that is intended to transcend, or at least neutralize, the intrusion of politics into processes of manuscript evaluation. Here, in contrast, these processes have been framed as a medium and outcome of asymmetrical power relations wherein opacity is seen to shelter elites – editors and publishers – from ‘sunshine’ and scrutiny. Openness is unwelcome, I conjecture, as it threatens to destabilize the status quo of learned journal publishing: ‘closed review’ protects the imprimatur of the peer reviewed journal that is so consequential for its symbolic and material value as a corporate asset treasured by commercial publishers. Recurrent and, it seems to me, intensifying and justified gripes by authors about processes of review being a ‘wicked labyrinth’ might be better directed at questioning the benefits ascribed to blinding that deprives this ‘labyrinth’ of sunlight.
The opacity of editorial and reviewer practices is particularly ugly and insidious, I have argued, when ethical principles (e.g. anonymity and confidentiality) are deployed in the name of protecting the vulnerable. Their invocation denies the possibility of subjecting manuscript evaluation processes to scrutiny, and thereby increases the likelihood of malconduct – such as editors’ self-promotion of a particular body of knowledge and/or the suppression of submissions that, despite being congruent with the espoused aims of the journal, pose an unwelcome challenge to editorial predilections.
Questioning the benign necessity of blinding peer review, and commending more open processes of manuscript evaluation, is congruent with the scholarly ‘grand challenge’ of ‘building a more progressive, equitable, freer, justice-centred world’ (Contu, 2020: 738). Processes of manuscript evaluation form a central part of ‘academic praxis (i.e. an embedded and embodied set of practices characterized by ideas, values, norms and theories)’ (Contu, 2020: 738) in which there are: ideas about justice; values associated with anonymity and confidentiality; norms of impartiality; and theories about the warranting of scientific knowledge.
By situating the institution of blinding within asymmetrical relations of power, where ideas, values, norms and theories of evaluation practices are promoted or impeded, this essay has sought to consider and critique how, as an invention of ‘dead generations’, secrecy in manuscript evaluation processes has come to weigh, in Marx’s (1852: 1) words, ‘like a nightmare on the brains of the living’. By challenging opacity, that is the host and protector of malpractice and abuse, it becomes possible to make ‘something visible in the object which it is orthodoxy’s secret purpose to keep invisible’ (Alvesson and Gabriel, 2013: 254 citing Adorno, 1954–1958/1984: 171, emphasis added). Fortunately, what may seem ‘unproblematic . . . is so only until further notice’ (Berger and Luckmann, 1967: 38, emphasis added). In MOS, this ‘notice’ is issued when recollecting that openness, not opacity, bestows upon science its distinctive claim to authority; and by honouring this claim, public confidence in, and understanding of, science is most effectively defended.
How, then, might we begin to wean ourselves off an apparently protective, but unaccountably oppressive, system into which, speaking for myself, I enrolled rather mindlessly? A modest beginning might be made by members of learned societies (e.g. Academy of Management; British Academy of Management) and journal editors acting for dominant publishers of MOS research (e.g. the Big 5, see note 7) initiating trials of open review, such as the one conducted here, that can then provide pointers for its continuous improvement. These trials would require the identities of authors and reviewers to be voluntarily revealed; and it would make review reports, authors’ responses and editorial decisions open to examination. It would parallel and complement current experiments with open access publishing.
I anticipate considerable resistance to this proposal, principally on the grounds that unblinding disadvantages the vulnerable (e.g. less established researchers, minority researchers and those affiliated to less prestigious institutions). I also predict that this resistance will be voiced not only by editors, prospective reviewers and authors but also, and more vigorously, if less obtrusively, by publishing houses and learned societies eager to protect their assets from unwelcome, potentially damaging, scrutiny. In other scientific fields (e.g. medical and natural sciences), admired and emulated by many exponents of MOS, open processes of peer review have been introduced and retained because they have been judged beneficial. The opacity of peer review is by no means the sole source of malaise currently affecting our field (Harley and Fleming, 2021), but greater transparency of manuscript evaluation processes can facilitate its illumination, and so enable relevant remedial action. Has the time now come to remove the blinkers that maintain our largely unexamined faith in blinding?
Supplemental Material
sj-pdf-1-hum-10.1177_00187267211016752 – Supplemental material for Critical essay: Blinding faith – Paradoxes and pathologies of opacity in peer review
Supplemental material, sj-pdf-1-hum-10.1177_00187267211016752 for Critical essay: Blinding faith – Paradoxes and pathologies of opacity in peer review by Hugh Willmott in Human Relations
Supplemental Material
sj-pdf-2-hum-10.1177_00187267211016752 – Supplemental material for Critical essay: Blinding faith – Paradoxes and pathologies of opacity in peer review
Supplemental material, sj-pdf-2-hum-10.1177_00187267211016752 for Critical essay: Blinding faith – Paradoxes and pathologies of opacity in peer review by Hugh Willmott in Human Relations
Supplemental Material
sj-pdf-3-hum-10.1177_00187267211016752 – Supplemental material for Critical essay: Blinding faith – Paradoxes and pathologies of opacity in peer review
Supplemental material, sj-pdf-3-hum-10.1177_00187267211016752 for Critical essay: Blinding faith – Paradoxes and pathologies of opacity in peer review by Hugh Willmott in Human Relations
Footnotes
Acknowledgements
I would like to thank the five reviewers of this essay for their helpful and engaging comments, which have been important for articulating, clarifying and amplifying its content (see Supplemental Material A). I would also like to recognize the support and guidance provided by Mark Learmonth, editor of Human Relations, during the process of revising the essay in the light of the reviewers’ suggestions and recommendations.
Funding
The author received no financial support for the research, authorship and/or publication of this article.
Supplemental material
It comprises editorial communications, reviewer reports and author responses (Supplemental Material A), Author reflections o the process of ‘open review’ (Supplemental Material B) and Notes for this essay (
).
Notes
Hugh Willmott (PhD, Manchester University; Honorary PhD, Lund University) is Professor of Management at Cass Business School and Research Professor in Organization Studies, Cardiff Business School. His research interests hinge around the application of critical social theories to the study and transformation of management and organizations. He is a past Associate Editor of Academy of Management Review and of Organization and has served on the editorial boards of Accounting, Organizations and Society, Journal of Management Studies and Organization Studies. Further details can be found on his homepage: . [Email:
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
