Perceived publication pressure and research misconduct: should we be too bothered with a causal relationship?

Abstract

Publication pressure has been touted to promote questionable research practices (QRP) and scientific or research misconduct (RM). However, logically attractively as it is, there is no unequivocal evidence for this notion, and empirical studies have produced conflicting results. Other than difficulties in obtaining unbiased empirical data, a direct causal relationship between perceived publication pressure (PPP) and QRP/RM is inherently difficult to establish, because the former is a complex biopsychosocial construct that is variedly influenced by multiple personal and environmental factors. To effectively address QRP/RM by tackling the sources of PPP would also be difficult because of the competitive nature of the reward and merit system of contemporary science. We might do better with efforts in enhancing knowledge in research ethics and integrity among the practitioners, as well as institutional infrastructures and mechanisms to fairly and efficiently adjudicate cases of QRP/RM.

Introduction

Research output, measured in terms of publications, is typically a major and critical component of performance assessment for scientists and researchers in the academia. Contemporary scientific research is inherently and declaratively competitive, both in general terms (such as grant funding), as well as from more personal perspectives (such as gainful employment and tenure). The pragmatic mandate and obligation to publish well to ensure career advancement, reinforced by a “publish or perish” culture (Neill 2008; Van Dalen and Henkens 2012), constitutes a psychological stress that has been termed perceived publication pressure (PPP) (Haven et al. 2019), on scientists and researchers.

The reproducibility crisis in research (Baker 2016) and the continuous rise in retractions (Brainard and You 2018; Steen et al. 2013) are worrisome and undermines the credibility of science and scientists. The above have been varyingly attributed (Brainard and You 2018; Fang et al. 2012; Wang et al. 2019) to acts of scientific or research misconduct (RM) (Gross 2016) and questionable research practices (QRP) (Andrade 2021). The prevalence of misbehavior in science has also been estimated (Fanelli 2009; Godecharle et al. 2018; Martinson et al. 2005), with a recent meta-analysis indicating a self-confessed commitment rate of 2.9% for RM and 12.5% for QRP, as well as astute awareness of commitment by others (15.5% for RM and 39.7% for QRP) (Xie et al. 2021).

Explanations for RM have indicated the involvement of both personal and systemic risk factors. In particular, a classical notion of RM being committed by flawed characters or errant individuals, or the “bad apples” notion, has been broadened to include systemic factors that are conducive for misconduct (Redman 2013; Sovacool 2008). The latter includes institutional factors such as a lack of clear scientific misconduct policies, research integrity training and/or peer/institutional oversight, as well as more general factors, such as unhealthy competition for resources, particularly research funding (Harvey 2020) and publication pressure (Gandevia 2018; Moreels 2018; Qiu 2010; Tijdink et al. 2016b).

That PPP could be a risk or predictive factor in QRP/RM is a popular and arguably logical notion (Haven et al. 2021; Johann 2022; Tijdink et al. 2013), but evidence obtained from empirical analyses with different strategies are conflicting (Fanelli et al. 2015, 2017, 2019; Gopalakrishna et al. 2022; Heuritsch 2021; Maggio et al. 2019; Paruzel-Czachura et al. 2021; Tijdink et al. 2014). In the paragraphs below, we briefly summarize these empirical findings and discuss why conclusions drawn from these would be persistently confounded by a lack of unbiased data and the complexity of PPP as a biopsychosocial construct. We posit that it would also be difficult to effectively tackle the sources of PPP, and that we would do better in attenuating QRP and RM by focusing on the other systemic factors.

Empirical assessments of a relationship between perceived publication pressure and scientific misconduct

Many different empirical analyses have been reported in the past years on the correlation between PPP and QRP/RM. Tijdink and colleagues noted a strong association between publication pressure and a composite scientific misconduct severity score based on self-reporting with a survey of Flemish biomedical scientists (Tijdink et al. 2014b). A cross-sectional survey of health researchers by Maggio and colleagues indicated that publication pressure is the strongest individual predictor of misconduct (Maggio et al. 2019). More recently, two studies by Paruzel-Czachura and colleagues on researchers, as well as those at the management level in a Polish university, showed a positive correlation between PPP and self-reported QRP/RM, as well as observed QRP/RM by those at the management level of the researchers (Paruzel-Czachura et al. 2021). A comprehensive survey of astronomers worldwide by Heuritsch found that perceived pressure to publish explains 10% of the variance in occurrence of misconduct (Heuritsch 2021). A large, nationwide survey of researchers in the Netherlands by Gopalakrishna and colleagues showed that publication pressure is associated with a higher engagement with QRPs (Gopalakrishna et al. 2022).

On the other hand, in performing comparative analyses of bibliographic and personal information of authors of retracted papers between 2010 and 2011 with authors of “control” papers matched by journal and issue, Fanelli and colleagues found a contradiction to the hypothesis that pressure to publish drives misconduct. The authors noted that “. . .high-impact and productive researchers, and those working in countries in which pressures to publish are believed to be higher, are less-likely to produce retracted papers, and more likely to correct them” (Fanelli et al. 2015). In another report by Fanelli and colleagues on bias-related patterns and risk factors with a large random sample of meta-analyses taken from different disciplines, they showed that highly published authors with many citations were not at greater risk of bias (Fanelli et al. 2017). Furthermore, conditional logistic regression analysis of a set of variables from papers containing problematic image duplications and those that do not (obtained from a previous screening of 8138 papers published in PLoS One in 2013–2014) also did not support pressure to publish as a hypothetical driver for misconduct (Fanelli et al. 2019). A more recent report by Fanelli and colleagues has also demonstrated the importance of a “country-level” effect (Fanelli et al. 2022). The authors found that for some countries where publications are rewarded with misaligned institutional incentives, such as cash rewards for papers published, the risk of problematic image duplication was indeed higher for more productive, more frequently cited, earlier-career researchers working in lower-ranking institutions. It is notable that the data analyzed by the authors who have reported a positive correlation between PPP and QRP/RM are fundamentally different from those used by Fanelli and colleagues. The former authors used mainly self-reported survey data, in that participants recorded their own role in QRP/RM and their own perceived intensity of publication pressure. In surveys like such, there is no running away from a preexisting bias, in that the participants’ responses are guided by how the survey is framed and worded by the investigators. Conceivably, those admitting to committing QRP/RM are also likely to be more intensely perceptive of publication pressure. On the other hand, analyses of Fanelli and colleagues of publication data appear to be relatively free of such a bias. However, the conclusion of Fanelli and colleagues are based on an intrinsic assumption that authors with higher number of publications and who are more highly cited would have a higher PPP. This assumption may be an oversimplification and may not be valid. It is entirely conceivable that the better published authors, being better endowed in terms of capabilities and resources, may instead have a lower PPP. Conversely, authors with higher PPP might be those that have low or marginal number of publications and are thus struggling for survival in academia. In any case, the assumption has not been independently verified, and is thus also a potential source of bias. The most recent report by Fanelli and colleagues also illustrated that PPP could indeed be made more complex by other individual risk factors such as perverted publication incentives, leading to a high incidence of QRP/RM.

It is not difficult to see why attempts to obtain unequivocal evidence for a causal link between PPP and QRP/RM may ultimately be futile because of the difficulty in getting unbiased data. More importantly, as discussed below, PPP is a complex and multivariate biopsychosocial construct that is variedly influenced by both personal and systemic factors and might not be accurately gauged in surveys.

Perceived publication pressure as a complex psychological construct influenced by multiple individual and systemic factors

Paruzel-Czachura and colleagues have defined PPP as a “Subjectively perceived psychological tension that is related to the requirement for a particular number of publications in a specified timeframe, which attests to one’s academic development, and is a condition of maintaining one’s position or even retaining one’s job”(Paruzel-Czachura et al. 2021). This subjective perception could nonetheless be qualitatively assessed by psychometric analysis methodologies. The Publication Pressure Questionnaire (PPQ) (Tijdink et al. 2014a) and the revised PPQr (Haven et al. 2019), with content and structural validity that provide sufficient psychological component coverage, represent such approaches. The use of the PPQ in a survey of medical scientists did indicate a strong association between publication pressure and scientific misconduct (Tijdink et al. 2014b). However, other surveys that attempted to examine explanatory or risk factors for QRP/RM may not have questions constructed with the same psychometric strength and depth of the PPQ or PPQr.

The issue though, is whether even survey questions with sufficient psychometric strength and depth could truly reflect PPP, particularly in a way that could relate, but not bias it toward a tendency for QRP/RM. Notably, there is no objective measure for PPP, and responses to PPP are conceivably influenced by a multitude of personal and environmental factors. In the first instance, personality traits could be an important determinant of one’s response to a given level of PPP, or conversely a determinant of the intensity of one’s PPP. Conceivably, the same intensity of PPP may result in varying degrees of tendency to moral transgression in individuals with differing moral virtues. Machiavellianism (but not narcissism, psychopathy and self-esteem) has been shown to be positively associated with self-reported research misbehavior (Tijdink et al. 2016a). Given an otherwise similar research climate and PPP, an unscrupulous machiavellian may have a disproportionate tendency to moral transgression. It is also likely that individuals with different cultural backgrounds (Davis 2003) may act differently given the same level of PPP. PPP intensity may also differ for scientists at different stages of their careers, and the ability to handle a given degree of PPP would presumably increase with age and experience (or conversely, apathy toward moral stringency may grow with age).

Wikström’s situational action theory in criminology posits that crimes are committed when people “. . .fail to act in accordance with their own personal morals (i.e., fail to exercise self-control) in circumstances when externally pressurized to act otherwise”, or “. . .find them viable and acceptable in the circumstance (and there is no relevant and strong enough deterrent). . .” (Wikström 2019). Analogously, it is not difficult to see that given the same PPP intensity, an individual with adequate training in responsible conduct of research, or one working in an environment where research integrity is prominently emphasized and that has in place efficient misconduct management infrastructure, might be less likely to err compared to another who is uninitiated and unguided. Likewise, researchers in a topic or field with heightened propensity for peer scrutiny or post-publication peer-review would be more careful with potential violations in terms of QRP/RM given the same intensity of PPP.

The above considerations suggest that there could be a good degree of decoupling between self-reported PPP and how this may relate to tendencies to commit acts of QRP/RM, and that multiple mechanisms exists whereby the former might not be empirically predictive of the latter.

Difficulties in effectively eliminating the sources of perceived publication pressure

PPP in the academia could arise from multiple sources, most commonly those associated with employment opportunity, career advancement (such as promotion and tenure) and research funding. The credential of practitioners in science and research are too often judged by their publication track records in an over-simplistic manner using bibliometric measures, such as one’s papers in journals with high impact factors. In the latter regard, in many fields, it is often difficult to publish negative or null results in such journals, rendering the often sizeable amount of effort and resources moot (Aczel et al. 2018). This understandably could prove to be very frustrating for researchers. In many research grant proposal evaluations, there is a demand for and bias toward productive researchers with a good track record. Often, the necessary support for project feasibility would be nothing less than a number of published papers. Indeed, funding opportunities could be made more inclusive, but given a fixed budget allocation, it would be a zero-sum game. The stiff competitions for faculty positions worldwide means that only those with an exceptional publication record would ever be shortlisted. Even for those who are merely looking to continue in research, a lag in one’s publication record within a contract may be detrimental to chances of securing their next employment. Criteria for promotion and tenure are almost never categorically spelled out for any universities and institutions, but it is common knowledge that these are critically dependent on publication output.

There are important movements for changes in evaluation of publications (Hatch and Curry 2020) beyond the traditional bibliometric assessment (Joshi 2014), such as the San Francisco Declaration on Research Assessment (DORA) (https://sfdora.org/read) and the Leiden manifesto for research metrics (http://www.leidenmanifesto.org/). There are also attempts to alleviate pressure to publish for junior investigators, such as the pilot schemes by the European Molecular Biology Organization (EMBO) to remove the requirement of a first-authored paper for fellowship consideration, or that a first-author refereed preprint will be considered as an equivalent (Lemberger et al. 2022). For the evaluation of eligibility for grants, positions, promotions or tenure there could be efforts made to emphasize on the quality rather than quantity of a smaller number of one’s representative publications. Examples of such include the Australian National Health and Medical Research Council (NHMRC)’s requirement of listing only up to 10 of one’s top publications in the past 10 years (https://www.nhmrc.gov.au/about-us/news-centre/changes-publication-assessment), and the Swiss National Science Foundation’s “narrative” curriculum vitae template (Sing Chawla 2022). The Hong Kong Principles for assessing researchers, formulated in 2019 at the 6th World Conference on Research Integrity (WCRI) in 2019, also proposed recognition of researchers in much broader terms than one’s publication list (https://wcrif.org/guidance/hong-kong-principles).

However, given the stiff competition for limited resources and positions, PPP shall continue to be exerted. It would be difficult to make wholesale changes to the reward and merit systems of modern science such that PPP could be effectively diminished. In this regard, it may be equally difficult, if not ultimately misplaced, to focus efforts in reducing PPP, just as in attempts to determine whether there is a direct causal role for PPP in QRP/RM.

Promoting knowledge and systemic improvements in research ethics and integrity

We posit that instead of struggling to decipher any causal relationships between PPP and QRP/RM with an intention of upstream intervention of the former, we might do better by simply focusing on more downstream remedies for the latter. Despite its negative connotations, PPP could in fact be a driving force for honest hard work. What would be needed is for any stress it induces to be handled with the right mindset and in the light of clear guidelines, as well for its negative consequences to be adequately and efficiently dealt with.

Perhaps from a haughtily stringent and uncompromising perspective, blaming QRP/RM on PPP is a show of weakness or even an attempt at moral disengagement (Moore 2015). In this regard, instilling essential moral values and virtues in science and research (Pennock 2015) among its practitioners with education and training would be a path forward. We should therefore put efforts in enhancing knowledge of research ethics and integrity among scientists and researchers (Shamoo and Resnik 2015; Yeo-Teh and Tang 2021), as well as closely associated managers and administrators. Zoloth has recently argued for the cultivation of “. . .classic values of veracity, courage, humility, and fidelity. . .” by researches to “. . .allow science to proceed ethically under conditions of deep uncertainty. . .”(Zoloth 2021). We think this would also be aptly applicable to dealing with pressure and stress in contemporary research settings.

To be fully prepared to tackle QRP/RM regardless of its cause, we should have in place institutional infrastructures and mechanisms to fairly and efficiently investigate and adjudicate suspicions, allegations and cases. These would include clear and well-disseminated research integrity guidelines or code, an authoritative protocol and trained personnel to handle whistleblowing and to investigate and adjudicate QRP/RM. These measures, we argue, would facilitate more effectively the productive output of reproducible research without laboring on poorly defined and potentially futile attempts to preempt any misconduct-inducing effects of PPP.