The need for verification markers on published content

Abstract

Ethics and integrity cases in submitted manuscripts is becoming an increasingly common problem, whilst retractions continue to grow at a rapid rate. These cases are caused by both intentional misconduct and fraud, as well as a host of unintentional poor research practices. Whilst most good journals undertake a range of checks to spot these issues ahead of peer review and publication, which checks have been conducted and passed are unclear to the reader, and most preprint servers conduct far more limited checks. As preprints become increasingly recognised as valued outputs of research, they will likely become a growing target for problematic submissions. This article explores the need to develop standards around the labelling of checks conducted and passed across all types of publication outlets, to better inform the reader how much to trust an article or preprint. It describes a couple of recent developments to create such labelling and more rigorously verified preprints, and explores the need for a more nuanced expectation around checks and peer review on different types of research outputs. Finally, it proposes that NISO (or a similar neutral entity) work to bring together relevant providers and members of the community to develop and agree a standard framework and metadata for these checks and verification.

The significant growth in ethics and integrity issues in submitted content is becoming well established. For example, data from Alam ¹ shows the very significant growth in submissions that needed to be investigated by the internal Publishing Ethics and Research Integrity team at Taylor & Francis, from 250 ethics cases in 2019 to over 900 in 2021, and then a large uptick to 3800 in 2023. This has grown in large part due to papermill activity and buying/selling authorship, but has been exacerbated more recently by the significant growth in (mis-)use of large language models (LLM) as their adoption has become rapidly more widespread.

Not all misconduct is malicious and intentional (Alam ¹ ). Media coverage naturally tends to focus on the more extreme end, such as papermills, fake/manipulated images and data, and nonsensical content. However, much problematic activity is unintentional and largely due to poor laboratory practice and/or lack of awareness, for example, some plagiarism, misuse of generative AI tools, inadequate/biased reporting of research, inappropriate citation practices, data sharing issues, p-hacking, ² etc. Whilst these might seem like less problematic issues, the net effect is the same: research outputs with questionable conclusions are being published and treated as accurate and then built on for future research. When the findings are subsequently found to be incorrect, and are contradicted by further research or even retracted, this can then have the problematic effect of eroding trust in new research.

Unfortunately, this issue does not just affect those working within the scholarly ecosystem. Scholars are more aware of these issues and of the need to self-verify findings prior to building on them and are more used to ongoing uncertainty of any scientific ‘fact’. However, the broader society has a greater expectation that published research is ‘correct’. This is further clouded in some communities by the (sometimes deliberate) misuse of certain misleading studies to further a particular unorthodox or extreme viewpoint. Combined, this mistrust of scholarly research is unfortunately growing in some communities and, as a consequence, has led to challenges in ensuring public uptake and action in emergencies. For example, the growth in vaccine scepticism leading to reduced take-up of COVID or measles vaccines ³ (White et al. 2023 ⁴ ; Parums DV 2024), and climate change scepticism leading to lack of support in some areas for positive action ⁵ (Bogert JM et al 2023). This then risks a growing lack of public support for investment and funding in scholarly research, especially as many governments work hard to address budgetary deficits.

Preprinting and checks

Another shift in the way research is first communicated is the growth in the use of preprint servers. This started with arXiv ⁶ in 1991, where preprinting in the physics, math, and computer sciences has now become standard practice ⁷ (Garisto D 2022). More recently, preprinting has grown in popularity in life and medical sciences with the launch of bioRxiv ⁸ in 2013 and medRxiv ⁹ in 2019, with a plethora of preprint servers now being established in most subject areas. Whilst uptake of preprinting in these fields has been steadier in pace, it is now estimated that preprints are posted for approximately six percent of all final publications ¹⁰ also supported by an analysis of 2023 and 2024 data in Dimensions). Scholar-led movements such as ASAPbio ¹¹ have accelerated this uptake, supported by funder policies that recognise preprinting as evidence of research output (e.g. NIH, Wellcome, and UKRI). More recently, some policies have started to recognise peer reviewed preprints at the same level as a peer reviewed journal article (e.g. Wellcome, EMBO, CZI, and cOAlition S).

However, to achieve the very fast publication times for preprint servers (most quote 24–48 hours), checks conducted on submissions are typically fairly light (more rigorous checks obviously being conducted on medical preprints on medRxiv). These checks are usually outsourced to groups of researchers in the community whose primary focus and expertise is on the research rather than on the intricacies of ethics and integrity issues, or the growing complexities of approaches used by nefarious actors to create and then submit problematic content. Combine this with the growing recognition and importance placed on preprints by funders and shifting research assessment practices, it should be anticipated that preprint servers will increasingly become targets for the scale of highly problematic content being seen by journals. This will likely be further exacerbated by the general low barriers and lack of rigorous integrity checks in place on these servers.

To complicate matters further, every preprint service and journal publisher conducts different checks (sometimes even between journals within one publisher or preprint host), and this information is typically either not publicly shared or is hard to locate. Consequently, even a well-trained researcher will find it difficult to understand what checks a piece of published content has gone through, and/or has passed. For a member of the broader public who is typically far less familiar with the intricacies of different publishing approaches, they are unlikely to understand what any of these models might imply about the level of verification of the content.

Editorial checks and peer review

Another common misconception (by both researchers and the broader community) is that peer review will catch any problems in an article and consequently, once an article has passed peer review, it can be deemed as ‘correct’. The reality is that articles are typically reviewed by two to three time-strapped experts, who may (or may not) conduct a detailed review and typically this process is anonymous to everyone but the Editor. Whilst this process can be highly effective and conducted very rigorously at some journals, at many others it may not be.

In addition, the typical publishing process sends every type of article through largely the same type of peer review by experts at a similar level of expertise. But different types of research outputs warrant different types of checks, and different checks require different types of experts. These might be primarily editorial or AI-based checks (e.g. editorial review), they could require subject-matter experts (e.g. traditional peer review), but they might also benefit from context-based review (e.g. by patients), or specific to the output type (e.g. data curators). All these types of review are just as important as each other, they just have different purposes, and each type of output may benefit from a different blend of checks.

What is needed is nuanced recognition of the type of verification an output requires, and a standard verification framework that can:

• Support more accurate understanding of how much to trust a piece of content;

• Support more effective transfer of content across the scholarly system, thereby minimising duplication of effort and cost between preprint servers, different receiving journals, etc;

• Consequently, enable an increase in sustainability of the peer review system through targeting of the right type and level of reviewer for the output rather than tasking the same community of reviewers with all research outputs;

• Support a shift towards responsible research assessment that recognises and incentivises research integrity as a key pillar of assessment.

VeriXiv and verification

VeriXiv ¹² was launched by F1000 in August 2024 with support from the Gates Foundation, after they announced their refreshed Open Access policy that included a requirement for Gates grantees to share a preprint of their work prior to journal publication, starting from January 2025. Due to a concern around the level of checks that typical preprints undergo, VeriXiv was built as the first Verified Preprint Server. This means that all preprints on VeriXiv undergo the same stringent editorial-based checks that all articles undergo for full publication in one of the F1000 Open Research Platforms or indeed many of the Open Access journals in Taylor & Francis. After preprints have passed the rigorous checks and the preprint is published (in full XML, not just PDF), the authors have three options:

• Leave as a preprint;

• Submit to any journal of their choice – we provide the authors with a package to take with them;

• Submit to Gates Open Research (for Gates grantees only) and therefore continue on to open peer review on their preprint. Those that pass peer review will be published as a Version of Record in Gates Open Research (which now becomes a more traditional, Open Access journal).

The Gates Foundation are the first partner utilising VeriXiv through a co-branded space on the platform, but over time, the expectation is that more partners will join, together with a growing range of Version of Record venues.

The verification checks that preprints on VeriXiv undergo include:

• ‘Verified’ checks: This covers a range of ethics and integrity checks including originality concerns, image manipulation, author verification, and competing interests. All preprints must pass all these checks to be published as a preprint on VeriXiv.

• ‘Open Research’ checks: Transparency checks for FAIR ¹³ data and methods to support reproducibility. In the case of the Gates partnership, these checks are conducted, but preprints do not have to pass these checks to be published. Future partners will have the flexibility to decide whether they want these checks conducting, and whether authors must pass them prior to publication of the preprint.

Badging of checks

To support readers of all types in being able to understand how verified an output is, we will be adding clear labelling to show which checks have been conducted and which have been passed (Figure 1).

Figure 1.

The four types of verification badges on VeriXiv. The checks involved around authorship, research integrity, and ethics must be passed for the content to be published as a preprint on VeriXiv.

Indeed, there are several similar efforts being developed by different parties aiming to achieve this. For example, a multi-stakeholder collaborative called United2Act ¹⁴ (supported by STM and COPE) includes a working group focussed on enabling the development of trust markers. Meanwhile, PKP have developed a Publication Facts label (similar to food labelling). Whilst it is good to start with a number of different approaches and initiatives to achieving this goal, and it is natural that every party wants to develop their own version with their own unique take on this issue, this will likely lead to considerable confusion for users and ultimately may mean we really do not solve the issues as laid out in this paper.

Proposal

The proposal is for NISO, or a similar neutral entity, to bring together the relevant providers and members of the community to develop and agree a standard framework and metadata for these checks and verification. This would enable users to compare checks conducted across different publishing venues so they can form an instantaneous sense of how much to trust what they are looking at, whilst leaving room for some individualism in design from each provider. Standards around the associated metadata are also crucial to provide machine-based users of content with adequate information around trust signals. This would not only address concerns around trust of content but could also reduce duplication of effort (and therefore cost) between providers as content moves across the scholarly communication system.

Integrity and trust in scholarly content is at a crucial juncture. Providers involved in scholarly communication have a pivotal role in maintaining integrity of the scholarly record, and it is therefore incumbent upon us to work together to bring clarity around the work we do to preserve this integrity to those who are using the content, whatever type of user they are.