Defining Trust Markers

Abstract

This paper is a visual summary of the key issues raised during a panel session entitled “Towards Standards for Trust Markers on Published Content” that was part of the 2024 NISO Plus Global virtual Conference held September 17–18, 2024. The author discusses paper mills and the efforts spearheaded by United2Act and the Committee on Publication Ethics (COPE) to mitigate the unethical behavior of paper mills that are seriously eroding research integrity and public trust in science.

United2Act ¹ is a group of international stakeholders committed to addressing the collective challenge of paper mills in scholarly publishing. United2Act is supported by the Committee on Public Ethics (COPE) ² and the International Association of Scientific, Technical, and Medical Publishers (STM). ³

Research paper mills, which have been discussed in the literature since 2013, are unethical organizations that produce manuscripts at scale using derivative, copied, and/or fabricated text or data sets. Manuscripts can be sold or traded to preexisting author teams, or individual authorship positions can be sold or traded before and/or after manuscript acceptance. Some paper mills may offer other services, including editorial handling and peer review, post-publication communications, and citations to their products ⁴ (Byrne et al., 2024).

United2Act developed five key initiatives, each associated with a specific working group: Education and Awareness; Improving post-publication corrections; Recommendations for research on paper mills; Facilitating dialogue between the various stakeholders; and Enabling the development of Trust Markers.

The Trust Markers working group is tasked with defining a “Trust Marker” and its cross-cultural function while creating a list of potential Trust Markers to be used by publishers and other stakeholders during the article submission process.

The group has determined that the objective of drafting definitions of Trust Markers in the context of paper mills would be most beneficial to the publishing community (this includes authors, reviewers, and editors) to enable cross-sector communications and strategy development. In this vein, the group determined that a Trust Marker provides assurance of item integrity, allowing for trust in the scholarly record. In other words, when evaluating research, these are areas where we assess trust. Further, Trust Markers allow the community to move from implicit trust to verifying and validating these markers at the right moments in the publishing process.

Specifically, Trust Markers relate to:

• People (authors, reviewers, editors)

• Organizations (research institutions, journals, funders)

• Processes (research timeline, publishing)

• Products or outputs (article, data, code, images)

As these are potential Trust Markers and refinement is ongoing to indicate positive or negative signals associated with them. For example, Trust Markers related to people center around identification which is applicable to all the people involved: authors, editors, and reviewers. These, along with the potential use of watch lists of known abusers, are key items to utilize as potential trust markers. Having an ORCID could be a Trust Marker. Similarly, having a Researcher ID or having a claimed profile on Web of Science could be as well. However, the absence of a positive signal should not be equated to a negative signal—not all profile systems are global or standardized in different arenas.

Trust in Organizations was another area of focus centered on clear journal policies with an emphasis on journals having publicly available peer-review data, requirements for ethics statements, and a retraction policy that they adhere to. Within the processes themselves, one of the specific items that came up early on in group discussions was the research process itself—was the timeline appropriate for the work being done, were the resources available that would have been used to do the work?

The most significant focus was then on the publishing process, submission screening, the peer review process and finally the production of the content. Potential future identifiers include Research Activity Identifiers (RAiDs) ⁵ —a persistent identifier for research projects. The final area of need for Trust Markers is in the products/outputs of the research process. This includes data, code, images and the article itself. Indicators of trust for these could be: that they are publicly available and stored in a repository, that they can be used to reproduce results, and that they have ROBUST metadata associated with them.