Abstract
Objective:
Data-sharing plays an essential role in advancing scientific understanding. Here, we aim to identify the commonalities and differences in data-sharing policies endorsed by otolaryngology journals and to assess their adherence to the FAIR (findable, accessible, interoperable, reusable) principles.
Methods:
Data-sharing policies were searched for among 111 otolaryngology journals, as listed by Scimago Journal & Country Rank. Policy extraction of the top biomedical journals as ranked by Google Scholar metrics were used as a comparison. The FAIR principles for scientific data management and stewardship were used for the extraction framework. This occurred in a blind, masked, and independent fashion.
Results:
Of the 111 ranked otolaryngology journals, 100 met inclusion criteria. Of those 100 journals, 79 provided data-sharing policies. There was a clear lack of standardization across policies, along with specific gaps in accessibility and reusability which need to be addressed. Seventy-two policies (of 79; 91%) designated that metadata should have globally unique and persistent identifiers. Seventy-one (of 79; 90%) policies specified that metadata should clearly include the identifier of the data they describe. Fifty-six policies (of 79; 71%) outlined that metadata should be richly described with a plurality of accurate and relevant attributes.
Conclusion:
Otolaryngology journals have varying data-sharing policies, and adherence to the FAIR principles appears to be moderate. This calls for increased data transparency, allowing for results to be reproduced, confirmed, and debated.
Introduction
Modern medicine and biomedical research face a reproducibility crisis, leading to issues with reliability and transparency. Research studies examining the extent of this crisis indicate that over 70% of scientists are unable to reproduce experiments conducted by another researcher, and over 50% report an inability to replicate results of their own previous research.1,2 Failures in reproducibility often stem from inadequate disclosure of research information such as sample data, methodology, and confounds. The estimated monetary losses from irreplicable research amount to $28 billion USD annually. 3
One promising answer is the required, comprehensive sharing of research data. Many researchers express that making all metadata and datasets available on public platforms is imperative to advancing literature.4 -7 External verification is crucial for building public confidence. 8 Sharing data stimulates broader meta-analyses and cross-discipline research. It can also curtail data manipulations and false publications, as irreplicable studies are subject to scrutiny.9 -11 Journals are inconsistent in data-sharing due to variations in how and what data is made available. One solution of interest is the FAIR (Findable, Accessible, Interoperable, Reusable) principles for data management and stewardship. These provide a gold standard for data-sharing, including metadata and methodology. 12
Here, we focus on otolaryngology, which relies heavily on clinical research. This cross-sectional review examines data-sharing patterns and adherence to the FAIR principles across otolaryngology journals. 12 Implementation of these standardized principles could assist in the culture shift toward data transparency among stakeholders. 13 Results serve to evaluate the reproducibility crisis in otolaryngology and provide guidance for future data policy.
Methods
Our protocol was uploaded apriori to Open Science Framework (https://osf.io/5evht/) along with all extracted data. We followed well-established extraction methodology frameworks.14 -16 This study was exempt from IRB scrutiny as it did not meet criteria outlined by the US Department of Health and Human Services’ Code of Federal Regulations. 17 To identify relevant journals, we searched Scimago rankings for Otorhinolaryngology journals.18,19 A smaller comparison group of top biomedical journals (per Google Scholar Metrics) was included. Our screening and inclusion process were outlined in a PRISMA flow diagrams (Figure 1). Journal data-sharing policies were evaluated on inclusion criteria: specific to data-sharing, written in the English Language, and freely accessible via website. Following screening, the data-sharing policies for every journal were extracted in accordance with items of the FAIR Principles as shown in Table 1. Two investigators pilot-tested the extraction framework for consistency. Disagreements were resolved with a third investigator present for adjudication. Interrater reliability was analyzed, and a kappa statistic of >.9 was met before full literature screening. Data were then extracted by 2 investigators in a masked and duplicate fashion, compiled into Google Sheets, and summarized for statistical analysis using STATA version 15.1.
Flow diagram of search results and inclusion criteria.
Comparison of Data Sharing Policies and Their Adherence to the FAIR Principles.
Results
A total of 100 otolaryngology journals (per h5-index) were included in our study sample and we found that 79% provided a data-sharing policy, while 21% did not. We compared this to the smaller sample of data-sharing policies from top biomedical journals. Of these 13 journals, 92% specified a data-sharing policy and 8% did not. Comparison of our 2 samples is well-visualized in Figure 2. We found most otolaryngology journals have a data-sharing initiative, with many utilizing aspects of the FAIR principles. The lack of standardization across data policies was apparent. Moreover, information was often not explicit or localized to a single page. Journals frequently voiced their support and recommendations for data-sharing, but few explicitly stated a requirement for publication.
Comparison of datapolicies for otolaryngology and biomedical journals.
Our comparison of otolaryngology journals against biomedical journals largely showcased homogeneity (Figure 2). In both samples, there was a low percentage of journals that established time frames, provided a data dictionary, or endorsed specific repositories. In contrast, both samples strongly adhered to protecting patient confidentiality. This is notable as previous studies found the most common public fears surrounding data-sharing are related to confidentiality. 20 There were differences between the samples, including a higher percentage of biomedical journals that outlined clinical trial data must be shared and had metadata retrievable by their identifier using a standardized protocol. The otolaryngology journals more often described datasets with rich metadata. Overall, results are promising; however, journals must specifically improve in terms of data accessibility and reusability.
Discussion
The reproducibility crisis persists despite increasing attention, as evidenced by investigations performed by the NIH and the US National Academy of Sciences, Engineering, and Medicine. 21 All US-funded research agencies receiving >$100 million USD must now have a systematic policy for easily accessible data. 22 The FAIR principles have laid down a key groundwork and shed light on deficits in community-wide data stewardship; a 2018 study suggests that responsible application of FAIR is a step forward. 23 Barriers and perceived risks of data-sharing have impeded universal adoption of any process. 24 Biomedical researchers may fear study disprovement or misinterpretation, rendering author-request data statements ineffective.25,26 Data-sharing motivations vary greatly among research stakeholders such as investigators, institutions, funders, and agencies. This incongruence has been a key force preventing the existence of a uniform and effective data-sharing approach. 12 A 2020 study demonstrates the exchange of data between researchers has been difficult due to a lack of standards, along with metadata that is insufficient in terms of accessibility or terminology. 27
A potential way to standardize shared data and increase adherence is implementing common data elements (CDEs). The NIH defines CDEs as “discrete, clearly defined and reusable data collection units.” 27 They allow for the standardization of data, through a consistent pairing of variables to responses sets across multiple studies that might otherwise be broad. They also utilize consistent terminologies and identifiers. 23 The facilitation of data-sharing increases opportunities for meta-analyses and comparisons across studies, improving data quality. 28 Despite CDEs great potential, there is currently no universal set in use. 29 The widespread use of a centralized, neutral data repository is paramount to the success of CDEs; a form of this exists with the NIH CDE Repository, a nationally resource of CDEs recommended by the NIH.27,30 With an increased commitment to this system comes data that is more accessible, interoperable, and secure through a clear process of authorizations.
Data findability is valuable; however, we must be a dialogue surrounding the ethical and legal concerns that arise with datasets containing personal information. 31 Our shift toward Open Science requires careful consideration and multiple strategies. The leadership of funders, journals, and publishers is crucial in creating data-sharing policies with clear instructions and enforcement. One option is to create data-editor positions, who ensure data is fully available in a repository or the materials. 32 Computational analysis also plays a role in reusability and reproducibility. 33 Open-source software can remove barriers such as proprietary code which inhibits researchers’ ability to properly replicate and examine studies. 32 Author incentives are also an avenue to improve transparency. For instance, the rewarding of open data badges to those in adherence with the best practices can motivate data-sharing through public recognition and citations. 34 These strategies can specifically increase data accessibility and reproducibility, supporting the maturation of research standards within otolaryngology and beyond.
Conclusion
The FAIR principles can improve transparency and reproducibility in the scientific community. Otolaryngology journals have inconsistent data-sharing policies and deficiencies in adherence. This study demonstrates specific gaps to be addressed and identifies multiple strategies to enhance compliance. Increasing data transparency can advance the field of otolaryngology, improving research reliability and quality of patient care.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
