Open science and conflict of interest policies of medical and health sciences journals before and during the COVID-19 pandemic: A repeat cross-sectional study

Abstract

Objectives

To audit the transparent and open science standards of health and medical sciences journal policies and explore the impact of the COVID-19 pandemic.

Design

Repeat cross-sectional study.

Setting

19 journals listed in Google Scholar's Top Publications for health and medical sciences.

Participants

Blood, Cell, Circulation, European Heart Journal, Gastroenterology, Journal of Clinical Oncology, Journal of the American College of Cardiology, Nature Genetics, Nature Medicine, Nature Neuroscience, Neuron, PLoS ONE, Proceedings of the National Academy of Sciences, Science Translational Medicine, The British Medical Journal, The Journal of the American Medical Association, The Lancet, The Lancet Oncology, and The New England Journal of Medicine.

Main outcome measures

We used the Transparency and Openness Promotion (TOP) guideline and the International Committee of Medical Journal Editors (ICMJE) requirements for disclosing conflicts of interest (COIs) to evaluate journals standards.

Results

TOP scores slightly improved during the COVID-19 pandemic, from a median of 5 (IQR: 2–12.5) out of a possible 24 points in February 2020 to 7 (IQR: 4–12) in May 2021, but overall, scores were very low at both time points. Journal policies scored highest for their adherence to data transparency and scored lowest for preregistration of study protocols and analysis plans and the submission of replication studies. Most journals fulfilled all ICMJE provisions for reporting COIs before (84%; n = 16) and during (95%; n = 18) the COVID-19 pandemic.

Conclusions

The COVID-19 pandemic has highlighted the importance of practising open science. However, requirements for open science practices in audited policies were overall low, which may impede progress in health and medical research. As key stakeholders in disseminating research, journals should promote a research culture of greater transparency and more robust open science practices.

Keywords

epidemiology NON-CLINICAL public health statistics and research methods journalology NONCLINICAL

Introduction

Scientific advancement in any field or discipline relies considerably on transparent research practices.¹ This includes medicine and public health. The ability to access, evaluate, and extrapolate the findings from research to clinical practice and policy relies on the transparency of available evidence.^1,2 Research transparency is also imperative for replicating scientific findings.¹ Given the often high-stakes circumstances of evidence-based medical and public health practice, the need for transparency is particularly critical.

Journal policies that support open science practices, also called open research or open scholarship, have improved transparency and replicability.^3,4 Open science is defined as research activities and outputs that are made publically available with little to no restrictions.⁵ Open and transparent practices include, but are not limited to, the preregistration of study protocols and analysis plans, the sharing of data, statistical code, and research materials, and the disclosure of authors’ conflicts of interests. Making code and data openly available allows for reuse, replication, and peer assessment, which is an important step in verifying the findings of studies and progressing science.⁶ Poor adherence to open practices produces unreliable outcomes and slows scientific progress.^7–9

Research published during the COVID-19 pandemic has highlighted the widespread impact of practising ‘closed’ research, where cases of undisclosed data and a lack of transparency led to high-profile retractions.^10,11 The pandemic has also demonstrated the positive impacts of open sciences practices, as demonstrated by quick data sharing used to identify the COVID-19 genome rapidly and journals removing paywalls for COVID-19 research, resulting in more equitable and widespread access to research.^12–14

There are various tools available to support and reward the practice of open science, including freely available repositories (e.g. GitHub, Figshare) and platforms (e.g. the Open Science Framework [OSF]), publication methods (e.g. registered reports), and open science badges.^15–18 Factors that are likely to contribute to poor adherence include a lack of knowledge or skills in using these tools to facilitate openness and the flawed incentive system that encourages the publication of more instead of better research.^19,20 For transparent research practices to be the default, stakeholders, including institutions, funders, publishers, and journals, can mandate and facilitate researchers to work in such ways. Since journals are the gatekeepers of disseminating research, strengthening journal policies to reflect transparent and open practices could incentivise researchers to upskill and practice open science. Various organisations such as the Committee of Publication Ethics, Directory of Open Access Journals (DOAJ), the Open Access Scholarly Publishers Association, and the World Association of Medical Editors have put forth guiding pricinples for scholarly publications to follow with the aim of improving transparency and best research practice.²¹

While efforts have been made to improve and promote the transparency of research,^1,19 further steps are needed to adopt and normalise this way of working. The Centre for Open Science (COS), which has pioneered the development of open tools and resources, developed the Transparency and Openness Promotion (TOP) guideline in 2015 to assess and improve journal policy requirements of publishing open and transparent research.²² Similar policy changes have improved open and transparent research practices, as was seen after the ICMJE trial registration policy in 2004.²³ Since the publication of the TOP guideline, several evaluations of journal policies using the TOP guidelines have been published,^24–26 which suggest poor compliance to TOP guideline standards across multiple fields of research.

Since the beginning of the COVID-19 pandemic, several journals have released statements and made efforts to improve the openness and transparency of COVID-19 research to support researchers during the pandemic.^27,28 To date, a peer-reviewed evaluation of high-ranking general medical and health science journals assessing open science practices before or during the COVID-19 pandemic has not been conducted. Therefore, we aimed to conduct this evaluation and explore the potential impact of the COVID-19 pandemic on the adherence to transparent and open science standards of leading health and medical journals.

Methods

We designed a repeat cross-sectional study based on methods developed by Cashin et al.²⁴ and preregistered the study protocol on the OSF.²⁹ We completed the first policy audit in early 2020 and repeated the same audit in 2021 to assess the potential impact of the COVID-19 pandemic on journal policies.

Sampling procedure

On 31 January 2020, we sampled 19 journals listed in Google Scholar's Top Publications for the health and medical sciences subcategory,³⁰ which at this time were (in alphabetical order): Blood, Cell, Circulation, European Heart Journal, Gastroenterology, Journal of Clinical Oncology, Journal of the American College of Cardiology, Nature Genetics, Nature Medicine, Nature Neuroscience, Neuron, PLoS ONE, Proceedings of the National Academy of Sciences (PNAS), Science Translational Medicine, The British Medical Journal (BMJ), The Journal of the American Medical Association (JAMA), The Lancet, The Lancet Oncology, and The New England Journal of Medicine (NEJM). On 10 February 2020, the policies for each journal were downloaded and saved with date and time stamps, defined as the pre-COVID-19 pandemic timepoint. We repeated this procedure for each journal to collect updated policies between 26 and 31 May 2021, defined as during the COVID-19 pandemic timepoint. All policies extracted are openly available on our OSF repository.³¹

Measurement tools and scoring

We used the TOP guideline to audit and score each journal's policy. All audits were performed in duplicate by two independent study authors (ADG, EH, CJ, GT, AGC, HL, DN, ET), and discrepancies were adjudicated with a third author (GCR). The TOP guideline has eight standards: data citation, data transparency, code transparency, materials transparency, design and analysis transparency, study preregistration, analysis preregistration, and replication.^22,32 For scoring, we used version 1 of the TOP rubric,³³ where 0 (zero) denotes “no mention” (of the standard in the policy wording), 1 “discloses”, 2 “requires”, and 3 “verifies”. The maximum score obtainable for TOP was 24. Justifications for scoring, along with quoted material from the journal policy, where appropriate, are provided in the Google Sheet master files on our OSF repository.³¹ Where a policy demonstrated varying requirements (e.g., different types of study design) or only mentioned requirements for one element (e.g., requiring the use of reporting guidelines or study preregistration for clinical trials only), a score was given based on the lowest-scoring element in accordance with the TOP guidelines.

Journals and funders can choose to become signatories of TOP to express their support, interest, and commitment in adopting such standards.³² Hence we also scored the included journals on whether they were a signatory of TOP, permitted registered reports, and used open science badges. For these metrics, we provided scores based on previously published methods (i.e. TOP signatory: no = 0, yes = 1; registered reports: no = 0, yes = 2; and open science badges: no = 0, yes = 2).²⁴ We summed the total TOP score (out of 24) with these three elements to calculate the overall TOP score (out of 29).

To examine the journal's requirements for authors to disclose conflict of interests (COIs), we used the four standards in the International Committee of Medical Journal Editors (ICMJE) disclosure form.³⁴ The four standards include: 1) recipient of payment or services from a third party (government, commercial, private foundation) for any aspect of the submitted work; 2) any financial relationships (regardless of the amount of compensation) with entities in the biomedical arena that could be perceived to influence, or that give the appearance of influencing, the submitted work; 3) any patents, whether planned, pending or issued, broadly relevant to the submitted work; and 4) any other relationships or activities that readers could perceive to have influenced, or that give the appearance of potentially influencing, the submitted work. We scored the journal policies for disclosing COIs on a 0 to 4 scale, with 0 being no statement of any of the standards and four being all standards or provision of the complete ICMJE disclosure form in accordance with published methods.²⁴

Data analysis

All scores were provided in separate Google Sheets that were combined, compared, and converted to a .csv file for analysis. We used descriptive statistics and calculated medians and interquartile ranges (IQRs) of scores where appropriate. Journals were ranked by their scores for the eight TOP standards (out of 24) and requirements for COIs (out of 4). To assess where journals engaged most with TOP standards, we summed scores across journals for each TOP standard, where the maximum total could be 57 (3 possible points for 19 journals). Scores were compared before and during the COVID-19 pandemic using absolute values and percentage changes.

Software and data sharing

Analyses and figures were created using pandas, seaborn, and matplotlib libraries in Jupyter Notebook with Python v3.7. Our study materials, data, code, and figures are accessible on our OSF repository³¹ and GitHub.³⁵

Protocol deviations

In the protocol, we planned to examine the 20 journals listed in Google Scholar's Top Publications for health and medical sciences as listed on the search date (31 January 2020). We excluded the Cochrane Database of Systematic Reviews during analysis as it is a specialized journal that commissions research groups to adhere to a prescriptive methodology and differs significantly from the other included journals. Therefore, our study includes the remaining 19 journals reported in Google Scholar's Top Publications for health and medical sciences at the time we reviewed the journal rankings (31 January 2020). After we commenced the study, iterations of TOP were made by COS (now called TOP Factor, removing the TOP signatory score from the total TOP score). Thus we present the sum of the eight TOP standards (score out of 24) as the primary outcome, but include the overall TOP score (out of 29) so comparisons can be made with previous similar studies.²⁹ All analyses are openly available at GitHub.⁴⁰

We preregistered a follow-up study to assess the impact of informing journals of their TOP scores in March 2020.³⁶ However, this second study was not conducted owing to the onset of the COVID-19 pandemic and the priority for journals to review and disseminate pressing COVID-19 related research. Instead, we repeated our evaluation in May 2021 to assess the adherence to the TOP guidelines during the COVID-19 pandemic, but did not pre-register the plans for this follow-up evaluation.

Results

For the eight TOP standards, journals had a median score of 5 (IQR: 2–12.5) out of a possible maximum score of 24 before the COVID-19 pandemic (February 2020), which increased to a median of 7 (IQR: 4–12) in May 2021. The highest scoring journals were Nature Medicine and Nature Neuroscience, with TOP scores of 14/24 for both evaluations (Figure 1). The Journal of the American College of Cardiology had the lowest score at each time point, receiving a TOP score of 0/24.

Figure 1.

Scores for 19 high-ranking health and medical sciences journal policies using the Transparency and Openness Promotion (TOP) guidelines in descending order for 2021 scores. The first audit was conducted on policies dated in February 2020 and the second set of policies were from May 2021. Policies had a median score of 5 (IQR: 2–12.5) out of a possible maximum score of 24 in February 2020, which increased to a median of 7 (IQR: 4–12) in May 2021. Eur.Heart J: European Heart Journal; Gastroenterol: Gastroenterology; J.Clin Oncol: Journal of Clinical Oncology; J.Am.Coll.Cardiol: Journal of the American College of Cardiology; Nat.Genet: Nature Genetics, Nat.Med: Nature Medicine; Nat.Neurosci: Nature Neuroscience; PNAS: Proceedings of the National Academy of Sciences; Sci.Transl.Med: Science Translational Medicine; BMJ: The British Medical Journal; JAMA: Journal of the American Medical Association; Lancet Oncol: The Lancet Oncology; and NEJM: The New England Journal of Medicine.

Eleven journals (58%) had no change in their TOP score following the onset of the COVID-19 pandemic. One-quarter of journals (26%, n = 5) improved their policies; Gasterenrology had the greatest improvement (from a score of 2 to a score of 9), followed by the European Heart Journal (score of 0 to a score of 5), with the remaining three journals improving by a score of 2 (The Lancet, The Lancet Oncology, and PLoS ONE). Two journals, Cell and Neuron, had a reduction in their TOP score during the COVID-19 pandemic (Figure 1).

Across the 19 medical and health sciences journal policies, the sum of scores for each TOP standard improved on average during the COVID-19 pandemic (May 2021) compared with before (Februrary 2020; Figure 2). Journal policies scored highest for their adherence to data transparency and design and analysis transparency. Policies engaged less with the requirements for preregistration of study protocols and analysis plans and the submission of replication studies. The scores for each TOP standard are summarised in Supplement 1 and Table 1.

Figure 2.

Cumulative scores on the eight Transparency and Openness Promotion (TOP) standards for 19 medical and health sciences journals evaluated before the COVID-19 pandemic (February 2020) and during the COVID-19 pandemic (May 2021) in descending order using 2021 scores.

Table 1.

Scores for the 19 health and medical sciences journal policies using the Transparency and Openness Promotion (TOP) guideline before the COVID-19 pandemic (February 2020) and during (May 2021), ordered alphabetically by Journal.

	Citation		Transparency								Study prereg		Analysis prereg		Replication		TOP signatory		Reg reports		OS badges		Overall* TOP (/29)		TOP only (/24)
Journal	Citation		Data		Code		Materials		Design & analysis		Study prereg		Analysis prereg		Replication		TOP signatory		Reg reports		OS badges		Overall* TOP (/29)		TOP only (/24)
Journal	2020	2021	2020	2021	2020	2021	2020	2021	2020	2021	2020	2021	2020	2021	2020	2021	2020	2021	2020	2021	2020	2021	2020	2021	2020	2021
Blood	0	0	2	2	0	0	2	2	0	0	0	0	0	0	0	0	0	0	0	0	0	0	4	4	4	4
BMJ	0	0	1	1	0	0	0	0	2	2	1	1	0	0	0	0	1	1	0	0	0	0	5	5	4	4
Cell	2	2	2	2	3	2	1	1	2	2	1	1	1	1	1	1	1	1	0	0	0	0	14	13	13	12
Circulation	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	0	0	0	0	9	9	8	8
European Heart Journal	0	2	0	1	0	1	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	5	0	5
Gastroenterol	0	2	1	2	0	1	1	1	0	1	0	1	0	1	0	1	0	0	0	0	0	0	2	9	2	9
J AM Coll Cardiol	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
J Clin Oncol	0	0	1	1	2	2	2	2	0	0	0	0	0	0	0	0	0	0	0	0	0	0	5	5	5	5
JAMA	0	0	0	0	0	0	0	0	2	2	0	0	0	0	0	0	0	0	0	0	0	0	2	2	2	2
Lancet	0	0	0	1	0	0	0	1	2	2	0	0	0	0	0	0	0	0	0	0	0	0	2	4	2	4
Lancet Oncol.	0	0	0	1	0	0	0	1	2	2	0	0	0	0	0	0	0	0	0	0	0	0	2	4	2	4
Nature Genetics	2	2	2	2	2	2	2	2	1	2	1	1	1	1	1	1	1	1	0	0	0	0	14	14	13	13
Nature Medicine	2	2	2	2	3	3	2	2	2	2	1	1	1	1	1	1	1	1	0	0	0	0	15	15	14	14
Nature Neuroscience	2	2	2	2	3	3	2	2	2	2	1	1	1	1	1	1	1	1	0	0	0	0	15	15	14	14
NEJM	0	0	1	1	0	0	0	0	1	1	0	0	1	1	0	0	0	0	0	0	0	0	3	3	3	3
Neuron	2	2	2	2	3	2	2	2	2	2	1	1	1	1	1	1	1	1	0	0	0	0	15	14	14	13
PLoS ONE	1	1	2	2	0	1	1	1	1	1	0	0	0	0	0	1	1	1	2	2	0	0	8	10	5	7
PNAS	1	1	2	2	2	2	2	2	1	1	1	1	1	1	1	1	0	1	0	0	0	0	11	12	11	11
Sci Transl Medicine	2	2	2	2	2	2	2	2	1	1	1	1	1	1	1	1	1	1	0	0	0	0	13	13	12	12

*The overall TOP score (out of 29) is the sum of all scores presented in the table (i.e. the eight TOP standards and three additional measures). TOP only (out of 24), sums the eight TOP standards. BMJ: The British Medical Journal; Gastroenterol: Gastroenterology; J Clin Oncol: Journal of Clinical Oncology; J Am Coll Cardiol: Journal of the American College of Cardiology; JAMA: Journal of the American Medical Association; Lancet Oncol: The Lancet Oncology; OS: open science; NEJM: The New England Journal of Medicine; PNAS: Proceedings of the National Academy of Sciences; Prereg: preregistration; Sci.Transl.Med: Science Translational Medicine; TOP: Transparency and Openness Promotion.

Additional measures

Before the COVID-19 pandemic, nine journals were signatories of TOP, which increased to 10 journals in May 2021 after PNAS joined as a TOP signatory. PLoS One was the only journal that accepted registered reports at both time points. No journals used open science badges to acknowledge the authors’ efforts to preregister their study or openly share their data, code, and study materials.

Conflicts of interests

The median ICMJE score of reporting COIs was 4 (IQR: 4–4) at both time points. Before the COVID-19 pandemic (February 2020), 84% (16/19) of journals fulfilled all four ICMJE standards for reporting COIs (Figure 3). In May 2021, this increased to 95% (18/19). Cell was the only journal that did not fulfil the fourth ICMJE criteria across both time periods, which requires authors to report “any other relationships or activities that readers could perceive to have influenced, or that give the appearance of potentially influencing, the submitted work.”

Figure 3.

Scores of high-ranking health and medical sciences journals requirements for authors to disclose conflict of interests (COIs) using the four standards in the International Committee of Medical Journal Editors (ICMJE) disclosure form. Scores were graded twice using policies dated in February 2020 (before the COVID-19 pandemic) and in May 2021 (during the COVID-19 pandemic). Eur.Heart J: European Heart Journal; Gastroenterol: Gastroenterology; J.Clin Oncol: Journal of Clinical Oncology; J.Am.Coll.Cardiol: Journal of the American College of Cardiology; Nat.Genet: Nature Genetics, Nat.Med: Nature Medicine; Nat.Neurosci: Nature Neuroscience; PNAS: Proceedings of the National Academy of Sciences; Sci.Transl.Med: Science Translational Medicine; BMJ: The British Medical Journal; JAMA: Journal of the American Medical Association; Lancet Oncol: The Lancet Oncology; and NEJM: The New England Journal of Medicine.

Discussion

The transparency and openness standards in the policies of high-ranking health and medical sciences journals were low before and during the COVID-19 pandemic. We found limited improvement in open science standards between evaluations, with most (58%) journals making no change to their policies. The preregistration of study protocols and analysis plans and encouraging replication studies were the standards in most need of improvement. In May 2021, ten journals were signatories of the TOP guidelines, with PNAS signing after the first evaluation. PLoS One was the only journal to accept registered reports, and open science badges were not used by any journals. The majority of journals endorsed ICMJE standards for disclosing COIs.

A lack of transparency in journal policies has been found across biomedical specialties.¹ Low engagement with transparent and open science standards in journal policies have been documented in research using the TOP guidelines to assess pain journals, sports science journals, and sleep and chronobiology journals. Those studies similarly found a limited requirement for authors to preregister their studies or analytic plans, and no journals assessed encouraged replication studies.^24–26

Incentives can improve engagement with transparency and openness standards.³⁷ Adopting open science badges may provide a valuable incentive for scientists to practice openly.³⁷ In a study by Kidwell et al., the authors found that four prominent psychology journals had less than 3% of articles reporting publicly accessible data before introducing open science badges. After these journals adopted open science badges, the same journals had nearly 40% of articles reporting publically accessible data.³⁸ Cashin et al.²⁴ and Spitschan et al.²⁵ found that no journal they assessed mentioned the use of open science badges, which was also the case for the health and medical sciences journals evaluated in our study.

In our study, a greater proportion of health and medical journals were TOP signatories by May 2021 (10/19) compared to the pain journals assessed by Cashin et al. in May 2019 (3/10). A positive correlation between being an ICMJE-member journals, using data-sharing statements, and expressing an intention to share individual patient data has been demonstrated previously demonstrated.³⁸ Future studies should explore whether being a TOP signatory improves the transparency and openness standards of journal policies.

TOP scores could be used to encourage journals to improve their policies. Researchers may be incentivized to submit their studies to journals that have high TOP scores, and the public may have more trust for research from such journals. However, this would require consensus for using such a metric.³⁹ Furthermore, there is a risk of journals having variable degrees of compliance with TOP guidance in practice, as demonstrated by Goldacre et al., who found extensive breaches to CONSORT guidelines that journals endorsed in their policies.⁴⁰

Journal policies that do not encourage or mandate transparent and open research reporting are likely attributable to multiple factors. To date, journals have prioritised positive results over replicability.⁴¹ The current academic publishing ecosystem incentivises quantity over quality, and as a result, researchers may not have had formal training in the tools to practice openly. While researchers may choose to use online repositories to deposit code and data, the responsibility for sharing access to study materials will remain with authors until journal policies require data citation and sharing prior to publication. If health and medical sciences journals mandate researchers to adhere to the transparency and openness standards of the TOP guidelines, this may incentivise researchers to upskill and improve the conduct and reporting of their research. Editors should use available resources to guide recommended policy wording for improved open science practice.⁴² Ongoing efforts encouraging journals to improve their transparency and openness standards will require further auditing.

Since the COVID-19 pandemic, research stakeholders have encouraged and supported improved open science practices for research specific to the pandemic.^{27,28,43–46} Despite these efforts, our study showed little improvement in the openness and transparency standards of high-ranking health and medical journals during the COVID-19 pandemic. Due to the multitude of factors that may contribute to a journal's decision to change its policies, it is difficult to determine which improvements directly result from the COVID-19 pandemic.

Strengths and limitations

While the TOP guideline is a valuable tool for assessing journals’ transparency and openness standards, there are some limitations. Firstly, the presence of imprecise and undefined terms when describing the required degree of adherence to a standard, such as ‘should’, ‘strongly encourage’, ‘recommend’ or ‘expect’, can lead to difficulties in allocating scores. To mitigate this, we conducted the scoring independently in dual and discussed scoring with a third study author to resolve differences in scoring. Our study assessed the policies of journals and not adherence of their published articles to these policies. The TOP guideline may not be applicable or flexible when examining the policies of specialized journals, such as the Cochrane Database of Systematic Reviews, which we excluded from our study. Nevertheless, the TOP guideline applies equally to journals that publish all study designs and provides a useful overview of how journals can improve the quality of research it publishes.

Future research

Our methods can be replicated by others to assess the transparency and openness standards of journals in different research fields. This information can be provided to journal editors to encourage them to improve their policies. Future research should review the degree of transparency of research published in journals to assess compliance with journal policies. Furthermore, the impact of poor transparency and open science practices on individual and societal health outcomes should be studied to demonstrate its real-world consequences.

Conclusions

We found that the 19 highly ranked health and medical sciences journals had minimal requirements for transparency and openness standards in their policies. During the COVID-19 pandemic, nominal improvements in the journal policies were observed. As the primary gatekeepers of research and evidence dissemination that impacts individual and societal health outcomes, journal policies should be regularly reviewed and improved to reflect the ongoing need for transparent and open research.

Declarations

Competing interests: ADG is an Ordinary Member of the Royal College of Physicians and Surgeons of Glasgow Trainees’ Committee. EJH, GT, CJ and ECT have no interests to disclose. AGC was financially supported by the Prince of Wales Clinical School, University of New South Wales and Neuroscience Research Australia to study for a Doctor of Philosophy (2017–2020), but no longer has any financial COIs. HL is funded by the National Health and Medical Research Council (grant no. APP1126767); NIHR Collaboration for Leadership in Applied Health Research and Care Oxford at Oxford Health NHS Foundation Trust; received project funding from the Berkeley Initiative for Transparency in the Social Sciences, a program of the Center for Effective Global Action (CEGA), with support from the Laura and John Arnold Foundation; and is a Catalyst for the Berkeley Initiative for Transparency in the Social Sciences. DN is a member of the Royal College of General Practitioners (RCGP) steering committee to support the Physical Activity and Lifestyle clinical priority. DN has received funding for research from the NHS National Institute for Health Research (NIHR) School for Primary Care Research (SPCR) and the RCGP for independent research projects related to physical activity and dietary interventions. GCR was financially supported by the NIHR SPCR, the Naji Foundation, and the Rotary Foundation to study for a Doctor of Philosophy (2017–2020) but no longer has any financial COIs. GCR is an Associate Editor of BMJ Evidence Based Medicine and is a COS Ambassador. The views expressed are those of the author and not necessarily those of the NHS, the NIHR, the RCGP or the Department of Health.

Supplemental Material

sj-docx-1-shr-10.1177_20542704221132139 - Supplemental material for Open science and conflict of interest policies of medical and health sciences journals before and during the COVID-19 pandemic: A repeat cross-sectional study

Supplemental material, sj-docx-1-shr-10.1177_20542704221132139 for Open science and conflict of interest policies of medical and health sciences journals before and during the COVID-19 pandemic: A repeat cross-sectional study by Antoni D. Gardener, Ellen J. Hick, Chloe Jacklin, Gifford Tan, Aidan G. Cashin, Hopin Lee, David Nunan, Elaine C. Toomey and Georgia C. Richards in JRSM Open

Footnotes

Funding

No funding was obtained or received for this study

Ethical approval

Not applicable

Guarantor

Dr Georgia Richards

Contributorship

GCR prepared the study protocol with HL. DN extracted the ranking of health and medical journals from Google Scholar. ADG, EH, CJ, GT, AGC, HL, DN and ECT extracted and scored journal policies. GCR resolved disagreement of TOP scores, analysed the data, and created the code and figures. GCR, HL, and DN provided supervisory support. ADG and GCR wrote the first draft manuscript, and all authors contributed, edited, and agreed to submit the final manuscript for publication. Authorship for EH, CJ, and GT (audit 1) and AGC, HL, DN, ECT (audit 2) was determined alphabetically.

ORCID iDs

Antoni D. Gardener

David Nunan

Georgia C. Richards

Acknowledgements

We acknowledge all members of the Open Pain Research Appraisal and Advocacy (OPeRA) group () for providing the idea for conducting this study. We thank Assistant Professor Sean Grant for providing clarity on scoring. No funding was provided or obtained for this study.

Supplemental material

Supplemental material for this article is available online.

References

Wallach

Boyack

Ioannidis

JPA

. Reproducible research practices, transparency, and open access data in the biomedical literature, 2015–2017. PLoS Biol [Internet] 2018;16(11):e2006930. doi:10.1371/journal.pbio.2006930.

Ioannidis

JPA

Greenland

Hlatky

, et al. Increasing value and reducing waste in research design, conduct, and analysis. The Lancet 2014;383(9912):166–175.

Vines

Andrew

Bock

, et al. Mandated data archiving greatly improves access to research data. FASEB J Off Publ Fed Am Soc Exp Biol 2013;27(4):1304–1308.

Kim

Adler

. Social scientists’ data sharing behaviors: investigating the roles of individual motivations, institutional pressures, and data repositories. Int J Inf Manag 2015;35(4):408–418.

Bueno de la Fuente

. What is Open Science? Introduction [Internet]. Foster Open Science. [cited 2021 Apr 20]. Available from: https://www.fosteropenscience.eu/content/what-open-science-introduction

Goldacre

Morton

DeVito

. Why researchers should share their analytic code. Br Med J 2019;367:l6365.

Baker

. 1,500 Scientists lift the lid on reproducibility. Nat News 2016;533(7604):452.

Ioannidis

JPA

. Why most published research findings are false. PLoS Med 2005;2(8):e124.

Peng

. The reproducibility crisis in science: a statistical counterattack. Significance 2015;12(3):30–32.

10.

Mehra

Ruschitzka

Patel

. Retraction—Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis. The Lancet 2020;395(10240):1820.

11.

Mehra

Desai

Kuy

Henry

Patel

. Cardiovascular Disease, Drug Therapy, and Mortality in Covid-19 | NEJM [Internet]. 2020 [cited 2021 Feb 15]. Available from: https://www.nejm.org/doi/full/10.1056/NEJMoa2007621

12.

Harrison

Lopez

Rahman

, et al. The COVID-19 data portal: accelerating SARS-CoV-2 and COVID-19 research through rapid open access data sharing. Nucleic Acids Res 2021;49(W1):W619–W623.

13.

Besançon

Peiffer-Smadja

Segalas

, et al. Open science saves lives: lessons from the COVID-19 pandemic. BMC Med Res Methodol 2021;21:117.

14.

Elsevier. COVID-19 resources for librarians, campuses and health professionals [Internet]. Elsevier Connect. [cited 2022 Jan 5]. Available from: https://www.elsevier.com/connect/coronavirus-initiatives

15.

GitHub: Where the world builds software [Internet]. GitHub. [cited 2020 Nov 26]. Available from: https://github.com/.

16.

OSF Registries [Internet]. [cited 2021 May 3]. Available from: https://osf.io/registries

17.

PROSPERO International prospective register of systematic reviews [Internet]. Available from: https://www.crd.york.ac.uk/prospero/

18.

Centre for Open Science. Open Science Badges enhance openness, a core value of scientific practice. [Internet]. [cited 2021 Apr 20]. Available from: https://www.cos.io/initiatives/badges.

19.

Bradley

DeVito

Lloyd

, et al. Reducing bias and improving transparency in medical research: a critical overview of the problems, progress and suggested next steps. J R Soc Med 2020;113(11):433–443.

20.

Bornmann

Mutz

. Growth rates of modern science: a bibliometric analysis based on the number of publications and cited references. J Assoc Inf Sci Technol 2015;66(11):2215–2222.

21.

DOAJ. Principles of Transparency and Best Practice in Scholarly Publishing, version 3 [Internet]. DOAJ News Service. 2018 [cited 2022 Jul 27]. Available from: https://blog.doaj.org/2018/01/15/principles-of-transparency-and-best-practice-in-scholarly-publishing-version-3/

22.

Nosek

Alter

Banks

, et al. Promoting an open research culture. Science 2015;348(6242):1422–1425.

23.

Zarin

Tse

Williams

Rajakannan

. The Status of trial registration eleven years after the ICMJE policy. N Engl J Med 2017;376(4):383–391.

24.

Cashin

Bagg

Richards

Toomey

McAuley

Lee

. Limited engagement with transparent and open science standards in the policies of pain journals: a cross-sectional evaluation. BMJ Evid-Based Med 2020;26(6):313–319.

25.

Spitschan

Schmidt

Blume

. Principles of open, transparent and reproducible science in author guidelines of sleep research and chronobiology journals. Wellcome Open Res [Internet]. 2021 Feb 26 [cited 2021 Mar 31];5. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7941095/

26.

Hansford

Cashin

Wewege

Ferraro

McAuley

Jones

. Evaluation of journal policies to increase promotion of transparency and openness in sport science research. Arthroscopy 2021;37(11):3223–3225.

27.

Health

TLD

. Transparency during global health emergencies. Lancet Digit Health 2020;2(9):e441.

28.

Waltman

Pinfield

Rzayeva

, et al. Scholarly communication in times of crisis: The response of the scholarly communication system to the COVID-19 pandemic [Internet]. Research on Research Institute; 2021 Dec [cited 2022 Jan 10]. Available from: https://rori.figshare.com/articles/report/Scholarly_communication_in_times_of_crisis_The_response_of_the_scholarly_communication_system_to_the_COVID-19_pandemic/17125394/1

29.

Lee

Richards

Nunan

, et al. Transparency and open science standards in the policies of medical and health science journals [Internet]. OSF 2020. [cited 2021 Apr 20]. Available from: https://doi.org/10.17605/OSF.IO/HX5CA

30.

Health & Medical Sciences - Google Scholar Metrics [Internet]. [cited 2020 Nov 26]. Available from: https://scholar.google.com/citations?view_op = top_venues&hl = en&vq = med

31.

Lee

Richards

Nunan

, et al. Transparency and open science standards in the policies of medical and health science journals [Internet]. OSF 2020. [cited 2021 May 3]. Available from: https://doi.org/10.17605/OSF.IO/H2XUD

32.

The TOP Guidelines were created by journals, funders, and societies to align scientific ideals with practices. [Internet]. [cited 2020 Nov 28]. Available from: https://www.cos.io/initiatives/top-guidelines

33.

Guidelines for Transparency and Openness Promotion (TOP) in Journal Policies and Practices “The TOP Guidelines” Version 1.0.1 [Internet]. Open Sciences Framework. Available from: https://osf.io/ud578/

34.

Disclosure of Interest (Updated February 2021) [Internet]. International Committee of Medical Journal Editors. [cited 2021 Apr 20]. Available from: http://www.icmje.org/disclosure-of-interest/

35.

Richards

. Transparency and open science standards in the policies of medical and health science journals: georgiarichards/TOP_medjournals [Internet]. Github. 2021. 2021 [cited 2021 Apr 20]. Available from: https://doi.org/10.5281/zenodo.5722554

36.

Lee

Richards

Nunan

, et al. Transparency and open science standards in the policies of medical and health science journals [Internet]. OSF 2020. [cited 2021 Dec 12]; Available from: https://doi.org/10.17605/OSF.IO/EHC27

37.

Kidwell

Lazarević

Baranski

, et al. Badges to acknowledge open practices: a simple, low-cost, effective method for increasing transparency. PLOS Biol 2016;14(5):e1002456.

38.

Siebert

Gaba

Caquelin

, et al. Data-sharing recommendations in biomedical journals and randomised controlled trials: an audit of journals following the ICMJE recommendations. BMJ Open 2020;10(5):e038887.

39.

Lash

. Getting over TOP. Epidemiology 2022;33(1):1–6.

40.

Goldacre

Drysdale

Dale

, et al. COMPare: a prospective cohort study correcting and monitoring 58 misreported trials in real time. Trials 2019 Feb 14;20(1):118.

41.

Hunter

. The reproducibility “crisis”. EMBO Rep 2017;18(9):1493–1496.

42.

Sample Implementation of Guidelines for Transparency and Openness Promotion (TOP) in Journal Policies and Practices. 2014 Aug 12 [cited 2022 Jan 11]; Available from: https://osf.io/edtxm/

43.

Reflections on a Year of COVID-19 Data Sharing | NIH: National Institute of Allergy and Infectious Diseases [Internet]. [cited 2022 Jan 10]. Available from: https://www.niaid.nih.gov/news-events/year-covid-19-data-sharing

44.

NIAID Clinical Trials Repository [Internet]. [cited 2022 Jan 10]. Available from: https://accessclinicaldata.niaid.nih.gov/

45.

Variant Therapeutic in vivo Variant Studies [Internet]. [cited 2022 Jan 10]. Available from: https://opendata.ncats.nih.gov/variant/in-vivo.

46.

Global coalition to accelerate COVID-19 clinical research in resource-limited settings. The Lancet 2020;395(10233):1322–1325.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.01 MB