Interventions for improving the design and conduct of scientific research: A scoping review

Abstract

Background

Research waste is a costly problem for scientific research, with poor design and conduct of the research being key elements which contribute to wastage. Interventions to address poor design and conduct may save time and money. The objective of the study was to map the interventions that have been evaluated for improving the design or conduct of scientific research to identify any gaps in the evidence.

Methods

We undertook a systematic scoping review. We searched MEDLINE, EMBASE, EconLit, ERIC, Social Policy and Practice, HMIC, ProQuest Dissertations and Theses Global and MetaArXiv from 1^st January 2012 to 13^th June 2022. Evaluated interventions that aimed to improve the design or conduct of scientific research by targeting researchers or research teams were included. Screening was completed by two reviewers and data charting by a single reviewer with another reviewer checking.

Results

A total of 81 evaluated interventions were included. Most of the interventions targeted research conduct, primarily focussed on registration, publishing, and reporting. Most included studies used observational evaluation methods. Categorising the interventions by the behaviour change wheel framework we found that most studies utilised restriction, coercion, and persuasion and fewer used enablement, training, or incentivisation to achieve their aims.

Conclusions

More evaluations of interventions aimed at how researchers design their research are needed, these should be developed appropriately and evaluated for effectiveness using experimental methods.

Keywords

Research designs & methods research conduct intervention study design research design scoping review planning the research the scientific method basic concepts of research

Introduction

Estimates suggest that around 85% of research is “waste”, meaning that the research is addressing unimportant questions, not addressing important questions properly, or the results are just not available.¹ Chalmers and Glaziou suggested four areas where this waste could be addressed: question and priority setting to identify research questions important to patients-public and clinicians; appropriate study design and methods; full reporting of results; and unbiased and usable reports.¹ Some steps have been taken to improve practice across these four areas. For example, awarding online badges to reward data sharing²; legal requirements for publication³; funder recommendations for design choices⁴; email reminders for increasing publication⁵; providing training and tools to researchers to allow them to design better studies⁶ However, research waste persists across all areas.^7,8

Hardwicke et al.⁹ proposed a four-stage framework for improving the efficiency, quality and credibility of research practice: 1) identify problems; 2) investigate problems; 3) develop solutions; 4) evaluate solutions. They note that many solutions are developed without any consideration for evaluation, resulting in mainly retrospective observational evaluations; where unintended consequences or lack of anticipated benefits may go undetected.

In advance of undertaking primary research to improve research practice in the development of appropriate design and conduct, we wanted to establish what interventions had previously been evaluated. There has been no overarching systematic review of interventions to improve the design and conduct of research or how these interventions are evaluated. Therefore, the aim of this research was to undertake a systematic scoping review to map existing evaluations of interventions which aim to improve the design or conduct of research.

We will use this review to identify knowledge gaps and to assess how this type of research could be improved.

Objectives

The aim of this scoping review is to address the below questions.

• What interventions, aimed at researchers or research teams, to improve the design and conduct of research have been evaluated?

o Where in the research pathway are these interventions implemented? (e.g. protocol development, registration, publishing)

o What specific aspects of research are being targeted? (e.g. research knowledge, improved methods used, compliance with a standard.)

o Who is implementing these interventions? (e.g. funders, ethics committees, research institutions, publishers.)

Methods

The protocol of this scoping review has been registered with the Open Science Framework registry¹⁰ and has also been published.¹¹ The PRISMA reporting guidance extension for scoping reviews ¹² has been used, a completed checklist is available in the supplementary materials.¹³ All supplementary materials are available on the Figshare repository with individual documents cited with links where relevant.

Data and analysis code for this study is available in the supplementary materials.^14,15

Eligibility criteria

Any interventions that aimed to improve the design or conduct of scientific research by targeting researchers or research teams were included.

Population: Researchers or research teams undertaking or developing research projects. Studies where the intervention is aimed at those participating in research (such as Studies Within A Trial) were excluded.

Concepts: For the purposes of this review an intervention was defined, as per the behaviour change wheel framework, as any coordinated sets of activities designed to change specified behaviour patterns.¹⁶ This could include policies, laws, and guidelines provided there was a reported evaluation of the intervention.

Interventions aimed at any aspect of design and conduct were included, from initial design (e.g. question setting, protocol development), through undertaking the research (e.g. registration, changes to research design), and publishing (e.g. timely publication, reporting standards, data sharing).

Context: This review included any interventions aimed at scientific research. It did not include interventions designed to improve hypothetical research projects. There were no other limitations on the context that the interventions are applied in.

Types of Studies: This review included any quantitative or qualitative evaluation of interventions. No limitations were made by study type provided the study was an evaluation of the intervention.

Searches

MEDLINE, EMBASE, EconLit, ERIC, and Social Policy and Practice were searched along with the following grey literature sources: HMIC, ProQuest Dissertations and Theses Global and articles submitted to the MetaArXiv preprint server. These were searched from 1^st January 2012 up to 13^th June 2022.

The full search strategy is available in the supplementary materials.¹⁷ No limits were applied relating to language or source of information. References within included articles were screened by a single reviewer to identify any further relevant articles that were undertaken.

Selection of sources of evidence

All records identified were de-duplicated using Endnote¹⁸ then imported into Covidence Systematic Review software.¹⁹

A two-stage screening process was used, first screening the titles and abstracts of all identified studies and then screening the full-texts of any that appeared eligible at the first stage. Both stages were completed independently by two researchers with any disagreements resolved by discussion.

This process was piloted initially to ensure agreement in decisions.

Data charting process

Data charting was completed using a data extraction template within Covidence. This process was done by one researcher with a second independently checking, with any disagreement being resolved through discussion. This process was piloted on a small number of initial studies to ensure consistency and that all pertinent data was captured using the data extraction template.

Data were collected on the details of the authors and report, interventions, methods of evaluation, and outcomes measured.

The intervention functions as assessed using the Behaviour Change Wheel Framework ¹⁶ were classified by two reviewers based on the below criteria:

• Education – providing knowledge of practices.

• Persuasion - communication for improving actions.

• Incentivisation – providing expectation of reward.

• Coercion – providing expectation of punishment.

• Training - providing skills.

• Restriction - rules or regulation

• Environmental restructuring - changing physical or social context.

• Modelling - showing examples of good practice.

• Enablement - reducing barriers/increasing means to practice.

• Other - Anything not captured by framework.

By using this framework, the types of intervention used can be assessed more broadly even if the research areas are not necessarily similar. It also allows identification of interventions which are potentially less which could be tested in future research.

Synthesis

A narrative synthesis is presented for each of the review aims. Relevant data are presented in tables and graphs. These are stratified by whether the intervention focussed on aspects of design, conduct, or both.

There was no planned critical appraisal as this was not one of the aims of this review, however the study design of all included studies was extracted in order to map what methods are used to evaluate these interventions.

Results

The titles and abstracts of 11,540 records and 178 full texts were screened. Eighty-one eligible studies were included in the review (Figure 1). A table of the included studies is provided in the supplementary materials.²⁰

Figure 1.

Flowchart of Study Selection process.

Details of the studies excluded at full-text review and the reasons for exclusion can be found in the supplementary materials.²¹ There were 30 that described an intervention but did not evaluate it; 29 described or investigated design or conduct problems with no reference to an intervention; 12 were not aimed at researchers or research teams (e.g. an intervention aimed at clinical teams or patients); and eight were review articles.

Characteristics

A summary of the characteristics of the included studies is provided in Table 1.

Table 1.

Characteristics of included studies.

Study characteristics (n = 81)	Count (%)
Study design
Audit	4 (4.9)
Before & after	13 (16)
Case study	1 (1.2)
Cohort	9 (11.1)
Cross sectional	38 (46.9)
Qualitative	3 (3.7)
Quasi-experimental	4 (4.9)
Randomised controlled trial	7 (8.6)
Survey	2 (2.5)
Intervention aim
Design	4 (4.9)
Conduct	73 (90.1)
Both	4 (4.9)
Continent
Asia	3 (3.7)
Australia & New Zealand	4 (4.9)
Central & south America	1 (1.2)
Europe	22 (27.2)
North America	28 (34.6)
Multiple countries	23 (28.4)
Funding
Publicly funded	41 (50.6)
Industry funded	2 (2.5)
Unfunded	23 (28.4)
Not reported	15 (18.5)

The majority of included studies (n = 67, 82.7%) used an observational study design seven used a randomised controlled trial and four used a quasi-experimental design. Most of the studies were conducted in Europe, North America, or across multiple countries. The source of funding for the studies was reported in 66 (81.5%) of the studies with 43 (53.1%) being funded and 23 (28.4%) being unfunded.

The 81 included studies assessed 41 different interventions, the most commonly assessed was journal reporting guidelines or endorsement of specific guidelines (e.g. CONSORT, PRISMA), which was assessed in 13 evaluations.

Synthesis

What Interventions have been evaluated?

Design or conduct

Seventy-three of the included studies assessed interventions focussed solely on the conduct of the research, four were focussed solely on the design, and four covered both design and conduct.

Types of intervention

Each of the interventions was characterised as per the behaviour change wheel intervention function framework.

Persuasion, restriction, and coercion were the most used intervention functions overall. Fewer interventions used training, enablement, or incentivisation. The most common intervention function utilised for design-focussed interventions was enablement in design and restriction for conduct-focussed interventions.

Some examples of the intervention functions used in the included studies were:

• Education: Expert consultation to advise and educate researchers²²

• Persuasion: Policies endorsing best practices²³

• Incentivisation: The offer of badges on publications to indicate use of good practices.²

• Coercion: The threat of fines for not publishing results.³

• Training: Training courses or standard operating procedures to follow²⁴

• Enablement: Provision of software or tools to enable good practices.⁶

• Modelling: provision of reporting checklists with best practice examples (e.g. CONSORT)²⁵

• Environmental Restructuring: Prompting emails to remind researchers of requirements with time limits.⁵

The number of uses of each intervention function stratified by design and conduct is presented in Table 2.

Table 2.

Frequency of behaviour change wheel functions by intervention aim.

	Education	Persuasion	Incentivisation	Coercion	Training	Enablement	Modelling	Environmental restructuring	Restriction
Design	1	1	0	1	1	2	0	1	1
Conduct	13	30	7	22	3	5	19	18	46
Both	3	1	0	0	3	1	1	0	0

Some interventions used multiple intervention functions.

Where in the research pathway have these interventions been evaluated?

Whilst there were interventions which targeted all stages of the research pathway the vast majority of those identified focussed on reporting (n = 36), publishing (n = 20) and data publishing (n = 12). A summary of the stages of the research pathway targeted by the interventions stratified by design and conduct is presented in table 3.

Table 3.

Frequency of interventions aimed at different stages in the research pathway.

	Idea development	Grant writing or development>	Protocol development	Study registration	Protocol publishing	Mid study conduct	Reporting	Publishing	Data publishing
Design	0	1	3	0	0	0	0	0	0
Conduct	0	0	0	19	1	2	35	29	12
Both	1	1	3	0	0	3	1	1	0

Some interventions focused on multiple stages.

What aspects of research have been targeted?

The outcomes assessed in the included studies were reviewed to inform mapping of the aspects of research targeted by the interventions. The outcomes measured are summarised in table 4.

Table 4.

Outcomes Assessed stratified by aim.

	Number of evaluations
Design
Appropriate inclusion of participants in research	1
Quality of protocol	1
Self-reported uptake of design practices	1
Time to produce protocol	1
Use of core outcome sets	1
Conduct
Reporting to a reporting guideline	24
Adherence to registration requirement	18
Data availability	11
Publication of results	10
Adherence to multiple open science practices	4
Data sharing statement/Commitment	4
Presence of spin in reporting	3
Appropriate inclusion of participants in research	1
Changes in reported attitude towards good practices	1
Code availability	1
Inclusion of PPI activity in reporting	1
Open access publication	1
Participant described experience of intervention	1
Parts or type of intervention utilised	1
Protocol availability	1
Statistical significance of published results	1
Both
Appropriate inclusion of participants in research	1
Research expectations	1
Research knowledge and skills	1
Topics discussed with experts	1
Usage of provided resources	1

Some studies measured multiple outcomes.

Most of the studies measured exactly what the intended effect of the intervention was, for example reporting to a particular standard for a reporting checklist/guidance²⁶ or meeting the study registration requirements of a policy.²⁷ Five studies assessed the researcher’s experience of the intervention either in how much it was used ,²⁴ what aspects of the intervention were most utilised,²⁸ or the participant’s described experiences of the intervention.²⁹

For studies focussed on design, the uptake of the target research design practice was assessed in several different ways: self-reported usage of design practices in being surveyed on their usage⁶; usage of the practice at the funding stage with funding applications being assessed⁴; uptake of design practices assessed in publications retrospectively ³⁰; and in a generic assessment of protocol quality and time to produce the protocol.³¹

One study used validated measures to assess changes in research knowledge and skills and research expectations following the intervention.³² They measured the Research Outcome Expectations Questionnaire and Research Knowledge and Skills Self-Assessment measure to assess a virtual laboratory intervention.

Who is implementing these interventions?

Most interventions were implemented by journals (n = 49), the different organisations and groups are summarised in Figure 2. Sixty-nine of the interventions involved a single organisation implementing them, 12 involved two organisations. Examples of this include a university initiative that aligns with government legislation³³ and government legislation and funder requirements aligning at the same time.

Figure 2.

Summary of implementing organisations.

Most of the evaluations were not completed by the organisation that implemented the intervention (n = 52, 64.2%). When the evaluations implemented by independent researchers are removed (n = 7), then 70% of the included evaluations were not completed by the implementing organisation.

Discussion

Summary of evidence

This review identified 81 evaluated interventions which were primarily aimed at aspects of research conduct rather than research design. This scoping review has identified a key gap in the evidence; whilst errors in publishing and reporting can be rectified after the fact it is often too late to fix errors in design once research is complete. Future research should have a stronger focus on how the design of research can be improved. This could be done by expanding upon the existing evidence in this review or development of new interventions. For example, Weeks et al.⁶ tested the effect of providing resources and training regarding research design within one institution, this could be replicated in other institutions as the resources used were freely available and provided by the authors.

The ways in which improvement in design is measured may also need to be considered as the studies identified in this research primarily relied upon targeting one specific design aspect or self-reporting of usage of best practices in design. Most of the research focussed on conduct used an outcome based on adherence to a set guideline or standard which covered multiple important aspects of the research (e.g. the CONSORT reporting standard), utilisation of similar standards for assessing study design would allow assessment of broader impact than on individual design aspects.

The most common intervention functions utilised were restriction, persuasion, and coercion, one possible explanation for this is that it may be easier to suggest or mandate a particular standard to be met than to provide resources, education, or training to help meet these standards. Development of any future interventions could draw on the guidance for development of complex health and care interventions by properly developing a theory of the problem to assess whether restrictive or persuasive techniques are appropriate or whether researchers do not have the skills or tools to meet new standards and therefore other intervention functions should be utilised.³⁴

There is also potential that some of the interventions already tested need to be developed further. The most common incentive intervention, a digital badge accompanying article publication, has had mixed results and has received criticism regarding its implementation.^35,36 Further research into what has caused the mixed results or how the intervention could be improved is needed.

One notable finding is that there were 30 records describing interventions which were not evaluated and were therefore excluded from the review as they did not meet the eligibility criteria. Some of these interventions have been evaluated articles which were included, such as the use of trial registration guidance³⁷ and the ICMJE data sharing requirements.³⁸ However, there are many that appear to have no evaluation at all such as new publishing models,³⁹ laws aimed at reducing conflicts of interest in research,⁴⁰ the use of specified staff for the oversight of animal research.⁴¹

Hardwicke et al⁹ suggested that most evaluations aimed at researchers would be limited to retrospective observational designs due to the lack of planning for evaluation. Our findings align with this as most of the included evaluations used observational methods. We also identified 29 records which detailed an evaluation without intervention, which would likely be categorised under “investigating problems” in Hardwicke et al’s framework. As with the articles that did not have evaluations, many of the problems identified have not had evaluations of interventions published to address them. Some of these include: solutions for reproducibility of results,⁴² increasing public involvement in research,⁴³ and implementation of Good Laboratory practices.⁴⁴ This suggests that whilst there is work to identify and characterise problems and develop solutions, the final steps of appropriate evaluation and implementation of these solutions are not being addressed for a large portion of potential solutions.

Furthermore, the implementing organisations were rarely involved in the evaluations; suggesting that not only had they not planned for an evaluation, but there was no planned framework to ensure that any issues identified during evaluation were dealt with to improve the implementation. One example of this occurring is identified by DeVito et al ,³ who assessed compliance with the Food and Drug Administration Amendments Act (FDAAA) of 2007. In their study they note that despite poor compliance, the implementing organisation (FDA) had yet to implement any of the enforcement actions available to it that were key aspects of the intervention, such as fines for not complying.

Most of the interventions evaluated were implemented by journals. The cause for this may be that the data for assessment is often publicly available so independent researchers can evaluate the effect of journal policies and practices more easily than that of funders, ethics committees, or governments who may not have such easily accessible data. However, this results in far more research being done on reporting and data sharing than on other aspects of research which may be more fundamental to the reliability of the results, such as design choices and data management.

Finally, there were some areas such as reporting or publishing where there was substantially more research than in other areas, often with unnecessary duplication of very similar work. This along with the lack of testing of some interventions suggests that perhaps a resource for helping researchers to identify what problems need further research and what areas have sufficient research either completed or ongoing. There has been some success with platforms for addressing this in other areas, such as the PROSPERO platform for pre-registration of systematic reviews.⁴⁵

Limitations

The searches in this review were limited to those published after 2012. It is possible that there are interventions evaluated prior to this and thus some evaluated interventions may not be included.

Whilst this review has identified many interventions that have not been evaluated there are potentially further interventions that have been implemented and not evaluated. These interventions may differ in the type of intervention, the organisations implementing them, and the area of research they target. It is also possible that many organisations internally evaluate their interventions but do not publish the results.

Conclusions

Very few interventions have been evaluated for improving how researchers do research, with only a small proportion focussing on the design of research. There are many suggested interventions that remain untested and identified problems with no developed solutions. The evaluations that are available are often insufficient to determine the effectiveness of the intervention. Future research in this area should focus on interventions aimed at the design of research.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Institute for Health Research (NIHR) under its Pre-Doctoral Fellowship (NIHR301991) to Andrew Mott. The views expressed in this publication are those of the author(s) and not necessarily those of the NIHR, NHS or the UK Department of Health and Social Care.

ORCID iDs

Andrew Mott

Helen Fulbright

References

Chalmers

Glasziou

. Avoidable waste in the production and reporting of research evidence. Lancet 2009; 374: 86–89.

Rowhani-Farid

Aldcroft

Barnett

. Did awarding badges increase data sharing in BMJ Open? A randomized controlled trial. R Soc Open Sci 2020; 7: 191818.

DeVito

Bacon

Goldacre

. Compliance with legal requirement to report clinical trial results on Clinicaltrials.gov: a cohort study. Lancet 2020; 395: 361–369.

Hughes

Kirkham

Clarke

, et al. Assessing the impact of a research funder’s recommendation to consider core outcome sets. PLoS One 2019; 14: e0222418.

Maruani

Boutron

Baron

, et al. Impact of sending email reminders of the legal requirement for posting results on Clinicaltrials.gov: cohort embedded pragmatic randomized controlled trial. BMJ 2014; 349: g5579.

Weeks

Henstridge

Salim

, et al. CORP: practical tools for improving experimental design and reporting of laboratory studies of cardiovascular physiology and metabolism. Am J Physiol Heart Circ Physiol 2019; 317: H627–H639.

Glasziou

Chalmers

. Research waste is still a scandal—an essay by Paul Glasziou and Iain Chalmers. BMJ 2018; 363: k4645. DOI: 10.1136/bmj.k4645, Epub ahead of print 12 November 2018.

Van Calster

Wynants

Riley

, et al. Methodology over metrics: current scientific standards are a disservice to patients and society. J Clin Epidemiol 2021; 138: 219–226. DOI: 10.1016/j.jclinepi.2021.05.018, Epub ahead.

Hardwicke

Serghiou

Janiaud

, et al. Calibrating the scientific ecosystem through meta-research. Annu Rev Stat Appl 2020; 7: 11–37.

10.

Mott

. Interventions for improving the design and conduct of scientific research: a scoping review. NIHR Open Res. 2022; 2: 4. Epub ahead of print 10 January 2022. DOI: 10.17605/OSF.IO/257XV.

11.

Mott

McDaid

Hewitt

, et al. Interventions for improving the design and conduct of scientific research: a scoping review protocol. NIHR Open Res 2022; 2: 4.

12.

Tricco

Lillie

Zarin

, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 2018; 169: 467–473.

13.

Mott

Reporting Checklist - PRISMA- SCR. 15 August 2023. DOI: 10.6084/M9.FIGSHARE.23957886.

14.

Mott

Project Analysis code. 15 August 2023. DOI: 10.6084/M9.FIGSHARE.23957910.

15.

Mott

Scoping review data. 15 August 2023. DOI: 10.6084/M9.FIGSHARE.23957916.

16.

Michie

Atkins

West

. The behaviour change wheel. Bream, England: Silverback Publishing, 2014.

17.

Mott

Scoping review full search strategy. 25 January 2022. DOI: 10.6084/M9.FIGSHARE.19053107.

18.

The EndNote Team . EndNote, 2013.

19.

Covidence, https://www.covidence.org 2022.

20.

Mott

Supplementary materials - included Studies. 18 August 2023. DOI: 10.6084/M9.FIGSHARE.23984505.

21.

Mott

Supplementary materials - excluded Records. 15 August 2023. DOI: 10.6084/M9.FIGSHARE.23957808.

22.

Carter-Edwards

Cook

McDonald

, et al. Report on CTSA consortium use of the community engagement consulting service. Clin Transl Sci 2013; 6: 34–39.

23.

Sarkis-Onofre

Poletto-Neto

Cenci

, et al. Impact of the CONSORT Statement endorsement in the completeness of reporting of randomized clinical trials in restorative dentistry. J Dent 2017; 58: 54–59.

24.

Rak

Matthews

Peña

, et al. Priority Populations Toolkits: enhancing researcher readiness to work with priority populations. J Clin Transl Sci 2020; 4: 28–35.

25.

Hopewell

Boutron

Altman

, et al. Impact of a web-based tool (WebCONSORT) to improve the reporting of randomised trials: results of a randomised controlled trial. BMC Med 2016; 14: 199.

26.

Hopewell

Ravaud

Baron

, et al. Effect of editors’ implementation of CONSORT guidelines on the reporting of abstracts in high impact medical journals: interrupted time series analysis. BMJ 2012; 344: e4178.

27.

Scott

Rucklidge

Mulder

. Is mandatory prospective trial registration working to prevent publication of unregistered trials and selective outcome reporting? An observational study of five psychiatry journals that mandate prospective clinical trial registration. PLoS One 2015; 10: e0133718.

28.

de Jong

van Zwieten

MCB

Willems

. Research monitoring by US medical institutions to protect human subjects: compliance or quality improvement? J Med Ethics 2013; 39: 236–241.

29.

Andresen

Laursen

Rosenberg

. Outlining and dictating scientific manuscripts is a useful method for health researchers: a focus group interview. SAGE Open Med 2018; 6: 2050312118778728.

30.

Geller

Koch

Roesch

, et al. The more things change, the more they stay the same: a study to evaluate compliance with inclusion and assessment of women and minorities in randomized controlled trials. Acad Med 2018; 93: 630–635.

31.

Juergens

Niemeijer

Jennings-Antipov

, et al. Evaluation of a novel cloud-based software platform for structured experiment design and linked data analytics. Sci Data 2018; 5: 180195.

32.

Stadtlander

Giles

Sickel

. The virtual research lab: research outcome expectations, research knowledge, and the graduate student experience. Journal of Educational Research and Practice 2013; 3: 120–138.

33.

Keyes

Mayo-Wilson

Nuamah

, et al. Creating a program to support registering and reporting clinical trials at johns Hopkins university. Acad Med 2021; 96: 529–533.

34.

Skivington

Matthews

Simpson

, et al. A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. BMJ 2021; 374: n2061. DOI: 10.1136/bmj.n2061, Epub ahead of print 30.

35.

Bastian

. Bias in open science advocacy: the case of article badges for data sharing. 2017 https://absolutelymaybe.plos.org/2017/08/29/bias-in-open-science-advocacy-the-case-of-article-badges-for-data-sharing/(accessed 5 February 2024).

36.

Research data . Why digital badges don’t work. https://www.nature.com/nature-index/news/digital-badges-not-enough-incentive-researchers-scientists-to-share-their-data-journal-publishing (accessed 5 February 2024).

37.

El-Boghdadly

Wiles

Atton

, et al. Adherence to guidance on registration of randomised controlled trials published in Anaesthesia. Anaesthesia 2018; 73: 556–563.

38.

Siebert

Gaba

Caquelin

, et al. Data-sharing recommendations in biomedical journals and randomised controlled trials: an audit of journals following the ICMJE recommendations. BMJ Open 2020; 10: e038887.

39.

Wright

Williams

Bryce

, et al. A novel approach to sharing all available information from funded health research: the NIHR Journals Library. Health Res Pol Syst 2018; 16: 70. DOI: 10.1186/s12961-018-0339-4, Epub ahead of print December 2018.

40.

Chatterjee

Xavier

Saha

. The role of the sunshine act in reducing conflict of interest in medical research and patient care. Ethics Biol Eng Med 2013; 4: 103–119.

41.

Mohan

Barbee

Silk

. Contribution of the breadth and depth of IACUC membership to experimental design as a factor in research reproducibility. J Am Assoc Lab Anim Sci 2018; 57: 104–109.

42.

Karczewski

Tatonetti

Manrai

, et al. Methods to ensure the reproducibility of biomedical research. Pac Symp Biocomput 2017; 22: 117–119.

43.

Boote

Twiddy

Baird

, et al. Supporting public involvement in research design and grant development: a case study of a public involvement award scheme managed by a National Institute for Health Research (NIHR) Research Design Service (RDS). Health Expect 2015; 18: 1481–1493.

44.

Jena

Chavan

. Implementation of Good Laboratory Practices (GLP) in basic scientific research: translating the concept beyond regulatory compliance. Regul Toxicol Pharmacol 2017; 89: 20–25.

45.

Stewart

Moher

Shekelle

. Why prospective registration of systematic reviews makes sense. Syst Rev 2012; 1: 7. DOI: 10.1186/2046-4053-1-7, Epub ahead of print December 2012.