Reducing bias and improving transparency in medical research: a critical overview of the problems,progress and suggested next steps

Abstract

In recent years there has been increasing awareness of problems that have undermined trust in medical research. This review outlines some of the most important issues including research culture, reporting biases, and statistical and methodological issues. It examines measures that have been instituted to address these problems and explores the success and limitations of these measures. The paper concludes by proposing three achievable actions which could be implemented to deliver significantly improved transparency and mitigation of bias. These measures are as follows: (1) mandatory registration of interests by those involved in research; (2) that journals support the ‘registered reports’ publication format; and (3) that comprehensive study documentation for all publicly funded research be made available on a World Health Organization research repository. We suggest that achieving such measures requires a broad-based campaign which mobilises public opinion. We invite readers to feedback on the proposed actions and to join us in calling for their implementation.

Keywords

Research and publication ethics statistics and research methods

Introduction

Significant problems in medical research undermine efficient scientific discovery and efforts to achieve improved outcomes for patients. Across scientific disciplines, a diverse range of issues has resulted in a situation where published research findings frequently cannot be replicated (‘the reproducibility crisis’). Meanwhile, avoidable methodological failings and biases lead to ‘research waste’, which is estimated to account for 85% of all medical research funding.¹

Many of these problems are rooted in incentives for individual medical researchers which are poorly aligned with the wider interests of science. Despite longstanding calls for ‘less research, better research, and research done for the right reasons’, concerns remain that quantity is valued over quality.² Cultural problems in medical research are important in themselves but also impede progress in tackling biases in how research is reported and objectively presented.

This review introduces some of the most pervasive problems in medical research along with an overview of the current efforts to address these issues. We begin by describing some of the most significant problems related to research culture, reporting biases, and statistical and methodological issues. We then outline some important measures that have been instituted to address these problems. We conclude with a proposed strategy to help restore confidence in the reproducibility of medical research.

Methods

This paper was informed by literature identified using a search strategy with the following terms: open science, open research, research quality, medic*, health*. Searches were conducted on CINAHL, EMBASE, MEDLINE and PsycINFO (Supplement 1).

What are the problems in medical research?

Research culture

Academic rewards system

Academics’ professional standing depends on demonstrating productivity through publication,³ with disproportionate rewards offered to those who attain publication in ‘luxury journals’ with high impact factors.⁴ Journal impact factors do little to capture the quality or value of individual research articles and can be manipulated.⁵ The extraordinary proliferation of research appears to reflect the pressure for academics to publish research rather than the development of genuine discovery which could lead to improved outcomes for patients.⁶

Conflicts of interests

Financial conflicts of interest have been shown to affect physician-prescribing habits, study conclusions and guideline recommendations.^7–10 Potential competing interests have traditionally been regarded as monetary, but may also include professional, political or personal considerations.¹¹ High-profile scandals have demonstrated undisclosed conflicts of interest undermine trust in the objectivity of research.^12–14 While non-disclosure of competing interests does not necessarily affect assessments of study quality, authors’ interests can have a bearing on findings.¹⁵

Interest disclosures for published research are frequently incomplete.^16,17 Conflict of interest recording and policies in institutions that host research are also poor, and in several cases journal editors, as well as researchers, have potential conflicts of interest.^18,19 Voluntary declarations from the pharmaceutical industry have been criticised as inadequate, due to the ability of individuals to opt out.²⁰ Meanwhile, other voluntary registers have limited coverage since they require eligible individuals to ‘opt in’. For example, a voluntary register of interests for doctors in the UK contains details of only 0.002% of all those registered with the General Medical Council.^21,22 In several countries, ‘sunshine acts’ require disclosure of physicians financial interests, and calls have grown for similar requirements to be introduced to the United Kingdom.^23,24

Reporting biases

‘Reporting bias’ encompasses several sub-biases caused by selective disclosure or withholding of information, either intentionally or unintentionally, related to study design, methods and/or findings.²⁵ While several types of reporting biases have been described, we will focus on two of the most widely studied: publication bias and spin.

Publication bias refers to the propensity of certain types of research to become published, while other types remain unpublished. This results in a distortion of the published record which disproportionately features findings that are deemed to be novel, striking or that provide evidence in favour of a proposed intervention.²⁶ While publication bias is commonly understood to be driven by the perception that journals are unlikely to accept so-called ‘negative’ or ‘uninteresting’ results, researchers also perpetuate this bias by failing to submit such research. Publication bias has also been demonstrated to affect regulatory decisions and ultimately clinical practice.²⁶ The problem appears to be culturally entrenched and, in some cases, conflicts of interest are implicated.^7,27 Some journals have instituted initiatives to encourage publication of ‘negative’ findings to help remedy publication bias.^28,29

Spin refers to the practice of distorting or misrepresenting results to appear more ‘positive’, newsworthy or interesting.³⁰ While ‘spin’ falls short of outright falsification or fraud, experiments show that readers of studies with spin draw more favourable interpretations of interventions than when results are presented more objectively.³¹ Of particular concern is that spin in journal abstracts influences press releases, and studies that obtain press coverage receive greater numbers of citations.^32,33 Spin manifests in a variety of ways including failure to mention study limitations, selective ordering and highlighting of outcomes, and drawing exaggerated inferences from study results.³⁴ Spin appears to be increasingly common while peer review has proved insufficient to counter it.^35,36 Reviewers who identify spin usually do not succeed in having it removed from manuscripts and some reviewers actually suggest the addition of spin.³⁷

Statistical and methodological issues

There is substantial debate throughout science about the role of p values in determining statistical significance.³⁸ Reporting of p values has become much more common in recent decades, with 96% of papers containing p values of 0.05 or less.³⁹ The ubiquity of published p values of <0.05 can be explained by the strong incentives for researchers to publish significant results.⁴⁰ p values are also often misinterpreted and it has been estimated that most claims based on p values are false.^38,41 Some scientists suggest that the threshold of significance for p values should be reduced or abandoned altogether, that greater emphasis be placed on effect sizes and confidence intervals, or that Bayesian statistics should be used to provide greater discriminatory utility when appropriate.^42,43

The availability of modern statistical software means that, in the absence of an accessible protocol, it is relatively easy to generate statistically significant results through repeated analyses, a practice that has been dubbed p-hacking.⁴⁴ P-hacking in turn facilitates so called HARKing (Hypothesising After the Result is Known), whereby researchers can retrospectively generate a matching hypothesis to a significant p value.⁴⁵ Researchers may also selectively report statistically significant results or exploit ‘undisclosed flexibility’ in how analyses are conducted, allowing researchers to provide evidence for virtually any hypothesis.⁴⁶

Existing measures addressing problems in medical research

Several measures have been introduced to improve the reproducibility and transparency of medical research. Although no individual measure would be a cure-all, many successes have been documented. These highlighted achievements demonstrate that coordinated action to improve the research landscape is possible and necessary.

A number of important measures are considered in this article with some additional initiatives outlined in Table 1 and Table S1 in Supplement 2. It is beyond the scope of this review to survey all relevant actions that have been undertaken to address problems in medical research.

Table 1.

Initiatives and organisations working to reduce waste and improve the openness and quality of research.

Category	Initiative/Organisation	Description	URL
Research culture	Sunshine UK	Voluntary register of doctors’ declared interests.	http://www.whopaysthisdoctor.org/
	Open Science Badges	Badges appended to publications to acknowledge and incentivise open science practices.	https://cos.io/our-services/open-science-badges/
	REWARD Alliance	Originated in 2014 Lancet series on waste in research. Promotes efforts to increase the value of research and reduce waste in research.	http://rewardalliance.net/
	San Francisco Declaration on Research Assessment (DORA)	Initiative which calls for improvement in how research quality is evaluated.	https://sfdora.org/
	UK Reproducibility Network (UKRN)	Initiative which promotes the practices of open science.	https://bristol.ac.uk/psychology/research/ukrn/
Reporting biases	AllTrials	Campaign to ensure all clinical trials are registered and published. Highlights problem of publication bias, e.g. through ‘unreported clinical trial of the week’ and trial trackers which monitor reporting performance.	https://www.alltrials.net/
	Enhancing the QUAlity and Transparency Of health Research (EQUATOR)	International network which promote transparent and accurate reporting and wider use of robust reporting guidelines.	https://www.equator-network.org
	TranspariMED	Campaigning organisation which advocates for registration and full reporting of clinical trials.	https://www.transparimed.org/
Improving methodological and statistical practices	Open Science Framework (OSF)	Online platform that facilitates open sharing and preregistration of research.	https://osf.io/
	Oxford – Berlin summer school on open research	Training for researchers organised by the QUEST Center for Transforming Biomedical Research and Reproducible Research Oxford.	https://www.medsci.ox.ac.uk/news/oxford-berlin-summer-school-2019
	Evidence-Based RESearch (EVBRES)	European network established to promote evidence based clinical research, particularly the need to use systematic reviews when planning new studies and when placing new results in context.	https://evbres.eu

Research culture

Training in medical research, as in clinical medicine, is based largely around an apprenticeship model. Much attention has therefore been given to providing positive mentorship that promotes integrity.⁴⁷ Several initiatives such as the UK Reproducibility Network have been established to promote values and practices in research conducive to the principles of open science within academic institutions.⁴⁸

Several other measures have been proposed in order to align academic promotion criteria with responsible research practices. Suggestions include appraising research quality and reproducibility, and placing limits on the volume of publications that can be periodically submitted for institutional appraisal.^49–52 The San Francisco Declaration on Research Assessment (DORA) and the Hong Kong Principles have been formulated improve the assessment of researchers while the Transparency and Openness Promotion guidelines (TOP) includes several standards by which to assess research and journals.^53–55

Reporting biases

Clinical trial registration

Many of the problems outlined above, such as publication bias, spin, HARKing and p-hacking occur after research has been conducted. ‘Pre-registering’ studies prior to collecting data establish a record of research scheduled to take place including the hypotheses, methods, analyses and outcomes. Researchers can then be held accountable to the methods and outcomes they prespecified and expected to justify any deviations.

Registration of clinical trials is promoted by the World Health Organization through a number of approved primary registries and has increasingly become an expectation of funders and publishers.^56,57 The requirement by the International Committee of Medical Journal Editors is that all trials should be registered before being considered for publication and the passage of the FDA Amendments Act 2007 mandating registration in the USA were major milestones in achieving widespread trial registration.^58,59 Mandatory registration requirements have in turn facilitated mandatory reporting requirements, with the expectation that full findings of all trials are disseminated in some form, be it through journal publication or alternate routes provided by regulatory bodies or trial registries.⁶⁰ As well as from clinical trials, systematic reviews are also routinely registered prior to study commencement.⁶¹

Registration should be understood as one tool to help ensure researchers share their results and to discourage unjustified deviations from research plans. However, registration alone is not a panacea. For example, undeclared ‘outcome switching’ between registration and publication is common among registered trials.⁶² Journals could routinely ensure that reported outcomes match pre-specified registered outcomes. Despite these caveats, the value of registration should not be understated, since without it such deviations would not be detectable at all.^63,64

Registration of non-trial research

Although registration is now an expectation for clinical trials and systematic reviews, it remains voluntary and is employed relatively infrequently for other research types. For example, observational studies would benefit from registration and pre-specification of hypotheses and methods. Free public platforms, such as the Open Science Framework, are a highly accessible means for all researchers to publish time-stamped protocols and analysis plans. We believe that registration or publication of a priori protocols should become an expectation, and justified when not present. Broad adoption of study registration and protocol publication could dissuade authors from presenting results from exploratory work as hypothesis-based research, combat p-hacking and HARKing, and create a permanent record of planned research that can mitigate publication and other reporting biases. ‘Registered Reports’ are a related initiative which is discussed below.

Publication checklists

One initiative employed to improve transparency and quality of reporting has been the widespread requirement by journals for authors to submit checklists along with manuscripts, indicating whether reporting standards have been met. Checklists now exist for all major study designs of which the Consolidating Standards of Reporting Trials (CONSORT) is perhaps the best known and endorsed by hundreds of journals.⁶⁵ CONSORT has been associated with some improvement in completeness of reporting, but instances of poor CONSORT compliance remain.^63,66 Even when discrepancies are clearly identified by journal readers, authors and journal editors infrequently make corrections.⁶⁴ While the existing self-regulation of checklist compliance has yielded imperfect results, journal editors, or perhaps specially trained editorial assistants, could vet publications to ensure accurate reporting before publication.⁶⁷

Statistical and methodological issues

Education

The importance of high-quality training in statistics and open science methods are increasingly recognised.^68,69 Several educational opportunities have been established, some of which are identified in Table 1 and Supplement 2. Although such training is welcome, only a small proportion of medical researchers will benefit from such initiatives with access limited in low- and middle-income countries.⁷⁰ Therefore, it is vital that capacity to deliver such training is developed and maintained within academic institutions.

Data sharing

For research to be fully appraised and potentially reproduced, additional information beyond a study publication is required. Registrations can provide some additional detail, but in many cases, other documentation such as study protocols, analysis plans and individual patient data are necessary to understand and assess or reproduce the study. Making such data available would also allow other researchers to use the same data to answer different research questions.⁷¹ Unfortunately, research data are often not made available, even in summary forms or through secure mechanisms that would minimise the risks to breaches of patient privacy and confidentiality.^72–75

The International Committee of Medical Journal Editors has instituted requirements for data sharing for published clinical trials. While such policies are welcome, research suggests that even with these requirements data are available in only about half of publications.⁷² Persuasive appeals have been made to institute greater data sharing for all study types, including the proposition that research papers become one of a series of ‘threaded documents’ with underlying data made available as a matter of course.^71,76 Several data-sharing platforms are available to researchers.^77,78

How to improve medical research?

Significant gains in transparency have been achieved in clinical trial registration through the combined efforts of researchers, journals, funders, campaigners and legislators. Meanwhile continued attention has been brought to the ‘reproducibility crisis’ through the Open Science movement, which has connected international collaborators who share a determination to improve scientific practice through meta-scientific study. While it remains vital that emphasis is placed on improving statistical and ethical education for researchers and on addressing cultural issues within research, there is a need for immediate and definitive action to improve the quality of medical research.

We propose a strategy that includes three measures to achieve further improvements in the transparency of medical research. These measures are:

mandatory registration of interests for all people and institutions who conduct and publish health research;

all journals and funders support uptake of registered reports; and

publicly funded research is pre-registered and published on a World Health Organization-affiliated research registry.

Box 1 outlines how this strategy was developed, and Table 2 summarises some of the ways this could improve research.

Table 2.

Problems in medical research and how they can be mitigated by authors’ proposed strategy.

Problem	Problem description	Relevant proposed solution(s)	How proposed solution(s) addresses problem
Publication bias	Tendency for results deemed ‘negative’ or ‘uninteresting’ to remain unpublished.	Registered Reports Research Registry	Study accepted for publication based on methods, not results Study results and documents made available, regardless of publication status
‘Spin’	Practice of presenting results of study as more striking, ‘positive’ or newsworthy than warranted.	Registered Reports Research Registry Mandatory Declaration of Interests	Reduced incentive to ‘spin’ to obtain publication Study documentation available to allow greater scrutiny of researchers’ claims Information on possible conflicts of interest allows peers to judge if researchers have vested interest in applying spin to study
Scientific fraud	Deliberate falsification of evidence, for example fabrication of results.	Research Registry	Availability of full study documentation allows peers to scrutinise results. Researcher compelled to demonstrate ‘not just the answer but their working out’
Non-adherence to reporting checklists	Inaccurate self-disclosure by researchers of fulfilment of checklist statements.	Research Registry	Peers can scrutinise methods from available study documentation
HARKing	Researchers generate hypotheses to fit results and present these as if formulated prior to obtaining results.	Registered Reports	Hypotheses and aims are agreed prior to undertaking research. Any further post hoc analyses are declared as such
P-Hacking	Researchers manipulate results until findings generated which satisfy statistical significance.	Registered Reports Research Registry	Analyses agreed prior to generation of results Analysis plans and code available to peers for scrutiny
Outcome switching	Researchers do not report certain outcomes, or switch primary and secondary outcome, to highlight favoured results.	Registered Reports Research Registry Mandatory Declaration of Interests	Outcomes of interest agreed prior to undertaking research Protocols and analysis plans made available to peers for scrutiny Conflicting interests which could engender bias made known to public and peers
Other questionable research practices	Practices including deciding to collect more data after inspecting results, selective rounding of p-values, selective reporting of dependent variables.	Registered Reports Research Registry	Methods are agreed prior to publication. Incentive to generate results which favour publication removed Protocol and analysis plan made available to peers for scrutiny
Undisclosed conflicts of interest	Researchers may have, or could be perceived to have, vested interest in obtaining certain outcome in their results.	Mandatory Declaration of Interests	Researchers compelled to made comprehensive statement of their pecuniary interests, gifts and hospitality received and non-pecuniary interests
Non-replicable research	Results unable to be replicated, either because of insufficient information to reproduce methods or because of biases in original study (including problems in this table) mean work not reproduced when attempted.	Research Registry	Adequate study documentation made available such that study can be repeated or analyses repeated.

Box 1.

Development of the authors’ strategy to achieve further improvements in medical research.

The concept of a campaign, with specific objectives to improve health research arose from a presentation made to the EBMLive conference in Oxford in July 2019. The conference organisers facilitated a further session to discuss the concept further, chaired by Georgia Richards. An online survey was distributed among attendees of the conference session. Survey results were used to inform selection of three demands. Drafts of a statement were prepared by Stephen Bradley, Peter Gill and Georgia Richards and distributed among conference attendees, who then provided feedback and became signatories. A draft statement was made available (https://osf.io/k3w7m/) in October 2019 and publicised via twitter (@TA_Declaration). The feedback received informed the second major revision of the declaration, which, following consultation with signatories, was made available in January 2020. Signing the statement and/or providing feedback is possible by completing this form: https://docs.google.com/forms/d/1wHWcw77fFtvvvY2tJpNrzflt3HE0tF8xvFOL3U55RV4/

Mandatory declaration of interests

To establish trust in the objectivity of research, researchers must become more open about conflicts or declarations of interests. The brief voluntary statements in publications are not sufficient to meet this expectation. A fully accessible database of interests should be established, with firm expectations of accurate, up-to-date and comprehensive disclosure from all researchers, doctors, institutions and patient advocacy groups. Such disclosures should feature both monetary payments, benefits such as travel, hospitality and conference fees, and non-monetary interests, such as memberships of committees. Researchers’ declaration of interests should be required by all academic institutions, funders and journals. The Open Research and Contributor ID (ORCID) system already catalogues the identities of over 7 million researchers. Hence, ORCID could be an option to index researchers’ interests.⁷⁹

Registered reports

Registered reports are a publication format that permits authors to submit their proposed methods and analysis plan to a journal prior to conducting the research. If the journal accepts the proposal, it commits to publication as long as the research is satisfactorily conducted, regardless of the findings.⁸⁰ This model can help limit the impacts of publication bias and other reporting biases. Registered reports also curtail both the ability of authors to undertake (or peer reviewers to request) HARKing or p-hacking, and reduce the incentive to do so. Underpowered research or questionable methodological or statistical decisions can be identified and addressed through peer-review prior to study conduct. Since journals commit to publication upon review of study plans, rather than finished papers, registered reports may reduce the incentive for authors to ‘spin’ and for reviewers to request such embellishments.

Authors are not restricted from undertaking additional exploratory analyses, but registered reports help clarify that such analyses are exploratory and not based on prior hypotheses. Null findings are more likely to be published through registered reports than traditional formats and registered reports are cited just as often as conventional papers.^81,82

At the time of writing, registered reports were accepted by 225 journals, which included only 68 (1.3%) of the 5250 journals indexed in MEDLINE (Supplement 3).⁸⁰ A precursor to the registered report format was introduced by the Lancet over 20 years ago, but was discontinued with the warning that some studies deviated substantially from pre-specified outcomes and analyses.⁸³ This experience suggests that consistent evaluation of registered reports and support for authors to use the format is required. Their successful adoption in other scientific disciplines suggests medical journals could adopt this format in much greater numbers than they currently do. Pressure from funders, authors, readers and editorial board members could help support journals to make this transition.

Comprehensive research registration and publication

To facilitate reproducibility of research findings and to assess the plausibility of scientific claims, it is essential that documentation, including protocols and analysis plans, are made available to peers. Making all study findings available is the only way to address publication bias. It is also a matter of fairness that research which is paid for by public or charitable funding, and upon which important healthcare decisions may be made, is made available for anyone to view.

For all publicly funded research, not just trials, comprehensive documentation including protocols, statistical analysis plans, statistical analysis code, raw or appropriately de-identified summary data, and results should be available on a World Health Organization-affiliated open access registry. In theory, the Food and Drug Administration already requires that protocols and statistical analysis plans for clinical trials are publicly shared.⁵⁹ Obtaining widespread compliance with this principle for most types of study would represent a significant, but achievable, advance in transparency and fairness. Funders should require that study documentation is made openly available, while governmental and national research institutions could support the development, or nomination of, an appropriate open platform where anyone can find comprehensive information about publicly funded research.

Important principles such as the range of documentation that should be shared and to what level of detail would need to be established. However, we believe that establishing the expectation that sufficient information should be available for research to be adequately appraised could be an important milestone in achieving greater transparency and reproducibility.

Implementation

Implementing the proposed measures would likely impose financial and opportunity costs. Registered reports could create additional workload for journals and peer reviewers, and shift substantial work to earlier in the lifecycle of a project which may have funding and resource implications for research teams. Additionally, establishing a register of interests and a dedicated research repository would require investment from research funders and regulators, as well as the participation of researchers and industry. We believe the measures are proportionate, particularly in the context of the profits generated by the publishing industry from research⁸⁴ and the large sum of public resources invested to produce those findings.¹ The implementation of registers elsewhere, even in highly decentralised healthcare settings like the USA, is a proof of concept for the viability on other settings. While researchers might be reluctant to accept additional administrative burdens, depositing interests and study data in central and accessible locations could help reduce duplicative reporting requirements.

Important stakeholders such as research funders, regulators and journal editors are likely to have the most influence in achieving change. Yet the experience of inconsistent compliance with existing requirements^85–87 suggests that cultural change is also vital. Campaigns that raise awareness and expectations of transparency from the public^88–90 and legislators⁹¹ along with social pressure from peers⁹² are likely to help with embedding improvements in research culture.

Conclusions

Significant progress is required to satisfy reasonable expectations that medical research is trustworthy, reproducible and represents value for money. The proposed strategy comprising mandatory registration of potentially competing interests, registered reports and requiring all publicly funded research is registered can be readily conveyed to policy makers and rapidly implemented. These ideas are not novel and we do not claim that they would solve all problems in medical research. But, while such profound problems persist in medical research, we believe that it is time to implement simple measures to achieve greater transparency, reduce reporting biases and deter poor methodological practices.

Supplemental Material

sj-pdf-1-jrs-10.1177_0141076820956799 - Supplemental material for Reducing bias and improving transparency in medical research: a critical overview of the problems, progress and suggested next steps

Supplemental material, sj-pdf-1-jrs-10.1177_0141076820956799 for Reducing bias and improving transparency in medical research: a critical overview of the problems, progress and suggested next steps by Stephen H Bradley, Nicholas J DeVito, Kelly E Lloyd, Georgia C Richards, Tanja Rombey, Cole Wayant and Peter J Gill in Journal of the Royal Society of Medicine

Footnotes

Declarations

ORCID iDs

Stephen H Bradley

Georgia C Richards

Peter J Gill

Supplemental material

Supplemental material for this article is available online.

References

Chalmers

Glasziou

. Avoidable waste in the production and reporting of research evidence. The Lancet 2009; 374: 86–89.

What Researchers Think About the Culture They Work In. See https://wellcome.ac.uk/reports/what-researchers-think-about-research-culture (last checked 19 August 2020)..

Grimes

Bauch

Ioannidis

JPA

. Modelling science trustworthiness under publish or perish pressure. R Soc Open Sci 2018; 5(1): 171511–171511.

Schekman R. How Journals Like Nature, Cell and Science Are Damaging Science. See https://www.theguardian.com/commentisfree/2013/dec/09/how-journals-nature-science-cell-damage-science (last checked 19 August 2020).

Time to remodel the journal impact factor. Nature 2016; 535: 466.

Bornmann

Mutz

. Growth rates of modern science: a bibliometric analysis based on the number of publications and cited references. J Assoc Inform Sci Technol 2015; 66: 2215–2222.

Lexchin

Bero

Djulbegovic

Clark

. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. 2003 326: 1167–1170.

Singh

Forman

Adamson

, et al. Impact of industry payments on prescribing patterns for tumor necrosis factor inhibitors among medicare beneficiaries. J Gen Intern Med 2019; 34: 176–178.

Sharma

Vadhariya

Johnson

Marcum

Holmes

. Association between industry payments and prescribing costly medications: an observational study using open payments and medicare part D data. BMC Health Serv Res 2018; 18: 236–236.

10.

Perlis

. Physician payments from industry are associated with greater medicare part D prescribing costs. PloS one 2016; 11: e0155474–e0155474.

11.

Wiersma

Kerridge

Lipworth

Rodwin

. Should we try to manage non-financial interests? BMJ 2018; 361: k1240–k1240.

12.

Gornall

. The trial that launched millions of mesh implant procedures: did money compromise the outcome? BMJ 2018; 363: k4155–k4155.

13.

Cohen D and Brown E. Heart doctors ‘held back stent death data'. BBC. See https://www.bbc.co.uk/news/health-51539112 (last checked 19 August 2020).

14.

Johnson

Rogers

. Joint issues – conflicts of interest, the ASR hip and suggestions for managing surgical conflicts of interest. BMC Med Ethics 2014; 15: 63–63.

15.

John

Loewenstein

Marder

Callaham

. Effect of revealing authors’ conflicts of interests in peer review: randomized controlled trial. BMJ 2019; 367: l5896–l5896.

16.

Wayant

Turner

Meyer

Sinnett

Vassar

. Financial conflicts of interest among oncologist authors of reports of clinical drug trials. JAMA Oncol 2018; 4: 1426–1428.

17.

Wallach

Boyack

Ioannidis

JPA

. Reproducible research practices, transparency, and open access data in the biomedical literature, 2015–2017. PLOS Biol 2018; 16: e2006930–e2006930.

18.

Feldman

DeVito

Mendel

Carroll

Goldacre

. A cross-sectional study of all clinicians’ conflict of interest disclosures to NHS hospital employers in England 2015–2016. BMJ Open 2018; 8: e019952–e019952.

19.

Janssen

Bredenoord

Dhert

de Kleuver

Oner

Verlaan

J-J

. Potential conflicts of interest of editorial board members from five leading spine journals. PloS one 2015; 10: e0127362–e0127362.

20.

Goldacre

. Problems with ABPI proposals to release data on payments to doctors. BMJ 2014; 348: g1301–g1301.

21.

Who Pays This Doctor? See http://www.whopaysthisdoctor.org/ (last checked 19 August 2020).

22.

Medical Register Reaches 300,000 for the First Time: General Medical Council. See https://www.gmc-uk.org/news/news-archive/medical-register-reaches-300000-for-the-first-time (last checked 19 August 2020).

23.

Fabbri

Santos

Mezinska

Mulinari

Mintzes

. Sunshine Policies And Murky Shadows In Europe: disclosure of pharmaceutical industry payments to health professionals in nine European countries. Int J Health Policy Manag 2018; 7: 504–509.

24.

Mahase

. RCGP calls on GMC to introduce mandatory and public declaration of interests register. BMJ 2019; 367: l6695–l6695.

25.

Richards G and Onakpoya I. Catalogue of Bias: Reporting Biases 2019. See www.catalogueofbiases.org/reportingbiases (last checked 19 August 2020).

26.

Turner

Matthews

Linardatos

Tell

Rosenthal

. Selective publication of antidepressant trials and its influence on apparent efficacy. New Engl J Med 2008; 358: 252–260.

27.

Rosenthal

. The file drawer problem and tolerance for null results. Psychol Bull 1979; 86: 638–641.

28.

Petty

Gross

. <em>Neurology</em>® null hypothesis. A special supplement for negative, inconclusive, or confirmatory studies. Neurology 2018; 91: 12–13.

29.

The Missing Pieces: A Collection of Negative, Null and Inconclusive Results: PLOS. See https://collections.plos.org/missing-pieces#id_Research%20Article (last checked 19 August 2020).

30.

Boutron

Dutton

Ravaud

Altman

. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA 2010; 303: 2058–2064.

31.

Boutron

Altman

Hopewell

Vera-Badillo

Tannock

Ravaud

. Impact of spin in the abstracts of articles reporting results of randomized controlled trials in the field of cancer: the SPIIN randomized controlled trial. J Clin Oncol 2014; 32: 4120–4126.

32.

Yavchitz

Boutron

Bafeta

, et al. Misrepresentation of randomized controlled trials in press releases and news coverage: a cohort study. PLoS Med 2012; 9: e1001308–e1001308.

33.

Phillips

Kanter

Bednarczyk

Tastad

. Importance of the lay press in the transmission of medical knowledge to the scientific community. N Engl J Med 1991; 325: 1180–1183.

34.

Boutron

Ravaud

. Misrepresentation and distortion of research in biomedical literature. Proc Natl Acad Sci USA 2018; 115: 2613–2619.

35.

Vera-Badillo

Shapiro

Ocana

Amir

Tannock

. Bias in reporting of end points of efficacy and toxicity in randomized, clinical trials for women with breast cancer. Ann Oncol 2013; 24: 1238–1244.

36.

Vinkers

Tijdink

Otte

. Use of positive and negative words in scientific PubMed abstracts between 1974 and 2014: retrospective analysis. BMJ 2015351: h6467–h6467.

37.

Lazarus

Haneef

Ravaud

Hopewell

Altman

Boutron

. Peer reviewers identified spin in manuscripts of nonrandomized studies assessing therapeutic interventions, but their impact on spin in abstract conclusions was limited. J Clin Epidemiol 2016; 77: 44–51.

38.

Wasserstein

Lazar

. The ASA Statement on p-values: context, process, and purpose. Am Stat 2016; 70: 129–133.

39.

Chavalarias

Wallach

AHT

Ioannidis

JPA

. Evolution of reporting P values in the biomedical literature, 1990–2015. JAMA 2016; 315: 1141–1148.

40.

Masicampo

Lalande

. A peculiar prevalence of p values just below .05. Q J Exp Psychol 2012; 65: 2271–2279.

41.

Ioannidis

JPA

. Why most published research findings are false. PLoS Med 2005; 2: e124–e124.

42.

Ioannidis

JPA

. The proposal to lower P value thresholds to .005. JAMA 2018; 319: 1429–1430.

43.

Etz

Vandekerckhove

. A Bayesian perspective on the reproducibility project: psychology. PLoS One 2016; 11: e0149794–e0149794.

44.

Head

Holman

Lanfear

Kahn

Jennions

. The extent and consequences of P-Hacking in science. PLoS Biol 2015; 13: e1002106–e1002106.

45.

Kerr

. HARKing: hypothesizing after the results are known. Personal Soc Psychol Rev 1998; 2: 196–217.

46.

Simmons

Nelson

Simonsohn

. False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci 2011; 22: 1359–1366.

47.

Straus SE, Sackett DL and Sawkins R. Mentorship in Academic Medicine. BMJ Books, 2013.

48.

The UK Reproducibility Network (UKRN). See https://bristol.ac.uk/psychology/research/ukrn/ (last checked 19 August 2020).

49.

Flier

. Faculty promotion must assess reproducibility. Nature 2017; 549: 133–133.

50.

Martinson

. Give researchers a lifetime word limit. Nature 2017; 550: 303–303.

51.

Finkel A. ‘There is a problem’: Australia’s top scientist Alan Finkel pushes to eradicate bad science. The Conversation. See https://theconversation.com/there-is-a-problem-australias-top-scientist-alan-finkel-pushes-to-eradicate-bad-science-123374 (last checked 19 August 2020).

52.

Frank

. N-best evaluation for academic hiring and promotion. Trends Cogn Sci 2019; 23: 983–985.

53.

TOP Guidelines. See https://www.cos.io/our-services/top-guidelines (last checked 19 August 2020).

54.

Moher

Bouter

Kleinert

, et al. The Hong Kong Principles for assessing researchers: Fostering research integrity. PLOS Biol 2020; 18: e3000737–e3000737.

55.

San Francisco Declaration on Research Assessment. See https://sfdora.org/ (last checked 19 August 2020).

56.

WHO Statement on Public Disclosure of Clinical Trial Results. See https://www.who.int/ictrp/results/WHO_Statement_results_reporting_clinical_trials.pdf (last checked 19 August 2020).

57.

NIHR Policy on Clinical Trial Registration and Disclosure of Results 2019. See https://www.nihr.ac.uk/documents/nihr-policy-on-clinical-trial-registration-and-disclosure-of-results/12252 (last checked 19 August 2020).

58.

Clinical Trials Registration: International Committee of Medical Journal Editors. See http://www.icmje.org/about-icmje/faqs/clinical-trials-registration/ (last checked 19 August 2020).

59.

FDAAA 801 and the final rule: U.S. National Library of Medicine, ClinicalTrials.gov. See https://clinicaltrials.gov/ct2/manage-recs/fdaaa (last checked 19 August 2020).

60.

Moorthy

Karam

Vannice

Kieny

M-P

. Rationale for WHO's new position calling for prompt reporting and public disclosure of interventional clinical trial results. PLoS Med 2015; 12: e1001819–e1001819.

61.

International Prospective Register of Systematic Reviews (PROSPERO). See https://www.crd.york.ac.uk/PROSPERO/ (last checked 19 August 2020).

62.

Jones

Keil

Holland

Caughey

Platts-Mills

. Comparison of registered and published outcomes in randomized controlled trials: a systematic review. BMC Med 2015; 13: 282–282.

63.

Goldacre

Drysdale

Dale

, et al. COMPare: a prospective cohort study correcting and monitoring 58 misreported trials in real time. Trials 2019; 20: 118–118.

64.

Goldacre

Drysdale

Marston

, et al. COMPare: qualitative analysis of researchers’ responses to critical correspondence on a cohort of 58 misreported trials. Trials 2019; 20: 124–124.

65.

Begg

Cho

Eastwood

, et al. Improving the quality of reporting of randomized controlled trials: the CONSORT statement. JAMA 1996; 276: 637–639.

66.

Blanco

Biggane

Cobo

, et al.

Are CONSORT checklists submitted by authors adequately reflecting what information is actually reported in published papers?

Trials 2018; 19: 80–80.

67.

Blanco

Altman

Moher

Boutron

Kirkham

Cobo

. Scoping review on interventions to improve adherence to reporting guidelines in health research. BMJ Open 2019; 9: e026589–e026589.

68.

Altman

. The scandal of poor medical research. BMJ 1994; 308: 283–284.

69.

Richards GC, Bradley SH, Dagens AB, et al. Challenges facing early-career and mid-career researchers: potential solutions to safeguard the future of evidence-based medicine. BMJ Evid-Based Med. doi: 10.1136/bmjebm-2019-111273.

70.

Gill PJ, Ali SM, Elsobky Y, et al. Building capacity in evidence-based medicine in low-income and middle-income countries: problems and potential solutions. BMJ Evid-Based Med. doi: 10.1136/bmjebm-2019-111272.

71.

Goldacre

Morton

DeVito

. Why researchers should share their analytic code. BMJ 2019; 367: l6365–l6365.

72.

Naudet

Sakarovitch

Janiaud

, et al. Data sharing and reanalysis of randomized controlled trials in leading biomedical journals with a full data sharing policy: survey of studies published in The BMJ and PLOS Medicine. BMJ 2018; 360: k400–k400.

73.

The FAIR Data Principles. See https://www.force11.org/group/fairgroup/fairprinciples (last checked 19 August 2020).

74.

Wilkinson

Dumontier

Aalbersberg

, et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data 2016; 3: 160018–160018.

75.

Ohmann

Banzi

Canham

, et al. Sharing and reuse of individual participant data from clinical trials: principles and recommendations. BMJ Open 2017; 7: e018647–e018647.

76.

Chalmers

Glasziou

. Should there be greater use of preprint servers for publishing reports of biomedical science? F1000Res 2016; 5: 272–272.

77.

Vivli: A Global Research Data Sharing Platform. See https://vivli.org/ (last checked 19 August 2020).

78.

Clinical Study Data Request. https://www.clinicalstudydatarequest.com/ (accessed 22 August 2020).

79.

Grundy

Dunn

Bero

. Improving researchers’ conflict of interest declarations. BMJ 2020; 368: m422–m422.

80.

Registered Reports: Peer Review Before Results Are Known to Align Scientific Values and Practices. See https://cos.io/rr/ (last checked 19 August 2020).

81.

Allen

Mehler

DMA

. Open science challenges, benefits and tips in early career and beyond. PLOS Biol 2019; 17: e3000246–e3000246.

82.

Hummer LT, Thorn FS, Nosek BA and Errignton TM. Evaluating Registered Reports: A Naturalistic Comparative Study of Article Impact 2019. See https://osf.io/5y8w7/ (last checked 19 August 2020).

83.

Hardwicke

Ioannidis

JPA

. Mapping the universe of registered reports. Nature Human Behav 2018; 2(11): 793–796.

84.

Buranyi S. Is the staggeringly profitable business of scientific publishing bad for science? The Guardian. See https://www.theguardian.com/science/2017/jun/27/profitable-business-scientific-publishing-bad-for-science (last checked 19 August 2020).

85.

DeVito NJ, Bacon S and Goldacre B. FDAAA TrialsTracker: a live informatics tool to monitor compliance with FDA requirements to report clinical trial results. bioRxiv 2018:266452.

86.

Bruckner T. TranspariMed: working to end evidence distortion in medicine. See https://www.transparimed.org/ (last checked 19 August 2020).

87.

Goldacre

DeVito

Heneghan

, et al. Compliance with requirement to report results on the EU Clinical Trials Register: cohort study and web resource. BMJ 2018; 362: k3218–k3218.

88.

All Trials. See http://www.alltrials.net/ (last checked 19 August 2020).

89.

Sense About Science. See https://senseaboutscience.org/ (last checked 19 August 2020).

90.

HealthWatch UK. See https://senseaboutscience.org/ (last checked 19 August 2020).

91.

Hawkes

. “Shocking” number of clinical trials are never reported, say MPs. BMJ 2018; 363: k4582–k4582.

92.

Goldacre B, DeVito N, Heneghan C, Lane S and Stephens R. Introducing unreported clinical trial of the week. BMJ Opinion. See https://blogs.bmj.com/bmj/2018/03/29/it-is-time-to-fix-the-problem-of-unreported-clinical-trials/ (last checked 19 August 2020).

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.24 MB