Tackling treatment uncertainties together: the evolution of the James Lind Initiative,2003

Abstract

At the end of 2003, the Journal of the Royal Society of Medicine published an article announcing the creation of the James Lind Initiative (JLI).¹ The article began by recalling the way that James Lind, an 18th century Scottish naval surgeon, had confronted uncertainty about how to treat scurvy by doing a controlled trial. This had shown that oranges and lemons were a very effective cure for an often lethal disease.^2,3 The 2003 article went on to suggest that therapeutic uncertainties today should also be confronted in research as an integral element of responsible healthcare, and that the JLI was being established to promote this principle. The current article describes the origins of the JLI and its work over the past 10 years.

The origins of the JLI

The idea for the JLI had its origins at a brainstorming meeting on 2 April 2001 convened by the UK Cochrane Centre to consider what might be done to increase general knowledge about why treatments need to be tested rigorously, and what rigorous testing of treatments entails. There were over 40 participants, from backgrounds including patient and public representation, education, lay media and medical journalism, government and parliament, medical ethics, history of medicine, medical practice, and research and research funding.⁴ Discussion and subsequent feedback generated a wide variety of suggestions, ranging from reducing research jargon in interactions with patients and the public, through more honest admission of uncertainties about the effects of treatments, to greater involvement of patients in shaping the health research agenda.⁵

The following year the Medical Research Council (MRC) established a working group to review the Council’s involvement in clinical trials. The section of the group’s report entitled ‘Promoting public engagement’ contained a number of recommendations, including a call for the creation of ‘a communications and discussion forum on randomized controlled trials, involving patients, practitioners, researchers, and others’, and stated that, with support from the Department of Health, the MRC had appointed Iain Chalmers to take this initiative forward.^6,7

After seeking views on how the challenge of engaging patients and clinicians in clinical trials might be pursued, it was decided that the JLI would take an indirect approach to the MRC’s brief: instead of promoting public and professional discussion of clinical trials directly, the JLI would instead promote discussion about how patients, clinicians and policy-makers should respond to uncertainties about the effects of treatments. The rationale for this theme was spelled out the in the JLI’s launch article in 2003 (Chalmers 2003a1), and in editorials published early in 2004.^8–10

Tens of thousands of uncertainties about treatment effects have never been investigated using systematic reviews of research evidence, and thousands of systematic reviews have made clear that many uncertainties about treatment effects have not been resolved by researchers. Furthermore, uncertainties reflected in the research choices of people in academia and industry cannot be assumed to address questions about the effects of treatments that are important to patients, clinicians and other users of research. For example, when patients, rheumatologists, physiotherapists and general practitioners were asked to identify their priorities for research on the management of osteoarthritis of the knee, there was little enthusiasm for yet more studies of the drugs that dominate the existing research on this condition. Patients and clinicians said they wanted more rigorous evaluation of the effects of physiotherapy and surgery, and better assessment of the educational and coping strategies that might help patients to manage this chronic, disabling and often painful condition.¹¹

Systematic reviews of available clinical research evidence prepared over the past two decades have made clear that this is not an isolated example of a mismatch between choices made by researchers and the priorities of patients and clinicians.¹² If the JLI was going to engage patients and clinicians in discussions about responding to uncertainties, it seemed sensible to begin by finding out which uncertainties mattered most to them.

Prioritizing uncertainties about the effects of treatments for further research

The James Lind Alliance (JLA): philosophy and early planning

To develop these ideas, Iain Chalmers sought help from Nick Partridge, chair of an organization promoting public involvement in health research (INVOLVE), and John Scadding, Dean of the Royal Society of Medicine, which has an education programme for clinicians. They agreed to establish a ‘James Lind Alliance’ to facilitate the identification of research priorities shared by patients, carers and clinicians.^13,14

The philosophy and values underpinning the JLA patient–clinician Priority Setting Partnerships (PSPs) were: (i) inclusion of a range of patient and clinician perspectives; (ii) transparency of the methods, outcomes and vested interests of those taking part in prioritization; and (iii) evidence-based knowledge – ‘known unknowns’ – on which to base prioritization. Research commissioned by the James Lind Alliance^15–17 suggested that identifying shared therapeutic research priorities by patient–clinician partnerships was rare, if not unique. To add momentum to the JLA’s ambitious plans, interested individuals and organizations were invited to become JLA ‘Affiliates’, and a website was established to promote the Alliance (www.lindalliance.org).

After ‘setting out its stall’ in articles and meetings, the JLA waited to be invited to facilitate the creation and work of PSPs. If people and organizations did not warm to the proposed concept and approach, the initiators were ready to acknowledge failure. In the event, there was no shortage of interest from patients and carers, although it was often difficult to engage clinicians.¹⁸

The UK Database of Uncertainties about the Effects of Treatments (UK DUETs)

Unsurprisingly, organizing the first PSP (for asthma) proved very challenging and it developed quite slowly. It was decided to create a public database in which uncertainties about the effects of treatments could be made explicit, as had been proposed at the brainstorming meeting in 2001.^19,20 This would provide a resource to help prioritize uncertainties for investigation in new research – either in the form of new, extended or updated systematic reviews; or in additional ‘primary’ research. ‘Treatment uncertainties’ were defined as ‘uncertainties that could not be resolved by reference to reliable, up-to-date systematic reviews of existing research evidence’. UK DUETs was conceptualized in 2005, developed with input from two advisory groups, and launched in 2006.²¹

After pilot work had been done, an infrastructure was needed to roll out UK DUETs, so that, as new uncertainties were identified, its coverage could be extended and maintained. In late 2006, it was agreed that the National Library for Health (NLH) would provide an appropriate infrastructure, particularly because of its commitment to address the needs of patients as well as clinicians. The following year NLH invitations to tender for specialist libraries specified that these would be expected to ‘identify and publish uncertainties about the effects of treatments, using agreed procedures, through DUETs’. This requirement was a clear recognition by the National Health Service that it is important to present information about what is not known as well as about what is known about the effects of treatments.

By the end of 2012, UK DUETs contained 4760 uncertainties, 67% derived from reports of systematic reviews and clinical guidelines, and 14% from protocols for systematic reviews (such as those published in the Cochrane Database of Systematic Reviews) or from registered information about ongoing clinical trials. Just under one-fifth (19%) of the treatment uncertainties came directly from patients or carers (14%), or from clinicians (5%).

James Lind Alliance Priority Setting Partnerships (JLA PSPs)

A review of published material on methods to achieve consensus and develop research priorities made clear that reports of existing experience were very limited.²² A JLA Strategy and Development Group and a Monitoring and Implementation Group debated how to formulate and implement JLA methods for identifying priorities shared by patients and clinicians.

UK DUETs proved to be an essential source of data for JLA PSPs, as well as being a repository for additional uncertainties revealed as a result of the surveys of patients and clinicians organized by PSPs. Figure 1 shows the process of refinement and prioritization, from ‘raw’ uncertainties to those prioritized for further research. Between 2005 and 2008 the JLA methodology was piloted in asthma and urinary incontinence. Reflection and discussion followed each key step of these initial PSPs, with refinements to the methods being used. Meetings and workshops were observed and recorded and both groups published their findings.^23,24 Subsequent PSPs added to an understanding of how to involve diverse groups in prioritization.

Figure 1

Identifying and prioritizing uncertainties about the effects of treatment

Initially, the PSPs were based on specific health problems, but this focus was sometimes extended to cover a wide range of conditions (for example, ‘sight loss’), as well as to include treatment settings (for example, intensive care units). Whereas the first PSP had only one patient partner (Asthma UK) and one clinician partner (the British Thoracic Society), some later PSPs have had over 40 partner organizations. By March 2013 there had been 13 completed PSPs and a further 12 were at various stages of completion (Table 1). There are no signs yet of any let up in requests for JLA support for additional PSPs.

Table 1

JLA priority setting partnerships

Completed	Current
Asthma	Acne
Urinary incontinence	Childhood disability
Vitiligo	Dementia
Prostate cancer	Dialysis
Schizophrenia	Inflammatory bowel disease
Type 1 diabetes	Multiple sclerosis
ENT aspects of balance Life after stroke Eczema Tinnitus	Pre-term birth Sight loss and vision Skin-suppurative hidradenitis Hips and knee replacement for osteoarthritis
Cleft lip and palate	Spinal cord injury
Lyme disease	Palliative care
Pressure ulcers
Epidermolysis Bullosa

Priority research themes emerging from JLA PSPs include emphasis on the need to assess long-term effects of treatments; safety and adverse effects of treatments; effects of complementary and non-prescribed treatments; and the effectiveness and safety of self-care;²⁵ and Table 2 shows the mismatch between the types of interventions that patients and clinicians wish to see evaluated compared with those being assessed by researchers.²⁶ Table 3 shows the implications for researchers of the priorities identified: additional primary research was needed to address under half of the priorities; new or updated systematic reviews of existing evidence were the appropriate response to the remainder.

Table 2

Interventions mentioned in research priorities identified by James Lind Alliance patient–clinician Priority Setting Partnerships, and among registered trials, 2003–2012 (Crowe et al. in preparation)

Type of intervention	JLA patient–clinician Priority Setting Partnerships % (of total of 113 interventions mentioned)	Registered non-commercial trials % (of total of 1036 interventions mentioned)	Registered commercial trials % (of total of 798 interventions mentioned)
Drugs, vaccines and biologicals	19.4 (22)	37.2 (397)	86.3 (689)
Radiotherapy, surgery and perioperative, devices, and diagnostic	23.1 (26)	29.8 (332)	11.1 (89)
Education and training, service delivery, psychological therapy, physical therapies, exercise, complementary therapies, social care, mixed or complex, diet, other	57.6 (65)	31.9 (307)	2.6 (20)

Table 3

James Lind Alliance: principal implications of top 10 research priorities

PSP	Prepare systematic review of existing evidence	Update or extend existing systematic review(s)	Design and do additional primary research
Asthma	4	3	4
Cleft lip and palate	5	0	7
ENT Balance	8	2	0
Eczema	4	2	8
Lyme disease	10	0	0
Prostate	0	0	11
Schizophrenia	3	4	3
Life after stroke in Scotland	3	3	4
Tinnitus	5	0	5
Type 1 diabetes	7	1	2
Urinary incontinence	4	1	5
Vitiligo	4	5	3
Total	57	21	52

Throughout its evolution, the JLA’s role has been as a neutral support/honest broker, and it encouraged flexibility as long as the founding principles of patient and clinician parity, transparency and systematic rigour were respected. The JLA Guidebook²⁷ has drawn on this experience, and has documented the variety of approaches adopted, showcased good practice, and encouraged PSPs to think independently. It describes in straightforward terms how to involve patients, carers and clinicians in research priority-setting. The Guidebook has been developed in consultation with its intended users because it had to be accessible to anyone – whether patient, clinician or researcher.

The mission, methods and output of the JLA have become quite widely reported and recognized during recent years and emulated in other countries. Furthermore, the MRC invited the JLA to consider how the JLA’s working principles might be applied to the Council’s support of pre-clinical research.

Mainstreaming UK DUETs and the JLA

An application in 2009 to extend support for the JLI for a further three years was approved on condition that efforts were made to ‘mainstream’ UK DUETs and the JLA. This process was facilitated by the support that UK DUETs had received from the National Library for Health, for which the National Institute for Health and Clinical Excellence (NICE) had become responsible, and the encouragement that plans for the JLA had received from its very first affiliate – the NIHR Health Technology Assessment Programme.

In 2010, it was agreed that formal responsibility for maintaining and developing UK DUETs should pass from the JLI to NICE. As far as we are aware, NICE thus became the first provider of health information to enable users to select information about uncertainties, and thus to identify gaps in knowledge and needs for additional research.^28,29 In 2012, the Swedish Council on Health Technology Assessment (SBU)³⁰ paid UK DUETs a compliment by announcing that it had established ‘a Swedish DUETs’. Finally, in 2011, it was agreed that, from April 2013, the NIHR Evaluation, Trials and Studies Coordinating Centre (NETSCC) would assume responsibility for managing the work of the JLA.

To mark the end of the JLI’s responsibility for UK DUETs and the JLA, it co-convened with NICE and the Association of Medical Research Charities a meeting to present information about national resources containing information about uncertainties and methods used for research priority setting.³¹ The responses from research charities new to these ideas ranged from scepticism to enthusiasm. A comment from one enthusiastic participant went to the heart of the matter:

‘A key lesson for us was to ask – always – “where is the patient in this uncertainty?” Charities don’t exist to give us jobs but to help those who give money. Asking them what to do with it should be normal.’

Enhancing general knowledge about testing treatments

Patients and members of the public more generally can play active parts in research addressing uncertainties about the effects of treatments. If patients and the public are to contribute effectively, however, they need help to enhance their general knowledge about how reliable information about the effects of treatments is generated.

This challenge has been addressed in the other element of the JLI’s programme of work – through articles, talks, interviews, radio and TV broadcasts, and committee and working party membership. The Initiative has also been involved in research showing that new treatments are as likely to be worse as they are to be better than existing treatments, evidence that is very relevant to creating an atmosphere in which research should flourish.^32–34

In addition, the JLI has: (i) coordinated the development of The James Lind Library (www.jameslindlibrary.org), a multilingual website introducing the reasons for and characteristics of fair tests of treatments; (ii) played a major part in co-authoring and making accessible two editions of Testing Treatments: better research for better healthcare, a book written for the public;^35,36 (iii) co-convened a meeting to establish an international Network to Support Understanding in Health Research (www.nsuhr.net); and (iv) established a new multilingual website called Testing Treatments interactive (www.testingtreatments.org).

The James Lind Library (www.jameslindlibrary.org)

The James Lind Library (JLL) is a website explaining and illustrating the evolution of fair tests of treatments. It was launched in the same year (2003) as the JLI, which also coincided with the 250th anniversary of the publication of Lind’s Treatise of the Scurvy.² The JLL contains historical material illustrating the evolution of fair tests of treatments, mostly provided by the Sibbald Library of the Royal College of Physicians of Edinburgh. Essays introducing the features of fair tests were added to the historical material, and this led Scientific American to select the JLL as one of only five medical websites to receive an award in 2003, and the only one based outside the USA.³⁷

The JLL has attracted widespread interest. In particular, the World Health Organization and the Pan American Health Organization funded the translation of the JLL explanatory essays into Arabic, Chinese, French, Portuguese, Russian and Spanish. The website has hosted for free download the texts of three books written for the public: Testing Treatments,³⁵ in six languages as well as in English; Smart Health Choices;³⁸ and Know your Chances.³⁹

Historical records and articles commissioned for publication in the JLL continue to be added to the website, and many of these articles have been republished in the Journal of the Royal Society of Medicine every month since October 2005. Copies of the website are being regularly ‘future proofed’ as part of the British Library’s Web Archiving Programme.

Testing treatments: better research for better healthcare

In December 2002, Imogen Evans, formerly an executive editor at The Lancet and then on the staff of the MRC, invited Iain Chalmers to co-author a book for the public on clinical trials. Planning this work began in earnest in July 2003, after IC had become responsible for taking forward elements of the MRC’s clinical trials strategy.⁶ IE and IC agreed that authorship would be strengthened by recruiting a ‘lay` viewpoint. On the basis of her decade of independent campaigning for citizen engagement in the research process, Hazel Thornton was recruited as a co-author.

The three co-authors all contributed to research and writing for the book, although the final drafting was done by Imogen Evans. The JLI co-ordinated the preparation of the manuscript for submission to the British Library. Testing Treatments was first published in 2006,³⁵ with a foreword written by the broadcaster and writer Nick Ross, and launched by the British Library at an event supported jointly by DH and the MRC.

The book was well received and the text was made available for free download from www.jameslindlibrary.org. After it had been downloaded 130,000 times, a new publisher – Pinter and Martin – published a second imprint, with an additional foreword by science journalist Ben Goldacre. Translations of the book were subsequently published in Arabic, Chinese, German, Italian, Polish and Spanish.

Because of the success of the first edition of Testing Treatments, discussions about a second edition began in 2008. Paul Glasziou, an Australian general practitioner, who had been director of the Centre for Evidence-Based Medicine at the University of Oxford, was recruited as a co-author of the new edition. The second edition of the book was published in print and ebook forms in October 2011 by Pinter and Martin.³⁶ As with the previous edition, the text of the new edition was made available for free download from a website www.testingtreatments.org, and, at the time of writing, it is one of only two books featured in the PubMedHealth ‘Bookshelf’ operated by the US National Library of Medicine. Translations seem likely to be freely available in at least a dozen languages other than English.

Testing treatments interactive

In 2010, consideration was given to upgrading the software of the James Lind Library (JLL), to take advantage of improvements in website functionality. Although the JLL will continue to assemble evidence about the evolution of fair tests of treatments, it became clear that, for technical reasons, it would be preferable to start from scratch with a new website designed primarily to enhance general knowledge about testing treatments. The JLI decided to base this around the text of the second edition of Testing Treatments. Accordingly, Testing Treatments interactive was established at www.testingtreatments.org. It is using video, audio, cartoons and other material to illustrate the messages and principles covered in the book.

Testing Treatments interactive (TTi) was piloted in late 2011, with sibling development sites established in Arabic, German and Turkish in late 2011 and early 2012. A complete version of TTi English was launched in August 2012, and a sibling site in Spanish – TTi Español – was launched in December 2012 (www.es.testingtreatments.org). In January 2013, a meeting was held to establish a TTi Editorial Alliance, bringing together editors involved in translating the second edition of Testing Treatments and in establishing and managing sibling interactive websites in Arabic, Turkish, German, Russian, Bahasa, Portuguese, Chinese, French, Norwegian, Croatian, Polish and Basque. The Editorial Alliance will provide a forum within which editors of all the sibling websites can exchange experiences and ways of evaluating the effectiveness of their work.

JLI in future

Enhancing general knowledge about testing treatments

Enhancing general knowledge about testing treatments will remain one of the JLI’s two main work themes in future. Testing Treatments interactive (TTi) will be a major component of this work, and will involve close collaboration with Paul Glasziou (Bond University, Australia), Douglas Badenoch (Minervation) and Amanda Burls (Oxford University).

TTi is already a key open learning resource for ECRAN⁴⁰ (the EU-supported European Communication on Research Awareness Needs Project), and SIHCLIC (Supporting Information for Health Care in Low Income Countries, funded by the Norwegian Government). To evaluate the impact of these learning resources, the JLI will collaborate with members of the TTi Editorial Alliance, especially colleagues at the Kunnskapssenteret in Oslo, and editors of the multilingual WHO Reproductive Health Library.

To remind people of the long history of efforts to obtain trustworthy evidence to inform and guide choices in healthcare,^41–52 the JLI will continue to maintain the James Lind Library,³⁷ with editorial input from Ulrich Tröhler (University of Bern, Switzerland) and Mike Clarke (Queen’s University, Belfast), in particular.

The JLI will continue to use opportunities to promote the principle that evaluative research addressing important uncertainties should be an expected element of usual clinical care,^53–59 as had been promoted by the GMC’s injunction that doctors ‘must work with colleagues and patients to help to resolve uncertainties about the effects of treatments’.⁶⁰ To encourage the application of this principle in practice, the JLI will continue to work with the Health Research Agency and others to reduce disproportionate regulation of clinical research.^61–69

Monitoring and reducing waste in research

Through a variety of mechanisms the public ends up meeting the cost of most medical research; but important questions remain about the value of the returns on this investment,⁷⁰ a matter of particular importance at a time of economic constraint. In a paper published in The Lancet in 2009, Iain Chalmers and Paul Glasziou⁷¹ estimated that 85% of the resources invested in medical research was being avoidably wasted. They have been involved in two important responses to their paper. First, they have had regular meetings with NETSCC to discuss and evaluate ways in which waste can be reduced in DH-funded research programmes, and to develop indicators for assessing progress. Second, after being asked by The Lancet to coordinate a series of articles on research waste to raise awareness of this problem, they have engaged co-authors to cover waste in pre-clinical as well as clinical and epidemiological research in a series of papers.

Monitoring and reducing waste in research will be the second of the two themes in the future work of the JLI. The Initiative will continue to focus on two ways to reduce waste in which it has an established interest: (i) ensuring that the design and interpretation of new research is informed by systematic reviews of existing evidence:^33,72–84 and (ii) working with others to expose and reduce the scandal of biased under-reporting of research,^{75,82,85–91} for example, through continued involvement in a campaign (alltrials.net) demanding that all clinical trials must be registered publicly at inception and reported publicly on completion.^92,93

In these various ways, the JLI will continue its mission to encourage acknowledgement of important uncertainties about the effects of treatments, and to promote better, more efficiently implemented research to address these.

Footnotes

DECLARATIONS

References

Chalmers

. The James Lind Initiative. J R Soc Med 2003; 96: 575–6.

Lind J. A Treatise of the Scurvy. In Three Parts. Containing an Inquiry into the Nature, Causes and Cure, of that Disease. Together with a Critical and Chronological View of what has been Published on the Subject. Edinburgh: A Kincaid and A Donaldson, 1753.

Milne

. Who was James Lind, and what exactly did he achieve?. J R Soc Med 2012; 105: 503–8.

Chalmers I, Tröhler U, Toth B, Milne I. Assessing the effects of treatments: what’s needed to inform the public? Background paper for an informal discussion, UK Cochrane Centre, Oxford, 2 April 2001.

Chalmers I. Assessing the effects of treatments: what’s needed to inform the public? Follow-up for people who were invited to attend the meeting at the UK Cochrane Centre on 2 April 2001, 4 September 2001.

Medical Research CouncilClinical Trials for Tomorrow, London: MRC Clinical Trials Series, 2003.

Warlow

. Clinical trials for tomorrow funded by the MRC. BMJ 2003; 327: 240–1.

Chalmers

. Well informed uncertainties about the effects of treatments: how should clinicians and patients respond?. BMJ 2004; 328: 475–6.

Chalmers I. Managers should help to address important uncertainties about the effects of treatments. BAMM (British Association of Medical Managers) News, 3–4 June 2004.

10.

Chalmers

. The James Lind Initiative. Cochrane Collaboration Methods Groups Newslett 2004; 8: 6–7.

11.

Tallon

Chard

Dieppe

. Relation between agendas of the research community and the research consumer. Lancet 2000; 355: 2037–40.

12.

Chalmers

. Letter from the editor: how often do researchers address questions of interest to clinicians and patients?. BMJ Clin Evid 2008.

13.

Partridge

Scadding

. The James Lind Alliance: patients and clinicians should jointly identify their priorities for clinical trials. Lancet 2004; 364: 1923–24.

14.

Scadding

Chalmers

. The James Lind Alliance: tackling treatment uncertainties together. RSM News Spring 2009, pp. 11–11.

15.

Oliver

Gray

. A Bibliography of Research Reports about Patients', Clinicians' and Researchers' Priorities for New Research, London: James Lind Alliance, 2006.

16.

Stewart R, Oliver S. A Systematic Map of Studies of Patients' and Clinicians' Research Priorities. London: James Lind Alliance, 2008. See http://www.lindalliance.org/pdfs/JLA%20Internal%20Reports/090712_summary_RS_map_%20studies_%20PPI%20 & %20Clincians_reasearch%20prioirites%20.pdf (last checked 21 May 2013).

17.

Staley K, Hanley B. Scoping research priority setting, and the presence of patient and public involvement, with UK clinical research organisations and funders. London: TwoCan Associates, 2008. See http://www.lindalliance.org/Scoping_research_priority_setting_PPI.asp (last checked 21 May 2013).

18.

Halls E. Where are the clinicians when you need them? BMJ 2010;340:c1845.

19.

Herxheimer A. In: Chalmers I. Assessing the effects of treatments: what’s needed to inform the public? Follow-up for people who were invited to attend the meeting at the UK Cochrane Centre on 2 April 2001. 4 September 2001.

20.

Fenton

Brice

Chalmers

Harvesting and publishing patients’ unanswered questions about the effects of treatments. In: Littlejohns

Rawlins

(eds). Patients, the Public and Priorities in Healthcare, Abingdon: Radcliffe, 2009, pp. 165–80.

21.

Brown

Brunnhuber

Chalmers

. How to formulate research recommendations. BMJ 2006; 333: 804–6.

22.

Crowe S. Setting Priorities for Treatment Uncertainties – A Review of Methods. Oxford: James Lind Alliance, 2009. See http://www.lindalliance.org/pdfs/Methods_page/JLA_Priority_Setting_approaches_V2_Nov_09.pdf (last checked 21 May 2013).

23.

Elwyn

Crowe

Fenton

. Identifying and prioritizing uncertainties: patient and clinician engagement in the identification of research questions. J Eval Clin Pract 2010; 16: 627–31.

24.

Buckley

Grant

Tincello

Wagg

Firkins

. Prioritizing research: patients, carers, and clinicians working together to identify and prioritize important clinical uncertainties in urinary incontinence. Neurourol Urodynam 2010; 29: 708–14.

25.

Chalmers I, Essali A, Rezk E, Crowe S. Is academia meeting the needs of non-academic users of the results of research? Lancet 2012;380:S43. See http://www.thelancet.com/health-in-the-occupied-palestinian-territory-2012 (last checked 23 May 2013).

26.

Crowe S, Fenton M, Hall M, Chalmers I. Researchers are not addressing the research needs of patients and clinicians. Submitted to BMJ, www.lindalliance.org.

27.

Cowan

Oliver

. The JLA Guidebook, Oxford: JLA, 2013.

28.

Chalmers I. Commentary: Registering, prioritizing and addressing treatment uncertainties. Eyes on Evidence November 2011, issue 31. https://www.evidence.nhs.uk/about-us/eyes-on-evidence/eyes-on-evidence-2011.

29.

Chalmers I. Commentary: Registering, Prioritizing and Addressing Treatment Uncertainties. Eyes on Evidence 2011;November:6. See https://www.evidence.nhs.uk/about-us/eyes-on-evidence/eyes-on-evidence-2011 (last checked 21 May 2013).

30.

SBU. Scientific Uncertainties: insufficiently Assessed Health Technologies, 2012. See www.sbu.se/uncertainties (last checked 21 May 2013).

31.

NICE/AMRC/JLI. Research prioritisation. Report of a Meeting held in London on 28 January 2013. See www.lindalliance.org (last checked 21 May 2013).

32.

Kumar

Soares

Wells

. Are experimental treatments for cancer in children superior to established treatments? Observational study of randomised controlled trials by the Children’s Oncology Group. BMJ 2005; 331: 1295–8.

33.

Chalmers

Matthews

. What are the implications of optimism bias in clinical research?. Lancet 2006; 367: 449–50.

34.

Djulbegovic

Kumar

Glasziou

. New treatments compared with established treatments in randomized trials. Cochrane Database Syst Rev 2012; 10: MR000024– MR000024.

35.

Evans

Thornton

Chalmers

. Testing treatments: Better Research for Better Healthcare, London: British Library, 2006.

36.

Evans

Thornton

Chalmers

Glasziou

. Testing Treatments: Better Research for Better Healthcare, London: Pinter & Martin Ltd., 2011.

37.

Chalmers

Milne

Tröhler

. The James Lind Library: explaining and illustrating the evolution of fair tests of medical treatments. J R Coll Physicians Edinb 2008; 38: 259–64.

38.

Irwig

Treveno

Sweet

. Smart Health Choices; Making Sense of Health Advice, London: Hammersmith Press, 2007.

39.

Woloshin

Schwartz

Walsh

. Know Your Chances; Understanding Health Statistics, California: University of California Press, 2008.

40.

ECRAN (European Communication on Research Awareness Needs). See http://cordis.europa.eu/search/index.cfm?fuseaction=proj.document & PJ_RCN=13177709 (last checked 21 May 2013).

41.

Chalmers

Clarke

. The 1944 Patulin trial: the first properly controlled multicentre trial conducted under the aegis of the British Medical Research Council. Int J Epidemiol 2004; 32: 253–60.

42.

Chalmers

The James Lind legacy: the present – the relevance of his work today. In: Rawlins

Littlejohns

(eds). Delivering Quality in the NHS 2004, Oxford: Radcliffe Medical Press, 2004, pp. 5–8.

43.

Chalmers

Statistical theory was not the reason that randomisation was used in the British Medical Research Council’s clinical trial of streptomycin for pulmonary tuberculosis. In: Jorland

Opinel

Weisz

(eds). Body Counts: Medical Quantification in Historical and Sociological Perspectives, Montreal: McGill-Queens University Press, 2005, pp. 309–34.

44.

Chalmers I. Why fair tests are needed: a brief history. Evidence-Based Med 2006;11:67–8 [republished in Evidence-Based Nurs 2007;10:4–5.

45.

Chalmers

Archibald leman cochrane. In: Bynum

Bynum

(eds). Dictionary of Medical Biography, Westport, CT: Greenwood Press, 2007, pp. 353–5.

46.

Chalmers

. Joseph Asbury Bell and the birth of randomized trials. J R Soc Med 2007; 100: 287–293.

47.

Forsetlund

Chalmers

Bjørndal

. When was random allocation first used to generate comparison groups in experiments to assess the effects of social interventions?. Econ Innovat New Technol 2007; 16: 371–84.

48.

Chalmers

. Historians and the history of medicine. Lancet 2008; 372: 1632–1632.

49.

Chalmers

. The development of fair tests of treatments in health care. HealthWatch Newsletter 2010; 76: 4–5.

50.

Chalmers

. Why the 1948 MRC trial of streptomycin for pulmonary tuberculosis used treatment allocation based on random numbers. J R Soc Med 2011; 104: 383–6.

51.

Chalmers

Dukan

Podolsky

Davey Smith

. The advent of fair treatment allocation schedules in clinical trials during the 19th and early 20th centuries. J R Soc Med 2012; 105: 221–7.

52.

Opinel

Tröhler

Gluud

. The evolution of methods to assess the effects of treatments, illustrated by the development of treatments for diphtheria, 1825–1918. Int J Epidemiol 2013.

53.

Chalmers

Jobling

Chalmers

. Is the NHS willing to help clinicians and patients reduce uncertainties about the effects of treatments?. Clin Med 2005; 5: 230–4.

54.

Chalmers

. Five dimensions of clinical care that should be understood by patients. Response to: Henderson GE, Churchill LR, Davis AM, Easter MM, Grady C, et al. Clinical Trials and Medical Care: Defining the Therapeutic Misconception. PLoS Med 2007; 4: e324–e324.

55.

Garattini

Chalmers

. Patients and the public deserve big changes in evaluation of drugs. BMJ 2009; 338: 804–6.

56.

Chalmers

. Addressing uncertainties about the effects of treatments offered to NHS patients: whose responsibility?. J R Soc Med 2007; 100: 440–1.

57.

Chalmers

. Better information systems are needed to help patients and clinicians integrate clinical research within everyday clinical practice. Otolaryngol – Head Neck Surg 2007; 137: S69–71.

58.

Van Staa

Goldacre

Gulliford

. Pragmatic randomised trials using routine electronic health records: putting them to the test. BMJ 2012; 344: e55–e55.

59.

Chalmers

. Make all research results public. MRC Network Spring 2013, pp. 22–3.

60.

General Medical Council. Good Medical Practice. London: General Medical Council, 2006.

61.

Chalmers

. Provision of consent. Lancet 2003; 362: 663–4.

62.

Glasziou

Chalmers

. Ethics review roulette: what can we learn?. BMJ 2004; 328: 121–122.

63.

Chalmers

. Regulation of therapeutic research is compromising the interests of patients. Int J Pharm Med 2007; 21: 395–404.

64.

Smith

Chalmers

Ethics and medical publishing. In: Ashcroft

Dawson

Draper

McMillan

(eds). Principles of Health Care Ethics, Chichester: John Wiley, 2007, pp. 751–8.

65.

Godlee

Chalmers

. Information about ongoing clinical trials for patients. BMJ 2010; 340: 456–7.

66.

Hey

Chalmers

. Mis-investigating alleged research misconduct can cause widespread, unpredictable damage. J R Soc Med 2010; 103: 132–7.

67.

Chalmers I. Research red tape contributes to the suffering and death of millions. The Guardian Notes and Theories, 16 November 2010. See http://www.guardian.co.uk/science/blog/2010/nov/16/research-red-rape-suffering-death/print (last checked 21 May 2013).

68.

Roberts

Prieto-Marino

Shakur

Chalmers

Nicholl

. Effect of consent rituals on mortality in emergency care research. Lancet 2011; 377: 1071–2.

69.

Roberts

Chalmers

Shakur

Prieto-Marino

Nicholl

. Consent in emergency care research – Authors’ reply. Lancet 2011; 378: 27–27.

70.

Chalmers

. Biomedical research: are we getting value for money?. Significance 2006; 3: 172–5.

71.

Chalmers

Glasziou

. Avoidable waste in the production and reporting of research evidence. Lancet 2009; 374: 86–9.

72.

Chalmers I, Lemon R, Dunnett S. Research synthesis. In: Tallis R (chair). Restoring Neurological Function: Putting the Neurosciences to Work in Neurorehabilitation. London: Academy of Medical Sciences, 2004: pp.34–5.

73.

Chalmers

. Academia’s failure to support systematic reviews. Lancet 2005; 365: 469–469.

74.

Chalmers

. The scandalous failure of science to cumulate evidence scientifically. Clin Trials 2005; 2: 229–31.

75.

Chalmers I. Written and oral evidence. In: House of Commons Health Committee. The Influence of the Pharmaceutical Industry. Fourth Report of the Session 2004–2005, Vol. II. London: Stationery Office, 2005, pp.194–208.

76.

Chalmers

. Role of systematic reviews in detecting plagiarism: case of Asim Kurjak. BMJ 2006; 333: 594–6.

77.

Chalmers

The lethal consequences of failing to make use of all relevant evidence about the effects of medical treatments: the need for systematic reviews. In: Rothwell

(eds). Treating Individuals: From Randomised Trials to Personalised Medicine, London: Elsevier, 2007, pp. 37–58.

78.

Clarke

Hopewell

Chalmers

. Reports of clinical trials should begin and end with up-to-date systematic reviews of other relevant evidence: a status report. J R Soc Med 2007; 100: 187–90.

79.

Clarke

Hopewell

Chalmers

. Clinical trials should begin and end with systematic reviews of relevant evidence: 12 years and waiting. Lancet 2010; 376: 20–1.

80.

Bastian

Glasziou

Chalmers

. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?. PLoS Med 2010; 7: e1000326–e1000326.

81.

Chalmers I. Systematic Reviews and Uncertainties about the Effects of Treatments. The Cochrane Library, 12 May, 2010. See http://www.thecochranelibrary.com/details/editorial/691951/Systematic-reviews-and-uncertainties-about-the-effects-of-treatments-by-Sir-Iain.html (last checked 21 May 2013).

82.

Chalmers I. Written evidence to Science and Technology Committee (PR47) – eighth report. Peer review in scientific publications. Ordered by the House of Commons to be printed, 18 July 2011.

83.

Chalmers I. Commentary on Robinson KA, Goodman SN. A systematic examination of the citation of prior research in reports of randomized, controlled trials. Ann Intern Med 2011;154:50–5.

84.

Chalmers I, Altman DG, McHaffie H, Owens N, Cooke R. Data sharing among data monitoring committees and responsibilities to patients and science. Trials [In Press].

85.

Chalmers I. In the dark. Drug companies should be forced to publish all the results of clinical trials. How else can we know the truth about their products. New Scientist 2004 March 6.

86.

Chalmers

. Government regulation is needed to prevent biased under-reporting of clinical trials. BMJ 2004; 329: 462–462.

87.

Chalmers

. Biased under-reporting of research. RSS News 2006; 34: 1–3.

88.

Chalmers

. From optimism to disillusion about commitment to transparency in the medico-industrial complex. J R Soc Med 2006; 99: 337–41.

89.

Chalmers

. For ethical, economic and scientific reasons, health-relevant degree theses must be made publicly accessible. Evidence-Based Med 2012; 17: 69–70.

90.

Chalmers I. Publish or Perish. Project Syndicate, 2012. See http://www.project-syndicate.org/commentary/publish-or-perish (last checked 21 May 2013).

91.

Chalmers

Dickersin

. Biased under-reporting of research reflects biased under-submission more than biased editorial rejection. F1000 Res 2012; 1: 69–69.

92.

Chalmers

Glasziou

Godlee

. All trials must be registered and the results published. BMJ 2013; 346: 8–8.

93.

Chalmers I. Acknowledging and researching treatment uncertainties in paediatric practice: an ethical imperative. Arch Dis Child Educ Pract Ed [In Press].

Tackling treatment uncertainties together: the evolution of the James Lind Initiative,2003–2013

Abstract

The origins of the JLI

Prioritizing uncertainties about the effects of treatments for further research

The James Lind Alliance (JLA): philosophy and early planning

The UK Database of Uncertainties about the Effects of Treatments (UK DUETs)

James Lind Alliance Priority Setting Partnerships (JLA PSPs)

Mainstreaming UK DUETs and the JLA

Enhancing general knowledge about testing treatments

The James Lind Library (www.jameslindlibrary.org)

Testing treatments: better research for better healthcare

Testing treatments interactive

JLI in future

Enhancing general knowledge about testing treatments

Monitoring and reducing waste in research

Footnotes

DECLARATIONS

References