Abstract
Introduction
For clinicians wanting to keep up to date, the proliferation of research is an impossible blessing. Medline, for example, adds over one million new records each year. Because of this tsunami of new information, it has been estimated, for example, that around 7% of the clinical conclusions from systematic reviews change every year. 1 Without some systematic assistance, keeping abreast of this vast and scattered research literature is simply not feasible for clinicians. As the problem has grown, attempts at systematic assistance to cope with it have evolved in two ways: collected summaries of texts and bibliographic databases, now electronic.
The systematic, evidence-based approach to clinical updates has four elements:
Collect the new research in an area, or the whole of medicine. Select the most relevant and valid research, the results of which might imply a change in clinical practice. Summarise the new research in the context of what is known already about this topic. Interpret this evidence in the broader context of clinical implications (e.g. comparison with other options, appropriate groups of patients).
The 17th century: ‘Weekly Memorials for the Ingenious’
The problem of changing medical knowledge is not new. Throughout medical history there have been attempts to cope with changes in practice, such as the publication on 16 January 1682 of an abstracting journal, thought to have been edited by a Mr Beaumont, called Weekly Memorials for the Ingenious: OR An Account of Books lately set forth in several Languages, With some other curious novelties relating to Arts and Sciences (printed for Henry Faithorne and John Kersey in St. Paul’s Churchyard), which drew much of its material from the French Journal des Sçavans. 2 It occasionally included medical material, an example of which is to be found at the start of the issue dated 3 April 1682, in which a book by Dr George Wolfgang Wedel, Disputatio de Arthritide vagâ Scorbutica, was summarised. Beaumont fell out with his publishers, who continued to publish the Weekly Memorials, while he turned to R Chiswel, W Crook, and S Crouch to produce a rival magazine with the same name, which folded after only 29 issues.
Although the Philosophical Transactions of the Royal Society (founded in March 1665, two months after the Journal des Sçavans) sometimes reported medical matters, Medicina Curiosa, which first appeared in 1684, is regarded as the first purely medical journal to be published in England. 3
The 18th century: ‘Commentaries’ on collections of the scattered literature
In 1752, Christian Gottlieb Ludwig founded the Commentarii de rebus scientia naturali et medicina gestis as a periodical published in Leipzig, Germany. 4 It continued until 1808 and contained abstracts of scientific and medical books. Gottlieb established an editorial group to disseminate current news from the world of medicine. The group met weekly to discuss what they had read, and to write reviews to communicate the contents of the book under discussion, particularly for readers with limited resources and no access to a medical library.
A related notable 18th century example from Edinburgh was Andrew Duncan’s Medical and Philosophical Commentaries (founded in 1773 and renamed Medical Commentaries in 1780), which attempted to collect, select, and summarise new books in medicine every quarter.5,6 In the first volume Duncan wrote:
From the liberal spirit of inquiry which universally prevails, it is not surprising that scarce a day should pass without something being communicated to the public as a discovery or an improvement in medicine. It is, however, to be regretted, that the information which can by this means be acquired, is scattered through a great number of volumes, many of which are so expensive, that they can be purchased for the libraries of public societies only, or of very wealthy individuals. No one, who wishes to practise medicine, either with safety to others, or credit to himself, will incline to remain ignorant of any discovery which time or attention has brought to light. But it is well known that the greatest part of those who are engaged in the actual prosecution of this art, have neither leisure nor opportunity for very extensive reading…. …The last section will consist of a list of new medical books … for the satisfaction of those who may be deprived of other methods of information …, published, both in this and other countries, during the three preceding months. We cannot, indeed, pretend that this list will in any case be a complete one; but it will be our endeavour to render it as much so as our situation will allow; and we are hopeful we shall be able to obtain intelligence of every material book. (Introduction, vol. 1, 1773)
The 19th century: responding to the growth of medical journals
The challenge of keeping up to date became more complex in the 19th century, with the growth of medical journals. 7
Duncan’s Commentaries were a remarkably innovative attempt to help clinicians keep up to date. As the Edinburgh Medical and Surgical Journal, subtitled ‘Exhibiting a concise view of the latest and most important discoveries in medicine, surgery, and pharmacy’, it contained three sections titled ‘Original Communications’, ‘Critical Analysis’, and ‘Medical Intelligence’ (i.e. news). In 1855, it was combined with the Monthly Journal of Medical Science to form the Edinburgh Medical Journal, which ceased publication only in 1954, when it was described as a ‘literary product of one of the older and certainly one of the most eminent centers of medical culture’. 8 It re-appeared as the Scottish Medical Journal in 1956, after merging with the Glasgow Medical Journal.
In 1840, William Braithwaite’s twice-yearly Retrospect of Practical Medicine and Surgery provided a similar service to Duncan Jr’s Annals of Medicine, but focused on new articles in medical journals rather than books.
In the USA, The National Library of Medicine began as a few books in the office of the army’s Surgeon General, Joseph Lovell, between 1818 and 1836. 9 After the Civil War, it became the nation’s largest medical library, under the direction of John Shaw Billings, Director of the Library of the Surgeon General’s Office from 1865 to 1895.10,11 Billings, inspired by the difficulty he had had in finding the literature he needed for his MD thesis, recognised the problems caused by the proliferation and scatter of new books and journals. His aim was that the Library should ‘contain every medical book published in this country [the USA] and every work relating to public health and state medicine’. He initiated the Index-Catalogue of the Library of the Surgeon General’s Office, which continued until 1950, and the Index Medicus.
Index Medicus, first published in 1879 by the Library of the Surgeon General’s Office, appeared monthly and listed the titles of current medical articles, books, reports, and other medical literature. It focused only on the initial ‘Collect’ element (see Step 1 above), but that made the other steps more feasible for those who wanted to select, summarise, and interpret the growing research literature. Index Medicus struggled to pay its way and was abandoned in 1899, but was revived in 1903 and continued until 1927, when the Quarterly Cumulative Index Medicus started publication under the auspices of the American Medical Association (AMA). Then in 1959, after the promulgation of the National Library of Medicine Act of 1956, it was agreed that Index Medicus would be revived again and be published by the National Library of Medicine (NLM), with the AMA responsible for an annual compilation of the accumulated data.
In the 1960s, computerisation led to the introduction by the NLM of the Medical Literature Analysis and Retrieval System (MEDLARS) and then an online version, Medline, in 1971. PubMed, the search engine that accesses the data in Medline, was initiated in 1996, run by the NLM.
In parallel development with these databases was the Excerpta Medica abstracting system, which a group of Dutch physicians began in 1946. It was taken over by Elsevier in 1972, and the online version, EMBASE, was launched in 1974. Over approximately the same time scale other databases have emerged, including Psycinfo from the American Psychological Association, the electronic version of Psychological Abstracts (1967, backdated to 1806), the Philosopher’s Index, published by the Philosopher’s Information Center (1967, backdated to 1902), and the Cumulative Index to Nursing Literature (CINL) in 1961, which became the Cumulative Index to Nursing and Allied Health Literature (CINAHL) in 1977 and went online in 1984.
The 20th century: emphasis on indexing and association
In 1938, the British author HG Wells collected a set of lectures, articles, and speeches, which he had contributed in various places in the UK and USA, under the general title World Brain, 12 in which he postulated the need for a world encyclopaedia. A few years later, Vannevar Bush, at that time Director of the US Office of Scientific Research and Development, coined the term memex, to describe ‘a sort of mechanized private file and library… a device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility’. 13
These ideas paved the way for the development of online bibliographic databases in the second half of the 20th century, starting in 1953 with the publication by Eugene Garfield 14 of Contents in Advance, which consisted of 50% photo-reduced contents pages from a range of science, documentation, and computer journals. Garfield, a chemist by training with no aptitude for bench work, had become interested in indexing, and was at that time working with the Welch Project at Johns Hopkins University, studying the application of machine methods to medical and scientific indexing. The project ended in 1955, as did Contents in Advance. It was succeeded by other similar publications, which also failed, until the publication in 1958 of Current Contents Life Sciences. Current Contents is now available online in different editions covering different subjects. Garfield founded the Institute for Scientific Information (ISI) in a converted chicken coop in Philadelphia in 1960, and in 1963 he launched the Science Citation Index, 15 which is now available online through various portals.
In a paper in Science, titled ‘Citation indexes for science’, Garfield
16
wrote: In this paper I propose a bibliographic system for science literature that can eliminate the uncritical citation of fraudulent, incomplete, or obsolete data by making it possible for the conscientious scholar to be aware of criticisms of earlier papers. It is too much to expect a research worker to spend an inordinate amount of time searching for the bibliographic descendants of antecedent papers. It would not be excessive to demand that the thorough scholar check all papers that have cited or criticized such papers, if they could be located quickly. The citation index makes this check practicable. Even if there were no other use for a citation index than that of minimizing the citation of poor data, the index would be well worth the effort required to compile it. If one considers the book as the macro unit of thought and the periodical article the micro unit of thought, then the citation index in some respects deals in the submicro or molecular unit of thought.
The ISI was acquired by Thomson Scientific & Healthcare in 1992, when it became known as Thomson ISI, part of Thomson Reuters; in 2016, it became Clarivate Analytics Web of Science (http://clarivate.com/news/ip-and-science-launched-as-independent-company). Rival databases include CiteSeerX, Google Scholar, and Elsevier’s Scopus.
Late 20th century: emphasis on validity and synthesis
As medical journals and research have grown, the challenges have become greater, and 20th century commentaries have developed. However, few of the services developed have been explicit about their methods, or have had well-developed tools for critical appraisal or synthesis of evidence. Examples that have used rigorous but quite different processes are The Medical Letter, Prescriber’s Journal, the ACP Journal Club, and the ODPT Newsletter.
The Medical Letter: emphasising clinical trials of new drugs
In 1959, Arthur Kallet and Harold Aaron founded the American Medical Letter on Drugs and Therapeutics (the Medical Letter). 21 Kallet was an engineer who co-authored the book 100,000,000 Guinea Pigs: Dangers in Everyday Foods, Drugs, and Cosmetics, a best-seller in the 1930s, 22 and started the publication Consumer Reports. 23 Aaron was an internist who suggested to Kallet that doctors could use something like Consumer Reports to help them judge the worth of new drugs coming onto the market. The Medical Letter, which is now published online at http://medicalletter.org, is a non-profit publication that does not accept advertisements, grants, gifts, or donations. Support comes from sales of subscriptions, books, software, continuing education materials, and licences. Published every two weeks, it evaluates almost all new drugs and gives priority to information from clinical trials. A first draft is sent for comments or corrections to the members of its Advisory Board, the first authors of all the articles cited in the text, eight to twelve consultants from specialist panels, the manufacturer of the drug being evaluated, the manufacturers of competing drugs, and the US Food and Drug Administration (FDA).
Prescribers’ Journal
Also in 1959, the UK’s Hinchliffe Committee on Costs of Prescribing recommended that a new journal should be established ‘to distribute to general practitioners up-to-date information about new drugs and preparations and the results of clinical trials’ (http://www.bmj.com/content/1/5231/1021). Prescribers’ Journal first appeared in April 1961, funded by the Department of Health and edited by a Management Committee that was initially chaired by Max (later Lord) Rosenheim. 24 Articles were commissioned and edited by the Committee. Funding was withdrawn in 2000, when it was decided that the evidence-based work of the National Institute for Clinical Excellence (NICE, now renamed the National Institute for Health and Care Excellence) made the journal redundant.
The growth of drug bulletins
The 1960s saw the emergence of a number of drug bulletins, in part triggered by the thalidomide disaster. Like the Medical Letter, these bulletins aimed to apprise clinicians of new drugs and new information about current drugs. In April 1962, the Consumers’ Association published the first issue of a British edition of the Medical Letter (http://dtb.bmj.com/content/40/4/25.long). A year later, formal ties with the parent journal were severed, the title was changed, and the Drug and Therapeutics Bulletin (DTB) was launched under the editorship of Andrew Herxheimer, 25 who had proposed its publication and who edited it for 30 years. Herxheimer oversaw the production of 778 issues (about two million words), and the circulation increased to 130,000 while he was the editor. Draft articles were written by invited authors and sent out to 30–40 reviewers, each of whom contributed comments. The reviewers included manufacturers, specialists, patients, representatives of the UK Department of Health, and members of an Advisory Council and Editorial Board. The final version was then prepared by the editors and published anonymously. Conclusions were based, whenever possible, on evidence from published randomised clinical trials or systematic reviews of these studies. The Drug and Therapeutics Bulletin, funded by the Department of Health, was sent to all UK prescribers, but funding was withdrawn in 2006 and the BMJ Group now publishes it under subscription. Later bulletins elsewhere, such as the Dutch Geneesmiddelenbulletin, used a similar model (http://geneesmiddelenbulletin.com/english/how-the-bulletin-works).
Countries in which bulletins that are members or associate members of the ISDB are published (number of bulletins in each).
Information found on the ISDB’s website (http://www.isdbweb.org/) on 27 May 2017.
The ACP Journal Club: validity filters for treatment, diagnosis and prognosis
Over a similar period, another approach to evidence alerts was developed by Brian Haynes at McMaster University.26,27 The ACP Journal Club scanned a set of major medical journals and provided summaries of the most valid, relevant, and newsworthy new research. A major advance over previous journal scanning services was the explicit set of rules of validity, which were used as the initial screen, before asking practising clinicians to assess relevance and newsworthiness. The small number of articles that passed this rigorous selection process was summarised, the results recalculated, and a commentary commissioned that set the results in the context of previous research, although without attempting a formal synthesis. The process was described in the first editorial: 28
“We prepare more informative abstracts for the articles that meet all criteria, so that you, the reader, can verify whether the conclusion is likely to be true and, if so, whether it applies to your own clinical practice. Abstracts are followed by a commentary from a physician with expertise in both critical appraisal and in the clinical content of the article in order to expand on key features of the methods and to set the article in the context of other key studies in the field.”
An evaluation of the ACP journal selection process in 2004 29 illustrates the problem that clinicians face in keeping up to date. Of 60,352 articles from 170 journal titles, 3059 original articles and 1073 review articles met the criteria of high-quality methods and clinically relevant material. The contributions differ vastly by journal. The top three titles (New England Journal of Medicine, JAMA, and Lancet) and the Cochrane Database of Systematic Reviews provided 57% of the articles abstracted, 28 other journals providing the other 43%. A sample of the ACP Journal Club’s ‘validity criteria’ is shown below.
The Oxford Database of Perinatal Trials: systematic reviews of clinical trials
The development of systematic reviews and meta-analyses in medicine in the 1980s30–33 led to the emergence of two new services. These emphasised the need to assess research validity before considering research relevance and to set the results of individual studies in the context of other studies that addressed the same or similar questions. 34
One of these services consisted of four linked publications presenting the results of analyses of research assessing the effects of care during pregnancy, childbirth, and early infancy. The first was a 1500-page, 2-volume book Effective Care in Pregnancy and Childbirth (ECPC). 35 The methods section of the book described how assessment of the methodological quality of potentially relevant evidence was sought, assessed, and analysed using meta-analysis if judged appropriate. A summary of the analyses and findings presented in ECPC was published concurrently in A Guide to Effective Care during Pregnancy and Childbirth (GECPC), a paperback produced for women using maternity services. 36 At the same time, the Oxford Database of Perinatal Trials (ODPT) – which had provided the management and analysis capacity for producing ECPC – was launched as a six-monthly electronic publication with accompanying newsletters, drawing subscribers’ attention to important new evidence. 37 In the early 1990s, these processes were extended to the synthesis of research evidence relevant to the care of newborn infants. 38 The success of the Oxford Database of Perinatal Trials (ODPT) and its novel processes influenced wider uptake of periodically updated systematic reviews, and formed the ‘pilot’ for the Cochrane Collaboration. 39 However, the Collaboration has never achieved the same coverage and periodic updating as the original ODPT model.
Clinical Evidence: summaries of systematic reviews
In 1995, Tom Mann et al. at the UK’s NHS Executive asked the BMJ to explore the possibility of developing an evidence-based resource along the lines of the British National Formulary. 40 The idea was to provide a pocket-sized book containing concise and regularly updated summaries of the best available evidence about clinical interventions. The BMJ enlisted the help of the American College of Physicians and convened an international advisory board, held focus groups of clinicians, and talked to patient support groups. The first issue of the resulting publication, Clinical Evidence, which appeared in 1999, contained summaries dealing with the prevention and treatment of about 70 common conditions, each based on a thorough search and appraisal of the literature, good systematic reviews and randomised controlled trials; the summaries were written by clinicians with skills in epidemiology and were extensively peer reviewed. The material is now available online (http://clinicalevidence.bmj.com/x/index.html).
Conclusions
The problem for clinicians (or their educators) wanting to keep up to date continues to grow. An analysis in 2010 suggested that 75 new trials and 11 systematic reviews were being published each day, 41 and those numbers continue to increase. In the past few decades, many evidence alert services have emerged, those mentioned above being just a sample of the pioneers. Organised efforts by the professions to identify the valid and important changes from research must continue to build on these pioneering efforts 42 (http://community.cochrane.org/review-production/production-resources/living-systematic-reviews).