Thomas C Chalmers (1917–1995): a pioneer of randomised clinical trials and systematic reviews

Introduction

Few people have contributed as much as Thomas (Tom) Chalmers to the fair assessment of treatment effects. His research was wide ranging, including both the randomised comparison of specific treatments and their effects and also studies about how randomised clinical trials and systematic reviews of randomised trials are and should be conducted and reported. Tom Chalmers’ research influenced many who have reflected publicly on this,^1–6 and many others who have not.

Tom Chalmers, a son and grandson of doctors, grew up in Forest Hills, New York, in Queens. He attended Yale University in 1939 and received his MD degree from Columbia University College of Physicians and Surgeons in 1943. He was an intern at Presbyterian Hospital in New York City and completed his residency at Harvard Medical Services of the Boston City Hospital in 1947.

Upon completion of his residency, Tom practised as a primary care internal medicine physician in Cambridge, Massachusetts, and also worked at the Thorndike Memorial Laboratory. He held appointments at Harvard Medical School from 1941 to 1975, and was Professor of Medicine at Tufts from 1961 to 1968. In 1968, he moved to the Washington, DC area, where he began work as head of the Research and Education Program of the Veteran’s Administration. Shortly afterwards, he was appointed Associate Director for Clinical Care and Director of the Clinical Center at the National Institutes of Health in Bethesda, Maryland. During this time he continued to work in an academic setting, as Professor of Medicine at George Washington School of Medicine.

In 1973, Tom began a 10-year term as President of Mount Sinai Medical Center and Dean of Mount Sinai Medical School in New York City. While continuing his promotion of the methodology of clinical trials there through the Clinical Trials Unit that he established, he is also credited with having created departments of Biomathematical Sciences and of Geriatrics and Adult Development. Citing ‘the increasing numbers of aging people and escalating medical costs [that] are creating a crisis with all sorts of ethical, medical, and moral problems,’ the Department of Geriatrics was one of the first of its kind at an American medical school. Under Tom’s leadership, the Mount Sinai Medical School not only became established as a leader in the study of biostatistics and geriatrics, but also became one of the highest ranked medical schools in the country.

Tom did a one-year sabbatical at Harvard School of Public Health in 1983–1984, with statistician Fred Mosteller. ⁷ Tom returned to Harvard in 1987 as part of the Health Care Technology Assessment Group, after completing his tenure as Chief of the Clinical Trials Unit at the Mount Sinai Medical Center. While in Boston, he also held a five-year post as Veterans’ Administration (VA) Distinguished Physician at the Boston VA Medical Center at Jamaica Plain (1987–1992). After leaving Harvard in 1992 he returned to Tufts University Medical Center as Adjunct Professor of Medicine.

In the early 1990s, a business, MetaWorks, was launched in response to a contract from a pharmaceutical company to perform a meta-analysis of research on the company’s calcium channel blocker drug. Tom was approached by MetaWorks in the summer of 1992 to serve as its chairman, and was involved with them as a major shareholder until his death. After Tom died, from 1997 to 2002, the company was designated by the Agency for Health Care Research and Quality as one of its 12 Evidence-Based Practice Centers.

Asking questions about the effects of inadequately tested treatments

Tom’s questioning approach to standard clinical practice began early in his clinical career. In 1951, he began his military service in Kyoto, Japan, as the principal investigator ⁸ in a randomised factorial trial ⁹ to assess the effects of then standard treatments for hepatitis among soldiers in the Korean War. The study found no evidence that the prolonged bed rest that was commonly prescribed at the time promoted recovery. The detailed, 70-page report is remarkable. Years later, the clinical epidemiologist David Sackett recounted the paper’s impact on him: ‘Reading this paper not only changed my treatment plan for my patient. It forever changed my attitude toward conventional wisdom, uncovered my latent iconoclasm, and inaugurated my career in what I later labelled “clinical epidemiology”’. ⁶

After returning from Japan, Tom continued his military service as a member of the metabolic unit of the Army Medical Services Graduate School at Walter Reed Medical Center in Washington, DC. He went on to serve as Chief of Medical Services at Lemuel Shattuck Hospital in Boston, Massachusetts, from 1955 to 1968, and also continued to practise clinically and do research in hepatology. He was obsessed with documentation and analysis, and challenged conventional clinical wisdom repeatedly.

‘Randomise the first patient’

Tom eventually left clinical practice to concentrate on refining and promoting randomised clinical trials and to ensuring that his students, colleagues and anyone else who would listen were encouraged to think critically and to question textbooks and other sources of authority in medicine. It was during this era that he began writing and speaking about randomisation as a decision-making technique, ¹⁰ and first proposed ¹¹ and subsequently illustrated how rigorous evaluation of new treatments could be integrated from the moment of their introduction into clinical practice by ‘randomizing the first patient’.^12,13 Tom was active and prominent in many aspects in the field of clinical trials. Indeed, his 1977 letter to the editor of the New England Journal of Medicine, ¹⁴ exhorting his colleagues to ‘randomize the first patient’, indicated prescience about trial registration when he said: ‘Also, there ought to be at the very least some better method of centrally recording the sporadic individual trials now going on.’ His colleagues from those days believe that Tom’s primary impact was his lifelong encouragement of colleagues in the medical profession to insist on evidence from clinical trials and on advising those performing clinical trials on how to make them better. ⁴

Exposing deficiencies in clinical trials

Tom’s teaching emphasised the scientific and ethical strengths of randomised clinical trials,^15,16 but he also drew attention to the weaknesses of many randomised studies.^17,18 His interest in the statistical power of controlled trials was longstanding. ¹⁹ In the late 1970s, he led a review of the statistical power of clinical trials published in major medical journals and showed that a high proportion were much too small to confirm or exclude treatment effects of clinical importance. ²⁰ In retrospect, Tom judged this paper to have had a net negative effect because it led some clinicians to cite it as an excuse for not embarking on randomised controlled trials to assess poorly evaluated aspects of their practice.

Concerned by the poor quality of the design, execution, analysis and reporting of clinical trials, Tom led an effort to promote improvements using a multi-item assessment to judge trial quality. ²¹ Although the checklist was complicated and its usefulness challenged by the CONSORT Group, ²² this early effort, together with Tom’s key role in organising the SORT group, one of two groups working concurrently on reporting standards for clinical trials, ²³ eventually led to the successful consensus reached by the CONSORT group. ²² Tom’s paper co-authored with Henry Sacks and Harry Smith involved the comparison of results of studies using randomised controls with those that had used historical controls. ²⁴ This paper was a reminder of how important it is to take steps to reduce allocation biases in tests of treatments, and another published the following year analysed bias in treatment assignment in clinical trials. ¹⁷

Systematic reviews, meta-analyses and reporting bias

The problem of trials with inadequate sample sizes exposed in the paper by Freiman et al. ²⁰ was one element in a growing recognition that more statistically reliable estimates of treatment effects could be obtained by synthesising the results of similar trials using meta-analysis, a term that had recently been introduced by Gene Glass, a social scientist in the USA. ²⁵ While he was President and Dean of Mount Sinai Medical School in New York from approximately 1978 to 1988, Tom led a seminal programme of research, funded by the National Library of Medicine on ‘Technical Evaluation of the Clinical Literature’ (LM-03116), which made extensive use of meta-analysis.^26,27 This grant also resulted in numerous publications on reporting bias ²⁸ a phenomenon to which Tom first drew attention in 1965, ²⁹ on searching for relevant clinical trials for systematic reviews using MEDLINE ³⁰ , and on the need for meta-analysis generally.^31,32

In this regard, Tom helped to initiate and contributed substantially to the wave of meta-analyses in medicine which took off during the late 1970s and 1980s, following his initial illustrations of systematic reviewing of the available data using examples from serum hepatitis, ²⁹ transaminase tests in liver disease, ³³ portacaval shunts, ¹² emergency surgical treatment of bleeding ulcer, ³⁴ trials of Vitamin C for the common cold, ³⁵ and trials of anticoagulants in myocardial infarction. ³⁶ Tom and his colleagues showed how synthesising the results of systematically collected, similar but separate studies could yield more useful information than examining study findings one by one, and provide statistically more robust estimates of treatment effects. ³¹

Tom’s role as a lecturer at the Harvard School of Public Health during his 1983–1984 sabbatical, and other teaching roles, is particularly noteworthy in terms of his leadership in establishing and refining meta-analysis as a necessary method in assessing treatment effects. At Harvard, he co-taught a seminar in meta-analysis with Fred Mosteller, ⁷ and initiated collaborations with Alessandro Liberati, Anne Jacquotte, Marc Buyse, and Kay Dickersin, among others. Tom was known for encouraging and publishing with students and colleagues across a wide variety of clinical specialties, one of whom later rated Tom’s greatest achievement as having been his cultivation of the careers of countless young investigators, as ‘a most generous and humble mentor’. ³ This mentorship lives on in the Student Scholarship Program established by the Society for Clinical Trials, and the Thomas C. Chalmers Prize awarded each year by the Cochrane Collaboration.

From then until the end of his life, Tom’s main research preoccupation became improving methods of research synthesis and of the studies on which they were based.^32,37,38 He and his colleagues at Mount Sinai Medical School in New York and Boston were among the most productive medical users of meta-analysis.^39–42

The most significant of Tom’s work on research synthesis was probably the series of retrospective cumulative meta-analyses of randomised trials, ⁴³ in particular those analysing treatments for myocardial infarction. Comparisons of these analyses with what had been written in contemporary textbooks and review articles showed that valid advice on some life-saving treatments had been delayed for more than a decade, and other forms of care had been promoted long after they had been shown to be harmful.^44,45 This report made it abundantly clear that the failure of researchers to prepare reviews of therapeutic research systematically could have very real human costs. ⁴⁶

Later years

Tom loved travelling and attended many national and international meetings and events. Around mid-1993 he developed leg oedema in one leg, and this turned out to be metastatic prostate cancer. Up to the week he died of prostate cancer in 1995, he continued to juggle the final stages of multiple projects while scheming about the next question to tackle. For example, in an interview in 1993 (two years before Tom’s death), Malcolm Maclure^1,2 asked whether he thought it would be better if the first course in medical school should be ‘the Anatomy of Evidence’. Tom replied as follows:

I have become convinced that this is the last crusade I am going on: the first two years of medical school have got to be changed. Students are spending more and more time understanding the difficult aspects of molecular biology, but we are kidding ourselves to think they use their knowledge of DNA – in my day it was the Krebs cycle – in making clinical decisions at the bedside. They do not. They make clinical decisions based on how the last patient did, how their friends are treating patients and what the latest article by an authority says they should do. And we have got repeated evidence now that authorities are way behind with regard to the data in clinical trials. (citing one of his most influential papers ⁴⁴ in support of this assertion)

Tom’s obsession with testing treatments was reflected in his personal and family life. He randomised his route to work through the city of Boston to identify the quickest average journey time. He based his timing of when to drink decaffeinated rather than caffeinated coffee on blinded randomised studies done with Frankie Talcott, his wife of 53 years, and Tom’s choice of wine was made after blinded wine tasting. His four children and six grandchildren have all been left with a keen sense of the importance of seeking reliable evidence. The blinded wine tastings continue to this day and the cheaper wines continue to do very well! And so when it came time to decide on a treatment plan for Tom’s metastatic prostate cancer, it did not come as a surprise that he insisted on being treated within a randomised trial.