Méta-analyse en médecine : the first book on systematic reviews in medicine

Editorial introduction by Iain Chalmers

Interest in systematic reviews and meta-analysis in medicine began in the late 1970s.^1–6 During the 1980s, these methods began to be adopted more widely by medical researchers, and in the late 1980s, expository journal articles began to appear,^7–11 and the first book about meta-analysis in medicine was published. ¹²

The book was published in 1987 by Milos Jenicek, a professor at the Université de Montréal. Too often, the anglophone world remains unaware of important contributions to science and other fields which have been published in languages other than English. So it was with this book, which was published in French. A bilingual friend – Michael Kramer, a professor of epidemiology at McGill University in Montréal – obtained a copy of the book for me in 1994. After reading and greatly enjoying it, I visited Montréal in October of that year and asked Milos to sign my copy. He wrote: ‘To Dr Iain Chalmers, with compliments and astonishment that he still believes that this book is worth reading’.

Méta-analyse en médecine was and remains well-worth reading, even for someone whose knowledge of French is not very strong. I regard the book as an important and insufficiently acknowledged milestone in the development of methods to assess the effects of medical treatments. Even if I or others had cited the book appropriately, however, it would have been difficult and probably impossible for our readers to have accessed copies of it, as its stock was shredded by the publisher not all that long after it had been published. It is for this reason that fairly long excerpts from the book, with translations, have been added to the James Lind Library.

Because of the importance of the book in the history of research synthesis in medicine, I wanted to find out from Milos Jenicek how he came to write it. What follows takes the form of an interview, although it is not a verbatim account of our conversations and communications.

IC: How far back does one have to go to identify the origins of the ideas that led to the book?

MJ: Almost half a century! I was born and went to medical school in Prague. In the 1960s, Patrick Hamilton, an epidemiologist at the London School of Hygiene and Tropical Medicine, visited me and gave me a copy of JN Morris’ Uses of Epidemiology. ¹³ The book reflected the beginning of a transition from the epidemiology of infectious diseases to its expanded scope to include ‘chronic diseases’. It opened my eyes to the population approach to thinking about health, disease and interventions.

From there, I read JE Gordon’s ¹⁴ paper Epidemiology in modern perspective, which put epidemiology in a philosophical framework applicable to almost all areas of medicine, followed by JN Paul’s ¹⁵ Clinical Epidemiology. It was through this immersion and influence that I overcame the rigid and stifling ideology then dominating Central Europe and embraced the concept of freedom of speech, movement, thought and association. Since that time, supported by an increasing array of quantitative and qualitative methodologies, I have devoted myself to this way of logical thinking and reasoning in medicine.

IC: What brought you from Prague to the Université de Montréal?

MJ: A plane! All jokes aside, the tanks of some Slavic brethren visiting my home country in 1968 convinced me to move to a place I could embrace with all my heart and soul. Thanks to the French Lycée in Prague, the Academic Lycée and the almost military guidance of my mother – a professor of French – I had become multilingual. This allowed me to land not only as an academic and professional ‘coopérant’ in North Africa, but also later (1970) as a warmly-received epidemiologist at the Université de Montréal, working on the growth and development of French Canadian children. The boundless welcome, interest, confidence, hospitality and attention of my Québec colleagues touched not only my heart and spirit, but also those of my entire family.

IC: Your first book was Introduction à l’épidémiologie. Tell me about that.

MJ: It was the first of over a dozen monographs (so far). In 1972, I was a young assistant professor in charge of a basic undergraduate course in epidemiology for medical students who were preparing for the licensure examination of the Medical Council of Canada. This meant that I regularly had to face about 200 very stressed individuals! To save my skin, I did everything I could to meet their and our general and specific needs, and this included writing a textbook. ¹⁶ The working language at the Université de Montréal is French, but existing French textbooks were not very suitable because they presented only a few statistical methods as the core of epidemiology. So I wrote my own book, trying to convey not only my ideas, but also responding to the shared objectives of teaching epidemiology to medical undergraduates in Canada. I later published a paper about the experience in co-authorship with Robert Fletcher, the American clinical epidemiologist. ¹⁷

IC: Something interesting seems to have happened in the mid-1980s in Montréal. What was it exactly, and how did it relate to Méta-analyse en médecine?

MJ: First, Walter Spitzer, then Chairman of the Department of Epidemiology at McGill, initiated summer clinical epidemiology programmes, for which he gathered together a number of people including Alvan Feinstein (from Yale), David Sackett (from McMaster) and me. Walter, later dubbed 1985 the ‘year of the Book’, since David, ¹⁸ Alvan ¹⁹ and I ²⁰ all published books on clinical epidemiology that academic year.

Second, there was the stimulating environment created in Le Cercle de Montréal. Mont Royal dominates the city of Montréal: the Université de Montréal is on its northern slope, and McGill University is on its southern slope. In an effort to bring together colleagues from the two universities in 1985, McGill’s Olli Miettinen convened a sort of ‘think tank’ called Le Cercle de Montréal to discuss issues relating to the theory of medicine. Meta-analysis was not the Cercle’s focus, but it was an excellent environment for lateral thinking, triangulation of ideas and the creation of new ones. On the one hand, there were innovations and developments directly applicable to health sciences, such as case–control study methodology in observational etiological research, increasingly rigorous clinical trials, medical technology assessments and electronic management, retrieval, evaluation and uses of medical and other information, including evidence. On the other hand, we looked at what was happening in other domains, initially unrelated to medicine, such as decision analysis, economics, information technology and informal logic and critical thinking from philosophy.

Research synthesis was an example of this second category of innovation in health research. We owe the basic idea of meta-analysis to American psychologists and social scientists, who had applied research synthesis to topics such as the effects of psychotherapy, diet for hyperactivity and perceptual skills and reading ability. The term ‘meta-analysis’ was coined by an American psychologist, Gene Glass, ²¹ and the word had been adopted by other psychologists, educationalists and statisticians in the United States, such as Richard Light and David Pillemer^22–24 and Robert Rosenthal. ²⁵

It was during the preparation of the clinical epidemiology book I co-authored with Robert Cléroux that this work in social sciences, and some of the early work in medicine led by Tom Chalmers in the USA and Richard Peto in the UK, began to shape my ideas about the need for systematic reviews and meta-analysis of scientific evidence in medicine – an ‘epidemiology of research findings’. When I became adjunct professor on ‘the other side of the mountain’ (that is, at McGill), my basic graduate course in clinical epidemiology at the Université de Montréal also bore a McGill course number, hence making it accessible to most graduate students in Montréal. My teaching responsibilities made it clear to me that methodologically sound research synthesis should be part of medical graduate training and subsequent practice, and that it should be taught to others as well. Some of my ‘students’ were professors and other medical academicians interested in advanced clinical epidemiology. It was my ideas for this course that first led me to conceptualise Méta-analyse en médecine. Beyond the issues the social scientists had tackled, I think our main contribution to research synthesis, as it became adopted in medicine was to insist that the methodological quality of the primary studies should always be taken into account.

IC: You acknowledge the involvement of several people in your book. Tell me how they helped

MJ: In many different ways. The development of the project and critical readers of the text included Sylvie Stachenko, then Associate Professor of Family Medicine at the Université de Montréal and later principal author of The Canadian Guide to Clinical Preventive Health Care; Xavier LeCoutour, one of my graduate students who initiated a review of the effectiveness of fetal monitoring; and Sammy Suissa, Professor of Biostatistics, my neighbor and friend in Montréal. Contacts with colleagues in France, like Jean-Pierre Boissel, had not developed at that time.

I commissioned my wife, Jana, a university-trained professional artist and painter, to create the cover for the book. I found her idea to be quite wonderful and symbolic: if you open up and look at the front and back covers simultaneously, you will discover two eyes looking at tiny particles, cells, atoms, blood cells, etc., like a sea of elements (original studies) allowing the mind to make some sense of their results – a representation of the process of research synthesis.

IC: What was the reaction to the book once it was published?

MJ: Lukewarm, to put it mildly. Few people understood the term ‘meta-analysis’ because the first expository articles about systematic reviews and meta-analysis for a general readership were published more or less concurrently with the book.^7–9 One of my colleagues in Montréal laughed in my face when I showed him the title of the book. “Do we really need this?”, he asked.

Even my publisher was lukewarm, but he was still prepared to take it on because my three preceding books had sold well. Fewer than one thousand copies were printed, but that should have been enough for a book written in French. However, he became even more lukewarm after witnessing the initial reactions to the book and the relatively limited sales. The book did not fit any francophone courses outside Montréal at that time. However, interest in the book did steadily increase, especially when systematic reviews and meta-analysis became an important element of the so-called ‘Evidence-Based Medicine (EBM)’ movement. Sadly, these developments occurred mainly after the publisher had shredded what remained of the initial printing, which is the main reason that it is more or less impossible to find a copy of the book today. Many years later, the publisher said to me: “Monsieur, one of your numerous faults, as far as this publisher and the market are concerned, is that you write things in which readers are interested only ten years later.”

English-speaking colleagues, who were geographically closer to me, often ignored and did not even cite the book, probably simply because it was in French. You were a notable exception to this trend, Iain. On a visit to McMaster in the 1990s, you were the first to call my book ‘a little jewel’ and this warmed my heart tremendously. Although I made a presentation based on the material in my book at the meeting of the International Epidemiological Association in Helsinki in 1987, it took a further two years for an expanded version of my paper to appear in print in English. ¹¹ I had sent my manuscript to my good friend Alvan Feinstein, editor-in-chief of the Journal of Clinical Epidemiology, and he forced me to rewrite the paper about three times. He disliked meta-analysis from the outset, and it may have been only because I was a member of his editorial board that he eventually accepted my manuscript. Off the record, he told me that he thought that meta-analysis was all bullshit, a view he confirmed in public in the journal he edited, where he characterised it as ‘statistical alchemy for the 21st century’. ²⁶ Undeterred by such attitudes, I have continued to publish and develop my thinking about systematic reviews and meta-analysis.^27,28

IC: How do you think systematic reviews and meta-analysis are regarded today?

MJ: Meta-analysis and systematic reviews have become part of mainstream medical research and thinking. They quickly became one of the workhorses of EBM and they are now an integral part of it. But how should we link and integrate research findings and patient preferences and values, clinical circumstances and settings, as well as other sources of clinical experience? This is a greater challenge than the development of new methods in systematic reviews and meta-analysis.

In my opinion, typical odds ratios or some other overall expressions of effect of medical interventions are much less important than the original ‘epidemiology of results’, analysis and interpretation of discrepant findings, formulation of new hypotheses on that basis and mapping out directions for the future. Much of the focus until now has been on synthesising the results of randomised trials, but what should we do about the integration of alternatives to RCTs, such as time series analyses, or n-of-1 trials? What about the integration of findings derived from analytical observational research? How can and should we integrate findings about screening and diagnostic tests, ‘simple’ incidence studies, or studies of prognosis? Should clinical case series reports not be presented as systematic reviews of cases? ²⁹

Are we investing too heavily in an excessively precise concept of some overall treatment effect instead of more closely examining the heterogeneity of findings, their nature, the biological explanation of such heterogeneity and what it really means for decision making? How should we refocus, expand or reduce research findings to particular subgroups of patients and community groups, or generalised policies for all? Knowing how to rationally and pragmatically use findings from research syntheses is just as important as methodologically brilliant research syntheses.

How simple life was for us when all this started in the 1980s! How everything looked crystal clear when we opened this Pandora’s box! Today, we can hardly imagine coping with the explosion of medical information without some process of research synthesis to deal with it all. Is there something out there to cope with this challenge which is better than systematic reviews and meta-analysis as we know them today? Place your bets!