Abstract
This is the second part of the two-part paper on the University Group Diabetes Program
Lessons
Publish first, present later
University Group Diabetes Program investigators, early on, agreed to a ‘publish first, present later’ policy for primary results from the trial. However, as often happens, there is backsliding when faced with reality. The first test of the policy came with the decision to stop the tolbutamide treatment. The investigators ultimately decided in favour of presentation with the expectation of having the results published by the time they were presented. Big mistake!
Abstracts for three presentations1–3 were submitted for the 1970 American Diabetes Association meeting early in 1970. The pair of papers ultimately comprising a separate supplemental issue of Diabetes were submitted about the same time to the journal. For a time, it looked as if the strategy was working, but things fell apart in late spring when the manuscripts were returned for revision.
In the end, the paper was published in November, about five months after the presentation in St. Louis. The intervening time meant that University Group Diabetes Program investigators stood helpless until the papers were published. The time gap was problematic. Diabetologists were deluged by calls from patients worried about the drug they were using. The fact that clinicians had to answer patients’ questions without the benefit of a published report made them hostile to the study. By the time the publication finally appeared, they had long since decided that the study was ‘no good’ and that there was no point in reading the published results.
Trust, but verify
Patients had to have a summary blood glucose tolerance test of ≥500 mg/100 mL to be eligible for enrolment. 4 The test consisted of a fasting value, and values at one-, two- and three-hour post-glucose challenge. Glucose determinations were done at the clinic level. There had been discussions about sending specimens to a central laboratory, but that approach was rejected because of logistics and cost. The issue to be settled was whether determinations should be done using blood or serum. After a fair amount of discussion, the issue was decided in favour of blood.
Things proceeded uneventfully until, about three years after the start of enrolment, an investigator made an offhand remark during an investigators’ meeting about the method used to determine glucose levels. Since the method cited was one requiring use of serum, another investigator questioned how the method could be used on whole blood. ‘Whole blood? We use serum’. ‘You do? The protocol specifies whole blood’. ‘It does?’
And so unfolded the ‘glucose story’, with the discovery that four of the twelve clinics were using serum instead of blood. When the smoke settled, the mistake was found to have affected entry determinations for 280 patients.
The mistake required the conversion of serum values to whole blood equivalents. Since serum glucose values are higher than whole blood values, the conversion resulted in 57 of the 280 patients having entry corrected summary glucose tolerance tests below the diagnostic cut point of 500 mg/100 mL.
4
On the meaning of ‘final’
Early on, I laboured producing forms for data collection. It was not my favourite activity, but I endured because I reasoned that it would be time-limited.
I was wrong!
Soon I learned that form changes and revisions of data collection forms are neverending. Often, before the ink was dry on one version there would be calls for revisions and additions. I smiled politely, playing deaf to the call; but I was ultimately outnumbered and overrun. The changes could range from correction of spelling errors to adding new sections to a form.
We can correct that
When a result is published and the world does not like it, people can always come up with some baseline variable that investigators failed to collect and attribute the result to imbalance in the variable, as was done with smoking history. Data were collected on current smoking habits but not on smoking history before enrolment. The Biometrics Committee characterised failure to include smoking history on enrolment as a blunder. (To my ear, an unfortunate characterisation because blunder means doing something stupid or careless.) The identification of smoking as a risk factor changed during the course of the University Group Diabetes Program. The foundation for data collection was laid in 1959, several years before the first report of the Surgeon General’s Advisory Committee on Smoking and Health (11 January 1964) and a year after that before warnings of health risks from smoking were required on cigarette packages.
In fact, the University Group Diabetes Program investigators did try to rectify the omission around 1972 with the collection of retrospective smoking histories. There were no major differences among the treatment groups attributable to smoking history. However, the results were never published because of questions involved in constructing baseline smoking histories long after patients had been enrolled and use of surrogate respondents for patients who had died.
Seek and ye shall find?
Many of the lessons one learns in trials are ‘lessons’ only because of shortsightedness. It should be apparent to anyone involved in long-term trials that one keeps track of everyone, even if they drop out, so that one can classify patients by whether they are alive or dead at the time of analysis. Anyone in charge of such efforts knows that clinic personnel have to keep up-to-date ‘locator’ information if there is to be any hope of tracing people. Even Inspector Clouseau knows that the chance of locating people lost to follow-up diminishes as a direct function of the time since last contact.
The protocol specified that clinics were to maintain ‘up-to-date’ locator information for dropouts, but many clinics ignored that requirement. Hence, when it came time to produce the publication describing the tolbutamide mortality results, nine years after the start of enrolment, vital status was unknown in 23 dropouts in tolbutamide-assigned patients and in 24 placebo-assigned patients.
Clearly, a differential mortality rate among those people could be large enough to explain the observed tolbutamide-placebo mortality difference. Hence, it was obvious that investigators would have to delay publication in order to track dropouts to find out whether they were alive or dead.
Ultimately, through those efforts, investigators were able to assess the vital status of all but five patients: one person assigned to tolbutamide, two persons assigned to placebo, and another two assigned to the insulin-variable treatment group. The hard core unlocatables included a person named Wong, who moved to Chinatown in San Francisco, and was lost among hundreds of other Wongs.
Who said you can vote?
The fateful day came in June 1969 when the Steering Committee was faced with an up or down vote on whether to stop use of tolbutamide. The voting policy (established early on) was two votes per centre – two for each of the twelve clinics and two for the coordinating centre (one vote for centre directors and one vote for deputy directors) – but there was no clear policy on proxy votes, ‘stand-in’ voters in the absence of the director or deputy director, or even the designation of ‘deputy director’. The ambiguities were noted when the policy was drafted, but considered unimportant because voting would be unnecessary in the expectation that major decisions would be by ‘consensus’.
The first vote was close: 13 to stop tolbutamide and 12 to continue. After a show of hands there followed a debate as to who had voting rights, a sort of a precursor to the ‘hanging chad’ problem at the 2000 presidential election in Florida.
What do you mean ‘The visit is missed?’
The patient visit schedule was in three-month intervals over the course of follow-up. Each visit consisted of a general examination and, depending on the quarter, an eye, heart, kidney or peripheral vascular examination. Visits were numbered by quarter, i.e. FU 1 for the third month after enrolment, FU 2 for the sixth month after enrolment, etc.
Well and good, except for what clinics did when patients missed a visit.
Suppose a patient does not show up for the six-month visit, but does for the nine-month visit, i.e. the second follow-up visit for the patient, but the third required visit according to the protocol. Does the clinic do the kidney exam or the heart exam? Some clinics did the kidney exam and labelled the exam FU 3; others did the heart exam and labelled it FU 2. Needless to say, counting visits to produce performance statistics by clinic was impossible without hard and fast rules as to when a visit was counted as missed.
Mortality: the trump outcome
The trial was designed to assess the value of different forms of anti-hyperglycaemic treatments for prevention or amelioration of the late complications of type 2 diabetes. The sample size was derived by pragmatic considerations of money and numbers that could be reasonably recruited. There was only passing mention of mortality in the protocol because investigators did not believe that the trial would be adequately sized to find differences in mortality, if indeed the drugs produced benefit in reduced mortality. This, however, is not to say that mortality was not tracked or that investigators did not look for differences in mortality. Indeed, it is the mortality differential in the tolbutamide-assigned group in contrast to the placebo-assigned group that ultimately led investigators to stop use of tolbutamide and to publish mortality results.
Interestingly and surprisingly, critics suggested investigators had no basis for acting on the mortality differential, since mortality was not specified as an outcome of interest in the study protocol.
Stopping a treatment
The decision to stop tolbutamide raised a series of questions.
How do you unmask a treatment without unmasking other treatments?
When should the tolbutamide treatment be stopped?
What should happen to patients after the stop?
Tolbutamide was administered double-masked. When tolbutamide was stopped, all patients receiving tolbutamide or the matching placebo were given new bottles of medication – all having the same bottle number (number 88) – to be taken on the same schedule as before. Investigators knew the bottles contained placebo but patients did not.
The options were to do it immediately by telephone or letter, or to wait until the next scheduled visits. The former approach was rejected as being unnecessary given the equivocal nature of the findings. Patients were told at their next regular clinic visit following the decision.
Follow-up and regular examinations continued.
What should the cut-off date be for the publication dataset?
What were other patients told about the decision?
The date used was 7 October 1967. That date corresponded roughly to the time required for patients to cycle through their next scheduled follow-up visit after the stop decision and providing adequate time for data harvests by the coordinating centre.
Nothing, but if patients asked, they were told of the decision.
Dealing with brickbats
The investigators plan was ‘mum’s the word’ about the tolbutamide results until they were published. But the plan fell apart when investigators decided to present the results at the American Diabetes Association meeting in June 1970 in anticipation of the results being published by then. They misjudged. The publication came months after the presentation.
The first report of results ran at 2:17 pm on Wednesday 20 May 1970 on the Dow Jones ticker. It was a report from a Kidder Peabody analyst warning investor of results adverse to Upjohn. That the first report was on a financial service wire was no surprise in retrospect, given that sales of tolbutamide accounted for nearly half of all prescriptions for oral hypoglycemic agents at the time. 6
That report was followed in the next few days by articles in major newspapers, including the Wall Street Journal, Washington Post and New York Times7–10 featuring headlines such as: Safety of Upjohn’s oral antidiabetic drug doubted in study: Firm disputes findings (21 May 1970, Wall Street Journal)
9
Antidiabetes pill held causing early death (22 May 1970, Washington Post)
8
Scientists wary of diabetic pill: FDA study indicates oral drug may be ineffective (22 May 1970, New York Times)
10
Discovery of diabetes drug’s perils stirs doubts over short-term tests (8 June 1970, Washington Post)
7
By the time of the meeting it seemed that everyone knew of the results, including patients calling their doctors to find out if they were on that ‘killer diabetes drug’.
The presentation in St. Louis did nothing to quell the criticisms.
We were crucified in the throwaway medical journals and accused of grandstanding, data dredging, malfeasance and fraud. We were to the Medical Tribune and Hospital Tribune what Jackie Kennedy Onassis at the time was to magazines at check-out counters in supermarkets – always on the front page. The box below contains sample headlines.
Investigators question study group’s findings (Monday, 29 June 1970) Experts challenge data, design of investigation (Monday, 6 July 1970) Irish study of antidiabetics contradicts findings in US (Wednesday, 15 December 1971) Why the conclusions of the UGDP are incorrect (Wednesday, 4 June 1975), Biometric Report on UGDP study stirs skepticism (Wednesday, 11 June 1975), A UGDP ‘Miracle’? … Some UGDP questions (Wednesday, 27 August 1975), Doctors’ debate. UGDP computer vs. clinical data (Wednesday, 23 June 1976)
2 Diabetes researchers quit over demand for ‘unanimity’ (Monday, 14 December 1970) Tolbutamide fiasco (Monday, 14 December 1970) ‘Misleading impression’ laid to UGDP report (Monday, 22 February 1971), Danger is seen in hasty action on antidiabetics (Monday, 22 March 1971), Canadian diabetes group rejects UGDP study (Monday, 19 April 1971) 3 Nonpartisan experts doubt worth of UGDP findings (Monday, 25 July 1971), Why the conclusions of the UGDP are incorrect (Monday, 16 June 1975), Europe skeptical of Biometric Study of UGDP (Monday, 14 July 1975)
The inclination was to respond to criticisms in the throwaway press, but it became clear that doing so would sap our energy, so we opted to sit on our hands. A more difficult question was what to do about criticisms published in peer reviewed journals. There were several over the years, starting with Schor
11
and Feinstein,
12
then Seltzer,
13
again Feinstein,14,15 Kilo et al.
16
and others. Most of these we answered. Responses are contained in the University Group Diabetes Program,
17
Schwartz
18
and Schwartz and Meinert,
19
and in Cornfield
20
(see Cornfield;
21
Schlesselman
22
).
Data sharing
The University Group Diabetes Program was into data sharing before it became an expected requirement of trialists. The publication of the tolbutamide results in 1970
23
contained a listing of data relating to deaths reported in the publication. A data listing for all 1027 persons enrolled in the study was available on request as per an announcement in the 1977 December issue of Diabetes.
24
The final publication in 198225 contained 30 pages of data listings for all patients enrolled in the University Group Diabetes Program as of the end of data collection, 31 August 1975. Paper listings and magnetic tapes of baseline and follow-up data for all study patients were deposited at the National Technical Information Service in 1983.26,27
After the fact consent to randomised treatments
Institutional Review Boards did not exist when the University Group Diabetes Program started. There were no consent forms for patients to read and sign. It was left to clinic personnel to decide what to tell patients why they were being approached for study.
In reality, the requirement for informed consent to research on people existed long before the start of the University Group Diabetes Program: it is the first item in a ten-point manifesto that emerged from the Nüremberg war crimes trials (known as the Nüremberg War Code) promulgated in 1947.
28
The voluntary consent of the human subject is absolutely essential. This means that the person involved should have legal capacity to give consent; should be so situated as to be able to exercise free power of choice, without the intervention of any element of force, fraud, deceit, duress, overreaching, or other ulterior form of constraint or coercion; and should have:
sufficient knowledge and comprehension of the elements of the subject matter involved as to enable him to make an understanding and enlightened decision. This latter element requires that before the acceptance of an affirmative decision by the experimental subject there should be made known to him the nature, duration, and purpose of the experiment; the method and means by which it is to be conducted; all inconveniences and hazards reasonably to be expected; and the effects upon his health or person which may possibly come from his participation in the experiment.
Although the requirement for consent existed well before the University Group Diabetes Program was planned, the requirement was largely ignored. The prevailing view in a then paternalistic medical profession was that discussions regarding such issues as randomisation to select the treatments patients were to receive would be anxiety-inducing and, hence, to be avoided.
That changed in the mid-1960s with accounts of a few ‘celebrated’ studies involving people in research without their consent. Among them, one involved infecting ‘mentally defective’ children in the Willowbrook State Hospital in New York with hepatitis and another involved the injection of live cancer cells into patients in the Jewish Chronic Disease Hospital in New York City. A publication by Beecher in the New England Journal of Medicine in 1966 32 focused attention on the issue of ethics in clinical research. The outrage led the Surgeon General of the U.S. Public Health Service to announce on 8 February 1966, that henceforth, in order to receive funding, National Institutes of Health grantees would have to provide evidence of procedures and practices designed to ensure documented informed consent. The order and its implementation eventually led to the creation of institutional review boards.
The problem for University Group Diabetes Program investigators was that the order came about when enrolment was complete. Memory no longer serves as to what investigators did to comply with the order, but whatever they did there is no evidence of widespread departures from the study based on seeking consents.
The label changes
The tolbutamide-placebo difference in cardiovascular mortality was striking. The conventional p-value for the difference was 0.005 when tolbutamide was stopped. But even with that, it is likely the results would have faded into obscurity had it not been for the efforts of the Food and Drug Administration to relabel the drug warning of cardiovascular risks associated with its use.
The opening salvo from the Food and Drug Administration was telegraphed in a Bulletin issued from the Food and Drug Administration by the Commissioner, Charles Edwards, and included as front matter in the Diabetes supplement containing the tolbutamide results.
The proposed relabeling had medical-legal implications in that it opened the door to legal action if persons on the drug experienced heart attacks. The concern regarding relabeling was a driving force behind the creation of the Committee on the Care of the Diabetic.
Efforts of the Committee on the Care of the Diabetic focused on forestalling the relabeling. The Committee on the Care of the Diabetic began its efforts in a request to the Commissioner of the Food and Drug Administration (7 October 1971) to delay the relabeling. A delay was granted on 7 November 1972 (after the label had already been printed and supplied to manufacturers).
Thirteen years after relabeling had been proposed it was eventually accomplished. The warning is reproduced below.
UGDP reported that patients treated for 5 to 8 years with diet plus a fixed dose of tolbutamide (1.5 g per day) had a rate of cardiovascular mortality approximately 2½ times that of patients treated with diet alone. A significant increase in total mortality was not observed, but the use of tolbutamide was discontinued based on the increase in cardiovascular mortality, thus limiting the opportunity for the study to show an increase in overall mortality. Despite controversy regarding the interpretation of these results, the findings of the UGDP study provide an adequate basis for this warning. The patient should be informed of the potential risks and advantages of tolbutamide and of alternative modes of therapy.
Although only one drug in the sulfonylurea class (tolbutamide) was included in this study, it is prudent from a safety standpoint to consider that this warning may also apply to other oral hypoglycemic drugs in this class, in view of their close similarities in mode of action and chemical structure. (Physicians’ Desk Reference (PDR); 39th edition; 1985; p. 2,130)
Reflections
The third aim of the University Group Diabetes Program was ‘development of methods applicable to cooperative clinical trials’.
It was a given that someone in the trial had to monitor results for quality control and for treatment differences. It was clear that responsibility for monitoring fell to the coordinating centre, but not who should see interim treatment results. Ultimately it was decided that the entire steering committee (comprising the director and deputy director of each of the 12 clinics and the director and deputy director of the coordinating centre) should see them.
The practice of monitoring and reporting to the steering committee in relation to its semi-annual meetings was well established when the mortality trend against the tolbutamide treatment group began to emerge. At first the trend was a matter of curiosity, but it came to be a focus of concern in 1968.
The fact that monitoring in the University Group Diabetes Program was done by the steering committee raised concerns of bias and conflicts of interest. Tom Chalmers, Associate Director of the National Institutes of Health and Director of the National Institutes of Health Clinical Center during the tolbutamide decision, was critical of the fact that investigators involved in the trial monitored results to decide if treatments should continue. He regarded investigator involvement as constituting a conflict of interest. 33
The issue raised ultimately (1998) led the National Institutes of Health to require monitoring bodies for all multicentre trials it funds. It is the policy of the NIH that each Institute and Center (IC) should have a system for the appropriate oversight and monitoring of the conduct of clinical trials to ensure the safety of participants and the validity and integrity of the data for all NIH-supported or conducted clinical trials. The establishment of the data safety monitoring boards (DSMBs) is required for multi-site clinical trials involving interventions that entail potential risk to the participants. The data and safety monitoring functions and oversight of such activities are distinct from the requirement for study review and approval by an Institutional Review Board (IRB). (10 June 1998)
34
The trend is an unfortunate legacy of the University Group Diabetes Program because isolation of monitors from study investigators reduces the competency of the monitors to the extent that investigators, who collect the data, know the protocol and quicksand traps in the data better than their external counterparts.
Even worse is the tendency to mask the monitors to treatment assignment, again because of desire to avoid bias that may creep in if monitors know treatment assignment. That practice is what led me to write, ‘Masked monitoring, blind stupidity?’ 35
John Hampton,
36
in his paper ‘Therapeutic fashion and publication bias: The case of anti-arrhythmic drugs in heart attack’, notes:
Disturbances of heart rhythm (arrhythmias) are common during and soon after heart attacks (myocardial infarctions), and these arrhythmias often precede and lead to early death. In the 1970s, it was found that the local anaesthetic drug lignocaine (lidocaine) suppressed arrhythmias, and it seemed obvious that giving anti-arrhythmic drugs would reduce the risk of early death after heart attack. The problem was that this obvious theory was wrong, but this was difficult to recognise from small clinical trials looking only at effects on arrhythmias, not outcomes that really matter, like deaths.
Similarly, for treatments used to control blood sugar in people with diabetes, it seems obvious that such control will confer benefit in reducing the risk of morbidity and mortality associated with the condition. Obvious – until the University Group Diabetes Program.
The reality is that most trials are too small and short-lived to produce results bearing on long-term safety and efficacy. The median sample size for National Institutes of Health-funded phase 3 and 4 trials is only 306 for phase 3 trials and 108 for phase 4 trials. 37
Testing diabetes drugs is akin to ‘whac-a-mole’ at fairs. As soon as you knock a mole down, another one pops up.
The profit-producing life of patented drugs is largely synonymous with the period of patent protection afforded drugs, typically 20 years from when the patent was issued. The period of protected marketing may be half that after subtracting time needed to bring a drug to market.
After patent protection expires, other manufacturers are free to market generic brands of the drug. This reality means that drug companies have to recover their development and marketing costs and make a profit before patents expire.
Tolbutamide (Orinase®) used in the University Group Diabetes Program has since been replaced by other members of the family of sulfonylurea compounds. The Food and Drug Administration label warning fashioned after the University Group Diabetes Program applies to all members of the family but there is no way of knowing if the same risks apply to other members of the family without additional trials.
The Centers for Disease Control and Prevention estimates that of the 21 million people with diabetes in the United States in 2010 being on treatment, over half were on oral drugs alone (14% on insulin alone). In 1990, there were 23.4 million such prescriptions and 91.8 million in 2001. 38 Glipizide and glyburide, sulfonylurea compounds, accounted for 77% of all prescriptions in 1990 and 33.5% in 2001.
Alexander et al., 39 in a 2008 publication on national trends in treatment of type 2 diabetes mellitus from 1994 through 2007, report a decrease in sulfonylurea use (decreased from 67% of treatment visits in 1994 to 34% in 2007). By 2007, biguanides (54% of treatment visits) and glitazones (thiazolidinediones) (28%) were leading therapeutic classes.
Perhaps an unsung contribution of the University Group Diabetes Program, because of the controversy it created, is what it did to stimulate the development of other long-term diabetes trials.
The Diabetes Complications Control Trial, started in 1983 and published in 1993, involved 1441 people, but is only of marginal relevance to the question addressed by the University Group Diabetes Program. Diabetes Complications Control Trial investigators excluded type 2 diabetics and the trial involved only insulin treatments (tight control via insulin pumps or by three or more injections of insulin based on frequent blood monitoring versus conventional insulin treatment involving one or two insulin injections per day). The investigators concluded that:
Intensive therapy effectively delays the onset and slows the progression of diabetic retinopathy, nephropathy, and neuropathy in patients with IDDM.
40
Intensive blood-glucose control by either sulphonylureas or insulin substantially decreases the risk of microvascular complications, but not macrovascular disease, in patients with type 2 diabetes. None of the individual drugs had an adverse effect on cardiovascular outcomes. All intensive treatment increased the risk of hypoglycaemia.
41
Last word
The University Group Diabetes Program, being my first exposure to trials, was an unparalleled learning and enriching experience. I am thankful to Chris Klimt for hiring me on for what was to become a life as a trialist. Thank you, Chris!