Abstract
Background:
Adjudicated cause-specific mortality has been used in major trials of chronic obstructive pulmonary disease. However, there is less experience with adjudicated major adverse cardiovascular events as a key efficacy outcome in chronic obstructive pulmonary disease trials. The Study to Understand Mortality and Morbidity in chronic obstructive pulmonary disease trial required a Clinical Endpoint Committee to adjudicate the outcomes of modified major adverse cardiovascular events and cause-specific mortality.
Methods and results:
A six-member Clinical Endpoint Committee reviewed adverse event and serious adverse event reports included in a list of 204 Medical Dictionary for Regulatory Activities terms. Adverse events were triaged by one Clinical Endpoint Committee member, and then reviewed by three reviewers (round 1). If these three disagreed on the adjudication, the event was discussed by the full committee to reach a consensus (round 2). Among 16,485 participants, 48,105 adverse events were reported, among which 3314 were reviewed by the Clinical Endpoint Committee. After triage, 1827 were adjudicated in round 1; 338 required committee consensus in round 2, yielding 450 myocardial infarctions, strokes, unstable anginas or transient ischaemic attacks. Only 20/1627 (1%) non-serious adverse events were adjudicated as cardiovascular events. Only 45/204 Medical Dictionary for Regulatory Activities terms reviewed yielded cardiovascular events. A total of 430 deaths were adjudicated in round 1 and 631 in round 2, yielding 459 cardiovascular deaths. Adjudication of chest pain and sudden death often required additional information from site investigators. Site assessment of cardiovascular death was moderately specific (501/602 = 83%) but not sensitive (256/459 = 56%).
Conclusion:
A Clinical Endpoint Committee is useful for adjudication of major adverse cardiovascular events in chronic obstructive pulmonary disease trials but requires considerable resources and effort by investigators. This process can be streamlined by reviewing only serious adverse events and filtering by selected Medical Dictionary for Regulatory Activities terms.
Keywords
Introduction
Cardiovascular co-morbidities are common in chronic obstructive pulmonary disease (COPD) and often lead to major adverse cardiovascular events. Cause-specific mortality has become a routine adjudicated outcome in large COPD clinical trials, and the methods and experience of the adjudication process have been reported previously.1–3 Although major adverse cardiovascular events are a common outcome measure in cardiovascular clinical trials, it has not been widely used as an adjudicated efficacy or safety outcome for COPD trials. The Study to Understand Mortality and Morbidity in COPD (SUMMIT) was a large COPD clinical trial that tested the secondary hypothesis that treatment of mild to moderate COPD with fluticasone furoate/vilanterol combination, in patients also with increased cardiovascular risk factors, would reduce the risk of major adverse cardiovascular events.4,5 To this end, SUMMIT required the development of a process and procedures for collecting and adjudicating cause-specific mortality and major adverse cardiovascular events. For SUMMIT, the definition of major adverse cardiovascular events included transient ischaemic attacks and unstable angina as well as myocardial infarction, stroke and cardiovascular death. To accomplish this task, a Clinical Endpoint Committee was established and principles of operation, based on published guidelines for cardiovascular trials, were adapted for review of SUMMIT events. From the extensive experience accrued from this study, we learned lessons that may be employed in future COPD trials in which major adverse cardiovascular events are an efficacy or a safety outcome. The purpose of this report is to provide post hoc details of the process, the efficiency of major adverse cardiovascular event ascertainment methods and the extent of agreement between adjudicated events and site investigator reports, and to provide recommendations for future adjudication committees.
Methods
SUMMIT study design
The SUMMIT study design and primary outcomes have been previously reported.4,5 Briefly, the trial was a randomised, double blind, parallel group event-driven trial comparing fluticasone furoate/vilanterol and placebo, inhaled once daily. Enrolled participants had moderate COPD (post-bronchodilator forced expiratory volume in 1 s 50%–70% predicted) and increased risk for cardiovascular disease. Each participant was followed from enrolment to at least the common end date, at which there were projected to be 1000 deaths. The primary outcome was all-cause mortality up to the common end date; however, all deaths reported before the database was locked were adjudicated. The secondary outcomes were rate of decline in forced expiratory volume in 1 s and a cardiovascular composite endpoint (major adverse cardiovascular events) comprising on-treatment myocardial infarction, stroke, transient ischaemic attack, unstable angina and cardiovascular death. All patients provided written informed consent. The study was approved by local ethics committees and was conducted in accordance with the Declaration of Helsinki and Good Clinical Practice guidelines.
Acquisition of medical information
When a participant died or reported any adverse event to the local site, the adjudication process was initiated. For each death, the site was asked to provide as much information as possible to facilitate the adjudication of the primary cause of death including death certificate, hospital correspondence, results of clinical investigations, procedure reports, witness interviews and autopsy results where available. When a site reported an adverse event into the electronic case report form, the verbatim term was coded by a central automatic coding procedure to a Medical Dictionary for Regulatory Activities (version 18) preferred term. If this Medical Dictionary for Regulatory Activities term matched a list of preferred terms of pre-defined events, then the adverse event was sent for adjudication for a cardiovascular event. These pre-defined events were chosen by a physician reviewing all Medical Dictionary for Regulatory Activities preferred terms prior to the adjudication process starting. For these events, sites were also requested to provide all available medical records to support the determination of causation of death and hence whether this was a component of major adverse cardiovascular events.
Operation of the Clinical Endpoint Committee
The Clinical Endpoint Committee is comprised of six physicians, two each from the following specialties: Pulmonology, Cardiology and Neurology. The Clinical Endpoint Committee members were not site investigators in the study. The committee members were selected by the sponsor and approved by the SUMMIT steering committee. The Clinical Endpoint Committee operated under a charter written by the sponsor and created a principle of operation document that was written by the Clinical Endpoint Committee members and was updated throughout the study to codify guidelines for adjudicating the cardiovascular events (see Online Supplementary Appendix A).
Data management
The virtual clinical adjudication system was a web-based application for central management of information retrieval and adjudication activities developed for SUMMIT (Parexel, Waltham, MA, USA). When an event occurred for a subject, relevant clinical documents were collected in the virtual clinical adjudication system to create an electronic dossier. Reviewers recorded their assessments in the virtual clinical adjudication system and if discordance was found, the case was automatically distributed for consensus review.
Adjudication of cause-specific death and major adverse cardiovascular events
The process for adjudicating the primary cause of death and assessing whether or not it was related to COPD was similar to that followed in recent large respiratory trials.1–3 In brief, the adjudicated cause of death was classified based on the underlying cause of death defined as the presenting illness that preceded the terminal events, not the terminal events just preceding death. Adjudication of major adverse cardiovascular events generally followed the Food and Drug Administration guidance for cardiovascular outcomes. 6
Sudden death is a term generally denoting a presumed arrhythmic death when the death is witnessed (i.e. the person is found dead within 1 h of being seen alive), and another cause could not be identified. In SUMMIT, it was included as a subcategory of cardiovascular deaths. If a death was unwitnessed and occurred within 1–24 h of the patient last being seen alive without evidence of clinical deterioration and no other cause of death was ascertained, it was also categorised as a sudden death as well as a cardiovascular death. If the interval between death and last being observed alive was greater than 24 h, and there was no other cause of death, the death was classified as unknown. This information was systematically obtained by the sites for deaths occurring outside of medical facilities by a standardised interview with a family member or witness.
Myocardial infarctions were adjudicated using published criteria using cardiac enzymes, electrocardiograph changes, imaging or pathologic evidence. 7 Stroke was diagnosed based on compatible neurologic symptoms supported by brain imaging or onset of a typical acute neurologic deficit. Unstable angina was diagnosed based on the need for unscheduled medical care associated with compatible electrocardiograph or imaging evidence of coronary stenosis or need for a revascularisation procedure. Transient ischaemic attack was defined as a witnessed, transient compatible neurologic deficit lasting less than 24 h in the absence of imaging evidence of a stroke. Amaurosis fugax, although representing ocular rather than brain ischaemia, was also classified as a transient ischaemic attack.
Adjudication procedures
Potential major adverse cardiovascular events were adjudicated in three stages: a triage round by one committee member, round 1 adjudication by three committee members and round 2 by the entire committee of six. All non-serious adverse events that matched the selected list of Medical Dictionary for Regulatory Activities preferred terms were assigned to a single member of the committee in the triage round. The committee member reviewed the available documentation and determined whether the event warranted further adjudication. If yes, the event was escalated to round 1. All deaths and serious adverse events were automatically assigned to round 1. In round 1, each event and all the accompanying information was sent to three Clinical Endpoint Committee members, one of each specialty. If all three members adjudicated the event in an identical fashion, the event was considered adjudicated. If there was disagreement on any field of the adjudication form, then the event was escalated to round 2, which was a face-to-face meeting or Internet conference with all six members discussing the event and associated source material to reach consensus.
Results
Overall, among the 16,485 participants, SUMMIT reported 48,105 on-treatment adverse events (includes fatal and non-fatal), of which 3314 (7%) matched the list of pre-defined Medical Dictionary for Regulatory Activities preferred terms. These were subsequently reviewed by the Clinical Endpoint Committee. Of these, 1687 were serious adverse events and 1627 were non-serious adverse events. 450 adverse events were adjudicated as myocardial infarctions, strokes, transient ischaemic attacks or unstable angina, of which 387 were non-fatal. Only 20 (1%) of the 1627 non-serious adverse events were finally adjudicated as cardiovascular events, whereas 430 (25%) of the 1687 serious adverse events were finally adjudicated as cardiovascular events. Of these 450 adjudicated events, 43% were myocardial infarctions, 30% were strokes, 19% were unstable angina and 8% were transient ischaemic attacks (Figure 1(a)). Out of 70 transient ischaemic attacks preferred terms, 30 (43%) were adjudicated as cardiovascular events.
Adjudication flow chart for (a) adverse events and (b) deaths. The 450 events in (a) refer to the four main components of the pre-defined major adverse cardiovascular events (MACEs): fatal or non-fatal myocardial infarction, fatal or non-fatal stroke, unstable angina (all non-fatal) and transient ischaemic attack (all non-fatal).
Of the 1061 deaths in the study, 1037 (98%) were in the intent-to-treat population and occurred on or before the common end date so were included in the SUMMIT primary analysis. 5 Of the 1061 total deaths, 4% were myocardial infarctions, 4% were strokes, 30% were sudden deaths, 1% were procedural deaths and 5% were of other cardiovascular causes (e.g. aortic aneurysm), giving a total of 459 (43%) classified as cardiovascular death. The other causes of death included cancer (23%), pulmonary (13%), other causes (8%) and unknown cause (12%) (Figure 1(b)).
Because the secondary outcome measure of interest was time to first event, participants who had two events had only the first event included in the primary analysis, consisting of 688 first on-treatment cardiovascular events (359 non-fatal and 329 fatal, Table 1).
Adjudicated first cardiovascular composite events (secondary endpoint). a
SUMMIT secondary endpoint was time to first on-treatment cardiovascular composite event (which includes on-treatment cardiovascular death). Patients may have experienced multiple cardiovascular events, but only the first was used in the analysis.
To capture all events that could potentially be a major adverse cardiovascular event, the committee reviewed events described by 204 Medical Dictionary for Regulatory Activities terms that occurred in the study. However, only 45 of the 204 Medical Dictionary for Regulatory Activities terms that were used to filter events for adjudication finally yielded an adjudicated major adverse cardiovascular event (Online Supplementary Appendix B, Figure S1 and Table S1). Some terms were very specific. For example, 82/97 (85%) events with the preferred term ‘acute myocardial infarction’ and 73/115 (63%) with the preferred term ‘myocardial infarction’ were adjudicated as major adverse cardiovascular events. In contrast, only 8/352 (2%) of events with the term ‘chest pain’ were finally adjudicated as a major adverse cardiovascular event.
Agreement among adjudicators
In general, the Clinical Endpoint Committee had good agreement and was able to adjudicate 317 (70%) of the non-fatal major adverse cardiovascular events with all three adjudicators agreeing on all elements of the adjudication in round 1. Of the 450 non-fatal major adverse cardiovascular events, 133 (30%) required promotion to the full six-member committee where a consensus was reached in all cases.
Overall, there was good agreement between individual adjudicators and the final committee consensus with respect to the primary class of death (e.g. cardiovascular, pulmonary and cancer). Among the six adjudicators, a total of 3878 records were reviewed initially and 3286 (85%) were ultimately consistent with the committee’s final adjudication. The percent agreement of individual adjudicators with the final cause of death category ranged from 75% to 91% (Online Supplementary Appendix B, Table S2).
Agreement with site investigators
As noted in prior COPD trials, the agreement between the Clinical Endpoint Committee and the site investigator for underlying cause of death varied by category. In general, there was good agreement for cancer-related deaths, the Clinical Endpoint Committee agreeing with the investigator in 192/194 (99%) of cases whereas the committee agreed with the site investigator in only 256/357 (72%) of cases initially classified by the site investigator as a cardiovascular death (Table 2). Overall, the site investigator classifications had a specificity of 83% (501/602) and a sensitivity of 56% (256/459) with regard to cardiovascular deaths.
Agreement (diagonal) of cause of death between Primary Investigator and Clinical Endpoint Committee. a
Percentages on diagonal show agreement by Clinical Endpoint Committee of Primary Investigator cause.
Discussion
In this article, we present the operations and outcomes of the SUMMIT Clinical Endpoint Committee to adjudicate major adverse cardiovascular events in patients with mild to moderate COPD and at risk for cardiac disease. SUMMIT was the first major COPD clinical trial to incorporate major adverse cardiovascular events as a key efficacy outcome measure, rather than evaluating this in the context of safety. 8 Because cardiovascular events are common in patients with COPD, and these events are often preceded by COPD exacerbations, it seems likely that future treatment trials targeting COPD exacerbations will place greater scrutiny on these as key clinical outcomes; 9 thus, we report here our experience in order to report lessons learned and provide recommendations for future studies.
In organising the Clinical Endpoint Committee for a trial the size of SUMMIT, it was necessary to construct a large infrastructure to collect, organise, translate and distribute the pertinent case report forms and clinical source material and documents that were reviewed by the committee, and to triage reports using a staged approach. Ultimately, the committee was able to reach a consensus in every case, although sometimes this required tabling an event for review at a second meeting. The initial in-person meetings, and discussion of example cases to establish rules for evaluation of cases, along with written principles of operation to codify these criteria were helpful for establishing the norms for the committee. It was also helpful to have specialised expertise in neurology, cardiology, pulmonary and critical care for evaluation of difficult cases.
We learned from this experience that the effort involved in reviewing non-serious reports of events yielded a few major adverse cardiovascular events, even after filtering for cardiovascular and related terms, with only 1.2% of these reviews ultimately yielding an outcome event, accounting for only 4% of the final total major adverse cardiovascular events. Therefore, in circumstances where resources are particularly constrained, it may be justified to review only serious adverse events, that is, those that lead to hospitalisation, death or are considered life-threatening.
Even among serious adverse events where medical records were available, some diagnostic terms were challenging for the committee. For example, the diagnosis of unstable angina presented a challenge to the committee requiring a second round of review in 39% of cases. We found that the hospital diagnosis of unstable angina was often used for patients with stable angina who were admitted to the hospital for elective procedures such as coronary arteriography. In line with the Food and Drug Administration draft guidance, the committee required three elements to adjudicate unstable angina: (1) a pattern of symptoms compatible with unstable angina, (2) an urgent, unscheduled admission to a healthcare facility, and (3) anatomic evidence of coronary artery disease or performance of a coronary intervention. With this definition, of 88 events with a severe adverse event coded as unstable angina, six (7%) were adjudicated as a myocardial infarction, and only 32 (36%) were adjudicated as unstable angina. Thus, a revascularisation procedure per se did not constitute a major adverse cardiovascular event unless it was in the context of an acute worsening of symptoms.
Transient ischaemic attacks were also challenging to adjudicate because of the requirement for definite evidence of neurologic symptoms and absence of findings of stroke. Among 45 severe adverse events that were reported as transient ischaemic attacks, only 24 (53%) were finally adjudicated as transient ischaemic attacks. We also note that congestive heart failure events were reported only 187 times and that only 2.7% were adjudicated as major adverse cardiovascular events. We speculate that this low yield may be due to the difficulty in distinguishing acute left ventricular failure from an exacerbation of COPD with cor pulmonale, emphasising the need for consistent adjudication of both cardiovascular events and COPD exacerbations.
In contrast, severe adverse events with the diagnosis of ‘acute myocardial infarction’ were more accurate – adjudicated as a myocardial infarction 81% of the time (78/96 events). If the severe adverse event Medical Dictionary for Regulatory Activities term was simply ‘myocardial infarction’, then only 73/114 (64%) events adjudicated as a myocardial infarction. Based on this experience, caution is warranted in the use of severe adverse events as reliable surrogates for major adverse cardiovascular events in clinical trials.
The committee also classified all myocardial infarctions as either type 1 (coronary artery obstruction) or type 2 (oxygen-demand ischaemia). COPD exacerbations increase the risk of a myocardial infarction about two- to fourfold after a COPD exacerbation.8,10 It has been hypothesised that these events are brought on by increased myocardial oxygen demands from the combination of hypoxemia and beta-agonist use raising the hypothesis that there would be an excess of type 2 myocardial infarctions in a COPD population. This was not found to be the case. In observational series of acute myocardial infarction, only 10%–14% are classified as type 2 events.11,12 This is comparable to the experience in SUMMIT where, of 173 myocardial infarctions, 20 (11.5%) were adjudicated as type 2 events. Thus, it seems plausible that other factors, such as increased platelet aggregation, are related to the risk of myocardial infarction and stroke immediately following a COPD exacerbation. 13
Early in the operation of the committee, we found that the adverse event of ‘chest pain’ or ‘chest discomfort’ was a frequent occurrence and that medical records were sparse. To better evaluate these reports, we provided site investigators with a specific questionnaire to assess whether this was likely due to ischaemic heart disease. Among 389 adverse events that were not considered serious, only three (0.7%) were adjudicated as a major adverse cardiovascular event (two myocardial infarction and one unstable angina). The yield was higher from the term ‘chest pain’ when it was associated with a serious adverse event. Among 53 such serious adverse events, six (11%) were adjudicated as major adverse cardiovascular events (four myocardial infarction and two unstable angina).
Assessment of sudden or unwitnessed death is a particularly difficult problem in COPD populations since some of these deaths may be the result of respiratory events as well as cardiovascular death. 5 In line with previous adjudication committees,1–3,14 we used arbitrary definitions of sudden death in the absence of a presenting acute illness and relied upon interviews with family or caretakers to help in this process. Among 320 deaths finally adjudicated as sudden death, only 189 (59%) were reported as such by site investigators. When the death was unwitnessed, but the participant was found dead within 24 h of being in usual health (often a nocturnal event), the committee had to review the case as a whole in round 2 in 111/156 cases (71%) which was the most frequent death event requiring full committee consensus with only 45/156 (29%) reaching agreement at round 1. When the death was witnessed within 1 h, there was greater consensus with 82/164 (50%) of these events confirmed on round 1, and 82/164 (50%) requiring discussion of the full committee.
In summary, we report here the procedures and experience of the SUMMIT Clinical Endpoint Committee with respect to cause-specific mortality and major adverse cardiovascular events. A major lesson that we have derived from this experience is that an extensive infrastructure and defined procedures are necessary to accomplish these adjudications. We also observed that only 22% of preferred Medical Dictionary for Regulatory Activities terms yielded adjudicated events and that adverse events that were not considered to be serious events rarely yielded major adverse cardiovascular events. Thus, in future studies, considerable efficiency could be achieved by tighter filtering of terms reviewed by the committee or even eliminating review of events not deemed serious by the site investigator. Although all-cause mortality is a more robust outcome measure in clinical trials than cause-specific mortality, assessment of cardiovascular death is an important component of major adverse cardiovascular events where cardiovascular safety or efficacy outcomes are critical. In this case, we believe that it is essential to rely on an adjudication committee for this outcome rather than site investigator reports. Considering the Clinical Endpoint Committee adjudication of cardiovascular deaths as the reference standard, site investigator classification was not particularly reliable with a specificity of 83% (501/602) and a sensitivity of only 56% (256/459). This contrasts with cancer-specific mortality where site investigators had a high degree of reliability.
Supplemental Material
Supplementary_File – Supplemental material for Adjudication of cardiovascular events in patients with chronic obstructive pulmonary disease: SUMMIT trial
Supplemental material, Supplementary_File for Adjudication of cardiovascular events in patients with chronic obstructive pulmonary disease: SUMMIT trial by Robert A Wise, Julie A Anderson, Pierre Amarenco, Nicholas J Cowans, Courtney Crim, Martin A Denvir, Camilo R Gomez, Matthew PA Jones, Andrea Morris, Dennis Niewoehner and Julie C Yates in Clinical Trials
Footnotes
Acknowledgements
The authors thank the SUMMIT Steering Committee, composed of academic experts and employees from GlaxoSmithKline plc., who were responsible for the scientific oversight of the trial, the study design and analysis, and for the review and interpretation of the data.
Declaration of conflicting interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship and/or publication of this article: R.A.W., P.A., M.A.D., C.R.G. and D.N. are members of the SUMMIT Clinical Endpoint Committee. J.A.A., C.C. and J.C.Y. are members of the SUMMIT Steering Committee. J.A.A., C.C., A.M. and J.C.Y. are employed by the GSK. N.J.C. and M.P.A.J. are employees of Veramed Limited – a contract research organisation that receives funding from the GlaxoSmithKline plc.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This work was supported by the GlaxoSmithKline plc. (HZC113782).
Members of the SUMMIT Clinical Endpoint Committee
Robert A. Wise (chair, USA), Dennis Niewoehner (USA), Camilo R. Gomez (USA), Sheldon Magder (Canada), Martin A. Denvir (UK) and Pierre Amarenco (France).
Members of the SUMMIT Steering Committee
Jørgen Vestbo (co-chair, UK), Robert D. Brook (USA), Peter M.A. Calverley (UK), Bartolome R. Celli (USA), Fernando J. Martinez (USA), David E. Newby (UK), Courtney Crim (co-chair, GlaxoSmithKline plc., USA), Julie A. Anderson (GlaxoSmithKline plc., UK) and Julie C. Yates (GlaxoSmithKline plc., USA).
Trial registration
Data accessibility
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
