Abstract
Summary
Objectives
To explore the relationship between in-hospital mortality following adult cardiac surgery and the time since primary clinical qualification for the responsible consultant cardiac surgeon (a proxy for experience).
Design
Retrospective analysis of prospectively collected national registry data over a 10-year period using mixed-effects multiple logistic regression modelling. Surgeon experience was defined as the time between the date of surgery and award of primary clinical qualification.
Setting
UK National Health Service hospitals performing cardiac surgery between January 2003 and December 2012.
Participants
All patients undergoing coronary artery bypass grafts and/or valve surgery under the care of a consultant cardiac surgeon.
Main outcome measures
All-cause in-hospital mortality.
Results
A total of 292,973 operations performed by 273 consultant surgeons (with lengths of service from 11.2 to 42.0 years) were included. Crude mortality increased approximately linearly until 33 years service, before decreasing. After adjusting for case-mix and year of surgery, there remained a statistically significant (
Conclusions
Consultant cardiac surgeons take on increasingly complex surgery as they gain experience. With this progression, the incidence of adverse outcomes is expected to increase, as is demonstrated in this study. After adjusting for case-mix using the EuroSCORE, we observed an increased risk of mortality in patients operated on by longer serving surgeons. This finding may reflect under-adjustment for risk, unmeasured confounding or a real association. Further research into outcomes over the time course of surgeon's careers is required.
Keywords
Introduction
Surgeons throughout the world are highly trained. 1 Surgeons currently undergoing training in the UK first undertake a 4–6 year Bachelor's Degree in Medicine and Surgery, followed by 2 years of foundation training, 2–3 years of core training, and more than 6 years of speciality training. Similar training times exist elsewhere. 2 Postgraduate research or fellowships may extend this training.
The association between surgical performance and cardiothoracic surgeon ‘experience’ has been investigated previously in trainee surgeons.3,4 The case-mix and complexity of surgery performed by newly appointed consultants is usually less complex than that of established consultants. However, as experience develops, a consultant will be assigned more complex caseloads, which inherently carry increased risk. 5
Most studies define surgeon experience according to case-volume for a fixed cross-sectional study period.6–8 The received wisdom is that high volume surgeons, taken as a proxy for ‘experienced’ surgeons, will have better outcomes after adjustment for clinical and baseline risk factors. This assumption has multiple flaws, including that it fails to account for any well-characterised relation between experience and outcome. Also, case-volume may depend on the career point of the surgeon; for example, a highly experienced surgeon close to retirement may reduce their caseload progressively. It is also possible that ‘performance’ may change over time.
In this paper, we quantify ‘experience’ of each surgeon as the time elapsed since the award of the first clinical degree from university. The aim of this research is to estimate the association of surgeon experience with in-hospital mortality in cardiac surgery, conditioning on an established risk score composed of known patient and operative factors to account for operative case-mix.
Materials and methods
Cardiac surgery registry data
Prospectively collected data by the Society for Cardiothoracic Surgery in Great Britain and Ireland (SCTS) were extracted from the National Adult Cardiac Surgery Audit (NACSA) registry on 20 November 2013 for all adult cardiac surgery procedures performed in the UK between 1 January 2003 and 31 December 2012. As described elsewhere, reproducible cleaning algorithms were applied to the data extract 9 : duplicate records and non-adult cardiac surgery entries were removed; transcriptional discrepancies were harmonised; clinical and temporal conflicts, and extreme values corrected or removed. If the recorded consultant was listed by name or initials, their General Medical Council (GMC) registration number was determined by means of direct communication with base hospitals and the GMC web portal (http://www.gmc-uk.org/doctors/register/LRMP.asp). In a handful of records, multiple GMC registration numbers were recorded, in which case we used the first one only. If a transcriptional error was identified (e.g. missing or extra digit, transposed numbers or partially incorrectly entered numbers), we manually corrected the GMC registration number by inspection of similar ones for the corresponding hospital. Summaries of the preprocessed data are regularly returned to each unit for local validation as part of the NACSA in the UK. Definitions of database variables used for the study are available at http://www.ucl.ac.uk/nicor/audits/adultcardiacsurgery/datasets.
Records were excluded if: (1) they were performed in a private hospital, due to incomplete case attainment; (2) the operation involved surgery other than coronary artery bypass grafting (CABG) or valve surgery; (3) they corresponded to a re-do cardiac operation within the same admission spell (in which case only the record corresponding to the primary operation was retained for analysis); (4) the outcome variable was missing or (5) the named responsible consultant surgeon GMC registration number was missing. In addition, records were excluded if they corresponded to a surgeon with (6) a mean case volume of <25 records per annum for contributing years or (7) the number of contributing study years was <3. The final two exclusion criteria were intended to allow a sufficient number of cases and length of time to accurately measure experience on for each individual surgeon.
GMC data
The entire List of Registered Medical Practitioners (LRMP) was downloaded on 27 November 2013. A list of all GMC registration numbers in the NACSA registry meeting the initial inclusion and exclusion criteria was used to extract a subset of the LRMP. Data collected included GMC registration number; primary medical qualification and year of qualification. Since only the year of qualification is recorded, for purposes of calculations, we set the date of award to be 15 July in the reported year.
Record linkage
A compulsory field in the NACSA registry is the ‘responsible consultant cardiothoracic surgeon’, which is recorded as the GMC registration number. Each record in the NACSA registry was deterministically linked by GMC registration number to a record in the LRMP subset. Following linkage, the time since qualification, denoted time since primary medical qualification (TSQ) from here onwards, is calculated for each record as the number of days between the operation and date of medical qualification for responsible consultant surgeon, divided by 365.25. All records corresponding to a consultant were excluded if: (1) there are <11 years between the date of award of primary medical qualification and first record operation in the NACSA registry; or (2) the surgeon graduated before 1970.
Outcome
The outcome for this study was in-hospital mortality. This was defined as death due to any cause during the admission to the hospital at which the cardiac surgery was performed. Missing and conflicting data for in-hospital mortality status were back-filled and validated via record linkage to the Office for National Statistics (ONS) census database, which records details of all deaths in England and Wales. Any remaining records with missing in-hospital mortality status were excluded.
Statistical analysis
Case-mix is known to vary between surgeons.
10
Therefore, for each record we calculated the logistic European System for Cardiac Operative Risk Evaluation (EuroSCORE), which yields a single score for each configuration of patient demographics, co-morbidities, cardiac status and operative details.
11
Due to the low number of missing data, if a variable was missing, we imputed each categorical variable with the baseline value and imputed age with the median value prior to calculation. Variables that are used to calculate the logistic EuroSCORE, as well as the actual EuroSCORE, are individually averaged and compared between three approximately equal TSQ width groups: 11 < TSQ ≤ 20, 20 < TSQ ≤ 30, 30 < TSQ ≤ 42. For each group, we calculate each surgeon's mean annual (per year of TSQ) volume as the total number of cases in the group, divided by the difference in TSQ for the first and last operation in the group (due to staggered entry and exit times). An analysis of variance
The relationship between TSQ and in-hospital mortality and between TSQ and logistic EuroSCORE is graphically assessed by: (1) plotting the mean values against the integer-part of TSQ; and (2) overlaying a fitted smoothing curve.
To quantify the association between TSQ and risk-adjusted mortality, a mixed-effects logistic regression model was fitted to the data, with the dependent binary variable as in-hospital mortality and TSQ as an independent variable. For all mixed-effects regression modelling, TSQ was centred at the mean value. 12 The logistic EuroSCORE was included in the regression model as an independent variable using a logit-transformation. To account for drift in calibration of the logistic EuroSCORE with increasing time in the UK population, an integer variable (2003 = 1, 2004 = 2, … , 2012 = 10) indicating the calendar year was included in the model as an independent variable. 13 A random intercepts model was used to account for outcomes being correlated within consultant surgeons and resulting extra-binomial variability. Wald statistics were used to assess the null hypotheses that model coefficients are zero.
A second model was fitted that included random intercepts and random slopes for TSQ, which allows for correlation of the random-effects for the same surgeon. This allows individual surgeons to have a separate slope associated with their intercept. A generalised likelihood ratio test was used to compare the two models. As the null hypothesis is testing a parameter on the boundary set of possible values (i.e. TSQ slope variance = 0), the determined
To assess the linearity assumption of TSQ, we plotted estimated restricted cubic spline functions with five knots between the outcome and mean-centred TSQ: (1) without any other adjustment variables; and (2) adjusted for logit-transformed logistic EuroSCORE and year of operation. The spline estimation was based on a standard (fixed-effects) logistic regression model.
As supportive analyses, we fitted fixed-effects logistic regression models for surgeon effects that do not account for any between-surgeon variation, and also models where no surgeon effects are included. In addition, we also fitted models with TSQ categorised as for the three groups above.
A
Results
Following the exclusion criteria and record linkage to the GMC register, a final dataset of 292,973 records performed by 273 consultant surgeons was available for analysis. A flowchart of the data is summarised in Figure 1. The maximum number of consultants operating in any one year occurred in 2010 with 242 surgeons (Figure 2). The distribution of TSQ ranged between 11.2 (6 surgeons) and 42.0 years (3 surgeons) (Figure 2), with the most common experience level being 18 years (131 surgeons).
Flowchart of study data. Summary of records included in the analysis. Top-left panel: number of unique consultants in each study year. Top-right panel: number of unique consultants for each year of time since qualification. Bottom-left panel: distribution of time since qualification by study year. Bottom-right panel: distribution of time since qualification across all records.
Missing data for the EuroSCORE risk factors were <3% for all variables except active infective endocarditis (3.6%); serum creatinine >200 µmol/L (5.2%) and pulmonary hypertension (6.0%). A total of 239,587 records (81.8%) had complete data for EuroSCORE calculation; 43,327 (14.8%) records had one missing variable and 10,059 (3.4%) had two or more missing variables.
Patient, operative and study characteristics.
LVEF: left ventricular ejection fraction; MI: myocardial infarction; SD: standard deviation; TSQ: time since primary medical qualification.
Figure 3 plots the observed mortality against TSQ (top) and the logistic EuroSCORE against TSQ (bottom). We note that the logistic EuroSCORE over-predicts the in-hospital mortality by a factor of 2.4 on average. The curves follow the same increasing approximate linear trend, each crossing the overall sample mean values at approximately TSQ = 21 years, before decreasing at a turning point of TSQ = 33 years.
Trends in time since primary qualification (TSQ) and observed mortality (top panel) and logistic EuroSCORE (bottom panel). Each black dot corresponds to the mean mortality (vertical axis) for all operations performed by consultant surgeons with a specific [integer] number of years experience (horizontal axis). The size of the black dots is proportional to the total number of operations performed by these surgeons with the given measure of experience. Note that each dot comprises multiple consultant surgeons, and each surgeon can contribute to >1 dot. The blue lines are fitted (generalised additive model) smoothing curves to illustrate the trend, and the grey-shaded areas denote approximate 95% confidence intervals. The red horizontal lines denote the overall mean observed mortality (2.62%) and logistic EuroSCORE (6.18%), respectively, for the 10-year study period.
Estimated regression model coefficients.
CI: confidence interval.
Random intercepts standard deviation 0.39.
Random intercepts standard deviation 0.38; random slopes standard deviation 0.02; correlation −0.28.
As a supportive analysis, we fitted a logistic regression model with fixed-effects for the surgeons. Data for one surgeon with zero deaths were excluded from this model. The odds ratio for TSQ was 0.984 (95% CI: 0.905–1.069;
Discussion
Statement of principal findings
CABG and/or valve surgery make up the majority of cardiac surgery in the UK. 10 In this analysis of 10 years of operations performed in NHS hospitals in the UK by 273 consultant cardiothoracic surgeons, we found that observed mortality increased with experience, as did baseline risk. After risk adjustment, a significant association remained, albeit modest, between the time since obtaining a primary clinical qualification and date of surgery, which spanned an interval of 11–42 years, and in-hospital mortality.
Strengths and weaknesses of the study
This study has been performed using national clinical registry data. Despite the data included being validated on a regular basis by each contributing hospital, some erroneous and missing data are inevitable. The number of records excluded due to either missing outcome data or missing GMC registration number was very small (<0.2% of records). The imputation assumptions for risk factors made here have been found to be robust during validation exercises.
Not having the full time course of all surgeons' career as a consultant is a limitation. The staggered appointment of consultants means that data in the most and least experienced regions are attributable to only a few established and newly appointed consultants, respectively. We also observed an association between study year and TSQ. Changes to cardiothoracic training in recent years means that the metric used might be subject to more complex temporal confounding. Unlike other studies, we have not included data on surgeons prior to appointment as a consultant, which might highlight non-linear associations. 5 It is also not possible to account for periods of absence during the study period. Finally, it is recognised that not all surgeons follow the pathway of school, medical education, postgraduate training in surgery and consultant appointment. This is a potential limitation to our study, but we feel that pathways to consultant grade that are markedly different to this are unusual.
It has also been shown that outcomes and patient risk profiles have changed over the past decade in England and Wales. We included the logistic EuroSCORE and year of surgery to adjust for these factors; however, it has been shown in general that risk adjustment in the ‘highest risk’ patients performs inadequately.21,22 There are likely important variables associated with the highest risk patients that have been omitted. Although partially reflected in the EuroSCORE II model, it was not possible to use this score here due to data not being collected by the national registry.23,24 Recent studies have shown that the number of variables predictive of in-hospital mortality dwarfs the 17 included in logistic EuroSCORE. 25 There are likely to be confounders that have gone unmeasured, or cannot be quantitatively measured, that an experienced surgeon will use in the decision-making process; for example, patient frailty. Furthermore, unmeasured confounders could also selectively bias the referral process in a unit, in favour of either the more or less experienced surgeon.
Strengths and weaknesses in relation to other studies
Clinical ‘experience’ is not a well-defined concept, and so there is not an omnibus measure. The experience of a consultant cardiothoracic surgeon depends on many factors, including the time spent practising cardiothoracic surgery; the number of cases performed (as a consultant and as a trainee); the cumulative complexity of cases performed and the amount/quality of formal postgraduate training.
Notwithstanding an exhaustive definition of ‘experience’, the most commonly reported measure is the case-volume for a cross-sectional study period, with researchers testing the hypothesis that ‘quality’ and ‘quantity’ are associated. For example, Birkmeyer et al. 8 showed that surgeon volume was inversely related to operative mortality for coronary artery bypass surgery and aortic valve replacement surgery. Volume-outcome studies compare only the average outcomes within the healthcare providers clustered by volume; they do not, however, examine the change in outcomes within these clusters.
Hartz et al. 26 reported that the observed-to-expected mortality ratio increased with the number of years of experience. In particular, the ratio significantly increased linearly from approximately 0.9 after 10 years of training completion, to 1.3 after 30 years training completion. Such a substantial change might be attributable to not including an overdispersion parameter in their model. A study by Duclos et al. 27 analysed the association of complications in thyroid surgery with surgeon age and time since qualification. They found 20 years of experience or more was significantly associated with increased odds of postoperative complications. However, their analysis was over just a single year and failed to account for trends within surgeons.
Meaning of the study
The study data contained operations performed under the care of consultant surgeons with between 11 and 42 years of service as clinicians (since primary qualification). Records corresponding to operations performed with TSQ > 35 years accounted for only 4.1% of records and are attributable to only a small number (
Mediating effects between experience and outcome in a causal pathway might include general fitness, diminishing eyesight and dexterity. Increasing experience might be associated with less contemporary knowledge of research or resistance to the introduction of evidence-based guidelines. It should be noted that in this study, each operation was attributed to the ‘responsible consultant’ rather than the ‘primary operator’. However, we have not accounted for the proportion of cases that the responsible consultant was the primary operator for, or the proportion of cases performed by dual-consultants, both of which may vary with TSQ. The most experienced consultants might have managerial, academic roles, or an increased private practice caseload. Academic seniority has shown to be associated with increased mortality in Hartz et al., 26 yet Baskett et al. 3 found no significant difference in mortality or postoperative complication rates between resident and staff cardiothoracic surgeons. As a result of changes to surgical services, the more experienced surgeons in this cohort may have dual cardiac and thoracic surgical practices compared to the less experienced surgeons. All of these factors may mean that ‘experienced’ surgeons have a lower annual case volume, as was observed here.
Outcomes also depend on the healthcare provider. Our primary analysis adjusted for this dependence on surgeons using random-effects modelling, which accounts for both within and between-surgeon variation. Supportive analyses that included fixed-effects rather than random-effects for surgeons, thus accounting only for within surgeon variability, yielded a negative coefficient for the association between experience and outcome after adjustment for EuroSCORE and year of surgery; however, this was statistically non-significant (
This study used in-hospital mortality as its measure of outcome. We have previously defended this as an important outcome measure.28,29 Nonetheless, other markers of quality of care are available and might show different associations; for example, long-term mortality, postoperative complications and patient-reported outcome measures.
Conclusions
The date of primary qualification is a time origin common to all clinicians and so reflects broad ‘experience’. Our findings describe how in-hospital mortality following cardiac surgery rises steadily over the first 20 years of a cardiac surgeon's consultant practice and then falls. At the same time, the average risk profile of the surgeons' patients rises and falls concordantly. After adjusting for baseline clinical and patient risk factors, as well as year of operation, we find a statistically significant association between mortality and experience. This risk may be real, or could be due to the failure of the model to account adequately with the case-mix of surgeons. The size of this association is modest – it may not be clinically significant on average, and both the size and significance was shown to be sensitive to model choice. Nonetheless, this study raises potentially important questions about increasing retirement age for surgeons in technically demanding specialties.
Footnotes
Declarations
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
