Abstract
Risk prediction is an important part of the management of emergency laparotomy (EL) patients. This study aims to investigate the current use of and future directions for EL risk prediction scores. New Zealand members of the Royal Australasian College of Surgery (RACS), Australian and New Zealand College of Anaesthetists (ANZCA) and College of Intensive Care Medicine (CICM) were invited to participate in an anonymous online survey. Responses were received from 316 clinicians (45 RACS, 253 ANZCA and 19 CICM), with 73% of them having >10 years’ experience as a consultant. Risk assessment scores were utilised by respondents for approximately 30% of EL cases. The most common EL risk scores used were Portsmouth Physiological and Operative Severity Score for the Enumeration of Mortality (P-POSSUM) and American College of Surgeons National Surgical Quality Improvement Programs (ACS-NSQIP). When used, respondents used risk assessment mostly preoperatively (RACS 100%, ANZCA 98% and CICM 78%), although P-POSSUM and ACS-NSQIP scores require intraoperative data (which can only be estimated crudely preoperatively by the clinician). Respondents on average ‘somewhat agreed’ that risk assessment scores should only include preoperative variables. The most common reasons for using P-POSSUM and ACS-NSQIP scores were familiarity and ease of use and availability of online/app calculators. The most important outcomes that the respondents would like to predict were quality of life and 30-day mortality rather than long-term impact from EL. These findings suggest that developing a new score may be required to improve utilisation and help in decision-making. This may require tailoring risk scores specifically for EL, and designing them to predict what is preferred by the clinicians making the decisions.
Introduction
Acute general surgery constitutes 8%–26% of all hospital admissions. A proportion of these admissions are for emergency laparotomy (EL), which refers to any abdominal operation performed as a part of an acute admission, including reoperation procedures.1–3 These patients have high rates of morbidity and mortality which range from 10% to 20% at 30 days.4–6 The overall mortality after EL in New Zealand is not known. However, a recent retrospective review at Middlemore Hospital revealed a mortality rate of 8%. 7 The high mortality associated with EL has led to several national and international collaborative efforts to record and reduce mortality. Amongst these are the National Emergency Laparotomy Audit (NELA) from the UK, 8 the Australian and New Zealand Emergency Laparotomy Audit and Quality Improvement programme and the American College of Surgeons National Surgical Quality Improvement Programs (ACS-NSQIP).
Accurate risk prediction can help clinicians to individualise care for higher-risk patients who may benefit from selective therapies such as critical care admission or goal-directed fluid therapy. Risk assessment, especially when made preoperatively, has other important implications beyond audit and quality improvement. It can help in counselling patients and families, targeting therapeutic interventions and determining the appropriate level of support. Patient-predicted risk can aid in deciding on the most appropriate operative or potentially non-operative therapy for a patient. Some patients with an unacceptably high risk may opt for palliative or non-operative treatment options. Risk scoring, when applied in an EL pathway risk assessment, has been shown to reduce mortality by accurately targeting treatment and by propagating the concept of risk between the multiple practitioners and specialties involved in patient care. 9 It is clear that clinical risk assessment tools could have profound implications for individual and population health.
There are multiple indices that can be used to estimate risk, but not all of these can be used in the preoperative setting. Some require estimation of variables only available postoperatively whereas others produce results that are not useful in counselling patients and their families. Risk prediction can be used postoperatively but will only be useful for prognostication and tailoring level of care and not for decision-making for surgery. There are no data about the use of risk assessment scores for EL outside of an audited laparotomy pathway such as the UK NELA. It is unknown how current risk assessment scores are utilised and when those scores are usually applied. It is also not known if current risk assessment scores are predicting the most important outcomes for the clinician. We aimed to explore the use of current risk prediction scores in EL and what clinicians prefer to be predicted by those scores in New Zealand.
Methods
The study was approved by the University of Auckland Human Participants Ethics Committee (UAHPEC ref 019115).
Survey design
The survey was novel and designed specifically for the study. No previous validated assessment tools were utilised. The survey was designed by the first author (A.B.), with editing and reviewing by three consultant clinicians and five registrar-level clinicians. The questions were developed and tailored around the primary aims. Questions were developed from a group session which was used to develop ideas for individual questions. The questions were written by the lead author (A.B.) and sent to the developing group in a Delphi session. The final survey was then sent to all the development group as a final check and adjustment. The survey questions are available in Supplemental Document 1.
Participants
We invited New Zealand trainees, Fellows and consultants of the Royal Australasian College of Surgeons (RACS), the Australian and New Zealand College of Anaesthetists (ANZCA) and the College of Intensive Care Medicine (CICM). The survey was sent electronically to RACS and ANZCA clinicians in New Zealand. Both the Australian and New Zealand CICM members were sent the survey through an e-bulletin newsletter. Only New Zealand CICM clinicians were included in the analysis. The distinction between solely CICM members and ANZCA members is not clear, given dual training or role with many clinicians.
Conduct
The survey was anonymously sent using Qualtrics™ (www.qualtrics.com) via an email link. The email was distributed to all Fellows and trainees of general surgery in New Zealand (approximately 200 members) through the New Zealand Association of General Surgeons. A random sample of 1000 New Zealand ANZCA members were contacted through the New Zealand arm of ANZCA. The survey was also sent via e-bulletin newsletter to approximately 120 members (some of those were Australian members due to the nature of distribution of the e-bulletin). The survey was open for two weeks, and one email reminder was sent during the second week. Partial answers to the survey were also accepted.
Analysis
The analysis of the results was descriptive only. Comparison between clinician specialist groups has also been kept descriptive. For ranking questions, the ranking was calculated by giving the greatest weighting to the highest-ranked item from each individual clinician. The weighting for each item was linear, with reduction in weighting the further the item is down the list from each individual clinician-ranked list. The average overall weighting was used to construct ranking. Reporting of the total number of responses was included if a question was not answered by all survey respondents (skipped question).
Results
A total of 316 clinicians responded to the survey. The respondents were 45 RACS, 253 ANZCA and 19 CICM members (eight of those were both ANZCA and CICM members, and an additional seven respondents did not identify a specialty; Table 1). This equates to response rates for RACS, ANZCA and CICM members of 22.5%, 25.4% and 15.8%, respectively (23.9% in total). The respondents were from all major centres in New Zealand (Figure 1). Not included in Figure 1 are the nine respondents currently practising for the short term in Australia. There was one clinician who did not respond to this question, six New Zealand clinicians undergoing Fellowship training in Australia and three consultant-level New Zealand clinicians currently practising for the short term in Australia.
Characteristics of responding clinicians.
aSeven non-respondents for specialty question.
RACS: Royal Australasian College of Surgery; ANZCA: Australian and New Zealand College of Anaesthetists; CICM: College of Intensive Care Medicine;
Distribution of surveyed clinicians. The number of clinicians responding in each centre is highlighted by the number over the location tag. (Map data @ 2019 Google LLC.)
To explore the distribution of consultant respondents, the number of years of experience for survey respondents and the years of post-Fellowship experience from the New Zealand Medical Council registry were compared in Table 2. There is an underrepresentation of the more junior clinicians (<5 years’ experience) for both RACS and ANZCA members. CICM members were not analysed due to the small number of New Zealand clinicians and the nature of dual ANZCA and CICM Fellowships.
Respondents compared to all specialists in New Zealand.
Years of post-Fellowship experience in respondents compared to the overall population in the New Zealand Medical Council registry. The percentage is from the total numbers given in the right-most column.
aOne ANZCA specialist did not report how many years of experience post specialisation.
Respondents were on average involved in just less than three emergency laparotomies the previous month, with RACS respondents reporting a mean of 3.1 and ANZCA and CICM reporting 2.0 and 3.1, respectively (range 0–10 laparotomies from 219 responses for this question). Respondents felt that the number of laparotomies performed in the previous month was slightly below what was normal for their practice.
Respondents indicated that they sometimes used the risk assessment scores, equating to an overall use of 30% of the time in an average month. Most respondents used risk assessment preoperatively—100% for RACS and 98.2% for ANZCA respondents—but CICM respondents reported lower preoperative use at 77.8% (from 219 total responses). Combined ANZCA and RACS respondents’ preoperative risk score use was greater at 98.4% than postoperative use which was only 11%.
Overall, the most utilised risk scoring system was Portsmouth Physiological and Operative Severity Score for the Enumeration of Mortality (P-POSSUM) for RACS and ANZCA respondents, whilst CICM favoured Acute Physiology, Age, Chronic Health Evaluation II as well (Table 3). The most common reason why a risk score was used across all specialties was ‘familiarity with the risk score’ and ‘availability of online apps/calculators’ (39% and 39.6% of respondents, respectively). ‘Ease of use in acute setting’ and ‘ability to apply preoperatively’ followed, with 26.9% and 24.4%, respectively. The least frequent reason for using a risk score was if it was the ‘most validated for emergency laparotomy’ (6.6%). Overall, respondents ‘moderately agreed’ that the chosen risk assessment would be useful in aiding in decision-making. When asked about the accuracy of risk assessment on average, RACS and ANZCA members stated that they are moderately accurate. CICM members felt that risk assessment is only slightly accurate, which is lower than RACS. We queried about the limitations of current risk scores and categorised those limitations into various subheadings. This categorisation was based on themes established from the responses. The most common limitation was a perceived lack of accuracy of the risk assessment (Table 4).
Most utilised risk scores for each clinician group.
POSSUM (including variants): Physiological and Operative Severity Score for the Enumeration of Mortality; ACS-NSQIP: American College of Surgeons National Surgical Quality Improvement Program; APACHE: Acute Physiology And Chronic Health Evaluation; MELD: Model of End stage Liver Disease; SORT: Surgical Outcomes Risk Tool.
Limitations of current risk assessment.
Responses when asked what the other limitations of risk assessment scores are (except for frailty and need for preoperative variables which were asked in the questions immediately prior).
We asked how likely assessment by a risk score changes decision-making. RACS respondents on average reported it was more likely to change decision than either ANZCA or CICM (199 responses to this question—no responses were given to this question by clinicians if they did not frequently use risk scores). Only three respondents reported that using a risk prediction tool would not alter decision-making at all (three ANZCA). Risk assessment was more likely to alter decision-making for ANZCA respondents than CICM respondents. Given there is potential to alter decision-making, we explored shared decision-making by asking if risk assessment scores were given to the patient and family. The overall response was a mean of 70.1%, which was higher in RACS (80.0%) respondents than either ANZCA (69%) or CICM (73.6%). Clinicians who did not routinely inform patients of the result of risk assessment scores responded with the following reasons (only four responses): patient too unwell to grasp concepts, there is no clear way to express outcomes to patients and scores are too inaccurate to be meaningful for patients. One clinician expressed that there was not enough time to explain risk scores to participants.
For clinicians who reported low use of risk assessment scores, the most common reason (24.5%; 12 respondents) for not using scores was that risk assessment is too cumbersome to use in a timely fashion in the acute setting. The available risk scores were also thought not to be indicated or would not change practice by eight (16.3%) respondents.
Clinicians were asked to comment on what risk scores should aim to predict by giving a list of possible prediction outcomes. There was consistency between all three specialty groups. The reasons (for at least 10% of respondents) ranked from most important to least important were: quality of life, 30-day mortality, functional status, complications, long-term survival and postoperative disposition.
When asked if risk assessment could be improved by adding an assessment of frailty, respondents somewhat to strongly agreed with that statement, with the majority stating that current risk assessment did not adequately assess frailty. Respondents somewhat agreed that preoperative variables are important to improve risk assessment. This was based on the mean score from a continuous six-point Likert scale from ‘strongly agree’ to ‘strongly disagree’ with the statements about frailty and preoperative variables.
Most respondents indicated that they would be extremely likely to apply an easy-to-use risk assessment tool developed in New Zealand if the tool used only preoperative variables and was prospectively validated. Only 15/197 respondents reported that they would be unlikely to use such a tool.
Discussion
We aimed to examine the current use of risk assessment indices and establish what clinicians would like predicted from current risk assessment scores. We found that assessment indices were used by respondents for only about 30% of their cases and usually only preoperatively. Risk prediction used in the postoperative setting would provide only prognostic information and guidance to level of care. Postoperative risk prediction cannot aid in decision-making about surgery or provide patient information, which was more relevant to the survey respondents. As expected, CICM members used preoperative risk assessment less often than RACS and ANZCA members did. This is likely due to the involvement of CICM members for some patients only postoperatively.
The scores most favoured across all specialties were P-POSSUM 10 and ACS-NSQIP, 11 both of which require intraoperative data for an accurate estimate of mortality risk. In the absence of intraoperative data, some variables must be estimated preoperatively. Respondents, in general, felt that intraoperative data were necessary for accurate risk assessment. However, the need for intraoperative data reduces the ability of a risk prediction score to inform decision-making accurately for operative intervention. The need for intraoperative data is in contrast with the preoperative use of risk prediction by most surveyed clinicians. Estimation of intraoperative variables preoperatively by the clinician may reduce the accuracy of risk prediction or potentially reduce the utilisation of risk prediction by the clinician. Interestingly, overall, the risk scores used were not based on their accuracy but on ease of use and accessibility. This may be because respondents perceived no differences in accuracy between scores rather than their perception of the importance of accuracy.
Our study suggests that the current risk assessment indices can potentially be improved by predicting what is desired by the clinician. Risk assessment focuses on predicting mortality, whilst the most important outcome to respondents (and potentially patients) is quality of life. An important caveat is that we have no evidence from this study to suggest that quality of life is more important than mortality for the EL patient—we report only clinician preferences. To improve current risk assessment, surveyed respondents agreed that some form of frailty assessment should be included. Therefore, risk assessment in its present form fails to fully provide what is needed by many end users. However, it must be kept in mind that the response rate was only 24%. We assume, but cannot be sure, that the responses received represent the opinion of the majority of clinicians involved in EL in New Zealand.
Despite the availability of multiple risk assessment scores and the development of new scores, 12 there is very little in the literature examining the current use of such scores. Several audit programmes exist that mandate the use of risk scoring. However, the use of such scores has not been examined outside such programmes. Currently, in New Zealand, there is no unified audit programme for EL, although one is currently in evolution. In the past, risk scores were developed using data from audit programmes 11 , 13 from either general medical or surgical patients and then generalised to other patient groups, including EL. 10 , 14 By examining our current use and perception of risk scores, we hope to develop an index which is better tailored to the end users, including our patients.
There are some limitations to the current study. We only surveyed New Zealand clinicians, which limited the absolute number of respondents, and the online methodology limited the overall response rate. The survey had to be run anonymously online to enable distribution to a wide audience of RACS, ANZCA and CICM members. The response rate was low, as was expected from the survey distribution. Furthermore, the response rate could not be accurately recorded for CICM members, as the survey was sent to some Australian members that were later excluded from analysis. We were unable to follow up with individual respondents and send reminders to fill in the survey. We also could not collect IP addresses to ensure respondents did not complete the survey twice. Given the large difference in clinicians in each group, we had significantly more ANZCA than RACS and CICM respondents, which skewed the overall results. However, the response rate was similar between ANZCA and RACS members, and we have reported specialties separately in the results. Another limitation in the study is the underrepresentation of more junior consultants in the survey. The number of consultants with less than five years’ experience is fewer than expected. Whilst there were minimal demographic differences between respondents and overall clinicians, we cannot exclude responder bias. It is possible that clinicians who were less likely to use risk assessment were also less likely to respond to the survey or vice versa. We are uncertain about the impact of such finding on the results of the survey, but it does raise important research questions for future surveys to explore.
Not all questions were answered by all respondents. We included the data for each question and did not discard the survey if some questions were not completed. This may reduce the applicability of the survey to the overall population. However, for most questions, the response rate was appropriate for the individual question. For example, a clinician who never uses risk scores is unlikely to respond to the question about reasons why they would use a risk prediction tool. Likewise, a clinician who does not use a risk prediction tool is unlikely to have picked a top tool for use in their practice. For the question about frequency of involvement in EL, incomplete responses may have caused us to overestimate results. We may also have overstated the frequency of EL, as clinicians who were not involved in EL may have simply skipped the question rather than reply with zero.
In summary, EL risk assessment scores appear to be poorly utilised by NZ clinicians: respondents used risk scores for only about 30% of cases. Most respondents stated that they chose scores based on ease of use and familiarity. Our findings suggest that currently available risk scores fail to predict the metric most important to respondents: quality of life. Our findings also suggest that the assessment of frailty needs to be improved: most respondents expressed that frailty assessment is needed but that current risk prediction indices do not adequately assess this. This indicates that there is a gap that needs to be bridged between the current risk assessment scores and what ideal risk assessment tools should contain according to the respondents. This requires improving current risk assessment scores or developing a new score to improve utilisation and help in decision-making whilst ensuring accuracy of risk prediction. This may require tailoring of risk scores specifically for EL, and designing them to predict what is preferred by the clinicians making the decisions.
Supplemental Material
sj-pdf-1-aic-10.1177_0310057X20907102 - Supplemental material for Multidisciplinary survey of current and future use of emergency laparotomy risk assessment scores in New Zealand
Supplemental material, sj-pdf-1-aic-10.1177_0310057X20907102 for Multidisciplinary survey of current and future use of emergency laparotomy risk assessment scores in New Zealand by Ahmed WH Barazanchi, Weisi Xia, Ashish Taneja, Andrew D MacCormick, Nicholas J Lightfoot and Andrew G Hill in Anaesthesia and Intensive Care
Footnotes
Acknowledgements
This paper has previously been presented at the American College of Surgeons Annual Scientific Congress in Boston in 2018.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: Health Research Council Clinical Training Fellowship.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
