Towards a national perioperative outcomes registry: A survey of perioperative electronic medical record utilisation to support quality assurance and research at Australian and New Zealand College of Anaesthetists Clinical Trials Network hospitals in Australia

Abstract

In Australia, 2.7 million surgical procedures are performed annually. Historically, a lack of perioperative data standardisation and infrastructure has limited pooling of routinely collected data across institutions. We surveyed Australian and New Zealand College of Anaesthetists (ANZCA) Clinical Trials Network hospitals to investigate current and potential uses of perioperative electronic medical record data for research and quality assurance.

A targeted survey was sent to 131 ANZCA Clinical Trials Network–affiliated hospitals in Australia. The primary aim was to map current electronic data collection methods and data utilisation in six domains of the perioperative pathway.

The survey response rate was 32%. Electronic data recording in the six domains ranged from 19% to 85%. Where electronic data exist, the ability of anaesthesiology departments to export them for analysis ranged from 27% to 100%. The proportion of departments with access to data exports that are regularly exporting the data for quality assurance or research ranged from 13% to 58%.

The existence of a perioperative electronic medical record does not automatically lead to the data being used to measure and improve clinical outcomes. The first barrier is clinician access to data exports. Even when this barrier is overcome, a large gap remains between the proportion of departments able to access data exports and those using the data regularly to inform and improve clinical practice. We believe this gap can be addressed by establishing a national perioperative outcomes registry to lead high-quality multicentre registry research and quality assurance in Australia.

Introduction

Australia has a population of approximately 25 million people, with one-quarter of hospital discharges involving surgery, and approximately 2.7 million procedures performed per year.^1,2 The Australian and New Zealand College of Anaesthetists (ANZCA) Clinical Trials Network (CTN) conducts investigator-led international multicentre clinical trials to inform best practice in anaesthesia, pain and perioperative medicine.^3–6 Perioperative outcomes research is a priority for the CTN, aimed at improving quality and safety of care. ⁴ To date a lack of perioperative data standardisation and infrastructure within and between Australia’s eight different states and territories has limited pooling of routinely collected data across institutions for research and quality assurance. In Australia, adoption of the electronic medical record (EMR) and anaesthesia information management system (AIMS) varies widely but is being increasingly integrated into perioperative care. ⁷ We surveyed ANZCA CTN hospitals to map current clinical data collection methods in six domains of the perioperative pathway to inform the development of a minimum dataset for a pilot national perioperative outcomes registry.

Methods

Our survey investigated the adoption and integration of the perioperative EMR at ANZCA CTN–affiliated hospitals in Australia. The primary aim was to map the method of perioperative data collection in six domains of the perioperative pathway. The study was approved by the Alfred Health human research ethics committee as a low-risk activity (project 162/20, approved 17 March 2020). The requirement for written consent from participants was waived by the ethics committee, with the submission of a survey response indicating respondent consent to participation. The study is reported in accordance with the Checklist for Reporting Results of Internet E-Surveys (CHERRIES) guideline. ⁸

The study was coordinated from the ANZCA CTN at Monash University, Melbourne, in March 2020. Public and private hospitals were invited to participate if they were ANZCA-accredited specialist training hospitals or had previously participated in ANZCA CTN research. Adult, paediatric and specialty hospitals were included. Peripheral hospitals within the network of an accredited specialist training hospital were excluded. The survey link was sent by email to the primary CTN contact at each hospital, or to the department director where there was no primary CTN contact, with an introductory email from the investigators outlining the purpose of the survey and the consent process.

Primary outcomes related to EMR data capture in six domains selected for clinical relevance based on different stages of the perioperative pathway. These domains included perioperative risk factors (recorded during the preoperative anaesthesia consultation), the intraoperative anaesthesia chart, post-anaesthesia care unit (PACU) data, medical emergency team (MET) calls, in-hospital major complications and mortality. Table 1 shows how these domains align with those collected by the Multicenter Perioperative Outcomes Group (MPOG) in the United States. For each domain the outcome included the current method and format of data collection, the ability to extract electronic data for analysis, any programme of regular or recurrent data extraction, and the purpose for which extracted data are currently being used within the anaesthesiology department. The process of care and critical care domains were excluded from our survey because these data are available from a standard hospital-level administrative dataset (length of stay, admission source and discharge destination and surgical and anaesthesia procedure codes) and by data linkage to the Australian and New Zealand Intensive Care Society (ANZICS) Centre for Outcome and Resource Evaluation (CORE) registries.^9,10

Table 1.

Data collection along six domains of the perioperative pathway. MPOG: Multicenter Perioperative Outcomes Group; ASA PS: American Society of Anesthesiologists physical status; PACU: post-anaesthesia care unit; MET: medical emergency team.

The survey was administered by email in late March 2020, with a direct link to the online survey response page. Respondents were asked to complete the survey within seven days. An automated reminder was sent on day 6 and day 7 to those who had not yet responded. No incentives were offered to complete the survey. The survey contained seven sections, including one section recording hospital and department details and one section on each of the six domains (Supplementary document 1). Each section contained between four and seven questions from which categorical responses were chosen. All sections were displayed on one screen. Adaptive questioning was used to display additional questions conditionally based on responses to previous questions. Survey items were presented in a logical rather than randomised order. No questions were mandatory, and where variations exist in the denominator presented in the results this is because some responses were incomplete. The name of respondent and hospital were collected to ensure no more than one response per hospital was submitted. Participants were able to view and edit their responses before final submission.

The survey was created in REDCap electronic data capture tools hosted and managed securely by Helix at Monash University.^11,12 Simple descriptive statistics were performed using REDCap version 8.10.15 (Vanderbilt University). ¹² No direct funding was required to conduct the survey.

The Organisation for Economic Co-operation and Development healthcare quality indicators expert group defines the electronic health record as the longitudinal electronic record of an individual patient that contains or virtually links records together from multiple EMRs, which can then be shared across healthcare settings. ¹³ Consistent with this terminology, our survey related to the perioperative EMR for a single episode of healthcare.

Results

Study population

The targeted survey was sent to 131 anaesthesiologists at 126 public and five private hospitals across Australia, using the latest contact list of the ANZCA CTN. A total of 42 responses (42/131, 32%) were received, representing public hospitals in six of eight states and territories. No responses were received from anaesthesiologists on behalf of private hospitals or at specialist paediatric hospitals. Half of respondents (21/42, 53%) were the primary CTN contact in their department, one quarter (11/42, 26%) were the departmental director and one quarter (9/42, 21%) were their delegates. The number of anaesthetics given in each department per year ranged from under 5000 (three hospitals) to over 30,000 (two hospitals), with the most common being 10,001 to 20,000 anaesthetics per year (20 hospitals) (Figure 1).

Figure 1.

Respondent hospital size.

EMRs and AIMS

The most common hospital EMR systems reported were manufactured by Cerner (Cerner Corporation, Kansas City, MO, USA) (20/42, 48%), Allscripts (Allscripts Healthcare Solutions Inc., Chicago, IL, USA) (4/42, 10%), EPIC (EPIC Systems Corporation, Verona, WI, USA) (2/42, 5%), Orion (Orion Health, Auckland, New Zealand) (1/42, 2%) and Oasis (Oasis Group, County Dublin, Ireland) (1/42, 2%) (Figure 2). Nineteen percent (8/42) of hospitals reported that medical records were stored electronically using scanned paper records. Nine hospitals reported currently using an AIMS, either within the EMR or as a stand-alone system. The AIMS currently in use included vendor systems from Cerner (8/9, 89%) and EPIC (1/9, 11%).

Figure 2.

Hospital electronic medical record (EMR) databases. Cerner (Cerner Corporation, Kansas City, MO, USA); Allscripts (Allscripts Healthcare Solutions Inc., Chicago, IL, USA); EPIC (EPIC Systems Corporation, Verona, WI, USA); Orion (Orion Health, Auckland, New Zealand); Oasis (Oasis Group, County Dublin, Ireland).

Perioperative domains

Risk factors

The pre-anaesthesia consultation was recorded electronically at 15 hospitals (15/42, 36%). Frailty was routinely recorded at four hospitals (4/41, 10%), using the clinical frailty scale at both hospitals that provided a detailed response to this question. Four anaesthesiology departments (4/15, 27%) had extracted data from the electronic pre-anaesthesia consultation, with two (2/15, 13%) doing so on a regular basis. These data were currently being used for quality assurance by four anaesthesiology departments (4/15, 27%) and for research by two anaesthesiology departments (2/15, 13%).

Intraoperative anaesthesia data

The intraoperative anaesthesia chart was recorded electronically at 12 hospitals (12/42, 29%). At the remaining hospitals it was kept on paper and scanned into the EMR or used a hybrid arrangement (30/42, 71%). Of those departments with electronic anaesthesia charts, all (12/12, 100%) reported having extracted electronic data from the chart, with seven doing so regularly (7/12, 58%). The data were being used for quality assurance by 11 departments (11/12, 92%) and for research by six departments (6/12, 50%).

Post-anaesthesia care unit data

PACU room data were recorded electronically at 16 hospitals (16/42, 38%), with a further eight hospitals (8/42, 19%) transitioning to electronic recovery data and using a hybrid system. Of those hospitals with electronic recovery room data (24/42, 57%), 13/24 (54%) had managed to extract the data and seven of 24 (29%) were doing so regularly. Anaesthesiology departments were using these data for quality assurance (13/24, 54%) and research (2/24, 8%).

Medical emergency team calls

Most respondents (35/41, 85%) reported that there was a MET call database in place at their institution. Further detail was provided by 34 respondents as follows. The MET call database was managed by the intensive care unit (ICU) at 26 hospitals (26/34, 76%), the anaesthesiology department at three hospitals (3/34, 9%) and another team in five hospitals (5/34, 15%), such as a dedicated rapid response team, multidisciplinary critical care group, or hospital administration. Fourteen anaesthesiology departments (14/34, 41%) had been able to extract data from the MET call database, and seven departments (7/34, 21%) were doing so regularly. The data were currently being used for quality assurance by 14 anaesthesiology departments (14/34, 41%) and research by four departments (4/34, 12%).

In-hospital major complications

Eight hospitals (8/42, 19%) reported that in-hospital complications were routinely recorded electronically by medical staff at the time of diagnosis or discharge using Systematized Nomenclature of Medicine–Clinical Terms (SNOMED CT) codes, ¹³ but only three anaesthesiology departments (3/8, 38%) had been able to extract these data and two departments (2/8, 25%) were doing so regularly. The data were being used for quality assurance (3/8, 38%) and research (3/8, 38%).

Mortality data

Responses to this question were incomplete. Seventy percent (26/37) of respondents reported that date of death was recorded in the hospital EMR; of these, 35% (9/26) were currently extracting mortality data. The data were being used for quality assurance by nine departments (9/26, 35%) and for research by three departments (3/26, 12%).

A bar chart diagram summarising the proportion of hospitals recording perioperative data electronically, access to data exports by anaesthesiologists for quality assurance or research, and regular use of data exports by anaesthesiologists for these purposes in each of the six perioperative domains is shown in Figure 3.

Figure 3.

Proportion of respondent hospitals recording perioperative data electronically, access to data exports by anaesthesiologists for quality assurance (QA) or research, and regular use of data exports by anaesthesiologists for these purposes. PACU: post-anaesthesia care unit; MET: medical emergency team.

Discussion

Electronic perioperative data recording varied from 19% to 85% across six domains, demonstrating wide variation in the integration of EMR and AIMS in our sample of 42 hospitals. More importantly, where electronic data exist, the proportion of anaesthesiology departments able to export the data in each domain ranged from 27% to 100%, and the proportion doing so regularly for quality assurance or research ranged from 13% to 58%. These results demonstrate that the existence of a perioperative EMR does not automatically lead to the data being used to measure and improve clinical outcomes. The first barrier is clinician access to data exports, with data in some domains being accessible to only one quarter (27%) of anaesthesiology departments. Even when this barrier is overcome, however, a large gap remains between the proportion of departments able to access data exports and those using the data regularly to inform and improve clinical practice through quality assurance and research.

To our knowledge there are no previous studies evaluating the perioperative EMR in Australia or its role in guiding improvements to clinical care, nor any published Australian clinical trials or cohort studies using routinely collected perioperative EMR data. In Sweden and the United States, the perioperative EMR is being used to drive improvements in clinical care and outcomes. The Swedish Perioperative Outcomes Registry (SPOR) is a national registry linking electronic data from 38 hospitals, covering an estimated 85% of surgical procedures nationally, aided by the use of a single EMR and AIMS. ¹⁴ Case-mix adjustments are performed and a national benchmarking programme identifies outlying hospitals with key metrics beyond two standard deviations. SPOR data have been used to validate a Swedish version of the Quality of Recovery score, ¹⁵ to validate the patient-centred outcome metric days alive and at home after surgery, ¹⁶ and to report process of care outcomes, mortality outcomes, and determinants of mortality in patients with acute hip fracture.^17,18 In the United States, 46 hospitals with a perioperative EMR contribute to the MPOG registry, a perioperative outcomes collaboration, with a data feed from a variety of EMR and/or AIMS systems supplemented by a minimum dataset entered by anaesthesiologists.^19–22 With the US population over 12 times greater than that of Australia, an equivalent sized outcomes collaboration to MPOG in Australia would involve about five hospitals.

When considering the gap between the proportion of our respondent departments able to access data exports and those using the data regularly to inform and improve clinical practice, factors to consider include the time available for these activities to clinical anaesthesiologists whose primary responsibility is patient care; clinician training and experience in research, quality assurance and statistical methodology; and the potential relevance and impact of a small, single-centre study compared to a large multicentre study. Common to both the Swedish and US approaches is the existence of an outcomes registry framework that provides leadership and infrastructure for data collection and analysis. The SPOR, MPOG, and many Australian registries in other disciplines offer infrastructure such as a centralised ethics and governance framework, a specified minimum dataset, a data dictionary, and expertise in data collection, management and risk-adjusted analysis.^23–27 Registry infrastructure facilitates wider participation in multicentre research and quality assurance by hospitals and clinicians who might otherwise not participate, and can generate a more meaningful sample size and higher quality results in a more efficient manner. The National Audit Projects in the United Kingdom and the National Surgical Quality Improvement Program in the US are further examples of how centralised infrastructure can support high-quality research and quality assurance, although these projects do not rely on the perioperative EMR. One of the desired outcomes from the 2021 Australian summit on ‘The hidden pandemic of post-operative complications’, involving stakeholders from all areas of perioperative care, was ‘to decrease and measure unwarranted variability in perioperative complications by effectively and accurately measuring patient outcomes, and transparently sharing data in a way that can be used to implement and drive change’. ²⁸ We believe a national perioperative registry in Australia will provide the ideal framework for using routinely collected perioperative EMR data for risk-adjusted process and outcomes measurement. Clinical quality registries in Australia have been shown to be highly cost-effective, with a return on investment between AU$2 and AU$7 for every dollar invested. ²⁹

The strengths of our study include the results representing hospitals providing approximately 400,000 anaesthetics per year, or 15% of surgical procedures in Australia. ² The results provide a snapshot of the perioperative EMR and AIMS in 2020 and their use by anaesthesiologists for quality assurance and research in ANZCA CTN-affiliated departments. There are several weaknesses of our study. The response rate of 32% was similar to many other published surveys and reflects a recognised limitation of survey research.^30,31 Responder bias was likely to have influenced our results, because anaesthesiologists with an interest in the EMR as a tool for research and quality assurance were more likely to have responded. ³¹ As this was a simple survey of departmental practice, reliability and validity testing of survey items was not undertaken. Our results reflect only public hospital practice in Australia. While 86% of emergency surgery in Australia is undertaken in public hospitals, 66% of elective surgery is performed in the private hospital system, and this population must ultimately be included in research and quality assurance. ² No specialist paediatric hospitals were included in the results; however, a parallel project is being planned by Australian and New Zealand paediatric anaesthetists.

Guided by the survey results, we have developed a minimum dataset for a pilot national perioperative outcomes registry for adults undergoing inpatient non-cardiac surgery.^32,33 Given the wide variation in access to perioperative EMR data, the proposed minimum dataset is based on Australia’s Admitted Patient Care National Minimum Dataset, a national hospital-level administrative dataset containing demographic, procedure, and process of care data. ³⁴ This administrative dataset can be supplemented by clinical data domains from the perioperative EMR at individual hospitals as they become available and accessible. Our survey identified a number of hospitals with whom this process can be initiated in multiple domains, and the next step will be to define minimum and ideal datasets in each clinical domain. Preoperative risk factors were the least developed domain in our survey, with few respondent hospitals recording these data electronically and an absence of data standardisation across institutions. The postoperative period provides more immediate scope for pooling of real-world data, with the majority of hospitals recording MET call and mortality data electronically, and some now implementing physician documentation of major in-hospital complications at the time of diagnosis using standardised data formats.

A minimum dataset collected on the ‘back end’ of a perioperative outcomes registry could be supplemented by a ‘front end’ digital case report form to facilitate cohort studies and registry randomised controlled trials.^35–38 In Australia, data linkage is available to state and national registries, the national death index, and data on healthcare utilisation including medication prescriptions, primary care utilisation and healthcare costs. ¹⁵ Collecting the ANZICS Adult Patient Database identifier would facilitate data linkage to critical care registry data for each episode of care. The proposed minimum dataset would enable the registry to provide risk-adjusted internal and external benchmarking of process of care outcomes (hospital length of stay and critical care utilisation), in-hospital mortality, and discharge destination (home, rehabilitation or a residential nursing facility)—an important patient-centred outcome—within a clinically relevant timeframe to participating hospitals, while clinical data domains are defined, developed and added over time. The proposed minimum dataset is contained in Supplementary document 2.

Our survey found wide variation in the adoption of the perioperative EMR and demonstrated that the existence of a perioperative EMR does not automatically lead to the data being used to measure and improve clinical outcomes. While lack of access to exported perioperative EMR data is one barrier, the primary clinical focus of anaesthesiologists is likely to be another, along with limited expertise in research methodology and limited supporting infrastructure. We believe the gap between accessible data and its use to drive improvement in patient outcomes can be addressed by establishing a national perioperative outcomes registry to support participation in high-quality multicentre registry research and quality assurance. The development and recording of a standardised minimum dataset of preoperative risk factors should be a priority, with a data dictionary specifying data structure, format and definitions. A future perioperative registry should start by using existing electronic hospital administrative data to measure the risk-adjusted process of care and mortality outcomes, supplemented in time by clinical data from the perioperative EMR and data linkage to other registries that will support routine measurement of risk factors and patient outcomes.

Supplemental Material

sj-pdf-1-aic-10.1177_0310057X211030284 - Supplemental material for Towards a national perioperative outcomes registry: A survey of perioperative electronic medical record utilisation to support quality assurance and research at Australian and New Zealand College of Anaesthetists Clinical Trials Network hospitals in Australia

Supplemental material, sj-pdf-1-aic-10.1177_0310057X211030284 for Towards a national perioperative outcomes registry: A survey of perioperative electronic medical record utilisation to support quality assurance and research at Australian and New Zealand College of Anaesthetists Clinical Trials Network hospitals in Australia by Jennifer R Reilly, Carolyn Deng, Wendy A Brown, Dianne Brown, Belinda J Gabbe, Carol L Hodgson and Paul S Myles in Anaesthesia and Intensive Care

Supplemental Material

sj-pdf-2-aic-10.1177_0310057X211030284 - Supplemental material for Towards a national perioperative outcomes registry: A survey of perioperative electronic medical record utilisation to support quality assurance and research at Australian and New Zealand College of Anaesthetists Clinical Trials Network hospitals in Australia

Supplemental material, sj-pdf-2-aic-10.1177_0310057X211030284 for Towards a national perioperative outcomes registry: A survey of perioperative electronic medical record utilisation to support quality assurance and research at Australian and New Zealand College of Anaesthetists Clinical Trials Network hospitals in Australia by Jennifer R Reilly, Carolyn Deng, Wendy A Brown, Dianne Brown, Belinda J Gabbe, Carol L Hodgson and Paul S Myles in Anaesthesia and Intensive Care