Abstract
Post-discharge phone calls are a widely used yet suboptimal method of ascertaining recovery of day surgery patients. We compared the efficiency of an automated electronic system of follow-up, the Day Care Anaesthesia Outcomes Recording Registry (DayCOR), and a telephone call system that was standard practice in our non-profit private healthcare organisation in Victoria, Australia. We also surveyed a group of clinicians to assess their acceptance of DayCOR compared with the telephone call system. DayCOR is a web-based system which collects, alerts, manages and analyses patient-reported outcomes. Patients may opt in to respond to a 15-question survey via a link sent by text message or email. DayCOR’s patient response rate was 77.5%, compared with 66.0% for the telephone call system. Both systems collected data on clinical, process, and experience outcomes. Completeness of data collection was 100% using DayCOR compared with 51%–61.4% of data items using the telephone call system. We estimated that replacing our telephone call system with DayCOR to follow up 60,000 day surgery patients a year would represent an annual cost reduction of AUD$101,345 (53%) using manual demographic data entry, and AUD$142,745 (74%) if DayCOR was integrated with the institution’s existing administrative software. Seventy-eight percent of day surgery nurses and 94% of anaesthetists preferred DayCOR to the telephone call system. All anaesthetists surveyed stated that DayCOR provided more valuable feedback, and almost one-fifth had changed their clinical practice as a result. DayCOR’s efficiency and acceptability will allow more effective collection of post-discharge patient outcomes than is currently possible in our institution, and will support interventional studies aimed at improving quality of recovery of day surgery patients.
Keywords
Introduction
In 2016–2017, 2.34 million anaesthetics were performed for day surgery procedures in Australia; 32% of procedures occurred in public hospitals, 48% in private hospitals and 20% in free-standing private day facilities. 1 However, there is no established method of determining the quality of recovery of this group of patients. This may be partly due to the challenges of collecting post-discharge patient-reported outcomes from large numbers of patients, and to the perceived but outdated perception of day surgery as low risk. 2 Additionally, reporting methods and requirements for healthcare organisations (HCOs) across Australia differ by funding mix and by state, and the clinical data that is collected is rarely available to clinicians. 3
The Australian Council on Healthcare Standards (ACHS) provides a national benchmarking service via a clinical indicator programme. 4 Reporting is voluntary, and in 2017 825 HCOs provided data. 5 The ACHS Clinical Indicator set for day patients (which includes day-admission haemodialysis and oncology patients) contains 14 clinical indicators, of which two are post-discharge indicators. Both are process measures—‘follow-up phone call within seven days’ and ‘follow-up phone call received by patient or carer within seven days’. In 2017, 280 HCOs submitted data for the Day Patient Clinical Indicator set. 6 The average reported rate for the first indicator was 88.0% from 59 HCOs. The rate for the second was 90.4% from 72 HCOs. Reporting is likely to be biased, as demonstrated by the low number of reporting organisations.
In order to adhere to the recommended clinical indicators, facilities use a telephone call system (TCS) for post-discharge follow-up of day surgery patients. Individual facility policy determines which patients are to be called. The information collected differs in each facility, but generally includes clinical outcome, process, or patient experience measures. Facilities may supply this information to treating clinicians, or make it available for group comparison at hospital or state level.
We do not reliably measure or report post-discharge clinical outcomes, therefore we do not know which groups of day surgery patients may be experiencing suboptimal outcomes. This evidence gap has implications for patient care and quality improvement initiatives at clinician, hospital and state levels.
A basic requirement of any alternative to the current telephone follow-up system is that it should have an improved response rate, efficiency of data collection and cost, and be acceptable to users. A Swedish study found that a smartphone-based application to evaluate patients after day surgery may be cost-effective by reducing expensive unplanned healthcare contacts. 7 The control group received standard care, which was no follow-up other than a planned surgical appointment. To our knowledge, there is no data comparing an alternative to the TCS for day surgery patients in Australia or elsewhere.
The Day Care Anaesthesia Outcomes Recording Registry (DayCOR) is an automated web-based system which collects, alerts, manages and analyses patient-reported data. A survey link is automatically sent to a patient via mobile phone text message and email 24 h after discharge, with a reminder at 48 h. Clicking on the embedded message link automatically redirects patients to a web-based form. After reading a short introduction, they may opt in to respond to a 15-question survey consisting of clinical, process and patient experience questions. Mean completion time is approximately 2 min. De-identified data is available for comparative reporting by any collected variable, at anaesthetist, subspecialty and hospital level. At the time of writing, it was used by four Australian hospitals.
Our aim was to determine whether an automated electronic system of follow-up (i.e. DayCOR) was more efficient and acceptable than a TCS of follow-up of day surgery patients. Our primary objectives were to compare the quality of data collected by each method, and to undertake a basic cost comparison study of the two methods. Our secondary objective was to assess the user acceptability of each method.
Our primary outcomes included patient response rate, data completeness and economic costs, including the cost of nursing labour and the cost of technology. Our secondary outcomes included the proportion of nurses and anaesthetists preferring one system over another overall, the proportion of users reporting a preferred system for providing the most valuable patient feedback and the proportion of users that have changed their practice as a result of feedback from either system.
Methods
Study design and setting
This was a comparison of two parallel systems of follow-up of day care patients already in use in the hospital, from 1 March 2018 to 31 August 2018. The study site was a tertiary non-profit private hospital in metropolitan Melbourne, Australia. It is one of seven acute care campuses within a Victorian HCO, which treats approximately 60,000 day surgery patients a year. DayCOR was introduced at the study site in February 2018. The hospital’s Clinical Governance Committee reviewed and approved DayCOR’s use for clinical quality purposes. The proposal for this comparison of two systems of follow-up (consisting of a review of administrative data and a clinician survey) was considered by the hospital’s Research Governance and Ethics Unit on 30 August 2018 and met all ethical requirements.
DayCOR and the TCS systems of follow-up ran in parallel in the hospital for the duration of the comparison period. Thirty-five anaesthetists were approached and all gave consent for their patients to be followed-up using DayCOR instead of the existing TCS. The anaesthetists were selected so that a variety of surgical specialties, and therefore patient demographic characteristics, would be represented in DayCOR. Patients of the remaining anaesthetists were followed up with the usual TCS, as were patients at the other campuses. Six hundred and eighty-five anaesthetists are credentialled to work at all campuses of the HCO; it was not possible to accurately determine the number of anaesthetists whose day surgery patients were followed up with the usual TCS at the study site during the study period. Day of Surgery Admission (DOSA) nurses at the study site used both DayCOR and the TCS.
The first part of this comparison involved review of administrative data from both systems to measure patient response rates and data completeness. All hospital patients provided consent for use of their data for the purposes of clinical quality improvement. The hospital’s Research Governance and Ethics Unit determined that use of such data for this study was within the scope of health service assessment, consistent with the primary purpose for which the data was collected. As such, use of this data was not in breach of the Privacy Act and therefore a waiver of the need for consent was not required.
The second part of this comparison involved a survey of nurses and anaesthetists. The act of voluntarily filling in a survey is implied consent. They could freely choose not to participate and were informed that only non-identifiable group data would be used in presentation or publication of this study’s findings. Research of this low risk nature may be reviewed through non–Human Research Ethics Committee processes as described in the Australian National Statement on Ethical Conduct in Human Research, Section 5.1.18–20. 8 Our institution permitted the survey to proceed through this pathway of review.
Telephone call system of follow-up
The standard method of post-discharge follow-up of day surgery patients at all campuses of the HCO is a nurse-initiated telephone call the day after discharge. Due to nursing labour time and cost constraints, certain patient groups are excluded from the planned telephone calls. These are patients undergoing procedures designated as ‘low risk’ (such as endoscopies and procedures under local anaesthesia with or without sedation) and patients whose surgeon/proceduralist has requested exclusion. Telephone calls are not made on weekends, public holidays or when clinical workload is high, therefore patients having procedures on Friday, Saturday and the day before a public holiday are also excluded.
Telephone calls are made by DOSA nurses, who are responsible for the admission and discharge of day surgery patients. Calls are made by one or more nurses when nursing workload is lowest, generally late morning or mid-afternoon. When a patient is contacted, the conversation is prompted by a list of questions on the hospital’s electronic administrative Inpatient Management (IPM) system (Supplementary Appendix 1). Patient responses are entered into a list of drop-down options; comments may be recorded in IPM or the paper hospital file. The DOSA Nurse Unit Manager is able to generate an IPM summary for a specific date range. Results are grouped by operating theatre and presented as a summed score of patient responses to four questions (pain, nausea, ambulation, surgical bleeding). These summaries may be reviewed by the hospital’s Clinical Quality committee. No direct clinician feedback is provided.
DayCOR system of follow-up
Patient demographic and procedural information is manually entered into the DayCOR website by the discharge DOSA nurse. Website links to the 15-question survey (Supplementary Appendix 2) are sent automatically via mobile phone text message and email. Patients can submit a survey once all clinical and process measure questions have been answered (patient experience comments and suggestions for improved healthcare are optional).
‘Alerts’ are triggered by patient responses such as a numerical pain score greater than 6 out of 10 which is not improving, or prolonged vomiting. Alerts can be viewed by designated DOSA nurses when logged into the hospital’s secure DayCOR account. The DayCOR software immediately forwards these alerts to the treating anaesthetist’s email address, and a dedicated hospital DayCOR email address. Anaesthesia-related alerts (such as pain, vomiting, anaesthesia-related negative comments) are managed by the treating anaesthetist. An allocated DOSA nurse checks for other alerts or patient comments during the day. Issues regarding nursing care are managed by the DOSA nurse or referred to nursing management.
The two systems collect comparable clinical outcome, process and patient experience data (Table 1).
Data collected by each system.
‘+’ indicates data collected.
DayCOR: Day Care Anaesthesia Outcomes Recording Registry.
Outcome measures
Due to the operational differences of the two systems, a direct comparison was not possible.
Response rate
We calculated the reported patient response rate of the DayCOR system, and the proportion of patients called and successfully contacted using the TCS.
Data completeness
We compared the proportion of patients who provided an answer to each item of patient-reported data collected by each system. Data for the DayCOR system was user-generated directly from the web application. Data for the TCS was extracted from IPM by a hospital data administrator.
Economic costs
To estimate the cost of the DayCOR system we measured the time required for five different DOSA nurses to enter a patient’s demographic and surgical details into DayCOR. Measured time was from log-in to log-out from DayCOR, and included time to consult the patient’s medical file. We also calculated the time required for a nurse to read and action patient alerts in the DayCOR system on three different days of the week. The fixed per patient fee to use the DayCOR infrastructure was AUD$0.52, which includes the cost of text messages. There were no hardware or other setup costs for DayCOR.
To estimate the cost of the TCS we measured the time required for nurse-initiated phone calls on three different days of the week. Measurement started from the time patient files were collected and concluded when the nurse logged out of IPM, and included time required to leave phone messages, provide healthcare advice, write progress notes and enter patient responses into IPM.
The cost of nursing time was based on an hourly rate of AUD$41.00. Annual cost estimates were based on a projected follow-up of 60,000 patients per year. We did not calculate costs of hospital phone calls or internet access, or those incurred by patients or anaesthetists.
User acceptability
We considered DOSA nurses and anaesthetists to be ‘users’ of the follow-up systems. At the end of the comparison period we surveyed DOSA nurses who had used both systems, and anaesthetists who had used both systems and had at least 20 patients followed up using the DayCOR system. We asked participants which system they preferred, which system provided the most valuable patient feedback and whether they had changed their clinical practice as a result of either system.
Results
During the six-month comparison period 30,637 patients underwent day surgery at all campuses; 11,016 were followed up using the TCS. The study site treated 8625 day surgery patients, of whom 1487 patients were chosen to be followed up using the DayCOR system (Figure 1). Due to technical limitations of the hospital’s IPM system we were not able to determine the number of patients who were followed up using the TCS at the study site (intersecting area marked X, Figure 1). There was no direct link between the TCS data and episodes of care in IPM, which meant that TCS data could not be linked to a hospital campus, or patient demographic and procedural data.
Study population.
Patient characteristics are presented in Table 2. For the same reason stated above, no data was available for patients followed up by the TCS.
Patient characteristics.
Data shown as mean (standard deviation) or percentage.
DayCOR: Day Care Anaesthesia Outcomes Recording Registry; NA: data not available; ASA: American Society of Anesthesiologists physical status classification; PACU: Post-Anaesthesia Care Unit; NRS: Numerical Rating Scale; PONV: postoperative nausea and vomiting.
Response rate
DayCOR surveys were sent to 1487 patients at the study site; 1152 patients (77.5%) submitted a completed survey (Table 3). Compared with responders, non-responders were older (mean age 56.6 compared with 49.6 years) and more likely to be male than female (52.1% compared with 42.1%). Of the 334 non-responders, 316 (94.6%) clicked on the survey link but did not complete the survey, 11 (3.3%) declined and seven (2.1%) surveys failed to send due to incorrect mobile phone number and email address.
Response rate.
DayCOR: Day Care Anaesthesia Outcomes Recording Registry.
Telephone calls were made to 11,016 patients at all campuses. We were not able to determine the number of telephone calls made to patients at the study site due to data linkage deficiencies described above. The number of patients successfully contacted using the TCS was 7274 (66.0% of patients who were called; 25.0% of day surgery patients at all campuses). IPM does not facilitate demographic analysis of non-responders. There were no recorded patient comments regarding preference or acceptability of either system.
Data completeness
Patients who submitted clinical and process measures using DayCOR numbered 1152 (100% of responders; 77.5% of surveys sent) (Table 4). With the TCS, clinical outcome measures were collected from 4365–4435 patients (60%–61% of patients who were successfully contacted; 40%–40.3% of patients who were called; 15.1%–15.2% of all day surgery patients). Process measures were collected from 3710–4466 patients (51%–61.4% of patients who were successfully contacted; 33.7%–40.5% of patients who were called; 12.7%–15.3% of all day surgery patients).
Completeness of each data item collected.
DayCOR, n=1152 (number of surveys returned). Telephone call system, n=7274 (number of patients successfully contacted).
DayCOR: Day Care Anaesthesia Outcomes Recording Registry; N/A: not applicable.
Using DayCOR, 650 (56.4%) responders reported a positive healthcare experience and 98 (8.5%) a negative healthcare experience. One hundred and twenty-three (10.7%) patients made a suggestion to improve care, compared with 283 (3.9%) patients using the TCS.
Economic costs
The total time required to follow up 28 patients (10, 10 and eight patients on three different days) using the TCS was 2 h 12 s, resulting in a mean time per patient of 4.7 min. Fifteen (54%) patients were successfully contacted. Based on an hourly rate of AUD$41.00, the mean cost per patient was AUD$3.21. The estimated cost of nursing labour to follow up 60,000 day surgery patients in one year was AUD$192,600 (Table 5).
Estimated annual cost.
DayCOR: Day Care Anaesthesia Outcomes Recording Registry; N/A: not applicable.
The mean time required to enter a patient’s demographic and procedural details into the DayCOR system was 66 s (mean cost of nursing labour per patient of AUD$0.69), an annual cost of AUD$41,400. We could not meaningfully measure the time required to read and action patient alerts on multiple different days because the times ranged from approximately 1 min on seven occasions (no alerts present) to 18 min on one occasion (collecting further information and acting on a patient response). We therefore allowed 15 min per weekday at each of seven campuses, which was AUD$18,655 per year. The annual cost of nursing labour was AUD$60,55. The fixed per patient fee to use the DayCOR infrastructure was AUD$0.52, an annual cost of AUD$31,200. The total estimated annual cost for the DayCOR system was AUD$91,255.
User acceptability
Nine of 11 nurses and 16 of 16 anaesthetists responded to our survey (Supplementary Appendix 3). Seven (78%) nurses and 15 (94%) anaesthetists preferred the DayCOR system to the TCS. Six (66%) nurses and 16 (100%) anaesthetists stated that DayCOR provided more valuable feedback compared with the TCS. Two (22%) nurses and three (19%) anaesthetists stated that they had changed their clinical practice as a result of patient feedback from the DayCOR system, compared with three (33%) nurses and one (6%) anaesthetist from the TCS. Using DayCOR feedback, one anaesthetist had ‘improved [my] bedside manner’ and two anaesthetists ‘prescribe discharge anti-emetics’. Using phone call feedback, one anaesthetist had ‘changed medication’.
Discussion
The DayCOR system for follow-up of day surgery patients had a higher patient response rate, a higher completion rate of similar patient-reported measures, lower cost and was more acceptable to users compared with the TCS.
One of DayCOR’s benefits is the ability to compare a user’s data with de-identified grouped data from other sites, by any collected variable. For example, during the comparison period the three other hospitals using DayCOR (in metropolitan and regional New South Wales and Queensland) reported a comparable response rate of 72.1% in a patient demographic which was similar except for funding mix (95.5% privately insured patients at the study site compared with 47.4% at the other sites) (unpublished data, DayCOR website). This implies that a similar response rate may apply to hospitals other than the study site. The generalisability of our study’s other findings (data completeness, cost and user acceptability) will depend on each facility’s current follow-up system. This data is difficult to obtain; anecdotal evidence suggests that some facilities do not have a process for patient follow-up, whereas other facilities routinely attempt to contact every patient. In particular, projected cost estimates for both methods of follow-up apply to our institution only, and are likely to be different in other facilities.
One reason that facilities do not routinely follow up all day surgery patients is the cost of nursing labour to make phone calls. To enable a more direct cost comparison, we extrapolated and estimated costs for our annual number of day surgery patients for both systems. Using DayCOR instead of a TCS to follow up all patients in our institution would represent a cost reduction of 53%. Furthermore, replacing manual data entry by DOSA nurses with direct imports of demographic information from IPM into DayCOR would eliminate data entry costs and errors. This would represent a cost reduction of 74%. We plan to introduce automated data importing in 2019 when DayCOR is sequentially implemented at the other six acute care campuses in the healthcare group.
The ACHS Clinical Indicator set for post-discharge follow-up of day patients has been revised. From January 2019 the two indicators will be the proportion of patients who were contacted according to facility policy within 48 h via telephone call, text message, email or a smartphone application, and the proportion of patients who received follow-up instructions on discharge. 9 DayCOR meets the reporting requirements of these indicators. It will also assist in the mandatory collection and review of patient experience data, which was introduced into Victorian Health Services legislation on 1 July 2018. 10
We were unable to directly compare the two follow-up systems, due to differences in operational requirements and the type of patient-reported data collected. Unexpectedly, we also could not directly compare the two study populations due to limitations of the HCO’s data management system (there was no direct link between the TCS data and episodes of care in IPM). To do so would require a manual search of individual patient episode numbers, or additional software coding by the IPM provider, neither of which were feasible for this study.
A randomised study design would have been more powerful, even without effective blinding of patients, nurses and anaesthetists. Given the limitations discussed above and logistical constraints, a parallel-group study design was considered to be the most appropriate. A stepped-wedge design may be preferred if further studies are undertaken when DayCOR is sequentially implemented at other healthcare sites.
A method of determining patient acceptability was outside the scope of this comparison, however DayCOR’s higher response rate may indicate increased patient acceptability compared with the TCS. A thorough assessment of user acceptability, defining users as patients, nurses, anaesthetists, surgeons, proceduralists and hospital managers, would entail an inductive qualitative approach utilising in-depth interviews and focus groups. Such a study may yield valuable information such as potential barriers to patient use of the DayCOR interface (lack of mobile telephone or internet access, poor vision) compared with the TCS (inconvenience or inability to answer the telephone during work or study hours, reluctance to reveal health status or criticise a healthcare provider in person).
Estimating the cost of nursing labour for follow-up of post-discharge patient issues was challenging due to the different operational nature of the two systems being compared. For DayCOR we estimated the time required to read and act on patient alerts, which included calling the patient if required. For the TCS some patient issues were resolved at the time of the call by the DOSA nurse. During the period of observation no patient issues were raised that required further follow-up; however, only 15 patients were successfully contacted by telephone. For both systems, concerning patient issues that required extensive further follow-up by the DOSA nurse or manager were rare, and therefore difficult to measure. This may change when more patients are surveyed. Additionally, we did not attempt to define and measure the opportunity cost of nursing labour, potential benefits of personal contact with a phone call or costs incurred by patients or anaesthetists. The costs of each system may be more accurately estimated using larger samples but are unlikely to change the direction of the result.
During the comparison period, patient-reported outcome data was collected from 2403 patients attending hospitals in three states using DayCOR. This cohort is too small to analyse for group differences or associations, particularly when baseline differences such as surgical specialty are considered. However, DayCOR’s efficiency and acceptability will enable follow-up of many more patients than is currently possible in our institution. Automated real-time capture of post-discharge outcome data from hundreds of thousands of patients at multiple facilities would allow the anaesthetic community to measure and compare de-identified outcomes and target areas for quality improvement.
Clinical quality registries collect health outcome data and generate risk-adjusted reports 11 and are increasingly used in quality improvement and interventional studies. 12 Additionally, a recent systematic review found that clinical registries improve healthcare processes and outcomes. 13 Utilising DayCOR, an efficient and acceptable method of post-discharge patient follow-up, is the first step towards improving recovery for patients undergoing day surgery.
Supplemental Material
Supplemental Material1 - Supplemental material for Efficiency and acceptability of an automated electronic system (DayCOR) compared with a telephone call system, for follow-up of day surgery patients
Supplemental material, Supplemental Material1 for Efficiency and acceptability of an automated electronic system (DayCOR) compared with a telephone call system, for follow-up of day surgery patients by Nicole LT Tan and John R Sestan in Anaesthesia and Intensive Care
Supplemental Material
Supplemental Material2 - Supplemental material for Efficiency and acceptability of an automated electronic system (DayCOR) compared with a telephone call system, for follow-up of day surgery patients
Supplemental material, Supplemental Material2 for Efficiency and acceptability of an automated electronic system (DayCOR) compared with a telephone call system, for follow-up of day surgery patients by Nicole LT Tan and John R Sestan in Anaesthesia and Intensive Care
Supplemental Material
Supplemental Material3 - Supplemental material for Efficiency and acceptability of an automated electronic system (DayCOR) compared with a telephone call system, for follow-up of day surgery patients
Supplemental material, Supplemental Material3 for Efficiency and acceptability of an automated electronic system (DayCOR) compared with a telephone call system, for follow-up of day surgery patients by Nicole LT Tan and John R Sestan in Anaesthesia and Intensive Care
Footnotes
Acknowledgements
The authors wish to thank the DOSA nursing unit for their assistance with testing and implementing DayCOR, in particular the DOSA project lead, Annabel Wallman. We also wish to acknowledge Dr Ken Sleeman, the driving force behind the concept of a national day care anaesthesia registry.
Declaration of conflicting interests
The author(s) declare the following potential conflicts of interest with respect to the research, authorship and/or publication of this article: JRS received personal fees from CAST, the parent website on which DayCOR is hosted.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
