Sage Journals: Discover world-class research

Abstract

An electronic audit and feedback (e-A&F) system was developed to support healthcare providers’ awareness of their own performance, improve delivery of care and ultimately the safety of patients while in hospital. The point-of-care e-A&F system provides healthcare providers, from a 600-bed tertiary hospital in Western Australia, with near real-time feedback via web-based dashboards. The aim of this evaluation is to determine the implications of e-A&F across multiple dimensions and domains of care in a tertiary hospital setting. The study also aims to address the paucity in the literature by validating hypothesised design and implementation mechanisms on its effectiveness. Key datasets to be examined include those related to patient outcomes, staff behaviour and costs. Quantitative methods, such as interrupted time series analysis and multiple logistic regression analysis, amongst other methods, will be employed to achieve these aims.

Keywords

dashboards electronic audit and feedback hospitals patient safety protocol

Introduction

Patients can experience unintentional but potentially avoidable harm in hospital, which, in addition to the impact on the individual patient, can have broad social and economic consequences.^1–3 In Australia, which is widely⁴ considered to have a well developed healthcare system, one in nine people admitted to hospital will experience an adverse event.⁵ A patient who contracts an infection during hospitalisation and requires a longer stay epitomises these all too common occurrences.^5,6 If this particular event were to be examined, it is possible that healthcare provider hand contamination could be found to represent a modifiable risk factor for this patient outcome, which another organisation, with greater compliance in professional practice standards, may have been able to avoid.⁷

A Cochrane review of 140 randomised control trials investigating the effect of audit and feedback on healthcare professional practice found that the median absolute improvement was 4.3%.⁸ Although this pooled estimate is arguably small a wide range of effect estimates was found. This included an upper quartile of 16.0% improvement in compliance, which suggests that audit and feedback can be a useful intervention to improve delivery of care and patient outcomes in some settings.^8–12 The range in study findings also supports the widely held view that the individual features of an intervention can substantially influence its effectiveness.^13–16

The Clinical Performance Feedback Intervention Theory (CP-FIT), developed to define these specific features, suggests that successful feedback is achieved through a series of sequential processes.¹³ The CP-FIT is a comprehensive framework based on a systematic review of qualitative audit and feedback studies and provides specific design recommendations.¹³ The theory suggests that the process of successful audit and feedback involves the identification of standards of clinical performance against which care will be measured; the collection and analysis of performance data; communication of the measured performance to healthcare providers; reception, comprehension and acceptance of these data by health professionals; a planned behavioural response and ultimately improvements to patient care.¹³

Given the unacceptable proportion of patients who continue to experience unintentional and potentially avoidable harm during hospitalisation, a reduction of this harm should be seen as a collective priority. Accreditation programmes are one approach that national governments have employed to address this unintentional and potentially avoidable harm and help provide quality assurance in hospitals.^5,17 Although global accreditation programmes differ in detail, there has been convergence over time towards a broadly similar model.^18–24 In Australia, hospitals and day procedure services are accredited against the National Safety and Quality in Health Service Standards (the ‘National Standards’).

Evidence of compliance with the National Standards is required by Australian hospitals to achieve accreditation status (see Table 1 below).^25,26 In 2017, a Western Australian tertiary hospital commenced development of a novel electronic audit and feedback (e-A&F) system, Consumer-Engaged National Standards Audits (CENSAs). This system was designed to collect evidence against the National Standards, support staff to learn about their performance, and improve delivery of care.

Table 1.

Summary of Australian National Standards.²⁵

National Standard	Intention
Standard 1 – Clinical Governance	To ensure that there are systems in place within health service organisations to maintain and improve the reliability, safety and quality of health care.
Standard 2 – Partnering with Consumers	To ensure patients are partners in their own care and that health service organisations are providing clear communication to patients.
Standard 3 – Preventing and Controlling Healthcare-Associated Infection	To improve infection prevention and control measures, and the spread of antimicrobial resistance through the appropriate prescribing and use of antimicrobials.
Standard 4 – Medication Safety	To ensure that clinicians safely prescribe, dispense and administer appropriate medicines and monitor medicine use.
Standard 4 – Medication Safety	To ensure that consumers are informed about medicines and understand their medicine needs and risks.
Standard 5 – Comprehensive Care	To ensure that patients receive comprehensive health care that meet their individual needs and consider the impact of their health issues on their life and wellbeing.
Standard 5 – Comprehensive Care	To ensure that risks of harm for patients during health care are prevented and managed through targeted strategies. Risks of harm include: pressure injuries, falls, malnutrition, self-harm, delirium, aggression and violence and restrictive practices.
Standard 6 – Communicating for Safety	To ensure timely, purpose-driven and effective communication and documentation that support continuous, coordinated and safe care for patients.
Standard 7 – Blood Management	To improve outcomes for patients by identifying risks and using strategies that optimise and conserve a patient’s own blood, as well as ensuring that any blood and blood products that patients receive are safe and appropriate.
Standard 8 – Recognising and Responding to Acute Deterioration	To ensure that a patient’s acute deterioration is recognised promptly and appropriate action is taken.

CENSAs offers healthcare providers with near real-time feedback via web-based dashboards which utilise the business intelligence tool Tableau© (Seattle, WA). Data are entered at the bedside on mobile devices by trained auditors using a web-based data collection tool and stored in the secure health service data warehouse. Data validation processes, governance oversight and auditor training provide quality assurance. The dashboards, which are refreshed daily, provide feedback for each work area in the form of quality indicators aligned to the eight overarching National Standards (see the example Figure 1: CENSAs Dashboard Standard 4 – Medication Safety).

Figure 1.

CENSAs Dashboard Standard 4 – Medication Safety.

CENSAs data collection and reporting tools were launched in May 2018 and August 2018 respectively. The quality indicators collected and reported by CENSAs were identified during a rigorous development process that was supported by expert clinical input. A set of criteria for selecting quality indicators was applied over a number of improvement cycles including: clinical impact and relevancy, monitoring feasibility and feasibility to action non-compliance. A process of validation and beta-testing was undertaken on three surgical and three medical wards. CENSAs was also linked to a quality improvement registry and management system to support staff addressing identified deficiencies in clinical practice.

There is some evidence that audit and feedback can improve compliance with desired clinical practice.²⁷ However, little is known about the effects of these programmes when coordinated at a hospital-level and directed at different domains of patient care, such as policy, patient safety culture, guideline adherence and patient outcomes.^8,12 The increasingly widespread availability of electronic health records and web-based technologies also necessitates closer examination of e-A&F, as compared to audit and feedback more broadly.²⁸ One systematic review²⁹ attempted to summarise findings from healthcare e-A&F studies but due to the small number of published studies available and the high level of heterogeneity, it was limited in its ability to make conclusions on the effect size or recommendations on implementation approach and design features.

Study aims and objectives

The first aim of this study is to assess the effectiveness of an e-A&F tool, CENSAs, on patient safety in a tertiary hospital setting. The study will address this aim by evaluating the rates of hospital-acquired complications (HACs) pre- and post- implementation of CENSAs, and will determine the cost-effectiveness of the intervention. The study also aims to examine the potential factors influencing the effectiveness of CENSAs on patient safety, and add evidence to the existing literature on design recommendations of e-A&F systems. This aim will be achieved by comparing the features of the e-A&F system against the hypothesised CP-FIT, such as the characteristics of feedback, characteristics of the organisation and characteristics of the recipients.

Methods

Study design

Interrupted Time Series (ITS) analysis, a quasi-experimental approach, will be used to assess the change in the rates of HACs before and after the introduction of the e-A&F system. Subsequently, the Incremental Cost-Effectiveness Ratio (ICER) methodology will be used to assess the cost-effectiveness of any detected change in the rate of HACs. To help explain causes for the effectiveness of the e-A&F both ITS analysis and descriptive analysis will be used to compare the features of CENSAs against the hypothesised CP-FIT.

Setting

The study will be conducted at a 600-bed tertiary teaching hospital in Western Australia in which the novel e-A&F has been implemented. The hospital is representative of the breadth of tertiary healthcare in Australian hospitals including being equipped with: a 24-hour emergency department; an Intensive-Care Unit and other specialised units such as cardiac surgery, neurosurgery, infectious diseases and organ transplant.³⁰

Outcomes

The intention of the National Standards, which predominately relate to professional practice and clinical processes, is to reduce the occurrence of unnecessary harm. For example, Standard 3 – Preventing and Controlling Healthcare-Associated Infection, requires the implementation of practices and processes to reduce healthcare-associated infections, such as healthcare-associated urinary tract infection. It is then unsurprising that compliance with the National Standards has been illustrated to reduce the occurrence of a set of nationally recognised hospital-acquired complications (HACs) (see Table 2).³¹ Outside the Australian setting comparable clinical practices and processes have also been shown to improve patient outcomes.³² Therefore, CENSAs, aligned to the National Standards, could be expected to mediate the rate of HACs, an important priority for both healthcare providers and patients to reduce.

Table 2.

Hospital-acquired complications.³¹

Complication	Diagnosis
Pressure injury	Stage III ulcer
	Stage IV ulcer
	Unspecified decubitus ulcer and pressure area
Falls resulting in fracture or intracranial injury	Intracranial injury
	Fractured neck of femur
	Other fractures
Healthcare-associated infection	Urinary tract infection
	Surgical site infection
	Pneumonia
	Blood stream infection
	Central line and peripheral line associated bloodstream infection
	Multi-resistant organism
	Infection associated with prosthetics/implantable devices
	Gastrointestinal infections
Surgical complications requiring unplanned return to theatre	Post-operative haemorrhage/haematoma requiring transfusion and/or return to theatre
	Surgical wound dehiscence
	Anastomotic leak
	Vascular graft failure
	Other surgical complications requiring unplanned return to theatre
Unplanned intensive care unit admission	Unplanned admission to intensive care unit
Respiratory complications	Respiratory failure including acute respiratory distress syndrome requiring ventilation
Respiratory complications	Aspiration pneumonia
Venous thromboembolism	Pulmonary embolism
Venous thromboembolism	Deep vein thrombosis
Renal failure	Renal failure requiring haemodialysis or continuous veno-venous haemodialysis
Gastrointestinal bleeding	Gastrointestinal bleeding
Medication complications	Drug related respiratory complications/depression
	Haemorrhagic disorder due to circulating anticoagulants
	Hypoglycaemia
Delirium	Delirium
Persistent incontinence	Urinary incontinence
Malnutrition	Malnutrition
Cardiac complications	Heart failure and pulmonary oedema
	Arrhythmias
	Cardiac arrest
	Acute coronary syndrome including unstable angina, STEMI and NSTEMI
Third/fourth degree perineal laceration during delivery	Third and fourth degree perineal laceration during delivery
Neonatal birth trauma	Neonatal birth trauma

We will assess, as the primary outcome of this study, what the longitudinal effect of the implementation of CENSAs has been in the rate of HACs. In addition, CENSAs, as a point-of-care audit system, has anecdotally permitted timely intervention from healthcare providers where unsafe practices have been detected. As a result, as a secondary outcome, we will evaluate the patient-level, or the average effect that could be expected for an individual patient, of CENSAs on HACs.

We will use the results of these HAC analyses to inform a cost-effectiveness analysis, by using the avoidance of adverse events as the effectiveness value in the denominator, also as a secondary outcome measure.³³

A key aspect of the CP-FIT is a planned change in behaviour which leads to improvements in care delivery.¹³ Thus, we will evaluate the change in the count of quality improvement activities commenced and completed by staff as a secondary outcome measure and a possible factor which could explain the effectiveness of the implemented system. Analysis of systematic staff behaviour change will be supplemented by descriptively exploring the features of the CENSAs system that could explain its effectiveness.

Data sources and sample

A number of retrospective datasets will be utilised for the analysis. These datasets will be extracted from the health service data warehouse. An inpatient administrative dataset, which contributes to the WA Hospital Morbidity Data Collection (HMDC), will be the primary data source for the HAC analysis.³⁴ The sample will include the 4.5-year pre-implementation phase (2014–2018) and 2.5-year post-implementation phase (2018–2020) for all overnight inpatients. Data fields to be extracted include perturbed medical record numbers, age, sex, admission and discharge dates, admission type, discharge specialty, diagnoses, procedure codes, hours spent in ICU and condition onset flags.

The entire CENSAs dataset will be extracted for analysis including perturbed medical record numbers, audit date, audit length, patient specialty, ward and categorical compliance measure fields. The CENSAs and inpatient datasets (for the post-implementation phase) will be linked for the patient-level HAC analysis. This data linkage process will utilise the unique patient identification codes and date fields with a probabilistic match algorithm.

We will utilise the approach of other cost-effectiveness studies by using the avoidance of adverse events as the effectiveness measure in the denominator.³³ Costs will be divided into intervention-related costs and outcome-related costs. The costs of HACs as adverse event outcomes will be approximated using the Australian Independent Hospital Pricing Authorities’ approach to HAC penalties.³⁵ The Australian Government approach to HAC penalties are that they reflect the extra cost of a hospital admission with a complication.³⁵ This additional cost may be the result of a more complex episode of stay or due to an increase in the length of an ordinary stay, and is adjusted based on a number of risk factors.³⁵ Key costs of the intervention will relate to the nursing time for audit data collection.³⁶ The additional nursing time for audit data collection will be determined by approximating the time required to complete an audit using the start and finish time stamps from the audit dataset. Other costs such as administrative, project, and overhead hospital costs will be estimated using the Australian National Hospital Cost Data Collection Cost Report.^37,38

To evaluate the change in the count of quality improvement activities commenced and completed by staff between the pre- and post- implementation periods we will undertake an extract from the state quality improvement registry and workflow software. This system supports staff organisation to create, manage and record quality improvement activities. All submitted quality improvement activities undergo review and assessment by quality improvement committees to promote consistent quality data being recorded in the system. Data fields such as the activity type, specialty undertaking the activity, staff participating in the activity, the National Standard the activity aligns to, proposal approval date and activity completion date will be extracted.

To complement the assessment in change in quality improvement activities we will examine dashboard usage logs as a possible mechanism explaining the effectiveness of the implemented system.³⁹ This approach benefits from quantitative and unobtrusive insight into clinical staff behavioural processes.⁴⁰ We will extract from Tableau© (Seattle, WA) individual access data and the accompanying professional profiles of staff, including the area of employment and profession.

A clinical staff survey will also be undertaken using an online questionnaire tool that we will develop. Clinical staff active in governance structures and committees, such as senior nursing staff who are most likely to have had involvement with the system, will be in scope for participation in the survey. Governance structures and committees will be invited to nominate names of staff for participation. We will initiate contact with potential participants via email. Information about the purpose of the study and how the results will be used will be provided in this initial contact. Survey questions will be based on concepts from the CP-FIT, including the reception and context of health professionals receiving feedback.¹³ Employee information (e.g. role, area of employment) will also be collected as part of the online questionnaire. We will pilot test the survey tool with clinicians for clarity, length and flow of questions with any amendments made as necessary. The online tool will be hosted on REDCap^® (Research Electronic Data Capture), a secure, web-based platform.

Covariates

HACs will be identified using nationally agreed specifications of International Classification of Disease – Tenth Revision – Australian Modification codes, and condition onset flags.^35,41 Condition onset flags can be used to help to distinguish pre-existing conditions from those arising during hospitalisation.⁴² While a widely available source of clinical information, the limitations of administrative datasets to accurately and comprehensively record information such as secondary diagnoses is well established.⁴³

Nationally agreed risk factors that will be used as covariates in the patient-related analyses include: emergency admission status (‘yes’ or ‘no’), patient age group (in 5 year increments), Major Diagnostic Category, hours spent in an Intensive Care Unit, Diagnostic Related Group type (‘medical’ or ‘intervention’), Charlson Comorbidity Index Score, sex and transfer status (‘transferred from another hospital’ or ‘other’).^35,41

Data analysis

We will use an ITS analysis to determine if specific time periods (across pre- and post-intervention periods) are associated with measurable changes in patient outcomes and staff behaviour. The benefits of the approach, as compared to other quasi-experimental designs, is that it avoids internal validity biases such as those related to secular trends.^44,45 Time series plots will be used initially to explore the unadjusted effects of the intervention and presence of trends, cyclical patterns and outliers.⁴⁶ A segmented regression analysis will then be undertaken using discrete intercepts and slopes for the periods before and after the implementation of CENSAs.^44,47 Results will be interpreted for both level and trend changes, from before and after the intervention, with 95% confidence intervals.⁴⁶ For the patient-level analyses we will perform multiple logistic regression analysis comparing the rate of HACs in audited and non-audited patients and controlling for risk factors.³⁵

We will assess the cost-effectiveness of CENSAs as an e-A&F intervention using the ICER methodology. ICER is useful in that it can examine both the marginal health benefits for increasing cost burden.³⁷ The ICER will be assessed using longitudinal multi-year time horizon perspective comparing the implementation of the e-A&F system to a scenario of a continuation of existing practice. All cost will be reported in 2020 Australian dollars.

Descriptive analyses will be undertaken to explore possible features of CENSAs that could explain its effectiveness. In reporting the behaviour of staff in their use of the dashboards we will focus on the extent to which each National Standard dashboard is used and how this varies between professions, clinical areas and over time.³⁹ In addition, CENSAs indicator data will be assessed to determine: if changes in patient outcomes are associated with performance with National Standards; how they vary between clinical areas and how they vary over time. These results will be reported alongside the results of the clinical staff survey that will focus on other aspects of the CP-FIT.

Statistical power

In the pre- and post- analysis the total rate of HACs across the intervention periods will be expressed monthly over 7 years (84 monthly time points in total). While there are no fixed recommendations on the minimum number of time points required for ITS analysis, as power is dependent on factors such as data variability and presence of confounding effects, some studies have suggested 40–50 observations as a guide.^48,49 The count of quality improvement activities will be expressed bi-monthly over the same time period (42 time points), therefore also meeting guidelines for robust statistical ITS design analysis.⁴⁶

In addition, a minimum of 100 observations can improve the stability at each time point.^46,50 From 2014 to 2020 there are expected to be approximately 210,000 episodes and 16,800 HAC endpoints. Therefore, a sufficient sample size to ensure stable estimates is expected. The minimum number of observations of quality improvement activities per time period is also expected to be met.

The analysis of the patient-level effect of CENSAs on the rate of HACs anticipates a small-to-moderate effect on patient outcomes (i.e. HACs). A 10% change from the baseline (of approximately 8%) is expected.^8,11 To detect even this small-to-moderate effect a total of 10,000 audited patients will be sufficient, using a type I error (alpha) of 5%, and 80% power.

Statistical power and sample size considerations for cost-effectiveness analysis require the calculation of confidence intervals for the incremental costs and effects and thus will be performed following the preliminary analyses.⁵¹ The remaining analyses will be reported descriptively and do not therefore do not require consideration of necessary sample sizes as no standardised rules exit.^52,53

Conclusion

We propose to evaluate the effectiveness of an e-A&F intervention for patient safety in tertiary hospital care in an Australian setting. Quantitative analysis of time series data, controlling for historical outcome trends, will be used to provide evidence of the intervention’s impact. Staff behaviour and views will also be used to better understand the mechanisms responsible for any change in the patient outcomes from the intervention.

This study will provide insight into e-A&F organised at a hospital-level and directed at different domains of patient care.^8,12 As an illustrative example and with many individual elements of CENSAs reproducible, results may provide a guide for other hospitals and healthcare organisations in the process of implementing and evaluating e-A&F. The results will also guide future research that aims to understand how to achieve improvements in patient safety more broadly.

Footnotes

Author contributions

JS conceptualised the project and drafted the manuscript. DP, KM and TM reviewed and edited the manuscript. All authors read and approved the final manuscript.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethical approval

The Research Governance Office of the Sir Charles Gairdner Osborne Park Health Care Group approved the study on 29 January 2020 (reference: RGS0000003231). Research governance approval was also received by the University of Western Australia (reference: RA/4/20/5794). A waiver of consent was provided to utilize information from retrospective datasets. Participants in the survey will however be individually consented.

Patient and public involvement

The CENSAs system development and implementation has been overseen by a key hospital governance committee represented by a member of the hospital Consumer Advisory Committee. Research findings, in addition to dissemination via traditional academic processes, will also be shared through non-academic processes including to the Consumer Advisory Committee and patients for their consumption while visiting hospital.

ORCID iD

James Soresi

References

Brennan

Leape

Laird

, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med 1991; 324: 370–376.

Hamilton

Gibberd

Harrison

. After the quality in Australian Health Care Study, what happened? Med J Aust 2014; 201: 23.

Wilson

Michel

Olsen

, et al. Patient safety in developing countries: retrospective estimation of scale and nature of harm to patients in hospital. BMJ 2012; 344: e832.

Australian Commission on Safety and Quality in Health Care. The state of patient safety and quality in Australian hospitals. Sydney: ACSQHC, 2019.

Duckett

Jorm

Danks

, et al. All complications should count: using our data to make hospitals safer. Melbourne: Grattan Institute, 2018.

Koff

Corwin

Beach

, et al. Reduction in ventilator associated pneumonia in a mixed intensive care unit after initiation of a novel hand hygiene program. J Crit Care 2011; 26: 489–495.

Eveillard

Raymond

Guilloteau

, et al. Impact of a multi-faceted training intervention on the improvement of hand hygiene and gloving practices in four healthcare settings including nursing homes, acute-care geriatric wards and physical rehabilitation units. J Clin Nurs 2011; 20: 2744–2751.

Ivers

Jamtvedt

Flottorp

, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2012; 6: CD000259.

Flottorp

Jamtvedt

Gibis

, et al. Using audit and feedback to health professionals to improve the quality and safety of health care. In: Belgian EU Presidency Conference on Investing in Europe’s health workforce of tomorrow: scope for innovation and collaboration, La Hulpe, 9–10 September 2010.

10.

Kiermeier

Babidge

McCulloch

GAJ

, et al. National surgical mortality audit may be associated with reduced mortality after emergency admission. ANZ J Surg 2017; 8: 830–836.

11.

Weiss

Dunn

Sprague

, et al. Effect of a population-level performance dashboard intervention on maternal-newborn outcomes: an interrupted times series study. BMJ Qual Saf 2018; 27: 425–436.

12.

Jamtvedt

Young

Kristoffersen

, et al. Does telling people what they have been doing change what they do? A systematic review of the effects of audit and feedback. Qual Saf Health Care 2006; 15: 433–436.

13.

Brown

Gude

Blakeman

, et al. Clinical Performance Feedback Intervention Theory (CP-FIT): a new theory for designing, implementing, and evaluating feedback in health care based on a systematic review and meta-synthesis of qualitative research. Implement Sci 2019; 14: 40.

14.

Hysong

Best

Pugh

. Audit and feedback and clinical practice guideline adherence: making feedback actionable. Implement Sci 2006; 1: 9.

15.

Hysong

. Meta-analysis: audit and feedback features impact effectiveness on care quality. Med Care 2009; 47: 356–363.

16.

Hysong

Teal

Khan

, et al. Improving quality of care through improved audit and feedback. Implement Sci 2012; 7: 45.

17.

Mumford

Greenfield

Hogden

, et al. Counting the costs of accreditation in acute care: an activity-based costing approach. BMJ Open 2015; 5: e008850.

18.

Braithwaite

Shaw

Moldovan

, et al. Comparison of health service accreditation programs in low- and middle-income countries with those in higher income countries: a cross-sectional study. Int J Qual Health Care 2012; 24: 568–577.

19.

Braithwaite

Westbrook

Johnston

, et al. Strengthening organizational performance through accreditation research - a framework for twelve interrelated studies: the ACCREDIT project study protocol. BMC Res Notes 2011; 4: 390.

20.

Mossialos

Djordjevic

Osborn

, et al. International profiles of health care systems. New York, NY: The Commonwealth Fund, 2017.

21.

The Joint Commission. America’s Hospitals: improving quality and safety—The Joint Commission’s Annual Report. Oakbrook Terrace, IL: The Joint Commission, 2014.

22.

Nicklin

Barton

. CCHSA accreditation: a change catalyst toward healthier work environments. Healthc Pap 2007; 7 Spec No: 58–63; discussion 109–119.

23.

Australian Commission on Safety and Quality in Health Care. National safety and quality health service standards guide for use in hospitals. Sydney: ACSQHC, 2011.

24.

Auras

Geraedts

. International comparison of nine accreditation programmes for ambulatory care facilities. J Public Health 2011; 19: 425–432.

25.

Australian Commission on Safety and Quality in Health Care. National safety and quality health service standards. 2nd ed. Sydney: ACSQHC, 2017.

26.

Bain

Bucknall

Weir-Phyland

, et al. Meeting National safety and quality health service standards – the role of the Point-of-Care (POC) audit application. IJEEEE 2013; 3: 507.

27.

Ivers

Grimshaw

Jamtvedt

, et al. Growing literature, stagnant science? Systematic review, meta-regression and cumulative analysis of audit and feedback interventions in health care. J Gen Intern Med 2014; 29: 1534–1541.

28.

Crofts

Moyo

Ndebele

, et al. Adaptation and implementation of local maternity dashboards in a Zimbabwean hospital to drive clinical improvement. Bull World Health Organ 2014; 92: 146–152.

29.

Tuti

Nzinga

Njoroge

, et al. A systematic review of electronic audit and feedback: intervention effectiveness and use of behaviour change theory. Implement Sci 2017; 12: 1–20.

30.

Australian Institute of Health and Welfare. Australian hospital peer groups. Report no. Cat. no. HSE 170, 2015. Canberra: AIHW.

31.

Australian Commission on Safety and Quality in Health Care. Hospital-acquired complications information kit. Sydney: ACSQHC, 2018.

32.

Rothman

Solinger

Rothman

, et al. Clinical implications and validity of nursing assessments: a longitudinal measure of patient condition from analysis of the Electronic Medical Record. BMJ Open 2012; 2: e000646.

33.

Cohen

Bakhai

Shi

, et al. Cost-effectiveness of sirolimus-eluting stents for treatment of complex coronary stenoses: results from the Sirolimus-Eluting Balloon Expandable Stent in the Treatment of Patients With De Novo Native Coronary Artery Lesions (SIRIUS) trial. Circulation 2004; 110: 508–514.

34.

Department of Health WA. Hospital morbidity data system reference manual part A. Perth: Department of Health WA, 2018.

35.

Independent Hospital Pricing Authority. Pricing and funding for safety and quality - risk adjustment model for hospital acquired complications. Sydney: IHPA, 2019.

36.

Karnon

Campbell

Czoski-Murray

. Model-based cost-effectiveness analysis of interventions aimed at preventing medication error at hospital admission (medicines reconciliation). J Eval Clin Pract 2009; 15: 299–306.

37.

Cohen

Reynolds

. Interpreting the results of cost-effectiveness studies. J Am Coll Cardiol 2008; 52: 2119–2126.

38.

Independent Hospital Pricing Authority. National hospital cost data collection cost report: round 20 financial year 2015-16. Sydney: IHPA, 2018.

39.

Gude

Roos-blom

van der Veer

, et al. Electronic audit and feedback intervention with action implmentation toolbox to improve pain management in intensive care: protocol for a laboratory experiment and cluster randomised trial. Implement Sci 2017; 12: 1–2.

40.

Gude

van der Veer

de Keizer

, et al. Optimizing digital health informatics interventions through unobtrusive quantitative process evaluations. In: Hoerbst

Hackl

de Keizer

, et al. (eds) Exploring complexity in health: An interdisciplinary systems approach. Amsterdam, The Netherlands: IOS Press eBooks, 2016, pp. 594–598.

41.

Australian Commission on Safety and Quality in Health Care. Hospital-acquired complications specifications 10th edition. V1.1 ed. Sydney: ACSQHC, 2018.

42.

Jackson

Michel

Roberts

, et al. Development of a validation algorithm for ‘present on admission’ flagging. BMC Med Inform Decis Mak 2009; 9: 48.

43.

Preen

Holman

Lawrence

, et al. Hospital chart review provided more accurate comorbidity information than data from a general practitioner survey or an administrative database. J Clin Epidemiol 2004; 57: 1295–1304.

44.

Wagner

Soumerai

Zhang

, et al. Segmented regression analysis of interrupted time series studies in medication use research. J Clin Pharm Ther 2002; 27: 299–309.

45.

Ramsay

Matowe

Grilli

, et al. Interrupted time series designs in health technology assessment: lessons from two systematic reviews of behavior change strategies. Int J Technol Assess Health Care 2003; 19: 613–623.

46.

Dunn

Sprague

Grimshaw

, et al. A mixed methods evaluation of the maternal-newborn dashboard in Ontario: dashboard attributes, contextual factors, and facilitators and barriers to use: a study protocol. Implement Sci 2016; 11: 59.

47.

Gillings

Makuc

Siegel

. Analysis of interrupted time series mortality trends: an example to evaluate regionalized perinatal care. Am J Public Health 1981; 71: 38–46.

48.

England

. How interrupted time series analysis can evaluate guideline implementation. Pharm J 2005; 275: 344–347.

49.

Bernal

Cummins

Gasparrini

. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol 2017; 1: 348–355.

50.

Wong

Smieliauskas

Pan

, et al. Interrupted time-series analysis: studying trends in neurosurgery. Neurosurg Focus 2015; 39: E6.

51.

Walter

Gafni

Birch

. Estimation, power and sample size calculations for stochastic cost and effectiveness analysis. Pharmacoeconomics 2007; 25: 455–466.

52.

Patton

. Qualitative research & evaluation methods. 3rd ed. Thousand Oaks, CA: Sage Publications, 2002.

53.

Kuzel

. Sampling in qualitative inquiry. In: Crabtree

Miller

(eds) Doing qualitative research. 2nd ed. Thousand Oaks, CA: Sage Publications, 1999, pp. 33–45.

An evaluation of an electronic audit and feedback system for patient safety in a tertiary hospital setting: A study protocol

Abstract

Keywords

Introduction

Study aims and objectives

Methods

Study design

Setting

Outcomes

Data sources and sample

Covariates

Data analysis

Statistical power

Conclusion

Footnotes

Author contributions

Declaration of conflicting interests

Funding

Ethical approval

Patient and public involvement

ORCID iD

References