Utilising routine clinical laboratory data to support quality improvement in health care: Application of a national acute kidney injury alert system as a proof of concept

Abstract

Background

Acute kidney injury (AKI) is a global health issue known to cause avoidable harm and death. Improvement in its prevention and management is therefore considered an important goal for the health-care sector. The work here aimed to develop a tool which could be used to robustly and reliably measure, monitor, and compare the effectiveness of health-care interventions related to AKI across the Welsh NHS, a mechanism which did not exist previously.

Methods

Using serum creatinine (SCr) as a biomarker for AKI and a validated national data-set collected from the all Wales Laboratory Information Management System, work involved applying Donabedian’s framework to develop indicators with which to measure outcomes related to AKI, and exploring the potential of statistical process control (SPC) techniques for analysing data on these indicators.

Results

Rate of AKI incidence and 30-day AKI-associated mortality are proposed as valid, feasible indicators with which to measure the effectiveness of health-care interventions related to AKI. The control chart, funnel plot, and Pareto chart are proposed as appropriate, robust SPC techniques to analyse and visualise variation in AKI-related outcomes.

Conclusions

This work demonstrates that routinely collected large SCr data offer a significant opportunity to monitor and therefore inform improvement in patient outcomes related to AKI. Moreover, while this work concerns utilisation of SCr data for improvement in AKI strategies, it is a proof of concept which could be replicated for other routinely collected clinical laboratory data, to improve the prevention and/or management of the conditions to which they relate.

Keywords

Acute kidney injury serum creatinine statistical process control quality improvement value-based health care outcomes

Background (context and rationale)

Acute kidney injury (AKI) is a global health issue known to cause avoidable harm and death. Improvement in its prevention and management is therefore reported as an important goal for the health-care sector, and a clinical and research priority.¹ By utilising routine clinical laboratory data, this work aimed to develop objective indicators and a tool with which to reliably (i) ascertain a baseline for potential improvement in the prevention and management of AKI and (ii) measure subsequent improvement in the prevention and management of AKI following the introduction of any change(s) in practice.

The work in this paper follows a National Confidential Enquiry into Patient Outcome and Death² which identified significant systemic deficiencies in the management of AKI across the UK and led to the development of new laboratory and clinical strategies. For the National Health Service in Wales (NHSW), this involved the implementation of an automated laboratory-based electronic alerting system for AKI, based on the assumption that this would aid early recognition and intervention, and thereby improve patient outcomes.³ The improvement science literature states that without measurement, improvement is not possible.⁴ It also states that with quality improvement (QI), improvement is the goal of measurement.⁵ Without measurement, it is impossible to determine if, and to what extent, interventions and changes in practice improve quality. If the effectiveness of health-care interventions related to AKI in Wales was to improve, then the improvement needed to be measurable.

In addition to its clinical and primary application, the aforementioned AKI alert system, which had been implemented by all laboratories in NHSW, also had the potential to be applied as a centralised system of data collection, to systematically collect national data on AKI. While this new large data source may have already been utilised to better understand the epidemiology of AKI, including by describing AKI identified in different patient populations and health-care settings,^6–12 and exploring its association with various risk factors,^13–17 it also offered further opportunity in the QI arena. To demonstrate this additional benefit, the work described here aimed to develop a tool which could be used to robustly and reliably measure, monitor, and compare the effectiveness of health-care interventions related to AKI across the entire Welsh NHS. A mechanism like this did not exist prior and may therefore offer a significant opportunity to inform strategies aimed at improving the prevention and/or management of AKI in Wales.

Methods (type of study, patients, materials, and techniques)

Development of data-set

Approved under ‘Service Evaluation Registration’, this work utilised the aforementioned validated national data-set described by the authors previously,⁷ which includes those fields listed in Table 1 and was produced using processes outlined in Figure 1 and Table 2. The data-set is based on electronic AKI alerts triggered by an algorithm built within the Welsh Laboratory Information Management System (WLIMS) (‘InterSystems TrakCare Lab’) and detects possible incident cases of AKI using one of three rules. These rules apply the ‘Kidney Disease Improving Global Outcomes’ diagnostic criteria for AKI¹⁸ and are based on absolute or relative increases in Serum Creatinine (SCr) from baseline values that have been drawn from the previous 48 h (Rule 1), 7 days (Rule 2), or 8–365 days (Rule 3). By using the same methods adopted by the authors previously to define and classify the clinical setting of AKI alerts,⁷ the data-set developed for this work thus includes all cases of AKI identified in the adult Welsh population for the period of 01 April 2015 to date, including those which are diagnosed in the hospital and community setting.

Table 1.

List of fields included in national AKI data-set developed by the authors.

Field	Source
NHS number	WLIMS
Gender	WLIMS
Date of birth	WLIMS
Age	WLIMS
Result date	WLIMS
SCr value	WLIMS
RV1	WLIMS
RV2	WLIMS
RV ratio	WLIMS
Baseline SCr	WLIMS
AKI alert code	WLIMS
AKI stage	WLIMS
AKI rule	WLIMS
Location code	WLIMS
Location description	WLIMS
Patient type code	WLIMS
Patient type description	WLIMS
Clinical speciality code	WLIMS
Clinical speciality description	WLIMS
Postcode	WDS
Date of death	WDS
LSOA	WDS
WIMD rank	WIMD
NHS Wales organisation	Calculated
NHS Wales site	Calculated

WLIMS, Welsh Laboratory Information Management system; SCr, Serum Creatinine; RV, Reference Value; WDS, Welsh Demographic Service; LSOA, Lower Super Output Area; WIMD, Welsh Index for Multiple Deprivation.

Figure 1.

Process developed by the authors to extract raw data on AKI alerts from the WLIMS. CSV, comma-separated values (a text file format); WLIMS, Welsh Laboratory Information Management system; WDS, Welsh Demographic Service; WIMD, Welsh Index for Multiple Deprivation.

Table 2.

Inclusion and exclusion criteria developed by the authors to validate and cleanse extracted data. Data fields indicated by italics.

Exclude from the data-set if:
1. SCr value is invalid (as defined by blank or not numeric data type), 2. NHS number is invalid (as defined by blank or not equalling 10 digits), 3. Gender is invalid (as defined by not ‘M’ or ‘F’), 4. Date of birth is invalid (as defined by blank or later than date of SCr result and/or not formatted as ‘dd/mm/yyyy’), 5. Date of death is invalid (as defined by earlier than date of SCr result and/or not formatted as ‘dd/mm/yyyy’), 6. Location code is invalid (as defined by blank or ‘Unknown’), 7. Alert code is ‘SAKI’, 8. Patient type description is ‘Ante-natal’, ‘Renal Dialysis’, ‘Renal’, or ‘Renal and Transplant’, 9. Alert code is ‘ABS1’, or 10. Alert code is ‘DELTA1’ and SCr value > 400 µmol/L

Exclude from the data-set if:

1. SCr value is invalid (as defined by blank or not numeric data type),
2. NHS number is invalid (as defined by blank or not equalling 10 digits),
3. Gender is invalid (as defined by not ‘M’ or ‘F’),
4. Date of birth is invalid (as defined by blank or later than date of SCr result and/or not formatted as ‘dd/mm/yyyy’),
5. Date of death is invalid (as defined by earlier than date of SCr result and/or not formatted as ‘dd/mm/yyyy’),
6. Location code is invalid (as defined by blank or ‘Unknown’),
7. Alert code is ‘SAKI’,
8. Patient type description is ‘Ante-natal’, ‘Renal Dialysis’, ‘Renal’, or ‘Renal and Transplant’,
9. Alert code is ‘ABS1’, or
10. Alert code is ‘DELTA1’ and SCr value > 400 µmol/L

SCr, Serum Creatinine; SAKI, Suspected AKI; ABS, Absolute.

Formulation of quality indicators

While several classifications for quality of care exist, Donabedian’s framework¹⁹ is the most widely accepted. It classifies quality indicators as structural, process, or outcome and theoretically hypothesises that ‘good’ structure increases the likelihood of ‘good’ process, which in turn increases the likelihood of ‘good’ outcome. Given there is limited evidence linking structure to outcomes,²⁰ a combination of outcome and process indicators is however recommended in practice by most.^21,22 Hence, while the omission of a process-based indicator in the work here may not have been ideal, the National Institute for Health and Care Excellence (NICE) advises that these types of indicators should only be measured if linked by scientific evidence to improved outcome.²³ Studies indicate that if the occurrence of what is being measured is common, and measurement of whole system performance is possible, outcome indicators are the most powerful detectors of differences in quality.^5,24 The following outcome-based indicators were therefore selected as valid, reliable, and feasible, to measure the effectiveness of health-care interventions related to AKI. For a given time period, these could be calculated as follows:

AKI incidence rate (per 1000 population at risk) = [∑(Incident AKI episodes)/∑(Population at risk)]*1000.

30-day AKI-associated mortality rate (% of AKI episodes) = [∑(Incident AKI episodes resulting in death within 30 days)/∑(Incident AKI episodes)]*100.

AKI incidence could be used to measure the effectiveness of interventions employed for patients at risk of developing AKI and therefore to measure improvement from strategies related to the prevention of AKI. Acute kidney injury-associated 30 day mortality could be used to measure the effectiveness of interventions employed in patients post-confirmed AKI and measure any systematic improvement following the implementation of strategies related to AKI management. It is also important for health-care providers to be aware of the number of patients developing AKI and who therefore require post-AKI care.²⁵

Stratification of data

To overcome potential ‘Simpson’s Paradox’, the disappearance of trends in groups of data when groups are inappropriately combined,²⁶ and which can affect the results of comparative and non-comparative analyses, the data were split. This enabled analysis of the data by the following strata:

1. Clinical setting of AKI (all-cause, hospital acquired [HA]-, or community acquired [CA]-),

2. severity of AKI (Stages 1, 2, and/or 3),

3. local health board (for which there are 7 in Wales),

4. hospital (for which there are 19 in Wales),

5. clinical speciality (for which there are 86 listed in the WLIMS), and

6. for HA-AKI only:

• Ward location (for which there are >1000 listed in the WLIMS) and

7. For CA-AKI only:

• Primary care cluster (for which there are 64 in Wales) and/or

• GP practice (for which there are approximately 400 in Wales).

While lower level data have the potential to provide more granular information in some cases, Mainz suggests outcome indicators become less useful as the number of data points reduces and the perspective narrows.²¹ Moreover, while not negating any potential benefits from stratifying by other possible ‘variables of interest’, the ‘national’ tool developed here is predominantly intent on measuring clinically important outcomes, mostly at the provider level.

Statistical process control methodology

Developed from traditional quality management theory and seminal work of Deming,⁴ statistical process control (SPC) methods are central to most QI methodology and through scientific analysis of data on quality indicators over time, enable the identification and investigation of variation in quality. They can therefore be used to evaluate the impact of, and identify opportunities for, QI. To analyse data on the two indicators described above, the authors explored and compared a number of such techniques, including control charts, funnel plots, and Pareto charts.

Results (main numerical data and statistical information)

Figure 2 includes screenshots of the prototype tool developed in Microsoft Excel software. The tool includes a series of interactive control charts, funnel plots, and Pareto charts which dynamically display data depending on selections made by the end-user using the dropdown menus pictured.

Figure 2.

Screenshots of prototype tool developed by the authors.

Control charts

Although a simple run chart may have offered an alternative SPC method for monitoring data on the effectiveness of health-care interventions related to AKI in Wales over time, Figure 3 shows control charts developed by the authors to plot data on the two quality indicators described above. While these example charts plot national all-cause AKI incidence and mortality rates, similar could also be produced for other variables which may be of interest, including any combination of the stratums listed earlier. Moreover, while Figure 3 is based on monthly time series data, also possible are similar plots based on the analysis of quarterly time series data.

Figure 3.

Sample control charts developed by the authors to monitor the effectiveness of health-care interventions related to AKI: (a) AKI incidence rate per month; (b) 30-day AKI-associated mortality rate per month; (c) AKI incidence rate for a selected variable of interest versus all Wales AKI incidence rate.

Funnel plots

Although overlaying data for a selected variable of interest onto the national mean using a control chart may have offered a method for comparing data on the effectiveness of health-care interventions related to AKI (Figure 3(c)), this is not suitable for comparing multiple data series simultaneously. Figure 4 shows funnel plots developed by the authors to plot data on the two outcome indicators described above. While these example charts compare the rates of HA-AKI incidence and mortality at different local health boards and hospitals, similar could also be produced for other variables which may be of interest. This could, for instance, include plotting by ward location or clinical speciality within individual hospitals, or for CA-AKI, plotting by primary care cluster and/or GP practice either for all of Wales or within individual local health boards. Moreover, while Figure 4 is based on data for a quarterly time period, similar plots based on the analysis of longer or shorter time periods, such as years, months, or seasons, are also possible.

Figure 4.

Sample funnel plots developed by the authors to compare the effectiveness of health-care interventions related to AKI: Comparison of HA-AKI incidence rates at local health boards (a) and hospitals (b). Comparison of 30-day HA-AKI-associated mortality rates at local health boards (c) and hospitals (d). The upper and lower bounds of the Poisson distribution for the funnel plots presented in (c) and (d) were calculated using Visual Basic programming routines made available by Pezzullo (StatPages, 2021). These show how control limits can be estimated for hypothetical population sizes starting from zero and results in more ‘smoothed’ looking funnels compared to those in (a) and (b).

Pareto charts

Based on the ‘Pareto principle’ and ‘law of the vital few’,²⁷ Pareto charts plot the frequency and cumulative frequency of an event against a selected variable of interest, and can be used to identify factors associated with the most, or highest proportions of an event. Figure 5 shows Pareto charts developed by the authors to plot data on the two outcome indicators described above. While these example charts rank national HA-AKI incidence and mortality by clinical speciality, similar could also be produced for other ‘relevant’ variables which may be of interest, and which do not necessarily relate to location of care. This could, for instance, include plotting by demographic factors such as patient age, gender and/or socioeconomic status, and/or other lower level data such as day and time of presentation. Moreover, while Figure 5 is based on data for a quarterly time period, also possible are similar plots based on the analysis of longer or shorter time periods, such as years, months, or seasons.

Figure 5.

Sample Pareto charts developed by the authors to investigate potential variables associated with variation in AKI-related outcomes: (A) Number and % of HA-AKI incident episodes by clinical speciality; (B) number and % of 30-day HA-AKI-associated deaths by clinical speciality.

Conclusions (main objective and verifiable conclusions)

The tool presented in this paper has used routine clinical laboratory data to develop a mechanism which could be used to robustly and reliably inform QI in health-care interventions related to AKI across the NHS in Wales. The tool proposes the adoption of:

1. The following outcome indicators to measure the effectiveness of health-care interventions related to AKI:

• Rate of AKI incidence, for patients at risk of developing AKI and therefore to measure the impact of strategies to improve the prevention of AKI and

• rate of AKI-associated mortality, in patients who develop AKI and therefore measure the impact of strategies to improve the management of AKI,

2. The following SPC techniques to analyse and graphically illustrate variation in the effectiveness of health-care interventions related to AKI:

• The control chart, to identify variation in rates of AKI incidence and AKI-associated mortality over time,

• the funnel plot, to identify variation in rates of AKI incidence and AKI-associated mortality between different health-care providers, and

• the Pareto chart, to identify potential causes of variation in rates of AKI incidence and AKI-associated mortality.

With capacity to identify areas of differing relative outcomes, the tool has potential to facilitate the sharing of good and/or best practice to areas where differences in outcomes exist. While the tool could be used to identify areas with most need and/or potential for QI, and therefore target interventions where they may have greatest impact, the tool could also be used to evaluate the impact of any intervention as an ongoing local self-assessment of effectiveness. To demonstrate this potential, the authors refer to the published Holmes et al. (2020) study.²⁸ Analysing the data of 132,599 patients with reported AKI, the described study evaluated whether implementation of the aforementioned national AKI alerting system was associated with changes in the number of patients in Wales developing AKI and/or changes in their outcome following AKI. By employing techniques and indicators similar to those applied by the tool presented in this paper, the published study is an example of the potential to evaluate further national and/or local QI initiatives.

While this work has centred on the utility of routine SCr data for QI in health-care interventions related to AKI, the authors present the work as a wider proof of concept which could be applied elsewhere. This includes scalability to other LIMS systems and populations, including at the national and/or more local level such as NHS Trust, Health Board, Clinical Commissioning Group, and/or hospital site. The work here is not only transferrable to health-care systems outside Wales, but it is also applicable to other common preventable and/or reversible health conditions which are not necessarily acute or related to the kidney, but which like AKI, are identifiable by routine laboratory biomarkers. Whereas this work utilises SCr as a biomarker for AKI and often subsequent chronic kidney disease, there are a number of other routinely undertaken laboratory-based quantitative investigations to which the authors anticipate an approach similar to that presented here could be applied. For instance, given non-alcoholic fatty liver disease (NAFLD) is speculated as becoming a major disease burden on NHS resources,²⁹ collecting, analysing, and presenting data on the aspartate aminotransferase (AST): alanine aminotransferase (ALT) ratio, which is used to identify suspected liver fibrosis and which can lead to chronic and serious liver disease, could help monitor the effectiveness of clinical interventions developed to reduce its incidence and/or its deterioration to a chronic condition. Another example could include Haemoglobin A1c (HbA1c), which is used to identify patients at risk of developing diabetes mellitus at a stage which could reverse their deterioration in glucose homeostasis. Replicating the concept presented here for these routine clinical laboratory data could help inform improvement in the prevention and management of the conditions to which they relate. Also, whereas this work has used internal benchmarks to identify variation in outcomes, defined as means derived from within the data-set itself, other work could use benchmarks which are based on external criteria as points of comparison. This could, for instance, include plotting against minimum care standards which, unlike for AKI, may exist for other conditions.

From a theoretical standpoint, the potential application of big data bioinformatics for health care is well documented,^30,31 and this work adds to the growing body of evidence of its utility. In highlighting the potential benefits of centralised laboratory-based data-sets which span the entire health-care system for supporting service improvement, the current work builds on data published by Shine and Barth³² and in the specific field of AKI, Sawhney and Fraser.³³ It also adds to the wider conversation regarding the emerging role of large databases, not only for epidemiological research and QI but other areas also, including disease surveillance and pandemic management,³⁴ intelligent population-based commissioning,³⁵ and population-specific reference ranges. Moreover, while the focus here may have been QI, it is likely data-sets and methods similar to those described in this paper could also inform policymaking and resource allocation for AKI. And while the work described may involve some collaboration and availability of staff to provide data and maintain SPC charts, by utilising data that are already routinely collected for clinical purposes, the proof of concept presented requires minimal additional resource. It therefore offers a cost-effective method for driving and measuring the impact of service improvements and therefore aligns with a value-based health-care approach.

As with all research, the conclusions of this work must be qualified by its limitations. These mostly relate to the AKI alert system which it has applied. Because the system is information technology driven and lacks linkage to other data-sets, it therefore lacks intelligence and does not apply any clinical context. The nature of its algorithm is also such that any patient without a measurement of SCr recorded in the previous 365 days is not included in the data-set developed for this work. The coding of certain parameters within WLIMS, such as clinical speciality, could have an effect on the data analysis and become apparent at the lower levels of granularity. Other issues which can also impact the reliability of implementing SPC methods in the health-care setting include variability in catchment populations and therefore case mix across the service, and variation in data quality and missing data across the service. Despite the described limitations, the authors maintain the view that once validated, and therefore able to be considered reliable, this approach offers a significant opportunity to utilise large data to inform potential QI in care.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethical approval

Work was approved under NHS Wales Service Evaluation Project Registration (Ref: CT/428/14).

Guarantor

Jennifer Holmes.

Contributorship

AOP and JG set up the programme of work. JH developed the data-set and conducted the analysis under supervision of AOP and JG. JH and JG wrote the manuscript. AOP and RH reviewed the manuscript.

ORCID iD

Jennifer Holmes

References

Kashani

Rosner

Haase

, et al. Quality improvement goals for acute kidney injury. Clin J Am Soc Nephrol 2019; 14(6): 941–953. DOI: 10.2215/CJN.01250119.

National Confidential Enquiry into Patient Outcome and Death (NCEPOD) . Acute kidney injury: adding insult to injury. London: NCEPOD, 2009.

Feehally

Gilmore

Barasi

, et al. RCPE UK consensus conference statement: management of acute kidney injury: the role of fluids, e-alerts and biomarkers. J R Coll Physicians Edinb 2013; 2013(43): 37–38. DOI: 10.4997/JRCPE.2013.109.

Deming

. Out of the crisis. Cambridge, MA: Center for Advanced Engineering Study, MIT, 1986.

Palmer

. Using health outcomes data to compare plans, networks and providers. Int J Qual Health Care 1998; 10(6): 477–483. DOI: 10.1093/intqhc/10.6.477.

Gubb

Holmes

Smith

, et al. Acute kidney injury in children based on electronic alerts. J Pediatr 2019; 220: 14–20.e4. DOI: 10.1016/j.jpeds.2019.11.019.

Holmes

Rainer

Geen

, et al. Acute kidney injury in the era of the AKI e-alert. Clin J Am Soc Nephrol 2016; 11(12): 2123–2131. DOI: 10.2215/CJN.05170516.

Holmes

Allen

Roberts

, et al. Acute kidney injury electronic alerts in primary care: findings from a large population cohort. QJM 2017a; 110(9): 577–582. DOI: 10.1093/qjmed/hcx080.

Holmes

Geen

Phillips

, et al. Community acquired acute kidney injury: findings from a large population cohort. QJM 2017b; 110(11): 741–746. DOI: 10.1093/qjmed/hcx151.

10.

Holmes

Roberts

May

, et al. The incidence of pediatric acute kidney injury is increased when identified by a change in a creatinine-based alert. Kidney Int 2017c; 92(2): 432–439. DOI: 10.1016/j.kint.2017.03.009.

11.

Holmes

Roberts

Geen

, et al. Utility of electronic AKI alerts in intensive care: a national multicentre cohort study. J Crit Care 2018a; 44: 185–190. DOI: 10.1016/j.jcrc.2017.10.024.

12.

Jones

Holmes

Stephens

, et al. Utility of electronic AKI alerts to define the epidemiology of acute kidney injury in renal transplants. J Nephrol 2020; 34(3): 829–838. DOI: 10.1007/s40620-020-00869-z.

13.

Holmes

Rainer

Geen

, et al. Adding a new dimension to the weekend effect: an analysis of a national data set of electronic AKI alerts. QJM 2018b; 111(4): 249–255. DOI: 10.1093/qjmed/hcy012.

14.

Holmes

Geen

Williams

, et al. Recurrent acute kidney injury: predictors and impact in a large population based cohort. Nephrol Dial Transplant 2019b; 35(8): 1361–1369. DOI: 10.1093/ndt/gfz155.

15.

Phillips

Holmes

Davies

, et al. The influence of socioeconomic status on presentation and outcome of acute kidney injury. QJM 2018; 111(12): 849–857. DOI: 10.1093/qjmed/hcy180.

16.

Phillips

Young

Holmes

, et al. Seasonal pattern of incidence and outcome of acute kidney injury: a National study of Welsh AKI electronic alerts. Int J Clin Pract. 2017; 71(9): e13000. DOI: 10.1111/ijcp.13000.

17.

Holmes

Phillips

Donovan

, et al. Acute kidney injury, age and socioeconomic deprivation: evaluation of a national data set. Kidney Int Rep 2019a; 4(6): 824–832. DOI: 10.1016/j.ekir.2019.03.009.

18.

Kidney Disease: Improving Global Outcomes (KDIGO) Acute Kidney Injury Work Group . KDIGO clinical practice guideline for acute kidney injury. Kidney Int Suppl 2012; 2(1): 1–138.

19.

Donabedian

. The quality of care. How can it be assessed? JAMA 1988; 260(12): 1743–1748. DOI: 10.1001/jama.260.12.1743.

20.

Brook

McGlynn

Shekelle

. Defining and measuring quality of care: a perspective from US researchers. Int J Qual Health Care 2000; 12(4): 281–295. DOI: 10.1093/intqhc/12.4.281.

21.

Mainz

. Methodology matters: defining and classifying clinical indicators for quality improvement. Int J Qual Health Care 2003a; 15(6): 523–530. DOI: 10.1093/intqhc/mzg081.

22.

Mainz

. Developing evidence-based clinical indicators: a state of the art methods primer. Int J Qual Health Care 2003b; 15(1): i5–i11. DOI: 10.1093/intqhc/mzg084.

23.

National Institute for Health and Care Excellence . NICE guideline 148 [NG148]: acute kidney injury: prevention, detection and management. London: NICE, 2018.

24.

Mant

. Process versus outcome indicators in the assessment of quality of health care. Int J Qual Health Care 2001; 13(6): 475–480. DOI: 10.1093/intqhc/13.6.475.

25.

Liu

Forni

Heung

, et al. Quality of care for acute kidney disease: current knowledge gaps and future directions. Kidney Int Rep 2020; 2020(5): 1634–1642. DOI: 10.1016/j.ekir.2020.07.031.

26.

Simpson

. The interpretation of interaction in contingency tables. J R Stat Soc Ser B Methodol 1951; 13(2): 238–241. DOI: 10.1111/j.2517-6161.1951.tb00088.x.

27.

Juran

. A quality trilogy: a universal approach to managing for quality. Qual Prog 1986; 19(August): 19–24. Available at: juran.com (Accessed: April 2021).

28.

Holmes

Donovan

Geen

, et al. Acute kidney injury demographics and outcomes: changes following introduction of electronic acute kidney injury alerts-an analysis of a national dataset. Nephrol Dial Transplant 2020; 36(8): 1433–1439. DOI: 10.1093/ndt/gfaa071.

29.

Lazarus

Colombo

Cortez-Pinto

, et al. NAFLD - sounding the alarm on a silent epidemic. Nat Rev Gastroenterol Hepatol 2020; 17: 377–379. DOI: 10.1038/s41575-020-0315-7.

30.

de Lusignan

Hague

van Vlymen

, et al. Routinely-collected general practice data are complex, but with systematic processing can be used for quality improvement and research. Inf Prim Care 2006; 14(1): 59–66. DOI: 10.14236/jhi.v14i1.615.

31.

Raghupathi

. Big data analytics in healthcare: promise and potential. Health Inf Sci Syst 2014; 2: 3. (Accessed: April 2021). DOI:10.1186/2047-2501-2-3.

32.

Shine Barth

. Big data in clinical biochemistry. Ann Clin Biochem 2019; 56(3): 308–309. DOI: 10.1177/0004563218800735.

33.

Sawhney

Fraser

. Epidemiology of AKI: utilizing large databases to determine the burden of AKI. Adv Chron Kidney Dis 2017; 24(4): 94–204. DOI: 10.1053/j.ackd.2017.05.001.

34.

Alamo

Reina

Millán Gata

, et al. Data-driven methods for present and future pandemics: monitoring, modelling and managing. Annu Rev Control 2021; 52: 448–464. DOI: 10.1016/j.arcontrol.2021.05.003.

35.

Dunn

de Belder

. Using data to drive service improvement: false dawns and a promised land? Future Healthc J 2022; 9(2): 150–153. DOI: 10.7861/fhj.2022-0058.