Abstract
Although significant improvements have been achieved in the renal replacement therapy of acute kidney injury (AKI), the mortality of patients with AKI remains high. The aim of this study is to prospectively investigate the capacity of Acute Physiology and Chronic Health Evaluation version II (APACHE II), Simplified Acute Physiology Score version II (SAPS II), Sepsis-related Organ Failure Assessment (SOFA) and Acute Tubular Necrosis Individual Severity Index (ATN-ISI) to predict in-hospital mortality of critically ill patients with AKI. A prospective observational study was conducted in a university teaching hospital. 189 consecutive critically ill patients with AKI were selected according Risk, Injury, Failure, Loss, or End-stage kidney disease criteria. APACHE II, SAPS II, SOFA and ATN-ISI counts were obtained within the first 24 hours following admission. Receiver operating characteristic analyses (ROCs) were applied. Area under the ROC curve (AUC) was calculated. Sensitivity and specificity of in-hospital mortality prediction were calculated. In this study, the in-hospital mortality of critically ill patients with AKI was 37.04% (70/189). AUC of APACHE II, SAPS II, SOFA and ATN-ISI was 0.903 (95% CI 0.856 to 0.950), 0.893 (95% CI 0.847 to 0.940), 0.908 (95% CI 0.866 to 0.950) and 0.889 (95% CI 0.841 to 0.937) and sensitivity was 90.76%, 89.92%, 90.76% and 89.08% and specificity was 77.14%, 70.00%, 71.43% and 71.43%, respectively. In this study, it was found APACHE II, SAPS II, SOFA and ATN-ISI are reliable in-hospital mortality predictors of critically ill patients with AKI. Trial registration number: NCT00953992.
Keywords
Significance of this study
What is already known about this subject?
Although significant improvements have been achieved in the renal replacement therapy of acute kidney injury (AKI), the mortality of patients with AKI remains high.
What are the new findings?
Acute Physiology and Chronic Health Evaluation version II (APACHE II), Simplified Acute Physiology Score version II (SAPS II), Sepsis-related Organ Failure Assessment (SOFA) and Acute Tubular Necrosis Individual Severity Index (ATN-ISI) are reliable in-hospital mortality predictors of critically ill patients with AKI.
How might these results change the focus of research or clinical practice?
Outcome prediction is of great importance in clinical practice. APACHE II, SAPS II, SOFA and ATN-ISI severity score systems are useful tools for the prediction of patients’ outcome.
Introduction
Although significant improvements in renal replacement therapy of critically ill patients with acute kidney injury (AKI) have achieved, the mortality remains high. AKI is frequent and is associated with poor outcomes.1 AKI elevates the risk of cardiovascular mortality and major cardiovascular events including heart failure and acute myocardial infarction.2 It increases the risk of other severe life-threatening complications.
Recently, several new urine and serum biomarkers for the early diagnosis of AKI such as cystatin C, kidney injury molecule-1, neutrophil gelatinase-associated lipocalin3 4 and interleukin-18 have been found. These new biomarkers have shown promising abilities to predict the occurrence of AKI. However, they cannot predict the in-hospital mortality of patients with AKI. Biomarker alone-based strategies are costly and prone to failure because of the clinical heterogeneity displayed by individual patients.5 Prediction of in-hospital mortality of critically ill patients with AKI has become an important issue facing clinicians. We have evaluated predictive capacity of two kinds of severity scoring systems including Acute Physiology and Chronic Health Evaluation version II (APACHE II) and Acute Tubular Necrosis Individual Severity Index (ATN-ISI) to predict hospital mortality of the elderly patients with AKI.6 Simplified Acute Physiology Score version II (SAPS II) and Sepsis-related Organ Failure Assessment (SOFA) were important severity score systems in clinical practice. The aim of the study is to prospectively evaluate the predictive capacity of in-hospital mortality of APACHE II, SAPS II, SOFA and ATN-ISI in critically ill patients with AKI.
Materials and methods
In a university teaching hospital, 189 consecutive critically ill patients with AKI in an emerging country were enrolled into this prospective study. One hundred and thirty-nine were male and 50 were female. The mean age was 63.33±18.29 years old. Clinical features and laboratory tests of 189 critically ill patients with AKI were shown in table 1. The inclusion criteria were on the basis of Risk, Injury, Failure, Loss, or End-stage kidney disease (RIFLE) criteria.7 8 R stage of the RIFLE criteria were inclusion criteria for the study. Patients with chronic kidney disease before AKI, critically ill patients with AKI with hospital stay of <24 hours, critically ill patients with AKI who were caused by postrenal obstruction and kidney transplantation were excluded in this study. APACHE II, SAPS II, SOFA and ATN-ISI counts were obtained within the first 24 hours following admission. APACHE II scores were assessed according to the method presented by Knaus et al.9 SOFA scores were assessed according to the method of Vincentet al.10 SAPS II scores were assessed according to the method of Le Gall et al.11 ATN-ISI scores were assessed according to the method presented by Liaño et al.12 Patients with severe AKI were treated with continuous renal replacement therapy and intermittent hemodialysis. The primary outcome was identified as survived and non-survived before the study began. According to their final clinical outcomes, all patients were divided into survived group (n=119) and non-survived group (n=70). Clinical features and laboratory test results of the critically ill patients with AKI were collected at admitting to the hospital or within the next 24 hours.
Clinical features and laboratory test results of 189 consecutive critically ill patients with acute kidney injury (AKI)
The various parameters were obtained within the first 24 hours following admission.
ALT, alanine amino transaminase; APACHE II, Acute Physiology and Chronic Health Evaluation version II; ATN-ISI, Acute Tubular Necrosis Individual Severity Index; BUN, serum urea nitrogen; Ca, serum calcium; Cho, serum total cholesterol Cl, serum chlorine; Fc, F stage of the Risk, Injury, Failure, Loss, or End-stage kidney disease criteria; HDL, High density lipoprotein; Ic, I stage of the Risk, Injury, Failure, Loss, or End-stage kidney disease criteria; K, serum potassium; LDL, low density lipoprotein; MO, monocyte; Na, serum sodium; P, serum phosphorus; RBC, total blood count of red blood cell; RIFLE, Risk, Injury, Failure, Loss, or End-stage kidney disease; Rc, R stage of the Risk, Injury, Failure, Loss, or End-stage kidney disease criteria; SAPS II, Simplified Acute Physiology Score version II ; SCr, serum creatinine; SOFA, Sepsis-related Organ Failure Assessment; WBC, total blood count of white cell count; TB, total protein; TG, triglyceride; Ua, uric acid;
In this study, receiver operating characteristic analyses (ROC), McNemar test and Kappa test were used. By applying the software of Statistical Product and Service Solutions V.16.0, the statistical analysis was performed. By means of summary measurements (mean±SD) or median (interquartile) of quantitative variables, the results were expressed. Comparison of parameters of the four groups was conducted by using Student's t-test, and the comparison of non-parametric data of the four groups was conducted by using Mann-Whitney test. We considered p value <0.05 as statistically significant. We calculated adjusted ORs and 95% CIs. After ROC analyses, area under the ROC curve (AUC) of these four scoring systems was calculated. Sensitivity and specificity of in-hospital mortality prediction of the four scoring systems were also calculated and compared.
It conforms to the provisions of the Declaration of Helsinki in 1975 (as revised in Edinburgh 1983). Informed, written consent was obtained from every patient.
Results
In this study, the in-hospital mortality of critically ill patients with AKI was 37.04% (70/189). There were gender differences in the in-hospital mortality of patients. The mortality of male patients was 36.7% and the mortality of female was 38.0%, which was a little higher than that of male patients. ROC curves of APACHE II, SAPS II, SOFA and ATN-ISI were drawn up to assess the effectiveness of the in-hospital mortality prediction on critically ill patients with AKI, as shown in figure 1. The AUC of APACHE II, SAPS II, SOFA and ATN-ISI was 0.903 (95% CI 0.856 to 0. 950), 0.893 (95% CI 0.847 to 0.940), 0.908 (95% CI 0.866 to 0.950) and 0.889 (95% CI 0.841 to 0.937). The AUC of SOFA was the largest among the four scoring systems (as shown in table 2).
Receiver operating characteristic analyses curves of APACHE II, SAPS II, SOFA and ATN-ISI severity score system on the in-hospital mortality prediction of critically ill patients with acute kidney injury. apache2ae, Acute Physiology and Chronic Health Evaluation version II (APACHE II); lianoae, Acute Tubular Necrosis Individual Severity Index (ATN-ISI); saps, Simplified Acute Physiology Score version II (SAPS II); sofaae, Sepsis-related Organ Failure Assessment (SOFA).
Area under the receiver operating characteristic analyses curves of APACHE II, SAPS II, SOFA and ATN-ISI severity score systems on in-hospital mortality prediction of critically ill patients with acute kidney injury
APACHE II, Acute Physiology and Chronic Health Evaluation version II; ATN-ISI, Acute Tubular Necrosis Individual Severity Index; SAPS II, Simplified Acute Physiology Score version II; SOFA, Sepsis-related Organ Failure Assessment.
In this study, it was found that mean value of APACHE II, ATN-ISI, SAPS II, SOFA scores of the non-survival group was higher than that of the survival group, p<0.001, as shown in table 3. Sensitivity of hospital mortality prediction of APACHE II, SAPS II, SOFA and ATN-ISI scoring systems was 90.76%, 89.92%, 90.76% and 89.08%, respectively. Specificity of hospital mortality prediction of APACHE II, SAPS II, SOFA and ATN-ISI systems was 77.14%, 70.00%, 71.43% and 71.43%, respectively. The sensitivity of in-hospital mortality prediction of APACHE II and SOFA were the highest. Specificity of in-hospital mortality prediction of APACHE II was the highest (as shown in table 4). The predicted mortality rate of APACHE II, ATN-ISI, SAPS, and SOFA was shown in table 5.
Mean value of APACHE II, SAPS II, SOFA and ATN-ISI severity score systems scores of survival group and non-survival group of critically ill patients with acute kidney injury
APACHE II, Acute Physiology and Chronic Health Evaluation version II; ATN-ISI, Acute Tubular Necrosis Individual Severity Index; SAPS II, Simplified Acute Physiology Score version; SOFA, Sepsis-related Organ Failure Assessment.
Sensitivity and specificity of in-hospital mortality prediction of APACHE II, SAPS II, SOFA and ATN-ISI severity score systems on critically ill patients with acute kidney injury
APACHE II, Acute Physiology and Chronic Health Evaluation version II; ATN-ISI, Acute Tubular Necrosis Individual Severity Index; SAPS II, Simplified Acute Physiology Score version II; SOFA, Sepsis-related Organ Failure Assessment.
Predicted in-hospital mortality rate and real in-hospital mortality rate of APACHE II, SAPS II, SOFA and ATN-ISI severity score systems on critically ill patients with acute kidney injury
APACHE II, Acute Physiology and Chronic Health Evaluation version II; ATN-ISI, Acute Tubular Necrosis Individual Severity Index; SAPS II, Simplified Acute Physiology Score version II; SOFA, Sepsis-related Organ Failure Assessment.
Discussion
In this study, capacity of in-hospital mortality prediction on critically ill patients with AKI of APACHE II, SAPS II, SOFA and ATN-ISI was compared. AUC of the four systems were all larger than 0.88, which suggested that the four systems have the capacity of prediction of in-hospital mortality on critically ill patients with AKI.
Critically ill patients with AKI are associated with significantly extends hospitalizations and increased mortality despite prevention efforts and improvements in therapeutics. Previous research found that in patients of intensive care unit (ICU), the in-hospital mortality of AKI was averaging 36.4%.13 In Italy, it was reported that the in-hospital mortality of critically ill patients with AKI was 39.1% in an intermediate nephrology care unit,14 which was a little higher than that of our study.
Renal function of patients on admission seems to be a significant independent prognostic factor for long-term mortality and new cardiovascular morbidity over a 10-year period.15 In a study on the performance of SAPS II, APACHE II, Logistic Organ Dysfunction, Organ Dysfunctions and/or Infection, ATN-ISI (Liano) and Mehta in predicting in-hospital mortality of critically ill patients with AKI, in-hospital mortality rate was found to be 75%,16 which was much higher than that of our research (37.04%). This is because the data in their study come from a developing country that owned less advanced technologies and equipment. In their study, mean APACHE II scores were 27.4±6.3 points and mean SAPS II scores were 48.5±11.2 points, which were much higher than that of our research, which was 16.00 (12.0, 26.0), 43.0 (31.0, 60.5), respectively. This confirmed that renal function on admission and the severity of critically ill patients with AKI in their study were more serious than that of our study.
The ROC is a comprehensive index that reflects the sensitivity and specificity of continuous variables. It reveals the relationship between sensitivity and specificity by mapping method. The greater the area under the curve, the higher diagnostic accuracy is. In the study of Maccariello et al,16 AUC of SAPS II was 0.72, and they concluded that general severity score system and AKI special severity score system were inaccurate in predicting outcomes of critically ill patients with AKI in ICU who needed renal replacement therapy (RRT). This result is quite different from our research that demonstrated that APACHE II, SAPS II, SOFA and ATN-ISI are useful for evaluating the prognosis of critically ill patients with AKI. Perhaps this is because the data of their study came from the critically ill patients with AKI and need for RRT in ICU. The clinical conditions of these patients were more serious.
ATN-ISI belongs to AKI special severity score system that mainly focus on scoring the severity of AKI and evaluates the risk of death of AKI itself completely. ATN-ISI is a linear model for analysis of hospital mortality of AKI, and the score is derived from the physical signs and laboratory test results of the disease. ATN-ISI was originally derived from the statistics of acute tubular necrosis and the parameters of AKI. ATN-ISI was easy to obtain and had good versatility.12 These score indices (gender, exposure to renal toxicity and oliguria) of ATN-ISI focus on the evaluation of the severity of AKI. Time points for parameter collection will lead to large differences in scores of ATN-ISI. For example, when patients use sedatives, analgesics and anesthetics, it will affect the evaluation of scores of coma and will also leads to the differences of final evaluation scores of ATN-ISI. A lot of factors affected the prognosis of critically ill patients with AKI. In addition to the main parameters in ATN-ISI system, serum urea, creatinine concentration, heart rate, serum albumin level, central venous pressure, coagulation time, immune system failure and hemodialysis model were the important influence factors. In fact, there are fewer parameters to evaluate these important influence factors in ATN-ISI that lead to the lack of comprehensiveness of ATN-ISI.
APACHE II system assigned little index scores to renal failure (AKI)9 and inclined to evaluation of the score indices such as blood coagulation, infection, anemia, blood concentration, arrhythmia, electrolyte, acid-base metabolism and so on. APACHE II takes into account the influence on prognosis of surgery, circulatory failure, nervous system disease, gastrointestinal bleeding, chronic liver disease, chronic kidney disease and tumor. SAPS II system was proposed by Le Gall et al on the basis of SAPS I in 1984.17 SAPS II was built on the foundation of a large sample of patients that included data from consecutive admissions to 13 152 patients in 137 ICUs in 12 countries. It includes 12 physiology variables and three underlying disease variables (AIDS, metastatic cancer and hematologic malignancy).11 SOFA is a relatively independent severity score system compared with the APACHE II and SAPS II, and the correlations among them were not significant. SOFA score was based on the degree of organ dysfunction of patients and was used in clinical practices to assess disease's severity and predict mortality and in clinical research. It includes serum creatinine and assigns more points to higher creatinine values, and as such, the scoring system and stages of AKI are directly connected. For this reason, previous authors have used scoring systems like SOFA. In the study of Carbonell et al,18 SOFA score was found to be a useful tool to categorize critically ill patients with AKI and to describe a sequence of complications of critically ill patients with AKI in ICU. In our study, it was found that the AUC of SOFA system was 0.908 (95% CI 0.866 to 0.950), which was the largest among the four scoring systems. AUC of SOFA system was bigger than that of SAPS II system, and the capacity to predict hospital mortality of critically ill patients with AKI was better than that of SAPS II. These results were similar to that of the study of Janssens et al,19 which concluded that SOFA and delta SOFA scores (total maximum SOFA score minus admission total SOFA) can assess the degree and progression of organ dysfunction and SOFA score was more important in the prediction of prognosis of critically ill patients with AKI than SAPS II scores. In a prospective study of 949 ICU patients, SOFA was found to be a reliable outcome predictors.20 Ferreira et al found that the mean and highest SOFA scores are particularly useful in prediction of prognosis in critically ill patients.21 In a prospective, multicenter study in ICU, it was found that SOFA score system had a good ability to evaluate organ dysfunction of patients.22 The accuracy of SOFA score system among clinical physicians was found good.23
Severity score systems such as APACHE II, SAPS II and SOFA belong tosystems that were usually derived from unselected ICU critically ill patients, it mainly focused on the general severity of diseases of patients and it was based on several organ failure sub-scores. In our study, AUC of APACHE II, SAPS II, SOFA and ATN-ISI was 0.903 (95% CI 0.856 to 0.950), 0.893 (95% CI 0.847 to 0.940), 0.908 (95% CI 0.866 to 0.950) and 0.889 (95% CI 0.841 to 0.937). AUC of APACHE II, SAPS II and SOFA were larger than that of ATN-ISI.
There are also some opposite results from other study. For example, Fiaccadori et al14 compared three general scoring system including APACHE II, SAPS II and Mortality Probability Model 24 II (MPM 24 II) and found APACHE II model was a slightly better calibrated predictor of group outcome in patients with AKI, as compared with SAPS II and MPM 24 II outcome prediction models and concluded that none of the models provided sufficient confidence for the prediction of outcome in individual patient. The result was different from that of ours since their study was conducted in an intermediate nephrology care unit and did not enroll the critically ill patients with AKI in other clinical departments such as department of cardiology, department of surgery and so on. So the results of their study were limited. The comparison of the main studies mentioned above can be seen in table 6.
Comparison of main results of studies on AKI mentioned in the discussion
AKI, acute kidney injury; APACHE II, Acute Physiology and Chronic Health Evaluation version II; ATN-ISI, Acute Tubular Necrosis Individual Severity Index; AUC, area under the receiver operating characteristic analyses; ICUs, intensive care units; SAPS II, Simplified Acute Physiology Score version II; SOFA, Sepsis-related Organ Failure Assessment.
In clinical practice, it is ideal to diagnose by using the gold standard, but restricted by the objective conditions of patients, clinicians sometimes cannot use the gold standard to diagnose and to make outcome prediction of patients. These severity score systems are useful tools for the prediction of patients’ outcome.
Conclusions
APACHE II, SAPS II, SOFA and ATN-ISI are reliable in-hospital mortality predictors of critically ill patients with AKI.
Footnotes
Contributors
All authors equally contributed to the conception and design of the research; YuG contributed to the design of the research; FD contributed to the acquisition and analysis of the data; YuG and FZ contributed to the analysis of the data; YoG contributed to the acquisition, analysis and interpretation of the data. All authors drafted the manuscript, critically revised the manuscript, agree to be fully accountable for ensuring the integrity and accuracy of the work and read and approved the final manuscript.
Funding
The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests
None declared.
Patient consent for publication
Not required.
Ethics approval
The protocol for this research project has been approved by the Ethics Committee of Huashan Hospital, Fudan University and the approval number is 2009-206.
Provenance and peer review
Not commissioned; externally peer reviewed.