Abstract
Introduction:
Very little artificial intelligence (AI) work has been performed to investigate acetaminophen-associated hepatotoxicity. The objective of this study was to develop an AI algorithm for analyzing weighted features for toxic hepatitis after acetaminophen poisoning.
Methods:
The medical records of 187 patients with acetaminophen poisoning treated at Chang Gung Memorial Hospital were reviewed. Patients were sorted into two groups according to their status of toxic hepatitis. A total of 40 clinical and laboratory features recorded on the first day of admission were selected for algorithm development. The random forest classifier (RFC) and logistic regression (LR) were used for artificial intelligence algorithm development.
Results:
The RFC-based AI model achieved the following results: accuracy = 92.5 ± 2.6%; sensitivity = 100%; specificity = 60%; precision = 92.3 ± 3.4%; and F1 = 96.0 ± 1.8%. The area under the receiver operating characteristic curve (AUROC) was approximately 0.98. The LR-based AI model achieved the following results: accuracy = 92.00 ± 2.9%; sensitivity = 100%; specificity = 20%; precision = 92.8 ± 3.4%; recall = 98.8 ± 3.4%; and F1 = 95.6 ± 1.5%. The AUROC was approximately 0.68. The weighted features were calculated, and the 10 most important weighted features for toxic hepatitis were aspartate aminotransferase (ALT), prothrombin time, alanine aminotransferase (AST), time to hospital, platelet count, lymphocyte count, albumin, total bilirubin, body temperature and acetaminophen level.
Conclusion:
The top five weighted features for acetaminophen-associated toxic hepatitis were ALT, prothrombin time, AST, time to hospital and platelet count.
Introduction
The popular phrase “artificial intelligence” (AI), first introduced in 1956 as “the science and engineering of making intelligent machines,” is applicable to a variety of items in medicine, such as the automated readings of chest radiographs and the detection of arrhythmias from wearable devices. What is so fascinating about AI is its potential to apply logical reasoning to data that is too large for the human mind to comprehend. An example of AI in toxicology is the prediction of irinotecan toxicity in metastatic colorectal cancer patients using a machine-learning approach with pharmacokinetic parameters. 1
Acetaminophen is one of the best-selling over-the-counter analgesic agents in the world. Although it is harmless at therapeutic doses, overdoses of acetaminophen can cause severe hepatic damage. Acetaminophen poisoning is the leading cause of acute liver injury and acute liver failure in the USA. 2 Most of the ingested acetaminophen is metabolized through glucuronidation and sulfation. Some, however, undergoes oxidative conversion through hepatic cytochromes, mainly cytochrome P450 2E1, generating the toxic metabolite N-acetyl-p-benzoquinone imine (NAPQI). 3 Normally, NAPQI is converted rapidly to nontoxic metabolites by glutathione. However, under circumstances where glutathione is depleted, such as acetaminophen overdose and severe malnutrition, NAPQI could produce liver injury. 4
The rationale for this study was based on an important but unsatisfactorily answered question that arose for many acetaminophen-poisoned patients treated at our hospital. Some patients suffered from severe toxic hepatitis and progressed rapidly to liver failure, whereas other patients suffered mild toxic hepatitis without overt clinical symptoms. Therefore, this raises the question of what are the best clinical predictors of toxic hepatitis? In addition, very little AI work has been performed to address this issue, which initiated our interest in this project. Although there have been studies on predictors for acetaminophen poisoning using statistical analysis (Table 1),5–15 investigations on the predictors of toxic hepatitis using the AI approach are relatively lacking. Therefore, we aimed to develop an AI algorithm for analyzing weight features for toxic hepatitis after acetaminophen poisoning. This machine-learning platform might help provide an alternative way to identify risk factors for toxic hepatitis after acetaminophen poisoning.
Published literature on risk factors for toxic hepatitis after acetaminophen poisoning using different analytical approaches.
Note: Acetaminophen psi parameter = 11.542[APAP]4 h (e-0.173ti - e-0.173tf) - [APAP]threshold (tf - ti), [APAP]4 h = serum acetaminophen concentration (mg/L) at 4 hour postingestion, [APAP]threshold = 45 mg/L, tf = time (hour) of initiation of n-acetylcysteine, ti = 6 hours; Acetaminophenpl plasma acetaminophen level; Acetaminophent threshold acetaminophen level at that time point on the treatment nomogram (based on a line through 100 mg/L at 4 hour) above which n-acetylcysteine would be administered; AST aspartate aminotransferase; ALT alanine aminotransferase; ALP alkaline phosphatase; AT aminotransferase; BMI body mass index; ICD-9-CM Classification of Diseases, Ninth Revision.
Materials and methods
Ethics statement
This retrospective observational study complied with the guidelines of the Declaration of Helsinki and was approved by the Medical Ethics Committee of Chang Gung Memorial Hospital, Linkou, Taiwan. The institutional review board number was 201800245B0.
Patients
We examined the medical records of 187 patients with intentional acetaminophen poisoning treated at Chang Gung Memorial Hospital between 2000 and 2011. Demographic, clinical, and laboratory data were collected for analysis. All patients were followed to investigate the risks of toxic hepatitis. The diagnoses of acetaminophen intoxication were based on clinical, physical and laboratory examinations and confirmed by blood acetaminophen levels (enzymatic method).
Inclusion and exclusion criteria
All adult patients aged 18 years and above who were diagnosed with acetaminophen poisoning at Chang Gung Memorial Hospital were eligible for inclusion in this study. Patients were excluded from analysis if they were younger than 18 years or did not have detectable acetaminophen levels in their blood despite a history of drug ingestion.
Clinical management
The protocols that were used to treat patients included gastric lavage with large amounts of normal saline, which was followed by the infusion of 1 g/kg activated charcoal and 250 mL magnesium citrate through a nasogastric tube. Magnesium citrate was used to prevent constipation after charcoal administration. The indication for N-acetylcysteine therapy was based on the Rumack-Matthew nomogram for acetaminophen poisoning to assess potential liver toxicity. 16
Definitions
Acetaminophen-associated toxic hepatitis was diagnosed if the ALT increase was more than 10 times the upper reference limit. 17 Acute respiratory failure has been defined as a condition of respiratory insufficiency requiring intubation and mechanical ventilation for more than 24 hours, regardless of the fraction of inspired oxygen. 18
Statistical analysis
Data are expressed as the mean ± standard deviation or number (percentage), unless otherwise stated. Student’s t-test was used to compare the means of continuous variables and normally distributed data. Categorical data were analyzed with the chi-square test. All statistical tests were two-tailed, with P values <0.05 considered statistically significant. Data were analyzed with SPSS 23.0 software for Windows (SPSS, Inc., Chicago, IL, USA).
Data collection and feature engineering
Demographics, vital signs, laboratory values, interventions, medications, and nurse documentation were accessed through the hospital information system of Chang Gung Memorial Hospital. In the data preprocessing, we first excluded all data containing missing values. For missing values in the other recording data, we filled them with the median values of each corresponding feature. This process guarantees the quality and consistency of the data as well as the model. Next, we conducted standard normalization of every feature, alleviating the influence of a different range of values. We selected 40 clinical and laboratory features recorded on the first day of admission for use in algorithm development.
Model selection
We applied several machine-learning methods, such as random forest classifier (RFC), K-nearest neighbor classification (KNN), logistic regression (LR), support vector machine, XGBoost, and neural network, and used fivefold cross-validation for the assessment of model performance. Cross-validation is a resampling procedure used to evaluate machine-learning models, and “5” refers to the number of groups the given data sample was split into. We found that random forest classifier, a decision-tree-based algorithm, and logistic regression had the best performance. Therefore, we adopted the random forest classifier and logistic regression for the following artificial intelligence algorithm development.
Performance evaluation
After we used random forest classifier and logistic regression to fit the training dataset and obtained the probability threshold, we computed the toxic hepatitis probability of the testing dataset as well as the accuracy, recall (sensitivity), specificity, precision and F1 score given that specific threshold. More importantly, we computed the receiver operating characteristic (ROC) curve for performance evaluation. We also performed a feature weight analysis to differentiate the contribution of each feature to the performance of the algorithm to evaluate the algorithm’s fitness for the clinical scenario. Feature weighting is closely associated with the AUROC, sensitivity, and precision of an algorithm and can be obtained during cross-validation in model training.
Results
Among the 187 patients with intentional acetaminophen poisoning, 35 (18.7%) were male, and their mean age was 28.94 ± 12.12 years (Table 2). Patients with toxic hepatitis not only had higher blood acetaminophen levels (88.14 ± 85.33 versus 44.73 ± 33.64 μg/mL, P < 0.001) and longer times to hospital (12.50 ± 11.88 versus 5.34 ± 4.10 hours, P < 0.001) but also suffered a higher incidence of respiratory failure (18.1 versus 1.8%, P = 0.004) than patients without toxic hepatitis. Laboratory analysis found higher blood concentrations of white blood cell count (10722.73 ± 7603.504 versus 8813.94 ± 3119.22 per μL, P = 0.032), AST (847.45 ± 2926.92 versus 27.67 ± 19.64 U/L, P < 0.001), ALT (450.50 ± 1014.10 versus 21.06 ± 20.83 U/L, P < 0.001), total bilirubin (1.35 ± 0.81 versus 0.88 ± 0.29 mg/dL, P < 0.001), bicarbonate (22.00 ± 4.36 versus 21.93 ± 1.25 mEq/L, P = 0.002), and prothrombin time (1.59 ± 0.80 versus 1.09 ± 0.06 INR, P < 0.001) in patients with toxic hepatitis than in those without toxic hepatitis. Furthermore, the blood concentration of platelets was lower in patients with toxic hepatitis than in those without toxic hepatitis (21.62 ± 10.70 versus 25.02 ± 5.83 103 per μL, P = 0.024).
Clinical data of patients with acetaminophen poisoning, stratified according to the status of toxic hepatitis (n = 187).
Note: AST aspartate aminotransferase; ALT alanine aminotransferase; *P < 0.05; ** P < 0.01; *** P < 0.001.
The algorithm development process included three steps (Table 3). During feature engineering, the 187 patients were divided into training data (n = 149) and testing data (n = 38). The RFC-based AI model was developed using 80% of the records as the training cohort and 20% of the records as the testing cohort. The established algorithm achieved the following results: accuracy = 92.5 ± 2.6%, sensitivity = 100%, specificity = 60%, precision = 92.3 ± 3.4%, and F1 = 96.0 ± 1.8%. The AUROC was approximately 0.98 (Figure 1).

The receiver operating characteristic (ROC) curve. The random forest classifier (RFC)-based artificial intelligence (AI) model achieved the following results: accuracy = 92.5 ± 2.6%; sensitivity = 100%; specificity = 60%; precision = 92.3 ± 3.4%; and F1 = 96.0 ± 1.8%. The area under the receiver operating characteristic curve (AUROC) was approximately 0.98.
Identification of the top 10 weighted features for toxic hepatitis using an artificial intelligence approach.
Note: AST aspartate aminotransferase; ALT alanine aminotransferase.
To compare the diagnostic performance of the RFC-based AI algorithm to the LR-based AI algorithm, the same 187 patients were used to evaluate the performance of toxic hepatitis using logistic regression. The LR-based AI model was developed using 80% of the records as the training cohort and 20% of the records as the testing cohort. The established algorithm achieved the following results: accuracy = 92.00 ± 2.9%, sensitivity = 100%, specificity = 20%, precision = 92.8 ± 3.4%, recall = 98.8 ± 3.4%, and F1 = 95.6 ± 1.5%. The AUROC was approximately 0.68. The weight features were calculated, and the top 10 were listed according to their categories (Table 3). The weight represents the influence of each feature. ALT, prothrombin time, AST, time to hospital and platelet count were the most highly weighted.
Discussion
These data are important because this is one of the few studies where an AI algorithm has been applied to analyze weight features for acetaminophen-associated toxic hepatitis. Using a binary mixed model tree artificial intelligence approach, 19 it was demonstrated that the three most important weight features for the daily outcome of patients with acetaminophen-induced acute liver failure were the use of pressors and bilirubin and creatinine levels. On the other hand, our analysis showed that the 10 most important weighted features for toxic hepatitis were ALT, prothrombin time, AST, time to hospital, platelet count, lymphocyte count, albumin, total bilirubin, body temperature and acetaminophen level. As shown in Table 1,5–15 several previous studies have identified risk factors for acetaminophen-associated toxic hepatitis using a traditional statistical analysis approach. In contrast, our data were analyzed using an AI algorithm to identify the top 10 weighted features for toxic hepatitis.
Liver disease is often reflected by abnormalities in liver function tests. While aminotransferase and bilirubin represent hepatocyte integrity and cholestasis, respectively, albumin and prothrombin time indicate liver function mass. 17 Several biomarkers, such as ALT, prothrombin time, AST, albumin and total bilirubin, ranked in the top 10 feature weights in our study. Aminotransferases, which are highly concentrated in the liver, are enzymes that catabolize amino acids, allowing them to enter the citric acid cycle. While AST is also represented in kidneys, brain, skeletal muscle and red blood cells, ALT has a lower concentration in skeletal muscle and kidneys. In one retrospective study including 304 subjects, 8 initial AST > 50 U/L predicted hepatic injury in patients with acetaminophen overdose with a sensitivity of 79.5% and a specificity of 82.6%. In another study including 410 subjects, 10 initial ALT > 50 U/L predicted hepatic injury in patients with acetaminophen overdose with a sensitivity of 100% and a specificity of 86%.
Prothrombin time measures the extrinsic pathway of coagulation and represents the synthetic capacity of the liver. In one retrospective study including 304 subjects, 8 prothrombin time predicted hepatic injury in patients with acetaminophen overdose with a sensitivity of 82.1% and a specificity of 63.6%. 8
Albumin is synthesized by the hepatocytes. A variety of factors regulate the synthesis of albumin, including nutritional status, cytokines and hormones. 20 In one study, 21 short-term fasting increased acetaminophen exposure in healthy subjects administered oral acetaminophen at 1000 mg. However, in another study including 197 subjects, 6 overweight-obesity status was a significant risk factor for acute liver injury secondary to acute acetaminophen overdose.
Bilirubin is the product of heme metabolism. It binds to albumin in the plasma and is transported throughout the body. In hepatocytes, bilirubin is conjugated with glucuronic acid by glucuronosyltransferase. In one animal study, 22 inhibition of hepatic bilirubin conjugation and excretion of conjugated bilirubin into the bile was observed after acetaminophen administration to rats.
Platelet count ranked as the fifth feature weight in this study. The maintenance of hematopoietic stem cells requires hepatic thrombopoietin. 23 Through a series of divisions, hematopoietic stem cells yield each of the hematopoietic lineages, including megakaryocytes, which ultimately give rise to platelets. Thrombopoietin is also involved in megakaryocyte development. 24 In one animal study, 25 acetaminophen poisoning caused the depletion of blood platelets and the accumulation of platelets in the liver. Notably, platelets have been found to contribute to the progression of acetaminophen-induced liver injury. 25
Time to hospital ranked as the fourth feature weight in our study. In one study including 322 pediatric subjects, 14 hepatocellular injury was associated with a presentation longer than 24 hours after ingestion. In another study including 75 subjects, 13 time to hospital was a risk factor for acetaminophen-induced hepatotoxicity. The acetaminophen psi parameter takes into account the acetaminophen level at 4 hours postingestion and the time to initiation of n-acetylcysteine therapy. Chomchai and Chomchai demonstrated in a retrospective study of 255 subjects 11 that the acetaminophen psi parameter predicts hepatotoxicity in overdose patients with a sensitivity of 96.9% and a specificity of 91.5%.
Lymphocyte count ranked as the sixth feature weight in our study. This is reasonable because acetaminophen overdose is always linked with blood toxicity. Rat lymphocytes carry cofactors and enzymes involved in the mechanism of acetaminophen hepatotoxicity,26,27 and furthermore, a rat study also demonstrated that blood lymphocyte count decreases during acetaminophen hepatotoxicity. 28
Body temperature ranked as the ninth feature weight in our study. Acetaminophen is widely used to treat fever. It is assumed that the antipyretic action of acetaminophen is mediated through the cyclooxygenase pathway. 29 In one animal study, 30 it was demonstrated that mild hypothermia attenuates liver injury and benefits survival in mice with acetaminophen toxicity. A previous study 31 revealed that hypothermia favors functional mitochondria and decreases oxidative stress to fight acetaminophen-induced hepatotoxicity.
Blood acetaminophen level ranked as the 10th feature weight in our study. Both the acetaminophen psi parameter and acetaminophen-aminotransferase multiplication product take the acetaminophen level into account. Several studies have demonstrated that they can predict acetaminophen hepatotoxicity.6,9,11 Another retrospective observational study included 545 subjects, 7 where acetaminophenpl was defined as the acetaminophen level and acetaminophent was defined as the threshold acetaminophen level at that time point on the treatment nomogram (based on a line through 100 mg/L at 4 hours) above which n-acetylcysteine was administered. An acetaminophenpl:acetaminophent ratio ≥3 predicted acetaminophen hepatotoxicity.
This AI algorithm has many potential applications in analyzing cases of poisoning. The machine-learning platform offers the following advantages over traditional statistical analysis. First, the variables are readily available in clinical practice and can be obtained upon admission. Second, the outcome can be predicted within a very short period of time using a simple laptop. Third, since the AI algorithm uses feature weighting to approximate the influence of individual features, it is not necessary to discard any variables as in statistical analysis because no null hypothesis is assumed. Fourth, the receiver operating characteristic curve obtained by artificial intelligence is based on multiple parameters, rather than a single parameter as in statistical analysis.
Finally, the limitations of this study are its small sample size and short follow-up duration. Further studies are warranted.
In summary, this study developed an AI algorithm to predict weighted features for toxic hepatitis after acetaminophen poisoning. The top five weighted features for toxic hepatitis are ALT, prothrombin time, AST, time to hospital and platelet count.
Footnotes
Author contributions
Data collection and manuscript writing JSY; Data analysis CCH, WHH; Patient care and management WHH, CWH, THY; Study design and supervision CHW.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by research grants from Chang Gung Memorial Hospital (CMRPG5J0201, CORPG3K0191).
