HDL-C levels added to the MELD score improves 30-day mortality prediction in Asian patients with cirrhosis

Abstract

Objectives

Lower high-density lipoprotein cholesterol (HDL-C) levels have been observed in chronic liver disease patients. The aim of this study was to develop a model that incorporates HDL-C levels and the Model for End-stage Liver Disease (MELD) score to predict 30-day mortality in Asian cirrhosis patients.

Methods

Cirrhosis patients were recruited from two hospitals in this retrospective observational study. Propensity score matching was used. The model’s performance was evaluated, including its ability to predict 30-day mortality, accuracy, and clinical utility. Net reclassification improvement (NRI) and integrated discrimination improvement (IDI) were calculated.

Results

The HDL-C + MELD model showed good ability to predict 30-day mortality (area under the curve, 0.784; sensitivity, 0.797; specificity, 0.632), which was better than that of the MELD score alone. It also showed good calibration and a net benefit for all patients, which was better than that of the MELD score, except at the threshold probability. NRI and IDI results confirmed that adding HDL-C levels to the MELD score improved the model’s performance in predicting 30-day mortality.

Conclusion

We added HDL-C levels to the MELD score to predict 30-day mortality in Asian patients with cirrhosis. The HDLC + MELD model shows good ability to predict 30-day mortality and clinical utility.

Keywords

Cirrhosis high-density lipoprotein cholesterol Model for End-stage Liver Disease score nomogram predictive model net reclassification improvement integrated discrimination improvement

Introduction

Cirrhosis involves necro-inflammation and fibrosis that is caused by different types of liver damage.¹ With the development of cirrhosis, portal pressure increases and liver function deteriorates, which leads to a series of complications such as ascites, portal hypertensive gastrointestinal bleeding, spontaneous bacterial peritoneum, and encephalopathy.² Cirrhosis is widely considered an end-stage liver disease that frequently results in death. Early intervention is required to stabilize cirrhosis progression and delay liver physiological function decompensation.³

Chronic liver disease is related to a lower level of serum high-density lipoprotein (HDL) cholesterol (HDL-C), which is the main protein component of HDL particles.⁴ Recent results show that decreased HDL-C levels were significantly associated with mortality in patients with non-cholestatic cirrhosis.⁵ HDL was confirmed to bind and neutralize bacterial lipopolysaccharide and promote their excretion, which suggests that HDL plays a key role in chronic liver failure progression.⁶

Trieb et al.⁷ recently studied the prognostic value of several HDL-related biomarkers in advanced liver disease. They suggested that HDL-C and other HDL-related biomarker levels reflect liver function deterioration and could reliably predict 90-day and 12-month mortality in patients with cirrhosis, regardless of their cirrhosis etiology. HDL-C in combination with the Model for End-stage Liver Disease (MELD) score showed better performance in predicting cirrhotic progression and survival.

In this multi-center retrospective observational study, we aimed to evaluate HDL-C levels in combination with the MELD score for predicting 30-day mortality in Asian patients with cirrhosis.

Methods

Study design

This was a retrospective observational study. Hospitalized patients with cirrhosis were recruited from the following two hospitals: i) the Department of Gastroenterology and Hepatology, The First Affiliated Hospital of Soochow University (center #1) between 1 January 2017 and 31 December 2020; and ii) the Department of Hepatology, The Fifth People’s Hospital of Suzhou (center #2) between 1 March 2018 and 31 December 2020. This study was approved by the local Institutional Review Boards (IRBs; approval number 2022098) and conducted in accordance with the Declaration of Helsinki. All participants provided written informed consent before they were included in the study.

Study entry criteria

At both centers, cirrhosis was diagnosed on the basis of either liver histology or clinical presentation, liver function tests, and abdominal imaging. Patients were excluded if they had malignancy, severe cardiopulmonary, renal, endocrine, immune, or inflammatory disease or cholestatic liver disease; if they were pregnant; or if they were taking lipid-lowering therapy. Figure 1 indicates the flowchart for the participants’ inclusion.

Figure 1.

Flowchart of participants’ inclusion at two centers.

Clinical data collection

Each participant fasted overnight. Blood samples were obtained from all participants. Biochemical analyses were preformed using a Hitachi 7600 Auto-Analyzer (Hitachi, Tokyo, Japan) or an Abbott-Architect Immunoanalyzer (Abbott Laboratories, Abbott Park, IL, USA). Anthropometric and biochemical examinations were performed in accordance with the standard procedures as previously reported.⁸ The outcome variable was defined as mortality at 30 days after enrollment.

Statistical analysis

The reporting of this study conforms to the STARD Initiative.⁹ Statistical analysis was performed using R (version 4.0.4, R Foundation for Statistical Computing, Vienna, Austria). Propensity score matching (PSM) analysis was used to reduce bias by equating the two groups on the basis of their age, sex, and cirrhosis etiology (the “tableone” package). PSM was performed using the nearest-neighbor 7:1 matching method with a caliper width of 0.1 of the pooled standard deviation of the logit of the propensity score. The binary logistic regression model was used to develop a predictive model. A nomogram was created using the “rms” package, and calibration plots were used to evaluate the nomogram accuracy. The nomogram was constructed on the basis of the independent risk factors in the predictive model that were identified in the multivariate analysis. The ability of the nomogram to predict 30-day mortality was evaluated using receiver operating characteristic (ROC) curve analysis (the “pROC” package) and the confusion matrix (the “InformationValue” package). Decision curve analysis (DCA) (the “rmda” package) was performed to assess the clinical usefulness of the nomogram. The net reclassification improvement (NRI) and integrated discrimination improvement (IDI) were also calculated (the “PredictABEL” package). A two-sided p value <0.05 was considered statistically significant.

Results

Participant characteristics

The study enrolled 913 patients with cirrhosis from two centers (center #1, n = 668 and center #2, n = 245; Figure 1). The PSM process is shown in Supplementary Figures 1 and 2 and Supplementary File 1. After PSM, 788 cirrhotic patients were included in the analyses. The baseline data are presented in Table 1. There were 399 men and 256 women with an average age of 61.20 ± 12.95 years in the survival group and 87 men and 46 women with an average age of 63.17 ± 12.77 years in the non-survival group.

Table 1.

Characteristics of patients with cirrhosis analyzed on the basis of 30-day mortality after propensity score matching.

	Survival (n = 655)	Non-survival (n = 133)	p
Sex, n (%)			0.382
Men	399 (60.9)	87 (65.4)
Women	256 (39.1)	46 (34.6)
Age (years)	61.20 (12.95)	63.17 (12.77)	0.11
Cirrhosis etiology, n (%)			0.318
HBV	552 (84.3)	104 (78.2)
HCV	15 (2.3)	3 (2.3)
Schistosomiasis	34 (5.2)	9 (6.8)
Others	54 (8.2)	17 (12.8)
Child–Pugh score	7.00 [6.00, 8.00]	10.00 [8.00, 11.00]	<0.001
Ascites (%)	467 (71.3)	87 (65.4)	0.211
Ascites (%)	188 (28.7)	46 (34.6)
Hepatic encephalopathy (%)	561 (85.6)	103 (77.4)	0.026
Hepatic encephalopathy (%)	94 (14.4)	30 (22.6)
Gastrointestinal bleeding (%)	482 (73.6)	95 (71.4)	0.685
Gastrointestinal bleeding (%)	173 (26.4)	38 (28.6)
Body mass index (kg/m²)	22.86 (3.43)	22.24 (3.17)	0.055
PLT (10⁹/L)	99.00 (70.44)	87.08 (62.33)	0.070
TBIL (mg/dL)	3.49 (4.72)	8.11 (8.66)	<0.001
Creatinine (mg/dL)	0.85 (0.47)	1.14 (1.12)	<0.001
ALT (U/L)	27.00 [18.20, 45.95]	42.00 [22.00, 109.40]	<0.001
AST (U/L)	39.00 [28.70, 64.00]	57.00 [36.40, 110.90]	<0.001
Albumin (g/L)	31.70 (5.98)	29.18 (5.87)	<0.001
HDL-C (mg/dL)	36.59 (16.08)	21.23 (14.61)	<0.001
PT (s)	14.90 [13.20, 16.85]	18.30 [15.60, 21.80]	<0.001
INR	1.29 [1.15, 1.48]	1.59 [1.30, 1.93]	<0.001
MELD	9.05 [5.41, 12.67]	15.12 [8.75, 21.99]	<0.001

Normally distributed variables are presented as the mean ± SD. Variables with a skewed distribution were presented as the median value (interquartile range).

ALT, alanine transaminase; AST, aspartate aminotransferase; HBV, hepatitis B virus; HCV, hepatitis C virus; HDL-C, high-density lipoprotein cholesterol; INR, international normalized ratio; MELD, Model for End-stage Liver Disease; PLT, platelet count; PT, prothrombin time; SD, standard deviation TBIL, total bilirubin.

Development of the HDLC + MELD model and its nomogram

A model that included both the HDL-C level and the MELD score was developed. It showed a good fit (Hosmer–Lemeshow goodness of fit test: χ² = −507.92; Wald test: HDL-C, F = 32.96, p < 0.001; MELD score, F = 19.29, p < 0.001; Supplementary Figures 3 and 4), and its nomogram is presented in Figure 2. The units on the x-axis indicate the probability of death at day 30.

Figure 2.

Nomogram to estimate the 30-day mortality risk.

A mortality risk nomogram was developed for the model incorporating both the HDL-C level and the MELD score. The points for the HDL-C level and MELD score variables are shown on the y-axis. The units on the x-axis indicate the percent probability of death at day 30.

The HDLC + MELD model’s prediction ability

The area under the ROC curve (AUC) is shown in Figure 3. The HDLC + MELD model showed good discriminative power to predict 30-day mortality (AUC = 0.784, sensitivity 0.797 and specificity 0.632), which was better than that for the MELD score alone (AUC = 0.716, sensitivity 0.737 and specificity 0.609; DeLong’s test p = 0.0479). Misclassification error was 0.146 for the HDLC + MELD model and 0.152 for the MELD score model.

Figure 3.

Receiver operating characteristic curve.

The AUC for the HDLC + MELD model was better than that for the MELD score alone (DeLong’s test, p = 0.0479).

The new model’s accuracy

The calibration curves are shown in Figure 4. The U index was −0.003, while the Brier score was 0.114. The HDLC + MELD model showed good calibration, with no significant difference between the observed and the predicted probability.

Figure 4.

HDLC+MELD calibration curve for 30-day mortality.

The y-axis represents the actual outcome (mortality), and the x-axis represents the predicted probability for the HDLC + MELD model. The calibration test showed a U index of −0.003 and a Brier score of 0.114.

Decision curve analysis

The DCA for the HDLC + MELD model and the MELD score is presented in Figure 5. The models evaluated the net benefit that patients could receive. As shown in Figure 5, the HDLC + MELD model had a net benefit for all the patients. Additionally, the HDLC + MELD model had a better net benefit compared with that of the MELD score alone, except at the threshold probability of 30% to 35%.

Figure 5.

Decision curve analysis for the nomogram.

The y-axis indicates the net benefit. The blue line represents the MELD+HDL-C model, and the red line represents the MELD score alone. The “All” line represents the assumption that all cirrhotic patients are mortal. The “None” line represents the assumption that all cirrhotic patients will survive. The decision curve showed that applying the HDLC + MELD model had a benefit for all the patients. Additionally, the HDLC + MELD model showed better benefit compared with that of the MELD score alone, except at the threshold probability of 30% to 35%.

Net reclassification improvement and integrated discrimination improvement

When HDL-C was added to the MELD score, the performance of the model improved compared with that of model using the MELD score alone (NRI =0.1728, p < 0.001 and IDI = 0.0455, p <0.001), as indicated in Supplementary File 2.

Discussion

Inspired by Trieb et al.’s study,⁷ a model that incorporated the HDL-C level and the MELD score was developed to predict 30-day mortality in patients with cirrhosis. Our results showed that the model where the HDL-C level was added to the MELD score improved the ability to predict 30-day mortality compared with using the MELD score alone.

Several blood markers and scoring systems have been developed to predict mortality in patients with cirrhosis.¹⁰ The Child–Pugh score was initially created to improve the clinical management of cirrhotic patients.¹¹ The MELD score was developed to predict the cirrhosis progression using serum creatinine and bilirubin levels and the international normalized ratio (INR).¹²

Recent evidence shows that the role of HDL-C levels in cirrhosis patients might be multifactorial. One of these roles is in malnutrition. Chronic liver dysfunction impairs lipoprotein composition and metabolism, which leads to hypercholesterolemia and hyperlipidemia.^5,13 Malnutrition is more common at the advanced stage, and it leads to decompensated cirrhosis. Thus, deterioration of the nutritional status in cirrhotic patients is significantly associated with disease progression because it usually appears in the decompensated stage of chronic liver disease.^14
–17 Insufficient protein and calorie supplementation in patients with chronic liver failure frequently results in inadequate cholesterol and lipoprotein synthesis. Bories and Campillo¹⁸ found that nutritional supplementation in cirrhotic patients could increase HDL-C levels, which was associated with improvements in other liver function parameters. Additionally, Wolf et al. reported that HDL-C levels increased after liver transplantation,¹⁹ which suggests that HDL-C levels are related to the degree and function of liver regeneration. Gao et al.²⁰ found that the HDL-C level was significantly correlated with liver function in cirrhotic patients, and it was a potential indicator for the prognosis of non-malignant cirrhotic patients with portal vein thrombosis. Habib et al.⁵ reported that the HDL-C level, but not the cholesterol or LDL-C level, was closely related to albumin, bilirubin, INR, and MELD scores. Moreover, HDL-C levels were suggested to be an independent predictor of 6-month and 12-month mortality in non-cholestatic cirrhosis. Some studies have also focused on the anti-inflammatory effect of HDL-C because it can regulate cholesterol excretion in macrophage foam cells, which leads to decreased lipid-induced proinflammatory cytokine secretion.²¹ Infection plays a vital role in the prognosis for cirrhosis patients and in these patients’ survival.²² Infection results from immune dysfunction and deterioration caused by liver disease, and it then leads to acute–chronic liver failure.^23,24 HDL may reduce inflammatory signaling transmission and regulate cholesterol levels in the cell membrane by clearing cholesterol and other lipids from the cells.^25,26 Additionally, HDL particles can reduce inflammatory cytokine production by binding to lipopolysaccharide, and they can also decrease mortality in various animal models.^27
–29 Gao et al.²⁰ reported that the HDL-C level was inversely correlated with the C-reactive protein level, which indicates its potential anti-inflammatory role.

This was a multi-center retrospective study to validate previous results. All participants were recruited from two medical centers and included in the analyses after PSM was performed. A predictive model including the HDL-C level and the MELD score was developed, and a nomogram was then created to assess the risk of mortality in cirrhotic patients. An ROC curve showed that the HDLC + MELD model had a good ability to predict 30-day mortality, which was better than that of the MELD score alone. This result was in accordance with the results reported in Trieb et al.’s study.⁷ Additionally, the new model showed good calibration. A decision curve analysis was also performed in our study, which assessed the nomogram’s clinical utility on the basis of the threshold probability.^30,31 The HDLC + MELD model had a net benefit for all patients, and the HDLC + MELD model showed a better net benefit compared with that of the MELD score, except at the threshold probability of 30% to 35%. Finally, NRI and IDI results confirmed that adding HDL-C levels to the MELD score improved the model’s performance in predicting 30-day mortality compared with that of the MELD score alone.

This study has several limitations. First, there was likely selection bias because of the study’s retrospective observational design. Second, the study had a small sample size despite recruiting participants from two centers and using the PSM method. Third, only one outcome, 30-day mortality, was investigated, and further evaluation of the patients’ long-term prognosis is required, such as using the 90-day mortality or investigation after a 12-month follow-up. Finally, the study entry criteria excluded patients with cardiopulmonary, renal, and inflammatory diseases that are associated with lipid metabolism, which limits the generalizability of our study.

In conclusion, our study added HDL-C levels to the MELD score to predict 30-day mortality in Asian patients with cirrhosis. The HDLC + MELD shows good ability to predict 30-day mortality and good clinical utility.

Supplemental Material

sj-pdf-1-imr-10.1177_03000605221109385 - Supplemental material for HDL-C levels added to the MELD score improves 30-day mortality prediction in Asian patients with cirrhosis

Supplemental material, sj-pdf-1-imr-10.1177_03000605221109385 for HDL-C levels added to the MELD score improves 30-day mortality prediction in Asian patients with cirrhosis by Yue Wang, Wenjuan Shen, Fang Huang, Chenyan Yu, Liting Xi, Jingwen Gao, Minyue Yin, Xiaolin Liu, Jiaxi Lin, Lu Liu, Huixian Zhang, Jinzhou Zhu Yu Hong in Journal of International Medical Research

Footnotes

Authors’ contributions statement

Y. Wang and L. Xi wrote manuscript; W. Shen, F. Huang, C. Yu, L. Liu, and H. Zhang collected clinical data; Y. Wang, J. Gao, M. Yin, X. Liu, and J. Lin analyzed clinical data; Y. Hong and J. Zhu contributed to the study design.

Declaration of conflicting interests

The authors declare that they have no conflict of interest.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Youth Program of Suzhou Health Committee (KJXW2021051 and KJXW2019001). No funding body had any role in the design of the study and collection, analysis, interpretation of data, or writing of the manuscript.

ORCID iD

Jinzhou Zhu

Supplemental material

Supplemental material for this article is available online.

References

Schuppan

Afdhal

NH.

Liver cirrhosis. Lancet 2008; 371: 838–851. 2008/03/11. DOI: 10.1016/S0140-6736(08)60383-9.

D'Amico

Morabito

D'Amico

, et al. Clinical states of cirrhosis and competing risks. J Hepatol 2018; 68: 563–576. 2017/11/08. DOI: 10.1016/j.jhep.2017.10.020.

D'Amico

Pasta

Morabito

, et al. Competing risks and prognostic stages of cirrhosis: a 25-year inception cohort study of 494 patients. Aliment Pharmacol Ther 2014; 39: 1180–1193. 2014/03/25. DOI: 10.1111/apt.12721.

Etogo-Asse

Vincent

Hughes

, et al. High density lipoprotein in patients with liver failure; relation to sepsis, adrenal function and outcome of illness. Liver Int 2012; 32: 128–136. 2011/11/22. DOI: 10.1111/j.1478-3231.2011.02657.x.

Habib

Mihas

Abou-Assi

, et al. High-density lipoprotein cholesterol as an indicator of liver function and prognosis in noncholestatic cirrhotics. Clin Gastroenterol Hepatol 2005; 3: 286–291. 2005/03/15. DOI: 10.1016/s1542-3565(04)00622-6.

Pirillo

Catapano

Norata

GD.

HDL in infectious diseases and sepsis.

Handb Exp Pharmacol 2015; 224: 483–508. 2014/12/20. DOI: 10.1007/978-3-319-09665-0_15.

Trieb

Rainer

Stadlbauer

, et al. HDL-related biomarkers are robust predictors of survival in patients with chronic liver failure. J Hepatol 2020; 73: 113–120. 2020/02/18. DOI: 10.1016/j.jhep.2020.01.026.

Xue

Zhang

, et al. Liver expressed antimicrobial peptide 2 is associated with steatosis in mice and humans. Exp Clin Endocrinol Diabetes 2020; 129: 601–610. 2020/09/16. DOI: 10.1055/a-1210-2357.

Bossuyt

Reitsma

Bruns

, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ 2015; 351: h5527. 20151028. DOI: 10.1136/bmj.h5527.

10.

Mahassadi

Nguieguia

JLK

Kissi

, et al. Systemic inflammatory response syndrome and model for end-stage liver disease score accurately predict the in-hospital mortality of black African patients with decompensated cirrhosis at initial hospitalization: a retrospective cohort study. Clin Exp Gastroenterol 2018; 11: 143–152. 2018/04/20. DOI: 10.2147/CEG.S140655.

11.

Wang

, et al. Risk stratification based on chronic liver failure consortium acute decompensation score in patients with Child-Pugh B cirrhosis and acute variceal bleeding. Hepatology 2020; 73: 1478–1493. 2020/07/25. DOI: 10.1002/hep.31478.

12.

Sheth

Riggs

Patel

Utility of the Mayo End-Stage Liver Disease (MELD) score in assessing prognosis of patients with alcoholic hepatitis.

BMC Gastroenterol 2002; 2: 2. 2002/02/12. DOI: 10.1186/1471-230x-2-2.

13.

Trieb

Horvath

Birner-Gruenberger

, et al. Liver disease alters high-density lipoprotein composition, metabolism and function. Biochim Biophys Acta 2016; 1861: 630–638. 2016/04/24. DOI: 10.1016/j.bbalip.2016.04.013.

14.

Nishikawa

Osaki

Liver cirrhosis: evaluation, nutritional status, and prognosis.

Mediators Inflamm 2015; 2015: 872152. 20150930. DOI: 10.1155/2015/872152.

15.

Tandon

Raman

Mourtzakis

, et al. A practical approach to nutritional screening and assessment in cirrhosis. Hepatology 2017; 65: 1044–1057. 20170206. DOI: 10.1002/hep.29003.

16.

Carey

Lai

Wang

, et al. A multicenter study to define sarcopenia in patients with end-stage liver disease. Liver Transpl 2017; 23: 625–633. 2017/02/28. DOI: 10.1002/lt.24750.

17.

Bering

Diniz

KGD

Coelho

MPP

, et al. Association between pre-sarcopenia, sarcopenia, and bone mineral density in patients with chronic hepatitis C. J Cachexia Sarcopenia Muscle 2018; 9: 255–268. 2018/01/20. DOI: 10.1002/jcsm.12269.

18.

Bories

Campillo

One-month regular oral nutrition in alcoholic cirrhotic patients. Changes of nutritional status, hepatic function and serum lipid pattern.

Br J Nutr 1994; 72: 937–946. 1994/12/01. DOI: 10.1079/bjn19940097.

19.

Wolf

Holmes

Fouraschen

, et al. Serum lipid expression correlates with function and regeneration following living donor liver transplantation. Liver Transpl 2016; 22: 103–110. 2015/07/24. DOI: 10.1002/lt.24220.

20.

Gao

Xiao

Zhang

, et al. High-density lipoprotein cholesterol for the prediction of mortality in cirrhosis with portal vein thrombosis: a retrospective study. Lipids Health Dis 2019; 18: 79. 2019/04/01. DOI: 10.1186/s12944-019-1005-8.

21.

Dogan

Karaman

Sonmez

, et al. Relationship between serum neutrophil count and infarct size in patients with acute myocardial infarction. Nucl Med Commun 2009; 30: 797–801. 2009/07/30. DOI: 10.1097/MNM.0b013e32832e3a16.

22.

Arvaniti

D'Amico

Fede

, et al. Infections in patients with cirrhosis increase mortality four-fold and should be used in determining prognosis. Gastroenterology 2010; 139: 1246–1256, 1256 e1241-1245. 2010/06/19. DOI: 10.1053/j.gastro.2010.06.019.

23.

Tandon

Garcia-Tsao

Bacterial infections, sepsis, and multiorgan failure in cirrhosis.

Semin Liver Dis 2008; 28: 26–42. 2008/02/23. DOI: 10.1055/s-2008-1040319.

24.

Gustot

Durand

Lebrec

, et al. Severe sepsis in cirrhosis. Hepatology 2009; 50: 2022–2033. 2009/11/04. DOI: 10.1002/hep.23264.

25.

Fessler

Parks

JS.

Intracellular lipid flux and membrane microdomains as organizing principles in inflammatory cell signaling.

J Immunol 2011; 187: 1529–1535. 2011/08/04. DOI: 10.4049/jimmunol.1100253.

26.

Iqbal

Barrett

Taylor

, et al. Acute exposure to apolipoprotein A1 inhibits macrophage chemotaxis in vitro and monocyte recruitment in vivo. Elife 2016; 5; e15190. 2016/08/31. DOI: 10.7554/eLife.15190.

27.

Levine

Parker

Donnelly

, et al. In vivo protection against endotoxin by plasma high density lipoprotein. Proc Natl Acad Sci U S A 1993; 90: 12040–12044. 1993/12/15. DOI: 10.1073/pnas.90.24.12040.

28.

Hubsch

Casas

Doran

JE.

Protective effects of reconstituted high-density lipoprotein in rabbit gram-negative bacteremia models. J Lab Clin Med 1995; 126: 548–558. 1995/12/01.

29.

McDonald

Dhadly

Cockerill

, et al. Reconstituted high-density lipoprotein attenuates organ injury and adhesion molecule expression in a rodent model of endotoxic shock. Shock 2003; 20: 551–557. 2003/11/20. DOI: 10.1097/01.shk.0000097249.97298.a3.

30.

Steyerberg

Vickers

Cook

, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010; 21: 128–138. 2009/12/17. DOI: 10.1097/EDE.0b013e3181c30fb2.

31.

Qiao

Zhang

, et al. Establishment and validation of a predictive nomogram for extended operation time following mandibular third molar removal. Clin Oral Investig 2021; 25: 1915–1923. 2020/08/12. DOI: 10.1007/s00784-020-03499-8.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.28 MB