Predicting Three-Year Survival in Patients Receiving Maintenance Dialysis: An External Validation and Updated Multivariable Prediction Model for iChoose Kidney in Ontario,Canada

Abstract

French

Background:

A significant barrier to kidney transplantation is limited knowledge about its potential benefits. To help patients who are receiving maintenance dialysis make more informed treatment decisions, a risk calculator (iChoose Kidney) was developed in the United States to provide individualized survival estimates for dialysis versus kidney transplantation. This tool was externally validated in Ontario, Canada, and was found to accurately predict mortality (Ontario version of the tool “Dialysis vs. Kidney Transplant-Estimated Survival in Ontario Risk Calculator”). The United States risk calculator has been updated to include additional variables (e.g., dialysis modality).

Objective:

To externally validate the updated iChoose Kidney risk calculator in patients from Ontario, Canada, with kidney failure using more recent data, removing race (race in clinical algorithms may perpetuate racial bias in medicine) and using a refined cohort definition (i.e., restricting to patients with no recorded contraindications to transplant).

Design:

External validation study.

Setting:

Linked administrative health care databases from Ontario, Canada.

Patients:

24 793 patients receiving maintenance dialysis and 5398 kidney transplant recipients from January 1, 2011, to August 31, 2021.

Measurements:

Three-year mortality.

Methods:

Model discrimination was evaluated using the C-statistic. Calibration was assessed by comparing the observed versus predicted mortality risks, and further assessed by using loess-smoothed calibration plots. To address over- or under-prediction (calibration-in-the-large), intercepts were adjusted using a correction factor. In our updated model, we used logistic regression to calculate mortality risk, incorporating the following variables: sex assigned at birth (male vs female), age (continuous), cardiovascular disease, hypertension, diabetes, time on dialysis (i.e., <6 months, 6 to 12 months, >1 to 2 years, >2 to 3 years, >3 to 5 years, >5 to 7 years, >7 to 10 years, >10 to 14 years, >14 years), and dialysis modality (peritoneal dialysis, home hemodialysis, in-center dialysis). In a post-hoc analysis, we used the simplified equations from our original Canadian external validation study of the iChoose Kidney tool (i.e., age, sex, hypertension, diabetes, cardiovascular disease, time on dialysis [<6 months, 6-12 months, >12 months]), with removal of the race variable as the only modification.

Results:

In the dialysis cohort, over a median follow-up of 2.5 years, 30.3% of patients died. In the kidney transplant recipient cohort, over a median follow-up of 2.9 years, 7.3% died. Our updated model had moderate discrimination (C-statistic for dialysis cohort: 0.67 [95% CI: 0.67, 0.68] and C-statistic for kidney transplant cohort: 0.76 [95% CI: 0.74, 0.79]). After recalibrating the intercepts, the observed and predicted mortality were similar between the dialysis cohort and the kidney transplant cohort. Similar results were found in a post-hoc analysis using the original model with the race variable removed.

Limitations:

Mortality risk estimates assume that all treatment options are readily accessible to patients. However, the average waiting time for a deceased donor kidney transplant in Ontario can be several years.

Conclusions:

After minor modifications, the iChoose Kidney risk calculator provides reliable survival estimates in patients with kidney failure from Ontario, Canada. Given the similarity in model performance between the updated model and the original model, with race removed, we will continue to use our original simplified model, but we will remove race. Our updated Dialysis vs. Kidney Transplant-Estimated Survival in Ontario Risk Calculator can continue to be a valuable tool for healthcare professionals to use with patients who are receiving maintenance dialysis to provide individualized survival estimates for dialysis versus transplantation, supporting informed decision-making about kidney transplantation.

Keywords

maintenance dialysis kidney transplantation risk calculator survival

Introduction

Compared with dialysis, a kidney transplant from a living or deceased donor offers patients a longer and better quality of life at a fraction of the cost—over five years, every 100 kidney transplants save the Canadian healthcare system ~$20 million in averted dialysis costs.^1,2 Unfortunately, there is a shortage of kidneys available for transplant, and many barriers impede patients’ access to transplants.^3-7 One significant barrier that patients report is limited education about kidney transplantation.⁸

The iChoose Kidney risk calculator was developed in the United States to provide prognostic information to help patients receiving maintenance dialysis make more educated treatment decisions (i.e., dialysis vs kidney transplantation). The risk calculator uses age, sex, race, ethnicity, time on dialysis, and comorbidities (i.e., diabetes, hypertension, cardiovascular disease, and low albumin) to provide patients with individualized three-year mortality estimates for different treatment options comparing dialysis to living or deceased donor kidney transplantation.⁹ The risk calculator was evaluated in a multicenter randomized controlled trial, finding that the calculator significantly increased knowledge about transplantation in patients undergoing the kidney transplant evaluation.¹⁰ We externally validated and adapted the iChoose Kidney risk calculator. With minor modifications, the calculator provided reliable survival estimates in patients with kidney failure from Ontario, Canada, using the variables age, sex, race, time on dialysis, and comorbidities. We titled our calculator the Dialysis vs. Kidney Transplant-Estimated Survival in Ontario Risk Calculator.¹¹

While our current Dialysis vs. Kidney Transplant-Estimated Survival in Ontario Risk Calculator was found to predict three-year mortality accurately,¹¹ our original model had some limitations. First, the model was validated based on data that is now several years old, and patient characteristics and survival rates may have changed. Second, the model included race. The use of race in clinical algorithms has come under scrutiny for perpetuating racial bias in medicine.^12,13 Third, the United States version of iChoose Kidney was updated to include dialysis modality given evidence of survival differences between different modality types.¹⁴ Last, our original model included all patients receiving dialysis. However, we know that most patients on dialysis are not eligible for transplant.¹⁵ A more accurate cohort may restrict to patients who are likely eligible for a transplant, which is the intended population for the risk calculator.¹⁵ Therefore, we conducted this study to externally validate the updated iChoose Kidney risk calculator¹⁴ in patients with kidney failure from Ontario, Canada, including additional variables (i.e., dialysis modality and additional dialysis categories), using more recent data, removing race, and restricting to patients who are likely eligible for transplant.

Methods

Design and Setting

Using linked administrative healthcare databases from Ontario, Canada, held at ICES (formerly known as the Institute for Clinical Evaluative Sciences; ices.on.ca), we conducted an external validation study. These datasets were linked using unique encoded identifiers and analyzed at ICES. ICES is an independent, non-profit research institute whose legal status under Ontario’s health information privacy law allows it to collect and analyze health care and demographic data, without consent for health system evaluation and improvement. The use of data in this project was authorized under section 45 of Ontario’s Personal Health Information Protection Act, which does not require review by a research ethics board. To report this study, we used the REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) Statement (Supplementary Table 1) and the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement (Supplementary Table 2).^16,17

Data Sources

We used several linked administrative databases, including the Canadian Organ Replacement Register (CORR) to identify patients receiving maintenance dialysis and kidney transplant recipients. CORR is 99% complete for dialysis records and has been found to accurately identify kidney transplant recipients with a sensitivity >95% compared to transplant center data.^18,19 To obtain information on vital statistics, we used the Registered Persons Database, while the Ontario Health Insurance Plan was used to identify physician diagnoses and billings. The Canadian Institute for Health Information Discharge Abstract Database was used to identify diagnostic and procedural information occurring during a hospitalization. Additional information on databases and coding definitions is available in Supplementary Table 3.

Study Populations

Maintenance dialysis

We included incident patients receiving maintenance dialysis (i.e., individuals on dialysis for at least 90 days) in Ontario, Canada, from January 1, 2011, to August 31, 2021. We excluded individuals who were aged <18 years or >80 years (it is relatively rare for older patients on maintenance dialysis to receive a kidney transplant)¹⁵ and individuals who previously received a solid organ transplant (including kidney). We also excluded anyone with a recorded contraindication to kidney transplant, including an end-stage kidney disease–modified Charlson Comorbidity Index score ≥7 (higher score indicates more comorbidity), home oxygen use, dementia, living in a long-term care facility, received at least one physician house call in the past year, and select malignancies (i.e., lung cancer, lymphoma, cervical cancer, colorectal cancer, liver cancer, active multiple myeloma, bladder cancer). Based on previous work, we found transplants are relatively rare in these individuals, with less than 3% of individuals with one or more of these conditions receiving a kidney transplant.¹⁵ The cohort entry date was the date of maintenance dialysis initiation.

Kidney transplant recipients

We included patients who received a kidney transplant in Ontario, Canada, from January 1, 2011, to August 31, 2021. Like the dialysis cohort, we excluded individuals aged <18 or >80 years and individuals who received a previous solid organ transplant (including kidney). To ensure comparability between the dialysis and kidney transplant recipient cohorts, we also excluded anyone with a recorded contraindication to transplant, as described earlier. A priori, we anticipated that a small proportion of kidney transplant recipients would have a recorded contraindication to kidney transplant in our databases given there can be inaccuracies in administrative databases. The cohort entry date was the date of the kidney transplant surgery.

The maintenance dialysis and kidney transplant cohorts were not mutually exclusive (i.e., individuals in the dialysis cohort could receive a kidney transplant during follow-up and then would start contributing person-time to the kidney transplant group).

Study Variables

Our outcome was three-year all-cause mortality, a fixed-time endpoint that is accurately ascertained in our databases.²⁰ To align with the United States version, we followed both cohorts from the cohort entry date for a maximum of three years (maximum follow-up date of August 31, 2024). However, unlike the United States version, low event numbers prevented us from examining one-year mortality. All individuals were followed from cohort entry (dialysis initiation or kidney transplantation) for up to three years. Completeness of follow-up is high in Ontario, with fewer than 0.5% of individuals emigrating annually, representing the only reason for loss to follow-up.²¹ Using the formulas derived from the United States version of the model (Supplementary Methods), which used logistic regression to calculate mortality risk, we incorporated the same variables used in our original model: sex assigned at birth (male vs female), age (continuous), cardiovascular disease (i.e., congestive heart failure, coronary artery disease, stroke/transient ischemic attack, peripheral vascular disease, myocardial infarction, chronic obstructive pulmonary disease, and other cardiac disease), hypertension, and diabetes.¹⁴ We modified the time on dialysis to include finer categories (i.e., <6 months, 6 to 12 months, >1 to 2 years, >2 to 3 years, >3 to 5 years, >5 to 7 years, >7 to 10 years, >10 to 14 years, >14 years) and added dialysis modality (peritoneal dialysis, home hemodialysis, in-center dialysis). The United States version of the iChoose Kidney risk calculator included the variables above but also included race (White, Black or African American, other), ethnicity (Hispanic, non-Hispanic), and low albumin (<35 g/L). Our original work that validated this model using Canadian data found that we had a large proportion of missingness for albumin, so we opted to exclude this variable.¹¹ From the original iChoose model, we also excluded ethnicity (our databases do not have this information), and we removed race. Best practice recommends the removal of race from medical calculators.^12,13 By excluding albumin, ethnicity, and race from our model, we assumed all individuals had normal albumin, all individuals identified as non-Hispanic, and as being non-Black race and non-other race. We found that 2.6% of individuals were missing donor type (living vs deceased donor kidney), which we imputed with receipt of a deceased donor kidney using mode imputation. The Supplementary Methods contain the formulas utilized to calculate mortality risk.

Statistical Analysis

We described continuous variables as medians (25th, 75th percentile) and categorical variables as counts (percentages). To determine how well the risk calculator could discriminate between individuals who did and did not die within three years, we used the area under the receiver operating curve (C-statistic). A C-statistic of 0.5 indicates that the risk calculator cannot discriminate mortality better than chance alone, and a higher value indicates a better ability to differentiate between those who did and did not die. To assess the agreement between the observed and predicted probabilities, we visually compared the observed and predicted mortality risk within deciles of predicted mortality probability (i.e., calibration). To further assess calibration graphically, we used loess-smoothed calibration plots.^22,23 Poor calibration occurs when the smoothed calibration curve diverges from the ideal 45° diagonal line (i.e., perfect calibration). We provided 95% confidence intervals around our risk estimates to indicate the degree of uncertainty.²⁴ To address over- or under-prediction (calibration-in-the-large), intercepts were adjusted using a correction factor where necessary.²⁵ No causal inference methods were applied, and all survival estimates comparing patients receiving maintenance dialysis to kidney transplant recipients are intended for risk communication rather than estimation of treatment effects. We conducted all analyses using SAS version 9.4 (SAS Institute, Cary, NC).

Sensitivity Analysis

In a sensitivity analysis, we restricted the accrual period from January 1, 2011, to March 1, 2017, stopping follow-up on March 1, 2020, and assessed the discrimination and calibration. This allowed us to stop follow-up when the COVID-19 pandemic started. Previous research has demonstrated that the pandemic substantially impacted mortality in patients receiving dialysis²⁶ and kidney transplant recipients.²⁷

Post-hoc Analysis

Prediction models should be parsimonious.^28-30 Therefore, we performed a post-hoc analysis using the simpler equations (i.e., equations with fewer variables) used in our original Canadian external validation study of the iChoose Kidney tool (i.e., age, sex, hypertension, diabetes, cardiovascular disease, time on dialysis [<6 months, 6-12 months, >12 months]),¹¹ with the only modification being the removal of the race variable. Table 1 summarizes the variables included in our updated model with additional variables, our more parsimonious post-hoc model, and the variables included in our original external validation study.¹¹

Table 1.

Description of Different Models to Predict 3-Year Survival.

	Updated model^a	Original model removing race-post-hoc analysis	Original model with race (previously published)^b
Variables	- Age (18-80 years)^c - Sex (Male vs Female) - Hypertension - Diabetes - Cardiovascular disease - Time on dialysis (<6 months, 6 to 12 months, >1 to 2 years, >2 to 3 years, >3 to 5 years, >5 to 7 years, >7 to 10 years, >10 to 14 years, >14 years)^d - Dialysis modality (peritoneal dialysis, home hemodialysis, in-center dialysis)^e	- Age (18-80 years)^c - Sex (Male vs Female) - Hypertension - Diabetes - Cardiovascular disease - Time on dialysis (<6 months, 6-12 months, >12 months)	- Age (18-80 years)^c - Sex (Male vs Female) - Hypertension - Diabetes - Cardiovascular disease - Race (white, black, other) - Time on dialysis (<6 months, 6-12 months, >12 months)

Updated model removed the race variable, added dialysis modality, and incorporated additional dialysis categories.

Model evaluated in our original external validation study. Results for the external validation study have been previously published.¹¹

Age is modeled as a continuous integer variable.

Variable modified from the original model to include additional dialysis categories.

New variable added.

Results

Baseline Characteristics

We included 24 793 patients receiving maintenance dialysis (Supplementary Figure 1) and 5398 kidney transplant recipients (3604 recipients of a deceased donor kidney [66.8%] and 1794 recipients of a living donor kidney [33.2%]) (Supplementary Figure 2). Table 2 provides the baseline characteristics for patients receiving maintenance dialysis and kidney transplant recipients, with kidney transplant recipients further stratified by individuals with a deceased or living donor kidney transplant. Compared with kidney transplant recipients, patients receiving maintenance dialysis had a higher median age (64 vs 52 years), were more likely to have diabetes (56.6% vs 36.0%), and were more likely to have cardiovascular disease (58.7% vs 36.4%). Compared with living donor kidney transplant recipients, recipients of a deceased donor had a higher median age (54 vs 47 years), were more likely to have diabetes (40.1% vs 27.8%), and had a longer time on dialysis (e.g., >7 to 10 years on dialysis: 11.1% vs 1.1%).

Table 2.

Baseline Characteristics for Patients Receiving Maintenance Dialysis and Kidney Transplant Recipients.

Characteristics	Maintenance dialysis (n = 24 793)	Kidney transplant recipients (n = 5398)	Living donor kidney transplant recipients (n = 1794)	Deceased donor kidney transplant recipients (n = 3604)
Age, years	64 (53, 72)	52 (41, 61)	47 (35, 57)	54 (44, 62)
18 to <40	2110 (8.5)	1165 (21.6)	583 (32.5)	582 (16.1)
40 to <50	2642 (10.7)	1211 (22.4)	412 (23.0)	799 (22.2)
50 to <60	4959 (20.0)	1518 (28.1)	458 (25.5)	1060 (29.4)
60 to <70	7256 (29.3)	1211 (22.4)	278 (15.5)	933 (25.9)
70 to <80	7826 (31.6)	293 (5.4)	63 (3.5)	230 (6.4)
Sex assigned at birth
Female	9107 (36.7)	2042 (37.8)	712 (39.7)	1330 (36.9)
Male	15 686 (63.3)	3356 (62.2)	1082 (60.3)	2274 (63.1)
Neighborhood income quintile^a
1 (lowest)	6852 (27.6)	1292 (23.9)	293 (16.3)	999 (27.7)
2	5568 (22.5)	1124 (20.8)	346 (19.3)	778 (21.6)
3 (middle)	4812 (19.4)	1060 (19.6)	344 (19.2)	716 (19.9)
4	4127 (16.6)	1025 (19.0)	393 (21.9)	632 (17.5)
5 (highest)	3434 (13.9)	897 (16.6)	418 (23.3)	479 (13.3)
Rural (population <10 000)^b	3092 (12.5)	489 (9.1)	190 (10.6)	299 (8.3)
Cause of end-stage kidney disease
Glomerulonephritis/ autoimmune diseases	2953 (11.9)	1458 (27.0)	519 (28.9)	939 (26.1)
Polycystic kidney disease	990 (4.0)	587 (10.9)	193 (10.8)	394 (10.9)
Diabetes	9395 (37.9)	1265 (23.4)	286 (15.9)	979 (27.2)
Renal vascular disease	2731 (11.0)	571 (10.6)	112 (6.2)	459 (12.7)
Other	6284 (25.3)	627 (11.6)	234 (13.0)	393 (10.9)
Unknown/Missing	2440 (9.8)	890 (16.5)	450 (25.1)	440 (12.2)
Dialysis modality
In-center hemodialysis	18 997 (76.6)	3075 (57.0)	794 (44.3)	2281 (63.3)
Home hemodialysis	265 (1.1)	130 (2.4)	39 (2.2)	91 (2.5)
Peritoneal dialysis	5531 (22.3)	1605 (29.7)	433 (24.1)	1172 (32.5)
Pre-emptive kidney transplant^c		588 (10.9)	528 (29.4)	60 (1.7)
Dialysis vintage^d		2.99 (1.30, 5.00)	0.89 (0.00, 1.99)	4.07 (2.54, 5.81)
<6 months		825 (15.3)	723 (40.3)	102 (2.8)
6 months to 12 months		321 (5.9)	238 (13.3)	83 (2.3)
>1 to 2 years		795 (14.7)	388 (21.6)	407 (11.3)
>2 to 3 years		762 (14.1)	199 (11.1)	563 (15.6)
>3 to 5 years		1342 (24.9)	171 (9.5)	1171 (32.5)
>5 to 7 years		783 (14.5)	45 (2.5)	738 (20.5)
>7 to 10 years		418 (7.7)	20 (1.1)	398 (11.0)
>10 years		152 (2.8)	10 (0.6)	142 (4.0)
Body mass index (kg/m²)	27.8 (24.1, 32.6)	26.7 (23.1, 31.0)	26.2 (22.8, 30.4)	26.9 (23.2, 31.1)
Missing body mass index	2925 (11.8)	926 (17.2)	611 (34.1)	315 (8.7)
Cardiovascular disease^e	14 546 (58.7)	1966 (36.4)	611 (34.1)	1355 (37.6)
Diabetes^f	14 039 (56.6)	1942 (36.0)	498 (27.8)	1444 (40.1)
Hypertension^g	21 620 (87.2)	4382 (81.2)	1391 (77.5)	2991 (83.0)
End-stage kidney disease–modified Charlson Comorbidity Index^h
0	11 130 (44.9)	3956 (73.3)	1386 (77.3)	2570 (71.3)
1	4582 (18.5)	738 (13.7)	232 (12.9)	506 (14.0)
2	2523 (10.2)	297 (5.5)	88 (4.9)	209 (5.8)
3	3426 (13.8)	282 (5.2)	58 (3.2)	224 (6.2)
4	1267 (5.1)	49 (0.9)	12 (0.7)	37 (1.0)
5	1335 (5.4)	76 (1.4)*	18 (1.0)*	58(1.6)*
6	530 (2.1)	76 (1.4)*	18 (1.0)*	58(1.6)*

Note. Data are presented as n (%) or median (25^th, 75^th percentile). All baselines were assessed using the dialysis initiation date (for non-pre-emptive kidney transplants, baselines were assessed from the date of dialysis initiation, not the date of kidney transplant) or the date of pre-emptive kidney transplant. The exception is dialysis vintage which was assessed looking backwards from the date of kidney transplant.

Neighborhood level income was utilized to create income quintiles. Less than 0.5% of individuals had missing data, which was imputed into category 3.

Less than 0.5% of individuals had missing data, which was imputed into urban.

Pre-emptive kidney transplant was defined as receiving a kidney transplant without any evidence of dialysis prior to the transplant.

The <6 months on dialysis category included pre-emptive kidney transplants. The dialysis vintage categories of 10 to 14 years and 14+ years were combined due to small cells; in accordance with ICES privacy policies, cell sizes less than or equal to five cannot be reported.

Cardiovascular disease was defined as having at least one of the following conditions: congestive heart failure, coronary artery disease, stroke/transient ischemic attack, peripheral vascular disease, myocardial infarction, chronic obstructive pulmonary disease, or other cardiac disease. Assessed in the five years prior to dialysis initiation except for pre-emptive kidney transplant (i.e., assessed in the five years prior to transplant date).

Diabetes was defined as three physician (Ontario Health Insurance Plan, OHIP) diagnosis codes for diabetes in a one-year period. Includes any evidence of diabetes meeting the aforementioned definition prior to cohort entry date.

Hypertension was defined as at least one hospitalization or two OHIP diagnosis codes in a two-year period or one OHIP code/hospitalization within two years. Includes any evidence of hypertension meeting the aforementioned definition prior to cohort entry date.

A higher score on the end-stage kidney disease–modified Charlson Comorbidity Index signifies increased comorbidity. Assessed in the five years prior to dialysis initiation except for pre-emptive kidney transplant (i.e., assessed in the five years prior to transplant date).

End-stage kidney disease–modified Charlson Comorbidity Index categories 5 and 6 were combined due to small cells. In accordance with ICES privacy policies, cell sizes less than or equal to five cannot be reported.

Compared with the original US population used to validate the iChoose Kidney model, patients in our study receiving maintenance dialysis were of similar age (64 vs 64 years), and a similar proportion had cardiovascular disease (58.7% vs 57.6%), but our study had a slightly lower percentage of females (36.7% vs 44.1%).⁹ Compared to the US study, the kidney transplant recipients in our study were slightly older (52 vs 49 years) and had a slightly lower proportion of females (37.8% vs 39.3%), but our study had a higher proportion with cardiovascular disease (36.4% vs 22.4%).⁹ It was difficult to compare diabetes and hypertension across studies given the US study only provided information on whether these conditions were the cause of their kidney failure versus whether the individual had any evidence of the condition.

Mortality

In the dialysis cohort, over a median follow-up of 2.5 years, 7521 (30.3%) patients receiving maintenance dialysis died. In the kidney transplant recipient cohort, the median follow-up was 2.9 years, and 393 kidney transplant recipients (7.3%) died. For living donor kidney transplant recipients, the median follow-up was 3.0 years, during which 41 (2.3%) living donor transplant recipients died, compared to 2.8 years in deceased donor recipients, during which 352 (9.8%) died.

Predictive Model Discrimination

Our updated model, which added dialysis modality, additional dialysis categories, and removed race, had moderate discriminative ability. The C-statistic for three-year mortality was 0.67 (95% CI: 0.67, 0.68) in the maintenance dialysis cohort, 0.76 (95% CI: 0.74, 0.79) in the kidney transplant recipient cohort, 0.73 (95% CI: 0.67, 0.80) in the living donor kidney transplant recipient cohort, and 0.73 (95% CI: 0.70, 0.75) in the deceased donor kidney transplant cohort (Table 3).

Table 3.

C-statistics for Three-Year Survival Prediction for Patients Receiving Maintenance Dialysis and Kidney Transplant Recipients, Presented by the Different Models.

	Updated model^a	Original model removing race: post-hoc analysis	Original model with race (previously published)^b
Cohort	C-statistic (95% CI)	C-statistic (95% CI)	C-statistic (95% CI)
Maintenance dialysis	0.67 (0.67, 0.68)	0.67 (0.66, 0.68)	0.70 (0.69, 0.70)
Kidney transplant recipients	0.76 (0.74, 0.79)	0.74 (0.72, 0.77)	0.72 (0.69, 0.75)
Deceased donor transplant	0.73 (0.70, 0.75)	0.71 (0.68, 0.73)	0.68 (0.64, 0.71)
Living donor transplant	0.73 (0.67, 0.80)	0.72 (0.65, 0.79)	0.71 (0.64, 0.78)

Updated model removed the race variable, added dialysis modality, and incorporated additional dialysis categories.

Model evaluated in our original external validation study. Results for the external validation study have been previously published.¹¹

Predictive Model Calibration

The 3-year observed mortality was lower than the predicted risk in the maintenance dialysis cohort (30.3% vs 44.6%). However, after we recalibrated the intercept, the observed and predicted three-year mortality were similar (30.3% vs 31.3%). Figure 1 provides the observed and predicted mortality risks in the maintenance dialysis population presented by deciles based on the predicted probability ranking after intercept recalibration. Calibration plots before and after intercept recalibration are provided in Figure 2a and b.

Figure 1.

Observed and predicted mortality probability for the updated model in patients receiving maintenance dialysis after intercept recalibration.

Figure 2.

Loess-smoothed calibration plots for the updated model comparing the observed and predicted mortality probability for patients receiving maintenance dialysis before (a) and after (b) intercept recalibration.

In the kidney transplant recipient cohort, the three-year observed mortality was lower than the predicted mortality risk (7.3% vs 9.8%). After recalibrating the intercepts, the observed and predicted mortality were nearly identical (7.3% vs 7.4%). Figure 3 demonstrates the observed and predicted mortality risks in the kidney transplant population presented by deciles based on the predicted probability ranking after intercept recalibration, while calibration plots before and after intercept recalibration are provided in Figure 4a and b.

Figure 3.

Observed and predicted mortality probability for the updated model in kidney transplant recipients after intercept recalibration.

Figure 4.

Loess-smoothed calibration plots for the updated model comparing the observed and predicted mortality probability for kidney transplant recipients before (a) and after (b) intercept recalibration.

When stratifying the kidney transplant recipient cohort into recipients of a living versus deceased donor kidney transplant, we found the three-year observed mortality was lower than the predicted mortality risk in the living donor kidney transplant recipient cohort (2.3% vs 4.5%) and in the deceased donor kidney transplant recipient cohort (9.8% vs 11.6%). However, after intercept recalibration, the observed and predicted mortality were identical (2.3% vs 2.3% in recipients of a living donor kidney transplant and 9.8% versus 9.8% in recipients of a deceased donor kidney transplant). Supplementary Figures 3 and 4 present the calibration plots before and after intercept recalibration in recipients of a deceased donor kidney and recipients of a living donor kidney, respectively.

Sensitivity Analysis

When we restricted our accrual period from 2011 to 2017, our cohort size decreased to 13 371 patients receiving maintenance dialysis and 2938 patients receiving a kidney transplant. Over the three-year follow-up, 3993 (29.9%) patients receiving maintenance dialysis died, and 185 (6.3%) kidney transplant recipients died. The C-statistics (Supplementary Table 4) and calibration were similar to our original cohort. For example, in the dialysis cohort, the c-statistic was 0.68 (95% CI: 0.67, 0.69), and after intercept calibration, the predicted three-year mortality risk was 30.9% compared to an observed mortality risk of 29.9%. In the kidney transplant recipient cohort, the c-statistic was 0.76 (95% CI: 0.73, 0.79), and after intercept recalibration, the predicted three-year mortality risk was 6.4% compared to an observed mortality risk of 6.3%.

Post-hoc analysis

In a post-hoc analysis, we used the model we used in our original external validation of the iChoose kidney calculator,¹¹ except we removed the race variable. We found that the discrimination (Table 3) and calibration were similar to our updated model. For example, the C-statistic was 0.67 (95% CI: 0.66, 0.68) for the dialysis population and 0.74 (95% CI: 0.72, 0.77) in the kidney transplant recipient population.

The 3-year observed mortality was lower than the predicted risk in the maintenance dialysis cohort (30.3% vs 36.7%). However, after we recalibrated the intercept, the observed and predicted 3-year mortality were similar (30.3% vs 30.7%). In the kidney transplant recipient cohort, the 3-year observed mortality was higher than the predicted mortality risk (7.3% vs 5.6%). After recalibrating the intercepts, the observed and predicted mortality were nearly identical (7.3% vs 7.2%). In the recipients of a living donor kidney transplant cohort, we found the 3-year observed mortality was lower than the predicted mortality (2.3% vs 3.1%). In the deceased donor kidney transplant recipient cohort, the observed mortality was higher than the predicted rate (9.8% vs 6.2%). However, after intercept recalibration, the observed and predicted mortality were similar (2.3% vs 2.3% in recipients of a living donor kidney transplant, and 9.8% versus 9.7% in recipients of a deceased donor kidney transplant). Supplementary Figures 6 and 8 to 10 present the loess-smoothed calibration plots for the post-hoc analysis before and after intercept recalibration in patients receiving maintenance dialysis, kidney transplant recipients, and recipients of a deceased donor kidney and recipients of living donor kidney, respectively. Supplementary Figures 5 and 7 demonstrate the observed and predicted mortality risks in the maintenance dialysis and kidney transplant population presented by deciles based on the predicted probability ranking after intercept recalibration.

Discussion

After minor modifications, we found both our updated model, which included additional variables, and our more parsimonious model, which removed race, were able to provide accurate mortality estimates in patients with kidney failure from Ontario, Canada. After updating the model to remove race, our results suggest we can continue to use our Dialysis vs. Kidney Transplant-Estimated Survival in Ontario Risk Calculator to support informed decision-making about kidney transplantation in patients receiving maintenance dialysis.

After recalibration of the intercepts in all our models, we found adequate calibration of our tool with similar observed and predicted 3-year mortality probabilities. We also found that the C-statistics in our updated model (i.e., removed race, incorporated dialysis modality and additional time on dialysis categories) and in our original model revised to remove race were similar to what we found in our original Canadian external validation study. For example, the C-statistic in kidney transplant recipients in our updated model was 0.76 and 0.74 in our model which removed race, compared to 0.72 in our original study.¹¹ In the maintenance dialysis population, the C-statistic was slightly lower, with a C-statistic of 0.67 in our updated model and 0.67 in our model, which removed race, compared to 0.70 in our original study.¹¹ Our results were also comparable to the updated US model which incorporated dialysis modality and additional dialysis vintage categories, with a C-statistic of 0.71 in the dialysis population and 0.71 in the kidney transplant population.¹⁴ Notably, the C-statistics in our study are higher than those of several commonly used prediction tools, including the Kidney Donor Risk Index and Fracture Risk Assessment Tool, which have C-statistics of 0.62.^31,32

Before incorporating additional variables (e.g., dialysis modality) to improve the model’s predictive ability, it is vital to consider data availability, simplicity, and accuracy. Given the predictive and discriminative abilities of the model did not improve meaningfully when we incorporated additional variables, in a post-hoc analysis, we examined our original Dialysis vs. Kidney Transplant-Estimated Survival in Ontario Risk Calculator model, with the removal of race as the only change. We found that C-statistics were similar to the original model and the model with the updated variables. For example, in the dialysis cohort, the C-statistic in our post-hoc model, which removed race, was 0.67 compared to 0.67 in the model with the additional variables and 0.70 in the model in our original publication.¹¹ Similar to the model that incorporated additional variables, after recalibrating the intercepts in the simpler model which removed race, the risk calculator was adequately calibrated. Given the C-statistics across models were similar and, after recalibration, the calibration between models was comparable, we opted to update our Dialysis vs. Kidney Transplant-Estimated Survival in Ontario Risk Calculator using our original model revised to remove race and not incorporate dialysis modality and the finer time on dialysis categories. We aimed to make the model easy for healthcare professionals to use without compromising predictive accuracy.

A significant, patient-reported barrier to accessing kidney transplantation is inadequate education about kidney transplantation.⁸ The US version of the risk calculator found that the tool significantly increased knowledge about mortality risks with dialysis versus transplantation, including deceased versus living donor transplant, in patients undergoing the kidney transplant evaluation.¹⁰ Specifically, a multicenter randomized controlled trial was conducted, where patients were randomized to standard transplant education versus the standard education with the iChoose kidney calculator. Future work in Ontario should evaluate the impact of the Ontario version of the calculator on patient knowledge and access to kidney transplants (e.g., the effect of the calculator on kidney transplant referrals). Our risk calculator helps healthcare professionals counsel their patients receiving maintenance dialysis who are likely eligible for transplant and under consideration for transplant referral. Providers use the survival estimates to visually demonstrate to patients the benefits of transplantation and to highlight the importance of finding a living donor. In the future, we will explore the possibility of having patients use the tool on their own to help further empower them to make decisions about their health.

Our work has some limitations. Our mortality risk estimates assume that all treatment options are readily accessible to patients. However, the average waiting time for a deceased donor kidney transplant in Ontario is 5 years depending on the patient’s blood type.³³ While our definition of no recorded contraindications to kidney transplant is accurate,¹⁵ administrative databases cannot capture all the complexities of transplant eligibility. We were not able to determine the model’s accuracy for predicting long-term mortality (e.g., 5- and 10-year risk); we opted to use more recent data to ensure patient characteristics were still relevant today instead of following patients for a longer period of time. Unlike the original US model, in an additional analysis, we did not use a time-to-event analysis to evaluate mortality. Instead, we deemed it appropriate to solely use logistic regression, with our outcome being captured over a fixed-time horizon with complete follow-up. In the original US model, they found c-statistics were unchanged when using a time to event analysis.⁹ Last, results may not be generalizable to other provinces in Canada.

Conclusions

Our study demonstrates that we have a risk calculator that can adequately predict three-year mortality in patients receiving maintenance dialysis with no recorded contraindications to transplant from Ontario, Canada. Once our website is updated (example in Supplementary Figure 11), the Dialysis vs. Kidney Transplant-Estimated Survival in Ontario risk calculator can continue to be used to help Ontario patients receiving maintenance dialysis make an informed decision about whether they should pursue a kidney transplant.

Supplemental Material

sj-docx-1-cjk-10.1177_20543581261437298 – Supplemental material for Predicting Three-Year Survival in Patients Receiving Maintenance Dialysis: An External Validation and Updated Multivariable Prediction Model for iChoose Kidney in Ontario, Canada

Supplemental material, sj-docx-1-cjk-10.1177_20543581261437298 for Predicting Three-Year Survival in Patients Receiving Maintenance Dialysis: An External Validation and Updated Multivariable Prediction Model for iChoose Kidney in Ontario, Canada by Kyla L. Naylor, Yuguang Kang, Eric McArthur, Amit X. Garg, Rachel E. Patzer, Susan McKenzie, S. Joseph Kim, Matthew Weir, Seychelle Yohanna, Gregory Knoll and Darin Treleaven in Canadian Journal of Kidney Health and Disease

Footnotes

Acknowledgements

Dr. Naylor is supported by a Health System Impact Embedded Early Career Researcher Award from the Canadian Institutes of Health Research. Dr. Garg was supported by the Kay Family Chair in Transformational Kidney Care. This study was supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health (MOH) and the Ministry of Long-Term Care (MLTC). This document used data adapted from the Statistics Canada Postal CodeOM Conversion File, which is based on data licensed from Canada Post Corporation, and/or data adapted from the Ontario Ministry of Health Postal Code Conversion File, which contains data copied under license from ©Canada Post Corporation and Statistics Canada. Parts of this material are based on data and/or information compiled and provided by the Canadian Institute for Health Information and the Ontario Ministry of Health. The analyses, conclusions, opinions, and statements expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement is intended or should be inferred. Parts of this material are based on data and information provided by Ontario Health (OH). The opinions, results, view, and conclusions reported in this article are those of the authors and do not necessarily reflect those of OH. No endorsement by OH is intended or should be inferred.

ORCID iDs

Kyla L. Naylor

Eric McArthur

Amit X. Garg

Seychelle Yohanna

Ethics Approval and Consent to Participate

ICES is an independent, non-profit research institute whose legal status under Ontario’s health information privacy law allows it to collect and analyze healthcare and demographic data, without consent, for health system evaluation and improvement. The use of data in this project was authorized under section 45 of Ontario’s Personal Health Information Protection Act, which does not require review by a research ethics board.

Consent for Publication

Consent for publication was obtained from all authors.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Availability of Data and Materials

The dataset from this study is held securely in coded form at ICES. While legal data-sharing agreements between ICES and data providers (e.g., healthcare organizations and government) prohibit ICES from making the dataset publicly available, access may be granted to those who meet prespecified criteria for confidential access, available at (email: das@ices.on.ca). The full dataset-creation plan and underlying analytic code are available from the authors upon request, understanding that the computer programs may rely upon coding templates or macros that are unique to ICES and are therefore either inaccessible or may require modification.

Supplemental Material

Supplemental material for this article is available online.

References

Chaudhry

Peracha

Sharif

Survival for waitlisted kidney failure patients receiving transplantation versus remaining on waiting list: systematic review and meta-analysis. BMJ. 2022;376:e068769. doi:10.1136/bmj-2021-068769.

Klarenbach

Tonelli

Chui

Manns

Economic evaluation of dialysis therapies. Nat Rev Nephrol. 2014;10:644-652. doi:10.1038/nrneph.2014.145.

Yohanna

Naylor

Mucsi

, et al. A quality improvement intervention to Enhance Access to Kidney Transplantation and Living Kidney Donation (EnAKT LKD) in patients with chronic kidney disease: clinical research protocol of a cluster-randomized clinical trial. Can J Kidney Health Dis. 2021;8:2054358121997266. doi:10.1177/2054358121997266.

Kazley

Simpson

Chavin

Baliga

Barriers facing patients referred for kidney transplant cause loss to follow-up. Kidney Int. 2012;82(9):1018-1023. doi:10.1038/KI.2012.255.

Garg

AX.

Helping more patients receive a living donor kidney transplant. Clin J Am Soc Nephrol. 2018;13(12):1918-1923. doi:10.2215/CJN.00760118.

Barnieh

McLaughlin

Manns

Klarenbach

Yilmaz

Hemmelgarn

BR.

Barriers to living kidney donation identified by eligible candidates with end-stage renal disease. Nephrol Dial Transplant. 2011;26(2):732-738. doi:10.1093/ndt/gfq388.

Knight

Teeter

Graviss

, et al. Barriers to preemptive renal transplantation: a single center questionnaire study. Transplantation. 2015;99(3):576-579. doi:10.1097/TP.0000000000000357.

Getchell

McKenzie

Sontrop

Hayward

McCallum

Garg

AX.

Increasing the rate of living donor kidney transplantation in Ontario: donor- and recipient-identified barriers and solutions. Can J Kidney Health Dis. 2017;4:205435811769866. doi:10.1177/2054358117698666.

Patzer

Basu

Larsen

, et al. iChoose kidney: a clinical decision aid for kidney transplantation versus dialysis treatment. Transplantation. 2016;100(3):630-639. doi:10.1097/TP.0000000000001019.

10.

Patzer

McPherson

Basu

, et al. Effect of the iChoose kidney decision aid in improving knowledge about treatment options among transplant candidates: a randomized controlled trial. Am J Transplant. 2018;18:1954-1965. doi:10.1111/ajt.14693.

11.

Tan

Garg

McArthur

, et al. Predicting 3-year survival in patients receiving maintenance dialysis: an external validation of iChoose kidney in Ontario, Canada. Can J Kidney Health Dis. 2018;5:2054358118799693. doi:10.1177/2054358118799693.

12.

Vyas

Eisenstein

Jones

DS.

Hidden in plain sight—reconsidering the use of race correction in clinical algorithms. N Engl J Med. 2020;383(9):874-882. doi:10.1056/NEJMms2004740.

13.

Eneanya

Yang

Reese

PP.

Reconsidering the consequences of using race to estimate kidney function. JAMA. 2019;322(2):113. doi:10.1001/jama.2019.5774.

14.

Gander

Basu

McPherson

, et al. iChoose kidney for treatment options. Transplantation. 2018;102(9):e370-e371. doi:10.1097/TP.0000000000002301.

15.

Wang

Naylor

Luo

, et al. Using administrative health care databases to identify patients with end-stage kidney disease with no recorded contraindication to receiving a kidney transplant. Can J Kidney Health Dis. 2022;9:205435812211117. doi:10.1177/20543581221111712.

16.

Benchimol

Smeeth

Guttmann

, et al. The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement. PLoS Med. 2015;12:1-22.

17.

Collins

Reitsma

Altman

Moons

KG.

Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med. 2015;162:55-63.

18.

Canadian Institute for Health Information. Canadian Organ Replacement Register Annual Report: Treatment of End-Stage Organ Failure in Canada, 2004 to 2013. Ottawa, ON: Canadian Institute for Health Information; 2015.

19.

Lam

Kim

Knoll

GA.

Validation of kidney transplantation using administrative data. Can J Kidney Heal Dis. 2015;2:20.

20.

Quinn

Laupacis

Austin

PPC

, et al. Using administrative datasets to study outcomes in dialysis patients: a validation study. Med Care. 2010;48(8):745-750. doi:10.1097/MLR.0b013e3181e419fd.

21.

Ontario Ministry of Finance. Ontario demographic quarterly: highlights of third quarter. https://www.ontario.ca/page/ontario-demographic-quarterly-highlights-third-quarter. Published 2026. Accessed February 1, 2026.

22.

Austin

Steyerberg

EW.

Graphical assessment of internal and external calibration of logistic regression models by using loess smoothers. Stat Med. 2014;33(3):517-535. doi:10.1002/sim.5941.

23.

Harrell

Jr.

Regression Modeling Strategies: With applications to Linear Models, Logistic Regression, and Survival Analysis. New York, NY: Springer-Verlag; 2001.

24.

Riley

Collins

Kirton

, et al. Uncertainty of risk estimates from clinical prediction models: rationale, challenges, and approaches. BMJ. 2025;388:e080749. doi:10.1136/bmj-2024-080749.

25.

Janssen

KJM

Moons

KGM

Kalkman

Grobbee

Vergouwe

. Updating methods improved the performance of a clinical prediction model in new patients. J Clin Epidemiol. 2008;61(1):76-86. doi:10.1016/j.jclinepi.2007.04.018.

26.

Taji

Thomas

Oliver

, et al. Covid-19 in patients undergoing long-term dialysis in Ontario. CMAJ. 2021;193(8):E278-E284.

27.

Naylor

Knoll

Treleaven

, et al. Comparison of COVID-19 hospitalization and death between solid organ transplant recipients and the general population in Canada, 2020–2022. Transplant Direct. 2024;10(7):e1670. doi:10.1097/TXD.0000000000001670.

28.

Steyerberg

EW.

Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York, NY: Springer Science & Business Media; 2009.

29.

Paynter

Cook

Everett

Sesso

Buring

Ridker

PM.

Prediction of incident hypertension risk in women with currently normal blood pressure. Am J Med. 2009;122:464-471. doi:10.1016/j.amjmed.2008.10.034.

30.

Chowdhury

MZI

Turin

TC.

Precision health through prediction modelling: factors to consider before implementing a prediction model in clinical practice. J Prim Health Care. 2020;12(1):3. doi:10.1071/HC19087.

31.

Rao

Schaubel

Guidinger

, et al. A comprehensive risk quantification score for deceased donor kidneys: the kidney donor risk index. Transplantation. 2009;88(2):231-236. doi:10.1097/TP.0b013e3181ac620b.

32.

Kanis

Oden

Johnell

, et al. The use of clinical risk factors enhances the performance of BMD in the prediction of hip and osteoporotic fractures in men and women. Osteoporos Int. 2007;18(8):1033-1046. doi:10.1007/s00198-007-0343-y.

33.

Trillium Gift of Life Network. Factors impacting wait times for a deceased donor kidney transplant. https://www.giftoflife.on.ca/resources/pdf/transplant/4_Estimating_Kidney_Transplant_Wait_Times.pdf. Published 2024. Accessed May 1, 2025.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

2.72 MB