Implementation and Evaluation of a Risk-Based Approach to Guide Chronic Kidney Disease Care: Protocol for a Multiphase Mixed-Methods Study

Abstract

French

Background:

Risk prediction tools are used in a variety of clinical settings to guide patient care, although their use in chronic kidney disease (CKD) care is limited.

Objectives:

To assess the association of a risk-based model of CKD care on patient care, satisfaction, outcomes, and cost.

Design:

Mixed-methods with a pre-post design.

Setting:

We will use mixed-methods and a pre-post design to evaluate use of the Kidney Failure Risk Equation (KFRE) to guide CKD care. The KFRE will be applied to patients currently followed in nephrology multidisciplinary CKD clinics in Alberta, as well as to new patients being considered for multidisciplinary care.

Patients:

Patients with a 2-year risk of kidney failure ≥10% or estimated glomerular filtration rate (eGFR) ≤15 mL/min/1.73 m² will be recommended care by a multidisciplinary team coordinated by a nurse clinician and nephrologist, with access to other multidisciplinary resources including dietitians, pharmacists, and social workers as required.

Measurements/Methods:

Focus groups and interviews will be conducted to qualitatively describe patient and provider perspectives of potential barriers and facilitators to implementation of the risk-based approach to CKD care. Patient and provider surveys will also be used to quantify patient and provider satisfaction before and after the intervention. Finally, administrative data will be used to evaluate the association between the risk-based approach to care and outcomes including health care resource use, frequency of testing, modality choice, and death.

Conclusions:

Use of a risk-based model of care has the potential to increase use of optimal treatments such as the use of home dialysis and preemptive kidney transplantation, while reducing costs and poor outcomes related to processes of care such as unnecessary laboratory testing; however, there is also potential for unintended consequences. Our mixed-methods approach will integrate perceptions and needs from key stakeholders (including patients with CKD, their families, and their providers) to guide implementation and ensure appropriate modifications.

Keywords

chronic kidney disease kidney failure risk nondialysis care risk prediction

What was known before

A mismatch exists between the care that many chronic kidney disease (CKD) patients require and the care they receive. In some cases, care intensity exceeds what is required, and in others, patients do not receive the care they require. This leads to inequitable access to care and inefficient resource use. Providing appropriate, timely care for CKD patients is complex due to the complicated nature of the disease process as well as various contributing factors, such as multimorbidity.

What this adds

Our protocol describes a mixed-methods approach to understand the barriers and facilitators to application of a systematic methodology to CKD patient triage, utilizing the Kidney Failure Risk Equation (KFRE), as well as a quantitative evaluation to assess the impact of this approach on patient care, satisfaction, outcomes, and cost. This protocol can serve as a resource for other multidisciplinary CKD clinics interested in applying a similar, systematic approach to CKD patient triage and management.

Background

Chronic kidney disease (CKD), defined by persistent estimated glomerular filtration rate (eGFR) <60 mL/min/1.73 m² or albuminuria ≥3 mg/mmol, affects approximately 10% to 12% of adults in Canada,¹ and is associated with adverse clinical outcomes, poor quality of life, and high health care costs.² While most CKD patients (90%-95%) at less advanced stages can be cared for in primary care,³ the remainder have varying levels of complex high needs and consume as much as 8% of the health care budget despite a prevalence of only 3%.⁴ Patients with advanced CKD are typically cared for by nephrologists during outpatient clinic visits, and those with more advanced CKD are cared for in multidisciplinary CKD clinics in conjunction with a team of health care providers. Multidisciplinary CKD care is typically coordinated by a nurse clinician (nurse with expertise in kidney disease) and a nephrologist, with access to dietitians, social workers, and pharmacists as needed, aiming to slow progression of kidney disease and prepare for renal replacement therapy or conservative kidney management.

There is, however, a mismatch between the intensity and complexity of care that many CKD patients receive as compared with what they actually need (ie, the risk-treatment paradox where low-risk patients are more likely to receive treatment compared with high-risk patients⁵). We examined administrative and laboratory data⁶ and identified a large number of patients in whom the intensity of delivered care exceeded what is required, and a similar number of patients who did not receive the care they did require—each are subgroups of patients that offer unique opportunities to improve outcomes. For example, the proportion of high-risk patients with CKD receiving guideline-concordant care (including use of indicated medications) in Alberta varies from 35% to 85%.⁷ Our current approach to care does not incorporate readily available information on patient risk. The current approach contributes to poor allocation of resources, poor clinical outcomes, inequitable access to care for remote dwellers, suboptimal patient experiences, and financial hardship.^8-13 Risk prediction tools have been used in a variety of settings to guide clinical care.^14,15 Thus, strategies to incorporate risk prediction into CKD care systems could improve outcomes and increase health care system efficiency.

The Kidney Failure Risk Equation (KFRE) was developed to predict the risk of kidney failure requiring dialysis. It was initially developed and validated in 2 Canadian cohorts,¹⁶ and has since been externally validated in diverse global cohorts.¹⁷ The version of the KFRE tool promoted, and also publicly available (www.kidneyfailurerisk.com), includes 4 variables: age, sex, eGFR, and albumin-creatinine ratio (ACR). The tool has since been validated in Manitoba,¹⁸ and its use has been expanded to include triage for nephrology referrals.¹⁹ However, the implementation of this risk-based approach to CKD care has not been rigorously evaluated.

We will implement a risk-based approach to guide CKD care using the KFRE prediction tool.^16,17 The KFRE will be applied to patients currently in multidisciplinary CKD clinics (prevalent patients) in Alberta, as well as to new patients being considered for multidisciplinary care (incident patients). The subsequent model of care received by each patient will depend on that patient’s predicted individual risk of progression to end-stage kidney disease based on the KFRE. The overall objective of this study is to investigate the effect of this approach on quality of care and outcomes, patient and provider satisfaction, and cost.

Methods

Study Design

We will use a multiphase mixed-methods design to evaluate the risk-based approach to guide CKD care²⁰ (Figure 1). Our mixed-methods design, based on philosophical assumptions driven by pragmatism, includes the collection and integration of qualitative and quantitative data for complementarity—to strengthen interpretation and provide a more complete understanding of a set of interconnected research questions.²¹ We will prospectively collect data over 2 phases, pre- and post-intervention. In the preintervention phase, focus groups and individual interviews will be conducted to qualitatively describe patient, family member(s), and provider perspectives of the CKD multidisciplinary clinic and potential barriers and facilitators to implementation of the risk-based approach to CKD care (the intervention). Patient and provider surveys will also be used to quantify patient and provider satisfaction before and after the intervention. Interviews, focus groups, and surveys will be repeated following implementation of the intervention. We will integrate both qualitative and quantitative findings post implementation to comprehensively evaluate and refine the intervention. Finally, we will use administrative and laboratory data to evaluate clinical outcomes and health care resource utilization related to the intervention (Table 1). All qualitative and quantitative strands (including survey and administrative data findings) are given equal priority and will be integrated at the interpretation stage to determine evidence-based clinical implications for KFRE tool implementation in multidisciplinary CKD clinics. Figure 2 summarizes participant involvement, and Figure 3 is an overview of the study timelines and components of data collection. The University of Calgary Conjoint Health Research Ethics Board has approved this research study.

Figure 1.

Mixed-methods study design.

Table 1.

Study Outcomes, Data Sources, Time Period for Assessment, and Analysis.

Outcome	Data source	Time period for assessment	Analysis
Patient experience/barriers and facilitators	Focus groups and survey	Pre- and post-intervention	Descriptive
Provider experience/barriers and facilitators	Interviews and survey	Pre- and post-intervention	Descriptive
Clinical outcomes: risk stratification, death, hospitalization, emergency department visits, home dialysis, transplantation	AKDN/ICDC data Alberta renal program data	Pre- and post-implementation of KFRE tool	Count, time-to-event, or modified Poisson models, as appropriate
Resource use: physician visits and lab tests	AKDN/ICDC data	Pre- and post-implementation of KFRE tool	Count models
Process-based quality indicators: albuminuria measure; ACE-I/ARB use and statin use	AKDN/ICDC data	Pre- and post-implementation of KFRE tool	Modified Poisson models
Costs	AKDN/ICDC data	Pre- and post-implementation of KFRE tool	Detailed costing analyses using administrative health data

Note. AKDN/ICDC = Alberta Kidney Disease Network/Interdisciplinary Chronic Disease Collaboration; KFRE = Kidney Failure Risk Equation; ACE-I/ARB = angiotensin-converting enzyme inhibitor/angiotensin receptor blocker.

Figure 2.

Participant involvement pre- and post-intervention.

Figure 3.

Overview of study timelines and components of data collection.

Intervention: Risk-Based Approach to Guide CKD Care

Currently, patients are referred to multidisciplinary CKD clinics based on need, and considering a variety of factors as determined by their nephrologist, with no standard set of criteria. The use of the KFRE will inform the referral to multidisciplinary CKD clinics based on an individual patient’s risk of developing kidney failure. The validated KFRE prediction tool¹⁷ will be integrated into the nephrology electronic medical records and applied in general nephrology clinics to patients being considered for multidisciplinary CKD care (incident patients), and patients currently in the multidisciplinary CKD clinics (prevalent patients), with a reassessment of risk annually. Patients with a 2-year risk of kidney failure ≥10% or eGFR ≤15 mL/min/1.73 m² will be recommended for multidisciplinary care coordinated by a nurse clinician. If their risk decreases over time, they may be recommended for care by a nephrologist only, as determined by their nephrologist and in consultation with the patient. Patients currently in the CKD clinic but recommended for care by a nephrologist only (2-year risk of <10%) will continue to have access to available resources as deemed necessary including dietitians, social workers, and pharmacists. A variety of tools that support patient-centered, evidence-based CKD care will be available for patients and providers. Patient tools include written materials, patient infographics, whiteboard video, and a decision aid (“My Kidneys, My Choice”) for kidney failure treatment options.²² These tools have been incorporated into an education strategy that includes 1:1 or group education targeted to all patients in multidisciplinary CKD clinics. Provider tools include provider infographics for timing of dialysis initiation and a decision aid (“Methods to Assess Treatment Choices for Home Dialysis [MATCH-D].”²³

Multidisciplinary CKD care has been associated with improved outcomes^24,25 supporting use of such clinics to deliver a suite of novel interventions. The overall objective of this study is to evaluate the association between a risk-based approach to guide CKD care and outcomes including health care resource use (physician visits, hospitalizations, emergency department visits, frequency of laboratory testing), clinical outcomes (modality selection, death), patient and provider satisfaction, and cost.

Qualitative component to assess patient and provider perceptions

Perceptions of CKD patients, family members, and health care providers will be explored though a qualitative descriptive methodology.²⁰ Focus groups and interviews will be conducted to gain an in-depth understanding of their perceptions and experiences with multidisciplinary CKD clinics, as well as to identify barriers and facilitators to implementation of a risk-based approach, both before and after implementation of the intervention. Results from the preintervention period will inform the implementation process, while results from the postintervention period will help to further refine the intervention, as well as determine patient and provider perceptions and satisfaction.

Participant Selection and Recruitment

Preintervention focus groups will include adult (age 18 and older) nondialysis patients with CKD and their family members currently attending a multidisciplinary CKD clinic in Calgary, Alberta. Patients will be approached at the clinic visit by a nurse clinician and those who express an interest will be asked to complete a consent to be contacted. A research assistant will contact potential participants to provide them with further details about the study, and assess eligibility and willingness to participate. Two to 3 focus groups will be conducted prior to the intervention, with 6 to 8 participants per group (n = 12-18 in total). Postintervention focus groups will include patients and their family members who experienced the multidisciplinary clinics in both the preintervention and postintervention periods (prevalent patients), those whose care was transferred from the multidisciplinary clinics to general nephrology, as well as incident patients who experienced the multidisciplinary clinics only in the postintervention period, to explore their perspectives independently. Four to 6 focus groups will be conducted following the intervention, with 6 to 8 participants per group (n = 24-36).

All providers in the CKD clinics will be invited to participate in a semistructured, one-on-one interview via an emailed invitation letter. Interviews will be conducted both before and after the intervention and will be scheduled in-person at the clinic site or by telephone based on participant preference.

Qualitative Data Collection

Basic demographic information will be collected from all participants. Focus group and interview questions will be open-ended and general to allow for further probing of participant responses; for example, “What do you like the best?” and “What would you change?” will be included in all discussions. Probing questions are explicitly designed to encourage participants to think about benefits and challenges of both CKD care models (ie, pre- and post-intervention). Field notes will be collected and used to inform subsequent focus group/interview questions as well as overall data analysis. Audio files will be transcribed verbatim by a transcriptionist.

Qualitative Data Analysis

We will use conventional qualitative content analysis to thematically analyze the transcribed data from the focus groups and interviews.^26,27 Data collection and analysis will occur iteratively to permit follow-up of ideas that emerge from the data. We will follow the Donabedian framework²⁸ involving structure, process, and outcomes to inform data analysis. Inductive analysis (identifying emerging themes from studying the transcripts) and deductive analysis (informed by the Donabedian framework) will occur in 3 phases: coding, categorizing, and developing themes.

Three members of the research team will independently review the transcripts to identify themes and develop a coding scheme. The preliminary themes will be discussed to ensure they capture the full range and depth of data (investigator triangulation). Themes from the different participant groups (patient and family member focus groups, as well as provider interviews) will be compared and contrasted, and each investigator will individually code all of the transcripts with the agreed upon schema. Following the analysis, themes will be presented for verification to available focus group and interview participants and the research team. Demographic data from the participant questionnaires completed prior to focus groups and interviews will be analyzed using descriptive statistics and reported in aggregate. Focus group and interview transcripts will be imported into NVivo software to assist with qualitative data analysis.²⁹

Survey component to assess patients and provider satisfaction

Participant Selection and Survey Administration

All adult nondialysis CKD patients and health care providers who attend or work at a multidisciplinary CKD clinic in Alberta will be invited to complete an anonymous survey during the preintervention and postintervention data collection phases. The patient survey will include a validated 5-question measure of patient experience of quality of care, the Care Experience Feedback Improvement Tool (CEFIT),³⁰ as well as general demographic questions and an open-ended comments section to record general comments about their satisfaction with the care experience at the CKD clinic. All eligible patients will be provided a paper-based, anonymous survey during their regular clinic visit in both the preintervention and postintervention data collection phases. Completed surveys will be collected in a secure box to ensure anonymity. Written consent will not be required; voluntary completion of the anonymous survey establishes implied consent.

The provider survey will include the validated 5-question Andrews and Withey Job Satisfaction questionnaire³¹ as well as basic demographics and open-ended questions relating to perceived barriers and facilitators of a risk-based approach to CKD care. The anonymous survey will be sent to all CKD multidisciplinary clinic health care providers via secure email link in both the preintervention and postintervention data collection phases. Similar to the patient survey, the provider survey will not require written consent; voluntary completion of the anonymous survey establishes implied consent.

Survey Analysis

Survey data from patients and providers will be analyzed using descriptive statistics and reported by means and frequencies. Patient and provider groups will be analyzed separately, and the preintervention/postintervention responses will be compared. Qualitative responses to open-ended questions will be imported into NVivo and analyzed alongside the focus group and interview content analysis.

Analysis to assess care and outcomes using administrative health data

Administrative, laboratory, and renal program data will be used to evaluate the effectiveness of the risk-based approach to guide CKD care using a pre-post cohort design in the Calgary zone. Our Alberta Kidney Disease Network (AKDN) data holdings will be used to define cohorts of adults (age 18 and older) with sustained eGFR <30 mL/min/1.73 m² (based on outpatient measures of serum creatinine, with GFR estimated using the Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) equation, who are followed by a nephrologist (at least 2 outpatient nephrology visits in the prior 2 years). Cohort accrual for the preperiod will span a 1-year time period from April 2015 to April 2016, to allow 1 year for outcome assessment prior to implementation of the kidney failure risk (KFR) tool (Figure 3). For the postperiod, cohort accrual will occur April 2017 to April 2018, with follow-up to April 2019 for outcomes. Patients receiving dialysis or with a kidney transplant prior to the study period will be excluded. Outcomes for the administrative data analysis will include clinical outcomes (hospitalization and emergency department visits, death), use of modalities that improve patient experience and outcomes (home dialysis, kidney transplantation), resource use (physician visits, laboratory tests), process-based quality indicators for appropriate CKD care (assessment of albuminuria, use of ACE-I/ARBs in those with albuminuria, and statins), and costs (Table 1). We will also quantify the proportion of patients risk-stratified and appropriately managed in the applicable settings.

Covariates and other variable definitions

Demographic data including age, sex, postal code (to identify residents of Calgary zone) and First Nations ethnicity will be determined from the Alberta Health registry file. Attendance at the clinic will be determined from renal program data. To estimate the 2-year KFR, we will use the index (most recent) eGFR and the most recent ACR measurement in the prior year; if there was no ACR measurement, we will use the most recent protein-to-creatinine ratio or protein dipstick measurement, along with other information, to impute an ACR. Socioeconomic status (neighborhood income quintile) and rural or nonrural location of residence will be determined from the Canadian Census or National Household Survey data,^32,33 as in our prior work.³⁴ We will use validated algorithms to define diabetes³⁵ and hypertension³⁶ from the Alberta Health physician claims and hospitalization databases. Other comorbid conditions based on the Deyo classification of Charlson comorbidities will be identified using validated coding algorithms.³⁷

Analysis

Patients will be followed for outcomes for at least 1 year, in both the pre- and postperiods, adjusting for potential confounders. We will stratify all analyses by risk, with high-risk defined as a 2-year risk of kidney failure ≥10%, or eGFR ≤15 mL/min/1.73 m², because it is possible that the effect of the KFR tool implementation may be different for high-risk people (some of whom may receive an increased level of care) than for low-risk people (some of whom may receive a reduced level of care). The primary exposure will be the pre- versus post-period. The analytic approach will vary depending on the outcomes. We will use count models to examine rates of all-cause hospitalizations, emergency department visits, outpatient visits to a primary care physician, outpatient visits to a nephrologist, and the frequency of routine outpatient laboratory tests (creatinine, electrolytes, hemoglobin). Initiation of home dialysis and receipt of a kidney transplant will be analyzed as binary outcomes, using modified Poisson regression models, while death will be analyzed using time-to-event models.38 We will classify drug use as new (no prescription in the prior year) or any (irrespective of prior use), and will analyze these as binary outcomes, again using modified Poisson regression models.

Costing analysis

We will calculate the total costs of care for patients managed before and after implementation of the KFRE tool. Using administrative data the cost of physician visits, emergency department visits, hospitalizations, laboratory testing, medications, and outpatient dialysis (including whether the modality was in-center hemodialysis, peritoneal dialysis, or home hemodialysis) will be measured for each individual. Alberta Health uses the Canadian Institute for Health Information (CIHI) case mix grouper methods and ambulatory case costing methods to estimate hospital and outpatient costs, respectively. Physician claims will be based on the amount paid by Alberta Health.^39,40 If needed, extrapolation over a patient’s lifetime will be made using decision analysis, considering the cost and outcomes for kidney transplantation, and the various dialysis modalities.

Discussion

Using mixed-methods and a pre-post design, we will evaluate the effectiveness of a risk-based approach to guide CKD care on patient care and clinical outcomes in Alberta. Specifically, we will evaluate the broader implications of allocating and treating patients based on their risk. We will also assess a variety of outcomes related to this risk-based approach including patient and provider satisfaction, clinical outcomes (hospitalization, emergency department visits, death), use of modalities that improve patient experience and outcomes (home dialysis, kidney transplantation), resource use (physician visits, laboratory tests), and process-based quality indicators for appropriate CKD care (assessment of albuminuria, use of ACE-I/ARBs, and statins), and costs.

Risk prediction tools have been used in other settings to guide clinical care, including the Framingham Risk Score, to predict risk of future cardiovascular events,¹⁴ and the Ottawa Ankle Rules to guide clinical decision making in the emergency department.¹⁵ Given the variability in kidney disease progression, predicting the clinical trajectories for patients with CKD can be challenging. A recent survey of nephrologists in Canada determined that the majority of nephrologists value risk scores for decision making and identified the need for such tools to guide clinical management of CKD.⁴¹ However, until recently, rigorously validated risk prediction tools have not been available.

If the intervention described here is successful, patients with CKD in Canada will interact with health care teams in a different way, making it easier for them to get the appropriate care with the type of information required. Appropriate risk-based stratification to specialized multidisciplinary CKD programs has the potential to increase the use of cost-saving treatments, such as home dialysis and preemptive kidney transplantation, which are associated with improved quality of life, while reducing costs and poor outcomes related to unnecessary hospitalization, laboratory testing, and the overuse of expensive medications, such as erythropoiesis-stimulating agents. Our mixed-methods approach will integrate perceptions and needs from key stakeholders, including patients with CKD, their families, and the providers involved in their care. Although evaluation at 1 year will be important to assess patient and provider satisfaction and modify the intervention accordingly, we may be limited by a small number of hard clinical outcomes (transplantation, dialysis, death) within this time period. Therefore, we will aim to continue follow-up beyond 1 year to permit assessment of these important outcomes.

However, it will also be important to rigorously assess potential negative consequences of a risk-based approach to care, including the potential for more rapid progression in kidney disease, increased hospitalizations, and poor patient satisfaction and experience with care. Our study will be uniquely positioned to evaluate this risk-based approach as it will include both patients who are currently in the multidisciplinary CKD clinic (and thus have the experience) and those who are newly referred to the CKD clinic. We will also incorporate the perceptions, experience, and satisfaction of health care providers to this risk-based approach to care. Together, this information will be used to modify the future model of care, aiming to improve the patient experience, enhance efficiency, and reduce resource use. Future work is needed to develop other prediction tools, including those that incorporate other important outcomes such as quality of life.

Footnotes

Ethics Approval and Consent to Participate

This study has been approved by the Conjoint Health Research Ethics Board (CHREB) at the University of Calgary. Informed consent will be obtained for focus group, interview and survey participants.

Consent for Publication

All authors consent to publication of this protocol.

Availability of Data and Materials

No data or material available as this is a study protocol.

Author Contributions

All authors in this study have all contributed to this manuscript and approve of this submission. BRH, MDS, RGW, BJM, MT, and RRQ contributed to the study design and drafted the article. All authors contributed to the design and provided critical revisions to this manuscript.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was also supported by the Canadian Institutes of Health Research (CIHR) and the Interdisciplinary Chronic Disease Collaboration (ICDC).

ORCID iDs

Michelle D. Smekal

Navdeep Tangri

References

Arora

Vasa

Brenner

et al . Prevalence estimates of chronic kidney disease in Canada: results of a nationally representative survey. CMAJ. 2013;185(9):E417-E423.

Hemmelgarn

Manns

Lloyd

et al . Relation between kidney function, proteinuria, and adverse outcomes. JAMA. 2010;303(5):423-429.

Manns

Tonelli

Culleton

et al . A cluster randomized trial of an enhanced eGFR prompt in chronic kidney disease. Clin J Am Soc Nephrol. 2012;7(4):565-572.

Manns

Scott-Douglas

Tonelli

et al . A population-based analysis of quality indicators in CKD. Clin J Am Soc Nephrol. 2017;12(5):727-733.

McAlister

. The end of the risk-treatment paradox? A rising tide lifts all boats. J Am Coll Cardiol. 2011;58(17):1766-1767.

Hemmelgarn

Clement

Manns

et al . Overview of the Alberta Kidney Disease Network. BMC Nephrol. 2009; 10:30.

Kidney Strategic Clinical Network. Quality of care in early stage chronic kidney disease: 2012-2013. Supplementary Report to the 2015 Alberta Annual Kidney Care Report, University of Calgary, Alberta, Canada; 2015.

Barrett

Garg

Goeree

et al . A nurse-coordinated model of care versus usual care for stage 3/4 chronic kidney disease in the community: a randomized controlled trial. Clin J Am Soc Nephrol. 2011;6(6):1241-1247.

Bello

Hemmelgarn

Manns

Tonelli

; Alberta Kidney Disease Network. Use of administrative databases for health-care planning in CKD. Nephrol Dial Transplant. 2012;27(suppl 3):iii12-iii18.

10.

Bello

Hemmelgarn

Lin

et al . Impact of remote location on quality care delivery and relationships to adverse health outcomes in patients with diabetes and chronic kidney disease. Nephrol Dial Transplant. 2012;27(10):3849-3855.

11.

Ronksley

Sanmartin

Campbell

et al . Perceived barriers to primary care among western Canadians with chronic conditions. Health Rep. 2014;25(4):3-10.

12.

Rucker

Hemmelgarn

Lin

et al . Quality of care and mortality are worse in chronic kidney disease patients living in remote areas. Kidney Int. 2011;79(2):210-217.

13.

Tonelli

Wiebe

Guthrie

et al . Comorbidity as a driver of adverse outcomes in people with chronic kidney disease. Kidney Int. 2015;88(4):859-866.

14.

Wilson

D’Agostino

Levy

Belanger

Silbershatz

Kannel

. Prediction of coronary heart disease using risk factor categories. Circulation. 1998;97(18):1837-1847.

15.

Stiell

McKnight

Greenberg

et al . Implementation of the Ottawa ankle rules. JAMA. 1994;271(11):827-832.

16.

Tangri

Stevens

Griffith

et al . A predictive model for progression of chronic kidney disease to kidney failure. JAMA. 2011;305(15):1553-1559.

17.

Tangri

Grams

Levey

et al . Multinational assessment of accuracy of equations for predicting risk of kidney failure: a meta-analysis. JAMA. 2016;315(2):164-174.

18.

Whitlock

Chartier

Komenda

et al . Validation of the Kidney Failure Risk Equation in Manitoba Can J Kidney Health Dis. 2017; 4:1–9. doi:10.1177/2054358117705372.

19.

Hingwala

Wojcie chowski

Hiebert

et al . Risk-based triage for nephrology referrals using the Kidney Failure Risk Equation Can J Kidney Health Dis. 2017; 4:1–9 doi:10.1177/2054358117722782.

20.

Sandelowski

. Whatever happened to qualitative description? Res Nurs Health. 2000;23(4):334-340.

21.

Creswell

. A Concise Introduction to Mixed Methods Research. London, England: Sage; 2015.

22.

Canadian Kidney Knowledge Translation and Generation Network. Patient educational tools—CANN-NET. Date unknown. http://www.cann-net.ca/patient-information/educational-tools. Accessed December 30, 2017.

23.

Canadian Kidney Knowledge Translation and Generation Network. Provider tools—CANN-NET. Date unknown. https://www.cann-net.ca/for-providers/educational-tools. Accessed December 30, 2017.

24.

Hemmelgarn

Manns

Zhang

et al . Association between multidisciplinary care and survival for elderly patients with chronic kidney disease. J Am Soc Nephrol. 2007; 18(3):993–999.

25.

Strand

Parker

. Effects of multidisciplinary models of care for adult pre-dialysis patients with chronic kidney disease: a systematic review. Int J Evid Based Healthc. 2012;10(1):53–59.

26.

Elo

Kyngas

. The qualitative content analysis process. J Adv Nurs. 2008;62(1):107-115.

27.

Hsieh

Shannon

. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277-1288.

28.

Donabedian

. The quality of care: how can it be assessed? JAMA. 1988;260(12):1743-1748.

29.

QIP. NVivo 11 Software. QSR International; 2015. Melbourne, Australia.

30.

Beattie

Shepherd

Lauder

Atherton

Cowie

Murphy

. Development and preliminary psychometric properties of the Care Experience Feedback Improvement Tool (CEFIT). BMJ Open. 2016;6(6):e010101.

31.

Rentsch

Steel

. Construct and concurrent validation of the Andrews and Withey Job Satisfaction Questionnaire. Educ Psychol Meas. 1992;52(2):357-367.

32.

Pitblado

Pong

Irvine

et al . Assessing Rural Health: Toward Developing Health Indicators for Rural Canada. Sudbury, Ontario, Canada: Centre for Rural and Northern Health Research; 1999.

33.

Statistics Canada. National Household Survey, 2011. Catalogue no. 99-0001-x2011001. Ottawa, Ontario: Statistics Canada; 2011.

34.

Tonelli

Muntner

Lloyd

et al . Risk of coronary events in people with chronic kidney disease compared with those with diabetes: a population-level cohort study. Lancet. 2012;380(9844):807-814.

35.

Hux

Ivis

Flintoft

Bica

. Diabetes in Ontario: determination of prevalence and incidence using a validated administrative data algorithm. Diabetes Care. 2002;25(3):512-516.

36.

Quan

Khan

Hemmelgarn

et al . Validation of a case definition to define hypertension using administrative data. Hypertension. 2009;54(6):1423-1428.

37.

Quan

Sundararajan

Halfon

et al . Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43(11):1130-1139.

38.

Zou

. A modified Poisson regression approach to prospective studies with binary data. Am J Epidemiol. 2004;159(7):702-706.

39.

Klarenbach

Tonelli

Pauly

et al . Economic evaluation of frequent home nocturnal hemodialysis based on a randomized controlled trial. J Am Soc Nephrol. 2014;25(3):587-594.

40.

McBrien

Manns

Chui

et al . Health care costs in people with diabetes and their association with glycemic control and kidney function. Diabetes Care. 2013;36(5):1172-1180.

41.

Chiu

Tangri

Djurdjev

et al . Perceptions of prognostic risks in chronic kidney disease: a national survey. Can J Kidney Health Dis. 2015;2:53.