Abstract
Background:
Quality indicators are important tools to measure and ultimately improve the quality of care provided. Performance measurement may be particularly helpful to grow disciplines that are underutilized and cost-effective, such as home dialysis (peritoneal dialysis and home hemodialysis).
Objective:
To identify and catalog home dialysis quality indicators currently used in Canada, as well as to evaluate these indicators as a starting point for future collaboration and standardization of quality indicators across Canada.
Design:
An environmental scan of quality indicators from provincial organizations, quality organizations, and stakeholders.
Setting:
Sixteen-member pan-Canadian panel with expertise in both nephrology and quality improvement.
Patients:
Our environmental scan included indicators relevant to patients on home dialysis.
Measurements:
We classified existing indicators based on the Institute of Medicine (IOM) and Donabedian frameworks.
Methods:
To evaluate the indicators, a 6-person subcommittee conducted a modified version of the Delphi consensus technique based on the American College of Physicians/Agency for Healthcare Research and Quality criteria. We shared these consensus ratings with the entire 16-member panel for further examination. We rated items from 1 to 9 on 6 domains (1-3 does not meet criteria to 7-9 meets criteria) as well as a global final rating (1-3 unnecessary to 7-9 necessary) to distinguish high-quality from low-quality indicators.
Results:
Overall, we identified 40 quality indicators across 7 provinces, with 22 (55%) rated as “necessary” to distinguish high quality from poor quality care. Ten indicators were measured by more than 1 province, and 5 of these indicators were rated as necessary (home dialysis prevalence, home dialysis incidence, anemia target achievement, rates of peritonitis associated with peritoneal dialysis, and home dialysis attrition). None of these indicators captured the IOM domains of timely, patient-centered, or equitable care.
Limitations:
The environmental scan is a nonexhaustive list of quality indicators in Canada. The panel also lacked representation from patients, administrators, and allied health professionals.
Conclusions:
These results provide Canadian home dialysis programs with a starting point on how to measure quality of care along with the current gaps. This work is an initial and necessary step toward future collaboration and standardization of quality indicators across Canada, so that home dialysis programs can access a smaller number of highly rated balanced indicators to motivate and support patient-centered quality improvement initiatives.
Introduction
The increasing burden of end-stage kidney disease (ESKD) has brought heightened interest in home dialysis modalities, including peritoneal dialysis (PD) and home hemodialysis (HHD). 1 Home-based dialysis therapies may be relatively underutilized in Canada, with reported prevalence rates of 17% for PD and 3% for HHD. 2 Both PD and HHD have also been shown to be equally effective and less costly relative to in-center hemodialysis in the Canadian health care context. 2 Therefore, growing high-quality home dialysis programs have become a national priority. One strategy to enable high-quality health care is through quality indicator measurement, which is then utilized by the applicable stakeholders to improve health care delivery for patients with ESKD interested in home dialysis.
Multiple frameworks for defining health care quality exist. The Institute of Medicine (IOM) identifies domains of health care quality as safe (free from harm), effective (using best available evidence), efficient (limits waste), timely (available when needed), patient-centered (focused on the patient), and equitable (equally available). 3 While the IOM domains are helpful in defining health care quality, the Donabedian framework is often used to describe how health care is delivered. In the Donabedian framework, the 3 components of health care quality are structure (the setting in which care occurs), process (the care that is done to the patient), and outcome (how the care ultimately affects the patient). 4 In addition to covering these elements of health care quality, it is also recommended that quality of care indicators be evidence-based, precisely specified, and feasible for subsequent quality improvement activities. 5
In Canada, it is unclear what home dialysis quality indicators exist and the degree that they characterize the different domains of health care quality. Overlap and repetition is also very likely given that health care is a provincial jurisdiction. Therefore, we sought to identify and catalog (by the IOM and Donabedian frameworks) home dialysis quality indicators currently used in Canada, as well as to evaluate these indicators based on the American College of Physicians/Agency for Healthcare Research and Quality criteria.5,6 Our aim is to provide a pan-Canadian resource of existing quality indicators in home dialysis to inform improvement initiatives, as well as serve as a starting point for future collaboration and standardization of quality indicators across Canada.
Methods
Indicator Identification and Categorization
We collected publicly available quality indicators currently in use by nephrology programs across the country (including British Columbia, Alberta, Saskatchewan, Manitoba, Quebec, Ontario, and the Atlantic Provinces). For indicators not publicly available, we contacted provincial data leads, division heads, and home dialysis content experts. We stopped the environmental scan once we achieved representation from all the aforementioned provinces.
We combined similar indicators into a single measure and characterized each indicator according to the IOM and Donabedian frameworks of health care quality; we also included balancing indicators so as to capture measures that look at potential adverse effects of home dialysis (eg, infectious complication). 7
Indicator Evaluation
We rated the identified indicators using a modified version of the American College of Physicians/Agency for Healthcare Research and Quality performance measure review criteria, which included the following dimensions (Supplemental Table 1)5,6,8:
Importance: The metric will lead to measurable and meaningful improvement or there is a clear performance gap.
Evidence-base: The metric is based on high-quality and high-quantity evidence.
Measure specifications: The metric can be clearly defined (ie, numerator and denominator) and reliably captured.
Feasibility and applicability: The metric is under the influence of health care providers and/or the health care system, with data collection and improvement activities both feasible and acceptable.
Based on these dimension ratings, each indicator then received a final global rating rather than an average score based on its overall ability to distinguish good quality from poor quality. 6
Modified Delphi Process
We then used a modified version of the Delphi consensus technique to evaluate the identified quality indicators, based on the RAND method and the above American College of Physicians/Agency for Healthcare Research and Quality performance measure review criteria. 9 This process involved a structured approach through which the expertise and knowledge of a group of individuals was systematically obtained and interspersed by opinion feedback. We used multiple stages of the modified Delphi technique, including individual rating, consensus meeting, and final ranking to allow for optimal, unbiased expression of opinions. This format is an established technique for developing and evaluating quality indicators in health care.8,10-16
First, we made the identified home dialysis quality indicators available to 6 members (home dialysis subcommittee) of a 16-member volunteer national quality indicator committee. The 16-member committee included representatives from 7 of 10 provinces, with most possessing advanced training in quality improvement. Each member of the home dialysis subcommittee then individually reviewed the quality indicators identified in the environmental scan in advance of a teleconference in which the indicators and their preliminary ratings were discussed. Through group discussion, the 6 panelists provided initial group ratings on whether the indicator met criteria within each of the American College of Physicians/Agency for Healthcare Research and Quality dimensions using a 9-point scale where 1 to 3 indicated “does not meet criteria,” 4 to 6 “meets some criteria,” and 7 to 9 “meets criteria.” For the global rating, we considered quality indicators as “necessary” if the median rating was 7, 8, or 9 and there was no disagreement by any member. We considered indicators as “unnecessary” if the median rating was 1, 2, or 3 and there was no disagreement by any member. We considered all other indicators as “supplemental.”
Next, we shared the group ratings with each home dialysis subcommittee member to compare with their initial rating and provide feedback as needed. Any disagreements prompted further discussion until we achieved consensus. These consensus ratings were then shared with the entire 16-member committee, with further discussion of any ratings that differed by ≥3 points. The final ratings were approved by the full 16-member committee prior to publication. Formal research ethics board review was not required by Queen’s University based on the Tri-Council Policy Statement for ethical human research, as the focus of the study involved quality indicators and not human participants.
Results
Our environmental scan revealed 40 home dialysis quality indicators across 7 provinces in Canada (Table 1). IOM domains covered included safety (n = 6, 15%), effective (n = 8, 20%), efficient (n = 15, 38%), timely (n = 3, 7%), patient-centered (n = 5, 13%), and equitable (n = 3, 7%). Donabedian categories covered included structure (n = 3, 7%), process (n = 16, 40%), outcome (n = 11, 28%), and balancing (n = 10, 25%).
Environmental Scan of Current Canadian Nephrology Quality Indicators.
Note. The denominator is 7 provinces (territories excluded and Atlantic provinces combined) and the table indicates the number of provinces currently using the listed indicator. PD = peritoneal dialysis; eGFR = estimated glomerular filtration rate; PTH = parathyroid hormone.
We found little overlap in the quality indicators being measured among provinces. Only 10 indicators were measured by multiple provinces, with only 4 indicators measured by most of the provinces (home dialysis prevalence, home dialysis incidence, rates of PD peritonitis, and home dialysis attrition). These common indicators focused on safe (n = 3), effective (n = 2), and efficient (n = 4) care and were primarily outcome (n = 4) and balancing measures (n = 5). The only patient-reported outcome/experience measure (PROM or PREM) identified was quality of life, consistently assessed by only a single province.
With respect to overall ability to distinguish good quality from poor quality (ie, necessary versus unnecessary for improvement), we rated 22 (55%) indicators as “necessary,” 11 (27%) as “supplemental,” and 7 (18%) as “unnecessary” (Table 2). The 22 “necessary” indicators focused on safe (n = 3, 14%), effective (n = 4, 18%), efficient (n = 7, 32%), timely (n = 2, 9%), patient-centered (n = 4, 18%), and equitable (n = 2, 9%) care and consisted of 2 (9%) structure measures, 8 (36%) process measures, 7 (32%) outcome measures, and 5 (23%) balancing measures.
Quality Indicators Rated by the American College of Physicians/Agency for Healthcare Research and Quality Performance Measure Criteria Using a Modified Delphi Technique.
Note. Each domain was rated on a 9-point scale where 1 to 3 indicated “does not meet criteria,” 4 to 6 “meets some criteria,” and 7 to 9 “meets criteria.” After considering and rating each of these domains, the panelists then rated the overall measure (1-3 = unnecessary, 4-6 = supplemental, 7-9 = necessary). PD = peritoneal dialysis; eGFR = estimated glomerular filtration rate; CKD = chronic kidney disease; HD = hemodialysis; PTH = parathyroid hormone.
Four common themes emerged during the rating process. First, the strength of evidence for most indicators was moderate, with only 10 indicators receiving ratings of 7 to 9. Second, most indicators could be precisely defined and specified, but definitions often varied between provinces. For example, some provinces measure “home dialysis incidence within 6 months of chronic initiation of dialysis,” whereas others use different time frames, exclusions, and risk adjustments. Third, feasibility of data collection varied across the indicators due to differing provincial infrastructure and electronic medical record (EMR) capabilities. This theme was particularly problematic for indicators that relied on patient or health care staff perspectives (eg, quality of life, reasons for home dialysis attrition). Last, the panel rated most indicators (n = 23, 58%) as usable for quality improvement (ie, under the influence of health care providers and/or the health care system, with data collection and improvement activities both feasible and acceptable). Notable exceptions included measures not necessarily attributable to nephrology, such as rehospitalizations.
Of the 10 indicators measured by multiple provinces, the panel rated 5 as “necessary.” These included home dialysis prevalence, home dialysis incidence, anemia target achievement, rates of PD peritonitis, and home dialysis attrition. None of these indicators captured the IOM domains of timely, patient-centered, or equitable care.
Discussion
In an environmental scan of home dialysis quality indicators across Canada, we identified 40 unique indicators. Our pan-Canadian panel with experience in both home dialysis and quality improvement rated just over half of these indicators as “necessary” to distinguish good quality from poor quality care based on the American College of Physicians/Agency for Healthcare Research and Quality criteria. However, we observed little overlap of indicators across provinces and only 5 indicators used by multiple provinces received global ratings ≥7. In addition, we noted several of the IOM domains of quality had little representation, specifically measures of timely, patient-centered, and equitable care. These results provide Canadian home dialysis programs with a starting point on how to measure quality of care, which we envision as an initial step toward future collaboration and standardization of quality indicators across Canada.
There is little published data on quality indicators in home dialysis. A recent review by the American Society of Nephrology (ASN) Quality Committee identified 60 national indicators for all aspects of kidney disease (except transplantation), of which only 3 specifically pertained to home dialysis (PD adequacy, PD catheter success rate, and PD catheter exit site infection rate). 8 Of their 60 indicators, the panel rated 29 (49%) as metrics with high validity using a similar approach as our group. These findings are consistent with our data for Canadian home dialysis indicators, showing approximately 50% of current indicators are highly rated. Also consistent with the ASN results are the identification of some common themes that affect quality of care measures. These included indicators based on questionable evidence and measures without a performance gap (ie, so-called “topped-out” measures). 17 Examples of “topped-out” home dialysis measures included dialysis adequacy and achievement of anemia targets. Given the large number of home dialysis indicators observed, it is important to continually assess measures to ensure they are based on the most current evidence and performance gaps exist.
Our work extends these lessons by also categorizing existing home dialysis indicators by the IOM and Donabedian frameworks, which highlights gaps and opportunities for new measure development. Most highly rated indicators used by multiple provinces focus on outcome or balancing measures in the domains of safe, effective, and efficient care. Notably missing are structure (the setting in which care occurs) and process (the care that is done to the patient) measures, along with measures of timely, patient-centered, and equitable care. As a first step, some of the identified indicators could be modified to measure other IOM domains (eg, by adding a time component to the definition).
Besides providing home dialysis programs with a starting point on how to measure quality of care, we believe the observed variation in such a large number of indicators combined with little overlap in the highly rated indicators suggests an opportunity exists for collaboration and standardization of quality indicators across Canada. Important considerations would include ensuring the selected indicators could be precisely defined and specified across provinces given different data infrastructures and EMRs, along with consensus on the included indicators and criteria for adding and removing indicators. Collaboration may also help programs to define and validate new indicators that have not been highly rated by demonstrating that adopting these measures into quality improvement initiatives lead to improvements in care. In this way, we may reduce measurement costs, data collection burden, and ensure that we are not “re-inventing the wheel” to solve similar problems.
Accordingly, we have proposed initial steps toward development of a balanced quality indicator scorecard for home dialysis (Table 3). This scorecard is intended to help programs focus on a small number of prioritized indicators across all domains of quality and stimulate further discussion. It should also be recognized that for many measures (particularly process measures), it may be preferable to stratify by home modality given the different barriers to success. The scorecard incorporates all aspects of the IOM and Donabedian frameworks, as well as highly rated indicators already in use by multiple provinces. These latter indicators include home dialysis incidence (which changes quicker than home dialysis prevalence), rates of PD peritonitis (under infectious complications), and home dialysis attrition. We have also included the presence of a multidisciplinary home dialysis team as a structural measure given its association with persistent use of PD at 1 year 18 ; however, the staffing components, full-time equivalents, and responsibilities of the multidisciplinary team remain to be determined. A process measure is also needed to capture efforts at home dialysis uptake, for which the proportion of offered patients who choose home dialysis was highly rated; although any quality indicator that captures this construct would be acceptable. For process measures (especially those related to education and decision-making), it is important to ensure that they accurately capture whether the process has been provided as intended (ie, fidelity). Otherwise, the measure could be susceptible to “gaming.” 19 Newly developed indicators are required to measure timely, patient-centered, and equitable care, and our suggestions are meant to elicit further discussion of how to routinely measure these in a manner that is useful for quality improvement initiatives and not overly burdensome to staff. In particular, a standardized approach is required to capture patient-reported outcome and experience measures (PROMs and PREMs) in a manner that is not overly burdensome to patients and staff, as well as being amenable to quality improvement interventions. The omission in available PROMs and PREMs was also noted by the ASN Quality Committee, 8 emphasizing the importance of this work.
First Step Toward Development of a Balanced Quality Indicator Scorecard for Home Dialysis.
Note. Several highly rated indicators from the environmental scan have been populated (in regular font), with indicator gaps (in bold) and additional work needed to complete the scorecard (in italics). HD = hemodialysis.
Strengths of this work include the structured approach to indicator categorization and evaluation, using the IOM and Donabedian frameworks along with the American College of Physicians/Agency for Healthcare Research and Quality criteria. Our panel also included members representing most regions of Canada to ensure relevance and feasibility of the indicators across different health care contexts, in addition to advanced training and real-life expertise in home dialysis and/or quality improvement to ensure applicability to frontline improvement efforts.
Our work does have some limitations. First, the environmental scan is a nonexhaustive list of quality indicators currently being used at the local and provincial levels. However, we did collect indicators from across the country, except for the 3 territories. Second, we did not examine how indicators were operationalized (ie, numerator, denominator, risk adjustment), as the focus of this work was on the indicator construct. Operational definitions for quality indicators must be clarified before implementation. 5 For example, the process measure “time from surgical clinic to catheter insertion” requires a clear definition to avoid interpretation being affected by purposeful “early” referrals with planned catheter deferral or unintentional “late” referrals.” Third, the differences between process and outcome measures can be subjective, especially for evidence-based surrogate outcomes (eg, vascular access, anemia). 19 Fourth, there is some degree of overlap in which IOM domains different indicators should be categorized. (eg, eGFR at time of home dialysis referral could be considered as a marker of efficient or timely access to care). Similarly, if home dialysis prevalence is stratified by geography, then this could measure equity. We classified indicators based on the description provided, but what really determines the IOM domain is how the indicator is used to drive frontline improvement efforts. Fifth, the evaluation of indicators was not anonymous, and so there is the potential for bias through the bandwagon effect. 20 Sixth, the panel was composed of 15 physicians and 1 nurse practitioner, and so the results may not represent the views of other important stakeholders in home dialysis quality, such as patients, administrators, and allied health professionals.
Conclusions
In summary, we identified 40 home dialysis quality indicators currently being measured across Canada. Of the 22 indicators rated as “necessary” to distinguish good quality from poor quality care, 10 were measured by multiple provinces (4 by the majority of provinces) and only 5 of these received global ratings ≥7. Furthermore, we noted that most home dialysis quality indicators focus on safe, effective, and efficient care, with gaps in assessment of timely, patient-centered, and equitable care. This work is intended to jump start discussions on the consolidation and standardization of quality indicator measurement and reporting in Canada. Future work will require stakeholder engagement to review data capabilities, operational definitions, risk adjustment, and targets, along with the development of new indicators to fill the identified gaps. Input from nephrology providers trained and committed to working in quality improvement will also be required to ensure candidate indicators remain useful to frontline staff. For example, selected indicators should inform the development of measurement sets and driver diagrams to guide improvement initiatives (ie, home dialysis educator → % educated → home dialysis incidence). In this way, Canadian home dialysis programs can begin to move beyond measuring 40 indicators of varying validity toward measuring a smaller number of balanced indicators that motivate and support patient-centered quality improvement initiatives.
Supplemental Material
sj-pdf-1-cjk-10.1177_2054358120977391 – Supplemental material for An Environmental Scan and Evaluation of Home Dialysis Quality Indicators Currently Used in Canada
Supplemental material, sj-pdf-1-cjk-10.1177_2054358120977391 for An Environmental Scan and Evaluation of Home Dialysis Quality Indicators Currently Used in Canada by Lisa Dubrofsky, Ali Ibrahim, Karthik Tennankore, Krishna Poinen, Sachin Shah and Samuel A. Silver in Canadian Journal of Kidney Health and Disease
Footnotes
Acknowledgements
The authors value the support and guidance from Adeera Levin throughout the design and execution of this work. The other 10 volunteer quality committee members include William Beaubien-Souligny, Daniel Blum, Tamara Glavinovic, Claire Harris, Jay Hingwala, Amber Molnar, Priya Mysore, Alison Thomas, Amanda Vinson, and Seychelle Yohanna. S.A.S. is supported by a Kidney Research Scientist Core Education and National Training (KRESCENT) Program New Investigator Award (co-funded by the Kidney Foundation of Canada, Canadian Society of Nephrology, and Canadian Institutes of Health Research).
Ethics Approval and Consent to Participate
No ethics approval or consent was required.
Consent for Publication
All authors approved the final version of this manuscript.
Availability of Data and Materials
The data and material are available from corresponding author upon reasonable request.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: S.A.S. has received speaking fees from Sanofi Canada. KT has participated in advisory boards for Otsuka, AstraZeneca, Baxter and Janssen and received unrestricted investigator-initiated grant funding for research from Otsuka and Astellas. The remaining authors have no conflicts of interest relevant to this study. All authors approved the final version of the submitted manuscript. We certify that neither this manuscript nor one with substantially similar content has been published or is being considered for publication elsewhere.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
