Abstract
Background:
Quality indicators can be used to identify gaps in care and drive frontline improvement activities. These efforts are important to prevent adverse events in the increasing number of ambulatory patients with advanced kidney disease in Canada, but it is unclear what indicators exist and the components of health care quality they measure.
Objective:
We sought to identify, categorize, and evaluate quality indicators currently in use across Canada for ambulatory patients with advanced kidney disease.
Design:
Environmental scan of quality indicators currently being collected by various organizations.
Setting:
We assembled a 16-member group from across Canada with expertise in nephrology and quality improvement.
Patients:
Our scan included indicators relevant to patients with chronic kidney disease in ambulatory care clinics.
Measurements:
We categorized the identified quality indicators using the Institute of Medicine and Donabedian frameworks.
Methods:
A 4-member panel used a modified Delphi process to evaluate the indicators found during the environmental scan using the American College of Physicians/Agency for Healthcare Research and Quality criteria. The ratings were then shared with the full panel for further comments and approval.
Results:
The environmental scan found 28 quality indicators across 7 provinces, with 8 (29%) rated as “necessary” to distinguish high-quality from poor-quality care. Of these 8 indicators, 3 were measured by more than 1 province (% of patients on a statin, number of patients receiving a preemptive transplant, and estimated glomerular filtration rate at dialysis start); no indicator was used by more than 2 provinces. None of the indicators rated as necessary measured timely or equitable care, nor did we identify any measures that assessed the setting in which care occurs (ie, structure measures).
Limitations:
Our list cannot be considered as an exhaustive list of available quality indicators at hand in Canada. Our work focused on quality indicators for nephrology providers and programs, and not indicators that can be applied across primary and specialty providers. We also focused on indicator constructs and not the detailed definitions or their application. Last, our panel does not represent the views of other important stakeholders.
Conclusions:
Our environmental scan provides a snapshot of the scope of quality indicators for ambulatory patients with advanced kidney disease in Canada. This catalog should inform indicator selection and the development of new indicators based on the identified gaps, as well as motivate increased pan-Canadian collaboration on quality measurement and improvement.
Trial registration:
Not applicable as this article is not a systematic review, nor does it report results of a health intervention on human participants.
Introduction
Chronic kidney disease (CKD) is an important global health concern, and it is associated with increased mortality, morbidity, and health care costs. 1 In Canada, approximately 12% of adults and 30% of older adults (≥65 years) are living with CKD (an estimated 2.9 million Canadians),2,3 Most of these patients are managed in primary care settings, 4 whereas patients with advanced CKD are often referred to multidisciplinary kidney clinics,5,6 In Canada, patients receive team-based care from nephrologists, nurses, dieticians, pharmacists, and/or social workers to slow CKD progression, reduce complications, and plan for patient-centered end-stage kidney disease management. These efforts may be further aided by quality indicators, which allow health care providers to assess their performance and identify opportunities for quality improvement.
Quality indicators are predefined quantitative or qualitative benchmarks that measure different domains of health care quality (ie, safe, effective, efficient, timely, patient-centered, and equitable care) and can be further classified as structure (the setting in which care occurs), process (how care is delivered), or outcome (how care affects patients) measures.7-9 These indicators can be used to set internal quality of care targets, compare performance with other centers, or implement payment-for-performance programs. 10 Furthermore, quality indicators can help motivate local quality improvement activities, which a recent meta-analysis demonstrated may improve surrogate outcomes and reduce dialysis incidence in patients with CKD. 11
Despite these potential benefits, it is unclear what quality indicators are measured in ambulatory patients with advanced CKD across Canada. Our objectives were to describe the scope of existing indicators and the domains of health care quality covered, as well as evaluate indicators using the American College of Physicians/Agency for Healthcare Research and Quality criteria. 12 This knowledge should help programs select indicators and promote national collaboration to fill any performance measurement gaps that are identified.
Methods
Indicator Identification and Categorization
Initially, we collected publicly available quality indicators from provincial and local nephrology programs. For indicators not publicly available, we used a snowball sampling methodology, where we relied on study participants to recruit and recommend future participants. We then contacted nephrology programs across the country (including British Columbia, Alberta, Saskatchewan, Manitoba, Quebec, Ontario, and the Atlantic Provinces) to query current provincial and local practices on performance measurement. This sampling took place from January 31, 2019, to March 29, 2019. Program representatives contacted included provincial data leads, division heads, and content experts, who provided data or identified regional leaders for assistance. We asked nephrology programs “which quality indicators are currently collected at the provincial and local levels to evaluate ambulatory kidney care practices?” The remainder of the discussion was free form. We stopped the environmental scan once we achieved representation from all the aforementioned provinces.
Once a list of indicators was collected, we consolidated similar indicators into a single measure. We categorized each indicator according to the Institute of Medicine (IOM; safe, effective, efficient, timely, patient-centered, equitable) and Donabedian (structure, process, outcome) frameworks of health care quality. 8 We also collected balancing indicators to look for inadvertent harms that may occur in the health care system (eg, hospitalizations for drug-related hyperkalemia). 13
Indicator Evaluation
We rated each indicator using a modified version of the American College of Physicians/Agency for Healthcare Research and Quality performance measure review criteria. These include the following dimensions (Supplemental Table 1):14-16
Importance: The metric will lead to measurable and meaningful improvement, or a clear performance gap exists.
Evidence-based: The metric evidence is robust and of high quality.
Measure specifications: The metric can be clearly defined (ie, numerator and denominator) and reliably captured.
Feasibility and applicability: The metric may have an influence on health care providers and/or the health care system, and data collection and improvement activities are achievable and acceptable.
We rated each dimension using a 9-point scale, where 1-3 indicates “does not meet criteria,” 4-6 “meets some criteria,” and 7-9 “meets criteria.” Based on the ratings, each indicator then received a final global rating (instead of an average score) based on its overall ability to distinguish good quality from poor quality. 15 For the global rating, we considered quality indicators as “necessary” if the median rating was 7, 8, or 9 and there was no disagreement between members. We considered indicators as “unnecessary” if the median rating was 1, 2, or 3 and there was no disagreement by any member. We considered all other indicators as “supplemental.”
Modified Delphi Process
We used a modified version of the Delphi consensus technique to evaluate quality indicators based on the RAND method. 17 This format is an established technique for developing and evaluating quality indicators in health care.16,18-24 The Delphi panel consisted of a 16-member volunteer national quality indicator committee with nephrology representatives from 7 of 10 provinces and most possessing advanced training in quality improvement. Four of the members formed an ambulatory kidney care subcommittee.
First, we made the identified ambulatory kidney quality indicators available to each member of the subcommittee. Each member then individually reviewed the indicators and rated them according to the above criteria. Next, the ambulatory kidney subcommittee met by teleconference to compare their initial ratings and provide feedback as needed. Any disagreements prompted further discussion until consensus was achieved. These consensus ratings were then shared with the entire 16-member committee, with further review of any ratings that differed by ≥3 points. The full 16-member committee prior to publication approved the final ratings. Formal research ethics board review was not required by Queen’s University based on the Tri-Council Policy Statement for ethical human research, as the focus of the study involved quality indicators and not human participants.
Results
We identified 28 ambulatory kidney quality indicators across 7 provinces in Canada (Table 1), of which 4 provinces collected province-wide standardized metrics. The IOM domains covered included safe (n = 5, 18%), effective (n = 9, 32%), efficient (n = 5, 18%), timely (n = 2, 7%), patient-centered (n = 6, 21%), and equitable (n = 1, 4%) care. Donabedian categories covered included process (n = 10, 36%), outcome (n = 14, 50%), and balancing (n = 4, 14%). We did not identify any structure indicators.
Environmental Scan of Current Canadian Ambulatory Care Quality Indicators for Patients With Advanced Kidney Disease.
Note. The number of provinces currently using the listed indicator is indicated in the adjacent parentheses. The denominator is 7 provinces (territories excluded and Atlantic provinces combined). ACEi = angiotensin-converting enzyme inhibitors; ARB = angiotensin receptor blocker; eGFR = glomerular filtration rate; PTH = parathyroid hormone.
We found significant heterogeneity in the quality-of-care indicators across provinces, with 5 indicators being collected in more than 1 province (achievement of anemia targets, % of patients on a statin, estimated glomerular filtration rate [eGFR] at dialysis start, number of patients receiving a preemptive kidney transplant, and number of days between home dialysis referral and assessment). More than 2 provinces measured no quality-of-care indicator.
With respect to overall ability to distinguish good-quality from poor-quality care (ie, necessary versus unnecessary for improvement), we rated 8 (29%) indicators as “necessary,” 14 (50%) as “supplemental,” and 6 (21%) as “unnecessary” (Table 2). The 8 “necessary” indicators focused on safe (n = 1, 13%), effective (n = 4, 33%), efficient (n = 2, 17%), and patient-centered care (n = 1, 13%) and consisted of 2 process and 6 outcome measures. None of the necessary rated indicators measured timely or equitable care. Of these 8 “necessary” indicators, 3 were measured by more than 1 province. These included % of patients on statin, eGFR at dialysis start, and number of patients receiving preemptive kidney transplant.
Quality Indicators Rated by the American College of Physicians/Agency for Healthcare Research and Quality Performance Measure Criteria Using a Modified Delphi Technique.
Note. Each domain was rated on a 9-point scale, where 1-3 indicated “does not meet criteria,” 4-6 “meets some criteria,” and 7-9 “meets criteria.” After considering and rating each of these domains, the panelists then rated the overall measure (1-3 = unnecessary, 4-6 = supplemental, 7-9 = necessary). ACEi = angiotensin-converting enzyme inhibitors; ARB = angiotensin receptor blocker; eGFR = glomerular filtration rate; PTH = parathyroid hormone; HD = hemodialysis.
Five common themes emerged during the rating process. First, the strength of evidence for most indicators was limited, with 9 (32%) indicators receiving ratings of 7-9. Second, feasibility of data collection was a major concern among committee members due to differing provincial infrastructures and electronic medical record (EMR) capabilities. Overall, 6 (21%) indicators received ratings of 7-9 for “feasible to collect.” Third, the majority of indicators were precisely defined and specified, with notable exceptions including necessary rated indicators related to the use of angiotensin-converting enzyme inhibitors (ACEi) or angiotensin receptor blockers (ARBs), statins, and patient-reported experience measures (PREMs). Fourth, there was unclear attribution of some metrics to nephrologists (ie, pathways for access creation, referrals not meeting standardized criteria) or lack of delineation from primary care (ie, albuminuria screening, goals of care discussions). Finally, the environmental scan did not identify any indicators related to kidney transplant awareness, education, or referral thresholds.
Discussion
Our environmental scan of the current ambulatory care quality indicators for patients with advanced kidney disease in Canada found 28 different indicators. Using the American College of Physicians/Agency for Healthcare Research and Quality criteria, our panel with expertise in nephrology and quality improvement rated 8 of the indicators as “necessary” to distinguish good-quality care from poor-quality care. We also revealed wide variation in indicator choices across Canada, with 5 indicators being identified as being collected by more than 1 province. Furthermore, very few indicators focused on the IOM domains of timely or equitable care, and there were no structure indicators. This work provides nephrology programs with examples of indicators from which to select, and we hope it motivates more pan-Canadian collaboration on quality measurement to avoid “re-inventing the wheel” when new indicators are developed.
Most published ambulatory care quality indicators for patients with CKD are aimed at primary care providers and not nephrologists.25-28 As a result, many indicators focus on CKD detection (ie, screening in high-risk patients), monitoring (ie, frequency of serum creatinine and proteinuria testing), and blood pressure/cardiovascular disease management, rather than CKD complications and kidney replacement therapy (KRT) planning. The American Society of Nephrology (ASN) Quality Committee recently evaluated measures from kidney and quality metric organizations, 16 identifying 20 ambulatory CKD indicators (17 unique indicators after removing duplicates). Overall, they rated 8 of 17 (47%) highly based on the American College of Physicians/Agency for Healthcare Research and Quality criteria, which was slightly higher than our rating (29%) due to their inclusion of vaccinations (influenza and pneumococcal) and smoking cessation as CKD indicators. Of their 5 remaining necessary rated indicators, our group similarly rated blood pressure control and use of ACEi/ARB as “necessary” metrics to distinguish good-quality from poor-quality care. These indicators are commonly cited in the primary care literature, along with our necessary rated indicators of albuminuria screening in diabetes and use of statins.25-28 This overlap between primary care and advanced CKD indicators may help motivate both groups to share limited resources in the pursuit of joint quality improvement initiatives. Also, both primary care and nephrologists should jointly develop and define indicators to ensure resources are deployed and used in the most efficient manner possible.
Our results also differed from the primary care literature and the ASN study in several important ways, mainly related to KRT planning metrics. It is not surprising that the CKD primary care literature lacks KRT metrics, but the only KRT indicators identified by the ASN study as necessary rated metrics were catheter rate at hemodialysis start and advance care planning. 16 We rated hemodialysis starts with permanent access (similar to catheter rate) as a “supplemental” indicator due to concerns about resource intensity and lack of patient-centeredness. 29 We did not rate advance care planning as a necessary indicator despite its importance, because as a binary measure it may not adequately capture the decision-making process with sufficient granularity, promoting checkbox completion as an unintended consequence. 30 However, these metrics still provide value by ensuring a standard is met for patient care, support staff adherence to care pathways, and may improve team-based models of care. Unlike the ASN study, more than 50% of the Canadian indicators focused on CKD complications (eg, anemia, uremic symptoms) or KRT planning (eg, modality decisions, eGFR at dialysis start). This disparity could be related to health care system differences, but the low provincial overlap suggests that quality-of-care measurement for advanced CKD is likely in its early stages, presenting an opportunity for provinces to collaborate on indicator selection that will help align quality improvement activities and promote shared learning.
Our environmental scan contributes to these goals by categorizing existing indicators according to the IOM and Donabedian frameworks. In this way, we highlight an absence of measures rated as necessary for timely and equitable care, as well as a complete lack of structural indicators. Large variation has been noted in ambulatory kidney clinic staffing ratios and consistency, which likely contributes to differences in team member roles, clinic processes, and potentially outcomes.5,6 While additional research is needed to determine the optimal clinic structure, some minimum standards may be helpful to ensure key processes (eg, home dialysis and transplant education) can be completed. In addition to further development of timely and equitable indicators, we identified 1 metric each for safe and patient-centered care. Potential indicators could focus on medication safety and patient-reported outcome measures (PROMs)/PREMs,31,32 with work needed so that the indicators are attributable to nephrology providers and usable by frontline staff for improvement efforts.
As an initial step in this process, we have provided an example of a balanced quality indicator scorecard for the ambulatory care of patients with advanced kidney disease that covers all of the IOM and Donabedian domains (Table 3). This incorporates several of the indicators rated as necessary, also found in the primary care literature and ASN study (albuminuria screening in diabetes, blood pressure achievement, and use of statins), many of which require specification of the numerator, denominator, and performance targets,16,25-28 Use of ACEi/ARBs is similarly rated as necessary by all groups and could replace blood pressure achievement depending on local data infrastructure and EMR capabilities. The eGFR at dialysis start metric is one of the few necessary rated indicators used by multiple provinces and supported by randomized trial evidence. 33 We also included a patient-reported measure, although work is needed to standardize its measurement, reporting, and staff responses. Other newly proposed indicators are intended to fill gaps in the clinical care of patients with advanced CKD (eg, rates of home dialysis and preemptive transplantation) and/or the IOM/Donabedian domains. For example, time to third next available appointment is a common measure of timely health care access. 34 These new indicator suggestions are meant to stimulate further discussions among administrators, frontline providers, and patients so that balanced scorecards may be developed that are feasible and usable for quality improvement in multiple different local and provincial settings.
First Step Towards Development of a Balanced Quality Indicator Scorecard for Ambulatory Patients With Advanced Kidney Disease.
Note. Several indicators rated as necessary from the environmental scan have been populated (in regular font), with examples of future potential indicators (in bold) and additional work needed to complete the scorecard (in italics).
The major strength our work is that we used established IOM and Donabedian frameworks to organize our indicators and the American College of Physicians/Agency for Healthcare Research and Quality criteria for indicator evaluation. To ensure our work was useful for frontline health care providers, our panel consisted of members from most provinces in Canada who had advanced training in quality improvement and experience with quality improvement activities in ambulatory settings.
There are some limitations to consider. First, the goal was not to survey all ambulatory centers that care for patients with advanced kidney disease across Canada, but rather to obtain an initial impression of which quality indicators exist in different provinces. Our snowball sampling methodology should not be considered exhaustive. Second, we targeted quality indicators that measure the care provided by nephrology providers and programs for ambulatory patients with advanced kidney disease. We did not intend to capture indicators that apply to the quality of care delivered to patients with kidney disease in the primary care setting, although some indicators may be relevant to that patient population. Next, our work focused broadly on the indicator constructs and not the details of inclusion/exclusion criteria or risk adjustment, which must be determined prior to application. 14 Last, our panel consisted of 15 physicians and 1 nurse practitioner, and so does not represent the views of other important stakeholders that will be needed when new indicators are developed and selected, such as patients, pharmacists, and administrators.
Conclusion
The 28 different quality indicators that our panel identified and evaluated across Canada illustrate the variation that exists and highlights current gaps in the measurement of the quality of care provided to ambulatory patients with advanced kidney disease. We rated 8 indicators as “necessary” to delineate good from poor quality of care, and 3 of these indicators were collected by more than 1 province. This work demonstrates that development of quality indicators for ambulatory patients with advanced kidney disease in Canada is in its early stages, and the small number of indicators rated as necessary presents an opportunity for pan-Canadian collaboration, as several new indicators are needed particularly in the domains of timely and equitable care. Our environmental scan should be considered a preliminary step to help guide this process, with the goal of creating indicators that address patient priorities and are useful for frontline quality improvement efforts.
Supplemental Material
sj-pdf-1-cjk-10.1177_2054358121991096 – Supplemental material for An Environmental Scan of Ambulatory Care Quality Indicators for Patients With Advanced Kidney Disease Currently Used in Canada
Supplemental material, sj-pdf-1-cjk-10.1177_2054358121991096 for An Environmental Scan of Ambulatory Care Quality Indicators for Patients With Advanced Kidney Disease Currently Used in Canada by Jay Hingwala, Amber O. Molnar, Priyanka Mysore and Samuel A. Silver in Canadian Journal of Kidney Health and Disease
Footnotes
Acknowledgements
The authors value the support and guidance from Adeera Levin throughout the design and execution of this work. The other 12 volunteer quality committee members include William Beaubien-Souligny, Daniel Blum, Lisa Dubrofsky, Tamara Glavinovic, Claire Harris, Ali Ibrahim, Krishna Poinen, Sachin Shah, Karthik Tennankore, Alison Thomas, Amanda Vinson, and Seychelle Yohanna. S.A.S. is supported by a Kidney Research Scientist Core Education and National Training (KRESCENT) Program New Investigator Award (co-funded by the Kidney Foundation of Canada, Canadian Society of Nephrology, and Canadian Institutes of Health Research).
Ethics Approval and Consent to Participate
No ethics approval or consent was required.
Consent for Publication
All authors consent to the publication of this study.
Availability of Data and Materials
Data and materials may be made available upon request to the corresponding author.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: S.A.S. has received speaking fees from Sanofi Canada. The remaining authors have no conflicts of interest relevant to this study. All authors approved the final version of the submitted manuscript. We certify that this manuscript nor one with substantially similar content has been published or is being considered for publication elsewhere.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
