Attitudes and Opinions of Canadian Nephrologists Toward Continuous Quality Improvement Options

Abstract

Background and objectives:

A shift to holding individual physicians accountable for patient outcomes, rather than facilities, is intuitively attractive to policy makers and to the public. We were interested in nephrologists’ attitudes to, and awareness of, quality metrics and how nephrologists would view a potential switch from the current model of facility-based quality measurement and reporting to publically available reports at the individual physician level.

Design, setting, participants, and measurements:

The study was conducted using a web-based survey instrument (Online Appendix 1). The survey was initially pilot tested on a group of 8 nephrologists from across Canada. The survey was then finalized and e-mailed to 330 nephrologists through the Canadian Society of Nephrology (CSN) e-mail distribution list. The 127 respondents were 80% university based, and 33% were medical/dialysis directors.

Results:

The response rate was 43%. Results demonstrate that 89% of Canadian nephrologists are engaged in efforts to improve the quality of patient care. A minority of those surveyed (29%) had training in quality improvement. They feel accountable for this and would welcome the inclusion of patient-centered metrics of care quality. Support for public reporting as an effective strategy on an individual nephrologist level was 30%.

Conclusions:

Support for public reporting of individual nephrologist performance was low. The care of nephrology patients will be best served by the continued development of a critical mass of physicians trained in patient safety and quality improvement, by focusing on patient-centered metrics of care delivery, and by validating that all proposed new methods are shown to improve patient care and outcomes.

What was known before

There is a societal shift toward holding individual physicians publically accountable for quality of patient care.

What this adds

Canadian nephrologists are engaged in quality improvement efforts. Nephrologists do not currently support public reporting of individual physician performance.

Introduction

The high-quality, modern health care system strives to deliver care that is safe, effective, efficient, equitable, patient centered, and timely. ¹ Attempts to measure these domains of quality and encourage physician engagement in the delivery of high-quality care dominate the health care landscape in North America. As a specialty, nephrology has been an active participant in developing metrics of quality care and incentivizing physicians to meet these goals.^2,3

To date, measurement and reporting of quality of care in nephrology has typically occurred at a health care facility level. This makes sense in the context of a hemodialysis unit where care is often team based and shared, with many physicians and other health care workers influencing the care of a single patient. External motivation for a facility to perform well may come through the public reporting of adherence to quality metrics, or through funding linked to performance.^4,5

A shift to holding individual physicians accountable for patient outcomes, rather than facilities, is intuitively attractive to policy makers and to the public. In the United States, comparative reports of individual surgeon outcomes are available online. ⁶ Overall, the effect of public reporting on clinical outcomes has been positive. ⁷ The utility of such initiatives, the reasons underlying their apparent successes, and the potential for adverse effects remain contentious.^8,9 However, the popularity of web-based platforms, such as Rate My MD, suggests that at a societal level, there is significant appetite for public comparison of health care providers. Furthermore, the Medicare Access and CHIP Reauthorization Act of 2015 legislation, recently passed in the United States, would link physician payment to care quality, resource use, and meaningful use of electronic health record technology.^10,11

Against this backdrop of increasing legislative oversight and public scrutiny of physician performance, we conducted a survey to assess nephrologists’ attitudes to, and awareness of, quality metrics. We were particularly interested in how nephrologists would view a potential switch from the current model of facility-based quality measurement and reporting, to publically available reports at the individual physician level. A secondary objective was to assess whether modality of treatment was associated with attitudinal differences toward quality improvement.

Methods

The study was conducted using a web-based survey instrument (Online Appendix 1). The survey, drafted by the authors, was initially pilot tested on a group of 8 nephrologists from across Canada and revised based on their feedback. The final survey was then e-mailed to 330 nephrologists through the Canadian Society of Nephrology (CSN) e-mail distribution list. Following a prenotification letter, the survey was e-mailed in early August 2015. Two follow-up reminder e-mails were sent in early and late September 2015, respectively.

The survey consisted of 3 main sections made up of questions targeted toward local quality improvement practices, general attitudes toward quality improvement in nephrology, and finally, a limited list of respondent demographic questions. The questions were either Likert scale questions (5-point from “strongly disagree” to “strongly agree”) or multiple choice questions.

The e-mailing, administration, data entry, and data analysis of the survey questionnaires were done by the authors. The survey was distributed by Survey Monkey. The results were analyzed using univariate statistics. This research project was approved by the Research Ethics Board of Humber River Hospital, Toronto, Ontario. There was no external funding. The authors have no conflicts of interest to disclose.

Results

After being sent to 330 nephrologists across Canada, a total of 137 nephrologists responded. Respondents were asked to confirm eligibility only if they were both a nephrologist currently practicing in Canada, and primarily a clinical nephrologist dealing with adult patients. Ten people indicated that they were not eligible, leaving 127 of 320 responders (43% response rate).

Demographic data are shown in Table 1. Respondents were distributed across provinces proportionately to population. The majority, 80%, were in a university-based group practice. Medical directors made up 33% of the total. The remainder was comprised of staff nephrologists.

Table 1.

Baseline Characteristics of Survey Respondent.

Demographics	n (%)
Location
Ontario	43
Quebec	17
British Columbia	13
Alberta	12
Manitoba	6
Nova Scotia, Saskatchewan	3 respectively
New Brunswick	2
Responsibility
Medical/dialysis director	33
Staff nephrologist	67
Affiliation
University based	80
Community based	20
Quality improvement training
Before fellowship	1
During fellowship	6
After fellowship	39
Never	54
Role in quality improvement
Received training in QI methods	29
Has expertise in QI methods	33
Participate in QI initiatives	89
Leader in QI initiatives	51

Note. QI = quality improvement.

The majority of participants in this survey rated their training in quality improvement as limited, with 54% answering “never” in answer to “when did you receive training in continuous quality improvement methods.” Similarly, only 33% agreed or strongly agreed that they had “expertise in quality improvement methods.” Despite this, the majority, 89%, participate in quality improvement initiatives, with 51% doing so in a leadership capacity (Table 1).

Figure 1 illustrates that 48% of responders agree or strongly agree that there is significant variation in physician performance in their facility, and 65% agree or strongly agree that reducing variation in physician practice is likely to improve patient outcomes.

Figure 1.

Distribution of responses to the statement: “To what degree do you agree with the following statements about quality improvement in nephrology.”

Figure 2 portrays potential differences in Continuous Quality Improvement (CQI) by treatment modality, as applied to predialysis chronic kidney disease (CKD) care, incenter hemodialysis care, and home dialysis care. There was very strong agreement that good measures of quality of care already exist in nephrology. Only 4% expressed disagreement with a statement that “there are some appropriate and valid measures that should form a basis for CQI activities” as applied to each of incenter hemodialysis, home dialysis, and predialysis care (Figure 2, panel A). In general, respondents felt that attention to traditional quality metrics had a positive effect on patient outcome, with 68% and 74%, respectively, agreeing that anemia management and vascular access type were clinically meaningful measures of quality (data not shown). There was also robust, comparable support for measuring more novel patient-centered metrics such as quality of life (69%), patient experience or satisfaction (77%), and predialysis education (78%) (data not shown).

Figure 2.

Distribution of responses, as applied to predialysis, incenter, and home dialysis care, to the statements: (A) In my opinion, there are some appropriate and valid measures that should form a basis for CQI activities; (B) Shifting from program to physician level measurement and reporting is likely to improve quality of care; (C) Participation of multiple physicians in the shared care of patients could confound physician specific measures; (D) Participation of nurses and/or other nonmedical professional staff in the shared care of patients could confound physician specific measures.

There was overall agreement that nephrologists should be held accountable for the quality of care they deliver. Figure 3A shows that 79% agreed or strongly agreed with a statement that “there are quality improvement metrics that nephrologists should be held accountable for.” Eighty-two percent of nephrologists surveyed would be interested in receiving a confidential personalized scorecard indicating their adherence with a set of quality measures (Figure 4A).

Figure 3.

Distribution of responses to the statement: “To what degree do you agree with the following statements about quality improvement in nephrology.”

Figure 4.

Distribution of responses to the statement: “To what degree do you agree with the following statements about quality improvement in nephrology.”

Support for public reporting of these measures was less unanimous; only 43% of nephrologists agreed that public reporting of facility-level performance would improve patient outcomes (Figure 3B), although this did improve to 63% when we asked about facility-level reporting of a theoretical set of ideal “valid and appropriate measures” (Figure 4C). Only 30% of survey participants were in favor of public reporting of individual performance (Figure 3C), and this did not improve even when they were asked about reporting on an ideal set of measures (Figure 4B). Many nephrologists felt that a shift from facility to physician-level reporting would not positively impact patient care either in the predialysis, hemodialysis, or home dialysis care setting, where 40% either disagreed or strongly disagreed with this suggestion (Figure 2, panel B). Furthermore, a majority (66%-86%) expressed concern that physician-specific measures could be confounded by participation of multiple physicians, nurses, and/or other nonmedical professional staff in shared care of patients in the 3 treatment modalities (Figure 2, panel C and D). Not shown in these figures is that 58% agreed or strongly agreed that another confounder is differences in patient demographics and comorbidities when comparing local physician’s patient populations with their own.

Discussion

We conducted this survey to assess the attitudes of Canadian nephrologists to quality improvement work and specifically to the nature of the measurement and reporting of quality measures. We hoped to learn how many of our colleagues were engaged in QI efforts and what training they had to do this work, what their perceptions of existing quality measures were, how they felt about public reporting of facility level results, and whether they would support a move to publically available individual physician-level reporting.

Like any survey, this study has strengths and limitations. To gain preliminary data, the survey methodology is inexpensive and lends itself well to this topic, which is difficult to investigate empirically. Limitations include a good but not optimal response rate despite 3 e-mail reminders. As with any survey, the answers of those who responded may be systematically different from those of nonresponders. The CSN e-mail list we used does not include every nephrologist in Canada. The exact number of practicing adult nephrologists in Canada is not known, and it is not possible to know the demographic and other differences between those who were e-mailed and those who were not. Similarly, answers may reflect socially acceptable opinions rather than actual ones. Nonetheless, we believe this study and its results are an important flag for the nephrology community to consider.

Several interesting themes emerged. The literature suggests that physicians participate in only 35% of quality improvement efforts. ¹² Barriers to physician involvement include lack of time, threats to autonomy, financial disincentives, and a lack of quality improvement skills. Our survey results confirm that Canadian nephrologists are generally supportive of quality improvement efforts. While almost all of the nephrologists we surveyed are involved in QI work, often as leaders, few felt they were adequately skilled with 54% saying they had never received any training in QI. The dissociation between wanting to improve care and knowing how to do so has plagued the quality and safety movement since its inception 15 years ago. ¹³ Poorly conceived QI projects have led to underwhelming results and unintended consequences. This effect may be magnified when an improvement initiative is attempted on a national or health care system level rather than locally. For example, a recent study demonstrated that the Centers for Medicare and Medicaid Services program linking nephrologist reimbursement for dialysis to frequency of visit inadvertently led to a reduction in the number of patients starting on home dialysis modalities. ¹⁴

Closing the gap between the desire and the ability to change the care delivered remains a significant challenge facing the nephrology community, and will require front-line nephrology staff trained in QI methodology as well as physician leaders with the influence and ability to direct health care policy decision making. This survey offers some cause for optimism on this front. The flip side of our finding that many nephrologists possessed little if any training is that 9% of respondents (10 individuals) possess a graduate-level qualification in improvement methodology. This number is quite significant in the context of Canadian nephrology, and augers well that leadership with the will and know how to improve care can be developed.

Indeed, training opportunities for physicians interested in pursuing a career in quality improvement now exist across Canada and the United States. Both the University of Toronto and Queen’s University offer master’s-level training in quality and patient safety. The Department of Medicine at the University of Toronto now recognizes applied scholarly activity in Quality Improvement through a new academic job description titled “Clinician in Quality and Innovation.” In the United States, the Veterans Health Administration has successfully established a national fellowship program in health care quality and safety. The Feinberg School of Medicine at Northwestern University has recently launched Master of Science and PhD programs in Healthcare Quality and Patient Safety. The engagement and interest of the academic medical community will continue to be critical in ensuring that quality improvement publications are properly vetted and those of merit disseminated. The SQUIRE guidelines have been pivotal in providing a rigorous framework upon which to plan and report QI endeavors. ¹⁵

Nephrology is fortunate in having well-developed, up-to-date clinical practice guidelines based on best available evidence and international expert consensus opinion. This has allowed for the development of measures of quality of care delivery based on these guidelines.^16,17 Our survey found that few nephrologists questioned the validity of these measures. It is encouraging to see that a majority of respondents expressed agreement with the currently proposed shift to more patient-centered measures of care delivery such as patient experience or satisfaction and quality of life.^18,19

There is a groundswell of opinion among nephrologists that we need to give more consideration to the preferences and values of our patients and their families. For example, a focus on mortality or hospitalization rates may lead us to initiate an elderly frail patient on hemodialysis. As Carson et al demonstrated, the extra life span that this notional patient would enjoy would largely be spent either in or traveling to and from the dialysis unit. ²⁰ It is encouraging therefore to see that 78% and 72%, respectively, of respondents to our survey agreed that predialysis education and dialysis modality selection were measures of care delivery which would likely improve patient outcomes.

The Achilles heel of quality performance measures is how to implement practice changes to improve performance over time. Approaches to this problem have recently been published in a series of Clinical Journal of the American Society of Nephrology (CJASN) articles.^21-24 Our survey shows that nephrologists are broadly in agreement with the idea that they should be held accountable for the quality of care they deliver, and most would favor being kept apprised of their performance via a confidential scorecard.

It is worth recognizing that the limitation of this (and any) survey is we are recording the views of a self-selected group who may be more interested in, and inclined toward, providing high-quality care. Forty-eight percent of respondents felt there was significant variation of practice in their facility (Figure 2) and 78% felt that participation of other physicians would confound results in the measurement of the quality of care delivered to their hemodialysis patients. So how does one reduce variation in physician practice and incentivize colleagues toward best practice? Public reporting is one potential strategy. Based on this survey results, Canadian nephrologists appear to be ambivalent to this approach at a facility level and opposed to it on an individual physician level. This opposition may be well founded. A recent publication in JAMA Cardiology showed that by removing public reporting of outcomes for coronary artery stenting in the setting of cardiogenic shock, more patients in the state of New York receive percutaneous intervention and mortality rates fell compared with other states. ²⁵ Similar concerns have been voiced in the surgical literature. ²⁶ One concern about physician-level public reporting is that nephrologists may be unconvinced that case-mix adjustments can properly reflect the influence of a high prevalence of frail or nonadherent patients and hence would avoid accepting dialysis patients who would negatively influence their “score” (cherry picking).

Ontario dialysis and kidney care centers are currently mandated to report on a variety of quality metrics upon which their funding is linked. While in the past, there was some discussion at the provincial Ontario Renal Network (ORN) about physician-level reporting, it was recognized that shared care models dominate dialysis care and preclude any meaningful analysis. For predialysis care, which is usually provided by a single nephrologist, the most responsible physician is identified and reported to ORN, but the current plan is to provide each physician with a decision support tool that allows them to examine their own practice, without public reporting.

The authors wish to acknowledge some biases and perceptions that were incorporated into the survey design. Nephrology is a particularly data-impoverished discipline, with fewer randomized controlled trials than other internal medicine subspecialties.^27,28 Consequently, it was our opinion that there are currently few (if any) entirely validated and appropriate performance measures that could be applied to incenter hemodialysis, predialysis CKD, and home dialysis. It was also our opinion that shifting to physician-based measures would be easier and more acceptable in predialysis CKD and home dialysis than in incenter hemodialysis. We reasoned that these 2 modalities use a single, most responsible physician-type model (without shared physician care) in most Canadian settings, in contrast to common models of incenter hemodialysis care. We are quite surprised and intrigued that our biases were not shared by the Canadian nephrology community, who believe that there are valid quality measures in all areas, and who do not distinguish differences in CQI approaches between incenter hemodialysis and the other 2 modalities.

This study is unique in that it is the only one we are aware of that examines physician attitudes toward a shift from facility-based CQI toward physician-level measurement and reporting. Given the nature of the common team-based practice of nephrology, there are unique potential problems with undertaking such a shift. This survey can be considered a preliminary report of many of the challenges that need to be overcome before broad nephrologist engagement can be achieved.

Modern medicine evolves best when hypotheses are tested and validated in well-designed and adequately powered empirical, prospective studies. We believe that the same principles should be applied to new approaches to CQI. Facility-based and/or physician-based public reporting may or may not lead to better patient outcomes in nephrology practice and may have both foreseen and unforeseen consequences. We hope that our study will stimulate health care system investigators to design and implement prospective evaluations of quality improvement methods as applied to nephrology.

Conclusion

This survey demonstrates that virtually all Canadian nephrologists are engaged in efforts to improve the quality of patient care. They feel accountable for this and would welcome the inclusion of patient-centered metrics of care quality. Support for public reporting as an effective strategy was not strong, particularly if this was on an individual level. A minority of those surveyed had training in quality improvement. We conclude that the care of nephrology patients will be best served by the continued development of a critical mass of physicians trained in patient safety and quality improvement, by focusing on patient-centered metrics of care delivery, and by validating that all proposed new methods are shown to improve patient care and outcomes.