Process Evaluation Alongside a Cluster-Randomized Trial of a Multicomponent Intervention Designed to Improve Patient Access to Kidney Transplantation

Abstract

French

Background:

In a cluster-randomized trial, we learned that a novel multicomponent intervention designed to improve access to kidney transplantation did not significantly increase the rate of completed steps toward receiving a kidney transplant. Alongside the trial, we conducted a process evaluation to help interpret our findings.

Objective:

To determine whether the intervention addressed targeted barriers to transplant and whether the implementation occurred as planned.

Design:

Mixed-methods process evaluation informed by implementation science theories.

Setting:

Chronic kidney disease (CKD) programs in Ontario, Canada. These programs, providing care to patients with advanced CKD, participated in the trial from November 1, 2017 to December 31, 2021 (either in the intervention or usual care group).

Participants:

Health care providers (eg, nurses, managers) at Ontario’s 27 CKD programs.

Methods:

We conducted surveys (n = 114/162 [70.4%]) and semi-structured interviews (n = 17/26 [65.4%]) with providers in CKD programs in Ontario, Canada. In both the intervention-group and control-group surveys, using the Theoretical Domains Framework, we assessed perceived barriers to transplant and how barriers changed throughout the trial period. In the intervention-group surveys and interviews, using the normalization process theory, we assessed the extent to which the intervention was embedded into daily routines. In the intervention-group surveys, and by completing an implementation checklist, we assessed fidelity of implementation.

Results:

Perceived barriers to transplant did not substantially differ between providers in the intervention and usual care groups, and both groups reported disagreeing or feeling neutral that the targeted barriers impeded transplant access. Intervention-group providers reported that intervention activities were becoming a regular part of their work and that they engaged with its components. However, they also felt the intervention was complex and described needing more resources, a better execution plan, and more buy-in from frontline staff. Fidelity was high for administrative support, quality improvement teams, delivery of educational resources, and patient peer support. The use of performance reports was low.

Conclusions:

We identified several possible reasons why the intervention was unsuccessful. Improving access to kidney transplantation remains a high priority for health care systems. We will continue to foster a quality improvement culture, and our results will guide future interventions.

Limitations:

Two of the 13 intervention-group CKD programs did not participate in this evaluation.

Trial Registration:

ClinicalTrials.gov Identifier: NCT03329521

Keywords

transplantation living kidney donation implementation science qualitative

Introduction

The best treatment option for most patients with advanced chronic kidney disease (CKD) is a kidney transplant.^1,2 Compared with the alternative treatment of maintenance dialysis, patients who receive a kidney transplant experience a better quality of life and survival and have lower health care systems costs.^1,3,4 Unfortunately, there are not enough deceased donor kidneys to meet the demand, and rates of living kidney donation have stagnated.^5,6 Many barriers prevent patients from receiving a kidney transplant.^7-9

The Enhance Access to Kidney Transplant and Living Kidney Donation cluster-randomized trial (EnAKT LKD) done in CKD programs in Ontario, Canada, found that a novel multicomponent intervention did not significantly increase the rate of patients completing steps toward receiving a kidney transplant when compared with usual care.¹⁰ We conducted a process evaluation alongside the EnAKT LKD trial to help understand the trial findings. We assessed whether the intervention addressed intended barriers and whether the implementation occurred as planned.

Methods

Overview of the EnAKT LKD Cluster-Randomized Trial

In 2023, in Ontario, Canada, 27 CKD programs delivered kidney care to over 24 000 patients with advanced CKD (ie, patients approaching the need for dialysis or receiving maintenance dialysis). In 2016, patients, health care professionals, and government agencies overseeing kidney and transplant care came together to develop a multicomponent intervention to address critical barriers preventing access to kidney transplantation and living kidney donation. We evaluated the effects of this intervention in a pragmatic, 2-arm, parallel-group, open-label, registry-based, superiority, cluster-randomized clinical trial conducted in all of Ontario’s CKD programs (clusters) (the EnAKT LKD trial).¹⁰ At the time of randomization, we allocated 13 of Ontario’s 26 CKD programs to receive the intervention from November 1, 2017 to December 31, 2021 (4.2-year trial period), while the remaining CKD programs received usual care. There were 26 CKD programs when the trial started in November 2017; in April 2018, 1 program split into 2. For the primary trial analysis, this program was treated as 1 program, but for this process evaluation, we treated it as 2 separate CKD programs.

The trial included over 20 000 potentially transplant-eligible patients with advanced CKD.¹⁰ The primary outcome was the rate of steps completed toward receiving a kidney transplant, with each patient being able to complete up to 4 steps, including (1) being referred to a transplant center for evaluation, (2) having a potential living donor contact a transplant center for evaluation, (3) being added to the deceased donor waitlist, and (4) receiving a transplant from a living or deceased donor. The intervention included 4 main components, which are summarized in Table 1 with more details in Supplemental Table 1. The 4 main components were (1) administrative support to establish local quality improvement (QI) teams, (2) transplant educational resources, (3) an initiative for transplant recipients and living donors to share stories and experiences (Transplant Ambassador Program [TAP]), and (4) program-level performance reports and oversight by administrative leaders. Chronic kidney disease programs had flexibility in implementing each of the intervention components. A provincial core operations group, which included a part-time medical lead and 3 full-time personnel from kidney and transplant government-funded agencies, supported QI teams at each CKD program. The provincial core operations group also organized monthly collaborative calls as a forum to address challenges and share best practices.

Table 1.

Description of the EnAKT LKD intervention components.

Intervention component	Description
1. Administrative support to establish local quality improvement (QI) teams	Each CKD program established a local quality improvement team that was provided with training, administrative support, and resources to implement the intervention. A central operations group, including a part-time medical lead and 3 full-time personnel (ie, manager, analyst, strategist) supported the QI team. Quality improvement team members were invited to attend provincial rounds monthly to share experiences, successes and barriers to the strategy. Financial support of $10 000 CAN per year was provided to each QI teams to support their activities
2. Transplant education and resources for staff, patients, and living donor candidates	Educational resources were developed and delivered to CKD program staff, patients, and living kidney donor candidates (ontariorenalnetwork.ca/en/kidney-care-resources/clinical-tools/transplantation)
3. Support from kidney transplant recipients and living donors	The Transplant Ambassador Program (TAP: www.transplantambassadors.ca) connects patients with kidney disease and their families to kidney transplant recipients and living kidney donors who can share their lived experience with transplantation and living kidney donation and provide emotional support and hope
4. Program-level performance reports and oversight by program leaders	Data reports were provided to CKD programs about their transplant-related performance (CKD program performance reports) and of patients who appeared to be eligible for transplant referral who were not yet referred (transplant referral eligibility reports). There were annual one-on-one performance meetings between leaders of each CKD program and the provincial renal agency

Note. Further details on the intervention components can be found in the trial protocol.¹¹

Overview of the EnAKT LKD Process Evaluation

We conducted a mixed-methods process evaluation where our objective was to assess health care professionals’ perception of how well the intervention addressed barriers to kidney transplantation and to determine how well the planned intervention was executed in practice (referred to as fidelity of implementation). We asked about perceived barriers to transplant over time to understand whether the intervention targeted the true barriers health care providers perceived they were experiencing and whether there was a change in those experienced barriers with the implementation of the intervention. We coupled the analysis of barriers with an analysis of fidelity to understand how well the intervention was implemented. Poor implementation of one or more intervention components would provide insight into why targeted barriers may not have changed over time. These data together would allow us to optimize the intervention by either addressing different barriers by modifying intervention components or by improving the implementation of certain intervention components.

We administered an online survey to health care professionals in both groups to examine the barriers and enablers experienced before the trial and during the trial ( Supplemental Appendices 1-2 ). We also administered surveys and semi-structured interviews with health care professionals in the intervention group to understand how the intervention activities became embedded into routine practice ( Supplemental Appendix 3 ). To assess fidelity of intervention implementation, we analyzed documents provided by the provincial core team that contained information on how the trial components were delivered ( Supplemental Appendix 4 ). We also evaluated potential usual care group contamination through an online survey ( Supplemental Appendix 2 ). Figure 1 provides an overview of the study objectives, methods of evaluation, and associated theoretical frameworks. We followed the UK Medical Research Council guidance on conducting and reporting process evaluations of complex interventions.¹² We conducted surveys and interviews from December 2020 to June 2021. Our protocol provides further details about the methods used.¹³ Online surveys were reported following the Checklist for Reporting Results of Internet E-Surveys (Supplemental Table 2) and interviews were reported following the Standard for Reporting Qualitative Research (SRQR) checklist (Supplemental Table 3).^14,15

Figure 1.

Design of the EnAKT LKD process evaluation.¹³

Theoretical Frameworks

To inform the process evaluation, we used 2 theoretical frameworks, the normalization process theory (NPT) and the theoretical domains framework (TDF). In brief, NPT describes factors that promote and inhibit implementing and embedding new processes into routine practice, such that, they are no longer visible as distinct from what is normally done.^16-18 Normalization process theory proposes 4 constructs to nuance how new practices become embedded in health care settings:¹⁶ (1) Coherence (sense-making)—the extent to which individuals and teams understand the intervention elements and why a new set of practices are being introduced in their workflow. (2) Cognitive participation (engagement)—the extent to which individuals believe and buy into the intervention and the relational work (eg, building and sustaining engagement and collaboration) they do to prepare for it. (3) Collective action (enacting)—the degree to which individuals and teams have the confidence, training, skills, resources, working relationships and management support to embed new practices into their workflow. (4) Reflexive monitoring (appraisal)—how individuals and teams track how they are doing.^17-20 The TDF synthesizes over 30 theories of behavior change into 14 domains that can be used as categories of barriers and enablers that are targeted by an intervention to assess whether the intervention affected the barriers it was designed to address.^21,22 Using the TDF, we assessed perceived barriers in the intervention group at the following time points: before the EnAKT LKD intervention began (prior to Fall 2017), after the EnAKT LKD intervention began but before the COVID-19 pandemic (Fall 2017 to March 2020), and barriers at the time of the survey (December 2020-June 2021). For CKD programs in the usual care group, participants reported on barriers perceived before the COVID-19 pandemic (prior to March 2020), during the first wave of the COVID-19 pandemic (March-May 2020) and barriers at the time of the survey (December 2020-June 2021). We embedded these frameworks into the surveys and interview guide.

Semi-Structured Interviews

We used an NPT-based directed content analysis to understand the process of implementing and integrating the EnAKT LKD intervention at CKD programs.²³ Interview transcript analysis involved 5 steps: (1) familiarization, (2) coding participant responses to specific NPT constructs, (3) inductively generating sub-themes within each construct, (4) grouping themes across constructs, and (5) defining and naming themes. Two analysts (MW and SY) reviewed the codebook to assess the accuracy of data representation. The analysts co-developed an overarching thematic framework throughout several iterative meetings. Where differences in interpretation of the data arose, the 2 analysts discussed interpretations until arriving at one that best accounted for participant views and experiences.

We obtained ethics approval from the Ottawa Health Sciences Research Ethics Board, and written informed consent was obtained from all participants (REB 20200426-01H).

Participants

For the online survey, we aimed to survey 162 health care professionals: 2 nephrologists, 2 dialysis nurses, and 2 specialized kidney clinic team members from each of the 27 CKD programs. For the semi-structured interviews (intervention group only), a research coordinator (MW) aimed to conduct 26 semi-structured interviews by phone or videoconference: 13 with QI team leads and 13 with other health care professionals.

Data collection

The online survey was administered from December 2020 to June 2021. The online survey for the intervention group collected information on perceived barriers to kidney transplant, implementation evaluation, and fidelity. The online survey for the usual care group collected information on perceived barriers to kidney transplant and contamination (use of intervention components in the usual care group). The semi-structured interviews were conducted with participants from the intervention group between December 2020 and June 2021; these interviews collected information on implementation evaluation.

Perceived barriers to kidney transplant (intervention and usual care groups)

As part of the online survey, participants were asked to recall current and past barriers at 3 points: before the trial before the pandemic, during the trial but before the pandemic, and during the trial and during the pandemic. Selected barriers were those intended to be addressed by the intervention. The presence of each barrier was rated using a 5-point Likert scale (strongly disagree to strongly agree).

Implementation evaluation (intervention group only):

Online survey. The extent to which the intervention was embedded into daily routines was evaluated using a validated 23-item survey²⁴ adapted for our trial. The survey included 1 item to assess the degree of normalization of the intervention using a 10-point scale (0 [not at all] to 10 [completely]), and 22 items to assess coherence, cognitive participation, collective action, and reflexive monitoring; these items were assessed on a 5-point Likert scale (strongly agree to strongly disagree, or not relevant).

Semi-structured interview. We developed interview guides to elicit content on 4 constructs (coherence, cognitive participation, collective action, and reflexive monitoring) and additional open-ended prompts to elicit participants’ reflection about their experience with each intervention component. We also asked specific questions to prompt views on how the COVID-19 pandemic influenced their use of the intervention.

Fidelity (intervention group only)

We included questions about the frequency of intervention activities, use of educational resources, and perceived usefulness of intervention components. For the TAP, ambassadors completed monthly reports that summarized the number of meaningful interactions with patients, family members or potential living donors. A meaningful interaction was defined as a conversation about kidney transplantation lasting at least 5 minutes.

Document analysis

Using data from the provincial core team, a research coordinator (MW) completed a document analysis to examine intervention fidelity within CKD programs in the intervention group. For example, how often members of the CKD program participated in the monthly collaborative calls (Supplemental Table 4).¹³

Analysis

Results are summarized using descriptive statistics. Each response was weighted equally, acknowledging the number of responses per CKD program varied. Categorical variables are presented as counts and proportions, and continuous variables as means (standard deviations [SDs]), medians (25th, 75th percentiles), or ranges, as appropriate. Where applicable, we assessed the internal consistency of items within each construct using Cronbach’s alpha for survey data.^24,25 Higher scores indicated greater integration of the intervention into the respondents’ workflow. We audio-recorded interviews and transcribed them verbatim, which were then verified by the interviewer (MW), de-identified and analyzed using NVivo 11.¹³

Results

Participants

In total, 114 health professionals completed the online survey (70% response rate). Of 78 invited professionals in the intervention group, 53 (68%) completed the online survey (10 nephrologists, 2 nurse educators, 11 specialized kidney clinic nurses, 13 dialysis nurses, 3 social workers, 3 managers, 3 coordinators, 8 other health professionals). Respondents represented 11 of the 13 CKD programs in the intervention group (2-7 respondents per program). Of 84 invited professionals in the usual care group, 61 (72%) completed the survey, representing 13 of the 14 programs in the usual care group (1-6 respondents per program). Of the 26 invited professionals in the intervention group, 17 (65%) participated in interviews, representing 11 of the 13 programs (the same 11 CKD programs that completed the online surveys). Of the 17 respondents, 11 were QI team leads, and 6 were other QI team members. On average, the respondents had worked at the CKD programs for 13 years (SD = 8).

Perceived Barriers to Kidney Transplant

Table 2 reports health care professionals’ perceived barriers to kidney transplant in the intervention and usual care groups. In the intervention group, there was a slight improvement over time in mean scores for all reported barriers, particularly between the pre-trial and post-trial pre-pandemic period. In most cases, responses were similar in both groups, including both groups generally reporting they disagreed or felt neutral that the intervention targeted barriers that impeded transplant access.

Table 2.

Potential Barriers to Kidney Transplantation at the Level of the Provider, Chronic Kidney Disease Program, Patient, and Living Donor in the Usual Care and Intervention Group as Reported by Health Care Professionals.

Level of barrier	Theoretical domains framework	Potential barriers	Group	Before trial (prior to Fall 2017)	During trial before pandemic (prior to March 2020)	During trial after pandemic (December 2020-June 2021)
Health care provider-related barriers	Knowledge	I am not sure which patients are eligible for transplant	Usual care	—	2.14 (1.09)	2.12 (1.04)
		I am not sure which patients are eligible for transplant	Intervention	2.21 (.86)	2.02 (.90)	2.06 (.94)
		I am not sure when to refer a patient to a transplant center	Usual care	—	2.05 (1.02)	2.12 (1.05)
			Intervention	2.19 (.99)	1.98 (.99)	1.98 (.97)
		I am not sure what the process is for referring eligible patients for transplant at my program	Usual care	—	1.81 (.96)	1.85 (.94)
			Intervention	2.02 (1.01)	1.83 (.96)	1.83 (.96)
		I don’t know enough about living kidney donation to promote it to my patients	Usual care	—	1.85 (.98)	1.80 (.91)
			Intervention	2.13 (1.08)	1.85 (1.06)	1.87 (1.04)
		I am not sure which patients would benefit from a living donor kidney transplant versus a deceased donor kidney transplant	Usual care	—	2.20 (1.17)	2.19 (1.15)
			Intervention	2.43 (1.12)	2.15 (1.12)	2.17 (1.13)
	Skills	I am not sure how to help my patients find a living kidney donor	Usual care	—	2.42 (1.16)	2.37 (1.10)
	Skills		Intervention	2.68 (1.09)	2.40 (1.04)	2.32 (1.02)
	Beliefs about capabilities	I am not confident in my ability to promote kidney transplant and living donation	Usual care	—	2.10 (1.14)	2.34 (1.14)
			Intervention	2.57 (1.16)	2.40 (1.25)	2.47 (1.25)
		I am not comfortable enough talking about living kidney donation with my patients	Usual care	—	1.78 (.93)	1.69 (.82)
			Intervention	2.13 (1.06)	1.79 (.95)	1.81 (.92)
Chronic Kidney Disease Program-related barriers	Environmental context and resources	I do not have access to the necessary data to understand how well patients in our program are completing steps toward receiving a kidney transplant	Usual care	—	2.59 (1.19)	2.66 (1.21)
			Intervention	2.83 (1.24)	2.68 (1.24)	2.62 (1.19)
		I do not have access to the necessary information to know if a patient is potentially eligible to receive a kidney transplant	Usual care	—	2.27 (.98)	2.29 (.98)
			Intervention	2.66 (1.01)	2.30 (.88)	2.26 (.92)
		I am not provided with effective educational resources to promote kidney transplant	Usual care	—	2.68 (1.18)	2.78 (1.18)
			Intervention	2.47 (1.25)	2.06 (1.01)	2.09 (1.08)
		Resources (including time) are not allocated to promoting transplant in our program	Usual care	—	2.49 (1.38)	2.71 (1.45)
			Intervention	2.74 (1.36)	2.43 (1.33)	2.36 (1.26)
	Social influences	There is insufficient communication with transplant centers to help move my patients through the evaluation process	Usual care	—	2.36 (1.08)	2.42 (1.09)
	Social influences		Intervention	2.68 (1.10)	2.34 (.98)	2.32 (.96)
	Motivation and goals	Transplant is not a priority in our program	Usual care	—	1.85 (1.26)	1.97 (1.38)
	Motivation and goals	Transplant is not a priority in our program	Intervention	2.00 (1.08)	1.64 (.82)	1.72 (.93)
Patient-related barriers	Knowledge	Patients do not demonstrate enough knowledge about transplant	Usual care	—	3.17 (1.10)	3.24 (1.14)
			Intervention	3.13 (1.06)	2.77 (1.03)	2.77 (1.03)
		Patients demonstrate misconceptions about their eligibility for transplant	Usual care	—	3.51 (.99)	3.61 (.93)
			Intervention	3.49 (.95)	3.36 (.87)	3.30 (.98)
	Environmental context and resources	Patients are not provided with transplant education that includes their family/support system	Usual care	—	2.41 (1.07)	2.53 (1.09)
			Intervention	2.70 (1.06)	2.13 (.82)	2.17 (.87)
		Patients do not have access to educational materials about kidney transplantation and living kidney donation	Usual care	—	2.29 (1.00)	2.46 (1.04)
			Intervention	2.49 (1.04)	1.91 (.65)	1.98 (.79)
	Social influences	Patients indicate the absence of interactions with past recipients and living kidney donors to understand the transplant process from a patient perspective	Usual care	—	3.08 (.77)	3.24 (.75)
	Social influences		Intervention	3.06 (.97)	2.47 (.80)	2.53 (.91)
Living donor-related barriers	Knowledge	Potential living donors demonstrate a lack of knowledge about living kidney donation	Usual care	—	3.29 (.95)	3.37 (.98)
	Knowledge		Intervention	3.47 (.75)	3.11 (.81)	3.09 (.86)
	Social influences	Potential living donors indicate a lack of peer support to help them through the living donor evaluation process	Usual care	—	3.27 (.78)	3.34 (.78)
	Social influences		Intervention	3.34 (.76)	3.04 (.75)	3.09 (.88)

Barriers were rated using a 5-point Likert scale from strongly disagree (1) to strongly agree (5). Data reported as mean (standard deviation). Respondents were asked to consider barriers from the perspective of the health care provider, CKD program, patients with CKD and living kidney donor candidates. All data were collected at one timepoint from December 2020 to June 2021; responses to earlier timepoints are retrospective accounts.

Extent to Which the Intervention was Embedded Into Daily Routines

In brief, when assessing the overall normalization of the intervention into daily work, respondents indicated they were relatively familiar with the intervention (mean = 7.1 of 10, with a score of 10 indicating they were thoroughly knowledgeable, SD = 2.9); most felt the intervention was currently a regular part of their work (mean = 6.9 of 10, with a score of 10 indicating completely integrated, SD = 3.0) and felt the intervention would become a more substantial part of their regular work in the future (mean = 8.1 of 10, SD = 2.6) (Table 3). When assessing key factors that led to successful intervention implementation, most participants were neutral about the work required to understand the intervention and the new activities associated with it, the work to understand who was responsible for different aspects of the implementation process, and the work needed to evaluate success.

Table 3.

NoMAD Instrument Results Presenting the Implementation and Integration of the EnAKT LKD Intervention.

Implementation and integration of the Access to Kidney Transplant (AKT) multicomponent intervention
General questions about the intervention
Item		Scale of 0-10 N = 46 Mean (SD)
When you engage in the Access Kidney Transplant (AKT)^¥ Strategy, how familiar does it feel? (0—still feels very new to 10—feels very familiar)		7.11 (2.95)
Do you feel that the AKT Strategy is currently a normal part of your work? (0—not at all to 5—somewhat to 10—completely)		6.91 (3.02)
Do you feel that the AKT Strategy will become a normal part of your work? (0—not at all to 5—somewhat to 10—completely)		8.09 (2.60)
Detailed questions about the intervention
NPT construct Mean (SD)	Item	Scale of 1—strongly disagree to 5—strongly agree Mean (SD)	Not relevant^€, n
NPT construct Mean (SD)	Item	N = 46
Coherence (α = .70) Mean score: 3.58 (.65)	I can see how the AKT Strategy differs from usual ways of working	3.33 (.76)	6
	Staff in this organization have a shared understanding of the purpose of the AKT Strategy	3.24 (.93)	1
	I understand how the AKT Strategy affects the nature of my own work	3.59 (.95)	2
	I can see the potential value of the AKT Strategy for my work	4.04 (.64)	1
Cognitive participation (α = .69) Mean score: 4.06 (.39)	There are key people who drive the AKT Strategy forward and get others involved	3.95 (.78)	2
	I believe that participating in the AKT Strategy is a legitimate part of my role	3.95 (.68)	2
	I’m open to working with colleagues in new ways to use the AKT Strategy	4.09 (.63)	1
	I will continue to support the AKT Strategy	4.11 (.49)	1
Collective action (α = .73) Mean score: 3.43 (.82)	I can easily integrate the AKT Strategy into my existing work	3.55 (.79)	2
	The AKT Strategy does not disrupt working relationships	3.77 (.86)	2
	I have confidence in other people’s ability to use the AKT Strategy	3.49 (.73)	1
	Work is assigned to those with skills appropriate to the AKT Strategy	3.50 (.76)	2
	Sufficient training is provided to enable staff to implement the AKT Strategy	3.02 (1.15)	2
	Sufficient resources are available to support the AKT Strategy	3.09 (1.04)	3
	Management adequately supports the AKT Strategy	3.60 (.84)	1
Reflexive monitoring (α = .84) Mean score: 3.75 (.38)	The staff agree that the AKT Strategy is worthwhile	3.51 (.66)	1
	I value the effects that the AKT Strategy has had on my work	3.81 (.59)	3
	Feedback about the AKT Strategy can be used to improve it in the future	3.96 (.56)	1
	I can modify how I work with the AKT Strategy	3.70 (.59)	2

Throughout the survey and interview questions, we refer to the EnAKT LKD intervention as the AKT Strategy, which is the name of the initiative used by the regional CKD programs.

€

For some questions, participants could indicate “not relevant to my role,” “not relevant at this stage” or “not relevant to EnAKT LKD.”

Interview Results

From the interview data, we developed overarching themes representing implementation challenges and opportunities as expressed by the health care professionals tasked with integrating the EnAKT LKD intervention activities into their daily work. We have summarized these 5 themes in Table 4, which includes a summary, an example, a representative quote, and suggestions for intervention improvement. Briefly, respondents indicated that the intervention’s complexity and breadth led some respondents to perceive it as being unfocused and lacking specificity, which some felt made it challenging to implement. A lack of clarity about the roles and responsibilities of each health care professional to engage with and deliver the different intervention components was noted by some participants. Several respondents reported feeling that there were not enough resources (time, funding, and staff) to be able to implement the EnAKT LKD intervention components. This was exacerbated by the COVID-19 pandemic and all participants reported this. Respondents identified that staff at CKD programs, especially those not involved in implementing the EnAKT LKD intervention, may not have enough transplant knowledge and experience to be comfortable having focused transplant discussions where they could advocate for the benefits of transplant over other modalities for treating kidney failure. This may have hindered buy-in to the EnAKT LKD intervention and transplant more broadly. The intervention did provide an opportunity for relationship building within their CKD program and with their corresponding transplant center, which was highly valued by respondents and was felt to lead to better patient outcomes.

Table 4.

Summary, Example, Representative Quote, and Next Steps for the 5 Themes Obtained From Interviews.

Theme 1: Considerable depth and breadth of the intervention
Summary	Example	Representative quote	Next steps
• The intervention’s breadth led some respondents to perceive it to be unfocused and lacking specificity, which some felt made it challenging to implement • Participants indicated the implementation of the intervention components was fragmented and delivered differently across units within a CKD program, possibly because not enough work was done to ensure all program areas were equally engaged, resulting in a perception of varying levels of awareness and buy-in	• The intervention education toolkits provided several types of educational materials that could be used by programs. Respondents reported that this made the intervention seem complex	• “. . . it took me a while to get a sense of what this project was all about, what is it exactly that they wanted us to do and how do I transition”	• More focused intervention and clear instructions to all staff may be needed in future iterations
Theme 2: Remaining areas of uncertainty
Summary	Example	Representative quote	Next steps
• A lack of clarity about the roles and responsibilities of each health care professional to engage with and deliver the intervention components was noted by some participants	• When referencing the education component, some respondents were not sure whether to use existing transplant education materials or new EnAKT LKD educational materials	• When a respondent was asked about how they supported the implementation of the Transplant Ambassador Program, they indicated that: “really, in my role, I feel greener than green and not totally sure what my role is”	• More role clarity may be needed to further guide health professionals toward new responsibilities and workflow changes
Theme 3: Resources
Summary	Example	Representative quote	Next steps
• Several respondents reported that there were not enough resources (time, funding, and staff) to implement the intervention. This was exacerbated by the COVID-19 pandemic and all participants reported this • Some respondents indicated that because this work was not tied to funding reimbursement, they did not see the implementation as a necessary responsibility • CKD programs that allocated a staff champion felt it helped to balance the intervention responsibilities and supported its implementation	• Respondents reported that responsibilities associated with new program initiatives often were tasked to CKD program managers: “Every time another . . .initiative rolls out—there’s not a whole lot that has decreased . . . it’s always the same managers”. • Staff turnover and the need to continually reorient new staff to the intervention was a challenge	• “We’re dealing with outbreaks. We’re dealing with cleaning in between patients getting dialysis and going out. We don’t have the staff to do that basically. That comes down to that with all the sick calls and everything. So this priority is not right now. Not because it’s not important, it’s just because it’s not feasible”	• Participants provided suggestions of resources needed to facilitate implementation: eg, dedicated personnel to support transplant education, transplant coordinator to support transplant work ups and data collection, financial support to host patient and staff education events, and clinic space to integrate TAP ambassadors
Theme 4: Buy-in
Summary	Example	Representative quote	Next steps
• Respondents identified that staff at CKD programs, especially those not involved in implementing the EnAKT LKD intervention, may not have enough transplant knowledge to be comfortable having transplant discussions where they could advocate for the benefits of transplant over other modalities. This may have hindered buy-in to the EnAKT LKD intervention and transplant more broadly • Respondents felt that some measures in the quarterly performance reports were not under their direct control or were too hard to achieve (eg, number of transplants), leading to frustration with the intervention which may have hindered buy-in and normalization of intervention activities	• There were challenges in the accuracy and timeliness of the quarterly performance reports and transplant eligibility lists, which led to participants losing confidence in their usefulness which potentially affected overall intervention buy-in	• Some staff did not think they should be promoting transplant as the best treatment option and preferred to present options in a more neutral way: “Some of them . . . felt that we were—it was too aggressive to being sharing that the option of living transplant, that we were offering a bias that it’s the best treatment . . . They thought that we should be providing all information in a neutral way, that you can do home hemo, home PD, that you can go to in-centre, you can be on a waitlist for a transplant, whether it’s deceased or living, but that they all get shared neutrally, equally, that one’s not better than the other, and that was such a sticky point for them.”	• Respondents reported that collaborative events held early in the EnAKT LKD intervention events re-ignited their motivation to focus on improving access to kidney transplant, suggesting that “booster” sessions may be needed to maintain momentum
Theme 5: Opportunities for relationship building
Summary	Example	Representative quote	Next steps
• Respondents valued strong relationships that were cultivated through the EnAKT LKD intervention, specifically within their CKD program, across other CKD programs and with their corresponding transplant center	• Respondents felt that a strong relationship with the transplant center led to better patient outcomes, such as a more efficient transplant evaluation process • Respondents were inspired by the connections they made during collaborative events to learn about the EnAKT LKD intervention, share implementation challenges and identify solutions	• “I think hearing about other people’s experiences, it shows us different options that are working for people and then we do a little bit of maybe that would work here, or maybe that wouldn’t and so on”	• Continue to foster this emergent community of practice

Note. Each theme draws upon 1 or more normalization process theory constructs.

Fidelity

We examined whether each of the 4 components of the intervention were implemented as planned.

Intervention component 1: local QI teams and administrative needs

All intervention-group CKD programs established a local QI team and created a team charter that was updated yearly. An average of 5 CKD program QI teams had a team member with formal QI training throughout the trial period. The provincial core operations group hosted regular collaborative calls for QI teams, for which attendance was moderate. For example, in 2019, there were 9 calls hosted, with a median attendance of 8 of the 13 CKD programs. With the onset of the COVID-19 pandemic, only 3 calls were hosted in 2020, with a median attendance of 3 of the 13 CKD programs. During the trial, the provincial core operations group conducted 2 transplant performance meetings and 1 check-in call with each CKD program, and the provincial medical lead conducted at least 1 in-person site visit at 7 of 13 programs for a total of 9 visits. The provincial core operations group met over 100 times. Table 5 describes how often the QI team meetings occurred before and during the pandemic. For example, while 60% of respondents indicated they always or often attended QI meetings before the pandemic, this decreased to 13% during the first pandemic wave.

Table 5.

Survey Results Examining the Frequency of Planned, Held and Attended Quality Improvement (QI) Team Meetings Prior to the COVID-19 Pandemic and During the COVID-19 Pandemic.^¥

	Prior to COVID-19 pandemic (prior to March 2020)*	During the first wave of the COVID-19 pandemic (March-May 2020)*
Planned QI team meetings	Monthly: 8 (26.7) Quarterly: 13(43.3) Bi-annually: 0 Annually: 0 Other: 2 (6.7) Not at all: 0 Unsure/Missing: 7 (23.3)	Monthly: 2 (6.7) Quarterly: 9 (30.0) Bi-annually: 0 Annually: 0 Other: 2 (6.7) Not at all: 10 (33.3) Unsure/Missing: 7 (23.3)
QI team meetings held	Monthly: 6 (20.0) Quarterly: 11 (36.6) Bi-annually: 2 (6.7) Annually: 0 Other: 2 (6.7) Not at all: 0 Unsure/Missing: 9 (30.0)	Monthly: 2 (6.7) Quarterly: 2 (6.7) Bi-annually: 0 Annually: 1 (3.3) Other: 1 (3.3) Not at all: 12 (40.0) Unsure/Missing: 12 (40.0)
Attendance at QI team meetings	Always: 9 (30.0) Often: 9 (30.0) Sometimes: 6 (20.0) Rarely: 2 (6.7) Never: 0 Not part of the QI team at that time: 3 (10.0) Missing: 1 (3.3)	Always: 3 (10.0) Often: 1 (3.3) Sometimes: 1 (3.3) Rarely: 1 (3.3) Never: 3 (10.0) Not part of the QI team at that time: 2 (6.7) Missing: 19 (63.3)

There were a total of 30 survey respondents (ie, individuals on the QI team).

Data are presented as n (%).

Table 6 provides survey results for participation in building a QI team, including participation and perceived helpfulness of creating a team charter, mapping the transplant referral process, and mapping the transplant education pathway. Participation in all 3 activities was ≥ 60%, and perceived helpfulness was ≥ 80%. However, participation in QI team activities was impacted by the pandemic, with an average of 1 (range = 0-6, SD = 1.3) Plan-Do-Study-Act cycles completed per quarter before the pandemic, which decreased to 0.1 per quarter (range = 0-1, SD = 0.34) during the first wave of the pandemic.

Table 6.

Survey Results for Participation in Building a Quality Improvement (QI) Team, Including the Participation and Perceived Helpfulness of Creating a Team Charter, Mapping the Transplant Referral Process, and Mapping the Transplant Education Pathway.

Activity	Participated in the activity (yes)^¥ n (%)	Perceived the activity to be helpful (yes)^† n (%)
Creating a QI team charter with specific aims	21 (70.0)	17 (81.0)
Mapping the transplant referral process	18 (60.0)	17 (94.4)
Mapping the transplant education pathway	19 (63.3)	18 (94.7)

There were a total of 30 survey respondents (ie, individuals on the quality improvement team). Response options included: yes, I participated; no, unsure/missing; or not part of the quality improvement team at the time.

†

The denominator was based on how many individuals responded to “yes the participated in the activity.”

Intervention component 2: transplant education and resources

In the fiscal year prior to the intervention launch, 297 patients with advanced CKD received greater than 30 minutes of transplant education. This increased after the EnAKT LKD intervention launched to 908 patients between April 2018 and March 2019 and 1452 patients from April 2019 to March 2020. With the onset of the COVID-19 pandemic, fewer patients received transplant education (eg, 1026 patients from April 2020 to March 2021). All programs had initial site visits by the education task group, and 8 programs had follow-up meetings (either in person or virtually). As shown in Table 7, 27% to 63% of QI team members participated in transplant educational activities offered and 75% to 100% of individuals who used them perceived them to be helpful. Educational resources were used by 8% to 58% of health care professionals, and their perceived helpfulness ranged from 75% to 100% (Table 8). As shown in Table 9, the use of all educational resources declined during the first wave of the pandemic, with resource use increasing later; however, usage never returned to pre-pandemic levels.

Table 7.

Survey Results for Examining Participation in Educational Activities and Perceived Helpfulness of Educational Activities for Quality Improvement Teams.^*

Educational activities for the quality improvement teams
Activity	Participation in activity (yes)^¥ n (%)	Perceived the activity to be helpful (yes)^† n (%)
Explore Transplant Ontario workshop in Fall 2017	19 (63.3)	17 (89.5)
Presentation on “How to Find a Living Kidney Donor?”	10 (33.3)	10 (100.0)
Teleconference with Dr Amy Waterman aimed at enhancing transplant education at the CKD program	16 (53.3)	15 (93.8)
In-person meeting with Dr Amy Waterman aimed at enhancing transplant education at your CKD program	8 (26.7)	6 (75.0)
Teleconference with any of the education support personnel	17 (56.7)	17 (100.0)
In-person meeting with any of the education support personnel	18 (60.0)	18 (100.0)

Data are presented as n (%).

†

The denominator was based on how many individuals responded to “yes the participated in the activity.”

Tables 8.

Survey Results for Examining Utilization of Educational Resources and Perceived Helpfulness of Educational Support for Quality Improvement Teams and Educational Resources for Other Health Care Professionals.^*

	Educational resources for quality improvement teams		Educational resources for health care professionals
Resource	Utilization of resource^€	Perceived the resource to be helpful (yes)^µ	Utilization of resource^¶	Perceived the resource to be helpful (yes)^µ
Provider Resource Hub (transplant resources) for health care providers on the Ontario Renal Network website	17 (56.7)	15 (88.2)	8 (30.8)	7 (87.5)
Infographic “Would your Patient Benefit from a Kidney Transplant Referral”	20 (66.7)	20 (100)	12 (46.2)	11 (91.7)
Infographic “How to Talk about Living Kidney Donation”	22 (73.3)	20 (90.9)	14 (53.8)	14 (100)
Infographic “Becoming a living kidney donor in Ontario”	20 (66.7)	19 (95)	10 (38.5)	10 (100)
Infographic “Living Kidney Donation outside Canada”	12 (40)	12 (100)	4 (15.4)	3 (75.0)
At least 1 video from the Core Transplant Curriculum Webinars (eg, risks and benefits of kidney transplantation compared with other renal replacement modalities)	12 (40)	12 (100)	6 (23.1)	6 (100)
Dialysis versus kidney transplant estimated survival in Ontario risk calculator website or app	5 (16.7)	5 (100)	2 (7.7)	2 (100)
At least 1 Explore Transplant Ontario video	23 (76.7)	18 (78.2)	12 (46.2)	10 (83.3)
Read the Explore Transplant Ontario brochures	23 (76.7)	19 (82.6)	15 (57.7)	15 (100)
Provided any of the Explore Transplant Ontario videos and/or brochures to patients or family members	19 (63.3)	Not applicable	14 (53.8)	Not applicable

Data presented as n (%)

€

There were a total of 30 survey respondents (ie, individuals on the quality improvement team). Response options included: yes, I participated; no, or unsure/missing.

The denominator was based on how many individuals responded to “yes they participated in the activity.”

There were a total of 26 survey respondents (ie, other health care professionals). Response options included: yes, I participated; no, or unsure/missing.

Table 9.

Educational Resource Usage Presented Using a Likert Scale, With 1 Indicating the Resource was Never Used and 5 Indicating the Resource was Always Used.^¥.

Question		Prior to the COVID-19 pandemic Mean ± SD (no. of respondents)	During the first wave of the COVID-19 pandemic (March 2020-May 2020) Mean ± SD (no. of respondents)	At the time of the survey (December 2020-June 2021) Mean ± SD (no. of respondents)
How often did you use the infographic: How to talk about living kidney donation?	Quality improvement team	3.23 ± 0.92 (22)	2.00 ± 1.07 (22)	2.77 ± 1.07 (22)
	Other health care professional	3.14 ± 0.77 (14)	2.14 ± 0.86 (14)	2.77 ± 0.83 (13)
How often did you use the infographic: Becoming a living kidney donor in Ontario?	Quality improvement team	3.25 ± 0.85 (20)	2.00 ± 1.08 (20)	2.65 ± 0.62 (20)
	Other health care professional	3.50 ± 0.53 (10)	2.70 ± 1.16 (10)	3.10 ± 0.88 (10)
How often did you use the infographic: living kidney donation outside Canada?	Quality improvement team	3.25 ± 0.62 (12)	2.25 ± 1.29 (12)	2.91 ± 0.94 (11)
	Other health care professional	2.75 ± 0.50 (4)	2.25 ± 0.96 (4)	2.75 ± 0.50 (4)
How often did you use the dialysis versus kidney transplant estimated survival in Ontario risk calculator website or app?	Quality improvement team	3.80 ± 0.45 (5)	1.80 ± 1.10 (5)	3.40 ± 0.89 (5)
	Other health care professional	3.00 ± 0.00 (2)	1.50 ± 0.71 (2)	2.50 ± 0.71 (2)
How often did you give any of the Explore Transplant Ontario videos and/or brochures to patients or family members?	Quality improvement team	4.16 ± 0.69 (19)	2.53 ± 1.31 (19)	3.63 ± 1.07 (19)
	Other health care professional	3.79 ± 0.80 (14)	2.21 ± 1.12 (14)	2.71 ± 1.07 (14)

Denominators are restricted to quality improvement team members and health care professionals who indicated that they used the resource.

Support from prior transplant recipients and living donors

Although it took some time to establish the patient-led TAP (https://transplantambassadors.ca/) in each CKD program, in 2018, a total of 1679 meaningful interactions occurred across all programs between transplant ambassadors and patients with advanced CKD, their family members or potential living kidney donors. Interactions decreased when hospital volunteer programs paused during the pandemic. At that time, ambassador interactions moved to a virtual environment. In 2020, there were 411 interactions, and in 2021, 452 interactions. In 2018, there was a median of 6 ambassadors per CKD program (range = 1-9), a value that decreased to 4 (range = 1-11) during the pandemic. In 2018, only 1 CKD program did not have a TAP lead, in all subsequent years (ie, 2019-2021), 2 CKD programs had no TAP lead.

Program-level performance reports and oversight

The CKD programs were to receive performance reports each quarter (ie, approximately 16 times during the trial period). However, the provincial renal agency only distributed 7 performance reports and 4 transplant referral eligibility reports. The work to create the reports was more than initially anticipated, which included the time needed to reconcile data concerns.

Chronic kidney disease program QI teams reviewed an average of 4.1 (SD = 2.7) of the 7 program-level performance reports and 1.6 (SD = 1.7) of the transplant referral eligibility reports. These reports were usually not distributed beyond the QI team; only 19% of surveyed health care professionals outside the team indicated they reviewed at least one performance report (Table 10). Using a 5-point Likert scale, health care professionals were mainly neutral regarding the ease of interpreting these reports (mean = 3.4) and their perceived usefulness (mean = 3.4) (Table 11).

Table 10.

Dissemination of Program-Level Performance Reports.

Fidelity of receipt and enactment	Health care professional reported participation and perceived helpfulness Yes, n (%)
Dissemination of the program-level performance reports
Shared information from the program-level performance reports with other members of your CKD program (besides QI team members). This question was only given to the quality Improvement team lead (n = 8)	6 (75)
Outside of your QI team, other health care professionals reviewed information from the program-level performance reports. This question was only given to quality improvement team members other than the QI team lead (n = 20)	5 (25)
Reviewed at least one of the program-level performance reports. This question was only given to health care professionals who were not a member of the quality improvement team (n = 26)	5 (19)
Found the program-level performance reports helpful.This question was only given to health care professionals who were not a member of the quality improvement team and who reviewed the report (n = 5)	4 (80)
Dissemination of the transplant referral eligibility reports
Shared information from the transplant referral eligible patient reports with other members of the CKD program (besides QI team members). This question was only given to the quality improvement team lead (n = 8)	2 (25)
Outside of your quality improvement team, other health care professionals reviewed the transplant referral eligible patient reports. This question was only given to quality improvement team members other than the QI team lead (n = 19)	4 (21)
Reviewed at least one of the transplant referral eligible patient reports. This question was only given to health care professionals who were not a member of the quality improvement team (n = 26)	4 (15)
Found the transplant referral eligible patient reports helpful. This question was only given to health care professionals who were not a member of the quality improvement team who reviewed the report (n = 4)	3 (75)

Table 11.

Program-Level Performance Reports Usage.

Quarterly performance reports	Mean ± SD	Range/scale
Number of quality improvement snapshot reports reviewed by the quality improvement team (n = 30)	4.1 ± 2.7	0-7; 0 represents 0 reports were reviewed and 7 represents all reports were reviewed
Quality improvement snapshot reports were easy to interpret (n = 21)	3.4 ± 0.9	Scale 1—strongly disagree to 5—strongly agree Respondents indicating they did not review any of the quality improvement snapshot reports did not respond to these follow-up questions
Quality improvement snapshot reports were useful for understanding how many patients in our CKD program took steps to receive a kidney transplant (n = 20)	3.5 ± 0.9
Quality improvement snapshot reports contributed to more patients in our CKD program being assessed for transplant referral eligibility (n = 21)	3.4 ± 0.8
Quality improvement snapshot reports contributed to more patients in our CKD program being referred to transplant centers (n = 21)	3.3 ± 0.7
Quality improvement snapshot reports contributed to more individuals contacting a transplant center about living kidney donation for a patient in our CKD program (n = 21)	3.1 ± 0.8
My CKD program is likely to continue using the quality improvement snapshot reports to track our progress in improving patient access to kidney transplant and living donation (n = 21)	3.5 ± 0.6
How often were quality improvement snapshot reports shared outside of the QI team (QI team leads) (n = 6). QI team leads only responded to this question if they indicated that reports were shared outside of the QI team	3.5 ± 0.5	Scale 1—rarely to 4—always
How often were quality improvement snapshot reports shared outside of the QI team (QI team members) (n = 3). QI team members only responded to this question if they indicated that reports were shared outside of the QI team	2.3 ± 0.6	Scale 1—rarely to 4—always
Transplant referral eligible patient report
Transplant referral eligible patient reports reviewed by QI team (n = 27)	1.6 ± 1.7	Range 0-4 (maximum number of transplant referral eligibility reports that could be received is 4)
Transplant referral eligible patient reports were easy to interpret^¥ (n = 14)	3.7 ± 0.9	Scale 1—strongly disagree to 5—strongly agree
Transplant referral eligible patient reports were useful^¥ (n = 14)	3.6 ± 0.9
Transplant referral eligible patient reports resulted in more patients being assessed for transplant referral^¥ (n = 14)	3.4 ± 0.8
Transplant referral eligible patient reports resulted in more patients being referred to transplant centers^¥ (n = 14)	3.4 ± 0.8
My CKD program is likely to continue using the transplant referral eligible patient reports to improve patient access to kidney transplant and living donation^¥ (n = 14)	3.6 ± 0.7

Respondents indicating they did not review any of the transplant referral eligible patient reports did not respond to these follow-up questions.

Additional Fidelity Results

Supplemental Table 4 includes additional fidelity indicators grouped by intervention component. Supplemental Table 5 includes fidelity indicators not reported due to unavailable data, and Supplemental Table 6 includes fidelity indicators not reported due to poor utilization of the transplant referral eligible patient report.

Contamination

The online survey confirmed a lack of contamination (ie, programs in the usual care group did not appear to use the intervention components during the trial; data not shown).

Discussion

The EnAKT LKD trial found that a novel multicomponent intervention compared with usual care did not significantly increase the rate of patients completing steps toward receiving a kidney transplant. Alongside the trial, we conducted a process evaluation. We learned that the intervention engaged providers; however, there were implementation challenges and a need to better integrate some components into routine care.

We surveyed CKD program staff on the perceived barriers to accessing kidney transplants. Over time, the views expressed by staff in the intervention and usual care groups were consistent; however, both groups disagreed or felt neutral that the targeted barriers impeded transplant access. We acknowledge the limitations of our approach, which include asking health care providers to rate barriers experienced by patients and living donors (rather than asking patients and living donors directly), asking providers to recall barriers over a long time period, and asking health care providers who were most experienced and knowledgeable in transplant and likely experience fewer barriers to helping their patients get a transplant.

Survey respondents were those most directly involved in key transplant activities, such as providing education and coordinating transplant referrals at the CKD program. Overall, respondents did not perceive a lack of transplant knowledge as a barrier. Still, in interviews, QI team leads expressed their belief that frontline dialysis nurses needed more transplant knowledge and experience to help their patients navigate the transplant process. While QI teams were to work out how to deliver health professional education to those who needed it the most, in the future, we can better guide them on how to do this. Specifically, we suggest assessing barriers to transplant in frontline staff that may not routinely engage in transplant processes (ie, those who are not part of the QI team), but who spend significant time with patients throughout their kidney care journey during which education, guidance and navigation are provided, and to tailor future educational interventions according to their perceived barriers. Fidelity data supported that while some educational resources had moderate-to-high use, others, such as the core curriculum webinars, were not used by health professionals as often as intended.

It makes intuitive sense that CKD programs must regularly review their performance on key transplant metrics to effectively guide their improvement efforts. There is good evidence that audit and feedback is helpful.²⁶ We established data-sharing agreements that allowed CKD programs to receive summary performance reports on how well patients in their program were completing key steps toward receiving a transplant. However, survey data showed that QI teams, on average, only reviewed 4 of 7 reports. Reports were also rarely distributed beyond QI team membership. The implementation may be improved using best practices in audit and feedback and recommending specific actions for the report recipient.^26,27

The COVID-19 pandemic hindered intervention delivery and the response rate for this process evaluation. Many CKD program staff left their positions or deployed to other hospital areas. The monthly collaborative calls, QI team meetings, and patient education efforts were all paused. Patient transplant ambassadors could not enter the hospital, and their interactions shifted to virtual.

The document analysis showed we successfully established a QI infrastructure. All 13 CKD programs in the intervention group formed QI teams, each with a team charter. Members participated in monthly provincial collaborative calls and, before the pandemic, held regular team meetings. Surveys and interviews show we engaged providers with the intervention. Respondents indicated optimism about its ongoing integration and valued the new relationships they built to support local process improvement. The findings support an assertion that initiatives to improve access to kidney transplantation and living kidney donation are acceptable to providers as a new part of their work. However, the breadth and complexity of the intervention led survey respondents to report difficulty making sense of it, understanding what work was required to implement it, and how it differed from usual processes. They indicated the substantial scope led to challenges in operationalizing and executing intervention components. Participants wanted more training on intervention components and suggested more staff and resources would be helpful to support implementation. For example, the provincial renal agency asked CKD programs to form QI teams and engage in process improvement work, yet only provided 1 training day at the start of the trial. We asked each QI team to include a local team member with formal QI training. However, such a person was only available in some programs (average of 5 of 13 CKD programs throughout the trial). Given the high staff turnover experienced at CKD programs over the 4-year trial period, many providers implementing the intervention were initially absent and never received training on its components. Future efforts should better delineate sustainability plans and address the need for retraining with staff turnover.

In 2021, the provincial renal agency provided a small amount of funding for transplant coordination at each CKD program. While helpful, during interviews, the QI leads stated they needed more support to complete transplant referrals and navigate patients through the transplant process efficiently. Quality improvement leads also cited complex communication processes with transplant centers to move their patients throughout the transplant process. These are targets for future QI efforts.

Limitations of our study are worth noting. Approximately, 30% of CKD program staff did not respond, and 2 of the 13 intervention CKD programs did not participate. Respondents were experienced in their roles and were amongst the CKD program staff most engaged with intervention implementation. Providers who did not participate in our process evaluation were likely less familiar with or involved with intervention implementation. Finally, we did not survey or interview patients.

In conclusion, we established a QI culture and infrastructure in Ontario to improve access to kidney transplantation and living kidney donation. While the intervention in its tested form was insufficient to improve access, this process evaluation provides several suggestions for intervention improvement.

Supplemental Material

sj-docx-1-cjk-10.1177_20543581251323959 – Supplemental material for Process Evaluation Alongside a Cluster-Randomized Trial of a Multicomponent Intervention Designed to Improve Patient Access to Kidney Transplantation

Supplemental material, sj-docx-1-cjk-10.1177_20543581251323959 for Process Evaluation Alongside a Cluster-Randomized Trial of a Multicomponent Intervention Designed to Improve Patient Access to Kidney Transplantation by Seychelle Yohanna, Mackenzie Wilson, Kyla L. Naylor, Amit X. Garg, Jessica M. Sontrop, Istvan Mucsi, Dimitri Belenko, Stephanie N. Dixon, Peter G. Blake, Rebecca Cooper, Lori Elliott, Esti Heale, Sara Macanovic, Rachel Patzer, Amy D. Waterman, Darin Treleaven, Candace Coghlan, Marian Reich, Susan McKenzie and Justin Presseau in Canadian Journal of Kidney Health and Disease

Supplemental Material

sj-docx-2-cjk-10.1177_20543581251323959 – Supplemental material for Process Evaluation Alongside a Cluster-Randomized Trial of a Multicomponent Intervention Designed to Improve Patient Access to Kidney Transplantation

Supplemental material, sj-docx-2-cjk-10.1177_20543581251323959 for Process Evaluation Alongside a Cluster-Randomized Trial of a Multicomponent Intervention Designed to Improve Patient Access to Kidney Transplantation by Seychelle Yohanna, Mackenzie Wilson, Kyla L. Naylor, Amit X. Garg, Jessica M. Sontrop, Istvan Mucsi, Dimitri Belenko, Stephanie N. Dixon, Peter G. Blake, Rebecca Cooper, Lori Elliott, Esti Heale, Sara Macanovic, Rachel Patzer, Amy D. Waterman, Darin Treleaven, Candace Coghlan, Marian Reich, Susan McKenzie and Justin Presseau in Canadian Journal of Kidney Health and Disease

Footnotes

Declaration of Conflicting Interest

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: AXG is supported by the Dr Adam Linton Chair in Kidney Health Analytics, a Clinician Investigator Award from the Canadian Institutes of Health Research (CIHR), a Clinician Research Salary Award from the CIHR, and he has received partnership grant funding from Astellas Canada for research grants funded by the CIHR. SY received partnership grant funding from Astellas Canada for a research grant funded by the CIHR. SQM received partnership grant funding from Astellas Canada for a research grant funded by the CIHR. IM was supported by investigator-initiated grants from CIHR, the Kidney Foundation of Canada, the Canadian Donation and Transplant Research Program, the Health Canada Health Policy Contribution Program, and the Mount Sinai Hospital-University Health Network Academic Medical Organization Innovation Funding; he also received an unrestricted education grant from Astellas Canada to adapt the Explore Transplant education program to the Ontario setting and from Paladin Labs. Inc., Canada to develop a self-management support website.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors thank the Executive Committee of Can-SOLVE CKD (Canadians Seeking Solutions and Innovations to Overcome Chronic Kidney Disease), a patient-oriented research network to transform the care of people affected by kidney disease (see ). They acknowledge the work of the Provincial Access to Kidney Transplantation and Living Donation Priority Panel and the Ontario Renal Network—Trillium Gift of Life Partnership. Astellas Pharma Canada Inc. provided partial funding to mass produce the educational program Explore Transplant Ontario and for the Transplant Ambassador Program. The analyses, conclusions, opinions, and statements reported in this paper are those solely of the authors and do not reflect those of the organizations or funding sources listed above; no endorsement should be intended or should be inferred.

Ethics Approval

We obtained ethics approval from the Ottawa Health Sciences Research Ethics Board, and written informed consent was obtained from all participants (REB 20200426-01H).

Consent to Participate

Written informed consent was obtained from all participants (REB 20200426-01H).

Consent for Publication

All authors consent to the publication of this study.

Availability of Data and Materials

The data for this study is held securely at the Ottawa Hospital Research Institute and are available upon reasonable request.

ORCID iDs

Seychelle Yohanna

Mackenzie Wilson

Kyla L. Naylor

Amit X. Garg

Jessica M. Sontrop

Istvan Mucsi

Stephanie N. Dixon

Supplemental Material

Supplemental material for this article is available online.

References

Tonelli

Wiebe

Knoll

, et al. Systematic review: kidney transplantation compared with dialysis in clinically relevant outcomes. Am J Transplant. 2011;11:2093-2109. doi:10.1111/j.1600-6143.2011.03686.x.

Rana

Gruessner

Agopian

, et al. Survival benefit of solid-organ transplant in the United States. JAMA Surg. 2015;150:252-259. doi:10.1001/jamasurg.2014.2038.

Ortiz

Aronen

Koskinen

, et al. Health-related quality of life after kidney transplantation: who benefits the most? Transpl Int. 2014;27:1143-1151. doi:10.1111/tri.12394.

Klarenbach

Manns

. Economic evaluation of dialysis therapies. Semin Nephrol. 2009;29:524-532. doi:10.1016/j.semnephrol.2009.06.009

Canadian Institute for Health Information. Organ Replacement in Canada: CORR Annual Statistics. Ottawa, ON: CIHI; 2020.

Horvat

Shariff

Garg

. Donor nephrectomy outcomes research, n global trends in the rates of living kidney donation. Kidney Int. 2009;75:1088-1098. doi:10.1038/ki.2009.20

Getchell

McKenzie

Sontrop

, et al. Increasing the rate of living donor kidney transplantation in Ontario: donor- and recipient-identified barriers and solutions. Can J Kidney Health Dis. 2017;4:2054358117698666. doi:10.1177/2054358117698666.

Horton

Nugus

Fortin

Landsberg

Cantarovich

Sandal

. Health system barriers and facilitators to living donor kidney transplantation: a qualitative case study in British Columbia. CMAJ Open. 2022;10:E348-E356. doi:10.9778/cmajo.20210049.

Kazley

Simpson

Chavin

Baliga

. Barriers facing patients referred for kidney transplant cause loss to follow-up. Kidney Int. 2012;82(9):1018-1023. doi:10.1038/ki.2012.255.

10.

Garg

Yohanna

Naylor

, et al. Effect of a novel multicomponent intervention to improve patient access to kidney transplant and living kidney donation: the EnAKT LKD cluster randomized clinical trial. JAMA Intern Med. 2023;183(12):1366-1375 doi:10.1001/jamainternmed.2023.5802.

11.

Yohanna

Naylor

Mucsi

, et al. A quality improvement intervention to enhance access to kidney transplantation and living kidney donation (EnAKT LKD) in patients with chronic kidney disease: clinical research protocol of a cluster-randomized clinical trial. Can J Kidney Health Dis. 2021;8:2054358121997266. doi:10.1177/2054358121997266.

12.

Moore

Audrey

Barker

, et al. Process evaluation of complex interventions: medical Research Council guidance. BMJ. 2015;350:h1258. doi:10.1136/bmj.h1258.

13.

Yohanna

Wilson

Naylor

, et al. Protocol for a process evaluation of the quality improvement intervention to enhance access to kidney transplantation and living kidney donation (EnAKT LKD) cluster-randomized clinical trial. Can J Kidney Health Dis. 2022;9:20543581221084502. doi:10.1177/20543581221084502.

14.

Eysenbach

. Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res. 2004;6:e34. doi:10.2196/jmir.6.3.e34.

15.

O’Brien

Harris

Beckman

Reed

Cook

. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. 2014;89(9):1245-1251. doi:10.1097/acm.0000000000000388.

16.

May

Cummings

Girling

, et al. Using Normalization Process Theory in feasibility studies and process evaluations of complex healthcare interventions: a systematic review. Implement Sci. 2018;13:80. doi:10.1186/s13012-018-0758-1.

17.

May

Finch

. Implementing, embedding, and integrating practices: an outline of normalization process theory. Sociology. 2009;43:535-554. doi:10.1177/0038038509103208.

18.

May

. A rational model for assessing and evaluating complex interventions in health care. BMC Health Serv Res. 2006;6:86. doi:10.1186/1472-6963-6-86.

19.

May

. Agency and implementation: understanding the embedding of healthcare innovations in practice. Soc Sci Med. 2013;78:26-33. doi:10.1016/j.socscimed.2012.11.021.

20.

May

Mair

Finch

, et al. Development of a theory of implementation and integration: Normalization Process Theory. Implement Sci. 2009;4:29. doi:10.1186/1748-5908-4-29.

21.

Hovden

EAS

Skudutyte-Rysstad

. Using Theoretical Domains Framework (TDF) to understand implementation of a pragmatic clinical trial in Public Dental Service in Norway. BMC Health Serv Res. 2021;21:698. doi:10.1186/s12913-021-06590-2.

22.

Atkins

Francis

Islam

, et al. A guide to using the Theoretical Domains Framework of behaviour change to investigate implementation problems. Implement Sci. 2017;12:77. doi:10.1186/s13012-017-0605-9.

23.

Hsieh

Shannon

. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277-1288. doi:10.1177/1049732305276687.

24.

Finch

Girling

May

, et al. Improving the normalization of complex interventions: part 2—validation of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT). BMC Med Res Methodol. 2018;18:135. doi:10.1186/s12874-018-0591-x.

25.

Rapley

Girling

Mair

, et al. Improving the normalization of complex interventions: part 1—development of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT). BMC Med Res Methodol. 2018;18:133. doi:10.1186/s12874-018-0590-y.

26.

Ivers

Jamtvedt

Flottorp

, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;2012(6):CD000259. doi:10.1002/14651858.CD000259.pub3.

27.

Brehaut

Colquhoun

Eva

, et al. Practice feedback interventions: 15 suggestions for optimizing effectiveness. Ann Intern Med. 2016;164:435-441. doi:10.7326/M15-2248.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.78 MB

1.80 MB