Implementation of a Novel Patient-Reported Outcome Measure for the Assessment of Symptoms in Children Living With Chronic Kidney Disease (PRO-Kid)—Program Report

Abstract

Purpose of the Program:

Children with chronic kidney disease (CKD) experience significant physical and psychological symptoms, necessitating patient-reported outcome (PRO) measurement tools to quantify symptoms, and to improve communication between children with CKD and their health care providers. This study aimed to implement the novel PRO-Kid tool into pediatric CKD and dialysis programs in Canada.

Sources of Information:

The PRO-Kid tool is a novel 14-item questionnaire that was developed and validated by our research team, in partnership with patients living with childhood-onset CKD and caregivers for children with G3-G5 CKD, including dialysis. PRO-Kid measures both the frequency and impact of CKD symptoms in children ages eight to 18 years. It showed strong internal consistency and validity, as demonstrated by a Cronbach alpha of .83 (95% confidence interval [CI], 0.78-0.88) for the frequency scale, and .84 (95% CI, 0.80-0.89) on the impact scale.

Methods:

The PRO-Kid tool is being implemented in seven Pediatric Nephrology Programs across Canada. Guided by the Consolidated Framework for Implementation Research (CFIR), organizational readiness at each of the sites was assessed via surveys and focus groups. To date, the PRO-Kid tool (eight- to 18-year-old version) and site-specific toolkits have been implemented at three sites that were ready to launch (HSC, Winnipeg; BC Children’s Hospital, Vancouver; and McMaster Children’s Hospital, Hamilton). Implementation outcomes, such as the number of patients reached and patient and provider satisfaction, were evaluated using audits, surveys, and focus groups guided by the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework.

Key Findings:

The organizational readiness assessment identified different electronic medical records and preferences for paper-based data collection methods across sites as well as the need to translate the PRO-Kid tool into other languages. Preliminary evaluation revealed that PRO-Kid is easy to use and is satisfactory to both patients and health care providers. Local champions were identified as key facilitators of implementation efforts. PRO-Kid can help improve communication between children with CKD and their health care providers in identifying symptoms that may not otherwise come up (be disclosed) during clinical encounters. However, language and low reading levels are barriers for some children, and unavailability of clinical staff can limit the use of the tool at some sites.

Implications:

Local teams will be required to play crucial roles in integrating the use of the tool in clinical settings. To ensure the tool can be applied across diverse populations, the validation of PRO-Kid tool two to four years old, five to seven years old, and French versions are ongoing. The availability of different versions of the tool will ensure equitable access and promote sustainability of the tool.

Purpose of the Program

The impacts of chronic kidney disease (CKD) and symptom burden on children living with this condition are well established. Compared with healthy children, children living with CKD experience significant physical, psychological, social, and functional symptoms,^1-3 along with lower self-reported quality of life. ⁴ Despite efforts to address CKD-related symptoms, follow-up can be difficult in the context of the subjectivity and complexity of symptoms, and the lack of objective measures to quantify them. Patient-reported outcomes (PROs) help to quantify the patients’ perspectives of their health, which can be incorporated into clinical care and used for research. ⁵ Moreover, it is now widely accepted that CKD-related symptoms, along with biological parameters, should be evaluated equally in guiding the timing of initiation of dialysis or kidney transplant,^6,7 making PROs even more important for longitudinal and objective assessment of symptoms.

The need and desire for improved methods to monitor CKD symptoms in children were identified by children with CKD and their families involved in our research program (Figure 1) as well as in previous studies. ⁸ This need was supported through our patient-oriented research network, Canadians Seeking Solutions and Innovations to Overcome Chronic Kidney Disease (Can-SOLVE CKD), in collaboration with clinicians within the network. ⁹ This led to the development ¹⁰ and validation ¹¹ of the PRO-Kid tool (Patient-Reported Outcome measure for the assessment of symptom burden in pediatric Chronic Kidney Disease).

Figure 1.

PRO-Kid questionnaire (8-18 years).

During this long journey leading to implementation, engagement and collaboration with CKD patients and parent partners were paramount in putting patient voices at the center of the development and implementation of this tool. Ke Fan Bei (K.F.B.), patient partner, and Kelly Loverock (K.L.), parent partner, both agree that in addition to the medical burden (appointments, medication, etc) of CKD, the disease itself significantly affects quality of life and limits life participation (LP), particularly in terms of engagement within the community. The treatment and management of CKD dominate both the personal and social aspects of a child’s life, making the disease a central part of their identity. It influences their interactions with peers and others in the community and plays a significant role in shaping their present experiences and future opportunities.

“From my experience with CKD, it becomes the starting point and end-point of my identity and how I see myself. When friends and peers share food, dietary restrictions become the barrier of a shared connection. Overnight or multi-day school trips become an impossibility with my dialysis regime and the need to take multiple types of medication. Holidays and birthdays are not celebrated with close family and friends but the health care staff. As I grew up in the hospital environment, managing CKD as a young teenager impacted my identity and my views on life possibilities. I start to wonder what is possible for me to do which affected my outlook on life and indirectly—my enthusiasm in my CKD care (K.F.B., patient partner).”

K.L. also emphasized aspects related to communication barriers children may have when communicating their symptoms:

“Children and young adults with CKD sometimes struggle to communicate their symptoms to their healthcare providers. This tool is intended to provide them with a way to consistently and more easily communicate what they’re feeling and experiencing to help inform their care and treatment options (K.L., parent partner).”

This study aims to fill a crucial gap in clinical practice by implementing the newly developed and validated PRO-Kid tool, which can be used by children and adolescents with CKD to report their symptoms. The PRO-Kid tool has the potential to empower them to be active participants in the decision-making processes related to their care and highlight the symptoms that impact their own life, thereby giving them a sense of control over the disease.

Sources of Information

Briefly, the research team, including patient partners, first conducted a thorough literature review and drafted the initial symptom questionnaire. Using cognitive interviews with children aged eight to 18 years living with CKD and their caregivers, this preliminary tool was refined and adopted as the PRO-Kid tool. ¹⁰ The PRO-Kid tool was then validated in 100 children across Canada. ¹¹ The outcome of the validation study revealed the PRO-Kid tool has a strong internal consistency and validity (Cronbach alpha of .83 (95% confidence interval [CI], 0.78-0.88) for the frequency scale, and .84 (95% CI, 0.80-0.89) on the impact scale).

“This study is designed around the need to better understand the non-medical impact of CKD, including quality of life and participation in social activities with friends/family, leisure activities and self-care. A focus on the pediatric patient as a whole is important for successful management of CKD (K.F.B., patient-partner; Figure 1).”

“The tool assesses both frequency and impact of symptoms in children and tracks symptom progression or improvement. I think it is a fantastic way to empower children to express themselves and be more forthcoming about their experiences, especially in cases where blood work or a physical exam doesn’t necessarily highlight an issue or anomaly (e.g., maybe they’re really fatigued but blood work doesn’t necessarily indicate a problem). It also helps track symptom burden over time and helps children articulate how they’re feeling that can impact their overall quality of life. My role as a patient (parent) partner has been informed by my experiences as a parent to a child with CKD. I’m also a communications professional and I bring this perspective to pretty much everything I do, including my role on this project (K.L., parent partner).”

To support research teams in their implementation efforts, Can-SOLVE CKD created a Pathway to Implementation guide. On the Can-SOLVE CKD Implementation Pathway, this research program is currently in Step 5—Implementation, Monitoring and Evaluation. ¹² This article describes the implementation process of the PRO-Kid tool in pediatric nephrology clinics across Canada (Figure 2) and shares reflections on the team’s journey in this knowledge mobilization effort.

Figure 2.

Patient partners involved in the PRO-Kid project emphasize how important symptom management is in the context of pediatric kidney care.

Methods

The implementation of the PRO-Kid tool includes seven pediatric nephrology sites across Canada (listed geographically from West to East), namely, the BC Children’s Hospital (Vancouver), the Alberta Children’s Hospital (Calgary), the Health Sciences Centre (Winnipeg), the McMaster Children’s Hospital (Hamilton), the Hospital for Sick Children (Toronto), the Montreal Children’s Hospital (Montreal), and the Centre hospitalier universitaire Sainte-Justine (Montreal).

Organizational Readiness and Implementation

Using the Organizational Readiness Theory ¹³ as an overarching theoretical framework, readiness of pediatric nephrology clinics and dialysis units at all participating sites was explored prior to implementing the validated PRO-Kid into clinical practice. The Organizational Readiness Theory facilitated a systematic exploration of the ability of each organization to accept a change or new process through the assessment of barriers and facilitators. ¹⁴

Readiness at the team level (readiness, motivation, team cohesion to use the tool) and at the organizational level (availability of systems, resources, etc) was assessed using surveys and focus groups. The Consolidated Framework for Implementation Research (CFIR) ¹⁴ guided the design of the survey and focus groups. The CFIR offers constructs, delineated into five domains (innovation, individuals, inner setting, outer setting, implementation process), that are identified as important for effective implementation of new programs.

Key interest holders at each site who are involved in the care of patients with CKD were invited to participate in focus groups. Personnel of interest included physicians, CKD nurses, dialysis nurses, medical office assistants, and data managers or information technology leads. Prior to the focus groups, local leads were invited to complete the readiness assessment survey for their site to provide information on clinic structure (including presence of a multidisciplinary team), time patients spend in clinic, technology and specific electronic medical record (EMR) systems available and CKD G3-G5 and dialysis population size. The focus groups, held in either English or French based on the primary language at the site, used a semi-structured focus group guide designed around the five CFIR domains and were led by a member of the research team (K.A.).

Quantitative data from the surveys and qualitative data from the focus groups were analyzed using a mixed-method approach, with the objective to expand on local experiences and identify local barriers and facilitators to implementation. The focus group field notes were used to ascertain the level of organizational readiness at each site. They were organized and coded deductively guided by CFIR, using NVivo Qualitative Data Analysis Software.

Asynchronous implementation was started based on site readiness. A toolkit was created, which included instructions for tool utilization and scoring to assist clinical staff with the consistent completion of PRO-Kid with patients/families. Information gathered through the surveys and focus groups helped tailor site-specific toolkits to each site’s local context. For example, the toolkits included center-specific information around the setting in which the tool would be administered, who would be administering the tool, and the process in which the scores would be recorded (scanned copy, generated score in the EMR, etc).

Implementation Evaluation

With the support of the evaluation team of the George & Fay Yee Centre for Healthcare Innovation (CHI) at the University of Manitoba, implementation evaluation was conducted over the six-month period following the launch of the implementation effort. The evaluation aimed to assess the success of implementation at each participating site, including barriers and facilitators to the ongoing implementation. The Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework ¹⁵ guided the evaluation of our implementation strategies. The RE-AIM framework is an evaluation model that assesses the outcomes necessary for effective population health impact and sustainability in five dimensions. Both CFIR and RE-AIM frameworks were used in this study because they are often paired together to strengthen implementation efforts. Consolidated Framework for Implementation Research is a determinant framework that helps identify what factors influence implementation. RE-AIM is an evaluation framework that guides how to measure the success of implementation.

To assess success of implementation efforts, we collected data using chart audits, surveys, and focus groups.

1. Chart audits: Chart audits at one-month and four-month timepoints postimplementation to determine the use of the PRO-Kid (proportion of users, representativeness, and frequency of completion by eligible patients). Data from the chart audits informed the Reach dimension.

2. Surveys: Surveys were administered to patients, parents/caregivers, and health care providers. The patient and parents/caregivers survey was developed with input from patient and parent partners to ensure that the survey was relevant and appropriate. We collected responses between the fourth and sixth month after starting implementation at each site. The patient and caregiver surveys assessed perception of ease of completion, the use of PRO-Kid results during patient encounters (ie, to improve communication), and the value of the tool, informing the Effectiveness domain. The health care provider survey was aimed at capturing enablers and barriers to reaching eligible patients (Reach dimension), perceptions on the utility of the PRO-Kid tool in practice (Effectiveness domain), attitudes toward adoption (Adoption domain), format and clinical use (Implementation domain), and sustainability of use (Maintenance domain).

3. Focus groups: Focus groups with health care providers were performed between the fourth and sixth month of implementation. The objectives of the focus group were (1) to gather health care providers’ perspectives regarding enablers and barriers to reach and adoption of PRO-Kid in clinical settings (Reach and Adoption domains); (2) to identify important considerations for the toolkit refinement (Implementation domain); and (3) to explore whether and how health care providers used the PRO-Kid tool during clinical encounters (Reach, Effectiveness, and Maintenance domains). A focus group guide informed by the RE-AIM framework facilitated discussions. Focus groups were led by LD, from the CHI team.

Quantitative data were entered into Research Electronic Data Capture (REDCap) ¹⁶ housed on the Survey Server at the University of Manitoba. Data were summarized using descriptive statistics (counts, ranges, averages) using Microsoft Excel (Microsoft Corporation, Microsoft 365) for analysis. Qualitative data were analyzed using a general deductive approach. Focus groups’ field notes were organized and analyzed using NVivo 15. The summary of each site’s focus group findings was shared with the site’s principal investigator (PI) to facilitate refinement of the toolkit and to promote adoption and reach.

Patient and parent partners were involved throughout the process of implementation and implementation evaluation.

“For this project, I attend check-in calls and other meetings, and I provide input and feedback on the tool and related products (e.g., posters, presentations). In 2023, I attended an in-person meeting in Toronto to discuss the knowledge mobilization plan for PRO-Kid and FILTR-CKD and provided a brief introduction of myself and my experience to a group of kidney specialist physicians, nurses, and researchers. Most recently, I attended a virtual call with researchers working on phase 3 analysis for PRO-Kid and provided a perspective on some of the results they are seeing from use of the tool in clinic. I try to provide constructive input from the perspective of a mother/parent of a child with CKD (I also try and place myself in my son’s shoes and consider how he might interpret various things). From a communications perspective, I try to offer suggestions that are simple, clear, and inclusive. I’m proud to contribute to this project because I think there is real strength in collaboration and bringing different perspectives to the table, including patient perspectives (K.L., parent partner).”

Key Findings

Findings From implementing Sites (Context)

The Organizational Readiness Assessments to investigate the individual and organizational readiness to implement the PRO-Kid tool for clinical use were performed at five English-speaking sites and two French-speaking sites. The assessments identified the following key findings:

1. Pediatric Nephrology clinics across Canada are motivated to use the PRO-Kid tool to improve communication with children living with CKD. Providers all agree that patient-reported outcome measure (PROM) tools are needed in practice.

2. Programs in different provinces use different EMRs. The available technology and EMRs were identified as important determinants of implementation, representing a facilitator in certain settings (eg, in programs PROMs have already been implemented in the past) and a barrier in others (eg, in programs using predominantly paper charts). Sites required different implementation plans tailored to their local context.

3. While some clinicians prefer having the PRO-Kid completed on paper and included in the physical chart for each patient encounter, most agreed that being able to see the PRO-Kid scores in the EMRs and to trend them over time would be necessary for optimal utilization of the clinical data.

4. The frequency of completion of the PRO-Kid will be different at CKD clinics compared with dialysis units, mainly because of the frequency of the appointments. The teams at all sites did not feel the need to have the tool completed outside of the clinic visits.

5. Teams highlight the value of having the PRO-Kid in the two main Canadian languages, but report the need to have the tool in other languages (such as Mandarin or Arabic) to better reach the multilanguage population they serve.

Preliminary Findings of the Implementation Evaluation

Three of the English-speaking sites (BC Children’s Hospital [Vancouver], the Health Sciences Centre [Winnipeg], and McMaster Children’s Hospital [Hamilton]) have commenced implementation, and two of these sites have completed implementation evaluation. Data were collected between October 2024 and May 2025.

Preliminary results suggest that PRO-Kid was offered to most children; however, some barriers such as unwillingness to participate, language, and low reading levels have been identified. PRO-Kid has been an effective tool to improve communication in the clinical setting, generating great interest among patients, caregivers, and health care providers, along with prompt adoption from the health care providers’ perspective (Figure 3). From their observations, health care providers report that PRO-Kid helps elicit symptoms that might not otherwise spontaneously come up during clinical encounters.

Figure 3.

A step-by-step breakdown of how the novel PRO-Kid symptom assessment tool is being implemented into practice in pediatric clinics across Canada.

While evaluating implementation efforts unfold, our team will continue to identify additional patient, staff, system, and intervention-level facilitators and barriers, to inform future implementation efforts across the country.

Patient Partner Reflections on the Implementation of the PRO-Kid Tool in Clinical Practice

With the ongoing inclusion of patient and parent partners in the implementation of the PRO-Kid tool in clinical settings, one of the partners reflected on these activities:

“I appreciate that the PRO-Kid study focuses on the person behind the CKD, on the non-medical aspects of CKD. Often, the psychosocial impact of CKD is not the focus of care and pediatric patients are left dealing with feeling left out on their own. With missed school trips and social events, children can become “distant” from their peers. Over the years, there will be many hospital visits and stays which further weaken the social bonds of those their age. School days will be missed as a result of hospital visits, which can lead to a tailored exam/coursework schedule that differs from the class, or missed assignments that become a point that singles out the student. It might not be direct, but the accumulation of many years of being “different” and unable to participate in school sports or events can widen the gap of a vulnerable population at a critical stage of their development. PRO-Kid encompasses physical symptoms, cognitive symptoms and emotional symptoms to determine the impact of CKD on life participation. The result of “feeling left out” strongly resonates with my personal experience of being unable to participate in school events and having to miss school on a regular basis because of dialysis. Therefore, I am glad to have the opportunity to contribute to this study and help to rephrases questions and add to the understanding of the importance of non-medical aspects of CKD as a former pediatric patient. Having aged out of pediatric care, many of the events that transpired have greatly shaped my adult life and my outlook on life. I appreciate the PRO-Kid team for giving me the chance to share my views and I hope that the children who had participated in the study will feel like their voices were heard. I am looking forward to seeing how we can mitigate the issues identified in the study and decrease feelings of left out for children living with CKD and help them (better) navigate school and social life (K.B., patient partner; Figure 4).”

Figure 4.

Patient partners report positive experiences with the tool, noting that it allows for better identification and communication of CKD symptoms.

Implementation Reflections and Implications for Scale and Sustainability

Preliminary results from the implementation evaluation at the Health Sciences Centre (Winnipeg) and the BC Children’s Hospital (Vancouver) suggest that health care providers see the value of the PRO-Kid tool to improve the care they deliver to children living with CKD. Patients and caregivers also report positive feedback regarding their perceived relevance and benefits of this tool. Working toward the sustainability of the implementation process, each local team at pediatric nephrology centers across the country will be a key player in promoting the tool and integrating it into their clinical practice, each in the way most suitable for their context. We foresee that developing an Implementation Checklist and Package, to be integrated as part of the toolkit, will help in supporting the scale and spread at other sites.

Going forward, equitable implementation considerations will be at the heart of the discussions. Accounting for the wide age range of children living with CKD and in anticipation of language barriers and different reading levels, the team had been working on modifying the PRO-Kid eight to 18 years old for the French-speaking population, for children two to four years old (parent-proxy completion) and for children five to seven years old (three-level visual scale). Validation of these modified versions is being completed, and once published, clinics will be able to offer the appropriate tool to children and/or parents and caregivers to complete, thereby addressing such barriers. Another important consideration for equitable access involves adapting the tool for use within virtual clinic settings, where reaching patients and administering the PRO-Kid can be burdensome because of time and logistics. This aspect will require discussion among teams who provide virtual care and may require shifts in practice to give PROM tools a place aligned with each teams’ values and beliefs regarding patient involvement in clinical encounters.

Upcoming Steps in the Implementation Process

While four sites (Alberta Children’s Hospital [Calgary], the Hospital for Sick Children [Toronto], the Montreal Children’s Hospital [Montreal], and the Centre hospitalier universitaire Sainte-Justine [Montreal]) have yet to start implementing the PRO-Kid tool, the team has observed some common barriers and facilitators to implementation. Specifically of importance is the role of a motivated site PI or champion who supports buy-in and leads and drives implementation efforts at a new site.

The French-speaking sites are crucial to the validation of the French PRO-Kid tool. However, because of the stepwise approach in developing and validating the French version of PRO-Kid after the English version had been validated, there were some inherent delays in implementation in the two French-speaking sites.

Organizational readiness assessments revealed the diverse medical record systems and different EMRs being used across Canada. This was anticipated and site-specific toolkits were created to facilitate implementation. However, the process of adding the PRO-Kid tool to the EMRs is quite different with varied timelines. This has led to each site having different implementation timelines.

Conclusion

Many learnings stem from this implementation process, which have the potential to inform and guide future implementation efforts of the PRO-Kid tool. Organizational readiness assessment and local perspective were crucial in developing adapted, relevant implementation plans at each site. Due to the wide range of organizational and logistics differences between sites (eg, EMRs, clinic setups, and workflow), tailored plans needed to be discussed with key interest holders to ensure optimal results. A motivated site PI/champion as a leader of the implementation process was a key facilitator. In our experience, having the site PI/chamption overseeing the implementation process and being available to support local teams strongly influenced the outcomes of implementation. Finally, the most important learning from this effort is the importance of engaging patients with lived experiences, along with their caregivers, at every step of the process. The development, validation, and implementation of PRO-Kid were done in partnership with patient partners. While working with stakeholders during the readiness assessment, all agreed that the most important outcome to quantify would be the patients’ and caregivers’ perspective. As we continue to evaluate our implementation efforts, feedback from our patients and their caregivers serves as our most valuable indicator of success. With their input, we can circle back to the original catalyst of this research: the need, identified by patients themselves, for a PRO tool for pediatric CKD.