Improving Children’s Rehabilitation Service Quality and Experience Through Innovative Programming: A Developmental Evaluation of the Bright Ideas Program

Abstract

Introduction:

Evidence-informed practice (EIP) is a widely accepted approach to providing rehabilitation services, however there are barriers to implementation. Bright Ideas is an innovative program developed by KidsAbility’s Rocket Discovery Centre, aiming enhance uptake of EIP through incubating ideas identified by KidsAbility staff linked to organizational priorities of enhancing service access and outcomes.

Objectives:

To evaluate Bright Ideas by identifying perceived successes and challenges associated with the program with the aim of supporting clinical innovation and the adoption of evidence-informed practices in children’s rehabilitation centres.

Methods:

A developmental evaluation approach was utilized to complete a real-time evaluation of Bright Ideas. Bright Ideas program materials were reviewed to describe the program. A focus group and interview were conducted with staff who led Bright Ideas projects (n = 7) to gain insight into their experiences. Additionally, interviews were conducted with the program coordinators (n = 2) to understand the evolution of the program. Data was analysed using qualitative content analysis.

Results:

Successes that drive the Bright Ideas program and roadblocks it has encountered were identified across three dimensions: the personal level (involving Bright Ideas project leads), the programmatic level (focusing on the Bright Ideas program), and the organizational level (pertaining to KidsAbility as a whole). Key roadblocks included a lack of knowledge about the program organizationally, restricted time resources, competing organizational priorities, and an absence of infrastructure to scale ideas from Bright Ideas into the organization.

Conclusions:

Bright Ideas is an innovative program aiming to make impactful change by enabling staff to lead implementation-driven projects. Our findings highlight many successes of the program and use the Theoretical Domains Framework and Behaviour Change Wheel to make recommendations to support its sustained implementation at KidsAbility.

Keywords

Developmental evaluation evidence-informed practice children’s rehabilitation health service innovation healthcare quality improvement theoretical domains framework behaviour change wheel

Introduction

Widely accepted as an ethical and accountable approach to clinical practice, evidence-informed practice (EIP) takes an inclusive approach to health care decision making, prioritizing empirical research, clinical experience, as well as clients’ knowledge, wants and needs.^1,2 Although there is consensus from clinicians and health service management about the importance of EIP, barriers such as a lack of time, limited resources and change fatigue impede the uptake of EIP interventions into clinical practice.^3,4 Conversely, prioritization of EIP by organizational leadership and a structured plan for implementation of EIP into organizational processes facilitates its adoption and embeds EIP as a core value of the organizational culture.^3-5

In their model of EIP, Carnochan et al⁶ identify implementation as an action required in the process. The Theoretical Domains Framework (TDF)⁷ and the Behaviour Change Wheel (BCW)⁸ use principles of behaviour change to understand how human behaviour impacts the implementation of health service changes, such as EIP interventions. The TDF explores barriers to implementation (e.g., lack of time,^9,10 difficulty interpreting research findings¹⁰) and the BCW guides intervention strategies that support the uptake and sustainability of change (e.g., engaging knowledge users, providing resources such as time, money, personnel^11,12). The BCW has been used in combination with the TDF to support implementation initiatives across a variety of health sectors including cancer care,¹³ chronic disease self-management,¹⁴ stroke rehabilitation¹⁵ and pediatric rehabilitation.¹⁶ These models can be used in tandem to develop a plan for the implementation and evaluation of EIP changes, such as new organizational programs.¹⁷

Social innovation, an often emergent and non-linear process, aims to address social needs through developing solutions that disrupt or challenge the context in which the need is situated.¹⁸ Collaboration amongst invested parties is prioritized in social innovation to reimagine both processes and outcomes addressing organizational challenges perpetuating inequities.¹⁸ Aligned with EIP, social innovation utilizes a variety of knowledge sources to guide decisions and prioritizes a plan for implementing and evaluating the solutions developed to address an identified social need.^18,19

Grounded in a social innovation approach, the Rocket Discovery Centre (Rocket) was established by KidsAbility Centre for Child Development (KidsAbility) in 2021 as a research and innovation centre with a mandate to engage in clinically impactful research that prioritizes the implementation of evidence-informed findings into clinical practice. KidsAbility is a publicly-funded Children’s Treatment Centre in Canada with five clinical sites that service a large region in Southwestern Ontario. KidsAbility provides family-centred, evidence-informed rehabilitation services to children and youth with disabilities and developmental delays from birth to secondary school exit. Despite providing services to over 14,000 children and youth each year, there continues to be a large waitlist for KidsAbility’s services restricting access to timely care. This wait time indicates that current resources and service structures are not meeting community demands for these services. As a result, innovation, access, and experience are all highlighted as strategic priorities for the KidsAbility organization.

Funded by a donation from the KidsAbility Foundation, Bright Ideas (BI) is a program developed and administered by Rocket to incubate innovative, evidence-informed ideas identified by KidsAbility staff, aiming to directly impact internal priorities related to enhancing service access and outcomes. BI empowers KidsAbility employees to engage in EIP by identifying a clinical problem, examining related evidence, proposing a solution, and testing that solution. If a BI project generates sufficient evidence of impact, the project becomes eligible for broad implementation across the organization.

This paper reports on a developmental evaluation of the first three rounds of BI occurring between the years 2021 and 2022 requested by Rocket for intended use by the BI program coordinators. The objective was to identify perceived successes and challenges associated with BI in its current phase of development to inform sustained implementation and future directions. The aim of reporting these findings is to support clinical innovation and the adoption of evidence-informed practices in children’s rehabilitation centres.

Methods

Study design and evaluation questions

Developmental evaluation as described by Patton²⁰ supports real-time evaluation of an innovative program currently in development and situated in a complex environment, where the evaluation design is responsive to the context. Developmental evaluation aims to make system-level change while simultaneously providing feedback that can be utilized to modify the program before the evaluation is complete.²⁰ Developmental evaluation is well suited for the evaluation of BI, as the program is a new innovation within a complex and ever-changing health service organization.²¹ In developmental evaluation it is common to include people who have both insider and outsider perspectives, such that the context can be fully understood, internal organizational needs can be met, and findings can be immediately used to inform ongoing projects.²⁰ Several authors held an ‘insider role’ on this project team. MR worked clinically at KidsAbility during the time of this project, author MP previously worked clinically at KidsAbility and holds an adjunct scientist role with the organization, authors CL and BWT are program coordinators with KidsAbility’s Rocket program and actively contributed to the developmental evaluation process. Author EC held an outsider perspective and supported the research activities required for this developmental evaluation. The ‘What?, So What?, Now What?’ framework was used to shape the evaluation questions (Table 1) and interview guides as it facilitates a reflective process that helps evaluators make sense of their findings and inform future actions.²⁰ This framework is well-suited for developmental evaluation because it recognizes that evaluations are both assessing past performance and shaping future actions. Given the quality improvement focus of this project, this paper followed the reporting guideline of the SQUIRE checklist.²²

Table 1.

That what? So what? Now what? Framework and evaluation questions.

Component of framework	Evaluation question(s)
What?: This question is aimed at understanding the innovation	What is the Bright Ideas program?
So What?: Here, we sought to interpret the results and impacts of innovative efforts	Has the implementation of Bright Ideas program: (1) supported personal and professional development of project leads? (2) influenced the culture of innovation at KidsAbility?
Now What?: This examines what findings reveal about considerations for improved and sustained implementation of the innovative program	What aspects of the Bright Ideas program need to be addressed to support its sustainable implementation at KidsAbility?

Project context – Innovative program of interest (Bright Ideas)

To become involved in BI, KidsAbility staff apply to an organization-wide call for proposals. The application asks staff to identify a problem they encounter, with implications for one of the following service priorities: reducing wait times for therapy, removing barriers to accessing service, and improving quality of care. All BI submissions are collated and ranked by an interdisciplinary committee composed of internal employees and individuals external to KidsAbility. The top-ranking ideas are brought forward to leadership for approval. The number of ideas approved depends on their ranking and alignment with organizational priorities. Innovations supported by BI are awarded up to $5000.00 to explore problem development and pilot implementation. As a group, Rocket provides project leads with innovation training and individual coaching throughout the project.

Participants and recruitment

A purposive sampling strategy was used. All project leads who participated in rounds 1 through 3 of BI (n = 9) and were currently employed received notice via email about the developmental evaluation process and were invited to participate in a focus group (rounds 1 and 2) or an interview (round 3). Participants were assured that all identifying information would be removed from the collected data, and every effort would be made to anonymize the information in any written reports. Participants were informed that while no names would be included in any final reports, it might still be possible for individuals from within the organization to recognize their contributions.

The two program coordinators (authors BWT and CL) were invited to complete an interview. Aligned with incorporating the insider perspective in developmental evaluation, the program coordinators were included as members of the research team. Incorporating researchers embedded in the study context helps to ensure relevance of the project, enhance uptake of findings into practice and build organizational capacity for future research.²³ Their participation was voluntary with the understanding that given their roles as BI coordinators, it might be possible to discern their contributions to the interview discussion.

Data collection

BI program materials

BI program materials were used as data for the purpose of describing details of rounds 1 through 3 including the number of applications received, the number of ideas selected to move into the program, a description of the ideas, and the project leads’ roles at KidsAbility.

Focus group and interview with BI project leads

In January 2022, a one-hour focus group was conducted over Zoom²⁴ by author MR with project leads from rounds 1 (n = 4) and 2 (n = 2) of BI. Of the 6 participants, 4 were clinical staff and 2 were managerial staff. The interview guide (see Supplementary Materials) was developed in collaboration with author CL, one of the coordinators of BI. Open-ended questions inquired about participants’ motivations for taking part in BI, successes and challenges, and insights on what factors might lead to the program’s future success or failure. Participant contributions were collated using Google Jamboard.²⁵ A one-hour interview was completed over Zoom²⁴ in October 2023 with a BI project lead from round 3. The same interview guide as was used for the focus group was used for the interview. Both the focus group and interview were audio recorded for analysis purposes and transcribed by author EC.

Interview with program coordinators

Two semi-structured interviews taking place in December 2022 and October 2023, were conducted virtually on Microsoft Teams,²⁶ by author MR with the two BI program coordinators (authors CL and BWT). Participants were asked a similar set of questions as the project leads (see Supplementary Materials). The purpose was to solicit reflections on lessons learned from rounds 1 through 3 of BI and provide contextual insights into the barriers and facilitators encountered. The interviews were audio recorded and transcribed by a Rocket student intern.

Data analysis

Qualitative content analysis served as the foundation for our analytical approach of the focus group and interview transcripts as well as the focus group Jamboard. We applied the methods of inductive content analysis as outlined by Elo and Kyngas.²⁷ In the preparation phase, we familiarized ourselves with the data to produce a detailed, direct description of participants’ experiences and insights that remained closely connected to the original data. During the organization phase, author EC conducted an open coding process. Descriptive codes were assigned line by line and were then grouped into higher-order categories. Finally, in the abstraction phase, data was synthesized first into a table format and then narratively across the descriptive code groups based on the emerging structural framework.

To maintain trustworthiness throughout the data analysis process, author EC wrote memos. These memos recorded the implications of success and roadblocks on the program’s sustainability, the extent to which these factors aligned with the program’s aims and intentions, and recommendations for addressing disparities between intentions and execution. A triangulation approach was employed to enhance credibility, whereby authors MR and EC met regularly to deliberate on the evolving organization of codes and data structures. These sessions provided a platform for addressing areas of consensus, divergent perspectives, and alternative interpretations. Importantly, these discussions persisted throughout the entire research process, culminating in a collective agreement on the components to be included in the findings. Peer debriefing took place by having authors MP and CL review and provide feedback on both the interim and final findings.

Ethical considerations

The research team sought clarification from the Hamilton Integrated Research Ethics Board and confirmed that ethics approval is not a prerequisite for this project, given its basis in quality improvement and program evaluation. While ethical regulations do not mandate that informed consent be obtained for the utilization of program evaluation data, participants had the option to abstain from participating.

Results

Table 2 describes rounds 1 through 3 of BI, providing counts of the number of submissions, how many ideas progressed into each round, who led projects in each round and a statement describing the focus of each project. These descriptive findings help to contextualize the qualitative data over time.

Table 2.

Describing Bright Ideas rounds 1 through 3.

	Round 1 (Fall 2021)	Round 2 (Spring 2021)	Round 3 (Fall 2022)
Number of submissions	n = 27	n = 18	n = 9
Number of ideas moved into program	n = 3 (two projects were co-led)	n = 3	n = 2 (one project lead withdrew part way through round for personal reasons)
Description of selected ideas	– Using a group-based assessment model to decrease wait times for service. – Increasing supports and resources available to families while they wait for initial assessment. – Exploring the viability of providing audiology services.	– Implementation of universal resources. – Use of technology to improve efficiency of client-based data collection. – Enhancing access to multi-lingual resources and supports for families.	– Empowering educators to use universally designed therapy supports for students in the school context.
Project lead’s role at KidsAbility	Clinician n = 3 Management n = 2	Clinician n = 1 Management n = 2	Clinician n = 1

The data analysis identified successes that drive the BI program and roadblocks it has encountered across three dimensions: the personal level (involving BI project leads), the programmatic level (focusing on BI specifically), and the organizational level (pertaining to KidsAbility as a whole). Table 3 summarizes the successes and roadblocks at each level.

Table 3.

Evaluation findings – Bright Ideas successes and roadblocks.

	Perceived successes	Perceived roadblocks
Personal (Bright Ideas project leads)	– Personal fulfilment gained through novel experiences and a sense of agency over project. – Professional growth gained through new learning opportunities and acquisition of skills.	– Lack of role clarity related to project decision making. – Unclear expectations about project development timelines.
Program (Bright Ideas)	– Implementation of formalized training and support structure for project leads. – Fostering internal connections through bridging frontline clinician knowledge to the broader organization. – Establishing external partnerships to support Bright Ideas projects. – Establishing a low-stakes space for experimentation.	– Program does not sufficiently protect time of the project leads resulting in demotivation of project leads and restrictions on how they are able to take part in the program. – Program coordinators experience restrictions in the time available within their roles to administer and monitor Bright Ideas. – Competing priorities across organization
Organization (KidsAbility)	– Bright Ideas service as a catalyst to foster an organizational culture of innovation. – Bright Ideas aims to diversify avenues for staff involvement in innovation.	– Lack of knowledge about Bright Ideas within the organization potentially impacting meaningful staff participation, clear understanding of program aims and program recognition. – Facilitating wide-scale implementation of impactful Bright Ideas projects across the organization.

Personal successes

Project leads expressed a sense of personal fulfilment gained through two avenues. Firstly, many individuals expressed satisfaction in their engagement with ‘new and exciting [experiences] compared to the everyday’ (lead-P5), relishing the opportunity to take on novel challenges that provided a ‘different sense of accomplishment than. . .other day-to-day work’ (lead-P2). Secondly, project leads expressed experiencing a sense of agency and control over their projects. Having a dedicated space to share ideas within the BI program made project leads feel valued and heard, fostering an environment where their input was acknowledged. The program gave project leads ‘a sense of trust . . . [they] were given autonomy to run with it and do the pieces [they] wanted’ (lead-P2). This dynamic environment empowered project leads to initiate and drive positive changes.

Project leads also highlighted professional growth as a success of participating in the program. Project leads perceived their engagement in new learning opportunities to broaden their perspectives on various facets of rehabilitation service delivery. This exposure offered insights into the ‘the business side of service delivery’ (Jamboard (JB)-successes), including financial (lead-P4), administrative (JB-successes), program management, research and evaluation (JB-successes). Furthermore, the program was perceived by participants to facilitate the acquisition of new skills through training in innovation. Participants expressed learning the importance of problem definition and validation, honing their skills in convergent and divergent thinking, and gaining the ability to reframe their ideas into a concise story.

Personal roadblocks

BI project leads shared a sense of uncertainty in decision-making about next steps in project progression. A project lead described the challenge, stating, ‘[It] was the lack of initial clarity around what this involved. . .It wasn’t clear to me in terms of starting this, that I would suddenly be expected to lead the project. . .I feel like there was a lot of support and explanation at the beginning, but then it was like “here you go”’ (lead-P4). BI coordinators described ambiguity as a feature of the program and a necessary component of innovative change. Recognizing the potential discomfort ambiguity can cause, the program coordinators described that the current BI innovation training supports project leads in planning for and tolerating ambiguity. Coordinators recognized that project leads might benefit from additional coaching, supporting them through periods of uncomfortable ambiguity and reinforcing that it is an expected and desired outcome of the process.

Program successes

A perceived program-level success as described by the program coordinators, involved the implementation of formalized support provided to project leads throughout the program. Recognizing the importance of structured support, one coordinator stated, ‘We really mapped it out so that we were more intentional, with our timing and making sure that they have the right information at the right time’ (coordinator-1). This intentionality was exemplified by the program coordinators’ describing their efforts to define learning goals and create training modules, geared towards equipping project leads with the knowledge needed to position their projects for success. The training process was described as maintaining a degree of flexibility through individual coaching. Program coordinators provided personalized guidance ‘on the fly’ (coordinator-1), addressing the unique aspects of specific projects, accommodating the diverse needs of project leads and maintaining project momentum.

The program coordinators described further enhancing the program structure by refining a phased approach. Program coordinators described that each phase entails deliverables, criteria for evaluation, and outputs. The phase emphasizing problem definition and validation was perceived by program coordinators to be particularly pivotal. A coordinator captured this sentiment when discussing a past project:

Their entire project [involved] just validating the problem. . .but the value of that has been huge. . .We’ve been able to identify what we need to do differently for families on the waitlist. And there [are] initiatives that we’re going to span out all across the organization, partly based on that. (coordinator-1)

This statement highlights the perception of program coordinators that problem definition and validation yielded valuable outputs aside from solution ideation, informing various levels of decision-making and inspiring change across the organization. As such, prioritizing problem definition and validation was an effective starting point for fostering innovative changes.

A second success of the BI program as perceived by participants was its role in fostering connections. Internally, the program was described to bridge knowledge within the organization, allowing previously untapped knowledge from frontline staff to be accessed and utilized. Moreover, it was thought to facilitate connections between different knowledge sources. One project lead described their experience:

We were told. . .to talk to so-and-so because they had already surveyed families. . . There was this information out there, we just needed to connect the dots a little bit. . .one manager we talked to was so excited that we were actually going to use the information that they had gathered because she wanted it actually used somewhere versus just being stored. (lead-P1)

Beyond organizational boundaries, BI was perceived by project leads to establish valuable external partnerships that enabled project leads to outsource the time and expertise they lacked internally. Collaboration within these partnerships was thought to elevate the quality of work as project leads and partners leveraged each other’s resources. For example, a project lead was paired with a student group from a university business program, utilizing their skills to assess the economic feasibility of the idea proposed by the lead. The project lead emphasized the crucial role of this support, stating, ‘Had I not had [the support], I have no idea what [the project] would have looked like or what I would have accomplished’ (lead-P4). According to project leads, this highlighted the critical role of coordinators in brokering these connections, aligning with project leads’ expectations that the program coordinators act as ‘the facilitators of that support’ (lead-P6).

Another perceived success of the program described by the program coordinators, was its establishment of a low-stakes space for experimentation, analogous to a ‘sandbox’ (coordinator-2). In this space, staff were encouraged by program coordinators to ‘play around’, (coordinator-1) offering a unique opportunity for those who have long ‘felt like they weren’t allowed to do it’ (coordinator-2) to initiate changes with support. Program coordinators shared that this setting provided project leads the freedom to test changes without the constraints of formal processes and gave the organization flexibility to pilot said changes before committing to large-scale implementation.

Program roadblocks

Participants perceived a significant roadblock perceived to be the lack of protected time. Without dedicated time, project leads who already felt ‘pushed to the limits of the time in the day’ (lead-P7) described finding themselves compelled to exert extra effort beyond their job requirements to manage BI project responsibilities. This situation was described as having dual impact: first, it generated discouragement and demotivation among those already engaged in the BI program. Second, it hindered the involvement of frontline staff members who are not part of the program. Participation in BI was described by project leads as beginning to ‘feel like just an extra thing’ (lead-P7) as project leads could not reduce their clinical workload to participate. This issue gains significance when considering the experience of a project lead whose manager opted to approve a project lead using time between clinical duties to complete BI project work. This participant expressed, ‘My manager’s been super supportive, loves the idea, and loves where we’re going with it.. . .So I would find the time. . .I just did it within my day, not in lieu of clinical, but just in and around my clinical’ (lead-P7). This testimony underscores the potentially transformative impact of explicit approval to integrate project work within existing clinical responsibilities, emphasizing its positive effect on participant experience.

Constrained resources are described to currently impact the time that program coordinators can dedicate to BI. Coordinators described being acutely aware of the pressing need to enhance communication and education about the BI program across the organization, as well as to improve the management and implementation of BI projects on a larger scale. ‘We don’t have enough time. We don’t have enough capacity. . . [an] increase in staffing would be an evolution. We’re resource-constrained at the moment’ (coordinator-2).

Program coordinators shared that operating with constrained resources was not unique to the BI program, it impacted the whole of KidsAbility as a publicly-funded organization experiencing a greater demand for service than resources can accommodate. Finite resources were described as contributing to the existence of competing priorities across organizational programs. Program coordinators shared that innovative approaches, such as the BI program, that prioritize experimental approaches and long-term impact (e.g., new models of service) can conflict with the immediate demands associated with service provision (e.g., provision of care to families and waitlist management).

A lack of dedicated time for project leads to complete project-related tasks was a roadblock perceived by program coordinators as closely linked to competing priorities within the organization. Time spent on leading a BI project was sometimes viewed as time taken away from addressing immediate service priorities. Program coordinators acknowledged there was a need to elevate how the organization viewed time dedicated to BI projects: ‘. . .that comes back to value within the organization. It’s valuing that this work is equally important to some of the other things that [BI leads] could do when they do have [time]’ (coordinator-1).

Program coordinators shared that they were actively working on improving communication and engagement throughout the organization. An engagement initiative involved inviting members of the leadership team to participate in the innovation training modules designed for the program. In describing the rationale for the initiative, one program coordinator stated, ‘Seeing that we really needed their support for some of these projects to move ahead, we figured bringing them into the process and having them feel more included and more aware of what we were doing would be helpful’ (coordinator-1). This strategic outreach emphasized the importance program coordinators placed on promoting involvement and understanding among the leadership team, recognizing their crucial role in facilitating effective project progress. Program coordinators, grappling with constrained resources and competing priorities across the organization, described striving to ensure that changes arising from BI projects resulted in positive, tangible outcomes.

As one program coordinator emphasized, ‘We have to justify our existence. . .I want to go back to say, we’ve done good work. We’ve changed the system. We’ve improved access or quality. And for us to continue to get gains, we need to get another three- [or] five-year investment. . .that is why we need outcomes. But it’s also why we need innovation generally. . .there is so little capacity to improve outcomes in meaningful ways in sectors like ours’ (coordinator-2). This perspective illustrated the importance of adequate resourcing to not only achieve immediate project success but to also demonstrate the broader impact and value of BI. According to the program coordinators, illustrating the value of BI was essential for justifying its continued existence and securing the ongoing funding required for sustained innovation.

Organizational successes

The BI program was described as a catalyst for fostering a culture of innovation within the organization. Program coordinators described that it brought about a shift towards evidence-based mindsets. A ‘bias towards action’ (coordinator-2) was evident, particularly among upper leadership, reflected in the emergence of an ‘agreement that we need to test and tinker with some of these things’ (coordinator-2). However, to ensure the continual growth of this culture, ongoing efforts are essential.

To promote innovative thinking, the program coordinators described three strategic approaches. Firstly, external perspectives were integrated into the organization through established partnerships, involving university students from diverse undergraduate departments (e.g., business, engineering) and postgraduate professional programs (e.g., speech-language pathology, occupational therapy), as well as research teams. A program coordinator described that an additional avenue for incorporating external insights was the expanded call for proposals, which included parents of children connected with KidsAbility services.

Secondly, program coordinators described aims of diversifying avenues for staff involvement in innovation beyond formal proposal submissions and project leadership. This included exploring alternative methods for idea generation, such as online submissions or hosting focus groups where individuals can participate in brainstorming sessions. Program coordinators hoped to extend these efforts earlier in the innovation process by running ‘programming to stimulate thinking ahead of the next intake, [which] will involve or require some interaction with external stakeholders of some kind to stimulate thinking in ways that our staff aren’t normally thinking or places where they’re not normally looking’ (coordinator-2).

Thirdly, coordinators described an active effort to further facilitate the creative process, encouraging more individuals to generate ideas that can feed into the BI pipeline. This involved expanding training opportunities to provide access to the innovation training modules to every person who submitted an idea, regardless of selection. The dissemination of knowledge created a shared language of innovation. Trained individuals became ambassadors of innovation, integrating newfound knowledge into their roles, teams and across the organization. One program coordinator envisioned a future where ‘the whole organization look[s] at our big problems, and everybody [is] in different pockets trying to attack it in different ways. . .we’d all be moving in the same direction, [trying] to address the same priorities’ (coordinator-1). Ultimately, the stated aim was for BI to contribute to cultivating an organizational culture of continuous improvement and collaborative creativity.

Organizational roadblocks

An organizational roadblock perceived by participants as impacting BI was the need for cross-organizational communication that promoted a shared understanding of the program. Participants described there being a lack of knowledge about the program’s position within the organization, its purpose, and its goals. Program coordinators have observed that the lack of clarity prevented meaningful staff participation, as individuals across the organization were unaware of the program’s intended audience, the reasons for participation, and the criteria defining suitable contributions to BI.

As a consequence of this communication gap, individuals with valuable ideas may bypass BI altogether. As one project lead noted, ‘you still hear people say, “Oh I went to a manager with [my idea], or I went to this place, I went to that place.” Well, why didn’t you go to BI? . . .that’s what it’s for’ (lead-P7). This tendency potentially results in missed opportunities for impactful ideas to receive the support and resources available through the program.

Program coordinators were implementing various measures to increase ongoing information sharing, such as showcases, staff events, workshops, and the launch of a new website. Program coordinators described that this multifaceted approach was designed to boost awareness, raise the program’s profile, and foster a deeper understanding of the program’s nature and activities across the organization.

Program coordinators envisioned that their communication efforts would bring about an evolution in the quality and relevance of proposals received by BI. One coordinator expressed, ‘hopefully, as more BI fit into the bigger picture, like when that becomes clear to everyone. . .that’ll change the types of BI that we’re getting’ (coordinator-1). Increased clarity about BI was intended to help individuals distinguish which projects aligned with the framework and which might be better suited for alternative pathways.

Project leads described expecting improved communication to alter attitudes and perceptions regarding BI, allowing for the celebration of successes and broader organizational acknowledgment of their work. One project lead expected increased awareness to enable ‘managers and whomever else [to be] empowered to direct us in the right way’ (lead-P7) regarding idea development. Related to their BI experience a project lead felt that by avoiding the risk of ideas getting lost or set aside due to constraints in managerial bandwidth, redirecting ideas towards the BI program would serve to sustain the nurturing of creativity and innovation.

As communication efforts persist, the hope was that employees across the organization would witness success stories, understand the potential for idea growth and development, and recognize the personal and organizational benefits of engagement. As one project lead put it, ‘from a bigger picture, this just strengthens that whole innovative, growth-mindset, big picture thinking, no matter what your role’ (lead-P7). Ultimately, the aspiration is for BI to become ‘just part of our DNA’, (lead-P7) fostering a more conducive environment for innovation across the organization.

A program coordinator described BI akin to a funnel, aiming to capture many ideas and, through experimentation, distil them down to a select few projects that have demonstrated positive impact and were suitable for wide-scale implementation. According to program coordinators, within its first three cycles, BI had successfully produced impactful projects prime for organizational implementation. This program success uncovered a second organizational roadblock described by program coordinators as being linked to the mechanisms by which BI projects were scaled across the organization. At the time of evaluation, there was a lack of established ‘procedures. . .protocols, [and] policy’ (coordinator-2) designed to guide an organized transfer of responsibilities from the BI program to clinical program management.

The factor of competing priorities resurfaced when considering the division of responsibility and accountability associated with scaling BI projects organizationally. Program coordinators described that while the BI program team was accountable for the outcomes of the projects, clinical program management was responsible for the organizational implementation process. This created a delicate balance for program coordinators, who described finding themselves navigating the challenges of accountability without having substantial control over the outcomes.

In response to these challenges, a suggested strategy from program coordinators emerged: rather than a straightforward transition of project ownership, the implementation phase should be entered into as a ‘joint project’ (coordinator-1) between the BI team and management. This collaborative approach was described to involve reaching a mutual ‘agreement of what those outcomes need to be so that everybody felt accountable and responsible for them’ (coordinator-1). In fostering shared accountability and responsibility, the hope of program coordinators was to ultimately build a cohesive vision for the success of the projects.

Discussion

BI was implemented at KidsAbility in 2021 as an innovative program aiming to enhance EIP through leveraging the ideas and skills of employees to address organizational priorities related to service access and experience. This developmental evaluation identified outcomes in the form of perceived personal, program and organization successes and roadblocks associated with the early phases of the BI program implementation. BI was reported to facilitate personal and progressional growth opportunities for project leads, create training opportunities and partnerships as well as catalyse innovative thinking across the organization. Conversely, limited knowledge about the program organizationally and a lack of clarity about roles and processes as well as competing organizational priorities were perceived as roadblocks.

Key successes and roadblocks will be discussed with consideration for how they may influence the likelihood of sustained implementation of BI within the organization and guide future directions of the program. Using The TDF and BCW implementation science frameworks, strategies for optimizing successes and managing roadblocks will be proposed. Table 4 synthesizes the discussion about key successes and roadblocks, their relation to the TDF and BWC and the practical ideas supporting sustained program implementation.

Table 4.

Bright Ideas findings and related implementation support strategies.

Bright Ideas findings	Related domain(s) of TDF	Related BCW intervention(s)	Strategies for addressing in context of Bright Ideas
Successes
Personal and professional growth of project leads	Optimism Goals Beliefs about capabilities	Education	– Training and mentoring from BI program coordinators
Personal and professional growth of project leads	Optimism Goals Beliefs about capabilities	Enablement	– Empower staff by enabling them to step outside the perceived boundaries of their role and take agency over the BI project.
External partnerships Space for experimentation Diverse avenues for staff involvement	Environmental context and resources	Environmental restructuring	– Develop novel infrastructure that embeds BI both within and beyond the organization
Roadblocks
Lack of knowledge about BI and lack of clarity regarding the roles and processes	Knowledge	Education	– Educate KidsAbility employees through: – visits to each clinical site for question-and-answer sessions – formal education opportunities offered regularly
Lack of protected time for project leads and program coordinators	Environmental context and resources	Environmental restructuring	– Develop policy and processes to restructure the KidsAbility environment to move projects from BI into the organization
	Environmental context and resources	Enabling	– Increase personnel and funding for BI
Lag of widespread organizational engagement with BI	Knowledge	Education	– Apply education ideas from above
	Environmental context and resources	Environmental restructuring	– Apply environmental restricting idea from above
	Social/professional roles and identity	Modelling	– Identify champions for BI who can share their experiences of participating in the program and discuss the impact of the projects on the organization
	Social influence	Modelling	– Apply modelling idea from above
		Enabling	– Empower staff by enabling them to step outside the perceived boundaries of their role and take agency over the BI project– Partner with clinical program managers to share responsibility for BI project implementation efforts to ensure they are equipped with the resources needed to scale ideas into clinical programs and use their influence to impact sustained change

Instances of personal and professional growth reported as successes by the project leads may be in part attributed to the education and mentorship provided by the BI program, enabling them to develop the skills, knowledge and confidence needed to lead their projects. These education and enablement BCW interventions connect to the optimism, goals and beliefs about capabilities domains of the TDF and could act to support the sustained implementation of BI organizationally.¹⁷ Other perceived successes that may contribute to sustaining BI within the organization involve addressing the environmental context and resources domain of the TDF through the BCW intervention of environmental restructuring.¹⁷ BI has started to restructure the organizational environment to enhance the likelihood of sustained implementation through developing partnerships with external agencies to support BI projects, diversifying the ways in which staff can become involved in the program and creating a space in the organization for low-stakes idea experimentation. Continued use of the implementation strategies linked to the perceived successes of BI is recommended to when considering future directions for program implementation within KidsAbility and for other children’s rehabilitation centres interested in developing innovative programming.

Evaluation findings describe a lack of knowledge about BI at the staff and management level of the organization along with a lack of clarity regarding the roles and processes to be followed by project leads. Trends in declining numbers of submissions from round 1 compared to round 3 may also be indicative of staff having a limited awareness about the existence or scope of BI. In a qualitative synthesis of clinician perceptions, a lack of knowledge was found to be a barrier to the implementation of clinical interventions, while having knowledge about these practices facilitated uptake.²⁸ To target the TDF knowledge domain, the BCW education-based interventions can be a mechanism to support change.¹⁷ A systematic review, examining the effectiveness of interventions promoting implementation of EIP behaviours in allied health professionals, educational outreach was found to be effective at increasing clinicians’ knowledge about the initiative.¹² BI program coordinators have recognized the need to heighten knowledge about BI across the organization. In response, they have initiated efforts to educate KidsAbility employees about BI using strategies such as visiting each KidsAbility site to share information and answer questions about BI. Formalized education about the program and the projects supported by BI should be offered regularly to keep staff up to date on program initiatives. BI roadblocks associated with protected time and a need for infrastructure to support scaling ideas across the organization align with the environmental context and resources domain of the TDF. Environmental restructuring and enabling are approaches to intervention from the BCW connected with this TDF domain.¹⁷ A perceived lack of time and resources has been identified as a barrier to implementing evidence into clinical practice.^28,29 Conversely, creating roles and allocating resources dedicated to implementation has been found to accelerate the adoption of change.¹¹ Enablement strategies such as increasing personnel and funding for BI are recommended to facilitate sustainable implementation of the program and enhance its capacity for organizational impact. Alongside enablement strategies, it is necessary that policy and processes are developed to restructure the KidsAbility environment to move projects beyond the limits of BI into the organization broadly.

The lag of widespread organizational engagement with BI could be attributed to TDF domains such as a: (1) lack of knowledge about the program or (2) an environmental context that views BI as adjacent to the organization instead of it being embedded into the culture, which could both be addressed by the BCW interventions associated with the TDF domains discussed above. Other TDF domains likely influencing the uptake of widespread organizational engagement with BI are: (1) social/professional roles and identity and (2) social influence. These domains emphasize the impact that leadership commitment and role boundaries have on implementation.⁷ To develop a culture of innovation and implementation, the translation of evidence into practice needs to be an accepted part of role responsibilities for clinicians and leadership.¹¹

Modelling is an approach to intervention identified by the BCW as effective at targeting the social/professional role and identity and social influence domains of the TDF.¹⁷ There is an opportunity for the impact and value of the BI program to be modelled throughout the organization by project leads and managers whose employees have led a BI project. These individuals can act as champions for BI by sharing their experiences of participating in the program and demonstrating the impact of the projects on the organization.

In addition to modelling, enabling is a BCW implementation intervention category linked with the social influence TDF domain.¹⁷ Enablement intervention strategies aim to increase opportunity through reducing barriers or enhancing resources.⁸ Leading a BI project enables staff to step outside the perceived boundaries of their role and empowers them through providing agency over the project. Engaging clinicians in innovation initiatives serves to improve the sustainability of the resulting change, the implementation of EIP, and the performance of health service organizations.^30-32 However, the sustainability of the innovation is at risk if it is not authentically prioritized by the organization in which it is situated, if clinicians are not adequately resourced (e.g., time, training, manager support) to complete the project, or if clinicians do not feel their efforts are valued.^30,31 The social influence domain could also be addressed through BI program coordinators partnering with clinical program managers to share responsibility for implementation efforts. This utilizes an enabling approach to ensure managers are equipped with the resources needed to scale ideas into clinical programs allowing them to use the influence attached to their leadership roles to impact sustained change.

A limitation of this evaluation is the potential bias in responses from the program coordinators during their interviews. This bias was mitigated by triangulating their responses with those of the project leads to analyse convergence and divergence in responses between these groups. The small sample size and local nature of this evaluation may limit transferability to other contexts. However, we believe the detailed description and contextual information will enable other health service organizations explore and adapt the development and implementation of an internal innovation program aimed at enhancing EIP. Future research should focus on evaluating the effectiveness of the BI program model once it is beyond its developmental phase and exploring the transferability to other similar healthcare contexts.

Conclusions

BI is an innovative program aiming to make impactful change for families by enabling staff to lead implementation-driven projects aligning with KidsAbility organizational priorities related to service access and experience. The program model aligns with evidence that encourages embedding frontline staff into innovation and implementation efforts. This developmental evaluation highlights the many successes of BI to date such as opportunities for personal and professional growth of project leads, establishing partnerships, and fostering a culture of innovation organizations. Implementation science frameworks are used to consider how successes can be optimized to enhance sustained implementation of BI. Recommendations for targeting implementation interventions to address key roadblocks such as a lack of clarity and knowledge about the BI program are offered to guide future directions of the program and the adoption of innovative programing and evidence-informed practices in children’s rehabilitation centres broadly.

Supplemental Material

sj-docx-1-his-10.1177_11786329241297910 – Supplemental material for Improving Children’s Rehabilitation Service Quality and Experience Through Innovative Programming: A Developmental Evaluation of the Bright Ideas Program

Supplemental material, sj-docx-1-his-10.1177_11786329241297910 for Improving Children’s Rehabilitation Service Quality and Experience Through Innovative Programming: A Developmental Evaluation of the Bright Ideas Program by Meaghan Reitzel, Emily Cheung, Cynthia Lennon, Brendan Wylie-Toal and Michelle Phoenix in Health Services Insights

Footnotes

Acknowledgements

The authors would like to express our gratitude to the BI project leads who generously shared their experiences taking part in the project with the evaluation team. We would like to thank KidsAbility and Rocket for their enthusiasm and participation in this evaluation initiative.

Funding:

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Declaration of conflicting interest:

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Authors CL and BWT are the program coordinators of BI. They participated in data collection interviews for this developmental evaluation and also hold paid positions at KidsAbility’s Rocket Discover Centre.

Author Contributions

Authors MR, CL, BWT and MP contributed to the conceptualization and design of this project. Author MR led the data collection and co-led data interpretation with author EC in consultation with authors CL and MP. Authors MR and EC drafted the manuscript and authors CL, BWT and MP provided substantive revisions. MP is the senior author of this publication.

Ethics Approval and Consent to Participate

Due to the quality improvement and program evaluation aims of this project, the Hamilton Integrated Research and Ethics Board determined that this project did not require research ethics board approval.

Consent to Participate

Not applicable. Due to the quality improvement and program evaluation aims of this project, the Hamilton Integrated Research and Ethics Board waived the need for informed consent.

Data Availability

The data sets generated and analysed during the current study are not available due to privacy risks associated with the small sample size and collection of data from a single organization. Therefore, data sets are not available on request.

ORCID iD

Meaghan Reitzel

Supplemental Material

Supplemental material for this article is available online.

References

Nevo

Slonim-Nevo

The myth of evidence-based practice: towards evidence-informed practice. Br J Soc Work. 2011;41:1176-1197.

Epstein

Promoting harmony where there is commonly conflict: evidence-informed practice as an integrative strategy. Soc Work Health Care. 2009;48(3):216-231.

Ellen

M E

Léon

Bouchard

Ouimet

Grimshaw

J M

Lavis

JN.

Barriers, facilitators and views about next steps to implementing supports for evidence-informed decision-making in health systems: a qualitative study. Implement Sci. 2014;9(179):1-12.

Smith-Miller

CA.

Implementing evidence informed practice changes: barriers, facilitators, and work environments. J Nurs Admin. 2022;52(4), 203-210.

Novotná

Dobbins

Henderson

Institutionalization of evidence-informed practices in healthcare settings. Implement Sci. 2012;112(7):1-8.

Carnochan

McBeath

Austin

MJ.

Managerial and frontline perspectives on the process of evidence-informed practice within human service organizations. Hum Serv Organ Manag Leadersh Gov. 2017;41(4):346-358.

Cane

O’Connor

Michie

Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012;7(37):1-17.

Michie

van Stralen

West

The behaviour change wheel: a new method for characterizing and designing behaviour change interventions. Implement Sci. 2011;6(42):1-11.

Upton

Knowledge and use of evidence-based practice by allied health and health science professionals in the United Kingdom. J Allied Health. 2006;35(3):127-133. https://www.ingentaconnect.com/content/asahp/jah/2006/00000035/00000003/art00002

10.

Heiwe

Kajermo

Tyni-Lenné

, et al. Evidence-based practice: attitudes, knowledge and behaviour among allied health care professionals. Int J Qual Health Care. 2011;23(2):198-209.

11.

Moore

Graham

ID.

Accelerating the implementation of evidence-based practice in physical medicine and rehabilitation. Arch Phys Med Rehabil. 2022;103:S252-S255.

12.

Goorts

Dizon

Milanese

The effectiveness of implementation strategies for promoting evidence informed interventions in allied healthcare: a systematic review. BMC Health Serv Res. 2021;21(241):1-11.

13.

Fahim

Acai

, et al. Use of the theoretical domains framework and behaviour change wheel to develop a novel intervention to improve the quality of multidisciplinary cancer conference decision-making. BMC Health Serv Res. 2020;20(578):1-19.

14.

Reidy

Foster

Rogers

A facilitated web-based self-management tool for people with type 1 diabetes using an insulin pump: intervention development using the behavior change wheel and theoretical domains framework. J Med Internet Res. 2020;22(5):e13980.

15.

Connell

McMahon

Tyson

Watkins

Eng

JJ.

Mechanisms of action of an implementation intervention in stroke rehabilitation: a qualitative interview study. BMC Health Serv Res. 2016;16(534):1-10.

16.

Zachry

Woods

Using the theoretical domains framework and the behavior change wheel to design a tummy time intervention. J Occup Ther Sch Early Interv. 2023;16(3):239-251.

17.

Michie

Atkins

West

The Behaviour Change Wheel A Guide to Designing Interventions. Silverback Publishing; 2014.

18.

Mulgan

. The theoretical foundations of social innovation. In: Nicholls

Murdock

, eds. Social Innovation Blurring Boundaries to Reconfigure Markets. 1st ed. Palgrave Macmillan; 2012:33-65.

19.

Mulgan

. Social innovation theories: can theory catch up with practice? In: Franz

Hockgerner

Howaldt

, eds. Challenge Social Innovation: Potentials for Business, Social Entrepreneurship, Welfare and Civil Society. Springer; 2012:19-42.

20.

Patton

MQ.

Developmental Evaluation Applying Complexity Concepts to Enhance Innovation and Use. The Guilford Press; 2011.

21.

Fagen

Redman

Stacks

, et al. Developmental evaluation: building innovations in complex environments. Health Promot Pract. 2011;12(5):645-650.

22.

Ogrinc

Davies

Goodman

Davidoff

Stevens

SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process. J Nursing Care Qual. 2016;1(31):1-8.

23.

Vindrola-Padros

Pape

Utley

Fulop

NJ.

The role of embedded research in quality improvement: a narrative review. BMJ Quality & Safety. 2017;26:70-80.

24.

Zoom Video Communications Inc. Zoom. Published online 2023.

25.

Google. Google Jamboard. https://www.jamboard.google.com

26.

Microsoft Corporation. Microsoft Teams. Published online 2017. https://www.microsoft.com/en-ca/microsoft-teams/download-app

27.

Elo

Kyngäs

The qualitative content analysis process. J Adv Nurs. 2008;62(1):107-115.

28.

Lipworth

Taylor

Braithwaite

Can the theoretical domains framework account for the implementation of clinical quality interventions?

BMC Health Serv Res. 2013;13(530):1-13.

29.

Paksaite

Crosskey

Sula

West

Watson

A systematic review using the theoretical domains framework to identify barriers and facilitators to the adoption of prescribing guidelines. Int J Pharm Pract. 2021;29(1):3-11.

30.

Ham

Improving the performance of health services: the role of clinical leadership. Lancet. 2003;361(9373): P1978-P1980.

31.

Martin

Weaver

Currie

Finn

McDonald

Innovation sustainability in challenging health-care contexts: embedding clinically led change in routine practice. Health Serv Manage Res. 2012;25:190-199.

32.

Moore

Shikako-Thomas

Backus

Knowledge translation in rehabilitation: a shared vision. Pediatr Phys Ther. 2017;29:S64-S72.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB