Abstract
Principles-focused evaluation reflects on the change process itself through examination of its underlying principles. The Centre for Advancing Collaborative Healthcare & Education (CACHE) worked to build interprofessional education programs and tools that attended to the Team Primary Care (TPC) principles. Our internally directed principles-focused evaluation, presented here, asks how CACHE adhered to these principles in the programs and tools it delivered to the TPC project. The article’s main contribution is the creation of a new concept, organizational critically reflective practice, which describes an approach health leaders can use to mitigate the limitations of short-term initiatives while pursuing transformational change. We propose specific tools and steps that will help health leaders attempting to enact organizational critically reflective practice.
Introduction
Leaders across the Canadian healthcare system have identified primary care as both a system in crisis1-4 and the backbone of equitable, universal, single-payer healthcare.5-7 Inequity in primary care permeates the Canadian system
8
and correlates with poorer health for persons from equity-deserving backgrounds including Indigenous people, recent immigrants, and low-income households.8-10 The Team Primary Care (TPC) initiative was a pan-Canadian effort proposing the most likely solution to the crisis comes from interprofessional teamwork based on clear principles.
11
While the TPC principles evolved over the course of the initiative, ultimately, the final principles grounding the work were social accountability; truth and reconciliation; psychological health and safety; equity, diversity, inclusion, and accessibility; and interprofessionalism (see Figure 1).
12
Team Primary Care principles as of March, 2024.
Our Centre, the Centre for Advancing Collaborative Healthcare and Education (CACHE), a leading organization in the advancement of interprofessional education, practice, research, systems, and policy, was a core partner in TPC. The Centre has long embodied interprofessionalism and we continue to learn, practice, and integrate the other TPC principles. To model critical yet generative reflection, this article describes our organization’s own attempt to more fully realize all five TPC principles in our work.
Self-critique is a sensitive and challenging scholarly modality. Evaluating transformative initiatives in complex systems such as primary care is doubly challenging.13,14 Our principles-focused evaluation represents an attempt to model reflexivity, to challenge ourselves, and, through this process, offer insights to health leaders collaborating in other transformational change projects. Principles-Focused Evaluation (PFE) is a useful approach to such work because PFE explores the developmental process of complex change initiatives. 15 It is first and foremost a learning opportunity; one that elevates the importance of principles to the ongoing design and development of education. Rather than asking traditional outcomes-oriented evaluation questions, such as how an academic group like CACHE impacted primary care across an entire country over the course of single year, our principles-focused evaluation asked: to what extent did CACHE as an organization adhere to the full scope of the TPC principles in the programs and tools it delivered during the TPC project? Most importantly, what lessons could be gleaned from the CACHE experience so leaders and collaborating organizations in the next phases of the work can best attend to these principles in the future?
Methods
Design and recruitment
We followed an emergent design combining the precepts of constructivist grounded theory16,17 and principles-focused evaluation. 18 Using a constant comparative approach, we iteratively collected and analyzed data based on the following question: to what extent have the TPC principles been adhered to the programs and tools CACHE delivered for TPC?
Reflexivity
Principles-focused evaluation uses ideas from participant observation and developmental evaluation where the researcher both participates in and documents the ongoing work.19,20 In this way, the analysis was shaped both by the perspectives of the CACHE team and the experiences of the evaluators involved in developing educational resources for interprofessional education in primary care.
Data collection and analysis
The data collected for this study included: 15 semi-structured interviews with leaders and educators from across CACHE and TPC; 3 focus groups with CACHE researchers and educators; 3 field observations from CACHE design meetings and implementation sessions; and 18 reflective memos.
Analysis of the data proceeded iteratively. We targeted sampling of key participants timed to milestones in the development and implementation process of CACHE’s educational intervention. Sampling of educators happened in two phases over the course of the project: once before development and implementation of new tools and teaching sessions over the course of the year and again after the implementation process. The coding and analysis of data followed constructivist grounded theory processes from the earliest interviews, and we refined the emergent categories throughout the analysis process until we agreed by consensus that sufficiency had been reached.21,22
Results
The CACHE team worked to find a balance between achieving clear deliverables and pursuing transformational change in accordance with the TPC principles. Ultimately, supporting principles of interprofessionalism and psychological health and safety was so intrinsic to the organization that little adjustment was needed. But attending to principles of social accountability; truth and reconciliation; and equity, diversity, inclusion, and accessibility required developmental effort and attention.
Description of the core category, balancing deliberate and emergent change, is accompanied below by its subcategories (Table 1). Data from focus groups are depicted with the subscript(FG#) and data from interviews with the subscript(I#).
Balancing deliberate and emergent change
Our participants described juggling attempts to develop new tools and programs that adhered to the principles while also attempting to adapt existing programs to meet deadlines. Some participants spoke explicitly about this tension, calling it a “kind of dance between emergence vs having the corporate structure tell you”(I16) what kind of work to do. In the case of emergence, participants focused on the complex nature of the planned change: “in enacting [the] principles, we don’t want to be tokenistic. We don’t want to appropriate. We want to be meaningful, we want to be thoughtful.”(FG1) Rather than immediate implementation of existing educational interventions, participants acknowledged “there’s so much learning that needs to happen in order to do the work.”(I1) Working from this approach, some expressed a need for both time to reflect and a network of resources: “to do that work justice, to not be not accidentally inappropriate or tick box [or] tokenistic, we need some help.”(FG1) In contrast, clearly focused systemic efforts towards a unified goal, or for partners to simply be “informed that ‘those are the principles’,”(FG1) may have “helped all teams be in better alignment from the start.”(I5) Our participants attempted to build tools that and programs that reflected the principles despite early ambiguity about both the principles themselves and the resources available to support the work.
Recognizing time pressure
Tight timelines pressed the balance of the work towards directed implementing tools (such as needs assessment streaming) and programs (such as professional development sessions) before optimal space for deep reflection about the principles could be afforded. When our participants were considering whether to implement existing tools vs attempting a principles-focused needs assessment, consultation process, or fundamental re-thinking “there were a lot of things that we couldn’t do, [that] we had to scratch, because timeline-wise we couldn’t do it.”(I2) Working on a large project with multiple partners led to compromises on implementation because “we would get a lot of requests and the deadline would be, like, tomorrow.”(I1) With the recognition that “enabling emergence is… very, very hard when the deliverables are so tight,”(I14) our participants leaned towards modifying existing tools rather than reimagining the tools to attend specifically to the TPC principles “because we [were] feeling that pressure of delivering and meeting all the milestones and the reporting deadlines.”(I7)
Searching for clarity
When pressed for time, our participants searched for ways to incorporate principles-oriented material into their educational tool development in a modified way. A first priority became clarifying which principles to work towards, to “make sure that we were living the principles and that it wasn’t just implicit in our heads”(I9) while acknowledging “our uncertainty as a team as to what were themes, what were principles, what was the difference? Who’s cross-cutting? Who isn’t? Even that was a bit messy.”(FG1) Identifying, for example, what measures to take to include equity or reconciliation into educational tools, which groups held responsibility for such changes, and whether the guiding principles may change lent significant complexity both “because we don’t know what we don’t know”(FG2) about how to best work towards the principles themselves and because “there was no concrete… documentation in terms of ‘these are the principles, this is what we want to focus on’.”(I3) Our participants described pushing forward building new educational tools under conditions of uncertainty was “a bit chaotic,”(I2) “felt like we were running on the spot,”(FG1) and like “treading water”(I7) while also acknowledging their trust in emergence: “we kept saying, as a team, that clarity will come. We’ve got to trust the process.”(FG1)
Revising expectations
As the project progressed, our participants began to reimagine the meaning and legacy of the work. The complexity of collaborating under conditions of uncertainty leads to recognition that “we were going to have to pivot.”(I2) Initial hopes for multi-partner education interventions shifted towards hopes for future collaboration: “people who were on board and got it, and whose strengths align with the strengths of this team, and the ethos of this project, they’ll continue the work.”(I12) Here, “emergence was helpful because… what we thought was going to be needed wasn’t actually really needed.”(I16) The initial framing of the project, as an act of principles-focused implementation, turned towards building a culture to support future change.
Deferring to future change
Representative quotes depicting core category and subcategories.
Discussion
We fused a grounded theory approach with principles-focused evaluation to explore how CACHE adhered to all five principles of TPC. Our findings suggest that, in national-scale healthcare change initiatives, achieving time-dependent deliverables and allowing for the time-independent emergence necessary for transformative change may inherently be in tension with one another. We propose that health leaders may learn from the CACHE experience and mitigate the impact of this tension through the practice of what we have called organizational critically reflective practice.
Principles under tension
Efforts at reforming modern healthcare practices suffer from a failure to consider the healthcare system as a complex adaptive system.13,23 Healthcare systems can be inherently unpredictable, embedded within other systems, and behave based on internal but ever-evolving rules. 24 Despite this complexity, health leaders are still required to attempt to address inefficiencies, inadequacies, and inequities. In this vein, Team Primary Care was funded based on the premise that the Canadian primary care system is in crisis1,6 and the solution is most likely to come from teamworking based on principles of social accountability; truth and reconciliation; psychological health and safety; equity, diversity, inclusion, and accessibility; and interprofessionalism.5,7,11
A crisis, especially one where health inequities are deepened by the healthcare system itself, 8 obliges reform with evidence of effectiveness. Demonstrating clear evidence of effectiveness in complex adaptive systems remains challenging, 25 but the findings of this study may account for some of these evidentiary challenges. The participants in this study were well-funded experts and national leaders in interprofessional education in healthcare. Nevertheless, even with this expertise, the expectation to demonstrate deliberate action towards a deliverable outcome in the short-term was in direct conflict with their recognition of the time-independent nature of transformational change. As suggested by the action scales model, 26 if evidence of effectiveness is required over the short-term then the findings of this study suggest it may be less likely that said change can be transformative. This inherent tension likely exists because assessing for deliberately sought short-term change does not adequately allow for emergence and poses intractable problems for the logic models and selection timing that characterize attempts to evaluate programs within complex adaptive systems. 25 Health leaders pursuing transformative change may be well served by seeking out resources that nudge collaborating organizations to co-create principles mitigating the tension between emergent and planned work earlier in the process.
Organizational critically reflective practice
Mitigating the impact of this inherent tension may be possible through the practice of what we will call organizational critically reflective practice. Reflexivity is the work of recognizing the limitations of our knowledge and critical reflection is the practice of challenging harmful processes and power dynamics.
27
Our findings support previous management scholarship suggesting that an organization undertaking well-intentioned reform without reflexivity runs the risk of reinforcing the very patterns of thought it seeks to transform.28,29 Organizational critically reflective practice, therefore, describes a process that both precedes and accompanies decisions about principles and outcomes in reform initiatives grounded in social accountability. Before, during, and after reform initiatives we recommend collaborating organizations attempt to tilt the balance of their focus towards emergence by considering the following practices: i. Take deliberate steps to understand how the organization interprets the prospective principles of the change project at the grassroots level early in the process. ii. Reflexively question the organization’s current practices regarding the principles while deliberately seeking out voices within the organization that may be missing. iii. Develop consensus around qualitative outcomes that reflect the principles and continuously reconsider potential harms of the reform during the process.
These suggestions are drawn from the literature on reflexivity and critically reflective practice in interprofessional education and research 30 and reimagine organizational fitness for change as an educational process.14,31 The concept of organizational critically reflective practice relies on valuing trust, respect, humility, and accountability to encourage an inward curiosity about well-intentioned work in areas as sensitive and complex as social accountability; truth and reconciliation; and equity, diversity, inclusion, and accessibility. These practices need to occur at all levels of an organization from values and visions through to operational processes. Organizations new to critically reflective practice may benefit from adapting existing resources in addition to taking the steps described above. The 3Rs (representation; roles/relationships; and responsibilities) 32 is one such resource drawn from literature on critically reflective practice 27 and adapted over the course of the TPC initiative 33 to help organizations continuously recalibrate their efforts. Given the inherent challenges of navigating complex power relationships in socially accountable reform projects, in the absence of critically reflective work, health leaders that rely primarily on short-term deliverables risk implementing reforms that recapitulate the very structures they hope to dismantle.
Conclusions
Health leaders designing reform initiatives should consider the inherent tension between deliverables and transformative change. The findings of this study suggest that deliberately sought short-term change efforts may catalyze future transformation but cannot be expected to deliver transformative change without allowing for emergence. Early attention to emergence of principles through organizational critically reflective practice may mitigate some of the impact of this inherent tension. We hope that organizations collaborating on large healthcare reform initiatives use the insights gleaned from this internal evaluation to focus earlier, more thoroughly, and more iteratively on the individual and organizational learning required to fully realize the principles in future initiatives.
Footnotes
Acknowledgements
The authors wish to thank the TPC Secretariat for their vision; the TPC evaluation team for their guidance; Dr. Michael Patton for his valuable insights; and the CACHE team for their bravery, brilliance, and hard work. The lead author is supported by an investigator award through the Department of Family & Community Medicine (University of Toronto) and a fellowship through AMS Healthcare.
Declaration of conflicting interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Team Primary Care (TPC) was supported by Employment and Social Development Canada through a grant to the Foundation for Advancing Family Medicine, co-led by the College of Family Physicians of Canada and the Canadian Health Workforce Network.
Ethical approval
The study was approved by the University of Alberta Research Ethics Board (Pro00128443).
