Sage Journals: Discover world-class research

Abstract

Billions of dollars are invested annually in leadership development globally; however, few programs are evidence-based, risking adverse outcomes, and wasted time and money. This article describes the novel Inspire Nursing Leadership Program (INLP) and the outcomes-based process of incorporating gold standard evidence into its design, delivery, and evaluation. The INLP design was informed by a needs analysis, research evidence, and by nursing, Indigenous, and equity, diversity, and inclusion experts. The program’s goals include enabling participants to develop leadership capabilities, cultivate strategic community partnerships, lead innovation projects, and connect with colleagues. Design features include an outcomes-based approach, the LEADS framework, and alignment with the principles of adult learning. Components include leadership impact projects, 360-assessments, blended interactive sessions, coaching, mentoring, and application and reflection exercises. The evaluation framework and subsequent proposed research design align to top-quality standards. Healthcare leadership programs must be evidence-based to support leaders in improving and transforming health systems.

Introduction

Internationally, the annual investment in leadership development is estimated at $50 billion.^1,2 Many organizations consider leadership development an obligatory cost,³ a source of competitive advantage,^4–7 and a key retention strategy.^8–12 Organizations across sectors offer internal leadership programs, along with an increasing number of external providers, including business schools, private corporations, and individual consultants.¹³

Evidence of leadership development outcomes

The increasing abundance of leadership training corresponds with evidence that programs can facilitate improved participant outcomes, such as increased confidence, knowledge, skills, capabilities, engagement, well-being, job satisfaction, self-efficacy, and performance.^14–16 Leadership interventions have also been correlated with organizational-level outcomes, including decreased absenteeism, increased staff retention, engagement, motivation, and sense of shared purpose, increased organizational performance, financially and clinically, and improved patient outcomes and patient safety.^14–16 There is also evidence that interventions can demonstrate financial Return-on-Investment (ROI).¹⁷

Uncertain quality of evidence and potential consequences

Despite this evidence and the extensive global investment, there is widespread ambiguity regarding which program designs are linked empirically to outcomes.^1,18–22 Even in academic literature, purported as the most credible source of evidence, the knowledge base is of predominantly poor quality.^15,23–25 This situation can obscure what works best, why, and with what reliability¹⁷, can perpetuate ineffective practices, and can be confusing, misleading, and potentially harmful.^22,23 Equally, designing interventions based on questionable evidence (or none) risks program underperformance or, worse, adverse outcomes.^22,26–30 For example, despite typically favourable program evaluations,^22,31,32 reports of the proportion of trainees applying their learning to the workplace—the enterprise’s currency of success¹³—are as low as 5%.¹⁹ The stakes of substandard interventions for organizations heighten when budgets are strained, priorities are competing, and capacity is limited.^33–35

Gold standard program elements and research quality

Establishing gold standards

To address the knowledge gap and to isolate evidence-based elements of leadership development programs linked to outcomes, the systematic review by Geerts et al.¹⁵ introduced a novel methodology. This involved building on the foundational and most-cited review of medical leadership development, that by Frich et al.,³¹ and applying a validated instrument, the Medical Education Research Study Quality Instrument (MERSQI), to all relevant studies from 2007 to 2016 (K = 25) to assess their quality.

These authors were the first in the field to create tiered categories of study quality (bronze, silver, and gold) based on MERSQI scores, and to present elements of program design, delivery, and evaluation exclusively from top-quality studies (the silver and gold categories, Table 1).

Table 1.

Gold standard elements of program design, delivery, and evaluation incorporated into the Inspire Nursing Leadership Program (INLP).

Category	Element^a	How elements are included in the INLP
Design	Conduct a pre-program needs and gaps analysis	The original design is based on:
		- Needs and gaps expressed by client organizational representatives,
		- CCHL’s ongoing research, and
		- A CNA report from their Dorothy Wylie Health Leadership Institute.^b
	Apply an outcomes-based design approach	We applied the 7-step outcomes- and evidence-based design process described in Geerts et al.¹⁵
		1) Conduct a needs and gaps analysis and establish an empirical foundation,
		2) Select ensuing desired outcomes,
		3) Select explicit program goals,
		4) Select participants intentionally to address the needs and gaps,
		5) Select program details and incorporate evidence-based elements according to their suitability to achieve identified targets,
		6) Develop a robust evaluation framework, and
		7) Embed an application of learning/training transfer strategy.
	Clearly describe explicit goals for the program	INLP goals enable participants to:
		- Develop an in-depth understanding of the LEADS framework and of how individuals, teams, and organizations can apply it effectively;
		- Develop their leadership capabilities, including self-awareness, and to increase their confidence as leaders;
		- Cultivate strategic and enduring relationships with key internal and community partners;
		- Implement innovative quality improvement projects; and
		- Form connections with each other and with a larger community of health leaders through the CCHL Circle Community for Practice.
	Participants select their own goals	At the outset, participants:
		- Select their own goals and desired outcomes for the INLP;
		- Identify 3 LEADS capabilities to develop during the program, aligned to their Leadership Development Plans (LDPs); and
		- Select goals and desired outcomes for their Leadership in Action (LiA) projects.
	Embed a leadership capability framework in the curriculum	The LEADS in a Caring Environment Framework is:
		- The curricular foundation and common leadership language of the program,
		- The basis for the 360-assessments and LDPs, and
		- Referenced explicitly in the LiA project details.
	Incorporate the principles of adult learning³⁶	(Pre-program): Motivation to learn: Participants apply or are recommended by their supervisors, which enhances their motivation.^35,37
		1. Self-directed: Participants select their own goals and desired outcomes for the program and their own LiAs;
		2. Participants’ experience as the basis: Participants’ expertise is considered a valued resource which enhances program learning;
		3. Content that is practical and relevant to participants: Having an RN design the full program, informed by a comprehensive needs and gaps analysis, cutting-edge research, and input from nurse leaders and Subject Matter Experts (SMEs), was intended to enhance validity and perceived relevance among program participants, as is prioritizing nurses as program faculty; and
		4. Outcomes-based: As described above, both in terms of how the INLP was designed and how each iteration is delivered.
	Embed an application of learning/training transfer strategy	Participants apply their learning to work through application exercises in between modules and through their LiAs. They are also held accountable by having to report their progress to program faculty and to their accountability teams mid-way, at the finale, and sometime after.
	Consider calculating the program return-on-investment	Tangible LiA outcomes, particularly economic, are helpful, complemented by self-reported and external rater-assessed outcomes.
Faculty	Mix: Internal and external to the organization and expert/practitioner	The design team included those with international academic expertise, CNA advisors, EDI specialists, Indigenous expert advisors, other SMEs, and organizational nurse leaders. Nurse faculty are prioritized, featuring the preferred mix.
Delivery and program components	Components/developmental activities/learning methods:³⁸ - Individual; - Educational; - Experiential; - Relational; - Resources; and - Credentials.	Individual: LEADS 360-assessments, debriefed by a LEADS Certified Coach, and an LDP;
		Educational: Interactive in-person and online sessions, small group discussions, guest speakers, and case study analyses;
		Experiential: Role plays, LiAs, and application exercises;
		Relational: Action learning triads, individual and peer coaching, mentoring, networking, and engagement in the CCHL Circle Community for Practice;
		Resources: Readings, media, and materials; and
		Credentials: The Certification in Nurse Leadership (Canada) and the Certified Health Executive (CHE).
Outcomes	Desired outcomes based on the Kirkpatrick framework levels:^c Satisfaction (1), attitudes and perceptions (2a), knowledge and skills (2b), subjective behaviour change (3A), objective behaviour change (3b), organizational change (4a), and benefit to patients (4b)	The INLP desired outcomes are:
		- Increased self-awareness and confidence as a leader (2a),
		- Knowledge, skills, and capabilities (2b), including developing and sustaining partnerships with local community members and leading improvement initiatives,
		- Behaviour change (3A and 3b),
		- Organizational change (4a),
		- Benefit to patients (4b), and
		- Benefit to communities, with consideration for economic/financial outcomes and sustainability.
Evaluation	Focus of evaluation: Both the program and participant outcomes	The evaluation framework includes assessments of:
		- Participant outcomes (as described above, plus unanticipated outcomes); and
		- The program itself (satisfaction overall and with its components for quality control, as well as participants’ assessments of which elements were most attributable to achieving certain outcomes).
	Type of data collected: Both quantitative and qualitative	Quantitative data are collected through Likert-scale participant self-ratings and external raters’ assessments, as well as through tangible LiA outcomes;
	Type of data collected: Both quantitative and qualitative	Qualitative data are gathered through free-text survey responses;
	Type of data collected: Both objective and subjective	Objective data involve tangible or statistical LiA project outcomes, external rater assessments, and any offered by participants through free-text responses; and
	Type of data collected: Both objective and subjective	Subjective data include participant self-reported outcomes and external rater free-text responses.
	Raters: Multiple (e.g., self, peer, program faculty, workplace supervisor)	The following raters are included in the 360-assessments and in evaluating participants’ leadership capabilities: Participants themselves (self), peers, direct reports, and workplace supervisors.
	Control group	No non-intervention control group was included in the initial cohort; however, this is possible in future iterations.
	When data were collected: Pre/baseline, post, and post-post^d	The evaluation frameworks for the program itself and for participant outcomes, including those from the LiAs, involve formal evaluations at outset (baseline), mid-way, at the finale (post), and 6 to 9 months after the finale (post-post).

^a“Elements” refers to components of leadership development programs from top-quality studies identified in Geerts et al. (2020)¹⁵ and Lyons et al. (2020)¹⁶ (k = 30).

^bLankshear, S. Dorothy Wylie Health Leadership Institute: Considerations for future program content and delivery. The Canadian Nurses Association (CNA); 2022:17.

^cKirkpatrick and Kirkpatrick’s (2006) four-part model categorizing the reported outcomes of training programs.

^d“Baseline” = at the outset, “post” = at the finale, “post-post” = sometime after the finale.

In summary, “gold standard” refers to (a) the methodological characteristics of highest standard research studies, whose findings are empirically anchored, and (b) the best scientific evidence regarding elements of leadership development programs that have been linked to outcomes.

Expanding the credible knowledge base

To augment the breadth of gold standard evidence, Lyons et al.¹⁶ applied MERSQI and the Joanna Briggs Institute (JBI) Critical Appraisal Tool^39,40 to 117 included studies from 2000 to 2020. This review found no correlation between program curriculum content and improved outcomes,^7,41 but identified a statistically significant correlation between having mixed faculty in terms of (a) internal/external to the organization and (b) experts/practitioners and organizational outcomes.¹⁶

These two reviews provide unique clarity regarding the most reliable evidence in the field based on 30 high-quality studies. This exclusive subset can inform future research and programming and potentially enhance their quality and maximize their impact.¹⁵

Context (professional and academic) and article purpose

The need for robust leadership development is particularly urgent in healthcare, especially since internationally, the sector is plagued by a severe Human Resource (HR) crisis, marked by massive staff and leader shortages, high turnover, expected retirements,⁴² and droves of rapidly promoted leaders who have not received adequate—or any—preparation for those roles.^8,22,43 Many of these “battlefield promotions” are early- to mid-career nurse leaders, who, having survived their trial by immersion, are now requesting formal training.^44,45

The need to provide leaders in healthcare and other sectors with evidence-based programs^20,22,46 is mirrored by a further knowledge gap regarding how top-quality evidence can be integrated into programs.

The Canadian Nurses Association (CNA) and the Canadian College of Health Leaders (CCHL), partner organizations, addressed this challenge when healthcare organizations approached them requesting leadership development specifically for nurses. In response, the design team created a customized⁴⁷ novel intervention, called the Inspire Nursing Leadership Program (INLP), which had not yet been delivered when this article was first written.

The purpose of this article is to describe the outcomes- and evidence-based process that the INLP creators implemented, including how they incorporated gold standard elements of program design, delivery, and evaluation, as well as of top-quality research, into the program design.

To the best of our knowledge, this is the only published account of this kind of application of evidence to either program or research designs. The intent is to provide a credible foundation for the design of other programs and studies in healthcare and beyond and to potentially improve their quality and impact.

Design of the Inspire Nursing Leadership Program

The core design team of the INLP included the CCHL Vice-President of Research and Leadership Development, the CCHL Senior Vice-President of Professional and Leadership Development, and a CCHL Faculty, with input provided by others described below. This section outlines how the team applied the 7-step outcomes-based design approach created by Geerts et al.¹⁵ and incorporated each of the gold standard elements listed in Table 1. The latter includes alignment with Knowles’s principles of adult learning, which are: (pre-program) motivation to learn, self-directed, participants’ experience as the basis, relevant and practical content, and outcomes-based.³⁶

1. Conduct a needs and gaps analysis and establish an empirical foundation

Step 1 involved conducting a comprehensive needs and gaps analysis based on meetings with representatives from client organizations, to ensure the program would address the prospective participants’ professional context precisely.^22,48

To establish an evidence foundation, two authors conducted a literature search for reviews of nurse leadership development from 2010 to 2024, guided by a University of Manitoba specialist librarian. This resulted in 14 unique reviews. Since none systematically isolated the best available evidence of program elements linked to outcomes, the team consulted Geerts et al.¹⁵ and Lyons et al.¹⁶ Despite their medical focus, many of included studies in these reviews included nurse participants. Further, research evidence confirms that empirically based components of effective leadership development translate considerably across sectors and professions within healthcare.^15,49

With this empirical foundation, the lead author drafted an initial program. To enhance its validity among nurse leaders, a Registered Nurse (RN) and CCHL Leadership Faculty designed the full program, with input from several Subject Matter Experts (SMEs), including from the CNA Indigenous Council, the Canadian Indigenous Nurses Association, and an Equity, Diversity, and Inclusion (EDI) specialist.

The purpose of the INLP is to increase the leadership capacity of nurse leaders in their organizational and community contexts, including to facilitate quality and system improvement.

2. Select program desired outcomes

Step 2, informed by step 1, involved selecting desired outcomes according to an extended version of the Kirkpatrick framework⁵⁰ to achieve this purpose. INLP desired outcomes include enhancing nurse leaders’:

- Self-awareness and confidence as leaders (Level 2A),

- Leadership capabilities (knowledge and skills, Levels 2B, and behaviours (Level 3)), including cultivating partnerships with local community members and leading improvement initiatives,

- Familiarity with a leadership framework as the common language to facilitate change (benefit to the organization, Level 4A), and

- Capacity to positively impact patients (Level 4B), staff, and communities, with consideration given to economic and sustainability impact.

A key curricular foundation is the LEADS in a Caring Environment Leadership Capability Framework (LEADS),⁵¹ which is comprised of 5 domains and 20 capabilities, with behavioural descriptors at 4 levels of leadership for each capability. This framework is the basis for much of the CCHL programming, including its 360-assessments, and is reported to be the most widely used resource of its kind in healthcare in Canada.^52,53 Integrating LEADS organization-wide has been empirically correlated in multiple sites with improved outcomes at the individual and organizational levels.⁵⁴

3. Select explicit program goals

To achieve the desired outcomes, the design team identified 5 program goals (Table 1), which are stated prior to registration and at the beginning of the program, are revisited throughout, and are included explicitly in the evaluation framework. At the outset, INLP participants also select personalized goals and desired outcomes,⁵⁵ including 3 leadership capabilities to develop, and goals and desired outcomes for their Leadership in Action (LiA) projects.^3,56 Personalization enhances perceived relevance and application of learning.^57–59

4. Select participants intentionally

The INLP features cohorts of 15–24 participants and is intended for early- to mid-career nurse leaders. Participants are usually nominated by their supervisors or apply and are selected based on their leadership potential to meet the priorities identified in step 1.^35,37

5. Select program details (structure, content, faculty, and components) and prioritize evidence-based elements

Step 5 involved incorporating the gold standard elements (Table 1) and selecting the corresponding program details to best achieve the program goals.

Structure

Structurally, the INLP involves 3 months of blended learning,⁶⁰ including 8 hour-long synchronous virtual sessions and a central 2-day residential component. The total time commitment, including asynchronous requirements inbetween modules, is 45–50 hours. The INLP can be delivered as an open-enrolment or organization-specific intervention.¹⁶

Content

The content covers the 5 domains of the LEADS Framework: Lead self, engage others, achieve results, develop coalitions, and systems transformation, with a focus on supporting participants to design and implement LiAs. EDI is interwoven explicitly throughout.

Faculty

Faculty with a nursing background are prioritized, along with a mix of internal/external and expert/practitioner.¹⁶

Delivery

In addition to effective facilitation techniques, the delivery aligns with the principles of adult learning.³⁶ This involves self-directed aspects, involving participants’ insights and experience as valuable learning resources throughout, and application exercises.

Components

Of all gold standard components, the INLP includes the following (based on a novel framework):³⁸

- Individual: A pre-program LEADS 360-assessment,^17,47 debriefed by a LEADS Certified Coach,^17,47 which is key for developing self-awareness,^20,61 a Leadership Development Plan (LDP),⁴⁷ and reflection exercises;⁶²

- Educational: Interactive in-person and online learning sessions^63,64 and small group discussions,⁶⁵ guest speakers,⁶⁵ and case study analyses;⁶⁵

- Experiential: Role plays,⁶⁴ leadership impact projects (LiAs),^22,64,66 and application exercises inbetween modules;⁶⁵

- Relational: Action learning triads,⁶⁷ individual^17,22,66 and peer coaching,⁶⁸ mentoring,^22,69,70 networking,⁷¹ and engagement in the CCHL Circle Community for Practice;⁷²

- Resources: Readings and materials;⁶⁵ and

- Credentials: Graduates qualify for the CNA’s Certification in Nursing Leadership (Canada) (CNL(C)) and the Certified Health Executive (CHE) credentials.⁶⁷

Along with selecting components specifically based on their intended efficacy in achieving program goals, it is important to provide a variety to appeal to different learning preferences.^59,61,73

6. Develop a robust complementary evaluation framework

Program evaluations typically only assess participant satisfaction (Level 1),¹ but a robust evaluation framework can enhance outcomes by providing focus, motivation, and accountability^5,32,74 and can demonstrate ROI.⁷⁵ For example, tangible LiA outcomes, particularly economic, such as cost savings, can enable organizations to calculate the program ROI, measured against the expense and opportunity cost of the program.

The INLP evaluation framework assesses both the program itself and participant outcomes at different levels (Table 2), involving subjective and objective data at the outset, mid-way, at the finale, and 6 to 9 months afterwards.¹⁵

Table 2.

The INLP evaluation framework: Participant outcomes.

Inspire Nursing Leadership Program evaluation framework: Participant outcomes (for each iteration)
No.	When	What	Who	What	Description
No.	Timing	Assessment	Raters (self, sup., and peer)^a	Types of data: subj., obj., qual., quant., and open^b	Assessment details
1	First session	Baseline goals and desired outcomes for the program, leadership capabilities, and LiAs	Self	Both	Participants select:
					1) Personalized program goals and desired outcomes, aligned with the program goals;
					2) Three leadership capabilities to develop during the program with specific objectives and metrics, aligned to their LDPs; and
					3) Goals and desired outcomes for their LiAs (project and personal), including tangible Levels 4A and 4B outcomes, plus benefit to community, economic, and sustainability-level outcomes.
2	First session	Baseline leadership capabilities assessments and LiA data	Self, sup., and peers (3) from the 360-assessment report^c	Both	Participants record ratings (self, sup., and peers) from their 360-assessment Report for the 3 capabilities they choose to develop during the program, as well as relevant tangible baseline LiA project data.
3	Residential component	Mid-way progress assessments	Self	Both	Participants self-assess their own progress regarding 1), 2) and 3) above, and share the results with program faculty and their teams. Part’s^d consider revising goals and desired outcomes based on progress and share any alterations with their teams.
4	Finale	End-of-program assessment	Self, sup., and peers (3, same as before)	Both	Participants self-assess their progress regarding 1), 2) and 3) above. External raters also assess 2). Participants and teams have an opportunity to provide open-ended feedback.
5	Sometime after (6 to 9 months later)	Sustained, diminished, or extended assessment	Self, sup., and peers (3, same as before)	Both	Same as above. Part’s additionally propose perceived correlations between program components and achieved outcomes. Finally, participants have the opportunity to offer unanticipated outcomes that were achieved.

^aSelf = participants; sup. = workplace supervisor of a participant; peer = a designated colleague of a participant.

^bSub. = subjective; obj. = objective; qual. = qualitative; quant. = qualitative; open = open-ended (free-text).

^cThese assessments are drawn from the 360-assessment report, rather than a second assessment by the same people at this time.

^dPart’s = participants.

7. Embed an application of learning/training transfer strategy

The quintessential hallmarks of successful development are improved tangible outcomes and yet, the process of participants applying their learning to work is not automatic or guaranteed.^7,13 Without effective transfer strategies, even outstanding programs can underperform or fail.^7,18

Accordingly, the INLP requires participants to apply their learning through exercises inbetween modules and through the LiAs, experiences which are debriefed afterwards and are reinforced by reflection activities. These iterative learning-cycle steps⁷⁶ can enhance learning, since participants can discuss their experiences and lessons learned after each cycle, with support from program faculty. Accountability is also augmented by requiring participants to report their progress to program faculty and to their accountability teams (see below) throughout.

Strategies to enhance outcomes and support organizational receptivity to change include involving participants’ colleagues through the 360-assessments and accountability teams, as well as requiring participants to engage with colleagues and community members before launching their LiAs to ensure that the importance and goals are shared. These measures can also enhance the support others provide.^77,78

Gold standard evaluation and research design

The following sections demonstrate how the gold standard elements were incorporated into the INLP evaluation framework (Table 2) and into a potential subsequent research design.

Evaluation framework: Program

Participants’ evaluation of the program is preceded by creating their own personalized goals. Mid-way, they self-assess progress toward achieving program and personal goals, as well as rating their satisfaction with the program and its components thus far. These evaluations are repeated during the final session and sometime after, along with enabling participants to propose program improvements. Participants are also asked to identify any program components that they perceive were most directly linked to specific improved outcomes.

Evaluation framework: Participant outcomes and accountabilities

Similarly, evaluation regarding participant outcomes is preceded by establishing baseline scores for the three designated leadership capabilities that they aim to develop and tangible LiA baseline data. INLP participants identify a core accountability team, called “the team”, ideally including their workplace supervisor, program colleagues, and three non-program colleagues, and share results periodically. This transparency establishes an informal social contract and can increase outcomes,^7,79 as well as prompting the team to offer any requisite support or resources to enable success.^77,78

Participants share their goals and baseline metrics with their teams, followed by their self-ratings mid-way, which can enable course correction and goal revisions or extensions based on progress.^80,81

At the program conclusion, participants repeat the process. Team members also rate the frequency with which participants demonstrate their 3 designated leadership capabilities effectively, along with providing qualitative feedback. Participants share the synthesized results with the program faculty and with their teams, along with any revised goals and metrics. They repeat this process 6–9 months later.

External ratings and tangible LiA outcomes provide objective data, which can demonstrate the program’s ROI and can increase external validity and minimize bias, compared to self-ratings alone.⁸²

Finally, at the finale and afterwards, participants may propose any unanticipated outcomes that were achieved, along with any supporting evidence.

Foundation for a subsequent high-quality research design

The majority of research in the field is of unreliable quality, which is problematic.^15,23–25 Robust study designs, conversely, can advance knowledge and potentially enhance the quality and impact of interventions,^15,22 minimizing the notorious research/practice divide.^83,84

Consequently, the INLP team embedded key elements in the design of the program to ensure that once it had been delivered and data had been collected, a subsequent research study would align with the criteria for gold standard quality in Geerts et al.¹⁵ These elements included collecting data from multiple iterations of the program to address generalizability, securing high response rates by administering evaluations during synchronous sessions, collecting objective data pre- and post-intervention through external raters and tangible LiA outcomes, targeting benefit to patients outcomes (Level 4B), and exploring the relationship among program variables (program components and outcomes).

In addition to potentially contributing to the scholarly knowledge base, aligning to top-quality research criteria bolstered the INLP design.

Discussion and limitations

The purpose of this article is to describe the process by which the INLP design team applied an outcomes-based approach to create the program, including how they incorporated gold standard evidence and components of top-quality research, concomitantly. This article is the first to demonstrate these applications, which is vitally important in healthcare, given the high stakes, considerable investment of money, time, and resources, and the predominantly unreliable knowledge base.

Despite limitations associated with publishing before initial program delivery and evaluation, given the robust evidence-based design, implementation lessons are anticipated to reveal nuances, rather than foundational flaws. First iterations of the program will yield early data, lessons, and opportunities to refine the design accordingly and to report these evolutions when publishing.

A second potential limitation concerns the extent to which lessons from this novel program for nurse leaders in Canada might apply to other contexts, such as different levels of leadership, professions, sectors, and geographical locations. Research indicates that credible evidence has the potential to be considerably generalizable.

Further research and practice

Further research could identify other gold standard studies, including those from other sectors, to expand the existing pool, and to analyze the extent to which key principles translate effectively to different contexts. Future research and practice could also explore alternative applications of the gold standard elements in various contexts, as well as which program components and formats correlate with specific desired outcomes, supported by objective data. It would also be interesting to explore how artificial intelligence can enhance development and training programs. Finally, beyond individual interventions, investigating evidence-based strategies for organization-wide leadership integration, development, and support toward becoming a Leadership Organization would be beneficial.

Conclusion

There is an urgent need to support leaders, particularly in healthcare, with programs based on high-quality research evidence. This calibre is crucial, given the context of the current health HR crisis, the risk of potentially wasted spending and other adverse consequences, and the parallel opportunity to potentially improve and transform health systems.

Footnotes

Acknowledgements

The authors would like to thank Tim Guest and Misty Fortier (CNA), Brenda Lammi and Alain Doucet (CCHL), Caroline Monnin (University of Manitoba), and Dr. Ibrahim Jahun (University of Manitoba) for their excellent contributions, as well as the guest and managing editors and the anonymous peer reviewers for their insightful suggestions.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethical approval

Institutional Review Board approval was not required.

ORCID iDs

Jaason M. Geerts

Sharon Bishop

References

Kellerman

. The End of Leadership. New York, NY: Harper Collins; 2012.

Kellerman

. Professionalizing Leadership. Oxford: Oxford University Press; 2018.

Samani

Thomas

. Your leadership development programme needs an overhaul. Harv Bus Rev. 2016;5:1-5. https://hbr.org/2016/12/your-leadership-development-program-needs-an-overhaul

Amagoh

. Leadership development and leadership effectiveness. Manag Decis. 2009;47(6):989-999.

Blume

Ford

Bladwin

Huang

. Training transfer: a meta-analytic review. J Manag. 2010;38(4):1065-1105.

Day

. Leadership development: a review in context. Leader Q. 2001;11(4):581-613. https://www.scopus.com/inward/record.url?eid=2-s2.0-0001159390&partnerID=40&md5=05a7d320be1f4cccace5c50c55e4228a

Salas

Tannenbaum

Kraiger

Smith-Jentsch

. The science of training and development in organizations: What matters in practice. Psychol Sci Publ Interest. 2012;13:74-101.

Geerts

Yiu

. The great optimization: What Canada’s health care leaders have learned about the new future of work - and what other companies can take away. The Globe and Mail. https://www.theglobeandmail.com/opinion/article-how-can-companies-thrive-in-the-new-future-of-work-by-understanding/. Published November 19, 2022. Accessed February 7, 2022.

Udod

MacPhee

Baxter

. Rethinking resilience: Nurses and nurse leaders emerging from the post-COVID-19 environment. J Nurs Adm. 2021;51(11):537-540. doi:10.1097/NNA.0000000000001060

10.

Almost

Mildon

. R-E-S-P-E-C-T: A key to nurse retention. Nurs Leadersh Tor Ont. 2022;35(2):12-28. doi:10.12927/cjnl.2022.26876

11.

Cowden

Cummings

Profetto-McGrath

. Leadership practices and staff nurses’ intent to stay: A systematic review. J Nurs Manag. 2011;19(4):461-477. doi:10.1111/j.1365-2834.2011.01209.x

12.

Hedenstrom

Spiva

Thurman

, et al. Developing and evaluating the effectiveness of a nursing leadership mentoring pilot program. Nurs Adm Q. 2023;47(2):173-181. doi:10.1097/NAQ.0000000000000557

13.

Lacerenza

Reyes

Marlow

Joseph

Salas

. Leadership training design, delivery, and implementation: A meta-analysis. J Appl Psychol. 2017;102(12):1686-1718. doi:10.1037/apl0000241

14.

Gagnon

Rithchie

Lynch

, et al. Job Satisfaction and Retention of Nursing Staff: The Impact of Nurse Management Leadership. Ottawa, ON: Canadian Health Services Research Foundation; 2006.

15.

Geerts

Goodall

Agius

. Evidence-based leadership development for physicians: A systematic literature review. Soc Sci Med. 2020;246:1-17. doi:10.1016/j.socscimed.2019.112709

16.

Lyons

George

Galante

, et al. Evidence-based medical leadership development: A systematic review. BMJ Lead. 2021;5(3):206-213. doi:10.1136/leader-2020-000360

17.

Orme

Campbell

. How leadership training saves money ‘service line leadership’ at Nottingham University Hospitals. BMJ Lead. 2019;3(2):29-36. doi:10.1136/leader-2018-000132

18.

Beer

Finnström

Schrader

. Why leadership training fails - and what to do about it. Harv Bus Rev. 2016;94:50-57. Published online October 2016.

19.

Gilpin-Jackson

Bushe

. Leadership development training transfer: A case study of post-training determinants. J Manag Dev. 2007;26(10):980-1004.

20.

Straus

Soobiah

Levinson

. The impact of leadership training programs on physicians in academic medical centers: A systematic review. Acad Med. 2013;88(5):1-15.

21.

DeRue

Wellman

. Developing leaders via experience: The role of developmental challenge, learning orientation, and feedback availability. J Appl Psychol. 2009;94(4):859-875. doi:10.1037/a0015317

22.

Cummings

Lee

Tate

, et al. The essentials of nursing leadership: A systematic review of factors and educational interventions influencing nursing leadership. Int J Nurs Stud. 2021;115:103842. doi:10.1016/j.ijnurstu.2020.103842

23.

Geerts

. Optimal Leadership Development for Professionals. [Unpublished doctoral thesis]. Cambridge: University of Cambridge;2018.

24.

Stephenson

Vaa

Wang

, et al. Conference presentation to publication: A retrospective study evaluating quality of abstracts and journal articles in medical education research. BMC Med Educ. 2017;17(1):193. doi:10.1186/s12909-017-1048-3

25.

Reed

Cook

Beckman

Levine

Kern

Wright

. Association between funding and quality of published medical education research. J Am Med Assoc. 2007;298(9):1002-1009.

26.

Boaden

. Leadership development: does it make a difference? Leader Organ Dev J. 2006;27(1):5-27. doi:10.1108/01437730610641331

27.

Rousseau

. Is there such a thing as “evidence-based management”. Acad Manag Rev. 2006;31(2):256-269. https://www.scopus.com/inward/record.url?eid=2-s2.0-33646405216&partnerID=40&md5=d4c5ac7466dd33353a5933121a467890

28.

Pfeffer

. Leadership BS. New York, NY: HarperCollins; 2015.

29.

Malling

Mortensen

Bonderup

Scherpbier

Ringsted

. Combining a leadership course and multi-source feedback has no effect on leadership skills of leaders in postgraduate medical education. An intervention study with a control group. BMC Med Educ. 2009;9(72):1-7.

30.

Kwamie

van Dijk

Agyepong

. Advancing the application of systems thinking in health: Realist evaluation of the Leadership Development Programme for district manager decision-making in Ghana. Health Res Pol Syst. 2014;12(29):1-12.

31.

Frich

Brewster

Cherlin

Bradley

. Leadership development programs for physicians: a systematic review. J Gen Intern Med. 2015;30(5):656-674. doi:10.1007/s11606-014-3141-1

32.

Steinert

Naismith

Mann

. Faculty development initiatives designed to promote leadership in medical education. A BEME systematic review: BEME guide no. 19. Internet J Med Technol. 2012;34(6):483-503. doi:10.3109/0142159X.2012.680937

33.

Leimbach

. Leadership development in the age of disruption. Training. https://trainingmag.com/leadership-development-in-the-age-of-disruption/. Published Online May 9, 2023. Accessed September 15, 2023.

34.

Antonakis

Fenley

Liechti

. Can charisma be taught? Tests of two interventions. Acad Manag Learn Educ. 2011;10(3):374-396. doi:10.5465/amle.2010.0012

35.

MacPhail

Young

Ibrahim

. Workplace-based clinical leadership training increases willingness to lead. Leader Health Serv. 2015;28(2):100-118.

36.

Knowles

. Andragogy in Action. Hoboken, NJ: Jossey-Bass; 1984.

37.

Berman

Bowman

West

Wart

. Human Resource Management in Public Service: Paradoxes, Processes, and Problems. 3rd ed. Berlin: Sage; 2010.

38.

Geerts

. Optimising leadership development: An evidence-informed framework. Published online, 2019.

39.

Lockwood

Munn

Porritt

. Qualitative research synthesis: methodological guidance for systematic reviewers utilizing meta-aggregation. Int J Evid Base Healthc. 2015;13(3):179-187. doi:10.1097/XEB.0000000000000062

40.

Hannes

Lockwood

Pearson

. A comparative analysis of three online appraisal instruments’ ability to assess validity in qualitative research. Qual Health Res. 2010;20(12):1736-1743. doi:10.1177/1049732310378656

41.

Saks

Belcourt

. An investigation of training activities and transfer of training in organizations. Hum Resour Manag. 2006;45:629-648.

42.

Foots

Swiger

Orina

, et al. Recommendations from a systematic review of leadership development to support a new nursing practice model. JONA J Nurs Adm. 2023;53(12):661-667. doi:10.1097/NNA.0000000000001363

43.

Ben-Ahmed

Bourgeault

. Sustaining the Canadian nursing workforce: Targeted evidence-based reactive solutions in response to the ongoing crisis. Nurs Leadersh Tor Ont. 2023;35(4):14-29. doi:10.12927/cjnl.2023.27076

44.

Gab Allah

. Challenges facing nurse managers during and beyond COVID-19 pandemic in relation to perceived organizational support. Nurs Forum. 2021;56(3):539-549. doi:10.1111/nuf.12578

45.

Forster

Patlas

Lexa

. Crisis leadership during and following COVID-19. Can Assoc Radiol J. 2021;71(4):421-422. doi:10.1177/0846537120926752. Published online May 12, 2020.

46.

McKimm

Swanwick

. Leadership development for clinicians: What are we trying to achieve? Clin Teach. 2011;8:181-185.

47.

Pradarelli

Jaffe

Lemak

Mulholland

Dimick

. A leadership development program for surgeons: First-year participant evaluation. Surgery. 2016;160(2):255-263. doi:10.1016/j.surg.2016.03.011

48.

Galuska

. Education as a springboard for transformational leadership development: Listening to the voices of nurses. J Cont Educ Nurs. 2014;45(2):67-76. doi:10.3928/00220124-20140124-21

49.

Edler

Adamshick

Fanning

Piro

. Leadership lessons from military education for postgraduate medical curricular improvement. Clin Teach. 2010;7(1):26-31. doi:10.1111/j.1743-498X.2009.00336.x

50.

Kirkpatrick

. Evaluating Training Programs. 3rd ed. Oakland, CA: Berrett Koehler Publishers; 2006.

51.

Dickson

Briscoe

Fenwick

MacLeod

Romilly

. LEADS in a caring environment: Health leadership capabilities framework. https://leadscanada.net/site/framework. Published online, 2007.

52.

Straatman

Matlow

Dickson

Aerde

Gautam

. The times are changing: Articulating the requisite leadership behaviours needed to embed equity, diversity and inclusivity into our healthcare systems. BMJ Lead. 2023;7(Suppl 2):e000767. doi:10.1136/leader-2023-000767

53.

Dickson

Tholl

, eds. Bringing Leadership to Life in Health: LEADS in a Caring Environment: Putting LEADS to Work. 2nd ed. Berlin: Springer Nature; 2020.

54.

Viches

Fenwick

Harris

Lammi

Racette

. Changing Health Organizations with the LEADS Leadership Framework: Report of the 2014-2016 LEADS Impact Study. Victoria, BC: Fenwick Leadership Explorations, the Canadian College of Health Leaders, and the Centre for Health Leadership and Research, Royal Roads University; 2016.

55.

Fassiotto

Maldonado

Hopkins

. A long-term follow-up of a physician leadership program. J Health Organisat Manag. 2018;32(1):56-68. doi:10.1108/JHOM-08-2017-0208

56.

Ardts

JCA

van der Velde

MEG

Maurer

. The influence of perceived characteristics of management development programs on employee outcomes. Hum Resour Dev Q. 2010;21(4):411-434.

57.

Blumenthal

Bernard

Fraser

Bohnen

Zeidman

Stone

. Implementing a pilot leadership course for internal medicine residents: Design considerations, participant impressions, and lessons learned. BMC Med Educ. 2014;14(257):1-11.

58.

Edmonstone

. Evaluating clinical leadership: A case study. Leader Health Serv. 2009;22(3):210-224.

59.

Leskiw

Singh

. Leadership development: Learning from best practices. Leader Organ Dev J. 2007;28(5):444-464. doi: 10.1108/01437730710761742

60.

Carney

Eiff

Green

, et al. Transforming primary care residency training: A collaborative faculty development initiative among family medicine, internal medicine, and pediatric residencies. Acad Med J Assoc Am Med Coll. 2015;90(8):1054-1060. doi:10.1097/ACM.0000000000000701

61.

McCauley

. Leader Development: A Review of Research. Greensboro, NC: Center for Creative Leadership; 2008.

62.

Bearman

O’Brien

Anthony

, et al. Learning surgical communication, leadership and teamwork through simulation. J Surg Educ. 2012;69(2):201-207. doi:10.1016/j.jsurg.2011.07.014

63.

Ten Have

ECM

Nap

Tulleken

. Quality improvement of interdisciplinary rounds by leadership training based on essential quality indicators of the Interdisciplinary Rounds Assessment Scale. Intensive Care Med. 2013;39(10):1800-1807. doi:10.1007/s00134-013-3002-0

64.

Patel

Brennan

Metlay

Bellini

Shannon

Myers

. Building the pipeline: The creation of a residency training pathway for future physician leaders in health care quality. J Assoc Am Med Coll. 2015;90(2):185-190. doi:10.1097/ACM.0000000000000546

65.

Kuo

Thyne

Chen

West

Kamei

. An innovative residency program designed to develop leaders to improve the health of children. Acad Med. 2010;85(10):1603-1608.

66.

McNamara

Fealy

Casey

, et al. Mentoring, coaching and action learning: Interventions in a national clinical leadership development programme. J Clin Nurs. 2014;23(17-18):2533-2541. doi:10.1111/jocn.12461

67.

Agius

Brockbank

Baron

Farook

Hayden

. The impact of an integrated medical leadership programme. J Health Organisat Manag. 2015;29(1):39-54. doi:10.1108/JHOM-09-2013-0188

68.

Haftel

Swan

Anderson

, et al. Fostering the career development of future educational leaders: The success of the association of pediatric program directors leadership in educational academic development program. J Pediatr. 2018;194:5-6.e1. doi:10.1016/j.jpeds.2017.11.066

69.

Day

Tabrizi

Kramer

Yule

Ahn

. Effectiveness of the AAOS leadership fellows program for orthopaedic surgeons. J Bone Jt Surg Am. 2010;92(16):2700-2708.

70.

McNamee

. Whose ethics, which research? J Philos Educ. 2011;35(3):309-327.

71.

von Vultée

Arnetz

. The impact of management programs on physicians’ work environment and health. A prospective, controlled study comparing different interventions. J Health Organisat Manag. 2004;18(1):25-37. doi:10.1108/14777260410532047

72.

Throgmorton

Mitchell

Morley

Snyder

. Evaluating a physician leadership development program - a mixed methods approach. J Health Organisat Manag. 2016;30(3):390-407. doi:10.1108/JHOM-11-2014-0187

73.

Hartley

Hinksman

. Leadership Development: A Systematic Literature Review. A Report for the NHS Leadership Centre. Warwick, UK: Warwick Institute of Governance and Public Management; 2003:1-77.

74.

Watkins

Lysø

deMarrais

. Evaluating executive leadership programmes: A theory of change approach. Adv Dev Hum Resour. 2011;13(2):208-239.

75.

Jeon

Simpson

, et al. Cluster randomized controlled trial of an aged care specific leadership and management program to improve work environment, staff turnover, and care quality. J Am Med Dir Assoc. 2015;16(7):629.e19. doi: 10.1016/j.jamda.2015.04.005

76.

Kolb

. Experiential Learning: Experience as the Source of Learning and Development. Hoboken, NJ: Prentice Hall; 1984.

77.

Hughes

Zajac

Spencer

Salas

. A checklist for facilitating training transfer in organizations. Int J Train Dev. 2018;22(4):334-345. doi:10.1111/ijtd.12141

78.

Avolio

Avey

Quisenberry

. Estimating return on leadership development investment. Leader Q. 2010;21(4):633-644.

79.

Latham

Locke

. Goal setting - a motivational technique that works. In: Hackman

Lawlor

Porter

, eds. Perspectives on Behavior in Organizations. New York, NY: McGraw Hill; 1983:296-304.

80.

Berenholtz

Schumacher

Hayanga

, et al. Implementing standardized operating room briefings and debriefings at a large regional medical center. Joint Comm J Qual Patient Saf. 2009;35(8):391-397.

81.

Brock

McManus

Hale

. Reflections today prevent failures tomorrow. Commun ACM. 2009;52:140-144.

82.

Husebø

Akerjordet

. Quantitative systematic review of multi professional teamwork and leadership training to optimize patient outcomes in acute hospital settings. J Adv Nurs. 2016;72(12):2980-3000. doi:10.1111/jan.13035

83.

Banks

Pollack

Bochantin

Kirkman

Whelpley

O’boyle

. Management’s science-practice gap: A grand challenge for all stakeholders. Acad Manag J. 2016;59(6):2205-2231. doi:10.5465/amj.2015.0728

84.

Simsek

Bansal

Shaw

Heugens

Smith

. From the editors—seeing practice impact in new ways. Acad Manag J. 2018;61(6):2021-2025. doi:10.5465/amj.2018.4006

Gold standard research and evidence applied: The Inspire Nursing Leadership Program

Abstract

Introduction

Evidence of leadership development outcomes

Uncertain quality of evidence and potential consequences

Gold standard program elements and research quality

Establishing gold standards

Expanding the credible knowledge base

Context (professional and academic) and article purpose

Design of the Inspire Nursing Leadership Program

1. Conduct a needs and gaps analysis and establish an empirical foundation

2. Select program desired outcomes

3. Select explicit program goals

4. Select participants intentionally

5. Select program details (structure, content, faculty, and components) and prioritize evidence-based elements

Structure

Content

Faculty

Delivery

Components

6. Develop a robust complementary evaluation framework

7. Embed an application of learning/training transfer strategy

Gold standard evaluation and research design

Evaluation framework: Program

Evaluation framework: Participant outcomes and accountabilities

Foundation for a subsequent high-quality research design

Discussion and limitations

Further research and practice

Conclusion

Footnotes

Acknowledgements

Declaration of conflicting interests

Funding

Ethical approval

ORCID iDs

References