Abstract
From conceptualization to application and evaluation, research is conducted in a context of increasing complexity of disciplines, goals, communities, and partnerships. Researchers often are challenged to demonstrate the rigor of their methods and results to audiences with diverse backgrounds and disciplinary expertise. This article illustrates the benefits of using mixed methods approaches in research designed to address issues in complex research projects. It outlines the implementation of a private, public, and academic partnership, where scientific merit of methods and results was a critical foundation to the development of public policy. The overall goal of the Public Health Leadership Competencies Project (the Project) was to identify public health leadership competencies that could apply to public health practice across the country. This research demonstrates how mixed methods research in public health might be of perceived benefit to complex projects. The Project included challenges and opportunities through multiple phases of data collection and participation of members from each of the seven disciplines in public health (i.e., public health dentists, physicians, dietitians, and nurses, as well as epidemiologists, health promoters, and environmental health inspectors). The discussion addresses challenges of a national project, the complex organizational framework within which we were directed to work, and the lessons associated with using multiple sources of data.
Public health research incorporates both quantitative and qualitative paradigms with a common goal for improved health equity at the population level. Each public health discipline has a different history grounded in different research paradigms (Rutty & Sullivan, 2010). For instance, environmental health and health inspection rely heavily on traditional epidemiology (Cushing, Morello-Frosch, Wander, & Pastor, 2015), whereas nursing has a strong background in the development and use of qualitative methodologies (Holloway & Wheeler, 2010). The very nature of public health research requires mixed methods capable of bridging contrasting paradigms and disciplinary perspectives (Harrits, 2011).
The debate over what constitutes a mixed methods research continues (Johnson, Onwuegbuzie, & Turner, 2007). The work plan developed for the Public Health Leadership Competencies Project, hereafter called the Project, involved the use of the following definition: “Mixed methods studies can either combine methods from different paradigms or use multiple methods within the same paradigm, or multiple strategies within methods” (Thurston, Cove, & Meadows, 2008, p. 3). A growing body of literature strongly supports the use of mixed methods as appropriate and necessary for research quality (Ivankova & Kawamura, 2010). Much has been written (e.g., Johnson & Onwuegbuzie, 2004; Weir & Fouche, 2015) about research designs that involve the use of mixed methods research approaches in order to investigate complex problems, such as those in health care and public health. One challenge of mixed methods, however, is methodological congruence (Richards & Morse, 2013; Thurston et al., 2008). Maintenance of methodological congruence requires deliberate strategies to ensure that all the elements of the research design fit, from research question to sampling, recruitment, data collection, data analysis, and rigor strategies as well as ethical consent. Strategies such as purposive changes (i.e., revising sampling techniques due to low participation rates, using Internet technology when face-to-face data collection is not possible) and consistency of standards help maintain methodological congruence. Fletcher and Marchildon (2014) argue that “trustworthiness can be enhanced by presenting a clear ‘decision trail’ that describes the appropriateness of the [research method] for the particular problem under study and for the research procedures” (p. 2). Herein lie key considerations when designing a multiphase project such as discussed in this article. For example, how to create opportunities and use standards for clearly documenting a decision trail, and how to establish deliberate maintenance of research standards to triangulate research results and to support trustworthiness for both researchers and practitioners need to be addressed (Fink, Kosecoff, Chassin, & Brook, 1991; Powell, 2003; Richards & Morse, 2013). Other key considerations of mixed methods research designs are: the relative timing of when each method is carried out (concurrently or sequentially) and the emphasis accorded to each component for addressing the purpose of the study (whether the components are equally weighted or whether one is primary and the other secondary). (Curry et al., 2013, p. 119)
Tracy (2010) presents a model for quality in qualitative research through eight Big-Tent criteria for excellence in qualitative research. These comprise a “worthy topic, credibility, sincerity, meaningful coherence and ethical research” (p. 837). Morse and Niehaus (2009) describe the point of interface as “the position in the research process in which the two components meet” (p. 55). Employing a mixed method research design is more than simply using two or more research approaches or parts thereof in a single study. It is the point of interface of those approaches and the consequent integration of the results of the various components in the research that makes the study a mixed methods research design.
Research Design and Method
The study design and methods presented in this article were used in the Project, with the aim of articulating leadership competencies for public health practice in Canada. Core competencies exist for public health in Canada (Public Health Agency of Canada [PHAC], 2008) and discipline-specific competencies also exist; however, a need has been recognized for leadership competencies specific to public health practice. In 2013, a request for proposals to undertake a national, consultative investigation to identify leadership competencies common to all public health disciplines was issued by the PHAC, for which our funder was the successful applicant.
The partnership developed for the Project included contractual sponsorship from the PHAC with the Community Health Nurses of Canada, Canadian Institute of Public Health Inspectors, and Manitoba Public Health Managers Network. Academic partners were contracted to complete much of the Project, which was guided by the input of an Expert Advisory Committee (EAC) and Project Steering Committee (PSC), consisting of members in leadership roles in each of the seven disciplines. An important feature of the Project was the team structure that made available multidisciplinary expertise and diverse methodological strengths among members. The training, research, and practice expertise of members included public health, population health, psychology, community health, sociology, primary care, education, anthropology, epidemiology, and qualitative research methods. In each Project phase, a lead investigator was chosen who was supported by other members as appropriate. The team also was supported by a professional transcriptionist and a professional editor.
Table 1 represents the elements of the Project designed to support congruence and to elucidate points at which reflection and decisions regarding Project progress were made. The table summarizes the components of the research design for the Project and the research questions(s) for each phase. The four phases focused on existing literature (literature review), national input (survey), public health leaders’ feedback (focus groups), and building consensus regarding leadership competencies (modified Delphi method). There was also an ongoing integrated knowledge translation (KT) strategy, and Project evaluations (midpoint and summative). Each of the phases was granted ethical approval from the Conjoint Health Research Ethics Board at the academic partners’ University.
Project Components and Associated Questions.
Phase I: Literature Review
Guided by the research questions noted in Table 1, the objective of Phase I was to identify what was documented in the published and grey literature about leadership competencies, leadership qualities, and the enablers and barriers to public health leadership (Arksey & O’Malley, 2005; Onwuegbuzie & Frels, 2016). The perceived benefit of the literature review for the Project was to establish a common foundation of understanding among the stakeholders. Published literature was identified using electronic databases (i.e., MEDLINE, CINAHL, EMBASE, Global Health, Business Complete [EBSCO], and PsychINFO) and key words that included leadership, leader, public health, community health, competencies, and discipline names. Resources were limited to those in English or French printed from 1995 to 2013. Original research, systematic reviews, and meta-analyses were included. Existing networks, such as the Project’s EAC, were used to identify and to retrieve grey literature.
Tools from the National Collaborating Centre for Methods and Tools were used to evaluate the strength of the literature (Vollman, Thurston, Meadows, & Strudsholm, 2014a), but less robust works were included if they generated helpful insights or otherwise informed analysis. A three-phase charting process was used to assess articles, capturing the process via Excel™ spreadsheets. In the first instance, after a calibration exercise, two research assistants independently assessed 3,228 citations. Of those citations, 750 were retained. Next, after the calibration exercise, the retained abstracts (N = 750) underwent a closer examination for relevance, and 414 citations were retained. Full-text articles then were retrieved for the 414 articles and indexed according to country of origin, primary specialty area/discipline, study type or design, brief summary, and reviewer comments. Of the 414 articles, 139 peer-reviewed articles were identified as focused on leadership in public health. The full texts of the 139 published articles and 68 works from the grey literature were coded and analyzed using NVivo™ software, guided by a framework recommended by the PSC (see Bazeley & Jackson, 2013 for information on the use of NVivo for data analysis and management).
Desirable personal characteristics of leadership in the literature were coded for personal qualities; knowledge, skills, and behaviors; and tasks and activities. Enablers to public health leadership were classified as personal, organizational, community, or system level enablers. Barriers also were coded as personal, organizational, community, or system level barriers.
The United States, United Kingdom, and Canada combined to account for three quarters of the retained literature. The most common topic and discipline represented in the literature was public health leadership (n = 43) and public health nursing (n = 41). The qualitative analysis revealed the following themes: the theoretical nature of the leadership competencies literature; conflation of the concepts of management and of leadership; how leadership and teamwork interface; how public health affects health outcomes; little mention of equity in the literature; gender bias toward men in the literature; and a lack of clarity about the performance or outcomes of effective leadership.
Phase II: Online Multidisciplinary Survey
The next step was to understand what leadership characteristics (i.e., knowledge, skills, and behaviors), enablers, and barriers were most relevant to public health professionals in Canada (Strudsholm, Vollman, & Thurston, 2014). The intended benefit of the survey for the overall Project was to capture a snapshot of the views of the public health workforce nationally.
The survey was developed using FluidSurveys™ software and based on the results of Phase 1, including lists of knowledge areas, skills, behaviors, enablers, and barriers identified in the literature. Participants were asked to rank the top 5 items in each list. The opportunity for comments was available in all components of the survey. An online consent form that included a link to the survey was distributed widely via e-mail to the membership of seven public health professional associations, remaining live for 2 months, from November 20, 2013, to January 20, 2014.
Responses were collected from all seven disciplines from across Canada. There were 821 total responses to the online survey. One third of respondents reported having worked in public health for more than 20 years. The greatest proportion of respondents was frontline workers (42%), 25% identified as “first line management” or “middle management,” and 9% identified as “senior administration.” The characteristics of leaders, barriers, and enablers of leadership that were most often ranked in the top five are in Table 2.
Top Five Public Health Leadership Characteristics, Enablers/Facilitators, and Barriers for Public Health Leadership.
Quantitative analysis showed that participants identified a unambiguous set of top five characteristics for each list, within which two or three statements were clearly the most important statements. Participant comments did not identify any gaps and affirmed that the choices provided in the survey were comprehensive. Recommendations from the research group, based on the online survey results, comprised development of a glossary of terms with robust definitions; avoidance of conflation of leadership with management in the Project; and leveling of the competencies across a hierarchy of leadership expertise, (i.e., at novice, intermediate, and advanced levels in public health). The literature review and online survey provided a foundation of existing knowledge regarding public health leadership competencies in Canada. Further clarification of the results of Phases I and II then was sought in Phase III through focus group discussions.
Phase III: Focus Group Discussion
The Phase III objective was to assemble responses from recognized leaders in public health about the results of the online survey (Table 2) and to capture details about the context of public health leadership. Data collection was undertaken using a focus group technique (Meadows, Strudsholm, Vollman, Thurston, & Henderson, 2014). Participants for the focus groups were nominated by members of the EAC as being experts in public health. The 92 nominees were from across Canada and worked at all levels of the public health hierarchy. Four focus groups were planned, originally with the intent that each group would consist of one or two disciplines in public health. Although it was recognized that recommendations for focus group sample sizes varied from 6 to 12 participants (Baumgartner, Strong, & Hensley, 2002; Johnson & Christensen, 2013; Krueger, 2000; Langford, Schoenfeld, & Izzo, 2002), for practical reasons, it turned out that groups of six to eight were more feasible for planning across Canada’s multiple time zones.
Recruitment for each focus group was undertaken through an e-mail invitation to nominees that briefly described the overall Project and provided a summary of the online survey results. A copy of the University-approved consent form also was sent with the invitation. When agreement to participate was confirmed, a tally of preferred times was conducted and the one chosen by the majority was selected. All those who had agreed to participate were then notified of the scheduling of their focus group. The day before the scheduled focus group, a reminder was sent to participants along with a request to review the summary points beforehand.
Informed consent was obtained orally at the beginning of each focus group. The focus groups used Chorus Call™ technology to support visuals through Power Point slides to communicate results from the online survey, and a conference call center for the audio portion. The technology audio recorded all of the focus groups. Members of the Project EAC participated in pilot tests of the focus groups in order to ensure that issues that might arise during data collection could be addressed beforehand.
Recruiting participants was a key challenge for this phase. In spite of sending follow-up reminders and receiving signed consent forms from those who stated that they would participate, each focus group had members who did not show up. As it became apparent that the number of participants was going to be low, the research team decided to invite all nominees again and added a fifth focus group. A total of 27 people participated from a sample of 92 nominees (29%). Although some focus groups had as few as three participants, discussion was lively and included suggestions for additions to the lists of top five characteristics of leaders as well as questions for clarification of some of the summary points. Standard methods of qualitative data analysis included identifying codes in each line or sentence, combining codes into categories, and identifying themes in the data (Charmaz, 2014; Crabtree & Miller, 1992; Downe-Wamboldt, 1992; Miller & Crabtree, 1994; Morse & Field, 1995; Simons, Lathlean, & Squire, 2008; Thurston et al., 2008). Because all members of the team had previously used QSR NVivo10™ software, the tool was used to support data management and analysis. The software provides a platform for multiple team members to access and to comment on the analysis: it posed a clear advantage because our team was located in diverse geographical locations during the focus group phase.
Our research assistant who was new to the Project at this phase and who brought an outside perspective undertook the initial coding. Summaries based on this coding then were reviewed by two other team members. This was followed by team meetings and further coding to discuss initial results (Thurston et al., 2008). We compared the results of the focus group discussions with those of the online survey as part of the process. As we moved to interpretation, the team meetings continued, including reflection from additional reading of the transcripts, disciplinary perspectives, and drawing on relevant literature. As part of the final interpretations and report writing, we explored congruence of the Phase III results with those of Phases I and II. Because the team remained geographically dispersed (from Africa to northern Canada), Internet technology (i.e., Skype™) also was an important resource to the Project. In each focus group, the participants stated that the organizing framework used (Table 2) to categorize competencies was an impediment rather than a furtherance to understanding the identified competencies. There was much discussion about differences among personal, occupational, and macro-level facilitators and barriers, with issues including personal agency raised. Participants identified competencies that needed to be added to those in the survey results and requested clarifications to some items. Particular concern was stated by focus group participants that knowledge of critical social theory was not identified as a necessary factor in the competencies. The issue of participants’ perspectives was raised in a team discussion noting that focus group participants were recognized by peers as representing high-level public health personnel, whereas the online survey had the highest percentage of response from frontline public health personnel.
The results of the focus group phase of the Project illustrated a general agreement of participants with the top five characteristics in each list that was generated in the online survey; revealed a need for more clarity in articulating the top five characteristics; questioned whether some of the enablers and barriers were really two sides of the same coin; and expressed serious concern with the organizing framework. As a consequence, the team investigated alternative frameworks for classifying the competencies and generated recommendations to address ambiguities. We also recommended that a glossary of terms accompany any future work on the Project.
Phase IV: Modified Delphi Process
The desired outcome of the modified Delphi process was a set of leadership competencies for public health practice that represent the shared opinion of public health professionals across seven public health disciplines in Canada. The intended benefit of the modified Delphi process was to engage leaders in public health with the leadership competencies. Phase IV addressed the question: “What is the degree of agreement among nominated public health leaders across Canada regarding leadership competencies for public health practice in Canada?” Specifically, the Delphi process engaged a panel of public health experts in a facilitated reflection on the results of work undertaken thus far in the Project and the meaning of public health leadership in Canada as put forth by PHAC: Leadership is described in many ways. In the field of public health it relates to the ability of an individual to influence, motivate, and enable others to contribute toward the effectiveness and success of their community and/or the organization in which they work. It involves inspiring people to craft and achieve a vision and goals. Leaders provide mentoring, coaching and recognition. They encourage empowerment, allowing other leaders to emerge. (PHAC, 2010, ¶ 28)
The Delphi survey presented a set of draft leadership competencies for public health practice. Each item in the survey contained a description of a competency statement followed by a 5-point Likert-format scale that allowed panelists to indicate the strength of their agreement regarding whether the competency is required to meet the accepted definition of public health leadership. Between each round, the competency statements were revised based on qualitative and quantitative analysis of the ratings and comments received from the panelists. Competency statements with which less than 80% of the panelists agreed were revised to reflect their comments and suggestions. In subsequent Delphi rounds, the panelists were presented with the revisions and were asked to rate the revised competency statements.
The Delphi panelists were recruited from the membership of each of the associations representing the seven participating disciplines and other public health professionals as identified by the Project’s EAC. The aim was to recruit at least 10 panelists from each discipline, with 73 panelists being recruited in total (Wall, Strudsholm, Vollman, & Betker, 2015). Participation rates varied throughout the Delphi: 71% (n = 52) in Round 1, 74% (n = 54) in Round 2, and 66% (n = 48) in Round 3. After three rounds of the survey, the Delphi panelists achieved consensus on all of the 49 competency statements that were presented. After Round 1, seven competency statements were revised. After Round 2, one competency statement was revised. No revisions were required after Round 3, because consensus had been achieved among the panelists.
Once the Delphi process had been completed, including a report of the results to the PSC, the data collection role of this Project was complete. The next steps will be for the Project leaders to determine a knowledge transfer and dissemination plan. This plan will guide the process of dissemination among the seven disciplines of public health in Canada across the country and across the many levels of public health practice.
Integrated KT Strategies
The integrated KT strategies consisted of several reports throughout the Project. A primary example was the Environmental Scan report. The objective of this report was to determine how the results from the Project’s Phases I to III compared (Table 1). The intended benefits of this KT strategy were to foster stakeholder engagement and to support a common understanding of the Project’s progress.
To synthesize the results of the three phases of the Project, team members first mapped the top five characteristics, skills, and behaviors from the literature review. Then, members reviewed the additions and clarifications brought forward from both the online survey and the focus group discussions. In some instances, the additions resonated with the 10 statements originally included as the online survey options. In other instances, new viewpoints emerged that required further investigation. At this point, another search of the literature was undertaken to ensure that the most up-to-date material was included (it had been a year since the review was initially conducted) and to interrogate any new concepts that had arisen during the Project (Vollman, Thurston, Meadows, & Strudsholm, 2014b).
It became evident that the organizing framework provided to the team at the outset of the Project (knowledge, skills, behavior, enablers, and barriers) was not robust enough to capture characteristics of leadership competencies. The notion of framing the competencies in terms of who leaders are (i.e., what they bring with them to the practice of leadership such as affective skills), the breadth of leaders’ knowledge (i.e., in addition to the public health knowledge, such as change management, creative, and critical thinking [cognitive] skills), how leaders perform their roles (i.e., how they act to engender trust, to communicate, and to lead change), and the outcomes of leadership (i.e., motivation and inspiration that enable public health work to be successful) was deemed more appropriate. This allowed us to take into consideration the broader social context in which public health leaders work and to de-emphasize the individual characteristics foregrounded by the framework.
Because successful public health work is measured not only by population health status but also by reduction in inequity, we assessed the PHAC (2010) definition of leadership in public health and found it wanting with regard to the ultimate outcome of public health leadership. Our recommendation was that PHAC should amend its definition of leadership in public health to include the desired outcomes of public health leadership (i.e., equity, social justice, engagement; Vollman et al., 2014b).
As our analysis in the environmental scan advanced, leadership competencies for deliberation in the Delphi process were drafted. It was noted that in order for the public health sector to communicate across the health care system, with reference to leadership, it needs to use familiar language. Therefore, a broadly based organizing framework would be beneficial. A Canadian leadership competency framework had been located during the literature search; and when compared to the results of Phases I–III, there was overlap. As a result, the LEADS model (Canadian College of Health Leaders, 2013) was recommended to frame the competencies for leadership in public health. This recommendation was intended as a means to improve cross-sector communication and to reduce competition, a major concern raised in Phases I through IV of the study. Further, the public health community might contribute to the LEADS framework by providing specific public health examples for discussion and by supporting public health leadership development in a multisectoral context.
Evaluations
The research design included evaluation at two points in the Project. The first was designed to provide a midterm evaluation to give feedback on reaching the objectives of each of the phases reported here. The second evaluation is ongoing and designed to assess the final outcomes of the project as outlined in Table 1. The evaluations also represent vital sources of information that provide feedback to the funders, EAC, and PSC regarding the current state of knowledge, expectations and attitudes toward the content, and the development process of public health leadership competencies.
The plan for evaluation of the Project began with what is referred to as an evaluability assessment (EA; Casebeer & Thurston, 1995; Davies & Payne, 2015; Wholey, Hatry, & Newcomer, 2010) at the beginning of the Project. The intended benefit of the EA was to reach agreement on goals and objectives of the Project and what an evaluation would support. The lead academic partner reviewed the research proposal for the Project, the logic model prepared by the applicants, and the work plan produced early in the Project development. Based on these, a description of the Project was written that included stakeholders, the governance structure, key definitions used, the goal and objectives, underlying assumptions in the logic model, and processes described as part of the Project. Some inconsistency in terminology was identified and a recommendation to change the end goal of the Project was made. A meeting was held via teleconference with the PSC that included representatives of the three partner organizations, the Project consultant, and the lead academic partner. The goal of this meeting was to gain consensus on the topics in the EA report. Consensus was achieved, including the goal change, and the elements of the EA became the introduction to the evaluation proposal.
The evaluation was designed to be participatory; thus, a proposal was drafted and circulated to the PSC for comments and suggestions that were later incorporated. The evaluation questions investigated the effectiveness of the Project management or governance structure for engaging the seven disciplines and whether the short-term and intermediate objectives of the Project were met. It was divided into two stages so that an interim assessment of the Project would be made and the leaders could use the information to make any changes in governance or engagement strategies that were central to meeting the Project goals.
As indicated earlier, the EA assisted in clarifying the Project in the early stages. Another outcome of the EA was clarification of the academic partner roles vis-à-vis some of the Project phases. The first phase of the evaluation methodology involved use of document review and telephone interviews with the key stakeholders representing the PSC and the seven disciplines.
The interim evaluation reported on the importance of strategic communication with various disciplines as a means of getting stakeholders “charged up about this exciting work” so that they in turn, could “charge up others” and contribute to the Project’s momentum and eventual uptake of competencies (Henderson, Thurston, & Strudsholm, 2014, p. 9). The Project could benefit with a clearer definition of what was expected from the discipline representatives on the EAC to address role ambiguity among the representatives. Regular, and ideally in person, consultation was identified as means with which start conversations about “what are you going to do with these competencies” (Henderson et al., 2014, p. 8). Such conversations were regarded as important means to secure long-term stakeholder investment and Project uptake. Overall, the evaluation results suggested that continued support of meaningful engagement with EAC was essential in order for the Project stakeholders to gain in-depth knowledge about processes, a more comprehensive understanding of project’s progression, and to support uptake of competencies.
The final evaluation is currently in progress, involving document review, survey, and telephone interviews with all members of the PSC and EAC. The interviews for the final summative evaluation are in process at the time of writing. Once the results are collated and report drafted, it will be circulated to the PSC. Discussion of the report resulting from a teleconference will serve as additional data for the final evaluation report.
Discussion
The mixed methods research design used in the leadership Project was necessary in order to address the complex character of leadership in a multifaceted public health context. One of the key elements of public health is improved health equity at the population level; therefore, equity among the seven disciplines was viewed by the academic partners as a principle to be upheld in the design. This principle, in addition to the research questions under examination, demanded a carefully planned mix of data collection strategies and their attendant techniques for each phase of the Project. Methodological congruence was essential to the Project, as was methodological purposiveness (Richards & Morse, 2013). McDaniel, Lanham, and Anderson (2009) have noted that research proposals are the recipes that are submitted for review and funding; once researchers are in the field, the proposal serves as a compass as the everyday circumstances and the context of field work is negotiated; therefore, the proposed Project evaluation that began with an EA rather than assuming agreement on what it would accomplish was an essential part of the Project, as was the integrated KT.
The multiphased research design was considered a benefit to the complex nature of the Project. Not only were the multiple phases required to address the long list of research questions (Table 1); each phase also was intended to benefit the Project’s stakeholders’ interests in different ways. Table 3 compares the intended and observed benefits of each phase of the Project.
How Mixed Methods Research in the Project were a Benefit.
Challenges in this Project were abundant—in no small part relating to the fact that public health in Canada brings together seven disciplines. Throughout the study, busy health practitioners were asked for participation (whether for the online survey or the focus groups). At times, practitioners were asked to reflect on their work and the guiding principles, values, and ethics that are not often explicitly articulated. One of the challenges for public health leadership identified from the results was the frequent work overload and burnout among public health professionals. Although public health personnel expressed excitement about the Project, that excitement did not always translate into participation.
Miller, Crabtree, Harrison, and Fennell (2013) identify the advantages of a fully integrated mixed methods research model, wherein all activities—including resource allocation and sharing, project and measure design, data collection, analysis, interpretation, dissemination, and so forth—require collaboration by and active engagement of every member of the research team. This is illustrated throughout our study, among other approaches to mixed methods research by Bowers et al. (2013) in the context of health services research. The Project team’s varied backgrounds drew together a number of lenses that allowed us to examine data and to develop interpretations that integrate several disciplinary perspectives. Continual monitoring of phases of the Project and a detailed work plan ensured that budgeted resources—both time and money—were available as needed. All team members were cognizant of the imperative to meet timelines to facilitate the cumulative phases of the Project. The EAC and PSC members were frequently updated with regard to progress of the phases as an important feature of transparency of the Project. Regular communication, including reminders of deadlines, fostered collaboration and engagement.
Neither the literature, nor the survey, nor the focus groups provided a complete picture of leadership competencies for public health practice. Different data sources investigated different theoretical and experiential sources, offering a rich analysis of competencies in a specialty field. Through iterative reflection on the analyses undertaken to this point, adoption of a more suitable framework occurred. The LEADS framework reflected an attempt to present a more parsimonious organization of both competencies and context of public health leadership through three prongs: personal qualities (both affective and cognitive), process competencies (actions), and outcomes (positive workplace culture). The Delphi phase of the Project was intended to refine further our understanding of leadership competencies.
The challenges of this national, complex, multidisciplinary Project were well served by the use of a mixed methods research design. The study provided a responsive research strategy that could reach a national, diverse, and busy population of public health professionals. Furthermore, the design supported integrated KT among Project partners. The Project was flexible enough to meaningfully incorporate consequent knowledge development. Mixed methods provided an alternative to rigid approaches that confine researchers to single paradigms (Harrits, 2011). The Project design supported a dynamic research approach that parallels the dynamic reality of public health practice in Canada.
Footnotes
Acknowledgment
We would like to acknowledge Helena Wall, of Innovative Solutions Health Plus, who acted as Project Consultant for the Project and led the Delphi data collection.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors were supported by the Community Health Nurses of Canada through funding provided by the Public Health Agency of Canada. The opinions expressed in this publication are those of the authors and do not necessarily reflect the official views of the Public Health Agency of Canada or the Community Health Nurses of Canada.