Abstract
Current pressures on public health systems have led to increased emphasis on restructuring, which is seen as a potential solution to crises of accessibility, quality, and funding. Leadership is an important factor in the success or failure of these initiatives. Despite its importance, health leadership evades easy articulation, and its study requires a thoughtful methodological approach. We used a modified Delphi method in a Participatory Action Research (PAR) project on health leadership in Canada. Little has been written about the combination of Delphi method with PAR. We offer a rationale for the combination and describe its usefulness in researching the role of leadership in a restructuring initiative in “real time” with the participation of health system decision makers. Recommendations are provided to researchers wishing to use the Delphi method qualitatively (i.e., without statistical consensus) in a PAR framework while protecting the confidentiality of participants who work at different levels of authority. We propose a modification of Kaiser's (2009) post-interview confidentiality form to address power differentials between participants and to enhance confidentiality in the PAR process.
Keywords
Health leadership has been identified as a critical factor in the success or failure of health system reforms in the United States, the United Kingdom, and Canada (Blackler, 2006; Borkowski, Deckard, Weber, Padron, & Luongo, 2011; Denis, Langley, & Rouleau, 2010). However, the distributed nature of a sector in which authority, legitimacy, resources, and influence are often highly dispersed and the ambiguous nature of distributed leadership make research on the topic extremely challenging (Buchanan, Addicott, Fitzgerald, Ferlie, & Baeza, 2007; Chreim, Williams, Janz, & Dastmalchian, 2010; Currie, Grubnic, & Hodges, 2011; Currie & Lockett, 2011). To address this challenge, a unique methodological approach was developed to study health system leadership in a major restructuring initiative in Canada that involved spreading leadership across numerous organizational boundaries (Denis, Langley, & Sergi, 2012). A modified Delphi method combined with a Participatory Action Research (PAR) framework was developed to facilitate the participation of health system decision-makers in the research project, which studied their leadership during the design and implementation of the restructuring initiative.
The purpose of this article is to present a methodological justification and model for using the Delphi method in qualitative, PAR research. Available research on the combined use of Delphi and PAR is very limited; as such, we provide a rationale for this particular methodological combination, describe a research design using the combination to study health leadership, and illustrate how significant ethical issues were addressed using a post-interview confidentiality form (Kaiser, 2009). PAR is change-oriented and should involve participants at every stage in the project; however, for these same reasons, PAR can impose trade-offs between participant involvement and participant confidentiality. We demonstrate the utility of the Delphi method for change-oriented research that fully involved health system leaders while maintaining their confidentiality as participants.
A second purpose of this article is to guide researchers in using the Delphi method for research that is primarily qualitative. Although commonly perceived as a quantitative method because of its focus on statistical consensus, a modified and open-ended Delphi method facilitated a qualitative understanding of participants' personal leadership experiences. Because the research project followed the restructuring initiative in real time, we used an iterative process with qualitative open-ended questioning in both Delphi rounds, a method that differs from the more common use of Delphi for numeric consensus seeking.
A significant amount of literature has described the procedures involved in doing Delphi research (e.g., Hasson, Keeney, & McKenna, 2000; Okoli & Pawlowski, 2004). However, less attention has been given to the specifically qualitative techniques in the process—particularly coding. As such, we provide a description of the qualitative coding procedures used during this modified Delphi project. There is strong debate about what constitutes methodological rigour in Delphi studies, in part because of the diverse applications of the method (Hasson & Keeney, 2011). However, many researchers argue that in Delphi studies, as in qualitative research generally, trustworthiness can be enhanced by presenting a clear “decision trail” that describes the appropriateness of the Delphi method for the particular problem under study and for the research procedures (Fink, Kosecoff, Chassin, & Brook, 1991; Powell, 2003). We present our rationale for selecting a modified Delphi method in the study of health leadership and describe its utility for understanding and refining elusive concepts such as leadership. We report several key findings to illustrate how the methodology produced useful information for health decision makers, and to show how it led us toward a more concrete articulation of the elusive concept “leadership.”
A Brief History of the Delphi Method: From Munitions to Modification
The origins of the Delphi method can be traced back to “Project DELPHI,” a Cold War study initiated by the RAND Corporation to identify potential American industrial targets and their vulnerability to Soviet munitions (Dalkey & Helmer, 1963). The method sought to “obtain the most reliable consensus of opinion of a group of experts” through “a series of intensive questionnaires interspersed with controlled opinion feedback” (Dalkey & Helmer, 1963, p. 458). The original Delphi study set forth several key features of the method, specifically the use of a purposively selected panel of experts to achieve consensus on a particular issue or question over multiple survey rounds. A typical Delphi study includes at least two rounds of questionnaires. Round one represents what Ziglio (1995) called the “exploration phase” (p. 9), in which the topic is fully explored using broad or open-ended questions. Each subsequent round then becomes part of an “evaluation phase” (p. 9), in which the results of the previous round are used to frame another set of questions. Each round provides an opportunity for the experts to respond and to revise their answer in light of the group members' previous responses. Over multiple rounds, the process can gradually lead to consensus or near-consensus.
Since its inception, the Delphi method has been used in several diverse fields, such as marriage and family therapy (Blow & Sprenkle, 2001), information systems (Okoli & Pawlowski, 2004), and project management (e.g., Chan, Yung, Lam, Tam, & Cheung, 2001), although it is perhaps most commonly used in health research. The method has been modified to serve a variety of purposes such as ordering issues by importance, defining an issue or concept, determining priorities, and identifying best practices. Modifications of the Delphi method abound. The large number of modifications and uses has prompted some researchers to speak of the “Delphi approach” rather than the more specific term “method” (Mead & Moseley, 2001). However, de Villiers, de Villiers, & Kent (2005) classified three main variants of Delphi: (a) conventional Delphi, which follows the format of the original RAND study (i.e., an open-ended exploratory phase followed by multiple consensus-seeking rounds) and usually seeks to prioritize issues or find solutions; (b) “real-time” Delphi, in which multiple rounds are temporally compressed to occur within a single meeting; and (c) policy Delphi, which creates a forum in which ideas are presented to decision-makers, who ultimately choose a solution from among a number of options.
Despite its diverse applications, the key purpose for using the Delphi method remains the collection of informed judgment on issues that are largely unexplored, difficult to define, highly context and expertise specific, or future-oriented (Helmer, 1967; Ziglio, 1995). In the case of future-oriented Delphi research, users have remained loyal to the etymology of the method's name, which refers to the ancient Greek oracle at Delphi that was believed to forecast the future. Because it is exploratory in nature, the method is not recommended for use in areas with abundant theory and empirical literature, or where topics are already well defined (Mead & Moseley, 2001; Ziglio, 1995).
Existing literature has described the strategies, benefits, and limitations involved in using the Delphi method (Adler & Ziglio, 1995; Goodman, 1987; Keeney, Hasson, & McKenna, 2006; Linstone & Turoff, 1975; McKenna, 1994; Murry & Hammons, 1995; Powell, 2003; Sackman, 1974). The approach has been criticized for its potential to force consensus. Although critics have argued that the Delphi method encourages conformity and neglects potentially illuminating information from alternative or outlying opinions (Goodman, 1987; Sackman, 1974), a key advantage of the method is that, because participants can be interviewed or surveyed individually, it can facilitate a “dialogue” between experts from diverse geographical locations while preserving anonymity (Linstone & Turoff, 1975; Ziglio, 1995). Unlike those in conventional focus groups, Delphi participants typically remain unknown to each other. Although it has been argued that the anonymity aspect reduces participant accountability and thus encourages “snap judgments” (Goodman, 1987; Sackman, 1974), it is useful for mitigating the effects of power relationships and can prevent the domination of group opinion by a particular individual or sector.
Mixing Prescriptions or a Healthy Combination? Using the Delphi Method in Participatory Action Research
Attention to power differentials makes the Delphi method conducive to studies using a PAR methodology. In PAR, the research participants also participate in the design and collection of research evidence. Participants are seen as collaborators with the knowledge and agency to contribute to an understanding of the research and its results. The participants, in turn, are able to use the research results for their organizational or advocacy purposes. As such, “the focus is on creating dialogue and generating knowledge through interaction” between researchers and participants (Brydon-Miller, Kral, Maguire, Noffke, & Sabhlok, 2011, p. 390; Walji, 2009).
In PAR, methods are selected by their ability to facilitate participatory, change-oriented research practice. The Delphi method is an iterative approach to research in which participants are presented with regular reports on the findings during each questionnaire round. This provides them with the ability to confirm or revise their previous answers and ensures that participants remain connected to, and involved in, the development of the research. Like PAR projects, the Delphi method produces information that can be put into practice by participants, making it particularly useful for policy- and decision-makers.
Delphi research occurs across several rounds and often over an extended period of time; as such, it requires a substantial commitment from participants. Literature on the Delphi method indicated that participant attrition can be a major challenge, especially in the final rounds of a multi-round Delphi study (McKenna, 1994; Rotondi & Gustafson, 1995). PAR emphasizes and requires full engagement and involvement of participants and, as such, can help to mitigate the attrition problem in Delphi studies by building participation into the research process.
Despite these commonalities, very little research exists on the combination of PAR methodology with the Delphi method. A similar combination was used by Totikidis (2010), who conducted community-based participatory action research using Nominal Group Technique (NGT) to explore community health issues. NGT is methodologically similar and often compared to the Delphi method or used in conjunction with it (e.g., Davis, Turner, Hicks, & Tipson, 2007). Like Delphi, NGT is a tool for gathering group opinion, in which participants generate ideas silently and independently of each other. The ideas of all participants are then subjected to discussion and evaluation by the entire group. Totikidis (2010) selected NGT for its ability to “generate ideas without distraction or influence from more dominant members in the group” as well as the “democratic voting/rating of ideas” (p. 20) it entails. The key difference between the Delphi method and NGT is that the latter typically occurs in person and, regardless of how democratically the collective ranking process is conducted, it requires participants to assess and rank other group members' ideas. For this reason, NGT might be inappropriate for situations involving power differentials between participants. The project discussed here involved participatory research with leaders and managers at different levels of authority in the same health system; therefore, it required the more anonymous approach offered by the Delphi method. The anonymity allowed participants to more openly discuss and evaluate issues of leadership in the health system without fear of reprisal.
Context: The “Leadership and Health System Redesign” Project
The research project discussed here was one of five regional studies or “nodes” in a broader pan-Canadian project called “Leadership and Health System Redesign.” The main purpose of the pan-Canadian project was to examine effective leadership practices in various cases of health system change across the country and to compare them to existing frameworks for best practices in health leadership, such as the LEADS framework discussed below. The project was designed to respond to current pressures within the Canadian health system, particularly a perceived tension between accessibility and quality in the context of escalating costs. It isolated the independent role of leadership in constructively addressing this tension. Once identified, knowledge about effective leadership can be translated into approaches, programs, tools, and techniques to enhance health leadership in Canada.
The project provided an opportunity to test and evaluate the accuracy of the new LEADS framework. The framework was adopted in 2009 by the Canadian Health Leadership Network, the Canadian College of Health Service Executives, and Leaders for Life as a standard for leadership development in Canada's health sector (Dickson, 2010). The framework “identifies the leadership and management capabilities required for individuals to effectively create change in the modern Canadian health environment” (Dickson, 2010, p. 1). The acronym LEADS is made up of five key elements: (a) leading self; (b) engaging others; (c) achieving results; (d) developing coalitions; and (e) systems transformation.
This article focuses on the unique methods used in one of these regional studies, the Prairie Node, and demonstrates how the methods led to several key findings on leadership and LEADS. Our study focused on a particular restructuring initiative currently underway in the Western Canadian province of Saskatchewan. The initiative, known generally as the Saskatchewan Shared Services Initiative or simply “Shared Services,” was initiated in 2010. Its purpose is to integrate and standardize delivery organizations, particularly “back office” services and procurement chains, to achieve economies of scale and scope without establishing a single, hierarchical health organization. Shared services are being created through a cooperative enterprise involving the province's 12 health regions and boards, independent health care organizations, a cancer agency, independent physicians, and the provincial health ministry.
In the Prairie Node research, we isolated and examined the leadership factor in this major health system redesign project. As a PAR project, a key goal was to provide feedback and information to health system leaders throughout the process of restructuring, allowing them to maintain an awareness of emerging leadership activities and needs. As such, the research is part of a dynamic and ongoing process of health system change.
The study was a two-round Delphi project carried out over a period of 9 months (see Figure 1). The first round involved interviews with 39 participants, categorized into three groups according to their level of management responsibilities within the health system. Although exercising very different levels of authority, from senior executive roles to front line implementation and delivery, all 39 were defined as change agents whose leadership characteristics and behaviours would influence the success or failure of Shared Services (Battilana, Gilmartin, Sengul, Pache, & Alexander, 2010). The interviews included both open-ended questions and quantitative Likert-type scale rating questions. Each interview was approximately one hour in length. The interview questions were designed to foster a deeper understanding of the leadership abilities that most accurately constitute a “best practices” framework in health system redesign. After collection, interview data were coded using a threefold strategy, further described below.
The Delphi Process in PAR
The second round acted as both research report and questionnaire. It allowed participants to view the aggregated interview themes and to revise, elaborate, or validate their previous responses in light of more recent events in the restructuring initiative. First, participants were asked to offer their own interpretations of areas where statistically significant differences were found between the average scale ratings of the three participant groups. Second, participants were asked to rank several leadership characteristics that emerged as dominant in the coding but which were not explicitly present in the LEADS framework and therefore had not been part of the scale rating exercise. An open comment field allowed participants to offer additional commentary or items they believed had been missed. A similar question was also asked about strategies that were seen as conducive to Shared Services thus far. A final section addressed four major observations that had emerged from the coded interview data. These observations spoke to four major challenges or problems in health leadership, and this section asked participants to rank a small set of suggested solutions or suggest other solutions to these particular problems. The results of both Delphi rounds are presented in more detail below.
Participation and Privacy: Ethical Issues Arising from PAR
Ethics approval was obtained from the University of Regina, the University of Saskatchewan, and four health regions, although only the two universities and one health region (the Regina-Qu'Appelle Health Region) had their own ethics boards. Our use of a post-interview confidentiality form proved instrumental in obtaining ethics approval. The use of PAR in this particular context posed three specific challenges related to confidentiality: first, the research context was one in which participants were likely to be acquainted with one another; second, participants represented various levels of authority within the system, which meant that at times they were asked to comment on the leadership styles of those senior to them; and third, the inclusion of four health system actors as research partners, although a key feature of PAR, necessitated a careful balance between confidentiality and disclosure.
Several measures were implemented in response to these challenges. All participants, including the four health system actors collaborating as research partners, were given access only to the aggregated research results without names or identifiers. However, because of the relatively interconnected network of actors in the Saskatchewan health system, the possibility remained that participants' views could be attributed to them based on anecdotes or quotations. In response to this concern, the researchers employed an adapted version of Kaiser's (2009) post-interview confidentiality form. Kaiser's original form contained four confidentiality options for participants to select from. These ranged from full disclosure of all information including the participant's name to a much more cautious option that required the researcher to contact the participant before using the data.
We adapted Kaiser's form and used it in addition to the standard pre-interview consent form required by the ethics boards. Our version of the post-interview confidentiality form allowed participants to select between two options for how their data would be utilized and presented. The first option authorized the research team to use all information provided by the participant, with the exception of the participant's name. In contrast to Kaiser's form, ours excluded names by default because we perceived little benefit, and even potentially negative consequences, from including names. The second option included the protections of the first but also allowed the participant to select other items they did not want published or presented to anyone except the academic researchers, for example, stories or anecdotes they had shared that could identify them. The post-interview confidentiality form allowed participants to speak more freely during the interview with the knowledge that they could determine how their data would ultimately be used. The majority of participants selected the first and most basic confidentiality option; however, four participants (at least one from each level of authority in the sample) selected the second option. Of these, three requested the exclusion of identifiers such as location, job title, or gendered pronouns. The remaining participant chose to read the transcript and selected several anecdotes for exclusion.
Applying a Modified Delphi in a PAR Framework
The PAR framework emphasizes participant involvement at all phases of the research project. As such, each round of the study included up to four health system actors as both research partners and participants. Representing different levels of authority within the system, these individuals were involved throughout the research process and acted as a decision-making reference group (see Figure 1). In the early stage of the research, they provided insight on problem identification, advised on the interview guide, and assisted with participant selection and recruitment. They also provided an initial list of potential participants who had sufficient involvement in the reform to speak about their experiences. The researchers then contacted these participants and a final list of 39 participants was established from the initial list.
Participants in the study represented three levels of authority within the health system and were categorized into three corresponding participant groups. Group 1 included governance and directional leadership at the ministry and health region executive level, Group 2 consisted of Shared Services senior leadership immediately below CEO-level at collective and individual project stream levels, and Group 3 included front-line leadership of existing business units within each project. After the first round of interviews, the decision-making reference group provided feedback as the aggregated interview data were converted into the round two report and Delphi questionnaire; for example, they ensured that data selected for follow-up in round two would ultimately be relevant and useful for decision-makers and participants. They also helped ensure that the final report was useful and useable for participants and health leaders more broadly.
The inclusion of health system participants as research colleagues, which is a key tenet of PAR, was beneficial for several reasons. First, it provided the academic researchers with consistent, up-to-date information about the progression of Shared Services. This was crucial to ensure that the questions being asked of participants were contemporary and relevant at all phases of the research project. Second, because Delphi research is premised on “informed judgment” (Ziglio, 1995, p. 4), it was important that the selected participants had been sufficiently involved in Shared Services to provide such informed judgment. The health system partners were best able to identify which actors should be consulted in this regard. Third, as discussed below, the Delphi method facilitated participants' use of the research results in their daily work on Shared Services. This fulfills the PAR goal of producing action-oriented, change-focused research (Berg, 2004).
There were four principal reasons the Delphi method was selected for this PAR project. First, the Delphi method is epistemologically conducive to PAR research. PAR creates space for “non-academic community members to contribute to knowledge construction about the issue being studied” and seeks to understand how the issue plays out in participants' everyday lives (Billies, Francisco, Krueger, & Linville, 2010, p. 278). Thus, PAR takes an epistemological position that values experiential knowledge as authoritative (Billies et al., 2010). As discussed previously, the Delphi method is suited to contexts where little academic literature exists but experiential knowledge is vast. The Delphi method positions community members or practitioners as experts on the issue being studied. We sought a clear articulation of the vague concept “leadership,” something known best by those who act as leaders in their daily lives. The Delphi method allowed us to draw on participants' experiential insight to help define effective leadership in health system change.
Second, the Delphi method facilitated confidentiality and inclusivity, which were absolutely necessary for this study. Because Delphi rounds occur without face-to-face meetings, individual participants' comments remain confidential while still facilitating “virtual” dialogue about the aggregated findings. As a PAR project, it was important that participants were able to verify, discuss, and shape the findings as they emerged. At the same time, confidentiality was paramount. Participants represented various levels of authority in the same health system, and they were asked to comment on the leadership behaviours of those leading them. The Delphi method allowed us to conduct participatory research with actors who might not normally have been able to participate due to confidentiality reasons. Furthermore, Saskatchewan is a large and sparsely populated province, which makes inclusivity difficult. Because of the “virtual” nature of the Delphi rounds, we were able to include participants located throughout the 12 geographically and administratively separate health regions in the province, many of which are rural or geographically remote.
Third, the Delphi method encourages researcher accountability to the participants during the analysis. After round one, the aggregated data were returned to participants along with the researchers' interpretations of it. This allowed participants to provide feedback on the findings and observations as they emerged. In contrast, more individualized methods of participant validation only allow individual participants to validate the accuracy of their own transcripts or findings, not the aggregated results or observations. In this way, the Delphi method encourages careful data processing and responsible interpretation by keeping the researcher accountable to the participants' meanings and intentions. Such consultation is a way to ensure participants' continuous involvement and their control over the results, which are key tenets of PAR (Berg, 2004).
A fourth reason for selecting Delphi methodology is related to the PAR goal of action and change. In PAR, the research project must address the needs of both academic and participant researchers (Billies et al., 2010). Although we sought to answer scholarly questions about leadership, the project was also designed to provide useful information for the decision-making reference group and participants, as well as to facilitate positive leadership outcomes in Shared Services and the health system more broadly. Our project focused on an in-progress case of health system change and, as such, followed the implementation of Shared Services in real time. The Delphi method necessarily involves a time lapse between rounds, as the results from round one are processed and aggregated into the round two questionnaire. The time lapse has been seen as a weakness of Delphi methodology (Murry & Hammons, 1995); however, in our study, it was in fact a strength because it allowed participants to revise or elaborate their answers based on events that had transpired since the previous round of research and, in so doing, enabled researchers to track changes in participants' perceptions over the course of the implementation.
Furthermore, the method allowed us to provide just-in-time information that participants could put into action. Because the findings were returned to all participants at a “halfway” point in the research process (i.e., between the first and second Delphi rounds), the participants could use the information to inform their actions over the next stage of Shared Services, reflect on their leadership, and make course corrections if necessary. After the first Delphi round, a high-level member of the decision-making reference group reported that, “the research project has given me a better understanding of the problems with the [Shared Services] Initiative … it was clear we had not engaged enough with front-line leaders and that is something that I am addressing now.” The Delphi method allowed us to return results to participants when they needed them, and not simply as a report presented after the fact.
Because our study focused on changing perceptions of leadership over the course of the Shared Services Initiative implementation, it deviated from the conventional use of Delphi for consensus seeking and is thus considered a “modified” Delphi method. Consensus building was the original purpose of the Delphi method as created by the RAND Corporation and is touted as one of its primary methodological advantages (Powell, 2003). Conventionally, open-ended questions are common in the initial exploratory phase of the Delphi, which is then followed by quantitative or statistical surveys in the latter phases; this is where consensus is produced. As previously mentioned, however, this aspect has drawn criticism for its potential to force consensus and for not allowing participants to elaborate on their responses (Goodman, 1987; Hasson et al., 2000; Sackman, 1974). Our study did not emphasize consensus as its primary goal; therefore, the second round of the research—the Delphi survey—sought open-ended participant reflections and interpretations of the previous interview results. Participants were given the opportunity to offer their interpretation of important findings from round one, rather than to simply further rank or rate items selected by the researchers for follow-up.
The Delphi Process and Findings
The following section provides a description of the Delphi process that focuses on the qualitative elements of the method. Some key findings are presented to illustrate how the findings were generated from the method. Although the literature provides advice on the use of quantitative and statistical measures in Delphi, particularly in the context of information systems research (Okoli & Pawlowski, 2004; Schmidt, 1997), little has been written about the process of qualitative data processing within the Delphi process, despite the fact that the first round typically involves an open-ended questionnaire or interviews to determine priority issues for subsequent rounds. Some studies have used alternative methods of determining priority issues, such as a literature review (Kutz, 2006) or pre-defined questions taken from national surveys (Oranga & Nordberg, 1993). The issues identified are then placed directly into a quantitative survey for ranking or rating by the expert panel. Murry and Hammons (1995) opted to eliminate the open-ended round altogether, moving directly into a structured survey because this “saves time and expense and allows the panel to immediately focus its attention on the study issues” (p. 431). Approaches that bypass the conventional use of an open-ended questionnaire in round one have been questioned on both theoretical and methodological grounds (Lemmer, 1998; Rowe, Wright, & Bolger, 1991).
With some notable exceptions (De Urioste-Stone, McLaughlin, & Sanyal, 2006; Hasson et al., 2000), many Delphi studies do not present a detailed discussion of how the results from the exploratory round are processed and converted into the round two questionnaire. Green, Jones, Hughes, and Williams (2002) expressed concerns about maintaining an inductive approach to data analysis when using the Delphi method; they found that the reduction and distillation of participants' words to facilitate consensus “emerged as one of the most problematic aspects of the study” (p. 201) given the authors' commitment to an inductive and qualitative approach to the data analysis process. The gradual removal of statements from their original contexts led the authors to notice an “increasingly quantitative turn” (p. 201) in their Delphi project. In a similar vein, Taylor (2008) critiqued the use of Delphi as a tool to produce codes for deductively organizing a larger body of legal data, promoting instead a more inductive approach to coding the data. As such, the method has been criticized for deductivism in both its internal processes and its end use.
Consensus was not a key priority in our study, a fact that allowed us to avoid several key challenges described in the literature. Indeed, our goal was to use the Delphi in a way that would provide participants with an opportunity to comment on substantial inter-group differences found in the first round. These areas were most likely to indicate key issues of disagreement and potential controversy within Shared Services, and therefore areas where leadership successes and failures were most likely to emerge. We also wanted to follow any qualitative changes in participants' views as the initiative progressed.
The open-ended interviewing in the exploratory phase produced a large amount of data that required careful analysis and presentation. The confidential interviews included both qualitative and quantitative portions. The first portion of the interview consisted of open-ended, general questioning, which allowed participants to identify key issues in health leadership without the constraints imposed by highly structured questions. The second part of the interview involved quantitative rating questions using a Likert-type scale of 0 to 10. The quantitative portion of the interview investigated which leadership styles and behaviours were most emphasized and enacted within this particular case of health system change. The 20 leadership styles and behaviours selected were based on the LEADS framework (Dickson, 2009, 2010).
After collection, the qualitative interview data were coded using NVivo 9 qualitative data processing software. Interviews from each participant group were coded separately to identify key areas of correspondence and difference between the groups. The coding strategy drew on both inductive and deductive approaches to create a unique strategy that suited the project. We began by identifying repeating ideas in the data, an inductive technique drawn from grounded theory (Auerbach & Silverstein, 2003). To do this, we used the word frequency query function of NVivo 9. The process resulted in 23 codes. These codes were then used deductively, with data coded directly to them as applicable. This strategy was useful because it allowed us to approach the data with a list of pre-established codes; however, these codes had been determined by the data itself and not by pre-existing literature or theory, which is common practice in deductive coding (Gilgun, 2011) but less useful in exploratory projects for which there is little existing literature. To be thorough, the deductive process was combined with inductive open coding (Auerbach & Silverstein, 2003). In other words, any data that did not fit our deductive code list were placed into new codes. This produced 52 codes that would otherwise have been missed if only the initial list was used. Indeed, as Gilgun (2011) noted, deductive codes must be treated flexibly, because “it is easy to find material that supports the prior codes, but it is just as important in many cases to find material that does not fit into your codes” (p. 2). Finally, data were also coded into separate nodes that corresponded with each interview question. This allowed us to easily view all data that spoke to any particular research question and provided a different way of viewing the data.
This mixed approach to coding proved productive. As mentioned, several important themes and inter-group differences emerged during the open coding process that would otherwise have been missed if coding had followed only pre-determined codes. For example, participants often spoke of government having the “appetite” for change, or they expressed uncertainty that government had the “stomach” for making politically controversial changes in the system. These metaphors all spoke to the same issue, political will, but participants had very different ways of describing the issue. As such, this important theme would have been missed if we had not combined our deductive codes with open coding. In contrast, the deductive approach, which applied the same coding categories to all groups, was crucial for showing areas of convergence and difference between the participant groups. For example, “communication” and “engagement” were identified as repeating ideas and were added to the list of deductive codes. Because data from each participant group were coded using the same list, we could see that “communication” was the most commonly coded node in each of the three groups. The same strategy showed that although “engagement” was the second most dominant code for Groups 2 and 3, it was not often mentioned amongst the high level leaders in Group 1. This proved an important finding and will be discussed further below. The deductive approaches also helped to ensure a sharp focus on issues of leadership, which otherwise might have been overshadowed by details and concerns about Shared Services itself.
All these first-level codes were analyzed and further narrowed into several major pattern codes using the details of the text. Pattern codes have been defined as “explanatory or inferential codes, ones that identify an emergent theme, configuration, or explanation” (Miles & Huberman, 1994, p. 69). Pattern coding allows for further organization of the data while allowing the researcher to attribute meaning and inference to it. For example, text coded to the “resources” and “commitment” nodes revealed a common theme. The information contained in these codes, in addition to the previously mentioned codes “appetite” and “stomach,” spoke primarily to the issue of political will. These first-level codes were thus combined into the pattern code “political will.” Drawing together our numerous first-level codes, we were able to identify four such patterns or themes, each of which indicated a particular problem or challenge of leadership in Shared Services. In the next section, we describe these four challenges and how the Delphi process was used to suggest practical solutions to each.
Four Problems or Challenges of Leadership in Shared Services
The four themes revealed particular problems or challenges of leadership in the context of Shared Services and helped to identify the most relevant leadership issues at play. The round two Delphi questionnaire then provided an opportunity to explore potential strategies and actions to help address these challenges. After identifying the four challenges, the researchers generated a list of potential solutions for each. The list was presented to the decision-making reference group for their feedback, and the finalized strategies were integrated into the round two Delphi questionnaire for all participants' feedback. In all cases, an open-ended comment window was available and participants were encouraged to provide additional comments and/or express disagreement with the strategies presented.
The first challenge of leadership we identified was related to vision, particularly the communication of the Shared Services vision. Although communication was often cited as a crucial leadership characteristic, especially during change initiatives, Group 3 participants reported that higher-level leaders had not communicated a well-articulated vision of the initiative. As a result, these front-line leaders felt unable to adequately convey the purpose of Shared Services to their own staff. This concerned them for two reasons: first, they felt it inhibited their ability to fully engage front-line staff in work related to Shared Services. Second, they were concerned about the veracity and accuracy of the information they had provided to staff concerning potential job losses. Indeed, concerns about job loss and potential impacts of the reform on employees were the most dominant themes in the open coding for all three groups. However, because these concerns were voiced in a variety of ways, this important theme would have been missed if we had relied only on the deductive codes derived from the NVivo word frequency search. It is also notable that Group 3 participants at the lowest level of authority in the study felt sufficiently comfortable to critically discuss senior leadership and express their fears about the reform. Participants had the opportunity to withdraw such statements through the post-interview confidentiality form; therefore, their decision to ultimately provide the information indicates trust in the confidentiality of the process.
The Delphi questionnaire presented potential strategies to address the problem of vision. Many participants responded that increased communication was a key part of the solution. As a result of these observations, four different options (all focused on different forms of communication) were presented in the Delphi questionnaire as possible strategies to address the vision problem. Notably, all groups selected a particular communication strategy—namely, increased communication between the central Shared Services office and the CEOs of the health regions—as the most ideal means to improve communication of the Shared Services vision. The least popular strategy was to increase communication between the Shared Services office and the Ministry of Health, possibly because this would bypass the health regions and might therefore be seen as less transparent.
The second challenge identified was engagement, which was defined by most as ensuring “buy in” and involvement amongst health system actors at all levels. There was a dramatic difference between Group 1 (i.e., those at the highest leadership level) and Groups 2 and 3 in the emphasis placed on the importance of engagement, with the latter two groups emphasizing it more strongly. Again, potential strategies were presented in the Delphi report for participant feedback. The most popular strategy choice among interviewees in Groups 2 and 3 was to increase CEO engagement with employees and stakeholders in the health system. For participants in Group 1, the most desirable strategy was to increase engagement between the central Shared Services office and the front-line leaders in Group 3.
A third challenge, as mentioned previously, involved political will. Shared Services is a controversial health restructuring initiative with potentially negative effects on health system employment and private sector as well as community-based suppliers of goods and services to individual health regions. Leaders in all three groups mentioned their concern about the government's political willingness to continue the reform into the implementation stage, once resistance becomes more evident to the politicians. Again, the Delphi questionnaire provided four potential strategies for addressing this leadership challenge. All three participant groups selected the same order of preference for the suggestions, with the most popular being a better clarification of the Shared Services vision. The consensus across groups helped to reinforce the observation that vision is perhaps the most important challenge in this particular case.
Finally, personal leadership was identified as a fourth challenge in the study. We observed some considerable weaknesses in terms of leaders' self-awareness. For example, many were unable to discuss their own leadership or provide examples based on their experience. Some had difficulty finding the words to describe leadership, which supports Dickson's (2009) assertion that “for many people, the qualities and actions that define quality leadership [are] elusive” (p. 296). Nonetheless, the recognized importance of leadership is growing in both the scholarly literature and in health institutions (Avolio, Walumbwa, & Weber, 2009; Dickson, 2009), and there is a documented link between personal leadership and long-term improvement in health services (Boaden, 2006). Many leaders across all three groups identified or requested future learning opportunities that would improve their own leadership development and enhance their change management skills. The Delphi questionnaire presented four potential areas for leadership development. Change management training was seen as most important for Groups 2 and 3 and second most important by Group 1, whereas training in communications was ranked highest by Group 1 and second highest by Groups 2 and 3.
Evaluating and Elaborating the LEADS Framework
The Delphi method also provided important insight into health leadership behaviours and strategies, and particularly the LEADS framework. This served to further define and delineate the elusive concept of leadership. Participants had been asked two types of questions in the interviews: very general, open-ended questions about effective leadership strategies and also closed, Likert-type scale questions based on LEADS. The open-ended questions allowed for an inductive, “organic” evaluation of the framework and the closed scale questions facilitated a more pointed and explicit evaluation of LEADS.
The open-ended questions, which were asked first, provided an opportunity to inductively test the leaders' own experiences and perceptions against the LEADS framework. For example, participants were asked: “what leadership capabilities do leaders need in order to initiate and to implement health system change?” Their responses to all questions were coded and the results were compared against LEADS to identify any gaps or particularly important elements of the framework. Coding revealed that participants emphasized two elements of LEADS in particular: “engage others” and “develop coalitions.” Several leadership characteristics that are not explicitly stated in the LEADS framework (but which fall within its general parameters) were also identified through the open coding; specifically, (a) credibility and trustworthiness, (b) long-term vision, (c) commitment and perseverance, and (d) humility and flexibility. These “new” characteristics were then included in the round two Delphi questionnaire to further explore their role in Shared Services and health leadership generally.
In contrast to the open-ended questions, the Likert-type scale questions were used to quantitatively evaluate the items already contained in the LEADS framework. Interview participants were asked to rate the importance placed on each LEADS characteristic by both themselves and other leaders involved with Shared Services. They were asked to use a scale that ranged from 0 (no importance) to 10 (the most importance) to indicate how much importance they had actually (not ideally) placed on each characteristic in their own leadership practice during Shared Services. Although many leaders commented on the importance of all the characteristics, the exercise showed which LEADS characteristics are actually emphasized by leaders experiencing time pressures in a real change scenario. Basic descriptive statistics (i.e., means and standard deviations) were calculated for the quantitative data. Overall, in both the self-assessment and assessment of others, the LEADS characteristic “modelling qualities such as integrity, honesty, resilience and confidence” was rated highest.
Conclusion
In this article we have presented a rationale and research design using a modified Delphi method in a PAR project on health leadership. The Canadian health system—like other publicly funded health systems globally—currently faces particular challenges at the intersection of accessibility, quality, and resource availability. These challenges often necessitate restructuring and change. Therefore, today's health leaders must lead in new and changing conditions, which include growing organizational complexity (Ford, 2009), distributed leadership structures (Buchanan et al., 2007; Chreim et al., 2010; Currie et al., 2011), and strategic change initiatives (Caldwell, Chatman, O'Reilly, Ormiston, & Lapiz, 2008). Under these conditions, traditional notions of managerial and administrative leadership are insufficient (Braithwaite, 2008). Scholars have identified a need for research on the concrete, experiential aspects of doing leadership in changing systems (Braithwaite, 2008; Denis et al., 2010). Our findings contribute to this literature by identifying the key challenges experienced by leaders engaged in a major restructuring initiative and identifying leadership strategies and solutions that helped them address these challenges.
Although the success of change initiatives often depends on the capabilities of health leaders at various levels, the conceptually elusive and often context-dependent nature of leadership makes it difficult to identify the characteristics most conducive to positive change. As one regional node in a pan-Canadian study on leadership in health system restructuring, this study sought to identify the characteristics of successful leadership in a particular case of health restructuring, and to test the existing knowledge on best practices in health leadership contained in the LEADS framework.
The study employed a modified Delphi method with the participation of health leaders as research partners. The method helped to ensure the involvement and investment of health leaders and the applicability of the results for those involved in this ongoing initiative. The method also led to issues around participant confidentiality, which were mitigated by using an adapted form of Kaiser's (2009) post-interview consent form.
This article has shown the modification of Delphi methodology for use in a significantly more open-ended and qualitative fashion than has previously been the case. With the modifications suggested, the method is well suited for participatory action research and for studies of change initiatives occurring in real time. This indicates that the Delphi method is not only suited to quantitative and consensus-building research but can also be applied to qualitative, participatory research.
There is a need for more research using a modified Delphi approach in participatory research in other contexts. Furthermore, there has been a tendency in existing Delphi literature to gloss over the qualitative data processing methods used between rounds. The detailed coding process described here helps to address this gap in the literature. This article has provided researchers with insight on the conversion of qualitative interview data, using qualitative data-processing software, into an open-ended Delphi survey. In the future, Delphi researchers should be careful to provide a detailed description of the process occurring between the first and second rounds of Delphi, wherein data are often converted from open-ended to survey or questionnaire format. This would not only enhance researcher transparency and accountability but would also provide useful strategies for other researchers using the Delphi method in qualitative research.