Abstract
In this article, we describe an approach for conducting participatory action research (PAR) in a longitudinal multicase study, with particular focus on cross-case analysis. Existing literature has documented the practice of PAR in single-case studies, but far less has been written on how to conduct PAR across multiple cases. There is also a need for instructional examples of multicase study application, particularly methods of cross-case analysis. In PAR, research methods—including data analysis methods—have the power to shape participant inclusion or exclusion, involvement or attrition, and mobilization of knowledge in real time. In response to these challenges, we discuss the analysis methods used in a PAR study of health leadership in Canada. The project, which consisted of six case studies of leadership in major health system change, involved health leaders as collaborators. We address the challenges of doing PAR with collaborators facing time limitations and suggest a project structure for involving collaborators at critical junctures. We present a detailed, two-part method for conducting cross-case data analysis. Our method involved targeted collaborator involvement in data interpretation while also ensuring faithfulness to the coded data. We describe our process for mobilizing study findings through a deliberative dialogue with health leaders.
Keywords
Introduction
As a general philosophy or framework for research, participatory action research (PAR) emphasizes the connection of research with action in a real-world setting, resulting in cogeneration of knowledge between researchers and participants. PAR is multidisciplinary and can be conducted using a variety of methods. Given its dual focus on research and action (Melrose, 2001), PAR methods should uphold standards of scholarly rigor while staying faithful to the experiential knowledge and goals of the participants. This balance can be challenging to achieve (Israel, Schulz, Parker, & Becker, 1998; Jacobs, 2010). In this article, we describe a multicase PAR project that engaged Canadian health leaders from different system levels while respecting their time constraints. We present a twofold process for cross-case analysis that balances collaborators’ interpretation with rigorous qualitative coding.
Health-care delivery occurs along a continuum of care that involves many sectors (e.g., acute care, community, long-term care, and hospice), and although care should be seamless and interconnected, structural, cultural, and political forces fragment care (Denis, Langley, & Rouleau, 2010). As a consequence, this project engaged academic researchers and health leaders located throughout the health system in an effort to collaboratively find ways to reconnect and lead complex health systems within a variety of Canadian health-care contexts.
Case Study and Multicase Study Research
Case study research has been defined as an intensive study of one case to better understand a population or larger class of cases (Gerring, 2007). Case studies can examine individual people, households, events, organizations, regions, or countries, for example (Gerring, 2007; Yin, 2009). The case study framework is useful for “in-depth appreciation of an issue, event or phenomenon of interest, in its natural real-life context” (Crowe et al., 2011, p. 1). Although the importance of single-case studies should not be underestimated (Flyvbjerg, 2006; Gerring, 2007), the use of multiple case studies can reveal common features of a phenomenon that persevere across different contexts, making multicase studies a useful approach for understanding vague or underexplored concepts and for facilitating generalization (Taylor, 2013; Yin, 2009). The process of analyzing across multiple cases is commonly referred to as “cross-case analysis.”
Case study research in general has been criticized for lack of methodological transparency (Gerring, 2007). Yin (2012) argued that of all the social science research frameworks, case study research is perhaps the least well-developed and understood. Case study researchers, therefore, should carefully describe their methods of data collection and analysis to enhance reliability and provide exemplars for other case study researchers (Crowe et al., 2011; Meyer, 2001).
With notable exceptions (Stake, 2006; Yin, 2012), there are few exemplars of applied cross-case analysis in the literature. The unique circumstances of a multicase study project raise different methodological questions than single-case analysis (Houghton, Casey, Shaw, & Murphy, 2013). For example, what is the unit of cross-case analysis—the individual units (e.g., participants) within each case study or the individual case study as a whole (Yin, 2009)? What material should be subjected to cross-case analysis—aggregated raw data from all case studies or case study reports? How should this material be coded and analyzed?
PAR in Multicase Studies
Additional questions are raised when a multicase study is conducted using a PAR framework, but there is remarkably little literature on this combination. PAR involves collaborative knowledge production by researchers and participants. It values experiential knowledge that can be used to address real-life problems and achieve positive social change (Reason & Bradbury, 2008). PAR is often situated in a context where change occurs (Smith, Bratini, Chambers, Jensen, & Romero, 2010). It typically involves a researcher or group of researchers partnering with members of an organization or community, for example, to better understand and enhance organizational or community functions through shared, participatory research (Brydon-Miller, Kral, Maguire, Noffke, & Sabhlok, 2011). In this article, we refer to the health leader partners as “collaborators” and the university researchers as “researchers.”
Many PAR studies are single-case studies focusing on a single organization or community. PAR is less commonly used across multicase studies. Existing PAR literature has shown how to involve participants in single-case studies (e.g., Smith et al., 2010), yet there is less guidance on how to effectively collaborate with participants during cross-case analysis of a multicase study. Another consideration is time constraints on PAR collaborators. Although the findings of a PAR study could prove highly beneficial for time-stressed collaborators (i.e., by identifying key priorities for their attention), those same time stressors limit their ability or inclination to participate. As Israel, Schulz, Parker, and Becker (1998) noted, “The active involvement of all partners in the research process, including questionnaire development, survey administration, and feedback and interpretation of data, exacts a tremendous commitment of time from all participants” (p. 188). In a research approach that emphasizes full participant involvement, what is the best way to involve collaborators without overwhelming them?
In the following sections, we describe an approach for doing participatory cross-case analysis in a multicase PAR project on health leadership. We describe how PAR principles were integrated throughout the project and discuss the strategic inclusion of our busy collaborators at key junctures in the research process. To address the lack of literature on methods for cross-case data analysis, we suggest a mixed inductive-deductive coding technique and we illustrate how the coding was combined with a participatory interpretation process that actively included collaborators.
Collaborative Structure in the Leadership in Health Systems Redesign (LHSR) Project
The LHSR research project was a 4-year, pan-Canadian PAR project examining best practices in leadership during health system redesign. The project arose as a response to concerns from health leaders about growing challenges in contemporary health systems leadership. The key purpose of the project was to assess health leadership capacity in Canada and identify practical ways to improve that capacity using applied research and knowledge mobilization (KM). The specific questions guiding the project were (1) What is the current state of health leadership capacity in Canada?; (2) Where are the gaps between current practices and leading practices?; and (3) How can knowledge of effective leadership be mobilized to enhance the development of quality health leaders?
The LHSR project was guided by the growing recognition that seamless care delivery within and across Canadian jurisdictions depends on collaborative leadership at different health systems levels (e.g., local/micro, provincial or regional/meso, and national/macro) (Dickson & Tholl, 2014a; Lazar, Forest, Lavis, & Church, 2013). In line with collaborative leadership models, the project was structured to ensure collaboration throughout its duration (see Box 1 for an overview of the project’s collaborative structure). LHSR was initiated as a practice-academic research collaboration between 17 health leadership researchers from nine universities across Canada; 14 senior health leaders from the Canadian Health Leadership Network (CHLNet), a not-for-profit network of more than 40 health organizations with a mandate to support and enhance health leadership in Canada; and 5 KM experts. This research collaboration was represented by two co-principal investigators (PIs): a university researcher and a prominent health leader from CHLNet. A secretariat was established to provide high-level governance and oversight of the project. National and provincial research funds and multiple ethics approvals were obtained for the project. Ethics approvals came from all involved universities and, when required, from health organizations with their own research ethics processes. All study participants signed an informed consent agreement. Although in some cases confidentiality was limited by the nature of group activities, confidentiality was ensured in all written documents through the use of pseudonyms.
Selecting and Structuring PAR Case Studies
The Canadian health system is highly complex. It is marked by dispersion of authority and responsibility throughout multiple organizations and levels of government (Dickson, 2009; Dickson, Lindstrom, Black, & Van der Gucht, 2012; Currie & Lockett, 2011). Because of this complexity, the plural term “health systems,” as opposed to the singular health system, is often used in discussions of the Canadian health system. Drawing on their knowledge of current health system transformation initiatives across the country, the LHSR steering group identified five initiatives underway in different geographic regions across Canada (e.g., Maritimes and Prairies). In each instance, initiatives being conducted at the local level with specific health-care organizations (i.e., micro-level) provided opportunities to engage in PAR with regional leadership at the meso-system level (e.g., provincial Ministries of Health) and local leadership at the micro-system level (e.g., hospital administrators). To understand health leadership at the macro-systems level, a sixth case study examined the initiatives on access, quality, and appropriateness (AQA) of health-care delivery at the national level. The steering group identified national health leaders to discuss their enactment of this AQA agenda.
After these six cases were identified, the steering group and secretariat recruited local university researchers and health leaders (i.e., collaborators) to study each case. The local researcher–collaborator case study teams reported regularly to researchers and collaborators from the steering group with regional affiliations. Table 1 provides an overview of the six case studies with brief situational context and examples of their study methods.
Overview of the Six Case Studies and Their Situational Contexts.
Together, the five local case studies and the national case study formed the six cases of the pan-Canadian LHSR project. Each case study team was responsible for developing and conducting a PAR study with significance to health-care leadership and transformative systems change. For each case, the case study teams purposively identified study participants to interview; participants from a variety of positions, perspectives, and levels of authority were chosen as long as they were involved in the change initiative. Although the collaborators also served as participants (e.g., by participating in interviews), not all participants served as collaborators.
Local health leaders involved with major change initiatives were well positioned to serve as collaborators in this project: they were experiencing the leadership dynamics involved in major system changes in real time. Epistemologically, such experiential insight is highly valued as a form of knowledge in PAR (Gayá Wicks, Reason, & Bradbury, 2008). Participation in this project also provided real-time information to busy health leaders facing persistent leadership challenges. For example, there were many instances in which individual participants reported on how they used the results from study cycles to change their own leadership practice. Collaborators also appreciated opportunities to reflect on their leadership styles and actions with regional health leaders and researchers from the steering group.
Method: Multicase PAR at Multiple Health Systems Levels
Working at different systems levels is a key challenge in multicase studies. Researchers must develop the best methods to capture the unique complexities of each individual case study while also identifying broader conceptual themes at the cross-case level. PAR adds the further challenge of ensuring meaningful participation of, and benefits for, collaborators at different levels. In the following sections, we discuss our process for balancing case-level imperatives—such as the imperative to usefully inform collaborators’ actions “on the ground” in real time—with the need to answer broader conceptual questions facing health leaders and researchers across the country. Table 2 provides an overview of our PAR process within three health systems levels of leadership (micro, meso, and macro). PAR for this study was designed to maximize collaborative engagement at these three levels.
PAR at Multiple Systems Levels.
Note. PAR = participatory action research; CCPAR = cross-case PAR.
aThe exception was the national case study conducted with health leaders at the macro health systems level.
Collaborative Structure and Functions in the LSHR Project.
The Pan-Canadian Steering Group: 17 university researchers from each geographic region of Canada; 14 senior health leaders from national/macro and regional/meso health systems levels; and 5 knowledge mobilization (KM) experts. This group initiated and directed the project after obtaining national funding.
The Co-Principal Investigators: A researcher and national health leader from the steering group were responsible for leading the project. They were also members of CHLNet, a Canadian health leadership network with leaders from local, regional, and national health systems levels.
The Secretariat: The co-PIs and three appointed national collaborators and researchers from the steering group provided high-level governance for the project.
The Cross-Case PAR (CCPAR) Analysis Team: Researchers and national/regional collaborators from the steering group were charged with overseeing the cross-case analysis process. Three researchers on this team took responsibility for conducting cross-case coding in NVivo.
The Case Study Teams: University researchers and local/micro-level collaborators carried out three PAR cycles related to transformative systems change within their regional/local contexts. Each local team worked closely with steering group researchers and collaborators from their same region (e.g., Prairies).
The Case Study PAR Process
Semistructured interviewing was the core method used in all case studies to promote a reflective process and informal, genuine conversations. In the early stages of the study, the steering group reached consensus on two tools to facilitate cross-case comparisons: a common template for reporting on each case study and a common set of interview questions to guide each case study. However, case study teams were encouraged to add additional questions to better understand their specific cases, and each case study team was welcome to use additional methods.
In addition to interviews, some research teams employed document analysis, observation (e.g., attending meetings about the change initiative), focus groups, a modified Delphi method (Fletcher & Marchildon, 2014), and critical incident technique (Dickson & Tholl, 2014b). Different methods were chosen depending on the unique situational context for each case and the best way to enable participants to share meaningful experiences related to health reform (see Table 1). Stringer (2007) advocated for the use of multiple methods to best capture and explore emerging themes through PAR; however, he also recognized that diverse methods can create challenges for cross-case analysis. Nonetheless, the steering group agreed that the situational uniqueness of leadership itself suggests that no specific set of methods will necessarily be relevant in each context; the methodological design should respond to context. Therefore, the cross-case analysis (described below) was designed to analyze data collected through many different combinations of methods.
For each case study, local teams collected data in three cycles over 2 years, with the findings of each cycle informing the next. After each cycle, case reports prepared by the researchers and collaborators were shared more broadly with respective regional collaborators and researchers from the steering group. Regional researcher–leader reviews would often generate feedback to local case study teams via the researchers. This review process across micro and meso levels allowed exchange of ideas about health systems issues between these two levels. In addition to capturing changes in leadership over time, the three research cycles ensured that findings could be mobilized in real time as the initiatives moved forward, allowing collaborators to make local changes as needed to improve their initiatives. This cyclical structure also aligned well with the PAR cycle of “planning, acting, observing, and reflecting” (Melrose, 2001, p. 162).
The Cross-Case PAR (CCPAR) Process
After completion of the six separate case reports, the cross-case analysis brought two distinct challenges: (1) how to analyze data collected in six different case studies using different methods of data collection and (2) how to analyze the data in a scientifically rigorous but participatory way that included busy collaborators who lacked the time, interest, and specialized skills to do conventional qualitative coding. In response to these challenges, we designed a unique two-part data analysis strategy. We began with a mixed inductive–deductive coding approach using NVivo9 software to organize data and compare findings across case study reports. Next, in-depth analysis and interpretation occurred at a 1-day participatory interpretation session that included researchers and collaborators from all system levels. The following sections describe this analytic/interpretative process. We conclude with a discussion of our KM based on a deliberative dialogue with national health leaders and our critical reflections of this project design.
Part 1: Mixed inductive–deductive coding of case reports
In qualitative research, a “code” is a summative word or phrase that captures the nature or character of the data (Saldaña, 2013). Coding is the process of organizing data into codes, which are then combined into broader, interpretive themes; this process is sometimes referred to as thematic analysis (Fereday & Muir-Cochrane, 2006). There are two main approaches to coding: inductive and deductive. Inductive coding is a data-driven, “bottom-up” approach in which the researcher does not begin with any preexisting codes and instead looks for repeating ideas in the data, which are gathered to become codes (see, e.g., Auerbach & Silverstein, 2003). In contrast, deductive coding is theory driven (Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005; Gilgun, 2011); it begins with a preliminary set of codes, which are usually drawn from theory or existing research. Deductively derived codes can be used flexibly and new codes added as necessary to account for additional ideas that emerge as coding proceeds (Gilgun, 2011).
In his guidebook on case study research, Yin (2009) described a deductive coding technique in which the researcher created an initial list of codes. This list was then used to drive a word frequency search and other automated functions with computer-assisted qualitative data analysis software, such as NVivo. Although Yin (2009) acknowledged that the researcher must provide a rationale for inductive or deductive coding, his text does not examine the methodological implications different coding approaches can have for case study research. Indeed, Dixon-Woods, Agarwal, Jones, Young, and Sutton (2005) argued that, “the failure of much writing on thematic analysis to distinguish adequately between these two approaches has resulted in a lack of transparency” in the literature on thematic analysis (p. 47).
The LHSR project was exploratory in nature. The goal was to articulate and better understand the practice of leadership, which is an underexplored and often vague concept. For this reason, deductive coding—which builds upon existing literature and research—was not entirely appropriate. At the same time, inductive coding across six case studies would have produced an unmanageable amount of codes, adding to the difficulty and complexity of cross-case comparison. Therefore, we selected a mixed inductive and deductive approach. Limited research is available on such mixed coding. In their study of performance evaluation in nursing, Fereday and Muir-Cochrane (2006) presented a coding model they described as a hybrid of inductive and deductive coding. They organized data according to research questions using a predefined codebook, and they added other codes for newly emerging ideas not captured by predetermined coding structures. According to Gilgun’s (2011) definition, however, this approach is still deductive. Even deductive coding requires the researcher to treat predetermined codes flexibly to allow for new ideas to emerge (Gilgun, 2011).
Unlike deductive approaches, we did not have a predefined codebook based on our research questions or existing literature. After each PAR cycle, the researchers for each case were responsible for inductively, thematically coding the raw data (e.g., interview, focus group, or documentary data) using NVivo. Next, researchers and collaborators in each case study team met to review the logic inherent in the themes (i.e., did the language of the theme make sense in the decision making world of the collaborators?); and substantiated their themes with rich descriptions and exemplar quotations drawn from the qualitative data.
After all six case study reports were completed for Cycle 1, a team of researchers and collaborators from the steering group (hereafter called CCPAR team) took charge of the cross-case level of analysis. The unit of analysis for the cross-case coding was the case study report (Yin, 2009). There were two key reasons for the choice to code reports instead of aggregated raw data in the cross-case analysis. First, the case reports had been coproduced by the case study teams before cross-case analysis. Second, each study used different combinations of qualitative methods, selecting those best suited to the context (Burns, 2010). The use of case reports helped to standardize the presentation of data and facilitate cross-case coding.
Although the case reports were thickly descriptive (Houghton et al., 2013; Lincoln & Guba, 1985), it is important to acknowledge that a certain degree of interpretation had already occurred before cross-case analysis, such as the selection of quotations. Coding of existing reports is an accepted strategy in cross-study synthesis (Rudel, 2008), a type of cross-case analysis that draws on preexisting studies. In Rudel’s cross-study synthesis, the author noted that a fundamental challenge was lack of access to original researchers, participants, and data; for this project, we had access to all three, ensuring we could clarify information in the reports as needed.
Coding of the first cycle of case reports was referred to as “Round 1.” Round 1 of coding began with three coders, a subset of the CCPAR team, independently and inductively analyzing the same three case study reports to establish codes based on repeating ideas in the data. The coders met to compare their in-progress coding. Key codes were identified, operationally defined, and compiled into a preliminary list that was used by the coders to deductively code the remaining three reports. Another coder meeting took place to gain consensus on the final list of codes. Using NVivo, the codes were combined and organized into subthemes and broader themes associated with each of the main research questions guiding the project.
Next, the CCPAR team met to discuss the cross-case analysis findings with respect to the study’s broader research questions. To assist the CCPAR team with this process, all subthemes and themes were placed into an Excel matrix. Rich background context and exemplar quotes were selected from across the six case reports; this helped to maintain connection between the cross-case themes and the unique contextual factors operating in each case study (Stake, 2006). We also included frequency counts to determine the prevalence of themes and subthemes across the six cases. These Round 1 findings were organized as a cross-case report for the steering group.
For Round 2 of cross-case coding, we used the Cycle 1 code list (which had been inductively derived from the Round 1 data) deductively; these deductive codes were treated flexibly and new codes were added as needed. As before, the three coders met to consolidate their codes into subthemes and themes according to the main study questions using merged NVivo projects, and then shared those with the CCPAR team for discussion. The participatory process for generating the final cross-case report will be described in the next section.
Overall, the mixed inductive–deductive coding process was well suited to exploratory research. Drawing codes from existing leadership research could simply have reinforced existing knowledge; this was not the goal of the project and would have negated the participants’ experiences as the key source of knowledge. Instead, all codes were data driven (i.e., inductive). Placing these data-derived codes into a deductive list prevented the three coders from creating an overwhelming number of codes with different names. The flexible use of the deductive list allowed us to capture unique contextual factors within each case study report. Furthermore, the deductive list facilitated comparison over time by easily identifying changes that had occurred on the same theme between the first and last PAR cycles.
Although PAR emphasizes the inclusion of collaborators throughout the project, the local health leaders in our project did not have the available time, interest, or technical expertise to participate in coding. Other PAR researchers have noted this issue. Westhues et al. (2008) did not include participants in coding their PAR study of community mental health projects, arguing that, “We did not understand this to be a lack of full participation in the project, but a way of prioritizing time and valuing the different skill sets that people brought to the collaboration” (p. 714). Smith, Bratini, Chambers, Jensen, and Romero (2010) found that participants became disengaged when they were included in overly abstract or technical conversations about research principles and methods. To mitigate this issue, regional and national health leaders from the steering group acted as collaborators on the CCPAR team, providing feedback and adjusting the language of the themes to reflect terminology used in the health-care leadership world.
Despite the challenges of including participants in cross-case analysis, Somekh (2006) argued that PAR researchers “need to involve [participants] in this process if we can, because analysis from the outside only will necessarily screen out some important insights” (p. 173). The challenge is to engage busy collaborators in data analysis by using methods that are time efficient and already familiar to them. Accordingly, the second part of our data analysis process drew on collaborator expertise from different systems levels through a 1-day participatory interpretation exercise.
Part 2: Participatory group discussion and interpretation
Analysis of research findings does not end with coding. Our second stage of data analysis and interpretation was designed to give collaborators and researchers equal input into the findings of the cross-case analysis. A full-day, in-person meeting was held with the LHSR researchers and collaborators; this purposeful selection of collaborators included health leader representation from macro (national) and meso (regional) levels. Researchers and collaborators from each case study team (micro-level representation) were invited to attend. The meeting began with an affinity diagramming exercise, a technique drawn from project management (Dale, Wiele, & Iwaarden, 2013). In this exercise, representatives (i.e., researchers and collaborators) from each case study summarized their case study’s key findings on colored note cards (one color per case study). Key findings ranged from 5 to 15. All the cards were pasted onto a wall and case study teams’ researchers and collaborators were asked to group the ideas together into related themes.
Next, the case study team researchers and collaborators discussed the groupings until consensus was reached on the best arrangement of themes. Several colors of paper on a single theme indicated that the theme had been found across case studies. Photos were taken to capture the groupings at several stages in the exercise. Collaborators reported that they found the exercise useful because it allowed them to express their interpretations in language commensurate with their experience and to “make meaning” of the data in the context of potential implications for subsequent action. It also gave the collaborators equal opportunity (to researchers) to engage in the production of the final cross-case report (B. Tholl, Personal Communication, November 22, 2013).
After the affinity exercise, all the participants were organized into small working groups and invited to compare the affinity exercise themes to the themes from Round 2, the final round of cross-case coding. In most cases, the cross-case coding matched the themes from the affinity exercise; however, the latter process involved slight linguistic nuances that better reflected the theme in the context of the health leader world. The themes included, for example, the importance of engaging stakeholders in a change initiative, a desire for a common vision to ground health reform, and a need for greater strategic leadership and systems thinking to guide leader behavior. However, the coded data also revealed themes that were not noted during the affinity exercise: one theme, for instance, that was coded frequently in NVivo was “alignment,” particularly alignment between local and regional levels of health systems. In all the final case reports, alignment was identified as a key precursor for transformative health system change, and it was also identified as a major challenge for health leaders. Small group work was followed by large group discussion and consensus-building between researchers and health leaders from different systems levels. The group discussed, for example, whether or not the alignment theme was actually present in the results of the affinity exercise; a decision was made that indeed it was found to be embedded in many of the themes as a “meta-theme.” In this way, cross-case coding allowed us to “catch” an important finding that would otherwise have been missed by the case study team affinity exercise.
The final cross-case report included the last set (Cycle 3) of case reports and a comparison and interpretation of similarities and differences between affinity exercise themes and those themes coded in NVivo by the three coders. The report also included a final interpretation or “answers” to the major research questions of this project. This report was written by the Co-PIs. Our collaborators across systems levels concurred that the findings resonated with the reality of their leadership roles and accountabilities. Collaborators also verified the importance of policy considerations related to the study’s key findings. Final refinements were made before publication of the report via the funding agency. The next step, in keeping with PAR, was to prompt action on the findings.
“Fresh Eyes”: A Deliberative Dialogue
In an effort to convert the findings into action, a structured process called a deliberative dialogue was employed (Boyko, Lavis, Abelson, Dobbins, & Carter, 2012; Moat, Lavis, Clancy, El-Jardali, & Pantoja, 2014). Working with KM staff at the Health Policy Forum at McMaster University, the LHSR steering group purposively selected 20 macro- and meso-level health leader decision makers to participate in the dialogue. Some of these leaders had participated in the project previously and others had not; this helped to ensure continuity while bringing new interpretations to the project findings. These leaders included national and provincial government representatives as well as executive health administrators and health practitioners. They were chosen through a process of selection stewarded by the secretariat and based on a number of criteria, the most important being their ability to “take action” based on the policy implications of the study.
A short issue brief summarizing the results of the LHSR project was prepared for attendees. The issue brief also included guiding questions to consider before attending the event (Moat et al., 2014). The purpose of the event was to bring in fresh eyes to review key findings and to suggest policy recommendations and actionable items from different stakeholder perspectives. At the deliberative dialogue, attendees discussed current challenges in Canadian health leadership, such as the challenge of leading in complex systems. They also made concrete commitments to support networks for leadership development and knowledge sharing (for a detailed report, see Moat & Lavis, 2014).
Although the deliberative dialogue resulted in agreement on the importance of leadership and concrete commitments to foster it, the deliberative dialogue did not produce a specific ongoing mechanism beyond creating networks and commitments. Insufficient time was left in the project after the deliberative dialogue to carry further actions forward on a formal basis (i.e., actions outside of what individual leaders were willing to do). There was no final observation and reflection phase to study the impact of the broader conceptual findings (i.e., the cross-case findings) after the deliberative dialogue. In this way, the cross-case analysis did not fully follow the cycle of “planning, acting, observing, and reflecting” to the same extent as the individual case studies (Melrose, 2001, p. 162). We recommend that future projects consider building in sufficient time to ensure there are ongoing formal mechanisms for continuing reflection and ongoing action across systems levels.
Conclusion
Like most action research, the goal of PAR is to inform practice while contributing to the scholarly knowledge on a topic (Bray, 2000; Townsend, 2013). Each LHSR case study examined a health transformation initiative in real time and over time—providing opportunities for PAR engagement at three health systems levels. Although evidence of collaborative learning is not the purpose of this article, we would like to provide one example of how the PAR cycles made a difference at the local/micro-level. Cycle 1 of the Prairie case study showed that the overarching vision behind the change initiative was not being effectively communicated to local health leaders and practitioners. After reviewing the Cycle 1 findings, one collaborator reported that “the research project has given me a better understanding of the problems with the [change] initiative … that is something that I am addressing now” (participant quoted in Fletcher & Marchildon, 2014, p. 13). There is evidence from at least three of the individual cases that the multicycle PAR process was an effective way to raise critical self-reflection among collaborators, positively influencing leadership behaviors as health systems transformation occurred in real time. We have evidence of similar “ah ha” moments for regional and national leaders who participated in this project.
Collaborators in the LHSR project were engaged in major health systems transformation that demanded a great deal of their time. To prevent attrition over the 4-year PAR project, participation had to be carefully balanced with a consideration of collaborators’ time constraints. To this end, the project was structured to ensure collaborators’ involvement throughout the process (e.g., through the secretariat, steering group, the CCPAR team, and research teams in each case study). The PAR process involved collaborators more heavily at certain key junctures (e.g., case study project design; cross-case interpretation activity).
A key issue in PAR is whether to include collaborators in data coding. Although the LHSR project was coinitiated by the health leader collaborators and their involvement was built into the project structure at many levels, the collaborators lacked the time, interest, and specialized skills to do qualitative coding. Rather than asking busy leaders to spend many hours doing either manual coding or learning how to use coding software, case study researchers took responsibility for coding data. However, in each case study, coded data were returned to collaborators after each cycle for interpretation and cocreation of the case study report.
At the cross-case level, we used a two-part coding and interpretation strategy to ensure collaborators’ equal participation and voice in the analysis process. First, mixed inductive–deductive coding was done by a team of three researchers in consultation with the CCPAR team. This mixed approach involved deductive use of codes that had been developed inductively from the data, which proved to be a useful approach for exploratory multicase research. Second, collaborator engagement was concentrated into a single-day participatory group interpretation event where researchers and collaborators engaged in an affinity exercise to theme the data. The results of the cross-case coding and the affinity exercise were then compared. We found this analytic approach to be a useful compromise between full inclusion or exclusion of collaborators with respect to data coding and analysis. With our approach, collaborators had a voice, and researchers did not have complete power over data interpretation. To gain additional stakeholder perspectives on findings, a final deliberative dialogue was held to mobilize knowledge and generate practical actions.
Some PAR studies have the final goal of implementing a project or initiative. In contrast, the LHSR project was exploratory in nature; it was aimed at clearly articulating the practice of leadership while informing leadership capacity building. However, this meant that little time was left at the project’s conclusion for building mechanisms to ensure ongoing action. The 4-year commitment of researchers and health leaders may indicate the usefulness of PAR for “stretching” beyond the typical research process and informing traditional change management approaches with real-time empirical data, but sufficient time should be allocated for continuing action at the broader (i.e., cross-case) level.
Methodologically, PAR requires a careful balance between scholarly rigor and faithfulness to experiential knowledge. It seeks to fully include participants as collaborators throughout the research project, generating both knowledge and action. Although PAR can be very beneficial for time-stressed participants, their availability is limited. This article has demonstrated how to effectively engage health leaders from different health systems levels as collaborators in multicase PAR, particularly in the data analysis process.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was funded by the Canadian Institutes of Health Research (CIHR) Partnerships for Health System Improvement (PHSI) program.