Abstract
Health services research is multifaceted and impacted by the multiple contexts and stakeholders involved. Hence, large data sets are necessary to fully understand the complex phenomena (e.g., scope of nursing practice) being studied. The management of these large data sets can lead to numerous challenges in establishing trustworthiness of the study. This article reports on strategies utilized in data collection and analysis of a large qualitative study to establish trustworthiness. Specific strategies undertaken by the research team included training of interviewers and coders, variation in participant recruitment, consistency in data collection, completion of data cleaning, development of a conceptual framework for analysis, consistency in coding through regular communication and meetings between coders and key research team members, use of N6™ software to organize data, and creation of a comprehensive audit trail with internal and external audits. Finally, we make eight recommendations that will help ensure rigour for studies with large qualitative data sets: organization of the study by a single person; thorough documentation of the data collection and analysis process; attention to timelines; the use of an iterative process for data collection and analysis; internal and external audits; regular communication among the research team; adequate resources for timely completion; and time for reflection and diversion. Following these steps will enable researchers to complete a rigorous, qualitative research study when faced with large data sets to answer complex health services research questions.
Introduction
Healthcare systems are very complex, multi component systems that are continually evolving. Given the complexity of these systems, health services research is multifaceted and impacted by the multiple contexts and stakeholders involved. Even when researchers study only a particular component of the healthcare system (e.g., scope of nursing practice in acute care), multiple contexts are encountered and many participants are included to better understand the complex phenomena being studied. Health services research on scope of practice is not well established and, given the research questions, qualitative research is often the focus of such exploratory research. Because data collection may occur across a number of sites by more than one research assistant, research teams encounter logistical issues and have difficulties maintaining predetermined timelines. The end result can be a very large amount of qualitative data that must be analyzed and interpreted appropriately to ensure that an accurate synopsis of the results will be presented. Organization of data and attention to rigour are essential when working with such large qualitative data sets. This article describes the management of a large qualitative data set generated from the research study entitled “A Systematic Approach to Maximizing Nurses' Scope of Practice.” More specifically, the purpose of this article is to reflect upon and describe the processes through which the research team managed a large qualitative data set to ensure that the final product would be judged as rigorous. One of the authors, Nelly D. Oelke, has considerable experience in qualitative research. This expertise and the application of the literature on qualitative data analysis guided the structures and processes used to collect and analyze our data. This article contributes to the literature about managing large qualitative data sets by providing concrete steps for ensuring rigour in data collection, analysis, and interpretation.
Background
Trustworthiness and data management are vital to the success of qualitative studies. Although literature on maintaining rigour in qualitative research is abundant, few articles have tackled doing so with large qualitative data sets (Knafl & Ayres, 1996) and few researchers have documented their process. A search of the literature was conducted and confirmed these findings. Search terms used for our literature search included qualitative research, data management, large data sets, and rigor, with coverage of the following databases: MEDLINE, CINAHL, and PsycINFO. Guba (1981) and others (Johnson & Waterfield, 2004; Whittemore, Chase, & Mandle, 2001) recommend general methodologies to ensure rigour in qualitative research. Although the descriptions of methodologies in the literature vary, most involve steps to maintain credibility, dependability, transferability, and confirmability (Guba, 1981).
To maintain credibility (Guba, 1981) or authenticity (Whittemore et al., 2001), researchers must adhere to methods accepted as scientifically sound in the qualitative and informational sciences. While transparency of methodology is important, Sandelowski (1997) cautions against focusing only on methods. Rather, researchers should maximize data utility to answer the research questions. The researcher must have a satisfactory cultural familiarity with the participating institution and use a comfortable approach in recruiting participants so that the sampling process is random and unbiased (Guba, 1981). Moreover, participants' input must be honest, clearly recorded, and accurately presented (Whittemore et al., 2001).
Dependability and transferability are related in that both ensure all research design and operations are clearly identified (Guba, 1981). These steps also allow for replication of the methodology with a larger population or by future researchers. However, it is important to differentiate between the dependability of a method in producing similar interpretations and the reliability of a method in producing identical results. Qualitative research focuses on describing participants' experience as accurately as possible (Sandelowski, 1997), rather than using numbers to describe the phenomena of interest. According to Sandelowski (1997), interpreting the results, providing valid applications of the findings, and accumulating knowledge as a foundation for other studies are essential for validating data from a qualitative study. Johnson and Waterfield (2004) explain:
Qualitative data are descriptive, unique to a particular context and therefore cannot be reproduced time and again to demonstrate ‘reliability’(Bloor, 1997). Instead of trying to control extraneous variables, qualitative research takes the view that reality is socially constructed by each individual and should be interpreted rather than measured; that understanding cannot be separated from context. (p. 122–123)
Whittemore et al.'s (2001) framework for enhancing rigour includes criticality and integrity components, which for Guba (1981) and Johnson and Waterfield (2004), are included as components of confirmability or an audit trail. It is recommended that researchers keep an accurate, comprehensive record of the approaches and activities employed in the study, both in data collection and analysis. This record includes highlighting shortcomings of the study in the research report and providing transparent links between study results and actual experiences of the participants in the study (Guba, 1981). Such audit trails not only provide a solid methodological reference for the reader, but also provide an opportunity for reflective reasoning (on themes or categories chosen, interpretations, etc.) and criticism for the researchers as the study progresses (Guba, 1981; Johnson & Waterfield, 2004; Whittemore et al., 2001). For example, if methodology changes at some point in the study, an audit trail would keep a record of when, why, and what changes were implemented. Such audit trails become especially useful in the management of large databases and for placing data points, methodology, and interpretation within the particular context in which they belong.
Knafl and Ayres (1996) offer researchers two data management steps for handling larger qualitative data sets. First, case summaries can save researchers great time and logistical resources while decreasing error. Core study researchers would summarize focus group or interview transcripts to a fraction of their original length, and also include relevant data organized into themes agreed upon beforehand as part of a summary guideline. This step not only allows core researchers, who will be interpreting the data, to work closely with the data, but also allows for critical insight. As a complement to the case summaries, it is recommended that researchers tackling a large data set create matrices using database management systems. Guided by themes and questions identified in the study, matrices provide a visual display of the data, including extracted themes. Such matrices simplify the data for researchers' discussions, while the case summaries provide more details on the data. Moreover, data can be reorganized quickly using electronic matrices, allowing for various perspectives and discussions on study outcomes (Knafl & Ayres, 1996).
Ensuring rigour in qualitative research is a priority when collecting, presenting, and interpreting data. Larger qualitative data sets can present a critical challenge for researchers in maintaining study trustworthiness and, therefore, special guidelines must be strictly followed to ensure transparency, logical reasoning, and criticality. As few sources in the literature have suggested methodology for managing large qualitative data sets, this article aims to outline the methods followed by our research team to maintain rigour in such circumstances.
Description of the Study
Numerous reports have highlighted the need to address the under-utilization of health human resources by maximizing professional scopes of practice (Advisory Committee on Health Human Resources, 2002; Fyke, 2001). The need to clarify and define the nursing scope of practice was recognized in Canada as well as internationally. However, there was a void in the research literature in terms of describing scope of practice (being able to practice to the full extent of one's education, knowledge, and experience) and examining barriers to the enactment of full scope of practice. This research study was unique in that it examined scope and boundaries in the practice of various categories of nursing personnel simultaneously, for example, registered nurses (RNs), registered psychiatric nurses (RPNs), and licensed practical nurses (LPNs). The overall goal of this research was to make rich and robust conclusions about the scope of practice of nurses, the barriers to and facilitators of scope, and the impact of contextual factors on scope of practice. Research findings have been reported elsewhere (Oelke, White, Besner, Doran, McGillis-Hall, & Giovannetti, 2008; White, Oelke, Besner, Doran, McGillis-Hall, & Giovannetti, 2008).
Study Methodology
This research study used a descriptive exploratory design with mixed methods (Creswall, 2009) to explain enactment of scope of practice among all categories of regulated nurses (e.g., RNs, LPNs, RPNs). Both quantitative (e.g., questionnaires) and qualitative (e.g., interviews with a variety of stakeholders) data were collected, and one informed the other in the data analysis and interpretation. This article focuses on the qualitative data set. To make our experience of managing this large qualitative data set truly transparent, underlying foundational components of the study will be discussed. According to Sandelowski's (2000) classifications, our study was a qualitative descriptive study. The methodological underpinnings of this study were eclectic. The qualitative component drew on tenets (e.g., importance of the setting and context, purposive sampling, and inductive analysis) situated within the naturalistic paradigm espoused by Lincoln and Guba (1985) and Miles and Huberman (1994). A subcomponent of the study (quantitative data from surveys and regional corporate databases) was positioned within a positivist paradigm. This latter component will not be addressed in this article.
Research questions focused on nurses' and other healthcare providers' perceptions of nurses working to their full scope of practice. Participants were also asked to identify personal, professional, and organizational barriers or facilitators that enabled or hindered their ability to work to their full scope of practice. These types of questions are typically associated with qualitative descriptive studies (Sandelowski, 2000). Data were collected on 14 acute care nursing units located within three western Canadian Health Regions. To ensure variability in sampling, patient care units from hospitals of various intensities and representing variability in provider and organizational characteristics were selected across the health regions to participate in the study. Types of units included in the study were intensive care, medicine, surgery, and psychiatry.
Individual, face-to-face, semi-structured interviews were conducted to gather information on enactment of nursing roles and perceived facilitators and barriers to maximizing scope of practice. A purposive volunteer sample of nursing personnel (e.g., RNs, LPNs, RPNs, and Patient Care Managers) and inter-professional healthcare team members were recruited on the study units. Patient interviews were also conducted with a small sample of volunteer patients from each health region to validate the extent to which patient experience reflected the expected focus of nursing defined in scope of practice documents. A total of 236 interviews were audio-recorded and transcribed. There were 167 interviews conducted with nursing staff: 85 RNs, 31 LPNs, 11 RPNs, 19 patient care managers and assistant patient care managers, and 21 nurses in specialized roles (e.g., nurse educators and nurse clinicians). The remainder of the interviews were completed with other healthcare providers (e.g., physicians, social workers, and physiotherapists) and patients. The establishment of rigour in this study became a daunting task when a large number of nurses and other healthcare providers were interviewed to discuss enactment of scope of practice of nurses and the influence of the work environment and other structures and processes on role enactment.
Ensuring Trustworthiness of the Data
Our study presented a myriad of challenges to ensure trustworthiness of the data. These challenges included collecting data from multiple sites using different research assistants, developing a process for analyzing the data, recruiting and retaining qualified individuals to complete coding and initial analysis, and completing in-depth analysis of the data. Despite the many challenges encountered while managing this large qualitative data set, we succeeded in reporting research results that met the criteria for data trustworthiness. The following sections of this article will outline the challenges and how they were addressed in handling the large amount of data collected in this study.
Credibility or Authenticity
First, participant recruitment was an important aspect to ensure credibility of research results. Participant selection required the incorporation of multiple perspectives (e.g., RNs, LPNs, managers, and interdisciplinary team members) to provide a clear and broad understanding of nurses' and other healthcare providers' perceptions of nursing scope of practice. Although participants volunteered to be interviewed, researchers facilitated a diverse range of perspectives by presenting in both posters and unit presentations the importance of capturing varied perception of scope of practice. Maximum variation (Polit & Hungler, 1999) was sought when recruiting units for the study. Variability in organizational characteristics, hospital intensity, and a variety of patient care units was desired (Aita & McIlvain, 1999; Morse & Richards, 2002). Once hospital units were selected, participants were recruited for interviews with attention to maximum variability in representing the various perspectives of nursing scope of practice. Given the geographical nature of the study, as well as the nature of the clinical environment, it was not possible to limit interviews to one unit with one group of providers at a single point in time. Rather, interviews were completed on various patient care units with a variety of providers in the same time frame, depending on their work schedules. Initially, both research assistants and several research team members were concerned that this process would result in more interviews being conducted because of the uncertainty about data saturation. However, bi-weekly discussions with interviewers confirmed that while they were hearing similar elements, data saturation had not been reached. The variability in units, desired variability in participants, and lack of data saturation until later in the study all led to the recruitment of a very large sample of participants for the study, which created a situation requiring the management of a large qualitative data set.
Second, the consistency of data collection was important to ensure credibility of research results. Data collection was conducted across three health regions with multiple research assistants. Semi-structured interview questions focused on components of the overarching research questions and guided the interview process with nurses and interdisciplinary team members (Sandelowski, 2000). These interviews were conducted by three different research assistants. To standardize interview tracking and scheduling and the entry of demographic data, an Access™ database, along with a user manual detailing the process and technical applications, were provided to each study site. To further establish consistency in data collection (Aita & McIlvain, 1999; Morse & Richards, 2002), a two day training session was conducted by the project manager with the research assistants to discuss the interview protocol, review interview questions and cues, address the concept of data saturation, and discuss the purpose of writing brief field notes following interviews. A second component of the training session was the completion and review of several interviews with each of the research assistants. Feedback about interviewing techniques (e.g., paraphrasing, clarity, utilization of cues for questions, and getting the interviewee to elaborate on their responses) was provided to each of the research assistants. At this time, research assistants also shared their experiences in completing the interview and provided valuable feedback contributing to the clarity of the interview questions and additional prompts for the questions. A training manual was also provided as a resource for the interviewers to complement training sessions. Given the magnitude of the study, research assistants were reminded to limit the number of interviews (3–4) completed in one day to avoid interviewer burden. Despite the focus on consistency of data collection, opportunity was provided through the field notes to co-author results as noted in Kvale (1996).
Finally, presentations were made to participants to ensure the credibility and authenticity of the research results. Ten to fifteen presentations were made to various groups of participants (e.g., nurses, allied healthcare professionals, Patient Care Managers, senior health leaders, and Chief Nursing Officers) wherein results were validated by participants.
Dependability and Transferability
Data Processing and Cleaning
Consistency between transcribers was ensured by utilizing a consistent template, which permitted easy transfer of documents into N6™, a computer program designed for qualitative data storage, indexing, and theorizing. Transcription guidelines to standardize expressions and formatting were provided to each transcriptionist. To ensure the accuracy of the transcription, each of the research assistants reviewed the transcripts while comparing them to audio files. Minor discrepancies, such as spelling errors and clarification of acronyms, were made in the transcripts following review by the research assistants.
Data Analysis
Preparation for the data analysis component of the study was intense, time consuming, and multi-focused. A phased approach (Gaskell & Bauer, 2000) was used in data analysis, with the completion of coding and initial analysis prior to in-depth analysis of the data. A conceptual framework was developed by the research team to begin a content analysis process. As an initial step in the development of the framework, two research team members and one research assistant independently reviewed four interviews to identify preliminary themes. The second step in the development of the framework was to have two of the original research team members and two new research team members independently analyze four new interviews utilizing the existing framework. With the analysis of this second set of interviews, consistencies were found in the themes identified between the team members. However, new themes also emerged resulting in an expansion of the conceptual framework. Miller and Crabtree (1992) have described this approach to content analysis as a “template” style (p.18). The conceptual framework served as the initial tree node structure to begin coding interviews in the N6™ software program.
As consistent with qualitative methods, the categories and nodes identified were not considered static. Several iterations of this conceptual framework and tree node structure evolved, particularly in the early stages of coding and analysis by the coders and the research team (Gaskell & Bauer, 2000). In developing the evolving tree, particular attention was paid to the semantic relationships of the parent and child nodes. A reference document defining each node and indicating placement in the hierarchy of the tree structure was developed and modified to reflect coding team discussions. This process assisted the dependability of the analysis of the large data set, which occurred across multiple coders. Documentation of the changes and the rationale for changes were maintained to establish an audit trail (Lincoln & Guba, 1985).
Recruiting and retaining qualified individuals to do coding and initial analysis was both an important and difficult task (Richards, 2005). Coders were recruited through the university and connections with other qualitative researchers. Coding was accomplished through collaboration and strengthened by the varying perspectives of multidisciplinary team members with backgrounds in clinical and academic nursing, psychology, social work, occupational therapy, and health services research. Most team members had previous qualitative research experience. A half day of training was provided to all coders with ongoing consultation and assistance provided by various members of the research team. Binders describing the technical aspects of N6 ™ were developed for each coder. Initially the coders completed the coding of the same two interviews. Coding was compared and the coding tree was discussed. Early in the coding process weekly meetings were held with the coders; as the study progressed meetings were decreased to every two weeks. These meetings provided an excellent opportunity both to discuss the development of new themes and to question and confirm saturation of themes.
In-depth analysis was completed by two experienced qualitative research team members with different healthcare backgrounds (nursing and occupational therapy). Analysis was completed by provider group; each researcher examined the data for specific groups of providers (e.g., RNs, LPNs, etc.). While each of the researchers examined descriptions of nursing scope of practice and barriers and facilitators to enactment of scope of practice, patterns across the data were also examined (Richards, 2005). Assigning data sets to different researchers (Gaskell & Bauer, 2000) was seen as an appropriate and manageable approach to in-depth data analysis of this large data set. Data overload, fatigue, and the potential for the researcher to “get lost in the data” posed real challenges for the analysis of the large amount of data collected for this study. Dedicating two researchers, who met and discussed regularly, to the process of in-depth analysis assisted in managing these challenges. Researchers frequently documented their processes and interpretations as memos directly in the software program. Summary analysis documents were also created for each of the data sets analyzed. Meeting regularly was also important in making meaningful sense of all the data in this study.
Although N6™ was an excellent program for organizing the qualitative data, challenges were encountered in merging projects between coders (initial analysis) and again between researchers once the in-depth analysis was completed. QSR Merge™ is designed to merge one project with another project, but with the coders and researchers each developing separate N6™ projects (five different projects in total) merging was not seamless. Duplication of transcripts and codes required that one individual be associated with one transcript to prevent duplication in the final N6™ for the complete analysis of all the data for the study. Although challenges were encountered in merging the projects, we were successful in creating one complete N6™ project.
Confirmability
Confirmability (Guba, 1981; Johnson & Waterfield, 2004) of research results was ensured via four key processes: the creation of an audit trail; an internal audit; an external audit; and the writing of the final research report.
Audit Trail
A detailed, comprehensive accounting of all data collection and data analysis activities was completed. Changes were documented as they were made, along with rationale for the change.
Accurate and comprehensive records of the methods employed in data collection and analysis by researchers in the study were recommended by qualitative research experts (Lincoln & Guba, 1985; Sandelowski, 2000). Such audit trails provided not only a solid methodological reference for the reader, but also provided an opportunity for reflective reasoning (on the themes or categories chosen, interpretations, etc.) for the researchers as the study progressed (Guba, 1981; Johnson & Waterfield, 2004; Whittemore et al., 2001). For example, if methodology changed at some point in the study, an audit trail would keep a record of when, why, and what changes were implemented. Such audit trails became especially useful for managing large data sets and placing data, methodology, and interpretation within the particular context in which they belonged.
Internal audit
Internal audits of coding and themes for the study were completed at three different intervals (after 10, 25, and 45 interviews were coded) during the analysis of the study. The purposes of these audits were to assess inter-rater reliability and to determine similarities and differences in key themes identified by coders and auditors. Audits were conducted by three auditors, each members of the research team. The audit included interviews of nurses (RNs, RPNs, and LPNs), nurse managers, interdisciplinary team members, and patients. The sample of interviews was based on a stratified selection by profession, education, unit, and health region to ensure maximum variability of codes and themes. Transcripts for review were then randomly selected from these stratified data sets. Internal audit results are outlined in Table 1.
Internal audit results
a=inter-rater reliability conducted on three randomly selected interviews from audit; compared coding of coders to coders and auditors to coders
b=inter-rater reliability conducted on one interview; compared coding among auditors and one coder
Overall, the internal audit showed positive results in inter-rater reliability of coding and common themes identified from data analysis. Although discrepancies were found between coding completed by coders and auditors, of note was the consistency in coding amongst coders. For the research team this reliability emphasized the importance of the regular meetings with coders to discuss node definitions and clarify where data elements best fit in the coding structure. The lack of consistency in coding between auditors and coders was not unexpected. Coding of qualitative data will be largely interpretive in nature; therefore, researchers' insight and language will be highly individual (Morse & Richards, 2002). The important finding in the internal audit was the consistency in the themes identified from the data, which reinforced for the research team that the right course was being pursued and the team should continue data analysis in the manner in which it was being conducted. The internal audit also facilitated the opportunity for researchers to engage with the data.
External audit
An expert in qualitative data analysis completed an external audit of the data. The reviewer was not associated with the study in any way. Audit questions were developed from the work of Flick (2002) and Miles and Huberman (1994) and reflected an assessment of the procedures undertaken in the process of conducting the study. Questions are outlined in Table 2.
External audit questions
Overall, the external audit review was very favourable. The reviewer noted that the sample was selected in a manner such that the units selected and the perspectives of various categories of nurses were obtained. The summary reports provided to the external reviewer by the project manager served as a complementary component to the data managed in N6™; the connections between identified categories and the data were easily accessible in a systematic manner. Team meetings demonstrated that the reports were discussed at length, resulting in decisions based on the data and documentation of changes to the framework.
Furthermore, the review confirmed that inferences made in the data were logical. More specifically, there was sufficient data for the thematic categorical structures of assessment, accountability, responsibility, coordination of care, general tasks, patient safety, patient education, role overlap and ambiguity, autonomy, working to full scope, facilitators, barriers, and recommendations for unit-based change. Conclusions drawn for these codes were very robust. The reviewer did note that data were less developed for the codes of critical thinking, problem solving, isolation, discontent, conflict, respect, and burnout.
The research team was commended for linking all inquiry decisions to the purpose and the strategies of the study. Specific activities, such as attention to the multidisciplinary nature of the research team; bi-weekly meetings with coders; creation of a detailed audit trail; documentation of the coding framework; and execution of an internal audit, were highlighted by the external auditor as important in increasing the trustworthiness of the study. In terms of research bias, while the researchers were commended for excellent use of follow-up questions to collect additional descriptive information, it was suggested that deliberate recruitment of participants who might hold contrary views to the researchers would have strengthened this component of the review. Overall, the research team was commended for the data collection, analysis, and interpretation of this very large qualitative study.
Research report
The final research report was written in such a way as to increase the confirmability of research results. The report highlighted the shortcomings of the study and provided transparent links between study results and the actual experiences of the participants in the study (Guba, 1981). To this end, limitations of the study were outlined and quotations from participants were included to represent themes identified in the study.
Strengths and Limitations
Several strengths of this study were noteworthy. First, given the large number of interviews completed, a robust description of the scope of practice of nurses in acute care and the barriers and facilitators impacting their ability to practice to full scope was clearly evident. During the management and analysis of the data, we, as researchers, were reflexive and engaged in many strategies that assisted us in questioning how our knowledge, position, and experience potentially influenced or shaped analysis and interpretation of research results (Pyett, 2003). When the findings of this study were discussed both formally and informally with nurses and other professionals from jurisdictions across Canada, we found that the results seemed to resonate with those colleagues. We, therefore, are reasonably confident that the findings from this research represented a current state that potentially characterizes many health care settings.
Several limitations in the methodology to manage the qualitative data for this study were identified. One key limitation was the inability to simultaneously analyze the data in an iterative manner to inform the interview process. This was difficult because data were collected by three research assistants across three geographically diverse sites. Working across sites was particularly challenging. Timelines were also difficult to manage given the magnitude of the study.
Conclusion
Although there were a variety of challenges in managing the large volume of data generated by the large number of interviews, the external audit report confirmed for the research team the strengths of the strategies implemented to manage the data and ensure the quality of the data analysis. We believe that the collective attention to data collection and analysis—via the training of interviewers and coders, careful development of the coding framework, expertise of the qualitative researchers in the analysis of the data, and attention to the development of an audit trail— has contributed to a rich description of the scope of practice of nursing providers and the barriers and facilitators to enactment of their scope of practice. Both the internal and external audit also demonstrated the researchers' commitment to remaining true to the findings. As emphasized in the external audit, researchers utilized rigorous methodology both to manage the data and to ensure that the data analysis captured the unique experience of participants (Ayres, Kavanaugh, & Knafl, 2003). These data management methodologies have been employed as a template for other large research studies in which data were collected across sites with multiple interviewers, participants, and coders.
The research team makes eight recommendations to help ensure rigour in the management of large scale qualitative studies. First, the importance of the organization of the study cannot be underestimated. One person must take on the role of managing the study. The organization of staff, scheduling, data collection, data analysis, and the data itself is essential to the success of the project. Second, diligent documentation of data collection and analysis details (e.g., changes in approach and rationale) is required. This responsibility is best assumed by one person on the research team. Third, ensuring a strict timeline for data collection, coding, and analysis is essential. Fourth, make every effort to use an iterative process for data collection and analysis. Fifth, conduct, at a minimum, a comprehensive internal audit at key points throughout the study. We would encourage researchers to undertake an external audit to further increase the credibility of the study. An external audit also provides an excellent learning opportunity for research team members. Sixth, regular communication between team members is critical to ensure quality completion of the study. Regular email contact, phone conversations, and face-to-face and teleconference meetings are recommended. Seventh, adequate resources are required to ensure timeliness and quality of results. Resources include both financial and human resources. Finally, maintain a good sense of humour and build in time to reflect and have fun. The commitment to large qualitative research is enormous and requires a team effort, with diversion from time to time.
There is a lack of scientific literature regarding the structures and processes for managing large qualitative data sets. This article provides concrete examples and recommendations for managing these large scale qualitative studies to ensure rigour of study results. The external audit completed by an expert qualitative researcher validates the processes and confirms the successful management of this large data set and research study. This information will be invaluable as researchers continue to answer complex health services research questions that inevitably result in large qualitative data sets.
Footnotes
Acknowledgements:
We want to thank all the participants for their valuable contributions to this study. In addition, we would like to thank our funders: Canadian Health Services Research Foundation, Calgary Health Region (now Alberta Health Services, Calgary), Capital Health Region (now Alberta Health Services, Edmonton), Saskatoon Health Region, and the University of Calgary, Faculty of Nursing.