Calling for a Shared Understanding of Sampling Terminology in Qualitative Research

Abstract

In this article we present the critical analysis of a recent methods overview, authored by McCrae and Purssell, as a means to highlight and address several important ambiguities and misunderstandings associated with terminology commonly used to describe sampling in qualitative research. We share several definitive understandings of sampling-related issues, which have been informed by a rigorous analysis of the methods literature from another earlier methods overview focused more broadly on sampling in qualitative research. Specifically, we address ambiguities and inconsistencies related to what can be sampled in qualitative research (the sampling unit), the concept of theoretical sampling, the term purposeful sampling, the appropriateness of initial sampling in grounded theory, and the need to distinguish between the functions of reporting one’s sampling methods and describing the final participant sample. Finally, we argue that a continued lack of clarity in the language we use to describe what we do erodes the real and perceived quality of qualitative research. We point to the important role of methods overviews both for focusing attention on underdeveloped research methods topics and as a source of solutions to methodological problems.

What is already known?

In their recent methods review, McCrae and Purssell (2016) concluded that only half of purported grounded theory studies from the journals surveyed used genuine theoretical sampling, according to their definition of this method. These authors interpret the meanings of several other terms commonly used sampling-related in qualitative research generally, according to their assumptions and understandings. A separate methods overview published earlier by other authors, which focused on the broader topic of sampling in qualitative research, has provided in-depth descriptions of several relevant sampling-related concepts derived from systematic analysis of the methods literature. This second review provides a basis for proposing clarifications to address ambiguous understandings of common sampling-related terminology used in qualitative research.

What this paper adds?

A more comprehensive understanding of theoretical sampling derived from the grounded theory methods literature is described. Four additional problematic assumptions about sampling-related terminology are identified and solutions proposed in each case, including ambiguous and inconsistent understandings of the sampling unit, the term purposeful sampling, the appropriateness of initial sampling in grounded theory, and the need to distinguish between the sampling process and the participant sample. This article also highlights the value of methods overviews both for focusing attention on underdeveloped research methods topics and for promoting clearer shared understandings of methods-related concepts.

Background

Reviews on methods topics, also called methods overviews, are welcome contributions to the literature because they stimulate important methodological discussion and help to advance research methods. The recently published methods overview by McCrae and Purssell (2016) serves an important purpose in this respect by drawing attention to the proportion of studies whose authors claim to use the grounded theory method, but which do not appear to use genuine theoretical sampling—an essential element of the method. Their review also illustrated, at least for us, some of the variable interpretations of terminology commonly used to describe sampling in qualitative research generally. Variable understandings do not facilitate clear communication, however, and we believe that shared understandings are particularly essential for communication about research methods. In this article, we present our critical analysis of McCrae and Purssell’s (2016) methods overview as a means to highlight and address some of the common ambiguities and misunderstandings associated with sampling-related terminology in qualitative research and to propose concrete clarifications in the hopes of developing a more shared understanding to facilitate communication. Thus, while the primary purpose of the McCrae and Purssell’s (2016) article was the use of theoretical sampling in grounded theory, we direct our analysis and discussion in this article to their use of terminology that has been commonly applied to describing sampling in qualitative research more generally.

The idea for this article was stimulated by a journal club-style discussion at a recent meeting of The Grounded Theory Club—a biweekly meeting space hosted at the University of Victoria where students and scholars from across Canada and beyond gather in person and by videoconference to discuss specific grounded theory research projects and methodological topics (Schreiber, 2001). In discussing McCrae and Purssell’s (2016) overview on theoretical sampling, we found ourselves debating and inspecting those authors’ understandings of sampling-related terminology, which many of us had taken for granted up to that point.

In their article, McCrae and Purssell (2016) reviewed the methods-relevant sections of the reports of published grounded theory studies using a rigorous systematic approach to identifying and selecting studies to review. A central aim of their review was to quantify the proportion of studies whose authors claim to use the grounded theory method that also feature genuine theoretical sampling approach. This is important because, as the authors point out, theoretical sampling is a defining and necessary part of the grounded theory method. The review authors found that only 50% of purported grounded theory studies from the journals surveyed used genuine theoretical sampling, according to their implicit definition of this technique.

Here, we follow up on several important issues related to the sampling terminology used in McCrae and Purssell’s (2016) methods overview that arose as deserving of detailed critical examination. To do this, we draw from several streams of grounded theory and other methods literature to inform and support our discussion. Specifically, we address ambiguities and inconsistencies regarding five topics: what can be sampled in qualitative research (the sampling unit), the concept of theoretical sampling, the term purposeful sampling, the appropriateness of initial sampling in grounded theory, and the need to distinguish between the functions of reporting one’s sampling methods and describing the final participant sample.

A key source of insight for our critical analysis is a recent methods overview on the topic of sampling in qualitative research (Gentles, Charles, Ploeg, & McKibbon, 2015). Methods overviews can involve reviewing either of two complementary categories of literature: the methods literature, as in the overview on sampling in qualitative research (Gentles et al., 2015), or the methods-relevant sections of empirical study reports, as in the case of McCrae and Purssell’s (2016) review—and both can produce valuable insights that help advance research methods (Gentles, Charles, Nicholas, Ploeg, & McKibbon, 2016).

What Can Be Sampled in Qualitative Research

In their review, McCrae and Purssell (2016) imply through the references they make to published studies in their review—including references to interview participants, participant recruitment processes, and human samples—that only people are sampled. This view of what a sample is may be more consistent with approaches in quantitative health research. It may also reflect that health researchers generally have to provide information about their selection of human participants to funding and regulatory bodies, a fact that institutionally predisposes researchers to define a sample as people. Sampling in qualitative research, however, has a much broader connotation. Systematic review of the methods literature (Gentles et al., 2015) reveals how sampling in two different qualitative approaches, case study and grounded theory, is generally understood to comprise the selection not only of multiple and widely varying data sources (people, accounts, events, sites, organizations, documents, etc.) but also units of analysis that do not qualify as data sources per se (i.e., cases to be analyzed for cases study, or examples of concepts for grounded theory). Thus, unlike the conventional realm of quantitative research, it is important to underscore that in qualitative research, the possibilities regarding what can be sampled (the sampling unit) are highly diverse, flexible, and rarely limited to people.

A Comprehensive Understanding of Theoretical Sampling

In attempting to define theoretical sampling for their review, McCrae and Purssell (2016) reference one of the multiple descriptions of theoretical sampling available in Glaser and Strauss’s initial book on grounded theory (1967, p. 45):

Theoretical sampling is the process of data collection for generating theory whereby the analyst jointly collects, codes, and analyzes his data and decides what data to collect next and where to find them, to develop his theory as it emerges.

However, there are numerous informative passages describing theoretical sampling throughout the foundational Glaser and Strauss book, many of which add meanings not encapsulated by the selected quote. For example, the above quote was selected from the first page of a 33-page chapter dedicated to theoretical sampling; however, this chapter addresses numerous topics not captured by the above quote such as selecting comparison groups (pp. 49–60), using theoretical sampling to develop categories (p. 51) including their properties (pp. 55–57), the relationship to theoretical saturation (pp. 60–65), and the use of multiple types or “slices” of data (pp. 65–69). Still more aspects of theoretical sampling are addressed outside this chapter, such as the use of library sources for theoretical sampling (pp. 171–172). Thus, piecing together a more comprehensive definition from that book is achievable only by more holistic consideration of all the passages referring to theoretical sampling, as can be achieved by rigorous review and extraction of all relevant content throughout the book. Similarly, other influential publications authored by the main developers of grounded theory (Charmaz, 2003, 2006, 2014; Corbin & Strauss, 2008, 2015; Glaser, 1978, 1992, 1998; Glaser & Strauss, 1967; Strauss & Corbin, 1998) contain extensive discussion of theoretical sampling throughout their pages—all of which can be rigorously reviewed for the same content.

This was the approach taken in the methods overview of sampling in qualitative research (Gentles et al., 2015). After extracting the multiple references from the multiple publications by grounded theory developers, the following definition was carefully constructed to be as inclusive and consistent as possible with the diversity of passages describing theoretical sampling (pp. 1779–1780):

A process in which data gathering is guided by the evolving theory and the aim is to develop categories in terms of their properties and dimensions and integrate those categories (i.e., relate them to each other within the theory being developed).

Similar to the referenced definition used by McCrae and Purssell, however, this definition fails to indicate what is sampled. This is because the review (Gentles et al., 2015) also revealed that the nature of what is selected in theoretical sampling (the sampling unit) is unclear and inconsistently described across these publications. Analysis of the passages that do address “what is sampled in theoretical sampling” revealed two possibilities (Gentles et al., 2015, p. 1780):

• Varied data sources—for example, “interviews, casual comments, observations, reports, manuals, files, newspaper articles, tables, and charts, etc.” (Glaser, 1978, p. 159)—or

• Examples of concepts or categories in the form of specific pieces of illustrative data—for example, Corbin and Strauss (2008) describe sampling examples of concepts by encouraging participants to share instances of relevant experiences by revising “the questions to be asked in the next interview or observation…based on what was discovered in the previous analysis” or by literally searching sources of previously collected data for “data about a concept” (p. 145). ¹

Developing and holding a clear shared understanding of what is sampled in theoretical sampling is becoming ever more important for being precise about one’s sampling methods not only within grounded theory, but also outside of it, as authors increasingly make reference to theoretical sampling in the general qualitative methods literature (Gentles et al., 2015). In our view, theoretical sampling refers to the generative selection of examples of concepts or categories in the form of specific pieces of illustrative data, which researchers achieve by multiple means including asking targeted questions in interviews and going to different types of data sources to seek out those examples.

The Inconsistent Meaning of Purposeful (Purposive) Sampling

Another significant issue that arises from reading of McCrae and Purssell’s (2016) review is their use of the term purposeful sampling and their explicit exclusion of it as appropriate in grounded theory. We start by referring to the methods literature to correct what we feel are two important misunderstandings regarding the semantics of this term. First, McCrae and Purssell (2016, p. 2286) explicitly state that purposeful sampling excludes theoretical sampling, drawing on the second version of Patton’s (1990) influential typology of purposeful sampling strategies. This is perhaps because they use the term, somewhat arbitrarily, to refer only to forms of sampling that involve a priori decisions, which would necessarily exclude theoretical sampling where decisions are made in response to prior data collection.

We counter that Patton’s definition and typology actually do include theoretical sampling. Although Patton did not use the term “theoretical sampling” explicitly in his 1990 description, his 10th purposeful sampling strategy, “theory-based or operational construct sampling,” is consistent with theoretical sampling—for example, operational construct sampling was defined as sampling “for study real-world examples (i.e., operational examples) of the constructs in which one is interested” (p. 178). Moreover, in the two subsequent versions of his typology, Patton (2002, 2015) explicitly uses the term “theoretical sampling” (alongside “theory-based, operational construct sampling”) in clarification of this form of purposeful sampling. Indeed, the founders of grounded theory consistently describe theoretical sampling as a highly purposeful process.

This relates to a larger problem that has been described regarding the semantics of purposeful sampling (Gentles et al., 2015): Any definition of what qualifies as purposeful is somewhat arbitrary since any form of deliberative sampling, even random sampling, is arguably purposeful (LeCompte, Preissle, & Tesch, 1993; Lincoln & Guba, 1985). This ambiguity is historically evident, too: for example, the term purposive sampling has been used to include random sampling (e.g., Chein, 1981). In addition, Patton never claimed to own the term purposeful sampling, belying the implicit authoritative certainty that McCrae and Purssell intimate. Early typologies also exist that vary from Patton’s typology (e.g., Chein, 1981; Goetz & LeCompte, 1984; LeCompte et al., 1993; Lincoln & Guba, 1985). Additionally, many users of the term, including methods authors, misinterpret earlier descriptions and definitions, as McCrae and Purssell did. In one example, the case study author, Merriam (2009, p. 82), excludes theoretical sampling which contrasts with Patton’s (1990, 2002, 2015) description. In a second example, Yin (2011, p. 311), another prominent case study methodology author, excluded snowball sampling in his earlier writing, which contrasts with the positions of Patton (1990, 2002, 2015) and Merriam (2009). Thus, the term purposeful (purposive) sampling has such potential for ambiguity that its meaning should never be assumed unless a clear definition is provided.

Appropriateness of Initial Sampling in Grounded Theory

As mentioned, McCrae and Purssell (2016) have used the term purposeful sampling in their article to refer only to forms of sampling that involve a priori decisions, which we have argued is an inconsistent use of the term. Instead, to promote clarity, we here use the term initial sampling or a priori sampling. We have previously defined initial sampling as any form of sampling in qualitative research in which sampling decisions are made a priori, or before data collection begins; this contrasts with ongoing sampling, in which sampling decisions are made after or in response to data collection, as is the case for theoretical sampling (Gentles et al., 2015). Here, we wish to clarify the issue of whether it is permissible or appropriate in grounded theory to use initial sampling.

While McCrae and Purssell (2016) suggest through selective references that the acceptability of initial sampling is an unresolved question, we point to the wider grounded theory literature to suggest that a more definitive position is already well established. Among all the main grounded theory developers, it is only Glaser who has refused to characterize initial sampling as acceptable (e.g., 1992). All other authors of the major grounded theory manuals (Charmaz, 2006, 2014; Clarke, 2005; Corbin & Strauss, 2008, 2015; Strauss & Corbin, 1998) agree that some form of initial sampling decisions, informed by a priori knowledge, are practically necessary to generate the data on which theoretical sampling can be based (see Gentles et al., 2015).

Moreover, the use of a priori knowledge is evident in Glaser and Strauss’s (1965) early research, in which they relied on the informed professional and scholarly opinions around them to direct their initial selection of research topic and data to collect. Consequently, one finds descriptions of initial sampling in the first description of the grounded theory method, on which Glaser himself was an author (Glaser & Strauss, 1967, p. 47): “Beyond the decisions concerning initial collection of data, further collection cannot be planned in advance of the emerging theory”; later on (p. 59), they include further illustration of decisions underlying initial data collection in their own seminal grounded theory research. Thus, there is broad consensus in the primary methods literature about the appropriateness and necessity of initial sampling in grounded theory, which dates from inception of the method.

Distinctions Between Sampling and the Participant Sample

We have observed a tendency by some authors of qualitative research reports to muddle the concepts of sampling (a research process) with the descriptions of the final sample of participants from whom interview data were collected. This muddling often occurs when authors embed claims about rigor of the sampling process within their descriptions of the participant sample—for example, by using phrases such as “a purposeful sample.” For example, McCrae and Purssell (2016) seemingly do this when they describe a study author who “did not use any terminology [related to the sampling process] for the study sample” (p. 2288; emphasis added). We believe this results in a misplaced critique of the study author in question, since sampling terminology should not be linked to descriptions of the participant sample for reasons we argue below.

As another example, McCrae and Purssell (2016) appear to further confound the concepts of study sample and sampling when they criticize primary study authors for providing “more detail on sampling demography than on theoretical sampling progress” (p. 2289), implying a position that the participant sample should be wholly derivative of the theoretical sampling process. As we have argued above, however, the characteristics of the participant sample are at least partly determined by initial sampling rather than theoretical sampling. Theoretical sampling, meanwhile, is more appropriately viewed as a research process used to support the ongoing analysis methods used primarily in grounded theory (Charmaz, 2003, 2006, 2014; Corbin & Strauss, 2008, 2015; Glaser, 1978, 1992, 1998; Glaser & Strauss, 1967), rather than the sole (or even primary) process that determines the identities and characteristics of the group of participants to be interviewed (discussed further below).

Finally, McCrae and Purssell (2016) go on to question the validity of describing one’s participant sample by referencing Morse (2008) for her criticism that “researchers often present plenty of detail on the demographic features of the sample, without explaining how these related to the development of theory” (p. 2290). We agree on the one hand with Morse (2008) that it is inappropriate to consider using descriptions of participant samples to increase study validity in the way that is required for generalizability in quantitative research. However, McCrae and Purssell fail to acknowledge Morse’s (2008) balancing statement that “Some demographic information may be pertinent” (p. 300). Below, we remind readers of the distinct rationale and precedent for describing the sample in qualitative research (including grounded theory), both with demographic information and other details that may be pertinent.

Descriptions of the participant sample and the process of sampling should be treated, for the most part, as separate elements of a study report for several reasons. First, since one’s participant sample is a product of the methods employed, it is appropriate to consider it a study finding. This is particularly true in grounded theory, where one’s final sample can only be known after theoretical sampling and data collection are complete. We therefore agree with many journal editors that it is preferable to describe one’s sample separately from one’s sampling methods (including theoretical sampling methods).

Second, no single sampling process ever wholly determines the participant sample. This is especially true of theoretical sampling, which was never intended to be primarily directed at sampling participants considering the other forms of data relevant to developing theoretical categories—including memoirs, reports, and other documents, incidents or events, observations, portions of interviews, or “slices of data” (Charmaz, 2006, 2014; Corbin & Strauss, 2008, 2015; Glaser, 1978; Glaser & Strauss, 1967). Additionally, participant samples are never determined entirely by theoretical sampling because every study must logically start with participant selection decisions made prior to data collection—that is, initial sampling (Charmaz, 2006, 2014; Clarke, 2005; Corbin & Strauss, 2008, 2015; Strauss & Corbin, 1998).

But even initial sampling processes, including those used in qualitative traditions besides grounded theory, can never entirely determine who participates in a study. This is because the participant sample is the outcome of sampling and recruitment processes combined, including aspects of the research context that are inevitably beyond the researcher’s control—for example, factors determining whom from the target population may be available at the time or in the geographical location of recruitment or reasons why people may be unwilling to volunteer. Thus, mixing the language of sampling processes (whether theoretical sampling or other purposeful sampling strategies) with descriptions of the participant sample implies a level of researcher control over who was selected for the study that is disingenuous, and thus, only serves to confound these two separate ideas.

A final reason to consider descriptions of the participant sample as distinct is that such descriptions provide unique value for understanding context and judging transferability of the research findings. This point has been made, for example, by Flyvbjerg (2006), who argues that in qualitative research, the opportunity to bring contextual details to the reader hones their understandings of the myriad of conditions reflective of real life, as well as honing their skills in utilizing non-rule-based (nongeneralized) knowledge. In other words, one strives for theoretical generalizability and not generalization based on representativeness of the sample. Flyvbjerg (2001) further describes how phronesis, or practical knowledge of the type supported by context-rich case studies, enables the reader to judiciously apply relevant findings and knowledge gained to similar known contexts. VanWynsberghe and Khan (2007) similarly argue (citing Donmoyer, 1990) that, in the example of case study, researchers skillfully draw out the meaning of the relationships inherent in contextual situations, enabling readers to engage and relate the material to their own contexts. Together, these arguments imply that authors must provide enough description of context to allow experience-referenced (phronetic) learning and application.

These arguments from case study fit well with ideas that have been described in grounded theory, such as Strauss’s (1993) discussion of context and action or Charmaz’ (2006) discussion related to the foundations of grounded theory in symbolic interaction and philosophical pragmatism. For example, Charmaz (2006) has argued that describing the context (including the participant sample) also helps to clarify the social meanings and processes that represent the theoretical focus of study. Furthermore, in the case of grounded theory, where the participant sample represents such an important aspect of context, providing relevant details in the final report about participant backgrounds, demographic and otherwise, helps to sharpen the reader’s understanding by situating the theory in a way that allows nuanced comparison with other studies (Charmaz, 2006).

In summary, describing one’s participant sample does not reflect the nature or rigor of one’s sampling method; to represent the findings clearly and appropriately, it is best to report one’s sampling process separately from any description of the participant sample.

Conclusion

In this critical analysis of McCrae and Purssell’s (2016) recent methods overview on theoretical sampling, we have highlighted some of the extant ambiguities and misunderstandings associated with terminology commonly used to describe sampling in qualitative research, and outline more definitive (i.e., specific and concrete) understandings informed by a rigorous analysis of the methods literature from an earlier methods overview, focused more broadly on sampling in qualitative research (Gentles et al., 2015). Just as McCrae and Purssell’s review stimulated a lively exchange of ideas at a biweekly meeting of the Grounded Theory Club, which prompted our written response here, we hope the ideas offered in our article will stimulate further critical thinking and discussions about qualitative sampling and invite others to participate in this ongoing conversation.

We have argued that continued ambiguity in the language we use to describe what we do erodes the real and perceived quality of qualitative research. We point to the important role of methods overviews both for focusing our attention on underdeveloped research methods topics and as a source of solutions to methodological ambiguities such as those identified here. Thus, they have the potential to help advance rigor and qualitative research methods themselves (Gentles et al., 2016).