Sage Journals: Discover world-class research

Abstract

Certain social groups are often difficult for researchers to access because of their social or physical location, vulnerability, or otherwise hidden nature. This unique review article based on both the small body of relevant literature and our own experiences as researchers is meant as a guide for those seeking to include hard-to-reach, hidden, and vulnerable populations in research. We make recommendations for research process starting from early stages of study design to dissemination of study results. Topics covered include participant mistrust of the research process; social, psychological, and physical risks to participation; participant resource constraints; and challenges inherent in nonprobability sampling, snowball sampling, and derived rapport. This article offers broadly accessible solutions for qualitative researchers across social science disciplines attempting to research a variety of different populations.

Keywords

hidden populations hard-to-reach populations vulnerable populations recruiting sampling methods

Certain social groups continue to be excluded from social research, and these include women, sexual minorities, and ethnic minorities, among others (Bailey, 2008; Cundiff, 2012; Liamputtong, 2007). Excluded groups are often difficult for researchers to access when traditional sampling methods are ineffective or inappropriate because of the group’s social or physical location, vulnerability, or otherwise hidden nature. In this article, we review sampling and recruitment challenges that have been identified in the small body of literature devoted to hard-to-reach, hidden, and vulnerable populations. We also draw on our own experiences as graduate students conducting qualitative research with such groups. Beyond offering case studies of a specific sampling method or a review relevant to one specific population, this article provides a unique perspective of practical insights relevant to researching various hard-to-reach, hidden, and vulnerable populations. This article offers broadly accessible solutions for qualitative researchers across disciplines attempting to include often excluded groups in their research and may be particularly useful for novice or graduate student researchers.

To begin, we frame the populations for which this article is relevant by defining key terminology that has been used interchangeably in the past. When a population is difficult to access, researchers generally describe them as hard to reach (Shaghaghi, Bhopal, & Sheikh, 2011; Sydor, 2013). Populations may be hard to reach because of their physical or social locations (e.g., remote geographical location, social elites), but they may also be hard to reach because they are vulnerable (i.e., disenfranchised, subject to discrimination or stigma; Liamputtong, 2007; Stone, 2003) or hidden (i.e., populations with no defined limits or sampling parameters; Faugier & Sargeant, 1997; Heckathorn, 1997). For example, abused women may be difficult for researchers to access because (a) of factors related to social location, such as being cut off from communication with outsiders like inquiring researchers (hard to reach); (b) they are disenfranchised and potentially at risk for greater harm if they identify their experiences (vulnerable); and (c) no record of their experience exists because many do not report their abuse (hidden). We assert that in most cases these classifications, while distinct, are not mutually exclusive and thus recommendations made in this article apply to hard-to-reach, hidden, and vulnerable groups alike and particularly to groups that occupy all three classifications at once.

The challenges in researching hard-to-reach, hidden, and vulnerable populations can be divided into two broad categories: individual barriers to participation and sampling issues. Because we draw on our own research experiences as well as published literature, we preface this article by positioning ourselves within the context of our experiences. We then examine the barriers that arise when trying to access hard-to-reach, hidden, and vulnerable populations and provide suggestions for overcoming these challenges. Finally, we discuss applied challenges of sampling hard-to-reach, hidden, and vulnerable populations, followed by our suggested solutions.

Authors’ Positioning

We are four graduate students in an applied social psychology PhD program. Our experiences stem from research we conducted individually at the master’s level (i.e., for theses and a practicum) in the same program. The populations we have developed expertise in are sexual minorities and female cisgender survivors of intimate partner violence (IPV). Specifically, our recruiting experiences focused on lesbian, gay, and bisexual (LGB) elementary school teachers; lesbian and gay (LG) first-generation immigrants; female IPV survivors; female IPV survivors who utilized IPV service agencies; and university women who experienced sexual IPV. Two of the authors self-identify with at least one of the identities targeted in their research and as such were considered “insiders” to the population.

We draw on our recruitment experiences with derived rapport through community agencies, Internet/listservs, snowball sampling, word of mouth, psychology participant pool, and posters. Our research methods all involved in-person interviews, and analytical frameworks included narrative, thematic analysis, phenomenology, and grounded theory. We position ourselves as researchers for social justice and were motivated by personal desires to give voice to silenced identities and to contribute to social change.

Individual Barriers to Participation

Labeling the Population

One of the first challenges researchers face when conducting research with hard-to-reach, hidden, and vulnerable populations is identifying potential participants within that population. For example, research demonstrates that identifying sexual assault survivors is difficult because women do not always label or identify their experiences as assault, abuse, or even a form of sexual victimization (Harned, 2004; Orchowski, Untied, & Gidycz, 2013; Wood & Rennie, 1994). Thus, recruitment may be limiting if study advertisements call for women who have been “sexually assaulted,” “raped,” or “sexually victimized” because this language does not capture women who do not identify their experiences in those ways. Similar challenges exist in the recruitment of other hidden populations. For example, the language that lesbian, gay, bisexual, and transgender (LGBT) individuals use to self-identify may vary greatly across race, culture, age, political affiliation, education level, and geographical region (Matthews & Cramer, 2008; Meyer & Wilson, 2009). Additionally, sexual identity, behavior, and attraction do not always overlap (Laumann, Gagnon, Michael, & Michaels, 1994), so, for example, research advertising for gay and lesbian-identified individuals may not reach all individuals with same-sex attraction or who engage in same-sex sexual behaviors.

Mistrust of the Research Process

A second well-documented barrier to vulnerable group participation in research is mistrust of the research process (Bonevski et al., 2014; Hynes, 2003; Jenkins et al., 1998; Rainbow Health Ontario, 2012). In many cases, this mistrust stems from historical violations by researchers and can affect willingness to participate and/or to disclose identities to researchers. For example, female victims of IPV and domestic violence outreach workers, who could serve as gatekeepers in the recruitment of the population, have reported mistrust of research due to negative assumptions about the academic process (Sullivan & Cain, 2004; Sutherland & Fantasia, 2012). In the second author’s work with these groups, she heard concerns that the research findings would be useless or even harmful to their community or that results would not reach the “right” people in order to foster any positive change (N. Jeffrey, personal communication, November 2013).

Ethnic minority groups share the concern that research findings may not benefit their community (Corbie-Smith, Moody-Ayers, & Thrasher, 2004; Jacklin & Kinoshameg, 2008) and a similar concern was heard by the third author in her research with LG first-generation immigrants—participants were concerned about who would have access to, and ultimately benefit from, study results (M. Choubak, personal communication, November 2013). Immigrant or ethnic minority populations may also be less likely to trust researchers who do not speak their language (Shedlin, Decena, Mangadu, & Martinez, 2011). Other sources of mistrust common to ethnic minorities, indigenous populations, and immigrants include a history of being mistreated in research, breaches of confidentiality, and a tendency of the research community to pathologize certain populations (Bonevski et al., 2014; Scharff, Mathews, Jackson, Hoffsuemmer, & Martin, 2010). Mistrust may also be compounded by intersectionality. For example, ethnic minority participants and older adults who also live in rural communities may be particularly hard to reach because some rural communities foster a cultural climate of mistrust of outsiders (Dibartolo & McCrone, 2003; Loftin, Barnett, Bunn, & Sullivan, 2005).

Participation Risks

Hard-to-reach, hidden, and vulnerable populations often face heightened social, psychological, and physical risks when identified as a member of a particular social group, thus making them more hesitant to identify themselves to researchers. Social risks include loss of status, privacy, or reputation if others learn about, for example, one’s stigmatized identity or illegal behavior as a result of research participation (Bonevski et al., 2014; Meyer & Wilson, 2009). For example, LGBT individuals often face discrimination in the workplace, harassment, and violence (Herek, 2009), and for those who keep their sexual/gender identities hidden, participation in research related to their identities puts them at increased risk for this negative treatment. As another example, intravenous drug users are very careful to avoid social contacts with those outside their drug using social group because they risk legal complications if exposed (Thompson & Collins, 2002). Also at risk for legal complications, undocumented immigrants may avoid participation in research for fear that their identification will lead to deportation (Shedlin et al., 2011).

Research with vulnerable or hidden groups often seeks to address sensitive research questions relating to their vulnerable or hidden status. In some cases, the sensitive nature of the research topic is what makes a population hidden or hard to reach (Sydor, 2013). In other cases, research topics of a sensitive nature compound researchers’ difficulty in reaching already hidden and hard-to-reach populations by making potential participants more hesitant to participate (Faugier & Sargeant, 1997; Lee, 1993; Tortu, Goldsamt, & Hamid, 2001). When potential participants avoid research because of its sensitive subject matter, they are often doing so to avoid the psychological risk of distress from recalling and retelling painful, frightening, or humiliating stories (World Health Organization, 2001). For example, in a study of women’s experiences using IPV services by the second author, women who declined participation reported reasons including wanting to move on, being tired of talking about their experiences, and wishing to forget about a negative period in their life (N. Jeffrey, personal communication, November 2013). Physical risks may also be heightened for hard-to-reach, hidden, and vulnerable groups. For example, within the context of survivors of IPV, participants may face additional physical violence from an abusive partner if their partner discovers their participation in such research (Langford, 2000; Sullivan & Cain, 2004).

It is important to remember that identities are often intersecting (creating doubly vulnerable populations) and therefore these social, psychological, and physical risks intersect as well. For example, risk is intensified for sexual minority elementary school teachers, above and beyond the vulnerability of sexual minorities in other professions. This is due to unfounded, stigmatizing stereotypes that create fear among parents in the school community (e.g., the stereotype that LG people are more likely to be pedophiles; Schlatter & Steinback, 2010; Wright, 2010). Heightened risk for this population includes psychological risk (e.g., emotional discomfort such as worry, anxiety, and shame) and social stigma (e.g., job loss or workplace discrimination; Duke, 2007; Gray, 2014; Hooker, 2010).

The risks outlined earlier become realities when anonymity or confidentiality is not maintained. Although maintaining confidentiality is a benchmark of ethical research with human participants, certain factors inherent in researching hard-to-reach, hidden, and vulnerable populations can challenge the feasibility of absolute anonymity or confidentiality. For example, qualitative research methods are well suited to studying hidden populations because research questions are often exploratory due to the understudied nature of the population (Rich & Ginsburg, 1999; Smith, 2008). Yet, many qualitative methods necessitate face-to-face interaction with participants (e.g., in-person interviews, focus groups, observational field research) and therefore disallow anonymity or place limits on confidentiality. Limits on anonymity and confidentiality also exist when populations are contained in small communities where members tend to know one another. This is particularly problematic when using snowball or chain-referral sampling, where each participant has been referred (and therefore identified) by another (Berg, 1988; Sutherland & Fantasia, 2012). For example, in using chain referral to recruit women who engaged in prostitution and other illegal activity, confidentiality had to be assured and tested every day to minimize impact on the relationships of participants who would not only see each other in the street but also in legal settings such as court or prison (Faugier & Sargeant, 1997).

Participant Resource Constraints

Attrition and absenteeism for research-related appointments may be worsened due to circumstances unique to vulnerable groups (Angucia, Zeelan, & De Jong, 2010; Bonevski et al., 2014). Common participant resource constraints affecting participation in research include finding child care and forgetting about or deprioritizing research when faced with other priorities and stresses of daily life (Bonevski et al., 2014). For example, victims of IPV and refugees often have to juggle many appointments related to abuse and the refugee claim process, respectively (N. Jeffrey, personal communication, December 2013; M. Choubak, personal communication, September 2013). Another common resource constraint is a lack of consistent transportation for those in socially disadvantaged positions (Dibartolo & McCrone, 2003; Loftin et al., 2005).

Solutions for Overcoming Individual Barriers to Participation for Hard-to-Reach, Hidden, and Vulnerable Populations

Despite the challenges that are often inherent in researching hard-to-reach, hidden, and vulnerable populations, there are several tactics that researchers can utilize at different stages of the research process to solve or lessen these challenges. For instance, from the early stages of study design, researchers need to narrow down sampling parameters and set eligibility requirements for the participants. In order to maximize recruitment success, researchers need to ensure that recruitment is inclusive of all who fit within sampling parameters. One way to accomplish this is to expand eligibility requirements to include as many labels as possible; for example, in recruiting advertisements for LG first-generation immigrants, the third author (Choubak, 2014) included descriptors pertaining to sexual behavior and attraction in addition to Western, self-identification labels such as LGBTQ, lesbian, and gay. In some cases, researchers can also distribute a prescreen questionnaire to a larger population and use survey responses to determine eligibility for research on a subhidden or vulnerable population. For example, rather than advertising specifically for women who had experienced sexual assault, the second author (Jeffrey & Barata, in press) distributed a survey to female university students and selected participants for the main qualitative study based on survey responses to questions about sexual experiences. Despite our suggestion to avoid narrow labels in recruitment, it is important to note that in some cases it is a specific identity that the research question is about. For example, the first author (Gray, 2014) was interested in how identifying with the often stigmatized labels of LGB impacted work life. In situations such as this, it may be acceptable to recruit using specific labels as long as the researcher justifies the reason for doing so and recognizes that labels may limit the diversity of the resulting sample.

Grov, Bux, Parsons, and Morgenstern (2009) conducted a qualitative evaluation to determine why recruitment of high-risk, drug-using, gay and bisexual men was moving slowly. Results indicated that the language on recruitment posters failed to attract participants for two reasons: (1) men who did not identify as drug using, engaging in high-risk sexual behaviors, or gay/bisexual did not feel eligible for the study and (2) the language was associated with stigma in their community. Recruitment improved when researchers changed the poster title to: “Ever used ecstasy? And are you a gay/bi man? Tell us about it. 1 hour research study—no judgments …” (p. 1860). Avoiding stigmatizing language may seem obvious, but what is perceived to be stigmatizing within a certain community may not be obvious to a researcher who is positioned outside that community. In the fourth author’s study of resilience in survivors of IPV (Crann, 2012), recruitment posters described the experience of participants (i.e., “adult women who have experienced abuse by an intimate partner”) rather than potentially limiting identities such as “victim” or “survivor.” This was particularly important in the context of resilience research because women who self-identify as something other than “survivor” may not have felt they were eligible for the study.

Once sampling parameters are known and recruitment is underway, other individual-level barriers, such as mistrust of the research process, may inhibit individuals from responding to recruitment efforts. Communication of study findings and greater community and participant involvement in the research process can also help alleviate some of the mistrust that certain groups have toward research (Jacklin & Kinoshameg, 2008; Shalowitz & Miller, 2008). Corbie-Smith, Moody-Ayers, and Thrasher (2004) suggested relaying research results in formats that are accessible by and useful for the community being researched. Even when the researcher cannot guarantee the positive impact of the results, honesty and transparency of the researcher can ease some mistrust. For example, in her research evaluating domestic violence victim services by interviewing the women who utilized them, the second author openly explained to concerned participants that the final report would be presented to the agency but that there was no guarantee about what would be done with the findings. Another strategy for gaining participant trust in the utilization of results is to give participants a sense of ownership in the research process. This can be done by allowing participants to be acknowledged for their contributions to the research (if they choose to be identified) and by allowing participants to review the final results before publication (Sullivan & Cain, 2004).

When possible, it is a good idea to build rapport with potential participants before attempting to recruit. This process requires becoming immersed in the community, which may be facilitated by representatives of the population of interest. Community partners who represent the population can help recruit participants who are hesitant due to mistrust of research (Bonevski et al., 2014; Dibartolo & McCrone, 2003; Shedlin et al., 2011). In this way, community partners become gatekeepers who help inform and protect the vulnerable population. In addition to utilizing community partners as gatekeepers for recruitment, the development of a community advisory board can help to develop a study design that is most appropriate for the population, as well as assist with dissemination of results (Corbie-Smith et al., 2004; Sullivan & Cailin, 2004). Thus, taking time to build trust is crucial not only for recruitment purposes but also for the researcher (particularly a novice one) to get to know the group and to learn about and address potential safety/risk issues of data collection for both participant and researcher.

Some researchers have come up with creative ways of influencing trust within the researched population, such as changing the language used to advertise the research and recruit participants. For example, Shedlin, Decena, Mangadu, and Martinez (2011) changed academic words such as “research” and “interview” to “conversation” and “dialogue,” respectively. While they were still clear about the processes and purposes of the work, the language may have sounded less intimidating to potential participants. Similarly, Sutherland and Fantasia (2012) changed their label of “researcher” into “the nurses doing the research” which was a person-centered identity more familiar to their population of interest. In the same study, Sutherland and Fantasia also noted that recruitment improved when researchers at the recruitment site began knitting instead of working on their laptops. This tactic made the researchers seem nonthreatening and familiar and increased participant trust and willingness to participate in the research.

Once members of the population have been identified and issues of mistrust have been addressed, researchers must attend to the heightened participation risks for hidden or vulnerable participants. The processes involved in carefully assessing threats to confidentiality will also serve to reduce many participation risks and, in doing so, can lead to more successful recruitment (Sutherland & Fantasia, 2012). Perhaps the most generally used measure of maintaining confidentiality is to deidentify data or to use pseudonyms in place of participant names. It is also important for researchers to use great caution when storing identifying documents. For example, the first author transferred audio recordings (which constitute identifying information) to a secure, encrypted computer system immediately after recording. By securing data while still on the site of data collection—which was often a public space—this author avoided traveling with unsecured data. In addition to deidentifying participant names, thorough deidentification includes altering or removing names of cities, service providers, and even unique storylines expressed during qualitative data collection that may be identifiable. If working with very small samples, researchers may choose to present demographic information in ranges (e.g., age range) and, if geographical location is an important focus of the study (not to be deidentified completely), small towns and cities may instead be presented as broad geographical regions.

When advertising the research for recruitment purposes, researchers may consider advertising only in private or safe spaces. For example, the first author (Gray, 2014) recruited LG teachers through LGBT community groups as opposed to places of employment. An alternative is to advertise using vague language that does not specify the research topic. For example, the second author (Jeffrey & Barata, in press) recruited women to participate in a study about sexual assault without specifically advertising the topic of sexual assault. Instead, she used the phrase “intimate experiences in dating relationships with men, including sexual experiences that may have been unwanted or distressing”. Although the fourth author (Crann, 2012) explicitly advertised the study as research on intimate partner violence on recruitment posters, the tear-away tabs with research contact information said “Resilience Study” to avoid “outing” women as survivors or increasing risk of violence from a partner if found in her possession. These precautions can help to avoid identification of the participants during initial contact attempts, such as via recruitment e-mails or through posters that participants may be seen interacting with.

Control of research materials beyond study advertisement and recruitment is also important. Other research materials that require careful consideration include consent forms, any documented communications with participants, interview guides, raw data, and transcripts. Control over these materials means that they will be securely stored, designed in a way that minimizes opportunity for identification of participants or details about the research topic, and shared with participants only under secure conditions. Secure conditions for sharing research materials with participants requires being acutely aware of the particular risks for a specific population; for example, when researching female victims of IPV, it is important to consider that abusers may find research materials and react with further abuse. It is important to keep information vague until the participant communicates that they will be uninterrupted and in a safe space to hear and talk about the details of the research topic (Sullivan & Cain, 2004). The second and fourth authors, when conducting research with this population, advised women take a copy of the consent form and other research documentation only if they felt it safe for them to have. In some cases, when e-mail is used as a recruitment tool, potential participants may be advised to use a trusted email account for research-related communications, such as an account created under a pseudonym or a personal email instead of a work e-mail, as was done by the first author. We suggest making this recommendation to participants before an initial e-mail is sent containing information that would link the recipient to the vulnerable group or to the sensitive research topic.

Finally, when research requires interaction with participants (e.g., interviews), the location of interaction is important to consider in order to reduce the risk of others learning about the nature of the research. When possible, participants may be invited to suggest locations for in-person procedures, as they are the experts on where they feel safest and most comfortable. For example, an in-person procedure at a participant’s home may be safer for those at risk of discrimination in public or workplace settings, while a location not in the participant’s home (and not on a shared home computer via Internet) may be a safer and more confidential environment for women who may be living with an abusive partner. In a creative use of location, Shedlin et al. (2011) conducted interviews with Hispanic immigrants in a restaurant serving their traditional foods: “a legitimized and proven ‘safe space’ for potential participants” (p. 355). It is also important to note that, while researchers should strive to maintain the safety of their participants (particularly important for vulnerable groups), researchers also need to be conscious of maintaining their own safety when conducting research in the field. This might include developing a safety plan with a colleague should issues arise and keeping a list of safety and counseling resources for both the researcher and the participants.

Once participants have been recruited and initial contact made, attrition is a heightened challenge with participants who are hard to reach. While it is crucial to respect participants’ autonomy in deciding to withdraw from participation, there are some creative steps that can be taken to lessen obstacles to participation. In our experience and that of other researchers (e.g., Loftin et al., 2005), researchers should provide transportation reimbursements (e.g., taxi fare, bus tickets, and gas coupons) or arrange for transportation through church vans, school busses, or medical transport vans if other transit systems are nonexistent or unreliable for participants living with low income and inconsistent access to transportation. In the same way, researchers can offer participants child care (or reimbursement for child care) during participation. For populations with constraints on their transportation, it may be desirable to conduct data collection procedures in a convenient location where participants already congregate (Dibartolo & McCrone, 2003). By reducing the requirements of transportation and external child care, two common barriers to participation are removed.

Beyond attrition, some populations are harder to schedule meetings or interviews with than others; for example, transient, homeless, or low-income populations without consistent access to e-mail or phones are decidedly more difficult to schedule and follow-up with beyond recruitment. Having multiple forms of contact for participants, including phone, e-mail, address, and contact persons such as community gatekeepers can aid with this (Loftin et al., 2005; Odierna & Schmidt, 2009; Sullivan, Rumptz, Campbell, Eby, & Dividson, 1996). For participants with no access to phones and no permanent address, it is also possible to provide a toll-free phone number that can be called from any pay phone, which connects the participant directly to the researcher (Lankenau, Sanders, Hathazi, & Bloom, 2010). In order to provide incentive for participation in phone interviews among a transient population, Des Jarlais, Perlis, and Settembrino (2005) distributed debit cards to participants that enabled electronic payment immediately following the interview. Beyond reducing resource-related barriers for participants, we found that researchers could improve attrition and absenteeism rates by making reminder and confirmation calls prior to scheduled meetings with participants who have access to phones. Researchers may decide to remind participants several days prior and follow-up with a confirmation call on the day of. Based on the second author’s experience conducting interviews, we also suggest scheduling extra time between back-to-back interviews to accommodate late participants.

Sampling Issues

Researchers can continue to learn how to best access and meet the needs of their population of interest throughout the process of recruiting. Therefore, development of the sampling strategy for hard-to-reach, hidden, or vulnerable populations should always be an ongoing, iterative part of the research process (Faugier & Sargeant, 1997; Hoppitt et al., 2012; Lee, 1993). We discuss limitations and benefits of the various sampling strategies that have been commonly relied upon for such populations. The specific procedures involved in individual recruitment methods are not described in this article, although we reference sources that include details on recruitment methods mentioned here.

Snowball and Respondent-Driven Sampling

Even when using small samples in qualitative research, researchers may want a nonhomogenous sample that represents the characteristics and diversity of the population. For example, many quantitative and qualitative studies with LGBT individuals have been criticized for the homogeneity of their samples in terms of race, ethnicity, socioeconomic status, and geography (Meezan & Rauch, 2005; Sullivan & Losberg, 2003). Some of the sampling methods commonly used with hard-to-reach and hidden populations can lead to undesired sample homogeneity. Snowball sampling, for example, involves acquiring an initial sample that then recruits their social networks to also participate in the study. Relying on referrals from the initial sample can be problematic when the initial sample itself is homogeneous (Erickson, 1979). These types of samples generally comprise of individuals who are more willing to speak about their experiences or who are more interested in the topic and can result in self-selection bias (Meyer & Wilson, 2009). Additionally, snowball sampling may underrepresent highly isolated individuals and may produce biased samples because individuals might wish to protect others by not referring them (i.e., masking; Erickson, 1979).

Derived Rapport

Just as we proposed developing rapport with community representatives to ease participant mistrust of the research process, this tactic can also be essential to reaching hard-to-reach groups. For some populations, researchers cannot ethically access the population without prior consent from an institution. For example, children and youth must be accessed through derived rapport, which unfolds as a complex negotiation between guardians, schools, and research ethics boards (Sippola, 2006). For other populations, derived rapport is an optional tool. We recruited LGB teachers with help from a teachers’ union, and survivors of IPV through women’s centers and shelters, violence against women agencies, community abuse councils, and immigration services (Crann, 2012; Gray, 2014). Like snowball sampling, community-based, derived rapport sampling can also produce biased samples, whereby individuals who are more engaged with the community are more likely to be sampled (Meyer & Wilson, 2009). Additionally, certain community venues may attract certain types of people or characteristics (Meyer & Wilson, 2009). Derived rapport can also be challenging, given that it amounts to “who you know” as a researcher and that it can be difficult to build trust and gain support from these gatekeepers in a short time frame.

When utilizing derived rapport, researchers may encounter institutional barriers outside their academic institutions (Sutherland & Fantasia, 2012). Researchers cannot assume that community organizations will have the time, resources, or willingness to assist with relationship building or recruitment, even if the organization or its patrons are likely to benefit from the research (Faugler & Sargeant, 1997; Hoppitt et al., 2012). In some cases, potential gatekeepers can be very protective of their community members. A prime example is school boards as gatekeepers for teachers or students. School boards often have strict ethics review processes that occur only once per year and even deterred the first author from applying for approval altogether. Conducting research on communities that are protected by an institution can require lengthy bureaucratic delays, and, particularly in the case of graduate research, these timelines can be unfeasible. With some populations, community gatekeepers may be willing to assist a researcher but may not have access to all members of the population (Faugler & Sargeant, 1997). When groups have multiple, intersecting marginalized identities, they are relegated to a position of intersectional invisibility (Purdie-Vaughns & Eibach, 2008). For example, LG first-generation immigrants represent an intersection of identities and may be unseen by the LGBT community, the immigrant community, and mainstream society.

Solutions for Overcoming Sampling Challenges When Working With Hard-to-Reach, Hidden, and Vulnerable Populations

Despite the potential biases and limitations of snowball sampling, when populations are truly hidden and/or the research topic is sensitive, snowball sampling may be the only feasible method available (Faugier & Sargeant, 1997; Henricks & Blanken, 1992). Some scholars argue that snowball sampling is the ideal method available for such populations, despite its challenges (Biernacki & Waldorf, 1981). Similar to snowball sampling, respondent-driven sampling has been suggested to reduce bias so that those with large, open networks are not oversampled (Heckathorn, 1997). Respondent-driven sampling begins with an initial sample and involves asking that sample to refer or recruit a specified number of peers, resulting in several waves of participants, each wave recruited by the last (Heckathorn, 1997; Johnston & Sabin, 2010). By requiring multiple waves of referrals and tracking these referrals to their source, respondent-driven sampling can reach beyond one social group and avoids relying on any one participant’s network over others. This technique requires less time and funding than some other methods (and is, therefore, more feasible for graduate student researchers) and may also be more confidential because potential participants are not required to divulge sensitive information directly to the researcher before agreeing to participate (Salganik & Heckathorn, 2004).

When utilizing derived rapport in sampling, the importance of building personal relationships with community organizations early on in the research process cannot be understated, and researchers must account for the time it takes when setting research deadlines. Meaningful relationships with the community being researched are long-term relationships (Scharff et al., 2010; Shedlin et al., 2011). For example, the third author spent several months prior to recruitment giving presentations about the research and its implications to stakeholders. In researching the homeless mentally ill population, Hough, Tarke, Renker, Shields, and Glatstein (1996) suggested that rapport building could take years prior to formal recruitment and data collection. Researchers must contend with the institutional problem that timelines accounting for such relationship building are not considered in typical research design.

To optimize productivity during this lengthy rapport building phase, researchers should focus their efforts on organizations or community connections that are the most specific to the population of interest and the identities within that population (Hoppitt et al., 2012; Temple, 2011). For example, the first author most successfully recruited LGB teachers through a community group specifically for LGBT teachers as opposed to general organizations for teachers or the larger LGBT community. Similarly, the third author most successfully recruited LG first-generation immigrants through groups specifically targeting LGBT newcomers as opposed to immigrant groups or the larger LGBT community.

Nonprobability Sampling

The sampling methods we have outlined thus far are nonrandom, nonprobability sampling methods. In fact, random or probability sampling is generally not possible among hidden populations because no sampling frame exists, and it is often not feasible among hard-to-reach populations because access is limited (Benoit, Jansson, Millar, & Phillips, 2005; Crosby, Salazar, DiClemente & Lang, 2010; Heckathorn, 1997). Despite concerns from traditional quantitative researchers, however, nonprobability sampling is less problematic in qualitative research. The epistemological underpinnings and goals of most qualitative research render random sampling inappropriate and irrelevant (Marshall, 1996), and both assumptions (e.g., normal distribution) and purposes (e.g., generalizability) relevant to quantitative studies do not apply to qualitative studies that are designed to explore the complexities of human experiences.

Although random and probability sampling are not always possible or necessary, there are some sampling methods that have been designed to reduce bias inherent in other nonprobability methods such as snowball sampling and derived rapport. One option is to attempt to recruit a representative sample. Target, adaptive, and time–space sampling involve ethnographically mapping a population of interest and recruiting a specified number of participants at identified locations (Martsolf, Courey, Chapman, Draucker, & Mims, 2006; Semaan, 2010; Watters & Biernacki, 1989). These methods are ideal when sampling frames do not exist (as for hidden populations), as they provide probability estimates (Meyer & Wilson, 2009; Semaan, 2010). Stueve, O’Donnell, Duran, San Doval, & Blome (2001) reported similar or higher response rates with time–space sampling in comparison to random-digit dialing of ethnic minority men who have sex with men.

Although these sampling methods offer quasi-probability sampling, they retain one of the main limitations of the nonprobability methods mentioned earlier—namely, targeting venues or geographic areas in which members of the population of interest are likely to congregate may result in accessing only a certain type of participant due to differences between those who are physically part of the community versus those who are isolated (Meyer & Wilson, 2009). In order to reduce this bias, researchers utilizing time–space sampling should maximize the variability in venues/geographic areas or set quotas for demographic variables to ensure a diverse sample (Meyer & Wilson, 2009).

Researchers may also distribute a screening survey to a larger (potentially random) sample in order to classify members of the hard-to-reach or hidden population (Kalton, 2003). For example, the second author (Jeffrey & Barata, in press) distributed a survey on sexual experiences to a sample of university women in order to identify women who had experienced sexual assault. We caution, however, that response bias may arise if participants can deduce the population of interest from screening questions (Kalton, 2003). A way to reduce the likelihood of such bias is to present the phases as separate studies and provide participants with incentives for participation in both the screening and the final phases.

Using Multiple Strategies

In addition to the solutions offered for common sampling challenges, perhaps the best way to maximize recruitment efforts is to use a combination of sampling strategies (Bonevski et al., 2014; Poorman, 2002; Shedlin et al., 2011). We mentioned earlier that development of the sampling strategy for hard-to-reach, hidden, or vulnerable populations should be an iterative process (Faugier & Sargeant, 1997; Hoppitt et al., 2012). The result of flexibility in sampling is often to try several sampling methods before knowing which method will work best for a specific population. We cannot prescribe the best strategy to find each hard-to-reach, hidden, or vulnerable population as the diversity in such populations is infinite. What we can prescribe, however, is to not to rely too heavily on a single sampling strategy. Flexibility and creativity is needed for successfully identifying and recruiting these populations.

Conclusion

This article is meant as a guide for those seeking to include hard-to-reach, hidden, and vulnerable populations in research and may be particularly useful for novice or graduate student researchers. We hope that by outlining and providing solutions to some of the challenges one can expect to encounter, researchers will be able to proactively design their studies around the unique needs of such populations.

We have made recommendations to ensure that recruitment efforts reach all who fit the sampling parameters. These recommendations included recruiting using vague or open-ended identity labels and being aware of and avoiding stigmatizing language in recruitment materials. We also recommended ways of addressing participant mistrust, such as building rapport with potential participants before attempting to recruit, and utilizing community partners as gatekeepers to the population. To protect hard-to-reach, hidden, and vulnerable populations from research related risk—and thereby to aid in successful recruitment—we recommended extreme caution and attention to maintaining confidentiality; for example, avoiding language in study advertisements that might link a potential participant to an incriminating or vulnerable identity. In order to combat attrition and scheduling difficulties that are exacerbated by circumstances related to a population’s vulnerability, we recommended that researchers provide transportation and/or childcare reimbursements or conduct the research in a location convenient to the participants.

Certain sampling strategies are commonly used for research with hard-to-reach, hidden, and vulnerable populations. We outlined the major challenges with snowball sampling and derived rapport (and related recruitment tactics). While these strategies may have weaknesses, they remain the most appropriate for the unique needs of such populations. We also addressed nonprobability sampling and how certain recruitment methods such as target, adaptive, or time–space sampling can provide probability estimates. We also suggested using multiple sampling strategies in tandem in order to maximize recruitment success.

Despite the inherent challenges, there is great benefit to researching hard-to-reach, hidden, and vulnerable populations. Recruitment and data collection should not be viewed as a means to an end but as a process of engagement with often silenced groups. Researchers can positively impact the well-being of such groups. For example, participants in our own research informed us that through the research process, we increased the visibility of the populations and their needs and in one case gave a participant confidence and a reason to value their marginalized identity (M. Choubak, personal communication, November 2013; A. Gray, personal communication, November 2013). Through our recruitment journeys, we were able to advocate for the populations we were working with. Initial networking during recruitment also facilitated future dissemination of study results. By disseminating through members and representatives of the community being researched, results were more relevant and empowering to the community than research results only published in academic journals. We encourage researchers to consider their goals for any research program involving a hard-to-reach, hidden, or vulnerable population and to be prepared and willing to address the inevitable challenges and complexities of such research. Without such preparedness, researchers can harm rather than empower the participants. While reaching and researching hard-to-reach, hidden, and vulnerable populations can be challenging, it is important that researchers continue to develop practices to foster better relationships with such groups so that their voices can be heard. This article is meant to continue this conversation.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Anguicia

Zeelen

De Jong

(2010). Researching the reintegration of formerly abducted children in Northern Uganda through action research: Experiences and reflections. Journal of Community & Applied Social Psychology, 20, 217–231.

Bailey

K. D.

(2008). Methods of social research (4th ed). New York, NY: The Free Press.

Benoit

Jansson

Millar

Phillips

(2005). Community-academic research on hard-to-reach populations: Benefits and challenges. Qualitative Health Research, 15, 263–282. doi:10.1177/1049732304267752

Berg

(1988). Snowball sampling. In Kotz

Johnson

N. L.

(Eds.), Encyclopedia of statistical sciences (Vol. 8, pp. 529–532). New York, NY: John Wiley.

Biernacki

Waldorf

(1981). Snowball sampling: Problem and techniques of chain referral sampling. Sociological Methods and Research, 10, 141–163.

Bonevski

Randell

Paul

Chapman

Twyman

Bryant

… Hughes

(2014). Reaching the hard-to-reach: A systematic review of strategies for improving health and medical research with socially disadvantaged groups. BMC Biomedical Research Methodology, 14, 1–29. doi:10.1186/1471-2288-14-42

Choubak

(2014). The experiences of lesbian and gay first- generation immigrants to Canada: Negotiating identities post-migration (Master’s Thesis). Retrieved from University of Guelph Atrium, http://hdl.handle.net/10214/8528

Corbie-Smith

Moody-Ayers

Thrasher

A. D.

(2004). Closing the circle between minority inclusion in research and health disparities. Archives of Internal Medicine, 164, 1362–1364.

Crann

S. E.

(2012). “Some days I don’t know how I did it, but I did”: The experience of resilience in survivors of intimate partner violence (Master’s thesis). Retrieved from University of Guelph Atrium, http://hdl.handle.net/10214/4145

10.

Crosby

R. A.

Salazar

L. F.

DiClemente

R. J.

Lang

D. L.

(2010). Balancing rigor against the inherent limitations of investigating hard-to-reach populations. Health Education Research, 25, 1–5.

11.

Cundiff

J. L.

(2012). Is mainstream psychological research “womanless” and “raceless”? An updated analysis. Journal of Sex Roles, 67, 158–173.

12.

Des Jarlais

D. C.

Perlis

T. E.

Settembrino

J. M.

(2005). The use of electronic debit cards in longitudinal data collection with geographically mobile drug users. Drug and Alcohol Dependence, 77, 1–5.

13.

Dibartolo

M. C.

McCrone

(2003). Recruitment of rural, community-dwelling older adults: Barriers, challenges, and strategies. Aging & Mental Health, 7, 75–82.

14.

Duke

T. S.

(2007). Hidden, invisible, marginalized, ignored: A critical review of the professional and empirical literature (or Lack Thereof) on gay and lesbian teachers in the United States. Journal of Gay & Lesbian Issues in Education, 4, 19–38.

15.

Gray

(2014). Institutionalized Sexual Prejudice: A narrative analysis of LGBQ teacher experiences (Master’s thesis). Retrieved from University of Guelph Atrium, https://hdl.handle.net/10214/8277

16.

Erickson

B. H.

(1979). Some problems of inference from chain data. Sociological Methodology, 10, 276–302.

17.

Faugier

Sargeant

(1997). Samping hard to reach populations. Journal of Advanced Nursing, 26, 790–797.

18.

Grov

Bux

Parsons

J. T.

Morgenstern

(2009). Recruiting hard-to-reach drug-using men who have sex with men into an intervention study: Lessons learned and implications for applied research. Substance Use & Misuse, 44, 1855–1871. doi:10.3109/10826080802501570

19.

Harned

M. S.

(2004). Does it matter what you call it? The relationship between labeling unwanted sexual experiences and distress. Journal of Consulting and Clinical Psychology, 72, 1090–1099. doi:10.1037/0022-006X.72.6.1090

20.

Heckathorn

(1997). Respondent-driven sampling: A new approach to the study of hidden populations. Social Problems, 44, 174–199.

21.

Hendricks

V. M.

Blanken

(1992) Snowball sampling: Theoretical and practical considerations. In Hendricks

V. M.

Blanken

Adriaans

(Eds.), Snowball sampling: A pilot study on cocaine use (pp. 17–35). Rotterdam, the Netherlands: IVO.

22.

Herek

(2009). Hate crimes and stigma-related experiences among sexual minority adults in the United States: Prevalence estimates from a National Probability Sample. Journal of Interpersonal Violence, 24, 54–74. doi:10.1177/0886260508316477

23.

Hooker

S. D.

(2010). Closeted or out? Gay and lesbian educators reveal their experiences about their sexual identities in K-12 schools (Unpublished doctoral dissertation), University of Cincinnati, OH.

24.

Hoppitt

Shah

Bradburn

Gill

Calvert

Pall

… Sackley

(2012). Reaching the ‘hard to reach’: Strategies to recruit black and minority ethnic service users with rare long-term neurological conditions. International Journal of Social Research Methodology, 15, 485–495.

25.

Hough

R. L.

Tarke

Renker

Shields

Glatstein

(1996). Recruitment and retention of homeless mentally ill participants in research. Journal of Consulting and Clinical Psychology, 64, 881–891.

26.

Hynes

(2003). The issue of ‘trust’ or ‘mistrust’ in research with refugees: choices, caveats and considerations for researchers (UN HCR Working Paper No.98) . Geneva, Switzerland: UNHCR.

27.

Jacklin

Kinoshameg

(2008). Developing a participatory aboriginal health research project: “Only if it’s going to mean something”. Journal of Empirical Research on Human Research Ethics, 3, 53–67. doi:10.1525/jer.2008.3.2.53

28.

Jeffrey

N. K.

Barata

P. C.

(in press). “He didn’t necessarily force himself upon me, but...”: Women’s lived experiences of sexual coercion in intimate relationships with men. Violence Against Women.

29.

Jenkins

R. A.

Chinaworapong

Morgan

P. A.

Ruangyuttikarn

Sontirat

Chiu

… Khambooruang

(1998). Motivation, recruitment and screening of volunteers for a phase I/II HIV preventive vaccine trial in Thailand. Journal of Acquired Immune Deficiency Syndromes & Human Retrovirology, 18, 171–177.

30.

Johnston

L. G.

Sabin

(2010). Sampling hard-to-reach populations with respondent driven sampling. Methodological Innovations Online, 5, 38–48. doi:10.4256/mio.2010.0017

31.

Kalton

(2003). Practical methods for sampling rare and mobile populations. Statistics in Transition, 6, 491–501.

32.

Langford

D. R.

(2000). Developing a safety protocol in qualitative research involving battered women. Qualitative Health Research, 10, 133–142.

33.

Lankenau

S. E.

Sanders

Hathazi

Bloom

J. J.

(2010). Recruiting and retaining mobile young injection drug users in a longitudinal study. Substance Use & Misuse, 45, 684–699.

34.

Lee

R. M.

(1993) Doing research on sensitive topics. London, England: Sage.

35.

Laumann

E. O.

Gagnon

J. H.

Michael

R. T.

Michaels

(1994). The social organization of sexuality: Sexual practices in the United States. Chicago, IL: University of Chicago Press.

36.

Liamputtong

(2007). Researching the vulnerable: A guide to sensitive research methods. London, England: SAGE.

37.

Loftin

W. A.

Barnett

S. K.

Bunn

P. S.

Sullivan

(2005). Recruitment and retention of rural African Americans in diabetes research: Lessons learned. Diabetes Education, 31, 251–29l.

38.

Marshall

M. N.

(1996). Sampling for qualitative research. Family Practice, 13, 522–526.

39.

Martsolf

D. S.

Courey

T. J.

Chapman

T. R.

Draucker

C. B.

Mims

B. L.

(2006). Adaptive sampling: Recruiting a diverse community sample of survivors of sexual violence. Journal of Community Health Nursing, 23, 169–182.

40.

Matthews

Cramer

E. P.

(2008). Using technology to enhance qualitative research with hidden populations. The Qualitative Report, 13, 301–315.

41.

Meezan

Rauch

(2005). Gay marriage, same-sex parenting, and America’s children. The Future of Children, 15, 97–115.

42.

Meyer

I. H.

Wilson

P. A.

(2009). Sampling lesbian, gay, and bisexual populations. Journal of Counseling Psychology, 56, 23–31. doi:10.1037/a0014587

43.

Odierna

D. H.

Schmidt

L. A.

(2009). The effects of failing to include hard-to-reach respondents in longitudinal surveys. American Journal of Public Health, 99, 1515–1520.

44.

Orchowski

L. M.

Untied

A. S.

Gidycz

C. A.

(2013). Factors associated with college women’s labeling of sexual victimization. Violence and Victims, 28, 940–958. doi:10.1891/0886-6708.VV-D-12-00049

45.

Poorman

P. B.

(2002). Perceptions of thriving by women who have experienced abuse or status-related oppression. Psychology of Women Quarterly, 26, 51–62.

46.

Purdie-Vaughns

Eibach

R. P.

(2008). Intersectional invisibility: The distinctive advantages and disadvantages of multiple subordinate-group identities. Sex Roles, 59, 377–391.

47.

Rainbow Health Ontario. (2012). Because LGBTQ health matters [Fact Sheet]. Retrieved from http://www.rainbowhealthontario.ca/admin/contentEngine/contentDocuments/CBR_in_LGBT_Communities.pdf

48.

Rich

Ginsburg

K. R.

(1999). The reason and rhyme of qualitative research: Why, when, and how to use qualitative methods in the study of adolescent health. Journal of Adolescent Health, 25, 371–378.

49.

Salganik

M. J.

Heckathorn

D. D.

(2004). Sampling and estimation in hidden populations using respondent-driven sampling. Sociological Methodology, 34, 193–239.

50.

Scharff

D. P.

Mathews

K. J.

Jackson

Hoffsuemmer

Martin

Edwards

(2010). More than Tuskegee: Understanding mistrust about research participation. Journal of Health Care for the Poor and Underserved, 21, 879–897. doi:10.1353/hpu.0.0323

51.

Schlatter

Steinback

(Winter, 2010). 10 anti-gay myths debunked. Southern Poverty Law Center’s Intelligence Report, 140. Retrieved from http://www.splcenter.org/get-informed/intelligence-report

52.

Semaan

(2010). Time-space sampling and respondent-driven sampling with hard-to-reach populations. Methodological Innovations Online, 5, 60–75.

53.

Shaghaghi

Bhopal

R. S.

Sheikh

(2011). Approaches to recruiting ‘hard to reach’ populations into research: A review of the literature. Health Promotion Perspectives, 1, 86–94.

54.

Shalowitz

D. I.

Miller

F. G.

(2008). Communicating the results of clinical research to participants: Attitudes, practices, and future directions. PLOS Medicine, 5, 0714–0720.

55.

Shedlin

M. G.

Decena

C. U.

Mangadu

Martinez

(2011). Research participant recruitment in Hispanic communities: Lessons learned. Journal of Immigrant Minority Health, 13, 352–360. doi:10.1007/s10903-009-9292-1

56.

Sippola

(2006). Ivory tower ethics: Potential conflict between community organizations and agents of the tri-council. In Leadbeater

B. J. R.

(Ed.), Ethical issues in community-based research with children and youth (pp. 111–135). Toronto, Canada: University of Toronto Press.

57.

Smith

J. A.

(2008). Qualitative psychology: A practical guide to research methods. Thousand Oaks, CA: Sage.

58.

Stueve

O’Donnell

L. N.

Duran

San Doval

Blome

(2001). Time-space sampling in minority communities: results with young Latino men who have sex with men. American Journal of Public Health, 91, 922–926.

59.

Stone

T. H.

, (2003). The invisible vulnerable: The economically and educationally disadvantaged subjects of clinical research. Journal of Law, Medicine, & Ethics, 31, 149–153.

60.

Sutherland

M. A.

Fantasia

H. C.

(2012). Successful research recruitment strategies in a study focused on abused rural women at risk for sexually transmitted infections. Journal of Midwifery & Women’s Health, 57, 381–385.

61.

Sullivan

C. M.

Rumptz

M. H.

Campbell

Eby

K. K.

Davidson

W. S.

II (1996). Retaining participants in longitudinal community research: A comprehensive protocol. Journal of Applied Behavioral Science, 32, 262–276.

62.

Sullivan

Losberg

(2003). A study of sampling in research in the field of gay and lesbian studies. Journal of Gay and Lesbian Social Services, 15, 147–162.

63.

Sullivan

C. M.

Cain

(2004). Ethical and safety considerations when obtaining information from or about battered women for research purposes. Journal of Interpersonal Violence, 19, 603–618.

64.

Sydor

(2013). Conducting research into hidden or hard-to-reach populations. Nurse Researcher, 20, 33–37.

65.

Temple

E. C.

(2011). A comparison of internet-based participant recruitment methods: Engaging the hidden population of cannabis users in research. Journal of Research Practice, 7, 1–20.

66.

Thompson

S. K.

Collins

L. M.

(2002). Adaptive sampling in research on risk-related behaviors. Drug and Alcohol Dependence, 68, S57–S67.

67.

Tortu

Goldsamt

Hamid

(2001). A practical guide to research and services with hidden populations. Boston, MA: Allyn and Bacon.

68.

Watters

J. K.

Biernacki

(1989). Targeted sampling: Options for the study of hidden populations. Journal of Social Problems, 36, 416–430.

69.

Wood

L. A.

Rennie

(1994). Formulating rape: The discursive construction of victims and villains. Discourse and Society, 5, 125–148.

70.

World Health Organization. (2001). Putting women first: ethical and safety recommendations for research on domestic violence against women. Geneva, Switzerland: World Health Organization.

71.

Wright

T. E.

(2010). LGBT educators’ perceptions of school climate: Administrators’ attitudes and district policies can make significant contributions to creating safe environments for LGBT educators. Phi Delta Kappan, 91, 49–53.

Finding the Hidden Participant

Abstract

Keywords

Authors’ Positioning

Individual Barriers to Participation

Labeling the Population

Mistrust of the Research Process

Participation Risks

Participant Resource Constraints

Solutions for Overcoming Individual Barriers to Participation for Hard-to-Reach, Hidden, and Vulnerable Populations

Sampling Issues

Snowball and Respondent-Driven Sampling

Derived Rapport

Solutions for Overcoming Sampling Challenges When Working With Hard-to-Reach, Hidden, and Vulnerable Populations

Nonprobability Sampling

Using Multiple Strategies

Conclusion

Footnotes

Declaration of Conflicting Interests

Funding

References