“You Can’t Look for Disparities Where Data Are Missing”: Interviews with Health Plans on Managing Race and Ethnicity Data in the Pursuit of Health Equity

Abstract

Introduction:

Many health plans in the United States are working to address disparities in care by race/ethnicity. A fundamental part of this work involves implementing processes for collecting race/ethnicity data to visualize where there are gaps for certain groups.

Methods:

A team of researchers at the National Committee for Quality Assurance interviewed health plans to understand how they collect and manage race/ethnicity data and to gain insights into how they overcome barriers to these practices. The research team purposively sampled and interviewed 13 health plans that report on the Healthcare Effectiveness Data and Information Set (HEDIS®), a widely used clinical quality measurement tool. Plans represented all product lines, and operate in a variety of United States geographic regions. Interview questions focused on how plans source race/ethnicity data, data granularity, processes for linking race/ethnicity data to quality performance metrics, and how data inform quality improvement efforts.

Results:

Participants highlighted four primary challenges to collecting and managing race/ethnicity data. First, there must be a balance between federal, state, and accreditation standards about which groups to include in data collection. Second, plans must balance the number of groups included with their ability to meaningfully report on those groups. Third, there are multiple sources of race/ethnicity data, requiring prioritization of certain data sources over others. Fourth, plans need an organizational structure conducive to such efforts, which includes the capacity for smooth interdepartmental collaboration and the organizational priority to collect, organize, and act on race and ethnicity data.

Discussion:

Despite the challenges associated with data wrangling, many plans emphasized interdepartmental collaboration as necessary for success in collecting and managing high-quality data to understand where there are health disparities by race/ethnicity, and to inform efforts aimed at addressing gaps in care.

Introduction

Racial and ethnic disparities in health outcomes exist throughout the U.S. health care system. Stratification of health care data by race/ethnicity is a critical tool; done correctly, it can inform strategies to address disparities and generate equitable health outcomes.^1–7 Incomplete and low-quality data, however, prevent clear assessment of where disparities exist, leading to disproportionate, poor outcomes among certain racial and ethnic populations. Health plans in particular have an important role in addressing disparities by identifying gaps in care, working with clinicians and patients to identify the cause of disparities, and providing proper and inclusive services and coverage to everyone.

Many health plans have begun collecting race/ethnicity data to understand and improve health outcomes and reduce disparities in the populations they manage, in line with state and federal policies incentivizing such outcomes.^8–10 But while there are many strategies to overcome limitations on collecting these data, simply mandating data collection is insufficient.^11–14 Health plans must also understand and implement strategies for managing and utilizing the data to provide more equitable health outcomes.

This study highlights health plans’ experience collecting and managing race/ethnicity data, and how these are working to improve data quality and completeness. Insights can help plans better understand how to use race/ethnicity data to develop initiatives that focus on improving the quality of care for patients, and can help plans in the early stages of their data collection journey.

Methods

This study reports qualitative findings from the Race/Ethnicity Stratification Learning Network (RESLN), a mixed-methods quality improvement and research program conducted by the National Committee for Quality Assurance (NCQA) between October 2022 and January 2023. The goal of the RESLN was to identify preliminary performance on race/ethnicity-stratified Healthcare Effectiveness Data and Information Set (HEDIS®)a quality measures, and pain points experienced by health plans in their efforts to manage quality data, along with strategies and opportunities to overcome barriers and achieve actionable quality insights. The quantitative arm of the program involved submitting stratified performance data on up to five HEDIS measures using a standardized data collection tool. For the qualitative arm, program staff conducted virtual, small-group interviews with plan-identified key personnel. Complete methods and findings from the quantitative analysis, as well as initial qualitative themes, were previously reported.¹⁵ No patient-level data were exchanged during this study. This study applied a rigorous qualitative framework to identify structured themes.

Participants were purposively sampled through email outreach to HEDIS-reporting health plans. Organizations offered all product lines (Commercial, Medicaid, Medicare, Exchange) and operated in a variety of states and geographic regions across the United States.

Plans that volunteered to participate were oriented to program aims and methods. Confirmed participants were asked to identify key personnel to be interviewed, based on a list of topics. Each interview had between three and five participants. All organizations were interviewed separately.

Interviews were 60 min long and semistructured. Interview questions focused on race/ethnicity data sources that plans relied on, the level of granularity in the data, the processes used to collect the data and link it to quality performance metrics, and how the data informed quality improvement efforts.

Qualitative analysis proceeded through several rounds of reading and coding, using an inductive approach to allow themes to emerge.^16–19 First, all interview transcripts and flagged sections were reviewed, according to the questions on the interview guide. Two researchers reviewed all transcripts to ensure that materials were captured logically and were appropriately tagged to a relevant question.

Next, we began a fresh review process to identify themes and patterns, which we labeled with formal codes. Each researcher independently read the collated documents to identify potential themes and codes for inclusion in the codebook. We discussed identified codes as a group and agreed on official code labels and descriptions, which were organized into a codebook.²⁰ We then generated an unmarked copy of the collated documents and applied the codes. At this stage, one team member performed an initial review, and another followed to verify agreement. Mismatches were discussed as a team to find agreement. As a final quality assurance step, codes were applied to relevant sections of the original transcripts. This allowed team members to reexamine the transcripts for materials that may have been missed in the earlier phases.

The final phase was analytic synthesis: extracting the coded transcript material into files organized by code and reviewing the material anew. Each researcher was assigned a set of codes to review and summarize, identifying key themes and patterns within the code, and pointing to especially representative examples. As we reviewed these summaries, we built toward broader patterns in the data, following an inductive approach.

Interview Sample Summary

Thirteen health plans participated in qualitative group interviews. Participating organizations varied in size, location, and product line (Table 1). Plan characteristics are masked. Interview participant roles included HEDIS manager, director/manager of quality improvement or data analytics, health equity officer, IT developer, and vice president.

Table 1.

Summary Characteristics of Participating Plans

Covered lives	Total organizations	States	Count of contracts by product line
Covered lives	Total organizations	States	Commercial	Medicaid	Medicare	Exchange
≤50,000	3	Florida, Minnesota	0	4	0	0
100,000–1,000,000	4	California, Massachusetts, Washington, Minnesota, Wisconsin	0	7	7	2
>1,000,000	6	California, Massachusetts, Pennsylvania, New York, Delaware, West Virginia, National	34	20	9	4

Results

Plans identified numerous challenges to managing and reporting race/ethnicity data. While some pointed to end goals such as visualization and reporting to appropriate audiences, most fell into two major categories: how to define and categorize race/ethnicity groups—addressing which race/ethnicity groups are considered, and the data sources available on those groups—and how best to achieve actionable data.

Part I: Defining and Categorizing Race and Ethnicity

There are three primary questions to address when considering how data are managed for quality data analysis. First, which race/ethnicity groups are recognized? Second, what data sources should health plans leverage? Third, how should plans prioritize certain data sources over others?

Race/ethnicity groups

Interviews identified multiple influencing forces on how race/ethnicity data are collected and harnessed for quality improvement. All plans indicated that state standards are involved, and most also referenced accreditation standards (e.g., NCQA) and federal standards (e.g., Office of Management and Budget) on which they are based. NCQA HEDIS specifications require collection of race/ethnicity data for certain measures, including Race and Ethnicity Diversity of Membership. Several states have instituted mandates, and they offer financial incentives to increase race/ethnicity data collection.

But when requirements differ, it can generate confusion and challenges. For example, federal (and accreditation) standards and state standards might direct reporting of race/ethnicity, but they might use different reporting categories. State-recognized categories are often already in Medicaid enrollment files, which means that plans must map data for different reporting outputs. This is further complicated for national or multistate organizations, because the structure of enrollment files can vary between states.

Misalignment is common in the “Hispanic/Latino” category. At the time of the qualitative interviews, federal standards recognized this category as an ethnicity independent from race categories and asked for a person’s race separately from their ethnicity. However, many intake forms include “Hispanic/Latino” within the broader list of races that could be indicated, making it difficult to disambiguate data: anyone who marked only “Hispanic” would be listed as such for ethnicity, but would be “Unknown” for race.

Similarly, not all intake forms include the same categories. For example, while federal standards separate the categories of Asian and Native Hawaiian, some intake forms collapse them into one item, creating a challenge when mapping to federal categories. Some forms allow an individual to check multiple race categories, while others indicate to select only one, or list a “Multiracial” option. When only one category can be indicated, individuals who identify with more than one race do not know how to respond: Do they select only one, select “Other,” or perhaps skip the question altogether? Uncertainty lowers data reliability. This suggests that the data collection instrument is critical to gathering good-quality race/ethnicity data, and that there may be room to better standardize intake of these data.

Multiple sources of race/ethnicity data

Plans can draw on an array of data sources for member race/ethnicity information—from member self-reporting, to enrollment files, to third-party records (e.g., state immunization registries). High-quality imputation algorithms are also available. Some plans have developed logic documents to facilitate prioritization of certain data sources based on data completeness and anticipated accuracy, often using criteria set by health plan staff.

The gold standard is member self-reported race/ethnicity data, and for the health plan to have full control over the collection method without involvement of an intermediary or third party. This ensures accurate assessment of how a person identifies their racial background. The member portal is the most prevalent method of collecting member self-reported data. Many RESLN participants stated that they are building, or have already built, a member portal that can collect an array of demographic information directly from members.

To ensure optimal outcomes from a portal, plans provided ideas to avoid or overcome key challenges. First, it is important to steer members to the member portal to ensure access and uptake. Once there, questions on race/ethnicity should be easy to understand and answer; this helps improve members’ digital experience. It also builds member trust if the health plan explains why it is asking for the information and assures members that their response does not affect their care or coverage.

Collecting member self-reported data requires investment of time and resources. When self-reported data are not immediately available, plans may have access to other data sources. Medicaid plans regularly receive enrollment files from their states, and Medicare plans may receive similar files from the Centers for Medicare & Medicaid Services (CMS). These files typically contain member race/ethnicity information. Some plans may use other types of state-level records, such as immunization registries, to obtain race/ethnicity data. Programs such as the Special Supplemental Nutrition Program for Women, Infants, and Children, or other state-run services, may also provide these data. Some plans partner with their provider networks to pull race/ethnicity data from the provider’s electronic health record system.

Many plans in the RESLN discussed leveraging imputation methods to fill in missing race/ethnicity data, although all plans prioritized direct sources over this approach. The most common imputation method used was the RAND Bayesian Improved Surname Geocoding method. A few plans stated they were hesitant to use imputation methods, largely out of a lack of trust in the methods’ accuracy.

Prioritizing data sources

Different sources can contradict each other, leading to questions about how to prioritize information or identify a source of truth. Most plans developed a hierarchy of data sources, although the structure of these hierarchies varied. Plans most often relied on state data, typically supplemented with either CMS data or data from providers and clinics. Some plans described their process for updating race/ethnicity data based on information in the member portal, or on interactions with case managers who work directly with the member. Some allowed member self-reported data to override all other information.

Not all organizations can do this. One plan shared how its state’s enrollment data contradicts the information the plan collected through member self-reporting, yet state reporting requirements prioritized the original file, making it difficult for the plan to update and reconcile the information. Another plan stated that it takes all the information it receives and rolls it up into state-defined output “buckets.” Deciding how to prioritize data includes not only data availability and quality but also state reporting requirements.

Plans that developed a source prioritization hierarchy were often also responsible for ensuring that staff understood how it worked, and/or when they were allowed to use certain data sources. This often meant expanding access to race/ethnicity data and training staff to ensure consistency of analyses across units. One plan instituted governance standards to ensure that only staff with training could access race/ethnicity data. Such standards clarified that the only allowable use of race/ethnicity data was to measure and address inequalities, and they also helped prevent inappropriate use that might exacerbate inequalities.

Illustrative quotes for Part I are described in Table 2.

Table 2.

Themes and Illustrative Quotes, Defining and Categorizing Race and Ethnicity

Theme	Illustrative quote
Race/Ethnicity groups	“…if HEDIS…didn’t require that we stratify based on race and ethnicity [I don’t think] that we would even be talking about collecting race and ethnicity again like we are… The most significant changes only came because of government mandates and… entities like the accreditation bodies requiring us to do things.”
	“The state’s requirement is what really kind of throws a wrench into the works because that’s not the same… [as] what we would want to report to NCQA.”
	“That’s a big barrier because you know health disparities are not only ethnicity, they’re also race. It’s harder to care for a member when you don’t know both… that’s definitely a big barrier because then we have a large population of race, we don’t know.”
Multiple sources of race/Ethnicity data	“…the gold standard data is critical for everything else. It’s very hard for a health plan to move forward without engaging in direct data collection, not through an intermediary.”
	“One of the big lessons learned that we did was to question the data more before it gets to your doorstep… You need to know how the data gets put together before it gets to you.”
	“We also are starting to get…EHR feeds directly from provider groups where … they are providing us race information as well. Again, not certain whether those race values that are in those EHR systems are because the provider in an office asked the Member specifically what their race was or whether they… documented that based on what they believed.”
	“We’re in the process of developing a Member portal to also collect that… because the data that we do get from the state… the race and ethnicity, is not always accurate and… we saw that one member’s race changed multiple times throughout the months, so we found that to be interesting.”
Prioritizing data sources	“Having all business owners have an involvement in this particular process because it does take the entire org to work through the race/ethnicity data and also through—from the process down to the interventions, everyone has to be involved“
	“We have one analyst—we’re asking her to do the HEDIS race/ethnicity disparities analysis. We’re asking her to do the complaints and grievances by race and ethnicity. So there’s a lot on her plate as one person, what we’re trying to do now to is overcome that by disseminating that knowledge about how to do disparities analysis across those business areas so they can actually teach it to their analytics teams and everyone can have that skill set and not rely on a few folks within an entire organization to kind of do that in the future.”
	“There’s a lot of responsibility put on our shoulders to tell the organization, hey, if you see data coming in from this source, that is what you should use instead of the data that’s coming in from that source.”

Part II: Achieving Actionable Data

In addition to defining race/ethnicity categories and determining sources to leverage, plans must address at least two overarching logistical questions on use of race/ethnicity data in assessing disparities. First, what is a plan’s ability to report on the data? Second, how can it be assured that departments across the health plan collect, store, and report these data in the most effective manner?

Reportability

When linked to quality performance metrics, analysis of race/ethnicity data must navigate between including an appropriate number of groups (granularity) and having large enough counts to adequately report on those groups. Smaller counts lower precision; since racial identity is inherent in the reported value, this also increases the risk of being able to identify individuals if a reported category is too small. Plans with very small counts of certain groups may not be able to report on those groups, despite otherwise valid data. Some health plans noted situations where smaller groups experienced considerably worse care than other groups, but plans were uncertain how to conduct valid disparity assessments or target proper resources to eliminate the disparities faced by those groups. Further, small totals make it more difficult to demonstrate return on investment to stakeholders when reducing gaps for smaller groups, which leads plans to focus on addressing gaps found in larger populations.

Analysis becomes more complicated for plans that want to analyze data at a more granular level than the federal categories. For example, one critique of the category “Asian” is that it comprises a diverse set of groups with a variety of experiences and needs. Plans with a large Asian population may want to examine differences between their sub-populations. Three plans noted that they run analyses using more granular data because it better suits their needs than the required aggregated levels. However, such an approach often runs into the problem of smaller cell sizes generating less reliable output.

One plan proposed a creative solution to this problem: rather than aggregating smaller groups and risk losing the nuance between groups, the plan aggregates three consecutive years of data on a measure, on the premise that disparities are not limited to a single measurement year. This approach may help identify and clarify trends in disparity while maintaining meaningful and reliable data.

The importance of organizational structure and priorities

Different departments provide different contributions to enhancing health equity. A diverse array of teams and perspectives necessitates collaboration across departments for both data management and data access. Most plans emphasized the importance of multiple teams and departments coordinating the process of collecting, storing, and reporting race/ethnicity data in support of equity initiatives. This approach minimizes duplication of effort across units, such as creating standards for reporting on race/ethnicity and allowable data sources.

The processes of acquiring, managing, and using race/ethnicity data in the pursuit of health equity is a complex task. Some plans mentioned the importance of having the direction of senior leadership to prioritize and advance initiatives, making health equity a priority across the organization. Asking plans how they define health equity, and who is responsible for establishing the definition, resulted in a fundamental insight: The best approaches to harnessing race/ethnicity data and striving for equity goals include strong leadership support, coupled with a broad team effort.

Over half the plans interviewed discussed the importance of leadership roles in setting equity goals and defining equity as a priority. Three plans stated that the board of directors established a commitment to equity, often by initiating a health equity committee or health equity campaign. Many units or departments in a plan have a stake in the data, and the best way to support them is to invest in resources at the top, and find efficient ways to coordinate units that work with race/ethnicity data.

But a successful strategy requires more than a top-down approach. One plan noted that even though its efforts were spearheaded by the chief equity officer, setting equity goals involved a strong collaborative component across multiple units. Other plans suggested a dialectic approach to how their equity goals were defined and set, relying on cross-departmental or multiparticipant efforts. One plan said equity decisions were made by a mix of top leadership, middle management, and subject matter experts from multiple departments, including population health, case management, medical management, member services, provider services, and human resources. Representation from all business units is necessary to motivate change to eliminate inequities.

Illustrative quotes for Part II are described in Table 3.

Table 3.

Themes and Illustrative Quotes, Achieving Actionable Data

Theme	Illustrative quote
Reportability	“We definitely want to preserve… that more detailed and more granular level as well. And I think we feel that… information is very important and there may be other initiatives where we need to get down to that granular level. So if we have the data, why wouldn’t we want to utilize that and have that available?”
	“We’re a very small plan. And so then when you split it out to what we have data on, it’s a lot smaller. And then when you start looking at differences, they’re not statistically significant.”
	“…if you don’t have a large population for a particular measure and then 30% of them [are] unknown and then the rest might be divided between 5 race-ethnicity groups. You’re really dealing with very small numbers.”
	If you’re looking at a particular subpopulation and the biggest denominator is under race unknown… then you can’t look for disparities where data [are] missing, right… one of the biggest challenges is data completion.”
The importance of intraorganizational collaboration	“Well, it really helped to have our CEO say this was important because that meant that it became a top priority for anybody we asked to help.”
The importance of intraorganizational collaboration	“There’s recognition within [the org] that health equity can’t be attained without everybody’s involvement.”
Using race/Ethnicity data	“We did close the gap for our black population in regards to the COVID-19 vaccinations… Being able to stratify the data and look at the race and ethnicity, that was a big key point for us.”

Discussion and Health Equity Implications

Incomplete, unstandardized race/ethnicity data prevent clear assessment of health care disparities. In this paper, we outline primary actions identified by health plans as critical to successful collection and use of race/ethnicity data. These include collecting high-quality race/ethnicity data and managing data effectively to address inequities in care. Plans must consider which race/ethnicity groups are recognized for data collection, in addition to their ability to meaningfully report on those groups, and have an effective data infrastructure and clear mechanisms for prioritizing inconsistent information. Plans also emphasized the importance of intraorganizational collaboration for success in collecting and managing high-quality data.

The ultimate goal in collecting race/ethnicity data is to be able to systematically track and address disparities. Despite the challenges presented in this paper, plans across the health care system are demonstrating creative ways to collect and manage these data, to understand where disparities exist, and to inform efforts aimed at addressing gaps.

Although some plans could point to specific examples of using race/ethnicity data to address an identified gap, there remains significant work to integrate data collection into efforts to address health disparities. Data collection and management processes described in this paper are a critical first step to facilitating efforts at closing racial disparities in health care. As health plans become more adept at managing race/ethnicity data, the next step is to understand how to use the data in the pursuit of health equity. Future research should consider how to link stratified outcomes by race/ethnicity with efforts to reduce gaps in care.

The health plans in this study have laid the foundation for eliminating health disparities by establishing effective methods of collecting and managing high-quality race/ethnicity data. While the early part of this decade saw an increased emphasis on equitable outcomes at the state and federal levels, we are currently seeing a retreat from such accountability. However, for organizations that intend to continue pursuing health equity work, the need for reliable data cannot be overstated. Health care organizations can implement findings from this study to build capacity and develop an infrastructure to collect, manage, and leverage race/ethnicity data to advance high-quality, equitable care.

Footnotes

a

HEDIS is a registered trademark of the National Committee for Quality Assurance.

Authors’ Contributions

S.A.T.: Conceptualization (equal); data curation (lead); formal analysis (equal); investigation (equal); methodology (lead); writing—original draft (lead); writing—review & editing (equal); K.T.: Conceptualization (equal); data curation (equal); formal analysis (equal); investigation (equal); methodology (equal); project administration (lead); writing-original draft (equal); writing—review & editing (lead); R.M.: Conceptualization (equal); data curation (equal); formal analysis (equal); investigation (equal); methodology (equal); writing-original draft (equal); writing—review & editing (equal); R.H: Conceptualization (equal); supervision (lead); writing—review & editing (equal).

Author Disclosure Statement

No competing financial interests exist.

Funding Information

This project received no external funding.

Supplementary Material

Abbreviations Used

References

AHC Media. Easier to Ask for Race, Ethnicity, and Language Data if Registrars Know Why. Hospital Access Management. [cited 2020 Sep 1]. Available from: https://www.reliasmedia.com/articles/146673-easier-to-ask-for-race-ethnicity-and-language-data-if-registrars-know-why

AHC Media. Tips to Help Collect Race, Ethnicity, and Language Data Hospital Access Management [Internet]. [cited 2020 Sep 1]. Available from: https://www.reliasmedia.com/articles/146674-tips-to-help-collect-race-ethnicity-and-language-data

Cook

, Sachs

, Weiskopf

. The quality of social determinants data in the electronic health record: A systematic review. J Am Med Inform Assoc, 2021; 29(1):187–196.

Portillo

, Rees

, Hartford

, et al. Research priorities for pediatric emergency care to address disparities by race, ethnicity, and language. JAMA Netw Open, 2023; 6(11):e2343791.

Renzaho

AMN

. The lack of race and ethnicity data in Australia-A threat to achieving health equity. Int J Environ Res Public Health, 2023; 20(8):5530.

Velmurugiah

, Gill

, Chau

, et al. Collection of patient race, ethnicity, and language data in emergency departments: A national survey. CJEM, 2022; 24(8):832–836.

Davidson

, Denning

, Thorp

, et al. Examining the effect of quality improvement initiatives on decreasing racial disparities in maternal morbidity. BMJ Qual Saf, 2022; 31(9):670–678.

Stratifying Data and Implementing Financial Incentives: How States Are Leveraging Medicaid Managed Care to Further Health Equity [Internet]. [cited 2024 Jun 13]. Available from: https://www.shvs.org/stratifying-data-and-implementing-financial-incentives-how-states-are-leveraging-medicaid-managed-care-to-further-health-equity/

Isaacson

, Dyer

, Taylor

. State Health & Value Strategies. 2022 [cited 2024 Jun 13]. Stratified Reporting | CMS. Available from: https://www.cms.gov/priorities/health-equity/minority-health/research-data/stratified-reporting

10.

Nau

. Department of Managed Health Care. 2022 [cited 2024 Jun 13]. Health Equity and Quality Measure Set and Reporting Process. Available from: https://www.dmhc.ca.gov/Portals/0/Docs/OPL/APL%2022-028%20-%20Health%20Equity%20and%20Quality%20Measure%20Set%20and%20Reporting%20Process%20(12_21_2022).pdf

11.

Ghirardelli

, Ubri

, Karunatileka

. Advancing Equity in Race & Ethnicity Data in Population Surveys: Findings from Expert Interviews. NORC at the University of Chicago [Internet]. Available from: https://www.norc.org/research/library/advancing-equity-race-ethnicity-data-population-surveys.html [Last accessed: April 2024 ].

12.

Quan

, Clark

, Costigan

, et al. JBI systematic review protocol of text/opinions on how to best collect race-based data in healthcare contexts. BMJ Open, 2023; 13(5):e069753.

13.

Abuelezam

. Collecting and reporting accurate data on race and ethnicity are necessary to achieve health equity. J Obstet Gynecol Neonatal Nurs, 2022; 51(6):562–565.

14.

Cruz

, Smith

. Health equity beyond data: Health care worker perceptions of race, ethnicity, and language data collection in electronic health records. Med Care, 2021; 59(5):379–385.

15.

Thompson

, Mundra

, Trivette

, et al. NCQA Race and Ethnicity Stratification Data Learning Network Summary Report. 2023.

16.

Charmaz

. Grounded Theory. In: Rethinking Methods in Psychology. London: Sage Publications; 1995. pp. 27–40.

17.

Emerson

, Fretz

, Shaw

. Writing Ethnographic Fieldnotes. Chicago: The University of Chicago Press; 1996.

18.

Rubin

, Rubin

. Qualitative Interviewing: The Art of Hearing Data. 2nd ed. Thousand Oaks: Sage; 2005.

19.

Wolcott

. Writing Up Qualitative Research. 3rd ed. Los Angeles: Sage; 2009.

20.

Bernard

, Ryan

. Analyzing Qualitative Data: Systematic Approaches. Los Angeles: Sage; 2010.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB