Abstract
Frontline communities and allies have long advocated for environmental protections that address cumulative impacts (CI) as a solution to the disproportionate and adverse pollution burden in low-income and communities of color. Cumulative impact assessments (CIAs) include the environmental health burden of a community from multiple sources, pollutants, and pathways as well as the existing conditions, including impacts from non-chemical stressors such as exposure to social adversity. Addressing the CI in communities requires a pivot away from the traditional risk assessment approach to include more qualitative information, particularly lived experience (LE) from impacted communities and populations. Mixed methods (MM) approaches have well-equipped structures and validity practices to inform how to integrate, process, and analyze both qualitative and quantitative data to answer research questions and provide examples for regulatory analyses that inform environmental decision making. This article describes an investigation of the existing MM studies in the environmental science and environmental health literature, MM parameters, data integration methodologies, and validity practices to ensure study rigor. The analysis of the included publications describes and highlights robust examples for community LE inclusion during data collection, analysis, and interpretation phases but found few examples of studies where impacted communities informed the initial research design. The authors consider how MM approaches can support the integration of quantitative and qualitative data, including LE, in applicable phases of CIAs.
INTRODUCTION AND BACKGROUND
Spatial analyses demonstrate that pollution is not equally distributed geographically or across the U.S. population1,2,3 and predominantly show higher pollution burdens in low-income 4 and communities of color. The extent of these disproportionate impacts cannot be properly assessed or regulated in isolation and without reference to social context. Holistic approaches that address cumulative impacts (CI) are needed. The U.S. Environmental Protection Agency (EPA) Office of Research and Development provides the following working definition of CI, “the totality of exposures to combinations of chemical and non-chemical stressors and their effects on health, well-being, and quality of life outcomes.” 5 Assessing CI is an effort to capture the complex real-life exposures and risks that frontline communities experience.
State and local governments are driving progress toward more comprehensive environmental health protections, as state laws directed at environmental permitting currently make up the majority of existing CI policies. 6 Within these policies, community engagement and public participation are predominantly directed at information sharing and commenting on already-formulated draft assessments and permits. There are few examples of community involvement throughout the process, with the Chicago cumulative impact assessment (CIA) 7 being an exception.
Risk-based decision making is currently the predominant tool agencies use to regulate environmental health. This approach determines the potential for harm from single chemical exposures using default and hypothetical estimates for chemical transport, transformation, and exposure from release into the environment to their biological effects. More recently, environmental and public health agencies have begun to use quantitative tools to screen and map CI and environmental justice (CI mapping tools) to prioritize geographic areas for resource allocation and program delivery.8,9,10 Even these tools, however, often fail to capture the full range of information necessary to understand CI. Qualitative data like the lived experience (LE) of frontline communities is also of critical importance, offering locally specific information about stressors like odors, emissions and discharges, noise, health status, and neighborhood activities and history. Moreover, such data can reveal where averaging indices over geographies larger than the impacted community dilutes localized hazards and impacts.
Currently, public comment periods are the predominant form of gathering LE in regulatory settings and can influence 11 environmental decisions resulting in improved environmental quality and other factors. 12 However, gathering LE to inform an analysis or assessment is not common possibly because there is little guidance and scientific literature about how to incorporate it in a regulatory setting.13,14
Mixed methods (MM) approaches offer strategies for integrating quantitative and qualitative data that are potentially useful in CIA. Quantitative research uses numeric data to provide trends estimation, generalize from a sample to a population, test quantitative hypotheses, and make predictions. Qualitative research uses non-numeric data on experiences, behaviors, and perceptions to provide insight into the phenomena being studied. These data can take the form of interview or survey responses, photovoice, ethnographic notes, and themes from focus groups and they offer a powerful source of expertise about the communities that are the subject of environmental decision making. It is often assumed that environmental decision making is fundamentally based in the quantitative sciences, but EPA’s legal authorities do not preclude qualitative approaches,15,16,17 and there are examples of EPA regulatory decisions using qualitative data as evidence. 18
MM approaches have been gaining in popularity since the 1980s. MM design determines the relationship between quantitative and qualitative elements of a study and whether data are collected sequentially or in parallel. A dominant MM quality criterion is the appropriateness of the approach to address the research question. 19 EPA states that enlisting the natural and social sciences, and local and Indigenous knowledge, 20 is necessary to assess CI. This clearly suggests the appropriateness of MM approaches in CIA. Notably, some MM principles are reflected in health impact assessment (HIA), which EPA is considering as a model process for CIA. 21 Moreover, the Chicago Department of Health 22 organized its CIA using three HIA phases and used MM approaches such as integrated document review, survey responses, interviews, and notes from engagement sessions to develop their CIA work. This article describes MM approaches used in the environmental science and health literature and considers how these approaches could be applied more broadly in CIAs to integrate quantitative and qualitative data, including LE.
MATERIALS AND METHODS
Study selection and eligibility criteria
The authors completed a literature search in January 2023 using the following search algorithm in PubMed:
((“Mixed Method”)) AND ((environment) OR (“environmental science”) OR (“environmental health”) OR (“cumulative impact”) OR (“cumulative risk”) OR (“health impact assessment”) OR (“risk assessment”) OR (“chemical risk assessment”) OR (“tribal ecological knowledge”) OR (pollution) OR (pollutant))
The authors then screened the titles and abstracts to filter the publications to MM studies from the environmental health and/or environmental sciences, resulting in 86 publications. A full-text review of those criteria resulted in 76 articles, and 61 were research studies and are the primary focus of this article (Supplementary Data S2, Supplementary Table S1, “included publications”).
Data extraction
Finally, we coded each included publication for MM parameters including the approach, design, purpose, qualitative and quantitative data collected, the data merging method, data timing integration, type, author conceptualization of community, purpose of information collected from stakeholders/community/participants, and the article topic. We developed a Tableau (version 2022.2) based citation tool to facilitate access to the included publications (Supplementary Data S1).
RESULTS
The included publications were multi-disciplinary, with topics including triangulation of water quality issues, behavioral changes from environmental risk communication, and environmental justice analyses (Table 1).
Typologies and Parameters of the Included Publications
Information was collected from professional/educational experts and not those potentially impacted.
Information/perceptions were either collected passively or were not of those potentially impacted.
Approach purpose
An MM purpose statement communicates why both qualitative and quantitative data were collected and their roles in answering the research question. 23 In 16 studies, the purpose was to validate the results between the data types, a technique known as triangulation. For example, in one study texting platforms collected community reports of stress, acute respiratory symptoms, and locations and were triangulated with daily air pollution levels. 24 The most common approach purpose in the included publications was complementarity (23), when both data types are applied jointly to answer the same research question more broadly than is possible with one data type. In one study using a complementarity approach, air pollution monitoring results were integrated with interview data about co-occurring psychosocial stressors to enhance and elaborate on cardiovascular disease etiology. Eleven of the included publications reflected a development purpose, where one data type informs the other in a sequential design. The expansion purpose, identified in eleven of the included publications, increases the breadth, range, or scope of information using different qualitative and quantitative approaches. There were no studies with an initiation purpose, which refers to one approach informing the need for an additional study using the other approach, potentially because such studies are typically published separately.
Study designs
Most of the included publications (45) followed a parallel-convergent design, where qualitative and quantitative data are collected in parallel and integrated. One example was, a study that used ethnographic observation, close-ended survey questions, and demographic data to understand the effectiveness of air pollution data report-back. 25 There were eight exploratory sequential studies, including one that used qualitative focus group themes to inform a quantitative survey of health care providers’ evaluation of coal ash exposures. 26 Explanatory sequential was the least used design, found in three studies, both of which used quantitative surveys to guide the qualitative approach of focus groups. Five articles described a nested design, including an embedded survey on environmental factors within a smartphone app linked to sensors that monitor medication use for managing asthma and COPD. 27
Purpose for gathering community and research participant experiences
The primary purpose for gathering experiences from those potentially impacted was to gain information about the community itself (23). Types of information collected were daily routines and behaviors, 28 including adaptation behaviors, 29 community locational information, 30 and resource use and use frequency. 31 Assessing perceptions or opinions about the research itself (e.g., health guidance materials) (18) was the next most common purpose, followed by testing for understanding of information (4).
Although there was direct engagement of potentially impacted people in many of the included studies, mainly during data collection (50), only five involved such people in research planning. One study engaged Navajo people and leadership and incorporated Navajo law provided by experts from the Diné Policy Institute to investigate alternative fuel recommendations for indoor heating. Community perception of fuels, a cultural assessment, and technical emissions information were ultimately integrated into recommendations. 32 There was no direct engagement of those potentially impacted in six of the included publications, either because authors focused on professional or educational experts or collected experiential data passively (e.g., through analysis of tweets). 33
Integrating qualitative and quantitative data
Data integration can involve weaving both data types into a narrative discussion, transforming qualitative data into quantitative data (word or theme counts), transforming quantitative data into qualitative data (descriptive categories), and joint displays of both data types. 34 Fifty-eight of the 61 included publications presented data integration, with 22 integrating data through narrative. In one study, authors integrated data in bivariate maps of environmental hazards joined with social adversity indicators developed from youth interviews (joint displays). 35 In another example, the authors split interview quotes into categories of exposure to greenspace development to illustrate the relationship between gentrification and sleep disturbance in African American community members (data transformation). 36 In yet another example, the authors triangulated water quality information and issues by integrating community mapping, source water data, and quantitative surveys. 37 Researchers triangulated knowledge of air pollution, perceptions of hazards and protective behaviors in a joint display with quantitative air pollution exposure data. 38 Researchers also used a joint display methodology by integrating community perception and air quality estimates into a “subjective environmental vulnerability index.” 39 Integrating qualitative and quantitative data forces data type interdependence. 40 Moreover, if the qualitative data includes LE, integration helps to ensure that recommendations from research reflect the needs of those impacted. An indoor air quality study incorporated participant behaviors during exposure report-back to inform improvements in understandability. 41 They found that non-respondents were more likely to be Spanish-speaking and experience higher indoor air pollutant concentrations, which is important to improving reduction measures and engagement.
Integrating community information and perspectives
For the purposes of this article, we define a community as a group of people with shared experiences. The community shared experiences described in the included publications were resources and services, geography, pollutant exposure or environmental impact, health conditions, profession or education, adverse experiences, and/or identity (Table 1). Each of these are intersectional but were coded based on the foundational shared experience. Of the publications where participants were included early in the research planning stage, these “communities” shared pollutant exposure and environmental impact (3), health conditions (1), and resources and services (1). In a study initiated by a community with shared pollution exposures, researchers integrated localized experience, CI mapping tool results, local health and environmental hazards, drinking water testing, and community mapping to target lead hazard interventions.
DISCUSSION
MM support involving those most impacted by decision-making throughout the CIA process
The needs of those most impacted by a final decision or recommendation are most likely to be taken into consideration by ensuring their meaningful involvement throughout the process, including as early as initial planning. This must be a central principle in CIA. MM designs provide structures for community member contribution from assessment planning to outcome evaluation.
MM support the integration of approaches and data
Some CIA will require an investigation of localized impacts since CI mapping tools dilute impacts through spatial averaging. MM approaches provide methodologies to integrate LE with the quantitative data in CI mapping tools.
Our literature search revealed many data integration methodologies that are relevant to CIA. There are other methodologies that would be equally useful but were not extracted in the search. For example, matrix methods were recommended for CIA 42 and are used to develop CI scores. 43 Matrix methods use data integration through uniform categorization, such as percentiles of raw data. The “Pillar Integration Process” 44 has promise for CIA since it transforms raw qualitative and quantitative data into matching categories and ultimately themes using a five-column matrix approach. Researchers can weave the resulting themes together into a narrative reflecting qualitative and quantitative data.
MM support decisions based on CIAs
MM approaches encourage thought and rigor in bringing together qualitative and quantitative data and synthesizing this into findings to evaluate or inform decisions. CIAs could inform a continuum of decisions that increase in regulatory authority from outreach and information sharing to permitting and rulemaking. As CIA is necessary to address cumulative and disproportionate burdens and impacts, decision evaluations are crucial to confirm on-the-ground health protection. Although there were no included publications describing regulatory decisions, several of the included publications informed outcome evaluations45,46,47 and a choice between alternatives. 48 In some situations, a full MM study will be warranted, in other cases methods and lessons learned from MM practitioners will be helpful. CIA practitioners may make micro-decisions or seemingly inconsequential decisions made over a short time frame, about process or environmental, health, or social stressors that incrementally modify the final regulatory decision. Acknowledging this and making these judgements clear may reveal new areas that could be better informed by MM approaches.
An important question within MM approaches with regulatory significance is how to assign relative weight to quantitative and qualitative data. MM approaches allow researchers to communicate whether the qualitative or quantitative approach is core, supplemental, or of equal status through a consistent notation system. 49 None of the included publications directly or quantitatively weighted or discussed the unintentional weighting of quantitative versus qualitative data. Open and direct acknowledgment of weighting methods is necessary for transparency and expectation management, especially since gathering LE requires community members’ time and expertise.
MM approaches can enhance rigor by using two data types to answer the same research question. However, validity issues raised by the two data types individually—for example, study population representativeness, validity of inferences, and integration—may also compound each other. MM validity literature first advocated for quality measures based on each individual approach 50 but more recently reflects validity considerations unique to MM 51 sometimes referred to as multiple validities legitimation. 52 More recent MM literature recommends transparently communicating methods and practices, ensuring congruence between design and execution and connecting inferences and results. 53
MM support resolving qualitative and quantitative data incongruence
Within MM literature there are procedures for resolving incongruence of qualitative and quantitative data. Researchers may attempt to reconcile the results by re-examining the data and methods (reconciliation), initiating an additional study, or gathering more data (initiation). Practitioners should only exclude data when there is clear evidence of improper data collection or other circumstances leading to significant errors. In cases where incongruence cannot be resolved, practitioners may discuss context or limitations. As MM approaches become more commonly applied in CIAs, it will be important to remember that contradictory findings may not be invalid but rather represent different perspectives. In fact, Bryman, 2006 54 proposed an MM purpose, “diversity of views,” that mixes qualitative perspectives from research participants and quantitative results from researchers.
CONCLUSIONS
MM approaches offer strategies for integrating quantitative and qualitative data within CIA, including the LE of frontline communities. Notable approaches encountered in our literature review include the triangulation of data, the visualization of disparate data in joint displays, and the use of qualitative data collected in one phase of a study to develop a subsequent quantitative phase of a study or vice versa. However, although MM research is more likely than traditional quantitative research to feature qualitative data provided by community members, the studies we analyzed did not always involve meaningful and consistent community engagement. Future research could evaluate ways of increasing this engagement within MM frameworks. In addition, future research could analyze the incorporation of MM into the HIA 55 process framework and explore how it might be similarly built into CIA processes.
The integration of quantitative and qualitative methods of inquiry can offer us a fuller and richer understanding of environmental harm in overburdened communities and, most importantly, point us toward more effective solutions, particularly in the regulatory sphere. Familiarizing regulators with proven MM tools may increase the likelihood that diverse kinds of data will be considered in decision making and that community LE will influence environmental outcomes.
AUTHORS’ CONTRIBUTIONS
K.E. conceptualized the idea for the article. S.W. and B.P. analyzed and reviewed the conceptualization and provided recommendations. K.E. determined the methodology for the literature search, and completed a formal analysis from a literature search, and wrote the original draft. K.E., B.P., and S.W. completed the writing with respect to review & editing.
Footnotes
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
