Planetary health: Creating rapid impact assessment tools

Abstract

Addressing current environmental, social, health, and democratic challenges requires projects and evaluations to be conceptualized differently. This article proposes a pathway to creating rapid impact assessment tools that consider the dimensions that matter the most to ensure positive impacts for a thriving future. This work is based on three premises for evaluation practice, which needs to: contribute to a positive ecosystem, adopt a holistic perspective, and engage participants in deliberative and democratic practices. We first review related evaluation approaches—health impact and environmental impact assessments—to learn from these experiences and avoid their pitfalls. Second, we present and illustrate how the Planetary Health Rapid Impact Assessment tools can be developed and used. This article, is intended to inspire and support policymakers, program designers, decision-makers, administrators, and evaluators willing to positively influence planetary health and introduce such tools in their projects.

Introduction

Addressing current environmental, social, health, and democratic challenges invites us to revisit mainstream practices in everything we do. Evaluation can contribute to the design and implementation of policies and programs that, in turn, contribute to a better future. At this time, the field of evaluation is exploring approaches to address concurrent existential crises (see, for example, Brousselle and McDavid, 2021; Brousselle et al., 2022; Patton, 2020; Rowe, 2018, 2019a; Uitto and Batra, 2020; Van Den Berg et al., 2021). With this article we aim to further this work.

This article proposes and illustrates a pathway for creating scalable rapid impact assessment tools aimed at fostering positive impacts on planetary health. Planetary health is an approach that considers the interdependence between the elements and living beings (plants, animals including humans), emphasizing the importance of cultivating a positive ecosystem to support the flourishing of all forms of life. The Planetary Health Framework and Rapid Impact Evaluation (Brousselle and McDavid, 2021; Brousselle et al., 2022; Rowe, 2019b) are foundational for this work. Such tools can be developed for all kinds of projects, programs, or policies (called interventions below)—whatever the topic or scope. In effect, we are advocating for a transformation in how we conceptualize all interventions and how we evaluate them. Everything we do has an impact on the environment and human health and well-being. As policymakers, program designers, decision-makers, administrators, or evaluators, we need to reveal and discuss these impacts in all our projects and engage with broader communities on these topics.

This work is based on several premises. First, we need to design and implement interventions contributing to a positive ecosystem, one that presents conditions for life to thrive and conditions that protect, restore, and regenerate (Hopkins, 2014; Wahl, 2016). By “regenerative” we mean that we need to think about and actively design projects, programs, and policies that contribute to the “good,” and reverse current depletion trends (Hopkins, 2014; Wahl, 2016). The status quo is harmful; evidence shows we are reaching tipping points that will create dangerous feedback loops that cause more harm. As professionals, we need to think about and design interventions that contribute to regeneration. Regeneration can take place at different levels and in different domains. It could mean, for example, preserving existing or creating new carbon sinks for absorbing carbon emissions; reducing and repairing economic, social, and political injustices; as well as promoting healthy environments.

Second, a holistic approach that recognizes the intrinsic interconnectedness of our being, rooted in the biosphere, is needed to tackle the multiple and embedded crises all societies are experiencing. Trying to address one crisis at a time, without considering broader impacts on the environment and community, increases risk potential in other areas (e.g. nuclear energy as a climate change solution). Furthermore, a siloed approach is likely to contribute to worsening environmental and population health and well-being because it creates blind spots at the seams in our solution design. All kinds of impacts should be considered: direct effects and externalities; short-term to long-term; and intended and unintended (Brousselle and McDavid, 2020; Rowe, 2019a).

Third, deliberative democratic practices (House and Howe, 2000) or interpretive approaches to policy making (Fischer, 2019; Fisher, 2021; Yanow, 2000) need to be implemented. In societies increasingly divided by distrust and strategic disinformation (Fisher, 2021; House, 2020; McIntyre, 2018), we will face resistance in making our work relevant and credible. Deliberative practices respond to this dynamic context and increase accountability; such practices give voice to those affected by policy and program decisions and, ultimately, empower communities. An approach that creates space for deliberation, dissent, and divergent thinking can generate new perspectives based on lived experiences, and particular values and representations.

This article presents a way to build and utilize Planetary Health Rapid Impact Assessment (PHRIA) tools which include critical dimensions for a thriving planet. First, we discuss existing related approaches in the evaluation field; in particular, we review the environmental and health impact assessment (EIA & HIA) approaches and their combinations to learn from their applications and pitfalls. We think it is important to draw from evaluation approaches and experiences which share a similar objective of analyzing coupled impacts on natural and human systems.

In the second part of this article, we present two PHRIA tools developed for local governments and describe the process by which they were designed. Using real examples, we hope to raise the evaluation community’s interest in developing similar tools for other contexts. We also hope readers will realize that it is possible to build accessible tools that engage and involve participants (from communities, beneficiaries, and evaluation commissioners) in the development and use of such tools. Finally, we discuss how such tools can be used to enhance multi-party engagement as a desirable deliberative practice. PHRIA tools can serve different purposes and be used at different stages: when designing interventions, for decision-making, and for post-implementation evaluation. They can be used as a stand-alone exercise or be integrated as a complement to evaluation project methodologies. They can easily be adapted to a diversity of contexts.

After reading this article, we hope people will feel inspired and have the skills to adapt or develop tools for their communities to make a positive impact on Planetary Health. As we extensively discuss EIA and HIA approaches one could legitimately ask whether we recommend replacing these with quick assessment tools. This would indeed be an interesting discussion to pursue. Our position is to encourage the consideration of life cycle impacts on natural and human systems in a variety of interventions, and not only for major infrastructure or industrial projects which is the dominant domain for HIA and EIA. Therefore, we are not arguing to replace these tools per se. We defend that to address the current polycrises, we collectively need to systematically consider the impacts of all projects, programs and policies. This will be feasible if and only if useful tools and accessible approaches other than HIA and EIA are made available to a diversity of actors. The recent IPCC report (Intergovernmental Panel on Climate Change, 2023) and United Nations report on progress toward the Sustainable Development Goals (United Nations, 2023) both paint a desperate portrait of the gap between what we need to achieve and our current collective (in)action. A transformation is needed, in everything we do, in all sectors, at macro and micro levels, and in the near-term. The PHRIA is one option in a range of possibilities for systematically raising awareness, engaging and taking individual and collective actions to positively impact natural and human systems. Deliberative and easy-to-use tools can be attractive to policymakers and organizations wanting to improve their impact and increase their accountability to communities. In this article, we show that introducing Planetary Health thinking into a diversity of evaluative projects and to support decision-making is both feasible and accessible. We offer guidance for those interested in developing assessment tools for their own context. We hope that generalizing such practice will contribute to transforming societies, so that our current social and environmental challenges can be addressed in everything we do, in a holistic way.

Learnings from evaluation approaches assessing natural and human impacts

The governance context is changing. Governments are committing to action on climate change and loss of biodiversity, and respect for Indigenous People’s rights. Legislation binding public institutions to systematic consideration of impacts on natural and human systems is being implemented. Examples include: the Canadian Treasury Board’s systematic integration of gender analysis and climate change impacts in their evaluations; the adoption of the United Nations Declaration on the Rights of Indigenous Peoples (UNDRIP) Act in Canada; and the recent “Wellbeing of Future Generations Act” in Wales. Each of these examples has created a need for systematic consideration of a range of policy and program impacts that extend beyond the program’s core objectives. However, the gap between the creation of legislation and the development of systematic assessment approaches to meet these new expectations is enormous.

EIA and HIA and their combined approach

Considering an intervention’s health and environmental consequences is not a new idea in the field of evaluation and in decision-making processes. Governments started to implement EIAs of major projects as early as the late 1950s (Turnbull, 1992). Harris et al. (2015: 1045) indicate that EIA is the regulatory process within 190 of the 193 member states of the United Nations, preventing and mitigating the potential negative environmental impacts of industry development projects, before they occur.

In the 1990s, we saw the emergence of a new approach, HIA, aimed at assessing impacts on health and equity of projects in the non-healthcare sectors, and suggesting mitigation measures for reducing harm and increasing positive impacts on human beings (Birley, 2011; Collins and Koplan, 2009; Wright et al., 2005). The creation of HIA took place within a larger movement defending the importance of health, a period marked by the adoption of the Ottawa Charter for Health Promotion (1986), the WHO Healthy Cities project, and the “Think Globally, Act Locally” movement (Santé Canada/Health Canada, 2004: 16). Since then, the field of public health has given more prominence to the importance of the ecological determinants of health (Lang and Rayner, 2015) which should be reflected in how HIA are conducted.

Many governments and organizations, such as Canada (under the Canadian Environmental Assessment Act), Europe (with the Strategic and Environmental Impact Assessment legislation), Switzerland, Australia, and the United States, are now combining EIA (or Strategic Environmental Assessment, as in Europe) and Health Impact Assessment (HIA) with the aim of identifying mitigation measures or alternatives leading to more positive health and environmental impacts (Fehr, 1999; Green et al., 2020; Gulis et al., 2022; Linzalone et al., 2019; McCallum et al., 2018; Turnbull, 1992). Such a combined approach is also considered the most likely avenue for translating the 2015 Welsh “Wellbeing of Future Generations Act” into evaluative action. Public consultation and engagement are components of both EIA and HIA (Bhatia and Wernham, 2008; Santé Canada/Health Canada 2004; Turnbull, 1992), and the expectation is that the general public and interest groups should be involved (Turnbull, 1992). EIA, HIA, and their combined approach is normally a prospective exercise conducted with the objective of identifying alternatives or suggesting program changes before implementation (Collins and Koplan, 2009). However, these approaches can involve monitoring or be conducted retrospectively to provide information for other prospective Environmental and Health Impact Assessments, particularly when a decision needs to be made rapidly (Green et al., 2021).

The application of these approaches varies and even if the legislation is clear, assessment agencies still find some margin to interpret their mandate, including the range and depth of the analytical work involved. For example, in Canada, the assessment of the Bay du Nord Project, a mega floating offshore oil production facility 500 km east of the coastline of Newfoundland and Labrador recently approved by the federal government (Minister of the Environment, 2022) clearly illustrates the limitations of the current application of existing legislation and methodologies. In the assessment conducted by the Government of Canada (all documents are available here: Sept 7, 2023, https://iaac-aeic.gc.ca/050/evaluations/proj/80154), the purpose of the project (oil production and consumption) has been omitted; instead, the assessment focuses only on assessing the facilities for extraction. However, in the Canadian Environmental Assessment Act (CEAA) (2019) and in the Impact Assessment Act (IAA) (2019) the language is clear. The CEAA defines effects as “changes to the environment or to health, social or economic conditions and the positive and negative consequences of these changes.” The objectives of the CEAA are, among others, to “foster sustainability,” “to ensure that impact assessments of designated projects take into account all effects — both positive and adverse — that may be caused by the carrying out of designated projects.” Furthermore, as indicated in IAA, impact assessments should take into consideration “the purpose of and need for the designated project,” “the extent to which the designated project contributes to sustainability” and “the extent to which the effects of the designated project hinder or contribute to the Government of Canada’s ability to meet its environmental obligations and its commitments in respect of climate change.” As for the context, the Government of Canada committed to reduce its greenhouse gas (GHG) emissions by 2030 to 40–45 percent below the 2005 level (Environnement et Changement climatique Canada, 2023). We also know that climate change has been identified as a major threat to human survival, and climate events are increasingly affecting the most vulnerable populations (Watts et al., 2015 and see also IPCC reports). However, notwithstanding the overall context and the language in the Acts, the primary purpose of the activity and its impacts has been excluded in this assessment. No lifecycle analysis has been conducted in relation to Canada’s commitments. The example shows that adequate legislation and methodologies are not enough. The fact that the EIA is an extensive technocratic process led by a few experts reduces the capacity for the public to understand how conclusions are drawn and to challenge the results themselves. This example illustrates the need for simpler, more accessible, and engaging processes.

Main observations and limitations of a combined Environmental and Health Impact Assessment approach

Several authors have analyzed the impacts of EIA and HIA used as stand-alone exercises or in combination. To improve the potential use of Planetary Health tools there are several lessons to be learned from these experiences.

First, according to the 1969 National Environmental Policy Act (NEPA) in the United States, HIA and EIA should be conducted in combination (Collins and Koplan, 2009); however, when they are, health impacts are not always systematically considered or a narrow definition of health impact is used which only focuses on toxic exposure and safety issues (Bhatia and Wernham, 2008; Linzalone et al., 2019; Noble and Bronson, 2005). The difficulty of integrating the consideration of health impacts in EIA has been raised in other contexts (Jabot and Rivadeneyra-Sicilia, 2022). Several difficulties are mentioned. If in principle HIA adopts a broad definition of health determinants, a comprehensive approach to health is still mostly non-existent (Bhatia and Wernham, 2008). One possible explanation cited for this omission is the lack of expertise and guidance (Jabot and Rivadeneyra-Sicilia, 2022; McCaig, 2005). In addition, HIA often excludes socio-economic determinants of health, leading to recommendations that potentially increase social inequities, which are known to have a negative impact on health (Buregeya et al., 2019; McCallum et al., 2018). Biophysical, mental, and social health determinants that more fully reflect human health and well-being should be simultaneously valued and integrated (Buregeya et al., 2019; Bhatia and Wernham, 2008; Jabot and Rivadeneyra-Sicilia, 2022; Linzalone et al., 2019).

Second, EIA tends to be scientifically very complex, often relying on complicated quantitative techniques. This approach tends to introduce various difficulties and risks. It tends to minimize or ignore judgments based on experience, which are often better represented using qualitative methods. In particular, cultural knowledge or Indigenous experience may be disregarded or misrepresented (Beaulieu-Guay, 2022; Gulis et al., 2022; Turnbull, 1992). For example, Beaulieu-Guay (2022) mentions that even if Indigenous people and organizations are systematically consulted, traditional ecological knowledge is often treated as marginal in impact assessments: “consultation with Indigenous Peoples and organizations in regulatory impact assessments is primarily to identify their preferences, answer technical questions, or validate support for the proposed regulatory change” (Beaulieu-Guay, 2022: 412). Similarly, another observation made by Gulis et al. (2022) is that, despite the existing guidance, people without specific expertise in health often experience difficulties in conducting or even interpreting scientifically complex analysis. These observations call for alternatives for compiling and synthesizing different sources of information, for bringing justice to different kinds of knowledge and worldviews, as well as making relevant and comprehensive assessments accessible to non-experts.

Third, conducting an impact assessment is meant to prioritize science-based knowledge, but inevitably, given the stakes, it is also a political process (Turnbull, 1992). Debate often results in trade-offs between monetized economic benefits and non-monetized, common pool environmental impacts, ignoring the embedded relationship between human health and natural systems and the complex and interrelated nature of environmental, social, economic, and health dimensions (Bhatia and Wernham, 2008). “Expert judgment” is meant to add value by identifying cumulative environmental and health impacts and by understanding health consequences of projects (Bhatia and Wernham, 2008). However, in the political context in which HIA and EIA are conducted, even if HIA demonstrate high scientific validity, “HIA practitioners should expect some skepticism regarding complex health arguments that involve environmental, social, and behavior pathways” (Bhatia and Wernham, 2008: 998). Furthermore, Bekker (2007) notes that adopting a “technocratic design and practice” to HIA creates a barrier to reshaping policies and is leading to a “politicisation of knowledge rather than to the rationalisation of policy” (Bekker 2007: 193–194). The lack of transparency is also identified as a barrier to engagement and participation (Iglesias-Merchan and Domínguez-Ares, 2020). The political dimensions of HIA and EIA assessment cannot be minimized. Instead, they should be anticipated and foster public engagement and deliberation, notwithstanding the challenges of meaningful public participation (Iglesias-Merchan and Domínguez-Ares, 2020), while also finding the right balance to preserve the validity and credibility of the knowledge component.

Fourth, neither EIA nor HIA consider a project’s complete life cycle. Without adopting a life cycle perspective to assess projects, there is a huge gap between legislative and regulative intentions and the results from actual practice. For example, the approval process of the Canadian Bay du Nord oil and natural gas project (Minister of the Environment, 2022) demonstrates that despite adequate legislation defining effects broadly (encompassing health and environmental impacts to foster sustainability), available methodological guidance, and current national commitments to reduce GHG, EIA are failing to meet their objectives by approving industrial projects that comply with only one phase of the project (i.e. extraction but not consumption); as such, the project creates high risks to climate change and human health. This example demonstrates that EIA and HIA fail their purpose when a life cycle perspective is not incorporated into the analysis. “Life Cycle Assessment (LCA) is a holistic, cradle-to-grave, environmental approach which provides a comprehensive view of the environmental aspects of a product or process throughout its life cycle” (Curran, 2015: 1). Technically, a life cycle assessment includes raw material acquisition, manufacturing which involves three steps—material manufacturing, product fabrication, filling/packaging/distribution; use/reuse/maintenance; recycle/waste management (Curran, 2015). With a Life Cycle Assessment perspective, environmental impacts, from the very start to the very end, are included (Curran, 2015: 1). Examples of evaluation using a life-cycle perspective exist (see Maloney et al., 2022; Stevens et al., 2023).

Finally, as EIA and HIA are used in different contexts and projects, their impacts on public policies vary. In Canada, both assessment tools have been used mainly at the federal level to analyze major industrial projects. In Quebec, HIA is used to implement the Health in All Policy Act at the municipal level, but not consistently. In order to maximize EIA and HIA’s positive impacts, assessments should be applied to support the design of all projects, programs and policies. Widespread application is unlikely based on how they are currently conducted (essentially as large-scale research projects) and the requirement for expert input. Bekker (2007) insists on the importance of being flexible throughout the process, allowing for rounds of discussion and redesign, so that policymakers feel supported in designing effective health policies and are not threatened by the exercise. This is, for example, where a rapid assessment tool—integrating environmental and health aspects in a holistic and integrated manner, accessible to anyone, independently of their level of expertise—can make a positive difference.

Principles for the development of Planetary Health Rapid Impact Assessment tools

Based on EIA and HIA implementation experiences, we can glean principles that can inform the effective design of PHRIA tools.

Co-design

Co-designing the tool in collaboration with targeted users ensures the assessment dimensions are contextualized according to their needs and that the language used to describe the dimensions is meaningful, understandable, and comprehensive. In addition, co-designing the tools according to the specificity of the context allows governments/organizations to address specific commitments. For example, it would be possible to develop a version specifically addressing the United Nations Sustainable Development Goals, if that was a priority for the organization (Green et al., 2020). However, critical dimensions for planetary health as identified in the Planetary Health framework should all still be considered as these were identified by existing scientific knowledge.

Easy-to-use tools

In EIA and in HIA, two barriers limit their use and impact: experts are needed to conduct the study and eco-social determinants of health are not easy to understand for non-experts. As we want to encourage widespread use of PHRIA tools, a simple tool can empower people to conduct their own assessments without planetary health-related expertise or knowledge. To make the assessment exercise accessible to non-experts, PHRIA tools should include some language describing what each planetary dimension encompasses. In particular, to circumvent the difficulties experienced in HIA and EIA, it is important how the dimensions of health, equity, and power relations are described.

Deliberation and engagement

Once created, the tool should be implemented and used in deliberation with different groups such as elected officials, administrators, and decision-makers, but also diverse interest groups, such as industrial groups, Indigenous leaders, beneficiaries, and community members. Sharing the tool with a diversity of individuals and groups early in the process is a way to increase the credibility of the exercise, and to enhance deliberation around potential trade-offs. The trade-offs among which dimensions to prioritize over others wouldn’t be the result of an expert’s decision, instead this would be informed by the positions of various groups and the debate taking place. Furthermore, assessing a project using a PHRIA tool may also surface alternatives with potentially higher positive impacts for consideration. Larger participation would also help in controlling individual rating bias.

Life-cycle perspective

Finally, with the purpose of the intervention in mind, we would like to re-emphasize that all kinds of impacts on natural and human systems, in both the short and long term, should be included. In addition, the impact assessment of projects, programs or policies shouldn’t be limited to their implementation or production processes. Limiting the scope of the assessment could derail the process. Pre-selecting a stage or part of the project for evaluation purposes inevitably leads to missing the purpose of exercise itself. A life-cycle perspective should be adopted.

Designing Planetary Health Rapid Impact Assessment tools: A case example

Developing PHRIA tools originated from discussions with elected local government officials who wanted to apply the Planetary Health framework (Brousselle and McDavid, 2021) to guide policy design and decision-making. They wanted easy-to-use and engaging tools applicable to any kind of project or policy. Suggestions were made to develop checklists that could be applied quickly, while referencing key dimensions of the Planetary Health framework in a systematic way. The objective was also to make such tools available to program and policy designers at an early stage so they could consider the planetary dimensions/criteria that decisions would be made against. Consequently, the tools would support the design stage. These tools were also created with the objective of fostering engagement of broader communities such as program participants, citizens, and different interest groups.

Two rapid assessment tools were developed. Below, we will present each tool, describe different aspects of the tool design and use, discuss their comparative advantages, and show how results are presented. The development of these tools is presented as an example to support potential users in designing their own tools to fit their own contexts.

The Planetary Health framework was created using scientific consensus on the dimensions that matter the most for human beings and other living species to thrive (Brousselle and McDavid, 2021). Its foundations rely on the work on the planetary boundaries and the critical dimensions of natural systems that create “safe operating space” for humans (Richardson et al., 2023; Rockström et al., 2009; Steffen et al., 2015), on socio-ecological determinants of health, on ecological economics (Alexander, 2015a, 2015b; O’Neill et al., 2018), and on the determinants for successfully implementing interventions. The tools have been inspired from existing work, in particular the tools developed by Andy Rowe (2019b) on Rapid Impact Assessment. We share with Andy Rowe’s (2019b) approach to the Rapid Impact Assessment, the goal of offering a pragmatic and rapid approach to assessing impacts of interventions. Similar to Rowe’s approach, PHRIA is a theory-based evaluation approach that can be used ex-ante or ex-post “providing utility for developmental and formative evaluation as well as the usual summative settings” (Rowe, 2019b: 1). Both approaches rely on the triangulation of different groups’ assessments of the impacts of the intervention. However, a major difference between the Rapid Impact Assessment approach as developed by Rowe and the tools we present below is that Rowe’s approach does not involve the systematic consideration of planetary impacts that matter the most for natural and human systems but rather focuses on the program’s goals as usually (and narrowly) defined in most evaluations. This addition is key. Furthermore, while Rowe’s metrics method was used to design one of the PHRIA tool (the numerical scale), the PHRIA scorecard and the envisioned process for use differ from Rowe’s suggested approach, hence making them different approaches. In the next sections we will describe how the PHRIA is designed and implemented.

Adapting the planetary health dimensions to evaluative contexts: Local government illustration

The Planetary Health framework includes eight dimensions (Brousselle and McDavid, 2021). Three address natural systems based on the planetary boundaries work of Rockström et al. (2009) and Steffen et al. (2015) (reducing pollutants, protecting biodiversity, and protecting and revitalizing lands and waters), three are determinants of human well-being (health, equity, and prosperity) and two relate to dimensions critical for the successful implementation of interventions (power and governance). For building PHRIA tools, the eight dimensions of the Planetary Health framework need to be adapted to evaluands. The challenge is to speak to the dimensions of the planetary health framework while finding language to make them pertinent to the context and accessible to people with no expertise in these areas.

In contextualizing the framework for local governments, one goal was to have language that would represent the Planetary Health framework’s dimensions and simultaneously make sense for municipalities. For example, all municipalities in Canada need to implement the UNDRIP. Hence, the equity and governance dimensions of the Planetary Health framework were contextualized to reflect the expectations set by UNDRIP. Another example is the separation of GHG emissions and pollution dimensions, which resulted from local government officials highlighting a legislative requirement. Although there is some overlap between the two dimensions, the revision made the tool more relevant to local governments.

Indigenous participants commenting on tool design indicated the implementation dimensions should be integrated to consider ways to redistribute power within a governance system still imposing colonial institutions and values on Indigenous communities. Integrating the concept of “reconciliation” and some specific language to describe the power dimension created room for this agenda which is relevant and important to the local context.

The purpose of the rapid assessment will also influence whether the implementation dimensions (power and governance) should be included in the tool or not. If the tool is guiding program design, these two dimensions should be included. If the tool’s purpose is to assess an intervention for summative purposes, these two dimensions may not be as pertinent.

Similarly, we want to mention that the overlaps between the dimensions of the Planetary Health Framework and the United Nations 17 Sustainable Development Goals (UN SDGs) would make it possible to design tools addressing both frameworks, while emphasizing each one of the UN SDGs.

Co-design

Draft versions of the tools were co-created by researchers and local government elected officials with the objective of translating the dimensions of the Planetary Health framework in a way that made the tools accessible and understandable. Although elected and appointed local government decision-makers have a wide range of backgrounds and expertise, these tools are primarily intended to engage users, offering opportunities to consider how their roles and responsibilities intersect with the broader Planetary Health context, in which their work is embedded. A medium-term goal was to transform their collective approach to designing and assessing the merits of policies and programs.

Once the first versions were developed, participants were recruited to test the tools. Participants were either elected officials, administrators in local governments, or Indigenous Peoples. Eight participants from Canadian local governments and Indigenous contexts were recruited for this stage. They were asked to choose an intervention they knew well and to assess it, using the scorecard option and then the likelihood by magnitude scales option. The intervention could be of any nature and scope. Tools were refined after each interview, as described in the next section.

Assessment tools

Two rapid assessment tools were developed. One is a scorecard which includes eight dimensions, each to be rated with a slider that ranges between a given intervention having an estimated strong negative impact on that dimension to having a strong positive impact (see Figure 1). Four planetary health dimensions address the environment and four address human well-being determinants. Each dimension is described in subtext to help the rater consider the elements and sub-dimensions that matter. As not everyone has the same familiarity and understanding of each concept, it is important to include a short description of the dimension to illustrate what should be considered by the rater. This subtext can also be used to speak more specifically to contextual characteristics. The scale is adapted for each dimension: on the left is doing harm; on the right is having positive impacts. Note that the scale is deliberately reversed for certain dimensions—going from high to low for pollution and greenhouse gas emissions, as both are undesirable, whereas other dimensions, such as health and equity, go from low to high as they are desirable.

Figure 1.

Planetary health policy-design tool for local governments.

The second tool includes the same dimensions but they are assessed using two scales for each dimension: interventions are assessed according to the likelihood/subjective probability that they have an impact on a given dimension (ranging from none to certainty; score from 0 to 3), and the estimated magnitude of the impact (including its valence, positive or negative; score from −3 to + 3), following the metric method described in Rapid Impact Evaluation (Rowe, 2019b). For each dimension, the likelihood is multiplied by the estimated magnitude impact to get a weighted score (from −9 to +9). A similar two-scale scorecard has been developed by the United Nations Environment Program (2020). The two scales are the probability of risk and impact of risk which is similar to Rowe’s approach. An overall risk is then estimated using a matrix compiling the results from the two scales. (Examples of this checklist approach can be seen online: https://www.ctc-n.org/system/files/dossier/3b/Zambia_CTCN%20SRIF_Signed.pdf; https://open.unep.org/docs/gef/ESS/10525_Safeguards%20Checklist.pdf).

Following data collection for both PHRIA tools, the scores were converted so that when aggregated for a given project, they formed a Planetary Health Index that ranged from −100 to +100, as well as indices for natural and human systems, separately. For the scorecard, visual ratings were translated from analogue unmarked scales to numbers for the purpose of calculating an index value. The distance between the start of the scale and the mark chosen by the respondent was measured and the distance was converted into a score. This approach is similar to the one used in cost-utility studies using visual analog scales (Drummond et al., 2005). In our case, the scale was 5.5 cm long. The mark on the scale was measured from 0 and then converted to a score of −9 to +9 in order to compare results obtained with the other tools. The calculation was ((mark distance * (9/5.5)) − 4.5) * 2.

For calculating the Planetary Health index, the scores of the eight dimensions on a −9 to +9 scale were translated to a scale of −100 to +100 using the following calculation: (((Σ dimensions scores + 72) * 100/144) − 50) * 2. We calculated a score for impacts on natural systems and on human systems using the same technique but adjusting for the number of dimensions referring to natural and human systems (n = 4 for each): (((Σ dimensions scores + 36) * 100/72) − 50) * 2.

We tested both tools during each interview, assessing one project identified by the participant. The results of the different assessments were compiled and analyzed to learn about their comparability. The two quick assessment tools are built differently and are not designed to measure the exact same elements of each dimension. Hence, it is not surprising that the Pearson correlation between the scorecard option and the weighted score approach was moderate (0.40).

Participants expressed different preferences regarding the two tools: the scorecard appeared to be more intuitive and easier to understand than the two-factor scale that included likelihood and magnitude. With the aim of reaching a diverse pool of non-expert users, this perception presents an advantage. However, some participants expressed a preference for the two-factor scale’s notion of magnitude of change, which is a desirable element to consider when thinking of the transformation that is needed to address climate change. Nevertheless, the magnitude of change can be problematic as it can lead to a lower valuing of interventions that focus on a singular community with smaller scale impacts, regardless of the transformative change for the community.

The results obtained were translated into composite indices as described earlier in this article and presented graphically (see Figure 2 for an example).

Figure 2.

Example of synthetic indices calculated from scorecard results.

Use and results presentation

These tools can be used for different purposes and at different stages: when designing interventions, for decision-making and for evaluation after the intervention’s implementation. We explore these three purposes below.

When used to support the design of interventions, these tools can help administrators, policymakers and other stakeholders to think systematically about all dimensions that matter the most, as well as their interactions. The tool can be used in an iterative way by participants to facilitate dialogue and to improve the intervention; in fact, it could be used to track interventions, post-implementation. When testing the tools, participants mentioned that the tools shed light on dimensions they didn’t consider initially and helped them think in a systemic way.

If used in decision-making, one needs to pay attention to the risk of bias, especially if only one respondent is rating the intervention. In particular, participants with high standards for some dimensions (on Truth and Reconciliation, for example) were rating projects more severely than participants without the same priorities. As a result, project ratings could be more generous or more severe depending on whether someone with direct interest in the dimension(s) perceived the project as contributing more positively it. A way to circumvent this potential bias is to invite different respondents (citizens, elected officials, different staff and administrators, experts, and business representatives) to rate the same intervention, and then compile or compare their scores. This would provide information on how the intervention is perceived by different groups and would also foster local government accountability.

If used for evaluation purposes, a small group of different constituencies such as the designers, the beneficiaries, and other interest groups could be convened and asked to independently assess the intervention. This strategy would be similar to the interest-based approaches to stakeholders recommended in Rapid Impact Evaluation (Rowe, 2019b) applied either ex-ante or ex-post.

The overall scores of impacts on natural and human systems and on planetary health are useful when willing to identify which interventions have the most desirable impacts. However, they are not as informative as scores presented individually, especially when an intervention is rated by several participants. As an example, we calculated the impact scores on planetary health of a 2023 international conference, based on the ratings of 28 participants. The scale used offered a choice among four responses—strong undesirable impact, small undesirable impact, small positive impact, and strong positive impact—avoiding a middle option and forcing choice. Figure 3 represents the general calculated impacts on natural and human systems. The scale goes from −100 to +100. The calculated impact on human systems is +23, while the calculated score on natural systems is −24 (see Figure 3). The calculated impact on planetary health is around 0. However, the disaggregated presentation, showing results for impacts on each dimension, shows a much richer portrait of the general impact (see Figure 4).

Figure 3.

General impacts on human and natural systems of a 2023 conference (scale: −100 to +100).

Figure 4.

Impacts on planetary health dimensions of a 2023 conference.

Engagement and deliberation

Coupling the rating process with a deliberative activity increases shared understanding of the intervention and of its impacts. Diverging perceptions, values, trades-offs, or promising alternatives can be made explicit and be discussed which ultimately reinforces the validity of the rapid impact assessment of the intervention, if made after the discussion.

As mentioned earlier, these tools can be used for different evaluation purposes: from supporting the design of projects and policies, for decision-making, to conducting summative evaluations. They can be used by individuals, or they can solicit the participation of a large group of people. When testing the tools, their usefulness for identifying trade-offs was highlighted. When implementing an intervention, choices sometimes result in harm to one system when trying to protect or regenerate another system. These trade-offs are often ignored, not out of ill intent, but because purposive interventions emphasize intended objectives and not the exploration of the harm an intervention could bring. There is value in identifying trade-offs and in making choices explicit. These trade-offs can be identified by individuals when using the tools and can also be brought to group deliberation.

Surveying a large group of participants and including some questions about participants’ characteristics can also help identify diverging opinions within sub-groups. For example, when asking conference participants to rate the event, we asked if respondents identified themselves as belonging to a marginalized group. When analyzing the results, we noticed that the respondents who answered yes to this question were also more likely to rate the conference as having an undesirable impact on health. If we had a chance to deliberate around the results, this difference could have been brought to the attention of the group or conference leaders, clarification could have been sought, and this issue could have been addressed.

The PHRIA tools have been designed to be accessible for a variety of people, even those with no specific expertise on any of the critical dimensions of Planetary Health. The multiple possibilities for integrating these tools into a design process or evaluation exercise create opportunities for debating more sensitive or complex issues related to intervention impacts and trade-offs, and diverging experiences among groups of participants and beneficiaries. One could imagine facilitating dialogue around lived experience or traditional knowledge using the tools as a starting point for deeper conversations, well beyond conventional approaches. The tools are one aspect of an evaluation project; thinking about how they are integrated into a larger process will open windows for engagement or debate around more complex questions that would be uncovered by using the tools. In addition, these tools could supplement or form part of larger processes. They could also be used at the level of organizations and governments in support of achieving their socio-ecological transition commitments.

Conclusion

This article aims to encourage policymakers, program designers, decision-makers, administrators, as well as evaluators to consider the whole range of impacts on Planetary Health, which includes the embedded relationships between natural and human systems. It advocates democratizing the use of Planetary Health tools and engaging with a diversity of people(s) to develop practical tools adapted to their own contexts.

Currently, approaches for impact evaluations of health and environmental interventions exist but they are typically demanding processes that can’t be applied to all kinds of project evaluations. They are generally used to assess major projects or specific industrial activities. With this article, we hope to show that developing easy-to-use tools, pertinent to different contexts, while addressing what matters most for flourishing ecosystems, is feasible, whatever the area of expertise and the level of knowledge one has. In the midst of embedded crises, we need engagement toward real transformation. Transformative practices can take place at all levels, in everything we do, in our personal and professional lives.