Beyond the Cone of Uncertainty: Exploring the Flexibility of Animated Risk Trajectories for Improving Hurricane Risk Communication

Abstract

Tropical cyclone storms can put lives at risk and produce extensive damage. In the USA, forecast paths of hurricanes are most commonly communicated to the public through the cone of uncertainty (CoU). This visualization has several recognized flaws, including being susceptible to the psychological bias known as the containment effect. The present work tested the CoU against a dynamic set of ensembles, referred to as Animated Risk Trajectories (ARTs). In the current study, ARTs depicted the possible hurricane trajectory using instances moving at slow, medium, or fast speeds. Participants viewed either the CoU or ARTs visualization and made judgments of risk. Overall, the ARTs consistently mitigated the containment effect compared to the CoU. However, variations in the speed of the ARTs did not influence perceived levels of risk. Further examining which visual features of ARTs, like color or size, can be used to convey additional storm information is important for developing more effective and nuanced ways to enhance public safety and response to hurricane-prone regions.

Introduction

Hurricanes are powerful storms that can have serious consequences, including loss of life and extensive, costly damage. For example, Hurricane Ian struck Florida in 2022 and caused widespread devastation, resulting in over 156 deaths and an estimated $112.9 billion in damages (National Oceanic and Atmospheric Administration [NOAA], 2024). This risk highlights the need for effective and accurate visualizations to guide decision-making during natural disasters. People who live along the potential path must prepare for evacuations but must do so based on inherently uncertain projections about the future trajectory of the storm. Evacuation decisions are influenced by visualizations that convey the spatial uncertainty of the storm’s path (Verma et al., 2022). However, research shows that people inherently struggle to understand spatial uncertainty (Wickens et al., 2020). Hurricane forecasts are often communicated to the public (see Millet et al., 2020) through a visualization known as the cone of uncertainty (CoU).

The CoU consists of a single midline conveying the most likely forecast path the storm might take. The width of the cone represents the forecasted range of possible storm paths (i.e., the uncertainty in the path) based on the National Hurricane Center’s (NHC) historical forecast errors over the past 5 years (NHC, n.d.). While the CoU is a common visualization used to communicate hurricanes, it is not without its limitations. For example, people often misinterpret the boundary of the CoU as representing areas at high risk within the boundary and low risk or safe outside of the boundary (Millet et al., 2020; Padilla et al., 2018). This is referred to as the containment effect, a psychological bias for which people fail to weigh peripheral locations as being at risk because they do not fall within the visual boundaries of the cone (Broad et al., 2007; Cox et al., 2013). Another aspect of the CoU that people misunderstand concerns the width of the cone. Prior work shows that some people misinterpret the width of the CoU as indicating the storm is growing in size instead of representing the increased uncertainty in location prediction at later time points (Padilla et al., 2018; Ruginski et al., 2016).

Recent work has found that visualizations exploiting properties of the visual system have more flexibility in how and what information can be conveyed with hurricane visualizations (Witt & Clegg, 2022; Witt et al., 2023; Ruginski et al., 2016). The visual system is sensitive to statistical properties of a set of objects (a.k.a. ensembles) through a process referred to as ensemble perception, which is the visual system’s natural ability to accurately and efficiently extract summary statistics, like the mean and variability, from an ensemble (for review, see Whitney & Yamanashi Leib, 2018). Several empirical studies have shown that the visual system can accurately extract the average size (Ariely, 2001), orientation (Dakin & Watt, 1997), spatial position (Alvarez & Oliva, 2008), motion (Watamaniuk et al., 1989), color (Maule & Franklin, 2015), and emotion of a crowd (Haberman et al., 2015). These features may improve hurricane visualizations by conveying additional information, such as the wind speed and storm surge of an approaching storm.

Prior work found that displays exploiting ensembles by conveying each unique track of the hurricane’s path helped reduce the tendency for people to view the storm as growing in severity and causing more damage at later time points (Ruginski et al., 2016). Other work has also shown that a set of dynamic ensembles, referred to as animated risk trajectories (ARTs), conveying the possible paths of a hurricane eliminated the containment effect that occurs with the CoU, (Witt et al., 2020; Witt & Clegg, 2022). Overall, research on ARTs suggests several advantages and increased flexible at conveying risk beyond the predicted central storm track. Specifically, increasing the number of icons in the ARTs display effectively conveyed higher risk whereas fewer icons conveyed lower risk (Witt et al., 2023).

The current research addresses the primary question of whether different features of the ARTs icons influence people’s perceptions of risk. Recent work has shown that the density of instances over a location, varied through the number of instances present in an ensemble, does influence the sense of risk (Witt et al., 2023). This raises the issue of whether other properties of the visualizations might impact perceptions of risk. More specifically, does the speed of the dynamic set of ensembles (ARTs) influence evacuation decisions and, therefore, perceptions of risk? For example, faster moving icons might also create a reduced sense of density at specific locations. Initial empirical evaluations of the ARTs (e.g., Witt et al., 2023) have used only a single speed for the instances within the ensemble. A secondary question is whether the ARTs reduce the magnitude of the containment effect compared to the cone of uncertainty as shown in prior work (Witt et al, 2023).

We predicted: (H₁) that the speed of the ARTs would influence evacuation decisions, with faster speeds showing different evacuation rates than slower speeds; and (H₂) that people would be more likely to evacuate towns beyond the cone’s boundaries when viewing the ARTs visualization than the cone of uncertainty visualization, which is consistent with the claim that the ARTs would reduce the containment effect.

Method

Participants

Sixteen participants enrolled in an introductory psychology course at Colorado State University received course credit in exchange for completing the experiment online via Qualtrics. All participants had self-reported normal-or-corrected to normal vision.

Stimuli and Apparatus

All stimuli were created in R (R Core Team, 2019). The maps displayed hypothetical hurricane storm predictions varying by the storm’s path and uncertainty level, with town positions marked relative to the predicted storm’s path toward the Gulf coastline.

The hypothetical storm predictions were generated by combining a set of angles representing the storm’s path (50, 70, 90, 110°) and the spread of the storm based on three levels of uncertainty (10, 20, 30°). The town was displayed along the coastline border with a red circle at 1 of 13 possible locations. The town positions ranged from −2 to 2 times the storm uncertainty level and were in units of standard deviation (SD), with zero representing the center of the predicted storm path. Towns located at the farthest locations on the coastline were two times the storm’s SD.

All storm predictions were displayed using two different types of visualizations: the cone of uncertainty (CoU) and a dynamic set of ensembles referred to as animated risk trajectories (ARTs). See Figure 1 for an example of each visualization condition.

Figure 1.

An illustration of a cone trial (top) and ARTs trial (bottom). For each trial, only one type of visualization was displayed. The ARTs visualization was an animated GIF that continued to loop until participants responded. The red dot indicates the location for which a judgment was made.

The CoU represents the possible range of hurricane outcomes in the forecast. The width of the cone represents the uncertainty in the storm’s path (i.e., a wider cone suggests greater uncertainty in the storm’s path). The central line within the cone represents the central storm’s most likely path, which is oriented to match the storm’s predicted angle.

The ARTs display was an animated GIF consisting of 50 small squares that moved along a linear path and at slow, medium, or fast speeds toward the coastline from the center of the bottom of the display. In slow ARTs each instance in the ensemble moved at a speed of 25 pixels per second (s), for a total movement time of 1.2 s per cycle; medium ARTS moved at 50 pixels per second, for a total movement time of 0.6 s; and fast ARTs at 100 pixels per second, for a total movement time of 0.3 s. Once the ARTs completed their path, the animation reset and replayed until the participant made their response.

The path of the ARTs varied based on the uncertainty in the storm’s possible paths. The path of each ART was determined through random sampling of a normal distribution with a mean equal to the storm’s path and a SD equal to the storm’s SD. Most of the ARTs were within 1 SD but could be beyond 1 SD.

Design

The experiment was a within-subjects design where participants completed one block for each visualization condition. Visualization conditions were counterbalanced and consisted of 144 trials each for a total of 288 trials. Within each visualization condition, the angle of the storm’s path, the spread of the storm, and the town’s position were randomized. For the ARTs, the speed of the icons was randomized. The entire experiment was approximately 1 hr.

Procedure

Participants followed self-paced instructions for either the cone or ARTs condition. The instructions for the CoU visualization read:

“Imagine it’s hurricane season and you are in charge of deciding whether to evacuate a town based on the predicted hurricane path. The town will be marked with a red circle. If you choose not to evacuate the town and a hurricane hits, damage will be extensive and costly. If you choose to evacuate the town and the hurricane does not hit there, money will be spent on the evacuation for nothing. Thus, there are benefits and costs to evacuating the town. Towns must be evacuated 12 hours in advance of when the hurricane will hit. For each decision, a hurricane is hovering and is approximately 12 hours away, so it will be time to make your decision. You will see a cone that shows the predictions of the hurricane’s path. The cone shows the probable path of the storm center but does not show the size of the storm. Hazardous conditions can occur outside of the cone.”

The instructions for the ARTs visualization were identical except that the italicized text read: “You will see several predicted hurricane paths, each presented as a black dot. These dots show the probable path of the storm center. Hazardous conditions can occur outside of these paths.” Participants responded by selecting the ratio button to indicate either “yes” or “no.”

Results

The evacuation rates were analyzed with a logistic mixed model. The dependent measure was evacuation decision coded as 1 when they chose to evacuate and 0 if they chose to note evacuate. Evacuation rates were used as a measure of risk perception, where higher evacuation rates correspond to higher levels of perceived risk and lower evacuation rates correspond to lower perceptions of risk. The fixed effects were town angle (i.e., town position), visualization condition, and their interaction. The random effects were included for participant, which included intercepts and the main effect for town position. The coefficients from the random effects were checked for outliers, but none were found.

As shown in Figure 2, evacuation rates dropped steeply at the edge of the cone and were more gradual in the ARTs conditions. A steeper drop (i.e., slope) indicates a strict shift in risk perceptions from high to low risk.

Figure 2.

Evacuation rates are plotted as a function of town angle and visualization condition. Town position “0” indicates the center of the storm. The red curve corresponds to the cone of uncertainty (CoU). The blue curves correspond to the various ARTs conditions representing slow, medium, and fast speeds. Curves are 1 standard error of the mean (SEM).

The primary research question is whether evacuation rates differed depending on the speed of the ARTs. Disconfirming H₁ and, as can be inferred in Figure 2, there were no differences in evacuation rates across the different speeds of the ARTs, ps > .20. This finding suggests that some properties might not matter as much when making evacuation decisions.

Further supporting the finding that the speed of the ARTs is irrelevant, we conducted a model comparison. Model comparison is one method used to support the null hypothesis rather than relying solely on p-values, which cannot support the null hypothesis. We compared the full model to the reduced model. The full model, as described above, included each level of the ART speed along with the cone of uncertainty as fixed effects. The reduced model included the visualization conditions (cone, ARTs), but treated the ART conditions as having only one level, thereby ignoring ART speed. The reduced model fit the data better than the full model, as shown by having a lower Bayesian Information Criterion (BIC). The reduced model had a BIC of 3,967 and the full model had a BIC of 3,991. The difference between the two models was 24. A difference in BIC of this magnitude in BIC is considered strong evidence (Wagenmakers, 2007). Thus, we found strong evidence that ART speed does not impact risk perception.

Supporting H₂ and aligning with prior work (Witt et al., 2023), the steep drop in evacuation rates (see Figure 2) is consistent with the idea of a containment effect indicating a strict transition from high to low risk with the cone but not with the ARTs. As town position moved further from the storm’s predicted path, the likelihood of deciding to evacuate decreased for both the CoU and ART conditions, β = −.146, SE = 0.006, z = −22.865, p < .001. This negative effect of town position on evacuation decisions was less pronounced with ART speeds of 5 (β = .065, SE = 0.010, z = 6.543, p < .001), 10 (β = .077, SE = 0.010, z = 7.942, p < .001, and 20 (β = .060, SE = 0.010, z = 6.123, p < .001) compared to the CoU. Overall, all the ART visualizations mitigate the containment effect more than the CoU.

Discussion

There is a critical need to communicate hurricane forecasts to the public so that people can better understand uncertainty in the projected storm path. Prior literature shows that the CoU leads to many biases, including the containment effect where areas outside the cone’s boundary are viewed as having no storm risk (Millet et al., 2020; Padilla et al., 2018). However, ARTs, which are a visualization technique that exploits properties of the visual system, can mitigate this bias (Witt et al., 2021; Witt et al., 2023).

The current experiment explored the flexibility of what information ARTs can convey by testing whether the visual feature of speed communicates additional information related to storm risk. Specifically, we varied the speed of the ARTs icons to assess whether increasing speeds impact perceptions of risk when making evacuation decisions. We also examined whether hurricane forecasts visualized via ARTs reduce the containment effect often seen with the CoU.

While slow and fast speeds both mitigate the containment effect compared to the COU, evacuation rates across the different ARTs visualizations were similar regardless of speed, disconfirming H₁. This suggests that speed alone does not inherently convey different levels of risk. That speed did not influence evacuation rates could suggest that participants focused on other key aspects of the visualization, such as the paths and distribution of the hurricane track conveyed by the ARTs rather than being distracted by the speed. The lack of variation across the different speeds does show that the ARTs visualization is robust in directing attention to the path of the storm without confusing participants.

Confirming H₂, the current findings show that ARTs significantly mitigate the containment effect compared to the CoU. This finding replicates previous work (Witt & Clegg, 2022; Witt et al., 2023), and shows that this finding generalizes to ARTs icons with various speeds. The more gradual decline in evacuation rates with the ARTs suggests a smoother transition in perceived risk indicating that ARTs may convey uncertainty in the storm’s path better.

As future developments of the visualization occur, distinguishing between features that do (e.g., the number of instances; Witt et al., 2023) and do not (as in the current findings for icon speed) influence perceptions of risk will be vital. Such knowledge allows for both intentional manipulation of features, for example to reduce perceptions of risk to ensure highly uncertain forecasts at long look-ahead times are used for preparation and not unnecessary evacuations; also, to avoid inadvertent consequences, for example, diminishing an intended communication of a high level of risk for an area by incorporating features into the visualization that reduce the sense of risk conveyed.

While these results suggest that manipulations of speed do not inherently impact perceptions of risk within ARTs, future development of visualizations will need to explore other aspects of the speed of the instances. For example, testing with other populations will be vital given that dynamic acuity may tend to degrade with aging (Burg, 1966).

Conclusions & Limitations

In summary, the current results support the finding that ARTs, unlike the CoU, help mitigate the containment effect, leading to better evacuation decisions and, therefore, perceptions of risk. The speed of the ARTs did not effectively convey additional information about the storm (i.e., a storm moving rapidly toward the coastline and, therefore, of higher risk). However, the ARTs visualization is robust when communicating the core information of the hurricane forecast, and can be used in evacuation planning and communication because they improve perceptions of risk compared to the CoU. Further examining which visual features of ARTs, like color or size, can be used to convey additional storm information is important for developing more effective and nuanced ways to enhance public safety and response to hurricane-prone regions.

One limitation of the study is the sample size. However, these effects were quite large, and it is unlikely that the sample size is the cause for the null effect of ART speed, given that the BIC analysis supported that there was no difference across speed. Another potential limitation is that the study was completed online, but people obtain weather forecasts via online distributions, so it is ecological valid and important to test the potential for ARTs to communicate forecasts in this manner.