Abstract
Objectives:
Exposure notification (EN) supplements traditional contact tracing by using proximity sensors in smartphones to record close contact between persons. This ledger is used to alert persons of potential SARS-CoV-2 exposure, so they can quarantine until their infection status is determined. We describe a model that estimates the impact of EN implementation on reducing the spread of SARS-CoV-2 and on the workload of public health officials, in combination with other key public health interventions such as traditional contact tracing, face mask wearing, and testing.
Methods:
We created an agent-based model, Simulated Automated Exposure Notification (SimAEN), to explore the effectiveness of EN to slow the spread of SARS-CoV-2. We varied selected simulation variables, such as population adoption of EN and EN detector sensitivity configurations, to illustrate the potential effects of EN. We executed 20 simulations with SimAEN for each scenario and derived results for each simulation.
Results:
When more sensitive versus more specific EN configurations were compared, the effective reproductive number, RE, was minimally affected (a decrease <0.03). For scenarios with increasing levels of EN adoption, an increasing number of additional infected persons were identified through EN, and total infection counts in the simulated population decreased; RE values for this scenario decreased with increasing EN adoption (a decrease of 0.1 to 0.2 depending on the scenario).
Conclusions:
Estimates from SimAEN can help public health officials determine which levels of EN adoption in combination with other public health interventions can maximize prevention of COVID-19 while minimizing unnecessary quarantine in their jurisdiction.
Traditional contact tracing methods are hindered by incomplete recall of contacts and delays in alerting identified contacts.1,2 During the COVID-19 pandemic, exposure notification (EN) technology augmented traditional contact tracing by leveraging proximity sensors on smartphones to notify recent close contacts of persons who later received a positive test result for COVID-19. 3 EN represented a new era of advanced technologies that emerged in the context of a global pandemic as researchers and engineers worked to address gaps in public health infrastructure. EN’s purpose was to provide prompt anonymous notification to potentially exposed persons, so they could quarantine until their infection status was determined. 4
However, EN’s effectiveness is difficult to evaluate because of its privacy-preserving design: it cannot link personal information or unique identifiers to proximity (ie, presumed exposure) events. Contacts themselves and even public health authorities in that jurisdiction cannot ascertain which new COVID-19 case triggered a notification. Effectiveness depends on how many persons chose to enroll in EN, the EN configuration settings chosen by public health authorities, and whether receiving a notification prompts contacts to quarantine and seek testing.4,5
We developed an agent-based model, Simulated Automated Exposure Notification (SimAEN), to evaluate the potential impact of EN in real-world–representative scenarios. SimAEN is structured as an open-access decision-support tool for public health jurisdictions. Our main objective was to understand the ability of EN to slow the spread of SARS-CoV-2, while taking into account concurrent traditional contact tracing and other nonpharmaceutical interventions, such as physical distancing and wearing face masks (hereinafter, masks). We examined how SimAEN can estimate the number of additional infections detected through EN alone.
Materials and Methods
Model Structure and Context
SimAEN is an agent-based model, where each agent represents an individual who interacts with a simulated environment according to a set of rules and range of probabilities.6 -8 Agent-based models are able to represent the heterogeneity and stochasticity that are observed in real life (ie, as events and workflows change) but typically have intensive computational demands and long execution times.9 -11 To achieve manageable runtimes, models must prioritize which aspects of reality are crucial to include in detail and which can be modeled more coarsely. The public health workflow used in SimAEN represents a simplified depiction of real-world COVID-19 case interviews and contact tracing (Figure).
Public health workflow shows people followed through close contact, testing, traditional contact tracing (within box), and exposure notification (EN). Simulated Automated Exposure Notification (SimAEN) modeled the real-life queues and time delays shown here with processing queues and delays and modeled yes/no decisions with probabilistic outcomes. SimAEN’s input parameters include the queue capacities, delays, and probabilities of proceeding down each path.
Each step related to testing, results notification, and case investigation depends on the previous step. Model parameters include the probability of a public health telephone call reaching the person, the proportion of a person’s close contacts identified during the case interview, and the likelihood that a person with a positive test result would inform EN of that result, triggering notifications to close contacts, including some who would not have been identified through traditional contact tracing. Progression from each step to the next is guided by a set of probabilities. 12
We created our model in early 2021, when a COVID-19 vaccine was not yet available to the general public. Nonpharmaceutical interventions to control COVID-19 included physical distancing, mask wearing, and traditional case investigation and contact tracing.13,14 The Centers for Disease Control and Prevention defined close contact as >15 minutes of exposure within 6 ft of an infectious case and, at the time, advised close contacts to quarantine for 7-14 days after exposure. 15 The more transmissible Delta and Omicron variants had not yet emerged. 16 We modeled mask wearing as reducing the risk of SARS-CoV-2 transmission by 65% if 1 person was masked and by the square (ie, 42%) if both persons were masked during the close contact encounter. 12
SimAEN uses a probabilistic determination of whether EN detected the close contact, termed the probability of detection, which is analogous to the public health surveillance concept of sensitivity. In addition to appropriate quarantines following correct detection and notification of close contacts, SimAEN models erroneous quarantines caused by inaccurate proximity sensors.12,17 These erroneous quarantines occur when a person has an encounter with an infected person who does not meet the close contact threshold yet is notified about the exposure and advised to quarantine. The false discovery rate (FDR) allows for such misclassification of close contact (ie, specificity <100%). We derived the probability of detection and FDR from in situ experiments with varying EN sensitivity and specificity parameters.
At the beginning of each model run, we set behavioral characteristics of the simulated population (eg, probabilities of mask wearing and quarantine compliance, rates of interaction, EN uptake). When a person becomes infected or receives a notification, personal behaviors could change according to a set of probabilities. Changing other model parameters allows public health officials to explore the predicted effects of various public health intervention scenarios in their community. Our description of the experiments, a full protocol description of the model, and the implementation source code can be found elsewhere.12,18
Population Assumptions
During the typical annual influenza season, at most 10% of the US population becomes infected. 19 Likewise, our early 2021 pre-Omicron model assumed that only a small fraction of the general population would become infected with SARS-CoV-2 during the 30-day simulation period. Another simplifying assumption was that uninfected persons would interact with at most 1 infected person within 14 days. This value was used as a baseline, because data on the frequency of infectious encounters per exposed individual, rather than per index case, were not available in the literature. Therefore, the number of persons modeled (ie, only first-degree contacts) is smaller than in other agent-based models, with a commensurate reduction in the computational demand. The model most similar to SimAEN had a similar abstraction method but did not account for the public health inputs or additional details associated with EN. 20 This assumption could be reprogrammed when running SimAEN for other variants of SARS-CoV-2 or for other respiratory pathogens.
The innovation that set SimAEN apart from other models was its lack of a latent population in the model implementation. Most modeling methods account for the entire population. In compartmental models, this approach takes the form of unreasonable assumptions of uniform mixing, whereas in agent-based models, it manifests as high computational loads as a result of modeling all persons, not just those relevant to the questions being answered.
Agent Interactions and Neighborhoods
SimAEN does not categorize interactions between persons (agents) based on exposure location (eg, office, home) or relationship (eg, stranger, coworker). Rather, we calibrated the default probability values for interaction based on documented EN experiences from the United Kingdom 21 and Switzerland 22 and from consultations with subject matter experts. These values could be modified per jurisdictional needs. 6 As a probabilistic simulation, events occur for each person depending on individually generated parameters. These include infection events during close contact with an infected person and notification events when contacts are notified. Time in SimAEN advances by the day, during which each person receives an updated status (eg, remains uninfected, becomes infected and infectious, changes to recovered and noninfectious), interacts with others, and potentially changes behavior (eg, enters quarantine, seeks testing).
The model creates an initial number of infected persons plus all the uninfected persons with whom they have the ability to interact. We refer to these first-degree relationships as the infected person’s neighborhood. Each day, each infected person interacts with some subset of the neighborhood. If any of these interactions resulted in infection, then a new neighborhood is created for the newly infected person. This process repeats each day of the simulation.
Model Simulations
Our simulations began with 50 000 active COVID-19 cases and ended after 30 days of simulation time (ie, 30 days = 1 model run). We derived the initial active case count from COVID-19 surveillance in early 2021 (ranging from 30 000 to 70 000 new cases reported daily in each US state). 23 Total cases modeled in SimAEN included all persons with infection regardless of whether they were identified by traditional contact tracing, EN, or both. We measured EN’s positive impact as the number of cases identified through EN only and its negative impact as the number of persons erroneously notified through EN (ie, due to the FDR). Each simulation generated mean daily counts of new infections, including cases identified through traditional contact tracing and EN, as well as mean daily counts of noncontacts erroneously advised to quarantine.
EN Effectiveness in the Context of Other Nonpharmaceutical Interventions
We defined 2 public health intervention scenarios, dubbed scenario 1 (low level of intervention) and scenario 2 (medium level of intervention). For both scenarios, we varied mask use and number of available contact tracers for traditional case investigation and contact tracing13,14; all other parameter settings remained fixed.12,24 For scenario 1, we set mask use to 25% of the population with 500 contact tracers in the public health jurisdiction. For scenario 2, we set mask use to 50% with 2000 contact tracers. During early 2021, the actual number of contact tracing staff in US states typically ranged from 30 to 60 per 100 000 persons. 23
Sensitivity Analysis
For each of the 2 intervention scenarios, we explored the effects of 2 EN model parameters: the configuration settings for the smartphone proximity sensors and the degree of EN adoption in the population.
Varying net configuration settings
In November 2020, a pair of template configurations was proposed for the EN’s proximity detector settings, termed the “wide net” (more sensitive) and “narrow net” (more specific). 25 A wide net classifies more exposures as meeting the close contact definition (implicitly, higher FDR resulting in more erroneous quarantines); a narrow net has better specificity but misses some close contacts. Our configuration sensitivity analysis used SimAEN to explore this tradeoff. For this analysis, we set EN adoption at 25% of the population. The net configurations used for this study are based on the version 1 settings. 25
Varying level of EN adoption
The greater the degree of EN adoption in the total population, the better EN can identify close contacts, but the greater the FDR.3,26 To explore this tradeoff, we used SimAEN to model EN adoption by 0%, 25%, 50%, and 75% of the total population. The 0% EN adoption is the baseline with traditional contact tracing still in place. For the EN adoption sensitivity analysis, we used the wide net configuration setting.
The parameter variation data illustrated how SimAEN could be used to explore the effects of EN detector settings and EN adoption levels on SARS-CoV-2 spread in the context of common nonpharmaceutical interventions. Following 20 model runs, we determined the mean and SD of the mean of each run’s daily case counts, as well as the number of noncontacts erroneously notified by EN with the recommendation to quarantine.
Results
Varying Net Configuration Settings
We examined the results of the 2 scenarios simulated with the narrow versus wide net detector setting (Table 1). The total case counts generated by the model were similar with either detector setting. Somewhat fewer persons were erroneously quarantined with the narrow net setting, and somewhat more persons were identified through EN with the wide net setting. For scenario 1, traditional contact tracing identified 2.7% of the total number of infected persons; EN identified an additional 1.3% and 2.1% of infected persons through narrow and wide nets, respectively. For scenario 2, traditional contact tracing identified 3.2% of the total number of infected persons; EN identified an additional 1.5% and 1.9% of infected persons through narrow and wide nets, respectively. The number of erroneous quarantines was similar with either net setting under scenario 2. Regardless of the decreased specificity of the wide net, the effective reproductive number, RE, was minimally affected (a decrease <0.03 when changing from narrow to wide settings).
Abbreviation: EN, exposure notification.
The narrow net is a more specific and less sensitive configuration template for EN’s Bluetooth detector, while the wide net is a less specific and more sensitive configuration template. 25
In scenario 1, face mask use was set to 25% of the population with 500 contact tracers in the public health jurisdiction. In scenario 2, face mask use was set to 50% with 2000 contact tracers. For both scenarios, EN adoption was set at 25% of the population. The mean (SD) of the mean daily case counts from the 20 simulations was calculated to produce each estimate.
Including those not identified by either traditional contact tracing or EN.
Varying Level of EN Adoption
We examined the results of the 2 scenarios simulated with 3 increasing levels of EN adoption, as well as a baseline where no EN was used (Table 2). With 0% EN adoption, average daily case counts continued to grow, reaching 9566 and 3145 at day 30 in scenarios 1 and 2, respectively. Increasing the public health intervention level in scenario 2 decreased the number of newly infected persons, which in turn reduced the number of those who needed to be contacted through traditional contact tracing or EN. EN adoption at 25% had a modest effect on reducing case counts. RE values decreased with increasing EN adoption (a decrease of 0.1 to 0.2 depending on the scenario). In scenario 2, as EN adoption increased to 75%, the cases trended downward, with RE reaching 0.90; however, it also resulted in more erroneous quarantines.
Comparisons of the number of SARS-CoV-2 cases identified and the effective reproduction number, when selecting various levels of EN adoption, in scenarios with low and medium levels of both face mask use and contact tracer staffing a
Abbreviation: EN, exposure notification.
In scenario 1, face mask use was set to 25% of the population with 500 contact tracers in the public health jurisdiction. In scenario 2, face mask use was set to 50% with 2000 contact tracers. For both scenarios, the wide net configuration setting was used. The mean (SD) of the mean daily case counts from the 20 simulations was calculated to produce each estimate.
Including those not identified by either traditional contact tracing or EN.
Baseline with traditional contact tracing in place.
The total number of cases decreased with higher EN adoption rates, which resulted in a lower RE and in the overall reduction of cases identified by EN and traditional contact tracing.
Discussion
SimAEN modeling confirmed that EN effectiveness depends on several factors, including EN detector settings and degree of EN adoption. The more sensitive (wider) the detector setting, the more cases were identified via EN; however, the number of persons unnecessarily instructed to quarantine also increased. EN adoption >50% had a noticeable effect by reducing new infections. Our results suggest that high adoption of EN can be especially effective in slowing the spread of SARS-CoV-2 when combined with other nonpharmaceutical interventions, such as increased mask use. The mask adoption parameter, which has a major influence on effective reproduction numbers (RE) in SimAEN, is an effective preventive intervention.27,28
By modeling both appropriate and erroneous quarantines, SimAEN represents the balance between detection benefits and costs, a trade-off not adequately explored and described in the literature. In addition to factors specific to EN, SimAEN considered the degree to which the community has adopted other nonpharmaceutical interventions. For this article, we focused on mask use and traditional contact tracing, but SimAEN could accommodate other interventions, such as vaccination.12,18 Additional parameters could be incorporated into the model, such as testing delays and time for reduced quarantine periods following negative test results.
SimAEN’s execution speed was achieved by modeling only the subset of persons who are infected and those who might encounter them as close contacts, rather than a fixed general population through which the disease spread. Notably, the nonoverlapping neighborhood assumption (ie, that 1 person encounters no more than 1 infected person during a 14-day period) is likely an underestimate in regions with a high prevalence of a more infectious variant and/or high social interaction plus low intervention and may overestimate the RE and case counts under such conditions. SimAEN was not able to provide estimates on disease prevalence directly, because it did not model a fixed general population. SimAEN did not model physical space to determine potential proximity but, rather, leveraged a calibrated distribution to determine exposure probability. Thus, it was not equipped to explore the impact of different physical locations on disease progression, such as indoor and outdoor spaces and home, work, and transit environments. Limited data were available from deployed EN systems at the time of our study. Because of the privacy-preserving design of EN, only surrogate performance metrics could be extracted—such as active EN users, aggregated notification event data, and case investigation survey information—which did not support a complete assessment of EN or model calibration. 29 Finally, we set the model parameter values prior to the evolution of the Delta and Omicron variants and prior to the wider availability of rapid antigen tests, which would affect both the transmission parameters and the agent behavior probabilities. As new variants of SARS-CoV-2 emerge and new net configurations are adopted, SimAEN will need to be modified to reflect current environments and EN settings.
Practice Implications
As new public health tools such as EN are developed, public health practitioners need mechanisms to understand and evaluate them. Typically, this evaluation occurs through structured experimental trials and real-world data collection, but some tools can be difficult to explore realistically. SimAEN was a way to experiment with EN adoption scenarios to understand which settings could be most effective for various jurisdictions.
SimAEN provided visibility into aspects of EN not normally visible to public health departments. SimAEN’s internal probabilities were calibrated to publicly available data related to disease progression and nonpharmaceutical intervention effectiveness.21,22 However, available data included only factors visible to public health, such as case counts, number of tests (with results), and number of notifications sent. Through the use of SimAEN, public health authorities can also estimate other variables, such as the number of persons potentially infected who choose not to test or quarantine. 12
The complex nature of large-scale public health responses can make it difficult to understand how an intervention such as EN produces its effects. SimAEN showed that using EN as a supplemental intervention in addition to other nonpharmaceutical interventions would help reduce disease transmission but is most effective when adopted at higher levels among the population. In addition, EN carries the cost of potential “false alarms” that could present a social burden that reduces quarantine adherence. The magnitude of this burden depends on the conditions in which EN is used and the severity of the quarantine constraints, which have changed as vaccination rates increased and more infectious variants emerged. Clear communication to the public about how to interpret EN notifications is critical. Many nonpharmaceutical interventions rely on individual choice to be effective in reducing transmission; therefore, considerations about the willingness to participate and take protective actions also directly affect the success of interventions in counteracting the spread of SARS-CoV-2. Although EN is less specific than traditional contact tracing, it is a less resource-intensive complement to traditional public health activities. This article highlights the potential benefits of implementing EN and provides public health officials with a tool to evaluate which EN settings are best for their jurisdiction.
Footnotes
Acknowledgements
The authors are grateful for the early conceptual contributions of Ted Londner, MSc (MIT Lincoln Laboratory), and Charlie Ishikawa, MSPH (Kahuina Consulting, LLC), whose advice was invaluable to the SimAEN project. They were not compensated for their contributions beyond their established salaries and contracts.
Disclaimer
The findings and conclusions of this article are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention. Use of product names does not imply their endorsement. This activity was reviewed by CDC and was conducted consistent with applicable federal law and CDC policy. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the US Air Force.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This material is based on work supported under Air Force contract FA8702-15-D-0001. Distribution statement: approved for public release; distribution is unlimited.
