A Sequential Calibration Approach to Address Challenges of Repeated Calibration of a COVID-19 Model

Abstract

Background

Mathematical models served a critical role in COVID-19 decision making throughout the pandemic. Model calibration is an essential, but often computationally burdensome, step in model development that provides estimates for difficult-to-measure parameters and establishes an up-to-date modeling platform for scenario analysis. In the evolving COVID-19 pandemic, frequent recalibration was necessary to provide ongoing support to decision makers. In this study, we address the computational challenges of frequent recalibration with a new calibration approach.

Methods

We calibrated and recalibrated an age-stratified dynamic compartmental model of COVID-19 in Minnesota to statewide COVID-19 cumulative mortality and prevalent age-specific hospitalizations from March 22, 2020 through August 20, 2021. This period was divided into 10 calibration periods, reflecting significant changes in policies, messaging, and/or epidemiological conditions in Minnesota. When recalibrating the model from one period to the next, we employed a sequential calibration approach that leveraged calibration results from previous periods and adjusted only parameters most relevant to the calibration target data of the new calibration period to improve computational efficiency. We compared computational burden and performance of the sequential calibration approach to a more traditional calibration method, in which all parameters were readjusted with each recalibration.

Results

Both calibration methods identified parameter sets closely reproducing prevalent hospitalizations and cumulative deaths over time. By the last calibration period, both approaches converged to similar parameter values. However, the sequential calibration approach identified parameter sets that more tightly fit calibration targets and required substantially less computation time than traditional calibration.

Conclusions

Sequential calibration is an efficient approach to maintaining up-to-date models with evolving, time-varying parameters and potentially identifies better-fitting parameter sets than traditional calibration.

Highlights

• This study used a sequential calibration approach, which takes advantage of previous calibration results to reduce the number of parameters to be estimated in each round of calibration, improving computational efficiency and algorithm convergence to best-fitting parameter values.

• Both sequential and traditional calibration approaches were able to identify parameter sets that closely reproduced calibration targets. However, the sequential calibration approach generated parameter sets that yielded tighter fits and was less computationally burdensome.

• Sequential calibration is an efficient approach to maintaining up-to-date models with evolving, time-varying parameters.