Meta-Modeling as a Variance-Reduction Technique for Stochastic Model–Based Cost-Effectiveness Analyses

Abstract

Purpose

When using stochastic models for cost-effectiveness analysis (CEA), run-to-run outcome variability arising from model stochasticity can sometimes exceed the change in outcomes resulting from an intervention, especially when individual-level efficacy is small, leading to counterintuitive results. This issue is compounded for probabilistic sensitivity analyses (PSAs), in which stochastic noise can obscure the influence of parameter uncertainty. This study evaluates meta-modeling as a variance-reduction technique to mitigate stochastic noise while preserving parameter uncertainty in PSAs.

Methods

We applied meta-modeling to 2 simulation models: 1) a 4-state Sick-Sicker model and 2) an agent-based HIV transmission model among men who have sex with men (MSM). We conducted a PSA and applied 3 meta-modeling techniques—linear regression, generalized additive models, and artificial neural networks—to reduce stochastic noise. Model performance was assessed using R² and root mean squared error (RMSE) values on a validation dataset. We compared PSA results by examining scatter plots of incremental costs and quality-adjusted life-years (QALYs), cost-effectiveness acceptability curves (CEACs), and the occurrence of unintuitive results, such as interventions appearing to reduce QALYs due to stochastic noise.

Results

In the Sick-Sicker model, stochastic noise increased variance in incremental costs and QALYs. Applying meta-modeling techniques substantially reduced this variance and nearly eliminated unintuitive results, with R² and RMSE values indicating good model fit. In the HIV agent-based model, all 3 meta-models effectively reduced outcome variability while retaining parameter uncertainty, yielding more informative CEACs with higher probabilities of being cost-effective for the optimal strategy.

Conclusions

Meta-modeling effectively reduces stochastic noise in simulation models while maintaining parameter uncertainty in PSA, enhancing the reliability of CEA results without requiring an impractical number of simulations.

Highlights

• When using complex stochastic models for cost-effectiveness analysis (CEA), stochastic noise can overwhelm intervention effects and obscure the impact of parameter uncertainty on CEA outcomes in probabilistic sensitivity analysis (PSA).

• Meta-modeling offers a solution by effectively reducing stochastic noise in complex stochastic simulation models without increasing computational burden, thereby improving the interpretability of PSA results.