Microsimulation Estimates of Decision Uncertainty and Value of Information Are Biased but Consistent

Abstract

Purpose

Individual-level state-transition microsimulations (iSTMs) have proliferated for economic evaluations in place of cohort state transition models (cSTMs). Probabilistic economic evaluations quantify decision uncertainty and value of information (VOI). Previous studies show that iSTMs provide unbiased estimates of expected incremental net monetary benefits (EINMB), but statistical properties of iSTM-produced estimates of decision uncertainty and VOI remain uncharacterized.

Methods

We compare iSTM-produced estimates of decision uncertainty and VOI to corresponding cSTMs. For a 2-alternative decision and normally distributed incremental costs and benefits, we derive analytical expressions for the probability of being cost-effective and the expected value of perfect information (EVPI) for cSTMs and iSTMs, accounting for correlations in incremental outcomes at the population and individual levels. We use numerical simulations to illustrate our findings and explore the impact of relaxing normality assumptions or having >2 decision alternatives.

Results

iSTM estimates of decision uncertainty and VOI are biased but asymptotically consistent (i.e., bias approaches 0 as number of microsimulated individuals approaches infinity). Decision uncertainty depends on 1 tail of the INMB distribution (e.g., P[INMB <0]), which depends on estimated variance (larger with iSTMs given first-order noise). While iSTMs overestimate EVPI, their direction of bias for the probability of being cost-effective is ambiguous. Bias is larger when uncertainties in incremental costs and effects are negatively correlated since this increases INMB variance.

Conclusions

iSTMs are useful for probabilistic economic evaluations. While more samples at the population uncertainty level are interchangeable with more microsimulations for estimating EINMB, minimizing iSTM bias in estimating decision uncertainty and VOI depends on sufficient microsimulations. Analysts should account for this when allocating their computational budgets and, at minimum, characterize such bias in their reported results.

Highlights

• Individual-level state-transition microsimulation models (iSTMs) produce biased but consistent estimates of the probability that interventions are cost-effective.

• iSTMs also produce biased but consistent estimates of the expected value of perfect information.

• The biases in these decision uncertainty and value-of-information measures are not reduced by more parameter sets being sampled from their population-level uncertainty distribution but rather by more individuals being microsimulated for each parameter set sampled.

• Analysts using iSTMs to quantify decision uncertainty and value of information should account for these biases when allocating their computational budgets and, at minimum, characterize such bias in their reported results.