Sage Journals: Discover world-class research

Abstract

Background

The expected value of sample information (EVSI) measures the expected benefits that could be obtained by collecting additional data. Estimating EVSI using the traditional nested Monte Carlo method is computationally expensive, but the recently developed Gaussian approximation (GA) approach can efficiently estimate EVSI across different sample sizes. However, the conventional GA may result in biased EVSI estimates if the decision models are highly nonlinear. This bias may lead to suboptimal study designs when GA is used to optimize the value of different studies. Therefore, we extend the conventional GA approach to improve its performance for nonlinear decision models.

Methods

Our method provides accurate EVSI estimates by approximating the conditional expectation of the benefit based on 2 steps. First, a Taylor series approximation is applied to estimate the conditional expectation of the benefit as a function of the conditional moments of the parameters of interest using a spline, which is fitted to the samples of the parameters and the corresponding benefits. Next, the conditional moments of parameters are approximated by the conventional GA and Fisher information. The proposed approach is applied to several data collection exercises involving non-Gaussian parameters and nonlinear decision models. Its performance is compared with the nested Monte Carlo method, the conventional GA approach, and the nonparametric regression-based method for EVSI calculation.

Results

The proposed approach provides accurate EVSI estimates across different sample sizes when the parameters of interest are non-Gaussian and the decision models are nonlinear. The computational cost of the proposed method is similar to that of other novel methods.

Conclusions

The proposed approach can estimate EVSI across sample sizes accurately and efficiently, which may support researchers in determining an economically optimal study design using EVSI.

Highlights

The Gaussian approximation method efficiently estimates the expected value of sample information (EVSI) for clinical trials with varying sample sizes, but it may introduce bias when health economic models have a nonlinear structure.

We introduce the spline-based Taylor series approximation method and combine it with the original Gaussian approximation to correct the nonlinearity-induced bias in EVSI estimation.

Our approach can provide more precise EVSI estimates for complex decision models without sacrificing computational efficiency, which can enhance the resource allocation strategies from the cost-effective perspective.

Keywords

expected value of sample information function approximation health economic evaluation Taylor series approximation value of information

Value-of-information (VoI) analysis involves calculating the economic benefits of reducing uncertainty in a decision model.^1–4 In general, VoI methods use Bayesian decision theory to integrate insights from health economic decision models with data from previous studies. This approach aids in informing decision making regarding future data collection efforts and resource allocation.^1,5 One of the metrics in VoI analysis involves the expected value of sample information (EVSI), which quantifies the expected economic benefits obtained from a specific data collection experiment. EVSI has a high potential to assist in selecting the optimal trial from a health economic standpoint and can guide public resource allocation to optimize future data collection efforts.^6–11

However, computing EVSI has been traditionally challenging due to its conceptual and computational burdens. This is because EVSI requires integrating over the conditional expectation of the net benefits, which is usually estimated through nested simulations.¹² Most simulation models used in practice are complex without the possibility of analytically computing the conditional expectation. Thus, EVSI is often computed numerically by sampling from the distribution of the conditional expectation of the net benefits a large number of times. Traditional estimation methods involve numerical methods such as Markov Chain Monte Carlo (MCMC) and may need considerable time to evaluate.¹² As a result, the computation burden of EVSI is often high, especially when the health economic decision model is complex. This has limited the application of EVSI in practice.^13,14

In recent years, several methods have been proposed to reduce the computational cost of estimating EVSI.^15–22 These methods have been applied in real-world studies.^7,13,23 In this exposition, we will be using the Gaussian approximation (GA) approach, which can easily identify the optimal design as it estimates EVSI for studies with distinct sample sizes at a low computational cost.²² Specifically, the GA approach involves estimating the conditional expectation of net benefit for each intervention by employing a regression model. This model uses the net benefits samples as the response variable and the parameters of a health economic decision model as the predictor variables.^22,24 However, the GA approach may produce biased estimates of EVSI when the net benefit function exhibits a nonlinear structure. This is because the GA approach estimates the conditional expectation of the net benefit using an estimate of the conditional expectation of the model parameters. Such a process may inadequately estimate the conditional expectation of the net benefit when the relationship between the parameters and the net benefit is nonlinear.²²

To improve the EVSI estimation accuracy of GA for nonlinear decision models, we extend the original GA by introducing higher-order correction terms. This is achieved by decomposing the conditional expectation of net benefits into 2 components using Taylor series expansions.^25–27 The first component is estimated using the original GA approach, while the second component is estimated using the fitted model, samples of parameters, and expected Fisher information to adjust for bias resulting from nonlinear net benefit functions.^28–30 The resulting method is named the spline-based Taylor series approximation and Gaussian approximation (TGA). This approach retains its efficiency in approximating EVSI across sample sizes once the prior effective sample sizes (ESSs) are obtained.

We begin this article by formally introducing the definition of EVSI and reviewing the methodology of the GA approach.²² Then, we present our extension to the GA methods by first presenting how to approximate the conditional expectation of the net benefits using Taylor series expansions, which is followed by an introduction to the approximation methods of the elements required for conducting Taylor series approximation. After that, we use stylized examples and the case study model included in the original GA article to demonstrate that our extended GA approach can provide more accurate EVSI estimates for nonlinear decision models.²² We conclude this article with a short discussion.

Methods

The EVSI

Health economic decision models assess the net monetary (or net health) benefits of different interventions to help decision makers choose the optimal decision from $D$ competing alternatives.³¹ Denoting the parameters in this health economic decision model by $θ$ , which includes all parameters used to compute the benefit of each intervention, the population-level net benefit for each decision $d = 1, \dots, D$ is a function of $θ$ and denoted ${NB}_{d} (θ)$ . ${NB}_{d} (θ)$ is derived from a health economic decision model, including the Markov and decision tree model.¹² The current knowledge about model parameters is summarized as a joint probabilistic distribution $p (θ)$ , which then induces uncertainty in the net benefit.ⁱ Note that $p (θ)$ can also be considered as the prior distribution of $θ$ in Bayesian statistics. The statistical uncertainty in the net benefits and model parameters is usually explored through a set of simulations with $M$ samples of $θ$ and the corresponding samples of ${NB}_{d} (θ), d = 1, \dots, D$ . This dataset is known as the probabilistic analysis (PA) dataset.

Using the information contained in the health economic decision model, the EVSI is defined as the difference between the expected net benefit provided by the optimal decision that is made after an additional dataset is collected, versus the expected benefit given by the optimal decision based on prior knowledge. Using our prior knowledge about $θ$ , the expected benefit given by the optimal decision is

max_{d} E_{θ} {{NB}_{d} (θ)} .

(1)

For a study with a sample size n, we plan to collect a dataset $X_{n} = (X_{1}, \dots, X_{n})$ to inform the parameters $θ$ . $X_{n}$ can be informative for all the model parameters $θ$ or only informative for a subset of $θ$ , $ϕ$ . If $X_{n}$ had been collected, we would then compute the conditional distribution of the parameters $p (θ | X_{n})$ , which would update the distribution of the net benefits and may change the optimal decision. The expected benefit of the optimal decision based on the updated parameter distribution is defined as

max_{d} E_{θ} [{NB}_{d} (θ) | X_{n}] .

As these data have not been collected, EVSI is defined by averaging over all potential dataset. The probabilistic distribution of potential dataset $X_{n}$ is determined by both the likelihood function $p (X_{n} | θ)$ and the prior distribution $p (θ)$ :

p (X_{n}) = \int_{θ} p (θ) p (X_{n} | θ) d θ .

(2)

We then average over the randomness of $X_{n}$ to assess the total expected benefit that can be obtained by collecting an additional dataset:

E_{X_{n}} [max_{d} E_{θ} [{NB}_{d} (θ) | X_{n}]] .

(3)

EVSI for a study design with a sample size n is defined as the difference between terms 3 and 1:

\begin{matrix} EVSI (n) = E_{X_{n}} [max_{d} E_{θ} [{NB}_{d} (θ) | X_{n}]] \\ - max_{d} E_{θ} {{NB}_{d} (θ)} \end{matrix}

(4)

\begin{matrix} = E_{X_{n}} [max_{d} E_{θ} [{NB}_{d} (θ) | X_{n}]] \\ - max_{d} E_{X_{n}} [E_{θ} [{NB}_{d} (θ) | X_{n}]], \end{matrix}

(5)

where equations 4 and 5 are equivalent because of the law of total expectation.¹⁷ Equation 5 is more commonly used in numerical approximation of EVSI since it can reduce the uncertainty introduced by Monte Carlo sampling.¹⁷

The traditional method for calculating the first component in equation 5 is Monte Carlo sampling, which requires a nested 2-stage process: first, we need to simulate a large number of samples of datasets $X_{n}$ using equation 2. Then, for each dataset, we generate the conditional distribution $p (θ | X_{n})$ and compute the maximized averaged net benefit $max_{d} E_{θ} [{NB}_{d} (θ) | X_{n}]$ .³² After that, we average $max_{d} E_{θ} [{NB}_{d} (θ) | X_{n}]$ over the samples of simulated datasets $X_{n}$ to approximate the outer expectation.¹² Since $p (θ | X_{n})$ does not always have a closed-form solution, an MCMC method may be required to approximate $p (θ | X_{n})$ . Moreover, due to the complexity of health economic decision models, after we obtain the samples of $θ$ from $p (θ | X_{n})$ , the evaluation of ${NB}_{d} (θ)$ may also take a long computational time.¹² These computational challenges motivate the development of more efficient EVSI calculation methods, such as the original GA approach.^5,13,22

Approximating the Conditional Expectation of a Prior Parameter Using the GA Approach

This section reviews the original GA approach. Assume we are aiming to approximate $E_{ϕ} [ϕ | X_{n}]$ , where $ϕ$ represents a univariate input parameter of our health economic decision model and $ϕ \in θ$ . The prior of $ϕ$ is assumed to be Gaussian distributed with mean $μ_{0}$ and variance $\frac{σ^{2}}{n_{0}}$ :

ϕ ~ N (μ_{0}, \frac{σ^{2}}{n_{0}}),

(6)

where $n_{0}$ is called the prior ESS representing the amount of information contained in $ϕ$ and $σ^{2}$ is the individual-level variance of the proposed data collection. $n_{0}$ can be estimated by the methods proposed in the original GA paper or the nonparametric regression-based method recently proposed by Li et al.^22,33

Moreover, we denote the sample mean of the simulated dataset with sample size $n$ by ${\bar{X}}_{n}$ . Assuming that, given $ϕ$ , each observation in the simulated dataset is also Gaussian distributed with mean $ϕ$ and variance $σ^{2}$ , the sample mean ${\bar{X}}_{n}$ given $ϕ$ is also Gaussian such thatⁱⁱ

{\bar{X}}_{n} | ϕ ~ N (ϕ, \frac{σ^{2}}{n}),

(7)

Since both $ϕ$ and ${\bar{X}}_{n} | ϕ$ are Gaussian, we can integrate out $ϕ$ and the marginal distribution of ${\bar{X}}_{n}$ is still Gaussian:

{\bar{X}}_{n} ~ N (μ_{0}, \frac{σ^{2}}{v n_{0}}), v = \frac{n}{n_{0} + n}

(8)

Because the prior distribution and likelihood function are conjugate, the conditional mean of $ϕ$ given the simulated dataset $X_{n}$ is a weighted sum between the sample mean and prior mean:

E_{ϕ} [ϕ | X_{n}] = (1 - v) μ_{0} + v {\bar{X}}_{n},

(9)

where $E_{ϕ} [ϕ | X_{n}]$ is a linear transformation of a Gaussian random variable ${\bar{X}}_{n}$ and thus follows a Gaussian distribution. After simplification, the marginal distribution of $E_{ϕ} [ϕ | X_{n}]$ becomes

E_{ϕ} [ϕ | X_{n}] ~ N (μ_{0}, \frac{v σ^{2}}{n_{0}}) .

(10)

Since the prior distribution of $ϕ$ is Gaussian with mean $μ_{0}$ and variance $\frac{σ^{2}}{n_{0}}$ , we can also rescale the samples of $ϕ$ drawn from the prior $p (ϕ)$ to obtain a Gaussian random variable with the same mean and variance as the distribution of $E_{ϕ} [ϕ | X_{n}]$ . To be specific, for $ϕ$ with the prior distribution $N (μ_{0}, \frac{σ^{2}}{n_{0}})$ , we have

(\sqrt{\frac{n}{n_{0} + n}} (ϕ - μ_{0}) + μ_{0}) ~ N (μ_{0}, \frac{v σ^{2}}{n_{0}}) .

(11)

Following equation 11, we can construct the distribution of $E_{ϕ} [ϕ | X_{n}]$ by linearly transforming the samples drawn from the prior distribution $p (ϕ)$ . Let us denote $M$ samples of $ϕ$ as $ϕ^{i} ~ p (ϕ), i = 1, \dots, M$ ; the distribution of $E_{ϕ} [ϕ | X_{n}]$ can be approximated by samples of $μ_{X}$ , which are defined as

μ_{X}^{i} = \sqrt{v} ϕ^{i} + (1 - \sqrt{v}) μ_{0},

(12)

Since the distribution in health economic evaluation often aims to reflect uncertainty, the shape of the distribution of $ϕ$ tends to be unimodal and concentrated around a specific value. In addition, as the sample size of the dataset increases, the Central Limit Theorem suggests that the likelihood function of ${\bar{X}}_{n}$ will approximate a Gaussian distribution. Therefore, approximating both the prior distribution and likelihood function by Gaussian distributions and updating the conditional expectation of $ϕ$ by rescaling the samples drawn from $p (ϕ)$ using equation 12 is reasonable, which makes the GA approach robust even in non-Gaussian settings.³⁴ For more discussion about the robustness of the GA approach, interested readers can refer to the original GA article by Jalal and Alarid-Escudero²² as well as the EVSI comparison article by Kunst et al.⁷ for details.

Lastly, Jalal and Alarid-Escudero suggest that $E_{θ} [{NB}_{d} (θ) | X_{n}]$ , required for EVSI estimation, can be estimated by directly plugging the $E_{ϕ} [ϕ | X_{n}]$ approximated by the GA into a fitted regression model. This method is named as the “linear meta-model” in the original GA approach. However, since the original GA approach uses only the expectation (first moment) of $ϕ$ to approximate $E_{θ} [{NB}_{d} (θ) | X_{n}]$ , the estimated $E_{θ} [{NB}_{d} (θ) | X_{n}]$ will be biased if the net benefit function is nonlinear, which will affect the accuracy of the estimated EVSI. In the next section, we will discuss how to provide more accurate estimates of $E_{θ} [{NB}_{d} (θ) | X_{n}]$ using the variance (second moment) of $ϕ$ .

Taylor Series Expansions for the Conditional Expectation of Net Benefit

Taylor series expansions can provide a more accurate estimation for $E_{θ} [{NB}_{d} (θ) | X_{n}]$ by introducing the higher-order correction terms instead of only using the first-order term of $ϕ$ as in the original GA.³⁵ To demonstrate the use of Taylor series expansions, we begin by rewriting $E_{θ} [{NB}_{d} (θ) | X_{n}]$ using the law of total expectation:

\begin{matrix} E_{θ} [{NB}_{d} (θ) | X_{n}] = E_{ϕ} [E_{θ} [{NB}_{d} (θ) | ϕ] | X_{n}] \\ = E_{ϕ} [g_{d} (ϕ) | X_{n}], \end{matrix}

(13)

where $E_{θ} [{NB}_{d} (θ) | ϕ]$ can be viewed as a function of $ϕ$ and denoted $g_{d} (\cdot)$ .

To approximate $E_{θ} [{NB}_{d} (θ) | X_{n}]$ using $E_{ϕ} [ϕ | X_{n}]$ , we can apply second-order Taylor series expansions to reexpress $E_{ϕ} [g_{d} (ϕ) | X_{n}]$ as a function of the conditional net benefit function $g_{d} (\cdot)$ and the conditional moments of $ϕ$ ²⁵:

\begin{matrix} E_{ϕ} [g_{d} (ϕ) | X_{n}] \approx g_{d} (E_{ϕ} [ϕ | X_{n}]) \\ + \frac{{Var}_{ϕ} [ϕ | X_{n}]}{2} g_{d ″} (E_{ϕ} [ϕ | X_{n}]), \end{matrix}

(14)

where $g_{d ″} (\cdot)$ denotes the second-order derivative of $g_{d} (\cdot)$ with respect to the input value.

Following equations 13 and 14, the estimation of $E_{θ} [{NB}_{d} (θ) | X_{n}]$ can be decomposed into the estimation of $E_{ϕ} [ϕ | X_{n}]$ , ${Var}_{ϕ} [ϕ | X_{n}]$ , $g_{d} (\cdot)$ and its second-order derivative $g_{d ″} (\cdot)$ . In the previous section, we have demonstrated that $E_{ϕ} [ϕ | X_{n}]$ can be estimated using the conventional GA approach. The estimation of ${Var}_{ϕ} [ϕ | X_{n}]$ , $g_{d} (\cdot)$ , and $g_{d ″} (\cdot)$ will be elaborated in the next 2 sections. After obtaining all of the approximated quantities, they can be combined using equation 14 to produce $E_{θ} [{NB}_{d} (θ) | X_{n}]$ for EVSI estimation.

Although equation 14 uses a second-order Taylor series approximation for estimating $E_{θ} [{NB}_{d} (θ) | X_{n}]$ , it is possible to use a higher-order Taylor series approximation to include additional correction terms of $ϕ$ for approximating $E_{θ} [{NB}_{d} (θ) | X_{n}]$ . However, we do not recommend this approach for 2 reasons. First, the second-order Taylor series approximation is generally sufficient for most scenarios because the influence of higher-order terms diminishes quickly as the dataset’s sample size increases. Therefore, including more correction terms may not significantly improve the accuracy of the approximation.²⁵ Second, we plan to use spline regression models in the upcoming sections to approximate the conditional net benefit function $g_{d} (\cdot)$ in equation 14. Undertaking a higher-order Taylor series expansion would necessitate precise estimations of the spline’s higher-order derivative and the higher-order conditional moments of $ϕ$ , which are challenging to acquire in a practical setting.^36,37 Therefore, we believe that applying the second-order Taylor series approximation strikes the best balance between ease of implementation and accuracy.

Finally, note that this article limits the parameter $ϕ$ to be univariate for ease of explanation, but our methodology can be generalized to the multivariate case.³⁸ The estimation method for $E_{θ} [{NB}_{d} (θ) | X_{n}]$ when $ϕ$ is multivariate is introduced in the appendix.

Approximating Conditional Variance of a Prior Parameter Using Expected Fisher Information

In this section, we introduce the methods that can be used to estimate the conditional variance ${Var}_{ϕ} [ϕ | X_{n}]$ . One way to estimate ${Var}_{ϕ} [ϕ | X_{n}]$ is using the numerical integration method such as MCMC. Nevertheless, because estimating EVSI requires us to estimate the conditional variance for a large number of simulated datasets, this means we need to repeat MCMC many times, and that will result in a high computational cost.

Alternatively, if the Gaussian assumption of $ϕ$ and $X_{n}$ is satisfied, we can show that the conditional variance is a fixed constant and independent of the dataset, which implies ${Var}_{ϕ} [ϕ | X_{n}] = E_{X_{n}} [{Var}_{ϕ} [ϕ | X_{n}]]$ . Then, using the iterated law of total variance, the conditional variance is equal to

\begin{matrix} {Var}_{ϕ} [ϕ | X_{n}] = E_{X_{n}} [{Var}_{ϕ} [ϕ | X_{n}]] \\ = {Var}_{ϕ} [ϕ] - {Var}_{X_{n}} [E_{ϕ} [ϕ | X_{n}]] \\ = \frac{v σ^{2}}{n_{0}} . \end{matrix}

(15)

However, since the Gaussian assumption of $ϕ$ is often not strictly satisfied, directly using $\frac{v σ^{2}}{n_{0}}$ to approximate the conditional variance may not be always accurate. Therefore, to balance the computational efficiency and estimation accuracy, we suggest using the expected Fisher information function to estimate ${Var}_{ϕ} [ϕ | X_{n}]$ .²⁸

The expected Fisher information is a crucial concept in statistical estimation theory. Utilizing asymptotic theory, it approximates ${Var}_{ϕ} [ϕ | X_{n}]$ based on the estimates of $ϕ$ obtained from the dataset. This information quantifies the amount of knowledge contained in the collected dataset, empowering us to draw meaningful inferences and make informed decisions based on the available information.

The expected Fisher information function is defined as the expectation of the second-order derivative of the log-likelihood function of $ϕ$ over the dataset $X_{n}$ .²⁹ When samples included in the dataset with the sample size $n$ are independent and identically distributed, the expected Fisher information function can be written as

I_{n} (ϕ) = - n E_{X} [\frac{d^{2}}{d ϕ^{2}} \log (f (X | ϕ))] .

(16)

The functional forms of $I_{n} (\cdot)$ for commonly used likelihood functions are included in the appendix. The functional form of $I_{n} (\cdot)$ can also be approximated based on the numerical methods implemented in the R package mle.tools, but that will typically require more computational time.³⁹

Using the asymptotic properties of the conditional distribution of $ϕ$ , we can plug the conditional expectation $E_{ϕ} [ϕ | X_{n}]$ into the reciprocal of the expected Fisher information function to approximate the corresponding conditional variance ${Var}_{ϕ} [ϕ | X_{n}]$ by $I_{n}^{- 1} (E_{ϕ} [ϕ | X_{n}])$ . Since in equation 12, we have shown that the marginal distribution of $E_{ϕ} [ϕ | X_{n}]$ can be approximated by the samples $μ_{X}^{1}, \dots, μ_{X}^{M}$ based on the conventional GA, it means that we can evaluate $I_{n}^{- 1} (μ_{X}^{i}), i = 1, \dots, M$ and take $I_{n}^{- 1} (μ_{X}^{i})$ as the conditional variance estimate associated with $μ_{X}^{i}$ .²⁸

Lastly, since the asymptotic conditional variance approximated by the expected Fisher information is usually greater than the true conditional variance,⁴⁰ we can further adjust the conditional variance provided by the expected Fisher information using the law of total variance. Because equation 15 suggests that the average of the conditional variance should be equal to $\frac{v σ^{2}}{n_{0}}$ , while $\frac{1}{M} \sum_{i = 1}^{M} I_{n}^{- 1} (μ_{X}^{i})$ is not always equal to that, we can additionally multiply each of $I_{n}^{- 1} (μ_{X}^{i})$ by a constant $C$ to make the averaged conditional variance of $ϕ$ equal to $\frac{v σ^{2}}{n_{0}}$ . Such $C$ can be solved by evaluating

C = \frac{(\frac{v σ^{2}}{n_{0}})}{(\frac{1}{M} \sum_{i = 1}^{M} I_{n} (μ_{X}^{i}))} .

(17)

After adjustments, the conditional variance associated with $μ_{X}^{i}, i = 1, \dots, M$ becomes

C I_{n} (μ_{X}^{i}), i = 1, \dots, M .

(18)

Approximating Conditional Expectation of Net Benefit Using Splines

This section introduces how to approximate the functional form of $E_{θ} [{NB}_{d} (θ) | ϕ]$ using splines and the PA dataset. The marginal distribution of $E_{θ} [{NB}_{d} (θ) | X_{n}]$ can then be approximated based on the estimated $E_{ϕ} [ϕ | X_{n}]$ , ${Var}_{ϕ} [ϕ | X_{n}]$ and the fitted spline.

Splines are a type of flexible regression model that can characterize the nonlinear relationship between the responses and predictors using a series of basis functions.^16,37 As Strong et al.¹⁷ introduced, using the PA datasets, we can regress the samples of net benefits given by the decision $d$ on the samples of the parameter of interest, $ϕ$ , to obtain the fitted spline ${\hat{g}}_{d} (ϕ)$ . ${\hat{g}}_{d} (ϕ)$ is an approximation of the function form of $E_{θ} [{NB}_{d} (θ) | ϕ]$ and reflects how much economic benefit can be obtained by eliminating all the uncertainty around the parameter of interest. This approach is computationally efficient and can accurately approximate the complex net benefit function, provided a sufficient number of probability analysis samples are available.¹³ For a detailed introduction to the implementation and model diagnostics of using splines to approximate the expected net benefit conditioning on the specific parameters, see Strong et al.⁴¹

After the functional form of $E_{θ} [{NB}_{d} (θ) | ϕ]$ is approximated by the fitted spline ${\hat{g}}_{d} (ϕ)$ , the second-order derivative of $E_{θ} [{NB}_{d} (θ) | ϕ]$ can be naturally approximated by the second-order derivative of the fitted splines, ${\hat{g}}_{d ″} (ϕ)$ . ${\hat{g}}_{d ″} (ϕ)$ is determined by the second-order derivative of the basis functions included in the fitted model and can be efficiently computed using the R package splines2.^27,iii

In equation 14, the conditional expectation of the net benefit can then be approximated using the fitted splines:

\begin{matrix} E_{θ} [{NB}_{d} (θ) | X_{n}] \approx {\hat{g}}_{d} (E_{ϕ} [ϕ | X_{n}]) \\ + \frac{{Var}_{ϕ} [ϕ | X_{n}]}{2} {\hat{g}}_{d ″} (E_{ϕ} [ϕ | X_{n}]) . \end{matrix}

(19)

Moreover, using equations 12 and 18, the marginal distribution of $E_{θ} [{NB}_{d} (θ) | X_{n}]$ can be approximated by the samples of the estimated conditional net benefit, $E_{θ} [{NB}_{d} (θ) | X_{n}^{i}], i = 1, \dots, M$ , which are defined as

\begin{matrix} E_{θ} [{NB}_{d} (θ) | X_{n}^{i}] \approx {\hat{g}}_{d} (μ_{X}^{i}) \\ + \frac{{C I}_{n}^{- 1} (μ_{X}^{i})}{2} {\hat{g}}_{d ″} (μ_{X}^{i}) . \end{matrix}

(20)

EVSI Calculation

Finally, we can repeat the above procedures to approximate the marginal distribution of $E_{θ} [{NB}_{d} (θ) | X_{n}]$ using samples $E_{θ} [{NB}_{d} (θ) | X_{n}^{i}]$ for all decisions $d = 1, \dots, D$ . Using equation 5, EVSI for a study design with the sample size $n$ can be estimated by the samples $E_{θ} [{NB}_{d} (θ) | X_{n}^{i}]$ such that

\begin{matrix} EVSI (n) \approx \frac{1}{M} \sum_{i = 1}^{M} (max_{d} E_{θ} [{NB}_{d} (θ) | X_{n}^{i}]) \\ - max_{d} \frac{1}{M} \sum_{i = 1}^{M} E_{θ} [{NB}_{d} (θ) | X_{n}^{i}] . \end{matrix}

(21)

The algorithm for estimating EVSI for k different sample sizes $n_{1}, \dots, n_{k}$ using spline-based Taylor series expansions and GA is summarized in algorithm 1. Once the prior ESS $n_{0}$ , the fitted splines ${\hat{g}}_{d} (\cdot)$ , and the expected Fisher information function $I_{n} (\cdot)$ are determined, the proposed algorithm can estimate EVSI across different sample sizes with low computational cost. Therefore, the proposed approach is more computationally efficient than nonparametric regression-based approaches and methods based on advanced Monte Carlo methods for identifying optimal study designs that maximize EVSI.^8,13,15–19

Algorithm 1:

Estimating EVSI Using GA and Spline-Based Taylor Series Expansions

EVSI, expected value of sample information; GA, Gaussian approximation; PA, probabilistic analysis.

Simulation Study

Case Study I: Gaussian Parameters and Nonlinear Net Benefit Functions

In the first case study, our augmented GA method, based on splines and Taylor series expansions (TGA), is used to evaluate the EVSI for 4 stylized examples with Gaussian-distributed parameters and nonlinear net benefit functions. We compare EVSI estimates from our TGA method with those from the conventional GA approach and the nonparametric regression-based method to demonstrate its accuracy. In addition, we have derived the analytic solution for the conditional net benefit and computed the EVSI based on this quantity. The EVSI derived from this approach is deemed to be the most accurate and is used as the benchmark in comparison with all 4 methods.

Incremental net benefit function

Our decision problem compares 2 potential interventions. To simplify the calculation, we can derive the incremental net benefit function using the net benefit functions of 2 decision options by subtracting one net benefit function from the other, that is, $INB (θ) = {NB}_{1} (θ) - {NB}_{2} (θ)$ . We test the robustness of TGA by specifying different functional forms for $INB (θ)$ in 4 hypothetical scenarios, including both linear and nonlinear scenarios. The incremental net benefit functions are summarized in Table 1.

Table 1

Incremental Net Benefit Functions for 4 Stylized Studies in Case Study I

Scenario	Incremental Net Benefit Function
1	$INB (θ) = - 100 + 5000 θ$
2	$INB (θ) = - 1000 + 5000 θ^{2}$
3	$INB (θ) = - 500 + 5000 θ^{4}$
4	$INB (θ_{1}, θ_{2}) = - 1500 + 5000 θ_{1}^{2} + 5000 θ_{2}^{4}$

Using the incremental net benefit function, EVSI can be calculated using the conditional expectation of $INB (θ)$ given the simulated dataset^21,iv:

\begin{matrix} EVSI (n) = E_{X_{n}} [max {E_{θ} [INB (θ) | X_{n}], 0}] \\ - max {E_{X_{n}} [E_{θ} [INB (θ) | X_{n}]], 0} . \end{matrix}

(22)

Therefore, we can approximate the distribution of $E_{θ} [INB (θ) | X_{n}]$ and compare it with $0$ to estimate EVSI.

Parameter of interest and dataset generation

For the first 3 scenarios in which the parameter of interest $θ$ is univariate, the prior distribution of $θ$ is set to be Gaussian with mean $μ_{0} = 0$ , $σ^{2} = 1$ , and $n_{0} = 5$ . Consequently, the variance of the Gaussian distribution is $\frac{σ^{2}}{n_{0}} = \frac{1}{5}$ .^v

θ ~ N (0, \frac{1}{5}) .

(23)

The likelihood function of $θ$ is also Gaussian distributed for the first 3 scenarios. $n$ collected data are drawn from a Gaussian likelihood function such that

X_{i} | θ ~ N (θ, 1), i = 1, \dots, n .

(24)

The prior distribution for the fourth bivariate incremental net function is an independent bivariate normal distribution with a mean of 0 and a variance of $\frac{1}{5}$ . The likelihood function of $(θ_{1}, θ_{2})$ is also modeled as an independent bivariate Gaussian distribution with a variance $1$ .^vi To reduce the parameter variability, the evaluation of EVSI for all 4 scenarios is based on the same PA dataset.

Method 1: Analytic method

For each data collection exercise, we draw $M = 10^{5}$ samples from the prior distribution $p (θ)$ and obtain samples $θ^{1}, \dots, θ^{M}$ . Next, we generate a simulated dataset for each sample of $θ$ and achieve $X_{n}^{1}, \dots, X_{n}^{M}$ . Due to the conjugacy, we can compute the analytic solution of the conditional expectation of the incremental net benefit $E_{θ} [INB (θ) | X_{n}^{i}]$ for each simulated dataset. The analytic solutions of $E_{θ} [INB (θ) | X_{n}^{i}]$ for the four case studies are summarized in the Appendix.

Using the simulated dataset with the sample size $n$ and equation 22, we can estimate EVSI for different $INB (θ)$ functions by

\begin{matrix} \hat{EVSI} (n) = \frac{1}{M} \sum_{i = 1}^{M} max {E_{θ} [INB (θ) | X_{n}^{i}], 0} \\ - max {\frac{1}{M} \sum_{i = 1}^{M} E_{θ} [INB (θ) | X_{n}^{i}], 0} . \end{matrix}

(25)

Note that deriving the closed-form solution of the conditional incremental net benefit function is usually unrealistic due to the complexity of the underlying health economics decision model. Therefore, the analytic method is rarely, if ever, applied in practical settings.

Method 2: Nonparametric regression-based method

We generate $M = 10^{5}$ samples of the dataset $X_{n}$ and use the sample mean ${\bar{X}}_{n}$ as the summary statistic for each simulated dataset. ${\bar{X}}_{n}$ is sufficient to summarize all the information in ${\bar{X}}_{n}$ if the data are normally distributed. We regress the incremental net benefit samples on all the samples of ${\bar{X}}_{n}$ using splines to estimate $E_{θ} [INB (θ) | X_{n}]$ . The fitted values are then extracted from the regression models and used to estimate $E_{θ} [INB (θ) | X_{n}]$ .

Method 3: Linear meta-modeling GA

Since the Gaussian assumption is strictly satisfied, we can derive values of prior ESS $n_{0}$ analytically. For the first 3 $INB (θ)$ functions, the prior ESS $n_{0}$ is $5$ . For the fourth incremental net benefit function, both ESSs for $θ = (θ_{1}, θ_{2})$ are $5$ . The linear meta-model used to produce $E_{θ} [INB (θ) | X_{n}]$ estimates is constructed by regressing the $10^{5}$ incremental net benefit samples on the samples of $θ$ drawn from the prior distribution using splines.

Method 4: Spline-based Taylor series GA

The prior ESSs $n_{0}$ of TGA are the same as the conventional GA. Since both the likelihood function and prior distribution are Gaussian, the closed-form solution of ${Var}_{ϕ} [ϕ | X_{n}]$ can be derived. For each $INB (θ)$ , we approximate its function form using a spline that is fitted by $M = 10^{5}$ incremental net benefit and $θ$ samples. We can then generate the samples of $E_{θ} [INB (θ) | X_{n}]$ via equation 20 and use them to approximate EVSI.

Case Study II: Calculating EVSI in a Markov Model

In the second case study, we test the robustness of TGA when the prior and likelihood are both non-Gaussian. We compare the accuracy of EVSI given by the nonparametric regression-based method, conventional GA, and TGA using a Markov model included in Jalal and Alarid-Escudero.²² Four different data collection processes are considered to reduce the uncertainty in this Markov model, and the corresponding EVSI are estimated. EVSI for each of the data collection processes is also computed using the nested Monte Carlo method for comparison.²²

Incremental net benefit function

A Markov model with 3 states (well, disabled, and dead) is used to simulate a group of 30-y-old patients suffering from a genetic disorder. To prevent the disorder from leading to permanent disability, 3 treatment options, labeled $A$ , $B$ , and $C$ , are available, with $C$ serving as the standard treatment. The ineffectiveness of any treatment reduces the patients’ quality of life. For simplification, it is assumed that these patients experience a steady annual mortality rate, with the genetic disorder contributing an additional absolute mortality increase of $0.5 %$ . Should a patient become disabled, their mortality rate is expected to rise by an extra $1 %$ . The model includes $15$ parameters, with 4 being uncertain: the average number of hospital visits required for treatments $A$ and $B$ (represented as $μ_{A}$ and $μ_{B}$ ) and the probability of failure for treatments $A$ and $B$ (represented as $P_{A}$ and $P_{B}$ ). The net benefit’s conditional value increases linearly with the number of hospital visits for treatments $A$ and $B$ , but its relationship with the failure probabilities of these treatments is nonlinear. For a comprehensive explanation of the model, refer to the work by Jalal and Alarid-Escudero.²²

Parameter of interest and dataset generation

For this case study, we consider the 4 different data collection exercises that are included in Jalal and Alarid-Escudero’s work.²² These 4 data collection exercises aim to reduce the uncertainty in $μ_{A}, μ_{B}, P_{A}$ , and $P_{B}$ , respectively. The detailed information of the prior and likelihood for these 4 data collection processes are included in Table 2.

Table 2

Prior Distribution and Likelihood Functions for the Markov Model in Case Study II^a

Scenario	Prior Distribution	Likelihood Function
1	$μ_{A} ~ G a m m a (α = 10, β = 10)$	$X_{i} ~ P o i s s o n (μ_{A}), i = 1, \dots, n$
2	$μ_{B} ~ G a m m a (α = 20, β = 10)$	$X_{i} ~ P o i s s o n (μ_{B}), i = 1, \dots, n$
3	$P_{A} ~ B e t a (α = 2, β = 8)$	$X_{i} ~ B e r n o u l l i (P_{A}), i = 1, \dots, n$
4	$P_{B} ~ B e t a (α = 3, β = 7)$	$X_{i} ~ B e r n o u l l i (P_{B}), i = 1, \dots, n$

$μ_{A}$ and $μ_{B}$ are used to indicate the average number of hospital visits associated with interventions A and B, respective4ly. $P_{A}$ and $P_{B}$ represent the failure probabilities for interventions A and B, respectively.

Method 1: Nested Monte Carlo method

The EVSI estimates from the nested Monte Carlo method are taken directly from Jalal and Alarid-Escudero’s article.²² Computing each EVSI estimate through this method required approximately 6 h of processing time on 16 parallel cores.²²

Method 2: Nonparametric regression-based method

We generate $M = 10^{4}$ samples of the parameter $θ$ from the prior $p (θ)$ , run the Markov model, and compute the incremental net benefit samples. For all 4 data collection exercises, the summary statistics are the sample mean of the simulated dataset with the sample size $n$ , ${\bar{X}}_{n}$ . The increment net benefit samples are then regressed on the samples of ${\bar{X}}_{n}$ using splines.

Method 3: Linear meta-modeling GA

We can derive that $n_{0} = 10$ for all 4 data collection exercises as the likelihood function and prior distributions are conjugate. For each data collection exercise, the linear meta-model used to generate $E_{θ} [INB (θ) | X_{n}]$ estimates is the spline that is fitted by $M = 10^{4}$ incremental net benefit and parameter samples.

Method 4: Spline-based Taylor series GA

Like the conventional GA approach, $n_{0} = 10$ for all data collection exercises. For each data collection study, we approximate the function form of $E_{θ} [{NB}_{t} (θ) | ϕ]$ by regressing the $10^{4}$ incremental net benefit samples on the samples of parameters using splines. The expected Fisher information functions for each scenario are derived analytically.

Results

Case Study I: Gaussian Parameters and Nonlinear Net Benefit Functions

Figure 1 compares the EVSI of 4 stylized net benefit functions with Gaussian distributed parameters computed using the analytic method, the conventional GA approach, TGA, and the nonparametric regression-based method for different sample sizes (between $10$ and $300$ in increments of $10$ ). The expected value of perfect information (EVPPI), which can quantify the economic benefit obtained by removing all the uncertainty around the parameter of interest,⁴¹ is computed by simulations and shown with the horizontal dashed line for each data collection exercise. Given its nature, EVPPI serves as the maximum limit for the EVSI.

Figure 1

The expected value of sample information (EVSI) computed by analytic method (Analytic), conventional Gaussian approximation (GA), spline-based Taylor series expansions and Gaussian approximation (GA and Taylor series approximation), and nonparametric regression-based method (non-nonparametric) for linear and nonlinear incremental net benefit functions with Gaussian distributed parameters. The expected value of partial perfect information (EVPPI) is shown with the horizontal dashed lines. (A) EVSI for Gaussian θ, INB(θ) = −100 + 5,000θ, (B) EVSI for Gaussian θ, INB(θ) = −1,000 + 50,00θ², (C) EVSI for Gaussian θ, INB(θ) = −500 + 50,00θ⁴, and (D) EVSI for Gaussian (θ₁, θ₂), INB(θ₁, θ₂) = −1,500 + $50, 00 θ_{1}^{2} + 50, 00 θ_{2}^{4}$ .

In subplot A of Figure 1, when $INB (θ)$ is univariate linear, all 4 methods can accurately estimate EVSI. However, when $INB (θ)$ is nonlinear, we can observe that GA largely overestimates EVSI. This is despite the Gaussian assumption being satisfied. Also, the nonparametric regression-based method may not accurately capture the correct functional form of $INB (θ)$ when $INB (θ)$ is highly nonlinear; thus, it also overestimates EVSI for the third and fourth $INB (θ)$ functions. Only EVSI curves given by TGA overlapped with the analytic method for all $INB (θ)$ , which suggests that when the Gaussian assumption is strictly satisfied, the TGA method can accurately estimate EVSI even if $INB (θ)$ is highly nonlinear. In addition, the total computation time required by the conventional GA approach, TGA, and the nonparametric regression-based method is $354.85$ s, $16.86$ s, and $101.04$ s, respectively, demonstrating the efficiency of the TGA approach.

Case Study II: Calculating EVSI in a Markov Model

Figure 2 compares the EVSI of the Markov models computed by the conventional GA approach, TGA approach, and nonparametric regression-based method for different sample sizes (between $5$ and $100$ in increments of $5$ ). EVSIs estimated by the nested Monte Carlo method for sample sizes equal to 5, 10, 50, and $100$ are taken from Jalal and Alarid-Escudero²² and denoted by the red cross. The EVPPI estimated by the nonparametric regression-based method proposed by Strong et al.⁴¹ is shown with the horizontal dashed lines for each data collection exercise.

Figure 2

The expected value of sample information (EVSI) computed by conventional Gaussian approximation (GA), spline-based Taylor series expansions and Gaussian approximation (GA and Taylor series approximation), and nonparametric regression-based method (nonparametric) for a Markov model across different sample sizes. EVSI estimated by standard nested Monte Carlo is denoted by the red cross. The expected value of partial perfect information (EVPPI) is shown by the horizontal dashed lines. (A) EVSI for non-Gaussian $μ_{A}$ in a nonlinear Markov model, (B) EVSI for non-Gaussian $μ_{B}$ in a nonlinear Markov model, (C) EVSI for non-Gaussian P_A in a nonlinear Markov model, and (D) EVSI for non-Gaussian P_B in a nonlinear Markov model.

When the relationship between conditional net benefits and parameters is nearly linear (subplots $A$ and $B$ ), the 4 approaches produce similar EVSI estimates. However, if the relationship is nonlinear (subplots $C$ and $D$ ), conventional GA may underestimate or overestimate EVSI compared with the other methods. By contrast, when sample sizes are greater than $30$ , TGA produces EVSI estimates that are similar to those generated by nested Monte Carlo and nonparametric regression-based methods. TGA’s accuracy is lower than that of nonparametric regression-based methods when sample sizes are less than $30$ . This is because when the sample size of the simulated dataset is small, the accuracy of the conditional net benefit estimated by Taylor series expansions, the conditional mean of the parameter estimated by the GA, and the conditional variance of the parameter estimated by the Fisher information decrease. The computation times for the conventional GA approach, TGA, and the nonparametric regression-based method are 30.17 s, 1.86 s, and 1,216.38 s, respectively. The inefficiency of the nonparametric regression-based method in this context stems from the extensive simulation required for the dataset.

Discussion

This article presents a new algorithm, spline-based Taylor series approximation and Gaussian approximation (TGA), for estimating EVSI. In the TGA method, we estimate EVSI by approximating the conditional expectation of net benefits using 2 steps. First, we use Taylor series expansion to approximate the conditional expectation of net benefits through the net benefit function and the conditional mean and variance of parameters. Subsequently, the net benefit function is approximated by the spline fitted to the PA dataset, and the conditional moments of the parameters are approximated by the conventional GA and expected Fisher information.

Strengths and Limitations

The TGA algorithm has several advantages over alternative EVSI estimation methods. First, once the prior ESS is estimated, TGA can estimate EVSI across multiple sample sizes with minimal computational cost. This is more efficient than EVSI estimation algorithms, in which EVSI must be estimated separately for each sample size (i.e., their computational time scales linearly with the number of sample sizes). Methods with linear scaling include the nonparametric regression-based method and other estimation algorithms based on advanced Monte Carlo methods.^15–19,21 In addition, EVSI estimates obtained using TGA are smooth with respect to the sample sizes and convenient for determining study designs that maximize economic benefit through numerical optimization.⁸ Finally, EVSI estimates from TGA are more accurate than conventional GA, especially when the net benefit function is highly nonlinear.^20–22

However, TGA’s efficiency and accuracy may be affected in certain scenarios. First, if the parameters of interest have a high dimension and complex interactions, a spline with a lot of interaction terms may be required to accurately approximate the function form of the conditional net benefit function. As a result, more computational resources are required for computing the second-order derivative of the net benefit samples, and this may reduce the efficiency of TGA. In this case, we can consider implementing the TGA methods using other nonparametric regression methods that are less affected by the “curse of dimensionality” (e.g., artificial neural network) to approximate the functional form of the conditional net benefit function. A future study might examine the efficacy of these nonparametric regression models in estimating EVSI, particularly when the number of the parameters of interest is large.^30,42

Second, although a closed-form solution for the expected Fisher information is typically accessible for most data-generating processes, there are exceptions in complex scenarios. For instance, evaluating the expected Fisher information function becomes problematic in cases like the generalized linear mixed effect model,⁴³ in which the likelihood function may not have a closed-form solution, or when identifying the likelihood function of the data-generating process proves to be challenging. In such scenarios, alternative approaches to estimating EVSI, such as nonparametric regression-based methods or moment matching,^17,20,21 may be more suitable.

Third, while our article demonstrates the proposed method’s accuracy through hypothetical and real-world case studies, this does not guarantee its effectiveness in every complex scenario. Future research should aim to evaluate the method’s adaptability across a broader range of conditions, including more complex prior, likelihood, and net benefit configurations.

Lastly, because the approximation of the conditional variance of the parameters of interest based on the expected Fisher information is more accurate when the sample size of the design is relatively large, EVSI provided by TGA is less accurate when the sample size of the design is relatively small. The nonparametric regression-based method may be preferred over TGA in that scenario. Alternatively, future research should consider investigating more precise methods to quantify the uncertainty of EVSI estimates given by TGA, especially for small sample sizes, possibly through Bayesian bootstrap or Taylor series approximation techniques.^25,44

Conclusion

We introduced a novel EVSI estimation method that combines Taylor series approximation and GA. As shown by the 2 case studies, the proposed algorithm can efficiently estimate EVSI for multiple sample sizes and is more accurate than conventional GA when the net benefit function is highly nonlinear. We believe that our method could aid in the evaluation and optimization of study designs using EVSI, particularly when the underlying health economic decision model is complex and includes a nonlinear structure.

Supplemental Material

sj-pdf-1-mdm-10.1177_0272989X241264287 – Supplemental material for Accurate EVSI Estimation for Nonlinear Models Using the Gaussian Approximation Method

Supplemental material, sj-pdf-1-mdm-10.1177_0272989X241264287 for Accurate EVSI Estimation for Nonlinear Models Using the Gaussian Approximation Method by Linke Li, Hawre Jalal and Anna Heath in Medical Decision Making

Supplemental Material

sj-pdf-2-mdm-10.1177_0272989X241264287 – Supplemental material for Accurate EVSI Estimation for Nonlinear Models Using the Gaussian Approximation Method

Supplemental material, sj-pdf-2-mdm-10.1177_0272989X241264287 for Accurate EVSI Estimation for Nonlinear Models Using the Gaussian Approximation Method by Linke Li, Hawre Jalal and Anna Heath in Medical Decision Making

Supplemental Material

sj-pdf-3-mdm-10.1177_0272989X241264287 – Supplemental material for Accurate EVSI Estimation for Nonlinear Models Using the Gaussian Approximation Method

Supplemental material, sj-pdf-3-mdm-10.1177_0272989X241264287 for Accurate EVSI Estimation for Nonlinear Models Using the Gaussian Approximation Method by Linke Li, Hawre Jalal and Anna Heath in Medical Decision Making

Supplemental Material

sj-pdf-4-mdm-10.1177_0272989X241264287 – Supplemental material for Accurate EVSI Estimation for Nonlinear Models Using the Gaussian Approximation Method

Supplemental material, sj-pdf-4-mdm-10.1177_0272989X241264287 for Accurate EVSI Estimation for Nonlinear Models Using the Gaussian Approximation Method by Linke Li, Hawre Jalal and Anna Heath in Medical Decision Making

Footnotes

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: LL was funded by the Canadian Statistical Sciences Institute (grant No. Collaborative Research Team 2023) and the Natural Sciences and Engineering Research Council of Canada (grant No. RGPIN-2021-03366)

Authors’ Note

Part of this work was presented at the 44th Annual Meeting of the Society for Medical Decision Making in October 2022.

ORCID iDs

Linke Li

Hawre Jalal

Anna Heath

Notes

References

Schlaifer

Raiffa

. Applied Statistical Decision Theory. New York: Wiley Interscience; 1961.

Howard

. Information value theory. IEEE Trans Syst Sci Cybern. 1966;2(1):22–6.

Pratt

Raiffa

Schlaifer

. Introduction to Statistical Decision Theory. London: MIT Press; 1995.

Briggs

Sculpher

Claxton

. Decision Modelling for Health Economic Evaluation. Oxford (UK): Oxford University Press; 2006.

Jackson

Baio

Heath

, et al. Value of information analysis in models to inform health policy. Annu Rev Stat Appl. 2022;9:95–118.

Willan

Pinto

. The value of information and optimal clinical trial design. Stat Med. 2005;24(12):1791–806.

Kunst

Wilson

Glynn

, et al. Computing the expected value of sample information efficiently: practical guidance and recommendations for four model-based methods. Value Health. 2020;23(6):734–42.

Fairley

Cipriano

Goldhaber-Fiebert

. Optimal allocation of research funds under a budget constraint. Med Decis Making. 2020;40(6):797–814.

Claxton

Sculpher

McCabe

, et al. Probabilistic sensitivity analysis for NICE technology assessment: not an optional extra. Health Econ. 2005;14(4):339–47.

10.

Canadian Agency for Drugs and Technologies in Health. Guidelines for the Economic Evaluation of Health Technologies: Canada. Ottawa: Canadian Agency for Drugs and Technologies in Health; 2006.

11.

Flight

Julious

Brennan

Todd

. Expected value of sample information to guide the design of group sequential clinical trials. Med Decis Making. 2022;42(4):461–73. DOI: 10.1177/0272989X211045036

12.

Ades

Claxton

. Expected value of sample information calculations in medical decision modeling. Med Decis Making. 2004;24(2):207–27.

13.

Heath

Kunst

Jackson

, et al. Calculating the expected value of sample information in practice: considerations from 3 case studies. Med Decis Making. 2020;40(3):314–26.

14.

Steuten

van de Wetering

Groothuis-Oudshoorn

Retèl

. A systematic and critical review of the evolving methods and applications of value of information in academia and practice. Pharmacoeconomics. 2013;31(1):25–48.

15.

Brennan

Kharroubi

. Efficient computation of partial expected value of sample information using Bayesian approximation. J Health Econ. 2007;26(1):122–48.

16.

Kharroubi

Brennan

Strong

. Estimating expected value of sample information for incomplete data models using Bayesian approximation. Med Decis Making. 2011;31(6):839–52.

17.

Strong

Oakley

Brennan

Breeze

. Estimating the expected value of sample information using the probabilistic sensitivity analysis sample: a fast, nonparametric regression-based method. Med Decis Making. 2015;35(5):570–83.

18.

Menzies

. An efficient estimator for the expected value of sample information. Med Decis Making. 2016;36(3):308–20.

19.

Hironaka

Giles

Goda

Thom

. Multilevel Monte Carlo estimation of the expected value of sample information. SIAM/ASA J Uncertain Quantif. 2020;8(3):1236–59.

20.

Heath

Manolopoulou

Baio

. Efficient Monte Carlo estimation of the expected value of sample information using moment matching. Med Decis Making. 2018;38(2):163–73.

21.

Heath

Manolopoulou

Baio

. Estimating the expected value of sample information across different sample sizes using moment matching and nonlinear regression. Med Decis Making. 2019;39(4):347–59.

22.

Jalal

Alarid-Escudero

. A gaussian approximation approach for value of information analysis. Med Decis Making. 2018;38(2):174–88.

23.

Kunst

Alarid-Escudero

Paltiel

Wang

. A value of information analysis of research on the 21-gene assay for breast cancer management. Value Health. 2019;22(10):1102–10.

24.

Doubilet

Begg

Weinstein

Braun

McNeil

. Probabilistic sensitivity analysis using monte carlo simulation. A practical approach. Med Decis Making. 1985;5(2):157–77.

25.

Wolter

. Introduction to Variance Estimation, Vol. 53. Berlin/Heidelberg (Germany): Springer; 2007.

26.

Oehlert

. A note on the delta method. Am Stat. 1992;46(1):27–9.

27.

Wang

Yan

. Shape-restricted regression splines with r package splines2. J Data Sci. 2021;19(3):498–517.

28.

Bernardo

Smith

. Bayesian Theory. Vol. 405. Chichester (UK): John Wiley & Sons; 2009.

29.

van der Vaart

. Asymptotic Statistics, Vol. 3. Cambridge (UK): Cambridge University Press; 2000.

30.

Gelman

Carlin

Stern

Dunson

Vehtari

Rubin

. Bayesian Data Analysis, Third Edition. Texts in Statistical Science Statistical Science. Boca Raton (FL): Chapman & Hall/CRC; 2013. Available from: https://books.google.ca/books?id=ZXL6AQAAQBAJ

31.

Stinnett

Mullahy

. Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Med Decis Making. 1998;18(suppl 2):S68–80.

32.

Heath

Strong

Glynn

Kunst

Welton

Goldhaber-Fiebert

. Simulating study data to support expected value of sample information calculations: a tutorial. Med Decis Making. 2022;42(2):143–55.

33.

Jalal

Heath

. A nonparametric approach for estimating the effective sample size in gaussian approximation of expected value of sample information. arXiv Preprint arXiv:2401.16660. 2024.

34.

Casella

Berger

. Statistical Inference. Noida (India): Cengage Learning; 2021.

35.

Madan

Ades

Price

, et al. Strategies for efficient computation of the expected value of partial perfect information. Med Decis Making. 2014;34(3):327–42.

36.

Hastie

Tibshirani

. Generalized Additive Models. Monographs on Statistics & Applied Probability. Boca Raton (FL): Chapman & Hall/CRC; 1990. Available from: https://books.google.ca/books?id=qa29r1Ze1coC

37.

Wood

. Generalized Additive Models: An Introduction with R. Boca Raton (FL): Chapman and Hall/CRC; 2006.

38.

Deka

Ha Nguyen

Amiri

Goulet

J-A

. The Gaussian multiplicative approximation for state-space models. Struct Control Health Monit. 2022;29(3):e2904.

39.

Mazucheli

Menezes

AFB

Nadarajah

. mle.tools: an R package for maximum likelihood bias correction. R J. 2017;9(2):268–90.

40.

Strimmer

. Statistical Methods: Likelihood, Bayes and Regression. 2022. https://strimmerlab.github.io/publications/lecture-notes/MATH20802/math20802-script.pdf

41.

Strong

Oakley

Brennan

. Estimating multiparameter partial expected value of perfect information from a probabilistic sensitivity analysis sample: a nonparametric regression approach. Med Decis Making. 2014;34(3):311–26.

42.

Heath

Manolopoulou

Baio

. Estimating the expected value of partial perfect information in health economic evaluations using integrated nested laplace approximation. Stat Med. 2016;35(23):4264–80.

43.

Pawitan

. In All Likelihood: Statistical Modelling and Inference Using Likelihood. Oxford (UK): Oxford University Press; 2001.

44.

Rubin

. The Bayesian bootstrap. Ann Stat. 1981;9(1):130–4.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.08 MB

0.35 MB

0.17 MB

0.18 MB