The Impact of Alternative Specifications of Uncertainty Relating to Extrapolation in Decision Models

Abstract

Economic evaluations that incorporate value-of-information analysis frequently conclude that the greatest information value relates to replicating short-term clinical trials. This study builds on recent guidance relating to extrapolation in economic evaluation by assessing the impact of alternative approaches to representing the uncertainty around unobserved/extrapolated data with respect to incremental outcomes and value of information. When the uncertainty over unobserved and observed data is considered distinct but correlated (i.e., has a joint distribution), it is demonstrated that the value to replicating short-term clinical studies is lessened and that further studies relating to the unobserved periods likely provide more value.

Highlights

Current practice in economic evaluation often involves the inappropriate specification of uncertainty with respect to unobserved data.

Appropriate specification of uncertainty will lead to more pertinent recommendations over future clinical studies.

Keywords

extrapolation uncertainty value of information

Introduction

Guidelines for the economic evaluation of health care interventions recommend that the time horizon should adequately capture all relevant costs and benefits.^1,2 This typically necessitates the adoption of a lifetime horizon. Analysts frequently have to incorporate assumptions relating to the long-term clinical progression for disease due to the limited length of follow-up in the available evidence base.^3,4

To overcome the limited duration of clinical data, methods for extrapolation are adopted to allow the estimation of unobserved data based on establishing the relationship between observed data for a parameter and time.^5–7 Extrapolation is typically required for both the long-term natural history of the disease and the effect of treatment on the disease course. Parametric survival models are frequently adopted within economic evaluation with respect to survival or time-to-event analysis as a means of establishing the relationship between time and the parameters of interest.^8,9 Prior to the use of parametric survival analysis, a common approach involved analysts adopting an estimate of a parameter for a specific period of time and then assuming that the parameter could be applied for the full time horizon.^10,11 Alternatively, analysis was restricted to a presentation of the costs and outcomes for the period covered by the clinical study.^12,13 Although these methods have become less common, they are still adopted in some studies, despite not being recommended within guidelines.^1,2

Parametric survival analysis is particularly relevant to analyses that are based on a direct comparison of 2 treatment options and where patient-level data from an appropriate clinical trial are available.¹⁴ When patient-level data are not available, methods for deriving such data from summary statistics have been developed.^15,16 Parametric models have the advantage over nonparametric methods as they allow extrapolation beyond the time horizon of the available data, incorporating any variation in parameter estimate over this time horizon. The estimated survival function relates to interpolation of the observed values within a clinical data set. Thus, the uncertainty around the function relates only to how good the model fits the data for the observed period and does not provide information regarding the uncertainty over how well the model can be used to provide reasonable values for the extrapolation period.¹⁷

In many studies, the same parameter is applied to both the periods for which data are observed and unobserved. This approach to considering uncertainty contradicts standard mathematics, in which the uncertainty around unobserved data should be considered both separately from and greater than the uncertainty around observed data.

The Canadian Agency for Drugs and Technologies in Health (CADTH) has recently provided guidance regarding the extrapolation of short-term clinical data.¹⁸ A specific recommendation is that different parameters should be adopted for the period for which data are available and are unavailable and that uncertainty should be assumed to be greater for the period for which data are unavailable.

In this technical note, we explore alternative approaches to representing the uncertainty around unobserved data through assuming a joint distribution between observed and unobserved parameters. The impact of adopting alternative approaches is assessed in terms of the potential impact on incremental outcomes and on the expected value of perfect partial information per patient (ppEVPPI) for model parameters.

Methods

Case Study

We designed a decision model relating to a progressive disease that can be represented by a simple Markov model with 3 states: moderate, severe, and dead. A clinical trial with a maximum follow-up of 1 year is assumed to suggest that treatment slows progression to severe disease from moderate disease but has no direct effect on mortality. Thus, analysis can address the decision problem relating to whether a public payer should reimburse the new treatment for patients with moderate disease.

The analysis adopted a lifetime horizon with a cycle length of 1 year. Data for the annual treatment management cost, utility weights, and mortality rates associated with both moderate and severe disease were assumed available (Table 1). The probability of progression to severe disease without treatment and the relative effect of treatment on progression were assumed to be derived from an appropriate parametric analysis of the clinical trial data. For simplicity, analysis assumed an exponential functional form for the rate of progression over time (rProg) without treatment and that the proportional hazards assumption was appropriate with respect to the relative effect (hrProg) of the new treatment on progression.

Table 1

Data Inputs

Parameter	Data Label	Expected Value	Probability Distribution
Utility values
Moderate disease	uModerate	0.9	1 − LN [Normal (−2.31, 0.12)]
Severe	uSevere	0.5	1 − LN [Normal (−0.69, 0.03)]
Costs
Cost of new treatment per month	cTRTmonth	CAN$450
Management of moderate disease	cModerateNoTrt	CAN$1,000	Gamma (2,500, 0.4)
Management of severe disease	cSevere	CAN$2,000	Gamma (1,600,1.25)
Baseline annual rates
Mortality rate in moderate disease	rMortMod	0.1	Beta (20, 180)
Mortality rate in severe disease	rMortSev	0.24	Beta (24, 76)
Rate of progression without treatment	rProg	0.1	Beta (25, 225)
Clinical effectiveness
Hazard ratio of progression with treatment	hrProg	0.5	LN [Normal (−0.69, 0.35) ]
Model design features
Discount rate		0.015
Cycle length		1 year
Time horizon		Lifetime
Joint distributions
Rate of progression without treatment
Base case	$(rPro g_{obs}, rPro g_{unobs}) ~ (0.1, 0.1, 0.00036, 0.00036, 0.0)$
Scenario 1a: same variance ρ = 0.0	$(rPro g_{obs}, rPro g_{unobs}) ~ (0.1, 0.1, 0.00036, 0.00036, 0.0)$
Scenario 1b: same variance ρ = 0.2	$(rPro g_{obs}, rPro g_{unobs}) ~ (0.1, 0.1, 0.00036, 0.00036, 0.2)$
Scenario 1b: same variance ρ = 0.6	$(rPro g_{obs}, rPro g_{unobs}) ~ (0.1, 0.1, 0.00036, 0.00036, 0.6)$
Scenario 2a: higher variance ρ = 0.0	$(rPro g_{obs}, rPro g_{unobs}) ~ (0.1, 0.1, 0.00036, 0.00072, 0.0)$
Scenario 2b: higher variance ρ = 0.2	$(rPro g_{obs}, rPro g_{unobs}) ~ (0.1, 0.1, 0.00036, 0.00072, 0.2)$
Scenario 2c: higher variance ρ = 0.6	$(rPro g_{obs}, rPro g_{unobs}) ~ (0.1, 0.1, 0.00036, 0.00072, 0.6)$
Scenario 3a: increasing variance ρ = 0.0	$(rPro g_{obs}, rPro g_{unobs}) ~ (0.1, 0.1, 0.00036, 0.00036 * {1.1}^{t - 1}, 0.0)$
Scenario 3b: increasing variance ρ = 0.2	$(rPro g_{obs}, rPro g_{unobs}) ~ (0.1, 0.1, 0.00036, 0.00036 * {1.1}^{t - 1}, 0.2)$
Scenario 3c: increasing variance ρ = 0.6	$(rPro g_{obs}, rPro g_{unobs}) ~ (0.1, 0.1, 0.00036, 0.00036 * {1.1}^{t - 1}, 0.6)$
Hazard rate of progression with treatment
Base case	$(lnhrPro g_{obs}, lnhrPro g_{unobs}) ~ (- 0.69, - 0.69, 0.123, 0.123, 1)$
Scenario 1a: same variance ρ = 0.0	$(lnhrPro g_{obs}, lnhrPro g_{unobs}) ~ (- 0.69, - 0.69, 0.123, 0.123, 0.0)$
Scenario 1b: same variance ρ = 0.2	$(lnhrPro g_{obs}, lnhrPro g_{unobs}) ~ (- 0.69, - 0.69, 0.123, 0.123, 0.2)$
Scenario 1b: same variance ρ = 0.6	$(lnhrPro g_{obs}, lnhrPro g_{unobs}) ~ (- 0.69, - 0.69, 0.123, 0.123, 0.6)$
Scenario 2a: higher variance ρ = 0.0	$(lnhrPro g_{obs}, lnhrPro g_{unobs}) ~ (- 0.69, - 0.69, 0.123, 0.246, 0.0)$
Scenario 2b: higher variance ρ = 0.2	$(lnhrPro g_{obs}, lnhrPro g_{unobs}) ~ (- 0.69, - 0.69, 0.123, 0.246, 0.2)$
Scenario 2c: higher variance ρ = 0.6	$(lnhrPro g_{obs}, lnhrPro g_{unobs}) ~ (- 0.69, - 0.69, 0.123, 0.246, 0.6)$
Scenario 3a: increasing variance ρ = 0.0	$(lnhrPro g_{obs}, lnhrPro g_{unobs}) ~ (- 0.69, - 0.69, 0.123, 0.123 * {1.1}^{t - 1}, 0.0)$
Scenario 3b: increasing variance ρ = 0.2	$(lnhrPro g_{obs}, lnhrPro g_{unobs}) ~ (- 0.69, - 0.69, 0.123, 0.123 * {1.1}^{t - 1}, 0.2)$
Scenario 3c: increasing variance ρ = 0.6	$(lnhrPro g_{obs}, lnhrPro g_{unobs}) ~ (- 0.69, - 0.69, 0.123, 0.123 * {1.1}^{t - 1}, 0.6)$

Alternative Specification of Uncertainty for Unobserved Data

In the base-case analysis, the same parameters are adopted for rProg and hrProg for both the observed and unobserved periods with appropriate probability distributions (Figure 1a).

P (rProg) ~ (μ_{rProg}, {σ^{2}}_{rProg})

P (hrProg) ~ (μ_{hrProg}, {σ^{2}}_{hrProg})

Figure 1

Uncertainty around the probability of progression: no treatment: (a) base case and scenario 1, (b) scenario 2, and (c) scenario 3.

We examined 9 alternative specifications of uncertainty for the observed versus unobserved periods for both rProg and hrProg. We adopted 3 alternative estimates of the variance within the unobserved period and 3 alternative specifications for the correlation between estimates for the observed and unobserved periods.

To incorporate the alternative specification of uncertainty, the rate of progression and the hazard ratio were represented by different parameters for the observed period (the first cycle of the model) and for the unobserved period (subsequent cycles), and the sampled estimates for these parameters were assumed correlated. Within these alternative scenarios, the same mean value was adopted for all parameters with different assumptions relating to the variance for the unobserved period and the correlation between parameter values for the 2 periods. Thus, parameters for the observed and unobserved periods are jointly distributed.

P (rPro g_{obs}, rPro g_{unobs}) ~ ({μ_{rProg}}_{obs}, {μ_{rProg}}_{obs}, {σ^{2}}_{rProg}_{obs}, {σ^{2}}_{rProg}_{unobs}, {ρ_{rProg}}_{obs},_{rPro g_{unobs}})

P (hrPro g_{obs}, hrPro g_{unobs}) ~ ({μ_{hrProg}}_{obs}, {μ_{hrProg}}_{obs}, {σ^{2}}_{hrProg}_{obs}, {σ^{2}}_{hrProg}_{unobs}, {ρ_{hrProg}}_{obs},_{hrPro g_{unobs}})

The 3 alternative scenarios regarding the variance were as follows:

Scenario 1: the same variance in both the observed and unobserved period (but distinct parameters for each period) (Figure 1a)

va r_{unobs} = va r_{obs}

Scenario 2: the variance for the unobserved period was assumed to be higher than in the observed period (for this example, the variance was doubled) (Figure 1b)

va r_{unobs} = 2 * va r_{obs}

Scenario 3: the variance for the unobserved period increases with the time from the observed period. The variance for each cycle is a function of the variance in the observed period, the cycle time (t) and the duration (d) that the evidence relates to (for this example, the cycle-specific variance is assumed to be 10% greater than in the previous cycle) (Figure 1c)

va r_{unob s_{t}} = va r_{obs} * 1 . 1^{t - d}

For each of the above, further distinct assumptions were made relating to the correlation between the sampled probabilities for the observed period and the unobserved period: ρ = 0, 0.2, and 0.6. The 9 alternative specifications for the joint distribution are provided in Table 1.

It is important to note that standard practice when analyzing survival or time-to-event data is to estimate a single survival function for the observed period and assume that the survival function can be applied to the extrapolated period.⁶ This approach does assume a joint distribution but with specific assumptions that are likely indefensible. First, that the uncertainty within the extrapolated period is the same as the uncertainty within the observed period and, second, that the values for the extrapolated period and observed period are perfectly correlated. Thus, the current approach assumes the following joint distribution:

\begin{matrix} P (rPro g_{obs}, rPro g_{unobs}) ~ \\ ({μ_{rProg}}_{obs}, {μ_{rProg}}_{obs}, {σ^{2}}_{rProg}_{obs}, {σ^{2}}_{rProg}_{obs}, 1) \end{matrix}

The alternative scenarios considered in this article challenge these 2 implicit assumptions.

Analysis

For each scenario, costs and QALYs associated with each treatment, the incremental cost per QALY gained (ICER), and the net health benefit (NHB) were estimated through a Monte Carlo simulation of 5,000 replications using a common random seed for all analyses.¹⁹ Costs are expressed in Canadian dollars. The ppEVPPI for the parameters were estimated using the Sheffield accelerated value-of-information method assuming a threshold value of a QALY of CAN$50,000.^20–22

Results

In the base-case analysis, the new treatment was associated with incremental QALYs of 0.75, incremental costs of $28,878, an ICER of $38,728, and an NHB of 0.19 QALYs (Table 2). Under scenario 1, the results were consistent with the base case; incremental QALYs ranged from 0.74 to 0.75, and incremental costs ranged from $28,747 to $28,891, with ICERs between $38,652 and $38,814 and NHBs of 0.17 QALYs. Under scenario 3, for all 3 analyses, there were slightly lower incremental QALYs (range of 0.72 to 0.73) with little change in incremental costs (range of $28,808 to $28,842), with ICERs between $39,567 and $39,794 and NHBs of 0.15 QALYs. Scenario 2 found the largest divergence from the base case in terms of incremental QALYs (0.71 for all levels of correlation), although incremental costs were broadly consistent with the base case (range of $28,709 to $28,747). Under scenario 2, ICERs were higher, with a range of $40,225 to $40,488 and NHBs lower, between 0.13 and 0.14 QALYs.

Table 2

Expected Outcomes for Base-Case and Alternative Scenarios

		Scenario 1: Different Variance			Scenario 2: Higher Variance			Scenario 3: Increasing Variance
	Base Case	ρ = 0^a	ρ = 0.2^a	ρ = 0.6^a	ρ = 0^a	ρ = 0.2^a	ρ = 0.6^a	ρ = 0^a	ρ = 0.2^a	ρ = 0.6^a
QALYs
No treatment	4.70	4.70	4.70	4.70	4.72	4.72	4.72	4.71	4.71	4.71
Treatment	5.45	5.45	5.45	5.45	5.43	5.43	5.43	5.44	5.44	5.44
Incremental	0.75	0.75	0.75	0.74	0.71	0.71	0.71	0.73	0.73	0.72
Costs
No treatment	$7,945	$7,946	$7,946	$7,946	$7,942	$7,942	$7,941	$7,940	$7,940	$7,940
Treatment	$36,819	$36,837	$36,827	$36,812	$36,688	$36,674	$36,651	$36,781	$36,768	$36,747
Incremental	$28,873	$28,891	$28,881	$28,866	$28,747	$28,732	$28,709	$28,842	$28,828	$28,808
Incremental cost per QALY gained	$38,728	$38,652	$38,718	$38,814	$40,225	$40,330	$40,488	$39,567	$39,658	$39,794
Net health benefit (QALYs)	0.19	0.17	0.17	0.17	0.13	0.14	0.14	0.15	0.15	0.15
EVPI per patient	$3,010	$2,219	$2,394	$2,739	$4,336	$4,615	$4,967	$3,476	$3,682	$4,072
EVPPI per patient
uModerate	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
uSevere	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
cModerateNoTrt	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
cSevere	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
rMortMod	$52	$47	$48	$52	$97	$101	$111	$69	$71	$78
rMortSev	$1	$1	$1	$2	$2	$2	$4	$2	$2	$3
rProg_obs– year 1	$73	$ 0	$ 0	$2	$ 0	$ 0	$78	$ 0	$ 0	$26
hrProg_obs– year 1	$2,738	$32	$213	$1,269	$54	$679	$2,541	$44	$382	$2,017
rProg_unobs - year 2+	N/A	$11	$20	$47	$221	$209	$401	$115	$154	$245
hrProg_unobs - year 2+	N/A	$1,886	$2,044	$2,394	$2,942	$3,820	$4,577	$3,127	$3,337	$3,726

EVPI, expected value of perfect information; EVPPI, expected value of perfect partial information; QALY, quality-adjusted life-year.

Correlation between rProg_obs and rProg_unobs and between hrProg_obs and hrProg_unobs.

In the base-case analysis, ppEVPPI was greatest for the parameters relating to the original clinical trial: rProg ($73 per patient) and hrProg ($2,738 per patient) (Table 2). However, for all alternative specifications of uncertainty, the results were substantially different.

For all specifications assuming lower correlation (ρ = 0 and ρ = 0.2), the ppEVPPI for rProg_obs was 0 (Table 2). When a higher correlation (ρ = 0.6) was assumed, the estimates of EVPPI for rProg_obs ranged from $2 to $78. Within each specification of uncertainty, the ppEVPPI for rProg_unobs was substantially greater than that for rProg_obs. The estimated ppEVPPI for rProg_unobs was largest for scenario 2 (range of $209 to $401), followed by scenario 3 (range $115 to $245) and then Scenario 1 (range $11 to $47). Within each scenario, the ppEVPPI for rProg_unobs was larger with high rates of correlation.

Across all scenarios, the ppEVPPI for hrProg_obs (range from $32 to $2,541) was lower than that for hrProg in the base-case analysis and lower than for the ppEVPPI for hrProg_unobs (range $1,886 to $4,577) (Table 2). As above, the estimated ppEVPPI for both hrProg_obs and hrProg_unobs was greater with a greater correlation and was greater for scenario 2, followed by Scenario 3 and then Scenario 1.

Discussion

We adopted a hypothetical economic model to demonstrate the impact of adopting alternative specifications of uncertainty for time periods for which there are no observed values for a data parameter. This approach has been recommended in recent guidance from CADTH and is consistent with accepted mathematical practice in that uncertainty around unobserved data should be considered greater than observed data. The main finding is that the information value from repeating short-term clinical trials may be much lower than previously suggested. The analysis did find that estimated ICERs will vary by the assumptions around uncertainty, but in most situations, this is unlikely to substantially alter decision making.

The finding that there may be limited information value from repeating short-term trials may be considered obvious. The current practice of assuming that the observed data for an input (e.g., rate and hazard ratio) can be used to estimate the expected value and uncertainty for a single input parameter, which relates to both the observed and unobserved periods, necessarily implies that there is no value from extending the time period for observed clinical data. Thus, the alternative approaches suggested within this article reach conclusions with respect to information value that more accurately reflect expectations than would be reached though adopting current accepted practice.

There are limitations to the study. Assumptions around the uncertainty for the unobserved data and the correlation between estimates for the observed and unobserved periods were subjective, although this will, by nature, always be the case. For example, doubling the variance around the rate of progression, which is equivalent to halving the person-time used to inform the unobserved period, may be considered too conservative. Thus, we suggest that scenario 3, with an increasing variance over the time horizon for unobserved parameters, is likely the more realistic scenario and the approach that we would advocate should be adopted within a base case. However, the study is meant to be illustrative, and the finding that conclusions around the information value from further research will be different if different specifications of uncertainty for the unobserved period are adopted will hold regardless of the assumptions adopted. What is clear from each of the scenarios is that the finding of limited value to repeating short-term studies is repeated regardless of how uncertainty is specified.

The analysis focuses on the use of parametric survival analyses within economic evaluations. Previously, a common approach would involve taking an estimate of a parameter for a specific period of time and applying the parameter for the full time horizon of the model. This approach is still adopted in some studies, particularly in relation to the effect of treatments on risk factors relating to future disease events.²³ Although not recognized as a form of extrapolation, the method by which it handles data for unobserved periods is analogous to current approaches with regard to handling the estimates from a parametric survival analysis. Thus, when analyses are conducted with this approach, alternative specifications of uncertainty between the observed and unobserved periods as explored in this article should be considered.

For ease of exposition, the analysis assumed that the long-term time to event could be represented by an exponential model. In most studies, more complex models are required. The general approach suggested can still be applied with more complex models, with uncertainty characterized by a joint distribution incorporating more than 2 parameters, that is, incorporating estimates of the parametric model parameters for both the observed and unobserved period and their underlying uncertainty and correlation.

In this article, 3 scenarios were considered with respect to the uncertainty in the unobserved period: the same variance, a higher variance, and an increasing variance. The purpose of the article was to illustrate the potential alternatives available and the impact of each of these. The increasing variance approach is intuitively the most appealing alternative as the uncertainty over a parameter would increase the further in the future that data are extrapolated. This has been demonstrated previously.^24–27 Previous studies have adopted similar approaches to modeling uncertainty within the unobserved period. In an economic evaluation examining the cost-effectiveness of screening for ovarian cancer among postmenopausal women, Kearns et al. adopted a model discrepancy method to assess the impact of alternative assumptions relating to the size and uncertainty around the treatment effect within the unobserved periods.²⁸ Scenarios 2 and 3 explored in this study are analogous to this approach without consideration of the potential of a waning of treatment effect. Similarly, Mahon²⁹ suggested the use of a temporal parameter that considers how the uncertainty over a parameter can change over the model’s time horizon. The suggested adoption of a Weiner process is akin to the approach within scenario 3 within this article.

Economic evaluations that incorporate value-of-information analysis frequently conclude that the greatest information value relates to estimates of transition probabilities and clinical effectiveness. However, such analyses do not distinguish between observed and unobserved periods and are therefore falsely concluding that “more of the same” clinical data are required. When uncertainty over unobserved and observed data are more fully represented, there may be limited value to repeating short-term studies, and further studies relating to the unobserved periods are required. As studies adopt the recent recommendations regarding unobserved data, future value-of-information analyses will be less likely to conclude that further clinical information of the same is required. Further analysis can focus instead on the optimal duration of further studies.

Footnotes

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. The authors received no financial support for the research, authorship, and/or publication of this article.

Ethical Considerations

No ethics approval was required for this study as it does not involve participants.

Consent to Participate

No consent was required for this study as it does not involve participants.

Consent for Publication

Consent for publication is granted by all authors.

ORCID iD

Doug Coyle

Data Availability

No data were used in this study.

References

CADTH. Guidelines for the Economic Evaluation of Health Technologies: Canada. 4th ed. Ottawa: CADTH; 2017.

National Institute for Health and Care Excellence. Guide to the Methods of Technology Appraisal 2013. London: National Institute for Health and Care Excellence; 2013.

Buxton

Drummond

Van Hout

, et al. Modelling in ecomomic evaluation: an unavoidable fact of life. Health Econ. 1997;6(3):217–27. DOI:10.1002/(sici)1099-1050(199705)6:3<217::aid-hec267>3.3.co;2-n

Tappenden

Chilcott

Ward

Eggington

Hind

Hummel

. Methodological issues in the economic analysis of cancer treatments. Eur J Cancer. 2006;42(17):2867–75. DOI:10.1016/j.ejca.2006.08.010

Latimer

. Survival analysis for economic evaluations alongside clinical trials—extrapolation with patient-level data: inconsistencies, limitations, and a practical guide. Med Decis Making. 2013;33(6):743–54. DOI:10.1177/0272989X12472398

Ishak

Kreif

Benedict

Muszbek

. Overview of parametric survival analysis for health-economic applications. Pharmacoeconomics. 2013;31(8):663–75. DOI:10.1007/s40273-013-0064-3

Crowther

Lambert

. A general framework for parametric survival analysis. Stat Med. 2014;33(30):5280–97. DOI:10.1002/sim.6300

Gerdtham

Zethraeus

. Predicting survival in cost-effectiveness analyses based on clinical trials. Int J Technol Assess Health Care. 2003;19(3):507–12. DOI:10.1017/S0266462303000436

Gelber

Goldhirsch

Cole

. Parametric extrapolation of survival estimates with applications to quality of life evaluation of treatments. Control Clin Trials. 1993;14(6):485–99. DOI:10.1016/0197-2456(93)90029-D

10.

Holtgrave

Kelly

. Cost-effectiveness of an HIV/AIDS prevention intervention for gay men. AIDS Behav. 1997;1(3):173–80. DOI:10.1023/B:AIBE.0000002978.57770.72

11.

Schulman

Kinosian

Jacobson

, et al. Reducing high blood cholesterol level with drugs: cost-effectiveness of pharmacologic management. JAMA. 1990;264(23):3025–33. DOI:10.1001/jama.1990.03450230061030

12.

Buxton

Acheson

Caine

Gibson

O’Brien

. Costs and Benefits of the Heart Transplant Programmes at Harefield and Papworth Hospitals. Department of Health and Social Security, Research Report No. 12. London: HMSO; 1985.

13.

Coyle

Drummond

. Costs of conventional radical radiotherapy versus continuous hyperfractionated accelerated radiotherapy (CHART) in the treatment of patients with head and neck cancer or carcinoma of the bronchus. Clin Oncol. 1997;9(5):313–21. DOI:10.1016/S0936-6555(05)80065-2

14.

Latimer

. NICE DSU Technical Support Document 14: Undertaking Survival Analysis for Economic Evaluations alongside Clinical Trials—Extrapolation with Patient-Level Data. 2011. Available from: http://www.nicedsu.ork.uk.

15.

Guyot

Ades

Ouwens

MJNM

Welton

. Enhanced secondary analysis of survival data: reconstructing the data from published Kaplan-Meier survival curves. BMC Med Res Methodol. 2012;12:9. DOI:10.1186/1471-2288-12-9

16.

Liu

Zhou

Lee

. IPDfromKM: reconstruct individual patient data from published Kaplan-Meier survival curves. BMC Med Res Methodol. 2021;21(1):1–22. DOI:10.1186/s12874-021-01308-8

17.

Grieve

Hawkins

Pennington

. Extrapolation of survival data in cost-effectiveness analyses: improving the current state of play. Med Decis Making. 2013;33(6):740–2. DOI:10.1177/0272989X13492018

18.

Coyle

Haines

Lee

. Extrapolating clinical evidence within economic evaluations. Can J Heal Technol. 2023;3(5):1–21.

19.

Stinnett

Mullahy

. Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Med Decis Mak. 1998;18(2 suppl):68–80. DOI:10.1177/0272989x98018002s09

20.

Felli

Hazen

. A Bayesian approach to sensitivity analysis. Health Econ. 1999;8(3):263–8. DOI:10.1002/(SICI)1099-1050(199905)8:3<263::AID-HEC426>3.0.CO;2-S

21.

Strong

Oakley

Brennan

. Estimating multiparameter partial expected value of perfect information from a probabilistic sensitivity analysis sample: a nonparametric regression approach. Med Decis Making. 2014;34(3):311–26. DOI:10.1177/0272989X13505910

22.

Strong

Brennan

Oakley

. How to calculate value of information in seconds using ‘savi’, the sheffield accelerated value of information web app. Value Health. 2015; 18(7):A725–26. DOI:10.1016/j.jval.2015.09.2759

23.

Hayes

Leal

Gray

Holman

Clarke

. UKPDS outcomes model 2: a new version of a model to simulate lifetime health outcomes of patients with type 2 diabetes mellitus using data from the 30 year United Kingdom Prospective Diabetes Study: UKPDS 82 Lipids in Diabetes Study. Diabetologia. 2013;56:1925–33. DOI:10.1007/s00125-013-2940-y

24.

Kearns

Stevens

Ren

Brennan

. How uncertain is the survival extrapolation? A study of the impact of different parametric survival models on extrapolated uncertainty about hazard functions, lifetime mean survival and cost effectiveness. Pharmacoeconomics. 2020;38(2):193–204. DOI:10.1007/s40273-019-00853-x

25.

Dietze

. Prediction in ecology: a first-principles framework: A. Ecol Appl. 2017;27(7):2048–60. DOI:10.1002/eap.1589

26.

Conn

Johnson

Boveng

. On extrapolating past the range of observed data when making statistical predictions in ecology. PLoS One. 2015;10(10):1–16. DOI:10.1371/journal.pone.0141416

27.

Blackwell

Honaker

King

. A unified approach to measurement error and missing data: overview and applications. Sociol Methods Res. 2017;46(3):303–41. DOI:10.1177/0049124115585360

28.

Kearns

Chilcott

Whyte

Preston

Sadler

. Cost-effectiveness of screening for ovarian cancer amongst postmenopausal women: a model-based economic evaluation. BMC Med. 2016;14(1):1–10. DOI:10.1186/s12916-016-0743-y

29.

Mahon

. Temporal Uncertainty in Cost-Effectiveness Decision Models. York: University of York; 2014.