Sage Journals: Discover world-class research

Abstract

Introduction

Economic evaluations identify the best course of action by a decision maker with respect to the level of health within the overall population. Traditionally, they identify 1 optimal treatment choice. In many jurisdictions, multiple technologies can be covered for the same heterogeneous patient population, which limits the applicability of this framework for directly determining whether a new technology should be covered. This article explores the impact of different decision frameworks within this context.

Methods

Three alternate decision frameworks were considered: the traditional normative framework in which only the optimal technology will be covered (normative); a commonly adopted framework in which the new technology is recommended for reimbursement only if it is optimal, with coverage of other technologies remaining as before (current); and a framework that assesses specifically whether coverage of the new technology is optimal, incorporating previous reimbursement decisions and the market share of current technologies (positivist). The implications of the frameworks were assessed using a simulated probabilistic Markov model for a chronic progressive condition.

Results

Results illustrate how the different frameworks can lead to different reimbursement recommendations. This in turn produces differences in population health effects and the resultant price reductions required for covering the new technology.

Conclusion

By covering only the optimal treatment option, decision makers can maximize the level of health across a population. If decision makers are unwilling to defund technologies, however, the second best option of adopting the positivist framework has the greatest relevance with respect to deciding whether a new technology should be covered.

Highlights

Traditionally, economic evaluations focus on identifying the optimal treatment choice.

This paper considers three alternative decision frameworks, within the context of multiple technologies being covered for the same heterogeneous patient population.

This paper highlight that if decision makers are unwilling to defund therapies, current approaches to assessing cost effectiveness may be non-optimal.

Keywords

decision making cost effectiveness uncertainty value of information

Background

The objective of economic evaluation of health care decisions is to identify which course of action by a decision maker would lead to the highest level of health within the population of interest given a budget constraint.¹ This requires estimation of the long-term costs and benefits associated with each potential treatment option and then deciding which treatment option to implement by considering whether the value generated would offset the value produced through other technologies, which would need to be forgone given the limited budget for health care.^1–3

The study of economics has traditionally been categorized into positive and normative economics, although the 2 subdisciplines are inherently linked.⁴ Positive economics describes what has occurred providing rationales for why, whereas normative economics focuses on what ought to occur subject to a decision maker’s objectives and constraints. In the context of health care decision making, positivist analyses describe and predict the consequences of policy change, whereas normative analyses determine which policy change would be optimal.

Economic evaluation is traditionally rooted within a normative framework, with the focus being on identifying which is the optimal treatment choice—the technology with the highest net benefit.⁵ This is consistent with normative theories relating to constrained optimization.⁶ Underlying this framework is the assumption that for each mutually exclusive decision, decision makers will fund the sole optimal treatment option. When placing economic evaluation in the context of whether a new technology should be covered, the normative framework assesses whether the new technology under consideration would maximize the level of the health of the population, if it were the only treatment choice covered for the condition under consideration.

However, in practice, there will be many different treatment options covered for a given indication and patient population.

Phelps recognized that heterogeneity within a patient population can lead to different estimates of expected outcomes and/or costs for treatment alternatives, thus leading to variations in cost-effectiveness ratios within a patient population and, hence, differences in optimal treatment choices.⁷ Building on this work, Coyle and colleagues⁸ developed the concept of stratified cost-effectiveness analysis, which highlighted that health gains would be maximized by making subgroup-specific decisions with respect to the coverage of a new technology. Kim and Basu⁹ further identified that when heterogeneity exists, there may be differential uptake of a technology in response to heterogeneous treatment effects. Thus, the net benefit of a technology is the weighted sum of the technology-specific net benefit for each subgroup where the weights are the products of the size of the subgroups and the rate of the adoption specific to that subgroup and technology.⁹ Thus, in situations where expected costs and outcomes vary across subgroups, Kim and Basu recognized that an analysis that incorporates adoption within each subgroup will provide more relevant information to decision makers.⁹

When patient populations can be defined such that there is no observed heterogeneity, multiple treatment options may remain for other reasons. These can include inefficient rationales such as inertia in clinical practice, poor information, perverse incentivesm, differences in the ability to pay, and political pressure. However, this may also be due to unobserved heterogeneity in outcomes and/or patient preferences, both of which are entirely consistent with principles of evidence-based medicine, combining best-quality evidence with clinical experience to determine the best treatment in consultation with the patient.^10,11 In this context, analyses that identify only the sole optimal treatment option are pertinent to the explicit decision relating to coverage only if a decision maker is willing to exclude all other technologies from coverage. If decision makers are not willing to exclude existing covered technologies from further coverage, interpreting the results of studies by adopting this framework has limited relevance. Current guidelines for economic evaluation, however, emphasize this approach, and if applied without direct consideration of the decision problem facing decision makers, this may lead to nonoptimal decision making.^3,12,13

The failure to consider previous coverage decisions has been recognized as one of the factors limiting the relevance of economic evaluation as it is currently interpreted.¹⁴ Alternative approaches have been suggested whereby economic evaluation should focus on the optimal reimbursement strategy, not necessarily the optimal sole treatment.¹⁵ Thus, with respect to the decision regarding coverage of a new technology, an alternative decision framework could take a positivist position regarding current treatment practice by accepting previous decisions relating to reimbursement and its effect on technology adoption (i.e., the market shares of the currently covered treatment choices). This is akin to the methods of Kim and Basu⁹ with respect to heterogeneity and differential treatment adoption.

The alternative decision framework, therefore, would focus specifically on determining whether covering the new technology is optimal in terms of whether it would increase the level of the health of the whole population, given the resultant change in market shares. This may be considered more pertinent to the actual decision-making environment in which such decisions are made. The alternative framework is consistent with how budget impact analysis is traditionally conducted, in that it recognizes the current market share of currently covered technologies and forecasts the change in market share over time with and without the additional coverage of a new technology.¹⁶

The focus of this article is to detail the different approaches to interpreting and applying the results of economic analysis and highlight the positive and negative consequences of each decision framework with respect to decision making and the impact on the health of the population.

Frameworks for Assessing Cost-Effectiveness

Three potential decision frameworks for assessing whether a new technology should be covered are detailed below. The first 2 decision frameworks adopt the same analytical framework, based on current accepted practice within economic evaluation, whereas the third decision framework adopts a revised analytical approach.

Normative Framework

The traditional normative approach for assessing cost-effectiveness, as recommended within guidelines for economic evaluation, involves estimating the expected costs and outcomes for each treatment alternative and identifying the optimal treatment choice based on the decision maker’s threshold value of the outcome obtained (framework 1: the normative framework).^3,17,18 We will assume that outcomes are measured in quality-adjusted life years (QALYs) and that the threshold represents the marginal productivity of health care. Thus, analysis assumes that only the optimal treatment choice would be covered; by finding the optimal technology, the level of health within the population will be maximized. As above, we assume that the patient population considered is homogenous in that the expected outcomes for patients are not expected to vary by patient characteristics.

When there are only 2 treatment options (e.g., an existing technology and a new technology, A and B), the value of reimbursing the new technology is simply assessed by estimating the incremental cost per QALY gained (incremental cost-effectiveness ratio; ICER) from the new technology.^2,3

ICE R_{a, b} = \frac{E (C_{b}) - E (C_{a})}{E (Q_{b}) - E (Q_{a})},

where a,b denotes the existing technology A and the new technology B.

ICER_a,b = incremental cost per QALY gained for b versus a

E(Q) = expected QALYs

E(C) = expected health care costs

The existing technology could relate to a nonactive treatment alternative such as best supportive care if the new technology is the first active treatment available for the condition of interest. With 2 treatment options, the normative decision framework allows a decision maker to determine whether to cover the new technology based on their threshold value of a QALY (λ), which should represent the marginal productivity of those technologies that would be foregone if the new technology were covered. If the new technology is covered, the existing technology should be defunded.

When there are multiple treatment options, the normative analytical approach involves conducting a sequential analysis to identify the optimal treatment.^2,3,19 This involves identifying which technologies cannot be optimal based on the concepts of dominance and extended dominance. Further to this, analysis requires the calculation of the ICER for a less effective comparator compared with the next most effective comparator (or, alternatively, the options can be ranked by cost and ICERs calculated: the mathematics are the same). This allows identification of the interventions that lie on the cost-effectiveness frontier (i.e., the technologies that may be optimal dependent on the threshold value of a QALY). Thus, applying a normative decision framework allows a decision maker to determine whether the new technology should be covered as the optimal treatment choice based on λ. As above, if the new technology is covered, all currently funded alternative technologies should be defunded.

Note that in both the above scenarios, if λ is known by the analyst, the optimal treatment can be determined by the treatment option (j) with the greatest net health benefit (NHB).^20,21

NH B_{j} = E (Q_{j}) - \frac{E (C_{j})}{λ},

where j denotes treatment.

Current Framework

The normative decision framework assumes an ideal world in which only the optimal treatment is covered and all others are explicitly defunded. However, in many circumstances, this scenario is not relevant to the decision problem faced by decision makers. Multiple treatment options are often already covered for a specific patient population, and decision makers are interested only in whether the new technology should be added to those covered technologies.

The second decision framework, therefore, uses the same analysis as the normative framework does but applies it to the world in which there have been previous nonoptimal decisions. Within this framework, it is recognized that the focus of decision makers relates solely to whether the new technology should be covered, not specifically to which technology is optimal (framework 2: current framework). This framework closely follows the deliberative processes adopted by health technology agencies such as CDA-AMC (formerly CADTH) and the NICE Single Technology Appraisal (STA) process. In this circumstance, evaluations conducted within this framework are interpreted in terms of determining whether the new technology is the optimal technology.^22,23 If optimal, the coverage of the technology may be permitted, and the new technology will be added to those therapies already covered. If the new technology is not optimal, coverage would be denied or a price reduction may be negotiated based on the required price for the technology to be optimal.

Positivist Framework

In the presence of multiple covered treatment options in which defunding technologies is not an option, the adoption of the current decision framework therefore does not address the true decision problem facing decision makers, namely, whether the new technology should be added to those technologies covered for the specific patient population. The third decision framework directly addresses this decision problem by estimating the expected long-term costs and QALYs for the patient population with and without coverage of the new technology based on the estimated market shares for technologies under both scenarios (framework 3: positivist framework).

QAL Y_{i} = \sum_{j - 1 \dots . n} p_{ij} . E (Q_{ij})

Cos t_{i} = \sum_{j - 1 \dots . n} p_{ij} . E (C_{ij}),

where E(Q) and E(C) reflect the total lifetime discounted expected QALYs and costs.

i = reimbursement scenario (0 = no coverage for the new technology, 1 = coverage for the new technology).

j denotes technology.

p_ij = market share for each technology dependent on reimbursement scenario.

Thus, the analysis simply compares 2 states of the world: the treatment of the patient population if the new technology is not covered and the treatment of the patient population if the new technology is covered. Decision makers can determine whether covering the new technology is the optimal scenario based on λ. If λ is unknown to the analyst, analysis can focus on presenting the ICER.

ICE R_{0, 1} = \frac{E (C_{1}) - E (C_{0})}{E (Q_{1}) - E (Q_{0})},

where 0 = no coverage for the new technology and 1 = coverage for the new technology.

If λ is known to the analyst, then the optimal decision can be presented based on whether the NHB for covering the new technology (NHB₁) is greater than not covering the technology (NHB₀). In other words, whether there is a positive incremental net benefit from covering.

N H B_{i} = E (Q_{i}) - \frac{E (C_{i})}{λ},

where i = reimbursement scenario (0 = no coverage for the new technology, 1 = coverage for the new technology).

Consider the situation in which there are 3 technologies currently covered for the condition of interest. The new technology may be forecasted to take market share from the 1 technology that it is comparatively cost-effective against but not take market share from the 2 other existing technologies for which it is inefficient. Covering the new technology may therefore actually result in an increase in overall health. Thus, the positivist framework, unlike the other 2 frameworks, reflects the real-world impact on outcomes from the decision on whether or not to allow coverage of the new intervention.

Worked Example of Coverage Decisions under the Different Frameworks

The worked example relates to a hypothetical situation in which a new technology is considered for reimbursement by a health technology assessment agency, in the context of other available technologies currently covered. We developed a simulated analysis to illustrate the implications of the 3 alternative frameworks for making a coverage decision with respect to the new technology. The simulated analysis compares 5 treatment options (no treatment; treatments A, B, and C; and a new treatment D) for a hypothetical progressive disease. Analysis was based on a Markov model with 3 health states: moderate disease, severe disease, and death. Analysis was conducted based on 3-mo cycles and a 50-y (lifetime) horizon. Data inputs are detailed in Table 1. Inputs relate to the rates of progression and death, treatment-specific hazard rates for progression, treatment costs, and state-specific disease costs and utility values.

Table 1

Input Parameters for Simulation

Variable	Expected Value	Probability Distribution
Annual rates
Rate of progression without treatment	0.4	Beta (40, 60)
Mortality rate with moderate disease	0.04	Beta (4, 96)
Mortality rate with severe disease	0.25	Beta (25, 75)
Disutility
Moderate disease	0.2	ln [Normal (−1.611, 0.060)]
Severe disease	0.5	ln [Normal (−0.694, 0.032)]
Disease costs per 3 mo
Managing moderate disease	1,000	Gamma (100, 10)
Managing severe disease	5,000	Gamma (100, 50)
Treatment costs per 3 mo
Treatment A	100	Fixed
Treatment B	3,200	Fixed
Treatment C	3,200	Fixed
Treatment D	4,600	Fixed
Hazard rates of progression versus no treatment
Treatment A	0.95	ln [Normal (−0.051, 0.05)]
Treatment B	0.76	ln [Normal (−0.051, 0.05)]
Treatment C	0.74	ln [Normal (−0.051, 0.05)]
Treatment D	0.5	ln [Normal (−0.051, 0.05)]
Market share without treatment D
No treatment	20%	Dirichlet (20, 20, 30, 30)
Treatment A	20%
Treatment B	30%
Treatment C	30%
Capture rates for treatment D
No treatment	20%	Beta (20, 80)
Treatment A	20%	Beta (20, 80)
Treatment B	50%	Beta (50, 50)
Treatment C	50%	Beta (50, 50)

Normative Framework

Table 2 presents the results of a standard sequential economic evaluation. Treatments B and C cannot be optimal as they are subject to extended dominance. If λ is less than $12,024, no treatment is optimal. Treatment A is optimal if λ is between $12,024 and $58,269. Treatment D will be optimal if λ is greater than $58,269. In our example, if we assume that λ is $50,000 per QALY, treatment A is optimal, which is also indicated by treatment A having the highest NHB. Thus, the normative decision framework would suggest that the decision maker should choose to fund only treatment A (i.e., treatment B and C should be defunded and treatment D should not be added to funding). Under optimal treatment patterns, all patients would receive the optimal treatment. However, for various potential reasons such as access or physician and patient choice, not all individuals will be treated with treatment A as a proportion will remain untreated (receive the no treatment option).

Table 2

Sequential Cost-Effectiveness Analysis

	QALYs	Total Costs	Sequential Cost-Effectiveness Results (Incremental Cost per QALY Gained)	Net Health Benefit (λ = $50,000)
No treatment	3.43	$78,225		1.864
Treatment A	3.50	$79,082	$12,024	1.918
Treatment D	4.73	$151,042	$58,269	1.714
Treatment B	3.84	$112,539	Subject to extended dominance	1.589
Treatment C	3.89	$113,443	Subject to extended dominance	1.621

QALY, quality-adjusted life-year.

Current Framework

When decision makers are not able to or are unwilling to defund existing covered technologies, interpretation of the sequential analysis leads to the conclusion that the new technology, treatment D, is not optimal (Table 2). Thus, under the current decision framework, the decision is made not to fund the new technology; the coverage status of other technologies remains the same (i.e., treatments A, B, and C would remain covered and treatment D would not be added to coverage).

Positivist Framework

Under the positivist decision framework, the analysis compares 2 states of the world: the treatment of the patient population without covering treatment D and the treatment of the patient population with covering treatment D. The expected costs and QALYs for the 2 reimbursement scenarios are the probability weighted sum of the assumed market shares for each technology under the different reimbursement scenarios and their expected costs and QALYs. The cost-effectiveness of covering treatment D is addressed by whether the coverage scenario meets the threshold value for a QALY.

In the hypothetical analysis, we have simplified the analysis for illustration by assuming the market share of existing products without covering the new treatment will remain constant and that there will be no unexpected product entry (Table 1). In real-world situations, this may be unlikely, and more complex modeling of market shares allowing for changing market shares over time and incorporating discount rates would be required. The expected values of the market shares are assumed to be 20% for treatment A, 30% for treatment B, and 30% for treatment C, with 20% untreated; note that there is uncertainty in the market shares, which is characterized by a Dirichlet distribution.²⁴ Similarly, we assume that, if covered, the new product will capture market share in the first year of coverage and the subsequent revised market shares will remain constant. For the simulation, treatment D is expected to capture 20% of the market for treatment A and 50% of the market for both treatments B and C and take 20% of the market, which was previously untreated. The percentage treatment D captures from each treatment is assumed independent, so uncertainty is modeled with independent beta distributions. Thus, for the scenario in which treatment D is covered, the expected values for the market shares will be 16% for no treatment, 16% for treatment A, 15% for treatment B, 15% for treatment C, and 38% for treatment D.

The positivist decision framework concludes that covering treatment D leads to greater costs and greater QALYs than not covering treatment D, with an incremental cost per QALY gained of $47,434 (Table 3). Thus, based on a threshold value of a QALY of $50,000, analysis concludes it would be optimal to include coverage of treatment D (i.e., treatment D would be covered alongside treatments A, B, and C).

Table 3

Cost-Effectiveness Analysis: Positivist Framework

	QALYs	Total Costs	Incremental Cost per QALY Gained	Net Health Benefit (λ = $50,000)
Do not fund treatment D	3.70	$99,259		1.719
Fund treatment D	4.07	$116,447	$47,434	1.738

QALY, quality-adjusted life-year.

Impact of Different Decision Frameworks

The 3 different decision frameworks can be assessed in terms of the difference in coverage decisions, their impact on NHBs to the general population, and the necessary price reduction required for the new technology to avoid a net reduction in population health if covered.

The impact on NHB of the different frameworks will be a function of the market shares of all alternative technologies after the coverage decision has been made (Table 4). Each decision framework leads to different combinations of treatment alternatives being covered with differential market shares. The impact on the health of the population from the implementation of each of the 3 scenarios for decision making can be compared by estimating the NHB obtained under each of the 3 potential decision frameworks. In this context, the NHB from each framework is a function of the expected costs and QALYs for each alternative technology, λ, and the estimated market share of each technology given the adoption of each framework.

NH B_{k} = \sum_{j = 1}^{n} p_{jk} * (E (Q_{j}) - \frac{E (C_{j})}{λ}),

where

k denotes the decision framework (normative, current, positivist).

j denotes technology (no treatment, treatment A, treatment B, treatment C, treatment D).

p denotes the market share of each technology under each decision framework.

E(Q) = expected QALYs.

E(C) = expected health care costs.

Table 4

Impact of Alternative Decision-Making Frameworks on the Health of the Population

	Expected QALYs	Expected Costs	Net Health Benefit (λ = $50,000)	Normative Framework	Current Framework	Positivist Framework
	Expected QALYs	Expected Costs	Net Health Benefit (λ = $50,000)	Market Shares
Untreated	3.43	$78,225	1.864	20%	20%	16%
Treatment A	3.50	$79,082	1.918	80%	20%	16%
Treatment B	3.84	$112,539	1.589	0%	30%	15%
Treatment C	3.89	$113,443	1.621	0%	30%	15%
Treatment D	4.73	$151,042	1.714	0%	0%	38%
Net health benefit				1.907	1.719	1.738
Incremental impact over current market shares				0.188	0	0.018

QALY, quality-adjusted life-year.

Based on the above, when λ = $50,000, the expected values for NHB are 1.907 under the normative decision framework, 1.719 under the current decision framework, and 1.738 under the positivist decision framework. For this example, the relative ranking of the 3 frameworks will hold regardless of λ; that is, the NHB under the normative decision framework will be greater than or equal to the NHB under the positivist decision framework, and the NHB under the positivist decision framework will be greater than or equal to the NHB under the current decision framework (Figure 1). The NHB derived under the normative decision framework is greater than under both frameworks, where nonoptimal treatments will not be defunded regardless of λ. The NHB derived under the positivist decision framework is greater than the current decision framework for values of λ between $47,434 (the ICER for covering treatment D versus no coverage) and $58,269 (the ICER for treatment D versus treatment A).

Figure 1

Impact of alternative decision frameworks on the health of the population. (a) Absolute impacts: the analysis presents the quality-adjusted life-year (QALY) gain for a patient population of 1,000, based on the implementation of each of the 3 decision frameworks and as a function of the threshold value of a QALY adopted by the decision maker. (b) Net impacts versus current decision framework: the analysis presents the difference in the QALY gain for a patient population of 1,000, when comparing the current decision framework to the other decision frameworks as a function of the threshold value of a QALY adopted by the decision maker.

The necessary price reductions to lead to the conclusion that treatment D should be covered will be the same under both the normative and current decision frameworks. The price reductions required under the positivist decision framework, however, will be either lower than or equal to the price reduction required for the normative and current decision frameworks. Assuming λ = $50,000, a price reduction of 14% for treatment D would be required under both the normative and current decision frameworks, but no price reduction is necessary under the positivist decision framework. For all values of λ less than $58,269, the necessary price reduction under the normative and current decision frameworks is greater than under the positivist decision framework (Figure 2). For values of λ greater than $58,269, no price reduction is required under any of the frameworks.

Figure 2

Impact of alternative decision frameworks on the necessary price reduction to lead to treatment D being covered.

Discussion

This article illustrates alternative decision frameworks that can be adopted with respect to determining the cost-effectiveness of reimbursing new technologies in the presence of covered therapies that are nonoptimal.

In the worked example, the decision frameworks will lead to different estimates of NHB. The implementation of the normative decision framework (i.e., funding only the optimal technology) is shown to lead to the largest NHBs of the 3 frameworks considered, followed by the positivist decision framework and finally the current decision framework. In all situations, this finding will hold, in that implementing the normative decision framework under ideal circumstances (covering only the optimal treatment) will always be as least as good as the positivist or current frameworks. Similarly, the positivist decision framework will always be at least as good as the current framework. If decision makers are willing to defund existing technologies that are nonoptimal, then all frameworks would lead to the same conclusion, and the application of the traditional normative approach to economic evaluation would be appropriate. Failure to choose only to fund the optimal technology, therefore, can lead to substantive reductions in health outcomes.⁵

In the worked example, the substantive opportunity costs of covering a range of technologies, rather than just the optimal technology, is demonstrated. Such reductions in population health due to nonoptimal decision making can be seen as the opportunity cost of choice. There may be differential barriers to treatment that can be based on factors such as level of knowledge, distortion of information, and access barriers, which can be geographical or financial. Thus, treatment options may vary across patients.

Where this is a question of clinical heterogeneity (that is, the treating clinician is able to use their knowledge and expertise to identify the individual patient as a member of a specific subpopulation) or access heterogeneity (that is, the treatment options will vary by subpopulations), these can in principle be handled by repeating the analysis for every conceivable subpopulation. Although this is often done for key subgroups in an economic evaluation (e.g., defining the population in terms of a particular phenotype or previous treatment responses), in practice a fully comprehensive analysis would be time-consuming, expensive, and excessively bureaucratic. However, by allowing the coverage of more than 1 technology, it need not follow that clinicians allocate patients to the treatment option that is optimal for their situation. This has been demonstrated by Kim and Basu,⁹ who have highlighted the need to consider differential adoption rates when assessing the value of a new technology in the presence of heterogeneity.

Our analysis assessed the implications for price reduction scenarios for the new technology on the NHB from each of the reimbursement frameworks. For simplicity, the analysis assumed that market share would not be a function of the price of the new technology. In scenarios in which there may be patient out-of-pocket expenses relating to accessing treatment (e.g., co-pays or deductibles) that may be differential cross treatments, changes in the price of the new technology could also have an impact on the subsequent market share. In scenarios in which this is the case, a more dynamic approach to accounting for market share would be required.

The adoption of the recommended normative analytical approach for economic evaluation in real-world situations in which currently covered technologies will not be defunded (framework 2) can lead to nonoptimal decisions with respect to whether a new technology should be covered. This failure to reflect the current decision-making environment may be limiting the acceptability of economic evaluations within decision-making processes.¹⁴ In such circumstances, the positivist decision framework that considers the adoption rates for all technologies should be preferred to the current decision framework. When nonoptimal treatments are currently covered, identifying the sole optimal treatment option may not be helpful with respect to the particular coverage decision. It may, however, be useful with respect to developing incentives to increase the optimal treatment’s market share, the impacts of which could be explored through a positivist framework.

Finally, the positivist decision framework (framework 3) accepts that a range of technologies may be currently covered and that decision makers may be unwilling to defund existing technologies, while still adopting the normative assumption that the key criterion in the coverage decision is whether it will increase the overall health of the population. However, this can be seen as a second best option in that it still fails to maximize (expected) population health subject to the budget. Furthermore, this approach opens the door to industry gaming over the market shares of competitors (particularly in assumptions as to which competitors the new drug will displace). This reliance on assumptions relating to market share is an obvious weakness of the positivist framework. Simply rejecting the adoption of this decision framework because of this weakness is, however, inappropriate given that the alternative decision frameworks do not adequately address the decision problems at hand. A more nuanced response to this concern is the recognition of the need to consider for whom other fields have addressed the issue of forecasting market shares and, if necessary, develop methods to address the challenges brought on by adequately forecasting market shares.

The differences between the frameworks can be illustrated by a recent CADTH review of romosozumab for the treatment of postmenopausal women with a history of osteoporosis-related fractures.^22,23 The review contained an economic evaluation comparing romosozumab to alendronate, denosumab, raloxifene, risedronate, and zoledronate.²² In the CADTH revised sequential analysis, denosumab, raloxifene, and risedronate were found to be subject to extended dominance or dominance. The ICER for zoledronate versus alendronate was $25,479 and for romosozumab versus zoledronate was $219,799. The analysis concluded that zoledronate was optimal, and for romosozumab to be optimal, a price reduction of 53% was required.

The recommendation from the Canadian Drug Expert Committee was to fund romosozumab on the condition of a 53% price reduction being achieved.²² This would be an appropriate recommendation, if, prior to consideration of romosozumab, only the optimal treatment for the patient population (zoledronate) had been covered. The accompanying budget impact analysis, however, suggested that the therapies currently covered and adopted were alendronate (40%), risedronate (40%), and denosumab (20%), with raloxifene and zoledronate having no market share. Within the budget impact analysis, it was further assumed that romosozumab would take market share only from alendronate and risedronate. If the analysis had adopted the positivist framework, the cost-effectiveness of covering romosozumab versus no coverage would be $123,000, and the necessary price reduction to suggest coverage would be optimal was 39%. Furthermore, if romosozumab took market share from only denosumab, the necessary price reduction would be 31%.

The above example illustrates that decisions based on the positivist decision framework are determined not just by the expected costs and benefits of each treatment option but also by the impact of coverage decisions on market shares for each treatment. In the worked example, covering treatment D is optimal under the positivist framework primarily because it captures most of its market share from treatments B and C. If treatment D had captured more market share from treatment A, the optimal treatment under the normative framework, treatment D would be unlikely to be cost-effective under the positivist framework. Thus, the positivist framework relies on reliable estimates of market share under different reimbursement decisions; otherwise, there is the potential for a high degree of bias.

Other alternative decision frameworks have been suggested. For example, stochastic league tables have been advocated as an alternate approach to considering uncertainty in decision making.^25–27 Under this approach, technologies with the highest probability of inclusion should be covered. This framework was not considered, however, as it has been demonstrated that this framework fails to consider the opportunity costs of obtaining increased health benefits, leading to a decision that may neither maximize net benefits nor fall within the desired budget constraint.^28,29

The sequential approach is the common approach advocated within economic evaluation guidelines. The approach is normative in that it assumes that decision makers will choose only to reimburse the optimal treatment option, given the assumed objective of maximizing the level of health in the population. Thus, analysis focuses on whether a new treatment should be covered as a replacement for the sole optimal treatment that is currently covered. Under this approach, in our worked example, treatment D will not be covered as treatment A is optimal. However, in many circumstances, decision makers chose to fund treatments for a variety of reasons, not just health maximization. In such circumstances, decision makers may be unwilling for several reasons to defund currently provided technologies; thus, a sequential analysis cannot determine if covering the new treatment will increase or decrease the level of health within the population.

Germany has adopted an efficiency frontier approach as an alternative to the traditional sequential analysis.³⁰ Under this approach, a new treatment not on the efficiency frontier would not be covered. If a new treatment falls on the efficiency frontier and its cost is not higher than the highest cost treatment currently covered, it would also be covered. If a new treatment falls on the frontier but its cost is higher than the highest-cost covered treatment under the German approach, the last segment of the existing efficiency frontier is linearly extrapolated, and the new intervention much falls on this, thus assuming the sequential ICER for the new treatment is no higher than the maximum sequential ICER with current treatments. An alternative to the German approach is that the new treatment would be covered only if the ICER versus the current highest-cost covered option is below a decision maker’s willingness to pay.³¹ In the worked example, the new treatment would fall under this latter scenario, and the same coverage decision would hold under both the German and revised approaches. However, in situations in which there are multiple treatments on the frontier that are covered, the positivist framework can lead to different coverage recommendations as the focus would be on from which alternative treatments the treatment will derive market share.

The positivist framework presented within this article relates more to the true decision facing the decision maker: should they fund the new treatment given the distribution of current treatment for this patient population? The positivist framework allows for previous coverage decisions that may not have been consistent with the goal of health maximization. The analysis takes a positive approach by accepting previous decision making; it provides a normative framework to address the actual decision problem facing the decision maker: should they fund the new intervention? Thus, in this context, the revised framework is consistent with the objective of maximizing the level of health within the population, given the possible constraints on the coverage options available to decision makers.

In conclusion, for economic evaluations to have more relevance to the decisions they aim to facilitate, the analysis could be rooted in a framework that recognizes both previous decision making and the constraints on current decision making. The above analysis, however, underlines that the unwillingness of decision makers to defund existing technologies that are nonoptimal leads to a substantive opportunity cost in terms of reductions in the health of the population.

Footnotes

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Doug Coyle

David Glynn

Jeremy D. Goldhaber-Fiebert

References

Weinstein

. Principles of cost-effective resource allocation in health care organizations. Int J Technol Assess Health Care. 1990;6(1):93–103. DOI: 10.1017/S0266462300008953

Karlsson

Johannesson

. The decision rules of cost-effectiveness analysis. Pharmacoeconomics. 1996;9(2):113–20. DOI: 10.2165/00019053-199609020-00003

CADTH. Guidelines for the Economic Evaluation of Health Technologies: Canada (4th Edition). Ottawa (Canada): CADTH; 2017.

Hausman

. The bond between positive and normative economics. Rev Econ Polit. 2018;128(2):191–208. DOI: 10.3917/redp.282.0191

Claxton

Sculpher

Drummond

. A rational framework for decision making by the National Institute For Clinical Excellence (NICE). Lancet. 2002;360(9334):711–5. DOI: 10.1016/S0140-6736(02)09832-X

Weinstein

Zeckhauser

. Critical ratios and efficient allocation. J Public Econ. 1973;2(2):147–57. DOI: 10.1016/0047-2727(73)90002-9

Phelps

. Good technologies gone bad: how and why the cost-effectiveness of a medical intervention changes for different populations. Med Decis Making. 1997;17(1):107–17. DOI: 10.1177/0272989X9701700113

Coyle

Buxton

O’Brien

. Stratified cost-effectiveness analysis: a framework for establishing efficient limited use criteria. Health Econ. 2003;12(5):421–7. DOI: 10.1002/hec.788

Kim

Basu

. New metrics for economic evaluation in the presence of heterogeneity: focusing on evaluating policy alternatives rather than treatment alternatives. Med Decis Making. 2017;37(8):930–41. DOI: 10.1177/0272989X17702379

10.

Sackett

Rosenberg

WMC

Gray

JAM

Haynes

Richardson

. Evidence based medicine: what it is and what it isn’t. BMJ. 1996;312:71–2.

11.

Basu

Meltzer

. Value of information on preference heterogeneity and individualized care. Med Decis Making. 2007;27(2):112–27. DOI: 10.1177/0272989X06297393

12.

National Institute for Health and Care Excellence. NICE Health Technology Evaluations: The Manual. Process and Methods [PMG36]. London: NICE; 2022.

13.

Sanders

Neumann

Basu

, et al. Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: second panel on cost-effectiveness in health and medicine. JAMA. 2016;316(10):1093–103. DOI: 10.1001/jama.2016.12195

14.

Williams

Bryan

. Understanding the limited impact of economic evaluation in health care resource allocation: a conceptual framework. Health Policy (New York). 2007;80(1):135–43. DOI: 10.1016/j.healthpol.2006.03.006

15.

Coyle

Lee

Mamdani

Sabarre

Tingley

. Reimbursement-based economics - what is it and how can we use it to inform drug policy reform? Headache. 2015;55(S4):236–47. DOI: 10.1111/head.12585

16.

Sullivan

Mauskopf

Augustovski

, et al. Budget impact analysis - principles of good practice: report of the ISPOR 2012 budget impact analysis good practice II task force. Value Health. 2014;17(1):5–14. DOI: 10.1016/j.jval.2013.08.2291

17.

Drummond

Sculpher

Claxton

Stoddart

Torrance

. Methods for the Economic Evaluation of Health Care Programmes. 4th ed. Oxford (UK): Oxford University Press; 2015.

18.

Briggs

Claxton

Sculpher

. Decision Modelling For Health Economic Evaluation. Oxford (UK): Oxford University Press; 2006.

19.

National Institute for Health and Care Excellence (NICE). NICE Health Technology Evaluations: The Manual. London: NICE; 2024. Available from: https://www.nice.org.uk/process/pmg36/resources/nice-health-technology-evaluations-the-manual-pdf-72286779244741

20.

Stinnett

Mullahy

. Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Med Decis Making. 1998;18(2 suppl):68–80. DOI: 10.1177/0272989x98018002s09

21.

Paulden

. Calculating and Interpreting ICERs and Net Benefit. Pharmacoeconomics. 2020;38(8):785–807. DOI: 10.1007/s40273-020-00914-6

22.

CADTH. CADTH reimbursement review. Romosozumab (Evenity). Can J Health Technol. 2022;2(1): 1–17.

23.

CADTH. CADTH reimbursement recommendation. romosozumab (evenity). Can J Health Technol. 2021;1(11):1–17.

24.

Briggs

Ades

Price

. Probabilistic sensitivity analysis for decision trees with multiple branches: use of the Dirichlet distribution in a Bayesian framework. Med Decis Making. 2003;23(4):341–50. DOI: 10.1177/0272989X03255922

25.

Hutubessy

RCW

Baltussen

RMPM

Evans

Barendregt

Murray

CJL

. Stochastic league tables: communicating cost-effectiveness results to decision-makers. Health Econ. 2001;10(5):473–7. DOI: 10.1002/hec.614

26.

Hutubessy

RCW

Baltussen

Tan Torres-Edejer

Evans

. WHO-CHOICE: choosing interventions that are cost-effective. Available from: http://books.google.com/books?hl=en&lr=&id=_HloWI6HXbcC&oi=fnd&pg=PR9&dq=edejer+baltussen&ots=hgTnNiuW9x&sig=ffrEjZ6TyLp1EM1y39gYfpVBno8.

27.

Baltussen

RMPM

Hutubessy

RCW

Evans

Murray

CJM

. Uncertainty in cost-effectiveness analysis: probabilistic uncertainty analysis and stochastic league tables. Int J Technol Assess Health Care. 2002;18(1):112–9.

28.

Coyle

. Who’s better not best: appropriate probabilistic uncertainty analysis. Int J Technol Assess Health Care. 2003;19(3):540–5. DOI: 10.1017/S0266462303000485

29.

Coyle

. Determining the optimal combinations of mutually exclusive interventions: a response to Hutubessy and colleagues. Health Econ. 2003;12(2):159–62. DOI: 10.1002/hec.717

30.

Sandmann

Mostardt

Lhachimi

Gerber-Grote

. The efficiency-frontier approach for health economic evaluation versus cost-effectiveness thresholds and internal reference pricing: combining the best of both worlds? Expert Rev Pharmacoecon Outcomes Res. 2018;18(5):475–86. DOI: 10.1080/14737167.2018.1497976

31.

Wilson

ECF

Yiu

ZZN

. Matching drug prices to their clinical benefit—the final frontier? JAMA Dermatol. 2022;160(4):387–8. DOI: 10.1136/bmj.o2163

Decision Frameworks for Assessing Cost-Effectiveness Given Previous Nonoptimal Decisions

Abstract

Introduction

Methods

Results

Conclusion

Highlights

Keywords

Background

Frameworks for Assessing Cost-Effectiveness

Normative Framework

Current Framework

Positivist Framework

Worked Example of Coverage Decisions under the Different Frameworks

Normative Framework

Current Framework

Positivist Framework

Impact of Different Decision Frameworks

Discussion

Footnotes

ORCID iDs

References