Enhancing the Value of the ASCO Value Framework

The past few years have seen the development of several “value frameworks” for assessing the relative merits and prices of cancer drugs, including American Society of Clinical Oncology’s (ASCO) own value framework. ¹ These frameworks can have many different aims and audiences.^2,3 For example they may be designed to support social decisions about value, as in the case of DrugAbacus. ⁴ ASCO’s framework is designed to support individual clinical decisions through comparative assessment of treatment alternatives, contemplated as part of the dialogue between patient and physician.

In this article, we inquire as to whether the ASCO framework can be considered as a valid system for measuring patient value, and thus for supporting decisions. We contrast it with some tools that have been developed by those within and affiliated with the medical decision-making community, with a view to making constructive suggestions for the ongoing development of the framework.

The ASCO Framework as a Tool for Measuring Value

Value frameworks should produce a score that measures value in a similar way to the way in which a thermometer measures temperature. In the ASCO approach, as the thermometer converts the true temperature to a temperature reading, the ASCO framework converts the hazard ratio from a comparative clinical trial to an estimate of a treatment’s value. But using the hazard ratio for this purpose poses a challenge, as the ratio says little or nothing about baseline risk, and therefore nothing about the magnitude of the potential benefit.

Consider the impact of moving from Texas to New York State, focusing on the change in the risk of death from car accidents and lightning strikes. The annual mortality risk from car accidents falls from 13.8 to 5.5 per 100,000⁵; the annual mortality risk from lightning strikes is about 7.1 per 100 million in Texas and 2 per 100 million in New York. ⁶ If one compares the risk reduction, which is analogous to the hazard ratio, the reduction in the risk of lightning strikes is nearly 4/10, and the risk reduction in car accidents is closer to 3/10. This ASCO framework gives 70 points for the lightning benefit, only 60 for the car fatality benefit, slightly favoring the former, even though from a lives saved perspective the car fatality benefit is 1,000 times as large. In other words, the framework produces precisely the reverse of what one would intuitively consider to be more valuable. While this may seem a made-up example, the problem with relying on a relative metric finds resonance in oncology examples—absolute benefits from adjuvant treatment are routinely larger than those from metastatic treatment, while the reverse pattern is often seen in their hazard ratios.

Another feature of the ASCO approach is that a treatment that offers no clinical benefit can actually perform quite well, due to the fact that toxicity is considered separately from the measure of clinical benefit. For instance, an effective but toxic treatment with a hazard ratio of 17/20 compared would earn 15 points on the clinical benefit score. But because having favorable side effects garners independent points, a piece of candy would actually garner more points (20), because it has no side effects, even though it has no benefits either.

Our point is not that patients may have different preferences for life extension versus toxicity—this is recognized by Schnipper and others ¹ —but rather that it seems wrong to assign points for nontoxicity independent of the clinical benefit offered. Comparing cancer treatment to candy is, of course, absurd, but the reality is that there are many treatments in oncology that have no or minimal evidence of clinical benefit. If no clinical benefit is offered, then being nontoxic is not by itself a positive feature. (This argument is developed more fully in Morton ⁷ .)

Some readers may object that the framework is meant to be used for the analysis of cancer drugs, not our cartoon examples. But it is hard to see why we should believe frameworks that do not give the right answer to “no-brainer” questions (would you prefer to reduce by 50% your mortality risk from a major cause of death like car accidents or a minor one like lightning strikes? Would you, as a cancer patient, prefer clinically effective medicine or clinically ineffective candy?) will give the right answer in situations where the answers are less obvious.

Opportunities for Development

Health economics and clinical decision science over the past few decades have developed several tools for value measurement.

In health economics, a core concept is the quality-adjusted life year or QALY.^8,9 This is a measure of clinical benefit that measures time alive adjusted for quality. In Figure 1, for example, we can see and compare two life trajectories, C and T. It can be seen that T gives longer life in better quality than C, and the degree of benefit can be quantified by the dark gray area of the chart between the two curves.

Figure 1

The QALY approach to measuring clinical benefit.

The QALY is not without its controversies, but it does capture what people want their medical treatment to provide, namely, longer life in better health. It explicitly incorporates treatment toxicity to the extent that the toxicity reduces quality of life. In this sense we would argue that it is a better measure of value than the hazard ratio.

A focus of life expectancy and health-related quality of life presupposes that all that matters to the patient in making a decision is health. But qualitative research has identified disease burden, alternative available treatments, quality of evidence, and convenience as factors that are also important when considering treatments. ¹⁰ In this case, a well-established tool that can be used to structure such considerations is multicriteria decision analysis or MCDA.^11,12 The basic idea of MCDA is a simple and familiar one: decision makers can be helped to analyze complex, multi-attributed choices by comparing the options one attribute at a time, assigning scores based on each attribute, and weighting and summing the scores.

The ASCO framework seems to be headed in the direction of MCDA (in particular it is stated on p. 2931 that the intention is to develop a software tool which will permit “weighting by the individual patient”). However, experience with MCDA shows that answering clearly expressed weighting questions in a thoughtful way requires careful upfront investment in defining the criteria and extensive deliberation in response to the questions themselves. Hence, a major focus in MCDA research has been the development of theoretically and psychologically well-grounded ways to ask for weights, and the provision of sensitivity analysis that can allow users to explore their preferences.^13,14

In our view, there is real scope to develop the ASCO value framework to add more value to the patient-physician interaction, drawing on the insights from the health economic and decision science disciplines. For example, one way to take the ASCO framework forward would be to replace the Clinical Benefit dimension with absolute risk reduction, years of life gained or even (if quality of life data are available), quality-adjusted life expectancy gains. Additional considerations (e.g., if the patient feels that it is important that there is treatment-free interval) could be taken into account using MCDA methods.

Conclusion

Organizations such as the ASCO have a critical role in formulating policy around cancer decision making. However, the ASCO value framework feels like it has been built “ground up” from data that are available through clinical trials. Although a natural way to proceed, we believe that a better and more scientific way to approach the development of such a framework is to focus on the decision itself: this means identifying the core construct that one wants to measure and building a framework that operationalizes that core concept. This is likely to revolve around what patients value in terms of their cancer care and treatment, and this would be consistent with a patient-centric approach to assessing value. Although evaluating cancer drugs present distinctive challenges, researchers in health economics and the decision sciences have produced robust and patient-centered frameworks to guide decision making. We believe that as the ASCO framework develops, there is much scope to learn from this wider body of research in the important search for value in cancer care.