Background. Although value of information (VOI) analyses are increasingly advocated and used for research prioritization and reimbursement decisions, the interpretation and usefulness of VOI outcomes depend critically on the underlying choices and assumptions used in the analysis. In this article, we present a structured overview of all items reported in literature to potentially influence VOI outcomes. Use of this overview increases awareness and transparency of choices and assumptions underpinning VOI outcomes. Methods. A systematic literature review was performed to identify aspects of VOI analyses that were found to potentially influence VOI outcomes. Identified aspects were grouped to develop a structured overview. Explanations were defined for all items included in the overview. Results. We retrieved 687 unique papers, of which 71 original papers and 8 reviews were included. In the full text of these 79 papers, 16 aspects were found that may influence VOI outcomes. These aspects related to the underlying evidence (bias, synthesis, heterogeneity, correlation), uncertainty (structural, future pricing), model (relevance, approach, population), choices in VOI calculation (estimation technique, implementation level, population size, perspective), and aspects specifically for assessing the value of future study designs (reversal costs, efficient estimator). These aspects were aggregated into 7 items to provide a structured overview. Conclusion. The developed overview should increase awareness of key choices underlying VOI analysis and facilitate structured reporting of such choices and interpretation of the ensuing VOI outcomes by researchers and policy makers. Use of this overview should improve prioritization and reimbursement decisions.
CaroJJBriggsAHSiebertUKuntzKMForce I-SMGRPT. Modeling good research practices—overview: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1. Med Decis Making. 2012;32(5):667–77.
2.
ClaxtonKSculpherMDrummondM.A rational framework for decision making by the National Institute for Clinical Excellence (NICE). Lancet. 2002;360(9334):711–5.
3.
McKennaCSoaresMClaxtonKet al. Unifying research and reimbursement decisions: case studies demonstrating the sequence of assessment and judgments required. Value Health. 2015;18(6):865–75.
4.
MinelliCBaioG.Value of information: a tool to improve research prioritization and reduce waste. PLoS Med. 2015;12(9):e1001882.
5.
ContiSClaxtonK.Dimensions of design space: a decision-theoretic approach to optimal research design. Med Decis Making. 2009;29(6):643–60.
6.
GriffinSWeltonNJClaxtonK.Exploring the research decision space: the expected value of information for sequential research designs. Med Decis Making. 2010;30(2):155–62.
7.
BriggsAHWeinsteinMCFenwickEAet al. Model parameter estimation and uncertainty analysis: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-6. Med Decis Making. 2012;32(5):722–32.
8.
SteutenLvan de WeteringGGroothuis-OudshoornKRetelV.A systematic and critical review of the evolving methods and applications of value of information in academia and practice. Pharmacoeconomics. 2013;31(1):25–48.
9.
ThornJCoastJAndronisL.Interpretation of the expected value of perfect information and research recommendations: a systematic review and empirical investigation. Med Decis Making. 2016;36(3):285–95.
10.
Zorginstituut Nederland. Guideline for Performing Economic Evaluations in Health Care [in Dutch]. Diemen: Zorginstituut Nederland, 2015.
BairdLGBankenREichlerHGet al. Accelerated access to innovative medicines for patients in need. Clin Pharmacol Ther. 2014;96(5):559–71.
13.
GruttersJPvan AsseltMBChalkidouKJooreMA.Healthy decisions: towards uncertainty tolerance in healthcare policy. Pharmacoeconomics. 2015;33(1):1–4.
14.
MohiuddinSFenwickEPayneK.Use of value of information in UK health technology assessments. Int J Technol Assess Health Care. 2014;30(6):553–70.
15.
YokotaFThompsonKM.Value of information literature analysis: a review of applications in health risk management. Med Decis Making. 2004;24(3):287–98.
16.
TuffahaHWGordonLGScuffhamPA.Value of information analysis in healthcare: a review of principles and applications. J Med Econ. 2014;17(6):377–83.
17.
MyersESandersGDRaviDet al. Evaluating the Potential Use of Modeling and Value-of-Information Analysis for Future Research Prioritization within the Evidence-Based Practice Center Program: AHRQ Methods for Effective Health Care. Rockville, MD: Agency for Healthcare Research and Quality; 2011.
18.
ChilcottJBrennanABoothAKarnonJTappendenP.The role of modelling in prioritising and planning clinical trials. Health Technol Assess. 2003;7(23):iii, 1–125.
19.
HeathAManolopoulouIBaioG.A review of methods for analysis of the expected value of information [published online April14, 2017]. Med Decis Making.
20.
JaimeCaro JEddyDMKanHet al. Questionnaire to assess relevance and credibility of modeling studies for informing health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report. Value Health. 2014;17(2):174–82.
21.
VemerPCorro RamosIvan VoornGAAlMJFeenstraTL.AdViSHE: a validation-assessment tool of health-economic models for decision makers and model users. Pharmacoeconomics. 2016;34(4):349–61.
22.
HusereauDDrummondMPetrouSet al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement. Value Health. 2013;16(2):e1–5.
23.
WeltonNJSoaresMOPalmerSet al. Accounting for heterogeneity in relative treatment effects for use in cost-effectiveness models and value-of-information analyses. Med Decis Making. 2015;35(5):608–21.
24.
WeltonNAdesAE.Research decisions in the face of heterogeneity: what can a new study tell us?Health Econ. 2012;21(10):1196–200.
25.
WillanAREckermannS.Accounting for between-study variation in incremental net benefit in value of information methodology. Health Econ. 2012;21(10):1183–95.
26.
BindelsJRamaekersBRamosICet al. Use of value of information in healthcare decision making: exploring multiple perspectives. Pharmacoeconomics. 2016;34(3):315–22.
27.
PetersJLAndersonRHoyleMHydeC.Evolution of a cost-utility model of donepezil for Alzheimer’s disease. Int J Technol Assess Health Care. 2013;29(2):147–54.
28.
PriceMJWeltonNJBriggsAHAdesAE.Model averaging in the presence of structural uncertainty about treatment effects: influence on treatment decision and expected value of information. Value Health. 2011;14(2):205–18.
29.
Corro RamosIRutten-van MolkenMHMAlMJ. Determining the impact of modeling additional sources of uncertainty in value-of-information analysis. Value Health. 2015;18(1):100–9.
30.
BreezePBrennanA.Valuing trial designs from a pharmaceutical perspective using value-based pricing. Health Econ. 2015;24(11):1468–82.
31.
GarrisonLPJrTowseABriggsAet al. Performance-based risk-sharing arrangements-good practices for design, implementation, and evaluation: report of the ISPOR good practices for performance-based risk-sharing arrangements task force. Value Health. 2013;16(5):703–19.
32.
McCabeCChilcottJClaxtonKet al. Continuing the multiple sclerosis risk sharing scheme is unjustified. BMJ. 2010;340:c1786.
33.
BrennanAKharroubiSO’HaganAChilcottJ.Calculating partial expected value of perfect information via Monte Carlo sampling algorithms. Med Decis Making. 2007;27(4):448–70.
34.
CoyleDOakleyJ.Estimating the expected value of partial perfect information: a review of methods. Eur J Health Econ. 2008;9(3):251–9.
35.
Groot KoerkampBMyriam HuninkMGStijnenTWeinsteinMC.Identifying key parameters in cost-effectiveness analysis using value of information: a comparison of methods. Health Econ. 2006;15(4):383–92.
36.
JalalHGoldhaber-FiebertJDKuntzKM.Computing expected value of partial sample information from probabilistic sensitivity analysis using linear regression metamodeling. Med Decis Making. 2015;35(5):584–95.
37.
MadanJAdesAEPriceMet al. Strategies for efficient computation of the expected value of partial perfect information. Med Decis Making. 2014;34(3):327–42.
38.
OakleyJEBrennanATappendenPChilcottJ.Simulation sample sizes for Monte Carlo partial EVPI calculations. J Health Econ. 2010;29(3):468–77.
39.
RojnikKNaversnikK.Gaussian process metamodeling in Bayesian value of information analysis: a case of the complex health economic model for breast cancer screening. Value Health. 2008;11(2):240–50.
40.
StrongMOakleyJE.An efficient method for computing single-parameter partial expected value of perfect information. Med Decis Making. 2013;33(6):755–66.
41.
StrongMOakleyJEBrennanA.Estimating multiparameter partial expected value of perfect information from a probabilistic sensitivity analysis sample: a nonparametric regression approach. Med Decis Making. 2014;34(3):311–26.
42.
TappendenPChilcottJBEggingtonSOakleyJMcCabeC.Methods for expected value of information analysis in complex health economic models: developments on the health economics of interferon-beta and glatiramer acetate for multiple sclerosis. Health Technol Assess. 2004;8(27):iii, 1–78.
43.
SadatsafaviMBansbackNZafariZNajafzadehMMarraC.Need for speed: an efficient algorithm for calculation of single-parameter expected value of partial perfect information. Value Health. 2013;16(2):438–48.
44.
BrennanAKharroubiSA.Efficient computation of partial expected value of sample information using Bayesian approximation. J Health Econ. 2007;26(1):122–48.
45.
ClaxtonKThompsonKM.A dynamic programming approach to the efficient design of clinical trials. J Health Econ. 2001;20(5):797–822.
46.
KharroubiSABrennanAStrongM.Estimating expected value of sample information for incomplete data models using Bayesian approximation. Med Decis Making. 2011;31(6):839–52.
47.
MaroufyVMarriottPPezeshkH.An optimization approach to calculating sample sizes with binary responses. J Biopharm Stat. 2014;24(4):715–31.
48.
MenziesNA.An efficient estimator for the expected value of sample information [published online April24, 2015]. Med Decis Making.
49.
StrongMOakleyJEBrennanABreezeP.Estimating the expected value of sample information using the probabilistic sensitivity analysis sample: a fast, nonparametric regression-based method. Med Decis Making. 2015;35(5):570–83.
50.
WeltonNJMadanJJCaldwellDMPetersTJAdesAE.Expected value of sample information for multi-arm cluster randomized trials with binary outcomes. Med Decis Making. 2014;34(3):352–65.
51.
TuffahaHWStrongMGordonLGScuffhamPA.Efficient value of information calculation using a nonparametric regression approach: an applied perspective. Value Health. 2016;19(4):505–9.
52.
HeathAManolopoulouIBaioG.Estimating the expected value of partial perfect information in health economic evaluations using integrated nested Laplace approximation. Stat Med. 2016;35(23):4264–80.
53.
BrennanAKharroubiSA.Expected value of sample information for Weibull survival data. Health Econ. 2007;16(11):1205–25.
54.
StrongMBrennanAOakleyJ.How to calculate value of information in seconds using ‘Savi,’ the Sheffield Accelerated Value of Information web app. Value Health. 2015;18(7):A725–6.
55.
AndronisLBartonP.Adjusting estimates of the expected value of information for implementation: theoretical framework and practical application. Med Decis Making. 2016;36(3):296–307.
56.
EckermannSKarnonJWillanAR.The value of value of information: best informing research design and prioritization using current methods. Pharmacoeconomics. 2010;28(9):699–709.
57.
FenwickEClaxtonKSculpherM.The value of implementation and the value of information: combined and uneven development. Med Decis Making. 2008;28(1):21–32.
58.
HoomansTFenwickEAPalmerSClaxtonK.Value of information and value of implementation: application of an analytic framework to inform resource allocation decisions in metastatic hormone-refractory prostate cancer. Value Health. 2009;12(2):315–24.
59.
KentSBriggsAEckermannSBerryC.Are value of information methods ready for prime time? An application to alternative treatment strategies for NSTEMI patients. Int J Technol Assess Health Care. 2013;29(4):435–42.
60.
McKennaCGriffinSKoffijbergHClaxtonK.Methods to place a value on additional evidence are illustrated using a case study of corticosteroids after traumatic brain injury. J Clin Epidemiol. 2016;70:183–90.
61.
WillanAREckermannS.Optimal clinical trial design using value of information methods with imperfect implementation. Health Econ. 2010;19(5):549–61.
62.
GrimmSEDixonSStevensJW.When future change matters: modeling future price and diffusion in health technology assessments of medical devices. Value Health. 2016;19(6):720–6.
63.
GrimmSEDixonSStevensJW.Assessing the expected value of research studies in reducing uncertainty and improving implementation dynamics. Med Decis Making. 2017;37(5):523–33.
64.
ShekellePEcclesMPGrimshawJMWoolfSH.When should clinical guidelines be updated?BMJ. 2001;323(7305):155–7.
65.
EspinozaMAMancaAClaxtonKSculpherMJ.The value of heterogeneity for cost-effectiveness subgroup analysis: conceptual framework and application. Med Decis Making. 2014;34(8):951–64.
66.
AndronisL.Analytic approaches for research priority-setting: issues, challenges and the way forward. Expert Rev Pharmacoecon Outcomes Res. 2015;15(5):745–54.
67.
HoyleM.Historical lifetimes of drugs in England: application to value of information and cost-effectiveness analyses. Value Health. 2010;13(8):885–92.
68.
PhilipsZClaxtonKPalmerS.The half-life of truth: what are appropriate time horizons for research decisions?Med Decis Making. 2008;28(3):287–99.
69.
AdesAELuGClaxtonK.Expected value of sample information calculations in medical decision modeling. Med Decis Making. 2004;24(2):207–27.
70.
EckermannSWillanAR.Globally optimal trial design for local decision making. Health Econ. 2009;18(2):203–16.
71.
SussexJTowseADevlinN.Operationalizing value-based pricing of medicines: a taxonomy of approaches. Pharmacoeconomics. 2013;31(1):1–10.
72.
WillanAR.Optimal sample size determinations from an industry perspective based on the expected value of information. Clin Trials. 2008;5(6):587–94.
73.
WillanAKowgierM.Determining optimal sample sizes for multi-stage randomized clinical trials using value of information methods. Clin Trials. 2008;5(4):289–300.
74.
ChenMHWillanAR.Determining optimal sample sizes for multistage adaptive randomized clinical trials from an industry perspective using value of information methods. Clin Trials. 2013;10(1):54–62.
75.
EckermannSWillanAR.Optimal global value of information trials: better aligning manufacturer and decision maker interests and enabling feasible risk sharing. Pharmacoeconomics. 2013;31(5):393–401.
76.
EckermannSWillanAR.Expected value of information and decision making in HTA. Health Econ. 2007;16(2):195–209.
77.
McKennaCClaxtonK.Addressing adoption and research design decisions simultaneously: the role of value of sample information analysis. Med Decis Making. 2011;31(6):853–65.
78.
HallPSEdlinRKharroubiSGregoryWMcCabeC.Expected net present value of sample information: from burden to investment. Med Decis Making. 2012;32(3):E11–21.
79.
WilsonECMugfordMBartonGShepstoneL.Efficient research design: using value-of-information analysis to estimate the optimal mix of top-down and bottom-up costing approaches in an economic evaluation alongside a clinical trial. Med Decis Making. 2016;36(3):335–48.
ClaxtonK.Pharmaceutical Pricing: Early Access, the Cancer Drugs Fund and the Role of NICE. York, UK: Centre of Health Economics, University of York; 2016.
82.
GrieveRAbramsKClaxtonKet al. Cancer Drugs Fund requires further reform. BMJ. 2016;354:i5090.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
