Multilevel models provide a flexible modelling framework for cost-effectiveness analyses that use cluster randomised trial data. However, there is a lack of guidance on how to choose the most appropriate multilevel models. This paper illustrates an approach for deciding what level of model complexity is warranted; in particular how best to accommodate complex variance–covariance structures, right-skewed costs and missing data. Our proposed models differ according to whether or not they allow individual-level variances and correlations to differ across treatment arms or clusters and by the assumed cost distribution (Normal, Gamma, Inverse Gaussian). The models are fitted by Markov chain Monte Carlo methods. Our approach to model choice is based on four main criteria: the characteristics of the data, model pre-specification informed by the previous literature, diagnostic plots and assessment of model appropriateness. This is illustrated by re-analysing a previous cost-effectiveness analysis that uses data from a cluster randomised trial. We find that the most useful criterion for model choice was the deviance information criterion, which distinguishes amongst models with alternative variance–covariance structures, as well as between those with different cost distributions. This strategy for model choice can help cost-effectiveness analyses provide reliable inferences for policy-making when using cluster trials, including those with missing data.
NICE. Methods for technology appraisal, London, UK: National Institute for Health and Clinical Excellence, 2008.
2.
CADTH. Guidelines for the economic evaluation of health technologies: Canada, 3rd ed. Ottawa, Canada: Canadian Agency for Drugs and Technologies in Health, 2006.
3.
PBCA. Guidelines for preparing submissions to the Pharmaceutical Benefits Advisory Committee, Canberra, Australia: Australian Government – Department of Health and Ageing, 2008.
4.
IQWIG. Methods for assessment of the relation of benefits to costs in the German Statutory Health Care System, Cologne, Germany: Institute for Quality and Efficiency in Health Care, 2009.
5.
BaioGDawidAP. Probabilistic sensitivity analysis in health economics. Stat Methods Med Res. Epub ahead of print 18 September 2011. Doi: 10.1177/0962280211419832.
6.
GomesMGrieveRNixonR. Statistical methods for cost-effectiveness analyses that use data from cluster randomized trials: A systematic review and checklist for critical appraisal. Med Decis Making2012; 32(1): 209–220.
7.
WillanARBriggsAHHochJS. Regression methods for covariate adjustment and subgroup analysis for non-censored cost-effectiveness data. Health Econ2004; 13(5): 461–475.
8.
BriggsAHO’BrienBJ. The death of cost-minimization analysis?Health Econ2001; 10(2): 179–184.
9.
HochJSBriggsAHWillanAR. Something old, something new, something borrowed, something blue: A framework for the marriage of health econometrics and cost-effectiveness analysis. Health Econ2002; 11(5): 415–430.
10.
NixonRMThompsonSG. Methods for incorporating covariate adjustment, subgroup analysis and between-centre differences into cost-effectiveness evaluations. Health Econ2005; 14(12): 1217–1229.
11.
MihaylovaBBriggsAO’HaganA. Review of statistical methods for analysing healthcare resources and costs. Health Econ2011; 20(8): 897–916.
12.
NixonRMThompsonSG. Parametric modelling of cost data in medical studies. Stat Med2004; 23(8): 1311–1331.
13.
ThompsonSGNixonRM. How sensitive are cost-effectiveness analyses to choice of parametric distributions?Med Decis Making2005; 25(4): 416–423.
14.
GrieveRNixonRThompsonSG. Bayesian hierarchical models for cost-effectiveness analyses that use data from cluster randomized trials. Med Decis Making2010; 30(2): 163–175.
15.
WillanARBriggsA. Statistical analysis of cost-effectiveness data, Chichester: Wiley-Blackwell, 2006, pp. 210–210.
16.
CeleuxGForbesFRobertCP. Deviance information criteria for missing data models. Bayesian Anal2006; 1(4): 651–673.
17.
GrieveRNixonRThompsonSG. Using multilevel models for assessing the variability of multinational resource use and cost data. Health Econ2005; 14(2): 185–196.
18.
ThompsonSGNixonRMGrieveR. Addressing the issues that arise in analysing multicentre cost data, with application to a multinational study. J Health Econ2006; 25(6): 1015–1028.
19.
BriggsANixonRDixonS. Parametric modelling of cost data: Some simulation evidence. Health Econ2005; 14(4): 421–428.
20.
O’HaganAStevensJW. Assessing and comparing costs: How robust are the bootstrap and methods based on asymptotic normality?Health Econ2003; 12(1): 33–49.
21.
ThompsonSGBarberJA. How should cost data in pragmatic randomised trials be analysed?BMJ2000; 320(7243): 1197–1200.
22.
BarberJThompsonS. Multiple regression of cost data: use of generalised linear models. J Health Serv Res Policy2004; 9(4): 197–204.
23.
SpiegelhalterDJBestNGCarlinBP. Bayesian measures of model complexity and fit. J Roy Stat Soc Ser B Stat Methodol2002; 64: 583–616.
24.
De BackerGAmbrosioniEBorch-JohnsenK. European guidelines on cardiovascular disease prevention in clinical practice – Third Joint Task Force of European and other Societies on Cardiovascular Disease Prevention in Clinical Practice. Eur Heart J2003; 24(17): 1601–1610.
25.
GillespiePO’SheaEMurphyAW. The cost-effectiveness of the SPHERE intervention for the secondary prevention of coronary heart disease. Int J Technol Assess Health Care2010; 26(3): 263–271.
26.
BrazierJRobertsJDeverillM. The estimation of a preference-based measure of health from the SF-36. J Health Econ2002; 21(2): 271–292.
27.
StinnettAAMullahyJ. Net health benefits: A new framework for the analysis of uncertainty in cost-effectiveness analysis. Med Decis Making1998; 18(2): S68–S80.
28.
NixonRMWonderlingDGrieveRD. Non-parametric methods for cost-effectiveness analysis: The central limit theorem and the bootstrap compared. Health Econ2010; 19(3): 316–333.
29.
HigginsJPThompsonSGDeeksJJ. Measuring inconsistency in meta-analyses. BMJ2003; 327(7414): 557–560.
30.
HarrisRBradburnMDeeksJ. Metan: Fixed- and random-effects meta-analysis. Stat J2008; 8(1): 3–28.
31.
HigginsJPTThompsonSG. Quantifying heterogeneity in a meta-analysis. Stat Med2002; 21(11): 1539–1558.
32.
LeveneH. Robust tests for equality of variances, Palo Alto, CA: Stanford University Press, 1960.
33.
BlandJM. An introduction to medical statistics, 3rd ed. Oxford: Oxford University Press, 2000.
34.
MorrellCJSladePWarnerR. Clinical effectiveness of health visitor training in psychologically informed approaches for depression in postnatal women: Pragmatic cluster randomised trial in primary care. BMJ2009; 338: a3045–a3045.
35.
GomesMNgESGrieveR. Developing appropriate methods for cost-effectiveness analysis of cluster randomized trials. Med Decis Making2012; 32(2): 350–361.
36.
BachmannMOFairallLClarkA. Methods for analyzing cost effectiveness data from cluster randomized trials. Cost Eff Resour Alloc2007; 5: 12–12.
37.
ManningWGMullahyJ. Estimating log models: to transform or not to transform?J Health Econ2001; 20(4): 461–494.
38.
ManningWGBasuAMullahyJ. Generalized modeling approaches to risk adjustment of skewed outcomes data. J Health Econ2005; 24(3): 465–488.
39.
McCullaghPNelderJA. Generalized linear models, 2nd ed. London: Chapman and Hall/CRC, 1989, pp. 532–532.
40.
MoranJLSolomonPJPeisachAR. New models for old questions: Generalized linear models for cost prediction. J Eval Clin Pract2007; 13(3): 381–389.
LunnDJThomasABestN. WinBUGS – A Bayesian modelling framework: Concepts, structure, and extensibility. Stat Comput2000; 10(4): 325–337.
43.
BrooksSPGelmanA. General methods for monitoring convergence of iterative simulations. J Comput Graph Stat1998; 7(4): 434–455.
44.
NtzoufrasI. Models for positive continuous data, count data, and other GLM-based extensions. Bayesian modeling using WinBUGS, Hoboken, NJ, USA: John Wiley & Sons, Inc, 2008, pp. 275–304.
45.
GelmanA. Prior distributions for variance parameters in hierarchical models (comment on an article by Browne and Draper). Bayesian Anal2006; 1(3): 515–533.
46.
SpiegelhalterDJBestNGCarlinBP. Bayesian measures of model complexity and fit. J Roy Stat Soc Ser B Stat Methodol2002; 64(4): 583–639.
47.
Information theory and an extension of the maximum likelihood principle. In: AkaikeH (eds). Second international symposium on information theory, Budapest: Akademiai Kiado, 1973.
48.
WardEJ. A review and comparison of four commonly used Bayesian and maximum likelihood model selection tools. Ecol Model2008; 211(1–2): 1–10.
49.
SchwarzG. Estimating dimension of a model. Ann Stat1978; 6(2): 461–464.
50.
MasonARichardsonSBestN. Two-pronged strategy for using DIC to compare selection models with non-ignorable missing responses. Bayesian Anal2012; 7(1): 109–146.
51.
Kumar V, Ligges U and Thomas A. ReliaBUGS User Manual: A subsystem in OpenBUGS for some statistical models, Version 1.0, OpenBUGS 3.2.1, 2010, http://openbugs.info/w/Downloads/.
52.
KassRERafteryAE. Bayes factors. J Am Stat Assoc1995; 90(430): 773–795.
53.
GeisserSEddyWF. Predictive approach to model selection. J Am Stat Assoc1979; 74(365): 153–160.
54.
StoneM. Cross-validatory choice and assessment of statistical predictions. J Roy Stat Soc Ser B Stat Methodol1974; 36(2): 111–147.
55.
JacksonCHSharplesLDThompsonSG. Structural and parameter uncertainty in Bayesian cost-effectiveness models. J Roy Stat Soc Ser C Appl Stat2010; 59: 233–253.
56.
MancaASculpherMJGoereeR. The analysis of multinational cost-effectiveness data for reimbursement decisions a critical appraisal of recent methodological developments. Pharmacoeconomics2010; 28(12): 1079–1096.
57.
LambertPCSuttonAJBurtonPR. How vague is vague? A simulation study of the impact of the use of vague prior distributions in MCMC using WinBUGS. Stat Med2005; 24(15): 2401–2428.
58.
TurnerRMOmarRZThompsonSG. Modelling multivariate outcomes in hierarchical data, with application to cluster randomised trials. Biometr J2006; 48(3): 333–345.
59.
NeuhausJMHauckWWKalbfleischJD. The effects of mixture distribution misspecification when fitting mixed-effects logistic-models. Biometrika1992; 79(4): 755–762.
60.
JonesAMLomasJRiceN. Applying beta-type size distributions to healthcare cost regressions. J Appl Econometr. Epub ahead of print 8 July 2013. DOI: 10.1002/jae.2334.
