This study presents an overview of conceptual and practical issues of a network meta-analysis (NMA), particularly focusing on its application to randomised controlled trials with a binary outcome of interest. We start from general considerations on NMA to specifically appraise how to collect study data, structure the analytical network and specify the requirements for different models and parameter interpretations, with the ultimate goal of providing physicians and clinician-investigators a practical tool to understand pros and cons of NMA. Specifically, we outline the key steps, from the literature search to sensitivity analysis, necessary to perform a valid NMA of binomial data, exploiting Markov Chain Monte Carlo approaches. We also apply this analytical approach to a case study on the beneficial effects of volatile agents compared to total intravenous anaesthetics for surgery to further clarify the statistical details of the models, diagnostics and computations. Finally, datasets and models for the freeware WinBUGS package are presented for the anaesthetic agent example.
GuyattGHSackettDLSinclairJC. Users’ guides to the medical literature. IX. A method for grading health care recommendations. Evidence-based Medicine Working Group. J Am Med Assoc1995; 274: 1800–1804.
2.
SackettDLRosenbergWMGrayJA. Evidence based medicine: what it is and what it isn’t. BMJ1996; 312: 71–72.
3.
BucherHCGuyattGHGriffithLE. The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. J Clin Epidemiol1997; 50: 683–691.
4.
SongFAltmanDGGlennyAM. Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses. BMJ2003; 326: 472–472.
5.
JansenJPFleurenceRDevineB. Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR task force on indirect treatment comparisons good research practices: Part 1. Value Health2011; 14: 417–428.
6.
CaldwellDMAdesAEHigginsJPT. Simultaneous comparison of multiple treatments: combining direct and indirect evidence. BMJ2005; 331: 897–900.
7.
LumleyT. Network meta-analysis for indirect treatment comparisons. Stat Med2002; 21: 2313–2324.
8.
LuGAdesAE. Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med2004; 23: 3105–3124.
9.
SmithTCSpiegelhalterDJThomasA. Bayesian approaches to random-effects meta-analysis: a comparative study. Stat Med1995; 14: 2685–2699.
10.
Spiegelhalter D, Thomas A, Best N, et al. WinBUGS User Manual. Version 1.4, http://www.mrc-bsu.cam.ac.uk/bugs (2003, accessed April 2012).
11.
EggerMSmithGDAltmanDG. Systematic reviews in health care: meta-analysis in context, London: BMJ Publishing Group, 2001, pp. 69–86.
12.
HawkinsNScottDAWoodsB. How far do you go? Efficient searching for indirect evidence. Med Decis Making2009; 29: 273–281.
13.
LuGAdesAE. Assessing evidence inconsistency in mixed treatment comparisons. J Am Stat Assoc2006; 101: 447–459.
14.
SalantiGHigginsJPAdesAE. Evaluation of networks of randomized trials. Stat Meth Med Res2008; 17: 279–301.
15.
SalantiGKavvouraFKIoannidisJP. Exploring the geometry of treatment networks. Ann Intern Med2008; 148: 544–553.
16.
MagurranAE. Ecological diversity and its measurement, Princeton: Princeton University Press, 1998.
17.
HamiltonAJ. Species diversity or biodiversity?J Environ Manage2005; 75: 89–92.
18.
TihoSJosensbG. Co-occurrence of earthworms in urban surroundings: a null model analysis of community structure. Eur J Soil Biol2007; 43: 84–90.
19.
StoneLRobertA. The checkerboard score and species distributions. Oncologia1990; 85: 74–79.
20.
SaltonG. Introduction to modern information retrieval, New York, NY: McGraw-Hill, 1983.
21.
SuttonAAdesAECooperN. Use of indirect and mixed treatment comparisons for technology assessment. Pharmacoeconomics2008; 26: 753–767.
22.
JansenJPCrawfordBBergmanG. Bayesian meta-analysis of multiple treatment comparisons: an introduction to mixed treatment comparisons. Value Health2008; 11: 956–964.
23.
HigginsJPTThompsonSG. Quantifying heterogeneity in a meta-analysis. Stat Med2002; 21: 1539–1558.
24.
BrockwellSEGordonIR. A comparison of statistical methods for meta-analysis. Stat Med2001; 20: 825–840.
25.
KontopantelisEReevesD. Performance of statistical methods for meta-analysis when true study effects are non-normally distributed: a simulation study. Stat Meth Med Res2012; 21: 409–426.
26.
KontopantelisEReevesD. Performance of statistical methods for meta-analysis when true study effects are non-normally distributed: a comparison between DerSimonian-Laird and restricted maximum likelihood. Stat Meth Med Res2012; 21: 657–659.
27.
HigginsJPWhiteheadA. Borrowing strength from external trials in a meta-analysis. Stat Med1996; 15: 2733–2749.
28.
TurnerRMDaveyJClarkeMJ. Predicting the extent of heterogeneity in meta-analysis, using empirical data from the Cochrane database of systematic reviews. Int J Epidemiol2012; 41: 818–827.
29.
SongFLokeYKWalshT. Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews. BMJ2009; 3: b1147–b1147.
30.
DoneganSWilliamsonPGambleC. Indirect comparisons: a review of reporting and methodological quality. PLoS One2010; 5: e11054–e11054.
31.
HardyRJThompsonSG. Detecting and describing heterogeneity in meta-analysis. Stat Med1998; 17: 841–856.
32.
HigginsJPTThompsonSGDeeksJJ. Measuring inconsistency in meta-analyses. BMJ2003; 327: 557–560.
33.
Higgins JPT and Green S. Cochrane handbook for systematic reviews of interventions. Version 5.1.0. The Cochrane collaboration, http://www.cochrane-handbook.org/ (2011, accessed April 2012).
34.
MittlböckMHeinzlH. A simulation study comparing properties of heterogeneity measures in meta-analyses. Stat Med2006; 25: 4321–4333.
35.
SpiegelhalterDJAbramsKRMylesJP. Bayesian approaches to clinical trials and health-care evaluation, New York, NY: Wiley, 2004.
36.
CooperNJSuttonAJMorrisD. Addressing between-study heterogeneity and inconsistency in mixed treatment comparisons: application to stroke prevention treatments in individuals with non-rheumatic atrial fibrillation. Stat Med2009; 28: 1861–1881.
37.
CaldwellDMWeltonNJAdesAE. Mixed treatment comparison analysis provides internally coherent treatment effect estimates based on overviews of reviews and can reveal inconsistency. J Clin Epidemiol2010; 63: 875–882.
38.
Dias S, Welton WJ, Sutton AJ, et al. NICE DSU Technical Support Document 4: inconsistency in networks of evidence based on randomised controlled trial, http://www.nicedsu.org.uk (2011, accessed March 2012).
39.
FranchiniAJDiasSAdesAE. Accounting for correlation in network meta-analysis with multi-arm trials. Res Synth Meth2012; 3: 142–160.
40.
Dias S, Welton NJ, Sutton AJ, et al. NICE DSU Technical Support Document 2: a generalised linear modelling framework for pairwise and network meta-analysis of randomised controlled trials, http://www.nicedsu.org.uk (2011, accessed March 2012).
41.
HoaglinDCHawkinsNJansenJP. Conducting indirect-treatment-comparison and network-meta-analysis studies: report of the ISPOR Task Force on indirect treatment comparisons good research practices: Part 2. Value Health2011; 14: 429–437.
42.
Dias S, Sutton AJ, Welton WJ, et al. NICE DSU technical support document 3: heterogeneity: subgroups, meta-regression, bias and bias-adjustment, http://www.nicedsu.org.uk (accessed March 2012).
43.
NixonRMBansbackNBrennanA. Using mixed treatment comparisons and meta-regression to perform indirect comparisons to estimate the efficacy of biologic treatments in rheumatoid arthritis. Stat Med2007; 26: 1237–1254.
LambertPCSuttonAJAbramsKR. A comparison of summary patient-level covariates in meta-regression with individual patient data meta-analysis. J Clin Epidemiol2002; 55: 86–94.
46.
BerlinJASantannaJSchmidCH. Anti-Lymphocyte Antibody Induction Therapy Study Group. Individual patient- versus group-level data meta-regressions for the investigation of treatment effect modifiers: ecological bias rears its ugly head. Stat Med2002; 21: 371–387.
47.
GreenlandSMorgensternH. Ecological bias, confounding, and effect modification. Int J Epidemiol1989; 18: 269–274.
48.
NtzoufrasI. Bayesian modeling using WinBUGS, New York, NY: Wiley, 2009.
49.
Adamakis S, Raftery CL, Walsh RW, et al. A Bayesian approach to comparing theoretic models to observational data: a case study from solar flare physics. astro-ph.SR 2012; arXiv:1102.0242v3, http://arxiv.org/abs/1102.0242 (accessed November 2012).
50.
Van DongenS. Prior specification in Bayesian statistics: three cautionary tales. J Theor Biol2006; 242: 90–100.
51.
GelmanA. Prior distributions for variance parameters in hierarchical models. Bayesian Anal2006; 1: 515–533.
52.
CrowderMJ. Maximum likelihood estimation for dependent observations. J Royal Stat Soc1976; B38: 45–53.
53.
GouriérouxCHollyAMonfortA. Likelihood Ratio Test, Wald Test, and Kuhn-Tucker Test in linear models with inequality constraints on the regression parameters. Econometrica1982; 50: 63–80.
54.
AkaikeHInformation theory as an extension of the maximum likelihood principle. In: PetrovBNCasaki (eds). Second international symposium on information, Budapest: Akademiai Kiado, 1973, pp. 267–281.
55.
SchwarzG. Estimating the dimension of a model. Ann Stat1978; 6: 461–464.
56.
Spiegelhalter DJ, Best NG and Carlin BP. Bayesian deviance, the effective number of parameters, and the comparison of arbitrarily complex models. Technical Report, MRC Biostatistics Unit, Cambridge, UK, 1998, http://yaroslavvb.com/papers/spiegelhalter-bayesian.pdf (accessed March 2012).
57.
DempsterAP. The direct use of likelihood for significance testing. Stat Comput1997; 7: 247–252.
58.
BergAMeyerRYuJ. Deviance information criterion for comparing stochastic volatility models. J Bus Econ Stat2004; 22: 107–120.
59.
SpiegelhalterDJBest ngCarlinBP. Bayesian measures of model complexity and fit. J Royal Stat Soc Ser B2002; 64: 583–639.
60.
YuanYJohnsonVE. Goodness-of-fit diagnostics for Bayesian hierarchical models. Biometrics2012; 68: 156–164.
61.
KassRERafteryAE. Bayes factors. J Am Stat Assoc1995; 90: 773–795.
62.
SalantiGAdesAEIoannidisJP. Graphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: an overview and tutorial. J Clin Epidemiol2011; 64: 163–171.
63.
LandoniGRodsethRSantiniF. Randomized evidence for reduction in perioperative mortality. JCVA2012; 26: 764–772.
64.
AdesAESculpherMSuttonA. Bayesian methods for evidence synthesis in cost-effectiveness analysis. Pharmacoeconomics2006; 24: 1–19.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
