Network Meta-Analysis for Single-Case Design Studies: An Illustration

Abstract

Single-case designs (SCDs) are used to evaluate the effects of interventions on individual participants. By repeatedly measuring participants under different conditions, SCD studies focus on individual effects rather than on group summaries. The main limitation of SCDs remains its generalisability to wider populations, reducing the relevance of their findings for practice and policy making. With this limitation in mind, methodological developments for synthesising SCD data from different studies that investigate the same research question have intensified in the past decades (e.g. multilevel modelling). However, these techniques are restricted to comparing two interventions at a time and can only incorporate evidence from studies that directly compare the two treatments of interest. These limitations could be addressed by using network meta-analysis that incorporates both direct and indirect evidence to simultaneously compare multiple interventions. Despite its potential, network meta-analytical techniques have yet to be applied to SCD data. Thus, in this paper, we argue that network meta-analysis can be a valuable tool to synthesise SCD data. We demonstrate the use of network meta-analysis in SCD data using a real dataset, and we conclude by reflecting on the challenges that SCD researchers might face when applying network meta-analysis methods to their data.

Keywords

network meta-analysis single-case design n-of-1 trial research synthesis research methods

Get full access to this article

View all access options for this article.

References

Beretvas

S. N.

Chung

(2008). A review of meta-analyses of single-subject experimental designs: Methodological issues and practice. Evidence-Based Communication Assessment and Intervention, 2(3), 129–141. https://doi.org/10/bfjmtk

Boyd

B. A.

Odom

S. L.

Humphreys

B. P.

Sam

A. M.

(2010). Infants and toddlers with autism spectrum disorder: Early identification and early intervention. Journal of Early Intervention, 32(2), 75–98. https://doi.org/10/c98b66

Bucher

H. C.

Guyatt

G. H.

Griffith

L. E.

Walter

S. D.

(1997). The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. Journal of Clinical Epidemiology, 50(6), 683–691. https://doi.org/10/b4qf2t

Catalá-López

Tobías

Cameron

Moher

Hutton

(2014). Network meta-analysis for comparing treatment effects of multiple interventions: An introduction. Rheumatology International, 34(11), 1489–1496. https://doi.org/10.1007/s00296-014-2994-2

Chaimani

Salanti

(2012). Using network meta-analysis to evaluate the existence of small study effects in a network of interventions. Research Synthesis Methods, 3(2), 161–176. https://doi.org/10.1002/jrsm.57.

Cipriani

Higgins

J. P.

Geddes

J. R.

Salanti

(2013). Conceptual and technical challenges in network meta-analysis. Annals of Internal Medicine, 159, 130–137. DOI:10.7326/0003-4819-159-2-201307160-00008.

Cooper

Patall

(2009). The relative benefits of meta-analysis conducted with individual participant data versus aggregated data. Psychological Methods, 14(2), 165–176. https://doi.org/10/ddcxmb

Cooper

N. J.

Peters

Lai

M. C.

Juni

Wandel

Palmer

Paulden

Conti

Welton

N. J.

Abrams

K. R.

Bujkiewicz

Spiegelhalter

Sutton

A. J.

(2011). How valuable are multiple treatment comparison methods in evidence-based health-care evaluation? Value in Health, 14(2), 371–380. https://doi.org/10/b5b9pv

Debodinance

Maljaars

Noens

Van den Noortgate

(2017). Interventions for toddlers with autism spectrum disorder: A meta-analysis of single-subject experimental studies. Research in Autism Spectrum Disorders, 36, 79–92. https://doi.org/10/f925hh

10.

Debray

T. P.

Schuit

Efthimiou

Reitsma

J. B.

Ioannidis

J. P.

Salanti

Moons

K. G.

(2018). An overview of methods for network meta-analysis using individual participant data: When do benefits arise? Statistical Methods in Medical Research, 27(5), 1351–1364. https://doi.org/10/gdfq29

11.

Declercq

Jamshidi

Castilla

B.-F.

Moayaert

Beretvas

S.N.

Ferron

J. M.

Noortgate

W. V. D

(2020). Multilevel meta-analysis of individual participant data of single-case experimental designs: One-stage versus two-stage methods. Submitted for Publication.

12.

Donegan

Welton

N. J.

Tudur Smith

D'Alessandro

Dias

(2017). Network meta‐analysis including treatment by covariate interactions: Consistency can vary across covariate values. Research Synthesis methods, 8(4), 485–495. https://doi.org/10.1002/jrsm.1257

13.

Donegan

Williamson

D'Alessandro

Garner

Smith

C. T.

(2013). Combining individual patient data and aggregate data in mixed treatment comparison meta‐analysis: Individual patient data may be beneficial if only for a subset of trials. Statistics in Medicine, 32(6), 914–930. https://doi.org/10.1002/sim.5584

14.

Donegan

Williamson

D'Alessandro

Tudur Smith

(2013). Assessing key assumptions of network meta-analysis: a review of methods: Assessing assumptions of network meta-analysis. Research Synthesis Methods, 4(4), 291–323. https://doi.org/10/gf8ddc

15.

Duan

Kravitz

R. L.

Schmid

C. H.

(2013). Single-patient (n-of-1) trials: A pragmatic clinical decision methodology for patient-centered comparative effectiveness research. Comparative Effectiveness Research, 66(8), S21–S28. https://doi.org/10/f46pwv

16.

Efthimiou

Debray

T. P.

van Valkenhoef

Trelle

Panayidou

Moons

K. G.

Reitsma

J. B.

Shang

Salanti

(2016). GetReal in network meta-analysis: A review of the methodology: ReviewNMA. Research Synthesis Methods, 7(3), 236–263. https://doi.org/10/gdqdcf

17.

Gabler

N. B.

Duan

Vohra

Kravitz

R. L.

(2011). N-of-1 trials in the medical literature. Medical Care, 49(8), 761–768. https://doi.org/10/dd3zfp

18.

Gleser

L. J.

Olkin

(1994). Stochastically dependent effect sizes. In Cooper

Hedges

L. V.

(Eds), The handbook of research synthesis (pp. 339–355). Russell Sage Foundation.

19.

Grant

E. S.

Calderbank-Batista

(2013). Network meta-analysis for complex social interventions: Problems and potential. Journal of the Society for Social Work and Research, 4(4), 406–420. https://doi.org/10.5243/jsswr.2013.25

20.

Higgins

J. P.

Jackson

Barrett

J. K.

Ades

A. E.

White

I. R.

(2012). Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies. Research Synthesis Methods, 3(2), 98–110. https://doi.org/10/gf8j2k

21.

Horner

R. H.

Carr

E. G.

Halle

McGee

Odom

Wolery

(2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71(2), 165–179. https://doi.org/10/gfpqfk

22.

Hutton

Salanti

Caldwell

D. M.

Chaimani

Schmid

C. H.

Cameron

Ioannidis

J. P.

Straus

Thorlund

Jansen

J. P.

Mulrow

Catalá-López

Gøtzsche

P. C.

Dickersin

Boutron

Altman

D. G.

Moher

(2015). The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: Checklist and explanations. Annals of Internal Medicine, 162(11), 777–784. https://doi.org/10.7326/M14-2385

23.

Jackson

Barrett

J. K.

Rice

White

I. R.

Higgins

J. P. T.

(2014). A design-by-treatment interaction model for network meta-analysis with random inconsistency effects. Statistics in Medicine, 33, 3639–3654. DOI:10.1002/sim.6188.

24.

Jamshidi

Declercq

Fernández-Castilla

Ferron

J. M.

Moeyaert

Beretvas

S. N.

Van den Noortgate

(2020). Multilevel meta-analysis of multiple regression coefficients from single-case experimental studies. Behavior research methods, 52(2), 2008–2019. https://doi.org/10/gg4d3k

25.

Jamshidi

Heyvaert

Declercq

Fernández-Castilla

Ferron

Moeyaert

J. M.

, et al. (2020). A systematic review of single- case experimental design meta-analyses: characteristics of study designs, data, and analyses. Evidence-based communication assessment and intervention.

26.

Jansen

J. P.

(2012). Network meta‐analysis of individual and aggregate level data. Research Synthesis Methods, 3(2), 177–190. https://doi.org/10.1002/jrsm.1048

27.

Jansen

J. P.

Naci

(2013). Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers. BMC Medicine, 11, 159. https://doi.org/10/f94fp5

28.

Kilgus

S. P.

Riley-Tillman

T. C.

Kratochwill

T. R.

(2016). Establishing interventions via a theory-driven single case design research cycle. School Psychology Review, 45(4), 477–498. https://doi.org/10/f98cc5.

29.

Koegel

L. K.

Koegel

R. L.

Ashbaugh

Bradshaw

(2014). The importance of early identification and intervention for children with or at risk for autism spectrum disorders. International Journal of Speech-Language Pathology, 16(1), 50–56. https://doi.org/10/ggjpgd

30.

Kratochwill

T. R.

Hitchcock

J. H.

Horner

R. H.

Levin

J. R.

Odom

S. L.

Rindskopf

D. M.

Shadish

W. R.

(2012). Single-Case Intervention Research Design Standards, 34(1), 26–38. https://doi.org/10/bwtw

31.

Moeyaert

Maggin

Verkuilen

(2016). Reliability, validity, and usability of data extraction programs for single-case research designs. Behavior Modification, 40(6), 874–900. https://doi.org/10.1177/0145445516645763

32.

Molloy

G. J.

Noone

Caldwell

Welton

N. J.

Newell

(2018). Network meta-analysis in health psychology and behavioural medicine: a primer. Health Psychology Review, 12(3), 254–270. https://doi.org/10.1080/17437199.2018.1457449

33.

Onghena

(2005). Single-case designs. In Everitt

B. S.

Howell

D. C.

(Eds) Encyclopedia of statistics in behavioral science, (Vol. 3, pp. 1850–1854). John Wiley & Sons. https://doi.org/10.1002/0470013192.bsa625

34.

Parker

R. I.

Brossart

D. F.

(2003). Evaluating single-case research data: A comparison of seven statistical methods. Behavior Therapy, 34(2), 189–211. https://doi.org/10/d42jqs.

35.

Pigott

T. D.

(2019). Handling missing data in meta-analysis. In Cooper

Hedges

L. V.

Valentine

J. F.

(Eds) The handbook of research synthesis and meta-analysis, (pp. 367–382). https://doi.org/10.7758/9781610448864.20

36.

Pustejovsky

J. E.

Ferron

J. M.

(2017). Research synthesis and meta-analysis of single-case designs. In: Handbook of special education (pp. 168–186). https://doi.org/10/gg4bvr

37.

R Core Team (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/

38.

Rücker

Krahn

König

Efthimiou

Schwarzer

(2020). Netmeta: Network meta-analysis using frequentist methods. https://CRAN.R-project.org/package=netmeta

39.

Rücker

Schwarzer

(2015). Ranking treatments in frequentist network meta-analysis works without resampling methods. BMC Medical Research Methodology, 15, 58. https://doi.org/10.1186/s12874-015-0060-8

40.

Salanti

(2012). Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Research Synthesis Methods, 3(2), 80–97. https://doi.org/10/gf8ddh

41.

Salanti

Ades

A. E.

Ioannidis

J. P.

(2011). Graphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: an overview and tutorial. Journal of Clinical Epidemiology, 64(2), 163–171. https://doi.org/10/fddq74

42.

Salanti

Kavvoura

F. K.

Ioannidis

J. P.

(2008). Exploring the geometry of treatment networks. Annals of Internal Medicine, 148(7), 544–553. https://doi.org/10.7326/0003-4819-148-7-200804010-00011

43.

Salanti

Marinho

Higgins

J. P.

(2009). A case study of multiple-treatments meta-analysis demonstrates that covariates should be considered. Journal of Clinical Epidemiology, 62(8), 857–864. https://doi.org/10.1016/j.jclinepi.2008.10.001

44.

Saramago

Sutton

A. J.

Cooper

N. J.

Manca

(2012). Mixed treatment comparisons using aggregate and individual participant level data. Statistics in Medicine, 31(28), 3516–3536. https://doi.org/10.1002/sim.5442

45.

Shadish

W. R.

(2014). Analysis and meta-analysis of single-case designs: An introduction. Journal of School Psychology, 52(2), 109–122. https://doi.org/10/ggdbrf

46.

Shadish

W. R.

Sullivan

K. J.

(2011). Characteristics of single-case designs used to assess intervention effects in 2008. Behavior Research Methods, 43(4), 971–980. https://doi.org/10/dxfjr6

47.

Smith

(2012). Single-case experimental designs: A systematic review of published research and current standards. Psychological Methods, 17(4), 510–550. https://doi.org/10/f4hg4t

48.

Tummers

(2006). DataThief III. https://datathief.org

49.

Van Den Noortgate

Onghena

(2003). Hierarchical linear models for the quantitative integration of effect sizes in single-case research. Behavior Research Methods, Instruments, & Computers, 35(1), 1–10. https://doi.org/10/fr6x4b

50.

Van den Noortgate

Onghena

(2008). A multilevel meta-analysis of single-subject experimental design studies. Evidence-Based Communication Assessment and Intervention, 2(3), 142–151. https://doi.org/10/c4qktn

51.

Viechtbauer

(2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1–48. https://doi.org/10/gckfpj.

52.

What Works Clearinghouse (2020). What works Clearinghouse standards handbook. Version 4.1. U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. https://ies.ed.gov/ncee/wwc/handbooks

53.

White

I. R.

Barrett

J. K.

Jackson

Higgins

J. P.

(2012). Consistency and inconsistency in network meta‐analysis: model estimation using multivariate meta‐regression. Research Synthesis Methods, 3, 111–125. DOI:10.1002/jrsm.1045.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.21 MB