In single-case research, the multiple baseline design is a widely used approach for evaluating the effects of interventions on individuals. Multiple baseline designs involve repeated measurement of outcomes over time and the controlled introduction of a treatment at different times for different individuals. This article outlines a general framework for defining effect sizes in multiple baseline designs that are directly comparable to the standardized mean difference from a between-subjects randomized experiment. The target, design-comparable effect size parameter can be estimated using restricted maximum likelihood together with a small sample correction analogous to Hedges’s g. The approach is demonstrated using hierarchical linear models that include baseline time trends and treatment-by-time interactions. A simulation compares the performance of the proposed estimator to that of an alternative, and an application illustrates the model-fitting process.
AlginaJ.KeselmanH. J. (2003). Approximate confidence intervals for effect sizes. Educational and Psychological Measurement, 63, 537–553. doi:10.1177/0013164403256358
2.
BaileyJ. S.BurchM. R. (2002). Research methods in applied behavior analysis. Thousand Oaks, CA: Sage.
3.
BaldwinS. A.FellinghamG. W. (2013). Bayesian methods for the analysis of small sample multilevel data with a complex variance structure. Psychological Methods, 18, 151–164. doi:10.1037/a0030642
4.
BeretvasS. N.ChungH. (2008). A review of meta-analyses of single-subject experimental designs: Methodological issues and practice. Evidence-Based Communication Assessment and Intervention, 2, 129–141. doi:10.1080/17489530802446302
5.
BorckardtJ. J.NashM. R.MurphyM. D.MooreM.ShawD.O’NeilP. (2008). Clinical practice as natural laboratory for psychotherapy research: A guide to case-based time-series analysis. The American Psychologist, 63, 77–95. doi:10.1037/0003-066X.63.2.77
6.
BusseR. T.KratochwillT. R.ElliottS. N. (1995). Meta-analysis for single-case consultation outcomes: Applications to research and practice. Journal of School Psychology, 33, 269–285.
7.
CenterB. A.SkibaR. J.CaseyA. (1985). A methodology for the quantitative synthesis of intra-subject design research. The Journal of Special Education, 19, 387–400.
8.
ChungY.Rabe-HeskethS.DorieV.GelmanA.LiuJ. (2013). A nondegenerate penalized likelihood estimator for variance parameters in multilevel models. Psychometrika, 78, 685–709. doi:10.1007/s11336-013-9328-2
9.
CooperH. M. (2009). Hypotheses and problems in research synthesis. In CooperH. M.HedgesL. VValentineJ. (Eds.), The handbook of research synthesis and meta-analysis (2nd ed., pp. 19–35). New York, NY: Russell Sage Foundation.
10.
CummingG.FinchS. (2001). A primer on the understanding, use, and calculation of confidence intervals that are based on central and noncentral distributions. Educational and Psychological Measurement, 61, 532–574. doi:10.1177/0013164401614002
11.
EfronB.TibshiraniR. J. (1998). An introduction to the bootstrap. Boca Raton, FL: Chapman & Hall/CRC.
12.
FerronJ. M.BellB. A.HessM. R.Rendina-GobioffG.HibbardS. T. (2009). Making treatment effect inferences from multiple-baseline data: The utility of multilevel modeling approaches. Behavior Research Methods, 41, 372–384. doi:10.3758/BRM.41.2.372
13.
GelmanA. (2006). Prior distributions for variance parameters in hierarchical models. Bayesian Analysis, 1, 515–533.
14.
GorsuchR. L. (1983). Three methods for analyzing limited time-series (N of 1) data. Behavioral Assessment, 5, 141–154.
15.
GottmanJ. M. (1981). Time-series analysis: A comprehensive introduction for social scientists. Cambridge, England: Cambridge University Press.
16.
HedgesL. V. (1981). Distribution theory for Glass’s estimator of effect size and related estimators. Journal of Educational Statistics. 6, 107–128.
17.
HedgesL. V. (2007). Effect sizes in cluster-randomized designs. Journal of Educational and Behavioral Statistics, 32, 341–370. doi:10.3102/1076998606298043
18.
HedgesL. V. (2011). Effect sizes in three-level cluster-randomized experiments. Journal of Educational and Behavioral Statistics. 36, 346–380. doi:10.3102/1076998610376617
19.
HedgesL. V.PustejovskyJ. E.ShadishW. R. (2012). A standardized mean difference effect size for single case designs. Research Synthesis Methods, 3, 224–239. doi:10.1002/jrsm.1052
20.
HedgesL. V.PustejovskyJ. E.ShadishW. R. (2013). A standardized mean difference effect size for multiple baseline designs across individuals. Research Synthesis Methods, 4, 324–341. doi:10.1002/jrsm.1086
21.
HedgesL. V.TiptonE.JohnsonM. C. (2010). Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods, 1, 39–65. doi:10.1002/jrsm.5
22.
HersenM. (1990). Single-case experimental designs. In BellackA. S.HersenM.KazdinA. E. (Eds.), International handbook of behavior modification and therapy (2nd ed., pp. 175–210). New York, NY: Plenum Press.
23.
HollandP. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81, 945–960.
24.
HornerR. H.CarrE. G.HalleJ.McGeeG.OdomS. L.WoleryM. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165–179.
25.
HornerR. H.SwaminathanH.SugaiG.SmolkowskiK. (2012). Considerations for the systematic analysis and use of single-case research. Education and Treatment of Children, 35, 269–290. doi:10.1353/etc.2012.0011
26.
HuitemaB. E.McKeanJ. W. (2000). Design specification issues in time-series intervention models. Educational and Psychological Measurement, 60, 38–58. doi:10.1177/00131640021970358
27.
KazdinA. E. (2011). Single-case research designs: Methods for clinical and applied settings. New York, NY: Oxford University Press.
28.
KenwardM. G.RogerJ. H. (1997). Small sample inference for fixed effects from restricted maximum likelihood. Biometrics, 53, 983–997.
29.
KenwardM. G.RogerJ. H. (2009). An improved approximation to the precision of fixed effects from restricted maximum likelihood. Computational Statistics & Data Analysis, 53, 2583–2595. doi:10.1016/j.csda.2008.12.013
30.
KratochwillT. R.HitchcockJ. H.HornerR. H.LevinJ. R.OdomS. L.RindskopfD. M.ShadishW. R. (2012). Single-case intervention research design standards. Remedial and Special Education, 34, 26–38. doi:10.1177/0741932512452794
31.
PinheiroJ. C.BatesD. M. (2000). Mixed-effects models in s and S-PLUS. New York, NY: Springer Verlag.
SAS Institute Inc. (2008). SAS/STAT(R) 9.2 user’s guide. Cary, NC:Author.
34.
SchutteN. S.MalouffJ. M.BrownR. F. (2008). Efficacy of an emotion-focused treatment for prolonged fatigue. Behavior Modification, 32, 699–713. doi:10.1177/0145445508317133
35.
ScruggsT. E.MastropieriM. A.CastoG. (1987). The quantitative synthesis of single-subject research. Remedial and Special Education, 8, 24–43.
36.
ShadishW. R.RindskopfD. M.HedgesL. V.SullivanK. J. (2013). Bayesian estimates of autocorrelations in single-case designs. Behavior Research Methods, 45, 813–821. doi:10.3758/s13428-012-0282-1
37.
ShadishW. R.SullivanK. J. (2011). Characteristics of single-case designs used to assess intervention effects in 2008. Behavior Research Methods, 43, 971–980. doi:10.3758/s13428-011-0111-y
38.
SingerJ. D.WillettJ. B. (2003). Applied longitudinal data analysis: Modeling change and event occurrence. New York, NY: Oxford University Press.
39.
SmithJ. D. (2012). Single-case experimental designs: A systematic review of published research and current standards. Psychological Methods, 17, 510–550. doi:10.1037/a0029312
40.
StataCorp. (2011). Stata, release 12: Longitudinal-data/panel-data reference manual. College Station, TX: Stata Press.
41.
WoleryM.BusickM.ReichowB.BartonE. E. (2010). Comparison of overlap methods for quantitatively synthesizing single-subject data. The Journal of Special Education, 44, 18–28. doi:10.1177/0022466908328009
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
