Single case experimental design (SCED) is an indispensable methodology when evaluating intervention efficacy. Despite long-standing success with using visual analyses to evaluate SCED data, this method has limited utility for conducting meta-analyses. This is critical because meta-analyses should drive practice and policy in behavioral disorders more than evidence derived from individual SCEDs. Even when analyzing data from individual studies, there is merit to using multiple analytic methods since statistical analyses in SCED can be challenging given small sample sizes and autocorrelated data. These complexities are exacerbated when using count data, which are common in SCEDs. Bayesian methods can be used to develop new statistical procedures that may address these challenges. The purpose of the present study was to formulate a within-subject Bayesian rate ratio effect size (BRR) for autocorrelated count data that would obviate the need for small sample corrections. This effect size is the first step toward building a between-subject rate ratio that can be used for meta-analyses. We illustrate this within-subject effect size using real data for an ABAB design and provide codes for practitioners who may want to compute BRR.
AlbertJ. H. (1992). Bayesian estimation of normal ogive item response curves using Gibbs sampling. Journal of Educational and Behavioral Statistics, 17(3), 251–269. https://doi.org/10.3102/10769986017003251
AnsariA.JedidiK.JagpalS. (2000). A hierarchical Bayesian approach for modeling heterogeneity in structural equation models. Marketing Science, 19, 328–347. https://doi.org/10.1287/mksc.19.4.328.11789
4.
BrieschA. M.BrieschJ. M. (2016). Meta-analysis of behavioral self-management interventions in single-case research. School Psychology Review, 45, 3–18. https://doi.org/10.17105/SPR45-1.3-18
5.
BrooksS.GelmanA. (1998). General methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics, 7, 434–455. https://doi.org/10.1080/10618600.1998.10474787
6.
BrossartD. F.ParkerR. I.OlsonE. A.MahadevanL. (2006). The relationship between visual analysis and five statistical analyses in a simple AB single-case research design. Behavior Modification, 30, 531–563. https://doi.org/10.1177/0145445503261167
7.
ChaffeeR. K.BrieschA. M.JohnsonA. H.VolpeR. J. (2017). A meta-analysis of class-wide interventions for supporting student behavior. School Psychology Review, 46(2), 149–164. https://doi.org/10.17105/SPR-2017-0015.V46-2
8.
CohenJ. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum.
CookB. G. (2014). A call for examining replication and bias in special education research. Remedial and Special Education, 35(4), 233–246. https://doi.org/10.1177/0741932514528995
11.
DartE. H.CollinsT. A.KlingbeilD. A.McKinleyL. E. (2014). Peer management interventions: A meta-analytic review of single-case research. School Psychology Review, 43(4), 367–384. https://doi.org/10.17105/SPR-14-0009.1
12.
DenwoodM. J. (2016). runjags: An R package providing interface utilities, model templates, parallel computing methods and additional distributions for MCMC models in JAGS. Journal of Statistical Software, 71, 1–25. https://doi.org/10.18637/jss.v071.i09
13.
GablerN. B.DuanN.VohraS.KravitzR. L. (2011). N-of-1 trials in the medical literature: A systematic review. Medical Care, 49, 761–768.
14.
GastD. L.LedfordJ. R. (2014). Single subject research methodology in behavioral sciences (2nd ed.). Routledge.
15.
GelfandA. E.SmithA. M. E. (1990). Sampling based approaches to calculating marginal densities. Journal of the American Statistical Association, 85, 398–409. https://doi.org/10.1080/01621459.1990.10476213
16.
GelmanA.CarlinJ. B.SternH. S.DunsonD. B.VehtariA.RubinD. B. (2013). Bayesian data analysis (3rd ed.). Chapman & Hall.
17.
GemanS.GemanD. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721–741. https://doi.org/10.1109/TPAMI.1984.4767596
18.
GreshamF. M.SugaiG.HornerR. H. (2001). Interpreting outcomes of social skills training for students with high-incidence disabilities. Exceptional Children, 67, 331–344. https://doi.org/10.1177/001440290106700303
19.
HarringtonM.VelicerW. F. (2015). Comparing visual and statistical analysis in single-case studies using published studies. Multivariate Behavioral Research, 50, 162–183. https://doi.org/10.1080/00273171.2014.973989
20.
HarropJ. W.VelicerW. F. (1985). A comparison of alternative approaches to the analysis of interrupted time-series. Multivariate Behavioral Research, 20, 27–44. https://doi.org/10.1207/s15327906mbr2001_2
21.
HedgesL. V.PustejovskyJ. E.ShadishW. R. (2012). A standardized mean difference effect size for single case designs. Research Synthesis Methods, 3, 224–239. https://doi.org/10.1002/jrsm.1052
22.
HedgesL. V.PustejovskyJ. E.ShadishW. R. (2013). A standardized mean difference effect size for multiple baseline designs across studies. Research Synthesis Methods, 4, 324–341. https://doi.org/10.1002/jrsm.1086
23.
HeidelbergerP.WelchP. D. (1983). Simulation run length control in the presence of an initial transient. Operations Research, 31, 1109–1144. https://doi.org/10.1287/opre.31.6.1109
24.
HitchcockJ. H.HornerR. H.KratochwillT. R.LevinJ. R.OdomS. L.RindskopfD. M.ShadishW. R. (2014). The What Works Clearinghouse single-case design pilot standards: Who will guard the guards?Remedial and Special Education, 35(3), 154–152. https://doi.org/10.1177/0741932513518979
25.
HornerR. H.CarrE. G.HalleJ.McGeeG.OdomS.WoleryM. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165–179. https://doi.org/10.1177/001440290507100203
26.
HornerR. H.KratochwillT. R. (2012). Synthesizing single-case research to identify evidence-based practices: Some brief reflections. Journal of Behavioral Education, 21, 266–272. https://doi.org/10.1007/s10864-012-9152-2
27.
HornerR. H.SwaminathanH.SugaiG.SmolkowskiK. (2012). Expanding analysis of single case research. Institute of Education Sciences, U.S. Department of Education.
28.
HuitemaB. E.McKeanJ. W. (1994). Two reduced-bias autocorrelation estimators: rF1 and rF2. Perceptual and Motor Skills, 78, 323–330. https://doi.org/10.2466/pms.1994.78.1.323
29.
KilgusS. P.Riley-TillmanC.KratochwillT. R. (2016). Establishing interventions via a theory-driven single case design research cycle. School Psychology Review, 45(4), 477–498. https://doi.org/10.17105/SPR45-4.477-498
KratochwillT. R.HitchcockJ. H.HornerR. H.LevinJ. R.OdomS. L.RindskopfD. M.ShadishW. R. (2013). Single-case intervention research design standards. Remedial and Special Education, 34, 26–38. https://doi.org/10.1177/0741932512452794
32.
KruschkeJ. K. (2013). Bayesian estimation supersedes the t-test. Journal of Experimental Psychology: General, 142, 573–603. https://doi.org/10.1037/a0029146
33.
KruschkeJ. K. (2015). Doing Bayesian data analysis: A tutorial with R, JAGS, and Stan (2nd ed.). Academic Press/Elsevier.
34.
LambertM. C.CartledgeG.HewardW. L.LoY. (2006). Effects of response cards on disruptive behavior and academic responding during math lessons by fourth-grade urban students. Journal of Positive Behavior Interventions, 8, 88–99.
35.
LanovazM. J.RappJ. T. (2016). Using single-case experiments to support evidence-based decisions: How much is enough?Behavior Modification, 40, 377–395. https://doi.org/10.1177/0145445515613584
36.
LipseyM. W.WilsonD. B. (2001). Practical meta-analysis. SAGE.
37.
MagginD. M.ChafouleasS. M. (2013). Introduction to the special series: Issues and advances of synthesizing single-case research. Remedial and Special Education, 34, 3–8. https://doi.org/10.1177/0741932512466269
38.
MagginD. M.ChafouleasS. M.GoddardK. M.JohnsonA. H. (2011). A systematic evaluation of token economies as a classroom management tool for students with challenging behavior. Journal of School Psychology, 49(5), 529–554. https://doi.org/10.1016/j.jsp.2011.05.001
39.
MagginD. M.O’KeefeB. V.JohnsonA. H. (2011). A quantitative synthesis of methodology in the meta-analysis of single-subject research for students with disabilities: 1985–2009. Exceptionality, 19, 109–135. https://doi.org/10.1080/09362835.2011.565725
40.
McKnightS. D.McKeanJ. W.HuitemaB. E. (2000). A double bootstrap method to analyze linear models with autoregressive error terms. Psychological Methods, 5, 87–101. https://doi.org/10.1037/1082-989X.5.1.87
41.
MeadanH.SnodgrassM. R.MeyerL. E.FisherK. W.ChungM. Y.HalleJ. W. (2016). Internet-based parent-implemented intervention for young children with autism: A pilot study. Journal of Early Intervention, 38, 3–23. http://dx.doi.org/10.1177/1053815116630327
42.
MoeyaertM.RindskopfD.OnghenaP.Van den NoortgateW. (2017). Multilevel modeling of single-case data: A comparison of maximum likelihood and Bayesian estimation. Psychological Methods, 22, 760–778. https://doi.org/10.1037/met0000136
43.
NastasiB. K.HitchcockJ. H. (2016). The new mixed methods research series: Mixed methods research and culture-specific interventions: Program design and evaluation. SAGE.
Natesan BatleyP.ContractorA.CaldasS. (under review). Bayesian Time-Series Models in Single Case Experimental Designs: A Tutorial for Trauma Researchers. Journal of Traumatic Stress.
46.
Natesan BatleyP.MinkaT.HedgesL. V. (2020). Investigating immediacy in multiple phase-change single case experimental designs using a Bayesian unknown change-points model. Behavioral Research Methods. https://doi.org/10.3758/s13428-020-01345-z
47.
NatesanP.BoedekerP.OnwuegbuzieA. J. (in press). Prevalence of Bayesian methods in the field of education. International Journal of Multiple Research Approaches.
48.
NatesanP.HedgesL. V. (2017). Bayesian unknown change-point models to investigate immediacy in single case designs. Psychological Methods, 22, 743–759. https://doi.org/10.1037/met0000134
49.
NatesanP.HedgesL. V. (2019, April). Accurate model vs. accurate estimates: A study of Bayesian single-case experimental designs [Paper presentation]. Annual Meeting of the American Educational Research Association, Toronto, Ontario, Canada.
50.
NatesanP.NandakumarR.MinkaT.RubrightJ. (2016). Bayesian prior choice in IRT estimation using MCMC and variational Bayes. Frontiers in Psychology, 7, Article 1422.
51.
OdomS. L.BrantlingerE.GerstenR.HornerR. H.ThompsonB.HarrisK. R. (2005). Research in special education: Scientific methods and evidence-based practices. Exceptional Children, 71, 137–148. https://doi.org/10.1177/001440290507100201
52.
ParkerR. I.VannestK. J. (2009). An improved effect size for single-case research: Nonoverlap of all pairs. Behavior Therapy, 40, 357–367. https://doi.org/10.1016/j.beth.2008.10.006
53.
ParkerR. I.VannestK. J.DavisJ. L. (2011). Effect size in single-case research: A review of nine nonoverlap techniques. Behavior Modification, 35, 303–322. https://doi.org/10.1177/0145445511399147
54.
PlummerM. (2003, March20–22). JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling [Conference session]. 3rd International Workshop on Distributed Statistical Computing, Vienna, Austria.
RoaneH. S.RihgdahlJ. E.KelleyM. E.GloverA. C. (2011). Single-case experimental designs. In FisherW. W.PiazzaC. C.RoaneH. S. (Eds.), Handbook of applied behavior analysis (pp. 132–147). Guilford Press.
57.
ShadishW. R.RindskopfD. M.HegdesL. V.SullivanK. J. (2013). Bayesian estimates of autocorrelations in single-case designs. Behavioral Research Methods, 45, 813–821. https://doi.org/10.3758/s13428-012-0282-1
58.
ShadishW. R.SullivanK. J. (2011). Characteristics of single-case designs used to assess intervention effects in 2008. Behavior Research Methods, 43, 971–980. https://doi.org/10.3758/s13428-011-0111-y
59.
SoaresD. A.HarrisonJ. R.VannestK. J.McClellandS. S. (2016). Effect size for token economy use in contemporary classroom settings: A meta-analysis of single-case research. School Psychology Review, 45(4), 379–399. https://doi.org/10.17105/SPR45-4.379-399
60.
VelicerW. F.MolenaarP. (2013). Time series analysis: Research methods in psychology. In SchinkaJ.VelicerW. F. (Eds.), Handbook of psychology (2nd ed., Vol. 2, pp. 628–660). John Wiley.
