The implementation of research-based practices by teachers in public school classrooms is required under federal law as expressed in the Individuals With Disabilities Education Improvement Act of 2004. To aid teachers in identifying such practices, researchers conduct systematic reviews of the educational literature. Although recent attention has been given to changes in the quality of these reviews, there has been minimal discussion about changes in the quality of the studies that comprise them. Specifically, to what extent have educational policies leading to the creation of experimental design standards resulted in a change in the rigor of educational research? Using a subset of the single-case literature commonly published in special education journals, we estimate the impact of What Works Clearinghouse single-case design standards on the trend in the rigor of single-case studies using a comparative interrupted time series framework. Within this subset of single-case studies, our estimation strategy did not detect a change in the trend of the rigor of single-case research following the establishment of What Works Clearinghouse single-case design standards. Implications are discussed for practitioners and researchers.
BeahmL. A.CookB. G.CookL. (2019). Proceed with caution: Using web-based resources for instructing students with and at risk for EBD. Beyond Behavior, 28, 13–20.
2.
BernalJ. L.CumminsS.GasparriniA. (2017). Interrupted time series regression for the evaluation of public health interventions: A tutorial. International Journal of Epidemiology, 46, 348–355.
3.
BurnsM. K.KlingbeilD. A.YsseldykeJ. E.Petersen-BrownS. (2012). Trends in methodological rigor in intervention research published in school psychology journals. Psychology in the Schools, 49, 843–851.
4.
CleaverS. L.WoodC. L. (2018). Using Pinterest to find and share evidence-based practices. Intervention in School and Clinic, 54, 111–117.
5.
CookB.BuysseV.KlingnerJ.LandrumT.McWilliamR.TankersleyM.TestD. (2014). Council for Exceptional Children: Standards for evidence-based practices in special education. Teaching Exceptional Children, 46(6), 206–212.
6.
FisherW. W.KodakT.MooreJ. W. (2007). Embedding an identity-matching task within a prompting hierarchy to facilitate acquisition of conditional discriminations in children with autism. Journal of Applied Behavior Analysis, 40, 489–499.
7.
GanzJ. B.AyresK. M. (2018). Methodological standards in single-case experimental design: Raising the bar. Research in Developmental Disabilities, 79, 3–9.
8.
HornerR. H.CarrE. G.HalleJ.McGeeG.OdomS.WoleryM. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165–179.
9.
Individuals With Disabilities Education Improvement Act, 20 U.S.C. § 1400 et seq. (2004).
10.
JamshidiL.HeyvaertM.DeclercqL.Fernandez-CastillaB.FerronJ. M.MoeyaertM.. . . Van den NoortgateW. (2018). Methodological quality of meta-analyses of single-case experimental studies. Research in Developmental Disabilities, 79, 97–115.
11.
KiuharaS. A.KratocwhillT. R.PullenP. C. (2017). Designing robust single-case design experimental studies. In KauffmanJ. M.HallahanD. P.PullenP. C. (Eds.), Handbook of special education (pp. 116–136). Routledge.
12.
KratochwillT. R.HitchcockJ.HornerR. H.LevinJ. R.OdomS. L.RindskopfD. M.ShadishW. R. (2010). Single-case designs technical documentation. What Works Clearinghouse. https://ies.ed.gov/ncee/wwc/Document/229
13.
LedfordJ. R.LaneJ. D.TateR. (2018). Evaluating quality and rigor in single case research. In LedfordJ. R.GastD. L. (Eds.), Single case research methodology (pp. 365–392). Routledge.
14.
LedfordJ. R.LaneJ. D.ZimmermanK. N.ChazinK. T.AyresK. A. (2016). Single case analysis and review framework (SCARF). http://ebip.vkcsites.org/scarf/
15.
MagginD. M.BrieschA. M.ChafouleasS. M.FergusonT. D.ClarkC. (2014). A comparison of rubrics for identifying empirically supported practices with single-case research. Journal of Behavioral Education, 23, 287–311.
16.
NelsonJ. R.RobertsM. L.MathurS. R.RutherfordR. B. (1999). Has public policy exceeded our knowledge base? A review of the functional behavioral assessment literature. Behavioral Disorders, 24, 169–179.
17.
OdomS. (1988). Research in early childhood special education: Methodologies and paradigm. In OdomS. L.KarnesM. B. (Eds.), Early intervention for infants and children with handicaps (pp. 1–22). Paul H. Brookes.
18.
O’KeefeB. V.SlocumT. A.BurlingameC.SnyderK.BundockK. (2012). Comparing results of systematic reviews: Parallel reviews of research on repeated reading. Education and Treatment of Children, 35, 333–366.
19.
ReichowB.BartonE. E.MagginD. M. (2018). Development and applications of the single-case design risk of bias tool for evaluating single-case design research study reports. Research in Developmental Disabilities, 79, 53–64.
20.
ReidR.NelsonJ. R. (2002). The utility, acceptability, and practicality of functional behavioral assessment for students with high-incidence problem behaviors. Remedial and Special Education, 23, 15–23.
21.
SackettD. L.RosenburgW. M. C.GrayJ. A. M.HaynesR. B.RichardsonW. S. (1996). Evidence-based medicine: What it is and what it isn’t. British Medical Journal, 312, 71–72.
22.
ShadishW.CookT. D.CampbellD. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.
23.
ShepleyC.AultM. J.OrtizK.VoglerJ. C.McGeeM. (2020). An exploratory analysis of quality indicators in adapted alternating treatments designs. Topics in Early Childhood Special Education, 39, 226–237. https://doi.org/10.1177/0271121418820429
24.
ShepleyC.LaneJ. D.AultM. J. (2019). A review and critical examination of the system of least prompts. Remedial and Special Education, 40, 313–327. https://doi.org/10.1177/0741932517751213
25.
SundellK.ÅhsbergE. (2018). Trends in methodological quality in controlled trials of psychological and social interventions. Research on Social Work Practice, 28, 568–576.
26.
SwainR.LaneJ. D.GastD. L. (2015). Comparison of constant time delay and simultaneous prompting procedures: Teaching functional sight words to students with intellectual disabilities and autism spectrum disorder. Journal of Behavioral Education, 24, 210–229.
27.
TateR. L.PerdicesM.RosenkoetterU.McDonaldS.TogherL.ShadishW.. . . SampsonM. (2016). The Single-Case Reporting Guideline in BEhavioural Interventions (SCRIBE) 2016: Explanation and elaboration. Archives of Scientific Psychology, 4, 10–31.
28.
WendtO.MillerB. (2012). Quality appraisal of single-subject experimental designs: An overview and comparison of different appraisal tools. Education and Treatment of Children, 35, 235–268.
WoleryM. (2013). A commentary: SCD technical document of the what works clearinghouse. Remedial and Special Education, 43, 39–43.
34.
WoleryM.AultM. J.DoyleP. M.GastD. L. (1986). Comparison of instructional strategies: A literature review (U.S. Department of Education, Grant No. G008530197). Department of Special Education.
ZimmermanK. N.LedfordJ. R.SeveriniK. E.PustejovskyJ. E.BartonE. E.LloydB. P. (2018). Single-case synthesis tools I: Comparing tools to evaluate SCD quality and rigor. Research in Developmental Disabilities, 79, 19–32.
37.
ZimmermanK. N.PustejovskyJ. E.LedfordJ. R.BartonE. E.SeveriniK. E.LloydB. P. (2018). Single-case synthesis tools II: Comparing quantitative outcome measures. Research in Developmental Disabilities, 79, 65–76.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
