Restricted accessResearch articleFirst published online 2017-11
Technology-Aided Instruction and Intervention for Students With ASD: A Meta-Analysis Using Novel Methods of Estimating Effect Sizes for Single-Case Research
The adoption of methods and strategies validated through rigorous, experimentally oriented research is a core professional value of special education. We conducted a systematic review and meta-analysis examining the experimental literature on Technology-Aided Instruction and Intervention (TAII) using research identified as part of the National Autism Professional Development Project. We applied novel between-case effect size methods to the TAII single-case research base. In addition, we used meta-analytic methodologies to examine the methodological quality of the research, calculate average effect sizes to quantify the level of evidence for TAII, and compare effect sizes across single-case and group-based experimental research. Results identified one category of TAII—computer-assisted instruction—as an evidence-based practice across both single-case and group studies. The remaining two categories of TAII—augmentative and alternative communication and virtual reality—were not identified as evidence-based using What Works Clearinghouse summary ratings.
*AchamadiD.KagoharaD. M.van der MeerL.O’ReillyM.LancioniG.SutherlandD.. . . SigafoosJ. (2012). Teaching advanced operation of an iPod-based speech-generating device to two students with Autism Spectrum Disorders. Research in Autism Spectrum Disorders, 6, 1258–1264.
2.
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Publishing.
3.
AyresK.LedfordJ. R. (2014). Dependent measures and measurement systems. In GastD. L.LedfordJ. R. (Eds.), Single case research design in behavioral sciences (2nd ed., pp. 124–153). New York, NY: Routledge.
4.
BartonE. E.LedfordJ. R.LaneJ. D.DeckerJ.GermanskyS. E.HemmeterM. L.KaiserA. (2016). The iterative use of single case research designs to advance the science of EI/ECSE. Topics in Early Childhood Special Education, 36, 4–14.
5.
*BeaumontR.SofronoffK. (2008). A multi-component social skills intervention for children with Asperger syndrome: The junior detective training program. Journal of Child Psychology and Psychiatry, 49, 743–753.
6.
*BoeschM. C.WendtO.SubramanianA.HsuN. (2013). Comparative efficacy of the Picture Exchange Communication System (PECS) versus a speech-generating device: Effects on social-communicative skills and speech development. Augmentative and Alternative Communication, 29, 197–209.
7.
ChazinK. T.LedfordJ. R.BartonE. E.OsbourneK. (2017). Antecedent exercise for young children: An examination of bias on experimental outcomes. Manuscript Submitted for Publication.
8.
*ChoiH.O’ReillyM.SigafoosJ.LancioniG. (2010). Teaching requesting and rejecting sequences to four children with developmental disabilities using augmentative and alternative communication. Research in Developmental Disabilities: A Multidisciplinary Journal, 31, 560–567.
9.
*CihakD. F.WrightR.AyresK. M. (2010). Use of self-modeling static-picture prompts via a handheld computer to facilitate self-monitoring in the general education classroom. Education and Training in Developmental Disabilities, 45, 136–149.
10.
The Cochrane Collaboration. (2014). Review Manager (RevMan) (Version 5.3) [Computer program]. Copenhagen, Denmark: The Nordic Cochrane Centre, The Cochrane Collaboration.
*DeThorneL.BetancourtM. A.KarahaliosK.HalleJ.BogueE. (2015). Visualizing syllables: Real-time computerized feedback within a speech-language intervention. Journal of Autism and Developmental Disorders, 45, 3756–3763.
13.
DrevonD.FursaS. R.MalcolmA. L. (2017). Intercoder reliability and validity of WebPlotDigitizer in extracting graphed data. Behavior Modification, 41, 323–339.
14.
DuvalS.TweedieR. (2000). A nonparametric “trim and fill” method of accounting for publication bias in meta-analysis. Journal of the American Statistical Association, 95, 89–98.
15.
EggerM.SmithG.SchneiderM.MinderC. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ: British Medical Journal, 315, 629–634.
16.
*FajaS.AylwardE.BernierR.DawsonG. (2007). Becoming a face expert: A computerized face-training program for high-functioning individuals with Autism Spectrum Disorders. Developmental Neuropsychology, 33, 1–24.
*FloresM.MusgroveK.RennerS.HintonV.StrozierS.FranklinS.HilD. (2012). A comparison of communication using the Apple iPad and a picture-based system. Augmentative and Alternative Communication, 28, 74–84.
19.
GanzJ. B.Earles-VollrathT. L.HeathA. K.ParkerR. I.RispoliM. J.DuranJ. B. (2012). A meta-analysis of single-case research studies on aided augmentative and alternative communication systems with individuals with Autism Spectrum Disorders. Journal of Autism and Developmental Disorders, 42, 60–74.
20.
GastD. L.SpriggsA. (2014). Visual analysis of graphic data. In GastD.LedfordJ. R. (Eds.), Single case research methodology: Applications in special education and behavioral sciences (pp. 234–375). New York, NY: Routledge.
21.
*GolanO.AshwinE.GranaderY.McClintockS.DayK.LeggettV.Baron-CohenS. (2010). Enhancing emotion recognition in children with autism spectrum conditions: An intervention using animated vehicles with real emotional faces. Journal of Autism and Developmental Disorders, 40, 269–279.
22.
*GolanO.Baron-CohenS. (2006). Systemizing empathy: Teaching adults with Asperger syndrome or high-functioning autism to recognize complex emotions using interactive multimedia. Development and Psychopathology, 18, 591–617.
23.
GrynszpanO.WeissP. L.Perez-DiazF.GalE. (2014). Innovative technology-based interventions for autism spectrum disorders: A meta-analysis. Autism, 18, 346–361.
24.
HedgesL. V. (1981). Distribution theory for Glass’s estimator of effect size and related estimators. Journal of Educational Statistics, 6, 107–128.
25.
HedgesL. V.PustejovskyJ. E.ShadishW. R. (2012). A standardized mean difference effect size for single case designs. Research Synthesis Methods, 3, 224–239.
26.
HedgesL. V.PustejovskyJ. E.ShadishW. R. (2013). A standardized mean difference effect size for multiple baseline designs across individuals. Research Synthesis Methods, 4, 324–341.
27.
HedgesL. V.TiptonE.JohnsonM. C. (2010). Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods, 1, 39–65.
28.
HigginsJ. P. T.AltmanD. G.SterneJ. A. C. (2008). Assessing risk of bias in included studies. In HigginsJ. P. T.GreenS. (Eds.), Cochrane handbook for systematic reviews of interventions (pp. 187–241). Chichester, UK: John Wiley.
29.
*HopkinsI. M.GowerM. W.PerezT. A.SmithD. S.AmthorF. R.WimsattF. C.BiasiniF. J. (2011). Avatar assistant: Improving social skills in students with an ASD through a computer-based intervention. Journal of Autism and Developmental Disorders, 41, 1543–1555.
30.
HornerR. H.CarrE. G.HalleJ.McGeeG.OdomS.WoleryM. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165–179.
31.
IrishJ. E. (2013). Can I sit here? A review of the literature supporting the use of single-user virtual environments to help adolescents with autism learn appropriate social communication skills. Computers in Human Behavior, 29, A17–A24.
32.
JonesP.WilcoxC.SimonsJ. (2016). Evidence-based instruction for students with Autism Spectrum Disorder: TeachTown Basics. In CardonT. A. (Ed.), Technology and the treatment of children with Autism Spectrum Disorder (pp. 113–130). New York, NY: Springer.
33.
*KagoharaD. M.van der MeerL.AchmadiD.GreenV. A.O’ReillyM. F.MulloyA.. . . SigafoosJ. (2010). Behavioral intervention promotes successful use of an iPod-based communication device by an adolescent with autism. Clinical Case Studies, 9, 328–338. doi:10.1177/1534650110379633
34.
KagoharaD. M.van der MeerL.RamdossS.O’ReillyM. F.LancioniG. E.DavisT. N.. . . GreenV. A. (2013). Using iPods® and iPads® in teaching programs for individuals with developmental disabilities: A systematic review. Research in Developmental Disabilities, 34, 147–156.
35.
*KasariC.KaiserA.GoodsK.NietfeldJ.MathyP.LandaR.. . . AlmirallD. (2014). Communication interventions for minimally verbal children with autism: A sequential multiple assignment randomized trial. Journal of the American Academy of Child & Adolescent Psychiatry, 53, 635–646.
36.
*KeF.ImT. (2013). Virtual-reality-based social interaction training for children with high-functioning autism. The Journal of Educational Research, 106, 441–461.
37.
*KingM. L.TakeguchiK.BarryS. E.RehfeldtR. A.BoyerV. E.MathewsT. L. (2014). Evaluation of the iPad in the acquisition of requesting skills for children with Autism Spectrum Disorder. Research in Autism Spectrum Disorders, 8, 1107–1120.
38.
KnightV.McKissickB. R.SaundersA. (2013). A review of technology-based interventions to teach academic skills to students with autism spectrum disorder. Journal of Autism and Developmental Disorders, 43, 2628–2648.
39.
*KodakT.FisherW. W.ClementsA.BouxseinK. J. (2011). Effects of computer-assisted instruction on correct responding and procedural integrity during early intensive behavioral intervention. Research in Autism Spectrum Disorders, 5, 640–647.
40.
KratochwillT. R.HitchcockJ. H.HornerR. H.LevinJ. R.OdomS. L.RindskopfD. M.ShadishW. R. (2013). Single-case intervention research design standards. Remedial and Special Education, 34, 26–38.
41.
LoflandK. B. (2016). The use of technology in the treatment of autism. In CardonT. A. (Ed.), Technology and the treatment of children with Autism Spectrum Disorder (pp. 27–35). New York, NY: Springer.
42.
*LorahE. R.ParnellA.SpeightD. R. (2014). Acquisition of sentence frame discrimination using the iPad™ as a speech generating device in young children with developmental disabilities. Research in Autism Spectrum Disorders, 8, 1734–1740.
43.
*McKissickB. R.SpoonerF.WoodC. L.DiegelmannK. M. (2013). Effects of computer-assisted explicit instruction on map-reading skills for students with autism. Research in Autism Spectrum Disorders, 7, 1653–1662.
44.
*MechlingL. C.GastD. L.CroninB. A. (2006). The effects of presenting high-preference items, paired with choice, via computer-based video programming on task completion of students with autism. Focus on Autism and Other Developmental Disabilities, 21, 7–13.
45.
*MechlingL. C.GastD. L.SeidN. H. (2009). Using a personal digital assistant to increase independent task completion by students with Autism Spectrum Disorder. Journal of Autism and Developmental Disorders, 39, 1420–1434.
46.
*MechlingL. C.SavidgeE. J. (2011). Using a personal digital assistant to increase completion of novel tasks and independent transitioning by students with Autism Spectrum Disorder. Journal of Autism and Developmental Disorders, 41, 687–704.
47.
*MineoB. A.ZieglerW.GillS.SalkinD. (2009). Engagement with electronic screen media among students with Autism Spectrum Disorders. Journal of Autism and Developmental Disorders, 39, 172–187.
48.
MoherD.LiberatiA.TetzlaffJ.AltmanD. G., & PRISMA Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Medicine, 6(7), e1000097.
49.
*MooreM.CalvertS. (2000). Brief report: Vocabulary acquisition for children with autism: Teacher or computer instruction. Journal of Autism and Developmental Disorders, 30, 359–362.
50.
*MylesB. S.FergusonH.HagiwaraT. (2007). Using a personal digital assistant to improve the recording of homework assignments by an adolescent with Asperger syndrome. Focus on Autism and Other Developmental Disabilities, 22, 96–99.
51.
*NeelyL.RispoliM.CamargoS.DavisH.BolesM. (2013). The effect of instructional use of an iPad® on challenging behavior and academic engagement for two students with autism. Research in Autism Spectrum Disorders, 7, 509–516.
52.
OdomS. L.BrantlingerE.GerstenR.HornerR. H.ThompsonB.HarrisK. R. (2005). Research in special education: Scientific methods and evidence-based practices. Exceptional Children, 71, 137–148.
53.
OdomS. L.ThompsonJ. L.HedgesS.BoydB. A.DykstraJ. R.DudaM. A.. . . BordA. (2015). Technology-aided interventions and instruction for adolescents with Autism Spectrum Disorder. Journal of Autism and Developmental Disorders, 45, 3805–3819.
54.
PenningtonR. C. (2010). Computer-assisted instruction for teaching academic skills to students with Autism Spectrum Disorders: A review of literature. Focus on Autism and Other Developmental Disabilities, 25, 239–248.
55.
*PenningtonR. C.StenhoffD. M.GibsonJ.BallouK. (2012). Using simultaneous prompting to teach computer-based story writing to a student with autism. Education and Treatment of Children, 35, 389–406.
56.
PloogB. O.ScharfA.NelsonD.BrooksP. J. (2013). Use of computer-assisted technologies (CAT) to enhance social, communicative, and language development in children with autism spectrum disorders. Journal of Autism and Developmental Disorders, 43, 301–322.
PustejovskyJ. E.FerronJ. M. (2017). Research synthesis and meta-analysis of single-case designs. In KaufmannJ. M.HallahanD. P.PullenP. C. (Eds.), Handbook of special education (2nd ed., pp. 168–186). New York, NY: Routledge.
59.
PustejovskyJ. E.HedgesL. V.ShadishW. R. (2014). Design-comparable effect sizes in multiple baseline designs: A general modeling framework. Journal of Educational and Behavioral Statistics, 39, 368–393.
60.
RamdossS.LangR.MulloyA.FrancoJ.O’ReillyM.DiddenR.LancioniG. (2011). Use of computer-based interventions to teach communication skills to children with Autism Spectrum Disorders: A systematic review. Journal of Behavioral Education, 20, 55–76.
61.
RamdossS.MachalicekW.RispoliM.MulloyA.LangR.O’ReillyM. (2012). Computer-based interventions to improve social and emotional skills in individuals with autism spectrum disorders: A systematic review. Developmental Neurorehabilitation, 15, 119–135.
62.
ReevesB. C.HigginsJ.RamsayC.SheaB.TugwellP.WellsG. A. (2013). An introduction to methodological issues when including non-randomised studies in systematic reviews on the effects of interventions. Research Synthesis Methods, 4, 1–11.
63.
ReevesL. M.UmbreitJ.FerroJ. B.LiaupsinC. J. (2013). Function-based intervention to support the inclusion of students with autism. Education and Training in Autism and Developmental Disabilities, 48, 379–391.
64.
ReichowB.BartonE. E.MagginD. (2017). Risk of bias assessment for single case designs. Unpublished manuscript, Anita Zucker Center for Excellence in Early Childhood Studies, University of Florida, Gainesville.
65.
ReichowB.DoehringP.CicchettiD. V.VolkmarF. R. (2011). Evidence-based practices in autism: Where we started. In ReichowB.DoehringP.CicchettiD. V.VolkmarF. R. (Eds.), Evidence-based practices and treatments for children with autism (pp. 3–24). New York, NY: Springer.
66.
*RichterS.TestD. (2011). Effects of multimedia social stories on knowledge of adult outcomes and opportunities among transition-aged youth with significant disabilities. Education and Training in Autism and Developmental Disabilities, 46, 410–424.
RothsteinH. R.SuttonA. J.BorensteinM. (2005). Publication bias in meta-analysis. In RothsteinH. R.SuttonA. J.BorensteinM. (Eds.), Publication bias in meta-analysis: Prevention, assessment, and adjustments (pp. 1–7). Chichester, UK: John Wiley.
69.
ShadishW. R.HedgesL. V.HornerR. H.OdomS. L. (2015). The role of between-case effect size in conducting, interpreting, and summarizing single-case research (NCER 2015-002). Washington, DC: National Center for Education Research, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies.ed.gov/ncser/pubs/2015002/
70.
ShadishW. R.HedgesL. V.PustejovskyJ. E. (2014). Analysis and meta-analysis of single-case designs with a standardized mean difference statistic: A primer and applications. Journal of School Psychology, 52, 123–147.
71.
ShadishW. R.ShamE.SmithT. (2014). Publication bias in studies of an applied behavior-analytic intervention: An initial analysis. Journal of Applied Behavior Analysis, 47, 663–678.
72.
ShadishW. R.ZelinskyN. A. M.VeveaJ. L.KratochwillT. R. (2016). A survey of publication practices of single-case design researchers when treatments have small or large effects. Journal of Applied Behavior Analysis, 49, 1–18.
73.
ShamE.SmithT. (2014). Publication bias in studies of an applied behavior-analytic intervention: An initial analysis. Journal of Applied Behavior Analysis, 47, 663–678.
74.
*ShihC. H.ChiangM. S.ShihC. T. (2015). Assisting students with autism to cooperate with their peers to perform computer mouse collaborative pointing operation on a single display simultaneously. Research in Autism Spectrum Disorders, 10, 15–21.
75.
*SilverM.OakesP. (2001). Evaluation of a new computer intervention to teach people with autism or Asperger syndrome to recognize and predict emotions in others. Autism, 5, 299–316.
76.
*SmithM. J.GingerE. J.WrightK.WrightM. A.TaylorJ. L.HummL. B.. . . FlemingM. F. (2014). Virtual reality job interview training in adults with Autism Spectrum Disorder. Journal of Autism and Developmental Disorders, 44, 2450–2463.
77.
*SoaresD. A.VannestK. J.HarrisonJ. (2009). Computer aided self-monitoring to increase academic production and reduce self-injurious behavior in a child with autism. Behavioral Interventions, 24, 171–183.
78.
SterneJ. A. C. (Ed.). (2009). Meta-analysis in Stata: An updated collection from the Stata journal. College Station, TX: Stata Press.
79.
StovoldE.BeecherD.FoxleeR.Noel-StorrA. (2014). Study flow diagrams in Cochrane systematic review updates: an adapted PRISMA flow diagram. Systematic Reviews, 3, 54–59.
80.
StromerR.MackayH. A.HowellS. R.McVayA. A.FlusserD. (1996). Teaching computer-based spelling to individuals with developmental and hearing disabilities: Transfer of stimulus control to writing tasks. Journal of Applied Behavior Analysis, 29, 25–42.
81.
Tanner-SmithE. E.TiptonE.PolaninJ. R. (2016). Handling complex meta-analytic data structures using robust variance estimates: A tutorial in R. Journal of Developmental and Life-Course Criminology, 2, 85–112.
82.
TateR. L.PerdicesM.RosenkoetterU.WakimD.GodbeeK.TogherL.McDonaldS. (2013). Revision of a method quality rating scale for single-case experimental designs and n-of-1 trials: The 15-item Risk of Bias in N-of-1 Trials (RoBiNT) Scale. Neuropsychological Rehabilitation, 23, 619–638.
83.
*TaylorB. A.HughesC. E.RichardE.HochH.CoelloA. R. (2004). Teaching teenagers with autism to seek assistance when lost. Journal of Applied Behavior Analysis, 37, 79–82.
84.
TiptonE. (2015). Small sample adjustments for robust variance estimation with meta-regression. Psychological Methods, 20, 375–393.
85.
TiptonE.PustejovskyJ. E. (2015). Small-sample adjustments for tests of moderators and model fit using robust variance estimation in meta-regression. Journal of Educational and Behavioral Statistics, 40, 604–634.
86.
*van der MeerL.KagoharaD.RocheL.SutherlandD.BalandinS.GreenV. A.SigafoosJ. (2013). Teaching multi-step requesting and social communication to two children with Autism Spectrum Disorders with three AAC options. Augmentative and Alternative Communication, 29, 222–234.
87.
WengP. L.MaedaY.BouckE. C. (2014). Effectiveness of cognitive skills-based computer-assisted instruction for students with disabilities: A synthesis. Remedial and Special Education, 35, 167–180.
88.
*WhalenC.MossD.IlanA. B.VaupelM.FieldingP.MacdonaldK.. . . SymonJ. (2010). Efficacy of TeachTown: Basics computer-assisted intervention for the intensive comprehensive autism program in Los Angeles Unified School District. Autism, 14, 179–197.
89.
What Works Clearinghouse. (2014). What Works Clearinghouse procedures and standards handbook (Version 3.0). Washington, DC: Institute for Education Sciences.
90.
WilliamsC.WrightB.CallaghanG.CoughlanB. (2002). Do children with autism learn to read more readily by computer assisted instruction or traditional book methods? A pilot study. Autism, 6, 71–91.
91.
WoleryM.BusickM.ReichowB.BartonE. E. (2010). Comparison of overlap methods for quantitatively synthesizing single-subject data. The Journal of Special Education, 44(1), 18–28.
92.
WongC.OdomS. L.HumeK. A.CoxA. W.FettigA.KucharczykS.. . . SchultzT. R. (2014). Evidence-based practices for children, youth, and young adults with autism spectrum disorder. Chapel Hill: Autism Evidence-Based Practice Review Group, Frank Porter Graham Child Development Institute, The University of North Carolina.
93.
WongC.OdomS. L.HumeK. A.CoxA. W.FettigA.KucharczykS.. . . SchultzT. R. (2015). Evidence-based practices for children, youth, and young adults with autism spectrum disorder: A comprehensive review. Journal of Autism and Developmental Disorders, 45, 1951–1966.
94.
*YakubovaG.Taber-DoughtyT. (2013). Brief report: Learning via the electronic interactive whiteboard for two students with autism and a student with moderate intellectual disability. Journal of Autism and Developmental Disorders, 43, 1465–1472.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
