Naturalistic Developmental Behavioral Interventions (NDBIs) are evidence-based interventions implemented by the adults in a child’s natural environment (e.g., by practitioners in classrooms). The social validity of this body of research may have an impact on the maintenance of practitioner implementation of NDBIs. We conducted a systematic review to evaluate the components (i.e., goals, procedures, and outcomes) and features of social validation assessment practices within studies where researchers trained practitioners to implement an NDBI with young children with disabilities in early childhood classroom contexts. Results of the systematic review indicated that social validity assessment within practitioner-implemented NDBI studies is lacking. Of the 23 studies identified, only 12 studies reported social validity assessment(s). Findings also highlight a lack of comprehensive social validity assessment and frequent use of biased respondents. An in-depth analysis of social validity assessment methods and features, as well as recommendations for the field, is provided.
BartonE. E.Meadan-KaplanskyH.LedfordJ. R. (2018). Independent variables, fidelity, and social validity. In Single Case Research Methodology (pp. 133–156). Routledge.
2.
CarterS. L. (2010). The social validity manual: A guide to subjective evaluation of behavioral interventions. Boston, MA: Academic Press.
3.
Division for Early Childhood. (2014). DEC recommended practices in early intervention/early childhood special education 2014. Retrieved from http://www.dec-sped.org/recommendedpractices
4.
Division for Early Childhood/National Association for the Education of Young Children. (2009). Early childhood inclusion: A joint position statement of the Division for Early Childhood (DEC) and the National Association for the Education of Young Children (NAEYC). Chapel Hill: The University of North Carolina, Frank Porter Graham Child Development Institute.
5.
*FetherstonA. M.SturmeyP. (2014). The effects of behavioral skills training on instructor and learner behavior across responses and skill sets. Research in Developmental Disabilities, 35, 541–562. doi:10.1016/j.ridd.2013.11.006
6.
GarfinkleA. N.SchwartzI. S. (2002). Peer imitation: Increasing social interactions in children with autism and other developmental disabilities in inclusive preschool classrooms. Topics in Early Childhood Special Education, 22, 26–39.
7.
GastD. L.LedfordJ. R. (2014). Applied research in education and behavioral sciences. Single case research methodology: Applications in special education and behavioral sciences, 1–18.
8.
*GianoumisS.SeiverlingL.SturmeyP. (2012). The effects of behavior skills training on correct teacher implementation of natural language paradigm teaching skills and child behavior. Behavioral Interventions, 27, 57–74. doi:10.1002/bin.1334
9.
GreshamF. M.LopezM. F. (1996). Social validation: A unifying concept for school-based consultation research and practice. School Psychology Quarterly, 11, 204–227. doi:10.1037/h0088930
10.
*HallL. J.GrundonG. S.PopeC.RomeroA. B. (2010). Training paraprofessionals to use behavioral strategies when educating learners with autism spectrum disorders across environments. Behavioral Interventions, 25, 37–51. doi:10.1002/bin.294
11.
HanleyG. P. (2010). Toward effective and preferred programming: A case for the objective measurement of social validity with recipients of behavior-change programs. Behavior Analysis in Practice, 3, 13–21. doi:10.1007/BF03391754
12.
HienemanM.DunlapG.KincaidD. (2005). Positive support strategies for students with behavioral disorders in general education settings. Psychology in the Schools, 42, 779–794. doi:10.1002/pits.20112
13.
HornerR. H.CarrE. G.HalleJ.McGeeG.OdomS.WoleryM. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165–179. doi:10.1177/001440290507100203
14.
HurleyJ. J. (2012). Social validity assessment in social competence interventions for preschool children: A review. Topics in Early Childhood Special Education, 32, 164–174. doi:10.1177/0271121412440186
15.
KaiserA. P.HesterP. P. (1994). Generalized effects of enhanced milieu teaching. Journal of Speech, Language, and Hearing Research, 37, 1320–1340. doi:10.1044/jshr.3706.1320
16.
KazdinA. E. (1977). Assessing the clinical or applied importance of behavior change through social validation. Behavior Modification, 1, 427–452. doi:10.1177/014544557714001
17.
KelleyM. L.HefferR. W.GreshamF. M.ElliottS. N. (1989). Development of a modified treatment evaluation inventory. Journal of Psychopathology and Behavioral Assessment, 11, 235–247.
18.
KennedyC. H. (2002). The maintenance of behavior change as an indicator of social validity. Behavior Modification, 26, 594–604. doi:10.1177/014544502236652
19.
KoegelR. L.O’dellM. C.KoegelL. K. (1987). A natural language teaching paradigm for nonverbal autistic children. Journal of Autism and Developmental Disorders, 17, 187–200. doi:10.1007/BF01495055
20.
KoegelR. L.SchreibmanL.GoodA.CernigliaL.MurphyC.KoegelL. K. (1989). How to teach pivotal behaviors to children with autism: A training manual. Santa Barbara: University of California.
21.
KramerJ. M. (2011). Using mixed methods to establish the social validity of a self-report assessment: An illustration using the Child Occupational Self-Assessment (COSA). Journal of Mixed Methods Research, 5, 52–76. doi:10.1177/1558689810386376
22.
KratochwillT. R.HitchcockJ. H.HornerR. H.LevinJ. R.OdomS. L.RindsckopfD. M.ShadishW. R. (2013). Single-case intervention research design standards. Remedial and Special Education, 34, 26–38. doi:10.1177/0741932512452794
23.
LaneJ. D.Lieberman-BetzR.GastD. L. (2016). An analysis of naturalistic interventions for increasing spontaneous expressive language in children with autism spectrum disorder. The Journal of Special Education, 50, 49–61. doi:10.1177/0022466915614837
24.
LangR.MachalicekW.RispoliM.RegesterA. (2009). Training parents to implement communication interventions for children with autism spectrum disorders (ASD): A systematic review. Evidence-based Communication Assessment and Intervention, 3, 174–190.
25.
LedfordJ. R.HallE.ConderE.LaneJ. D. (2016). Research for young children with autism spectrum disorders: Evidence of social and ecological validity. Topics in Early Childhood Special Education, 35, 223–233. doi:10.1177/0271121415585956
26.
LekoM. M. (2014). The value of qualitative methods in social validity research. Remedial and Special Education, 35, 275–286. doi:10.1177/0741932514524002
27.
MartensB. K.WittJ. C.ElliottS. N.DarveauxD. X. (1985). Teacher judgments concerning the acceptability of school-based interventions. Professional Psychology: Research and Practice, 16, 191–198. doi:10.1037/0735-7028.16.2.191
28.
MoherD.LiberatiA.TetzlaffJ.AltmanD. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6, e1000097. doi:10.1371/journal.pmed.1000097
29.
MrachkoA. A.KaczmarekL. A. (2017). Examining paraprofessional interventions to increase social communication for young children with ASD. Topics in Early Childhood Special Education, 37, 4–15. doi:10.1177/0271121416662870
30.
*NeelyL.RispoliM.BolesM.MorinK.GregoriE.NinciJ.Hagan-BurkeS. (2018). Interventionist acquisition of incidental teaching using pyramidal training via telehealth. Behavior Modification. doi:10.1177/0145445518781770
31.
*NeelyL.RispoliM.GerowS.HongE. R. (2016). Preparing interventionists via telepractice in incidental teaching for children with autism. Journal of Behavioral Education, 25, 393–416. doi:10.1007/s10864-016-9250-7
32.
*RobinsonS. E. (2011). Teaching paraprofessionals of students with autism to implement pivotal response treatment in inclusive school settings using a brief video feedback training package. Focus on Autism and Other Developmental Disabilities, 26, 105–118. doi:10.1177/1088357611407063
33.
*RyanC. S.HemmesN. S.SturmeyP.JacobsJ. D.GrommetE. K. (2008). Effects of a brief staff training procedure on instructors’ use of incidental teaching and students’ frequency of initiation toward instructors. Research in Autism Spectrum Disorders, 2, 28–45. doi:10.1016/j.rasd.2007.02.002
34.
SchreibmanL.DawsonG.StahmerA. C.LandaR.RogersS. J.McGeeG. G.. . . HalladayA. (2015). Naturalistic developmental behavioral interventions: Empirically validated treatments for autism spectrum disorder. Journal of Autism and Developmental Disorders, 45, 2411–2428. doi:10.1007/s10803-015-2407-8
35.
*SchwartzI. S.BaerD. M. (1991). Social validity assessments: Is current practice the state of the art?Journal of Applied Behavior Analysis, 24, 189–204.
36.
SeiverlingL.PantelidesM.RuizH. H.SturmeyP. (2010). The effect of behavioral skills training with general-case training on staff chaining of child vocalizations within natural language paradigm. Behavioral Interventions, 25, 53–75. doi:10.1002/bin.293
37.
SnodgrassM. R.ChungM. Y.MeadanH.HalleJ. W. (2018). Social validity in single-case research: A systematic literature review of prevalence and application. Research in Developmental Disabilities, 74, 160–173. doi:10.1016/j.ridd.2018.01.007
38.
SnyderP. A.RakapS.HemmeterM. L.McLaughlinT. W.SandallS.McLeanM. E. (2015). Naturalistic instructional approaches in early learning: A systematic review. Journal of Early Intervention, 37, 69–97. doi:10.1177/1053815115595461
39.
SpearC. F.Strickland-CohenK.RomerN.AlbinR. W. (2013). An examination of social validity within single-case research with students with emotional and behavioral disorders. Remedial and Special Education, 34, 357–370. doi:10.1177/0741932513490809
40.
StrainP. S.BartonE. E.DunlapG. (2012). Lessons learned about the utility of social validity. Education and Treatment of Children, 35, 183–200. doi:10.1353/etc.2012.0007
41.
*SuhrheinrichJ. (2015). A sustainable model for training teachers to use pivotal response training. Autism, 19, 713–723. doi:10.1177/1362361314552200
42.
U.S. Department of Education. (2018). Annual report to congress on the implementation of the individuals with disabilities act. Washington, DC: Author.
43.
VerschuurR.DiddenR.LangR.SigafoosJ.HuskensB. (2014). Pivotal response treatment for children with autism spectrum disorders: A systematic review. Review Journal of Autism and Developmental Disorders, 1, 34–61. doi:10.1007/s40489-013-0008-z
44.
*VismaraL. A.YoungG. S.RogersS. J. (2013). Community dissemination of the early start Denver model: Implications for science and practice. Topics in Early Childhood Special Education, 32, 223–233. doi:10.1177/0271121411409250
45.
*VismaraL. A.YoungG. S.StahmerA. C.GriffithE. M.RogersS. J. (2009). Dissemination of evidence-based practice: Can we train therapists from a distance?Journal of Autism and Developmental Disorders, 39, 1636–1651. doi:10.1007/s10803-009-0796-2
46.
*WainerA. L.PickardK.IngersollB. R. (2017). Using web-based instruction, brief workshops, and remote consultation to teach community-based providers a parent-mediated intervention. Journal of Child and Family Studies, 26, 1592–1602. doi:10.1007/s10826-017-0671-2
WolfM. M. (1978). Social validity: The case for subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11, 203–214.
49.
WongC.OdomS. L.HumeK. A.CoxA. W.FettigA.KucharczykS.. . . SchultzT. R. (2015). Evidence-based practices for children, youth, and young adults with autism spectrum disorder: A comprehensive review. Journal of Autism and Developmental Disorders, 45, 1951–1966. doi:10.1007/s10803-014-2351-z
