Decades of clinical psychological science have produced empirically supported treatments that are now undergoing dissemination and implementation (DI) but with little guidance from a science that is just taking shape. Charting a future for DI science (DIS) and DI practice (DIP), and their complex relationship, will be complicated by significant challenges—the implementation cliff (intervention benefit drops when tested practices are scaled up), low relevance of most clinical research to actual practice, and differing timetables and goals for DIP versus DIS. To address the challenges, and prepare the next generation of clinical psychological scientists, we propose the following: making intervention research look more like practice, solving the “too many empirically supported treatments” problem, addressing mismatches between interventions and their users (e.g., clients, therapists), broadening the array of intervention delivery systems, sharpening outcome monitoring and feedback, incentivizing high-risk/high-gain innovations, designing new professional tracks, and synchronizing and linking the often-insular practice and science of DI.
AaronsG. A.GlissonC.GreenP. D.HoagwoodK.KelleherK. J.LandsverkJ. A., & Research Network on Youth Mental Health. (2012). The organizational social context of mental services and clinician attitudes toward evidence-based practice: A United States national study. Implementation Science, 7, 56. Retrieved from http://www.implementationscience.com/content/7/1/56
2.
AccursoE. C.TaylorR. M.GarlandA. F. (2011). Evidence-based practices addressed in community-based children’s mental health clinical supervision. Training and Education in Professional Psychology, 5, 88–96. doi:10.1037/a0023537
3.
AddisM. E. (2002). Methods for disseminating research products and increasing evidence-based practice: Promises, obstacles, and future directions. Clinical Psychology: Science and Practice, 9, 367–378. doi:10.1093/clipsy/9.4.367
BakerT. B.McFallR. M.ShohamV. (2009). Current status and future prospects of clinical psychology: Toward a scientifically principled approach to mental and behavioral health care. Psychological Science in the Public Interest, 9, 67–103.
6.
BarlowD. H.FarchioneT. J.FairholmeC. P.EllardK. K.BoisseauC. L.AllenL. B.Ehrenreich-MayJ. (2011). The unified protocol for transdiagnostic intervention of emotional disorders: Therapist guide. New York, NY: Oxford University Press.
7.
BearmanS. K.HerrenJ.Dorf-CaineR.MichlL.ZolothE.JheG.WeiszJ. R. (2011, November). Taking care (with usual care): Common elements across evidence-based interventions for youth depression and intervention as usual in school settings. In Higa-McMillanC. K. (Chair) and GarlandA. (Discussant), Comparing usual care and evidence-based care for children and adolescents: Advancing dissemination in the 21st century. Symposium conducted at the annual meeting of the Association of Behavioral and Cognitive Therapies, Toronto, Ontario, Canada.
8.
BearmanS. K.WeiszJ. R.ChorpitaB. F.HoagwoodK.WardA.UguetoA. M., . . . Research Network on Youth Mental Health. (2013). More practice, less preach? The role of supervision processes and therapist characteristics in EBP implementation. Administration and Policy in Mental Health and Mental Health Services Research. Advance online publication. doi:10.1007/s10488-013-0485-5
9.
BeckerC. B.SticeE.ShawH.WodaS. (2009). Use of empirically supported interventions for psychopathology: Can the participatory approach move us beyond the research-to-practice gap?Behaviour Research and Therapy, 47, 265–274.
10.
BeidasR. S.EdmundsJ. M.MarcusS. C.KendallP. C. (2012). Training and consultation to promote implementation of an empirically supported intervention: A randomized trial. Psychiatric Services, 63, 660–665.
11.
BeidasR. S.KendallP. C. (2010). Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice, 17, 1–30.
12.
BickmanL. (1996). A continuum of care: More is not always better. American Psychologist, 51, 689–701.
13.
BickmanL.KelleyS. D.BredaC.deAndradeA.RiemerM. (2011). Effects of routine feedback to clinicians on mental health outcomes of youths: Results of a randomized trial. Psychiatric Services, 62, 1423–1429.
14.
BradshawC. P.PasE. T.BloomJ.BarrettS.HershfeldtP.AlexanderA.. . . LeafP. J. (2012). A state-wide partnership to promote safe and supportive schools: The PBIS Maryland initiative. Administration and Policy in Mental Health and Mental Health Services Research, 39, 225–237. doi:10.1007/s10488-011-0384-6
15.
Brookman-FrazeeL.StahmerA. C.LewisK.FederJ. D.ReedS. (2012). Building a research-community collaborative to improve community care for infants and toddlers at-risk for autism spectrum disorders. Journal of Community Psychology, 40, 715–734. doi:10.1002/jcop.21501
16.
BuchananR.ChamberlainP.PriceJ. M.SprengelmeyerP. (2013). Examining the equivalence of fidelity over two generations of KEEP implementation: A preliminary analysis. Children and Youth Services Review, 35, 188–193. doi:10.1016/j.childyouth.2012.10.002
17.
ChamberlainP.RobertsR.JonesH.MarsenichL.SosnaT.PriceJ. M. (2012). Three collaborative models for scaling up evidence-based practices. Administration and Policy in Mental Health and Mental Health Services Research, 39, 278–290. doi:10.1007/s10488-011-0349-9
18.
ChambersD. A.RingeisenH.HickmanE. E. (2005). Federal, state, and foundation initiatives around evidence-based practices for child and adolescent mental health. Child and Adolescent Psychiatric Clinics of North America, 14, 307–327.
19.
ChamblessD. L.BakerM. J.BaucomD. H.BeutlerL. E.CalhounK. S.Crits-ChristophP.. . . WoodyS. R. (1998). Update on empirically validated therapies, II. Clinical Psychologist, 51, 3–15.
20.
ChorpitaB. F.ReiseS.WeiszJ. R.GrubbsK.BeckerK. D.KrullJ. L., & Research Network on Youth Mental Health. (2010). Evaluation of the brief problem checklist: Child and caregiver interviews to measure clinical progress. Journal of Consulting and Clinical Psychology, 78, 526–536.
21.
ChorpitaB. F.WeiszJ. R. (2009). MATCH-ADTC: Modular approach to therapy for children with anxiety, depression, trauma, or conduct problems. Satellite Beach, FL: PracticeWise.
22.
ClarkD. M.LayardR.SmithiesR.RichardD. A.SucklingR.WrightB. (2009). Improving access to psychological therapy: Initial evaluation of two UK demonstration sites. Behaviour Research and Therapy, 47, 910–920. doi:10.1016/j.brat.2009.07.010
23.
CurtisN. M.RonanK. R.BorduinC. M. (2004). Multisystemic intervention: A meta-analysis of outcome studies. Journal of Family Psychology, 18, 411–419. doi:10.1037/0893-3200.18.3.411
24.
DimeffL. A.WoodcockE. A.HarnedM. S.BeadnellB. (2011). Can dialectical behavior therapy be learned in highly structured learning environments? Results from a randomized controlled dissemination trial. Behavior Therapy, 42, 263–275. doi:10.1016/j.beth.2010.06.004
25.
FarchioneT. J.FairholmeC. P.EllardK. K.BoisseauC. L.Thompson-HollandsJ.CarlJ. R.. . . BarlowD. H. (2012). The unified protocol for the transdiagnostic intervention of emotional disorders: A randomized controlled trial. Behavior Therapy, 3, 666–678.
26.
FeilE. G.SprengelmeyerP. G.DavisB.ChamberlainP. (2012). Development and testing of a multimedia Internet-based system for fidelity and monitoring of multidimensional intervention foster care. Journal of Medical Internet Research, 14, 239–248.
27.
FoaE. B.HembreeE. A.CahillS. P.RauchS. A. M.RiggsD. S.FeenyN. C.YadinE. (2005). Randomized trial of prolonged exposure for posttraumatic stress disorder with and without cognitive restructuring: Outcome at academic and community clinics. Journal of Consulting and Clinical Psychology, 73, 953–964.
28.
GarlandA. F.Brookman-FrazeeL.HurlburtM. S.AccursoE. C.ZoffnessR. J.Haine-SchlagelR.GangerB. (2010). Mental health care for children with disruptive behavior problems: A view inside therapists’ offices. Psychiatric Services, 61, 1–8.
29.
GlissonC.HemmelgarnA.GreenP.DukesD.AtkinsonS.WilliamsN. J. (2012). Randomized trial of the Availability, Responsiveness, and Continuity (ARC) organizational intervention with community-based mental health programs and clinicians serving youth. Journal of the American Academy of Child and Adolescent Psychiatry, 51, 780–787.
30.
GlissonC.LandsverkJ.SchoenwaldS.KelleherK.HoagwoodK. E.MaybergS.GreenP. (2008). Assessing the organizational social context (OSC) of mental health services: Implications for research and practice. Administration and Policy in Mental Health and Mental Health Services Research, 35, 98–113. doi:10.1007/s10488-007-0148-5
31.
GlissonC.SchoenwaldS. K.HemmelgarnA.GreenP.DukesD.ArmstrongK. S.ChapmanJ. E. (2010). Randomized trial of MST and ARC in a two-level evidence-based intervention implementation strategy. Journal of Consulting and Clinical Psychology, 78, 537–550. doi:10.1037/a0019160
32.
GlissonC.SchoenwaldS. K.KelleherK.LandsverkJ.HoagwoodK. E.MaybergS., . . . Research Network on Youth Mental Health. (2008). Therapist turnover and new program sustainability in mental health clinics as a function of organizational culture, climate, and service structure. Administration and Policy in Mental Health and Mental Health Services Research, 35, 124–133. doi:10.1007/s10488-007-0152-9
33.
GreenL. A.SeifertC. M. (2005). Translation of research into practice: Why we can’t “just do it.”Journal of the American Board of Family Medicine, 18, 541–545. doi:10.3122/jabfm.18.6.541
34.
GreenhalghT.MacfarlaneF.BateP.KyriakidouO. (2004). Diffusion of innovations in service organizations: Systematic review and recommendations. The Milbank Quarterly, 82, 581–629.
35.
HerschellA. D.KolkoD. J.BaumannB. L.DavisA. C. (2010). The role of therapist training in the implementation of psychosocial interventions: A review and critique with recommendations. Clinical Psychology Review, 30, 448–466.
36.
HoagwoodK.KolkoD. J. (2009). Introduction to the special section on practice contexts: A glimpse into the nether world of public mental health services for children and families. Administration and Policy in Mental Health and Mental Health Services Research, 36, 35–36.
37.
HoagwoodK.OlinS.CleekA. (2013). Beyond context to the skyline: Thinking in 3D. Administration and Policy in Mental Health and Mental Health Services Research, 40, 23–28.
38.
IgnatiusA. (Ed.). (2011). The failure issue: How to understand it, learn from it, and recover from it [Special issue]. Harvard Business Review, 89(4).
39.
IsraelB. A.EngE.SchulzA. J.ParkerE. A. (2005). Methods in community-based participatory research for health. San Francisco, CA: Jossey-Bass.
40.
KarlinB. E.BrownG. K.TrockelM.CunningD.ZeissA. M.TaylorC. B. (2012). National dissemination of cognitive behavioral therapy for depression in the Department of Veterans Affairs health care system: Therapist and patient-level outcomes. Journal of Consulting and Clinical Psychology, 80, 707–718. doi:10.1037/a0029328
41.
KarlinB. E.RuzekJ. I.ChardK. M.EftekhariA.MonsonC. M.HembreeE. A.. . . FoaE. B. (2010). Dissemination of evidence-based psychological interventions for posttraumatic stress disorder in the Veterans Health Administration. Journal of Traumatic Stress, 23, 663–673. doi:10.1002/jts.20588
42.
KazdinA. E.BlaseS. L. (2011). Rebooting psychotherapy research and practice to reduce the burden of mental illness. Perspectives on Psychological Science, 6, 21–37. doi:10.1177/1745691610393527
43.
LandsverkJ.BrownC. H.ReutzJ. R.PalinkasL.HorwitzS. M. (2011). Design elements in implementation research: A structured review of child welfare and child mental health studies. Administration and Policy in Mental Health and Mental Health Services Research, 38, 54–63.
44.
LefflerJ. M.JacksonY.WestA. E.McCartyC. A.AtkinsM. (2012). Training in evidence-based practice across the professional continuum. Professional Psychology: Research and Practice, 44, 20–28.
45.
LewisB. L.HatcherR. L.PateW. E.II. (2005). The practicum experience: A survey of practicum site coordinators. Professional Psychology: Research and Practice, 36, 291–298. doi:10.1037/0735-7028.36.3.291
46.
ManuelJ. K.HagedornH. J.FinneyJ. W. (2011). Implementing evidence-based psychosocial intervention in specialty substance use disorder care. Psychology of Addictive Behaviors, 25, 225–237. doi:10.1037/a0022398
47.
MargisonF.BarkhamM.EvansC.McGrathG.Mellor-ClarkJ.AudinK.ConnellJ. (2000). Measurement and psychotherapy: Evidence-based practice and practice-based evidence. British Journal of Psychiatry, 177, 123–130.
48.
McGrewJ. H.BondG. R.DietzenL.SalyersM. (1994). Measuring the fidelity of implementation of a mental health program model. Journal of Consulting and Clinical Psychology, 62, 670–678. doi:10.1037/0022-006X.62.4.670
49.
McHughR. K.BarlowD. H. (2010). The dissemination and implementation of evidence-based psychological interventions: A review of current efforts. American Psychologist, 73, 73–84.
50.
McLeanC. P.FoaE. B. (2011). Prolonged exposure therapy for post-traumatic stress disorder: A review of evidence and dissemination. Expert Reviews, 11, 1151–1163.
51.
McLeodB. D.WeiszJ. R. (2005). The Therapy Process Observational Coding System–Alliance Scale: Measure characteristics and prediction of outcome in usual clinical practice. Journal of Consulting and Clinical Psychology, 73, 323–333.
52.
McLeodB. D.WeiszJ. R. (2010). Therapy Process Observational Coding System for Child Psychotherapy Strategies Scale. Journal of Clinical Child and Adolescent Psychology, 39, 436–443.
53.
MillerW. R.YahneC. E.MoyersT. B.MartinezJ.PirritanoM. (2004). A randomized trial of methods to help clinicians learn motivational interviewing. Journal of Consulting and Clinical Psychology, 72, 1050–1062.
54.
NathanP. E.GormanJ. M. (Eds.). (2007). A guide to interventions that work (3rd ed.). Oxford, England: Oxford University Press.
PalinkasL. A.AaronsG. A.HorwitzS.ChamberlainP.HurlburtM.LandsverkJ. (2011). Mixed method designs in implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 38, 44–53. doi:10.1007/s10488-010-0314-z
59.
PalinkasL. A.SchoenwaldS. K.HoagwoodK.LandsverkJ.ChorpitaB. F.WeiszJ. R., . . . Research Network on Youth Mental Health. (2008). An ethnographic study of implementation of evidence-based interventions in child mental health: First steps. Psychiatric Services, 59, 738–746.
60.
PerepletchikovaF.TreatT. A.KazdinA. E. (2007). Intervention integrity in psychotherapy research: Analysis of the studies and examination of the associated factors. Journal of Consulting and Clinical Psychology, 75, 829–841.
61.
RaineR.SandersonC.HutchingsA.CarterS.LarkinK.BlackN. (2004). An experimental study of determinants of group judgments in clinical guideline development. Lancet, 364, 429–437.
62.
Rotheram-BorusM. J.SwendemanD.ChorpitaB. F. (2012). Disruptive innovations for designing and diffusing evidence-based interventions. American Psychologist, 67, 463–476. doi:10.1037/a0028180
63.
RuzekJ. I.KarlinB. E.ZeissA. (2012). Implementation of evidence-based psychological interventions in the Veterans Health Administration. In McHughR. K.BarlowD. H. (Eds.), Dissemination and implementation of evidence-based psychological interventions (pp. 78–96). New York, NY: Oxford University Press.
64.
SafranJ. D.AbreuI.OgilvieJ.DeMariaA. (2011). Does psychotherapy research influence the clinical practice of researcher-clinicians?Clinical Psychology: Science and Practice, 18, 357–371.
65.
SchoenwaldS. K. (2010). From policy pinball to purposeful partnership: The policy contexts of multisystemic therapy transport and dissemination. In WeiszJ. R.KazdinA. E. (Eds.), Evidence-based psychotherapies for children and adolescents (2nd ed., pp. 538–556). New York, NY: Guilford Press.
66.
ScottS. (2010). National dissemination of effective parenting programmes to improve child outcomes. British Journal of Psychiatry, 196, 1–3.
67.
SharkinB. S.PlagemanP. M. (2003). What do psychologists think about mandatory continuing education? A survey of Pennsylvania practitioners. Professional Psychology: Research and Practice, 34, 318–323. doi:10.1037/0735-7028.34.3.318
68.
ShimokawaK.LambertM. J.SmartD. W. (2010). Enhancing treatment outcome of patients at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy quality assurance system. Journal of Consulting and Clinical Psychology, 78, 298–311.
69.
ShohamV.InselT. R. (2011). Rebooting for whom? Portfolios, technology, and personalized intervention. Perspectives on Psychological Science, 6, 478–482.
70.
SholomskasD. E.Syracuse-SiewartG.RounsavilleB. J.BallS. A.NuroK. F.CarrollK. M. (2005). We don’t train in vain: A dissemination trial of three strategies of training clinicians in cognitive-behavioral therapy. Journal of Consulting and Clinical Psychology, 73, 106–115.
71.
SilvermanW. K.HinshawS. P. (2008). Evidence-based psychosocial interventions for children and adolescents: A ten-year update. Journal of Clinical Child and Adolescent Psychology, 37, 1–7. doi:10.1080/15374410701817725
72.
SmithD. K.ChamberlainP. (2010). Multidimensional intervention foster care for adolescents: Processes and outcomes. In WeiszJ. R.KazdinA. E. (Eds.), Evidence-based psychotherapies for children and adolescents (2nd ed., pp. 243–258). New York, NY: Guilford Press.
73.
SmithJ. L.CarpenterK. M.AmrheinP. C.BrooksA. C.LevinD.SchreiberE. A.. . . NunesE. V. (2012). Training substance abuse clinicians in motivational interviewing using live supervision via teleconferencing. Journal of Consulting and Clinical Psychology, 80, 450–464. doi:10.1037/a0028176
74.
StewartR. E.ChamblessD. L. (2007). Does psychotherapy research inform intervention decisions in private practice?Journal of Clinical Psychology, 63, 267–281. doi:10.1002/jclp.20347
75.
StewartR. E.StirmanS. W.ChamblessD. L. (2012). A qualitative investigation of practicing psychologists’ attitudes toward research-informed practice: Implications for dissemination strategies. Professional Psychology: Research and Practice, 43, 100–109. doi:10.1037/a0025694.supp
76.
StirmanS. W.KimberlyJ.CookN.CallowayA.CastroF.CharnsM. (2012). The sustainability of new programs and innovations: A review of the empirical literature and recommendations for future research. Implementation Science, 7, 17. Retrieved from http://www.implementationscience.com/content/7/1/17
WampoldB. E.BudgeS. L.LaskaK. M.Del ReA. C.BaardsethT. P.FluckigerC.. . . GunnW. (2011). Evidence-based interventions for depression and anxiety versus intervention-as-usual: A meta-analysis of direct comparisons. Clinical Psychology Review, 31, 1304–1312. doi:10.1016/j.cpr.2011.07.012
79.
WeiszJ. R. (2004). Psychotherapy for children and adolescents: Evidence-based interventions and case examples. New York, NY: Cambridge University Press.
80.
WeiszJ. R.ChorpitaB. F.FryeA.NgM. Y.LauN.BearmanS. K., . . . Research Network on Youth Mental Health. (2011). Youth top problems: Using idiographic, consumer-guided assessment to identify intervention needs and track change during psychotherapy. Journal of Consulting and Clinical Psychology, 79, 369–380.
81.
WeiszJ. R.ChorpitaB. F.PalinkasL.SchoenwaldS. K.MirandaJ.BearmanS. K., . . . Research Network on Youth Mental Health. (2012). Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: A randomized effectiveness trial. Archives of General Psychiatry, 69, 274–282.
82.
WeiszJ. R.GrayJ. S. (2008). Evidence-based psychotherapies for children and adolescents: Data from the present and a model for the future. Child and Adolescent Mental Health, 13, 54–65.
83.
WeiszJ. R.Jensen-DossA.HawleyK. M. (2005). Youth psychotherapy outcome research: A review and critique of the evidence base. Annual Review of Psychology, 56, 337–363.
84.
WeiszJ. R.Jensen-DossA.HawleyK. M. (2006). Evidence-based youth psychotherapies versus usual clinical care: A meta-analysis of direct comparisons. American Psychologist, 61, 671–689.
85.
WeiszJ. R.KazdinA. E. (2010). Evidence-based psychotherapies for children and adolescents (2nd ed.). New York, NY: Guilford Press.
86.
WeiszJ. R.KuppensS.EckshtainD.UguetoA. M.HawleyK. M.Jensen-DossA. (2013). Performance of evidence-based youth psychotherapies compared with usual clinical care: A multilevel meta-analysis. JAMA Psychiatry, 70, 750–761.
87.
WeiszJ. R.UguetoA. M.CheronD. M.HerrenJ. (2013). Evidence-based youth psychotherapy in the mental health ecosystem. Journal of Clinical Child and Adolescent Psychology, 42, 274–286. doi:10.1080/15374416.2013.764824
YatesB. T. (2011). Delivery systems can determine therapy costs and effectiveness, more than type of therapy. Perspectives on Psychological Science, 6, 498–502.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
