Psychosocial interventions have advanced remarkably for the treatment of a broad range of psychiatric disorders and related sources of impairment among children, adolescents, and adults. By one count, over 320 interventions have been identified as evidence-based in light of rigorously controlled studies and replication of treatment effects. Yet, how evidence-based psychotherapies have been evaluated and reported raises questions about their impact. This article evaluates both methodological and substantive issues that limit what can be stated about evidence-based psychotherapies and their effects. Among the methodological topics are the control conditions to which evidence-based psychotherapies are compared, selective reporting of measures, and the limited evidence that evidence-based psychotherapies have clinically significant impact. Among the substantive issues are the paucity of research on moderators that would help us better direct patients to treatments from which they are likely to profit and our limited understanding of the mechanisms responsible for therapeutic change. The issues discussed are fundamental to what can be stated about the impact of evidence-based psychotherapies and impact and the bases for their effects. It is not clear at present whether concerted efforts are in place to alter research in ways that would redress the issues. There are, however, novel new directions for research that build on the evidence-based psychotherapies, and these are addressed in a companion article.
AchenbachT. M.KrukowskiR. A.DumenciL.IvanovaM. Y. (2005). Assessment of adult psychopathology: Meta-analyses and implications of cross-informant correlations. Psychological Bulletin, 131, 361–382.
2.
American Psychological Association Presidential Task Force on Evidence-Based Practice. (2006). Evidence-based practice in psychology. American Psychologist, 61, 271–285.
3.
BaldwinS. A.ChristianS.BerkeljonA.ShadishW. R. (2012). The effects of family therapies for adolescent delinquency and substance abuse: A meta-analysis. Journal of Marital and Family Therapy, 38, 281–304.
4.
BaskinT. W.TierneyS. C.MinamiT.WampoldB. E. (2003). Establishing specificity in psychotherapy: A meta-analysis of structural equivalence of placebo controls. Journal of Consulting and Clinical Psychology, 71, 973–979.
5.
BenedettiF. (2009). Placebo effects: Understanding the mechanisms in health and disease. New York, NY: Oxford University Press.
6.
BlantonH.JaccardJ. (2006). Arbitrary metrics in psychology. American Psychologist, 61, 27–41.
7.
BootW. R.SimonsD. J.StothartC.StuttsC. (2013). The pervasive problem with placebos in psychology: Why active control groups are not sufficient to rule out placebo effects. Perspectives on Psychological Science, 8, 445–454.
8.
BullockJ. G.GreenD. P.HaS. E. (2010). Yes, but what’s the mechanisms? Don’t expect an easy answer. Journal of Personality and Social Psychology, 98, 550–558.
9.
CahillJ.BarkhamM.StilesW. B. (2010). Systematic review of practice-based research on psychological therapies in routine clinic settings. British Journal of Clinical Psychology, 49, 421–453.
10.
CavanaghK.StraussC.ForderL.JonesF. (2014). Can mindfulness and acceptance be learnt by self-help? A systematic review and meta-analysis of mindfulness and acceptance-based self-help interventions. Clinical Psychology Review, 34, 118–129.
11.
ChamblessD. L.OllendickT. H. (2001). Empirically supported psychological interventions: Controversies and evidence. Annual Review of Psychology, 52, 685–716.
12.
ChristophersonE. R.VanscoyocS. (2013). Treatments that work with children: Empirically supported strategies for managing childhood problems (2nd ed.). Washington, DC: American Psychological Association.
13.
CohenJ. S. (1988). Statistical power analysis in the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
14.
CooperH.VandenBosG. R. (2013). Archives of Scientific Psychology: A new journal for a new era. Archives of Scientific Psychology, 1(1), 1–6.
15.
DavisM. (2011). NMDA receptors and fear extinction: Implications for cognitive behavioral therapy. Dialogues in Clinical Neuroscience, 13, 463–474.
16.
DawsonG.RogersS.MunsonJ.SmithM.WinterJ.GreensonJ.. . . VarleyJ. (2010). Randomized controlled trial of an intervention for toddlers with Autism: The Early Start Denver Model. Pediatrics, 125, e17–e23.
17.
De AngelisC. D.DrazenJ. M.FrizelleF. A.HaugC.HoeyJ.HortonR.WeydenM. B. V. D. (2005). Is this clinical trial fully registered?—A statement from the International Committee of Medical Journal Editors. New England Journal of Medicine, 352, 2436–2438.
18.
De Los ReyesA.KazdinA. E. (2006). Conceptualizing changes in behavior in intervention research: The Range of Possible Changes Model. Psychological Review, 113, 554–583.
19.
De Los ReyesA.KazdinA. E. (2008). When the evidence says, “yes, no, and maybe so”: Attending to and interpreting inconsistent findings among evidence-based interventions. Current Directions in Psychological Science, 17, 47–51.
20.
De Los ReyesA.ThomasS. A.GoodmanK. L.KundeyS. M. (2013). Principles underlying the use of multiple informants’ reports. Annual Review of Clinical Psychology, 9, 123–149.
21.
DeRubeisR. J.CohenZ. D.ForandN. R.FournierJ. C.GelfandL. A.Lorenzo-LuacesL. (2014). The Personalized Advantage Index: Translating research on prediction into individualized treatment recommendations: A demonstration. PloS One, 9(1), e83875.
22.
DoehrmannO.GhoshS. S.PolliF. E.ReynoldsG. O.HornF.KeshavanA.. . . GabrieliJ. D. (2013). Predicting treatment response in social anxiety disorder from functional magnetic resonance imaging. JAMA Psychiatry, 70, 87–97.
23.
DriessenE.VanH. L.DonF. J.PeenJ.KookS.WestraD.. . . DekkerJ. J. M. (2013). The efficacy of cognitive-behavioral therapy and psychodynamic therapy in the outpatient treatment of major depression: A randomized clinical trial. American Journal of Psychiatry, 170, 1041–1050.
24.
DrussB. G.HwangI.PetukhovaM.SampsonN. A.WangP. S.KesslerR. C. (2009). Impairment in role functioning in mental and chronic medical disorders in the US: Results from the National Comorbidity Survey Replication. Molecular Psychiatry, 14, 728–737.
25.
DumanC. H.SchlesingerL.RussellD. S.DumanR. S. (2008). Voluntary exercise produces antidepressant and anxiolytic behavioral effects in mice. Brain Research, 1199, 148–158.
26.
DuncanB. L.MillerS. D.WampoldB. E.HubbleM. A. (Eds.). (2010). The heart and soul of change: Delivering what works in therapy (2nd ed.). Washington, DC: American Psychological Association.
27.
DwanK.AltmanD. G.ArnaizJ. A.BloomJ.ChanA. W.CroninE.. . . WilliamsonP. R. (2008). Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PloS One, 3, e3081.
28.
EichE. P. R. (2014). Business not as usual. Psychological Science, 25, 3–6.
29.
FinnissD. G.KaptchukT. J.MillerF.BenedettiF. (2010). Biological, clinical, and ethical advances of placebo effects. Lancet, 375, 686–695.
30.
FrankE.CassanoG. B.RucciP.ThompsonW. K.KraemerH. C.FagioliniA.. . . ForgioneR. N. (2011). Predictors and moderators of time to remission of major depression with interpersonal psychotherapy and SSRI pharmacotherapy. Psychological Medicine, 41, 151–162.
31.
FreedlandK. E.MohrD. C.DavidsonK. W.SchwartzJ. E. (2011). Usual and unusual care: Existing practice control groups in randomized controlled trials of behavioral interventions. Psychosomatic Medicine, 73, 323–335.
32.
FrewenP. A.DozoisD. J. A.LaniusR. A. (2008). Neuroimaging studies of psychological interventions for mood and anxiety disorders: Empirical and methodological review. Clinical Psychology Review, 28, 228–246.
33.
FureyM. L.DrevetsW. C.HoffmanE. M.FrankelE.SpeerA. M.ZarateC. A. (2013). Potential of pretreatment neural activity in the visual cortex during emotional processing to predict treatment response to scopolamine in major depressive disorder. JAMA Psychiatry, 70, 280–290.
34.
GrantP. M.BeckA. T. (2013). Cognitive therapy in patients with schizophrenia—Reply. JAMA Psychiatry, 70, 543–544.
35.
GrissomR. J. (1996). The magical number .7 +/− .2: Meta-meta-analysis of the probability of superior outcome in comparisons involving therapy, placebo, and control. Journal of Consulting and Clinical Psychology, 64, 973–982.
36.
GrissomR. J.KimJ. J. (2011). Effect sizes for research: Univariate and multivariate applications (2nd ed.). New York, NY: Taylor & Francis.
37.
HarveyA. G.LeeJ.WilliamsJ.HollonS. D.WalkerM. P.ThompsonM. A.SmithR. (2014). Improving outcome of psychosocial treatments by enhancing memory and learning. Perspectives on Psychological Science, 9, 161–179.
38.
HillC. E.ChuiH.BaumannE. (2013). Revisiting and re envisioning the outcome problem in psychotherapy: An argument to include individualized and qualitative measurement. Psychotherapy, 50, 68–76.
39.
JacobsonN. S.RobertsL. J.BernsS. B.McGlincheyJ. (1999). Methods for defining and determining the clinical significance of treatment effects in mental health research: Current status, new applications, and future directions. Journal of Consulting and Clinical Psychology, 67, 300–307.
40.
JohnL. K.LoewensteinG.PrelecD. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23, 524–532.
41.
JooberR.SchmitzN.AnnableL.BoksaP. (2012). Publication bias: What are the challenges and can they be overcome?Journal of Psychiatry & Neuroscience, 37, 149–152.
42.
Kam-HansenS.JakubowskiM.KelleyJ. M.KirschI.HoaglinD. C.KaptchukT. J.BursteinR. (2014). Altered placebo and drug labeling changes the outcome of episodic migraine attacks. Science Translational Medicine, 6, 218ra5.
43.
KaplanR. M. (2008). Special issue: Mediation and moderation. Health Psychology, 27(2, Suppl.), S99–S100.
44.
KazantzisN. (2000). Power to detect homework effects in psychotherapy outcome research. Journal of Consulting and Clinical Psychology, 68, 166–170.
45.
KazdinA. E. (2010). Problem-solving skills training and parent management training for Oppositional Defiant Disorder and Conduct Disorder. In WeiszJ. R.KazdinA. E. (Eds.), Evidence-based psychotherapies for children and adolescents (2nd ed., pp. 211–226). New York, NY: Guilford Press.
46.
KazdinA. E. (2013). Evidence-based treatment and usual care: Cautions and qualifications. JAMA Psychiatry, 70, 666–667.
47.
KazdinA. E. (2014). Moderators, mediators, and mechanisms of change in psychotherapy. In LutzW.KnoxS. (Eds.), Quantitative and qualitative methods in psychotherapy (pp. 87–101). New York, NY: Taylor & Francis.
48.
KazdinA. E. (in press a) Evidence-based psychotherapies II: New directions to extend impact of treatment. South African Journal of Psychology.
49.
KazdinA. E. (in press b) Research design in clinical psychology (5th ed.). Boston, MA: Pearson.
50.
KazdinA. E.BassD. (1989). Power to detect differences between alternative treatments in comparative psychotherapy outcome research. Journal of Consulting and Clinical Psychology, 57, 138–147.
51.
KazdinA. E.WhitleyM. K. (2006). Comorbidity, case complexity, and effects of evidence-based treatment for children referred for disruptive behavior. Journal of Consulting and Clinical Psychology, 74, 455–467.
52.
KesslerR.Aguilar-GaxiolaS.AlonsoJ.ChatterjiS.LeeS.OrmelJ.. . . WangP. (2009). The global burden of mental disorders: An update from the WHO World Mental Health (WMH) Surveys. Epidemiologia e Psichiatria Sociale, 18, 23–33.
53.
KesslerR. C.DemlerO.FrankR. G.OlfsonM.PincusH. A.WaltersE. E.. . . ZaslavskyA. M. (2005). Prevalence and treatment of mental disorders, 1990 to 2003. New England Journal of Medicine, 352, 2515–2523.
54.
KesslerR. C.WangP. S. (2008). The descriptive epidemiology of commonly occurring mental disorders in the United States. Annual Review of Public Health, 29, 115–129.
55.
KraemerH. C. (2013). Discovering, comparing, and combining moderators of treatment on outcome after randomized clinical trials: A parametric approach. Statistics in Medicine, 32, 1964–1973.
56.
LaineC.GoodmanS. N.GriswoldM. E.SoxH. C. (2007). Reproducible research: Moving toward research the public can really trust. Annals of Internal Medicine, 146, 450–453.
57.
LambertM. J. (Ed.). (2013). Bergin and Garfield’s handbook of psychotherapy and behavior change (6th ed.). New York, NY: John Wiley & Sons.
58.
LambertM. J.OglesB. M. (2013). The efficacy and effectiveness of psychotherapy. In LambertM. J. (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (6th ed., pp. 169–218). New York, NY: John Wiley & Sons.
59.
LeijtenP.RaaijmakersM. A.de CastroB. O.MatthysW. (2013). Does socioeconomic status matter? A meta-analysis on parent training effectiveness for disruptive child behavior. Journal of Clinical Child and Adolescent Psychology, 42, 384–392.
60.
LutzW.HillC. E. (2009). Quantitative and qualitative methods for psychotherapy research: Introduction to special section. Psychotherapy Research, 19, 369–373.
61.
MilesM. B.HubermanA. M.SaldañaJ. (2014). Qualitative data analysis: A methods sourcebook (3rd ed.). Thousand Oaks, CA: SAGE.
62.
MorrisB. (2005). Discovering bits and pieces of me: Research exploring women’s experiences of psychoanalytic psychotherapy. London, England: Women’s Therapy Centre.
63.
MulcahyR.ReayR. E.WilkinsonR. B.OwenC. (2010). A randomised control trial for the effectiveness of group interpersonal psychotherapy for postnatal depression. Archives of Women’s Mental Health, 13, 125–139.
64.
NathanP. E.GormanJ. M. (Eds.). (in press). Treatments that work (4th ed.). New York, NY: Oxford University Press.
65.
NelsonJ. C.DevanandD. P. (2011). A systematic review and meta-analysis of placebo-controlled antidepressant studies in people with depression and dementia. Journal of the American Geriatrics Society, 59, 577–585.
66.
NilssonT.SvenssonM.SandellR.ClintonD. (2007). Patients’ experiences of change in cognitive-behavioral therapy and psychodynamic therapy: A qualitative comparative study. Psychotherapy Research, 17, 553–566.
67.
OglesB. M. (2013). Measuring change in psychotherapy research. In LambertM. J. (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (6th ed., pp. 134–167). Hoboken, NJ: John Wiley & Sons.
68.
PaulG. L. (1966). Insight versus desensitization in psychotherapy: An experiment in anxiety reduction. Stanford, CA: Stanford University Press.
69.
PaulG. L. (1967). Outcome research in psychotherapy. Journal of Consulting Psychology, 31, 109–118.
70.
PortoP. R.OliveiraL.MariJ.VolchanE.FigueiraI.VenturaP. (2009). Does cognitive behavioral therapy change the brain? A systematic review of neuroimaging in anxiety disorders. Journal of Neuropsychiatry and Clinical Neurosciences, 21, 114–125.
71.
QuidéY.WitteveenA. B.El-HageW.VeltmanD. J.OlffM. (2012). Differences between effects of psychological versus pharmacological treatments on functional and morphological brain alterations in anxiety disorders and major depressive disorder: A systematic review. Neuroscience and Biobehavioral Reviews, 36, 626–644.
72.
RichardsD.RichardsonT. (2012). Computer-based psychological treatments for depression: A systematic review and meta-analysis. Clinical Psychology Review, 32, 329–342.
73.
RoffmanJ. L.MarciC. M.GlickD. M.DoughertyD. D.RauchS. L. (2005). Neuroimaging and the functional neuroanatomy of psychotherapy. Psychological Medicine, 35, 1385–1398.
74.
RothA.FonagyP. (2005). What works for whom: A critical review of psychotherapy research (2nd ed.). New York, NY: Guilford Press.
75.
RutledgeT.LohC. (2004). Effect sizes and statistical testing in the determination of clinical significance in behavioral medicine research. Annals of Behavioral Medicine, 27, 138–145.
76.
SenS.DumanR.SanacoraG. (2008). Serum brain-derived neurotrophic factor, depression, and antidepressant medications: Meta-analyses and implications. Biological Psychiatry, 64, 527–532.
77.
SimmonsJ. P.NelsonL. D.SimonsohnU. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359–1366.
78.
SpeerD. C.GreenbaumP. E. (1995). Five methods for computing significant individual client change and improvement rates: Support for an individual growth curve approach. Journal of Consulting and Clinical Psychology, 63, 1044–1048.
79.
SpielmansG. I.KirschI. (2014). Drug approval and drug effectiveness. Annual Review of Clinical Psychology, 10, 741–766.
80.
Substance Abuse and Mental Health Services Administration (SAMHSA). (2011). SAMHSA announces a working definition of “recovery” from mental disorders and substance abuse disorders. Retrieved from www.samhsa.gov/newsroom/advisories/1112223420.aspx
81.
SunS.PanW.WangL. L. (2010). A comprehensive review of effect size reporting and interpreting practices in academic journals in education and psychology. Journal of Educational Psychology, 102, 989–1004.
82.
US Department of Health and Human Services (USDHHS). (2014). Substance Abuse and Mental Health Services Administration (SAMHSA), National Registry of Evidence-based Programs and Practices (NREPP). Retrieved from http://www.nrepp.samhsa.gov/AboutNREPP.aspx
83.
WallaceM. L.FrankE. J.KraemerH. C. (2013). A novel approach for developing and interpreting moderator profiles in randomized controlled trials. JAMA Psychiatry, 70, 1241–1247.
84.
WampoldB. E.BudgeS. L.LaskaK. M.Del ReA. C.BaardsethT. P.GunnW. (2011). Evidence-based interventions for depression and anxiety versus intervention-as usual: A meta-analysis of direct comparisons. Clinical Psychology Review, 31, 1304–1312.
85.
WeiszJ. R.Jensen-DossA. J.HawleyK. M. (2006). Evidence-based youth psychotherapies versus usual clinical care: A meta-analysis of direct comparisons. American Psychologist, 61, 671–689.
86.
WeiszJ. R.KazdinA. E. (Eds.). (2010). Evidence-based psychotherapies for children and adolescents (2nd ed.). New York, NY: Guilford Press.
87.
WeiszJ. R.KuppensS.EckshtainD.UguetoA. M.HawleyK. M.Jensen-DossA. J. (2013). Performance of evidence-based youth psychotherapies compared with usual clinical care: A multilevel meta-analysis. JAMA Psychiatry, 70, 750–761.
88.
WhitleyR.DrakeR. (2010). Recovery: A dimensional approach. Psychiatric Services, 61, 1248–1250.
89.
World Federation for Mental Health. (2011). The great push: Investing in mental health. Occoquan, VA: Author.
90.
World Health Organization (WHO). (2008). The global burden of disease: 2004 update. Geneva, Switzerland: Author.
