Understanding how mental health treatments benefit people who receive treatment comes with a challenge: Often different people involved in treatment have different impressions of the treatment’s ultimate effects. How do people reconcile these different reports to understand the true benefit of treatment? In a series of four experiments, we tested people’s beliefs about how to integrate information from multiple informants for the treatment improvement of child clients. We found that laypeople (Experiments 1, 2, and 3) and professional mental health clinicians (Experiment 4) trust informants they believe to be insightful about the specific disorder but pessimistic about overall improvement. Our findings suggest important future research avenues to better understand how intuitions about reconciling informants influence the process of weighting information from clients and other people involved in their care.
AchenbachT. M. (2006). As others see us: Clinical and research implications of cross-informant correlations for psychopathology. Current Directions in Psychological Science, 15, 94–98. doi:10.1111/j.0963-7214.2006.00414.x
2.
AchenbachT. M.KrukowskiR. A.DumenciL.IvanovaM. Y. (2005). Assessment of adult psychopathology: Meta-analyses and implications of cross-informant correlations. Psychological Bulletin, 131, 361–382. doi:10.1037/0033-2909.131.3.361
3.
AchenbachT. M.McConaughyS. H.HowellC.T. (1987). Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin, 101, 213–232. doi:10.1037/0033-2909.101.2.213
4.
American Psychiatric Association. (2006). American Psychiatric Association Practice Guidelines for the treatment of psychiatric disorders: Compendium 2006. Washington, DC: Author.
5.
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: Author.
ArditteK. A.ÇekD.ShawA. M.TimpanoK. R. (2016). The importance of assessing clinical phenomena in Mechanical Turk research. Psychological Assessment, 28, 684–691. doi:10.1037/pas0000217
8.
BaumeisterR. F.BratslavskyE.FinkenauerC.VohsK. D. (2001). Bad is stronger than good. Review of General Psychology, 5, 323–370. doi:10.1037/1089-2680.5.4.323
9.
BeidasR. S.StewartR. E.WalshL.LucasS.DowneyM. M.JacksonK.. . . MandellD. S. (2015). Free, brief, and validated: Standardized instruments for low-resource mental health settings. Cognitive and Behavioral Practice, 22, 5–19. doi:10.1016/j.cbpra.2014.02.002
10.
Brown-JacobsenA. M.WallaceD. P.WhitesideS. P. H. (2011). Multimethod, multi-informant agreement, and positive predictive value in the identification of child anxiety disorders using the SCAS and ADIS-C. Assessment, 18, 382–392. doi:10.1177/1073191110375792
11.
BuhrmesterM. D.TalaifarS.GoslingS. D. (2018). An evaluation of Amazon’s Mechanical Turk, its rapid rise, and its effective use. Perspectives on Psychological Science, 13, 149–154. doi:10.1177/1745691617706516
12.
Canadian Psychiatric Association. (2005). Clinical practice guidelines: Treatment of schizophrenia. Canadian Journal of Psychiatry, 50, 7S–57S. doi:10.1176/appi.books.9780890425596
13.
ChamblessD. L.OllendickT. H. (2001). Empirically supported psychological interventions: Controversies and evidence. Annual Review of Psychology, 52, 685–716. doi:10.1146/annurev.psych.52.1.685
14.
CronbachL. J.FurbyL. (1970). How we should measure “change”: Or should we?Psychological Bulletin, 74, 68–80. doi:10.1037/h0029382
15.
De Los ReyesA.AlfanoC. A.BeidelD. C. (2011). Are clinicians’ assessments of improvements in children’s functioning “global”?Journal of Clinical Child and Adolescent Psychology, 40, 281–294. doi:10.1080/15374416.2011.546043
16.
De Los ReyesA.AugensteinT. M.WangM.ThomasS. A.DrabickD. A. G.BurgersD.RabinowitzJ. (2015). The validity of the multi-informant approach to assessing child and adolescent mental health. Psychological Bulletin, 141, 858–900. doi:10.1037/a0038498
17.
De Los ReyesA.BunnellB. E.BeidelD.C. (2013). Informant discrepancies in adult social anxiety disorder assessments: Links with contextual variations in observed behavior. Journal of Abnormal Psychology, 122, 376–386. doi:10.1037/a0031150
18.
De Los ReyesA.CookC. R.GreshamF. M.MakolB. A.WangM. (2019). Informant discrepancies in assessments of psychosocial functioning in school-based services and research: Review and directions for future research. Journal of School Psychology, 74, 74–89. doi:10.1016/j.jsp.2019.05.005
19.
De Los ReyesA.HenryD. B.TolanP. H.WakschlagL. S. (2009). Linking informant discrepancies to observed variations in young children’s disruptive behavior. Journal of Abnormal Child Psychology, 37, 637–652. doi:10.1007/s10802-009-9307-3
20.
De Los ReyesA.KazdinA. E. (2005). Informant discrepancies in the assessment of childhood psychopathology: A critical review, theoretical framework, and recommendations for further study. Psychological Bulletin, 131, 483–509. doi:10.1037/0033-2909.131.4.483
21.
De Los ReyesA.KazdinA. E. (2006). Conceptualizing changes in behavior in intervention research: The range of possible changes model. Psychological Review, 113, 554–583. doi:10.1037/0033-295X.113.3.554
22.
De Los ReyesA.KazdinA. E. (2008). When the evidence says, “Yes, no, and maybe so”: Attending to and interpreting inconsistent findings among evidence-based interventions. Current Directions in Psychological Science, 17, 47–51. doi:10.1111/j.1467-8721.2008.00546.x
23.
De Los ReyesA.KundeyS. M.WangM. (2011). The end of the primary outcome measure: A research agenda for constructing its replacement. Clinical Psychology Review, 31, 829–838. doi:10.1016/j.cpr.2011.03.011
24.
De Los ReyesA.LernerM. D.KeeleyL. M.WeberR.DrabickD. A. G.RabinowitzJ.GoodmanK. L. (2019). Improving interpretability of subjective assessments about psychological phenomena: A review and cross-cultural meta-analysis. Review of General Psychology, 23, 293–319. doi:10.1177/1089268019837645
25.
De Los ReyesA.ThomasS. A.GoodmanK. L.KundeyS. M. A. (2013). Principles underlying the use of multiple informants’ reports. Annual Review of Clinical Psychology, 9, 123–149. doi:10.1146/annurev-clinpsy-050212-185617
26.
DemyttenaereK.BruffaertsR.Posada-VillaJ.GasquetI.KovessV.LepineJ.. . . PolidoriG. (2004). Prevalence, severity, and unmet need for treatment of mental disorders in the World Health Organization World Mental Health Surveys. JAMA, 291, 2581–2590. doi:10.1001/jama.291.21.2581
27.
DerosD. E.RaczS. J.LiptonM. F.AugensteinT. M.KarpJ. N.KeeleyL. M.. . . De Los ReyesA. (2018). Multi-informant assessments of adolescent social anxiety: Adding clarity by leveraging reports from unfamiliar peer confederates. Behavior Therapy, 49, 84–98. doi:10.1016/j.beth.2017.05.001
28.
DiBartoloP. M.AlbanoA. M.BarlowD. H.HeimbergR. G. (1998). Cross-informant agreement in the assessment of social phobia in youth. Journal of Abnormal Child Psychology, 26, 213–220. doi:10.1023/A:1022624318795
29.
DoodyR. S.StevensJ. C.BeckC.DubinskyR. M.KayeJ. A.GwytherL. M. S. W.. . . CummingsJ. L. (2001). Practice parameter: Management of dementia (an evidence-based review) Report of the Quality Standards Subcommittee of the American Academy of Neurology. Neurology, 56, 1154–1166. doi:10.1212/WNL.56.9.1154
30.
GrillsA. E.OllendickT. H. (2003). Multiple informant agreement and the Anxiety Disorders Interview Schedule for parents and children. Journal of the American Academy of Child and Adolescent Psychiatry, 42, 30–40. doi:10.1089/cap.2011.0114
31.
HawleyK. M.WeiszJ. R. (2003). Child, parent, and therapist (dis)agreement on target problems in outpatient therapy: The therapist’s dilemma and its implications. Journal of Consulting and Clinical Psychology, 71, 62–70. doi:10.1037//0022-006X.71.1.62
32.
HolmbeckG. N.LiS. T.SchurmanJ. V.FriedmanD.CoakleyR. M. (2002). Collecting and managing multisource and multimethod data in studies of pediatric populations. Journal of Pediatric Psychology, 27, 5–18. doi:10.1093/jpepsy/27.1.5
33.
HunsleyJ.LeeC. M. (2014). Introduction to clinical psychology (2nd ed.). Hoboken, NJ: Wiley.
34.
HunsleyJ.MashE. J. (2007). Evidence-based assessment. Annual Review of Clinical Psychology, 3, 29–51. doi:10.1146/annurev.clinpsy.3.022806.091419
35.
HunsleyJ.MashE. J. (Eds.). (2008). A guide to assessments that work (1st ed.). Oxford, England: Oxford University Press.
36.
ItoT. A.LarsenJ. T.SmithN. K.CacioppoJ. T. (1998). Negative information weighs more heavily on the brain: The negativity bias in evaluative categorizations. Journal of Personality and Social Psychology, 75, 887–900. doi:10.1037/0022-3514.75.4.887
37.
KazdinA. E. (2017). Addressing the treatment gap: A key challenge for extending evidence-based psychosocial interventions. Behaviour Research and Therapy, 88, 7–18. doi:10.1016/j.brat.2016.06.004
38.
KazdinA. E.BlaseS. L. (2011). Rebooting psychotherapy research and practice to reduce the burden of mental illness. Perspectives on Psychological Science, 6, 21–37. doi:10.1177/1745691610393527
39.
KesslerR. C.BerglundP.DemlerO.JinR.MerikangasK. R.WaltersE. E. (2005). Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry, 62, 593–602. doi:10.1001/archpsyc.62.6.593
40.
KramerT. L.PhillipsS. D.HargisM. B.MillerT. L.BurnsB. J.RobbinsJ. M. (2004). Disagreement between parent and adolescent reports of functional impairment. Journal of Child Psychology and Psychiatry, 45, 248–259. doi:10.1111/j.1469-7610.2004.00217.x
41.
LairdR. D.De Los ReyesA. (2013). Testing informant discrepancies as predictors of adolescent psychopathology: Why difference scores cannot tell you what you want to know and how polynomial regression may. Journal of Abnormal Child Psychology, 41, 1–14. doi:10.1007/s10802-012-9659-y
42.
LairdR. D.WeemsC. F. (2011). The equivalence of regression models using difference scores and models using separate scores for each informant: Implications for the study of informant discrepancies. Psychological Assessment, 23, 388–397. doi:10.1037/a0021926
43.
LernerM. D.De Los ReyesA.DrabickD. A. G.GerberA. H.GadowK. D. (2017). Informant discrepancies define discrete, clinically useful autism spectrum disorder subgroups. Journal of Child Psychology and Psychiatry, 58, 829–839. doi:10.1111/jcpp.12730
44.
LilienfeldS. O. (2007). Psychological treatments that cause harm. Perspectives on Psychological Science, 2, 53–70. doi:10.1111/j.1745-6916.2007.00029.x
45.
LoeberR.GreenS. M.LaheyB. B. (1990). Mental health professionals’ perception of the utility of children, mothers, and teachers as informants on childhood psychopathology. Journal of Clinical Child Psychology, 19, 136–143. doi:10.1207/s15374424jccp1902_5
46.
MarshJ. K.RomanoA. L. (2016). Lay judgments of mental health treatment options: The mind vs. body problem. Medical Decision Making: Policy & Practice, 1, 1–12. doi:10.1177/2381468316669361
47.
MasonW.SuriS. (2012). Conducting behavioral research on Amazon’s Mechanical Turk. Behavior Research Methods, 44, 1–23. doi:10.3758/s13428-011-0124-6
48.
McMahonR. J.FrickP. J. (2005). Evidence-based assessment of conduct problems in children and adolescents. Journal of Clinical Child and Adolescent Psychology, 34, 477–505. doi:10.1207/s15374424jccp3403_6
49.
OglesB. M.LambertM. J.WeightD. G.PayneI. R. (1990). Agoraphobia outcome measurement: A review and meta-analysis. Psychological Assessment, 2, 317–325. doi:10.1037/1040-3590.2.3.317
50.
PeerE.VosgerauJ.AcquistiA. (2014). Reputation as a sufficient condition for data quality on Amazon Mechanical Turk. Behavior Research Methods, 46, 1023–1031. doi:10.3758/s13428-013-0434-y
51.
PeetersG.CzapinskiJ. (1990). Positive-negative asymmetry in evaluations: The distinction between affective and informational negativity effects. European Review of Social Psychology, 1, 33–60. doi:10.3389/fpsyg.2018.00213
52.
PelhamW. E.Jr.FabianoG. A.MassettiG. M. (2005). Evidence-based assessment of attention deficit hyperactivity disorder in children and adolescents. Journal of Clinical Child and Adolescent Psychology, 34, 449–476. doi:10.1207/s15374424jccp3403_5
53.
RosenA.ProctorE. K. (2003). Developing practice guidelines for social work intervention: Issues, methods, and research agenda. New York, NY: Columbia University Press.
SkowronskiJ. J.CarlstonD. E. (1989). Negativity and extremity biases in impression formation: A review of explanations. Psychological Bulletin, 105, 131–142. doi:10.1037/0033-2909.105.1.131
56.
Southam-GerowM. A.PrinsteinM. J. (2014). Evidence base updates: The evolution of the evaluation of psychological treatments for children and adolescents. Journal of Clinical Child and Adolescent Psychology, 43, 1–6. doi:10.1080/15374416.2013.855128
57.
VolkmarF.SiegelM.Woodbury-SmithM.KingB.McCrackenJ.StateM. (2014). Practice parameter for the assessment and treatment of children and adolescents with autism spectrum disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 53, 237–257. doi:10.1016/j.jaac.2013.10.013
58.
WeiszJ. R.Jensen DossA.HawleyK. M. (2005). Youth psychotherapy outcome research: A review and critique of the evidence base. Annual Review of Psychology, 56, 337–363. doi:10.1146/annurev.psych.55.090902.141449
59.
WeiszJ. R.WeissB.AlickeM. D.KlotzM. L. (1987). Effectiveness of psychotherapy with children and adolescents: A meta-analysis for clinicians. Journal of Consulting and Clinical Psychology, 55, 542–549. doi:10.1037/0022-006X.55.4.542
60.
YoungstromE. A.FindlingR. L.CalabreseJ. R. (2004). Effects of adolescent manic symptoms on agreement between youth, parent, and teacher ratings of behavior problems. Journal of Affective Disorders, 82, S5–S16. doi:10.1016/j.jad.2004.05.016
61.
YzerbytV. Y.SchadronG.LeyensJ. P.RocherS. (1994). Social judgeability: The impact of meta-informational cues on the use of stereotypes. Journal of Personality and Social Psychology, 66, 48–55. doi:10.1037/0022-3514.66.1.48
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
