Previous research has yielded mixed findings regarding the validity of the Child Behavior Checklist (CBCL) DSM-Oriented Scales (DOSs) in identifying DSM diagnoses. The present study evaluates the concurrent validity of the DOSs by comparing scale classifications with a gold-standard clinical interview.
Methods
Participants (N = 11,851) were children in the NIH Adolescent Brain Cognitive Development (ABCD) study whose caregivers completed the CBCL and the Kiddie Schedule for Affective Disorders and Schizophrenia (KSADS) semi-structured interview at baseline (9.92 years, SD = 0.63), two-year follow-up (12.03 years, SD = 0.67), or both.
Results
2,321 (19.59%) children at baseline and 1,708 (15.57%) participants at two-year follow-up met criteria for one or more KSADS disorders. Sensitivity of the CBCL relative to KSADS diagnoses was generally low (10.36% - 75.00%), with only two scales above 50%. Specificity was high (92.47% - 97.07%). Positive predictive values were low (0.71% - 48.20%), and negative predictive values were high (89.84% - 99.90%). Area under the curve (AUC) ranged from 0.55 to 0.84.
Discussion
In the present sample, the DOSs showed poor concurrent validity with a gold standard clinical interview. Caution should be exercised when using the CBCL due to the high rates of false negatives.
Plain Language Summary
The Child Behavior Checklist is a commonly used questionnaire for caregivers to rate symptoms of mental health conditions in children. Previous research shows conflicting support for the accuracy of the questionnaire results. This study examined how well scores on the questionnaire matched diagnoses based on a gold-standard diagnostic interview. Results showed the questionnaire identified fewer children with mental health concerns than the diagnostic interview. Given that the CBCL under identified children with mental health concerns, this study urges caution when using the questionnaire as part of a diagnostic evaluation.
AchenbachT. M. (2009). The Achenbach System of Empirically Based Assessment (ASEBA): Development, findings, theory, and applications. University of Vermont Research Center for Children, Youth, & Families.
2.
AchenbachT. M. (2014). DSM-oriented guide for the Achenbach System of Empirically Based Assessment (ASEBA). University of Vermont Research Center for Children, Youth, & Families.
3.
AchenbachT. M.DumenciL.RescorlaL. A. (2003). DSM-oriented and empirically based approaches to constructing scales from the same item pools. Journal of Clinical Child and Adolescent Psychology, 32(3), 328–340. https://doi.org/10.1207/S15374424JCCP3203_02
4.
AebiM.Winkler MetzkeC.SteinhausenH.-C. (2010). Accuracy of the DSM-oriented attention problem scale of the child behavior checklist in diagnosing attention-deficit hyperactivity disorder. Journal of Attention Disorders, 13(5), 454–463. https://doi.org/10.1177/1087054708325739
BarchD. M.AlbaughM. D.AvenevoliS.ChangL.ClarkD. B.GlantzM. D.HudziakJ. J.JerniganT. L.TapertS. F.Yurgelun-ToddD.Alia-KleinN.PotterA. S.PaulusM. P.ProutyD.ZuckerR. A.SherK. J. (2018). Demographic, physical and mental health assessments in the Adolescent Brain and Cognitive Development study: Rationale and description. Developmental Cognitive Neuroscience, 32, 55–66. https://doi.org/10.1016/j.dcn.2017.10.010
7.
BellinaM.BrambillaP.GarzittoM.NegriG. A. L.MolteniM.NobileM. (2013). The ability of CBCL DSM-oriented scales to predict DSM-IV diagnoses in a referred sample of children and adolescents. European Child & Adolescent Psychiatry, 22(4), 235–246. https://doi.org/10.1007/s00787-012-0343-0
ButrynT.BryantL.MarchionniC.SholevarF. (2017). The shortage of psychiatrists and other mental health providers: Causes, current state, and potential solutions. International Journal of Academic Medicine, 3(1), 5. https://doi.org/10.4103/IJAM.IJAM_49_17
10.
CollishawS. (2015). Annual research review: Secular trends in child and adolescent mental health. Journal of Child Psychology and Psychiatry, 56(3), 370–393. https://doi.org/10.1111/jcpp.12372
11.
ÇorbacıoğluŞ. K.AkselG. (2023). Receiver operating characteristic curve analysis in diagnostic accuracy studies: A guide to interpreting the area under the curve value. Turkish Journal of Emergency Medicine, 23(4), 195–198. https://doi.org/10.4103/tjem.tjem_182_23
12.
DuffyK. A.GandhiR.FalkeC.WiglesworthA.MuellerB. A.FiecasM. B.Klimes-DouganB.LucianaM.CullenK. R. (2023). Psychiatric diagnoses and treatment in nine-to ten-year-old participants in the ABCD study. JAACAP Open, 1(1), 36–47. https://doi.org/10.1016/j.jaacop.2023.03.001
13.
DuongM. T.BrunsE. J.LeeK.CoxS.CoifmanJ.MaywormA.LyonA. R. (2021). Rates of mental health service utilization by children and adolescents in schools and other common service settings: A systematic review and meta-analysis. Administration and Policy in Mental Health and Mental Health Services Research, 48(3), 420–439. https://doi.org/10.1007/s10488-020-01080-9
14.
EbesutaniC.BernsteinA.NakamuraB. J.ChorpitaB. F.Higa-McMillanC. K.WeiszJ. R.Research Network on Youth Mental Health. (2010). Concurrent validity of the Child Behavior Checklist DSM-oriented scales: Correspondence with DSM diagnoses and comparison to syndrome scales. Journal of Psychopathology and Behavioral Assessment, 32(3), 373–384. https://doi.org/10.1007/s10862-009-9174-9
15.
EgJ.BilenbergN.CostelloE. J.WesselhoeftR. (2018). Self- and parent-reported depressive symptoms rated by the mood and feelings questionnaire. Psychiatry Research, 268, 419–425. https://doi.org/10.1016/j.psychres.2018.07.016
16.
EwingS. W. F.DashG. F.ThompsonW. K.ReuterC.DiazV. G.AnokhinA.ChangL.CottlerL. B.DowlingG. J.LeBlancK.ZuckerR. A.TapertS. F.BrownS. A.GaravanH. (2022). Measuring retention within the adolescent brain cognitive development (ABCD) SM study. Developmental Cognitive Neuroscience, 54, Article 101081. https://doi.org/10.1016/j.dcn.2022.101081
17.
FletcherM.SilvaS.PanW.Reuter-RiceK. (2025). Trajectory moderators of functional outcomes and ADHD symptoms in children with ADHD. Journal of Attention Disorders, Advanced Online Publication. https://doi.org/10.1177/10870547251367284
18.
GaravanH.BartschH.ConwayK.DecastroA.GoldsteinR. Z.HeeringaS.JerniganT.PotterA.ThompsonW.ZahsD. (2018). Recruiting the ABCD sample: Design considerations and procedures. Developmental Cognitive Neuroscience, 32, 16–22. https://doi.org/10.1016/j.dcn.2018.04.004
19.
HaslamN.McGrathM. J.ViechtbauerW.KuppensP. (2020). Dimensions over categories: A meta-analysis of taxometric research. Psychological Medicine, 50(9), 1418–1432. https://doi.org/10.1017/S003329172000183X
20.
HenriksenM. G.EnglanderM.NordgaardJ. (2022). Methods of data collection in psychopathology: The role of semi-structured, phenomenological interviews. Phenomenology and the Cognitive Sciences, 21(1), 9–30. https://doi.org/10.1007/s11097-021-09730-5
21.
HunsleyJ.MashE. J. (2018). A guide to assessments that work. Oxford University Press.
22.
InselT.CuthbertB.GarveyM.HeinssenR.PineD. S.QuinnK.SanislowC.WangP. (2010). Research Domain Criteria (RDoC): Toward a new classification framework for research on mental disorders. American Journal of Psychiatry, 167(7), 748–751. https://doi.org/10.1176/appi.ajp.2010.09091379
23.
JarbinH.AnderssonM.RåstamM.IvarssonT. (2017). Predictive validity of the K-SADS-PL 2009 version in school-aged and adolescent outpatients. Nordic Journal of Psychiatry, 71(4), 270–276. https://doi.org/10.1080/08039488.2016.1276622
24.
JiangZ.WangD.XuH.ZhangA.ZhaoQ.YanJ.WangX.ZhangW.LiY.YangK.HuS.CuiY.LiY. (2023). Diagnostic efficiency and psychometric properties of CBCL DSM-oriented scales in a large sample of Chinese school-attending students aged 5–16. Asian Journal of Psychiatry, 88, 103724. https://doi.org/10.1016/j.ajp.2023.103724
25.
KasiusM. C.FerdinandR. F.Van Den BergH.VerhulstF. C. (1997). Associations between different diagnostic approaches for child and adolescent psychopathology. Journal of Child Psychology and Psychiatry. https://doi.org/10.1111/j.1469-7610.1997.tb01689.x
26.
KaufmanJ.BirmaherB.BrentD.RaoU.FlynnC.MoreciP.WilliamsonD.RyanN. (1997). Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime Version (K-SADS-PL): Initial reliability and validity data. Journal of the American Academy of Child & Adolescent Psychiatry, 36(7), 980–988. https://doi.org/10.1097/00004583-199707000-00021
27.
KnepleyM. J.KendallP. C.CarperM. M. (2019). An analysis of the Child Behavior Checklist Anxiety Problems Scale’s predictive capabilities. Journal of Psychopathology and Behavioral Assessment, 41(2), 249–256. https://doi.org/10.1007/s10862-019-09722-5
28.
KotovR.KruegerR. F.WatsonD.AchenbachT. M.AlthoffR. R.BagbyR. M.BrownT. A.CarpenterW. T.CaspiA.ClarkL. A.EatonN. R.ForbesM. K.ForbushK. T.GoldbergD.HasinD.HymanS. E.IvanovaM. Y.LynamD. R.MarkonK.WrightA. G. C. (2017). The hierarchical Taxonomy of Psychopathology (HiTOP): A dimensional alternative to traditional nosologies. Journal of Abnormal Psychology, 126(4), 454–477. https://doi.org/10.1037/abn0000258
29.
KuhnM. (2008). Building predictive models in R using the caret package. Journal of Statistical Software, 28(5), 1–26. https://doi.org/10.18637/jss.v028.i05
30.
LacalleM.EzpeletaL.DoménechJ. M. (2012). DSM-oriented Scales of the Child Behavior Checklist and Youth Self-Report in clinically referred Spanish children. Spanish Journal of Psychology, 15(1), 377–387. https://doi.org/10.5209/rev_SJOP.2012.v15.n1.37344
31.
LavigneJ. V.MeyersK. M.FeldmanM. (2016). Systematic review: Classification accuracy of behavioral screening measures for use in integrated primary care settings. Journal of Pediatric Psychology, 41(10), 1091–1109. https://doi.org/10.1093/jpepsy/jsw049
32.
McDonaldE.WhitneyS.HorricksL.LipmanE. L.FerroM. A. (2021). Parent-child agreement on the mini international neuropsychiatric interview for children and adolescents (MINI-KID). Journal of the Canadian Academy of Child and Adolescent Psychiatry, 30(4), 264–272.
33.
NakamuraB. J.EbesutaniC.BernsteinA.ChorpitaB. F. (2009). A psychometric analysis of the child behavior checklist DSM-oriented scales. Journal of Psychopathology and Behavioral Assessment, 31(3), 178–189. https://doi.org/10.1007/s10862-008-9119-8
34.
ObuchowskiN. A.LieberM. L. (1998). Confidence intervals for the receiver operating characteristic area in studies with small samples. Academic Radiology, 5(8), 561–571. https://doi.org/10.1016/s1076-6332(98)80208-0
35.
ParkerA.CorkumP. (2016). ADHD diagnosis: As simple as administering a questionnaire or a complex diagnostic process?Journal of Attention Disorders, 20(6), 478–486. https://doi.org/10.1177/1087054713495736
36.
PliszkaS.AACAP Work Group on Quality Issues. (2007). Practice parameter for the assessment and treatment of children and adolescents with Attention-Deficit/Hyperactivity disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 46(7), 894–921. https://doi.org/10.1097/chi.0b013e318054e724
37.
PriceM.Higa-McMillanC.EbesutaniC.OkamuraK.NakamuraB. J.ChorpitaB. F.WeiszJ. (2013). Symptom differentiation of anxiety and depression across youth development and clinic-referred/nonreferred samples: An examination of competing factor structures of the Child Behavior Checklist DSM-oriented scales. Development and Psychopathology, 25(4pt1), 1005–1015. https://doi.org/10.1017/S0954579413000333
38.
R Core Team. (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/
39.
ReillyM.WeitzmanC. (2024). Mental health screening and measurement in children and adolescents. Pediatric Clinics of North America, 71(6), 1013–1026. https://doi.org/10.1016/j.pcl.2024.07.010
40.
RevelleW. (2024). psych: Procedures for psychological, psychometric, and personality research. Northwestern University, Evanston, Illinois. R package version 2.4.6. https://CRAN.R-project.org/package=psych
41.
RishelC. W.GreenoC.MarcusS. C.ShearM. K.AndersonC. (2005). Use of the Child Behavior Checklist as a diagnostic screening tool in community mental health. Research on Social Work Practice, 15(3), 195–203. https://doi.org/10.1177/1049731504270382
42.
RobinX.TurckN.HainardA.TibertiN.LisacekF.SanchezJ.MüllerM. (2011). pROC: An open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics, 12, 77. https://doi.org/10.1186/1471-2105-12-77
43.
SaitoT.RehmsmeierM. (2015). The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS One, 10(3), e0118432. https://doi.org/10.1371/journal.pone.0118432
44.
SciberrasE.EfronD.SchilpzandE. J.AndersonV.JongelingB.HazellP.UkoumunneO. C.NicholsonJ. M. (2013). The Children’s attention project: A community-based longitudinal study of children with ADHD and non-ADHD controls. BMC Psychiatry, 13(1), 18. https://doi.org/10.1186/1471-244X-13-18
45.
SheldrickR. C.BenneyanJ. C.KissI. G.Briggs-GowanM. J.CopelandW.CarterA. S. (2015). Thresholds and accuracy in screening tools for early detection of psychopathology. Journal of Child Psychology and Psychiatry, 56(9), 936–948. https://doi.org/10.1111/jcpp.12442
46.
SpoelmaM. J.SicouriG. L.FrancisD. A.SongcoA. D.DanielE. K.HudsonJ. L. (2023). Estimated prevalence of depressive disorders in children from 2004 to 2019: A systematic review and meta-analysis. JAMA Pediatrics, 177(10), 1017–1027. https://doi.org/10.1001/jamapediatrics.2023.3221
47.
SteinerH.RemsingL.Work Group on Quality Issues. (2007). Practice parameter for the assessment and treatment of children and adolescents with Oppositional Defiant Disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 46(1), 126–141. https://doi.org/10.1097/01.chi.0000246060.62706.af
48.
StewartL. C.AsadiS.Rodriguez-SeijaC.WilsonS.MicheliniG.KotovR.CiceroD. C.OlinoT. M. (2024). Measurement invariance of the Child Behavior Checklist (CBCL) across race/ethnicity and sex in the Adolescent Brain and Cognitive Development (ABCD) Study. Psychological Assessment, 36(8), 441–451. https://doi.org/10.1037/pas0001319
49.
TanakaM. (2024). Beyond the boundaries: Transitioning from categorical to dimensional paradigms in mental health diagnostics. Advances in Clinical and Experimental Medicine, 33(12), 1295–1301. https://doi.org/10.17219/acem/197425
50.
ThomasR.SandersS.DoustJ.BellerE.GlasziouP. (2015). Prevalence of attention-deficit/hyperactivity disorder: A systematic review and meta-analysis. Pediatrics, 135(4), e994–e1001. https://doi.org/10.1542/peds.2014-3482
51.
ThöneA.-K.Görtz-DortenA.AltenbergerP.DoseC.GeldermannN.HautmannC.JendreizikL. T.TreierA.-K.von WirthE.BanaschewskiT.BrandeisD.MillenetS.HohmannS.BeckerK.KetterJ.HebebrandJ.WenningJ.HoltmannM.LegenbauerT.DöpfnerM. (2020). Toward a dimensional assessment of externalizing disorders in children: Reliability and validity of a semi-structured parent interview. Frontiers in Psychology, 11, 1840. https://doi.org/10.3389/fpsyg.2020.01840
52.
TkaczJ.BradyB. L. (2021). Increasing rate of diagnosed childhood mental illness in the United States: Incidence, prevalence and costs. Public Health in Practice (Oxford, England), 2, 100204. https://doi.org/10.1016/j.puhip.2021.100204
53.
TownsendL.KobakK.KearneyC.MilhamM.AndreottiC.EscaleraJ.AlexanderL.GillM. K.BirmaherB.SylvesterR.RiceD.DeepA.KaufmanJ. (2020). Development of three web-based computerized versions of the kiddie schedule for affective disorders and schizophrenia child psychiatric diagnostic interview: Preliminary validity data. Journal of the American Academy of Child & Adolescent Psychiatry, 59(2), 309–325. https://doi.org/10.1016/j.jaac.2019.05.009
54.
TsamakisK.TsiptsiosD.OuranidisA.MuellerC.SchizasD.TerniotisC.NikolakakisN.TyrosG.KympouropoulosS.LazarisA.SpandidosD. A.SmyrnisN.RizosE. (2021). COVID-19 and its consequences on mental health (Review). Experimental and Therapeutic Medicine, 21(3), 1. https://doi.org/10.3892/etm.2021.9675
55.
VolkowN. D.KoobG. F.CroyleR. T.BianchiD. W.GordonJ. A.KoroshetzW. J.Pérez-StableE. J.RileyW. T.BlochM. H.ConwayK.DeedsB. G.DowlingG. J.GrantS.HowlettK. D.MatochikJ. A.MorganG. D.MurrayM. M.NoronhaA.SpongC. Y.WeissS. R. B. (2018). The conception of the ABCD study: From substance use to broad NIH collaboration. Developmental Cognitive Neuroscience, 32, 4–7. https://doi.org/10.1016/j.dcn.2017.10.002
56.
WalterH. J.AbrightA. R.BuksteinO. G.DiamondJ.KeableH.Ripperger-SuhlerJ.RockhillC. (2023). Clinical practice guideline for the assessment and treatment of children and adolescents with major and persistent depressive disorders. Journal of the American Academy of Child & Adolescent Psychiatry, 62(5), 479–502. https://doi.org/10.1016/j.jaac.2022.10.001
57.
WalterH. J.BuksteinO. G.AbrightA. R.KeableH.RamtekkarU.Ripperger-SuhlerJ.RockhillC. (2020). Clinical practice guideline for the assessment and treatment of children and adolescents with anxiety disorders. Journal of the American Academy of Child & Adolescent Psychiatry, 59(10), 1107–1124. https://doi.org/10.1016/j.jaac.2020.05.005
58.
WhitneyD. G.PetersonM. D. (2019). US national and state-level prevalence of mental health disorders and disparities of mental health care use in children. JAMA Pediatrics, 173(4), 389–391. https://doi.org/10.1001/jamapediatrics.2018.5399
59.
WongH. B.LimG. H. (2011). Measures of diagnostic accuracy: Sensitivity, specificity, PPV and NPV. Proceedings of Singapore Healthcare, 20(4), 316–318. https://doi.org/10.1177/201010581102000411
60.
World Health Organization. (2019). International statistical classification of diseases and related health problems (11th ed.). https://icd.who.int/
61.
YeeA. M.Perez AlgortaG.YoungstromE. A.FindlingB. B.FristadM. A.The LAMS Group (2015). Unfiltered administration of the YMRS and the CDRS-R in a clinical sample of children. Journal of Clinical Child & Adolescent Psychology, 44(6), 992–1007. https://doi.org/10.1080/15374416.2014.915548
62.
YuG.WuX.LiuZ.BeckerB.KuangN.KangJ.ZhangJ. (2021). Common and disorders-specific cortical thickness alterations in internalizing, externalizing and thought disorders in the preadolescents of the ABCD study. medRxiv, 2021-09. https://doi.org/10.1101/2021.09.02.21263005
