The purpose of the present study was to further investigate the clinical utility of individual and composite indicators within the CPT-3 as embedded validity indicators (EVIs) given the discrepant findings of previous investigations.
Methods:
A total of 201 adults undergoing psychoeducational evaluation for ADHD and/or Specific Learning Disorder (SLD) were divided into credible (n = 159) and non-credible (n = 42) groups based on five criterion measures.
Results:
Receiver operating characteristic curves (ROC) revealed that 5/9 individual indicators and 2/4 composite indicators met minimally acceptable classification accuracy of ≥0.70 (AUC = 0.43–0.78). Individual (0.16–0.45) and composite indicators (0.23–0.35) demonstrated low sensitivity when using cutoffs that maintained specificity ≥90%.
Conclusion:
Given the lack of stability across studies, further research is needed before recommending any specific cutoff be used in clinical practice with individuals seeking psychoeducational assessment.
AlbrechtB.Uebel-von SanderslebenH.WiedmannK.RothenbergerA. (2015). ADHD history of the concept: The case of the continuous performance test. Current Developmental Disorders Reports, 2, 10–22https://doi.org/10.1007/s40474-014-0035-1
2.
BenjaminiY.HochbergY. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1), 289–300. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x
3.
BergerC.LevA.BrawY.ElbaumT.WagnerM.RassovskyY. (2021). Detection of feigned ADHD using the MOXO-d-CPT. Journal of Attention Disorders, 25(7), 1032–1047. https://doi.org/10.1177/1087054719864656
4.
Bing-CanarH.PhillipsM. S.ShieldsA. N.Ogram BuckleyC. M.ChangF.KhanH.SkymbaH. V.OvsiewG. P.ReschZ. J.JennetteK. J.SobleJ. R. (2022). Cross-validation of multiple WAIS-IV Digit Span embedded performance validity indices among a large sample of adult Attention Deficit/Hyperactivity Disorder clinical referrals. Journal of Psychoeducational Assessment, 40, 678–688. https://doi.org/10.1177/07342829221081921
5.
BooneK. B. (2013). Clinical practice of forensic neuropsychology. Guilford Press.
6.
BooneK. B.LuP.HerzbergD. (2002). The Dot Counting Test manual. Los Angeles, CA: Western Psychological Services.
7.
BusseM.WhitesideD. (2012). Detecting suboptimal cognitive effort: Classification accuracy of the Conner’s continuous performance test-II, brief test of attention, and trail making test. The Clinical Neuropsychologist, 26(4), 675–687. https://doi.org/10.1080/13854046.2012.679623
8.
ConnersC. K. (2004). Conners’ continuous performance test (CPT-II): Technical guide and software manual. Multi-Health Systems.
9.
ConnersC. K. (2014). Conners’ continuous performance test – 3rd edition (CPT-3) manual. Multi-Health Systems.
10.
ErdodiL. A.AbeareC. A.LichtensteinJ. D.TysonB. T.KucharskiB.ZuccatoB. G.RothR. M. (2017). Wechsler Adult Intelligence Scale-fourth edition (WAIS-IV) processing speed scores as measures of noncredible responding: The third generation of embedded performance validity indicators. Psychological Assessment, 29(2), 148–157. https://doi.org/10.1037/pas0000319
11.
ErdodiL. A.PelletierC. L.RothR. M. (2018). Elevations on select Conners’ CPT-II scales indicate noncredible responding in adults with traumatic brain injury. Applied Neuropsychology: Adult, 25(1), 19–28. https://doi.org/10.1080/23279095.2016.1232262
12.
ErdodiL. A.RothR. M.KirschN. L.Lajiness-O’NeillR.MedoffB. (2014). Aggregating validity indicators embedded in conners‘ CPT-II outperforms individual cutoffs at separating valid from invalid performance in adults with traumatic brain injury. Archives of Clinical Neuropsychology, 29(5), 456–466. https://doi.org/10.1093/arclin/acu026
13.
FrazierT. W.FrazierA. R.BuschR. M.KerwoodM. A.DemareeH. A. (2008). Detection of simulated ADHD and reading disorder using symptom validity measures. Archives of Clinical Neuropsychology, 23(5), 501–509. https://doi.org/10.1016/j.acn.2008.04.001
14.
GreenP. (2004). Green’s Medical Symptom Validity Test (MSVT) for Microsoft Windows: User’s manual. Edmonton, Canada: Green’s Publishing.
15.
GreveK. W.BianchiniK. J. (2004). Setting empirical cut-offs on psychometric indicators of negative response bias: A methodological commentary with recommendations. Archives of Clinical Neuropsychology, 19, 533–541.
16.
HarrisonA. G.BealA. L.ArmstrongI. T. (2021). Predictive value of performance validity testing and symptom validity testing in psychoeducational assessment. Applied Neuropsychology Adult. Advance online publication. https://doi.org/10.1080/23279095.2021.1943396
17.
HarrisonA. G.EdwardsM. J.ArmstrongI.ParkerK. C. (2010). An investigation of methods to detect feigned reading disabilities. Archives of Clinical Neuropsychology, 25(2), 89–98.
18.
HarrisonA. G.EdwardsM. J.ParkerK. C. (2008). Identifying students feigning dyslexia: Preliminary findings and strategies for detection. Dyslexia, 14(3), 228–246.
19.
HosmerD. W. J.LemeshowS.SturdivantR. X. (2013). Applied logistic regression. New York: John Wiley & Sons.
20.
LangeR. T.IversonG. L.BrickellT. A.StaverT.PancholiS.BhagwatA.FrenchL. M. (2013). Clinical utility of the Conners’ continuous performance test-II to detect poor effort in U.S. Military personnel following traumatic brain injury. Psychological Assessment, 25(2), 339–352. https://doi.org/10.1037/a0030915
21.
LippaS. M. (2018). Performance validity testing in neuropsychology: A clinical guide, critical review, and update on a rapidly evolving literature. The Clinical Neuropsychologist, 32(3), 391–421. https://doi.org/10.1080/13854046.2017.1406146
22.
LovettB. J.HarrisonA. G. (2021). Assessing adult ADHD: New research and perspectives. Journal of Clinical and Experimental Neuropsychology, 43(4), 333–339. https://doi.org/10.1080/13803395.2021.1950640
23.
MarshallP.SchroederR.O’BrienJ.FischerR.RiesA.BlesiB.BarkerJ. (2010). Effectiveness of symptom validity measures in identifying cognitive and behavioral symptom exaggeration in adult attention deficit hyperactivity disorder. The Clinical Neuropsychologist, 24(7), 1204–1237. https://doi.org/10.1080/13854046.2010.514290
24.
MieleA. S.GunnerJ. H.LynchJ. K.McCaffreyR. J. (2012). Are embedded validity indices equivalent to free-standing symptom validity tests?Archives of clinical Neuropsychology, 27(1), 10–22. https://doi.org/10.1093/arclin/acr084
25.
OrdA. S.MiskeyH. M.LadS.RichterB.NagyK.ShuraR. D. (2021). Examining embedded validity indicators in Conners continuous performance test-3 (CPT-3). The Clinical Neuropsychologist, 35(8), 1426–1441. https://doi.org/10.1080/13854046.2020.1751301
26.
OrdJ. S.BoettcherA. C.GreveK. W.BianchiniK. J. (2010). Detection of malingering in mild traumatic brain injury with the Conners’ continuous performance test–II. Journal of Clinical and Experimental Neuropsychology, 32(4), 380–387. https://doi.org/10.1080/13803390903066881
27.
SchroederR. W.MartinP. K.HeinrichsR. J.BaadeL. E. (2018). Research methods in performance validity testing studies: Criterion grouping approach impacts study outcomes. The Clinical Neuropsychologist, 33(3), 466–477. https://doi.org/10.1080/13854046.2018.1484517
28.
ScimecaL. M.HolbrookL.RhoadsT.CernyB. M.JennetteK. J.ReschZ. J.ObolskyM. A.OvsiewG. P.SobleJ. R. (2021). Examining Conners continuous performance test-3 (CPT-3) embedded performance validity indicators in an adult clinical sample referred for ADHD evaluation. Developmental Neuropsychology, 46(5), 347–359. https://doi.org/10.1080/87565641.2021.1951270
29.
SharlandM. J.WaringS. C.JohnsonB. P.TaranA. M.RusinT. A.PattockA. M.PalcherJ. A. (2018). Further examination of embedded performance validity indicators for the Conners’ continuous performance test and brief test of attention in a large outpatient clinical sample. The Clinical Neuropsychologist, 32(1), 98–108. https://doi.org/10.1080/13854046.2017.1332240
30.
ShermanE.SlickD. J.IversonG. L. (2020). Multidimensional Malingering Criteria for Neuropsychological Assessment: A 20-Year Update of the Malingered Neuropsychological Dysfunction Criteria. Archives of Clinical Neuropsychology, 35(6), 735–764. https://doi.org/10.1093/arclin/acaa019
31.
ShuraR. D.MiskeyH. M.RowlandJ. A.Yoash-GantzR. E.DenningJ. H. (2016). Embedded performance validity measures with post deployment veterans: Cross-validation and efficiency with multiple measures. Applied Neuropsychology: Adult, 23(2), 94–104. https://doi.org/10.1080/23279095.2015.1014556
32.
SuhrJ. A.BerryD. T. R. (2017). The importance of assessing for validity of symptom report and performance in attention deficit/hyperactivity disorder (ADHD): Introduction to the special section on noncredible presentation in ADHD. Psychological Assessment, 29(12), 1427–1428. https://doi.org/10.1037/pas0000535
33.
SweetJ. J.BensonL. M.NelsonN. W.MobergP. J. (2015). The American Academy of Clinical Neuropsychology, National Academy of Neuropsychology, and Society for Clinical Neuropsychology (APA division 40) 2015 TCN professional practice and ‘salary survey’: Professional practices, beliefs, and incomes of U.S. neuropsychologists. The Clinical Neuropsychologist, 29(8), 1069–1162. https://doi.org/10.1080/13854046.2016.1140228
34.
SweetJ. J.KlipfelK. M.NelsonN. W.MobergP. J. (2021). Professional practices, beliefs, and incomes of U.S. neuropsychologists: The AACN, NAN, SCN 2020 practice and “salary survey”. The Clinical Neuropsychologist, 35(1), 7–80. https://doi.org/10.1080/13854046.2020.1849803
35.
TombaughT. N. (1996). TOMM: Test of memory malingering. Multi-Health Systems.
36.
WhiteD. J.KorinekD.BernsteinM. T.OvsiewG. P.ReschZ. J.SobleJ. R. (2020). Cross-validation of non-memory-based embedded performance validity tests for detecting invalid performance among patients with and without neurocognitive impairment. Journal of Clinical and Experimental Neuropsychology, 42(5), 459–472. https://doi.org/10.1080/13803395.2020.1758634
37.
WinterD.BrawY. (2022). Validating embedded validity indicators of feigned ADHD-associated cognitive impairment using the MOXO-d-CPT. Journal of Attention Disorders. Advance online publication. https://doi.org/10.1177/10870547221112947
