This study examined concordance between symptom and performance validity among clinically-referred patients undergoing neuropsychological evaluation for Attention-Deficit/Hyperactivity Disorder (ADHD).
Method:
Data from 203 patients who completed the WAIS-IV Working Memory Index, the Clinical Assessment of Attention Deficit-Adult (CAT-A), and ≥4 criterion performance validity tests (PVTs) were analyzed.
Results:
Symptom and performance validity were concordant in 76% of cases, with the majority being valid performance. Of the remaining 24% of cases with divergent validity findings, patients were more likely to exhibit symptom invalidity (15%) than performance invalidity (9%). Patients demonstrating symptom invalidity endorsed significantly more ADHD symptoms than those with credible symptom reporting (ηp2 = .06–.15), but comparable working memory test performance, whereas patients with performance invalidity had significantly worse working memory performance than those with valid PVT performance (ηp2 = .18).
Conclusion:
Symptom and performance invalidity represent dissociable constructs in patients undergoing neuropsychological evaluation of ADHD and should be evaluated independently.
AaseD. M.SobleJ. R.ShepardP.AkagiK.SchrothC.GreensteinJ. E.ProescherE.PhanK. L. (2020). Concordance of embedded performance and symptom validity tests and associations with mild traumatic brain injury and posttraumatic stress disorder among post-9/11 veterans. Archives of Clinical Neuropsychology. Advance online publication. https://doi.org/10.1093/arclin/acaa053
2.
AlversonW. A.O’RourkeJ. J. F.SobleJ. R. (2019). The word memory test genuine memory impairment profile discriminates genuine memory impairment from invalid performance in a mixed clinical sample with cognitive impairment. The Clinical Neuropsychologist, 33(8), 1420−1435.
3.
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed., DSM-5). American Psychiatric Publishing.
4.
BaileyK. C.SobleJ. R.BainK. M.FullenC. (2018). Embedded performance validity tests in the Hopkins verbal learning test – revised and the brief visuospatial memory test – revised: A replication study. Archives of Clinical Neuropsychology, 33(7), 895−900.
5.
BainK. M.SobleJ. R. (2019). Validation of the advanced clinical solutions word choice test (WCT) in a mixed clinical sample: Establishing classification accuracy, sensitivity/specificity, and cutoff scores. Assessment, 26(7), 1320−1328.
6.
BainK. M.SobleJ. R.WebberT. A.MesserlyJ. M.BaileyK. C.KirtonJ. W.McCoyK. J. M. (2019). Cross-validation of three advanced clinical solutions performance validity tests: Examining combinations of measures to maximize classification of invalid performance. Applied Neuropsychology: Adult. Advance online publication. https://doi.org/10.1080/23279095.2019.1585352
7.
Ben-PorathY. S.TellegenA. (2008/2011). The Minnesota multiphasic personality inventory-2 restructured form (MMPI-2-RF): Manual for administration, scoring, and interpretation. University of Minnesota Press.
8.
BenjaminiY.HochbergY. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B (Methodological), 57(1), 289−300.
9.
BooneK. B. (2012). Clinical practice of forensic neuropsychology: An evidence-based approach. Guilford Press.
10.
BooneK.LuP.HerzbergD. S. (2002a). The b test manual. Western Psychological Services.
11.
BooneK. B.LuP.HerzbergD. S. (2002b). The dot counting test manual. Western Psychological Services.
12.
BooneK. B.LuP.WenJ. (2005). Comparison of various RAVLT scores in the detection of noncredible memory performance. Archives of Clinical Neuropsychology, 20(3), 301−319.
13.
BrackenB. A.BoatwrightB. S. (2005). Clinical assessment of attention-deficit-adult (CAT-A) professional manual. Psychological Assessment Resources.
14.
CookC. M.BolingerE.SuhrJ. (2016). Further validation of the Conner’s adult attention deficit/hyperactivity rating scale infrequency index (CII) for detection of non-credible report of attention deficit/hyperactivity disorder symptoms. Archives of Clinical Neuropsychology, 31(4), 358−364.
15.
CritchfieldE.SobleJ. R.MarceauxJ. C.BainK. M.BaileyK. C.WebberT. A.AlversonW. A.MesserlyJ.GonzalezD. A.O’RourkeJ. J. F. (2019). Cognitive impairment does not cause performance validity failure: Analyzing performance patterns among unimpaired, impaired, and noncredible participants across six tests. The Clinical Neuropsychologist, 33(6), 1083−1101.
16.
DenningJ. H. (2012). The efficiency and accuracy of the test of memory malingering trial 1, errors on the first 10 items of the test of memory malingering, and five embedded measures in predicting invalid test performance. Archives of Clinical Neuropsychology, 27(4), 417−432.
17.
FischerA. B.WatkinsM. W. (2008). ADHD rating scales’ susceptibility to faking in a college student sample. Journal of Postsecondary Education and Disability, 20(2), 81–92.
18.
FuermaierA. B. M.TuchaO.KoertsJ.ButzbachM.WeisbrodM.AschenbrennerS.TuchaL. (2018). Susceptibility of functional impairment scales to noncredible responses in the clinical evaluation of adult ADHD. The Clinical Neuropsychologist, 32(4), 671−680. https://doi.org/10.1080/13854046.2017.1406143
19.
FuermaierA. B. M.TuchaL.KoertsJ.WeisbrodM.GrabemannM.ZimmermannM.MetteC.AschenbrennerS.TuchaO. (2016). Evaluation of the CAARS infrequency index for the detection of noncredible ADHD symptom report in adulthood. Journal of Psychoeducational Assessment, 34(8), 739−750. https://doi.org/10.1177/0734282915626005
20.
GlickmanM. E.RaoS. R.SchultzM. R. (2014). False discovery rate control is a recommended alternative to Bonferroni-type adjustments in health studies. Journal of Clinical Epidemiology, 67(8), 850−857.
21.
GreenP. (2003). Green’s word memory test for Microsoft windows: User’s manual. Green’s Publishing.
22.
GreveK. W.HeinlyM. T.BianchiniK. J.LoveJ. M. (2009). Malingering detection with the Wisconsin card sorting test in mild traumatic brain injury. The Clinical Neuropsychologist, 23(2), 343−362.
23.
HarrisonA. G.ArmstrongI. T. (2016). Development of a symptom validity index to assist in identifying ADHD symptom exaggeration or feigning. The Clinical Neuropsychologist, 30(2), 265−283. https://doi.org/10.1080/13854046.2016.1154188
24.
HirschO.ChristiansenH. (2018). Faking ADHD? Symptom validity testing and its relation to self-reported, observer-reported symptoms, and neuropsychological measures of attention in adults with ADHD. Journal of Attention Disorders, 22(3), 269–280. https://doi.org/10.1177/1087054715596577
25.
JasinskiL. J.HarpJ. P.BerryD. T. R.Shandera-OchsnerA. L.MasonL. H.RanseenJ. D. (2011). Using symptom validity tests to detect malingered ADHD in college students. The Clinical Neuropsychologist, 25(8), 1415−1428. http://doi.org/10.1080/13854046.2011.630024
26.
LarrabeeG. J. (2008). Aggregation across multiple indicators improves the detection of malingering: Relationship to likelihood ratios. The Clinical Neuropsychologist, 22(4), 666−679.
27.
LarrabeeG. J. (2012). Performance validity and symptom validity in neuropsychological assessment. Journal of the International Neuropsychological Society, 18(4), 625−630. https://doi.org/10.1017/s1355617712000240
28.
LarrabeeG. J. (2014). False-positive rates associated with the use of multiple performance and symptom validity tests. Archives of Clinical Neuropsychology, 29(4), 364−373.
29.
MarshallP.SchroederR.O’BrienJ.FischerR.RiesA.BlesiB.BarkerJ. (2010). Effectiveness of symptom validity measures in identifying cognitive and behavioral symptom exaggeration in adult attention deficit hyperactivity disorder. The Clinical Neuropsychologist, 24, 1204−1237. https://doi.org/10.1080/13854046.2010.514290
30.
MeyersJ. E.MillerR. M.ThompsonL. M.ScaleseA. M.AllredB. C.RuppZ. W.DupaixZ. P.Junghyun LeeA. (2014). Using likelihood ratios to detect invalid performance with performance validity measures. Archives of Clinical Neuropsychology, 29(3), 224−235. https://doi.org/10.1093/arclin/acu001
31.
MussoM W.GouvierW. D. (2014). “Why is this so hard?” A review of detection of malingered ADHD in college students. Journal of Attention Disorders, 18(3), 186−201.
32.
NelsonJ. M.LovettB. J. (2019). Assessing ADHD in college students: Integrating multiple evidence sources with symptom and performance validity data. Psychological Assessment, 31(6), 793–804. https://doi.org/10.1037/pas0000702
33.
ParkerA.CorkumP. (2016) ADHD diagnosis: As simple as administering a questionnaire or a complex diagnostic process?Journal of Attention Disorders, 20(6), 478−486.
34.
PoynterK.BooneK. B.ErmsharA.MioraD.CottinghamM.VictorT. L.ZieglerE.ZellerM. A.WrightM. (2019). Wait, there’s a baby in this bath water! Update on quantitative and qualitative cut-offs for Rey 15-item recall and recognition. Archives of Clinical Neuropsychology, 34(8), 1367–1380.
35.
QuinnC. A. (2003). Detection of malingering in assessment of adult ADHD. Archives of Clinical Neuropsychology, 18(4), 379−395.
36.
RabinL. A.PaolilloE.BarrW. B. (2016). Stability in test-usage practices of clinical neuropsychologists in the United States and Canada over a 10-year period: A follow-up survey of INS and NAN members. Archives of Clinical Neuropsychology31(3), 206-230. https://doi.org/10.1093/arclin/acw007
37.
RamsayJ. R. (2015). Psychological assessment of adults with ADHD. In BarkleyR. A. (Ed.), Attention-deficit hyperactivity disorder: A handbook of diagnosis and treatment (4th ed., pp. 455–474). Guilford Press.
38.
ReschZ. J.PhamA. T.AbramsonD. A.WhiteD. J.DeDios-SternS.OvsiewG. P.CastilloL.SobleJ. R. (2020). Examining independent and combined accuracy of embedded performance validity tests in the California verbal learning test-II and brief visuospatial memory test-revised for detecting invalid performance. Applied Neuropsychology: Adult. Advance online publication. https://doi.org/10.1080/23279095.2020.1742718
39.
SchwartzE. S.ErdodiL.RodriguezN.GhoshJ. J.CurtainJ. R.FlashmanL. A.RothR. M. (2016). CVLT-II forced choice recognition trial as an embedded validity indicator: A systematic review of the evidence. Journal of the International Neuropsychological Society, 22(8), 851−858.
40.
ShuraR. D.DenningJ. H.MiskeyH. M.RowlandJ. A. (2017). Symptom and performance validity with veterans assessed for attention-deficit/hyperactivity disorder (ADHD). Psychological Assessment, 29(12), 1458−1465.
41.
Silk-EglitG. M.LynchJ. K.McCaffreyR. J. (2016). Validation of victoria symptom validity test cutoff scores among mild traumatic brain injury litigants using a known-groups design. Archives of Clinical Neuropsychology, 31(3), 231−245.
42.
SilverbergN. D.WertheimerJ. C.FichtenbergN. L. (2007). An effort index for the repeatable battery for the assessment of neuropsychological status (RBANS). The Clinical Neuropsychologist, 21(5), 841−854.
43.
SobleJ. R.BainK. M.BaileyK. C.KirtonJ. W.MarceauxJ. M.CritchfieldE. A.McCoyK. J. M.O’RourkeJ. J. F. (2019). Evaluating the accuracy of the Wechsler memory scale-fourth edition (WMS-IV) logical memory embedded validity index for detecting invalid test performance. Applied Neuropsychology: Adult, 26(4), 311−318.
44.
SollmanM. J.RanseenJ. D.BerryD. T. (2010). Detection of feigned ADHD in college students. Psychological Assessment, 22(2), 325–335. https://doi.org/10.1037/a0018857
45.
SuhrJ. A.BerryD. T. R. (2017). The importance of assessing for validity of symptom report and performance in attention deficit/hyperactivity disorder (ADHD): Introduction to the special section on noncredible presentation in ADHD. Psychological Assessment, 29(12), 1427−1428.
46.
SuhrJ.HammersD.Dobbins-BucklandK.ZimakE.HughesC. (2008). The relationship of malingering test failure to self-reported symptoms and neuropsychological findings in adults referred for ADHD evaluation. Archives of Clinical Neuropsychology, 23(5), 521−530. http://doi.org/10.1016/j.acn.2008.05.003
47.
SullivanB. K.MayK.GalballyL. (2007). Symptom exaggeration by college adults in attention-deficit/hyperactivity disorder and learning disorder assessments. Applied Neuropsychology, 14(3), 189−207. http://doi.org/10.1080/09084280701509083
48.
Van DykeS. A.MillisS. R.AxelrodB. N.HanksR. A. (2013). Assessing effort: Differentiating performance and symptom validity. The Clinical Neuropsychologist, 27(8), 1234–1246. https://doi.org/10.1080/13854046.2013.835447
49.
WebberT. A.CritchfieldE. A.SobleJ. R. (2020). Convergent, discriminant, and concurrent validity of non-memory-based performance validity tests. Assessment, 27(7), 1399−1415.
WhiteD. J.KorinekD.BernsteinM. T.OvsiewG. P.ReschZ. J.SobleJ. R. (2020). Cross-validation of non-memory-based embedded performance validity tests for detecting invalid performance among patients with and without neurocognitive impairment. Journal of Clinical and Experimental Neuropsychology, 42(5), 459−472. https://doi.org/10.1080/13803395.2020.1758634
52.
WilliamsonK. D.CombsH. L.BerryD. T. R.HarpJ. P.MasonL. H.EdmundsonM. (2014). Discriminating among ADHD alone, ADHD with a comorbid psychological disorder, and feigned ADHD in a college sample. The Clinical Neuropsychologist, 28(7), 1182−1196. http://doi.org/10.1080/13854046.2014.956674
