In their article on specific learning disorders (SLDs), Toffalini, Giofré, and Cornoldi conducted a profile analysis of index scores from individuals diagnosed with an SLD. Their data interpretation—and subsequent clinical recommendations—have many of the same problems that have plagued the research in subscale score interpretation for decades. In this article, I describe some problems with Toffalini et al.’s analysis and conclusions as well as provide some recommendations for future research and clinical use of subscale scores.
Get full access to this article
View all access options for this article.
References
1.
AdamsH. E.LuscherK. A.BernatJ. A. (2002). The classification of abnormal behavior: An overview. In SutkerP. B.AdamsH. E. (Eds.), Comprehensive handbook of psychopathology (3rd ed., pp. 3–28). New York, NY: Kluwer.
2.
American Educational Research Association, American Psychological Association, National Council on Measurement in Education. (2014). Standards for educational and psychological testing (4th ed.). Washington, DC: Author.
3.
BeaujeanA. A. (20017). Simulating data for clinical research: A tutorial. The Journal of Psychoeducational Assessment. Advance online publication. doi:10.1177/0734282917690302
4.
BradenJ. P.ShawS. R. (2009). Intervention validity of cognitive assessment: Knowns, unknowables, and unknowns. Assessment for Effective Intervention, 34, 106-115. doi:10.1177/1534508407313013
5.
BulutO.DavisonM. L.RodriguezM. C. (2016). Estimating between-person and within-person subscore reliability with profile analysis. Multivariate Behavioral Research, 52, 86–104. doi:10.1080/00273171.2016.1253452
6.
CanivezG. L. (2013). Psychometric versus actuarial interpretation of intelligence and related aptitude batteries. In SaklofskeD. H.SchweanV. L.ReynoldsC. R. (Eds.), The Oxford handbook of child psychological assessment (pp. 84–112). New York, NY: Oxford University Press.
7.
CanivezG. L. (2016). Bifactor modeling in construct validation of multifactored tests: Implications for understanding multidimensional constructs and test interpretation. In SchweizerK.DiStefanoC. (Eds.), Principles and methods of test construction: Standards and recent advancements (pp. 247–271). Gottingen, Germany: Hogrefe.
8.
CrawfordJ. R.GarthwaiteP. H.GaultC. B. (2007). Estimating the percentage of the population with abnormally low scores (or abnormally large score differences) on standardized neuropsychological test batteries: A generic method with applications. Neuropsychology, 21, 419–430. doi:10.1037/0894-4105.21.4.419
9.
GarbH. N.BoyleP. A. (2003). Understanding why some clinicians use pseudoscientific methods: Findings from research on clinical judgment. In LilienfeldS. O.LynnS. J.LohrJ. M. (Eds.), Science and pseudoscience in clinical psychology (pp. 17–38). New York, NY: Guilford Press.
10.
GiofrèD.CornoldiC. (2015). The structure of intelligence in children with specific learning disabilities is different as compared to typically development children. Intelligence, 52, 36–43. doi:10.1016/j.intell.2015.07.002
11.
HayesS. C.NelsonR. O.JarrettR. B. (1987). The treatment utility of assessment: A functional approach to evaluating assessment quality. American Psychologist, 42, 963–974. doi:10.1037/0003-066X.42.11.963
12.
KamphausR. W.WinsorA. P.RoweE. W.KimS. (2012). A history of intelligence test interpretation. In FlanaganD. P.HarrisonP. L. (Eds.), Contemporary intellectual assessment (3rd ed., pp. 56–70). New York, NY: Guilford.
13.
LivingstonR. B.JenningsE.ReynoldsC. R.GrayR. M. (2003). Multivariate analyses of the profile stability of intelligence tests: High for IQs, low to very low for subtest analyses. Archives of Clinical Neuropsychology, 18, 487–507. doi:10.1016/S0887-6177(02)00147-6
14.
LuechtR. M.GierlM. J.TanX.HuffK. (2006, April). Scalability and the development of useful diagnostic scales. Paper presented at the annual Meeting of the National Council on Measurement in Education, San Francisco, CA.
15.
MiciakJ.TaylorW. P.StuebingK. K.FletcherJ. M. (2016). Simulation of LD identification accuracy using a pattern of processing strengths and weaknesses method with multiple measures. Journal of Psychoeducational Assessment. doi:10.1177/0734282916683287
16.
PaunonenS. V. (1991). On the accuracy of ratings of personality by strangers. Journal of Personality and Social Psychology, 61, 471–477. doi:10.1037/0022-3514.61.3.471
17.
ReiseS. P.BonifayW. E.HavilandM. G. (2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95, 129–140. doi:10.1080/00223891.2012.725437
18.
SilversteinA. B. (1989). Reliability and abnormality of scaled-score ranges. Journal of Clinical Psychology, 45, 926–929. doi:10.1002/1097-4679(198911)45:6<926::AID-JCLP2270450615>3.0.CO;2-O
19.
SinharayS. (2010). How often do subscores have added value? Results from operational and simulated data. Journal of Educational Measurement, 47, 150–174. doi:10.1111/j.1745-3984.2010.00106.x
20.
SinharayS.PuhanG.HabermanS. J. (2011). An NCME instructional module on subscores. Educational Measurement: Issues and Practice, 30, 29–40. doi:10.1111/j.1745-3992.2011.00208.x
21.
StuebingK. K.FletcherJ. M.Branum-MartinL.FrancisD. J. (2012). Evaluation of the technical adequacy of three methods for identifying specific learning disabilities based on cognitive discrepancies. School Psychology Review, 41, 3–22.
22.
SwetsJ. A.DawesR. M.MonahanJ. (2000). Psychological science can improve diagnostic decisions. Psychological Science in the Public Interest, 1, 1–26. doi:10.1111/1529-1006.001
23.
TaylorW. P.MiciakJ.FletcherJ. M.FrancisD. J. (in press). Cognitive discrepancy models for specific learning disabilities identification: Simulations of psychometric limitations. Psychological Assessment. doi:10.1037/pas0000356
24.
ToffaliniE.GiofréD.CornoldiC. (2017). Strengths and weaknesses in the intellectual profile of different subtypes of specific learning disorder. Clinical Psychological Science. doi:10.1177/2167702616672038
25.
WatkinsM. W. (2003). IQ subtest analysis: Clinical acumen or clinical illusion?The Scientific Review of Mental Health Practice: Objective Investigations of Controversial and Unorthodox Claims in Clinical Psychology, Psychiatry, and Social Work, 2, 118–141.
26.
WatkinsM. W. (2006). Orthogonal higher order structure of the Wechsler Intelligence Scale for Children–Fourth Edition. Psychological Assessment, 18, 123–125. doi:10.1037/1040-3590.18.1.123
27.
WatkinsM. W.GluttingJ. J.Pui-WaL. (2007). Validity of the full-scale IQ when there is significant variability among WISC-III and WISC-IV factor scores. Applied Neuropsychology, 14, 13–20. doi:10.1080/09084280701280353
28.
WechslerD. (2003). WISC-IV technical and interpretive manual. San Antonio, TX: The Psychological Corporation.