Although often applied in practice, clinically based cognitive subtest profile analysis has failed to achieve empirical support. Nonlinear multivariate subtest profile analysis may have benefits over clinically based techniques, but the psychometric properties of these methods must be studied prior to their implementation and interpretation. The current study posed the following question: Is Wechsler Intelligence Scale for Children–Third Edition (WISC-III) cluster membership based on nonlinear multivariate subtest profile analysis stable over a 3-year period? Membership stability to a subtest taxonomy, including constancy of displaying an unusual profile, was based on data from 585 students. General (.39) and partial (.26 to .51) kappa coefficients either failed to reach statistical significance or indicated poor classification stability, with the exception of two profile types. It was concluded that, with these two possible exceptions, profile-type membership to an empirically derived WISC-III subtest taxonomy should not be used in interpretation or educational decision making.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
2.
Canivez, G. L., & Watkins, M. W. (1998). Long-term stability of the Wechsler Intelligence Scale for Children– Third Edition. Psychological Assessment, 10, 285-291.
3.
Canivez, G. L., & Watkins, M. W. (1999). Long term stability of the Wechsler Intelligence Scale for Children– Third Edition among demographic subgroups: Gender, race, and age. Journal of Psychoeducational Assessment, 17, 300-313.
4.
Canivez, G. L., & Watkins, M. W. (2001). Long term stability of the Wechsler Intelligence Scale for Children– Third Edition among students with disabilities. School Psychology Review, 30, 361-376.
5.
Cattell, R. B. (1949). rp and other coefficients of pattern similarity. Psychometrika, 14, 279-298.
6.
Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284-290.
7.
Cronbach, L. J., & Gleser, G. C. (1953). Assessing similarity between profiles. Psychological Bulletin, 50, 456-473.
8.
Davidow, J., & Levinson, E. M. (1993). Heuristic principles and cognitive bias in decision making: Implications for assessment in school psychology. Psychology in the Schools, 30, 351-361.
9.
Donders, J. (1996). Cluster subtypes in the WISC-III standardization sample: Analysis of factor index scores. Psychological Assessment, 8, 312-318.
10.
Elliott, C. D. (1990). Differential Ability Scales: Introductory and technical handbook. San Antonio, TX: Psychological Corporation.
11.
Faust, D. (1986). Research on human judgment and its application to clinical practice. Professional Psychology: Research and Practice, 17, 420-430.
12.
Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76, 378-382.
13.
Glutting, J. J., & McDermott, P. A. (1990a). Patterns and prevalence of core profile types in the WPPSI standardization sample. School Psychology Review, 19, 471-491.
14.
Glutting, J. J., & McDermott, P. A. (1990b). Score structures and applications of core profile types in the McCarthy Scales standardization sample. Journal of Special Education, 24, 212-233.
15.
Glutting, J. J., McDermott, P. A., & Konold, T. R. (1997). Ontology, structure, and diagnostic benefits of a normative subtest taxonomy from the WISC-III standardization sample. In D. P. Flanagan, J. L. Genshaft, & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (pp. 349-372). New York: Guilford.
16.
Glutting, J. J., McDermott, P. A., Konold, T. R., Snelbaker, A. J., & Watkins, M. W. (1998). More ups and downs of subtest analysis: Criterion validity of the DAS with an unselected cohort. School Psychology Review, 27, 599-612.
17.
Glutting, J. J., McDermott, P. A., Prifitera, A., & McGrath, E. A. (1994). Core profile types for the WISC-III and WIAT: Their development and application in identifying multivariate IQ-achievement discrepancies. School Psychology Review, 23, 619-639.
18.
Glutting, J. J., McDermott, P. A., Watkins, M. W., Kush, J. C., & Konold, T. R. (1997). The base rate problem and its consequences for interpreting children's ability profiles. School Psychology Review, 26, 176-188.
19.
Glutting, J. J., McGrath, E. A., Kamphaus, R. W., & McDermott, P. A. (1992). Taxonomy and validity of subtest profiles on the Kaufman Assessment Battery for Children. Journal of Special Education, 26, 85-115.
20.
Holland, A. M., & McDermott, P. A. (1996). Discovering core profile types in the school-age standardization sample of the Differential Ability Scales. Journal of Psychoeducational Assessment, 14, 131-146.
21.
Kamphaus, R. W. (1998). Intelligence test interpretation: Acting in the absence of evidence. In A. Prifitera & D. Saklofske (Eds.), WISC-III clinical use and interpretation: Scientist-practitioner perspectives (pp. 39-57). San Diego, CA: Academic Press.
22.
Kamphaus, R. W., Petoskey, M. D., & Rowe, E. W. (2000). Current trends in psychological testing of children. Professional Psychology: Research and Practice, 31, 155-164.
23.
Kaufman, A. S. (1994). Intelligent testing with the WISC-III. New York: John Wiley.
24.
Kaufman, A. S., & Kaufman, N. L. (1983a). K-ABC administration and scoring manual. Circle Pines, MN: American Guidance Service.
25.
Kaufman, A. S., & Kaufman, N. L. (1983b). K-ABC interpretive manual. Circle Pines, MN: American Guidance Service.
26.
Kaufman, A. S., & Lichtenberger, E. O. (2000). Essentials of WISC-III and WPPSI-R assessment. New York: John Wiley.
27.
Kavale, K. A., & Forness, S. R. (1984). A meta-analysis of the validity of Wechsler scale profiles and recategorizations: Patterns or parodies?Learning Disability Quarterly, 7, 136-156.
28.
Konold, T. R., Glutting, J. J., McDermott, P. A., Kush, J. C., & Watkins, M. W. (1999). Structure and diagnostic benefits of a normative subtest taxonomy developed from the WISC-III standardization sample. Journal of School Psychology, 37, 29-48.
29.
Livingston, R. B., Jennings, E., Reynolds, C. R., & Gray, R. M. (2003). Multivariate analyses of the profile stability of intelligence tests: High for IQs, low to very low for subtest analyses. Archives of Clinical Neuropsychology, 18, 487-507.
30.
Lubinski, D. (2004). Introduction to the special section on cognitive abilities: 100 years after Spearman's (1904) “‘General intelligence,’objectively determined and measured.”Journal of Personality and Social Psychology, 86, 96-111.
31.
McCarthy, D. (1972). Manual for the McCarthy Scales of Children's Abilities. New York: Psychological Corporation.
32.
McDermott, P. A. (1998). MEG: Megacluster analytic strategy for multistage hierarchical grouping with relocations and replications. Educational and Psychological Measurement, 58, 677-686.
33.
McDermott, P. A., Fantuzzo, J. W., & Glutting, J. J. (1990). Just say no to subtest analysis: A critique on Wechsler theory and practice. Journal of Psychoeducational Assessment, 8, 290-302.
34.
McDermott, P. A., Fantuzzo, J. W., Glutting, J. J., Watkins, M. W., & Baggaley, A. R.(1992). Illusions of meaning in the ipsative assessment of children's ability. Journal of Special Education, 25, 504-526.
35.
McDermott, P. A., Glutting, J. J., Jones, J. N., & Noonan, J. V. (1989). Typology and prevailing composition of core profiles in the WAIS-R standardization sample. Psychological Assessment: A Journal of Consulting and Clinical Psychology, 1, 118-125.
36.
McDermott, P. A., Glutting, J. J., Jones, J. N., Watkins, M. W., & Kush, J. (1989). Core profile types in the WISC-R national sample: Structure, membership, and applications. Psychological Assessment, 1, 292-299.
37.
Milligan, G. W., & Hirtle, S. C. (2003). Clustering and classification methods. In J. A. Schinka & W. F. Velicer (Eds.), Handbook of psychology: Vol. 2. Research methods in psychology (pp. 165-186). Hoboken, NJ: Wiley.
38.
Osgood, C. E., & Suci, G. J. (1952). A measure of relation determined by both mean differences and profile information. Psychological Bulletin, 49, 251-262.
39.
Pfeiffer, S. I., Reddy, L. A., Kletzel, J. E., Schmelzer, E. R., & Boyer, L. M. (2000). The practitioner's view of IQ testing and profile analysis. School Psychology Quarterly, 15, 376-385.
40.
Reschly, D. J. (1997). Diagnostic and treatment utility of intelligence tests. In D. P. Flanagan, J. L. Genshaft, & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (pp. 437-456). New York: Guilford.
41.
Sattler, J. M. (2001). Assessment of children: Cognitive applications (4th ed.). San Diego, CA: Jerome M. Sattler.
42.
Spengler, P. M., Strohmer, D. C., Dixon, D. N., & Shivy, V. A. (1995). A scientist-practitioner model of psychological assessment: Implications for training, practice, and research. The Counseling Psychologist, 23, 506-534.
43.
Spruill, J. (1998). Assessment of mental retardation with the WISC-III. In A. Prifitera & D. Saklofske (Eds.), WISC-III clinical use and interpretation: Scientist-practitioner perspectives (pp. 73-90). San Diego, CA: Academic Press.
44.
Stinnett, T. A., Havey, J. M., & Oehler-Stinnett, J. (1994). Current test usage by practicing school psychologists: A national survey. Journal of Psychoeducational Assessment, 12, 331-350.
45.
Tatsuoka, M. M., & Lohnes, P. R. (1988). Multivariate analysis: Techniques for educational and psychological research (2nd ed.). New York: Macmillan.
46.
U.S. Department of Education. (2001). To assure the free appropriate public education of all children with disabilities: Twenty-fourth annual report to Congress on the implementation of the Individuals With Disabilities Education Act (USDOE Contract No. HS97020001 with Westat). Washington, DC: Author.
47.
Ward, J. H., Jr. (1963). Hierarchical grouping to optimize an objective function. American Statistical Association Journal, 58, 236-244.
48.
Watkins, M. W. (1998). MacKappa [Computer software]. University Park, PA: Author.
49.
Watkins, M. W. (2003). IQ subtest analysis: Clinical acumen or clinical illusion?Scientific Review of Mental Health Practice, 2, 118-141.
50.
Watkins, M. W., & Canivez, G. L. (2004). Temporal stability of WISC-III subtest composite: Strengths and weaknesses. Psychological Assessment, 16, 133-138.
51.
Watkins, M. W., & Glutting, J. J. (2000). Incremental validity of WISC-III profile elevation, scatter, and shape information for predicting reading and math achievement. Psychological Assessment, 12, 402-408.
52.
Wechsler, D. (1967). Manual for the Wechsler Preschool and Primary Scale of Intelligence. New York: Psychological Corporation.
53.
Wechsler, D. (1974). Manual for the Wechsler Intelligence Scale for Children–Revised. New York: Psychological Corporation.
54.
Wechsler, D. (1981). Manual for Wechsler Adult Intelligence Scale–Revised. New York: Psychological Corporation.
55.
Wechsler, D. (1991). Manual for the Wechsler Intelligence Scale for Children–Third Edition. San Antonio, TX: Psychological Corporation.
56.
Wechsler, D. (2003a). Wechsler Intelligence Scale for Children–Fourth Edition administration and scoring manual. San Antonio, TX: Psychological Corporation.
57.
Wechsler, D. (2003b). Wechsler Intelligence Scale for Children–Fourth Edition technical and interpretive manual. San Antonio, TX: Psychological Corporation.
