Independent Examination of the Factor Structure of the Cognitive Assessment System (CAS): Further Evidence Challenging the Construct Validity of the CAS

Abstract

This study is the first to examine independently the factor structure of the Cognitive Assessment System (GAS; Naglieri & Das, 1997) with a primary dataset not collected by its authors. Participants were 155 students (59 boys, 96 girls), ages 8 to 11 (M = 9.81 years, SD = 0.88), in Grades 3 to 6. Confirmatory factor analysis (CFA) was used to compare the fit provided by the planning, attention, and simultaneous-successive (PASS) model, the theoretical model underlying the CAS, with alternative models of cognitive ability suggested by previous research. Results of this study indicated that the PASS model did not provide a better fit to the data than did alternative hierarchical and nonhierarchical models. Not only were the Planning and Attention factors of the PASS model virtually indistinguishable (r = .88), but they demonstrated inadequate specificity for meaningful interpretation. The model reflecting the actual hierarchical structure of the CAS was found to fit the data no better than alternative models based on different theoretical orientations. Of the hierarchical models examined in this study, the best fitting was a hierarchical (PA)SS model with one second-order general factor, psychometric g, and three first-order factors reflecting Fluid Intelligence/Visual Processing (Simultaneous), Memory Span (Successive), and Processing Speed (Planning/Attention). In sum, results of this study support Kranzler and Keith's (1999) conclusion that the CAS lacks structural fidelity, which means that the CAS does not measure what its authors intended it to measure. Results of this study, therefore, provide further evidence challenging the construct validity of the CAS.

Get full access to this article

View all access options for this article.

References

American Psychological Association (1992). Ethical Principles of psychologists and code of conduct. Washington, DC: Author.

Benson, J. (1998). Developing a strong program of construct validation: A test anxiety example. Educational Measurement: Issues and Practices, 1 7, 10-17.

Bracken, B. A. , & McCallum, R. S. (1998). Universal Nonverbal Intelligence Test. Chicago: Riverside.

Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-analytic studies. New York: Cambridge University Press.

Carroll, J. B. (1995). [Review of Das, J. P., Naglieri, J. A., & Kirby, J. R. (1994). Assessment of cognitive processes: The PASS theory of intelligence]. Journal of Psychoeducational Assessment, 13, 397-409.

Carroll, J. B. (1997). The three-stratum theory of cognitive abilities. In D. P. Flanagan , J. L. Genshaft , & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (pp. 122-130). New York: Guilford.

Das, J. P. , Naglieri, J. A. , & Kirby, J. R. (1994). Assessment of cognitive processes: The PASS theory of intelligence. Boston: Allyn & Bacon.

Flanagan, D. P. , Andrews, T. J. , & Genshaft, J. L. (1997). The functional utility of intelligence tests with special education populations. In D. P. Flanagan , J. L. Genshaft , & P. L. Harrison , (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (pp. 457-483). New York: Guilford Press.

Flanagan, D. P. , McGrew, K. S. , & Ortiz, S. O. (2000). The Wechsler Intelligence Scales and Gf-Gc theory: A contemporary approach to interpretation. Boston: Allyn & Bacon.

10.

Horn, J. L. (1994). Theory of fluid and crystallized intelligence. In R. J. Sternberg (Ed.), Encyclopedia of intelligence (pp. 443-451). New York: Macmillan.

11.

Hu, L. , & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.

12.

Hunt, E. , & Lansman, M. (1986). Unified model of attention and problem solving. Psychological Review, 93, 446-461.

13.

Jensen, A. R. (1980). Bias in mental testing. New York: Free Press.

14.

Kaufman, A. S. (1990). Assessing adolescent and adult intelligence. Boston: Allyn & Bacon.

15.

Kaufman, A. S. , & Kaufman, N. L. (1993). Manual for the Kaufman Adolescent and Adult Intelligence Test (KAIT). Circle Pines, MN: American Guidance Service.

16.

Keith, T. Z. (1997). Using confirmatory factor analysis to aid in understanding the constructs measured by intelligence tests. In D. P. Flanagan , J. L. Genshaft , & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (pp. 373-402). New York: Guilford.

17.

Keith, T. Z. , & Kranzler, J. H. (1999). The absence of structural fidelity precludes construct validity: Rejoinder to Naglieri on what the Cognitive Assessment System does and does not measure. School Psychology Review, 28, 303-321.

18.

Kranzler, J. H. , & Keith, T. Z. (1999). Independent confirmatory factor analysis of the Cognitive Assessment System (CAS): What does the CAS measure? School Psychology Review, 28, 117-144.

19.

Kranzler, J. H. , & Weng, L. (1995a). The factor structure of the PASS cognitive tasks: A reexamination of Naglieri et al. Journal of School Psychology, 33, 143-157.

20.

Kranzler, J. H. , & Weng, L. (1995b). A reply to the commentary by Naglieri and Das on the factor structure of a battery of PASS cognitive tasks. Journal of School Psychology, 33, 169-176.

21.

Luria, A. R. (1966). Human brain and psychological processes. New York: Harper & Row.

22.

Luria, A. R. (1973). The working brain: An introduction to neuropsychology. New York: Basic Books.

23.

Luria, A. R. (1980). Higher cortical functions in man (2nd ed.). New York: Basic Books.

24.

McGrew, K. S. , & Flanagan, D. P. (1998). The intelligence test desk reference (ITDR): Gf-Gc cross-battery assessment. Boston: Allyn & Bacon.

25.

Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (pp. 13-103). New York: Macmillan.

26.

NaglieriJ. A. (1997). Planning, attention, simultaneous, and successive theory and the Cognitive Assessment System: A new theory-based measure of intelligence. In D. P. Flanagan J. L. Genshaft , & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (pp. 247-267). New York: Guilford.

27.

Naglieri, J. A. (1999a). Essentials of CAS assessment. New York: Wiley.

28.

Naglieri, J. A. (1999b). How valid is the PASS theory and CAS? School Psychology Review, 28, 145-162.

29.

Naglieri, J. A. , Braden, J. , & Gottling, S. (1993). Confirmatory factor analysis of the planning, attention, simultaneous, successive (PASS) cognitive processing model for a kindergarten sample. Journal of Psychoeducational Assessment, 11, 259-269.

30.

NaglieriJ. A. , & DasJ. P. (1997). Cognitive assessment system. Chicago: Riverside.

31.

Naglieri, J. A. , Das, J. P. , & Jarman, R. F. (1990). Planning, attention, simultaneous, successive (PASS) cognitive processes: A model for assessment. Journal of Psychoeducational Assessment, 36, 35-48.

32.

Naglieri, J. A. , Das, J. P. , Stevens, J. J. , & Ledbetter, M. F. (1991). Confirmatory factor analysis of planning, attention, simultaneous, and successive cognitive processing tasks. Journal of School Psychology, 29, 1-17.

33.

Reschly, D.J. (1997). Diagnostic and treatment utility of intelligence tests. In D. P. Flanagan , J. L. Genshaft , & P. L. Harrison , (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (pp. 437-456). New York: Guilford Press.

34.

Sternberg, R. L. (1996). Myths, countermyths, and truths about intelligence. Educational Researcher, 25(2), 11-16.

35.

Wechsler, D. (1991). Wechsler Intelligence Scales for Children-Third Edition. New York: The Psychological Corporation.

36.

Woodcock, R. W. , & Johnson, M. B. (1989). Woodcock-Johnson Psychoeducational Battery-Revised. Chicago: Riverside.