Memory and Reasoning Abilities Assessed by the Universal Nonverbal Intelligence Test: A Reliable Component Analysis (RCA) Study

Abstract

The Universal Nonverbal Intelligence Test (UNIT) is an important advancement in intelligence assessment, but the equal-weighting method used to compute scores for the memory and reasoning abilities results in some psychometric weaknesses. The equally weighted memory and reasoning scores are highly correlated, leading to a lack of discriminant validity, a potential lack of incremental validity, and poor reliability for their score differences. Reliable component analysis (RCA) was performed on the UNIT normative data to allow for the computation of reliable, uncorrelated memory and reasoning scores. The RCA scores were highly replicable across samples and ages, have good convergent validity, and have greater discriminant validity than the equally weighted scores. Furthermore, the difference between the memory and reasoning scores is much more reliable when using RCA. This results in narrower confidence intervals around difference scores and more powerful tests of the null hypothesis that there is no true difference between memory and reasoning abilities.

Get full access to this article

View all access options for this article.

References

Bracken, B. A. , & McCallum, R. S. (1998). Universal Nonverbal Intelligence Test examiner’s manual. Itsaka, IL: Riverside.

Brown, L. , Sherbenou, R. J. , & Johnsen, S. K. (1990). Test of Nonverbal Intelligence—Second edition. Austin, TX: PRO-ED.

Caruso, J. C. (in press). Reliable component analysis of the Stanford-Binet: Fourth Edition for 2-6 year olds. Psychological Assessment.

Caruso, J. C. , & Cliff, N. (1999). The relative properties of equally and differentially weighted WAIS-III factor scores. Psychological Assessment, 11, 198-206.

Caruso, J. C. , & Cliff, N. (in press-a). Increasing the reliability of WISC-III difference scores with reliable component analysis. Psychological Assessment.

Caruso, J. C. , & Cliff, N. (in press-b). Reliable component analysis (RCA): A new/old method for exploratory data reduction. In B. Thompson (Ed.), Advances in social science methodology (vol. 6). Stamford, CT: JAI.

Cattell, R. B. (1982). The clinical use of difference scores: Some psychometric problems. Multivariate Experimental Clinical Research, 6, 87-98.

Cliff, N. , & Caruso, J. C. (1998). Reliable component analysis through maximizing composite reliability. Psychological Methods, 3, 291-308.

Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304-1312.

10.

Cronbach, L. J. , & Furby, L. (1970). How should we measure “change”—Or should we? Psychological Bulletin, 74, 68-80.

11.

Dudek, F. J. (1979). The continuing misinterpretation of the standard error of measurement. Psychological Bulletin, 86, 335-337.

12.

Glutting, J. J. , Youngstrom, E. A. , Ward, T. , Ward, S. , & Hale, R. L. (1997). Incremental efficacy of WISC-III factor scores in predicting achievement: What do they tell us? Psychological Assessment, 9, 295-301.

13.

Joint Committee on Standards for Educational Evaluation . (1994). The program evaluation standards: How to assess evaluations of educational programs (2nd ed.). Thousand Oaks, CA: Sage.

14.

Korth, B. , & Tucker, L. R. (1975). The distribution of chance congruence coefficients from simulated data. Psychometrika, 40, 361-372.

15.

Lord, F. M. , & Novick, M. R. (1968). Statistical theories of mental text scores. Reading, MA: Addison-Wesley.

16.

Naglieri, J. A. (1985). Matrix Analogies Test—Expanded Form: Examiner’s manual. San Antonio, TX: Psychological Corporation.

17.

Naglieri, J. A. , & Prewett, P. N. (1999). Nonverbal intelligence measures: A selected review of instruments and their use. In C. R. Reynolds & R. W. Kamphaus (Eds.), Handbook of psychological and educational assessment of children: Intelligence and achievement (pp. 348-370). New York: Guilford.

18.

Raven, J. C. , Court, J. H. , & Raven, J. (1986). Manual for Raven’s Progressive Matrices and Vocabulary Scales (Section 3)—Standard progressive matrices (1986 edition, with U.S. norms). London: Lewis.

19.

Sattler, J. M. (1992). Assessment of children (3rd ed., revised). San Diego, CA: Author.

20.

Stanley, J. C. (1967). General and special formulas for the reliability of differences. Journal of Educational Measurement, 4, 249-252.

21.

Sullivan, P. M. , & Burley, S. (1999). Mental testing of the hearing impaired child. In C. R. Reynolds & R. W. Kamphaus (Eds.), Handbook of psychological and educational assessment of children: Intelligence and achievement (pp. 761-788). New York: Guilford.

22.

Thorndike, R. L. , Hagen, E. P. , & Sattler, J. M. (1986). Technical manual, Stanford-Binet Intelligence Scale: Fourth Edition. Chicago: Riverside.

23.

Tucker, L. R. (1951). A method for synthesis of factor analysis studies. Personnel Research Section Report, No. 984. Washington, DC: Department of the Army.

24.

U.S. Bureau of the Census. (2000). Language use [Online]. Available: http://www.census.gov/population/www/socdemo/lang_use.html

25.

Wechsler, D. (1991). Wechsler Intelligence Scale for Children—Third Edition manual. San Antonio, TX: Psychological Corporation.

26.

Wilkinson, L. , & APA Task Force on Statistical Inference . (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604.

27.

Zumbo, B. D. (1999). The simple difference score as an inherently poor measure of change: Some reality, much mythology. In B. Thompson (Ed.), Advances in social science methodology (vol. 5, pp. 269-304). Stamford, CT: JAI.