Validity and Reliability Evidence for the Comprehensive Test of Nonverbal Intelligence–Second Edition Scores From an Independent Sample

Abstract

In this study, we examined the factor structure, reliability, and external validity of scores from the Comprehensive Test of Nonverbal Intelligence–Second Edition (CTONI-2) using an independent sample of young adults currently enrolled in a postsecondary institution. Although the subtests appear to be measuring general intelligence, the aggregate Full Scale score should be interpreted with caution when using the instrument with adults suspected of having above average intelligence. The item difficulties were too low to capture individual differences in adults with higher intelligence. Moreover, the scores’ reliability and the relations to measures of academic achievement are lower than what is typically expected for intelligence measures.

Keywords

Comprehensive Test of Nonverbal Intelligence test validity test reliability item response theory

Get full access to this article

View all access options for this article.

References

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing (4th ed.). Washington, DC: American Educational Research Association.

Athanasiou

M. S.

(2000). Current nonverbal assessment instruments: A comparison of psychometric integrity and test fairness. Journal of Psychoeducational Assessment, 18, 211-229. doi:10.1177/073428290001800302

Bishop

S. L.

Farmer

Thurm

A. J.

(2015). Measurement of nonverbal IQ in autism spectrum disorder: Scores in young adulthood compared to early childhood. Journal of Autism and Developmental Disorders, 45, 966-974. doi:10.1007/s10803-014-2250-3

Bracken

B. A.

(1987). Limitations of preschool instruments and standards for minimal levels of technical adequacy. Journal of Psychoeducational Assessment, 5, 313-326.

Bracken

B. A.

Naglieri

J. A.

(2003). Assessing diverse populations with nonverbal tests of general intelligence. In Reynolds

C. R.

Kamphaus

R. W.

(Eds.), Handbook of psychological and educational assessment of children: Intelligence, aptitude, and achievement (2nd ed., pp. 243–274). New York, NY: Guilford.

Braden

J. P.

Athanasiou

M. S.

(2005). A comparative review of nonverbal measures of intelligence. In Flanagan

D. P.

Harrison

P. L.

(Eds.), Contemporary intellectual assessment: Theories, tests, and issues (2nd ed., p. 557-577). New York, NY: Guilford Press.

Bradley-Johnson

(1997). Comprehensive Test of Nonverbal Intelligence. Psychology in the Schools, 34, 289-292a.

Brown

Maydeu-Olivares

(2011). Item response modeling of forced-choice questionnaires. Educational and Psychological Measurement, 71, 460-502. doi:10.1177/0013164410375112

Chalmers

R. P.

(2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1-29. Retrieved from http://www.jstatsoft.org/v48/i06/

10.

Costa

P. T.

Jr. McCrae

R. R.

(1992). Revised NEO Personality Inventory (NEO-PI-R) and NEO Five Factor Inventory (NEO-FFI) professional manual. Odessa, FL: Psychological Assessment Resources.

11.

Kaya

Ritter

N. L.

(2012). Test review: Comprehensive Test of Nonverbal Intelligence–Second Edition (CTONI-2). Journal of Psychoeducational Assessment, 30, 209-213. doi:10.1177/0734282911415614

12.

DeThorne

L. S.

Schaefer

B. A.

(2004). A guide to child nonverbal IQ measures. American Journal of Speech-Language Pathology, 13, 275-290. doi:10.1044/1058-0360(2004/029)

13.

DiCerbo

K. E.

de Barona

M. S.

(2001). Test review: Comprehensive Test of Nonverbal Intelligence. Journal of Psychoeducational Assessment, 19, 175-180.

14.

Drossman

E. R.

Maller

S. J.

McDermott

P. A.

(2001). Core profiles of school-aged examinees from the national standardization sample of the Comprehensive Test of Nonverbal Intelligence. School Psychology Review, 30, 586-598.

15.

Dziuban

C. D.

Shirkey

E. C.

(1974). When is a correlation matrix appropriate for factor analysis? Some decision rules. Psychological Bulletin, 81, 358-361. doi:10.1037/h0036316

16.

Embretson

S. E.

Reise

S. P.

(2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.

17.

Enders

C. K.

(2011). Applied missing data analysis. New York, NY: Guilford Press.

18.

Floyd

F. J.

Widaman

K. F.

(1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7, 286-299. doi:10.1037/1040-3590.7.3.286

19.

Garrido

L. E.

Abad

F. J.

Ponsoda

(2011). Performance of Velicer’s minimum average partial factor retention method with categorical variables. Educational and Psychological Measurement, 71, 551-570. doi:10.1177/0013164410389489

20.

Gharaibeh

A. M.

Al-Natour

M. M.

El-Mneizel

A. F.

(2010). Examining the item effectiveness, reliability and validity of the CTONI for the Jordanian environment. Dirasat: Educational Sciences, 33, 382-399.

21.

Gignac

G. E.

(2014). Fluid intelligence shares closer to 60% of its variance with working memory capacity and is a better indicator of general intelligence. Intelligence, 47, 122-133. doi:10.1016/j.intell.2014.09.004.

22.

Goldberg

L. R.

Johnson

J. A.

Eber

H. W.

Hogan

Ashton

M. C.

Cloninger

C. R.

Gough

H. G.

(2006). The international personality item pool and the future of public domain personality measures. Journal of Research in Personality, 40, 84-96. doi:10.1016/j.jrp.2005.08.007

23.

Hammill

D. D.

Pearson

N. A.

Wiederholt

J. L.

(1997). Comprehensive Test of Nonverbal Intelligence. Austin, TX: PRO-ED.

24.

Hammill

D. D.

Pearson

N. A.

Wiederholt

J. L.

(2009). Comprehensive Test of Nonverbal Intelligence (2nd ed.). Austin, TX: PRO-ED.

25.

Harman

H. H.

Jones

W. H.

(1966). Factor analysis by minimizing residuals (minres). Psychometrika, 31, 351-368. doi:10.1007/BF02289468

26.

Harwell

M. R.

Baker

F. B.

(1991). The use of prior distributions in marginalized Bayesian item parameter estimation: A didactic. Applied Psychological Measurement, 15, 375-389. doi:10.1177/014662169101500409.

27.

Kelley

(2007). Methods for the behavioral, educational, and social sciences: An R package. Behavior Research Methods, 39, 979-984. doi:10.3758/BF03192993

28.

Klein

Herzberg

(2010). Quick Picture Reading Test. Los Angeles, CA: Western Psychological Services.

29.

Kranzler

J. H.

Floyd

R. G.

(2013). Assessing intelligence in children and adolescents: A practical guide. New York, NY: Guilford Press.

30.

Lassiter

K. S.

Harrison

T. K.

Matthews

T. D.

Bell

N. L.

(2001). The validity of the Comprehensive Test of Nonverbal Intelligence as a measure of fluid intelligence. Assessment, 8, 95-103. doi:10.1177/107319110100800109

31.

Lassiter

K. S.

Matthews

T. D.

Feeback

(2007). An examination of the CTONI utilizing gc-gf theory: A comparison of the CTONI and WJ-III. Psychology in the Schools, 44, 567-577.

32.

McCallum

R. S.

(Ed.). (2003). Handbook of nonverbal assessment. New York: Kluwer Academic/Plenum Publishers.

33.

McCallum

R. S.

(2013). Assessing intelligence nonverbally. In Geisinger

K. F.

Bracken

B. A.

Carlson

J. F.

Hansen

J. I. C.

Kuncel

N. R.

Reise

S. P.

Rodriguez

M. C.

(Eds.), APA handbook of testing and assessment in psychology, Vol. 3: Testing and assessment in school psychology and education (pp. 71–99). Washington, DC: American Psychological Association.

34.

McDonald

R. P.

(1999). Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum.

35.

McGill

R. J.

(2016). Investigation of the factor structure of the Comprehensive Test of Nonverbal Intelligence-Second Edition (CTONI-2) using exploratory factor analysis. Journal of Psychoeducational Assessment, 34, 339-350.

36.

Messick

(1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performance as scientific inquiry into score meaning. American Psychologist, 50, 741-749. doi:10.1037/0003-066X.50.9.741

37.

Nicholson

C. L.

(1999). Comprehensive Test of Nonverbal Intelligence (CTONI). Diagnostique, 24, 57-68.

38.

R Development Core Team. (2016). R: A language and environment for statistical computing (Version 3.2.3) [Computer software]. Vienna, Austria: R Foundation for Statistical Computing.

39.

Revelle

(2016). psych: Procedures for psychological, psychometric and personality research [Computer program]. Evanston, IL: Northwestern University.

40.

Reynolds

C. R.

Kamphaus

R. W.

(2003). Reynolds Intellectual Assessment Scales. Lutz, FL: Psychological Assessment Resources.

41.

Reynolds

M. R.

Hajovsky

D. B.

Pace

J. R.

Niileksela

C. R.

(2016). What does the Shipley-2 measure for children and adolescents? Integrated and conjoint confirmatory factor analysis with the WISC-IV. Assessment, 23, 23-41. doi:10.1177/1073191115572695

42.

Richardson

Abraham

Bond

(2012). Psychological correlates of university students’ academic performance: A systematic review and meta-analysis. Psychological Bulletin, 138, 353-387. doi:10.1037/a0026838

43.

Rossen

E. A.

Shearer

D. K.

Penfield

R. D.

Kranzler

J. H.

(2005). Validity of the Comprehensive Test of Nonverbal Intelligence (CTONI). Journal of Psychoeducational Assessment, 23, 161-172. doi:10.1177/073428290502300205

44.

Roth

Becker

Romeyke

Schäfer

Domnick

Spinath

F. M.

(2015). Intelligence and school grades: A meta-analysis. Intelligence, 53, 118-137. doi:10.1016/j.intell.2015.09.002.

45.

Sackett

P. R.

Yang

(2000). Correction for range restriction: An expanded typology. Journal of Applied Psychology, 85, 112-118. doi:10.1037/0021-9010.85.1.112

46.

Sheng

Wikle

C. K.

(2007). Comparing multiunidimensional and unidimensional item response theory models. Educational and Psychological Measurement, 67, 899-919. doi:10.1177/0013164406296977

47.

Shipley

W. C.

Gruber

C. P.

Martin

T. A.

Klein

A. M.

(2009). Shipley-2. Los Angeles, CA: Western Psychological Services.

48.

Shrout

P. E.

Lane

S. P.

(2012). Reliability. In Cooper

Camic

P. M.

Long

D. L.

Panter

A. T.

Rindskopf

Sher

K. J.

(Eds.), APA Handbook of Research Methods in Psychology: Vol. 1. Foundations, planning, measures, and psychometrics (pp. 643-660). Washington, DC: American Psychological Association.

49.

Sireci

S. G.

Sukin

(2013). Test validity. In Geisinger

K. F.

Bracken

B. A.

Carlson

J. F.

Hansen

J. I. C.

Kuncel

N. R.

Reise

S. P.

Rodriguez

M. C.

(Eds.), APA Handbook of Testing and Assessment in Psychology: Vol. 1. Test theory and testing and assessment in industrial and organizational psychology (pp. 61-84). Washington, DC: American Psychological Association.

50.

Sotelo-Dynega

Dixon

S. G.

(2014). Cognitive assessment practices: A survey of school psychologists. Psychology in the Schools, 51, 1031-1045. doi:10.1002/pits.21802

51.

Spearman

C. E.

(1904). “General intelligence,” objectively determined and measured. The American Journal of Psychology, 15, 201-293. doi:10.2307/1412107

52.

Spiegelhalter

D. J.

Best

N. G.

Carlin

B. P.

Van Der Linde

(2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 64, 583-639. doi:10.1111/1467-9868.00353

53.

Wasserman

J. D.

Bracken

B. A.

(2003). Psychometric characteristics of assessment procedures. In Weiner

I. B.

(Ed.), Handbook of psychology (2nd ed., pp. 43-66). Hoboken, NJ: John Wiley.

54.

Wilkinson

G. S.

(1993). WRAT3: Wide Range Achievement Test administration manual. Wilmington, DE: Wide Range.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB