A Meta-Analytic Evaluation: Investigating Evidence for the Validity of the Cognitive Abilities Test

Abstract

Cognitive Abilities Test (CogAT) is one of the most frequently used gifted identification tools. In this meta-analytic study, we investigated empirical evidence of the validity of CogAT, in relation to different types of instruments. After reviewing 1,480 studies, a total of 24 with 33 effect sizes were included in the meta-analysis. According to our findings, the average effect size of r was found to be .63 with a 95% confidence interval [.57, .69]. Based on the heterogeneity test, significant variation due to the systematic between-study differences exists among the included correlations. Egger’s test for funnel plot asymmetry also indicates that no obvious publication bias exists in our study pool, which indicates there might not be a serious threat to alter the obtained results with publication bias. The moderator analysis revealed Lohman’s authorship and publication type influenced the effect size differences among studies. CogAT’s overall correlation with other identification tools (.63) might suggest using at least one more identification tool besides CogAT.

Keywords

Cognitive Abilities Test CogAT gifted identification validity check

Get full access to this article

View all access options for this article.

References

Acar

Sen

Cayirdag

(2016). Consistency of the performance and nonperformance methods in gifted identification: A multilevel meta-analytic review. Gifted Child Quarterly, 60(2), 81–101. https://doi.org/10.1177/0016986216634438

Ahn

Ames

Myers

(2012). A review of meta-analyses in education. Review of Educational Research, 82, 436–476. https://doi.org/10.3102/0034654312458162

*Alodat

A. M.

Zumberg

M. F.

(2018). Standardizing the cognitive abilities screening test (CogAT 7) for identifying gifted and talented children in kindergarten and elementary schools in Jordan. Journal for the Education of Gifted Young Scientists, 6(2), 1–13.

Angrist

Djankov

Goldberg

P. K.

Patrinos

H. A.

(2021). Measuring human capital using global learning data. Nature, 592(7854), 403–408. https://doi.org/10.1038/s41586-021-03323-7

*Bildiren

Kiziltepe

G. I.

(2018). Analysis of the relation between the coloured progressive matrices test and the Cognitive Abilities Test form-6 in the preschool period of gifted children. Journal of Education and Learning, 7(4), 215–224.

*Blue

(2009). An investigation into the relationship between cognitive ability, standardized achievement, and grades in middle school (Publication No. 3359614) [Doctoral dissertation, Temple University]. ProQuest Dissertations & Theses Global.

Borenstein

Hedges

Rothstein

(2007). Meta-analysis: Fixed effect vs random effects. Meta-analysis.com.

Borenstein

Hedges

L. V.

Higgins

J. P.

Rothstein

H. R.

(2010). A basic introduction to fixed-effect and random-effects models for meta-analysis. Research Synthesis Methods, 1(2), 97–111. https://doi.org/10.1002/jrsm.12

Borenstein

Hedges

L. V.

Higgins

J. P.

Rothstein

H. R.

(2011). Introduction to meta-analysis. John Wiley.

10.

Cao

T. H.

Jung

J. Y.

Lee

(2017). Assessment in gifted education: A review of the literature from 2005 to 2016. Journal of Advanced Academics, 28(3), 163–203.

11.

Card

N. A.

(2015). Applied meta-analysis for social science research. Guilford Publications.

12.

Carman

C. A.

Walther

C. A.

Bartsch

R. A.

(2018). Using the Cognitive Abilities Test (CogAT) 7 nonverbal battery to identify the gifted/talented: An investigation of demographic effects and norming plans. Gifted Child Quarterly, 62(2), 193–209. https://doi.org/10.1177/0016986217752097

13.

Carman

C. A.

Walther

C. A.

Bartsch

R. A.

(2020). Differences in using the Cognitive Abilities Test (CogAT) 7 nonverbal battery versus the Naglieri Nonverbal Ability Test (NNAT) 2 to identify the gifted/talented. Gifted Child Quarterly, 64(3), 171–191. https://doi.org/10.1177/0016986220921164

14.

Carroll

J. B.

(1993). Human cognitive abilities: A survey of factor-analytic studies. Cambridge University Press.

15.

Cattell

R. B.

(1971). Abilities: Their structure, growth, and action. Houghton Mifflin.

16.

Cook

D. J.

Guyatt

G. H.

Ryan

Clifton

Buckingham

Willan

Mcllroy

Oxman

A. D.

(1993). Should unpublished data be included in meta-analyses? Current convictions and controversies. Journal of the American Medical Association, 269(21), 2749–2753.

17.

Cooper

(2015). Research synthesis and meta-analysis: A step-by-step approach (Vol. 2). Sage.

18.

Cooper

Hedges

L. V.

Valentine

J. C.

(2009). The handbook of research synthesis and meta-analysis. Russell Sage Foundation.

19.

Cooper

H. M.

(1998). Synthesizing research: A guide for literature reviews (3rd ed.). Sage.

20.

*Daniel

K. M.

(2015). The relationship between middle grades Cognitive Abilities Test and high school advanced placement exam scores (Publication No. 3712229) [Doctoral dissertation, Liberty University]. ProQuest Dissertation & Theses.

21.

*Edmonds

M. M.

(2015). Identifying gifted minorities using nonverbal tests: The Cognitive Abilities Test, form 7, versus the Naglieri nonverbal ability test, second edition (Publication No. 3730349) [Doctoral dissertation, Regent University]. ProQuest Dissertations & Theses.

22.

Egger

Smith

G. D.

Schneider

Minder

(1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315(7109), 629–634.

23.

*Emens

R. A.

(2008). The use of a behavior screener to predict outcomes on high stakes tests for elementary school students (Publication No. 3315198) [Doctoral dissertation, Mississippi State University]. ProQuest Dissertations & Theses.

24.

*Foreman

J. L.

Gubbins

E. J.

(2015). Teachers see what ability scores cannot: Predicting student performance with challenging mathematics. Journal of Advanced Academics, 26(1), 5–23.

25.

*Gittens

C. A.

(2015). Assessing numeracy in the upper elementary and middle school years. Numeracy, 8(1), 15–28.

26.

Gucciardi

D. F.

Lines

R. L. J.

Ntoumanis

(2022). Handling effect size dependency in meta-analysis. International Review of Sport and Exercise Psychology, 15(1), 152–178. https://doi.org/10.1080/1750984X.2021.1946835

27.

Hall

S. M.

Brannick

M. T.

(2002). Comparison of two random-effects methods of meta-analysis. Journal of Applied Psychology, 87(2), 377.

28.

Hedges

L. V.

Tipton

Johnson

M. C.

(2010). Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods, 1(1), 39–65. https://doi.org/10.1002/jrsm.5

29.

*Hichens

(2009). College readiness of 11th grade students: Identifying characteristics related to success on the ACT (Publication No. 3381288) [Doctoral dissertation, Aurora University]. ProQuest Dissertations & Theses.

30.

Higgins

Thompson

S. G.

(2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21(11), 1539–1558. https://doi.org/10.1002/sim.1186

31.

*Hlasny

J. G.

(2008). Creativity: A comparison of gifted students (Publication No. 3316344) [Doctoral dissertation, Capella University]. ProQuest Dissertations & Theses.

32.

Hodges

Tay

Maeda

Gentry

(2018). A meta-analysis of gifted and talented identification practices. Gifted Child Quarterly, 62(2), 147–174. https://doi.org/10.1177/0016986217752107

33.

Horn

J. L.

(1989). Models of intelligence. In Linn

(Ed.), Intelligence: Measurement, theory, and public policy (pp. 29–73). University of Illinois Press.

34.

Hunter

J. E.

Schmidt

F. L.

(2004). Methods of meta-analysis: Correcting error and bias in research findings. Sage.

35.

*Jarvis

J. M.

(2009). The relationship between adolescents’ domain knowledge and creative performance on an ill-defined physics task (Publication No. 3351204) [Doctoral dissertation, University of Virginia]. ProQuest Dissertations & Theses.

36.

JASP Team. (2022). JASP (Version 0.16.3) [Computer software].

37.

*Kong

(2013). Socioemotional competencies, cognitive ability, and achievement in gifted students (Publication No. 3604369) [Doctoral dissertation, Arizona State University]. ProQuest Dissertations & Theses.

38.

Kurtz

Lloyd

Harwin

Chen

Furuya

(2019). Gifted education: Results of a national survey. Editorial Projects in Education. https://epe.brightspotcdn.com/5c/8d/b982dd8a48638a57e8857a0f5c29/gt-survey-report-final-11.25.19.pdf

39.

*Lakin

J. M.

(2012). Assessing the cognitive abilities of culturally and linguistically diverse students: Predictive validity of verbal, quantitative, and nonverbal tests. Psychology in the Schools, 49(8), 756–768.

40.

Lakin

J. M.

(2017, August 15). Introducing form 8 for CogAT. https://blog.riversideinsights.com/introducing-form-8-CogAT

41.

*Lakin

J. M.

Gambrell

J. L.

(2012). Distinguishing verbal, quantitative, and figural facets of fluid intelligence in young students. Intelligence, 40(6), 560–570.

42.

Lee

Karakis

Olcay Akce

Azzam Tuzgen

Karami

Gentry

Maeda

(2021). A meta-analytic evaluation of Naglieri nonverbal ability test: Exploring its validity evidence and effectiveness in equitably identifying gifted students. Gifted Child Quarterly, 65(3), 199–219. https://doi.org/10.1177/0016986221997800

43.

*Lindsey

C. D.

(2012). An investigation of the combined assessments used as entrance criteria for a gifted English middle school program (Publication No. 3525542) [Doctoral dissertation, University of New Orleans]. ProQuest Dissertations & Theses.

44.

*Lohman

D. F.

(2003a). The Wechsler intelligence scale for children III and the Cognitive Abilities Test (form 6): Are the general factors the same? https://www.semanticscholar.org/paper/The-Wechsler-Intelligence-Scale-for-Children-III-Lohman/1730b8339f3f5472596eff694a6f543bc34f3252

45.

*Lohman

D. F.

(2003b). The Woodcock–Johnson III and the Cognitive Abilities Test (form 6): A concurrent validity study. https://www.semanticscholar.org/paper/The-Woodcock-Johnson-III-and-the-Cognitive-Test-(-6-Lohman/bd2eed7e216abd6ffa9d71b15d2920583ed955d1

46.

Lohman

D. F.

(2005). The role of nonverbal ability tests in identifying academically gifted students: An aptitude perspective. Gifted Child Quarterly, 49(2), 111–138. https://doi.org/10.1177/001698620504900203

47.

Lohman

D. F.

(2011). Cognitive Abilities Test, form 7. Riverside Publishing.

48.

*Lohman

D. F.

(2012). Cognitive Abilities Test, form 7, research and development guide. Riverside Publishing.

49.

Lohman

D. F.

Gambrell

J. L.

(2012). Using nonverbal tests to help identify academically talented children. Journal of Psychoeducational Assessment, 30(1), 25–44. https://doi.org/10.1177/0734282911428194

50.

Lohman

D. F.

Hagen

E. P.

(2001). Cognitive Abilities Test (Form 6). Riverside Publishing.

51.

*Lohman

D. F.

Hagen

E. P.

(2002). Cognitive Abilities Test (Form 6): Research handbook. Riverside Publishing.

52.

*Lohman

D. F.

Korb

K. A.

Lakin

J. M.

(2008). Identifying academically gifted English-language learners using nonverbal tests: A comparison of the Raven, NNAT, and CogAT. Gifted Child Quarterly, 52(4), 275–296. https://psycnet.apa.org/doi/10.1177/0016986208321808

53.

López-López

J. A.

Page

M. J.

Lipsey

M. W.

Higgins

J. P. T.

(2018). Dealing with effect size multiplicity in systematic reviews and meta-analyses. Research Synthesis Methods, 9(3), 336–351. https://doi.org/10.1002/jrsm.1310

54.

Lorge

Thorndike

R. M.

(1954). Lorge-Thorndike intelligence tests. Houghton Mifflin.

55.

*Marchesini

(2020). Relationships between group administered ability tests and individual academic achievement (Publication No. 27960760) [Doctoral dissertation, George Mason University]. ProQuest Dissertations & Theses.

56.

McBee

M. T.

Peters

S. J.

Waterman

(2014). Combining scores in multiple-criteria assessment systems: The impact of combination rule. Gifted Child Quarterly, 58(1), 69–89. https://doi.org/10.1177/0016986213513794

57.

Naglieri

J. A.

(2003). Naglieri nonverbal ability test: Individual administration manual. Multi-Health Systems.

58.

Naglieri

J. A.

(2011). Naglieri nonverbal ability test manual: Technical information and normative data (2nd ed.). NCS Pearson.

59.

Naglieri

J. A.

(2018). Naglieri nonverbal ability test manual levels A-D (3rd ed.). Pearson.

60.

Ouzzani

Hammady

Fedorowicz

Elmagarmid

(2016). Rayyan—a web and mobile app for systematic reviews. Systematic Reviews, 5(1), 1–10. https://doi.org/10.1186/s13643-016-0384-4

61.

Page

M. J.

McKenzie

J. E.

Bossuyt

P. M.

Boutron

Hoffmann

T. C.

Mulrow

C. D.

Shamseer

Tetzlaff

J. M.

Akl

E. A.

Brennan

S. E.

Chou

Glanville

Grimshaw

J. M.

Hróbjartsson

Loder

E. W.

Mayo-Wilson

McDonald

McGuinness

L. A.

. . .Moher

(2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Systematic Reviews, 10(1), 1–11. https://doi.org/10.1136/bmj.n71

62.

*Patterson

J. S.

(2012). Predicting SAT performance from advanced course content and timing of matriculation (Publication No. 3528462) [Doctoral dissertation, Mercer University]. ProQuest Dissertations & Theses.

63.

*Perham

H. J.

(2012). Interpersonal skills of gifted students: Risk versus resilience (Publication No. 3505402) [Doctoral dissertation, Arizona State University]. ProQuest Dissertations & Theses.

64.

Roberts

J. L.

Inman

T. F.

Robins

J. H.

(Eds.) (2022). Introduction to gifted education. Taylor & Francis.

65.

Schellenberg

S. J.

(2004). Test bias or cultural bias: Have we really learned anything? [Paper presentation]. Annual Meeting of the National Council for Measurement in Education 2004, San Diego, the USA. https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=6554a7eefc3bd8614137f8adc15342ab20cc4783

66.

Soares

D. L.

Lemos

G. C.

Primi

Almeida

L. S.

(2015). The relationship between intelligence and academic achievement throughout middle school: The role of students’ prior academic performance. Learning and Individual Differences, 41, 73–78.

67.

Software for Statistics and Data Science. (2021). Stata statistical software: Release 17 [Computer Software]. StataCorp LLC.

68.

Van den Noortgate

López-López

J. A.

Marín-Martínez

Sánchez-Meca

. (2013). Three-level meta-analysis of dependent effect sizes. Behavior Research Methods, 45(2), 576–594. https://doi.org/10.3758/s13428-012-0261-6

69.

Van den Noortgate

López-López

J. A.

Marín-Martínez

Sánchez-Meca

. (2015). Meta-analysis of multiple outcomes: A multilevel approach. Behavior Research Methods, 47(4), 1274–1294. https://doi.org/10.3758/s13428-014-0527-2

70.

Warne

R. T.

(2015). Test review: Cognitive Abilities Test, form 7 (CogAT7). Journal of Psychoeducational Assessment, 33(2), 188–192. https://doi.org/10.1177/0734282914548324

71.

Warne

R. T.

(2020). In the know: Debunking 35 myths about human intelligence. Cambridge University Press.

72.

*Warnimont

C. S.

(2010). The relationship between students’ performance on the Cognitive Abilities Test (CogAT) and the fourth and fifth grade reading and math achievement tests in Ohio (Publication No. AAI3424545) [Doctoral dissertation, Bowling Green State University]. Linguistics and Language Behavior Abstracts.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB