Norming of the NIH Toolbox Cognition Battery Version 3

Abstract

The NIH Toolbox^® for Assessment of Neurological and Behavioral Function Cognition Battery (NIHTB-CB) is a brief neuropsychological assessment tool created as part of the NIH Neuroscience Blueprint. Since its inception, NIHTB-CB has been widely used in a variety of clinical and research settings, including large-scale epidemiological studies. The NIHTB-CB was recently updated and re-normed to Version 3 (V3). We describe the approach to establish normative reference values. The NIHTB-CB tests were administered to a large English-speaking sample of n = 3,904 (average age = 25.7 years, 52.1% female) individuals from the U.S. population, stratified by age, race and ethnicity, sex assigned at birth, and education level within the four U.S. census regions. Normative data were raked via iterative proportional fitting (e.g., by sex assigned at birth, race, ethnicity, and educational attainment nested within geographic region) to derive sampling weights that match the demographic proportions from the U.S. Census targets. Through regression-based continuous norming and bootstrap techniques, age-adjusted and age-and-education-adjusted normative scores were created for individual measure-level and composite scores.

Keywords

cognition continuous norming lifespan NIH Toolbox psychometrics sample weighting neuropsychology

Get full access to this article

View all access options for this article.

References

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (Eds.). (2014). Standards for educational and psychological testing. American Educational Research Association. https://www.aera.net/publications/books/standards-for-educational-psychological-testing-2014-edition

Beaumont

J. L.

Havlik

Cook

K. F.

Hays

R. D.

Wallner-Allen

Korper

S. P.

Lai

J. S.

Nord

Zill

Choi

Yost

K. J.

Ustsinovich

Brouwers

Hoffman

H. J.

Gershon

(2013). Norming plans for the NIH Toolbox. Neurology, 80(11 Suppl. 3), S87–S92. https://doi.org/10.1212/WNL.0b013e3182872e70

Casaletto

K. B.

Umlauf

Beaumont

Gershon

Slotkin

Akshoomoff

Heaton

R. K.

(2015). Demographically corrected normative standards for the English version of the NIH toolbox cognition battery. Journal of the International Neuropsychological Society, 21(5), 378–391. https://doi.org/10.1017/S1355617715000351

DeBell

Krosnick

J. A.

(2009). Computing weights for American national election study survey data (ANES Technical Report series no. nes012427). American National Election Studies.

delCacho-Tena

Christ

B. R.

Arango-Lasprilla

J. C.

Perrin

P. B.

Rivera

Olabarrieta-Landa

(2024). Normative data estimation in neuropsychological tests: A systematic review. Archives of Clinical Neuropsychology, 39(3), 383–398. https://doi.org/10.1093/arclin/acad084

Delis

D. C.

Kramer

J. H.

Kaplan

Ober

B. A.

(2017). California Verbal Learning Test, Third Edition (CVLT-3). The Psychological Corporation.

Dickinson

M. D.

Hiscock

(2011). The Flynn effect in neuropsychological assessment. Applied Neuropsychology, 18(2), 136–142. https://doi.org/10.1080/09084282.2010.547785

Embretson

S. E.

Reise

S. P.

(2013). Item response theory. Psychology Press.

Farmer

R. L.

McGill

R. J.

Dombrowski

S. C.

McClain

M. B.

Harris

Lockwood

A. B.

Powell

S. L.

Pynn

Smith-Kellen

Loethen

Benson

N. F.

Stinnett

T. A.

(2020). Teleassessment with children and adolescents during the coronavirus (COVID-19) pandemic and beyond: Practice and policy implications. Professional Psychology: Research and Practice, 51(5), 477–487. https://doi.org/10.1037/pro0000349

10.

Flynn

J. R.

(1987). Massive IQ gains in 14 nations: What IQ tests really measure. Psychological Bulletin, 101(2), 171–191. https://doi.org/10.1037/0033-2909.101.2.171

11.

Flynn

J. R.

(1998). WAIS-III and WISC-III gains in the United States from 1972 to 1995: How to compensate for obsolete norms. Perceptual and Motor Skills, 86(3, Pt 2), 1231–1239. https://doi.org/10.2466/pms.1998.86.3c.1231

12.

Flynn

J. R.

(2000). The hidden history of IQ and special education: Can the problems be solved? Psychology, Public Policy, and Law, 6(1), 191–198. https://doi.org/10.1037/1076-8971.6.1.191

13.

Fox

R. S.

Zhang

Amagai

Bassard

Dworak

E. M.

Han

Y. C.

Kassanits

Miller

C. H.

Nowinski

C. J.

Giella

A. K.

Stoeger

J. N.

Swantek

Hook

J. N.

Gershon

R. C.

(2022). Uses of the NIH Toolbox® in clinical samples: A scoping review. Neurology Clinical Practice, 12(4), 307–319. https://doi.org/10.1212/CPJ.0000000000200060

14.

Gershon

R. C.

(2016). NIH Toolbox norming study (Harvard Dataverse, V.4). https://doi.org/10.7910/DVN/FF4DI7

15.

Gershon

R. C.

Wagster

M. V.

Hendrie

H. C.

Fox

N. A.

Cook

K. F.

Nowinski

C. J.

(2013). NIH toolbox for assessment of neurological and behavioral function. Neurology, 80(11 Suppl 3), S2–S6. https://doi.org/10.1212/WNL.0b013e3182872e5f

16.

E. H.

Han

Y. C.

LaForte

Kaat

A. J.

Ece

Dworak

E. M.

Olsen

J. B.

Hook

Gershon

(2025). Development and validation of the NIH Toolbox version 3.0 for children [Manuscript under submission, OSF Preprint: https://osf.io/2w83f_v1]. Feinberg School of Medicine, Northwestern University.

17.

International Test Commission. (2001). International guidelines for test use. International Journal of Testing, 1(2), 93–114.

18.

Kalton

Flores-Cervantes

(2003). Weighting methods. Journal of Official Statistics, 19, 81–97.

19.

Kiselica

A. M.

Karr

J. E.

Mikula

C. M.

Ranum

R. M.

Benge

J. F.

Medina

L. D.

Woods

S. P.

(2024). Recent advances in neuropsychological test interpretation for clinical practice. Neuropsychology Review, 34(2), 637–667. https://doi.org/10.1007/s11065-023-09596-1

20.

LaForte

E. M.

Dailey

McGrew

K. S.

(2025). Woodcock-Johnson V technical manual. Riverside Assessments, LLC.

21.

LaForte

E. M.

Hook

J. N.

Giella

A. K.

(2024). National Institutes of Health (NIH) Toolbox® version 3 technical manual. https://nihtoolbox.org/app/uploads/2024/04/NIHTB-V3-Technical-Manual_040524.pdf

22.

Lenhard

(2020). Improvement of norm score quality via regression-based continuous norming. Educational and Psychological Measurement, 81(2), 229–261. https://doi.org/10.1177/0013164420928457

23.

Manly

J. J.

(2005). Advantages and disadvantages of separate norms for African Americans. The Clinical Neuropsychologist, 19(2), 270–275. https://doi.org/10.1080/13854040590945346

24.

Marden

J. R.

Tchetgen Tchetgen

E. J.

Kawachi

Glymour

M. M.

(2017). Contribution of socioeconomic status at 3 life-course periods to late-life memory function and decline: Early and late predictors of dementia risk. American Journal of Epidemiology, 186(7), 805–814. https://doi.org/10.1093/aje/kwx155

25.

Mather

M. A.

E. H.

Bedjeti

Karpouzian-Rogers

Rogalski

E. J.

Gershon

Weintraub

(2024). Measuring multidimensional aspects of health in the oldest old using the NIH Toolbox: Results from the ARMADA study. Archives of Clinical Neuropsychology, 39(5), 535–546. https://doi.org/10.1093/arclin/acad105

26.

McGrew

K. S.

LaForte

E. M.

Schrank

F. A.

(2014). Woodcock-Johnson IV technical manual. Riverside Assessments, LLC.

27.

Mungas

Heaton

Tulsky

Zelazo

P. D.

Slotkin

Blitz

Lai

J. S.

Gershon

(2014). Factor structure, convergent validity, and discriminant validity of the NIH Toolbox Cognitive Health Battery (NIHTB-CHB) in adults. Journal of the International Neuropsychological Society, 20(6), 579–587. https://doi.org/10.1017/S1355617714000307

28.

Mungas

Widaman

Zelazo

P. D.

Tulsky

Heaton

R. K.

Slotkin

Blitz

D. L.

Gershon

R. C.

(2013). National Institutes of Health Toolbox Cognition Battery (NIH Toolbox CB): Validation for children between 3 and 15 years: VII. NIH Toolbox Cognition Battery (CB): Factor structure for 3 to 15 year olds. Monographs of the Society for Research in Child Development, 78(4), 103–118. https://doi.org/10.1111/mono.12037

29.

Oosterhuis

H. E.

van der Ark

L. A.

Sijtsma

(2016). Sample size requirements for traditional and regression-based norms. Assessment, 23(2), 191–202. https://doi.org/10.1177/1073191115580638

30.

Pietschnig

Voracek

(2015). One century of global IQ gains: A formal meta-analysis of the Flynn effect (1909-2013). Perspectives on Psychological Science, 10(3), 282–306. https://doi.org/10.1177/1745691615577701

31.

Possin

K. L.

Tsoy

Windon

C. C.

(2021). Perils of race-based norms in cognitive testing: The case of former NFL players. JAMA Neurology, 78(4), 377–378. https://doi.org/10.1001/jamaneurol.2020.4763

32.

Price

J. C.

Lee

J. J.

Saraiya

Lei

Mintz

C. D.

(2023). An update on NIH programs relevant to child brain health research: ECHO, ABCD, HBCD, and MIRA. Journal of Neurosurgical Anesthesiology, 35(1), 119–123. https://doi.org/10.1097/ANA.0000000000000875

33.

Rentz

D. M.

Amariglio

R. E.

Becker

J. A.

Frey

Olson

L. E.

Frishe

Carmasin

Maye

J. E.

Johnson

K. A.

Sperling

R. A.

(2011). Face-name associative memory performance is related to amyloid burden in normal elderly. Neuropsychologia, 49(9), 2776–2783. https://doi.org/10.1016/j.neuropsychologia.2011.06.006

34.

Roid

G. H.

Pomplun

(2012). The Stanford-Binet Intelligence Scales, fifth edition. In Flanagan

D. P.

Harrison

P. L.

(Eds.), Contemporary intellectual assessment: Theories, tests, and issues (3rd ed., pp. 249–268). The Guilford Press.

35.

Shono

Ece

E. H.

Kaat

A. J.

La Forte

E. M.

Ayturk

Gershon

(2024). A comparison of scoring algorithms for the NIH Toolbox Executive Function Tasks in a U.S. norming sample. Psychological Assessment, 36(12), 760–771. https://doi.org/10.1037/pas0001350

36.

Timmerman

M. E.

Voncken

Albers

C. J.

(2021). A tutorial on regression-based norming of psychological tests with GAMLSS. Psychological Methods, 26(3), 357–373. https://doi.org/10.1037/met0000348

37.

Torres

V. A.

Ashford

J. M.

Wright

Zhang

Merchant

T. E.

Conklin

H. M.

(2021). The impact of socioeconomic status (SES) on cognitive outcomes following radiotherapy for pediatric brain tumors: A prospective, longitudinal trial. Neuro-Oncology, 23(7), 1173–1182. https://doi.org/10.1093/neuonc/noab018

38.

Trahan

L. H.

Stuebing

K. K.

Fletcher

J. M.

Hiscock

(2014). The Flynn effect: A meta-analysis. Psychological Bulletin, 140(5), 1332–1360. https://doi.org/10.1037/a0037173

39.

U.S. Census Bureau. (2017a). 2017 American Community Survey 1-year estimates. https://www.census.gov/programs-surveys/acs/technical-documentation/table-and-geography-changes/2017/1-year.html

40.

U.S. Census Bureau. (2017b). Current population survey (CPS). https://www.census.gov/programs-surveys/cps.html

41.

U.S. Census Bureau. (2019). American Community Survey 5-year estimates. https://www.census.gov/data/developers/data-sets/acs-5year.html

42.

U.S. Census Bureau. (2020). 2020 Census Results. https://www.census.gov/programs-surveys/decennial-census/decade/2020/2020-census-results.html

43.

Victorson

Manly

Wallner-Allen

Fox

Purnell

Hendrie

Havlik

Harniss

Magasi

Correia

Gershon

(2013). Using the NIH Toolbox in special populations: Considerations for assessment of pediatric, geriatric, culturally diverse, non–English-speaking, and disabled individuals. Neurology, 80(11 Suppl 3), S13–S19. https://doi.org/10.1212/WNL.0b013e3182872e26

44.

Wang

Fang

Zheng

Zhao

Wang

Ning

(2024). Associations of socioeconomic status and healthy lifestyle with incident dementia and cognitive decline: Two prospective cohort studies. eClinicalMedicine, 76, 102831. https://doi.org/10.1016/j.eclinm.2024.102831

45.

Wei

McKinlay

J. D.

Harding

J. E.

Wouldes

T. A.

Rogers

Brown

T. L.

Franke

(2025). NIH toolbox for assessment of neurocognitive, motor and emotional-behavioral function in childhood: A systematic review. Child Neuropsychology, 31(6), 948–983. https://doi.org/10.1080/09297049.2024.2447444

46.

Weintraub

Dikmen

S. S.

Heaton

R. K.

Tulsky

D. S.

Zelazo

P. D.

Bauer

P. J.

Carlozzi

N. E.

Slotkin

Blitz

Wallner-Allen

Fox

N. A.

Beaumont

J. L.

Mungas

Nowinski

C. J.

Richler

Deocamp

J. A.

Anderson

J. E.

Manly

J. J.

Borosh

. . . Gershon

R. C.

(2013). Cognition assessment using the NIH Toolbox. Neurology, 80(11 Suppl 3), S54–S64. https://doi.org/10.1212/WNL.0b013e3182872ded

47.

Wechsler

(2008). Wechsler Adult Intelligence Scale–Fourth Edition (WAIS-IV). APA PsycTests. https://doi.org/10.1037/t15169-000

48.

Wechsler

(2009). Wechsler Memory Scale – Fourth Edition (WMS-IV). Pearson

49.

Wechsler

(2012). Wechsler Preschool and Primary Scale of Intelligence (4th ed.). Pearson.

50.

Wechsler

(2014). Wechsler Intelligence Scale for Children—Fifth Edition (WISC-V). Pearson.

51.

Wood

S. N.

Pya

Säfken

(2016). Smoothing parameter and model selection for general smooth models. Journal of the American Statistical Association, 111(516), 1548–1563. https://doi.org/10.1080/01621459.2016.1180986

52.

Zachary

R. A.

Gorsuch

R. L.

(1985). Continuous norming: Implications for the WAIS-R. Journal of Clinical Psychology, 41(1), 86–94.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.30 MB