Psychometric Properties of Controlled Oral Word Association (COWA) Test and Associations With Education and Bilingualism in American Indian Adults: The Strong Heart Study

Abstract

The Controlled Oral Word Association (COWA) test is used to assess phonemic fluency and executive function. Formal validation of test scores is important for accurate cognitive evaluation. However, there is a dearth of psychometric validation among American Indian adults. Given high burden of dementia risk and key contextual factors associated with cognitive assessments, this represents a critical oversight. In a large, longitudinal population-based cohort study of adult American Indians, we examined several validity inferences for COWA, including scoring, generalization, and extrapolation inferences, by investigation of factor structure, internal consistency, test–retest reliability, and differential test functioning. We found adequate unidimensional model fit, with high factor loadings. Internal consistency reliability and test–retest reliability were 0.88 and 0.77, respectively, for the full group. COWA scores were lowest among the oldest, lowest education, bilingual speakers; group effects for sex and bilingual status were small; age effect was medium; and education effect was largest. However, Wide Range Achievement Test (WRAT) score effect was stronger than education effect, suggesting better contextualization may be needed. These results support interpretation of total COWA score, including across sex, age, or language use strata.

Keywords

verbal fluency validity reliability differential item functioning American Indian

Get full access to this article

View all access options for this article.

References

Adesope

O. O.

Lavin

Thompson

Ungerleider

(2010). A systematic review and meta-analysis of the cognitive correlates of bilingualism. Review of Educational Research, 80(2), 207–245. https://doi.org/10.3102/0034654310368803

Axelrod

B. N.

Henry

R. R.

(1992). Age-related performance on the Wisconsin card sorting, similarities, and controlled oral word association tests. Clinical Neuropsychologist, 6(1), 16–26. https://doi.org/10.1080/13854049208404113

Baraldi

A. N.

Enders

C. K.

(2010). An introduction to modern missing data analyses. Journal of School Psychology, 48(1), 5–37. https://doi.org/10.1016/j.jsp.2009.10.001

Barry

Bates

M. E.

Labouvie

(2008). FAS and CFL forms of verbal fluency differ in difficulty: A meta-analytic study. Applied Neuropsychology, 15(2), 97–106. https://doi.org/10.1080/09084280802083863

Bassuk

S. S.

Murphy

J. M.

(2003). Characteristics of the modified mini-mental state exam among elderly persons. Journal of Clinical Epidemiology, 56(7), 622–628. https://doi.org/10.1016/S0895-4356(03)00111-2

Benjamini

Hochberg

(1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1), 289–300. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x

Benton

A. L.

De Hamsher

S. K.

Sivan

A. B.

(1994). Multilingual aphasia examination (3rd. ed.). Psychological Assessment Resources.

Bialystok

Craik

F. I.

Freedman

(2007). Bilingualism as a protection against the onset of symptoms of dementia. Neuropsychologia, 45(2), 459–464. https://doi.org/10.1016/j.neuropsychologia.2006.10.009

Bialystok

Craik

F. I.

Luk

(2008). Lexical access in bilinguals: Effects of vocabulary size and executive control. Journal of Neurolinguistics, 21(6), 522–538.

10.

Bolla

K. I.

Lindgren

K. N.

Bonaccorsy

Bleecker

M. L.

(1990). Predictors of verbal fluency (FAS) in the healthy elderly. Journal of Clinical Psychology, 46(5), 623–628. https://doi.org/10.1002/1097-4679(199009)46:5<623::aid-jclp2270460513>3.0.co;2-c

11.

Boone

K. B.

(1999). Neuropsychological assessment of executive functions: Impact of age, education, gender, intellectual level, and vascular status on executive test scores. In The human frontal lobes: Functions and disorders (pp. 247–260). Guilford Press.

12.

Brown

T. A.

(2015). Confirmatory factor analysis for applied research (2nd ed.). Guilford Press.

13.

Browne

M. W.

Cudeck

(1992). Alternative ways of assessing model fit. Sociological Methods & Research, 21(2), 230–258. https://doi.org/10.1177/0049124192021002005

14.

Casaletto

K. B.

Umlauf

Beaumont

Gershon

Slotkin

Akshoomoff

Heaton

R. K.

(2015). Demographically corrected normative standards for the English version of the NIH toolbox cognition battery. Journal of the International Neuropsychological Society, 21(5), 378–391. https://doi.org/10.1017/S1355617715000351

15.

Centers for Disease Control and Prevention. (2019). Alzheimer’s disease and healthy aging. Subjective cognitive decline among American Indian/Alaska native adults. https://www.cdc.gov/aging/data/infographic/2017/american-Indian-alaska-native-cognitive-decline.html

16.

Cheng

Shao

Lathrop

Q. N.

(2016). The mediated MIMIC model for understanding the underlying mechanism of DIF. Educational and Psychological Measurement, 76(1), 43–63. https://doi.org/10.1177/0013164415576187

17.

Cizek

Rosenberg

Koons

(2008). Sources of validity evidence for educational and psychological tests. Educational and Psychological Measurement, 68, 397–412. https://doi.org/10.1177/0013164407310130

18.

Cole

E. R.

(2009). Intersectionality and research in psychology. American Psychologist, 64(3), 170–180. https://doi.org/10.1037/a0014564

19.

Costa

Hernandez

Sebastian-Galles

(2008). Bilingualism aids conflict resolution: Evidence from the ANT task. Cognition, 106(1), 59–86. https://doi.org/10.1016/j.cognition.2006.12.013

20.

Craik

F. I.

Bialystok

Freedman

(2010). Delaying the onset of Alzheimer disease: Bilingualism as a form of cognitive reserve. Neurology, 75(19), 1726–1729. https://doi.org/10.1212/WNL.0b013e3181fc2a1c

21.

Crane

P. K.

Gruhl

J. C.

Erosheva

E. A.

Gibbons

L. E.

McCurry

S. M.

Rhoads

. . . White

(2010). Use of spoken and written Japanese did not protect Japanese-American men from cognitive decline in late life. Journal of Gerontology: Series B, 65(6), 654–666. https://doi.org/10.1093/geronb/gbq046

22.

Crenshaw

K. W.

(1989). Demarginalizing the intersection of race and sex: A Black feminist critique of antidiscrimination doctrine, feminist theory and antiracist politics. University of Chicago Legal Forum, 1989(1). Article 8.

23.

Enoch

M. A.

Albaugh

B. J.

(2017). Review: Genetic and environmental risk factors for alcohol use disorders in American Indians and Alaskan Natives. American Journal of Addictions, 26(5), 461–468. https://doi.org/10.1111/ajad.12420

24.

Grace

Nadler

J. D.

White

D. A.

Guilmette

T. J.

Giuliano

A. J.

Monsch

A. U.

Snow

M. G.

(1995). Folstein vs modified Mini-Mental State Examination in geriatric stroke. Stability, validity, and screening utility. Archives of Neurology, 52(5), 477–484. https://doi.org/10.1001/archneur.1995.00540290067019

25.

Harvey

Siegert

R. J.

(1999). Normative data for New Zealand elders on the controlled oral word association test, graded naming test, and the recognition memory test. New Zealand Journal of Psychology, 28, 124.

26.

Jack

A. S.

, & Secwepemc Cultural Education Society. (2000). Behind closed doors: Stories from the Kamloops Indian Residential School. Secwepemc Cultural Education Society.

27.

Kane

M. T.

(2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. https://doi.org/10.1111/jedm.12000

28.

Kenny

D. A.

Kaniskan

McCoach

D. B.

(2015). The performance of RMSEA in models with small degrees of freedom. Sociological Methods & Research, 44, 486–507. https://doi.org/10.1177/0049124114543236

29.

Kiselica

A. M.

Johnson

Lewis

K. R.

Trout

(2021). Examining racial disparities in the diagnosis of mild cognitive impairment. Applied Neuropsychology: Adult. Advance online publication. https://doi.org/10.1080/23279095.2021.1976778

30.

Lakens

(2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontier in Psychology, 4, Article 863. https://doi.org/10.3389/fpsyg.2013.00863

31.

Lee

E. T.

Welty

T. K.

Fabsitz

Cowan

L. D.

N. A.

Oopik

A. J.

. . . Howard

B. V.

(1990). The Strong Heart Study a study of cardiovascular disease in American Indians: Design and methods. American Journal of Epidemiology, 132(6), 1141–1155. https://doi.org/10.1093/oxfordjournals.aje.a115757

32.

Lenhard

(2016). Computation of effect sizes. https://doi.org/10.13140/RG.2.2.17823.92329

33.

Lezak

M. D.

(2012). Neuropsychological assessment (5th ed.). Oxford University Press.

34.

Libon

D. J.

Glosser

Malamut

B. L.

Kaplan

Goldberg

Swenson

Prouty Sands

(1994). Age, executive functions, and visuospatial functioning in healthy older adults. Neuropsychology, 8(1), 38–43. https://doi.org/10.1037/0894-4105.8.1.38

35.

Lissitz

R. W.

(2009). The concept of validity: Revisions, new directions, and applications. Information Age.

36.

Little

R. J. A.

(1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83(404), 1198–1202. https://doi.org/10.1080/01621459.1988.10478722

37.

Liu

(2021). Lifelong bilingualism functions as an alternative intervention for cognitive reserve against Alzheimer’s disease. Frontiers in Psychiatry, 12, Article 696015. https://doi.org/10.3389/fpsyt.2021.696015

38.

Lomawaima

K. T.

McCarty

T. L.

(2006). Table of contents for “To remain an Indian”: Lessons in democracy from a century of Native American education. Teachers College Press. http://www.loc.gov/catdir/toc/fy0702/2006043864.html

39.

Loonstra

A. S.

Tarlow

A. R.

Sellers

A. H.

(2001). COWAT metanorms across age, education, and gender. Applied Neuropsychology, 8(3), 161–166. https://doi.org/10.1207/S15324826AN0803_5

40.

Lynch

P. D. C. M

. (1990). The emergence of American Indian leadership in education. Journal of American Indian Education, 29, 1–10.

41.

Manly

J. J.

Jacobs

D. M.

Sano

Bell

Merchant

C. A.

Small

S. A.

Stern

(1998). Cognitive test performance among nondemented elderly African Americans and Whites. Neurology, 50(5), 1238–1245. https://doi.org/10.1212/wnl.50.5.1238

42.

McDonald

R. P.

(1999). Test theory: A unified treatment. Lawrence Erlbaum.

43.

McDowell

Kristjansson

Hill

G. B.

Hébert

(1997). Community screening for dementia: The Mini Mental State Exam (MMSE) and modified Mini-Mental State Exam (3MS) compared. Journal of Clinical Epidemiology, 50(4), 377–383. https://doi.org/10.1016/S0895-4356(97)00060-7

44.

Mehta

K. M.

Yeo

G. W.

(2017). Systematic review of dementia prevalence and incidence in United States race/ethnic populations. Alzheimers Dementia, 13(1), 72–83. https://doi.org/10.1016/j.jalz.2016.06.2360

45.

Mervis

(2009). Informal education: Report calls for fresh look at what happens outside school. Science, 323(5914), 572–573. https://doi.org/10.1126/science.323.5914.572a

46.

Muthén

B. O.

(1989). Latent variable modeling in heterogeneous populations. Psychometrika, 54, 557–585.

47.

Muthén

B. O.

(1998-2004). Mplus technical appendices.

48.

Nunnally

J. C.

Bernstein

I. H.

(1994). Psychometric theory. McGraw-Hill.

49.

Pedraza

Clark

J. H.

O'Bryant

S. E.

Smith

G. E.

Ivnik

R. J.

Graff-Radford

N. R.

. . . Lucas

J. A.

(2012). Diagnostic validity of age and education corrections for the Mini-Mental State Examination in older African Americans. Journal of the American Geriatricks Society, 60(2), 328–331. https://doi.org/10.1111/j.1532-5415.2011.03766.x

50.

Prior

A. M. B.

(2010). A bilingual advantage in task switching. Bilingualism: Language and Cognition, 13(2), 253–262. https://doi.org/10.1017/S1366728909990526

51.

Rivera Mindt

Byrd

Saez

Manly

(2010). Increasing culturally competent neuropsychological services for ethnic minority populations: A call to action. The Clinical Neuropsychologist, 24(3), 429–453. https://doi.org/10.1080/13854040903058960

52.

Rodriguez-Aranda

Martinussen

(2006). Age-related differences in performance of phonemic verbal fluency measured by Controlled Oral Word Association Task (COWAT): A meta-analytic study. Developemental Neuropsychology, 30(2), 697–717. https://doi.org/10.1207/s15326942dn3002_3

53.

Rosenthal

(2016). Incorporating intersectionality into psychology: An opportunity to promote social justice and equity. American Psychologist, 71(6), 474–485. https://doi.org/10.1037/a0040323

54.

Ruff

R. M.

Light

R. H.

Parker

S. B.

Levin

H. S.

(1996). Benton controlled oral word association test: Reliability and updated norms. Archives of Clinical Neuropsychology, 11(4), 329–338. https://www.ncbi.nlm.nih.gov/pubmed/14588937

55.

Running Bear

Thayer

Z. M.

Croy

C. D.

Kaufman

C. E.

Manson

S. M.

Team

A.-S.

(2019). The impact of individual and parental American Indian boarding school attendance on chronic physical health of northern plains tribes. Family and Community Health, 42(1), 1–7. https://doi.org/10.1097/FCH.0000000000000205

56.

Sayegh

Arentoft

Thaler

N. S.

Dean

A. C.

Thames

A. D.

(2014). Quality of education predicts performance on the Wide Range Achievement Test-4th Edition word reading subtest. Archives of Clinical Neuropsychology, 29(8), 731–736. https://doi.org/10.1093/arclin/acu059

57.

Saykin

A. J.

Gur

R. C.

Gur

R. E.

Shtasel

D. L.

Flannery

K. A.

Mozley

L. H.

. . . Mozley

P. D.

(1995). Normative neuropsychological test performance: Effects of age, education, gender and ethnicity. Applied Neuropsychology, 2(2), 79–88. https://doi.org/10.1207/s15324826an0202_5

58.

Scribner

Cole

(1973). Cognitive consequences of formal and informal education: New accommodations are needed between school-based learning and learning experiences of everyday life. Science, 182(4112), 553–559. https://doi.org/10.1126/science.182.4112.553

59.

Selnes

O. A.

Jacobson

Machado

A. M.

Becker

J. T.

Wesch

Miller

E. N.

. . . McArthur

J. C.

(1991). Normative data for a brief neuropsychological screening battery. Multicenter AIDS cohort study. Perceptual and Motor Skills, 73(2), 539–550. https://doi.org/10.2466/pms.1991.73.2.539

60.

Settles

I. H.

O’Connor

R. C.

Yap

S. C. Y.

(2016). Climate perceptions and identity interference among undergraduate women in STEM: The protective role of gender identity. Psychology of Women Quarterly, 40(4), 488–503. https://doi.org/10.1177/0361684316655806

61.

Shi

DiStefano

Maydeu-Olivares

Lee

(2022). Evaluating SEM model fit with small degrees of freedom. Multivariate Behavioral Research, 57(2–3), 179–207. https://doi.org/10.1080/00273171.2020.1868965

62.

Suchy-Dicey

A. M.

Eyituoyo

O'Leary

Cole

S. A.

Traore

Verney

. . . Whitney

(2022). Psychological and social support associations with mortality and cardiovascular disease in middle-aged American Indians: The Strong Heart Study. Social Psychiatry and Psychiatric Epidemiology, 57(7), 1421–1433. https://doi.org/10.1007/s00127-022-02237-7

63.

Suchy-Dicey

A. M.

Howard

Longstreth

W. T.

Jr. Reiman

E. M.

Buchwald

(2022). APOE genotype, hippocampus, and cognitive markers of Alzheimer's disease in American Indians: Data from the Strong Heart Study. Alzheimers Dementia, 18, 2518–2526. https://doi.org/10.1002/alz.12573

64.

Suchy-Dicey

A. M.

Muller

C. J.

Madhyastha

T. M.

Shibata

Cole

S. A.

Zhao

. . . Buchwald

(2018). Telomere length and magnetic resonance imaging findings of vascular brain injury and central brain atrophy: The Strong Heart Study. American Journal of Epidemiology, 187(6), 1231–1239. https://doi.org/10.1093/aje/kwx368

65.

Suchy-Dicey

A. M.

Muller

C. J.

Shibata

Howard

B. V.

Cole

S. A.

Longstreth

W. T.

Jr. . . . Buchwald

(2021). Comparing vascular brain injury and stroke by cranial magnetic resonance imaging, physician-adjudication, and self-report: Data from the Strong Heart Study. Neuroepidemiology, 55(5), 398–406. https://doi.org/10.1159/000517804

66.

Suchy-Dicey

A. M.

Oziel

Sawyer

Olufadi

Ali

Fretts

A. M.

Umans

J. G.

Shibata

D. K.

Longstreth

W. T.

Rhoads

Buchwald

D.S.

(2022). Educational and clinical associations with longitudinal cognitive function and brain imaging in American Indians: The Strong Heart Study. Neurology, 99(24), e2637–e2647. https://doi.org/10.1212/WNL.0000000000201261

67.

Suchy-Dicey

A. M.

Shibata

Best

L. G.

Verney

S. P.

Longstreth

W. T.

Jr. Lee

E. T.

. . . Buchwald

(2016). Cranial magnetic resonance imaging in elderly American Indians: Design, methods, and implementation of the Cerebrovascular disease and its consequences in American Indians study. Neuroepidemiology, 47(2), 67–75. https://doi.org/10.1159/000443277

68.

Suchy-Dicey

A. M.

Shibata

Cholerton

Nelson

Calhoun

Ali

. . . Verney

S. P.

(2020). Cognitive correlates of MRI-defined cerebral vascular injury and atrophy in elderly American Indians: The Strong Heart Study. Journal of the International Neuropsychological Society, 26(3), 263–275. https://doi.org/10.1017/S1355617719001073

69.

Suchy-Dicey

A. M.

Verney

S. P.

Nelson

L. A.

Barbosa-Leiker

Howard

B. A.

Crane

P. K.

Buchwald

D. S.

(2020). Depression symptoms and cognitive test performance in older American Indians: The Strong Heart Study. Journal of the American Geriatrics Society, 68(8), 1739–1747. https://doi.org/10.1111/jgs.16434

70.

Tombaugh

T. N.

Kozak

Rees

(1999). Normative data stratified by age and education for two measures of verbal fluency: FAS and animal naming. Archives of Clinical Neuropsychology, 14(2), 167–177. https://www.ncbi.nlm.nih.gov/pubmed/14590600

71.

Tombaugh

T. N.

McDowell

Kristjansson

Hubley

A. M.

(1996). Mini-Mental State Examination (MMSE) and the Modified MMSE (3MS): A psychometric comparison and normative data. Psychological Assessment, 8(1), 48–59. https://doi.org/10.1037/1040-3590.8.1.48

72.

Veizel

Zibulsky

(2013). Wide range achievement test–fourth edition. In Reynolds

C. R.

Vannest

K. J.

Fletcher-Janzen

(Eds.), Encyclopedia of special education: A reference for the education of children, adolescents, and adults with disabilities and other exceptional individuals. Wiley.

73.

Verney

S. P.

Suchy-Dicey

A. M.

Cholerton

Calhoun

Nelson

Montine

T. J.

. . . Buchwald

(2019). The associations among sociocultural factors and neuropsychological functioning in older American Indians: The Strong Heart Study. Neuropsychology, 33(8), 1078–1088. https://doi.org/10.1037/neu0000574

74.

Widaman

K. F.

(2006). Best practices in quantitative methods for developmentalists: III. Missing data: What to do with or without them. Monographs of the Society for Research in Child Development, 71(3), 42–64. https://doi.org/10.1111/j.1540-5834.2006.00404.x

75.

Zahodne

L. B.

Manly

J. J.

Smith

Seeman

Lachman

M. E.

(2017). Socioeconomic, health, and psychosocial mediators of racial disparities in cognition in early, middle, and late adulthood. Psychology and Aging, 32(2), 118–130. https://doi.org/10.1037/pag0000154

76.

Zahodne

L. B.

Sharifian

Kraal

A. Z.

Zaheed

A. B.

Sol

Morris

E. P.

. . . Brickman

A. M.

(2021). Socioeconomic and psychosocial mechanisms underlying racial/ethnic disparities in cognition among older adults. Neuropsychology, 35(3), 265–275. https://doi.org/10.1037/neu0000720

77.

Zhang

Galloway

J. M.

Welty

T. K.

Wiebers

D. O.

Whisnant

J. P.

Devereux

R. B.

. . . Lee

E. T.

(2008). Incidence and risk factors for stroke in American Indians: The Strong Heart Study. Circulation, 118(15), 1577–1584. https://doi.org/10.1161/CIRCULATIONAHA.108.772285