Normative data for the CERAD verbal episodic memory tests in a sample of older illiterate Korean women

Abstract

Background

Verbal episodic memory tests in the Korean version of the Consortium to Establish a Registry for Alzheimer’s Disease (CERAD) neuropsychological battery require literacy. While modified administrations are provided for illiterate individuals, there are no norms for evaluating the performance of illiterate subjects.

Objective

Assessing the performance of illiterate individuals has limitations, since existing norms were developed on data from literate populations. The purpose of this study is to provide normative data for illiterate Korean older women.

Methods

The participants analyzed in this study were all illiterate. Among a sample of 109 non-demented older adults recruited from community centers for dementia, only 9 were men. Thus, the current analysis developed norms for women. Descriptive statistics were used to summarize participants’ characteristics and test scores. Linear regressions were conducted to examine the effect of age on verbal episodic memory tests.

Results

Age was found to be significantly associated with performance on all word list tests. Age was divided into two groups based on the average value for sufficient sample size and showed a significant difference in test performance: 60–75, ≥76 years.

Conclusions

Evaluating illiterate women using existing standards has the limitation of underestimating their performance relative to their actual skills. The present normative data are expected to enable researchers and clinicians to more accurately evaluate the results of the three verbal episodic memory tests, including the Word List Memory, Word List Recall, and Word List Recognition in older illiterate Korean women.

Keywords

Alzheimer's disease cognitive ageing memory neuropsychological tests

Introduction

The Consortium to Establish a Registry for Alzheimer's Disease (CERAD) neuropsychological assessment battery is a very widely used neuropsychological test battery for the evaluation of cognitive impairment in older adult.^1–3 The Korean version of the CERAD neuropsychological battery [CERAD-K(N)] has also been developed and is commonly used in Korea.^4,5

The CERAD-K(N) includes three verbal episodic memory tests—the Word List Memory (WLM), Word List Recall (WLR), and Word List Recognition (WLRc).^2,4,5 The WLM test assesses learning ability for new verbal information. After the participant reads the 10 words, they are asked to recall them immediately. The WLRc test measures the recognition of the target words presented in the WLM task when presented with the 10 distractor words. Since the WLM and WLRc utilize word list cards, and the WLR requires recalling words from the WLM, these tests require the ability to read words. Therefore, a modified form of administration is provided for those who are illiterate: the administrator reads each word aloud, and the participant repeats it as the card is displayed.

Declines in episodic memory can be observed in both normal and pathological ageing.⁶ Episodic memory impairment is one of the most common early cognitive symptoms of dementia and is particularly disrupted in Alzheimer's disease.⁷ The word list tests in the CERAD-K(N) measure verbal memory function and are thus critical for assessing progression of Alzheimer's disease. Applying norms obtained from individuals with full reading ability to those with no reading skills risks overestimation of cognitive impairment. Therefore, the norms developed for the literate population should not be used for the illiterate subjects. In non-Western countries, attempts have been made to develop norms for both illiterate and low-educated populations.^8,9 In Korea, 10.6% of the population is illiterate; when only women are considered, the rate rises to 15.3%.¹⁰ Despite relatively large groups of illiterate older women in Korea, there are no separate normative data available for this group.

In order to address this limitation, we aimed to provide a modified administration method for the verbal episodic memory tests in the CERAD-K(N) and to provide the normative information on the three tests for illiterate Korean older women.

Methods

Participants

The data were obtained from illiterate older adults who participated in the dementia screening program provided by two regional public centers for dementia located in Seoul, Republic of Korea, from 2010 to 2018. Specifically, the study participants were elderly individuals residing in Jongno-gu and Nowon-gu, Seoul, South Korea. Individuals who voluntarily visited the centers for dementia prevention or management took part in the early screening program upon the recommendation of a staff member from a local dementia care center and underwent the CERAD-K(N) assessment. Following this, each subject was examined by a psychiatrist with advanced training in neuropsychiatry and dementia research according to the protocol of the Korean version of the CERAD clinical assessment battery for clinical diagnosis.⁵ Diagnoses of dementia, probable Alzheimer's disease, and mild cognitive impairment (MCI) were made according to the criteria and guidelines of the fourth edition of the Diagnostic and Statistical Manual of Mental Disorders¹¹ and the National Institute on Aging and the Alzheimer's Association (NIA-AA).¹² A diagnosis of MCI further required a Clinical Dementia Rating (CDR) score of 0.5.¹³ An interview with a reliable informant was also considered in making a final diagnosis. All subjects satisfied the strict entry criteria excluding dementia and other serious medical, psychiatric, and neurological disorders that could affect mental function. Only non-demented individuals including MCI and normal participants were included in the study. Illiteracy was defined as the inability to independently read the 10 words of the WLM test in the CERAD-K(N). This procedure identified 109 individuals without a diagnosis of AD, who were illiterate. Data from this group formed the basis for the current study.

The total number of people who underwent CERAD-K(N) was 1,329, and the proportion of illiterate participants among these subjects was estimated to be 8.2%. Due to past educational practices in Korea, where women received less education, illiterate women outnumber illiterate men in this sample (Men, n = 9; Women, n = 100). The number of men present in the current sample is too small for current analyses, focus is therefore on data from the women.

All participants had adequate vision and hearing, although many used glasses and some relied on hearing aids. Individuals with minor physical conditions, such as diabetes without serious complications, essential hypertension, or mild hearing loss, were not excluded from the study. The study protocol was approved by the institutional review board of the Seoul National University Hospital.

Procedure

The tests listed below were administered by trained psychometrists or research nurses under the supervision by a licensed neuropsychologist. They were blinded to the psychiatrists’ clinical evaluation. The battery includes the eight tests listed below, which are presented in the order of administration of the battery.

Word List Memory test (modified for illiterate persons). WLM is a task to assess learning ability and free-recall memory.^4,14 The task consists of 10 common nouns, each printed on a separate card, and while the list of words remains the same across all 3 trials, the order of presentation is randomized in each trial to minimize the primacy effect. Literate participants are required to read aloud these 10 words; each word is presented for no more than two seconds. Three trials are administered, and after each trial, participants are asked to recall as many words as possible without any prompts.

For the illiterate participants, the administration was modified; an administrator would read aloud each word to a subject, who then will repeat after the administrator as the card is visually presented. Immediately following presentation and oral repetition of the 10 words, the participants are asked to recall as many words as possible without any cues present. The word list used is identical to the word list of CERAD-K(N),⁴ which is validated and currently used.

The total score is calculated by summing the number of correctly recalled words across the three trials, with a maximum of 10 points per trial and 30 points in total. In this study, we report the sum of correctly recalled words across the three trials.

Word List Recall. WLR assess the ability to recall the 10 words given in the WLM, after a delay of few minutes (the duration to complete the Constructional Praxis task, approximately 3∼5 min). A maximum of 90 s is allowed for the recall.

Word List Recognition (modified for illiterate persons). The WLRc task is administered immediately after the WLR task and evaluates the ability to recognize the target words from the WLM task. 20 words are presented in a mixed sequence. Ten of them are target words seen in WLM, and the remaining ten are distractor words.

The participants are asked if they recognize the target words as the test administrator reads out each word while presenting the words on the cards; participants, after hearing the words (and seeing the card), are required to provide a response (yes or no). Consistent with the original administration, the final score is calculated as the total number of correct answers for both the 10 target words and the 10 distractor words minus 10 (the number of correct “yes” responses + the number of correct “no” responses – 10 = recognition score). A score of zero is given if the result is smaller than zero. Therefore, the maximum score is 10.

Statistical analyses

Descriptive statistics were used to summarize the participants’ characteristics, including the mean age and the mean scores with standard deviations for each test: WLM, WLR, and WLRc. A separate linear regression analysis was performed to assess the relative contribution of age to each verbal episodic memory test score. Age was entered as a continuous variable. Participants were then categorized into age-based groups, and the differences in scores for WLM, WLR, and WLRc between the groups were examined. Finally, normative data for each age group were obtained. Statistical analyses were performed using the statistical software Statistical Package for the Social Sciences (SPSS) (version 25, IBM, New York, NY, USA). The graphs and violin plots were created using the MedCalc software (version 20.2, MedCalc Software Ltd, Ostend, Belgium).

Results

Demographic characteristics

The demographic characteristics of the subjects are presented in Table 1. One hundred illiterate women without dementia participated in this study. The mean age was 77.6 (SD = 7.6). The mean score of each test was, WLM 7.3 (SD = 4.5); WLR 1.7 (SD = 1.9); WLRc 4.8 (SD = 3.4) in the total group.

Table 1.

Demographic characteristics of the participants.

Variable	Total (N = 100)
Age (y)	77.6 ± 7.6^a
60–75 (N = 40)	70.4 ± 4.23^a
76–90 (N = 60)	82.3 ± 5.1^a

M ± SD (M: mean; SD: standard deviation).

Effect of age on the word list tests

In normative study, the effect of age is analyzed to determine its significance and whether age-related differences should be considered in establishing normative data. Therefore, the effect of age on each verbal episodic memory test score was analyzed by a linear regression test. As shown in Table 2, there was a significant age effect on the three tests. Therefore, we decided to stratify the subjects into two age groups based on the frequency distribution of the current data to ensure sufficient size of each cell. The age group was divided into two strata; 60–75, and ≥76 years group.

Table 2.

Regression of age on the WLT scores in CERAD-K(N).

Test	Intercept	B (SE)	β	R²
Word list memory	25.65	−0.24 (0.06)	−0.40***	0.16
Word list recall	9.89	−0.11 (0.02)	−0.42***	0.17
Word list recognition	14.39	−0.12 (0.05)	−0.26*	0.07

WLT: word list test; B: regression coefficient; SE: standard error of B; β: standardized regression coefficient; R²: percent variance explained by age. Age was entered as a continuous variable. ***p < 0.001, *p < 0.05. To predict an individual's test score, the following regression equation can be used: Predicted score of word list memory = 25.65 − 0.24 × age; predicted score of word list recall = 9.89 − 0.11 × age; and predicted score of word list recognition = 14.39 − 0.12 × age.

As presented in Table 3 and Figure 1, there was a significant difference between the two age groups for all three tests. Compared to the ≥76 years group, the 60–75 years group performed significantly better on the WLM, WLR, and WLRc. The violin plots illustrate score distributions across age groups. The 60–75 years group displays a narrower, taller distribution, reflecting more consistent performance, whereas the ≥76 years group shows a wider, flatter distribution, indicating greater variability in test scores and a broader range of cognitive abilities.

Figure 1.

Performance on the word list memory, word list recall, and word list recognition tasks, with violin plots showing the distribution of scores across the two age groups.

Table 3.

Normative table of the WLT scores by age.

			Age (y)
Variables		Total	60–75	≥76	t-test
Word List Memory	n	99	40	59
	M	7.26	9.30	5.88	<0.001***
	SD	4.49	4.13	4.21
Word List Recall	n	100	40	60
	M	1.74	2.68	1.12	<0.001***
	SD	1.91	1.90	1.65
Word List Recognition	n	97	39	58
	M	4.78	6.18	3.84	0.001**
	SD	3.41	3.13	3.29

WLT: word list test; y: years; n: numbers; M: mean; SD: standard deviation; **p < 0.01; ***p < 0.001 by t-test.

Normative data of WLM, WLR, WLRc

Based on the results of above analyses, we decided to provide the norms for the two age groups. The normative data for the WLM, WLR, and WLRc tests are presented in Table 3. Slight differences in sample numbers are observed for each test. In the WLM, one participant responded to trials 1 and 2 but refused to respond to trial 3 (female, 76 years old), but readily responded to the WLR. For the WLRc, there were a total of 3 cases in which scores could not be obtained. The reasons were case of not performing the test, refusing to continue to participate, and having difficulty in understanding the instruction (female, 63 years old; female, 92 years old; female, 94 years old, respectively); for those who completed the WLM and WLR, the recognition trial was given.

Discussion

The current study was aimed to provide normative data for the CERAD-K(N) verbal episodic memory tests (WLM, WLR, as well as WLRc) for older illiterate women in Korea. Our analyses revealed that all the three episodic memory tests were influenced by age. Based on the results, normative data stratified only by age are provided.

Our findings on the association between age and the scores of the three verbal episodic memory tests align with previous studies on literate individuals.⁴ Beyond age, gender is commonly cited as an important demographic factor influencing test performance. A prior study of literate older Korean adults found a significant gender effect on the episodic memory tests of the CERAD-K(N).⁴ However, the current study, based on data from a local dementia care center, included only 9 male participants, making the data unsuitable for analyzing gender effects across the combined male and female sample. It is worth noting that recruiting a sufficient number of illiterate older males from diverse regions in Korea is challenging, as the majority of the illiterate older population consists of women (approximately 85.3% of 900,000, according to the Korea National Statistical Office report in 2020).¹⁵ Thus, the norms developed in this study are expected to better reflect the actual phenomenon, where illiteracy is predominantly concentrated among women, and to enhance clinical usefulness when evaluating illiterate individuals.

The mean value of numerous published normative data for literate older adults was around 20/30 for the WLM (immediate recall) or above 7/10 for the WLR (delayed recall).^16,17 In contrast, the current norms for illiterate Korean older women were 7.26 for WLM, 1.74 for WLR, and 4.78 for WLRc. Given these considerations, the use of normative data derived from literate populations may lead to an increased risk of misclassifying cognitively intact illiterate older adults as cognitively impaired. A recent study reported that healthy illiterate elders showed lower performance than low-functioning or cognitively impaired educated/literate individuals. This discrepancy may reflect differences in neural substrates and learning strategies.¹⁸ Additionally, formal education and training in reading and writing enable people to develop intellectual skills necessary for successful cognitive test performance, which relies heavily on language function.^19,20 Therefore, illiterate individuals with limited opportunities to develop such skills are more likely to exhibit relatively lower verbal memory performance. In Korea, specifically, males tend to have more opportunities for social activities that can influence cognitive test performance, even with the same level of education.²¹

Considering the significant impact of literacy, educational differences, and the unique characteristics of illiterate women on cognitive test performance, establishing separate normative data that account for these factors is crucial. In non-Western populations, where lower educational attainment may be more prevalent, conventional cognitive assessment tools often fail to accurately reflect the abilities of these individuals. Previous studies have emphasized the need for distinct normative data to assess verbal memory function in the illiterate elderly population.⁸ In Korea, while the population is less ethnically diverse than in Western countries, rapid ageing has resulted in a substantial proportion of illiterate individuals among the elderly. This underscores the importance of separate normative data to address educational disparities in this demographic group.

In the current CERAD-K(N) norms, data are provided for individuals with zero years of formal education. These norms have traditionally been used to assess the cognitive performance of illiterate individuals. However, the normative data were derived from individuals who, despite lacking formal education, still possessed basic literacy skills. This differs significantly from the present sample, which includes individuals with no formal education and no literacy skills. Applying the existing CERAD-K(N) norms to a fully illiterate population is problematic and may result in inaccurate assessments, particularly for early cognitive decline, such as episodic memory deficits commonly seen in Alzheimer's disease. In typical Alzheimer's disease, episodic memory decline is more consistently observed compared to other cognitive domains.²² Considering that both illiteracy²³ and female gender²⁴ are significant risk factors for dementia, the new norms developed in this study are expected to be more accurate in assessing the episodic memory of illiterate women. Identifying the preclinical stages of cognitive decline is particularly important. Since subjective memory complaints are prevalent in this population, appropriate normative data are essential for distinguishing normal ageing from true cognitive decline. Typically, a score 1.5 standard deviations below the matched control group indicates a decline in the specific cognitive domain.²⁵ Furthermore, core clinical criteria for cognitive impairment include deficits in one or more cognitive domains.^26,27

This study has some limitations that should be considered. First, we provide illiterate-based norm only for the verbal episodic memory tests (WLM, WLR, and WLRc). We selected these tests among the eight tests in the CERAD-K(N) battery because they are important for objectively identifying cognitive declines. Performance of the other cognitive tests that are not dependent on reading skill, such as Verbal Fluency test, Boston Naming test, Constructional Praxis, and Constructional Recall, may also be negatively influenced by illiteracy, given that visual perception, visuo-constructional ability, executive function, and visual memory functions are also improved by formal educations.^19,28,29 Future research to investigate the influence of illiteracy on the performance of the other tests will be useful. Second, the sample size was limited because the samples were selected based on convenience sampling; the data were collected from a nearby regional public center for dementia. Collecting data from dementia centers in other regions in the future would be highly beneficial for developing improved normative data for illiterate participants for other tests in the CERAD-K(N).

Footnotes

Acknowledgments

We sincerely thank the subjects in this study and supporting team members.

ORCID iDs

Hyejin Ahn

Dahyun Yi

Dong Young Lee

Author contributions

Hyejin Ahn (Writing – original draft; Writing – review & editing); Haejung Joung (Writing – original draft); Min Soo Byun (Data curation); Jee Eun Park (Data curation); Gijung Jung (Data curation); Dahyun Yi (Supervision; Writing – review & editing); Dong Young Lee (Supervision; Writing – review & editing).

Funding

This study was supported by grants from the Ministry of Science, ICT, and Future Planning, Republic of Korea (Grant No: NRF-2014M3C7A1046042, NRF-2020R1G1A1099652 & RS-2022-00165636) and grants of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (Grant No: HI18C0630, HI19C0149 & HU23C0140), a grant from New Faculty Startup Fund from Seoul National University, and a grant from the National Institute of Aging, United States of America (U01AG072177). The funding source had no role in the study design, data collection, data analysis, data interpretation, writing of the manuscript, or decision to submit it for publication.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data availability

The data supporting the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.

References

Consortium to Establish a Registry for Alzheimer's Disease (CERAD), https://agingcenter.duke.edu/cerad (2023, accessed 24 November 2023).

Welsh-Bohmer

Mohs

. Neuropsychological assessment of Alzheimer's disease. Neurology 1997; 49: S11–S13.

Morris

Heyman

Mohs

, et al. The consortium to establish a registry for Alzheimer's disease (CERAD). part I. Clinical and neuropsychological assessment of Alzheimer's disease. Neurology 1989; 39: 1159–1165.

Lee

, et al. A normative study of the CERAD neuropsychological assessment battery in the Korean elderly. J Int Neuropsychol Soc 2004; 10: 72–81.

Lee

, et al. Development of the Korean version of the consortium to establish a registry for Alzheimer's disease assessment packet (CERAD-K) clinical and neuropsychological assessment batteries. J Gerontol B Psychol Sci Soc Sci 2002; 57: P47–P53.

Tromp

Dufour

Lithfous

, et al. Episodic memory in normal aging and Alzheimer disease: insights from imaging and behavioral studies. Ageing Res Rev 2015; 24: 232–262.

Beck

Gagneux-Zurbriggen

Berres

, et al. Comparison of verbal episodic memory measures: consortium to establish a registry for Alzheimer's disease—Neuropsychological Assessment Battery (CERAD-NAB) versus California Verbal Learning Test (CVLT). Arch Clin Neuropsychol 2012; 27: 510–519.

Franzen

van den Berg

Goudsmit

, et al. A systematic review of neuropsychological tests for the assessment of dementia in non-western, low-educated or illiterate populations. J Int Neuropsychol Soc 2020; 26: 331–351.

Paddick

S-M

Gray

McGuire

, et al. Cognitive screening tools for identification of dementia in illiterate and low-educated older adults, a systematic review and meta-analysis. Int Psychogeriatr 2017; 29: 897–929.

10.

Lee

Kim

Hwang

, et al. Survey of living conditions and welfare needs of Korean older persons. https://www.mohw.go.kr/board.es?mid=a10411010100&bid=0019&act=view&list_no=366496 (2020).

11.

American Psychiatric Association. Diagnostic and statistical manual of mental disorders: DSM-IV. Washington, DC: American Psychiatric Association, 1994.

12.

Albert

DeKosky

Dickson

, et al. The diagnosis of mild cognitive impairment due to Alzheimer's disease: recommendations from the National Institute on Aging-Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's disease. Alzheimers Dement 2011; 7: 270–279.

13.

Morris

. The Clinical Dementia Rating (CDR) current version and scoring rules. Neurology 1993; 43: 2412–2414.

14.

Welsh

Butters

Mohs

, et al. The Consortium to Establish a Registry for Alzheimer's Disease (CERAD). Part V. A normative study of the neuropsychological battery. Neurology 1994; 44: 609–609.

15.

Korean Statistical Information Service (KOSIS). Population and housing census, elderly people (over 60 years old) by gender/age/education level - city, county, and district. https://kosis.kr/statHtml/statHtml.do?orgId=101&tblId=DT_1PG2001&conn_path=I2 (2020).

16.

Kirsebom

B-E

Espenes

Hessen

, et al. Demographically adjusted CERAD wordlist test norms in a Norwegian sample from 40 to 80 years. Clin Neuropsychol 2019; 33: 27–39.

17.

Luck

Pabst

Rodriguez

, et al. Age-, sex-, and education-specific norms for an extended CERAD neuropsychological assessment battery—results from the population-based LIFE-adult-study. Neuropsychology 2018; 32: 461.

18.

Mandyla

M-A

Yannakoulia

Hadjigeorgiou

, et al. Identifying appropriate neuropsychological tests for uneducated/illiterate older individuals. J Int Neuropsychol Soc 2022; 28: 862–875.

19.

Ardila

Ostrosky-Solis

Rosselli

, et al. Age-related cognitive decline during normal aging: the complex effect of education. Arch Clin Neuropsychol 2000; 15: 495–513.

20.

Byrd

Jacobs

Hilton

, et al. Sources of errors on visuoperceptual tasks: role of education, literacy, and search strategy. Brain Cogn 2005; 58: 251–257.

21.

Chung

J-S

. Women's unequal access to education in South Korea. Comp Educ Rev 1994; 38: 487–505.

22.

Gagliardi

Epelbaum

Houot

, et al. Which episodic memory performance is associated with Alzheimer’s disease biomarkers in elderly cognitive complainers? Evidence from a longitudinal observational study with four episodic memory tests (Insight-PreAD). J Alzheimers Dis 2019; 70: 811–824.

23.

Arce Rentería

Vonk

Felix

, et al. Illiteracy, dementia risk, and cognitive trajectories among older adults with low education. Neurology 2019; 93: e2247–e2256.

24.

Gong

Harris

Lipnicki

, et al. Sex differences in dementia risk and risk factors: individual-participant data analysis using 21 cohorts across six continents from the COSMIC consortium. Alzheimers Dement 2023; 19: 3365–3378.

25.

Petersen

Smith

Waring

, et al. Mild cognitive impairment: clinical characterization and outcome. Arch Neurol 1999; 56: 303–308.

26.

Albert

DeKosky

Dickson

, et al. The diagnosis of mild cognitive impairment due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Focus 2013; 11: 96–106.

27.

American Psychiatric Association. Diagnostic and statistical manual of mental disorders: DSM-5. Washington, DC: American Psychiatric Association, 2013.

28.

Dekhtyar

Wang

H-X

Fratiglioni

, et al. Childhood school performance, education and occupational complexity: a life-course study of dementia in the Kungsholmen project. Int J Epidemiol 2016; 45: 1207–1215.

29.

Balduino

de Melo

BAR

da Silva

LSM

, et al. The “SuperAgers” construct in clinical practice: neuropsychological assessment of illiterate and educated elderly. Int Psychogeriatr 2020; 32: 191–198.