Sage Journals: Discover world-class research

Abstract

Background

With the arrival of new disease-modifying treatments for Alzheimer's disease (AD), feasible cognitive tests, also for illiterate patients, are needed to screen those requiring deeper evaluation among individuals presenting with memory complaints. The TMA-93, a brief binding memory test, has proven useful for diagnosing early AD, and is supported by normative data that accounts for age and cognitive reserve.

Objective

To compare the sensitivity of different TMA-93 cutoffs in detecting AD pathology.

Methods

A retrospective analysis was performed on a biobank sample of patients with confirmed AD pathology via amyloid PET or cerebrospinal fluid (CSF) biomarkers. The sensitivity of six TMA-93 cutoffs was evaluated: the 10th, 15th, and 20th percentiles based on traditional norming (TN) and regression-based norming (RBN). False negatives (FN) characteristics were also analyzed.

Results

A total of 270 AD-positive patients (96 by amyloid-PET, 174 by CSF biomarkers) were included, comprising 224 with mild cognitive impairment and 46 with mild dementia. The 15th percentile using RBN demonstrated substantial sensitivity (80.4%), higher than that of the 10th percentile, and also provided a more uniform distribution across normative groups compared to the TN approach. Higher global cognition (Mini-Mental State Examination score) and, in patients over 70, lower cognitive reserve (Cognitive Reserve Questionnaire), were linked to a greater likelihood of FN results.

Conclusions

The 15th percentile cutoff based on RBN, accounting for age and cognitive reserve, improves sensitivity for detecting AD pathology, making it a valuable screening tool for memory complaints. Future normative data from biomarker-negative subjects may enhance the sensitivity of cognitive tests.

Keywords

Alzheimer's disease biomarkers cognition cognitive reserve memory neuropsychological tests normative data

Introduction

Within the current context, due to the development of fluid and neuroimaging biomarkers, Alzheimer's disease (AD) is defined from a biological perspective, as reflected in the current diagnostic criteria.^{1, 2} This allows for the diagnosis in earlier stages of the disease, even in the preclinical phase. However, across much of the world, patients with memory complaints typically receive their initial assessments in general neurology or primary care settings, where consultation times are often severely limited. Furthermore, when referred to specialized memory clinics, patients may experience significant delays before evaluation. Compounding these challenges, in many regions there is a high proportion of illiterate patients, which complicates cognitive assessments.^{3, 4} Consequently, there is a pressing need for sensitive memory tests that can effectively discriminate patients who would benefit from more comprehensive evaluations, including biomarker studies. This need is becoming increasingly critical with the development of new disease-modifying therapies.⁵ Ideally, these tests should have published normative data that account for sociodemographic variables that impact the performance, such as educational attainment, be feasible in settings with limited time for consultations, and be well-accepted by patients.⁶

Associative or binding memory involves recalling pairs of semantically-related words or pictures.⁷ Previous studies established that this binding memory is promptly affected in prodromal or even preclinical stages of AD, and tests assessing it may be more sensitive than other memory tests.^8–10 The Memory Associative Test of the district of Seine-Sant-Denis (TMA-93) is a relational binding test that evaluates the ability to remember pairs of drawings of semantically related common objects.¹¹ This makes it useful in patients with lower educational level, where tests using verbal material are challenging to interpret. The test is remarkably feasible, well-accepted by patients, can be administered by non-specialized staff, and has an average administration time of less than 3 min in healthy controls and less than 6 min in patients with mild cognitive impairment (MCI).¹²

In a first work, TMA-93 demonstrated to be as accurate as the Free and Cued Selective Reminding Test (FCSRT) in discriminating patients with amnestic MCI from healthy controls.¹³ A posterior validation study, using biomarkers as ground truth, confirmed a comparable accuracy of TMA-93 to FCSRT in diagnosing early AD in patients with memory complaints and Mini-Mental State Examination (MMSE) scores of 22 or higher out of 30.¹⁴

Normative data for TMA-93 were obtained from a large sample of cognitively unimpaired subjects,¹⁵ with a broad representation of illiterate participants, considering variables that demonstrated a significant influence on its scores (age and educational level). In a subsequent study employing a new, large sample, cognitive reserve, measured by an ad-hoc questionnaire (Cognitive Reserve Questionnaire, CRQ), exhibited a greater influence than the isolated educational level on TMA-93 performance.¹⁶ New normative data, accounting for age and cognitive reserve, revealed wide variations in the 10th percentile, often used as a cutoff point, with scores ranging from 15 out of 30 for older individuals with lower cognitive reserve to 28 out of 30 for younger individuals with higher reserve.¹⁶

To date, normative data for cognitive tests have been calculated in cohorts of healthy controls without considering their biomarker status, which may have impacted the sensitivity of these tests. The use of biomarkers as the ground truth may optimize the selection of cutoff points in such normative data. The objective of this study was to analyze the sensitivity of various TMA-93 cutoff sets, all based on cognitive reserve, and select the most optimal approach for diagnosing early AD in a biobank sample of patients from a tertiary hospital Memory Unit.

Methods

Design

This is a cross-sectional, observational, and retrospective study, focused on evaluating the sensitivity of various approaches of the TMA-93 test for detecting AD pathology within a large biobank sample of biomarker-confirmed AD patients, with the goal of selecting the most optimal approach. Additionally, it aimed to analyze the demographic and clinical characteristics of patients with false negative (FN) outcomes and identify variables that increase the likelihood of such results on the test.

Study population

Between November 2018 and February 2024, patients were attended at the Outpatient Memory Clinic of Virgen del Rocío University Hospital, a tertiary referral center in Southern Spain, and underwent thorough clinical and biomarker assessments. The sample comprised patients who sought consultation for cognitive complaints, where AD was suspected, and a biomarkers study was requested based on the individualized interpretation of the neurological and neuropsychological evaluation in the clinical setting and in accordance with the appropriate recommendations for using biomarkers by Shaw et al. (2018).¹⁷ Patients were selected from the registry of the “Alzheimer and other dementias” collection (C330022) within the Andalusian biobank, using the following inclusion criteria: 1) age 50 years or above, 2) cognitive complaints corroborated by a reliable informant, 3) a MMSE score of 18 or higher, 4) available TMA-93 score from the initial visit, 5) available CRQ total score, 6) a positive result on AD biomarkers, either amyloid position emission tomography (PET) or cerebrospinal fluid (CSF) concentrations of the amyloid β peptide 42 to 40 ratio (Aβ₄₂/Aβ₄₀) and 7) being in a clinical stage of MCI or mild dementia, as defined by a Global Deterioration Scale (GDS) score of 3 or 4, determined with the support of the standard cognitive and functional battery of our center, previously described,¹⁸ and independently of biomarker status.

Cognitive reserve questionnaire

The CRQ, a quick assessment tool taking approximately two minutes to complete, assess cognitive reserve based on a validated eight-item scale for the Spanish population.¹⁹ With a maximum score of 25, it encompasses areas including educational level (0–5 points), educational level of parents (0–2 points), number of training courses taken (0–3 points), professional occupation (0–4 points), musical training (0–2 points), number of spoken languages (0–3 points), reading frequency (0–4 points), and intellectual games playing frequency (0–2 points).

TMA-93

The TMA-93 assessment was conducted as recommended by its authors.¹¹ Initially, participants engaged in an encoding phase by naming and memorizing ten pairs of semantically-related drawings depicting common objects (Figure 1). During the recall phase, performed immediately after, one item from each pair is displayed, requiring participants to recall the missing one. The pair is displayed as a reminder after each answer or after up to 5 s. A maximum score of 30 points is given if the participant identifies all pairs correctly on the first try, eliminating the need for further attempts. Failing this, scores between 0 and 9 are assigned based on the first attempt's accuracy, prompting a second attempt. Achieving full recall in the second attempt yields 20 points (added to those of the first try), covering both the second and an unneeded third attempt. If not, a third attempt is displayed, and its score added to the previous ones.

Figure 1.

Example of TMA-93 sheet. During the encoding phase, semantically related drawings are presented in pairs (left). In the recall phase, the subject is required to recall the missing object (right).

To analyze the sensitivity of the TMA-93 for AD diagnosis, we considered two different sets of normative data. Firstly, we used normative data derived from a traditional norming (TN) approach of our prior study,¹⁶ where TMA-93 percentiles were estimated across 12 normative groups based on age and cognitive reserve, the variables that significantly influenced the test scores. Secondly, we calculated new normative data using a regression-based approach on the same cohort of 902 cognitively unimpaired individuals, also using CRQ total score and age as predictors. Here, age was standardized by subtracting the mean age of the group from each subject's chronological age. The resulting percentiles are detailed in the Supplemental Table, which organizes percentiles in rows and CRQ scores in columns for different ages. In the analysis conducted for the present study, we utilized six distinct cutoff points: the 10th, 15th, and 20th percentiles from both TN and regression-based norming (RBN) methods for the TMA-93 test. These cutoff points were employed to classify patients as either “Normal” or “Altered” according to each approach.

AD biomarkers assessment

Patients had a positive result for AD pathology based on amyloid PET scans or AD CSF biomarkers.

For amyloid-PET, amyloid-β's cortical deposition was evaluated using florbetaben PET scans. Following a 300 MBq florbetaben intravenous injection, PET scans were conducted within 90 to 110 min, employing the cerebellar cortex for SUV calculations. The scanning period lasted 20 min. DGS, a Nuclear Medicine expert, reviewed the images in line with the European Association of Nuclear Medicine recommendations, determining each as either “Positive” (A+) or “Negative” (A-) for presence of amyloid pathology.

For AD diagnoses based on CSF biomarkers, a lumbar puncture was performed at the L3/L4 or L4/L5 levels to obtain CSF. Samples were collected into polypropylene tubes and stored in the IBIS Biobank (C330022 collection), preserved at below −80°C, proceeding only with thawing for the determination, following the international consensus recommendations.²⁰ The CSF Aβ₄₂/Aβ₄₀ and phosphorylated tau 181 (p-tau 181) levels were determined using either Lumipulse or Euroimmun methods, following the manufacturers’ recommendations. AD diagnosis was made depending on the Aβ₄₂/Aβ₄₀ levels (A+), using as threshold 0.069 in the case of Lumipulse and 0.095 in the case of Euroimmun. If p-tau 181 resulted higher than 55.9 pg/ml (Lumipulse) or higher than 61 pg/ml (Euroimmun), the patient was classified as “Tau positive” (T+).

Ethics

Ethical approval for this study was granted by the ethics committee of Virgen del Rocío University Hospital, with all participants providing written informed consent before engagement.

Statistical design

Descriptive results were reported as frequency (percent) for dichotomous and categorical variables, mean (±SD, range) for normally distributed continuous variables, and median (interquartile range, P10-P90, range) for non-normally distributed continuous variables. Normality was assessed using the Shapiro-Wilk test. For between-group comparisons of continuous variables, we conducted a Student's t-test or one-way analysis of variance (ANOVA), or their nonparametric counterparts: the Mann-Whitney U-test and the Kruskal-Wallis ANOVA, respectively. Categorical variables were compared using the chi-squared test. Additionally, to study the association between different continuous variables, Pearson correlation was used for normally distributed data, and Spearman correlation was employed for data not following a normal distribution.

To study the effect of cognitive reserve on the relationship between AD pathology and visual binding capacity, we carried out generalized additive models (GAM), fitted with a smoothing parameter of k = 10, using the z-score of the CSF biomarkers (Aβ₄₂/Aβ₄₀ and p-tau 181) as predictor variables and the log-transformed score of the TMA-93 as the response variable. We assessed this relationship in two groups according to the score below or above the median CRQ score in the total sample, and displayed it using the ggplot 2 package in R.

For the sensitivity analyses, we firstly examined the proportion of patients with an “Altered” result for each of the six cutoff sets tested, considering the total sample, as well as the MCI and mild dementia subgroups. Subsequently, we analyzed the sensitivity of each set within each of the 12 normative groups from the previous work (4 by age and 3 by CRQ score).¹⁶ Based on these findings, we selected the approach that seemed most useful for application in clinical practice. Lastly, we described the characteristics of patients with a FN result in the selected approach, investigating the variables that differed from those in true positive (TP) patients, through group comparisons. Furthermore, using logistic regression on the total sample, we explored which variables were associated with a higher probability of a FN result (response variable: 0= TP, 1= FN).

All analyses were conducted using R version 4.2.2. Statistical significance was set at p < 0.05.

Results

270 AD-positive patients from the biobank database finally met the inclusion criteria (Table 1). All of them were primarily classified as “early AD” stage at diagnosis: 83% with MCI (GDS 3) and 17% with mild dementia (GDS 4). The median age was 73 years and women (54.8% of the total) were significantly more prevalent in the mild dementia group (χ²(1) = 5.6149, p = 0.011). 20.1% of patients had not completed primary studies. There were no differences by age (W = 5588, p = 0.4) or educational level (χ²(1) = 5.9474, p = 0.051) between MCI and mild dementia groups, but mild dementia patients had a lower cognitive reserve, as measured by the CRQ (W = 7021, p < 0.001). The median TMA-93 score in the sample was 17 out of 30 points, and the two cutoff points that delimited the 10% with the worst and best scores (10th and 90th percentiles, respectively) were less than or equal to 4 points and greater than or equal to 27. As expected, both MMSE (W = 7069, p < 0.001) and TMA-93 (W = 6624, p = 0.002) scores were lower in the mild dementia group than in the MCI one.

Table 1.

Demographic and cognitive variables of the total sample and subgroups with mild cognitive impairment and mild dementia.

	Total sample, N = 270	MCI (GDS 3), N = 224	Mild dementia (GDS 4), N = 46	p
Age (y)	73, 68–76, (63–78), [55–84]	73, 68–76, (63–78), [56–83]	72, 68–74, (63–79), [55–84]	0.4
Sex				0.011
Female	148 (54.8%)	115 (51.3%)	33 (71.7%)
Educational level				0.051
Less than primary education	54 (20.1%)	40 (17.9%)	14 (30.4%)
Primary education	87 (32.3%)	70 (31.4%)	17 (37.0%)
More than primary education	128 (47.6%)	113 (50.7%)	15 (32.6%)
CRQ total score	9, 6–14, (3–17), [1–23]	10, 7–15, (4–18), [1–23]	6, 5–10, (3–14), [2–20]	<0.001
MMSE score	24, 22–26, (20–27), [18–29]	24, 22–26, (21–28), [18–29]	22, 20–24, (19–26), [18–28]	<0.001
TMA-93 score	17, 10–23, (4–27), [1–30]	18, 11–24, (6–28), [1–30]	13, 5–19, (3–24), [1–29]	0.002

Median, IQR, (p10-p90), [Range]; N (%).

Group comparisons: Wilcoxon rank sum test; Pearson's Chi-squared test.

MCI: mild cognitive impairment; GDS: Global Deterioration Scale; CRQ: cognitive reserve questionnaire; MMSE: Mini-Mental State Examination.

96 patients were classified as A + in accordance to a positive amyloid-PET scan. The other 174 were diagnosed based on the CSF Aβ₄₂/Aβ₄₀ levels (136 using Euroimmun and 38 Lumipulse). Among the patients classified by CSF biomarkers, 91% were categorized as T + due to its p-tau 181 level.

In Figure 2, the TMA-93 performance in relation to the AD CSF biomarkers (Aβ₄₂/Aβ₄₀ and p-tau 181) levels in patients with CRQ total score below and above the median of 9 points is shown. The curves are more closely superimposed in the case of Aβ₄₂/Aβ₄₀, while a trend toward worse performance on the TMA-93 is observed with higher p-tau levels in patients with lower cognitive reserve.

Figure 2.

Performance of TMA-93 in relation to AD CSF biomarkers (Aβ₄₂/Aβ₄₀ on the left and p-tau 181 on the right) across cognitive reserve groups.

Table 2 shows the sensitivity (proportion of “Altered” results) of each TMA-93 cutoff set in the total sample, and in the MCI and mild dementia groups. As expected, sensitivity in the group of patients with mild dementia was higher than in those with MCI for all cutoff sets. As can be observed, in the total sample, the use of the 15th percentiles improved sensitivity by more than 4 points compared to the 10th percentiles, reaching 80%, while the use of the 20th percentiles caused a smaller increase in sensitivity, from 1 to less than 3 points. Each percentile (10, 15 or 20th) calculated using RBN showed a slightly better sensitivity than its homologous calculated using TN, except in the case of the 20th percentile in the total sample and in the MCI group. Notably, none of the sets reached 90% sensitivity, meaning they all failed to detect AD pathology in the 10% of the sample with the highest scores (27 points or more).

Table 2.

Sensitivity (proportion of “Altered” results) of the 10th, 15th, and 20th percentiles of TMA-93 obtained by traditional norming and regression-based norming for diagnosing AD pathology.

	N	p10 TN	p10 RBN	p15 TN	p15 RBN	p20 TN	p20 RBN
Total sample	270	204 (75.6%)	205 (75.9%)	215 (79.6%)	217 (80.4%)	223 (82.6%)	221 (81.9%)
MCI	224	165 (73.7%)	166 (74.1%)	176 (78.6%)	177 (79.0%)	183 (81.7%)	180 (80.4%)
Mild dementia	46	39 (84.8%)	39 (84.8%)	39 (84.8%)	40 (87.0%)	40 (87.0%)	41 (89.1%)

MCI: mild cognitive impairment; TN: traditional norming; RBN: regression-based norming.

Next, we analyzed the sensitivity of these different TMA-93 cutoff sets in each of the 12 normative groups from the previous work (according to combinations of 4 age ranges and 3 CRQ total score ranges (Table 3).¹⁶ Of note, sensitivity was consistently lower in patients older than 69 years, especially in those over 75 years, with lower cognitive reserve (CRQ score below 7). As was the case in the overall sample, the choice of the 15th percentiles clearly increased the sensitivity of the diagnosis of AD pathology in the majority of normative groups, but the benefit of using the 20th percentiles was more limited. On the other hand, in addition to a greater global sensitivity as mentioned above, the percentiles according to RBN showed a more homogeneous sensitivity throughout the different normative groups.

Table 3.

Sensitivity (proportion of “Altered” results) of the 10th, 15th, and 20th percentiles of TMA-93 obtained by traditional norming and regression-based norming for diagnosing AD pathology in normative groups by age and cognitive reserve.

Normative groups	N	p10 TN	p10 RBN	p15 TN	p15 RBN	p20 TN	p20 RBN
<61 y, CRQ<7	4	3 (75%)	3 (75%)	3 (75%)	3 (75%)	3 (75%)	3 (75%)
<61 y, CRQ 7–11	2	2 (100%)	2 (100%)	2 (100%)	2 (100%)	2 (100%)	2 (100%)
<61 y, CRQ > 11	3	3 (100%)	2 (66.7%)	3 (100%)	3 (100%)	3 (100%)	3 (100%)
61–68 y, CRQ < 7	24	20 (83.3%)	20 (83.3%)	22 (91.7%)	23 (95.8%)	23 (95.8%)	23 (95.8%)
61–68 y, CRQ 7–11	21	16 (76.2%)	16 (76.2%)	18 (85.7%)	17 (81.0%)	20 (95.2%)	19 (90.5%)
61–68 y, CRQ > 11	25	18 (72.0%)	17 (68.0%)	20 (80.0%)	17 (68.0%)	20 (80.0%)	17 (68.0%)
69–75 y, CRQ < 7	31	17 (54.8%)	22 (71.0%)	19 (61.3%)	23 (74.2%)	21 (67.7%)	24 (77.4%)
69–75 y, CRQ 7–11	46	36 (78.3%)	35 (76.1%)	38 (82.6%)	36 (78.3%)	39 (84.8%)	37 (80.4%)
69–75 y, CRQ > 11	35	32 (91.4%)	29 (82.9%)	32 (91.4%)	32 (91.4%)	32 (91.4%)	32 (91.4%)
>75 y, CRQ < 7	18	7 (38.9%)	9 (50.0%)	7 (38.9%)	9 (50.0%)	8 (44.4%)	9 (50.0%)
>75 y, CRQ 7–11	28	23 (82.1%)	23 (82.1%)	23 (82.1%)	24 (85.7%)	24 (85.7%)	24 (85.7%)
>75 y, CRQ>11	33	27 (81.8%)	27 (81.8%)	28 (84.8%)	28 (84.8%)	28 (84.8%)	28 (84.8%)

TN: traditional norming; RBN: regression-based norming; CRQ: Cognitive Reserve Questionnaire.

Finally, taking into account the 15th percentile cutoff set according to RBN, we closely examined the FN (individuals with AD-positive biomarkers yet classified as “Normal” by the TMA-93 test) in comparison to the TP (who showed “Altered” test results; Table 4). With this set, all FN (53 out of 270, 19.6%) scored above 23 on the TMA-93, with 5.9% scoring between 23 and 26, and 13.7% scoring 27 or higher (Figure 3A). Age (W = 5548, p = 0.6913), sex (χ²(1) = 0.56808, p = 0.451), and educational level (χ²(2) = 3.0756, p = 0.0.2149) did not differ significantly between the TP and FN groups. Interestingly, the MMSE scores were significantly elevated in the FN group (W = 3377.5, p < 0.01; Figure 3B). Indeed, in the total sample, there was a positive and moderate correlation between the TMA-93 and MMSE scores (rho = 0.39). The proportion of patients in the MCI clinical stage (GDS 3) was slightly higher in the FN group (88.7% versus 81.6%), though not significant (χ²(1) = 1.0628, p = 0.3026). A tendency towards lower cognitive reserve was observed within the FN group (CRQ total score of 8 versus 10), yet this did not achieve statistical significance (W = 5997.5, p = 0.6278; Figure 3C). Nonetheless, significant differences in cognitive reserve became evident among participants older than 70 years (W = 3342, p = 0.02095; Figure 3D). In the logistic regression analysis focusing on the likelihood of a FN outcome in the total sample, only higher MMSE scores significantly predicted the FN classification (B = 0.35, p < 0.001), with age (B = −0.0047, p = 0.87), sex (B = −0.58, p = 0.09), and cognitive reserve (B = −0.043, p = 0.20) showing no significant effect. However, for individuals aged over 70, both higher MMSE scores (B = 0.31, p < 0.001) and lower CRQ scores (B = −0.10, p = 0.018) significantly increased the risk of a FN outcome.

Figure 3.

Distribution of scores in true positives (TP) and false negatives (FN) of TMA-93: (a) TMA-93 score, (b) MMSE score, (c) CRQ score, and (d) CRQ score in individuals over 70 years. Statistical significance between groups is represented by asterisks: p < 0.05 (*), p < 0.01 (**), p < 0.001 (***), p < 0.0001 (****), and “ns” for non-significant results.

Table 4.

Characteristics of true positives and false negatives at the 15th percentile of TMA-93 obtained by regression-based norming.

	TP, N = 217	FN, N = 53	p
Age (y)	72, 68–76, [55–84]	73, 68–76, [59–83]	0.7
Sex			0.4
Female	116 (53.5%)	32 (60.4%)
Educational level			0.2
Less than primary education	43 (19.9%)	11 (20.8%)
Primary education	65 (30.1%)	22 (44.5%)
More than primary education	108 (50.0%)	20 (37.7%)
CRQ total score	10, 6–14, [1–23]	8, 6–14, [3–21]	0.6
MMSE score	23, 22–25, [18–29]	26, 24–27, [20–29]	<0.001
TMA-93 score	14, 9–19, [1–26]	27, 26–29, [23–30]	<0.001
GDS			0.2
GDS 3 (MCI)	177 (81.6%)	47 (88.7%)
GDS 4 (Mild dementia)	40 (18.4%)	6 (11.3%)

Median, IQR, [Range]; N (%).

Group comparisons: Wilcoxon rank sum test; Pearsońs Chi-squared test.

TP: true positives; FN: false negatives; CRQ: cognitive reserve questionnaire; MMSE: Mini-Mental State Examination; GDS: Global Deterioration Scale; MCI: mild cognitive impairment.

Discussion

This study represents, to our knowledge, the first evaluation of a cognitive test—specifically the TMA-93—with normative data established based on cognitive reserve, rather than solely on educational level, to assess sensitivity for detecting AD pathology using specific biomarkers. Utilizing the 15th percentile obtained through RBN demonstrated a significant increase in sensitivity compared to the traditionally employed 10th percentile, along with subtler variations across different normative groups, relative to the cutoffs obtained through TN. Notably, a higher MMSE score and, among patients over 70 years of age, a lower cognitive reserve, were both associated with a higher risk of receiving a FN result in the test (a ‘Normal’ result despite AD pathology confirmed by biomarkers).

In the present work, we analyzed a large sample of AD patients identified through biomarker status, with a significantly higher representation of patients in the prodromal stage compared to those with mild dementia. Such distribution is expected in clinical settings, since milder the cognitive impairment is, greater may be the diagnostic uncertainty, prompting an increased use of biomarkers. Moreover, as anticipated, the TMA-93 test showed increased sensitivity in the mild dementia subgroup across all sets analyzed.

Cognitive reserve is a concept that is garnering increasing interest and has been incorporated into current models of AD.^{1, 21} Based on this framework, patients with higher cognitive reserve can cope with the pathology for a longer period and exhibit a delayed onset of symptoms.²² Intriguingly, once symptoms begin, those with higher reserve experience a more rapid decline due to more advanced stages of pathology at symptom onset compared to those with lower reserve.²² According to the most recent operational definition,²³ cognitive reserve should be elucidated by a variable (CRQ score) that moderates the relationship between a measure of brain pathology (AD CSF biomarkers) and one of cognitive change (TMA-93 score). The demographic composition of our sample, predominantly comprising participants with low cognitive reserve (median CRQ score of 9 out of 25, with 75% scoring below 14), coupled with the utilization of cross-sectional CSF biomarker measures exclusively during the symptomatic stages of the AD continuum, may limit the ability to discern certain associations. Yet, two points warrant consideration. First, within this large cohort of AD-positive patients, no differences were observed in age or educational level between the MCI and mild dementia subgroups. However, the latter group exhibited lower cognitive reserve, supporting the idea that those with less reserve, beyond their educational level, have a reduced tolerance to AD pathology. Second, among patients with cognitive reserve below the median, a trend toward poorer performance on the TMA-93 was noted in relation with higher levels of tau pathology (i.e., elevated CSF p-tau 181 levels) compared to those with greater reserve. For amyloid pathology (CSF Aβ₄₂/Aβ₄₀ levels), we did not observe a clear effect, consistent with previous evidence demonstrating a more substantial impact on cognition of tau than amyloid pathology.^{24, 25}

In this sample of 270 AD-positive patients, we compared the sensitivity (ability to classify as “Altered”) of three cutoffs points (10th, 15th, and 20th percentiles) of the TMA-93, utilizing normative data derived from both TN and RBN. On one hand, in general, cutoffs calculated by RBN demonstrated slightly higher sensitivity compared to those using TN, with the exception of the 20th percentile in both the total sample and the MCI subgroup. More significantly, when analyzing the sensitivity in each of the normative groups from the previous work according to age and cognitive reserve,¹⁶ the use of cutoff sets by RBN facilitated a more uniform distribution of sensitivity across different strata. Indeed, the use of RBN instead of TN has previously shown to provide a smoother and less staggered distribution of the data, in addition to other advantages such as reduced sample size requirements and the non-categorization of continuous predictors like age.^{26, 27} On the other hand, as expected, employing higher percentiles enhanced sensitivity to detect AD pathology, with the increase from the 10th to the 15th percentile being particularly significant (exceeding a 4-point increase and surpassing 80%) compared to the increment from the 15th to the 20th. In this sense, it is important to note that historically, normative data for cognitive tests have been derived from populations of cognitively unimpaired individuals without consideration of biomarker status. In recent years, some studies have confirmed that using normative data obtained from populations with negative biomarker results yields higher sensitivity due to more stringent thresholds,^{28, 29} and it is anticipated that the adoption of biomarker-negative norms will become standard practice in near future. Our findings support that utilizing the 15th percentile as a cutoff in normative data not based on biomarkers enhances sensitivity and appears more suitable than the traditionally used 5th or 10th percentiles. Consequently, we propose the 15th percentile, obtained through RBN, as the optimal cutoff for clinical application of the TMA-93.

In our study, the TMA-93 achieved a sensitivity of 80.4% when using this 15th percentile threshold determined by RBN. Importantly, this level of sensitivity was observed in patients with early AD. Traditionally, the sensitivity of cognitive tests has been evaluated in patients diagnosed with MCI or dementia based on clinical criteria, rather than biomarker status. Therefore, achieving such sensitivity, considering biomarkers as the ground truth and the early stage of the disease in the patients, could indeed be deemed satisfactory. This sensitivity greater than 80%, demonstrated in the current sample and another distinct, smaller sample of AD patients using different normative data,¹⁴ is particularly noteworthy for the TMA-93, taking into account that the test offers the advantage of being both time-efficient and suitable for illiterate patients.¹² Of note, relational binding (e.g., Memory Binding Test)^{30, 31} refers to the association between an object and its semantic category or context, apparently relying on the hippocampus, while conjunctive binding (e.g., Short-Term Memory Binding Test)⁸ involves the integration of an object's features (e.g., color, shape) into a single representation, apparently supported by the entorhinal and perirhinal cortex and occipitoparietal regions.¹⁰ Remarkably, previous studies have shown that relational binding experiences a more pronounced age-related decline than conjunctive binding,^{32, 33} or exhibits a greater susceptibility to interference as age increases.³⁴ This suggests that tests assessing relational binding might be less accurate than those evaluating conjunctive binding in detecting early cognitive changes in AD. Although the TMA-93 is primarily a relational binding test, it evaluates a combination of semantic and memory functions, and we hypothesize that this combined assessment could account for its sensitivity. Indeed, research has shown that along with memory deficits, some degree of semantic impairment is also present in both the prodromal³⁵ and preclinical stages of AD.^{36, 37} Consequently, the use of diagnostic tools that assess both memory and semantic functions could significantly improve sensitivity in detecting early cognitive changes of the disease. Further studies incorporating biomarkers are essential to explore the sensitivity of the different tests examining binding in the preclinical stage of AD, where it is anticipated that tests assessing conjunctive binding will surpass those assessing relational binding.

Besides using biomarkers as the gold standard, several factors may have influenced the sensitivity of the test and deserve consideration. In this study, we conducted a comprehensive analysis of individuals with a FN result using the 15th percentile cutoff of TMA-93 obtained by RBN. No significant differences in age, sex, or educational level were found between the FN and TP groups, though the proportion of prodromal patients (GDS 3) was slightly higher in the FN group, as would be presumed. By contrast, both logistic regression and group comparisons revealed that two variables had a significant impact. Firstly, a higher MMSE score in the total sample was associated with a greater likelihood of a FN result, indicating that individuals with higher global cognition exhibit less memory and/or semantic impairment and, therefore, more preserved visual binding ability, as expected. Secondly, among individuals over 70, a higher MMSE score combined with lower cognitive reserve, measured by the CRQ, was linked to an increased risk of FN outcomes. The influence of cognitive reserve requires deeper examination. In our previous work,¹⁶ we observed a moderate, positive correlation between CRQ scores and TMA-93 performance (Rho = 0.34) and found that the model incorporating cognitive reserve explained a greater percentage of TMA-93 score variability than the model based solely on educational level. Additionally, in this cognitively unimpaired population, test scores practically overlapped across different cognitive reserve groups until the age of 70, but beyond this age significant differences emerged, as individuals with lower cognitive reserves exhibited a more pronounced age-related decline in TMA-93 scores. We hypothesize that older patients with lower cognitive reserve may be subject to a less stringent threshold, facilitating their classification as ‘Normal.’ Consistent with this hypothesis, our prior normative study showed that the gap in the 10th percentile of TMA-93 scores between the lowest and highest reserve groups widened with age—from 4 points in those under 61 to 9 points in those over 75. Another factor to consider is that in a sample drawn from real clinical practice like this, there might be a bias toward requesting biomarkers for patients with persistent cognitive complaints despite normal results on practical screening tests like the TMA-93, which could disproportionately affect certain groups, such as older individuals with lower cognitive reserve. Moreover, our sample was heterogeneous, consisting of patients from real clinical practice with cognitive complaints and a biological diagnosis of AD, likely including non-amnestic phenotypes such as posterior cortical atrophy or logopenic primary progressive aphasia, where memory may be relatively preserved. Lastly, no participant scoring below 23 was classified as FN. In contrast, all patients scoring 27 or above invariably received an FN result. These latter patients, representing the top 10% of the sample based on their TMA-93 scores, were not correctly classified as ‘Altered’ by any set, as none reached 90% sensitivity. They likely constitute the upper half of the FN group, potentially representing individuals with higher cognitive functioning for whom the test may not be challenging enough to detect an impairment. We suggest that those variables (a higher MMSE score and, for those over 70, a lower cognitive reserve) associated with a higher risk of an FN outcome might predominantly affect the lower half of the FN group (those scoring between 23 and 26).

As stated, one of the primary advantages of the TMA-93 is its proven feasibility and reliability in patients with a low educational level,¹² often encountered in healthcare settings with limited access to some resources, such as comprehensive neuropsychological evaluations or specific AD biomarkers. In such contexts, the TMA-93 could serve as a valuable screening tool for identifying patients who require more detailed assessments. Based on the discussion above, we propose the following recommendations for using the TMA-93 in clinical practice when attending to patients with cognitive complaints where there is a high suspicion of AD as the underlying etiology:

- Score below 23 out of 30 points: directly request AD biomarkers.

- Score of 27 out of 30 points or higher: conduct a more demanding memory test. Some possible alternatives include the Free and Cued Selective Reminding Test (FCSRT)³⁸ for verbal memory assessment; tests involving semantic interference, such as the Loewenstein-Acevedo Scale for Semantic Interference and Learning (LASSI-L)³⁹ or the Memory Binding Test (MBT);^{30, 31} or tests assessing conjunctive binding, such as the Short Term Memory Binding Test,⁸ which has shown higher sensitivity. The latter, utilizing non-verbal material, may be particularly useful for patients with lower educational levels.

- Score between 23 to 26 points: Utilize the 15th percentile cutoff of TMA-93 established by RBN:

If the patient is classified as ‘Altered': request AD biomarkers.

If classified as ‘Normal': conduct a more demanding memory test.

One of the strengths of the present study is its substantial sample size, which was derived from a biobank collection. This sample specifically included patients who presented with cognitive complaints in a real clinical setting and received a biomarker-based diagnosis of AD, thereby utilizing biomarkers as a definitive ground truth to estimate the sensitivity of various TMA-93 test sets—another significant strength of this research. Moreover, the study featured a wide representation of individuals with a low educational level and, notably, of those with low cognitive reserve, often underrepresented in biomarker cohorts. Conversely, there are some limitations that must be considered. Firstly, in the current study, we did not include subjects without AD pathology and, consequently, did not examine the specificity of the test. This resulted from the absence of specific biomarkers for other pathologies, such as limbic-predominant age-related TDP-43 encephalopathy (LATE),⁴⁰ which could potentially impact the memory of AD-negative subjects, whose impairments might be accurately detected by the test. In fact, while the test demonstrated high specificity for diagnosing patients with amnestic MCI against healthy controls,¹³ we anticipate lower specificity for detecting AD pathology when considering biomarkers, as evidenced by the previous validation study.¹⁴ That study reported specificity rates of 41% and 29% at the 5th and 10th percentiles, respectively, based on normative data that considered the educational level. In essence, the present study focuses on analyzing the sensitivity of the TMA-93 for the diagnosis of early AD. In the future, improved characterization of the amyloid-negative group using biomarkers will enable sensitivity studies to be extended to these other entities. Secondly, the use of cross-sectional data precludes establishing causality between observed relationships, such as those between CSF biomarkers and TMA-93 performance across different cognitive reserve groups. Additionally, some normative groups from our previous work contained very few participants in the current sample, particularly those under 61 years of age, limiting the ability to draw precise conclusions about sensitivity within this age range. Finally, the construct of cognitive reserve may not be fully captured by the CRQ, as it does not account for some aspects as socialization or using of technology that could impact cognitive reserve. Furthermore, while validated in a Spanish population, the questionnaire may not accurately measure cognitive reserve in other populations.

Conclusions

Using biomarkers as ground truth, the 15th percentile cutoff obtained through RBN of the TMA-93 appears to be the most optimal approach. This cutoff demonstrates greater sensitivity and more uniform distribution across normative groups compared to the 5th or 10th percentiles, or when using thresholds calculated via TN. Notably, the risk of a FN result may be higher in patients with elevated MMSE scores and in those over 70 years of age with low cognitive reserve. Based on these findings, we provide recommendations for using the TMA-93 in clinical practice to help identify which patients presenting with cognitive complaints in resource-limited settings may benefit from a more in-depth evaluation for suspected AD.

Supplemental Material

sj-xlsx-1-alz-10.1177_13872877251325759 - Supplemental material for TMA-93 (binding by images): Cutoffs optimization based on Alzheimer's disease biomarkers

Supplemental material, sj-xlsx-1-alz-10.1177_13872877251325759 for TMA-93 (binding by images): Cutoffs optimization based on Alzheimer's disease biomarkers by José Enrique Arriola-Infante, Ernesto García-Roldán, David García-Solís, Alba Marta Marín-Cabañas, Andrea Luque-Tirado, Ángela Almodóvar-Sierra, María Bernal Sánchez-Arjona, Didier Maillet and Emilio Franco-Macías in Journal of Alzheimer's Disease

Footnotes

Acknowledgments

We are grateful to all the patients and caregivers for their participation in this research. Maillet is the author of the TMA-93.

ORCID iDs

José Enrique Arriola-Infante

Ernesto García-Roldán

David García-Solís

Alba Marta Marín-Cabañas

Andrea Luque-Tirado

Ángela Almodóvar-Sierra

Didier Maillet

Emilio Franco-Macías

Author contributions

José Enrique Arriola Infante (Conceptualization; Data curation; Formal analysis; Methodology; Project administration; Writing – original draft); Ernesto García-Roldán (Conceptualization; Data curation; Methodology; Writing – review & editing); David García-Solís (Data curation; Writing – review & editing); Alba Marta Marín-Cabañas (Data curation; Writing – review & editing); Andrea Luque-Tirado (Data curation; Writing – review & editing); Ángela Almodóvar-Sierra (Data curation; Writing – review & editing); María Bernal Sánchez-Arjona (Data curation; Writing – review & editing); Didier Maillet (Writing – review & editing); Emilio Franco-Macías (Conceptualization; Data curation; Formal analysis; Methodology; Project administration; Supervision; Writing – original draft).

Ethical considerations

Not applicable

Consent to participate

Not applicable

Consent for publication

Not applicable

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: JEAI and EGR are recipients of the Río Hortega Grant from the Carlos III Health Institute of the Spanish Government.

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Emilio Franco-Macías is an Editorial Board Member of this journal but was not involved in the peer-review process of this article nor had access to any information regarding its peer-review.

The remaining authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data availability

The data supporting the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.

Supplemental material

Supplemental material for this article is available online.

References

Jack

Jr Andrews

Beach

, et al. Revised criteria for diagnosis and staging of Alzheimer's disease: Alzheimer's Association Workgroup. Alzheimers Dement 2024; 20: 5143–5169.

Dubois

Villain

Schneider

, et al. Alzheimer disease as a clinical-biological construct-an international working group recommendation JAMA Neurol 2024; 81: 1304–1311.

Julayanont

Ruthirago

. The illiterate brain and the neuropsychological assessment: from the past knowledge to the future new instruments. Appl Neuropsychol Adult 2018; 25: 174–187.

Mandyla

Yannakoulia

Hadjigeorgiou

, et al. Identifying appropriate neuropsychological tests for uneducated/illiterate older individuals. J Int Neuropsychol Soc 2022; 28: 862–875.

Cummings

Zhou

Lee

, et al. Alzheimer's disease drug development pipeline: 2023. Alzheimers Dement 2023; 9: e12385.

Costa

Bak

Caffarra

, et al. The need for harmonisation and innovation of neuropsychological assessment in neurodegenerative dementias in Europe: consensus document of the Joint Program for Neurodegenerative Diseases Working Group. Alzheimers Res Ther 2017; 9: 27.

Mayes

Montaldi

Migo

. Associative memory and the medial temporal lobes. Trends Cogn Sci 2007; 11: 126–135.

Parra

Abrahams

Fabi

, et al. Short-term memory binding deficits in Alzheimer’s disease. Brain 2009; 132: 1057–1066.

Parra

Abrahams

Logie

, et al. Visual short-term memory binding deficits in familial Alzheimer's disease. Brain 2010; 133: 2702–2713.

10.

Pavisic

Suarez-Gonzalez

Pertzov

. Translating visual short-term memory binding tasks to clinical practice: from theory to practice. Front Neurol 2020; 11: 458.

11.

Maillet

Narme

Amieva

, et al. The TMA-93: a new memory test for Alzheimer’s disease in illiterate and less educated people. Am J Alzheimers Dis Other Dement 2017; 32: 461–467.

12.

Franco-Macías

Rodrigo-Herrero

Luque-Tirado

, et al. Reliability and feasibility of the memory associative test TMA-93. J Alzheimers Dis Rep 2020; 4: 431–440.

13.

Rodrigo-Herrero

Carnero-Pardo

Méndez-Barrio

, et al. TMA-93 for diagnosing amnestic mild cognitive impairment: a comparison with the Free and Cued Selective Reminding Test. Am J Alzheimers Dis Other Dement 2019; 34: 322–328.

14.

Rodrigo-Herrero

Luque-Tirado

Méndez-Barrio

, et al. TMA-93 validation by Alzheimer’s disease biomarkers: a comparison with the Free and Cued Selective Reminding Test on a biobank sample. J Alzheimers Dis 2021; 82: 401–410.

15.

Rodrigo-Herrero

Sánchez-Benavides

Ainz-Gómez

, et al. Norms for testing visual binding using the memory associative test (TMA-93) in older educationally-diverse adults. J Alzheimers Dis 2020; 75: 871–878.

16.

Arriola-Infante

García-Roldán

Montiel-Herrera

, et al. Using cognitive reserve to create norms for the TMA-93 (relational binding of images). J Alzheimers Dis 2023; 95: 119–129.

17.

Shaw

Arias

Blennow

, et al. Appropriate use criteria for lumbar puncture and cerebrospinal fluid testing in the diagnosis of Alzheimer’s disease. Alzheimers Dement 2018; 14: 1505–1521.

18.

Méndez-Barrio

Medina-Rodríguez

Mendoza-Vázquez

, et al. Prodromal Alzheimer's disease: global cognition, cue efficiency, and cerebrospinal fluid neurofilament light values predict short-term conversion to dementia. J Alzheimers Dis 2024; 101: 877–887.

19.

Rami

Valls-Pedret

Bartres-Faz

, et al. Cuestionario de reserva cognitiva. Valores obtenidos en población anciana sana y con enfermedad de Alzheimer. Rev Neurol 2011; 52: 195–201.

20.

Hampel

Frank

Broich

, et al. Biomarkers for Alzheimer’s disease: academic, industry and regulatory perspectives. Nat Rev Drug Discov 2010; 9: 560–574.

21.

Jack

Knopman

Jagust

, et al. Tracking pathophysiological processes in Alzheimer’s disease: an updated hypothetical model of dynamic biomarkers. Lancet Neurol 2013; 12: 207–216.

22.

Stern

. Cognitive reserve in ageing and Alzheimeŕs disease. Lancet Neurol 2012; 11: 1006–1012.

23.

Stern

Albert

Barnes

, et al. A framework for concepts of reserve and resilience in aging. Neurobiol Aging 2023; 124: 100–103.

24.

Ossenkoppele

Smith

Ohlsson

, et al. Associations between tau, Aβ, and cortical thickness with cognition in Alzheimer disease. Neurology 2019; 92: e601–e612.

25.

Hanseeuw

Betensky

Jacobs

HIL

, et al. Association of amyloid and tau with cognition in preclinical Alzheimer disease: a longitudinal study. JAMA Neurol 2019; 76: 915–924.

26.

Oosterhuis

HEM

van der Ark

Sijtsma

. Sample size requirements for traditional and regression-based norms. Assessment 2016; 23: 191–202.

27.

Lenhard

. Improvement of norm score quality via regression-based continuous norming. Educ Psychol Meas 2021; 81: 229–261.

28.

Bos

Vos

SJB

Jansen

, et al. Amyloid-β, tau, and cognition in cognitively normal older individuals: examining the necessity to adjust for biomarker status in normative data. Front Aging Neurosci 2018; 10: 193.

29.

Brugulat-Serrat

Cañas-Martínez

Canals-Gispert

, et al. Enhancing the sensitivity of memory tests: reference data for the Free and Cued Selective Reminding Test and the Logical Memory Task from cognitively healthy subjects with normal Alzheimer's disease cerebrospinal fluid biomarker levels. J Alzheimers Dis 2021; 84: 119–128.

30.

Buschke

. Rationale of the memory binding test. In: Nilsson

Ohta

(eds) Dementia and memory. Hove, East Sussex: Psychology Press, 2014, pp.55–71.

31.

Gramunt

Sánchez-Benavides

Buschke

, et al. Psychometric properties of the memory binding test: test-retest reliability and convergent validity. J Alzheimers Dis 2016; 50: 999–1010.

32.

Kirmsse

Zimmer

Ecker

UKH

. Age-related changes in working memory: age affects relational but not conjunctive feature binding. Psychol Aging 2018; 33: 512–526.

33.

Bastin

. Differential age-related effects on conjunctive and relational visual short-term memory binding. Memory 2018; 26: 1181–1190.

34.

Van Geldorp

Parra

Kessels

RPC

. Cognitive and neuropsychological underpinnings of relational and conjunctive working memory binding across age. Memory 2015; 23: 1112–1122.

35.

Marra

Piccininni

Masone Iacobucci

, et al. Semantic memory as an early cognitive marker of Alzheimer's disease: role of category and phonological verbal fluency tasks. J Alzheimers Dis 2021; 81: 619–627.

36.

Papp

Rentz

Orlovsky

, et al. Optimizing the preclinical Alzheimer's cognitive composite with semantic processing: the PACC5. Alzheimers Dement 2017; 3: 668–677.

37.

Nation

. Neuropsychological profiles and trajectories in preclinical Alzheimer's disease. J Int Neuropsychol Soc 2018; 24: 693–702.

38.

Buschke

. Cued recall in amnesia. J Clin Neuropsychol 1984; 6: 433–440.

39.

Loewenstein

Curiel

Greig

, et al. A novel cognitive stress test for the detection of preclinical Alzheimer disease: discriminative properties and relation to amyloid load. Am J Geriatr Psychiatry 2016; 24: 804–813.

40.

Nelson

Dickson

Trojanowski

, et al. Limbic-predominant age-related TDP-43 encephalopathy (LATE): consensus working group report. Brain 2019; 142: 1503–1527.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.09 MB