TMA-93 for Diagnosing Amnestic Mild Cognitive Impairment: A Comparison With the Free and Cued Selective Reminding Test

Abstract

Background:

TMA-93 examines binding by images, an advantage for the less educated individuals.

Aim:

To compare the discriminative validity of TMA-93 against the picture version of Free and Cued Selective Reminding Test (FCSRT) to distinguish patients with amnestic mild cognitive impairment (aMCI) from normal controls (NCs) without excluding less educated individuals.

Methods:

Design:

Phase I diagnostic evaluation study.

Participants:

A total of 30 patients with aMCI and 30 NCs matched for sociodemographics variables.

Statistical Analysis:

The diagnostic accuracy for each test was calculated by conducting receiver operating characteristic curve analysis. Hanley and McNeil method was used to compare diagnostic accuracy of different tests on the same sample.

Results:

Up to 41.7% of the sample had less than a first grade of education. Both tests showed excellent diagnostic accuracy. The comparisons did not show significant differences.

Conclusions:

TMA-93 is so accurate as FCSRT to differentiate aMCI from controls including less educated individuals. The test could be considered as a choice in this sociodemographic context.

Keywords

TMA-93 FCSRT binding amnestic MCI diagnostic accuracy ROC curves

Introduction

Diagnosis of Alzheimer’s disease (AD) in its prodromal phase is a global aim in dementia research. This diagnosis is currently based on memory tests and pathophysiological markers.¹

When patients have a low level of education or are emigrants from countries with different languages, the usual neuropsychological assessment procedures are not possible or reliable. Classical neuropsychological tests often require abilities that have been acquired at school such as reading and writing and are not useful for the less educated individuals. These patients are more difficult to diagnose and are usually excluded from clinical trials. To overcome this limitation, tests that use pictorial material to evaluate memory could be administered. Between these tests are the picture version of the “Free and Cued Selective Reminding Test” (FCSRT)^2,3 and the more recent “Memory Associative Test of the District of Seine-Saint-Denis” (TMA-93).⁴

Free and Cued Selective Reminding Test is a classical test based on testing the cueing.² This test controls for a successful encoding (achieved by cued recall) and it facilitates retrieval processing with the same semantic cues.³ A low total recall performance, despite retrieval facilitation with cueing, has an excellent specificity for AD.⁵ Dual coding gives advantage to the picture version of the test against the word one.⁶ This picture version may be more adequate for patients with a low level of education. There are different ways to administer the test. In one of them,³ the study phase is followed by 3 recall trials and the usual 30-minute delayed recall is deleted. This deletion improves applicability without losing the more relevant information that is obtained from the 3 recall trials.³

TMA-93 is based on testing the binding, the ability to form associations.⁴ Binding refers to representational elements in memory that can be recalled together in a unitized way when a specific episode or fact is retrieved.^7,8 Experiments that test binding support the hypothesis that the deficit in binding information in patients with AD is more pronounced than their memory impairment for unrelated information.⁸ In animal literature,⁹ computational models,¹⁰ neuroimaging studies,¹¹ and neuropsychological literature,¹² it is posited that associative learning, or relational association, is a low mechanism which is dependent on functional integration in the medial temporal lobe structures, particularly the hippocampus.⁷

On neuropsychological evaluation, binding has been examined in different ways. The Wechsler Memory Scale evaluates it by learning and recalling of verbal paired associates.¹³ The subset distinguishes easy (eg, north/south) from difficult (eg, school/cellar) associations.¹³ The more recent Memory Binding Test (MBT) examines binding by recalling paired items (eg, flea/ant) from each shared category (eg, insect) from 2 different lists.¹⁴ However, studying binding by images could be more suitable for patients with a low level of education. This is exactly what has been developed by the TMA-93.⁴ Drawings of familiar objects of everyday life are displayed in semantically related pairs during the encoding phase (Figure 1A). Then participants have to recall the missing drawing when the associated one is provided (Figure 1B). This retrieval phase is repeated 3 times successively to assess participants’ learning abilities. The test has shown diagnostic accuracy to differentiate patients with AD from normal controls (NCs) in a less educated and culturally diverse population.⁴

Figure 1.

A, A semantically related pair of drawings (a tree and a bird). B, The bird is missing. It is the drawing the participant has to recall.

The aim of this study was to compare the diagnostic accuracy of the recently described TMA-93 against the classical FCSRT to differentiate patients with amnestic mild cognitive impairment (aMCI) from NCs in a sample that does not exclude participants with low educational levels.

Materials and Methods

Design

This study was designed following the guidelines for a preliminary evaluation of a diagnostic test.¹⁵ A cross-sectional, case–control study with convenience sampling and pretest prevalence of 50% was planned. The aim was to compare on the same sample the discriminative validity of 2 picture memory tests (TMA-93 vs FCSRT) to distinguish individuals with aMCI from NCs.

Study Population

The sample consisted of 60 participants from an urban area of Spain. They comprised 2 groups: 30 patients with aMCI and 30 NCs matched for age, gender, and educational level. All participants were older than 60 years and spoke Spanish as their native language. The following sociodemographic variables were considered: age, gender, educational attainment (less than first grade, first grade, and more than first grade), and literacy (illiterate, able to read and write in Spanish but not fluently, and able to read and write fluently in Spanish).

All patients were selected by convenience sampling of consecutive cases who had been diagnosed of aMCI at the Memory Unit of the Hospital Virgen del Rocio (Seville, Spain). The procedures had consisted of general, neurological, neuropsychological, laboratory, and neuroimaging examinations. Neuropsychological evaluation had included “Spanish Version of the Informant-Based AD8 Questionnaire,”^16,17 “Phototest,”^18,19 “Delayed Matching-to-Sample Task 48” (DMS-48),^20,21 “Geriatric Depression Scale 15 items,”²² and “Interview for Deterioration in Daily Living Activities in Dementia” (IDDD).²³ The diagnosis of aMCI had been made according to the National Institute on Aging and Alzheimer's Association (NIA-AA) recommendations²⁴ and operationally put into practice as follows: (a) memory complaint corroborated by a reliable informant, (b) objective memory impairment measured by a score equal to or below the 10th percentile on set 2 of DMS-48 (this score being lower than that on set 1), and (c) no significant functional decline for activities of daily living (score up to 39 on IDDD was allowed).

Normal controls were recruited among the caregivers and relatives of patients attending the center. They met the following inclusion criteria: (a) absence of memory complaints, (b) absence of objective memory impairment (DMS-48 set 2 score equal to or above the 25th percentile), and (c) intact level of independence in activities of daily living (score between 33 and 36 on IDDD).

The following exclusion criteria were considered for both groups: (1) absence of reliable informant, (2) current history of other neurological diseases that potentially cause cognitive impairment, (3) poor vision or hearing despite correction, (4) clinically significant, advanced, or unstable systemic disease that might interfere with cognitive evaluation, (5) current Diagnostic and Statistical Manual of Mental Disorders, 5th Edition (DSM-V), diagnosis of active major depression, schizophrenia, or bipolar disorder, and (6) history of abusing alcohol or other substances.

Procedures

Both tests were administered on different days and by different raters blinded to participants’ diagnostic group and results of their cognitive testing. TMA-93 was administered following the instructions given by the authors.⁴ During the encoding phase, participants were asked to orally name the 10 paired drawings successively and were explicitly told to memorize them (Figure 1A). After this encoding phase, the first cued recall was administered: For each pair of associates, only 1 drawing was displayed and participants had to recall the missing associated drawing (Figure 1B). Following the participant’s response, regardless of its accuracy after a 5-second delay, the previously encoded paired drawing was presented again. This procedure was repeated for the 9 other paired associates. If participants did not accurately recall all of the 10 paired associates during the first cued recall (ie, score <10), a second and, in case of obtaining again a score lower than 10, a third cued recall were administered, following the same procedure, resulting in a total score of maximum 30 points (by summing the number of correct responses on 3 recall trials). On the contrary, when participants obtained the maximum score of 10 after the first or the second cued recall, the procedure was stopped and 20 points (or the remaining 10 points) were automatically credited for a total score of 30. Several kinds of errors were distinguished: (i) “errors,” all responses corresponding to an object that belonged to 1 of the 9 other paired associates, (ii) “intrusions,” responses that did not belong to the 10 paired associates, and (iii) “perseverations,” referred to repeatedly produced errors during the whole procedure.

Free and Cued Selective Reminding Test was administered following the instructions given by the author.³ It begins with a search procedure in which participants are asked to examine a card containing line drawings of easily recognizable objects for an item that goes with a unique category cue. The 16 items to be learned were presented 4 at a time on a card, 1 picture in each quadrant. The participant was asked to search each card and point to and name aloud each item after its cue was aurally presented. After 4 items were identified correctly, the card was removed, and immediate cued recall of just those 4 items was tested by presenting the cues again. The participant was reminded of any item he or she failed to retrieve by presenting the cue and the item together. Once immediate recall for a group of 4 items was completed, the next set of items was presented for studying. The study phase was followed by 3 recall trials, each one preceded by 20 seconds of participants counting backward to obtain recall from long-term memory. Each recall trial consists of 2 parts. First, each participant had up to 2 minutes to freely recall as many items as possible. Next, verbally presented category cues were provided for items not retrieved by free recall. If participants failed to retrieve the item with the category cue, they were reminded by presenting the cue and the item together. The following variables were scored: total free recall (TFR; the sum of free recall from 3 memory trials), total recall (TR; the sum of free and cued recall from 3 memory trials), and Index of Sensitivity of Cueing (ISC), which was determined by the score of (TFR − TR)/(TFR − 48).

Ethics

This study was approved by the ethics committee of the Hospital Virgen del Rocio (Seville, Spain) and conducted according to the World Medical Association Declaration of Helsinki. All participants accepted the study procedures by signing an informed consent.

Statistical Study

Comparative analysis between groups included independent sample t tests and χ² tests, depending on the variables. Descriptive results were shown as frequencies (percentage) or means (±standard deviation) for categorical or continuous variables, respectively. The diagnostic accuracy of the TMA-93 and the FCSRT was estimated by the area under curve (AUC) using receiver operating characteristic (ROC) curve analysis. The diagnostic accuracy was classified as excellent (>0.9), good (>0.8), fair (>0.7), or poor (>0.6).²⁵ The Youden index was used to determine the optimum cutoff scores to provide the best balance between sensitivity and specificity.²⁶ The method suggested by Hanley and McNeil was used to compare AUCs between the memory measures.²⁷ Analysis was performed using SPSS version 24. Statistical significance was set at P < .05 and 95% confidence intervals were calculated.

Results

Sociodemographics characteristics and neuropsychological background for aMCI and NC groups are shown in Table 1. There were no significant differences in age, gender, educational attainment, or degree of literacy between groups. Up to 41.7% of the sample was comprised of individuals with less than a first grade of educational attainment.

Table 1.

Sociodemographic Characteristics and Neuropsychological Tests Results by Diagnostic Group.

	NC	aMCI	Total	P	Cohen d Effect Size
Number of participants	30	30	60
Gender (female %)	19 (63.3%)	22 (73.3%)	41 (68.2%)	.40
Age (years)	75.4 (4.7)	73.9 (6.6)	74.7 (5.7)	.32
Educational attainment
<First grade	10 (33.3%)	15 (50.0%)	(41.7%)	.29
First grade	11 (36.7%)	6 (20.0%)	17 (28.3%)
>First grade	9 (30.0%)	9 (30.0%)	18 (30.0%)
Degree of literacy
Illiterate	1 (3.3%)	3 (10.0%)	4 (6.7%)	.43
R&W not fluently	11 (36.7%)	13 (43.3%)	24 (40.0%)
R&W fluently	18 (60.0%)	14 (46.7%)	32 (53.3%)
Phototest (total score)	36.2 (5.2)	25.7 (6.1)	30.9 (7.8)	<.001	1.85
DMS-48
Set 1 score	46.6 (1.5)	37.7 (4.5)	42.2 (5.6)	<.001	2.65
Set 2 score	46.3 (1.7)	32.2 (4.1)	39.2 (7.8)	<.001	4.49
TMA-93
Total score	27.4 (3.8)	11.2 (7.6)	19.3 (10.1)	<.001	2.70
Errors	0.3 (0.6)	2.1 (3.6)	1.2 (2.7)	<.001	−0.48
Intrusions	0.7 (2.3)	3.7 (5.1)	3.7 (5.1)	<.001	−0.76
Perseverations	0.1 (1.4)	1.4 (2.4)	1.4 (2.4)	<.001	−0.66
FCSRT
TFR	28.1 (5.2)	11.0 (6.2)	19.5 (10.3)	<.001	2.99
TR	46.8 (1.3)	31.0 (10.9)	28.9 (11.1)	<.001	2.04
ISC	0.94 (0.06)	0.57 (0.24)	0.76 (0.26)	<.001	2.11

Abbreviations: NC, normal controls; aMCI, amnestic mild cognitive impairment; DMS-48, Delayed Matching-to-Sample Task 48; FCSRT, Free and Cued Selective Reminding Test; ISC, Index of Sensitivity of Cueing; R&W not fluently, able to read and write in Spanish but not fluently; R&W fluently, able to read and write fluently in Spanish; TFR, total free recall; TR, total recall.

All the participants were able to complete both TMA-93 and FCSRT tests. With respect to TMA-93, the aMCI group had significantly lower TMA-93 total scores than the NC group (11.2 ± 7.6 vs 27.4 ± 3.8, P < .001; Cohen d = 2.7) and made significantly more errors (2.1 ± 3.6 vs 0.3 ± 0.6, P < .001; Cohen d = 0.4), intrusions (3.7 ± 5.1 vs 0.7 ± 2.3, P < .001; Cohen d = 0.7), and perseverations (1.4 ± 2.4 vs 0.1 ± 1.4, P < .001; Cohen d = 0.6; Table 1). In the NC group, 21 of 30 participants needed only 1 or 2 trials to complete the test. On the contrary, all 30 of 30 patients needed all 3 trials. Of 30, 18 individuals in the NC group scored 29 or 30 on the test. The ROC curve analysis determined an AUC of 0.97 (95% CI, 0.89-1.00, P < .001) for the TMA-93 total score (Figure 2). A score of 19/20 (Youden index, J = 0.83) was revealed as the optimal threshold to distinguish between patients with aMCI and NC with a sensitivity of 0.87 (95% CI, 0.69-0.96) and a specificity of 0.97 (95% CI, 0.83-1.00).

Figure 2.

TMA-93 total score: receiver operating characteristic (ROC) curve. Area under the curve (AUC) of 0.97.

With respect to FCSRT, the aMCI group had significantly lower TFR (11.2 ± 6.2 vs 28.1 ± 5.2, P < .001; Cohen d = 2.9), TR (31.0 ± 10.9 vs 46.8 ± 1.3, P < .001; Cohen d = 2.0), and ISC (0.57 ± 0.24 vs 0.94 ± 0.06, P < .001; Cohen d = 2.1) scores than NC group (Table 1). The ROC curve analysis determined an AUC of 0.99 (95% CI, 0.92-1.00) for TFR, 0.95 (95% CI, 0.86-0.99) for TR, and 0.93 (95% CI, 0.83-0.98) for ISC. A score of 21/22 (Youden index, J = 0.93) was shown as the best cutoff for TFR to discriminate between groups with a sensitivity of 1.00 (95% CI, 0.88-1.00) and a specificity of 0.93 (95% CI, 0.78-0.99). A score of 43/44 (Youden index, J = 0.83) was revealed as the optimal threshold for TR with a sensitivity of 0.83 (95% CI, 0.65-0.94) and a specificity of 1.00 (95% CI, 0.88-1.00). For ISC, the best cutoff was <0.77 (Youden index, J = 0.76) with a sensitivity of 0.77 (95% CI, 0.58-0.90) and a specificity of 1.00 (95% CI, 0.88-1.00).

The ROC curves comparison showed an apparent superiority for TFR (Figure 3) but, following the Hanley and McNeil method, the comparisons of AUC between TMA-93 total score and each variable FCSRT did not show significant differences (Table 2).

Figure 3.

Comparison of receiver operating characteristic (ROC) curves: total score TMA-93 (purple); total free recall of Free and Cued Selective Reminding Test (FCSRT-TFR; blue); total recall of FCSRT (FCSRT-TR green); and Index of Sensitivity of Cueing of FCSRT (FCSRT-ISC, gray).

Table 2.

Comparison of aROC between TMA-93 and FCSRT by Hanley and McNeil Method.

	aROC	FCSRT-TFR	FCSRT-TR	FCSRT-IS
TMA-93	0.97 (0.02)	0.02 (0.02); ns	0.02 (0.03); ns	0.04 (0.03); ns
FCSRT-TFR	0.99 (0.01)		0.04 (0.03); ns	0.06 (0.03); ns
FCSRT-TR	0.94 (0.03)			0.02 (0.01); ns
FCSRT-ISC	0.93 (0.04)

Abbreviations: aROC, The area under curve (AUC) using receiver operating characteristic (ROC) curve analysis; FCSRT, Free and Cued Selective Reminding Test; FCSRT-ISC, Index of Sensitivity of Cueing on FCSRT; FCSRT-TFR, Total Free Recall on FCSRT; FCSRT-TR, Total Recall on FCSRT; ISC, Index of Sensitivity of Cueing; ns, not significant.

Discussion

Even in the most developed countries, there are patients with memory complaints who have a low educational attainment or are migrants from other cultures and have a different native language. It seems inappropriate to use classical memory tests based on list learning or memorizing paragraphs in these patients. Diagnostic delays and exclusions from clinical trials are consequences of this handicap. Picture memory tests seem more suitable in this context.

Here, TMA-93 diagnostic accuracy was first tested to differentiate patients with aMCI from NC without excluding less educated participants (up to 41% of the sample). Results showed an optimal diagnostic accuracy of 0.97, not different from that obtained by the picture version of the FCSRT on the same sample. This result indicates that TMA-93 may be so reliable as FCSRT to assist in the diagnosis of aMCI in this sociodemographic context.

This study also supposed the preliminary validations of the TMA-93 and the picture version of the FCSRT in Spain. The best cutoff scores for both tests were lower than the previously described.^4,28 Total scores on TMA-93 for patients with aMCI here were similar to those previously reported for patients with dementia in the French validation study.⁴ Apparently, this result could indicate that while TMA-93 can be useful in differentiating patients with aMCI or dementia from NC, it may not be so useful in distinguishing patients with aMCI from patients with dementia. With respect to FCSRT, the best cutoff (43/44) for TR was also lower than that recently reported for American population (45/46).²⁸ So, it is probable that the relatively low cutoff scores for both tests in this study could be better explained by differences in the sociodemographic features of the population or by differences in the method used for the validation.

High total scores with a relatively small standard deviation suggest a ceiling effect for the TMA-93 in the control group. This ceiling effect could be an advantage for the test in diagnosing aMCI since a small number of errors can be a worrisome result for a patient. In some way, this ceiling effect has also been described for FCSRT and has been considered as a problem in studies focused on the preclinical phase of AD.²⁹ In fact, MBT has been recently developed to overcome this problem with the FCSRT.^14,29 This ceiling effect seems to be even most robust for the TMA-93. Normative studies should be undertaken to explore whether this ceiling effect for controls remains when only the oldest or the less educated individuals are considered.

The strength of the FCSRT is its validation with AD biomarkers.^30,31 TMA-93 has not been validated using AD biomarkers yet. Unavailability of biomarkers is a limitation for this study. The diagnosis of aMCI was based on clinical criteria according to NIA-AA recommendations.²⁴ The evidence obtained from this preliminary validation study highlights the need for inclusion of biomarkers in the design of future studies focused on validations or comparisons of these picture memory tests in order to increase the specificity of the diagnosis.

Another important question is about the applicability and acceptability of these tests. Both FCSRT and TMA-93 were well tolerated by the participants in this study including those with less than a first grade of educational attainment. The acceptability usually emerges as a problem when patients have severe memory impairment and there is a floor effect for the test. In this situation, a short test that requires less examination time is better tolerated by the patients. Applicability is very important in contexts that are not so specialized as dementia units where patients with memory complaints are usually first attended. The shorter the test, the more applicable it will be. Right there, in terms of acceptability and applicability, TMA-93 would have an advantage since it takes shorter time than FCSRT and DMS-48 to be administered. Future studies comparing these picture memory tests should also focus on applicability or acceptability, not only on diagnostic accuracy.

This study presents some limitations. In relation to design, there may be a selection bias. Convenience sampling is not representative of the normal population or the population with dementia. However, for a preliminary evaluation of a diagnostic test, the main requirement is that the 2 groups that are compared, patients and NC, do not differ in the sociodemographic variables and only differ in the diagnosis.³² Here, this requirement was fulfilled.

The standard Mini-Mental State Examination (MMSE) was not used here as screening test before making the diagnosis by the DMS-48. The MMSE scores may have better defined the cognitive status of the sample and would have allowed comparisons with other studies. However, MMSE may be not reliable for participants with low educational level^33,34 and was deliberately avoided in this study. The Phototest,^18,19 a picture screening test with normative data collected in Spain,³⁵ was used instead. This test was shown to be robust to educational level.³⁶

In conclusion, the difficulty to evaluate the memory in emigrants or individuals with a low educational attainment can be solved by using picture memory tests. TMA-93 may be as discriminative as FCSRT for diagnosing aMCI. Future studies including AD biomarkers will strengthen the validity of the test. Studies focused on applicability and acceptability could place this new test as a good choice in some sociodemographic and clinical care contexts.

Footnotes

Authors' note

Didier Maillet is now affiliated with Service de Neurologie, Hôpital Saint-Louis (AP-HP), Paris, France.

Acknowledgments

The authors thank Aida Suárez-González (University College London) and Elena Tsoy (UCSF Memory and Aging Center) for their contribution to the elaboration of this manuscript.

Declaration of Conflicting Interests

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Carnero-Pardo is the author of the Phototest. Maillet is the author of the TMA-93.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Cristóbal Carnero-Pardo

Didier Maillet

Emilio Franco-Macías

References

Dubois

Feldman

Jacova

. Advancing research diagnostic criteria for Alzheimer’s disease: the IWG-2 criteria. Lancet Neurol. 2014;13(6):614–629.

Buschke

. Cued recall in amnesia. J Clin Exp Neuropsychol. 1984;6(4):433–440.

Grober

Lipton

Hall

Crystal

. Memory impairment on free and cued selective reminding predicts dementia. Neurology. 2000;54(4):827–832.

Maillet

Narme

Amieva

. The TMA-93: a new memory test for Alzheimer’s disease in illiterate and less educated people. Am J Alzheimers Dis Other Demen. 2017;32(8):461–467.

Tounsi

Deweer

Ergis

. Sensitivity to semantic cuing: an index of episodic memory dysfunction in early Alzheimer disease.Alzheimer Dis Assoc Disord. 1999;13(1):38–46.

Zimmerman

Katz

Wang

. Comparison of “Word” vs. “Picture” version of the Free and Cued Selective Reminding Test (FCSRT) in older adults. Alzheimers Dement (Amst). 2015;1(1):94–100.

Parra

Abrahams

Fabi

Logie

Luzzi

Della Sala

. Short-term memory binding deficits in Alzheimer’s disease. Brain.2009;132(Pt 4):1057–1066.

Gramunt

Sánchez-Benavides

Buschke

. Psychometric properties of the Memory Binding Test: test-retest reliability and covergent validity. J Alzheimers Dis. 2016;50(4):999–1010.

Brown

Warburton

. Association and dissociation in recognition memory systems. In: Zimmer

Mecklinger

Lindenberger

, eds. Handbook of Binding and Memory, Perspective From Cognitive Neuroscience. New York, NY: Oxford University Press; 2006:193–220.

10.

Cer

O’Reilly

. Neural mechanisms of binding in the hippocampus and neocortex: insights from computational models. In: Zimmer

Mecklinger

Lindenberger

, eds. Hanbook of Binding and Memory, Perspective From Cognitive Neuroscience. New York, NY: Oxford University Press; 2006:193–220.

11.

Luo

Niki

. Does hippocampus associate discontinuous events? Evidence from event-related fMRI. Hippocampus. 2005;15(2):141–148.

12.

Mayes

Montaldi

Migo

. Associative memory and the medial temporal lobes. Trends Cogn Sci.2007;11(3):126–135.

13.

Wechsler

. Wechsler Memory Scale. Revised Edition. New York, NY: The Psychological Corporation; 1987.

14.

Buschke

. The rationale of the Memory Binding Test. In: Nilsson

Ohta

eds. Dementia and Memory. New York, NY: Psychology Press; 2014:55–71.

15.

Sackett

Haynes

. Evidence base of clinical diagnosis: the architecture of diagnostic research. BMJ. 2002;324(7336):539–541.

16.

Galvin

Roe

Powlishta

. The AD8: a brief informant interview to detect dementia. Neurology. 2005;65(4):559–564.

17.

Carnero Pardo

De la Vega Cotarelo

López Alcalde

. Assessing the diagnostic accuracy (DA) of the Spanish version of the informant-based AD8 questionnaire. Neurologia. 2013;28(2):88–94.

18.

Carnero-Pardo

Lopez-Alcalde

Allegri

. Diagnostic accuracy of the phototest for cognitive impairment and dementia. Neurologia. 2007;22(10):860–869.

19.

Carnero Pardo

López Alcalde

Allegri

Russo

. A systematic review and meta-analysis of the diagnostic accuracy of the phototest for cognitive impairment and dementia. Dement Neuropsychol. 2014;8(2):141–147.

20.

Barbeau

Didic

Tramoni

. Evaluation of visual recognition memory in MCI patients. Neurology. 2004;62(8):1317–1322.

21.

Rulier

Matharan

Barbeau

. The DMS 48: norms and diagnostic proprieties for Alzheimer´s disease in elderly population from the AMI cohort study. Geriatr Psychol Neuropsychiatr Vieil. 2014;12(3):321–330.

22.

Yesavage

Brink

Rose

Lum

. Development and validation of a geriatric depression scale: a preliminary report. J Psychiat Res. 1983;17(1):37–49.

23.

Teunisse

Derix

. Measurement of activities of daily living in patients with dementia living at home: development of a questionnaire. Tijdschr Gerontol Geriatr. 1991;22(2):53–59.

24.

Albert

DeKosky

Dickson

. The diagnosis of mild cognitive impairment due to Alzheimer’s disease: recommendations from the National Institute on Aging and Alzheimer’s Association Workgroup. Alzheimers Dement. 2011;7(3):270–279.

25.

Gifford

Cummings

. Evaluating dementia screening tests: methodologic standards to rate their performance. Neurology. 1999;52(2):224–227.

26.

Youden

. Index for rating diagnostic tests. Cancer. 1950;3(1):32–35.

27.

Hanley

McNeil

. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology. 1983;148(3):839–843.

28.

Grober

Veroff

Lipton

. Temporal unfolding of declining episodic memory on the Free and Cued Selective Reminding Test in the predementia phase of Alzheimer’s disease: implications for clinical trials. Alzheimers Dement (Amst). 2018;10:161–171.

29.

Gramunt

Sánchez-Benavides

Buschke

. The Memory Binding Test: development of two alternate forms into Spanish and Catalan. J Alzheimers Dis. 2016;52(1):283–293.

30.

Swerdlow

Jicha

. Alzheimer disease: can the exam predict the pathology? Neurology. 2012;78(6):374–375.

31.

Derby

Burns

Wang

. Screening for predementia AD: time-dependent operating characteristics of episodic memory tests. Neurology. 2013;80(14):1307–1314.

32.

Carnero Pardo

. Evaluación de las pruebas diagnósticas. Rev Neurol. 2005;40(11):641–643.

33.

Scazufca

Almeida

Vallada

Tasse

Menezes

. Limitations of the Mini-Mental State Examination for screening dementia in a community with low socioeconomic status: results from the Sao Paulo Ageing & Healthy Study. Eur Arch Psychiatry Clin Neurosci. 2009;259(1):8–15.

34.

Zunzunegui

Otero

. Screening for dementia in populations with little education. Int J Geriatr Psychiatry. 2010;25(11):8–15.

35.

Carnero Pardo

Carrera Muñoz

Triguero Cueva

López Alcalde

Vílchez Carrillo

. Normative data for the Fototest from neurological patients with no cognitive impairment [published online ahead of print May 28, 2018]. Neurologia. 2018. doi:10.1016/j.nrl.2018.03.001.

36.

Carnero-Pardo

Saez-Zea

Montiel-Navarro

Feria-Vilar

Gurpegui

. Normative and reliability study of Fototest. Neurologia. 2011;26(1):20–25.