Detection Test for Language Impairments in Adults and the Aged—A New Screening Test for Language Impairment Associated With Neurodegenerative Diseases: Validation and Normative Data

Abstract

To date, there is no quick screening test that could be used during routine office visits to accurately assess language disorders in neurodegenerative diseases. To fill this important gap, we developed the Detection Test for Language impairments in Adults and the Aged (DTLA), a quick, sensitive, standardized screening test designed to assess language disorders in adults and the elderly individuals. In Study 1, we describe the development of the DTLA. In Study 2, we report data on the DTLA’s validity and reliability. Finally, in Study 3, we establish normative data for the test. The DTLA has good convergent and discriminant validity as well as good internal consistency and test–retest reliability. Norms for the DTLA obtained from a sample of 545 healthy, community-dwelling, French-speaking adults from 4 French-speaking countries (Belgium, Canada (Quebec), France, and Switzerland) are provided. The development, validation, and standardization of the DTLA constitute a significant effort to meet the need for a language screening test adapted to neurodegenerative diseases.

Keywords

language abilities screening instrument dementia primary progressive aphasia normative data validity

Background

The number of people aged 60 years and above is expected to at least double by 2050, reaching approximately 2 billion older adults worldwide.¹ This dramatic demographic growth has important economic, political, and societal implications. Normal aging is accompanied by changes in cognitive functions. These changes do not occur in a uniform way in all cognitive domains. Most of the cognitive changes observed in normal aging have been attributed to a reduction in processing speed.² Due to changes in the nervous system, older people often perform less well than younger adults on tests measuring attention, working memory, episodic memory, and executive functions.³ Compared to these domains, language appears to be mostly resistant to age-related decline in normal aging. Older adults maintain or even improve knowledge of words and word meanings.⁴ However, difficulties retrieving spoken and written forms of words usually become more frequent with age.⁵

From a public health perspective, the demographic aging trend is accompanied by a tremendous growth in neurodegenerative diseases. The number of individuals with dementia worldwide is estimated to nearly double every 20 years, from 35.6 million in 2010 to 65.7 million in 2030 and 115.4 million in 2050.⁶ Dementia is a decline in cognitive function affecting memory, visuospatial abilities, executive functions, praxis, gnosis, and thinking. Language is also fragile in pathological aging. Individuals diagnosed with mild cognitive impairment, that is, the condition between healthy aging and dementia, often show language deficits affecting word comprehension, word production, syntax, and discourse (for a review, see Taler and Phillips⁷). In dementia, clinical linguistic profiles have been described following neuroepidemiological and neuropsychological studies. Recent neuropsycholinguistic studies have also made a major contribution to improving the characterization of language deficits in dementia by specifically identifying impaired and preserved processing components of language processing in the early stages of the disease (for a review, see Macoir et al⁸). In addition to progressive episodic memory loss, people in the early stage of Alzheimer’s disease (AD) usually present with semantic memory impairment leading to word-finding and word comprehension deficits. Articulatory, phonological, and syntactic abilities are usually considered unscathed until the final stage of AD. Some studies, however, showed that disease progression might impact these abilities.⁹ Syntax processing, required for sentence production and sentence comprehension, may also be compromised in some cases of AD, even in the early stages of the disease.^10,11 Whether this deficit is really syntactic in nature or is more readily explained by working memory difficulties or semantic interpretive processing remains unresolved. With respect to written language, studies in AD also reveal the presence of surface dyslexia and surface agraphia, although different patterns of written language impairment may also be observed. Language is also affected in the early stages of other major forms of dementia, including vascular cognitive impairment (VCI) and dementia with Lewy bodies (DLB).

There are very few studies that have systematically described language disorders in VCI. Most of them were conducted in an attempt to differentiate VCI from AD. Many of these studies suggested that there was no significant difference in language processing between the 2 syndromes.¹² However, other authors reported contradictory results. For example, according to most studies, the comprehension of complex sentences is affected in people with VCI because of impaired working memory and executive functions. In some of these studies, there was little or no difference in syntactic comprehension between individuals with AD and VCI, while in others, individuals with VCI performed better than those with AD.^13,14 Compared to AD, the spontaneous speech of individuals with VCI is less empty and conveys more information but they tend to produce shorter and less grammatically complex phrases.¹⁵ To date, very little is known about the nature and prevalence of reading and writing deficits in VCI. In newly diagnosed patients, Chan et al¹⁶ found more reading difficulties in VCI than in AD, while no differences between the 2 groups in reading tasks were reported in other studies.¹³ For written production, a few investigations suggest peripheral impairments (eg, difficulties to write letters and copy sentences) in VCI.¹⁷ Others found that, as compared to individuals with AD, patients with VCI produced more spelling errors and grammatically less complex sentences, therefore suggesting a central origin for this deficit.¹⁸

Dementia with Lewy bodies is a frequent form of dementia, commonly regarded as a Parkinson plus syndrome that has the clinical and pathological features of both AD and Parkinson disease.¹⁹ Dementia with Lewy bodies is associated with impairment in all areas of cognition, with predominance of visuoperceptual/spatial deficits. Almost no study has specifically addressed language deterioration in DLB. Language is usually considered unimpaired in the early stages of the disease but impairments in confrontation naming have been reported with disease progression.²⁰ Individuals with DLB also present with syntactic impairments in sentence comprehension, a deficit attributed to working memory and executive dysfunctions.²¹ A similar executive origin was proposed to explain their difficulty in organizing narrative speech in both production and comprehension modalities.^22,23 Finally, up-to-date, very little is known about the reading and spelling abilities in DLB.

Finally, language deficits are at the core of the clinical portrait of primary progressive aphasia (PPA). Primary progressive aphasia is a progressive language disorder associated with atrophy of the frontal and temporal regions, typically resulting from a neurodegenerative disease. Primary progressive aphasia is a heterogeneous condition, in which the most prominent clinical feature is difficulty with language (progressive impairment of language production, syntax, or word comprehension), while other cognitive domains, including memory, visuospatial skills, and executive abilities, are not affected at the onset and in the early stages of the disease.²⁴ A broad-ranging International Consensus Group recently published recommendations for the diagnosis and classification of PPA.²⁵ These recommendations provide a classification of PPA and its 3 main variants, namely the nonfluent/agrammatic variant (nfvPPA), semantic variant (svPPA), and logopenic variant (lvPPA). According to this classification, at least 1 of the following 2 core features must be present in nfvPPA: (1) effortful, halting speech with inconsistent speech sound errors and distortions (apraxia of speech), and (2) agrammatism in language production. Moreover, at least 2 of the following 3 additional features must be present: (1) impaired comprehension of syntactically complex sentences, (2) spared single-word comprehension, and (3) spared object knowledge. The svPPA is a clinical syndrome that results from progressive atrophy of the temporal lobes, leading to the selective impairment of semantic memory. According to the classification proposed by Gorno-Tempini et al²⁵, the following 2 core features must be present to establish a diagnosis of svPPA: (1) impaired confrontation naming and (2) impaired single-word comprehension. Moreover, at least 3 of the following additional features must be present: (1) impaired object knowledge, particularly for low-frequency or low familiarity items, (2) surface dyslexia or dysgraphia, (3) spared repetition, and (4) spared speech production (grammar and motor speech). Finally, the 2 core features essential to the diagnosis of lvPPA are the presence of anomia in spontaneous speech and confrontation naming and impaired repetition of sentences and phrases. At least 3 of the following additional features must be present: (1) production of phonological errors, (2) preservation of semantic memory, (3) preservation of articulation and prosody, and (4) the absence of agrammatism. There are also some supporting features and specific imaging abnormalities for each PPA variant.²⁵

Early diagnosis of dementia relies on various types of assessment performed to detect the disease (screening) or rule out other possible causes of cognitive impairment (ie, differential diagnosis). With respect to cognition and language, the main goal of screening is to determine whether a patient has a problem or not. The output of this type of assessment is a “pass” or “fail” result, based on an established criterion that could lead to a more exhaustive or follow-up assessment. Cognitive screening tests are used by primary care physicians to diagnose dementia. However, this type of assessment is still not extensively done in standard practice and numerous patients who have dementia remain undiagnosed.²⁶ When faced with complex clinical presentations such as primary language symptoms, physicians refer patients to specialists, such as geriatricians or neurologists, who also commonly use cognitive screening tests such as the Mini-Mental State Examination (MMSE)²⁷, Montreal Cognitive Assessment (MoCA)²⁸; or Alzheimer’s Disease Assessment Scale-Cognitive Subscale (ADAS-Cog).²⁹ In the majority of these screening tests, the focus of the assessment is memory dysfunction, the hallmark of AD, while impairments in other cognitive domains such as praxis, executive functions, and language are largely underestimated. However, according to the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders, neurocognitive dysfunction in adults is not limited to learning and memory problems but also includes difficulties with complex attention, executive functions, perceptual and motor abilities, social cognition, and language.³⁰ Moreover, studies have shown that screening tests for dementia could be misleading when used with patients with language problems.³¹

Even though most cognitive screening tests involve subtests of language function, these are often limited to naming (eg, MoCA) or verbal fluency (MoCA, MMSE, etc), a task mainly relying on executive functions. Moreover, it is difficult to disentangle language from other cognitive abilities. The role and influence of language abilities in various screening tests designed to assess cognitive functions is well known. For example, the immediate and delayed recall of words used in the MMSE and the MoCA to assess episodic memory rely on verbal stimuli. Thus, these subtests also involve lexical and semantic abilities. Impaired performance on these subtests could originate from a memory as well as a language deficit. Compared to the MMSE and the MoCA, a large part of Addenbrooke’s Cognitive Examination (ACE)³² and its revised version (ACE-R)³³ is devoted to assessing language, with subtests covering not only picture naming and verbal fluency but also language comprehension, semantic matching, repetition, reading, and spelling.^19,20 The ACE-R has proven to be effective in detecting and tracking the evolution of neurodegenerative diseases in which the core symptoms are language-related, such as nfvPPA and svPPA.³⁴ However, considering the clinical presentation of PPA variants, some language subdomains are missing from the ACE-R: syntactic comprehension (affected in nfvPPA) and reading and spelling of pseudowords (preserved in svPPA). Additionally, the ACE-R includes a sentence repetition subtest but stimuli are limited to 2 short syntactically complex sentences, while one of the most distinctive features of lvPPA is impairment in the repetition of long sentences because of short-term memory deficits. Finally, the average administration time for this test is around 16 minutes, which is lengthy for primary care clinicians.³³ There are other brief measures of cognitive status in adults, such as the Cognitive Linguistic Quick Test.³⁵ This test has similar limitations than those described for the MMSE and the MoCA. Moreover, its administration is lengthy (from 15 to 30 minutes) for primary care clinicians. Finally, there are some screening tests for language impairment such as the Frenchay Aphasia Screening Test and the Mississippi Aphasia Screening test, but they were developed for aphasia and are not suitable for language deficits associated with neurodegenerative diseases.^36,37

As a result, primary care providers are frequently faced with patients whose main complaints concern language problems in everyday and professional life. To date, however, there is no quick screening test that could be applied during routine office visits to accurately assess language disorders in neurodegenerative diseases. Such a test would improve referrals to specialized resources (memory clinic, geriatrics, neurology, speech-language pathology, neuropsychology), where the diagnosis could be confirmed and the appropriate health care provided. Also, the use of such a test would improve the diagnosis of neurodegenerative diseases, especially those in which language is primarily affected. Ultimately, this will permit patients and their families to receive services at an earlier stage of the disease.

To fill this important gap, we developed a quick, sensitive, standardized French screening test designed to assess language disorders in adults and the elderly individuals. In this article, we first describe the development of the “Dépistage des troubles du langage chez l’adulte et la personne âgée” (DTLA—Detection Test for Language impairments in Adults and the Aged), a new screening test developed in 4 French-speaking countries (Belgium, Canada (Quebec), France and Switzerland). We also studied the validity and reliability of the test and established normative data from a representative sample of adults and elderly participants. Thus, 3 studies are presented in this article. In Study 1, we describe the design and development of the DTLA. In Study 2, we report data on the convergent and discriminant validity of the DTLA as well as on its test–retest reliability and internal consistency. Finally, in Study 3, we provide normative data for healthy, community-dwelling, French-speaking people from the 4 French-speaking countries. The 3 studies were approved by the local Research Ethics Boards of the 4 participating countries and all participants gave their written informed consent to participate.

Study 1. Development of the DTLA

The purpose of Study 1 was to create the tasks and to choose the stimuli of the DTLA, building on the scientific literature and the steps for assessment tools development.

Methods

We followed a comprehensive test development approach, including the establishment of translational validity. Translational validity refers to the transformational aspect of construct validity and includes content and face validity.³⁸ The content validity of the DTLA was established on the basis of the scientific literature. The research team first identified the language domains and abilities most frequently affected in neurodegenerative diseases, with special attention paid to those with the best discrimination value among clinical syndromes. Next, the most sensitive and easy-to-use assessment tasks were selected for each domain of interest. The research team then determined the number of items as well as their psycholinguistic characteristics in each assessment subtest, with the objective of developing a short test, which could be administered in approximately 5 minutes.

The following language assessment tasks were selected for the screening test: (1) object picture naming; (2) word, nonword, and sentence repetition; (3) verbal fluency; (4) spelling to dictation of words and nonwords; (5) spontaneous written sentence production; (6) reading aloud of words and nonwords; (7) auditory sentence-to-picture matching; (8) written word matching; and (9) alpha-span. Lexical access is consistently affected in almost all neurodegenerative diseases. Deficits in this domain lead to substantial difficulties in confrontation naming, which is why a spoken picture-naming task was chosen. A phonemic verbal fluency task was also selected to assess lexical access. This task requires the generation of a lexical strategy, sustained by executive functions, which guides the search for words in the mental lexicon. Impairment in phonemic verbal fluency has been consistently reported in the majority of neurodegenerative diseases, including the 3 PPA variants.^39,40 Performance on sentence repetition tasks is essential to differentiate lvPPA from the 2 other PPA variants. Moreover, apraxia of speech, which is 1 of the 2 core features of nfvPPA, could be exacerbated in nonword and sentence repetition tasks and helped to differentiate individuals with nfvPPA presenting with apraxia of speech from those with agrammatism.⁴¹ A word, nonword, and sentence repetition tasks were therefore selected. Impairment of written production abilities usually occurs very early in the course of the majority of neurodegenerative diseases (for a review, see Macoir et al⁸). For example, individuals with AD or svPPA often develop surface dysgraphia.^42,43 Dysgraphia involves written agrammatism and the production of nonphonologically plausible paragraphias in nfvPPA, while patients with lvPPA usually present with phonological dysgraphia.^44,45 Spontaneous writing may also be affected in neurodegenerative diseases.^46,47 Impairment in this task usually mimics the manifestations observed in spoken production (ie, difficulties in lexical access, inflectional morphology, syntactic structure). A task involving spelling to dictation of words and nonwords and a spontaneous written sentence production task were selected to assess written production abilities. Reading difficulties are key features in AD and PPAs. They are observed in AD and svPPA in the form of surface dyslexia. In nfvPPA, disease progression is sometimes accompanied by the production of phonological errors in reading, while a pattern of phonological dyslexia is observed in lvPPA.^41,48 A word and nonword reading aloud tasks were chosen to assess reading. Sentence comprehension is another domain that can be compromised in neurodegenerative diseases. Deficits in this domain have been reported in cases of AD and Lewy body dementia, even in the early stages of the disease.^10,21 The impaired comprehension of syntactically complex sentences is 1 of the 3 additional features that must be present to detect nfvPPA.²⁵ Moreover, impairment in sentence comprehension was also found in lvPPA patients due to a reduction in verbal short-term memory resources.⁴⁹ A syntactically complex sentence-to-picture matching task was chosen to assess sentence comprehension. Also, semantic processing is consistently affected in AD.⁵⁰ Semantic impairment is also at the core of the clinical manifestations of svPPA, while semantic processing is preserved in the other 2 PPA variants.²⁵ A semantic written word-matching task was selected to assess semantic processing. Finally, some of the language deficits observed in neurodegenerative diseases are caused by phonological short-term memory and/or verbal working memory impairments. Such an origin was posited to explain difficulties in sentence production and sentence comprehension observed in AD and Lewy body dementia.^10,21 Reduction in phonological short-term memory resources was also proposed to account for the deficit in sentence repetition and sentence comprehension in lvPPA.^25,49 An alphabetization span task (“alpha-span task”), which requires subjects to recall presented words in alphabetical order, was selected to assess phonological short-term/working memory. Phonology is usually preserved in the early stages of AD, VCI, and DLB. As mentioned above, phonological abilities are affected in nfvPPA and lvPPA. In the DTLA, phonological impairment can be identified in all subtests involving spoken production: picture naming, word, nonword, and sentence repetition, verbal fluency, and word and nonword reading aloud. The cognitive domains, tasks, and characteristics of the items chosen as a result of the development process are presented in Table 1.

Table 1.

Cognitive Domains, Tasks, Item Characteristics, and Scoring of the DTLA.

Cognitive Domain	Task	Item Characteristics (Number of Stimuli)	Psycholinguistic Variables	Scoring
Lexical access in production	Picture naming	Pictures (6)	Semantic category: biological concepts (3); man-made concepts (3) Lexical frequency: low Subjective frequency: low	2 points/item: 12 points
Spoken production and phonological short-term memory	Repetition	Words (3) Nonwords (3) Long sentences (3)	Phonological complexity: high Syllable length (words and nonwords: 3 syll; sentences: 2 × 17 and 1 × 18 syll.)	2 points/item: 18 points
Spoken production and executive functions	Verbal fluency	Phonemic verbal fluency with letter D in 1 minute		15 points: EL ≤11 years = 8 words EL ≥12 years = 10 words
Written production	Spelling to dictation	Words (3) Nonwords (3)	Words: lexical frequency: low; orthographic complexity: irregular Syllable length (words: 2 syll; nonwords: 1 syll)	2 points/item: 12 points
Written production	Spontaneous writing	Written sentence spontaneous production (subject–verb complement)		4 points
Lexical and nonlexical reading	Reading aloud	Words (3) Nonwords (3)	Lexical frequency: low Orthographic complexity: irregular words Syllable length (words and nonwords: 2 syll)	1 point/item: 6 points
Syntactic comprehension	Auditory sentence-to-picture matching	Syntactically complex reversible sentences (3)	Syntactic structure: cleft object sentence (1); agentless passive sentence (1); passive sentence with agent (1)	4 points/item: 12 points
Semantic memory	Written word matching	Word triplets (4)	Semantic category: biological concepts (2); man-made concepts (2) Semantic relationship: associative (2); categorial (2)	4 points/item: 16 points
Verbal working memory	Alpha-span	Word triplet (1)	Lexical frequency: high Syllable length: 1 syll Imageability: high	5 points

Abbreviations: DTLA, Detection Test for Language impairments in Adults and the Aged; EL, education level.

Once the tasks and items were chosen, a pilot study was conducted to develop the final version of the DTLA. A screening test with 4 times as many stimuli as required for the final version of the test was created (except for verbal fluency, spontaneous writing and alpha-span: 2 three-item lists, 1 four-item list, and 1 five-item list). This preliminary version also comprised the administration protocol for each subtest including the language domain assessed, the stimuli, the psycholinguistic variables controlled and manipulated, and the instructions. Clinical experts then reviewed this version to establish the instrument’s face validity. A total of 18 professionals in the 4 countries (1 geriatrician, 2 neurologists, 7 neuropsychologists, 1 psychiatrist, and 7 speech-language pathologists) were invited to review the preliminary version of the DTLA by means of a questionnaire specifically developed to evaluate its appropriateness, usefulness, ease-of-use, and clarity (administration sheet, instructions). All informants agreed wholeheartedly with the purpose and usefulness of the instrument. After their review, adjustments were made to all the elements where a lack of clarity in the instructions or the procedure was identified.

Following these adjustments, the preliminary version of the DTLA was administered to 106 healthy participants (normal age- and education-adjusted MMSE scores) in the 4 French-speaking countries (Belgium = 40, France = 25, Canada = 23, Switzerland = 18) in order to proceed to the final selection of stimuli. All participants spoke French as their primary language. Their mean age was 65.2 years (SD = 9.5, range = 50-88), their mean level of education was 12.56 years (SD = 3.34, range = 6-23), and the gender distribution was 37 males and 69 females. They all self-reported good mental and physical health (ie, no history of neurological disease, current untreated psychiatric illness, traumatic brain injury or untreated medical conditions that could interfere with cognitive performance). All participants were tested individually in a quiet room at their home or one of the participating research centers, and the tasks were administered without any time constraints.

For the final selection of stimuli, the items kept were those for which the best scores were obtained in the 4 countries. The average success rates for each task were as follows: object picture naming = 99.83% (SD = .41); repetition: words = 100%, nonwords = 95% (SD = 3.61), and sentences = 87% (SD = 6.24); spelling to dictation: words = 91.33% (SD = 3.39), nonwords = 96.67% (SD = 3.2); reading aloud: words and nonwords = 100%; auditory sentence-to-picture matching = 100%; written word matching = 100%. On average, the participants produced 12.31 (SD = 1.35) words beginning with the letter D in the fluency task. In 96.33% (SD = 3.51) of them, the spontaneous production of a sentence was correct. Finally, for the alpha-span, the average success rates were low for the 4-item (47%, SD = 10.47) and 5-item (11%, SD = 3.7) lists, so the 3-item list was retained (95%, SD = 5.97).

The final version of DTLA is presented on a single double-sided sheet held in portrait orientation. The 5 of the 6 pictures in the naming task were taken from the study of Snodgrass and Vanderwart⁵¹, and 1 was taken from the study of Bonin et al, while the picture for the auditory sentence-to-picture matching task is original artwork.^38,52 The psycholinguistic parameters controlled and manipulated in the final version of the test are presented in Table 1. Data for lexical frequency were taken from the study of New⁵³ and those for subjective frequency and imageability were taken from the study of Desrochers and Bergeron.⁵⁴

The scoring method was established according to the relative value of each subtest for the detection of language impairment for a total maximum score of 100 points. We chose to use a 100-point scale for 2 reasons. First, its use and interpretation are more intuitive (ie, people are used to considering 100 points a perfect score). Second, it enabled us to better distribute the relative weight of each subtest in the total score by avoiding the usage of decimals. The scoring method and the maximum score for each subtest are presented in Table 1. Each item was allotted a number of points, except for verbal fluency for which fifteen points are given according to the following cutoff scores for 2 education levels, calculated on the basis of data collected in the normative study (see below): production of 8 words for individuals who have 11 or fewer years of formal education and production of 10 words for individuals with 12 or more years of formal education. The DTLA protocol, administration procedure, and instructions are available upon request from the first author.

Study 2. Validity and Reliability of the DTLA

The purpose of Study 2 was to provide data on the DTLA’s convergent and discriminant validity as well as on its test–retest reliability and internal consistency. The DTLA was designed to assess language impairment in neurodegenerative diseases. However, it is conceivable that it could also be used to detect language deficits in other neurological populations. Therefore, convergent validity, discriminant validity, and internal consistency were established by including participants presenting with neurodegenerative diseases and participants with post-stroke aphasia. A schematic representation of the DTLA validation process is presented in Figure 1.

Figure 1.

Schematic representation of the Detection Test for Language impairments in Adults and the Aged (DTLA) validation process. AD indicates Alzheimer’s disease; BNT, Boston Naming Test; HC, healthy control; MEC, Protocole Montréal d’Évaluation de la Communication; MT-86, Protocole Montréal-Toulouse d’examen linguistique de l’aphasie; PPA, primary progressive aphasia; PPTT, Pyramids and Palm Trees Test; WMS, Wechsler Memory Scale.

Methods

Participants, materials, and procedure

In all the studies of the psychometric properties of the DTLA hereafter described, participants were tested individually in a quiet room at their home or at one of the participating research centers. The external measures (ie, the other tests) and the DTLA were administered without any time constraints. Written protocols for the tests were collected by research assistants.

Convergent validity

Thirty-one patients were recruited. Twenty patients had a diagnosis of probable AD (mean age = 77.75, SD = 7.85 years; mean education = 13, SD = 3 years; mean MMSE score = 21.10, SD = 4.15). Probable AD was determined according to current diagnostic criteria and based on medical records and history (eg, diagnosis of AD from a medical doctor and/or participants taking an approved pharmacological treatment for dementia [memantine, donepezil, galantamine, or rivastigmine]).⁵⁵ These patients were in the mild-to-moderate stage of AD. Seven patients had chronic (ie, more than 1 year post-stroke) aphasia (mean age = 66.29, SD = 9.25 years; mean education = 13, SD = 3.26 years; mean MMSE score = 21.29, SD = 4.35), and 4 patients had been diagnosed by a neurologist with PPA (nonfluent/agrammatic variant = 2; logopenic variant = 1; semantic variant = 1) over the last 2 years (mean age = 76, SD = 7 years; mean education = 9, SD = 3.47 years; mean MMSE score = 23, SD = 3). Primary progressive aphasia was determined according to current diagnostic criteria and based on medical records and history (eg, diagnosis from a neurologist).²⁵ Exclusion criteria for all participants were as follows: (1) history of moderate or severe traumatic brain injury, (2) history of cerebrovascular disease (this criterion did not apply to post-stroke aphasia patients), (3) history of delirium (in the last 6 months), (4) history of intracranial surgery, (5) history of neurological disorder of cerebral origin (other than those examined in the study), (6) history of encephalitis or bacterial meningitis, (7) unstable metabolic or medical condition (eg, untreated hypothyroidism or diabetes), (8) history or actual psychiatric diagnosis according to the DSM-V,³⁰ (9) alcoholism or substance abuse (in the last 12 months), (10) uncorrected vision or audition problems, and (11) inability of the participant to provide consent due to inaptitude.

The following tests were administered to all the participants: the DTLA, Boston Naming Test;⁵⁶ repetition, comprehension, reading, spelling, and written questionnaire subtests of the Protocole Montréal-Toulouse d’examen linguistique de l’aphasie (MT-86); fluency subtest of the Protocole Montréal d’Évaluation de la Communication (MEC); digit span subtest of the Wechsler Memory Scale—Fourth Edition; and Pyramids and Palm Trees Test.^57

-60 The association of the performance of the participants on the different tasks was analyzed using Pearson correlation coefficients.

Discriminant validity

We tested whether the DTLA score distinguished between the performances of controls and patients with AD in the mild to moderate stage of the disease and between the performances of controls and patients with chronic post-stroke aphasia. Inclusion and exclusion criteria of participants were identical than those used for convergent validity. To do so, 2 groups were compared by means of independent samples t tests: (1) a group of 12 patients with a diagnosis of probable AD, matched to a group of 24 healthy control participants by age (AD: mean age = 74.5, SD = 4.83 years; Controls: mean age = 73.5, SD = 5.24 years; t (34) = −.553, P = .58) and education (AD: mean education = 13.58, SD = 3.06 years; Controls: mean education = 13.71, SD = 3.71 years; t (34) = .101, P = .92) and (2) a group of 12 patients with post-stroke aphasia, matched to a group of 24 healthy control participants by age (aphasia: mean age = 62.67, SD = 6.44 years; Controls: mean age = 64.33, SD = 5.80 years; t (34) = .784, P = .44) and education (aphasia: mean education = 12.08, SD = 2.87 years; Controls: mean education = 12.08, SD = 3.84 years; t (34) < .001, P = .1). Both groups of patients differed significantly from controls in terms of their score on the MMSE (AD: mean score = 21.42, SD = 3.73; Controls: mean score = 28.42, SD = 1.35; t (34) = 8.27, P < .001; Aphasia: mean score = 24.25, SD = 4.58; Controls: mean score = 28.42, SD = 1.18; t (34) = 4.24, P < .001).

Test–retest reliability

The DTLA was administered twice to twenty healthy participants (mean age at first testing = 62.4, SD = 6.73 years; mean education = 12.7, SD = 2.87 years; mean MMSE score = 28.55, SD = 1.73) with a 6-month interval. The 2 measures (T1 and T2) were compared by means of a paired samples t test.

Internal consistency

This type of reliability was studied with a sample of 602 participants divided into 4 groups: (1) healthy controls (n = 561; mean age = 63.96, SD = 9.21 years; mean education = 12.5, SD = 3.39 years; mean MMSE score = 28.53, SD = 1.30), (2) patients with AD (n = 20; mean age = 77.75, SD = 7.85 years; mean education = 13.15, SD = 3.07 years; mean MMSE score = 21.10, SD = 4.15), (3) patients with post-stroke aphasia (n = 17; mean age = 68.06, SD = 10.86 years; mean education = 11, SD = 3.20 years; mean MMSE score = 22.71, SD = 6.14), and (4) patients with PPA (n = 4; mean age = 76, SD = 7.07 years; mean education = 9, SD = 3.46 years; mean MMSE score = 23, SD = 3). Internal consistency was calculated by means of the Cronbach α coefficient.

Results

Convergent Validity

Table 2 shows the correlation matrix between the external measures and DTLA subtests. The relevant correlations are highlighted in gray. As expected, the external measures that assessed the same construct correlated significantly and positively with the corresponding DTLA subtests, except for 2 of them. First, the Pyramids and Palm Trees Test did not correlate significantly with the written word matching subtest of the DTLA, even though both tests focus on semantic processing.⁶⁰ However, the Boston Naming Test, which also includes semantic processing, and the written word matching DTLA subtest correlated positively and significantly.⁵⁶ Second, the digit span subtest of the Wechsler Memory Scale—Fourth Edition and the alpha-span subtest of the DTLA, both measures of working memory, failed to show a significant correlation.⁵⁹

Table 2.

Correlations Between External Measures and DTLA Subtests for Convergent Validity.

External Measures
DTLA Subtests	Boston Naming Test	Repetition (MT-86)	Fluency (MEC)	Span Task (WMS-IV)	Reading (MT-86)	Comprehension (MT-86)	Spelling to Dictation (MT-86)	Written Questionnaire (MT-86)	PPTT
Object picture naming	.817^a, ^b	.127	−.102	.282	.249	.319	.326	.307	.487^b
Repetition	.330	.788^a, ^b	.476^b	.605^b	.735^b	.707^b	.691^b	.481^b	.215
Fluency	.266	.247	.733^a, ^b	.681^b	.348	.581^b	.546^b	.413^c	.140
Alpha-span	.220	−.225	.151	.228^a	−.171	.141	−.132	−.200	.227
Reading	.295	.681^b	.486^b	.603^b	.846^a, ^b	.758^b	.802^b	.730^b	.206
Auditory sentence-to-picture matching	.239	.332	.105	.213	.255	.378^a, ^c	.147	−.011	.464^b
Spelling to dictation	.487^b	.661^b	.442^c	.608^b	.895^b	.734^b	.922^a ^,b	.853^b	.044
Spontaneous sentence production	.132	.389^c	.613^b	.311	.389^c	.482^b	.412^c	.510^a, ^b	.135
Written word matching	.510^a, ^b	−.148	−.161	.081	−.012	−.013	.138	.258	0.301^a

Abbreviations: DTLA, Detection Test for Language impairments in Adults and the Aged; MEC, Protocole Montréal d’Évaluation de la Communication; MT-86, Protocole Montréal-Toulouse d’examen linguistique de l’aphasie; PPTT, Pyramids and Palm Trees Test.

^aIndicates relevant correlations.

^b P < .01.

^c P < .05.

Discriminant Validity

The mean DTLA score of patients with AD was significantly lower than that of healthy controls (AD: mean score = 81.75, SD = 15.51; Controls: mean score = 95.17, SD = 5.78; t (34) = 4.43, P < .001). Also, the mean DTLA score of patients with post-stroke aphasia was significantly lower than that of healthy controls (Aphasia: mean score = 58.83, SD = 15.46; Controls: mean score = 93.79, SD = 8.20; t (34) = 8.92, P < .001).

Test–Retest Reliability

The performance of healthy controls on the DTLA did not differ between the first (T1) and second testing (T2), conducted 6 months after the first (T1 mean score = 94.10, SD = 7.36; T2 mean score = 96.20, SD = 5.56; t (19) = −1.63, P = .12).

Internal Consistency

The Cronbach α coefficient obtained was .84 for the 36 elements that make up the DTLA’s stimuli (except for fluency since it does not have values of 0 or 1 but is composed of the number of words produced by participants). According to Cortina⁶¹, a coefficient between .8 and .9, like the one we found, is considered to indicate good internal consistency.⁴⁵

Summary

From the analyses carried out, we can conclude that, overall, the DTLA has good convergent validity. The screening test can also distinguish between the performance of controls and patients with AD and controls and patients with post-stroke aphasia and therefore presents good discriminant validity. The stability of the DTLA over time is good (test–retest reliability) as is its internal consistency.

Study 3. Normative Data

The purpose of Study 3 was to provide normative data for the DTLA, adapted to adult and aged populations from the 4 French-speaking countries.

Method

Participants

A total of 545 healthy, community-dwelling, French-speaking adults, whose mother tongue and currently used language was French, were recruited in the 4 French-speaking countries (Belgium: n = 76, 13.09%; Quebec, Canada: n = 99, 18.2%; France: n = 255, 46.8%; Switzerland: n = 115, 21.1%). All participants had normal age- and education-adjusted MMSE scores (MMSE ≥26; mean score = 28.54, SD = 1.3), indicating normal cognition.²⁷ All participants self-reported good mental and physical health (ie, no history of neurological disease, current untreated psychiatric illness, traumatic brain injury, or untreated medical condition that could interfere with cognitive performance).

The sample was composed of 235 men (43%) and 310 women (57%), aged between 50 and 80 (mean age = 63.32 years, SD = 8.53 years), with an education level varying between 2 and 26 years (mean education = 12.5 years, SD = 3.37 years). Based on the education systems of the 4 countries and in previous experiences with tests for French-speaking populations, participants were recruited by speech-language pathology students through public advertisements and among relatives to form 4 mutually exclusive age and education groups: (1) 50 to 64 years of age and 11 or fewer years of formal education, (2) 50 to 64 years of age and 12 or more years of formal education, (3) 65 to 80 years of age and 11 or fewer years of formal education, and (4) 65 to 80 years of age and 12 or more years of formal education. Table 3 shows the descriptive statistics of the 4 groups of participants for the normative study as a function of age and education.

Table 3.

Descriptive Statistics (Mean and Standard Deviation) of the 4 Groups of Participants for the Normative Study as a Function of Age (≥65 and 65+ Years) and Education (≥11 and 12+ Years).^a

	Age
	50-64 years				65-80 years
	Education				Education
	≤11 years		12+ years		≤11 years		12+ years
	Mean	SD	Mean	SD	Mean	SD	Mean	SD
Age	57.45	4.42	55.93	3.95	71.74	4.73	70.45	4.83
Education	9.86	1.77	14.81	2.42	9.40	1.43	15.04	2.47
% Women	59		54		63		53
N	126		166		124		129
DTLA	93.88	6.99	97.73	4.19	91.58	8.23	96.35	5.88
Alert	84		94		83		92
Cutoff	78		85		75		81
Total N	545

^aSD, standard deviation; Age, age in years; Education, formal education in years; Women, percentage of women; N, number of participants in the group; DTLA, Detection Test for Language impairments in Adults and Aged people score/100; Alert, suggested alert score calculated as the 15th percentile (score below it and above the cutoff may indicate “at risk”); Cutoff, suggested cutoff score calculated as the fifth percentile (score equal to or below it is under normal performance limits); Total N, total number of participants in the normative study.

Materials and procedure

All participants were tested individually in a quiet room at their home or a research center. Tasks were administered without any time constraints. All of the DTLA’s visual stimuli (pictures, written words) were presented on the test sheet. Written protocols for the tests were collected by research assistants and data were entered in the analyses.

Results

Two-way analyses of variance (ANOVA) were carried out with age (≤65 and 65+ years) and education (≤11 and 12+ years) as between-subject factors and the total score on the DTLA (max 100 points) as the dependent variable. Results showed that age, F (1, 541) = 11.28, P < .01, η² = .020, and education, F (1, 541) = 61.84, P < .001, η² = .10, significantly affected performance on the DTLA. The interaction of age x education did not reach significance, with F (1, 541) = .70, P = .40.

We calculated the 5th, 15th, 25th, and 50th percentiles for the DTLA score for each group. After visual exploration of the whisker plots and according to the usual criteria,⁶² we chose the fifth percentile, which approximately corresponds to one and a half standard deviations below the mean, as the most reliable cutoff score.⁶³ A score equal to or below the suggested cutoff score can be considered to be under normal performance limits. We also proposed an alert score based on the 15th percentile, frequently considered as representing borderline performance.⁶⁴ A score below this alert score is not necessarily, but might be, under normal performance limits. Further language and cognitive testing is suggested in such cases. Table 3 shows the suggested cutoff and alert scores for each group of participants. The average time to administer the DTLA is about 5 minutes. Scoring is performed as the test is administered, so that the average time to complete the DTLA (ie, administration, scoring and interpretation) is about 6 minutes.

Discussion

Aging is the most important risk factor for cognitive decline, and the detection of cognitive impairment in at-risk middle-aged and elderly individuals has become a societal priority. In frontline services as well as specialized clinics, this detection is done by means of cognitive screening tests such as the MMSE or MoCA.^27,28 In the majority of these tests, the assessment focuses on memory dysfunction, the hallmark of AD, while other cognitive domains such as praxis, executive functions and language are largely ignored. Furthermore, studies have shown that screening tests for dementia could be misleading when used with patients with language problems.³¹ Language deficits are an integral part of clinical symptoms in all the major forms of neurodegenerative diseases. They are also the core features of the 3 variants of PPA. Most cognitive screening tests involve subtests of language function (eg, picture naming in MoCA, verbal fluency in MoCA and MMSE). However, these tests are not sensitive enough to capture the entire spectrum of language manifestations in dementia or they are too time-consuming to administer (eg, ACE-R³³) for them to be used in primary care services.²⁰ In short, there is no quick, sensitive screening test that could be administered during routine office visits to assess language deficits in neurodegenerative diseases.

The DTLA was explicitly developed to meet the need for an assessment tool specifically addressing the language impairment encountered in the majority of neurodegenerative diseases. This test comprises 9 sensitive, easy-to-administer tasks designed to quickly (5 minutes) assess the language domains and abilities most frequently affected in these diseases, with particular attention paid to those with the best discrimination value among clinical syndromes. The results of the psychometric study of this new tool showed that it has good convergent and discriminant validity as well as very good internal consistency and test–retest reliability. This study also provides norms for the DTLA obtained from a sample of 545 healthy, community-dwelling, French-speaking adults. These individuals, aged between 50 and 80 years, with an education level varying between 2 and 26 years, were recruited in 4 French-speaking countries (Belgium, Canada (Quebec), France, and Switzerland). The development, validation, and standardization of such a screening instrument constitute a significant effort to meet the need for a language screening test adapted to neurodegenerative diseases.

As a screening test, the main goal of the DTLA is to determine whether an individual has language impairment. Although scores on specific subtests can provide information concerning the possible etiology of deficits, the tests were not designed to make differential diagnoses. The suggested cutoff and alert scores provided in this study for the DTLA should be used, respectively, to confirm the presence of a language impairment or to raise a flag and prompt further and more extensive language assessment. Other information collected during the medical visit may supplement the DTLA results, for example, specific complaints about language, word finding, speech, or syntactic problems apparent during the interview.

The large group of participants is a major strength of this normative study. Despite this significant number, however, further studies are needed to, for example, extend the DTLA normative data to include people aged 81 and above. Another limitation of the present study was the use of an incidental sampling method, which could have resulted in selection bias. Although a random sampling method would have been preferable, this study is a practical and relevant starting point for establishing DTLA norms for the French-speaking population.

Culture has an impact on cognition, and therefore, it is important to use normative data specific to the population to which they are applied. This is particularly true for the assessment of language functions, considering the possible psycholinguistic (eg, vocabulary, familiarity of concepts) biases. The DTLA was created by selecting stimuli culturally adapted to the populations in the 4 French-speaking countries. Further studies are now necessary to develop DTLA versions in other languages.

Conclusion

To conclude, this study provides psychometric and normative data for the DTLA, a new screening test for the quick assessment of language abilities in adults and elderly people. These norms, established from a wide sample of individuals selected from the community in 4 French-speaking countries, will be useful to primary care and specialized clinicians in detecting language impairments associated with neurodegenerative diseases.

Footnotes

Authors’ Note

J.M. was responsible for the coordination of all aspects of the study. All the authors contributed to the study design. J.M. was responsible for the recruitment of participants in Canada. M.F. was responsible for the recruitment of participants in Switzerland. T.M.T. and A.R. were responsible for the recruitment of participants in France. L.F. was responsible for the recruitment of participants in Belgium. J.M. and M.W. analyzed and interpreted the data. All authors reviewed and gave final approval of the manuscript.

Declaration of Conflicting Interests

The authors declared no potential conflict of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

References

United Nations. World population ageing 2013. In: United Nations, Department of Economic and Social Affairs, Population Division (World Population Ageing 2013. ST/ESA/SER.A/348., ed. New York, NY: United Nations; 2013.

Salthouse

. Aging and measures of processing speed. Bio Psychol. 2000;54(1):35–54.

Fjell

Walhovd

. Structural brain changes in aging: courses, causes and cognitive consequences. Rev Neurosci. 2010;21(3):187–221.

Glisky

. Changes in cognitive function in human aging. In: Riddle

, ed. Brain Aging: Models, Methods, and Mechanisms. Boca Raton, FL: CRC Press/Taylor & Francis; 2007.

Craik

Bialystok

. Cognition through the lifespan: mechanisms of change. Trends Cogn Sci. 2006;10(3):131–138.

Prince

Bryce

Albanese

Wimo

Ribeiro

Ferri

. The global prevalence of dementia: a systematic review and metaanalysis. Alzheimers Dement. 2013;9(1):63–75.e62.

Taler

Phillips

. Language performance in Alzheimer’s disease and mild cognitive impairment: a comparative review. J Clin Exp Neuropsychol. 2008;30(5):501–556.

Macoir

Turgeon

Laforce

. Language processes in delirium and dementia. In: Wright

, ed. International Encyclopedia of the Social & Behavioral Sciences. Vol 13. 2nd ed. Oxford, UK: Elsevier; 2015:360–367.

Croot

Hodges

Xuereb

Patterson

. Phonological and articulatory impairment in Alzheimer’s disease: a case series. Brain Lang. 2000;75(2):277–309.

10.

Waters

Caplan

. Working memory and on-line sentence comprehension in patients with Alzheimer’s disease. J Psycholinguist Res. 1997;26(4):377–400.

11.

Bates

Harris

Marchman

Wulfeck

Kritchevsky

. Production of complex syntax in normal ageing and Alzheimer’s disease. Lang Cogn Processes. 1995;10(5):487–539.

12.

Fahlander

Wahlin

Almkvist

Bäckman

. Cognitive functioning in Alzheimer’s disease and vascular dementia: further evidence for similar patterns of deficits. J Clin Exp Neuropsychol. 2002;24(6):734–744.

13.

Vuorinen

Laine

Rinne

. Common pattern of language impairment in vascular dementia and in Alzheimer Disease. Alzheimer Dis Assoc Disord. 2000;14(2):81–86.

14.

Grossman MD’Esposito

Hughes

. Language comprehension profiles in Alzheimer’s disease, multi-infarct dementia, and frontotemporal degeneration. Neurology. 1996;47(1):183–189.

15.

Powell

Cummings

Hill

Benson

. Speech and language alterations in multi-infarct dementia. Neurology. 1988;38(5):717–719.

16.

Chan

Lim

Sahadevan

. Stage-independent and stage-specific phenotypic differences between vascular dementia and Alzheimer’s Disease. Dement Geriatr Cogn Disord. 2008;26(6):513–521.

17.

Forbes

Venneri

. A case for case: handling letter case selection in written spelling. Neuropsychologia. 2003;41(1):16–24.

18.

Graham

. Dysgraphia in dementia. Neurocase. 2000;6:365–376.

19.

McKeith

Dickson

Lowe

. Diagnosis and management of dementia with Lewy bodies: third report of the DLB Consortium. Neurology. 2005;65(12):1863–1872.

20.

Ferman

Smith

Boeve

. Neuropsychological differentiation of dementia with Lewy bodies from normal aging and Alzheimer’s disease. Clin Neuropsychol. 2006;20(4):623–636.

21.

Gross

McMillan

Chandrasekaran

. Sentence processing in Lewy body spectrum disorder: the role of working memory. Brain Cogn. 2012;78(2):85–93.

22.

Ash

McMillan

Gross

. Impairments of speech fluency in Lewy body spectrum disorder. Brain Lang. 2012;120(3):290–302.

23.

Ash

McMillan

Gross

. The organization of narrative discourse in Lewy body spectrum disorder. Brain Lang. 2011;119(1):30–41.

24.

Mesulam

Wieneke

Thompson

Rogalski

Weintraub

. Quantitative classification of primary progressive aphasia at early and mild impairment stages. Brain. 2012;135(pt 5):1537–1553.

25.

Gorno Tempini

Hillis

Weintraub

. Classification of primary progressive aphasia and its variants. Neurology. 2011;76(11):1006–1014.

26.

Lin

O’Connor

Rossom

Perdue

Eckstrom

. Screening for cognitive impairment in older adults: a systematic review for the U.S. Preventive Services Task Force. Ann Intern Med. 2013;159(9):601–612.

27.

Folstein

McHugh

. “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12(3):189–198.

28.

Nasreddine

Phillips

Bédirian

. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc. 2005;53(4):695–699.

29.

Rosen

Mohs

Davis

. A new rating scale for Alzheimer’s disease. Am J Psychiatry. 1984;141(11):1356–1364.

30.

American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 5th ed. Arlington, VA: American Psychiatric Association; 2013.

31.

Osher

Wicklund

Rademaker

Johnson

Weintraub

. The mini-mental state examination in behavioral variant frontotemporal dementia and primary progressive aphasia. Am J Alzheimers Dis Other Demen. 2007;22(6):468–473.

32.

Mathuranath

Nestor

Berrios

Rakowicz

Hodges

. A brief cognitive test battery to differentiate Alzheimer’s disease and frontotemporal dementia. Neurology. 2000;55(11):1613–1620.

33.

Mioshi

Dawson

Mitchell

Arnold

Hodges

. The Addenbrooke’s Cognitive Examination Revised (ACE-R): a brief cognitive test battery for dementia screening. Int J Geriatr Psychiatry. 2006;21(11):1078–1085.

34.

Leyton

Hornberger

Mioshi

Hodges

. Application of Addenbrooke’s cognitive examination to diagnosis and monitoring of progressive primary aphasia. Dement Geriatr Cogn Disord. 2010;29(6):504–509.

35.

Helm-Estabrooks

. Cognitive Linguistic Quick Test. San Antonio, TX: PsychCorp; 2001.

36.

Enderby

Wood

Wade

Hewer

. The Frenchay Aphasia Screening Test: a short, simple test for aphasia appropriate for non-specialists. Int Rehabil Med. 2009;8(4):166–170.

37.

Nakase-Thompson

Manning

Sherer

Yablon

Gontkovsky

Vickery

. Brief assessment of severe language impairments: initial validation of the Mississippi aphasia screening test. Brain Inj. 2005;19(9):685–691.

38.

Trochim

. The Research Methods Knowledge Base, 2nd Edition. Cincinnati, OH: Atomic Dog Publishing; 2000.

39.

Marczinski

Kertesz

. Category and letter fluency in semantic dementia, primary progressive aphasia, and Alzheimer’s disease. Brain Lang. 2006;97(3):258–265.

40.

Teichmann

Kas

Boutet

. Deciphering logopenic primary progressive aphasia: a clinical, imaging and biomarker investigation. Brain. 2013;136(pt 11):3474–3488.

41.

Rohrer

Rossor

Warren

. Syndromes of nonfluent primary progressive aphasia: a clinical and neurolinguistic analysis. Neurology. 2010;75(7):603–610.

42.

Neils-Strunjas

Groves-Wright

Mashima

Harnish

. Dysgraphia in Alzheimer’s disease: a review for clinical and research purposes. J Speech Lang Hear Res. 2006;49(6):1313–1330.

43.

Macoir

Bernier

. Is surface dysgraphia tied to semantic impairment? Evidence from a case of semantic dementia. Brain Cogn. 2002;48(2-3):452–457.

44.

Sepelyak

Crinion

Molitoris

. Patterns of breakdown in spelling in primary progressive aphasia. Cortex. 2011;47(3):342–352.

45.

Faria

Crinion

Tsapkini

. Patterns of dysgraphia in primary progressive aphasia compared to post-stroke aphasia. Behav Neurol. 2013;26(1-2):21–34.

46.

Rodriguez-Ferreiro

Martinez

Perez-Carbajal

Cuetos

. Neural correlates of spelling difficulties in Alzheimer’s disease. Neuropsychologia. 2014;65:12–17.

47.

Turner

Kenyon

Trojanowski

Gonatas

Grossman

. Clinical, neuroimaging, and pathologic features of progressive nonfluent aphasia. Ann Neurol. 1996;39(2):166–173.

48.

Brambati

Ogar

Neuhaus

Miller

Gorno-Tempini

. Reading disorders in primary progressive aphasia: a behavioral and neuroimaging study. Neuropsychologia. 2009;47(8-9):1893–1900.

49.

Amici

Brambati

Wilkins

. Anatomical correlates of sentence comprehension and verbal working memory in neurodegenerative disease. J Neurosci. 2007;27(23):6282–6290.

50.

Rogers

Friedman

. The underlying mechanisms of semantic memory loss in Alzheimer’s disease and semantic dementia. Neuropsychologia. 2008;46(1):12–21.

51.

Snodgrass

Vanderwart

M. A

standardized set of 260 pictures: Norms for name agreement, image agreement, familiarity, and visual complexity. J Exp Psychol Hum Learn. 1980;6(2):174–215.

52.

Bonin

Peereman

Malardier

Méot

Chalard

. A new set of 299 pictures for psycholinguistic studies: French norms for name agreement, image agreement, conceptual familiarity, visual complexity, image variability, age of acquisition, and naming latencies. Behav Res Methods Instrum Comput. 2003;35(1):158–167.

53.

New

. Lexique 3: Une nouvelle base de données lexicales. Conference proceeding: Traitement Automatique des Langues Naturelles, Louvain, Belgique, 2006.

54.

Desrochers

Bergeron

. Valeurs de fréquence subjective et d’imagerie pour un échantillon de 1,916 substantifs de la langue française. Can J Exp Psychol. 2000;54(4):274–325.

55.

McKhann

Knopman

Chertkow

. The diagnosis of dementia due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011;7(3):263–269.

56.

Kaplan

Goodglass

Weintraub

. Boston Naming Test. Philadelphia, PA: Lee & Febiger; 1983.

57.

Nespoulous

Lecours

Lafond

. Protocole Montréal-Toulouse d’examen linguistique de l’aphasie. édition revisée par Béland

Giroux

ed. Isbergues, France: Ortho-Edition; 1992.

58.

Joanette

Ska

Côté

. Protocole Montreal d’Evaluation de la Communication MEC. Isbergues, France: Ortho Editions; 2004.

59.

Wechsler

, Wechsler Memory Scale—Fourth Edition . San Antonio, TX: The Psychological Corporation; 2009.

60.

Howard

Patterson

. Pyramids and Palm Trees: A Test of Semantic Access from Pictures and Words. Bury St. Edmunds, England: Thames Valley Test Company; 1992.

61.

Cortina

. What is coefficient alpha? An examination of theory and applications. J Appl Psychol. 1993;78(1):98–104.

62.

Crawford

Garthwaite

. Percentiles please: the case for expressing neuropsychological test scores and accompanying confidence limits as percentile ranks. Clin Neuropsychol. 2009;23(2):193–204.

63.

Benton

Sivan

Hamsher

Varney

Spreen

. Contributions to Neuropsychological Assessment: A Clinical Manual. 2nd ed. New York, NY: Oxford University Press; 1994.

64.

Hebben

Milberg

. Essentials of Neuropsychological Assessment. 2nd ed. Chichester, UK: John Wiley & Sons Ltd; 2009.