Abstract
The Alzheimer’s Disease Assessment Scale–Cognitive Subscale (ADAS-Cog) was developed in the 1980s to assess the level of cognitive dysfunction in Alzheimer’s disease. Advancements in the research field have shifted focus toward pre-dementia populations, and use of the ADAS-Cog has extended into these pre-dementia studies despite concerns about its ability to detect important changes at these milder stages of disease progression. If the ADAS-Cog cannot detect important changes, our understanding of pre-dementia disease progression may be compromised and trials may incorrectly conclude that a novel treatment approach is not beneficial. The purpose of this review was to assess the performance of the ADAS-Cog in pre-dementia populations, and to review all modifications that have been made to the ADAS-Cog to improve its measurement performance in dementia or pre-dementia populations. The contents of this review are based on bibliographic searches of electronic databases to locate all studies using the ADAS-Cog in pre-dementia samples or subsamples, and to locate all modified versions. Citations from relevant articles were also consulted. Overall, our results suggest the original ADAS-Cog is not an optimal outcome measure for pre-dementia studies; however, given the prominence of the ADAS-Cog, care must be taken when considering the use of alternative outcome measures. Thirty-one modified versions of the ADAS-Cog were found. Modification approaches that appear most beneficial include altering scoring methodology or adding tests of memory, executive function, and/or daily functioning. Although modifications improve the performance of the ADAS-Cog, this is at the cost of introducing heterogeneity that may limit between-study comparison.
INTRODUCTION
The Alzheimer’s Disease (AD) Assessment Scale-Cognitive Subscale (ADAS-Cog) is considered the gold standard for assessing the efficacy of antidementia treatments. Since its development in the 1980s, theoretical and clinical developments have shifted research interest earlier in the natural history to include pre-dementia syndromes, especially mild cognitive impairment (MCI) [1 –6]. It is now thought that intervening to slow or stop the progression of disease will be more effective than waiting until severe neuropathology and dysfunction have developed [1, 4 , 7–9]. Thus, many research studies, both observational and experimental, are being conducted in pre-dementia populations, whereby cognitive ability may range from normal cognition (NC) to MCI [5 , 10–14]. The ADAS-Cog is often employed in these pre-dementia studies; however, the ADAS-Cog was developed for use in studies of dementia where cognitive impairments are more severe. Concerns have been raised about whether the ADAS-Cog is able to detect important changes at earlier stages of disease progression [2 , 16]. If not, studies may miss disease progression or regression, or miss potential treatment benefits. Using an outcome measure in a population or context where it does not perform well threatens both internal and external study validity.
This narrative review has two main objectives: first, to review the performance of the ADAS-Cog in subjects with pre-dementia status such as MCI, and second, to assess responsiveness of modified ADAS-Cog versions in dementia and pre-dementia populations.
DEVELOPMENT OF THE ALZHEIMER’S DISEASE ASSESSMENT SCALE
The ADAS was originally designed as a rating scale to assess the severity of cognitive and non-cognitive dysfunction from mild to severe AD [17]. Twenty-one tasks were selected from a pool of forty candidate tasks based on reliability and validity comparisons between 27 subjects with AD and 28 subjects with NC [17]. The full ADAS takes about 45 minutes to administer, and is scored from 0 to 150 by summing the number of errors made on each task so that higher scores indicate worse performance [17].
The ADAS comprises two subscales. The non-cognitive subscale (ADAS-Noncog) includes 10 tasks, scored from 0 to 50, which consider mood and behavioral changes [17]. The ADAS-Noncog is not widely used and will not be reviewed further.
The cognitive subscale (ADAS-Cog) includes 11 tasks that include both subject-completed tests and observer-based assessments [17]. Together these tasks assess the cognitive domains of memory, language, and praxis [17]. Specific tasks (Table 1) include Word Recall, Naming Objects and Fingers, Commands, Constructional Praxis, Ideational Praxis, Orientation, Word Recognition, and Language [17].
Summary of ADAS-cog tasks
Initially the two memory tasks (Word Recall and Word Recognition) were viewed as a separate memory subscale [17], but this distinction is not commonly seen in research studies. Usually studies administer all 11 tasks and score them as a single scale from 0 to 70; we use “ADAS-Cog-11” to refer to this 11-task version.
In the original development sample, concurrent criterion validation was suggested by statistically significant correlations with well-established outcome measures, including the Sandoz Clinical Assessment-Geriatric, the Memory-Information Test, and the Dementia Rating Scale [17]. Point-biserial correlations further demonstrated that subjects with AD had significantly higher scores on the ADAS-Cog-11 (r = 0.754, p < 0.0001) as well as on all individual ADAS-Cog-11 tasks (all p < 0.0001) than subjects with NC [17]. The mean (SD) ADAS-Cog score for the 27 subjects with AD was 19.3 (12.1) [17].
ASSESSMENT OF THE ADAS-COG 11 IN PRE-DEMENTIA POPULATIONS
Objective 1 literature search
We performed bibliographic searches to locate all studies (e.g., psychometric analyses, observational studies, clinical trials) that used the ADAS-Cog-11 with a sample or subsample of older adults with pre-dementia syndromes or normal cognitive abilities. MEDLINE, Embase, Cinahl, PsychINFO, PsychTests, and Proquest Psychology were searched using subject headings and key words related to ADAS-Cog (full and abbreviated spelling), pre-dementia, cognition, mild cognitive impairment, subjective cognitive impairment, and normal cognition. Original searches were performed in June 2016 without date restriction, and updated in January 2018. Citation lists of relevant articles were considered. Studies were excluded if results were not presented separately for pre-dementia subsamples (e.g., if subjects with dementia were also included in the study), or if a non-English version of the ADAS-Cog-11 was used. Results from included studies were organized according to the type of information they provide about the ADAS-Cog-11, as presented in the following sections.
Ceiling effects
Seven of the eleven ADAS-Cog-11 tasks demonstrate severe ceiling effects in MCI and NC samples (Table 2) [1 , 19]. Two tasks show milder ceiling effects [1 , 19]. Accordingly, 84% of errors made by subjects with NC and 71% of errors made by subjects with MCI occur on Word Recall and Word Recognition [18]. These two tasks have also been ranked as the most difficult ADAS-Cog-11 tasks by Item Response Theory (IRT) analyses [20].
ADAS-Cog 11 ceiling effects
, no ceiling effect; 
, mild ceiling effect; 
, severe ceiling effect; ADCS, Alzheimer’s Disease Cooperative Study; ADNI, Alzheimer’s Disease Neuroimaging Initiative; BCM-ADC, Baylor College of Medicine Alzheimer’sDisease Study; N/A, not available; Q1,Word Recall; Q2, Naming Objects and Fingers; Q3, Commands; Q4, Constructional Praxis; Q5, Ideational Praxis; Q6, Orientation; Q7, Word Recognition; Q8, Language; Q9, Comprehension of Spoken Language; Q10,Word Finding Difficulty; Q11, Remembering Test Instructions; SIU-NAS, Southern Illinois University Normal Aging Study. All studies included participants with mild cognitive impairment except BCM-ADC [3], which included exclusively cognitively normal controls.
Information content
IRT analyses suggest the most informative ADAS-Cog-11 tasks for assessing cognitive ability in MCI are Word Recall, Orientation, Word Recognition, and Naming Objects and Fingers [7]. The remaining tasks are more informative for people with more severe impairment than MCI [7]. Furthermore, an in-depth evaluation of the Word Recall Task suggests the Pole response item is abnormally easy as it has a higher recall probability than other response items for NC, MCI, and AD groups [21]. These types of in-depth analyses have not been published for other individual ADAS-Cog-11 tasks.
Invariances
The ADAS-Cog-11 as a whole and all individual tasks have shown measurement invariance with respect to age and education in MCI samples, meaning people with different ages or education levels should score similarly on the ADAS-Cog-11 if their true cognitive abilities are equal [18]. Measurement invariance to gender has also been found for the ADAS-Cog-11 in MCI samples [18]. In samples with NC, the ADAS-Cog-11 showed measurement invariance with respect to gender and education, but not age [3 , 22]. In these NC samples, age was significantly correlated with total ADAS-Cog-11 scores as well as with the Word Recall task [3 , 22]. Altogether these results suggest that the only threat to measurement invariance is the age of subjects with NC, however it is important to note that these studies did not cover the full range of possible education levels.
Reliability
Significant variance in administration procedures and materials used for the ADAS-Cog-11 across clinical trials has been found, which threatens inter-observer, intra-observer, and test-retest reliability [23]. Learning effects may also be a concern as Herholz et al. (2011) found a statistically significant decline in ADAS-Cog-11 scores in subjects whom otherwise did not appear to be progressing in symptoms [24].
Baseline discrimination
The ADAS-Cog-11 has demonstrated the ability to discriminate between the diagnostic categories of NC, MCI, and AD at one point in time, and the scores for subjects with NC are lower than for those with MCI and subsequently for those with AD [18 , 25–31]. All 11 tasks have shown statistically significant differences between NC and MCI subgroups, and all but three tasks (Commands, Ideational Praxis, Language) have demonstrated significantly higher scores in AD than MCI subgroups [18]. Furthermore, statistically significant differences in the scores for three tasks have been found between subjects with MCI and none versus one versus two APOE ɛ4 alleles, a genetic risk factor for AD [32].
Responsiveness to disease progression
Rosen et al. (1984) found a statistically significant worsening on ADAS (p = 0.02) and ADAS-Cog-11 (p = 0.01) scores over a 12-month period for ten subjects with AD but not for ten subjects with NC (all p > 0.05) in their original development sample [17]. Further studies have also detected worsening of ADAS-Cog-11 scores over time [2 , 34]. Statistically significant differences have been found between the magnitude of change scores for subjects with NC, MCI, and AD [26 , 33]. In general, subjects with AD have the largest change scores, followed by those with MCI, and then those with NC. Importantly, the magnitude of change scores detected in NC and MCI samples is very small [2 , 33]. At the item level, all individual ADAS-Cog-11 tasks have been found to have smaller Standardized Response Means (SRM = mean difference score/SD of difference scores) than the ADAS-Cog-11 total score [15]. The three tasks demonstrating the largest SRM were Word Recall, Orientation, and Word Recognition [15]. The magnitude of 12-month and 24-month change scores for six ADAS-Cog-11 tasks produced smaller change scores in MCI than AD groups [15]. A separate study found groups of subjects with NC compared to MCI had statistically significant different 12-month change scores on the Word Recall and Word Recognition tasks [26].
Responsiveness to exposure-based differences in cognitive ability
We used a rudimentary approach to assess the performance of the ADAS-Cog-11 as an outcome measure for tests of association between an exposure or intervention and cognition in pre-dementia samples, by summarizing when the ADAS-Cog-11 found statistically significant results for hypothesized associations in comparison with other outcome measures.
Responsiveness to group-level between-person differences in observed level of disease severity based on exposure status
Twenty-six studies made forty-nine statistical comparisons to assess cross-sectional associations between exposure status and ADAS-Cog-11 scores in older adults with pre-dementia status, as summarized in Supplementary Table 1 [26 , 34–57]. There were twenty-three (47% ) statistically significant associations between exposure status and ADAS-Cog-11 scores. Considering NC and MCI analyses separately, 18% and 54% of statistical comparisons were significant, respectively. There were seventeen (35% ) non-statistically significant associations between the ADAS-Cog-11 and an exposure where any other cognitive or brain imaging outcome measures used to assess the same association also produced statistically non-significant results; this result occurred for 45% and 31% of NC and MCI analyses, respectively. The ADAS-Cog-11 produced statistically non-significant result for nine (18% ) potential associations that were statistically significant for at least one other cognitive or brain imaging outcome measure, which included 36% of NC and 14% of MCI statistical comparisons.
Responsiveness to group-level between-person differences of within-person observed change in those estimated to be different based on baseline exposure status
Twenty-five studies were found which made seventy-one statistical comparisons to test for associations between baseline exposure status and change in ADAS-Cog-11 scores, as summarized in Supplementary Table 2 [19 , 57–61]. There were forty-nine (69% ) statistically significant associations between baseline exposure status and ADAS-Cog-11 scores over time, which can be broken down into 63% and 75% of NC and MCI statistical comparisons, respectively. There were nineteen (27% ) statistically non-significant associations between baseline exposure status and ADAS-Cog-11 scores over time whereby any other cognitive or brain imaging outcome measures also produced statistically non-significant results; this included 37% and 17% of NC and MCI statistical comparisons, respectively. The ADAS-Cog-11 produced a further three (4% ) statistically non-significant results for associations found to be statistically significant by at least one other cognitive or brain imaging outcome measure across groups, whereby 0% of NC and 8% of MCI statistical comparisons were non-significant.
Responsiveness to treatment effects
Twenty clinical trials using the ADAS-Cog-11 as an outcome measure in pre-dementia samples were found (Supplementary Table 3) [62 –81]. The ADAS-Cog-11 was able to detect ten (48% ) statistically significant treatment effects. The ADAS-Cog-11 did not find a statistically significant effect for eleven interventions, four (19% ) of which demonstrated a statistically significant treatment effect for at least one other outcome measure. Note that only results from longest follow-up periods are reviewed, and subgroup analyses only are presented when the primary analyses did not include the ADAS-Cog-11 in a sample composed completely of subjects with pre-dementia levels of disease severity.
Sample size estimates from simulation studies
Four studies estimated the sample size needed to detect a treatment effect using the ADAS-Cog-11 in a clinical trial of pre-dementia syndromes. As summarized in Supplementary Table 4, the ADAS-Cog-11 was never (0% ) the outcome measure requiring the smallest sample size [82 –85]. A separate study found that increasing the proportion of APOE ɛ4 allele carriers in clinical trial simulations, a method employed to try to increase the level of impairment of a sample, did not lead to meaningful increases in power to detect a treatment effect with the ADAS-Cog-11 [41]. Furthermore, the ADAS-Cog-11 failed to produce statistically significant treatment effects in several situations where one was hypothesized to be present based on other indicators of disease progression [42].
Summary of ADAS-Cog-11 performance in pre-dementia studies
ADAS-Cog-11 scores in pre-dementia populations are driven primarily by the Word Recall and Word Recognition tasks, and age may influence scores for older adults with NC. Despite this, ADAS-Cog-11 scores generally appear able to detect differences in cognitive ability in groups defined by an exposure that is expected to be associated with cognitive ability (for example, brain glucose metabolism), although the magnitude of the differences detected tends to be small. Responsiveness of the ADAS-Cog-11 to treatment effects appears low compared to other global outcome measures and compared to outcome measures designed to assess subdomains of cognition or other aspects of dementia and pre-dementia syndromes. Nonetheless, caution must be maintained when interpreting these findings because an in-depth exploration of whether there truly should be an association between cognition or disease severity and any given exposure or treatment, and the potential magnitude and direction of these associations, was not explored. Overall, the ADAS-Cog-11 seems able to provide a measure of disease severity in pre-dementia syndromes, but there is room for improvement.
OUTCOME MEASURE MODIFICATION
There are two main approaches to modifying, or attempting to improve, a pre-existing outcome measure: altering scoring methodology and adding additional test items [25]. These can be used alone or in combination [25].
Objective 2 literature search
To locate all modified versions of the ADAS-Cog-11, electronic databases (MEDLINE, Embase, Cinahl, PsychINFO, PsychTests, and Proquest Psychology) were searched using subject heading and key word searches related to ADAS-Cog (full and abbreviated spelling), dementia, pre-dementia, and cognition. Original searches were performed without date restriction in June 2016, and updated in January 2018. Citation lists of relevant articles were considered. Articles were included if they presented an outcome measure that contains at least one task of the original ADAS-Cog-11 and that is intended for use in populations with dementia or pre-dementia syndromes.
Figure 1 presents the results of our search for modified versions of the ADAS-Cog-11 chronologically by publication date. Below, the different versions are organized by modification strategy; within each subsection measures are presented chronologically by publication date. Following a brief introduction and description of each modification, we summarize available information on the performance of that measure compared to the ADAS-Cog-11. The basis for these comparisons are summarized in Supplementary Table 5, which contains a chart with results of analyses that assessed responsiveness to baseline discrimination, to disease progression, or to treatment effects. We provide a qualitative comparison of measures that rely on the same modification strategy at the end of each subsection.
Timeline of ADAS-Cog 11 modifications.
Altering scoring methodology
This section reviews modifications that contain the 11 original tasks but derive final scores by an alternative method to straight summation of points across tasks. Note that the removal of tasks is analogous to multiplying the removed task score by zero and would therefore constitute a scoring modification.
ADAS-Rasch
Wouters et al. (2008) identified three problematic aspects of the ADAS-Cog-11 scoring methodology [86]. Specifically, they found ADAS-Cog-11 tasks do not have equal measurement precision, several subtask item response categories are disordered in terms of difficulty, and a difference of X points at the low end of the scoring range does not equal the same amount of difference in cognitive ability as a difference of X points at the higher end of the scoring range [86].
Wouters et al. (2008) used Rasch analysis to develop a scoring method that overcomes the first two limitations. In brief, response categories with the same level of difficulty for each task on the ADAS-Cog-11 were collapsed so that the ADAS-Rasch has hierarchically ordered categories, and each task is weighted by its measurement precision [86]. The total possible score for each task of the ADAS-Rasch is the product of the number of different categories of difficulty present for the items of that task and the weight assigned to the task: Word Recall (total possible points for ADAS-Cog-11 = 10 versus total possible points for ADAS-Rasch = 12), Naming Objects and Fingers (5 versus 6), Commands (5 versus 8), Constructional Praxis (5 versus 4), Ideational Praxis (5 versus 6), Orientation (8 versus 6), Word Recognition (12 versus 3), Remembering Test Instructions (5 versus 5), Language (5 versus 5), Word Finding Difficulty (5 versus 4), Comprehension of Spoken Language (5 versus 5), total score (70 versus 64) [86]. ADAS-Rasch scores are backwards-compatible to a Classical Test Theory-derived ADAS-Cog-11 sum score.
The ADAS-Rasch was developed from baseline data of the placebo arms of three clinical trials that included 706 subjects with mild to moderate dementia [86]. External criterion validation analyses in a similar 456 subject trial included correlating ADAS-Rasch scores with ADAS-Cog-11 (r = 0.93) and MMSE (r = –0.72) scores [86].
Performance summary. The ADAS-Rasch appears to demonstrate baseline discrimination, although statistical tests of scores between groups have not been performed [25, 35]. Responsiveness to disease progression and treatment effects was worse for the ADAS-Rasch than the ADAS-Cog-11 in NC, MCI, and AD groups [25, 35].
ADAS-Cog-5-Subset and ADAS-Cog-6-Subset
Ihl et al. (2012) used subsetting analysis to develop subsets of ADAS-Cog-11 tasks based on the ability of individual tasks to detect a treatment effect in three 24 week randomized controlled trials of 855 subjects with mild to moderate AD [87]. First, they calculated pre-post difference scores for all ADAS-Cog-11 tasks [87]. A binary (0,1) response variable was created and coded 1 if the pre-post difference on the task score was less than or equal to 0 (treatment responder; did not get worse) and coded 0 if the pre-post difference was greater than 0 (treatment non-responder; subject got worse) [87]. Importantly, “responders” were defined as those who showed no worsening on any task of a given subset of tasks over the course of the study; responders could be in the placebo or treatment group [87]. Second, an algorithm was used to identify subsets of ADAS-Cog-11 tasks that could identify groups of responders, and Wilcoxon rank-sum tests were used to find subsets with statistically significant differences in the proportion of responders between treatment and placebo groups [87]. The subset with the lowest p-value was selected as the collection of ADAS-Cog-11 tasks with the most potential for detecting a treatment response [87].
The primary result of this analysis was the ADAS-Cog-5-Subset: Ideational Praxis, Remembering Test Instructions, Language, Comprehension, and Word Finding Difficulty [87]. Internal consistency (Chronbach’s alpha = 0.81) was close to that of the ADAS-Cog-11 (Chronbach’s alpha = 0.82) [87].
Tasks not selected for the primary subset were combined into the ADAS-Cog-6-Subset: Word Recall, Naming Objects and Fingers, Commands, Constructional Praxis, Orientation, and Word Recognition [87].
Performance summary. Both ADAS-Cog-5 and 6 subsets were more responsive than the ADAS-Cog-11 to treatment effects in AD [87]. The 5-Subset was the most sensitive for detecting a memantine treatment response in mild AD, and the 6-Subset was the most sensitive for detecting a memantine treatment response in moderate AD [87].
ADAS-Cog-IRT
Balsis et al. (2012) developed an IRT scoring methodology for the ADAS-Cog-11 using data from 1,240 subjects with varying levels of dementia severity [88]. Although their primary focus was identifying limitations to traditional ADAS-Cog-11 scoring methodology, Balsis et al. (2012) also showed how using IRT to model a subject’s score along with the difficulty of individual items can increase precision for estimating cognitive ability [88].
Verma et al. (2015) found, using multidimensional IRT on data from a large sample of older adults with NC to AD, that the ADAS-Cog-11 is most appropriately modelled using three latent factors corresponding to the cognitive domains of memory (Word Recall, Orientation, and Word Recognition tasks), language (Naming Objects and Fingers, Language, Comprehension of Spoken Language, Word Finding Difficulty, and Remembering Test Instructions tasks), and praxis (Commands, Constructional Praxis, and Ideational Praxis tasks) [89]. The finding that the ADAS-Cog-11 was best modelled using multiple latent cognitive domains suggests that the unidimensional assumption used in Classical Test Theory may not be appropriate for assessing the ADAS-Cog-11.
Verma et al. (2015) evaluated their multidimensional IRT scoring methodology for the ADAS-Cog-11 using data from the treatment arms of 11 clinical trials that enrolled older adults with AD [89]. Their ADAS-Cog-IRT uses ICCs from patient responses on the ADAS-Cog-11 to provide an assessment of cognitive impairment based on maximum likelihood estimation [89]. Differential Item Functioning analyses were used to adjust item slopes and intercepts so that patient characteristics other than cognitive ability do not cause large variations in scores [89]. To maintain non-negative final scores, summary scores for the memory, language, and praxis domains were linearly scaled [89]. This ADAS-Cog-IRT scoring methodology demonstrated good accuracy as assessed by root mean squared error of observed compared to predicted ADAS-Cog-11 scores (6.05 points) [89]. Precision was assessed using item information functions. Memory tasks showed good precision across the entire range of memory impairment, however precision for measuring language and praxis impairment was suboptimal at lower levels of cognitive ability [89].
The application of IRT scoring methodology to the ADAS-Cog-11 provided the same score to all subjects with the same cognitive ability in contrast to traditional scoring whereby two subjects with equal cognitive ability can score differently, and two subjects with different scores on the ADAS-Cog-11 can have equal underlying cognitive ability [89]. The increased precision in IRT scoring methodology is expected to improve responsiveness to baseline discrimination.
Performance summary. The ADAS-Cog-IRT demonstrated greater responsiveness to moderately large treatment effects in AD than the ADAS-Cog-11 [89].
ADAS-Cog-3
Podhorna et al. (2016) removed eight tasks from the ADAS-Cog-11 that demonstrate ceiling effects in MCI, resulting in the three-task ADAS-Cog-3: Word Recall, Orientation, and Word Recognition [2]. The ADAS-Cog-3 assesses only memory and has a scoring range of 0 to 30 [2].
Performance summary. The ADAS-Cog-3 appeared able to discriminate between groups of subjects with MCI and mild AD, although statistical tests were not performed [2]. Responsiveness to disease progression in MCI or mild AD was not superior to the ADAS-Cog-11 [2].
Overview of scoring modifications approaches
Advantages of the ADAS-Rasch and ADAS-Cog-3 are that they are backwards compatible to the ADAS-Cog-11 and rely on an intuitive scoring method; however, neither consistently demonstrated better responsiveness than the ADAS-Cog-11. Both ADAS-Cog-Subsets and the ADAS-Cog-IRT demonstrated better responsiveness than the ADAS-Cog-11 to treatment effects, but responsiveness to baseline discrimination and disease progression was not evaluated. Nonetheless, the more rigorous evaluation of the ADAS-Cog-IRT and fact this measure has a clearly interpretable final score, may make it the preferable ADAS-Cog-11 modification based solely on statistical modification.
Additional task content
This section reviews outcome measures that contain the 11 original tasks of the ADAS-Cog-11 as well as at least one additional task. Although the additional tasks will increase the scoring range, final scores are the simple sum of tasks as for the ADAS-Cog-11.
ADAS-Cog-13
Mohs et al. (1997) identified cognitive domains hypothesized to be important treatment targets of antidementia drugs that are not assessed by the ADAS-Cog-11: attention and concentration, planning and executive function, verbal memory, nonverbal memory, and praxis [90]. From a pool of candidate tasks relevant for those domains, Mohs et al. (1997) made recommendations about which tasks to add to the ADAS-Cog-11 based on assessments of reliability, influence of age and education on change scores, learning effects over a one month re-test interval, range of dementia severity coverage, floor and ceiling effects, and ability to measure change over 12 months for 207 subjects with cognitive abilities ranging from NC to moderately severe AD [90]. Final recommendations by Mohs et al. (1997) inform the ADAS-Cog-13, which includes all ADAS-Cog-11 items as well as a test of delayed word recall and a number cancellation or maze task. ADAS-Cog-13 scores range from 0 to 85.
Performance summary. The ADAS-Cog-13 appeared able to discriminate between groups of subjects with MCI and mild AD at one point in time, but statistical tests were not performed [2, 25]. The responsiveness of the ADAS-Cog-13 to disease progression was better than that of the ADAS-Cog-11 in subjects with AD but had similar or only slightly better responsiveness in subjects with pre-dementia syndromes to disease progression [2 , 25]. Responsiveness to treatment effects in MCI was better for the ADAS-Cog-13 than the ADAS-Cog-11 [15, 25].
Vascular Dementia Assessment Scale (VaDAS)
To address the need for a primary outcome measure for clinical trials in vascular dementia (VaD), Ferris et al. (1999) suggested the ADAS-Cog-11 as a starting point because many of the cognitive domains affected by VaD such as memory, attention, processing speed, visuospatial function, language, executive function, and abstraction are also affected in AD [91]. The original VaDAS included items suggested by Mohs et al. (1997) for the ADAS-Cog-13 [90, 91]. An updated version added items to access frontal lobe function: all tasks from the ADAS-Cog-11 with a delayed recall portion added to the Word Recall task (memory), two number cancellation tasks (attention), a maze (executive function), symbol digit modalities (attention/concentration), backwards digit span (working memory), and animal category retrieval (verbal fluency) [92].
Performance summary. Evaluation of the VaDAS was left for future clinical trials [92].
ADAS-Cog-12
The ADAS-Cog-12 contains a delayed word recall task, an addition to the Word Recall task whereby after a period of time (delay) a subject is given three trials to recall as many of the ten words from the Word Recall task as possible [9]. The number of words not recalled (errors) is added to the score from the other 11 tasks, giving a final ADAS-Cog-12 score from 0 to 80.
Lowe et al. (2015) used IRT methods with 788 subjects with diagnoses ranging from Subjective Cognitive Impairment (SCI) to severe stages of AD to determine where along the continuum of disease progression the Delayed Word Recall task provides the most information about cognitive ability, and to determine if it alters the measurement precision of the ADAS-Cog-11 [93]. Item Characteristic Curves (ICC)s showed that in general, memory tasks of the ADAS-Cog-12 are the most sensitive to earlier stages of disease progression, and the Delayed Word Recall task provides the most information in the mildest range of cognitive impairment with its ICC being significantly different from that of the other memory tasks [93]. The Delayed Word Recall task does not however have much sensitivity for more severe cognitive dysfunction such as that seen with AD [93]. Floor effects (10 errors) on the Delayed Word Recall task were seen for 9% of the MCI group and 52% of the AD group at baseline [93]. Test information curves show that both the ADAS-Cog-11 and ADAS-Cog-12 are maximally precise around mild to moderate AD, but the ADAS-Cog-12 holds more information about underlying cognitive impairment in earlier stages of disease progression [28]. The two versions have similar precision in more severe stages of cognitive impairment, namely AD [28].
Performance summary. The ADAS-Cog-12 has demonstrated the ability to discriminate between groups of subjects with MCI and AD, as well as between MCI subtypes [28, 94]. The ADAS-Cog-12 demonstrated more responsiveness to disease progression and treatment effects in MCI than the ADAS-Cog-11 [28]. Further along the disease continuum, responsiveness to disease progression and treatment effects of the two versions were comparable [28].
Computerized ADAS-Cog
The National Institute on Aging funded the development of a computerized version of the ADAS-Cog (cADAS-Cog) to try to increase consistency and decrease errors among administrators of the ADAS-Cog [94]. The cADAS-Cog incorporates all ADAS-Cog-11 items plus Delayed Recall, Number Cancellation, and Maze tasks in a Computerized Multiphasic Interactive Neurocognitive Dual Display System (CMINDS) [94]. A preliminary Perception Response Evaluation (PRE) module allows subjects to practice using the CMINDS interface, and also ensures that subjects have sufficient perceptual and response abilities to take the computerized test [94]. Next, the cADAS-Cog is administered on one monitor display while the administrator uses a second monitor to control the speed of the testing, provide repeated test instructions, and receive information on the subject’s progress throughout the test [94].
A sample of 88 subjects with mild to moderate AD were administered both the computerized and paper ADAS-Cog versions three times, four months apart [94]. Different versions were given on alternate time points, each one month apart [94]. Both versions took approximately 44 minutes to administer [94]. High concurrent criterion validity between the cADAS-Cog and paper version total scores, and all individual task scores, was suggested by ICCs (all p < 0.001), Pearson’s correlation coefficients (all p < 0.01), and paired sample t-tests of differences between intra-subject scores (all p > 0.10) [94]. High test-retest reliability was found over approximately five month (p < 0.001) and ten month periods (p < 0.001) [94]. Furthermore, the reliability across cADAS-Cog scores was significantly better than that of the paper administration method over five and ten month periods (5 month: mean cADAS-Cog ICC = 0.87, mean paper ICC = 0.80, t = 2.88, p < 0.02; 10 month: mean cADAS-Cog ICC = 0.83, mean paper ICC = 0.77, t = 2.54, p < 0.03) [94]. Agreement was also demonstrated with a Bland-Altman plot of the differences between total scores [94].
Performance summary. Tests of responsiveness were not performed.
Overview of task addition approaches
The ADAS-Cog-12 and ADAS-Cog-13 are both simple modifications that result in apparent improvements in responsiveness over the ADAS-Cog-11. Since the ADAS-Cog-13 is backwards compatible to the ADAS-Cog-12, which is backwards compatible with the ADAS-Cog-11, administering the ADAS-Cog-13 may be preferable as time and resources allow. Further evaluation is needed to assess whether the increased standardization in test administration promoted by the cADAS-Cog translates into an advantage in responsiveness over the ADAS-Cog-11. Since design and evaluation of the VaDAS was specific to VaD, use as a universal dementia or pre-dementia outcome measure is not recommended.
Altered scoring methodology and additional task content.
This section includes modifications of the ADAS-Cog-11 that combine scoring and content modification approaches, and therefore contain at least one ADAS-Cog-11 task and at least one additional task.
Test for the Early Detection of Dementia from Depression-Cognitive (TE4D-Cog)
The Test for the Early Detection of Dementia from Depression (TE4D) was initially developed in the German language to differentiate early dementia from depression [27]. Mahoney et al. (2005) modified the TE4D to screen for MCI in English-speaking populations (TE4D-Cog) [27]. The TE4D-Cog is scored from 0 to 45, and has eight tasks that together assess immediate recall, semantic memory, clock drawing, category fluency, orientation, and following commands (from ADAS-Cog 11) [27]. The TE4D-Cog was tested in a sample of 178 subjects with AD and 25 subjects with NC and was found to have good concurrent criterion validity with the ADAS-Cog-11 (r = –0.90, p < 0.001) and MMSE (r = 0.92, p < 0.001), high inter-rater reliability, and good internal consistency [27].
Performance summary. The TE4D-Cog demonstrated the ability to discriminate between NC and AD groups [27].
Pooled Index
Carusone et al. (2006) were the first to add measures of functionality to the ADAS-Cog-11 [95]. They used data from a clinical trial involving 101 subjects with mild to moderate AD, and combined six scales using a pooled index approach: ADAS-Cog-11, Geriatric Depression Scale, Dysfunctional Behaviour Rating Instrument (DBRI), MMSE, Activities of Daily Living (ADL), and DBRI frequency [95].
Performance summary. The Pooled Index was more responsive to treatment effects than the ADAS-Cog-11 in a clinical trial for mild to moderate AD [95].
ADAS-Tree
Llano et al. (2011) developed an alternative weighting scheme for scoring the ADAS-Cog-13 to identify subjects with MCI at high risk of converting to AD [1]. The rationale was to increase the efficiency of a clinical trial by using conversion from MCI to AD as an outcome, and then enrolling subjects with a particularly high risk of this conversion [1]. A second purpose of the ADAS-Tree, pertinent to our review, was to discriminate between subjects with different levels of disease severity at the start of a clinical trial [1].
To develop the ADAS-Tree, the Random Forests (RF) tree-based algorithm was used to derive weights for each task of the ADAS-Cog-13 based on their ability to discriminate between subjects with NC, MCI, and AD. Briefly, ten thousand bootstrap datasets were taken from baseline data of 229 subjects with NC, 397 subjects with MCI, and 193 subjects with AD [1]. The RF algorithm was applied in each dataset to develop a classification tree for NC, MCI, and AD diagnostic categories [1]. Each bootstrap dataset was the same size as the original sample, but because datasets were obtained using random sampling with replacement, about one third of the original sample was not selected for any given bootstrap, and some observations were sampled multiple times [1]. These excluded samples were used to obtain an estimate of predictive accuracy by comparing diagnoses predicted by the majority of classification trees (RF model) with original diagnoses [1]. Weights for each task of the ADAS-Cog-13 were derived by comparing the predictive accuracy of the RF model fit using the full ADAS-Cog-13 to the predictive accuracy of a RF model fit when one ADAS-Cog-13 task was replaced by noise, repeated for all tasks [1]. Tasks that led to the largest decreases in predictive accuracy when excluded were given the highest weights in the ADAS-Tree [1].
ADAS-Tree item weights were: 1.05 Word Recall, 0.38 Commands, 0 Construction, 1.17 Delayed Word Recall, 0.61 Naming, 0.13 Ideational Praxis, 1.13 Orientation, 0.41 Word Recognition, 0.54 Recall Instructions, 0.49 Spoken Language, 0.69 Word Finding, 0.39 Comprehension, 0.69 Number Cancellation [1].
Performance summary. The ADAS-Tree demonstrated responsiveness to baseline discrimination between NC, MCI, and AD diagnostic categories [1 , 35]. The ADAS-Tree also had better responsiveness than the ADAS-Cog-11 to disease progression and treatment effects for MCI and AD, but not NC samples [25, 35].
Touch Panel-Type Dementia Assessment Scale (TDAS)
Inoue et al. (2011) created a computerized version of a modified ADAS-Cog-11 that can be administered in 30 minutes and is scored from 0 to 101 [96]. This Touch Panel-Type Dementia Assessment Scale (TDAS) has a 14” touch panel display and includes the seven computerizable parts of the ADAS-Cog-11 (Word Recognition, Commands, Orientation, visual-spatial perception (modified Constructional Praxis), Naming Fingers, object recognition (modified Naming Objects), accuracy of the order of a process (modified Ideational Praxis)) as well as tests for money calculation and analog clock time recognition [96].
Thirty-four subjects with AD were administered both the TDAS and a paper version for concurrent criterion validation analyses (r = 0.69, p < 0.01) [96]. Kendall coefficients of concordance were calculated to assess agreement between six of the TDAS tasks and six of the paper ADAS-Cog-11 tasks [96]. Three tasks showed acceptable concordance [Word Recognition (0.57), Orientation (0.41), and Naming Objects and Fingers (0.32)], while three showed poor concordance [Following Commands, Constructional Praxis, Ideational Praxis (all Kendall’s coefficients < 0.3)] [96].
Performance summary. Responsiveness was not evaluated.
Computerized Adaptive Testing of the Cambridge Cognitive Examination-Plus (CAMCOG-Plus)
The Cambridge Cognition Examination (CAMCOG)-Plus comprises a battery of neuropsychological tests including the ADAS-Cog-11 [97]. Wouters et al. (2011) used Computerized Adaptive Testing (CAT) to administer the CAMCOG and CAMCOG-Plus to 41 subjects with NC, 21 subjects with MCI, and 22 subjects with dementia to see whether the CAT version maintains diagnostic accuracy while decreasing administration time [97].
The CAT procedure begins with a series of standard questions that an internal algorithm uses to estimate cognitive ability [97]. Each time a question is answered (i.e., during CAMCOG or CAMCOG-Plus administration) the algorithm updates the estimate of cognitive ability and uses the response to select the difficulty of the next question [97]. Correct responses lead to more difficult questions while incorrect responses result in easier ones [97]. Difficulty levels of test items were initially estimated using a one parameter logistic model [97]. The updating process is continued until 25 items are administered or a standard error of measurement corresponding to 90% reliability for cognitive ability is reached [97].
In the original sample an acceptable estimate of cognitive ability was reached using the CAT CAMCOG-Plus after administering 53% fewer items than the full test battery [97]. Time to administer was reduced by 54% [97]. The CAT CAMCOG-Plus demonstrated agreement with the paper CAMCOG-Plus (ICC 0.98, p < 0.001) and the paper CAMCOG (ICC 0.99, p < 0.001) [97]. Concurrent criterion validation was assessed between the CAMCOG-Plus and MMSE (Spearman’s rho = 0.80, p < 0.001) and Informant Questionnaire on Cognitive Decline in the Elderly (Spearman’s rho = –0.54, p < 0.025) [97].
Performance summary. Responsiveness was not evaluated.
ADAS-Cog-Plus (ADAS-Bifactor, ADAS-Plus-EF, ADAS-Plus-EF&FA)
Skinner et al. (2012) tried two strategies to modify the ADAS-Cog-13 for improved responsiveness to changes in MCI [25]. First, alternative weights were applied to tasks of the ADAS-Cog-13 based on IRT analyses, resulting in a bi-factor model that accounted for correlations between Word Recognition and Word Recall tasks, and for correlations between the four administrator-rated tasks [25]. The variance of the primary factor was fixed at one, and loadings were freely estimated [25]. Scores for follow-up visits were computed using item parameters from this baseline model transformed to a standard normal distribution (mean = 0 and SD = 1) [25]. Second, two other variants of the ADAS-Cog-13 were created by adding tasks to assess Executive Functioning (EF) and informant reports of daily function (FA) [25]. The ADAS-Plus-EF consists of the ADAS-Cog-13 bi-factor model plus an additional task for category (vegetable) fluency [25]. The ADAS-Plus-EF&FA consists of the ADAS-Cog-13 bi-factor model plus category (vegetable) fluency, Trail Making Tests (TMT) A and B, Digit Symbol Substitution (DSS) Test, and five Pfeffer Functional Assessment Questionnaire (FAQ) items [25]. These modifications were developed using data from 811 subjects with a range of cognitive abilities, validated in a subset of 394 subjects with MCI, and then compared to the ADAS-Cog-11, ADAS-Cog-13, ADAS-Rasch, and ADAS-Tree [25].
Plots of test information curves (Monte Carlo integrated test information versus cognitive ability) showed that the ADAS-Plus-EF&FA model had the highest test information over all levels of cognitive ability, followed by the ADAS-Plus-EF, and then the ADAS-Bifactor [25]. In general, the amount of information any of the three variants held about cognitive ability increased as cognitive ability worsened [25].
Performance summary. The ADAS-Bifactor, ADAS-Plus-EF, and ADAS-Plus-EF&FA all showed superior responsiveness to disease progression in MCI than the ADAS-Cog-11, but not to the previously developed ADAS-Tree [25]. Of all measures assessed, the ADAS-Plus-EF&FA appeared to be the most responsive to treatment effects in MCI [25]. The ADAS-Plus-EF and ADAS-bifactor were both more responsive to treatment effects in MCI than the ADAS-Cog-11, but less responsive than the ADAS-Tree [25].
Common Item Pooling
Wouters et al. (2012) pooled data from 1863 subjects with a range of cognitive abilities from multiple data sets which each included some or all of the CAMCOG, modified ADAS-Cog (ADAS-Cog-12 plus a concentration task), and MMSE [20]. Data pooling was performed using “common item equating”, and Rasch measurement models were used to estimate the difficulty of each test item and the cognitive ability of each participant [20]. The purpose was to locate an underlying dimension of cognitive ability common to all three outcome measures so that their scores could be compared [20]. Items showing systematic differences in level of difficulty between data sets, or for which valid estimates of difficulty level could not be obtained, were excluded [20].
Rasch measurement theory was also applied to assess whether adding neuropsychological tests of episodic or semantic memory and executive function to the modified ADAS-Cog, CAMCOG, and MMSE increased precision for discriminating between levels of early cognitive decline and detecting mild dementia [20]. Neuropsychological tests were found to be more difficult than the modified ADAS-Cog, MMSE, and CAMCOG items with difficulty levels compatible with NC to MCI and mild dementia [20]. In contrast, the modified ADAS-Cog had only a few tasks with difficulty levels appropriate for pre-dementia cognitive abilities [20].
The measurement precision for assessing levels of latent cognitive ability varied between the individual outcome measures and between different combinations of them. At mild levels of cognitive impairment, adding neuropsychological tests to the MMSE without the modified ADAS-Cog was recommended to maximize measurement precision, but for more severe levels of cognitive impairment adding the modified ADAS-Cog to the MMSE exceeds the modified ADAS-Cog alone [20].
Performance summary. Responsiveness was not evaluated.
Alzheimer’s Disease Neuroimaging Initiative Memory Composite (ADNI Memory Composite)
Crane et al. (2012) used modern psychometric approaches to develop and test the validity of a composite score for memory (ADNI Memory Composite) made up of the Rey Auditory Verbal Learning Test (RAVLT), ADAS-Cog-13 Delayed Word Recall task, ADAS-Cog-11 Word Recognition task, MMSE three word memory task with distractors, and the Logical Memory test [35]. Initial analyses of the ADNI Memory Composite included 803 subjects with NC, MCI, or AD [35]. Psychometric approaches determined that a bi-factor model was not a substantially better fit than a single factor model, so a single factor model was maintained. Concurrent criterion validation with the RAVLT was assessed [35].
Performance summary. The ADNI Memory Composite appeared able to discriminate between NC, MCI, and AD diagnostic categories, but statistical tests were not performed [35]. Although the ADNI Memory Composite demonstrated responsiveness to disease progression in NC, MCI, and AD samples, this performance did not exceed the ADAS-Cog-11 [35]. The ADNI Memory Composite was more responsive to treatment effects in subjects with NC than the ADAS-Cog-11, but not more responsive to treatment effects for MCI and AD levels of disease severity [35].
ADAS-3b
Raghavan et al. (2013) aimed to improve sensitivity to change and reduce variability of the ADAS-Cog-11 for MCI and early AD trials by replacing uninformative items with more responsive measures of cognition or function [15]. A total of six novel measures were derived based on analyses of cognitive and functional measures in 798 subjects with NC, MCI, or AD [15]. For a new item to be considered for inclusion, it needed to demonstrate a standardized two-year change score of at least 0.4 for MCI participants [15]. Three of the novel measures were composed solely of cognitive items, and three included cognitive items and measures of daily function. Bootstrap validation was performed for the entire selection process. The six novel measures were compared with each other and with other outcome measures, including the ADAS-Cog-11 and ADAS-Cog-13, using data from two years of follow-up of 606 subjects with NC, MCI, or AD [15].
The first cognitive measure, the ADAS-3b, includes ADAS-Cog-11 tasks which did not exhibit ceiling effects and surpassed the 0.4 threshold for standardized two-year change scores: Word Recall, Delayed Word Recall, and Orientation [15].
Performance summary. The ADAS-3b was more responsive to disease progression than the ADAS-Cog-11 for MCI participants [15].
Cognitive Composites 1 and 2 (CC1, CC2)
The second novel composite developed by Raghavan et al. (2013), the Cognitive Composite 1 (CC1), adds the RAVLT immediate recall test and the MMSE to the ADAS-3b [15].
The third novel composite developed by Raghavan et al. (2013), the CC2, consists of the ADAS-3b and the cognitive portion of the Clinical Dementia Rating Scale–Sum of Boxes (CDR-SB) [15].
Criterion validation was performed for both the CC1 and CC2 using Spearman’s correlations between two-year change scores for each of the composites and reference standards such as the ADAS-Cog-11 and CDR-SB. Factor analysis was used to assess the latent structure of each composite measure [15].
Performance summary. The CC1 and CC2 both appeared able to discriminate between groups of subjects with MCI and AD, although statistical tests were not performed [15]. Both the CC1 and CC2 were more responsive than the ADAS-Cog-11 to disease progression in subjects with MCI [15]. Sample size estimates in simulated clinical trials also suggest that both the CC1 and CC2 are more responsive to treatment effects in subjects with MCI and with AD than the ADAS-Cog-11 [15].
Cognitive Functional Composites 1 and 2 (CFC1, CFC2)
Three novel measures developed by Raghavan et al. (2013), the Cognitive Functional Composites (CFC) include measures to assess both cognition and daily functioning [15]. The CFC1 combines the CC1 and the FAQ [15]. The CFC2 combines the CC2 and the FAQ [15]. The CC3 (CDR-SB and FAQ) did not incorporate any ADAS-Cog-11 items, so it will not be reviewed further [15]. Correlations between the cognitive portions of each of the CFCs with the FAQ demonstrated that change scores on the CFCs were due to changes on both cognitive and functional sub-tasks [15].
Performance summary. The CFC1 and CFC2 appeared able to discriminate between MCI and AD groups, although no statistical tests were performed [15]. Both were more responsive than the ADAS-Cog-11 to disease progression in subjects with MCI [15]. It was also suggested that both the CFC1 and CFC2 would be more responsive than the ADAS-Cog-11 to treatment effects in subjects with MCI and in subjects with AD based on sample size estimates in a clinical trial [15]. Overall, the CFC2 was the most responsive measure developed by Raghavan et al. (2013).
Item Response Theory and Pharmacometric ADAS-Cog-13
Ueckert et al. (2014) used IRT and pharmacometric modelling to analyze ADAS-Cog-13 scores [7]. First, an IRT model to estimate baseline cognitive ability was created using data from 2,744 subjects with NC, MCI, or mild AD [7]. The IRT baseline model models cognitive ability as a subject-specific random effect following a standard normal distribution (Z-score), with no limits on the upper or lower extremes of cognitive ability [7]. The probability of a subject responding a certain way on an ADAS-Cog-13 task (or sub-item), given their underlying cognitive ability, was described using four different test item specific models. First, tasks that are scored as correct or incorrect (e.g., Orientation: correctly state the month) were modeled with a three-parameter binary model that accounts for item discrimination, item difficulty, and the probability that a subject with no cognitive disability would get the item incorrect [7]. Second, tasks involving words were modeled with a binomial model, with slightly different failure probabilities depending on the task (Word Recall failure probability: three-parameter binary model described above; Word Recognition failure probability: same as for Word Recall plus a fourth parameter to account for the maximal probability that a subject with severe cognitive dysfunction would correctly categorize words as seen or not) [7]. All words were assumed to hold the same amount of information about underlying cognition [7]. Third, the Number Cancellation task was modelled using a generalized Poisson model, which included the same three test item parameters as the three-parameter binary model plus a fourth parameter for dispersion, and a factor to ensure predicted scores are in the range of 0 to 40 [7]. Fourth, tasks on the ADAS-Cog-13 that are rated by the examiner (e.g., Comprehension of Spoken Language) were modeled using a proportional odds, ordered categorical model with five ordinal categories (none to severe impairment) and parameters for item difficulty and discrimination [7].
Next, three different methods of measuring cognitive change over time were assessed. The latter two are extensions of the baseline IRT model: A Least-Square Mean Analysis Model used change in ADAS-Cog-13 score as the outcome variable, treatment as the exposure variable, visit as a repeated factor, baseline ADAS-Cog-13 score as a covariate, a treatment-by-visit interaction term, and a grouping factor of subjects nested within treatment [7]. This represents a more “traditional” ADAS-Cog scoring methodology. The baseline IRT model was extended a Longitudinal IRT Model by adding a hidden variable to account for disease progression over time [7]. Disease progression was assumed to be linear, subject-specific, and modelled through random-effects [7]. A hazard function for the probability that a subject will drop out of a longitudinal study was also included [7]. To assess the performance of the longitudinal IRT model, Z-score estimates of underlying cognitive ability for an 18-month long clinical trial were translated back to the original ADAS-Cog-13 scale and compared with observed ADAS-Cog-13 scores from 322 real subjects in the 18-month clinical trial [7]. Specifically, two-hundred Monte-Carlo simulations from the IRT model and the original clinical trial data were used to compare the proportion of subjects from the original data whose task-level scores would fall in the 95% prediction interval from the score produced by IRT models [7]. Total ADAS-Cog-13 score comparisons were done in a similar manner [7]. ADAS-Cog-13 scores for the clinical trial were plotted with the median, 2.5th, and 97.5th percentile of the real scores observed in the clinical trial. When plotted on top, the median value of the real scores fell within the 95% CI predicted by the IRT model for all but the final 18-month follow-up assessment [7]. The Pharmacometric Total ADAS-Cog Score Model was based on a previously published disease progression model, modified according to the results of goodness of fit plots, residual plots, and visual prediction checks [7]. This model was further refined and tested using a simulated data set from the longitudinal IRT model [7]. Similar to the longitudinal IRT model analyses whereby estimated ADAS-Cog-11 total scores were compared to observed scores in a real clinical trial, the performance of the pharmacometric total ADAS-Cog-13 score model was assessed with visual predictive checks of whether the 95% CI for the ADAS-Cog-13 scores estimated from the pharmacometric model included the 2.5th, 97.5th, and median ADAS-Cog-13 scores from the ADAS-Cog Longitudinal IRT model based simulated data set [7]. The final pharmacometric total ADAS-Cog-13 score model assumes a linear progression of cognitive dysfunction (increasing scores), and models individual subject baseline scores with a Box-Cox distribution and normally distributed individual slope parameters correlated with baseline random effect [7].
Performance summary. Both IRT and pharmacometric modelling appeared more responsive to treatment effects in AD than traditional ADAS-Cog scoring and methods of analysis [7].
Integrated Alzheimer’s Disease Rating Scale (iADRS)
Wessels et al. (2015) used a theoretical framework to combine existing scales of cognition and function into a single outcome measure for AD studies [98]. For subjects with mild AD (n = 181) and MCI (n = 380), preliminary tests found the combination of ADAS-Cog-13 and the FAQ was most sensitive, and the combination performed better than either scale alone [98]. Further analyses relied on data from treatment trials in AD that did not have those two measures specifically, so they were approximated with the ADAS-Cog-14 and the ADCS-instrumental Activities of Daily Living (iADL), which together make the integrated Alzheimer’s Disease Rating Scale (iADRS) [98].
iADRS score = [-1(ADAS-Cog-14) + 90] + iADL, where the ADAS-Cog-14 (ADAS-Cog-11, Delayed Word Recall, Maze, and Digit Cancellation tasks) and iADL are summed normally and the total range of the iADRS is 0 to 146 with lower scores indicating worse performance [98].
Psychometric analyses showed that the iADRS is composed of two principal components (cognition and instrumental function) for assessment at one point in time, and the majority of the variability for subjects with MCI was due to cognitive items of the ADAS-Cog [98]. For change over time, the iADRS items load on a single component, and variance of change scores was driven by both cognitive and function items [98].
Performance summary. The iADRS was more responsive to disease progression in MCI and AD than the ADAS-Cog-11, however it was not more responsive than the ADAS-Cog-14 for treatment effects in MCI and mild AD [98].
Straightforward Sensitive Scale (SSS)
Huang et al. (2015) designed a scale including cognitive and functional measures to track disease progression over time and detect potential treatment effects in clinical trials for MCI and early AD, while maintaining good reliability and validity as subjects progress to more severe stages [99]. Selection of measures to include in the composite scale was performed in a stepwise manner, using SRM calculations [99]. In general, the minimum SRM for a candidate measure to be considered was 0.45 for the group of 397 subjects with MCI, 0.50 for an APOE enriched subgroup, and 0.55 for hippocampal volume and Aβ enriched subgroups [99]. The SRMs of all possible combinations of candidate measures were calculated to determine the composite scale most sensitive to disease progression and treatment effects [99]. This “straightforward sensitive scale” (SSS) consisted of the CDR-SB, FAQ, and three ADAS-Cog-13 items (Word Recall, Delayed Word Recall, Orientation) [99].
Performance summary. The SSS appeared more responsive to disease progression and treatment effects in MCI than the ADAS-Cog-13 [99].
ADAS-Cog-5
Podhorna et al. (2016) created the ADAS-Cog-5 by adding to the ADAS-Cog-3 (presented under Scoring Modification section) Delayed Word Recall and Digit Cancellation tasks [2]. The ADAS-Cog-5 is scored from 0 to 45 [2].
Performance summary. The ADAS-Cog-5 appeared able to discriminate between groups of subjects with MCI and mild AD, although statistical tests were not performed [2]. Responsiveness to disease progression in MCI was not superior to the ADAS-Cog-11 [2].
ADAS-13 Re-Weighted (ADAS-13RW)
Grochowalski et al. (2016) created three versions of the ADAS-Cog-13 using data from 505 subjects with AD or MCI in an effort to improve reliability of ADAS-Cog-13 change scores [100]. Improved reliability would reduce variability and ultimately improve the ability of the ADAS-Cog-13 to track changes over time. The three versions included a re-weighted, a lengthened, and a re-weighted and lengthened ADAS-Cog-13 [100]. To obtain these three different variants the ADAS-Cog-13 was divided into three subsections based on task scoring procedures [100]. Each section was given a separate weight, calculated as the number of tasks in that section divided by the total number of tasks in the test [100]. This resulted in a section of verbal memory with weight 0.10, a section of clinician-rated tasks with weight 0.45, and a section for general cognitive tasks with weight 0.45 [100]. Lengthening the test, either with or without re-weighting, did not substantially improve score reliability so the authors concluded that the ADAS-Cog-13 with re-weighted sections (ADAS-13RW) was best for improving reliability of change scores [100]. Criterion validation was performed by analyzing correlations between the ADAS-13RW and the ADAS-11, ADAS-13, RAVLT, and MMSE [100].
Reliability of change scores of the ADAS-13RW was better than the ADAS-Cog-13, but only of an “acceptable” magnitude for change scores defined by cut-score dependability (compare subject’s scores to pre-set criterion value of 4 points change) [100]. Relative change score reliability (rank subject’s change relative to another subject’s change) and absolute change score reliability (estimate of subject’s true individual change score) were not of an acceptable magnitude [100].
Performance summary. Responsiveness was not evaluated.
Alzheimer’s Disease Composite Score (ADCOMS)
Wang et al. (2016) developed the Alzheimer’s Disease Composite Score (ADCOMS) from outcome measures previously shown to be sensitive to AD-specific clinical decline and treatment effects using pooled data from the placebo arm of four aMCI clinical trials (n = 1,160) [101]. A partial least squares procedure was used to fit a linear model characterizing disease progression and variable importance projections (VIP) for candidate items [101]. The ADCOMS was derived by combining the twelve items with a VIP value of 0.8 or greater, weighted by their partial least squares coefficients [101]. Specific items included four ADAS-Cog-12 tasks (Delayed Word Recall, Orientation, Word Recognition, Word Finding Difficulty), two MMSE items (Orientation time, Drawing), and six CDR-SB items (Personal Care, Community Affairs, Home and Hobbies, Judgement and Problem Solving, Memory, Orientation) [101].
Performance summary. The ADCOMS demonstrated better responsiveness to disease progression and treatment effects in MCI than the ADAS-Cog-12 [101].
Overview of combined scoring method and task content modifications
The ADAS-3b removed all tasks with suggested ceiling effects in pre-dementia populations, while the CC1 and CC2 add back on to the ADAS-3b assessments of additional cognitive domains that were left out of the reduced measure. Other modifications that remove some items with ceiling effects from the ADAS-Cog-11 but also include additional cognitive tasks are the ADAS-5 and TE4D. One modification approach, ADNI-Mem, focused solely on combining tests of memory; this measure may be beneficial for certain research questions, but not those that include some indication of overall cognitive ability.
Four modification approaches highlighted that certain tasks are more informative than others and incorporated these differences into the scores. The ADAS-Tree and ADAS-RW provide easier weighting schemes than the ADAS-IRT/Pharmacometric and Common Item Pooling methods, but these weighting schemes may be optimized for the development study sample, potentially limiting generalizability.
Computerization modifications, including the TDAS and CAMCOG-Plus, may trade off reduced variability in test administration for increased research costs and less accessibility for some subjects.
Adding functionality assessments to ADAS-Cog-11 tasks (possibly with other cognitive measures) aligns with the present understanding of dementia and pre-dementia syndromes as including both cognitive and functional deficits. These may have better content validity of overall disease severity than purely cognitive measures. However, these composite measures will be of less interest when research questions focus on cognition or functionality as separate constructs; hence, while the SSS, CFC1, CFC2, and ADCOMS may have the advantage of being shorter assessments or incorporating a variety of cognitive tasks (e.g., not all from ADAS-Cog-11) in addition to functionality tasks, the Pooled Index, iADRS, and ADAS-Plus-EF&FA have the advantage of administering all tasks of the ADAS-Cog-11 so that those scores could be isolated, if needed.
Summary of modifications made to the ADAS-Cog-11
Of the 31 modifications of the ADAS-Cog-11, 5 altered the original scoring methodology, 4 maintained the scoring methodology and added additional tasks, and 22 both altered scoring methodology and added tasks.
In general, adding memory items to the ADAS-Cog-11, occasionally accompanied by the removal of other tasks, improved responsiveness to disease progression and to treatment effects [2 , 35]. Adding assessments of EF to the ADAS-Cog-11, with or without modifying scoring methodology, and with or without additional measures of memory, was found to improve responsiveness to disease progression and to treatment effects in MCI populations in all but one instance [2 , 99]. The results of our review also support recent statements that ideal measures for MCI and early AD should include both cognitive and functional assessments [98, 99]. Modifications that added items to assess functionality, alone or in combination with other cognitive tests or alternative scoring methods, demonstrated superior responsiveness to disease progression and to treatment effects in MCI populations than ADAS-Cog-11 modification approaches that only modified cognitive content alone or in combination with scoring modification techniques [15 , 101]. The only exception was that the ADAS-Tree outperformed the ADAS-Plus-EF&FA for responsiveness to disease progression, although the ADAS-Plus EF&FA demonstrated superior responsiveness over the ADAS-Tree to treatment effects [25]; this point also serves to demonstrate the context specificity of responsiveness. The CDR-SB alone, which includes assessments of cognition and functionality, was also found to be more responsive to measured decline over two years in MCI and NC samples than several ADAS-Cog-11 modifications that re-weighted scores and/or added cognitive tests but did not include any assessment of functionality [35].
CONCLUSION
Summary
The ADAS-Cog-11 was developed to assess cognitive function in subjects with moderate to severe AD. Subsequent developments in dementia research have shifted attention toward pre-dementia syndromes, especially MCI. The ADAS-Cog-11 continues to be used in these studies in spite of ceiling effects, suboptimal scoring methodology, and limited content validity across domains affected in pre-dementia syndromes. The ADAS-Cog-11 has been improved through modifications, notably scoring methodology and added assessments of executive function, delayed recall, and/or daily functioning to detect early cognitive changes seen in the continuum from MCI to Dementia; these findings align with previous studies that shown these domains are relevant clinical features that begin deteriorating in advance of dementia. While modification approaches may improve within-study quality, important limitations need to be outlined. The large variety of modified versions limits between-study comparisons. Modified versions that are backwards compatible to the ADAS-Cog-11 allow comparison of results to the original ADAS-Cog-11 scale, but it is not clear how to deal with incompatible findings between the original and modified versions. Modified versions may also lack standardized administration procedures laid out for the ADAS-Cog-11, which may add to total measurement error. Although this review demonstrates that the ADAS-Cog-11 is not optimally responsive in pre-dementia populations, abandoning it may lead to a proliferation of outcome measures particularly in the absence of a consensus-based new standard. Results from decades of research do however suggest that there is little more to be gained from exploring additional modifications of the ADAS-Cog-11.
Practical suggestions
When selecting an outcome measure for a given study, there is a trade-off between measurement performance and the length or challenge of administering the measure. Furthermore, the performance of an outcome measure depends on the stage of natural history of a study sample, the desired form of responsiveness, and the study purpose. For these reasons and because not all ADAS-Cog-11 modifications have been compared with each other, it is difficult to recommend a single ‘best’ ADAS-Cog-11 modification based on our review. In general, removing tasks from the ADAS-Cog-11 that demonstrate ceiling effects in pre-dementia populations is expected to improve performance for pre-dementia syndromes and be simpler to administer, but probably with poorer measurement properties for dementia syndromes. Therefore, for studies across the spectrum of cognitive abilities, the appropriate outcome measure may be one that assesses cognitive domains that deteriorate later in the natural history (e.g., language), without including too many tasks that will lower responsiveness in pre-dementia studies due to ceiling effects. In addition, since the ability to perform activities of daily living relies upon intact executive function along with other cognitive abilities, and is now considered an important component of disease severity, it is expected that including assessments of these cognitive domains and functionality would improve responsiveness in pre-dementia syndromes while also capturing important deficits at later stages of the natural history. By expanding the ADAS-Cog-11 with cognitive and functional outcomes, better responsiveness should be possible.
Future directions
An area of growing interest is including motor function assessments, like gait speed, with cognitive outcome measures, because motor biomarkers have been associated with progression to dementia [102, 103]. Another emerging assessment method that appears responsive to changes in pre-dementia populations includes proactive semantic influence [104 –106]. Future research should build on the findings of our review and consider the potential value of the aforementioned domains that have demonstrated responsiveness in pre-dementia populations but have not been included in a published modification of the ADAS-Cog-11, with the ultimate goal of creating a comprehensive outcome measure for studies of dementia or pre-dementia populations.
