Abstract
Alzheimer’s disease dementia (AD dementia) is one of the most common neurodegenerative diseases worldwide, with a growing incidence during the last decades. Clinical diagnosis of cognitive impairment and presence of AD biomarkers have become important issues for early and adequate treatment. We performed a systematic literature search and quality appraisal of AD dementia guidelines, published between 2005 and 2011, which contained diagnostic recommendations on AD dementia. We also analyzed diagnostic recommendations related to the use of brief cognitive tests, neuropsychological evaluation, and AD biomarkers. Of the 537 retrieved references, 15 met the selection criteria. We found that Appraisal of Guidelines Research and Evaluation (AGREE)-II domains such as applicability and editorial independence had the lowest scores. The wide variability on assessment of quality of evidence and strength of recommendations were the main concerns identified regarding diagnostic testing. Although the appropriate methodology for clinical practice guideline development is well known, the quality of diagnostic AD dementia guidelines can be significantly improved.
Background
The world population is rapidly aging, which causes a significant increase in chronic and neurodegenerative diseases, with serious consequences in terms of global public health. The prevalence of diseases such as dementia is nearly 42 million patients in 2012, with approximately 4.6 million new cases a year. 1 These figures would increase up to 300% by 2040, with an impact on expenditure of over US$ 422 billion.1,2
Alzheimer’s disease dementia (AD dementia), the most common dementia in the Western world, is defined as a cognitive decline documented by standardized testing on 2 or more domains, which interferes with daily function and represents a decline in previous levels. 3 Recently, the National Institute of Neurological and Communicative Disorders and Stroke–Alzheimer's Disease and Related Disorders Association criteria have been updated and presented 3 different types of AD dementia according to presence of core clinical criteria only (probable), an atypical course without differential diagnosis (possible), or neuropathological evidence (definitive). 4
The presence of cognitive impairment, especially in the memory domain, is a fundamental part of the clinical criteria used to diagnose AD dementia. This impairment must be detected and diagnosed through a combination of clinical history analysis and objective cognitive assessment by means of a brief mental evaluation or comprehensive neuropsychological testing. 4 The latter reflects the clinical dimension of the disease and characterizes the type of presentation. However, new trends in AD dementia diagnosis suggest the need to reflect upon the biological dimension of the disease, including biomarkers related to brain amyloid-β protein deposition and downstream neuronal degeneration. 5
Several clinical practice guidelines (CPGs) for the management of AD dementia have been published in different countries. These offer users (professionals and health administrators, patients and caregivers, among others) clinical recommendations for daily practice based on the best available evidence. 6 Although CPGs have become a useful clinical tool, their quality can vary and could be improved. 7 The CPG assessment must be conducted in an objective manner using standardized instruments that address issues such as rigor of development, clarity of recommendations, and applicability, among others. 8 It is also important to analyze the differences in recommendations among guidelines focused on the same topic, which can vary and even be contradictory.9,10
There are previous studies related to the assessment of the quality and content of AD dementia CPGs, but these were not conducted systematically and did not assess recommendations on AD dementia diagnosis.11–14 The purpose of this review was to assess the quality of CPGs that focus on the diagnosis of AD dementia as well as to examine the recommendations on 3 critical aspects of AD diagnosis: initial testing with brief cognitive screening tools, use of neuropsychological assessment, and use of biomarkers of amyloid-β accumulation and neuronal injury.
Methods
We performed a systematic review of AD dementia CPGs retrieved from the Web sites of guideline developers (Scottish Intercollegiate Guidelines, National Institute for Clinical Excellence, among others), CPG compilers (National Guidelines Clearinghouse, National Library of Guidelines, GIN Web site, among others), and institutions that conduct research related to dementia.
We complemented this search in MEDLINE, using the terms “dementia,” “Alzheimer,” and “elderly,” with a methodological filter to search for CPGs (the full search strategy is available upon request). The CPGs with diagnostic recommendations in English, Spanish, and Portuguese published between January 2005 and December 2011 were included. Previous versions of the same guideline, letters to the editor, or guidelines with a focus on population screening or general mental health were excluded.
Two authors independently selected all potentially eligible studies by reading the titles and abstracts of the retrieved references. Subsequently, 4 trained authors independently assessed the quality of each CPG using the Appraisal of Guidelines Research and Evaluation (AGREE)-II tool. 8 This instrument includes the evaluation of 6 domains: Scope and Purpose, Stakeholder Involvement, Rigor of Development, Clarity of Presentation, Applicability, and Editorial Independence. The intraclass correlation coefficient (ICC) and its 95% confidence interval (95% CI) were calculated to assess interrater agreement. Statistical analyses were performed using SPSS® 15.0 (SPSS Inc, Chicago, IL).
For the analyses of recommendations regarding diagnostic testing, we only included guidelines of high quality, defined as those with scores equal to or greater than 60% in the “Rigor of development” domain. We used this cutoff point as an adequate figure to reflect that a valid and transparent process had been adopted in the development of recommendations. 15 For each guideline, 3 authors extracted recommendations concerning: (1) the brief cognitive instruments recommended for initial evaluation of patients with suspected AD dementia, (2) the use of neuropsychological assessment, and (3) the use of biomarkers in patients with suspected AD dementia. The recommendations (sentences) of each CPG were extracted along with the quality of evidence, strength of the recommendation, and type of studies included. This information was synthesized in a recommendation matrix for further analysis. All disagreements were resolved by consensus.
All paper and pencil tests used for the initial assessment of patients with suspected AD dementia were classified as brief cognitive instruments. Neuropsychological assessment was defined as all comprehensive and standardized cognitive evaluations, including appraisal of memory, language, executive functions, and visual–spatial abilities conducted by trained clinicians. Recommendations on the following AD biomarkers were included: low cerebrospinal fluid (CSF) amyloid β42 levels, positive retention of tracer in positron emission tomography (PET), elevated Tau protein levels in CSF (total or phosphorylated), decreased fluorodeoxyglucose (FDG) uptake on PET reflected in the temporoparietal cortex, and presence of patterns of atrophy on magnetic resonance imaging (MRI), involving medial, basal and lateral temporal lobes and medial parietal cortex. 4
Results
We retrieved a total of 537 citations and excluded 518 that were not related to AD dementia. We also excluded letters to the editor as well as CPGs in languages other than English, Spanish, and Portuguese, those with recommendations concerning only a subset of patients with dementia and those with general recommendations for mental health. In all, 4 CPGs containing therapeutic recommendations only were also excluded (Figure 1). We assessed 15 CPGs distributed among 22 individual references.16–37

Flowchart of the systematic search.
We found that the domains with the highest scores were “Scope and Purpose” and “Clarity of Presentation” (75.4% and 80%, respectively), whereas “Applicability” and “Editorial Independence” had the lowest figures (32.4% and 47%, respectively). Only 1 (6.7%) CPG had scores above 60% in all domains. 18 All the 6 (40%) guidelines had ratings equal to or above 60% for the “Rigor of Development” domain18,19,25,30,34,36 (Table 1). We classified these as guidelines of high quality. Agreement among reviewers was considerable (ICC = 0.78, 95% CI: 0.60-0.91).
The AGREE II Domain-Standardized Scores for CPG on Alzheimeŕs Dementia Diagnostic Recommendations.
Abbreviations: CPG, clinical practice guideline; EFNS, European Federation of Neurological Societies.
Of the 15 identified guidelines, 4 did not report a system to grade evidence or recommendations. Conversely, the 6 high-quality CPGs used 4 different grading systems. The most commonly used system, out of these 4, was that developed by the Scottish Intercollegiate Guidelines Network (66%). 38 Only 1 CPG used a tool specifically designed for diagnostic questions 30 (Table 2). One CPG presented a diagnostic classification system in the Methods section, but after concluding that there was no enough information to address the diagnostic topics, they based their recommendations on an expert consensus and presented them without graduation of strength. 18
Quality of Evidence and Strength of Recommendation Systems Used.
Abbreviations: EFNS, European Federation of Neurological Societies; SIGN, Scottish Intercollegiate Guidelines Network; RCT, randomized controlled trials.
All the 15 included guidelines had information on at least 1 of the 3 critical aspects of AD dementia diagnosis included in this review. In all, 14 (93%) provided recommendations related to the use of brief cognitive tests in the diagnosis of patients with suspected AD, 9 (60%) presented recommendations on the use of neuropsychological tests, and 4 (26%) presented explicit recommendations on use of β-amyloid accumulation/neuronal injury biomarkers.
Brief Cognitive Test Recommendations
All 6 (100%) high-quality guidelines provided recommendations on the use of brief cognitive tests in the diagnosis of patients with suspected AD dementia. They emphasized the importance of conducting a formal cognitive assessment using objective and valid instruments, although 3 (50%) of them did not recommend any test in particular (Table 3). A total of 19 brief instruments were recommended in the guidelines retrieved, the Mini-Mental State Examination (MMSE) being the most frequently recommended. Of the 6 high-quality guidelines, 5 (83%) considered information on national validation and standardization of recommended tests, and 2 (33%) of them, recommended the use of MMSE versions validated in the country where the CPG had been developed.34,36
Recommendations for Brief Cognitive Tests in AD Dementia.
Abbreviations: AD, Alzheimer’s disease; MMSE, Mini-Mental State Examination.
Observational designs, especially cohorts and cross-sectional studies, were the most frequently included study designs. We found disagreements between the quality assessment of evidence and the confidence of recommendation among the high-quality guidelines. The former varied from very low to high depending on the system used. The strength of recommendation ranged from A to C (Table 3).
Neuropsychological Assessment Recommendations
All the 6 (100%) high-quality guidelines presented recommendations related to the use of neuropsychological tests. They recommended neuropsychological testing only when the diagnosis is unclear, when there is uncertainty about the stage of the disease, or when a differential diagnosis needs to be ruled out. Again, observational studies were the most frequently included type of design included. The quality of the evidence varied from very low to high and the strength of recommendations from A to D, similar to brief cognitive recommendations. None of the CPGs proposed a specific neuropsychological battery of instruments and only 1 guideline included a recommendation about the assessment of major cognitive domains (memory, executive functions, language, attention, and visuospatial skills; Table 4). 30
Neuropsychological Assessment Recommendations in AD Dementia Guidelines.
Abbreviation: AD, Alzheimer’s disease.
The AD Biomarkers Recommendations
Only 4 (66%) of the 6 high-quality CPGs presented explicit recommendations for the use of β-amyloid accumulation/neuronal injury biomarkers (Table 5). In all, 3 of these emphasized a lack of clinical studies on these biomarkers and presented recommendations against their use.19,34,36 The fourth high-quality CPG found FDG–PET and Tau levels to be useful, 30 although the direction of recommendation (for or against) remains unclear. Again, observational designs were the most frequently included studies. The quality of evidence varied widely, from low to high, depending on the system used. Similarly, the strength of recommendation ranged from B to C.
Recommendations for the Use of Alzheimer’s Dementia Biomarkers.
Abbreviations: CSF, cerebrospinal fluid; FDG, fluorodeoxyglucose; EFNS, European Federation of Neurological Societies; MRI, magnetic resonance imaging; PET, positron emission tomography.
a Inside cells: level of evidence, grade of recommendation.
b Recommendation against use of biomarker.
c Recommendation unclear for use of biomarker.
Discussion
This review shows that CPGs addressing diagnostic issues related to AD dementia published between 2005 and 2011 are very heterogeneous in terms of quality according to their AGREE-II scores. Domains such as applicability and editorial independence had very low scores and deserve special attention. Previous reviews of AD dementia CPGs highlight similar difficulties on these topics, even though such shortcomings could be easily improved.7,12 For example, if the applicability domain addresses organizational barriers, cost implications, and monitoring criteria related to guideline implementation,39–42 it is important that guideline developers identify a priori factors related to transferability, such as values, costs, availability of resources, and influence of economic and intellectual interest on implementation. Regarding editorial independence issues,41,43,44 a complete and valid documentation of conflicts of interest for all participants in all stages of guideline development is required. 45
Issues concerning CPG rigor of development deserve closer attention. In our review, we noticed that 26% of the CPGs did not use a system to grade the quality of the evidence and the strength of recommendations. However, among those that did use a system, only 1 included diagnostic-specific elements in their framework (eg, appropriate reference standard or adequate spectrum of patients). 46 This figure is disturbing because CPG users need to know how much they can trust the evaluated information and whether adherence to the recommendations will yield more benefits than harm. 47 In our opinion, the use of generic systems developed by therapeutic studies would lead to inaccurate ratings of the quality of the evidence and would generate uncertainty about the recommendations. An example is the use of neuropsychological testing. Although recommendations were fairly homogeneous and specific in terms of content, their strength and quality of the evidence on which they were based varied from very low (experts opinion, D) to very high (1++, A). This situation is worrying because although we selected only high-quality guidelines for these comparisons, it is not clear whether the evidence related to neuropsychological tests is of good quality or whether developers are confident about implementing the recommendations provided.
We found a similar situation regarding the use of brief cognitive tests, which added to other concerns about the clinical utility of these tools. In our review, we found a total of 19 tests recommended for initial assessment, but 3 of the high-quality guidelines only prescribed a cognitive assessment, although without identifying any instrument to achieve that goal. Once again, we encountered difficulties in assessing the quality of the diagnostic evidence and the strength of recommendations, but this variability also reflects doubts about the advantages and limitations of the tests. 25 We think that valid and global figures about accuracy of these tools, especially those related to subgroups such as patients with mild cognitive impairment or the general population, are imperative. This information could help us determine which test(s) would be more useful for the initial assessment of patients with suspected AD dementia.
We found that only a few (26%) CPGs addressed the use of biomarkers. Three of them did not recommend their use in daily clinical practice and only one suggested that they could be potentially useful. 30 Based on the currently available evidence, the uncertainty with respect to the applicability and benefit of these biomarkers in clinical settings does not support their inclusion in CPGs. The CPG users, who still wish to adopt these diagnostic tools in daily practice, could face difficulties related to local standardization, development of cutoff points for different stages of AD, and availability of trained staff (eg, radiologists in measurement of MRI structures). Once again, we noticed that formal evaluation of clinical accuracy of these biomarkers is necessary. 48
A previous review by Beck et al also found considerable differences among diagnostic recommendations in CPGs on dementia published until 1999. 11 They attributed these differences to the methods used to formulate recommendations and to the need for an evidence-based approach for future guideline development. We found that disagreement persisted even in rigorously developed evidence-based CPGs, causing significant consequences in terms of interpretation, comparison, and generalizability of the recommendations. 12 In our opinion, adopting a system that took into account the nature of the evidence evaluated would be helpful. The framework outlined by Grading of Recommendations Assessment, Development and Evaluation approach has been developed to provide a common, rigorous, and comprehensive framework with a specific consideration of diagnostic issues that could be helpful in solving these difficulties. 49
Our review has several strengths and limitations. We conducted an exhaustive and systematic search strategy of the available literature that was not limited to indexed databases, given that it included references that had not been used in other reviews on AD dementia. However, we had to exclude 6 CPGs in languages such as German and Chinese as well as 4 references with only therapeutic recommendations, which could have facilitated a more complete evaluation of AD dementia guidelines. Similarly, we only included guidelines published between 2005 and 2011. Although we selected this time span in order to be more certain about the appropriateness of the guidelines, CPG developers may have had different interpretations of the same evidence.
We used a valid and widely disseminated instrument (AGREE-II) to assess guideline quality as well as a reliable process based on independent assessment by 4 appraisers. Nevertheless, the use of the AGREE instrument has potential drawbacks related to a distinctive focus on the development process of the CPG instead of on the quality of individual recommendations within the guideline. 50 In addition, we only used 1 domain of the AGREE-II tool to select the guidelines for individual analyses, whereas other reviews recommend the assessment of 2 domains, namely rigor of development and editorial independence. In our review, we aimed to assess the methodological issues related to the development of CPGs in order to highlight the issues associated with using generic grading systems, but we also noticed that all selected guidelines obtained adequate scores on the editorial independence domain (≥50%).
As a conclusion, it is necessary that clinicians have access to high-quality and up-to-date recommendations from the best available evidence for the adequate management of patients with AD dementia. The quality of CPGs on the diagnosis of AD dementia can be improved not only through the inclusion of ethical aspects, the participation of stakeholders, and a clear strategy for their implementation, but also through a thorough evaluation of the quality of the available evidence using a unified framework to present recommendations.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Pablo Alonso-Coello is funded by a Miguel Servet investigator contract from the Instituto de Salud Carlos III. Other authors received no financial support for this research, authorship, and/or publication of this article.
