Abstract
Many new therapies for dementia target a specific pathologic process and must be applied early. Selection of specific therapy is based on the clinical etiologic diagnosis. We sought to determine the stability of the clinical etiologic diagnosis over time and to identify factors associated with instability. We identified 4141 patients with dementia or mild cognitive impairment who made at least 2 visits approximately a year apart to a dementia research center, receiving a clinical etiologic diagnosis on each visit. We assessed concordance of etiologic diagnoses across visits, κ-statistics, and transition probabilities among diagnoses. The primary clinical etiologic diagnosis remained stable for 91% of patients but with a net shift toward dementia with Lewy bodies and Alzheimer’s disease. Lower diagnostic stability was significantly associated with older age, nonwhite race, milder disease at presentation, more underlying conditions contributing to cognitive decline, lack of a consistent spouse/partner informant, and being evaluated by different clinicians on different visits. Multistate Markov modeling generally confirmed these associations. Clinical etiologic diagnoses were generally stable. However, several readily ascertained characteristics were associated with higher instability. These associations may be useful to clinicians for anticipating when an etiologic diagnosis may be more prone to future change.
Introduction
Dementia can result from any of several underlying disease processes, including Alzheimer’s disease (AD), Lewy body disease, frontotemporal dementia (FTD), cerebrovascular disease, and other less common disorders. 1 Much current research is aimed at finding effective ways to prevent or treat dementia, and most drugs in development are aimed at a specific etiologic type of dementia. For example, experimental vaccines have been developed to stimulate immune system clearance of amyloid protein in AD. 2,3 Other drugs seek to block the accumulation of α-synuclein that leads to Lewy body disease. 4,5 Gene therapy has been proposed to counteract a genetic abnormality that leads to hyperphosphorylated τ protein in FTD. 6 Drugs that inhibit platelet aggregation may prevent progression of vascular dementia (VaD). 7,8
Many new drugs may need to be given quite early in the disease course, before widespread neuronal loss occurs, to be effective. 9,10 Because neuropathological confirmation of a dementia diagnosis is rarely possible during life, selection of an appropriate disease-specific drug must almost always depend on a clinical etiologic diagnosis, based on symptoms, signs, and test results, none of which have perfect sensitivity and specificity. The clinical etiologic diagnosis is the underlying neurodegenerative process to which the physician ascribes the patient’s neurocognitive abnormalities—for example, AD, cerebrovascular disease, or Lewy body disease, among others. Often the accuracy of that clinical etiologic diagnosis will not be known until autopsy. However, due to the chronic and progressive nature of most dementing illnesses, the true underlying condition itself is unlikely to change over time. Thus, if there are inconsistencies in the clinical diagnosis over time for the same patient, it is unlikely that all of those different diagnoses are correct. Thus, consistency or stability of the clinical etiologic diagnosis can be regarded as a necessary but not sufficient requirement for accuracy.
This study had 3 aims: (1) to quantify the stability of clinical etiologic diagnoses over time in a large sample of patients with cognitive impairment; (2) to describe changes in the distribution of etiologic diagnoses over time; and (3) to identify factors associated with greater instability of the clinical etiologic diagnosis.
Methods
Setting
Study patients were drawn from all 32 AD centers (ADCs) funded by the US National Institute on Aging. These centers conduct research and provide clinical evaluation and treatment for patients with mild cognitive impairment (MCI) or cognitive impairment from AD or other forms of dementia. The Uniform Data Set (UDS), a set of standardized data collection forms and guidelines, was used at all centers. 11 Data collection was overseen by the National Alzheimer’s Coordinating Center, which also served as a common data repository. The present analysis was approved by the institutional review board of the University of Washington. Informed consent was obtained from all participants.
Study Patients
We studied 4141 ADC patients who were judged clinically to have either MCI or dementia. All were evaluated approximately annually. For this study, a patient had to have made at least 2 UDS visits about a year apart between September 2005 and November 2009, at which a clinical etiologic diagnosis of cognitive impairment had been recorded. The first study visit for each patient was when he or she was first recorded as having dementia or MCI. Some patients had made up to 5 annual visits thereafter, depending on their date of enrollment and retention in study.
Data
The UDS data collection forms allowed any of 21 different etiologies to be recorded, either as the primary clinical diagnosis or as a contributing condition. Written guidelines for use with UDS forms (available at http://www.alz.washington.edu) incorporated diagnostic criteria for several etiologic diagnoses for which explicit criteria have been developed, after review and approval by the UDS Clinical Task Force. 11 The UDS did not mandate that each patient receive specific imaging procedures or biomarker tests, several of which were performed as part of center-specific clinical research protocols and not at all centers.
Characteristics evaluated as possible risk factors for diagnostic instability included demographic factors (age, gender, and race); severity of cognitive impairment at presentation and time since first symptoms began; presence or absence of other comorbid conditions; whether multiple dementing illnesses were thought to be contributing to cognitive decline; type of informant who accompanied the patient to ADC visits; whether the patient was evaluated by the same clinician on each visit or by different clinicians; and whether the clinical etiologic diagnosis was assigned by a single clinician or through a consensus process after the visit.
Analysis
Many of the 21 possible etiologic diagnoses proved to be rare. Accordingly, etiologic diagnoses were grouped into 2 ways, resulting in an 8-category list and a 4-category list. The 8-category list included AD, dementia with Lewy bodies (DLB), FTD, primary progressive aphasia, Parkinson’s disease, VaD, other etiology (other), or unknown etiology (unknown). The shorter 4-category list included AD, DLB, FTD, and other/unknown. Each patient fell into 1 category on each list at each visit, based on the primary etiologic diagnosis recorded at that visit.
Initial descriptive analyses involved cross tabulating the primary etiologic diagnosis from visit 1 with that from visit 2. Changes in the overall (marginal) distribution of diagnoses across successive visits were also examined.
Considering 2 successive visits for each patient, 1 measure of etiologic-diagnosis stability is the proportion of patients who received the same etiologic diagnosis on both visits—a measure known as concordance. However, our main measure of stability was instead the κ-statistic, 12 which corrects for the amount of agreement expected by chance if a diagnosis were simply chosen at random from the overall distribution of etiologic diagnoses at each visit. The κ-statistics were then compared among subgroups defined by the potential risk factors for diagnostic instability listed earlier.
Comparisons of κ value between exposure groups can be confounded if the groups differ on other factors that also affect etiologic-diagnosis stability. We developed a relatively simple method to control for other key patient characteristics by applying inverse-probability-of-exposure weights. Let E be a categorical exposure variable and
Finally, discrete-time Markov models
16,17
were applied. Let
Results
Table 1 shows how the primary clinical etiologic diagnoses on visit 1 corresponded to those on visit 2, using the 8-category grouping scheme. The top panel shows the number of patients in each cell; the bottom panel shows the percentage distribution of etiologic diagnoses at visit 2 among patients with each etiologic diagnosis at visit 1. Not surprisingly, AD was the most common form of dementia, and most patients received that etiologic diagnosis on each visit. Most patients kept the same diagnosis on both visits, but those with an initial diagnosis of FTD, VaD, or “other” were more likely to receive a different diagnosis the following year than other patients did. Overall, the concordance was 0.91, with a κ value of 0.77, representing substantial agreement beyond chance. Grouping into only 4 diagnosis categories made little difference to these results (concordance = 0.92, κ = 0.78).
Cross Tabulation of Primary Diagnosis (in 8 Groups) at Visit 1 and Visit 2 for All Patients.a
Abbreviations: AD, Alzheimer’s disease; DLB, dementia with Lewy bodies; FTD, frontotemporal dementia; PPA, primary progressive aphasia; PD, Parkinson’s disease; undet, undetermined; VaD, vascular dementia.
a Concordance = 0.91, κ = 0.77 (0.75-0.79).
Comparing the row and column totals in the top panel of Table 1 shows that the overall distribution of primary clinical etiologic diagnoses remained similar from visit 1 to visit 2. Nonetheless, given the large sample, some changes were statistically significant. DLB and AD both gained “market share,” and for DLB, the shift was statistically significant (P =.007). The diverse other category decreased significantly (P =.047). Additional analyses, not shown, revealed that similar trends in the mix of diagnoses continued to visit 3 in the smaller subset of patients who had made 3 or more annual visits.
For many patients, other disease processes besides the primary one were recorded as also contributing to a patient’s cognitive impairment. Table 2 shows concordance and κ-statistics for whether each of 21 disease processes was mentioned as involved (as either a primary or contributing cause) on visits 1 and 2. These process-specific concordances were generally high, but the corresponding κ-statistics were more variable. Lower κ values were observed for dementing processes of a more transitory nature, such as depression (κ = 0.47), medical illness (κ = 0.44), psychiatric illness (κ = 0.38), and medication effects (κ = 0.25).
Reliability of Whether Each of Several Etiologies was Judged to be Involved, Either as a Primary or as a Contributing Cause of Cognitive Impairment, Across 2 Consecutive UDS Visits.
Abbreviations: CI, confidence interval; Est, estimated.
Using κ value as the measure of diagnosis stability between visits 1 and 2, several statistically significant differences were found among subgroups of patients (Table 3). Diagnosis stability was higher in younger patients and lower in older ones. The κ value was also higher in patients of white race than in Blacks or patients of other races. In light of these demographic differences, the κ-statistics for other characteristics were adjusted for age and race. No statistically significant association was found with education, before or after adjustment for age and race. Etiologic diagnoses tended to be less stable among patients with less severe disease at visit 1, as reflected by having MCI rather than dementia or a Clinical Dementia Rating (CDR) of less than 1. Those with a Mini-Mental State Examination score of greater than 24 at visit 1 also had less stable clinical etiologic diagnoses but not to a statistically significant extent. Little association was found with length of time between onset of symptoms and visit 1. Diagnostic stability tended to be lower among patients who were judged to have more underlying disease processes contributing to their cognitive decline. The type of informant who accompanied the patient at clinic visits also mattered: diagnostic stability was highest when the informant was the patient’s spouse or live-in partner on both visits and lowest when 2 different informants (one of them not a spouse or partner) accompanied the patient. The etiologic diagnosis was less stable if 2 different clinicians evaluated the patient on successive clinic visits, rather than the same clinician both times. However, there was little association between the stability of the etiologic diagnosis and whether the diagnosis was assigned by an individual clinician or via a consensus process on visit 1.
Stability of Etiologic Diagnosis From Visit 1 to Visit 2 by Selected Exposures, Before, and After Adjustment for Age and Race.
Abbreviations: CI, confidence interval; CDR, Clinical Dementia Rating; MCI, mild cognitive impairment; MMSE, Mini-Mental State Examination.
Presenting full results from multistate Markov modeling is difficult because of the large volume of numerical data. Even using only 8 diagnostic categories, 56 transitions from one diagnostic state to another are possible, resulting in 56 logistic regression models, each of which includes multiple coefficient estimates and associated standard errors. Instead, a summary of results is shown in Table 4 for the 4-category diagnostic grouping scheme, which involves 12 possible transitions. In this analysis, all pairs of consecutive visits for each patient were included, not just the first 2 visits. Following the visit at which each patient had first been recorded as having MCI or dementia, 2269 patients made 1 subsequent visit, 1266 made 2 visits, 558 made 3 visits, 17 made 4 visits, and 1 made 5 visits. The 9 covariates shown in the first column were included in all models. For each covariate value except the reference category, a 4 × 4 table is shown at right. Each number within that table can be interpreted as the adjusted risk ratio of transitioning from the row diagnosis (made at the earlier visit of the pair) to the column diagnosis (at the later visit) for patients with that covariate value. For example, the value of 2.11 in the DLB row and AD column for age 80+ means that, relative to a patient age <70 years (the reference age category), a patient age of 80+ years was 2.11 times more likely to transition from DLB on the earlier visit to AD on the later visit, adjusting for all other covariates shown in the table. Boldface value denotes that it was statistically significantly different (at p < 0.05) from 1.0.
Markov Model Results: Adjusted Risk Ratios for Transitioning From One Clinical Etiologic Diagnosis (in 4 Categories) to Another Between Adjacent Visits.
Abbreviations: AD, Alzheimer’s disease; DLB, dementia with Lewy bodies; FTD, frontotemporal dementia; CDR, Clinical Dementia Rating. Boldface value denotes that it was statistically significantly different (at p < 0.05) from 1.0.
For age, most risk ratio estimates were statistically significantly different from 1.0, suggesting that age was quite important as a predictor of most of the possible transitions. The risk ratio estimates uniformly greater than 1.0 in the first column suggest that advanced age favored a net shift toward a diagnosis of AD over successive visits. A roughly similar pattern is seen for DLB in the second column, suggesting that age favored a shift in diagnosis toward DLB (unless the patient had a diagnosis of AD).
Among the other covariates in Table 4, the CDR on the earlier visit appeared quite important as a predictor of several transitions in diagnosis. Relative to patients with CDR <1, higher CDR scores favored net migration out of the diverse other category and net migration into the FTD category. The number of contributing conditions also appeared important but with a more complex pattern. Patients with more contributing conditions tended to shift away from an initial diagnosis of AD toward DLB, FTD, or other. The DLB in particular tended to “win out” and become the primary diagnosis on the later visit in patients with multiple contributing conditions.
A pertinent negative finding is the absence of statistically significant adjusted risk ratios for the last covariate shown, that is, the UDS visit numbers that corresponded to the sequence of consecutive visits used in this analysis. This pattern suggests that pooling data across pairs of successive visits, regardless of when those visits occurred in a patient’s UDS participation history, were a defensible analytic strategy.
Finally, Table 5 shows how the primary clinical etiologic diagnosis compared with the primary neuropathological diagnosis for the subgroup of 526 (13%) study patients who died and underwent brain autopsy. Separate panels are shown for the primary clinical etiologic diagnosis on clinic visits 1 and 2. In general, agreement between clinical and neuropathological diagnoses was high but not perfect, and slightly better if the clinical diagnosis came from visit 2 rather than visit 1. Note, however, that these results concern only the nonrandom subsample who died and underwent brain autopsy and that the elapsed time between clinical visits and death could have been up to several years.
Primary Clinical Etiologic Diagnosis on Visits 1 and 2 in Relation to Primary Neuropathological Diagnosis Among 526 Decedents With Brain Autopsy.
Abbreviations: AD, Alzheimer’s disease; DLB, dementia with Lewy bodies; FTD, frontotemporal dementia; VaD, vascular dementia; PPA, primary progressive aphasia; PD, Parkinson’s disease.
Comment
Using data on over 4000 patients evaluated on 2 or more annual visits at 32 dementia research centers in the United States, we found that the primary clinical etiologic diagnosis remained stable for about 9 of 10 patients. Nonetheless, there was a gradual shift in the distribution of etiologic diagnoses, with an increasing share of patients diagnosed as having DLB or AD. Moreover, we found several subgroups in whom the clinical etiologic diagnosis was significantly less stable, as reflected by a lower κ-statistic. Clinical etiologic diagnoses were less stable in older patients, nonwhites, those with clinically milder disease at presentation, those judged to have more etiologic processes contributing to cognitive decline, patients unaccompanied by a spouse or partner to serve as an informant, and those who were evaluated by different clinicians on different visits.
Results of multistate Markov modeling are harder to summarize but generally supported the importance of the same factors as affecting the likelihood of transition from one etiologic diagnosis to another. This observed pattern of associations may help to identify patients with early dementia in whom the clinical etiologic diagnosis is more likely to change over time, which may in turn affect the balance of risks and benefits of early therapies aimed at a specific underlying pathologic process.
To our knowledge, no previous published studies have examined the stability of clinical etiologic dementia diagnosis in relation to the characteristics studied here. The demographic factors that we found to be associated with lower stability—older age and nonwhite race—may simply be markers for other underlying causal factors. It has been suggested 18 that informants for older or African American patients with dementia may be less available and/or knowledgeable than those for other groups.
Two factors associated with the stability of the clinical etiologic diagnosis may be of special interest because they are potentially modifiable. First, having a spouse or partner accompany the patient at both visits was associated with greater diagnostic stability. This finding suggests that someone who is in constant close contact with the patient can provide better and more consistent information about disease manifestations, leading to a more stable (and possibly more accurate) diagnosis. These results thus provide empirical support for the importance of obtaining information from a knowledgeable informant in clinical evaluation of patients with early dementia. 18,19
Second, the primary etiologic diagnosis was less likely to change if the same clinician evaluated the patient on repeated visits. This result suggests that individual clinicians may be reluctant to change their own initial diagnostic impressions or that a “second opinion” can often lead to rethinking and changing a patient’s clinical etiologic diagnosis. Despite recently published preliminary evidence that a consensus diagnostic process involving multiple evaluators can yield improved accuracy over diagnoses assigned by individual experts, 20 we found no significant difference in stability of the etiologic diagnosis in relation to whether a consensus diagnostic process was used.
The finding that diagnostic stability was higher for patients with dementia than for those with MCI agrees with the common clinical observation that classic manifestations of a disease often become more apparent as the disease progresses. This pattern may be especially true of neurodegenerative diseases, which can evolve over many years. It may pose a challenge for clinicians; however, even though etiologic diagnosis of MCI may be more subject to change, therapies aimed at slowing particular forms of neurodegeneration may need to be applied at the MCI stage to be effective.
In interpreting results of this study, it should be kept in mind that consistency of the clinical etiologic diagnosis is not the same as accuracy of that diagnosis. An etiologic diagnosis can be consistent across clinical encounters but consistently wrong. The “gold standard” for accuracy of an etiologic diagnosis remains neuropathology, which unfortunately is available only on a subset of patients with dementia, namely, those who die and then undergo brain autopsy. The present study focused instead on diagnostic stability, which could be addressed in a larger patient population. Future advances in imaging and biomarker development will almost certainly lead to new research opportunities to investigate accuracy in live patients as well as to improvements in diagnostic stability and accuracy themselves. 21 Nonetheless, even the most recently revised criteria for AD rely on clinical features, treating evidence from biomarkers as complementary but not required. 10
Other study limitations should also be noted. Patients at participating ADCs are best regarded as a large clinical case series, not a truly population-based sample, so generalizability of these findings is uncertain. Although the content of the UDS was standardized across centers, use of imaging modalities and biomarkers varied among centers, depending in part on each center’s own research focus. Most disease-specific drugs are still under development, and we do not yet know what might be the consequences of using such a drug when the clinical etiologic diagnosis is incorrect.
These limitations notwithstanding, it is reasonable to expect that the stability and accuracy of the clinical etiologic diagnosis for dementia will be increasingly important for proper treatment. Information about risk factors for diagnostic instability, such as that generated in this study, can help clinicians to identify patients in whom an early diagnosis may be most likely to change, and it can help alert researchers to possible diagnostic misclassification.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by grant U01 AG016976 from the National Institute on Aging.