Abstract
Background:
Persons with dementia have higher mortality than the general population. Objective, standardized predictions of mortality risk in persons with dementia could help with planning resources for care close to the end of life.
Objective:
To systematically review prediction models for risk of death in persons with dementia.
Methods:
The Medline and PsycInfo databases were searched on November 29, 2020, for prediction models estimating the risk of death in persons with dementia. Study quality was assessed using the Prediction model Risk Of Bias ASsessment Tool.
Results:
The literature search identified 2,828 studies, of which 18 were included. These studies described 16 different prediction models with c statistics mostly ranging from 0.67 to 0.79. Five models were externally validated, of which four were applicable. There were two models that were both applicable and had reasonably low risk of bias. One model predicted risk of death at six months in persons with advanced dementia residing in a nursing home. The other predicted risk of death at three years in persons seen in primary care practice or a dementia specialty clinic, derived from a nationwide registry in Sweden but not externally validated.
Conclusion:
Valid, applicable models with low risk of bias were found in two settings: advanced dementia in a nursing home and outpatient practices. The outpatient model requires external validation. Better models are needed for persons with mild to moderate dementia in nursing homes, a common demographic. These models may be useful for educating persons living with dementia and care partners and directing resources for end of life care.
Registration:
The study protocol is registered on PROSPERO as RD4202018076.
Keywords
INTRODUCTION
In most cases dementia is caused by incurable, progressive neurodegenerative diseases that will lead to death. The average duration of survival with dementia is seven to ten years [1]. Shorter survival has been associated with older age, male sex, lower cognition, neuropsychiatric symptoms, greater impairments in function, and more comorbidities [2]. Patients and care partners are usually aware that dementia will shorten the lifespan, and often ask their clinicians how long the patient has to live. The answer to this question can help persons living with dementia and their care partners understand the illness and what the future holds, motivate care planning for the future, and, in later stages, determine eligibility for palliative care services. Additionally, an emerging application during the Coronavirus Disease in 2019 (COVID-19) pandemic may be to identify patients with limited remaining lifespan that may not warrant triage to critical care resources when such resources are scarce [3].
Our objective was to determine whether there are validated risk scores that predict the risk of death in persons with dementia, suitable for application in either the general population, a primary care practice, a specialty dementia clinic, or in a nursing home.
METHODS
The CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) [4] was used to guide the definition of the research question and design of the systematic review. The objective was to systematically identify prognostic models that would allow the user to calculate an estimated probability of death at 6 months or later in persons with dementia. Models that estimated median survival were also considered potentially applicable. The required setting was a nursing home, outpatient practice, or in the general population. The study protocol is registered on PROSPERO as CRD42020180760.
Study selection criteria required that all participants had dementia at inception, that the dementia was either caused by Alzheimer’s disease or included all types of dementia, that outcomes were assessed at 6 months or later, and that either all-cause mortality or duration of survival was reported as an outcome. Studies were required to report a multivariable-adjusted risk score or a multivariable model from which the user could make individualized predictions of 6-month, 1-year, or longer-term estimated percent mortality or estimated survival. Studies were excluded if they predicted mortality only in non-Alzheimer’s causes of dementias (e.g., Lewy body dementia) or in subpopulations based on risk factors (e.g., hypertension), acute conditions (e.g., pneumonia) or other factors; if they used longitudinal information (e.g., rate of change on test scores) as a predictor; or if they analyzed risk factors for mortality without developing or validating a prognostic model to predict risks in individuals. The included settings were a population-based study, family practice, dementia specialty practice, or long-term care home. Studies of model development, external validation, or comparison were included.
The Medline and PsycInfo databases were searched on November 29, 2020, using keywords and medical subject headings for dementia, risk, prediction, death, and mortality (full search terms are provided in Supplementary Table 1). There were no restrictions on study dates or language. Two reviewers (ES and ZI) screened each abstract and read each full text article for eligibility. The Covidence website (http://www.covidence.org) was used to organize the screening. Reasons for excluding full text articles were recorded in these pre-specified categories: unable to retrieve, letter or abstract only, wrong population, review article, duplicate, wrong study design, wrong setting, wrong outcomes, pediatric population.
Data were extracted independently by two reviewers (ES and ZI) with discrepancies resolved by consensus. If consensus was not achieved, the data element were described as unclear and the reasons for the lack of clarity was recorded. These data were extracted: year of publication, years of data collection, setting, sample size, number of outcomes (deaths), country of origin, population selection criteria, percent women, age (central tendency and distribution), type of model (development, validation, or both), candidate predictors, final predictors, analysis methods, measures of discrimination, measures of calibration, internal validation, external validation, other significant results, and limitations. Studies were considered to undergo external validation only when tested in an independent dataset, not when the original sample was split.
Quality was assessed using the Prediction model Risk Of Bias ASsessment Tool (PROBAST) [5], following guidance for its use [6]. Risk of bias and applicability was rated by two reviewers, using the recommended categories of Yes, Probably Yes, No, Probably No, or No Information. For risk of bias, the number of Yes and Probably Yes responses for 20 signaling questions were tabulated, and reported as ≥18, 15–17, and <17. Studies were considered overall good quality if they were applicable to the study question, reported both discrimination and calibration, and had a quality score of ≥18. Publication bias was suspected for studies that reported high discrimination with small samples (less than 20 events per predictor).
The research question was developed based on the work of a taskforce on COVID-19 and dementia sponsored by the Alzheimer Society of Canada, a charitable organization working on behalf of persons with dementia. A subcommittee of this taskforce has published guidance for triaging persons with dementia for critical care, including recommendations that survival from underlying dementia should be estimated based on objective, evidence-based models [7].
RESULTS
The search returned 3,070 articles, of which 73 were read in full and 18 were ultimately selected for inclusion (Fig. 1). One study [8] retrospectively analyzed administrative health data and provided a figure showing mortality rates stratified by age, sex and organ failure; however, the authors wrote that they “did not aim to build a prediction model” and therefore the paper was excluded.
Study selection.
Characteristics of the studies are shown in Table 1. Sixteen studies included development of a model of which three also included an external validation in the same paper, while two reported external validation of a previously published model. The models were developed in nursing home populations (4 studies), in administrative or electronic health record databases of claims (3), dementia specialty clinics (2), geriatrics inpatient units or day programs (2), convenience samples from the community (2), incident dementia cases in a population-based study (1), prevalent dementia in a community survey (1), and participants in a randomized controlled trial of case management (1). The mean age of participants ranged from 76 to 86 years. The proportion that were women ranged from 53 to 82%.
Study Characteristics
“Development” refers to model development without external validation, while “Validation” refers to external validation. RCT, randomized controlled trial. *Defined as Cognitive Performance Score 5 or 6.
Study quality according to PROBAST criteria is shown in Table 2. Eight of the studies had concerns about applicability, either because they did not provide a usable probability table or provided only a simple cut-off with sensitivity and specificity (3 studies) [9–11], included hospitalized inpatients (2) [12, 13], or used predictors that were difficult to obtain, poorly defined, subjective, or non-standardized (3) [14–16].
Study Quality
+ indicates low risk of bias (ROB)/low concern regarding applicability; –indicates high ROB/high concern regarding applicability. Risk of bias for determining the outcome (death) was deemed to be low for all studies, and therefore is not shown.
Most studies were deemed to be at some risk for bias, usually for analytic reasons. Common reasons included lack of information on discrimination and calibration, lack of internal validation, insufficient numbers of events, lack of accounting for missing data or using complete case analysis, and selecting final candidates based on the results of univariate p value testing (more details available in Supplementary Table 2). Of the 16 studies reporting model development, 2 had low risk of bias for ≥18 PROBAST signaling questions [17, 18], 7 had low risk of bias for 15–17 signaling questions[9, 19–21], and 7 had low risk of bias for <15 signaling questions [11, 23]. The two highest quality studies were the Advanced Dementia Prognostic Tool (ADEPT) [17], derived to predict risk of death at 6 months among persons with advanced dementia in nursing homes, and a prognostic model for risk of death at 6 months among persons with dementia seen in outpatient primary care or dementia specialty clinics contributing data to a nationwide registry [18].
Results of model development are shown in Table 3. C statistics ranged from 0.68 to 0.79, with the exception of a paper that described machine learning-based models with very high c statistics of ≥0.94 [16]. However, that machine learning paper was deemed to be at high risk of bias due to overfitting because of the large number of predictors (all information in the electronic health records) and was of low applicability to the research question because data from all medical encounters were used rather than data from a single inception point. Model calibration was generally reported to be good, when it was assessed. However, many studies only reported the Homer-Lemeshow test, rather than tabulating or plotting observed versus expected deaths as recommended by recent guidelines [6].
Results of Model Development
NR, not reported; O-E, plot or table comparing observed versus expected. If studies reported split samples the total size and total number of deaths is reported. If studies reported a derivation and external validation cohort, the size of the derivation cohort is shown (details of external validation are shown in Table 4). ADEPT, Advanced Dementia Prognostic Tool; ADL, activity of daily living; BEHAVE-AD, Behavioral Pathology in Alzheimer’s Disease; BMI, body mass index; COPD, chronic obstructive pulmonary disease; mMMS, modified Mini-Mental State Examination; MMSE, Folstein Mini-Mental State Examination; SOB, shortness of breath; WHO-DAS II, World Health Organization Disability Assessment Schedule.
The results of external model validation are shown in Table 4. There were five studies that validated four models. The most extensively validated model was developed by Mitchell and colleagues to predict death at 6 months in persons with advanced dementia on admission to a nursing home [22]. The candidate predictors were taken from the Minimum Data Set, which is collected by regulation on admission to a nursing home and at regular intervals, at minimum every year thereafter. It was validated externally in 4,631 persons in the original derivation paper [22], and then subsequently in data from two other multicenter nursing home studies (n = 288 and n = 289) [24]. The C statistics from external validation (0.70, 0.65, 0.64) were somewhat lower than the original derivation cohort (0.74). The model was re-derived, and re-named ADEPT, in a 2010 publication that included persons already admitted to the nursing home, with a new variable for newly admitted versus existing resident [17]. In a prospective cohort of 606 persons recruited to validate ADEPT, the validation c-statistic (0.67) [25] was essentially the same as in the original derivation cohort (0.68). The two other external validation studies validated models derived from incident dementia cases in a population-based study and in outpatients with dementia.
Model external validation
NR, not reported; O-E, plot or table comparing observed versus expected.
DISCUSSION
This review identified two models at reasonably low risk of bias that were applicable to the research question. The ADEPT score can be used to predict risk of death at 6 months in persons with advanced dementia residing in a nursing home [17], and has been externally validated [25]. A prognostic model derived from the Swedish Dementia Registry can be used to predict risk of death at 3 years in persons with dementia seen in specialty clinics or primary care practice. It was derived with good quality methods but has not yet been externally validated. Another 14 prognostic models were returned by the search but suffered from either methodological limitations that conferred higher risk of bias, or had concerns about applicability, or both.
The best validated prognostic model was the ADEPT score for predicting risk of death in persons with dementia in a nursing home [17]. The intended purpose for the model was to help identify persons meeting United States Medicare rules for eligibility for hospice care, which requires a life expectancy of less than 6 months. The ADEPT score was closely based on a previous model derived and externally validated in 2004 by the same author group [22]. ADEPT uses 12 predictor variables derived from the Minimum Data Set, a component of the federally mandated Resident Assessment Instrument (RAI) which is collected on nursing home residents on admission and annually each year thereafter. It displayed good discrimination and calibration in a cohort of 606 patients prospectively collected to test its validity [25]. A limitation of ADEPT is that, although deemed applicable to our research question, it is only valid for persons with very advanced dementia. It was derived and validated in persons with a RAI Cognitive Performance Score of 5 or 6, indicating very severe loss of function including eating problems. Most people with this level of impairment have a Folstein Mini-Mental State Exam (MMSE) score of 5 or less [26]. Within this population, ADEPT seems valid for continued use although additional new research on the calibration of the score would be welcome, as it was calibrated ten years ago and may not reflect contemporary trends in medical care and preferences regarding palliative care.
A prognostic model from the Swedish Dementia Registry was applicable to predicting death in patients with dementia attending either a primary care practice or a dementia specialty clinic. The score was derived from a nationwide registry including all dementia specialty clinics and about 75%of primary care practices in Sweden, with a large sample size. In the primary care setting, three-year risk of death was predicted by age, sex, MMSE, and Charlson comorbidity score, while in the specialty clinic setting cause of dementia was another predictor (classified as Alzheimer’s disease; vascular, mixed dementia, or other; or Lewy Body disease, Parkinson’s disease dementia, or frontotemporal dementia). The inclusion of dementia diagnosis in the specialty clinic model presumably reflects better clinical characterization by specialists, although the discrimination was similar in primary care (c statistic 0.70 for men and 0.71 for women) as in specialty care (c statistic 0.71 for men and 0.72 for women). This model seems suitable for use in the outpatient setting, although more research is required to externally validate it. Internal validation by bootstrapping was promising, with low shrinkage factors and little evidence of overoptimism [18].
Thirteen other models were identified in the systematic review, but eight were deemed not applicable to the research question and five others were deemed applicable but to be at high risk of bias due to various methodological shortcomings.
Among all the models, C statistics ranged from 0.67 to 0.79 (with one overly optimistic exception), suggesting that it is possible to predict risk of death in persons with dementia with sufficiently good discrimination to use this information to make individual prognoses.
There are several potential uses for prognostic model of death in persons with dementia. Patients and their care partners typically understand that dementia is irreversible and progressive. They deserve to have the best evidenced-based prognosis of survival and risk of death in this condition. This information can help reinforce the need for advanced planning for medical care, finances, and designating power of attorney. It could also be useful for determining eligibility for palliative care services, such as hospice, close to the end of life.
An additional prognostic need has emerged in the context of the COVID-19 pandemic. When access to ICUs and mechanical ventilation is limited by scarce resources, eligibility for ICU care should be determined by objective evidence of likelihood of survival. One perspective recommends giving priority to patients with “reasonable life expectancy” [27]. However, all persons living with dementia deserve good general medical care—including, where appropriate, palliative care—even if intensive care, which can be burdensome, is scarce. Guiding principles for triaging persons with dementia state that dementia is a heterogenous condition with a wide range of prognoses and recommends basing decisions on objective evidence of estimated survival rather than speculations about quality of life, which are more subjective and prone to bias [7]. The prognostic models systematically reviewed in this paper have some utility for this triaging application, but also some limitations. The ADEPT model [17] predicts risk of death from dementia in an appropriate time frame for triage decisions but only in persons with very advanced dementia, where functional status alone probably conveys sufficient information for triage. The Swedish Dementia Registry model [18] includes outpatient and milder dementia patients but predicts death over a longer time period (3 years) that may not be as relevant for immediate triage decisions. However, it does clarify that younger patients (age 65 or 75) with dementia without other comorbidities generally have greater than 80%chance of surviving to three years and beyond, such that mortality risk from underlying dementia alone seems insufficient to justify a blanket exclusion from ICU care during the pandemic. What is absent from the published literature are models that predict the risk of death in persons with mild to moderate dementia residing in nursing homes.
In conclusion, this systematic review identified two useful prognostic models for mortality risk in persons with dementia, in nursing home residents with advanced dementia and outpatients with dementia attending primary care or seen in a specialty clinic. However, there remain many questions to be answered by future research. Both models would benefit from further validation, including the first external validation for the latter model in the outpatient setting. There are no validated, applicable, high quality models for patients residing in nursing homes with mild to moderate dementia, which is a common demographic. Models in this population should be derived and validated. For all models, research is needed on how they can be implemented in clinical practice. Patients, and their care partners, deserve individualized prognoses based on high quality, objective evidence. The impact of obtaining this information—on their knowledge, personal choices, clinical care, and quality of life—should be investigated.
-4pt
DISCLOSURE STATEMENT
Authors’ disclosures available online (https://www.j-alz.com/manuscript-disclosures/20-1364r1).
