Abstract
Frailty is one of the major challenges for intensive care, affecting one-third of intensive care unit patients and being associated with a range of poor health outcomes. Determination of frailty in critical illness using the Clinical Frailty Scale has recently been adopted by the Australian and New Zealand Intensive Care Society, but it is not known whether this is able to be measured from the clinical record without interviewing patients or their relatives. The aims of this retrospective cohort study were to test whether a Clinical Frailty Scale score could be assigned in an intensive care unit population from the clinical record, and to assess the inter-rater reliability of frailty measured in this manner. A total of 144 patients were enrolled. Of these, 137 (95%) were able to have a Clinical Frailty Scale score assigned, and 22 (15%) were scored as frail (Clinical Frailty Scale ≥5). Cohen’s kappa coefficient for inter-rater reliability between assessors was 0.67, confirming substantial agreement. Consistent with other critically ill cohorts, frailty was associated on multivariate analysis with age, Charlson comorbidity score, dependence with activities of daily living, and limitation of medical treatment, indicating validity of this approach to frailty measurement. Our results imply that frailty measurement is possible and feasible from the intensive care unit clinical record, which is of importance as routine measurement and reporting of frailty in intensive care units in our region increases. Future work should seek to validate an assigned Clinical Frailty Scale score with that obtained directly from patients or their next of kin.
Introduction
As critically ill populations around the world age, patient frailty has become one of the major challenges in intensive care. Over the next decade in Australia, more than a quarter of all patients admitted to the intensive care unit (ICU) will be aged >80 years. 1 Based on similar European cohorts, >40% of this population will have coexisting frailty. 2 Frailty is associated with an increased risk of death, functional dependence, hospital readmission and new discharge to residential care.3–5 Measuring frailty in the ICU is challenging, but one validated tool—the Clinical Frailty Scale (CFS)—is increasingly being used due to its ability to be measured at the bedside, with no requirement for functional testing, which is challenging in the ICU environment. 6 Frailty measured according to this scale, a nine-item categorical measure, has been correlated with increased mortality, adverse events and functional dependence in ICU survivors.2,3 Given these associations, the Australian and New Zealand Intensive Care Society (ANZICS) Centre for Outcome and Resource Evaluation has recently added frailty case finding using the CFS to the data collected at admission for ICU patients in our region.
Despite the importance of frailty to the risk stratification and outcomes of critically ill patients, gathering these data presents a particular challenge in the ICU. Unlike other data points for the ANZICS Adult Patient Database (APD), which are for the most part based on objective criteria able to be gathered from the clinical record, frailty measurement via the CFS requires a degree of subjectivity in assessment. Interviewing the patient or the next of kin is also often necessary in order to determine aspects of functional performance and physical dependency that allow granularity of frailty measurement. Prior frailty studies have employed dedicated research personnel for this purpose, trained in the collection and interpretation of the CFS.3,7 However, more recent attempts have been made to collect these data routinely using clinical staff. A single-centre Australian study illustrated the challenges posed with this approach, wherein only 59% of eligible patients had a CFS score assigned, and of these, three-quarters were assigned by the nurse in charge without next of kin involvement. 8 This approach thus raises concerns regarding data completeness and accuracy.
CFS collection in Australasian ICUs will therefore likely fall to existing ICU data collectors, posing significant challenges to workload. In particular, contemporaneous bedside frailty measurement will be problematic for data collectors, who currently rarely (if ever) are required to interview patients or next of kin, and also often enter data retrospectively using the convenience of medical chart review. It is not known whether this is possible for frailty in critical illness. No prior literature exists to inform whether frailty measurement and scoring is possible from the routine medical admission records of ICU patients. Accordingly, we therefore conducted a retrospective cohort study to test the primary hypothesis that a CFS score could be assigned in an ICU population from interrogation of the clinical record. Second, we hypothesised that the inter-rater reliability of this score between two separate clinician assessors as scored by Cohen’s kappa coefficient would be at least 0.6, implying substantial agreement, and that associations with other patient characteristics and frailty found in other ICU populations would remain when data were collected using this methodology.
Materials and methods
This study was a secondary analysis of the previously published retrospective case-control study examining contributory factors leading to persisting critical illness. 9 Approval was obtained from the local Human Research Ethics Committee as a quality assurance project (QA2016110). In this previous single-centre study, 72 adult patients between 1 January 2013 and 31 December 2014 with an ICU length of stay (LOS) of >10 days were matched to 72 patients with an ICU LOS of <10 days admitted within the same time frame. The matching hierarchy was as follows: ANZICS APD diagnostic code, sex, age within 10% and Acute Physiology and Chronic Health Evaluation III risk of death within 10%. Patients admitted to the ICU for purposes of organ donation were excluded. In the case of patients readmitted to the ICU, only the first ICU episode was eligible for inclusion. Four study members (JD, TB, JN or DM) interrogated the paper medical record after discharge and collected data including age, sex, Charlson co-morbidity score, ICU admission source and diagnosis, presence of treatment limitation on admission, dependence with any activities of daily living (ADLs) as determined by the Katz scale (bathing, dressing, toileting, feeding, continence and transferring), 10 daily ICU supports and requirement for ICU care, and outcomes including mortality, discharge destination (home, other acute hospital or chronic care/rehabilitation) and cause of death. In addition, a CFS score was assigned for all patients by the lead intensive care specialist investigator (JD), with a randomly selected subset of 100 patients also independently assigned a CFS score by one of the other three other intensive care resident medical officer investigators (TB, JN or DM) to assess inter-rater reliability. Accepted cut-offs for frailty were used, with frailty defined as a CFS score ≥5, non-frail as a CFS score of <4 and vulnerable as a CFS score of 4. 6 Prior to study commencement, all data collectors were trained in the role of the CFS and its measurement, with calibration on a sample of five patients. CFS scores were assigned after interrogation of the entire clinical record, with particular emphasis on the allied health review documentation (IP49 hospital form) in conjunction with the ‘social history’ aspect of the admission note. Data collectors were blind to each other’s CFS scores.
Statistical analysis
All data were initially assessed for normality. Data were reported as numbers (%), means (standard deviation) or median (interquartile range) as appropriate. Univariable and multivariable linear regression models were fitted to investigate the association of other variables with frailty and with in-hospital mortality. Inter-rater agreement between CFS scores was examined via Cohen’s kappa coefficients, using quadratic weighting due to the increased magnitude of clinical difference with ascending CFS categories. Cohen’s kappa were categorised using the scale of Landis and Koch. 11
Results
During the study period, 72 patients with an ICU LOS of >10 days were matched to 72 control patients with complete medical records and with a LOS of <10 days from a total of 3874 patients admitted to the ICU. 9 The baseline data of the cohort are presented in Table 1. CFS scores could be assigned for 137 of the 144 (95.1%) patients; seven patients were unable to have a CFS score assigned due to absent social details in the admission note and no allied health review. Twenty-two (15.3%) patients were scored as frail (CFS score ≥5), 37 (27.0%) patients were scored as vulnerable (a CFS score of 4) and 78 (54.2%) patients were scored as non-frail (a CFS score of ≤3; Figure 1). As previously reported, there were no differences between persistently critically ill cases and control patients in frailty prevalence. 9
Baseline characteristics of the cohort.
ADLs: activities of daily living (bathing, dressing, toileting, feeding, continence and transferring)
10
; APACHE: Acute Physiology and Chronic Health Evaluation; IQR: interquartile range;
Clinical Frailty Scale scores of study participants.
For the 100 patients with a dual CFS assessment, Cohen’s kappa coefficient for inter-rater reliability between investigators was 0.67, indicating substantial agreement. Forty-five (45%) of 100 scores agreed perfectly, with a further 41 (41%) differing by only one point and 13 (13%) by two points. Only one pair of scores differed by more than two points: a patient with a CFS score of 7 by one investigator and a CFS score of 4 by another.
On univariate analysis, frailty (a CFS score ≥5) was associated with age, Charlson comorbidity score, dependence with ADLs, limitation of medical treatment on admission, mortality, and discharge to chronic care/rehabilitation (Table 2). On multivariate analysis, age, Charlson comorbidity score, dependence with ADLs and limitation of medical treatment on admission remained significantly associated with frailty (Table 2). Although frailty was associated with univariate mortality, there were no statistically significant associations when considered in the multivariable model.
Univariate and multivariate association of variables with frailty (CFS score ≥5).
CFS: Clinical Frailty Scale.
Discussion
In this retrospective cohort study, we found that the assignment of frailty status with a CFS score from the clinical record of critically ill patients was feasible, able to be completed in 95% of patients and had substantial inter-rater reliability. As seen in overseas cohorts, frailty was associated with increasing age, medical comorbidity, functional dependence and limitation of medical treatment on admission, providing evidence of the concurrent validity of this approach to measuring the CFS.
One prior (non-ICU) study examined the feasibility of retrospectively assigning a CFS score from the clinical record of 41 geriatric outpatients who had both undergone and had documented a comprehensive geriatric assessment, with similarly substantial inter-rater reliability (Cohen’s κ = 0.64). 12 Unlike our investigation, however, this study used the gold-standard comprehensive geriatric assessment, with multiple data points able to be assessed in arriving at a frailty score. In contrast, the current study has demonstrated that CFS assignment is possible from the usual clinical record of an ICU admission, without specific geriatric assessment. The inter-rater reliability of prospective CFS assignment in the ICU from interviews with patients’ surrogates was also demonstrated in a recent study of 101 critically ill patients in Wales and Scotland, with a linear weighted kappa between assessors of 0.74, and a similar level of score agreement seen in our study (53% versus 45% of scores in perfect agreement; 40% versus 41% of scores differing by one point). 1 Taken together, our study and these prior studies suggest that the CFS has acceptable inter-rater reliability, including in the ICU environment.
Similar to recent literature using the CFS to measure frailty in critical illness, we also observed poor health status and ICU outcomes in frail patients. A recent study of 421 critically ill Canadian patients demonstrated similarly increased comorbid disease and functional dependence with frailty, as well as a higher rate of limitation of medical treatment (34% frail patients versus 12% non-frail,
The strengths of our study include the assessment of a mixed medical/surgical/trauma ICU population using a paper charting system, thus implying our findings are likely applicable to other similar ICUs in our region. A further strength is that our study investigators were non – geriatric specialists, thus enhancing external validity to data collection by similar non-frailty experts in other ICUs. Our study had several limitations. It was conducted in a single centre, with the possibility that differences in data recorded and documented may exist between different hospitals. Given that much documentation is likely common to many Australasian ICUs, however, including mention of patients’ relevant social circumstances and the integral role (and hence related documentation) of allied health in ICU, we consider that the standard of our medical records could be reasonably expected to reflect that of units around our region. A further limitation is that we did not compare the CFS assigned through clinical record interrogation to that measured contemporaneously through interviews with patients or their next of kin directly. Thus, we cannot compare the two techniques. Future research should seek to validate this and to assess the accuracy of the methodology we have chosen in this study. We note, however, that one such study exists which demonstrated substantial agreement between prospective and retrospective CFS scores when derived from a documented comprehensive geriatric assessment. 12 Whether this same level of agreement exists when compared to documentation typical of an ICU admission requires further research. Our methodology is also not translatable to the determination of frailty at ICU admission. We assigned a CFS score after access to the entire chronology of the clinical record, in particular (and most usefully) after allied health documentation some days into the admission, which frequently revealed details related to functional capacity and physical dependency, allowing granular frailty assessment. Although a limitation for the early measurement of frailty in critical illness, this still has relevance to our hypothesis, allowing determination of patient frailty for benchmarking, data reporting and audit. There is, however, the potential for bias as a result of assignment of a higher CFS score in those who were sicker or who had a limitation of medical treatment order, although this applied to only 13% of the total cohort. A final limitation is we did not stratify patients by pre-ICU residential location. It is possible that the increased association of discharge to chronic care/rehabilitation was influenced by a smaller proportion of frail patients residing at home prior to the onset of critical illness. There is, however, a strong association in past studies with new onset of residential care admission. 16
In conclusion, we have demonstrated the feasibility of measuring frailty using the CFS from the clinical record in a cohort of critically ill patients. This has significant implications for the ability to measure and report frailty routinely in Australian and New Zealand ICUs, which is likely achievable without extra resourcing for specific bedside data collection. As accurate and complete frailty assessment becomes integral to risk stratification and ongoing treatment decisions made in the ICU, this is an important development.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by grants from the Veterans Administration Health Services Research and Development (17-045) to TJI.