Abstract
Background:
Informal carers face many challenges in caring for patients with palliative care needs. Selecting suitable valid and reliable outcome measures to determine the impact of caring and carers’ outcomes is a common problem.
Aim:
To identify outcome measures used for informal carers looking after patients with palliative care needs, and to evaluate the measures’ psychometric properties.
Design:
A systematic review was conducted. The studies identified were evaluated by independent reviewers (C.T.J.M., M.B., M.P.). Data regarding study characteristics and psychometric properties of the measures were extracted and evaluated. Good psychometric properties indicate a high-quality measure.
Data sources:
The search was conducted, unrestricted to publication year, in the following electronic databases: Applied Social Sciences Index and Abstracts, Cumulative Index to Nursing and Allied Health Literature, The Cochrane Library, EMBASE, PubMed, PsycINFO, Social Sciences Citation Index and Sociological Abstracts.
Results:
Our systematic search revealed 4505 potential relevant studies, of which 112 studies met the inclusion criteria using 38 carer measures for informal carers of patients with palliative care needs. Psychometric properties were reported in only 46% (n = 52) of the studies, in relation to 24 measures. Where psychometric data were reported, the focus was mainly on internal consistency (n = 45, 87%), construct validity (n = 27, 52%) and/or reliability (n = 14, 27%). Of these, 24 measures, only four (17%) had been formally validated in informal carers in palliative care.
Conclusion:
A broad range of outcome measures have been used for informal carers of patients with palliative care needs. Little formal psychometric testing has been undertaken. Furthermore, development and refinement of measures in this field is required.
The involvement of informal carers is essential for the provision of palliative care, but informal caregiving can have a major impact on carers’ outcomes.
Studies of informal carer outcomes use a wide range of endpoints.
Selecting suitable and appropriate carer outcome measures seems problematic.
An increasing number of studies are conducted in informal carers looking after patients with palliative care needs.
Only four outcome measures have been formally developed and validated within this population, and limited psychometric information is available on most measures.
While there has been an increasing trend since 2008 in the use of outcome measures for informal carers in palliative care research, most measures used in these studies were developed more than 20 years earlier and may not adhere to current standards for measure development.
Existing carer outcome measures need to be validated for the palliative care setting and new measures need to be developed in accordance with current guidelines in order to meet the requirements of the growing number of studies, including intervention studies, of informal carers looking after patients with palliative care needs.
When using an existing outcome measure, the authors should report their rationale for selecting it and should refer to the publications that report the original development of the measure.
Interventions for supporting informal carers should be evaluated using outcome measures for which appropriate psychometric properties have been reported before they are implemented as policy.
Introduction
The World Health Organization (WHO) 1 defines palliative care as an approach that focuses on the quality of life of patients and their relatives facing problems associated with life-threatening illness, through prevention and relief of suffering. Annually, around 20 million people worldwide need palliative care, 2 and an ageing population and increases in long-term conditions mean that need is likely to continue to rise.3,4
Informal carers make an important contribution in the provision of palliative care and are regarded as integral to its delivery.5,6 Informal carers are defined as carers who are not financially compensated for their services typically spouses, children, siblings or friends. 7 In 2011, the contribution of approximately six million informal carers in the United Kingdom was estimated at the equivalent of £119 billion a year. 8 About half a million people are caring for patients during the end-of-life phase and this number is expected to increase to 3.4 million in the coming 30 years. 9
Palliative care has become an important component of health care, and policy makers are putting more emphasis on informal carers. 10 Informal caregiving may provide emotional benefits and togetherness for carers,11,12 but it also involves considerable challenges including adverse psychological, physical, social and financial consequences.13,14 Studies indicate that informal caregiving affects carers’ wellbeing and their own health resulting in isolation, fatigue, sleeping problems, exhaustion, weight loss, depression and anxiety.15–19 It is therefore important that carer outcomes are assessed in order to be able to provide effective support and to reduce negative consequences of caregiving. Carer outcomes refer to a range of concepts including quality of life, burden and strain. While these terms are not well defined and frequently get used interchangeably, it is generally accepted that they comprise multiple dimensions such as physical impact, mental strain and social functioning. It has been argued that quality of life is a broader concept as it assesses a wider spectrum of wellbeing, whereas burden and strain suggest a more direct measure of duty of care. 20
Evidence on effective strategies to reduce the burden of caring and improve their quality of life of informal carers is limited.21,22 Although interventions have been developed that aim to improve outcomes for informal carers, their results are difficult to compare as studies focus on a wide range of endpoints. 23 One systematic review identified 62 questionnaires used among informal family carers in various palliative care settings. 24 These questionnaires included instruments on carer satisfaction, experience (of health services and support), needs bereavement and outcomes. Previous reviews on interventions for informal carers concluded that it was unclear what kind of support was beneficial, partly due to the lack of appropriate outcome measures.21,25
In order to assess the impact of the caring role on carers, an appropriate choice of outcome measures is required; however, selecting suitable and appropriate measures seems a common problem.25 –27 This requires reliable and valid measures with robust psychometric properties, which are appropriate for a palliative care context, as this forms the foundation for evaluating caregiver interventions.
This systematic review aimed to identify and evaluate outcome measures that have been used for informal carers in palliative care studies. The measures used in palliative care are described and their psychometric properties (e.g. reliability, validity, feasibility and precision), when available, are evaluated.
Methods
Search strategy
We conducted a systematic review of carer outcome measures used in palliative care, according to Cochrane guidelines. 28 The databases, Applied Social Sciences Index and Abstracts (ASSIA), the Cochrane Library, Cumulative Index to Nursing and Allied Health Literature (CINAHL), EMBASE, PubMed, PsycINFO, Social Sciences Citation Index and Sociological Abstracts, were searched using four main terms: palliative care, informal carers, outcomes and measures. The search strategy is presented in Table 1 and further detailed search histories are available from the corresponding author on request. All identified citations were imported into the bibliographic database of EndNote, version X5 (Thomas Reuters, New York, NY). Reference lists of the retrieved articles were screened for additional studies.
Search strategy employed in systematic review of studies on psychometric properties of carer-reported outcome measures in palliative care.
Study selection
All types of multidimensional measures (generic, carer-specific for any condition and carer-specific for patients with a specific condition) were eligible for inclusion. The study focused on multidimensional measures as we were interested in measures that assess the overall impact of caring in palliative care rather than measures that assess one specific dimension of outcome or impact. A study was included if all of the following were fulfilled: (1) the study used a self-reported multidimensional measure that assessed caregiver outcomes (i.e. burden, strain or quality of life), (2) measures were directed at unpaid informal carers (e.g. spouse, relatives, siblings, friends or neighbours), (3) the patients they supported were diagnosed with an advanced progressive illness or were receiving palliative care (end-of-life care, terminal care or hospice care), (4) both carers and patients were ⩾18 years old and (5) the study was reported in English.
A study was excluded if any of the following were fulfilled: (1) only unidimensional measures were used; (2) only subscales or individual items and not the full measure were included; (3) only clinician-assessed measures or patient-reported measures were used; (4) all measures completed by carers were on behalf of the patient or (5) it was a qualitative study, comment, editorial, protocol, conference article or grey literature. There were no restrictions regarding publication date and research methods.
Data extraction and analysis
After retrieving all records, the duplicates were removed. All studies were initially screened on the basis of title and abstract, and then on the basis of full-text. Three authors (C.T.J.M., M.B. and M.P.) independently assessed the eligibility of studies: C.T.J.M. assessed all articles, M.B. and M.P. each assessed half of the articles. Any uncertainties were discussed with the other two authors (A.A. and B.W.) and resolved by consensus. C.T.J.M. extracted the data on study characteristics (publication year, country, sample size, research setting, type of disease, intended outcome measure and information on measure) and psychometric charac-teristics. The following information on psychometrics was collected: content validity, internal consistency, construct validity, reproducibility (agreement and reliability), res-ponsiveness, floor or ceiling effects, acceptability and feasibility. As guidance, we used the definitions given by Terwee et al. 29 and Fitzpatrick et al. 30 Additionally, when an included study did not report any psychometric information but referred to other articles regarding a measure or its psychometric values, we assessed these additional articles in order to evaluate the evidence they provided.
Results
Our electronic search, performed on 4 September 2014, identified 8569 studies. Figure 1 provides an overview of the number of studies identified at each stage of the search. After duplicates were removed, 4505 studies were screened on the basis of titles and abstracts, and 231 studies were screened on the basis of full text. This identified 112 studies using 38 different measures for informal carers in palliative care.

PRISMA flow diagram of study selection.
Study and measure characteristics
A total of 112 studies (18 randomized controlled trials (RCTs), 78 observational studies and 16 methodological studies) were included. The methodological studies included translation, development and validation studies about an outcome measure for informal carers in palliative care. The patient population mainly consisted of cancer patients (n = 67, 60%) or a mixture of conditions (n = 29, 26%). Of the studies, 37% were conducted in the United States. Most studies included a mix of spouses, children, parents or friends (n = 99, 88%) and a small number of studies included only spouse carers (n = 4, 3%).
Most studies used only one outcome measure that fit our selection criteria (n = 91, 81%) and 19% of the studies administered two outcome measures to carers. Studies mainly used carer-specific measures only (n = 69, 62%), a quarter used a generic measure (n = 29, 26%), and 14 studies used both types (i.e. generic and carer-specific). In total, 38 measures were identified, including 25 carer-specific measures and 13 generic measures. The main study characteristics are presented in Table 2 and in detail in Supplement 1.
Study characteristics of the included studies (n = 112).
RCT: randomized controlled trial; ALS: amyotrophic lateral sclerosis; ESRD: end-stage renal disease; MND: motor neurone disease; MS: multiple sclerosis.
The most frequently used generic measure was the SF-36 (n = 16, 14%). The most frequently used carer-specific measures were the Caregiver Reaction Assessment (n = 21, 19%), Caregiver Quality of Life Index–Cancer (n = 14, 13%) and the Zarit Burden Inventory (n = 10, 9%). The primary focus of studies using a carer-specific measure was burden (n = 14, 13%), followed by quality of life (n = 8, 7%) and strain (n = 3, 2.6%). An overview of the identified measures and their frequency of use are presented in Table 3.
Identified outcome measures and frequency of use in the included studies.
SF: short form; EORTC QLQ-C30: European Organization for Research and Treatment of Cancer quality-of-life–30-item questionnaire; EQ-5D: EuroQol–5 dimensions; QOLS: Quality of Life Scale; MS: Montgomery Scale; MQOL: McGill Quality of Life Questionnaire; SWED-QOL: Swedish Health-Related QOL Survey; OQOLI: Overall Quality of Life Index; QOLI: Quality of Life Index; WHOQOL: World Health Organization Quality of Life; WHOQOL-BREF: World Health Organization Quality of Life–brief form; CRA: Caregiver Reaction Assessment; ZBI: Zarit Burden Inventory; CBS: Caregiver Burden Scale; BASC: brief assessment scale for caregivers; BCOS: Bakas Caregiving Outcomes Scale; BSFC: Burden Scale for Family Caregivers; CBS-EOLC: Caregiver’s Burden Scale in end-of-life care; CBI: Caregiver Burden Inventory; CIS: Caregiver Impact Scale; FACS: Feelings about Caregiving Scale; HP: Hausliche Pflegeskala; MBCBS: Montgomery Borgatta Caregiver Burden Scale; RCAS: Revised Caregiving Appraisal Scale; BIC: burden index of caregivers; CQOLI: Caregiver Quality of Life Index; QOLLTI-F: Quality of Life in Life-Threatening Illness–Family Carer Version; AQOL-EOL: Assessment Quality of life–End of life–Spouses; CH-QOL-F: City of Hope–QOL Scale–Family Version; FACT: Functional Assessment of Cancer Therapy; HQOLI: Hospice Quality of Life Index; QOL: quality of life; CSI: Caregiver Strain index; FACQ-PC: Family Appraisal of Caregiving Questionnaire for Palliative Care; FSQ: Family Strain Questionnaire.
No information reported and available on psychometric properties.
Psychometrics of measures
More than half of the 112 (n = 60, 54%) studies reported no information on psychometric properties. The 52 (46%) studies that did included 33 observational studies, 15 methodological studies and 4 RCTs. Psychometric data were available for only 23 of the 38 measures including 7 generic measures (i.e. McGill Quality of Life Questionnaire, 142 World Health Organization Quality of Life, 143 Quality of Life Scale, 144 Quality of Life Index, 94 SF-36, 145 SF-12 145 and Swedish Health-Related Quality of life 146 ) and 17 carer-specific measures. These measures consisted of 4–64 items, with a median of 16 items. Table 4 presents an overview of the 24 measures with the available psychometric information. This consisted mainly of information on the Cronbach’s alpha (n = 45, 40%), construct validity (n = 27, 24%), reliability (n = 14, 12%), content validity (n = 8, 7%), responsiveness (n = 8, 7%) and acceptability and feasibility (n = 8, 7%).
Identified psychometric information in studies identified from the search (n=52).
Measure not used in study, +Conference article, $Grey literature, *Non English, ^Review, NR=Not Reported, MES=Methodological study, OBS=Observational study, RCT=Randomized Controlled Trial.
BCOS=Bakas Caregiving Outcomes Scale, BDI=Beck Depression Inventory, BIC=The Burden Index of Caregivers, BSFC=Burden Scale for Family Caregivers, CBI= Caregiver Burden Inventory, CBS=Caregiver Burden Scale, CBS-EOLC=Caregiver’s Burden Scale in End-of-Life Care, CES-D=Center for Epidemiologic Studies Depression Scale, CFA=Confirmatory factor analysis, CGI= Clinical Global Impression, CIS= Caregiver Impact Scale, CQOLI=Caregiver Quality of Life Index, CQOLI-C=Caregiver Quality of Life Index-Cancer, CQOLI-R=Caregiver Quality of Life Index–Revised, CRA=Caregiver Reaction Assessment, CSI=Caregiver Strain Index, ECOG=Eastern Cooperative Oncology Group, EFA=Exploratory factor analysis, ESRD=End stage renal disease, FA=Factor Analysis, FACQPC=Family Appraisal of Caregiving Questionnaire for Palliative Care, FACT=Functional Assessment of Cancer Therapy, HADS= Hospital Anxiety and Depression Scale, IHS =Integrative Hope Scale, HCQ-C= Holistic Comfort Questionnaire – Caregiver, ICC= Intra-class Correlation, ITC=Corrected Item-Total Correlation, LASA= Linear Analogue Scale Assessment, MBCBS=Montgomery Borgatta Caregiver Burden Scale, MCSDS=Marlowe-Crowne Social Desirability Scale, MDD=Major depressive disorder, MOS-SS= Medical Outcomes Study- Social Support, MQOL=McGill Quality of Life Questionnaire, MSPSS=Multidimensional Scale of Perceived Social Support, PSR=Performance Status Rating, PSS=Perceived Stress Scale, QOL=Quality of life, QOLI=Quality of Life Index, QOLLTI-F=Quality of Life in Life Threatening Illness-Family Carer Version, QOLS=Quality of Life Scale, SF=Short Form, STAI= State-Trait Anxiety Inventory, SWED-QOL=Swedish Health-Related QOL Survey, WHO-QOL=World Health Organization Quality of Life, ZBI=Zarit Burden Inventory.
Of the 24 measures, four were originally developed in a palliative care context, that is, the Quality of Life in Life-Threatening Illness–Family Carer Version (QOLLTI-F), 34 the Family Appraisal of Caregiving Questionnaire for Palliative Care (FACQ-PC), 35 the Caregiver Burden Scale in end-of-life-care (CBS-EOLC) 36 and the Caregiver Quality of Life Index (CQOLI). 94 The content validity (which examines the extent to which the concepts of interest are represented by the items 201 ), internal consistency (which measures the extent to which items in a scale are inter correlated 29 ) and construct validity (the extent to which scores relate to other similar measured concepts 29 ) were adequate in all four measures. Interpretability (the degree to which one can assign qualitative meaning to quantitative scores) was not reported in all four studies. The reliability (which concerns the degree to which repeated measurements in stable persons provide similar answers 29 ) was positive in two measures34,94 and negative for FACQ-PC. 35 Floor and ceiling effects (considered to be present if more than 15% of respondents achieved the lowest or highest possible score, indicating that it is likely that extreme items are missing in the lower or upper ends of the scale 202 ) was negative for QOLLTI-F 34 and not reported for the other three measures.35,36,94
For studies (n = 60) that did not report psychometric properties but referred to previous publications about the measure, C.T.J.M. additionally extracted psychometric information from the referenced articles (see Supplement 2). An additional 139 references were assessed for study type, study population and psychometric properties. Although this provided information on how the measures were originally developed, it did not result in additional psychometric information for the measures in the context of carers in a palliative care setting.
Discussion
The aim of this systematic review was to identify and evaluate the psychometric properties of self-reported measures used in informal carers in palliative care studies. A total of 112 studies were found, which used 38 different outcome measures for informal carers. The most commonly used generic measure was the SF-36 (n = 27) and the most commonly used carer-specific measure was the Caregiver Reaction Assessment (n = 21). Psychometric information was available for only 24 carer outcome measures (52 studies). We identified only four measures that were formally tested in a palliative care context.
Measures were mainly used in descriptive studies (n = 78) and the overall study sample sizes tended to be quite small. This could be due to methodological and structural challenges in palliative care research. 203 For example, uncertainties in patients’ prognosis, heterogeneity of the palliative care population, relatively small palliative care centres, ethical concerns or attrition of patients during the study could inhibit research in palliative care.
We noted an increasing trend in the use of measures in informal carers in palliative care. The majority of the included studies were published relatively recently, with more than 70% published since 2008. However, the majority of measures were developed much longer ago, including the most frequently used such as the Caregiver Reaction Assessment 165 or the Zarit Burden Interview. 155 It is therefore unclear whether measures adhere to the current development guidelines, such as those set by the Food and Drug Administration for patient-reported outcome measures. 204 Evaluating publications on the development of these outcome measures was beyond the scope of our review, and the information would have been of limited value as the measures were mainly developed in other carer populations.
Due to the wide range of identified carer outcome measures and the variety of versions of the measures (e.g. Zarit Burden Interview; Table 3), it is difficult to draw overall conclusions about psychometric properties. The most commonly reported psychometric information was Cronbach’s alpha (n = 45, 40%), which is a psychometric property that is commonly used, relatively easy to calculate and easy to interpret. In all, 60 did not report any psychometric information. It was not expected that all studies would contain psychometric information, as the lack of psychometrics was not an exclusion criterion. For studies that did not report psychometric properties but referred to previous publications about the measure, we screened an additional 139 references for information on psychometrics. However, these resulted in limited extra psychometric data, and none of the studies met the inclusion criteria of this systematic review.
Although psychometric information was generally limited, it was even more limited in relation to some psychometric properties such as responsiveness. Responsiveness (or sensitivity to change) is particularly important to highlight as carer-reported outcome measures may be used to assess the effectiveness of interventions. Interventions to support carers in palliative care settings are likely to be complex and require measures that are able to detect change following the intervention.
We identified only four carer-specific measures that were formally developed and tested in this population: QOLLTI-F, 34 FACQ-PC, 35 CBS-EOLC 36 and CQLI. 94 These four measures were used less frequently than either the Caregiver Reaction Assessment or the Zarit Burden Interview that have not been validated in this population.
Regarding the generic measures, none have been formally validated in this carer population but we found psychometric information on seven94,142–146 measures. As these have been widely validated in a large number of different populations, it seems reasonable to assume that they are applicable for carers in a palliative context as well.
It is interesting that limited psychometric information was reported for the most widely used carer-specific measure, the Caregiver Reaction Assessment. 165 This suggests that psychometric properties of the measures may not be the key factor in researchers’ choice of outcome measures. It would be worthwhile exploring in further studies what considerations researchers take into account when selecting their measures and why some carer-specific measures are used more frequently than others, particularly those developed specifically for carers in a palliative context.
Choosing the right measure for a particular study can be challenging because there may be a number of relevant measures from which to choose. 205 A systematic review would be appropriate valuable method to identify the most suitable measure, but it may not always be feasible to conduct a systematic review. Alternatively, as our systematic review highlights, no measure may seem entirely appropriate due to a lack of psychometric information. Additionally, measures may include items irrelevant to the study population, but developing new measures is costly and time consuming. Measure listings such as the Mapi research trust 206 and published systematic reviews can assist in selecting an appropriate measure. 205 Studies in this review did not always reference the measures used or when a reference was provided, it was frequently not the reference of the development of the measure. We encourage authors to reference the original development paper(s) of the measure(s) used and to justify their choice of instrument.
The findings of this systematic review are in line with previously published reviews. Hudson et al. 24 identified 62 tools covering a range of topics including satisfaction, experience, bereavement, needs, preparedness, family functioning and outcomes. Hudson et al. 24 identified a larger number of tools than we did as they included instruments, which we specifically excluded. The review concluded that appropriate tools were lacking but the authors only gave a broad critical appraisal across substantially different types of instruments. In 2009, Whalen and Buchholz 207 identified 74 caregiver burden screening tools for children or adults providing informal care, not specific to a palliative care context. Whalen and Buchholz 207 reported that burden measures might seem appropriate for informal carers but many are lacking psychometric information. Deeken et al. 208 searched MEDLINE and PubMed from 1966 to 2002 and identified 28 tools on burden (n = 17), needs (n = 8) and quality of life (n = 3). Neither Whalen and Buchholz nor the Deeken et al. reviews focused on palliative care. In contrast, our systematic review was conducted in a broader range of databases, specifically focused on self-reported multidimensional carer outcome measures in a palliative care context.
A strength of this systematic review is the comprehensive search of eight databases using four main search terms and no date restrictions, which meant we could collate and examine the variety of outcome measures that have been used with informal carers in a palliative care context. This review shows that although there is an increasing number of studies of informal carers in palliative care, most of the outcome measures used have not been formally validated within this carer population.
Another strength of the review is the care that was taken with regard to the inclusion criterion of palliative care. Palliative care is a complex process and involves a broad spectrum of health care services and treatments. Not all palliative care studies are labelled as such but refer to ‘hospice care’ or ‘end-of-life care’. These search terms were included but provided some challenges. For example, end-stage renal failure is for some patients a chronic disease but when dialysis or treatment is no longer effective, patients need a palliative approach. Two palliative care experts (A.A. and B.W.) independently assessed each study where there was uncertainty to determine whether or not it was in a palliative care population.
A limitation of the review is the exclusion of the grey literature and literature in languages other than English. It is likely that this meant we missed measures published outside the standard academic field or validation studies of translated measures, which might have provided further psychometric information.
A second limitation is rooted in the limitations of literature itself. Limited psychometric information was available, as more than half of the studies (n = 60) did not report any psychometric data. We included all studies that used multidimensional outcome measures in informal carers in palliative care, rather than only development or validation studies, as this corresponded to our study aims. We did not intend to include only development or validation studies, but this may be more appropriate for assessing psychometrics. However, if our inclusion criteria had been limited to development or validation studies alone, only four studies34–36,94 would have been identified. Trends regarding the increasing number of publications on carer outcomes in palliative care would have been missed. As most of the studies did not include psychometric information, we could not critically assess the quality of most of the measures.
Conclusion
Support for patients receiving care is likely to continue to be devolved to informal carers. The WHO has called for health care provision to be extended to families, ensuring their needs, coping and outcomes are addressed alongside those of patients receiving health care services at the end of life. 209 As more interventions are developed to support carers, carers’ outcomes will increasingly be assessed in palliative care context. Although a wide range of measures have already been used in this context, very limited formal psychometric testing has been undertaken. The frequently used measures contain limited psychometric information, while the outcome measures developed or validated in this context are not frequently used in research. Hence, further development and refinement of measures for informal carers in palliative care is required in order to be able to sufficiently support informal carers.
Footnotes
Acknowledgements
The authors would like to thank librarian Nia Roberts (Bodleian Health Care Libraries, University of Oxford, Oxford, UK) for helping us create the syntaxes for the electronic databases.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship and/or publication of this article. Michele Peters is a senior researcher of the Department of Health funded Policy Research Unit on Quality and Outcomes of Person-centred Care Policy Research Unit (QORU), a collaboration between the London School of Economics and Political Science (LSE) and the Universities of Kent and Oxford.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
