Abstract
Objectives
Positive psychology outcome measures aim to quantitatively document the character strengths that people use to maintain their wellbeing. Positive aspects of caregiving including the use of character strengths is gaining credence in dementia carer literature but there remain few psychometrically robust tools by which to capture this. The current study evaluated the psychometric properties of a newly developed measure of hope and resilience for family carers of people living with dementia.
Methods
An online study where family carers (n = 267) completed the newly adapted Positive Psychology Outcome Measure – Carer version (PPOM-C), the Hospital Anxiety and Depression Scale – Depression subscale (HADS-D), The Short Form Health Survey (SF-12), and The Multidimensional Scale of Perceived Social Support (MSPSS).
Results
Psychometric analysis indicated strong properties for the PPOM-C in family carers, with two items dropped to improve the internal consistency. Convergent validity was established, with strong correlations between the hope, resilience, depression symptomology, quality of life and social support. A Confirmatory Factor Analysis indicated acceptable model fit.
Discussion
The PPOM-C is a psychometrically robust tool that can be recommended for use in large scale psychosocial research. The use of this measure in research and practice will provide a more nuanced understanding of the caregiving role and how to support wellbeing in this population.
Introduction
Many interventions with carers of people with dementia evaluate their efficacy using instruments based on the reduction of negative emotions or states. Frequently, these include burden, depression and stress (Dickinson et al., 2017). In addition, research often includes instruments measuring family carers’ quality of life. The relationship between reducing scores on instruments measuring negative states and an increase in quality-of-life measures has been established (Piersol et al., 2017). However, there are a wide variety of positive and negative constructs and emotions contributing to quality of life (Keyes et al., 2012). Contemporary research often omits robust measurements of positive psychological traits. This is important as if intervention seeks only to alleviate negative emotional states, an opportunity to make meaningful positive changes to people’s lives is missed (Seligman, 1998).
Positive psychology researchers seek to determine which positive emotions or traits are assisting the individual (Seligman et al., 2005). Whilst this research began in the United States of America, it has spread across the globe and across clinical populations (Saeedi et al., 2019). For example, a sense of fulfilment has been shown to lessen negative states such as burden in family carers (Grant & Nolan, 1993). Through targeting an increase in positive domains, it is hoped specific interventions will develop to bolster these traits and thus, increase quality of life.
Hope has been defined as ‘a confident yet uncertain expectation of achieving future good, which to the hoping person is realistically possible and personally significant’ (Dufault & Martocchio, 1985, p. 380). This seems relevant to the caring context, and indeed, is a key resource for family carers of people with dementia (Snyder et al., 2000). Qualitative interviews with carers suggest an important coping mechanism involves renewing their sense of hope on a daily basis. This process has been speculated to include acceptance, looking for the positives and seeing possibilities for the future (Duggleby et al., 2009). Despite hope being viewed by family carers as valuable, it has been used sparingly in research with this population (Duggleby et al., 2013).
Resilience is a multi-faceted construct but usually refers to an ability to recover or bounce back after an adverse event (Windle, 2011). Other definitions embed resilience within social and cultural contexts in addition to resources and assets possessed by the individual (Windle & Bennett, 2011). Resilience is a relevant factor for family carers as stressors for this population have been well documented (Pinquart & Sörensen, 2003). Despite resilience serving to mitigate stressors faced by family carers, it is not measured routinely (Gaugler et al., 2019).
A newly established positive measure in use for people living with dementia is the Positive Psychology Outcome Measure (PPOM) (Stoner et al., 2018). This was developed due to a lack of positive instruments available for people living with dementia. It was developed following a systematic review of the positive psychology literature and interviews with people with dementia and their carers. The PPOM includes two subscales measuring hope and resilience. There has been a lack of consensus in outcome measurement in dementia care making comparisons amongst interventions problematic (Clarke et al., 2020). In addition, Clarke et al. (2020) noted a paucity of robust positive measures available for family carers. This study aims to fill this gap by administering the PPOM to a sample of family carers to obtain an independent psychometric analysis. It is necessary to obtain such analyses such as reliability, floor and ceiling effects, convergent validity, and sensitivity to change (Rothrock et al., 2011). These data provide meaningful information to researchers about the validity of the measure and can be helpful when inferring change has occurred within interventions. In this context, validity refers to ‘building a case that an instrument functions effectively in a particular population for a specific purpose’ (Rothrock et al., 2011, p. 3).
The validity and reliability of the PPOM was analysed with 225 people with dementia as part of its original development with this population (Stoner et al., 2017, 2018). An exploratory factor analysis and subsequent confirmatory factor analysis (CFA) yielded a two-factor model with acceptable fit indices. Each factor contained eight items and were labelled hope and resilience. The PPOM achieved good temporal stability in a test-retest sample of 48 people and excellent internal consistency. Construct validity was achieved through statistically significant correlations with the Control, Autonomy, Self-realisation and Pleasure scale (Hyde et al., 2003), Geriatric Depression Scale short form (Yesavage et al., 1982) and the Quality of Life in Alzheimer’s Disease Scale (Logsdon et al., 1999).
Aims
The purpose of this study was to complete a series of psychometric analyses assessing the validity and reliability of the PPOM instrument in carers of people with dementia, henceforth termed ‘PPOM Carer version (PPOM-C)’. This analysis included a CFA to determine if the sixteen-item, two-factor model previously found (Stoner et al., 2018) remained consistent in a sample of family carers. Construct validity was determined by convergent validity testing with theoretically linked constructs and a binary logistic regression.
Method
Design
An online study was conducted with people who self-identified as carers of people living with dementia. Ethical approval was obtained from the University College London (UCL) Research Ethics Committee (Ref: 15,139/001). This study consisted of a battery of measures for respondents to complete at baseline and a retest for a subsample of 50 participants 4 weeks later. Invitations to participate in the retest were sent out to completers of the first battery until 50 responses had been collated. This was optional for the participants. The number for the retest was selected in accordance with guidelines on instrument development and validation (Terwee et al., 2007), which was adequate to establish test-retest reliability of the PPOM in the original study (Stoner et al., 2018). All measures were self-completed online by participants using Qualtrics.
Participants
Participants were recruited from the Join Dementia Research (JDR) register (www.joindementiaresearch.nihr.ac.uk) and non-governmental organisations (Age UK and the Alzheimer’s Society). To participate, respondents must have met the criteria below. The JDR is a joint venture between the National Institute for Health Research and several Alzheimer’s research organisations. It promotes dementia research with affected persons and their carers.
Inclusion criteria
To be eligible for the study, participants must have self-identified as an unpaid family carer for a person living with dementia. They must have been over 18 years old, able to communicate in English and be capable of providing informed consent.
Procedure
The inclusion criteria were applied to the JDR database and a list of eligible volunteers was obtained. Volunteers were contacted via email inviting them to participate. Within the advert was a link directing them to the battery of measures hosted on Qualtrics. This advert was also circulated to other organisations including Age UK and the Alzheimer’s Society. These organisations independently disseminated the link to its members.
After respondents answered questions relating to the inclusion criteria, they were presented with information regarding General Data Protection Regulation (GDPR, 2018) and the Data Protection Act (2018). All participants had summary sheets relating to these policies appear on screen and had several opportunities to download them. Participants also had the opportunity to contact the research team prior to giving consent. The battery of outcome measures was then presented in a counterbalanced fashion to control for order effects.
Outcome measures
Respondent demographics
Data were collected with respect to the respondent’s age, gender, ethnicity, marital status, employment, education, diagnosis the person living with dementia had received, kinship to the person with dementia, and the length of time they had supported the person with dementia. Respondents were asked whether they lived with the person they provided care for and if they identified as the primary carer.
The positive psychology outcome measure (PPOM)
The PPOM is a measure of hope and resilience developed for people with dementia. It was developed using both qualitative and quantitative methods involving people with dementia, their carers, healthcare professionals and experts (Stoner et al., 2017). It has excellent internal consistency α = 0.94 (Stoner et al., 2018). It demonstrated significant positive correlations with the Quality of Life Alzheimer’s Disease scale (Logsdon et al., 1999) (QoL-AD; r = 0.627) and significant negative correlations with the Geriatric Depression Scale (Yesavage et al., 1982) (GDS; r = −0.699). Test-retest reliability was ‘good’ for the PPOM over a 1-week period (Intraclass Correlation Coefficient = 0.88). Items are scored on a five-point Likert scale (0–4) and are based on the last month. A sample item from the hope subscale is ‘I have a positive outlook on life’. A sample item from the resilience subscale is ‘I can bounce back’. A score of zero indicates this is ‘not true at all’ whilst a score of four implies this is ‘true nearly all of the time’.
Hospital anxiety and depression scale (HADS; Zigmond & Snaith, 1983)
The HADS is a 14-item instrument measuring anxiety and depression. In a recent psychometric analysis, the anxiety subscale was found to be unsuitable for family carers (Stott et al., 2017). Thus, only the depression subscale was used to evidence convergent validity between the PPOM and the HADS. The HADS depression (HADS-D) subscale is a self-report measure consisting of seven items. Each item is scored from 0 to 3 with several questions negatively scored. Total scores between 0 and seven are labelled ‘normal’ whilst a total score between 8 and 10 are considered ‘borderline’. A total score of 11 or over indicates ‘clinical caseness for depression’.
The short form health survey (SF-12; Ware et al., 1996)
Given the lack of a gold standard quality of life measure in use with family carers, the SF-12 was selected as it has been used extensively with this population (Farina et al., 2017). The SF-12 is a health-related quality of life scale measuring eight concepts in health questionnaires: physical functioning, role functioning, physical pain, general health, vitality, social functioning, emotional and mental health. The SF-12 is psychometrically sound, comprehensive and correlates highly with the SF-36 (Ware et al., 1996). It consists of two subscales: the physical component score and the mental component score.
The multidimensional scale of perceived social support (MSPSS; Zimet et al., 1988)
The MSPSS is a 12-item self-report instrument designed to measure perceived social support. Each item is scored on a 7-point Likert scale ranging from 1 (very strongly disagree) to 7 (very strongly agree). These scores are summed with higher scores indicating greater perceived levels of social support. The MSPSS was developed using a sample of undergraduates (Zimet et al., 1988) but has been used with family carers (Charlesworth et al., 2008). It possesses good psychometric properties used across a range of different target populations (Zimet et al., 1990).
Statistical power
There are no concrete guidelines with respect to the number of participants required for a validation study. Given this study made use of Pearson’s r correlations to ascertain convergent validity, G*Power 3.1 (Faul et al., 2007) was used to calculate that a minimum 193 respondents would be required to detect a small effect size (r = 0.2). This sample size provides 80% power with a corresponding alpha level of 0.05. With respect to factor analyses, Terwee et al. (2007) recommend multiplying the number of items in the measure by seven (n = 112) as a minimum figure. The final sample size exceeds both criteria.
Emails were sent to participants who had completed the initial battery to complete a subset of the measures 4 weeks later on an optional basis. Invitations were sent until 50 responses had been collected. A 4-week timeframe was selected to reduce the chance of practice effects and new life stressors interfering with scores at the second timepoint.
Statistical analyses
All analyses were computed using SPSS version 26. If respondents terminated their participation before completion of the battery, this was assumed to be a withdrawal of consent and all data for the participant was deleted. Descriptive and frequency statistics including means and standard deviations were obtained for all demographic information and scores on the PPOM-C. Floor and ceiling effects were evaluated by analysing the percentages of the lowest and highest scores on the PPOM-C and its subscales. Floor and ceiling effects were considered absent if less than 15% of participants scored the minimum or maximum score possible (Terwee et al., 2007).
To evaluate temporal stability, an Intraclass Correlation Coefficient (ICC) was used to assess agreement over a 4-week period. ICC scores would be deemed excellent (>0.90), good (0.75–0.90), moderate (0.50–0.75) or poor (<0.50) (Koo & Li, 2016). It was hypothesised the ICC would be good over a 4-week period based on previous findings (Stoner et al., 2018).
Cronbach’s alpha analyses were used to determine internal consistency for the PPOM-C and its subscales. Internal consistency would be deemed excellent (α = 0.90 < 0.95), good (α = 0.80–0.89), or acceptable (α = 0.70–0.79) (Terwee et al., 2007).
Pearson’s r correlations were used to assess convergent validity of the PPOM-C (the extent to which theoretically related constructs are observed to be correlated through instruments of these constructs). It was hypothesised the PPOM-C and its subscales would be negatively correlated with the HADS-D and positively correlated with the MSPSS and both subscales of the SF-12.
The PPOM-C has undergone acceptable exploratory factor analysis and CFA in a previous study (Stoner et al., 2018). Therefore, another CFA was deemed appropriate for this dataset. CFA analysis used SPSS Amos version 25. Model of fit indices included examining the (i) chi-square and p value (low chi-square and high p value indicate good fit), (ii) standardised root mean square error of approximation (RMSEA; RMSEA <0.08 indicates good fit), (iii) comparative fit index (CFI; > 0.90 indicates good fit), (iv) standardised root mean square residual (SRMR; SRMR > 0.08 indicate good fit) and (v) Average Value Explained (AVE; > 0.50 indicates acceptable amount of variance explained by factor/s) (Hooper et al., 2007; O'Rourke et al., 2013).
Results
Participants
Carer demographics.
Carer characteristics.
Initial internal consistency of the PPOM-C
The internal consistency of the PPOM-C indicated multicollinearity with α = 0.951. The hope and resilience subscales demonstrated excellent internal consistency with scores of α = 0.915 and α = 0.927 respectively. The analysis demonstrated that any deleted item would have brought the internal consistency score to an acceptable score of less than α = 0.95 (Terwee et al., 2007). As such, further analysis in the form of a CFA was needed to identify items to be deleted.
Confirmatory factor analysis
Variations in instrument type and indices of fit.
This resulted in a 14-item PPOM-C (Table 3). Factor loadings indicated all items loaded onto their respective factors significantly (Figure 1). The CFA also demonstrated that hope and resilience shared some variance, as in keeping with the original factor structure in people with dementia. Factor loadings varied from 0.755 to 1.408 and the CFI, SRMR and AVE statistics all fell within acceptable ranges for the PPOM-C. The average value explained by each factor was R
2
= 0.61 for hope and R
2
= 0.62 for resilience. The RMSEA and chi-square statistics fell outside of accepted guidelines, but further exploratory analysis failed to identify a superior model fit. As all items had significant loadings for the PPOM-C, and the CFI, SRMR and AVE indicated acceptable model fit, the PPOM-C was considered adequate, and all subsequent analyses were conducted on this 14-item version. CFA Factor Loadings.
Revised internal consistency of the PPOM-C
By removing two items that were contributing to multicollinearity and lower standardised factor loadings, internal consistency for the revised PPOM-C was improved (α = 0.948). The hope and resilience subscales also had excellent internal consistency with scores of α = 0.912 and α = 0.918 respectively (Table 3). No deleted items would have improved the internal consistency score further.
Descriptive statistics
The possible score of the PPOM-C ranged from 0 to 56. The observed range varied from 2 to 56, with a mean of 38.65 and standard deviation of 10.98. The distribution of the PPOM-C (Figure 2), showed the skewness of the data to be approximately symmetric (−0.455), with the kurtosis value being −0.014. No respondents scored the lowest possible score (0), and 13 participants attained the highest score (56). This number equated to 4.9% of the sample and thus, floor and ceiling effects were deemed to be absent. Distribution of the PPOM-C.
Test-retest reliability
The number of days between the test and the retest for the 52 respondents ranged from 28 to 43. Most of the respondents (75%) had completed the retest 29 days after time 1. Reliability across a 4-week period was found to be ‘excellent’ (ICC = 0.908, 95%CI = 0.845, 0.946). Reliability was ‘good’ for both the hope subscale (ICC = 0.891, 95%CI = 0.817, 0.936) and the resilience subscale (ICC = 0.874, 95%CI = 0.790, 0.926).
Convergent validity
Pearson’s r correlations were employed with an adjusted p-value of 0.004 (0.05/12) calculated using the Bonferroni correction. The PPOM-C was significantly negatively correlated with HADS-D (r = −0.66, p < 0.001) in addition to the hope and resilience subscales (r = −0.67, p < 0.001; r = −0.58, p < 0.001). The hope and resilience subscales were positively correlated with the SF-12 mental component score (r = 0.62, p < 0.001; r = 0.57, p < 0.001) in addition to the PPOM-C (r = 0.63, p < 0.001). The PPOM-C, and its hope and resilience subscales were significantly correlated with the SF-12 physical component score (r = 0.19, p = 0.002; r = 0.17, p = 0.004; r = 0.19, p = 0.002). Lastly, the MSPSS was significantly correlated with the PPOM-C (r = 0.39, p < 0.001), the hope (r = 0.45, p < 0.001) and resilience (r = 0.29, p < 0.001) subscales.
Discussion
The newly evaluated 14-item PPOM-C possesses acceptable psychometric properties and is suitable for use in further research with family carers. Two items from the PPOM were dropped to improve both the internal consistency and the goodness of fit indices. Internal consistency of the PPOM-C and its subscales were excellent. It also possessed excellent test-retest reliability over a 4-week period. This suggests that whilst there was a small fluctuation over 4 weeks, family carers’ levels of hope and resilience remained stable.
All correlation coefficients were statistically significant and in the expected direction whilst using a Bonferroni correction to control for type 1 error. The correlation coefficient between the PPOM-C and the depression construct was similar to previous findings (Stoner et al., 2018). It was noted the PPOM-C held larger correlations with the mental component score of the SF-12 than the physical component score. This suggests hope and resilience have more in common with a person’s mental health quality of life than physical health quality of life. Correlations between the PPOM-C, the HADS-D and the mental component score were all in excess of r = 0.60 and therefore, considered strong (Evans, 1996).
The comparative fit index (CFI), standardised root mean square residual (SRMR) and the average value explained (AVE) by each factor each indicated good model fit. In addition, all items loaded significantly onto their respective factors. Only the chi-square and standardised root mean square error of approximation (RMSEA) statistics indicated less than adequate model fit. The significant chi-square test (indicating poor model fit) is a common phenomenon within CFA as it is influenced by sample size (Kline, 2011). Some structural equation theorists have proposed models should be evaluated according to the theory underlying the measures in addition to using fit indices as a guide (Hooper et al., 2007). The PPOM has previously demonstrated satisfactory evidence of its items being developed by the theories underpinning hope and resilience (Stoner et al., 2017). Thus, on the balance of the evidence collated in this study, the two-factor model proposed for the PPOM-C was concluded to be adequate, although this could be explored further in future large-scale studies.
Implications for practice
This paper is to the authors’ knowledge the first time an instrument measuring two positive psychology constructs has been validated in carers of people with dementia. Given the limited positive psychology measures available for family carers (Stansfeld et al., 2017), the arrival of an instrument capable of measuring two constructs is timely. Due to a lack of robust alternate measures, widely adopting the PPOM-C in practice could aid in comparing outcomes in interventions internationally (Moniz-Cook et al., 2008).
The PPOM-C represents a prompt method of assessing family carers who may be low in hope and resilience. This information could be vital for auxiliary services to mobilise around such carers and offer intervention or support. Conversely, it offers an opportunity to learn more from carers with high levels of these traits. Qualitative approaches could shed further light on which psychosocial resources and assets lead to differing levels of hope and resilience.
There were significant correlations between the HADS-D, MSPSS, the mental component score and the PPOM-C subscales indicating good convergent validity. This indicates that hope and resilience are linked to the wellbeing of family carers. This suggests interventions targeting such positive constructs may also bring about positive changes in quality of life, perceived social support and depression. The literature comprehensively supports this given that hope is a key psychosocial factor in being a family carer (Snyder et al., 2000), managing dementia (Moniz-Cook et al., 2009), Acquired Immune Deficiency Syndrome (Moon & Snyder, 2000) and cancer (Magaletta & Oliver, 1999). Resilience has also been shown to be useful for family carers (Gaugler et al., 2007), safeguarding against post-traumatic stress disorder (Waysman et al., 2001) and preventing developmental psychopathology (Masten et al., 1990).
Methodological problems and limitations
The psychometric analysis here was quantitative in approach. Though the majority of validation criteria depend on quantitative statistics, criteria such as content validity also require qualitative engagement with the targeted population.
The respondents within this survey aligned with what Heinrich et al. (2010) classified as WEIRD (White, Educated, and lived in Industrialised, Rich, and Democratic countries). The demographics of the sample did not reflect the United Kingdom diversity with respect to race or education levels (Office for National Statistics; ONS, 2016). For example, 96.6% of this sample were white compared to 86% of the UK population. In an attempt to counteract this, the researchers sent research invitations to all black and minority ethnic carers listed on the JDR database. The recruited sample stayed in education for longer compared to UK census data with 57.7% educated to degree level or higher compared to 27% of the UK population. A majority of participants here were also female and caring for a parent with dementia. Whilst daughters as oppose to spouses may have been over-represented here, this is broadly in line with carer demographics in the UK, in which female carers make up 60%–70% of informal carers. (Alzheimer's Research UK, 2015). It is therefore not known how generalisable these findings are to other populations within the UK.
Future research
A qualitative study designed to canvass family carers’ views on the suitability of each item on the PPOM-C would be valuable. This would serve to increase its content validity with family carers. Such research could take the form of a focus group with family carers to ascertain their views on each item.
This study has demonstrated the PPOM-C has excellent temporal stability but was unable to establish its responsiveness. Responsiveness is an important psychometric property that determines whether an outcome measure can detect clinically important changes following intervention (Terwee et al., 2007). Including the PPOM-C in research-based interventions would ascertain if this criterion is acceptable. The PPOM is currently being used in the Promoting Independence in Dementia ‘PRIDE’ Research Programme (Shafayat et al., 2019) which may indicate whether the PPOM has adequate responsiveness in people living with dementia. Research has found hope and resilience are responsive to intervention in family carers (Duggleby et al., 2018; MacCourt et al., 2017). Thus, the PPOM-C would be expected to demonstrate this quality.
Conclusions
The PPOM-C is a short self-report measure that gives insights into the degree of hope and resilience a carer possesses. It possesses excellent internal consistency, excellent temporal stability, good convergent validity, and adequate factor structure. The PPOM-C illustrated significant correlations with quality of life, perceived social support and depression indicating that hope and resilience play a role in the wellbeing and assets possessed by family carers. Utilising the PPOM-C in interventions will give data regarding its responsiveness. It is hoped that adopting the PPOM-C will influence future interventions to be more strengths focused in their approach. A copy of the PPOM-C is obtainable from the corresponding author on request. It is free to use but should be appropriately referenced.
Footnotes
Acknowledgements
The authors would like to thank all the respondents who took the time to complete their study. In addition, we would like to thank the Join Dementia Research register who were immensely helpful during the data collection phase of this study.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Data availability
Data is stored with University College London (UCL) Digital Collections, in line with [blinded for review] records retention policy. Access to this data within the European Economic Area will be considered on a case-by-case basis following an email to the corresponding author. A copy of the PPOM-C can be obtained from the corresponding author on request.
Ethical Approval
Ethical approval was obtained from University College London (UCL) Research Ethics Committee (Ref: 15,139/001).
This involves working with over 34 countries and supporting CST for dementia training centres in Hong Kong, the US, Denmark, Norway, Germany and New Zealand. Her expertise lies in the global development and evaluation of complex interventions for dementia. This includes currently being PI for the MRC funded ‘CST-International’ in Brazil, India and Tanzania and holding a UCL-PKU (Peking University) Global Engagement award. Her research includes the evaluation of a range of interventions for dementia including Cognitive Behavioural Therapy, Compassion Focused Therapy and Mindfulness. She has published over 140 peer-reviewed papers, 4 book chapters and 10 books. She is international lead for University College London’s Clinical Psychology doctorate course.