Abstract
Exercise can improve physical function and slow the progression of dementia. However, uncertainty exists around the costeffectiveness of exercise programmes for people with early dementia. The aim of this study was to determine whether a home-based supervised exercise programme (PrAISED – promoting activity, independence, and stability in early dementia) could generate a positive social return on investment (SROI). SROI analysis was conducted as part of a randomised controlled feasibility trial comparing PrAISED with usual care. Wellbeing valuation was used to compare the costs of the programme with the monetised benefits to participants, carers, and healthcare service providers. The PrAISED programme generated SROI ratios ranging from £3.46 to £5.94 for every £1 invested. Social value was created from improved physical activity, increased confidence, more social connection and PrAISED participants using healthcare services less often than usual care. This study found that home-based supervised exercise programmes could generate a positive SROI for people with early dementia. Trial registration: ClinicalTrials.gov: NCT02874300 (first posted 22 August 2016), ISRCTN: 10,550,694 (date assigned 31 August 2016).
Keywords
Introduction
Dementia is a common and debilitating disease resulting in high costs to individuals and society. Public Health England has recognised dementia as a major priority. Studies show that delaying the onset of dementia by 2 years can have substantial economic and societal benefits (Rakesh et al., 2017). Research indicates that interventions that promote exercise and physical activity may slow the rate of dementia progression. A study from Finland reported that a twice-weekly, 12-month, supervised exercise programme at home significantly reduced the rate of decline in activities of daily living for people with dementia (Pitkala et al., 2013). This study reported reductions in hospital admissions and overall costs, suggesting that intensive exercise, with the right support, may be achievable and sustainable for improving the quality of life for people with dementia.
PrAISED is a novel intervention in the UK to determine whether an individually tailored, home-based supervised exercise programme for people with early dementia can be clinically effective and cost-effective (Harwood et al., 2018). In alignment with World Health Organisation guidelines, the PrAISED exercise programme involves a minimum of 150 minutes of moderate to vigorous physical activity per week. In addition, the PrAISED programme involves referring people with early dementia to relevant community groups to encourage physical activity and social participation.
Between September 2016 and March 2018, a feasibility randomised controlled trial (RCT) of PrAISED was conducted at two sites, Derby and Nottingham. Patients with early dementia were randomised to a PrAISED programme consisting of moderate to high supervision (12–50 home sessions within 12 months), or to a usual care group consisting of a brief assessment of falls prevention (1 or two sessions within 12 months). Delivered by physiotherapists, occupational therapists and rehabilitation support workers, the PrAISED programme included assessment of patients, creation of individualised exercise plans, delivery of supervised exercises and activities, and referral to appropriate community activities. To encourage good habit-formation and the continuation of self-directed exercise, home visits were tapered over the 12-month study period with more frequent sessions during the first 3 months.
Patients with early dementia were assessed at baseline and 12 months. The primary outcome measure was activities of daily living (ADLs) measured by the Disability Assessment for Dementia (DAD) scale. Secondary outcomes included physical activity, balance, mobility, fear of falling, quality of life, carer strain, and health service use. Additional methodological details are found in the study protocol (Harwood et al., 2018) and the feasibility trial findings relating to effectiveness found elsewhere (Goldberg et al., 2019). A social return on investment (SROI) analysis of the feasibility trial was specified in the study protocol and undertaken to help assess cost-effectiveness and potential value for money.
Methods
SROI is a pragmatic form of Social Cost–Benefit Analysis (Social CBA), which is recommended in the HM Treasury Green Book to assess interventions that impact social welfare (New Economics Foundation, 2012). Using quantitative and qualitative methods, Social CBA measures and values all relevant costs and outcomes (HM Treasury, 2018). First documented in the early 2000s, SROI methodology has been refined and described in the Cabinet Office ‘A Guide to Social Return on Investment’ (Cabinet Office, 2012). SROI considers what outcomes are relevant and significant to stakeholders and then assigns financial proxies to outcomes which often do not have market values enabling findings to be reported in a common metric (£s).
Using a benefit-cost ratio comparing the value of outcomes with the value of inputs, the reporting of findings is easy to interpret and understandable to a wide audience. With the introduction of the Public Services (Social Value) Act 2012, the UK government is increasingly interested in evaluation methods which capture the social, economic and environmental outcomes of health and social care interventions. In taking a societal perspective, SROI considers the costs and benefits to key stakeholders. A societal approach is useful for informing decision-making within the NHS, which has budgets to be allocated between various programmes.
In this paper, SROI methodology was used to generate a range of SROI ratios using quantitative and qualitative data collected during the PrAISED feasibility trial. SROI methodology involves the following stages: identifying stakeholders, developing a theory of change, evidencing outcomes, valuing outcomes, calculating costs and estimating the SROI ratio. The SROI analysis in this study was conducted in accordance with the 21-item SROI Quality Assessment Framework Tool (Appendix 1).
The first stage was to identify the key stakeholders, which are the people or organisations significantly affected by the PrAISED programme. The PrAISED feasibility study research team, which included patient and public involvement representatives, identified three key stakeholders: patient participants, carer participants and the NHS. It was expected that patient participants would benefit from PrAISED exercises and community referrals; carer participants would benefit from the additional support provided by home visits from PTs, OTs and RSWs; and the NHS would benefit from less frequent health service use from patients participating in the PrAISED programme.
A Theory of Change (ToC) model was created to identify the expected benefits experienced by key stakeholders. The ToC model illustrates the links between the inputs, outputs, outcomes and impacts of PrAISED (Figure 1). Theory of change model.
Outcome Measures for Three Key Stakeholders.
For patient participants and carer participants, two different outcome scenarios were estimated: a base case and a conservative case. The base case considered patient or carer participants who stayed the same or improved over the 12-month study period for a particular outcome. Because dementia is a progressive condition, no change in outcome over 12 months can be considered a positive result. The conservative case, however, counted only those patient or carer participants who improved for a particular outcome over 12 months.
The next stage of SROI methodology was to determine the value of outcomes for each of the three key stakeholder groups. To monetise the outcomes for patient and carer participants, financial proxies were assigned from the Social Value Bank (SVB), which is a databank of methodologically consistent unit costs for outcome indicators (Trotter et al., 2014). SVB is based on ‘wellbeing valuation’, which is recognised in HM Treasury’s Green Book as a robust method for financial appraisal and evaluation.
Wellbeing valuation uses thousands of large UK national surveys to isolate specific variables and to determine the effect of those variables on a person’s wellbeing. Wellbeing valuation then establishes the equivalent amount of money needed to increase a person’s wellbeing by the same amount. In this study, wellbeing valuation was applied to quantify and monetise significant patient and carer participant outcomes for both the base case and conservative case scenarios.
Total costs of the PrAISED programme included the following categories: training costs for therapists; transportation and staffing costs required for therapists to deliver the programme in patient homes, and equipment costs for instructional materials and exercise equipment.
SROI ratios for base case and conservative case scenarios were generated by dividing the social value per participant by the cost per participant. To avoid overestimating the SROI ratio, ‘deadweight’ (i.e., quantity of outcomes that would have happened anyway) is usually considered (Cabinet Office, 2012). However, due to the randomised controlled study design used in the feasibility trial, calculating deadweight was not necessary as the data collected from the usual care patient participants represented ‘what would have happened anyway’. Discounting was also not necessary in this evaluation as all benefits measured were within 1 year of the baseline assessment.
Results
After receiving NHS ethics approval, participant recruitment occurred between September 2016 and March 2017 (Goldberg et al., 2019). Early dementia patients (n = 60) and carers (n = 54) were enlisted from memory assessment clinics, community health venues and online registers. The mean Standardised Mini-Mental State Examination (SMMSE) score for patient participants was 25.6/30. The mean age of participants was 76 years for patients and 68 years for carers. Patient participants were mostly male (57%) and carer participants were mostly female (65%).
Between September 2017 and March 2018, 19 PrAISED patients (with their carers) completed interviews (Appendix 1). Of the 60 patient participants, 82% (49/60) completed both baseline and 12-month questionnaires, which made it possible to determine the percentage who improved, stayed the same, or worsened for each outcome measure. A significant outcome was based on a >10% difference between the percentage of PrAISED and usual care participants who improved for each outcome at 12-months. At 12 months, four significant outcomes were identified:
Outcome 1: Increased Physical Activity Resulting in Improved Activities of Daily Living
Valuing Outcomes for Patient and Carer Participants.
Outcome 2: More Confidence Resulting in Less Fear of Falling
At 12-months, 58% (15/26) of PrAISED patient participants reported no change or less concern about falling compared with 33% (4/12) of usual care patient participants. The conservative case indicated 31% (8/26) of PrAISED patient participants with improved falls efficacy scores compared to 25% (3/12) of usual care participants (Table 2).
Outcome 3: More Social Connection Resulting in Improved Quality of Life
At 12-months, 76% (22/29) PrAISED patient participants reported no change or improved health-related quality of life compared with 46% (6/13) of usual care patient participants. The conservative case indicated 52% (15/29) of PrAISED patient participants with improved EQ5D scores at 12-months compared to 15% (2/13) of usual care patient participants (Table 2).
Outcome 4: Less Carer Strain (for carer participants)
At 12-months, 68% of PrAISED carer participants reported no deterioration or less carer strain compared to 42% of usual care participants. The conservative case indicated 40% (10/25) of PrAISED carers with reduced carer strain compared to 25% (3/12) of usual care carers (Table 2).
Valuing Outcomes for the NHS.
The NHS was responsible for the training and delivery costs for the 12-month PrAISED programme. The equivalent of one full-time therapist (PT/OT) and four full-time RSWs delivered the PrAISED programme to 33 patient participants over the 12-month study period. The therapist and RSWs received two full days of PrAISED training prior to the start of the feasibility study. Training costs were estimated at £583 per therapist/RSW (Appendix 2). The difference in training and delivery costs between PrAISED and usual care patient participants was estimated at £1351 (Appendix 2).
Social Return On Investment Ratios.
Discussion
The results indicated that the PrAISED programme generated a positive SROI ratio ranging from £3.46 (conservative case) to £5.94 (base case) for every £1 invested. These ratios (£3.46 - £5.94: £1) were similar to SROI ratios generated in other related studies. For example, a study of peer support groups for people with dementia generated social value ratios ranging from £1.17 to £5.18 for every £1 invested (Willis et al., 2018); an evaluation of arts activities for people with dementia revealed social value ratios ranging from £3.20 to £6.62 for every £1 invested (Jones, Windle, et al., 2020); and a physical activity intervention for older people with chronic health conditions showed SROI ratios ranging from £2.60 to £5.16 for every £1 invested (Jones, Hartfiel, et al., 2020).
In this SROI of PrAISED, patient participants gained social value from improved physical activity, higher confidence, and more social connection. Carer participants acquired social value from less carer strain. The NHS obtained social value from reduced demand on health services by patient participants during the 12-month study.
In the base case scenario, greater percentages of social value were derived from patient participants experiencing higher confidence (40% of total social value), carer participants reporting less carer strain (22% of total social value), and the NHS benefitting from reduced demand on health services (15% of total social value).
In the conservative case, greater percentages of social value were derived from patient participants experiencing more social connection (29% of total social value), the NHS benefitting from reduced demand on health services (25% of total social value) and carer participants reporting less carer strain (22% of total social value).
Although the PrAISED intervention focused on providing supervised physical activity at home, greater percentages of social value were due to mental and social outcomes (i.e., higher confidence, less carer strain, and more social connection) experienced by patient participants and their carer. Financial values in the Social Value Bank indicate that mental health has a high impact on overall wellbeing. Home visits by PrAISED therapists and subsequent referral to community activities enabled patient participants to keep socially connected to the wider community, which can be crucial for sustaining mental wellbeing and quality of life (Birt et al., 2020).
Strengths of This Study
Although previous SROI evaluations have investigated the social value of arts activities and peer support groups for people with dementia, this is the first study to estimate the social value of physical activity and community referral to people with early dementia and their carer. A strength of this evaluation is the societal perspective and the inclusion of multiple stakeholder groups. Second, the results were strengthened by the randomised controlled study design, which is rare in SROI evaluations. Third, the validity of the results was strengthened from both quantitative and qualitative data collected from baseline and follow-up questionnaires completed by patients and their carer. Finally, the social value ratios calculated in this study were generated from value sets derived from wellbeing valuation, a consistent and robust method recommended in HM Treasury’s Green Book (2018) for measuring social CBA.
Limitations of This Study
First, there are only a limited number of pre-determined outcome values in the SVB derived from wellbeing valuation (Social Value UK, 2015). Not all outcome values in the SVB fit perfectly with the outcomes measured in health-related interventions. For example, high confidence in the SVB is valued at £13,080 per person per year. In our study, high confidence was derived from improvement in the Falls Efficacy Scale and applied to those patient participants who became more confident in walking and doing daily activities. SVB values, however, are typically binary, meaning that they are either applied in full or not at all. SVB valuations do not allow for situations where there may be varying degrees of improvement (Social Value UK, 2015).
Second, a common issue is that social value researchers working with the same data may arrive at different SROI ratios (Fujiwara, 2015). More social connection, for example, could be matched in the SVB with either ‘feeling belonging to neighbourhood’ (£3753 per person per year) or ‘member of a social group’ (£1850 per person per year). Matching outcomes from the study data with the most appropriate SVB value depends on the researcher’s discretion. This can introduce potential researcher bias and the possibility that social value estimates are upward-biased (Fujiwara, 2015).
Third, the PrAISED programme was individualised. Although many of the exercises prescribed were the same, each patient received a personalised programme involving home exercises and referral to relevant community activities. Due to the specific skills of therapists and RSWs, and to the varied needs and abilities of patients, the individual PrAISED programmes differed considerably in the content delivered. For example, participant benefit could have been due to the PrAISED exercises or from community activities such as the parkrun, dementia-friendly swimming or ballroom dancing. The variety of activities prescribed makes it inappropriate to determine whether particular PrAISED activities or referrals were most responsible for participant outcomes.
Finally, although the SROI analysis was mentioned in the feasibility trial protocol (Harwood et al., 2018), the methods of analysis were not pre-specified. In addition, the SROI ratios reported in this study need validation from larger multi-centre trials with higher sample sizes.
Conclusion
This SROI analysis is based on quantitative and qualitative data from the PrAISED feasibility study with 60 patient participants and 54 carer participants. The results showed that a home-based supervised exercise programme with ongoing referral to community activities could generate a positive SROI ratio for people with early dementia. By delivering positive social value for money, PrAISED can help people with early dementia stay more independent, active, and socially connected. To confirm these results, a further SROI analysis will be required on the larger multi-centred randomised controlled trial of the PrAISED programme currently in progress (Bajwa et al., 2019). In addition, future SROI analysis could compare the social value ratios generated from face-to-face and online delivery of PrAISED.
Footnotes
Author Contributions
R.H and R.T.E developed the study concept, acquired funding and contributed to the study design. N.H. analysed the data. N.H. and J.G. drafted the manuscript. All authors contributed to reading and approving the final draft. All authors have read and agreed to the published version of the manuscript.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Institute for Health Research (NIHR) under its Programme Grants for Applied Research funding scheme (RP-PG-0614-20007). The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care. This research was also supported by the NIHR Nottingham BRC and the NIHR Applied Research Collaboration East ,Midlan,ds.
Ethical Approval
Ethical approval was provided by the Yorkshire and The Humber - Bradford Leeds National Research Ethics Service (NRES) Committee Reference 16/YH/0040.
