Abstract
Since 2009, patient-reported outcome measures (PROMs), measuring symptoms and health-related quality of life, have been routinely completed by NHS patients before and after varicose vein, groin hernia and hip and knee replacement surgery. 1 The arising data are collated by the Health & Social Care Information Centre and published in quarterly and annual reports (see www.hscic.gov.uk/proms). The aim of the programme is to provide patient-centred information with regard to the outcomes of NHS care at provider level, encouraging poorer performing organisations to implement changes focused on maximising patient benefits and outcome.1–3
In theory, such change ought to take place as a result of two pathways.1,2 First, patients may ‘vote with their feet’, using the data to inform decisions regarding the location of their care. This so-called ‘selection pathway’ assumes that resulting losses to the market share of the poorest performing providers will act as a strong stimulus for service improvement. 2 Second, identification of negative outliers, i.e. providers with outcomes significantly worse than the national average, will motivate underperforming trusts to alter models of care: known as the ‘change pathway’.
National PROMs programme participation and linkage rates, April 2013 to March 2014. 5
While the Department of Health has indicated its commitment to supporting the use of PROMs in the NHS in the future, there is recognition that the current national PROMs programme requires review. Consequently, NHS England recently held a public consultation regarding the value of the current system and to gain consensus on the preferred direction of future national PROMs collection. 4
The Centre for Patient-Reported Outcomes Research at the University of Birmingham and Birmingham Health Partners 6 (a strategic alliance between the University of Birmingham and three major teaching hospitals, including the University Hospitals Birmingham NHS Foundation Trust) rapidly convened a stakeholder taskforce to coordinate the joint response to this important NHS consultation.
The taskforce – including patient advocates and individuals with expertise in NHS management, informatics, clinical practice and PROMs research – focused its discussion on the following topics: whether the national PROMs programme had influenced provider performance; how the data were currently used at a local level within the University Hospitals Birmingham Trust; benefits and drawbacks of the system; and opinions regarding the future use of routine PROM collection at a national and local level.
Has the national PROMs programme influenced provider performance?
The available evidence suggests the answer to this question is ‘no'. In 2013, Varagunam et al. 2 determined that the initiative had brought very little change: both inter-provider performance and the proportion of providers deemed to be outliers had not altered significantly over time. Some authors have theorised this apparent lack of impact may be due in part to inadequate provision of feedback to providers/clinicians/patients, that may have failed to meet users’ needs. 2
The taskforce felt another reason could be that clinicians have yet to be persuaded of the evidence underpinning the use of PROMs to inform service change. One of the underlying challenges faced by this and any future PROMs programme will be to convince clinicians – who may be far more comfortable making decisions using quantitative ‘hard’ clinical endpoints – of the worth of PROMs as a driver for change.
In addition, current outlier thresholds may require adjustment to increase the effectiveness of the programme. In 2014, NHS England implemented a ‘best practice tariff’ pay-for-performance scheme for hip and knee replacement surgery, aimed at reducing provider variation. 7 Payment of the tariff is conditional on providers demonstrating an average health gain that does not fall significantly below the ‘alarm’ rate, defined as greater than three standard deviations below the national average. Providers who failed to meet these criteria received procedure payments 10% (£550) below the tariff. While a commitment to link provider pay to patient-centred outcomes is laudable, it seems clear that in its current form, the best practice tariff initiative is only likely to influence the behaviour of a very small minority of trusts. Using the current three standard deviations cut-off, this in fact equates to just 0.1% of providers.
How national PROMs data are used on the ground – a NHS trust viewpoint
Within University Hospitals Birmingham, considerable time and effort is invested in the collection of national PROMs data, both in terms of staff training and time spent on ensuring high completion rates. Logistically, the data are fully integrated into the trust quality assurance infrastructure: National PROMs results are routinely ‘pulled’ from the Health & Social Care Information Centre database and internal reports collated by the trust Health Evaluation Data team, on a quarterly basis, in a dedicated benchmarking PROMs module. The data are then disseminated to trust the Head of Quality Development within an annual quality account report. Crucially, however, it became apparent to the taskforce that, as the trust had never been identified as a negative outlier, the national PROMs data do not appear to have impacted on trust and operational management resource priorities. Moreover, while PROMs information had been disseminated to service leads, the data were not routinely discussed with/utilised by the clinical teams, and thus had no perceived influence on clinical service change or patient care.
It is clear that non-outlying NHS providers may spend considerable effort collecting and collating PROMs data that is, in reality, aimed at influencing a small fraction of the total provider population. Thus, thousands of patients receiving surgery at these trusts may have spent valuable time completing questionnaires for no discernable purpose. The taskforce agreed that future iterations of the PROMs programme should explore ways in which all providers can find value in the data, which leads to both quantitative and qualitative improvements.
How are national PROMs data currently used by patients?
There was little local evidence that PROMs data had influenced patient decisions regarding their choice of provider. The experience of the taskforce was: (i) patients do not know that national PROMs data are available for this purpose; (ii) the data available via the Health & Social Care Information Centre site are not delivered in a format that is accessible to patients, therefore (iii) other aspects influence patient choice such as clinician recommendation, peer recommendation and especially locality. These findings appear to concur with existing research. A systematic review failed to find conclusive evidence that public release of healthcare performance results was a significant driver of patient behaviour with regards to provider selection. 8
Benefits and drawbacks of the current national PROMs programme?
The national PROMs programme has been an important first step in attempting to assess patient-centred health gain within the NHS. Taskforce members recognised the potential benefits of the current programme as a method of comparing providers, to encourage delivery of high-quality care, and as way of generating important research questions aimed at highlighting opportunities for further service improvement. However, the existing system clearly has a number of drawbacks.
PROM questionnaires, predominately collected using paper-based methods, remain resource-intensive to collate and in some cases there is duplication with existing data capture. This was recognised as inefficient for staff/providers and burdensome/unhelpful for patients. Moreover, although PROM results are included in national online HSCIC reports, the taskforce felt their content was not necessarily useful at the local level, with experience suggesting the circle is not closed locally, as clinical staff/patients are not directly informed of the results in an easily accessible format. Thus, patients and clinicians may not appreciate the relevance of the exercise as they do not ‘see’ the results.
The future of PROM collection in the NHS
As a group, the taskforce members strongly supported the continued collection of PROMs in the NHS as a method of putting patient-centred outcomes at the heart of healthcare delivery, but felt significant improvements were required to ensure that the data were capable of facilitating patient benefit.
To encourage full integration of PROMs benchmarking data into service delivery there is a need to:
Improve the efficiency of data collection, utilising electronic capture wherever appropriate. Ensure there is clarity on how the data will be used. This needs to be made explicit in communications with patients. Give providers guidance on how best to interpret and utilise the data. Explore (i) whether the current performance-related financial incentives represent the optimal method of encouraging service change, and (ii) if monitoring other sources of variation, for example consultant-level performance,
9
may be more effective than the existing provider-based model. Ensure the evidence base for PROMs selected for use in the clinical setting is definitive and includes patient input.
The development of current PROM tools requires a rigorous iterative process of expert clinical provider and patient engagement to ultimately deliver a questionnaire capable of accurately measuring outcome. What appears to have been omitted in the current era of national PROMs measurement is a similar rigorous process for determining the methodology of dissemination of the results. All stakeholders, and most importantly patients, should be involved in deciding the nature of PROM result reporting. In addition, the many different ways patients can currently access information about their providers can be confusing, these should be streamlined.
The taskforce believed greater patient benefit/cost-effectiveness could be derived by shifting focus from the current ‘top-down’ national PROMs initiative, to a more efficient ‘bottom-up’ clinic-based collection of PROMs data that could be used for multiple purposes. Such data could be utilised at a macro level, not only to monitor outcomes, but also to inform big-data research, prognostic modelling, post-marketing surveillance and development of patient decision aids. But, importantly, this model may allow instantaneous feedback of PROMs data to patients and clinicians aimed at informing shared decisions surrounding individual patient care, especially in those with long-term conditions. 10
Over 15 million people in England have a long-term condition and the number with multiple conditions is expected to rise to 2.9 million in 2018. 11 Care of such patients absorbs approximately 70% of hospital/primary care budgets in England. 11 Efficient PROM use at the individual patient level, using electronic PROMs, may play a vital role in mitigating the effects of these disorders by providing more meaningful real-time PROMs data with which the clinician and patient can better manage care.12–15
Evidence suggests electronic PROM use in routine care may aid patient–provider communication, support shared decision-making, help patients to feel more involved/empowered in decisions around their care, improve the accuracy of symptom assessment and enhance symptom management.12,13,15–17 electronic PROM interventions have proven efficacious in US cancer patients – significantly reducing accident and emergency attendance, improving health-related quality of life and increasing quality-adjusted survival 12 – and are currently under development in the NHS: in chronic kidney disease and cardiovascular care at the University of Birmingham and in cancer care at the University of Leeds, for example.18,19 The potential use of electronic PROMs to collect data that could be used locally to inform both individual patient care and service evaluation, but that could simultaneously be ‘pulled’ into a national dataset for benchmarking and research purposes, is an exciting prospect and we eagerly await the results of these initiatives.
Finally, next year Greenhalgh et al. are due to publish their groundbreaking realist synthesis providing evidence-based guidance aimed at improving the implementation of PROMs feedback to ensure delivery of the intended service benefits. 20 It is essential that NHS England absorb the lessons provided in this key document when considering the next iteration of PROMs measurement at both a national and local level.
Conclusion
The Centre for Patient-Reported Outcomes Research and Birmingham Health Partners support the routine collection of PROMs as a method of putting the patient voice at the heart of UK healthcare delivery. The national PROMs programme has been an important first step in integrating collection of this valuable patient-centred data within the NHS. However, significant improvements are required to ensure that PROMs data are captured and utilised more efficiently for multiple purposes and, crucially, that it is actually used to improve patient care and outcomes. It is vital that patients, carers, clinicians, service managers and other key stakeholders are fully involved/engaged in the development of any future NHS PROM collection programme to ensure these benefits are realised.