Abstract
Objectives
To demonstrate a statistical method to enable the identification of key drivers of quality from a patient perspective that can be used by service providers to help drive improvement.
Design
Crosstabulation, Chi-square analysis and Cramer's V calculation using SPSS software of NHS Inpatient Surveys 2006 and 2007.
Setting
The NHS Inpatient Survey is a standardized survey designed by the Picker Institute conducted on a sample of patients across all acute care hospital trusts in England.
Participants
The surveys (available from the UK Data Archive) provide anonymized patient data for over 77,000 patients in 2006 and 72,000 patients in 2007.
Main outcome measures
Cramer's V score testing associations between patient ratings on multiple components of care and ratings on the overall quality of care.
Results
Of the 58 questions analysed, some questions correlate more strongly with overall satisfaction of care than others and there is strong agreement of results over the two years. Of the top 20 rated components, communication (both between professionals and between professionals and patients) and trust engendered by that communication is a recurring theme.
Conclusions
Hospital trusts are required to develop quality indicators and collate detailed feedback from patients in addition to the annual inpatient survey to measure these. To make best use of resources, additional data collection should focus on those aspects of care of most importance to patients locally. This analysis demonstrates a statistical technique that can help to identify such priority areas by showing those aspects of care most strongly associated with the overall rating of care. The analysis uses national level data to demonstrate how this can be achieved. This shows the importance to patients of being treated with dignity and respect, and good communication between staff and between staff and patients.
DECLARATIONS
None declared
None
Not applicable
CD
CD conducted the statistical analysis and produced draft text for comment; JR and DB provided substantial comment and advice on revisions to text and interpretation of results; TW provided expert statistical advice on analysis and interpretation of results
Veena Raleigh
Introduction
In the UK, the measurement of patient experience of healthcare is becoming increasingly important. In policy terms it now has equal billing with patient safety and clinical care as a driver of quality.1,2 The collection and rigorous analysis of data on patient experience are seen as necessary to identify strengths and weaknesses in service delivery, to transform working practices and drive quality improvements. 3 Under the ‘NHS Next Stage Review’, by June 2010 every trust needs to produce ‘quality accounts’ to report on safety, effectiveness and patients’ experience of care. 3
The NHS Inpatient Survey, administered by acute care trusts since 2002, is an important source of data on patients’ perception of care. In this standardized survey, patients rate multiple aspects of their care from admission to discharge. Trusts use the results to identify their strengths and weaknesses and receive annual reports identifying where they score above or below a national average on various components of care. This provides useful data for local trusts looking to improve the quality of their service. The items measured in the NHS survey have been validated as important indicators of quality from patients’ perspective and below-average scores should be a cause for concern. But as a way of identifying key priorities for improvement there are limitations to this approach. Several components of care may score below average but this study contends that not all are equally important as drivers of quality. For example, patients who did not like one aspect of care (such as hospital food) may still think highly of the care they received overall because of how doctors and nurses communicated with them.
Service providers need additional analysis to identify what areas matter most to their patients and where scarce resources should be concentrated to help raise quality. This study demonstrates a straightforward statistical analysis of NHS Inpatient Surveys that can help provide more direction and precision to the important search for key drivers of quality.
Methods
For the NHS survey, patients choose a point on a scale to indicate how positively or negatively they perceive quality of components of care such as communication, pain management or cleanliness (‘component’ questions). They are also asked to give an overall rating of care (‘the overall question’, stated as ‘Overall, how would you rate the care you received?’).
This study assumes first that while all aspects of care measured in the NHS survey are important in their own right, some aspects of care will matter more to patients than others. A previous study conducted by the Picker Institute, where patients were asked to rate the comparative importance of different aspects of care, supports this assumption. 4 The second assumption is that the relative importance of some aspects of care to how a patient feels about their care overall will be partly reflected in the strength of the statistical association between the component questions and the overall question.
Patient-level data from the NHS Inpatient Survey were obtained directly from the UK Data Archive 5 for the years 2006 and 2007 (the most recent data available at the time of analysis) representing over 77,000 and 72,000 individual responses, respectively. These data were statistically analysed to measure the strength of association between answers to questions on components of care and the overall rating of care.
Using SPSS, cross-tabulation was applied to show in tabular form the relationship between answers to the component questions and the overall question. Chi-square analysis was then applied to measure how much the scores conformed or deviated, followed by Cramer's V calculation. Cramer's V is a commonly used measure of the strength of association between variables in a Chi-square analysis. Scores indicate the strength of association between two variables with 0 as the weakest possible association and 1 the strongest.
The study assumes the stronger the association, the more important it is likely to be for patients as a determinant of quality. To validly compare Cramer's V scores, answers to questions with varying scales were first re-categorized to ensure all cross-tabulations were standardized as 3 x 2 tables. For example, 3-point and 4-point scales were re-categorized as 2-point scales such as ‘positive’ and ‘negative’. The study also assessed the conformity of responses between the two years.
Results
There were 58 concordant questions between 2006 and 2007 representing 86% of the question pool and these questions were used in the analysis. The gender profiles were similar in both years. The age profile differed slightly with a higher percentage in the 16-35 years age category and 81+ years category in 2007 compared to 2006 (5.6% and 2.1% higher, respectively).
For the 58 questions tested Cramer's V scores ranged from 0.04 to 0.63. For illustration, Table 1 presents the ranking by strength of association for the 20 component questions most closely related to the overall question. All measures are statistically significant with p < 0.01.
Ranking of component questions against overall questions
This shows consistency of results over the two years, with the same 20 questions ranked in the top 20 for both years, with minor differences in the weighting.
Explaining differences in Cramer's V scores
A higher Cramer's V value indicates that positive or negative answers in a component question more closely reflect positive or negative answers in the overall question than a lower score. To illustrate what the difference between Cramer's V scores means in this context, cross-tabulation tables for two variables are compared. Table 2 shows a crosstabulation between patients’ experience of being treated with dignity and respect and their overall experience of care (Cramer's V score of 0.64) and Table 3 compares their views on privacy against overall care (Cramer's V score of 0.29). Table 2 shows that 53% of those who felt they were not treated with dignity and respect also rated the care they received as poor. For Table 3, 12% who were not given adequate privacy rated overall care as poor. This difference of 40 percentage points suggests that the first component of care is currently a bigger priority. This does not mean to suggest that those with lower scores are unimportant to patients. They may reflect an aspect of care that is being delivered well and patients now take for granted. In the example provided, a failure to maintain standards of privacy for patients may then be reflected in a higher score.
Crosstabulation of ‘treated with dignity and respect and overall rating of care (Cramer's V score of 0.64)
Crosstabulation of ‘privacy when being examined or treated’ and overall rating of care (Cramer's V score of 0.29)
Discussion
Principal findings
These results demonstrate that an analytical approach can help identify key drivers of quality from the patient's perspective and thereby provide a more effective focus for quality improvement for hospital trusts. This aggregated national-level analysis shows the importance to patients of being treated with dignity and respect, good communication both between health professionals and between professionals and patients or their families and having trust in doctors and nurses treating them. Pain control, help with eating meals, cleanliness, staff numbers and privacy also score highly.
Strengths and weaknesses of the study
This study assumes that when a patient completing the inpatient survey considers how they feel overall about their episode of care, the answer will reflect to some degree the opinions expressed about the components of care. The ‘overall’ question was chosen only as the one most likely to reflect people's experiences of individual components. It was not specifically designed to provide a precise measure of individual components of care and the interpretation of results, therefore, need to be treated with caution.
The analysis shows the likelihood but not the certainty that some variables are bigger drivers of patients’ perceptions of quality than others at a point in time. It demonstrates associations between component questions and the overall questions but is not conclusive evidence that one causes another. To conduct the Cramer's V analysis some of the scales in the questions have been collapsed into binary categories. Also, this study has only included data from two annual surveys and hence may miss other aspects of patient experience that could be as important. However, this analysis assesses the responses of over 140,000 patients and shows remarkable consistency between the two years.
Strengths and weaknesses in relation to other studies
Bearing in mind the caveats expressed, comparison with previous research partly supports the assumptions made in the study that tests of association between component questions and the ‘overall’ question will help distinguish between more and less important issues for patients at a point in time. A comprehensive survey conducted by the Picker Institute, asking patients to rank the importance of components of care, produced similar results to this analysis with an emphasis on trust in hospital staff, effective communication, privacy and cleanliness. 4 There were some exceptions to this, with the highest ranking items for this study (dignity and respect and doctors and nurses working well together) ranked 28th and 24th in the Picker study. 4 This may be due to methodological differences. While the NHS Inpatient Survey asked patients to rate their experience of their most recent episode of care, the Picker Institute survey asked patients to consider quality of care more broadly by scoring the importance of 82 components of care using a Lickert scale.
Key drivers of quality, such as communication, dignity and respect and trust have also been identified in other studies.6,7
The Care Quality Commission (CQC) produce annual analyses of the NHS Inpatient Survey at a national and hospital trust level. 8 This shows the percentage of patients who rate questions positively or negatively and highlight for trusts how these headline scores compare to national averages. Testing the relationships between component questions and the overall question, this study complements and adds value to the CQC analysis enabling a clearer assessment of the most likely important drivers of care at a point in time that service providers may need to urgently investigate.
Meaning – clinicians, policy and NHS management
Emerging policy suggests that acute hospital trusts will need to agree quality indicators locally on patients experience and gather more detailed real-time patient feedback to measure these. 9 This process will place increasing demands on staff and patients’ time and, given the current financial climate, needs to be focused on key areas that matter most to patients locally. 10 Without a targeted approach, trusts risk drowning in data and creating ‘survey fatigue’ among patients.
The technique demonstrated in this study provides a pragmatic tool to help address these concerns. It attempts to add value to the NHS annual inpatient survey by enabling trusts to focus on a smaller number of apparently key patient concerns. While evidence from Table 1 indicates broadly what issues trusts should prioritize, the relative importance of components of care will vary locally.
A high Cramer's V score, especially if it is combined with a comparatively low national average CQC rating, is likely to signify an aspect of care in need of immediate extra management attention. This could help identify aspects of care where additional monthly or quarterly data collection to complement annual survey results are likely to be most beneficial. Regular detailed patient feedback on a small pool of questions could help staff better understand causes of patient discontent and devise action to address them.
Service providers should interpret results with caution. A high Cramer's V score is not a precise measure of importance. Also a low Cramer's V score does not signify an aspect of care that can be safely ignored or neglected. It could signify an area where the quality of care is good and where standards now taken for granted need to be maintained.
Footnotes
Acknowledgements
None