Abstract
Objectives
To investigate the relationship between patient experience assessed through surveys of random samples of practice populations and intermediate outcome targets in those patients with diabetes, collected in the Quality and Outcomes Framework pay-for-performance scheme.
Design
Cross-sectional study.
Setting
The East Midlands region of England.
Participants
Six hundred and twenty-nine general practices.
Main outcome measures
Logistic regression models were used to assess whether practice-level reports of patient experience of access and consultations were associated with achievement of treatment targets (HbA1c of 7.5% and 10% or lower, BP 145/85 mmHg or lower, and cholesterol 5 mmol/L or lower) in people with diabetes. Survey respondent characteristics (ethnicity, age, sex) and practice size, deprivation, and prevalence of diabetes and obesity were also assessed within the models.
Results
Patient experience of practice populations explained little of the variation in diabetes treatment targets. In the practice survey, the proportion of respondents who had seen a nurse in the last 6 months was associated with increased likelihood of achieving HbA1c of 7.5%, and being involved in decision-making or having tests and treatment explained were associated with achievement of HbA1c of 10% or less, cholesterol of 5 mmol/L or less, and BP of 145/85 or less.
Conclusions
Although patient experience at practice level should be included in monitoring outcomes, it should not replace monitoring clinical outcomes in diabetes. A mix of clinical and patient experience measures will have to be used to monitor outcomes in general practice.
Introduction
The proportion of people with diabetes reaching treatment targets for control of blood pressure, cholesterol and HbA1c is variable. 1 Factors affecting the achievement of these intermediate outcomes include patient characteristics such as age, co-morbidity, ability to self-care, lifestyle and deprivation. 2 Quality of healthcare is also important in achievement of outcomes.
Quality of healthcare has been divided into access to care and the effectiveness of care. 3 In the care of diabetes, there is evidence that access to a nurse providing enhanced structured care, and patient-oriented consultations that include education and involvement, can improve intermediate outcomes. 4–8 The Quality and Outcomes Framework (QOF), a pay-for-performance scheme introduced in the UK in 2004, includes treatment targets for people with diabetes, and achievement of the targets for blood pressure, HbA1c and cholesterol has improved. 9 An annual survey of patient experience of each general practice has been undertaken in England from 2006. 10 The survey provides information at practice level on patient perceptions of the service provided by their practice. The survey scheme is incorporated into the QOF, contributing to the calculation of payments to practices.
Following a change in government in 2010, increased emphasis is being placed on outcome measures in the monitoring and management of services, 11 and it is proposed that patient experience should form one of the five key domains for monitoring outcomes. It is anticipated that patient experience surveys will assess whether care is safe, high quality and coordinated. However, it is not known how patient experience reported by samples of practice populations relates to treatment targets based on intermediate clinical outcomes among subsets of those populations. Furthermore, use of a large set of outcomes measures would place an undue burden on practices, and the interpretation of the wealth of findings would be difficult. If outcome monitoring is to be used in managing services, it is important to understand how different outcome measures relate to each other, and how much weight should be placed on individual measures. For example, should healthcare organizations devote greater effort and resources to meeting targets for clinical outcomes than to improving patient experience? Therefore, we undertook a study to investigate the relationship between outcome treatment targets among patients with diabetes and reports of patient experience of practice populations. Both of these measures have potential for use in the national monitoring scheme.
The study is part of a programme of work investigating means of monitoring outcomes in primary care populations, in relation to health service innovations such as the collaborations for leadership in applied health research and care (CLAHRCs). 12,13 Our hypothesis was that patient experience of access and consultations, measured through surveys of practice populations, would be associated with achievement of targets for intermediate outcomes among the subset of patients with diabetes. Specifically, we hypothesized that practices characterized in patients' surveys as providing better access to nurses and greater patient education and involvement in consultations would be more likely to achieve targets for intermediate outcomes in people with diabetes (including both type 1 and type 2 diabetes mellitus). If the hypothesis was shown to be correct, it would suggest that the numbers of measures used in monitoring could be limited by including those that predict findings in a range of other outcomes.
Method
Setting
The study was based on publicly available data relating to nine primary care trusts (PCTs) in the East Midlands of England. All 629 general practices in these PCTs were included; we excluded primary care walk-in centres.
Access to care, patient and consultation characteristics
The GP patient survey is a nationally administered survey of random samples of registered patients aged 18 or over in general practices. 10,14 We used the 2008–2009 survey that included questions on access and consultations. The national survey was administered by the market research company Ipsos MORI, and was sent to 5.7 million patients registered with 8273 practices in England in January 2009. Patient samples were obtained for each practice using PCT registration records, including only patients aged 18 years or over who had been registered continuously with the practice throughout the previous 6 months. The sample size was chosen to deliver a confidence interval of +/–7 percentage points for questions asked of patients who had an appointment in the last six months, patients being selected on a ‘1 in n’ basis after sorting into age and gender bands. The average sample size per practice was 689. Alternative methods of questionnaire completion included online and in 13 non-English languages.
In our study, we included questions from the survey on access (including access to nurses, as an indicator of structured care) and on consultations. Questions on access were: getting through on the telephone, trying to see a doctor fairly quickly (i.e. within 2 working days), being able to book an appointment more than two full days in advance, last seeing a doctor more than six months ago, preferring to a see a particular doctor and how frequently, satisfaction with opening hours or would like additional opening times, ease of getting an appointment with a nurse and whether they had seen a practice nurse in the last six months. Questions on consultations, for both doctors and nurses, were: giving you enough time, asking about your symptoms, explaining tests and treatments, and involvement in decision-making. The survey also collected data on respondents' ethnicity, sex and age group, overall satisfaction, and the patient's perception of their own health. We used the practice index of multiple deprivation (IMD 2007) as the indicator of deprivation. 15
Diabetes intermediate outcomes
Publicly available data were obtained from the 2008–2009 QOF on practice list size, prevalence of diabetes (the proportion of patients recorded on practice diabetes registers) and the recorded prevalence of obesity (proportion of adults in practice obesity registers, obesity defined as BMI of 30 or more). 16,17 From the QOF, we used the proportion of patients in each practice achieving the targets for intermediate outcomes, specifically cholesterol of 5 mmol or less in the last 15 months, blood pressure of 145/85 mmHg or less, HbA1c at 7.5% or less and HbA1c 10% or less in the last 15 months. It should be noted that both the survey data and the diabetes data were available at the practice level only and hence those answering the survey do not necessarily correspond with those with diabetes outcomes.
Statistical methods
Descriptive analysis was undertaken to summarize the data using means and confidence intervals. Univariate logistic regression was then undertaken in order to assess whether practice characteristics (list size, deprivation, prevalence of diabetes and obesity) and findings from the patient experience survey including the proportion of survey respondents aged less than 65 years, the proportion who were men, patient experience of access and consultations, and response rate to the survey were associated with the outcome of interest. Survey response rate varied between practices and we wished to determine whether there was any evidence of bias, particularly any association between response rate and achievement of outcome targets. Outcome measures were: (1) HBA1c <7.5%; (2) HBA1c <10%; (3) cholesterol 5 mmol or less; (4) BP 145/85 or less, all in the last 15 months.
The patient experience survey was completed by samples of practice populations rather than only by patients with diabetes. In the analysis, the patient experience variables were regarded as characteristics of the practice, and we sought to determine whether these characteristics also explained intermediate outcomes in diabetes. Therefore, multivariate models were constructed using stepwise procedures with a P value of 0.05 taken to indicate that a variable should be included in the model. All analyses were carried out using SPSS for Windows (version 16).
Results
The response rate to the GP patient survey varied between the 629 practices from 5.4% to 65.6% (mean 44.7%). The mean practice diabetes prevalence was 4.36%. Table 1 presents descriptive data on practice characteristics and patient experience. Although patient experience was generally positive for some questions (e.g. 94.2% [95% CI 86.5–98.5%] expressed trust in the doctor), for others there was wide variation between practices (e.g. 67.1% [95% CI 0.0–85.5%] reported that the nurse involved the respondent in decisions). Univariate associations of practice characteristics and patient experience with intermediate outcomes are shown in Tables 2 and 3. For HbA1c 7.5% or less, there were significant associations with deprivation, diabetes prevalence, survey response rate, trying to see a doctor fairly quickly (i.e. within 2 working days), last seeing a doctor more than 6 months ago, trust in doctor, seeing a nurse in the past 6 months, the consultation with the nurse, and perception of good health. For HbA1c 10% or less, there were significant associations with deprivation, practice list size, response rate, able to get an appointment more than two days in advance, satisfaction with opening times, features of consultations with doctors and nurses, and perception of good health. For cholesterol 5 mmol or less, there were significant associations with deprivation, prevalence of obesity, response rate, access (getting through on the telephone, opening times, appointments), and features of consultations with doctors and nurses. For BP 145/85 mmHg or less there were significant associations with the prevalence of obesity and having seen a nurse in the past 6 months.
Characteristics of practices, proportion of patients with diabetes achieving quality and outcomes framework targets, and responses to the patient survey
Univariate analyses of practice features, patient experience, and intermediate outcomes. HBA1c 7.5% or less in the last 15 months and HbA1c 10% or less in the past 15 months
Univariate analyses of practice features, patient experience, and intermediate outcomes. Cholesterol 5 mmol less in the last 15 months and BP 145/85 or less in the last 15 months
Results for the multivariate logistic regression are shown in Table 4. The findings explained only a small proportion of the variation in achievement of outcome targets among practices, the highest proportion of variance explained being only 14%, for HbA1c 10% or less. For HbA1c of 7.5% or less, increasing deprivation was associated with fewer patients achieving the target, although higher prevalence of diabetes and higher proportion of patients reporting seeing a nurse in the past six months were associated with higher achievement. For HbA1c of 10% or less, increasing deprivation, increasing proportion of survey respondents who were white and higher prevalence of diabetes were associated with lower achievement, but higher survey response rate and a higher proportion of respondents reporting that nurses involved them in decisions were associated with higher achievement.
Multivariate analysis of practice features and patient experience associated with intermediate clinical outcomes
Better cholesterol control was significantly associated with higher response rate to the survey, proportion of patients able to book an appointment with their practice in advance, increased recording of obesity on practice registers, and a higher proportion of respondents reporting that nurses had explained tests and treatments. Blood pressure of 145/85 mmHg or less was more likely if respondents reported that doctors involved them in decisions and if practices recorded more people with obesity, but less likely with increasing prevalence of diabetes and if respondents reported that doctors gave them enough time.
Discussion
Principal findings
In this study, we related patient experience of samples of practice populations with achievement of diabetes intermediate outcome targets in people with diabetes. The findings provide some support for our hypothesis that practices characterized as providing better patient experience would be more likely to achieve diabetes intermediate outcomes targets. Having seen a nurse and being able to arrange an appointment in advance, both aspects of access to care, were associated with some outcomes. Being involved in decisions and receiving explanations about tests and treatments were aspects of consultations also associated with achieving targets. However, the strength of the associations was weak, and the total variance explained by the models was less than 10% for three of the four indicators. The finding that patient reports of consultations being long enough was associated with lower achievement of the blood pressure target may appear to contradict our hypothesis. However, the finding may reflect evidence that patient-centredness is less effective in achieving treatment targets in diabetes than structured disease management. 18
Strengths and weaknesses of the study
Some practice and organizational features potentially predicting diabetes outcomes may have been omitted from the study. For example, postgraduate experience and education of staff and levels of equipment may influence intermediate outcomes. The study is limited to describing associations, and does not allow conclusions on causation. The GP access survey contained patient perceptions of access rather than objective measurement of specific features of access such as consultation availability. Furthermore, only a few aspects of access were addressed in the survey, mainly the ability to call on services. The survey response rate varied widely between practices and was low (mean of 47%) although typical of the levels now achieved in postal surveys in primary care. The survey is administered to random samples of practice populations, and not exclusively to patients with diabetes, and therefore we have investigated characteristics of practices as perceived by patients rather than the specific experiences of patients with diabetes. This may partly explain why the models accounted for only small proportions of the variance. The findings should not be generalized to the outcomes of patients with long-term conditions other than diabetes.
Implications and future research
This study has a number of implications for the monitoring of outcomes in primary care, as proposed recently for the NHS. First, the finding of an albeit weak association between the findings of a practice survey of patient experience and achievement of targets for intermediate outcomes in a subgroup of those populations lends some justification to the use of surveys to monitor practices. It also provides some justification for attempts by practices to improve patient experience, particularly features of structured disease management, including the provision of information and involvement in decision-making in consultations. Second, the finding should also encourage those developing patient experience surveys to include questions on aspects of care shown by research to be associated with clinical outcomes. Both doctors and nurses in practices should be assisted in developing these aspects of their consultation skills.
Third, it is necessary to continue to monitor clinical outcomes when it is practical to do so; in view of the weakness of the association between clinical outcomes and experience, it would be unwise to deduce that a practice with positive patient experience also has good intermediate outcomes in diabetes, or that a practice with good clinical outcomes is providing a positive patient experience. Our findings do not suggest that the burden of monitoring outcomes can be reduced by recourse to a general measure such as patient experience, but instead a mix of measures is required, including clinical outcomes. Consequently, continued research is required to develop practical measures of outcome in general practice.
DECLARATIONS
Competing interests
None declared
Funding
RB is partly supported by the NIHR CLAHRC for LNR, and this project was undertaken while JS and MA were attached to the CLAHRC. The views expressed in this paper do not necessarily reflect those of the Department of Health or NIHR
Ethical approval
Not applicable
Guarantor
RB
Contributorship
JR prepared the data, undertook the analysis and prepared the first draft of the paper; MSA advised on the conduct of the study, and took part in the interpretation of the findings and revision of the paper; CW advised on and completed the statistical analysis, and took part in the interpretation of the findings and revision of the manuscript; RB supervised the study and took part in the interpretation of the findings and revision of the paper
Acknowledgements
None
Reviewer
Azeem Majeed
