Patients’ attitudes and perceptions of two health-related quality-of-life questionnaires used to collect patient-reported outcome measures in the English National Health Service: A qualitative study of patients undergoing cardiac interventions

Abstract

Objectives:

To explore patients’ views on the EuroQol-5D and Coronary Revascularisation Outcome Questionnaire, tools currently used for collecting patient-reported outcome measures in the English National Health Service. The key questions were as follows: (1) whether patients consider them sensitive enough to detect change in their health after cardiovascular disease interventions and (2) whether they consider the health-related quality-of-life questions as meaningful.

Methods:

Data were collected on patients’ views using focus groups. We held four focus groups selecting participants on the basis of their baseline and follow-up EuroQol-5D scores. Data were analysed using framework analysis and grounded theory.

Results:

Focus group participants confirmed that they had derived substantial health benefits from their cardiac interventions despite the lack of measurable effects on the EuroQol-5D scores. Participants felt that the EuroQol-5D questionnaire was limited because of the following reasons:

• Their health fluctuates from day to day.

• They had difficulty assessing their general health status on the visual analogue scale.

They felt that the Coronary Revascularisation Outcome Questionnaire was limited because of the following reasons:

• They did not understand the clinical terms used.

• The impact of tiredness on their quality of life was not captured.

• They were unable to distinguish between the effects of their heart condition and other health issues.

Additionally, neither questionnaire considers the adjustments people have made to their domestic arrangements to improve their health-related quality of life.

Conclusion:

This study provides evidence that the two questionnaires do not capture some aspects of health that patients consider important. Furthermore, the presence of co-morbidities masks the symptoms relating to the heart disease and the effect of their cardiac interventions. Future work on patient-reported outcome measures should consider developing new questionnaires that address these major concerns.

Introduction

Internationally, there is a growing desire to understand how health can be measured. The traditional ways of reporting health and treatment outcomes are gradually being complemented by patient-reported health-related quality-of-life (HRQL) outcomes. ¹ HRQL is a general term that describes the overall impact of a disease, illness or condition on the health and well-being of the affected individual. ²

There is a growing understanding that patients are likely to be the best source of information about how they feel. ² In the United Kingdom, some health-care organisations have been measuring HRQL routinely. ² The Department of Health in England is developing a programme of patient-reported outcome measures (PROMs) using HRQL questionnaires to allow benchmarking of hospital performance. ³ The Department of Health in England decided to collect PROMs using the EuroQol-5D (EQ-5D) measure ^1–7 as the generic/utility questionnaire together with disease-specific questionnaires for most of the procedures in the national programme. PROMs for cardiac interventions have used both EQ-5D and the more recent Coronary Revascularisation Outcome Questionnaire (CROQ) as the disease-specific tool. ⁸

In a local survey using an EQ-5D questionnaire, HRQL was determined for 822 patients undergoing cardiac interventions including coronary artery bypass graft (CABG) and angioplasty or other coronary interventions. The survey questionnaires were administered in patients 1–2 weeks before and then repeated at 6 months after cardiac interventions. The EQ-5D scores indicated that approximately 20% of patients had very good HRQL before receiving their cardiac intervention and reported the same at the post-operative stage (i.e. suggesting there was no measurable benefit). This observation highlighted a possibility that the EQ-5D questionnaire may not be sufficiently sensitive to some patients’ perception of their health status and the health benefits of their cardiac intervention. Our observation from the survey results reflected recent concerns that PROMs may not fit the purpose of collecting patient care outcomes.^9–12 Controversies have come to light in terms of the science, the scales used and also the measurement approaches taken.^9–12 The most common concern is that many of the scales used in PROMs were originally developed for use in the context of clinical trials to assess the treatment effectiveness rather than for use to assess individual patients’ clinical care outcomes. ¹² Quality-of-life scores were therefore used as a proxy measurement of clinical performance of an individual hospital or health-care provider.

The number of health status questionnaires has increased dramatically over the past decades, and many investigators are faced with the difficult dilemma of selecting the right one that fits the purpose. ¹³ Indeed, some studies have highlighted the variability in the relationship between quality-of-life measurements obtained from generic questionnaires compared with clinical outcomes. ¹⁴ In addition, it was observed in clinical studies that the performance of different questionnaires in the measurement of quality of life varied considerably depending on patients’ socio-demographic and morbidity factors. ¹⁵ Despite the existing guidelines from a number of consensus groups,^16–18 many studies have reported problems with identifying the right tool.^9–12 Selecting instruments is based on the reported measurement performance, including evaluation of the validity of constructs being measured and the ability of the instrument to detect meaningful change. Patient acceptability and feasibility of data collection are key factors in maximising participation and response rates.^16–18

In a preliminary discussion with our service users’ representatives within our hospital, the sensitivity of a selection of questionnaires (EQ-5D,^1–7 Seattle Angina Questionnaire (SAQ), ¹⁹ and MacNew Quality of Life Questionnaire ²⁰ ) in capturing participants’ perception of their health outcomes following an intervention was questioned.

Subsequently, the Department of Health announced the use of the CROQ for the PROMs programme for cardiac interventions; hence, we are interested in exploring its performance among patients undergoing cardiac interventions. We therefore conducted an exploratory qualitative study of patients’ attitudes and perceptions of the EQ-5D and the CROQ with regard to capturing their health status before and after their cardiac intervention.

Methods

Ethics

Favourable ethical approval was granted by the local Research Ethics Committee (North West 1 Research Ethics Committee, Cheshire) and by our institution’s Research Committee in 2010.

Study design

The study was carried out in a single centre, tertiary cardiac referral centre in the North West of England. Focus groups were used to obtain data on the views of participants. The dynamics of focus group discussion allows participants to be prompted to consider issues that they may not have done in a one-to-one interview. However, the public nature of focus groups may mean that people are less open than in a one-to-one interview. Data from focus group discussions may not be as rich as data from one-to-one interviews, but it is a cost-effective and time-effective means of collecting data.

There were four focus groups designed to reflect the pre- and post-operative patient population and their baseline EQ-5D scores:

• Focus group 1: Post-operative patients with the highest baseline EQ-5D scores

• Focus group 2: Post-operative patients with the lowest baseline EQ-5D scores

• Focus group 3: Pre-operative patients with the highest baseline EQ-5D scores

• Focus group 4: Pre-operative patients with the lowest baseline EQ-5D scores

Sample

We chose a sample from patients undergoing CABG and angioplasty because these procedures are high-cost, high-volume cardiac interventions performed at many tertiary cardiac referral centres and also because they were the subject of a national PROMs pilot project. ¹⁸ People were not excluded on the basis of existing co-morbidities in order to reflect ‘real-world’ conditions. The sample was purposively selected from the geographical areas closer to the hospital to optimise participation.

Recruitment

Invitations were sent out to 176 patients in total (Appendix 1). Of these patients, 103 did not respond or declined participation, 55 patients gave informed consent to participate but could not attend for various reasons and 18 of these attended the focus groups. Summary of characteristics of patients who were invited to participate in the focus group are presented in Appendix 1. Details of the focus groups are also summarised in Figure 1 and Appendices 1 and 2.

Figure 1.

A summary flow chart of the way the participants were recruited into the study.

People who consented to take part were mailed two questionnaires, that is, the EQ-5D and the CROQ (pre- and post-procedure versions for CABG and percutaneous coronary intervention (PCI)), 2 weeks before the focus groups and were asked to study them in detail. Key domains are summarised in Appendix 1. The full questionnaires EQ-5D and CROQ can be viewed online at the following websites, respectively:

1. http://www.euroqol.org/fileadmin/user_upload/Documenten/PDF/Products/Sample_UK__English__EQ-5D-3L.pdf

2. http://heart.bmj.com/content/90/12/1460/suppl/DC1

Conduct of the focus groups

Each participant was asked to give his or her opinion on each of the questions in turn to ensure that everyone’s views were captured. With the participant’s permission, the focus groups were recorded using a digital recorder from which verbatim transcripts were made. Field notes arising from the interview process were documented.

Analysis

Final coding and analysis were then undertaken by an experienced qualitative researcher (S.H.). In this study, a framework analysis ²¹ approach was taken. Each individual questionnaire item provided the ‘framework’ for analysing and reporting the data. Additional themes were developed using grounded theory from general discussion in the focus groups. Analysis of the data was undertaken using ATLAS.ti (computer-aided qualitative data analysis) software.

Findings were checked by several members of the focus groups and the Service User Research Endeavour Group (the Service Users Research Endeavour Group is made up of former patients interested in supporting research activities with the hospital) for resonance with their knowledge and experience and for readability.

Findings

The main finding of the study was that people who had scored highly on the EQ-5D questionnaire before their cardiac intervention and therefore could not show any improvement on the EQ-5D scores nevertheless reported during the focus groups that they had derived substantial health benefits from their procedures (Box 1). This observation highlighted the lack of consistency between patients’ perception of HRQL after an intervention and their scores in the EQ-5D questionnaire.

Box 1.

Participants with prior good/excellent EQ-5D scores

Q: How did the operation benefit you?

A: Less angina, easier activities, strenuous activities, that’s the main benefit. (Don)

Q: OK, can you give a bit more detail, on what the difference was before and after your operation.

A: Er, some tightness after the heart attack, and then total satisfaction afterwards from the point of view of nothing in tightness and being able to walk and knowing you weren’t going to be short of breath. (Marjorie)

Q: So could you walk further then?

A: Prior to the operation I had chest pain that came on when I was either rushing or if it was very cold or if I was under some kind of stressful situation and that hindered my getting around quite a bit. That’s completely gone now. (Bernard)

Q: What sort of thing couldn’t you do before but you can do now?

A: Er, run up steps two at a time, I used to do that and get breathless, well now I can do it and it’s no problem. (Norman)

Similarly, participants who had ‘very poor’ EQ-5D scores prior to and after their cardiac intervention and therefore could not show any improvement on the EQ-5D scores also reported during the focus groups that they derived substantial health benefits (Box 2). This observation also highlighted the lack of consistency between patient’s perception of HRQL after an intervention and their scores in the EQ-5D questionnaire.

Box 2.

Participants with prior poor EQ-5D scores

A: Prior to the heart attack, I had a full and active life, it came on suddenly, I was admitted to hospital, on two occasions, the third time they referred me here to hospital … I had the angioplasty and I’ve been fine and well ever since. And very satisfied with the treatment I received. (Malcolm)

Q: What sort of problems did you have initially before the operation?

A: Er, I couldn’t do things that I wanted to do, you know with having heart problems, but since I’ve had the heart operation I feel more relaxed, better in myself. Not that I’m going to run a hundred miles like you know but, it’s given me life, and you know I’m getting on and I’m grateful for it. That’s a life saver and I’m just grateful. (John)

Q: You can play football now?

A: Yeah, all sorts. All strenuous exercise, jogging things like that, I was getting this sort of stitch that I get in your side, I was getting that in the chest. And I thought, I wasn’t out of breath or anything, I just thought that wasn’t right. Anyway, since Dr S got this blockage enlarged, or removed, they put a stent in don’t they, and since then no problem whatsoever. (David)

Patient’s perception of EQ-5D questions

This EQ-5D questionnaire required the participants to indicate which of the options best described their health state in terms of their mobility, self-care, engaging in usual activities, managing pain/discomfort and anxiety/depression and scores on the visual analogue scale. The results from all focus groups representing a sample of maximum variation are reported here, showing no differences in the nature of responses between the participants of the different focus groups. Overall, all the patients expressed opinions that the response options were inadequate (Boxes 3–7) and that they failed to account for the patient’s perspective. They fail to take into consideration relevant issues in relation to the patient’s quality of life: the behavioural change and the adaptations in the life conditions that some patients have undertaken.

Box 3.

Self care

The self-care question, any problems?

I just wonder if there isn’t more to do with self-care than those three answers, particularly with more subtle things like getting out of breath, unable to stand up quickly, dizziness and so on, because washing and dressing yourself you can do at your own pace, you can take time. (Bob)

As I say, I don’t think it’s comprehensive enough, you know to cover the varying things. Because one day you can have a wonderful day and the next you can have a lousy day. (Martin)

I put ‘I have moderate problems washing and dressing myself’. My problem is that I do not have a bath anymore because I cannot get myself up out of the bath because of the spondylitis in my spine. (Martin)

That is exactly the same for me, I had to get my bath taken out and a shower put in, you know, a walk in shower. (Joan)

Box 4.

Usual activities

I think, in my opinion you should, as regards leisure activities, that should be different from household activities, or house work. Because for some people leisure activities might just be going for a walk where as other people might go to the gym or swim. I think it’s entirely different to what you normally do at home. (Laura)

I concur completely with the comments on gardening because I do quite a lot of it but pacing yourself, you have to physically, but studying or working at the sort of administration of the home is no problem so I, when I first saw that I was inclined to think that word ‘study’ should perhaps be in a category of its own, … I don’t find it difficult to study but I find it difficult digging trenches! (Bob)

I’ve put no. 2 (response) because I like DIY but over the last two years my health has deteriorated that much, that I can’t do the normal things. I’ve even sold a three bedroomed house that I used to do gardening in, for a bungalow that’s got no gardens. It’s that bad. (Martin)

Box 5.

Pain and discomfort

You see, there again, today I’m having a reasonable day. Tomorrow with my cervical spondylitis, I could turn my head and could be in agony. (Martin)

The only time I get a little bit of pain is when I go swimming, a little bit of pain on the top here, and what I do, I just ease off for a bit, maybe 10 minutes something like that then I carry on and I think, seem to think it clears itself but the most pain I ever get, any sort of pain is when I walk up an incline. (Bert)

It’s because it (EQ-5D) comes from the heart hospital, you just assume it’s cardiac. If it had come from a general hospital then, you know, you would have put things differently. (Laura)

If I say I have no pain or discomfort I’d want to add a little bit there say, occasionally yes, but it’s completely under control which it wasn’t before … I suppose I’m looking for something in between, that it may be too bald as a question. (Bob)

Box 6.

Anxiety and depression

I’m sure I ticked (3) on this and the only reason I did that is I’ve never suffered from depression before to my knowledge and even when my doctor told me that I’m suffering with it, I had problems believing it, simply because I don’t understand it. (Adrian)

But what causes you to be depressed? What causes the anxiety? … If today was Saturday and my team lost at football I’d be depressed (laughter). (David)

Box 7.

Visual analogue scale

Yeah, it makes sense but it’s all done on particular day though isn’t it … you know you could be having an off day or a roaring day, yeah. (Alistair)

I was only going to say I have real difficulty with this kind of question, I would say at the beginning of the day I’d put it at 90 and at the end of the day I’d put it at 1 (laughter). Seriously though, I find it very, very hard to know where to put a cross, so I went optimistic and went about 60 but it was because I’m an optimist, not because I could really answer the question. (Bob)

I just, under my own pace I felt alright I’d say 80 but if you’d asked me earlier this morning when I went for the papers I’d say about 40, you know, yeah. (Gordon)

I feel perfectly well but I’m aware there’s a heart condition that hasn’t been cured, the symptoms are being managed, therefore that would bring it down a bit, I wouldn’t know where to put, I’d probably put 85 or something like that. It might not be the right answer. (Bernard)

It says, on the scale ‘good or bad your own health is today’ – I’m sure other people have health problems apart from the heart problems, and is that related just to the heart problems or is it just to general health? (Don)

I marked it 95 on the day because that’s was the day I had my operation on my foot and I thought it was just cardiac (laughter). (Laura)

And I would put 99 because from this point of view (pointing to heart) fine, this point of view, (pointing to hip) not fine. (Marjorie)

Some limitations of the EQ-5D questionnaire

In summary, there are two major limitations of EQ-5D identified by the focus groups. The first limitation identified was the period of reference chosen for the questionnaire, that is, ‘today’. This was not considered appropriate because patients often experience good and bad days and also in the same day there can be great variations on the perceived health state. Therefore, to comment only about ‘today’ is not relevant to them as it is not giving any useful indication of how their overall HRQL is. In addition, there were problems using the visual analogue scale to evaluate the general state of health ‘today’ and in translating the feelings about their health onto a numbered scale.

CROQ

The results presented here are focused on questions 1–3, 5 and 7 where most controversial comments were made as summarised in Boxes 8–13. Participants considered some of the questions to lack specificity as summarised in Box 8, for example, under what conditions they have shortness of breath.

Box 8.

Question 1

See, tomorrow I could put ‘quite a bit’ and today I’d put ‘moderate’. (Martin)

But have you noticed that the question says ‘over the past 4 weeks’ just at the top of that question here on the side?

  Oh yeah. (Martin and Joan)

I have a couple of comments on that one regarding, ‘shortness of breath and angina pain’ and what I’ve said on shortness of breath, but under what conditions is it walking, climbing stairs or sport? You’d be short of breath sometimes with that. (Bernard)

Box 9.

Question 1 (continued)

Sometimes you’re not awfully sure that you’ve got this angina. (Angela)

You see in my case I can’t tell whether it’s my spondylitis or my angina, or these (points to chest) which are blocked. (Martin)

I have a little shortness of breath. I don’t know whether that’s the cold weather as well though. (Alistair)

Now ‘angina pain radiates to other parts of your body’, I didn’t even realise it did that, I mean all these pains I’m getting from down my shoulders and to my arms could be angina, I don’t know. (Joan)

Probably ‘a little’, I’ve no angina pain, where it says palpitations, I’ve put ‘a lot’ because I’ve atrial fibrillation is it called, so I have an irregular heartbeat all the time, there’s nowhere on here for that like but, I don’t have, I don’t know what palpitations are … I don’t know what, I’ve never had palpitations. (Alistair)

I’m wondering if there’s much difference between ‘quite a bit’ and ‘moderately’, maybe they’re almost the same. (Bob)

Box 10.

Question 2

Anyhow, I was talking to one of the chaps, doctors when I went into the hospital when I started to go on like this, the breathing, and he said ‘do you use it 3 or 4 times a day?’ I said ‘no I don’t use it at all’, he said ‘why?’ I said ‘well I don’t get pains in my chest’. He said, ‘I know but it can help your breathing as well’. (Joan)

I’m not getting it, I’m getting my breathlessness but that doesn’t cover it does it, I don’t get chest pain, I’m not getting, I don’t get the tightness so that question actually is wrong, for me. (Gordon)

If it’s helped you breathe, you know, all well and good, in my case it’s never helped me breathe. (Martin)

Box 11.

Question 3

Mine’s all my breathlessness. (Joan)

It’s the only problem, I don’t get chest pain or angina, I mean I’m not even getting the palpitations which I thought were angina attacks, I’m not getting them since I’ve gone on them tablets so it doesn’t happen the same, mine’s my breathing. (Gordon)

I think it is straightforward but I think it might be worth putting a box for indigestion or something because somebody might not know whether that’s angina or not. (Bernard)

Box 12.

Question 5

The only one that’s limited a little bit is walking half a mile, I’ve got to get a second wind sort of thing but that might be being overweight as well, that probably won’t help. The only thing is probably, like climbing several flights of stairs, that’s put without rest afterwards, and the half mile was without rest. You know some people can climb several flights of stairs and rest on every landing might they and still climb several flights of stairs, I don’t know. (Alistair)

Yeah, limited a little bit on the moderate activities, lifting or carrying groceries, how I’ve got around that now I’ve got a blue badge, we go to the … and what we do put, we’ve got more bags to put stuff in so you’re not lifting and carrying the same amount of weight, you know when you put them in the car. There’s only say 5 or 6 tins in one bag, that’s the way we’ve been doing it for the last year. (Martin)

Yeah but it’s alright you see, J is with me when I’m shopping, she’s always with me so we put the stuff in the carrier, in the trolley. (Martin)

Yeah I could walk without resting yeah and the same with the 100 yards, I could walk 100 yards providing there’s no incline, on the straight no problem … If there’s an incline I would say I’d have problems. (Bert)

The first one, but it doesn’t apply to me because I don’t do any hoovering and I don’t do any vacuum cleaning! (laughter) … And I don’t play golf so that doesn’t apply to me. The groceries now and again, a little bit, I would say. (Bert)

I’ve restricted myself, I don’t go swimming now, I don’t do gardening now, I don’t come in and say oh we’ll change the ceiling, I’ll paint the ceiling or something like that or decorate the walls. (Martin)

Box 13.

Question 7

I’m a bit concerned about that there’s this ‘depressed’ because there’s quite a few others that have ‘worried’ in them, and they’re all a bit similar, but I’m actually wondering what are you really trying to find out when you say ‘depressed’ because they won’t know whether it’s clinical depression or just fed up or afraid, or are you actually trying to find out whether the anaesthetic has had a long-term effect or something. (Bernard)

That ‘depressed’ I put all the time because It’s the only way I feel, even with good news it’s very hard to make me feel any better, and most of that is down to the fact that I’m not working, you know, it’s a simple as that. I’ve always worked, and I … I would answer it differently if I was working, you know, there’s no doubt about that. (Adrian)

A: All the time depressed. Yeah. (Ken)

Q: but you’ve written in ‘getting better’?

A: No, I feel like I’m getting better, you know what I mean, I’m getting treatment because I was in a month in hospital getting psychiatric, and wanted to end it all. (Ken)

In addition, participants considered that responding accurately to question 1 may be problematic in describing their health status in a situation where the patient might have multiple conditions, for example, participants were sometime confused about their symptoms and which conditions they related to (Box 9).

Participants also raised concerns that question 2 made no attempt to establish whether or not they tolerate the medication before asking them to comment on how many times they take it daily (Box 10). Recording their responses as ‘None over the past four weeks’, both before and after the cardiac intervention gives a fairly inaccurate view of their condition.

Participants also commented that in question 3 it would be helpful to include a choice between angina and indigestion as both may sometimes result in similar symptoms (Box 11).

Question 5 did not take into account the adaptations people had already made to their ‘usual activities’ in order to improve their quality of life such as taking a break during physical activities or carrying less groceries (Box 12).

Overall, participants found it hard to describe their feelings about their heart condition during the past 4 weeks in question 7, in particular feelings of depression (Boxes 13 and 14).

Box 14.

Question 7 (continued)

Personally, I think I would sooner die suddenly than end up in a nursing home for 10 years (general agreement). (Alistair)

You see, I wouldn’t put that in, die suddenly, I’d put it in about having a heart attack or, you know what I’m terrified of, really terrified of, and that’s having a stroke … I couldn’t cope, I really couldn’t cope if I had a stroke. (Martin)

Apart from that no I don’t worry about that. Frustrating no, no, your heart condition interfering with the enjoyment of life, no I don’t worry about that, I just carry on as normal, take a different … position overlooking your health, I don’t even worry about that one, the bottom one, the very different, plans ahead e.g. vacation, no don’t worry about that, none of them really. I mean I don’t think I, there’s questions in there I wouldn’t be bothered even answering if that came to me. (Bert)

There’s some of the things that you need to include in this, like family, whether I’m married, single. (Martin)

Some limitations of the CROQ

Some patients did not understand clinical terms such as angina making it difficult for them to complete some of the questions. Others were not sure whether their physical symptoms related to their cardiac conditions, for example, breathlessness caused by being overweight, by a cold or by angina. Therefore, there is a marked element of variation in their responses. Our evidence suggests that patients view their health status as ‘mishmash’ or conglomeration of symptoms and not as a sum of distinct and distinguishable health conditions or related symptoms. Participants were also confused about the timescale of the question sometimes thinking it referred to 1 day instead of the previous 4 weeks. Participants were also concerned that the questionnaire did not take into account the adaptations they had already made to improve their HRQL such as removing a bath and installing a shower or moving into a bungalow. Participants were also concerned that the impact of tiredness and lack of vitality on their quality of life was also not captured in the CROQ.

Discussion

In this study, we sought to explore patients’ perception of EQ-5D questionnaire and CROQ. Additionally, we were interested in exploring whether participants gained substantial benefits from their cardiac procedures despite the lack of improvement in their EQ-5D scores. The findings of the study indicate that the EQ-5D may not explicitly detect the substantial health gains attained by each patient after a cardiac procedure. It could be argued that the response options are too limited in scope. In contrast, important symptoms such as lack of energy (vitality), common among patients with cardiac conditions, are not included in the EQ-5D questionnaire and CROQ but have a perceived serious impact on their quality of life.

Findings from this focus group have also highlighted the issue of recall time. The EQ-5D asks people to answer questions based on how they feel on the day they are completing the questionnaire, while the CROQ focuses on how they felt in the past 4 weeks. This was problematic for every one of the participants because how they feel physically varies from day to day and the variations could sometimes be dramatic. How people feel physically also varies from hour to hour during any one day, for example, experiencing severe pain or tiredness.

The UK Department of Health has commissioned a number of projects to capture the change in mean scores on a short health status measure from year to year using EQ-5D, as one of its key measures of quality in the National Health Service (NHS), ²² with scores adjusted for case mix or clinical complexity. There is concern that the PROMs project has overestimated the scale and scope of its applicability. ²³ To allay these concerns, the UK Department of Health issued an assurance that PROMs data will only be used to inform commissioning decisions rather than being used for performance management. ²⁴

There are also concerns that although EQ-5D is a widely used generic health status measure, it has never been used longitudinally before. ²⁵ The data from the Department of Health PROMs report 2009–2010 showed that the utility scores did not match up with the visual analogue scores for some conditions. ²⁶ This was reflected in the findings of this study that people found it difficult to report their perceived HRQL on the visual analogue scales.

From a theoretical perspective, it is known that prior knowledge and comprehension influence peoples’ expectations and help them interpret the current input of information. ²⁷ This influence of prior knowledge is known as top-down or conceptual-driven processing, which helps people interpret the bottom-up flow of information from the world. ²⁷ It is important that clinicians and service managers realise that they should not make assumptions about the level at which patients understand their conditions or the clinical terminology. For example, some people did not relate their breathlessness to symptoms of angina and therefore believed that they did not have angina.

In this study, participants pointed out that the two questionnaires studied do not explore the changes that they have made in their home lives in order to cope with living with a chronic condition (response shift), the issue of constant tiredness as a significant health burden, the importance of the family situations and their impact on HRQL and patients’ concerns about significant events such as having a stroke and being severely disabled or ending their days in a nursing home. In conclusion, our study provides some evidence that the EQ-5D questionnaire and CROQ may not necessarily be appropriate for a national PROMs programme. The validation of questionnaires is context specific, and no PROMs can thus claim to be fully validated for all health conditions and there is room for improvement. ²⁸ We suggest that in addition to rigorous psychometric testing, future questionnaires should capture as much as possible patient experiences at the beginning of the development process and during further consultation. ²⁹

Study limitations

We acknowledge that the study is limited by the small sample size due to problems with recruitment. We chose focus groups as the more pragmatic and cost-effective way of gathering people’s opinions in an exploratory study.

We acknowledge that the information gathered may be less detailed as compared to that of one-to-one interviews; however, the dynamics of focus group discussions encourages greater interaction and prompts members of the group to talk about issues they may otherwise have not considered. There is, however, the risk that some people may feel inhibited in an open discussion. To minimise this concern, each individual was given a chance in turn to give their opinion on each of the questions.

We recommend that future study should consider face-to-face interviews using cognitive interview methods. These could be conducted in places convenient to the participant or by telephone and therefore allow a larger study sample to be recruited.