Sage Journals: Discover world-class research

Abstract

Background. Patient-centered measures of the inpatient neurorehabilitation experience are needed to assess services. Objective. The objective of this study was to develop a valid and reliable Neurorehabilitation Experience Questionnaire (NREQ) to assess whether neurorehabilitation inpatients experience service elements important to them. Methods. Based on the themes established in prior qualitative research, adopting questions from established inventories and using a literature review, a draft version of the NREQ was generated. Focus groups and interviews were conducted with 9 patients and 26 staff from neurological rehabilitation units to establish face validity. Then, 70 patients were recruited to complete the NREQ to ascertain reliability (internal and test-retest) and concurrent validity. Results. On the basis of the face validity testing, several modifications were made to the draft version of the NREQ. Subsequently, internal reliability (time 1 α = .76, time 2 α = .80), test retest reliability (r = 0.70), and concurrent validity (r = 0.32 and r = 0.56) were established for the revised version. Whereas responses were associated with positive mood (r = 0.30), they appeared not to be influenced by negative mood, age, education, length of stay, sex, functional independence, or whether a participant had been a patient on a unit previously. Conclusions. Preliminary validation of the NREQ suggests promise for use with its target population.

Keywords

rehabilitation reliability validity patient experience patient satisfaction

Introduction

Health care organizations, researchers, and professionals have a commitment to the provision of effective and efficient health services.¹ This requires feedback from patients who use services,² but tools that can be used during inpatient neurological rehabilitation require special consideration, given the array of physical, cognitive, and social problems encountered and the range of underlying neurological conditions.^3,4

The concept of patient satisfaction has been criticized as ill defined.⁵ This has led to the development of statistically invalid and unreliable measures⁶ that are often oriented to staff and service interests over those of the patient^3,7 and elicit outcomes that do not indicate the processes or procedures requiring improvement.⁸ Based on the evidence that patients often are not formally evaluating their care,⁹ a more specific alternative to patient satisfaction—patient experience—has been developed. This seeks to obtain discrete information on what important elements occurred for patients, rather than their evaluation of the process.¹⁰ Such data are more useful to inform service improvement because they quantify and pinpoint the problems more precisely. Although some research has addressed patient satisfaction in neurological rehabilitation services,^11,12 to date, none has done this with appreciation that patient experience,¹⁰ rather than satisfaction, should be the focus.

The objective of this study was to develop a valid and reliable Neurorehabilitation Experience Questionnaire (NREQ) focusing on patient experience rather than satisfaction, which would have a clearly defined, patient-focused framework.

Methods

Questionnaire Content and Design

The content of the NREQ originated from themes identified by 3 sources: a qualitative study of patient experience in a neurological rehabilitation setting,¹³ patient feedback questionnaires used in local neurological rehabilitation services, and a literature search. Priority was given to the themes from the previous study¹³ because these were derived directly from interviews with ex-patients and were therefore of most relevance. Individual questions were generated based on these themes and those highlighted from the literature search. Where possible, patients’ own language was used. Questions were also adopted from previous questionnaires with permission. Sentences were kept short and simple in order to maximize the ease of completion by patients with cognitive impairment. In line with the focus on experience over satisfaction, each question focused on whether certain processes or events occurred, rather than asking the patient to make an evaluation.¹⁴

Traditionally, questionnaire design has included a balance of positive and negative statements to combat response acquiescence. However, negative statements produce inconsistent results in a neurological rehabilitation population.¹² On this account, it was decided that a list of positive statements would be used, on which patients could comment. Because the use of rating scales can also be problematic in neurological rehabilitation,¹⁵ discrete response options were provided. It was also considered important to keep the options as simple as possible on account of the nature of the client group while allowing for the important possibility of nonevaluation.⁹ Thus, the categories in the initial draft questionnaire were mostly agree, mostly disagree, not sure, and not applicable, the latter 2 options reflecting nonevaluation. An open-ended question was also included to allow respondents to identify any additional aspects of care requiring change.¹⁶

Self-reported satisfaction and experience have been shown to be influenced by the person who completes the questionnaire (patient or family member),¹¹ age,^17,18 sex,^11,19 and education.²⁰ Therefore, additional questions to elicit background and demographic information were included at the end of the questionnaire to aid interpretation of findings. The questionnaire was designed to be completed anonymously, so that those surveyed would not alter their responses on account of concerns about being associated with negative feedback or that it might affect their future rehabilitation or other treatment.

Face Validity Testing

Participants

Participants were recruited from 3 neurological rehabilitation units. One unit provides neurological rehabilitation for individuals who are mostly 18 to 65 years old and concentrates on persons with physical disability with or without cognitive deficits—for example, multiple sclerosis and stroke. The second is a stroke unit providing rehabilitation for predominantly older stroke patients. The third unit provides neurological rehabilitation for individuals who are mostly 18 to 65 years old, concentrating on persons with predominantly cognitive disability—for example, acquired head injury. All patients on the units at the time of the study who met the eligibility criteria were invited to participate.

The inclusion criteria required participants to be medically stable and capable of understanding the questionnaire and taking part in a discussion group. Patients were excluded if they had dysphasia, cognitive impairment (indicated by a score of less than 8 on the Abbreviated Mental Test,²¹ less than 24 on the Mini-Mental State Examination,²² or as stated by the multidisciplinary team), or a comorbid psychiatric disorder (as defined by the DSM-IV²³ or an ICD-10 classification,²⁴ other than an anxiety or mood disorder).

All members of staff on the units were eligible to take part, with the exception of members of the research team. Informed consent was obtained before participation commenced.

Procedure

Face validity testing took place with patient and staff focus groups and interviews with specialist medical doctors. A preestablished guide was used to facilitate discussions. This included directions to consider whether the questionnaire covered everything of importance and could be easily understood and also the appropriateness of the different response categories and their number. Each session was recorded, checked for accuracy, and transcribed. The comments were grouped into themes based on similarity. The content, layout, and wording of the NREQ were revised on the basis of these themes. Where patient and staff opinion differed, priority was given to patient opinion. Consistent with best practice,²⁵ an experienced qualitative researcher, who also acted as assistant facilitator for the focus groups, validated the changes that were made to the questionnaire.

Reliability and Concurrent Validity

Participants

Current inpatients were recruited from the same locations as in the face validity stage. The same inclusion and exclusion criteria were applied.

Procedure

Following informed consent, participants completed a set of questionnaires 1 week before their planned discharge. Patient experience was assessed by comparing the revised (face validated) version of the NREQ with the Faces Scale²⁶ and a set of brief questions about patients’ overall experience.²⁷ Because self-reported satisfaction and experience have been shown to be influenced by mood,^28,29 patients also completed the Positive and Negative Affect Scale (PANAS).³⁰ Then, 2 weeks postdischarge, participants were posted the revised NREQ to complete for a second time. Questionnaires were returned in a prepaid envelope.

Because self-reported satisfaction and experience have also been shown to be influenced by the duration of admission,³¹ relative state of functional independence,³² current physical health profile,³³ and the length of stay, admission and discharge scores on the Functional Independence Measure (FIM)³⁴ were also collected.

Measures

Draft Neurorehabilitation Experience Questionnaire

A prior qualitative study¹³ identified a superordinate theme—person centeredness—that included 4 key themes that expanded the concept: ownership, personal value, holistic approach, and therapeutic atmosphere. Taking account of these themes, considering patient feedback from local neurological rehabilitation services, and informed by a literature search, a 16-item self-report measure was generated. The pilot questionnaire asked participants to mark 1 of 3 faces to indicate if they “mostly agree” were “not sure,” or “mostly disagree” with respect to 16 statements such as “I felt able to talk to staff about any problems I had.” A “not applicable” option was also included. For 1 question, participants could elect to rate a number of different therapies they may have received. It was suggested that to score this question, responses be summed and divided by the number of items endorsed.

Demographic information

A demographic information sheet was located at the end of the experimental questionnaire. The information collected was about age, sex, ethnicity, marital status, living circumstances, current/previous occupation, level of education, and reason for the neuro-rehabilitation.

The Delighted-Terrible FACES Scale²⁶

The FACES scale required participants to rate 1 statement regarding their satisfaction with care on a 7-point scale. Participants were asked to place a mark on 1 of 7 picture faces: very satisfied, satisfied, slightly satisfied, neither, slightly dissatisfied, dissatisfied, and very dissatisfied. Responses are scored on a 7-point scale, ranging from 1 = very dissatisfied up to 7 = very satisfied.

A summary section of the National Health Service of Great Britain and Northern Ireland (NHS) Adult Inpatient Survey²⁷

This consisted of 3 questions taken from the summary section of the NHS adult inpatient survey.²⁸ The first question asks, “Overall, did you feel you were treated with respect and dignity while you were in the unit?” Responses available were scored as follows: “yes always” scored 100%, “yes sometimes” scored 50%, and “no” was scored as 0%. The 2 further questions are, “How would you rate how well the staff worked together?” and “Overall, how would you rate the care you received?” Responses to these 2 questions were rated on 5-point scale (range “excellent,” scored 100%, to “poor,” scored 0%). The average percentage score for all questions was taken as the overall score.

Positive and Negative Affect Scale³⁰

The PANAS is a 20-item self-report measure with 10 positive and 10 negative affective descriptors. Participants are asked to rate their feeling for each affective descriptor at the present time. Responses are scored on a 5-point scale, ranging from 1 = very slightly or not at all, up to 5 = very much or extremely. The PANAS has an affect scale range from 10 to 50, where higher scores indicate higher affect. The PANAS was included to consider the effect of mood on patient experience.

Analyses

A review of the research literature and a power analysis (using the computer package G Power³⁵) was used to establish the sample size required for statistical power. The calculation was conducted for a 1-tailed, a priori analysis of a correlation. The power analysis specified a sample size of 21 participants where a large effect size is expected and a sample size of 64 for a medium effect size. The research literature recommends a sample size of 50.³⁶ For this reason, a sample size of 50 was decided on to fit with the power analysis calculation and the literature.

Internal consistency of the patient experience questionnaire was calculated using Cronbach α. Correlating participants’ scores on the questionnaire at initial testing and then 2 weeks later assessed test retest reliability. Comparing outcomes on the NREQ with the 2 other patient experience questionnaires, also via correlation, measured concurrent validity. The relationship between the demographic and mood data on patients’ reported experience was also calculated. All correlations apart from those to establish internal consistency were calculated using the Pearson product–moment correlation formula.

Missing data

In some cases, questionnaires were returned but incomplete. A maximum limit of 2 missing questions was set for NREQ responses to be considered usable. No questions could be missed for either of the comparison questionnaires. For the purposes of analysis, in cases where the NREQ was returned incomplete but usable, the missing data were substituted logically based on responses to surrounding questions. This was possible in most cases because participants were largely consistent in their responses. Where there was variability in surrounding answers, the overall mean score was substituted.

Ethical Approval

The face validity testing phase of the study was approved by the West Surrey Local Research Ethics Committee. The reliability and concurrent validity testing was approved by the West Kent Local Research Ethics Committee.

Results

Face Validity

Participants

In total, 9 white British patients (4 male) participated in the study. They ranged in age from 41 to 81 years (mean, 57 years). The reasons for neurorehabilitation included: stroke (4), multiple sclerosis (2), Miller Fischer Syndrome (1), acquired brain injury (1), and frontal lobotomy (1). The duration on the unit at the time of the research ranged from 3 to 20 weeks (mean, 8.5 weeks). For 8 patients, it was their first admission to the units.

A total of 27 staff participated: specialist/acting-specialist medical doctors (2), associate specialist medical doctor (1), ward managers/nurses (2), staff nurse (1), enrolled nurse (1), nursing auxiliaries (2), occupational therapists (5), physiotherapists (5), clinical psychologist (1), speech and language therapist (1), and therapy/rehabilitation assistants (6).

Neurorehabilitation Experience Questionnaire

Modifications to the NREQ based on the focus groups and interviews included the following: changes to make questions more relevant or applicable to the environment and the process of rehabilitation, clarifications and simplifications, inserting questions to address areas that had been omitted, and the removal of questions deemed less applicable. Some suggestions were made for additional questions. If space had permitted, then questions would have been included to address the following: availability of staff, information provision (ie, medical condition and prognosis, the routine on the ward, and personal daily rehabilitation timetable), catering and whether dietary requirements were met, night staff’s role, and the degree of independence/responsibility fostered. One patient focus group suggested that some of the NREQ items might not be as relevant for patients who regularly used the service to manage chronic conditions. On this account, a question was inserted into the demographics section to ask if the patient had previously stayed on the unit. The “not applicable” category was seen as irrelevant and potentially confusing, so this was removed from the questionnaire. Feedback on the length of the questionnaire was positive, such that overall, with the incorporated changes, the total questionnaire length increased by only 1 question. As noted previously, it was proposed to obtain the therapies question total score by summing and dividing by the number of items endorsed. Given the emphasis participants put on therapies in their inpatient rehabilitation, this score was also doubled. As can be seen in Table 1, the questionnaire to be subject to the reliability and validity trial then consisted of 17 items. The responses available were mostly agree = 2, not sure = 1, and mostly disagree = 0. A patient experience score, range 0 to 36 (when the double score for the therapies question is included), is able to be generated from the questionnaire. Higher scores indicate a more positive rehabilitation experience.

Table 1.

Questions Included in Scoring the Neurorehabilitation Experience Questionnaire

Question Number	Question
1	When I arrived I was given information about the unit and what would happen during my stay
2	The facilities on the unit were good (eg, washing facilities, toilets)
3	There was somewhere secure to keep my belongings
4	I was able to discuss personal matters in private
5	There were enough things to do in my free time (eg, watch television, join in with a group, sit in the garden)
6	There was a friendly atmosphere on the unit
7	I felt the staff really cared about me
8	The staff worked well as a team
9	I felt able to talk to the staff about any problems I had
10	I was asked what I wanted to achieve during my stay
11	I felt as though the staff and I were partners in the whole process of care
12	The staff kept me informed every step of the way
13	My family or carer was involved in discussions about my treatment if I wanted them to be
14	I am happy with the amount of therapy or treatment I received for: (a) swallowing problems, (b) speech and communication, (c) improving mobility (physiotherapy), (d) independent living (eg, washing, dressing, cooking), (e) continence (eg, bladder & bowels), (f) other (please state)
15	I received enough emotional support (eg, if I was feeling low or finding it hard to cope)
16	I am feeling well supported and prepared for my discharge
17	I am satisfied with the progress I have made during my stay

Internal Reliability, Test-Retest Reliability, and Concurrent Validity

Participants

In total, 70 patients were recruited to complete this element of the study. On account of missing data, the actual sample size varied between analyses. However, only 2 NREQ questionnaires had to be discarded at time 1 because of missing data and only 1 at time 2. Demographic characteristics for those who completed questionnaires at both time 1 and time 2 are summarized in Table 2. The reasons for neurorehabilitation included stroke (33), multiple sclerosis (11), head injury (4), and other (10). For 44 patients, it was their first admission to the unit. Table 3 describes the distribution of actual scores achieved on the NREQ at time 1, and Table 4 provides a summary of scores for the NREQ and all the other measures. As can be seen in Table 4, the NREQ scores indicated that a large number of patients experienced what was important to them.

Table 2.

Demographic Characteristics for the 59 Participants Who Completed Questionnaires at Time 1 and Time 2

	Mean/Number	Standard Deviation/Percentage
Age, y; range 20-87 years	57.39	14.7%
Length of stay on the unit
Less than 1 month	n = 14	23.7%
1 to 3 months	n = 38	64.4%
4 to 6 months	n = 4	6.8%
More than 6 months	n = 2	3.4%
		1.7%^a
Sex
Female	n = 30	50.8%
Male	n = 29	49.2%
Marital status
Single	n = 11	18.6%
Married	n = 37	62.7%
Separated	n = 1	1.7%
Divorced	n = 4	6.8%
Widowed	n = 5	8.5%
		1.7%
Ethnicity
Asian/Asian British	n = 2	3.4%
White	n = 56	94.9%
		1.7%
Education
No qualifications	n = 11	18.6%
Secondary	n = 9	15.2%
Further	n = 12	20.3%
First degree	n = 11	18.6%
Higher degree	n = 4	6.7%
Other (not reported)	n = 10	16.9%
		3.4%^a

Missing data.

Table 3.

Distribution of Individual Items on the Neurorehabilitation Experience Questionnaire for Time 1^a

	Frequency, Percentage (n = 59)
Question Number	Mostly Disagree	Not Sure	Mostly Agree	Mean	Standard Deviation	Missing Data	Percentage Not Applicable
1.	8.5	13.6	78.0	1.69	0.62	0
2.	11.9	1.7	86.4	1.75	0.65	0
3.	11.9	16.9	71.2	1.59	0.69	0
4.	3.4	8.5	88.1	1.85	0.44	1
5.	13.6	8.5	78.0	1.64	0.71	0
6.	0	1.7	98.3	1.98	0.13	0
7.	3.4	1.7	94.9	1.92	0.38	0
8.	1.7	5.1	93.2	1.92	0.37	0
9.	5.1	1.7	93.2	1.88	0.45	0
10.	8.5	11.9	79.7	1.71	0.61	0
11.	3.4	8.5	1.7	1.84	0.44	0
12.	6.8	18.6	74.6	1.68	0.60	0
13.	3.4	5.1	91.5	1.88	0.41	2
14a.	1.7	5.1	28.8	1.76	0.53		62.7
14b.	5.1	3.4	45.8	1.75	0.62		45.8
14c.	1.7	0	91.5	1.96	0.27		6.8
14d.	3.4	3.4	81.4	1.88	0.42		11.9
14e.	0	10.2	57.6	1.85	0.36		32.2
14f.	1.7	3.4	13.6	1.64	0.67		81.4
15.	5.1	11.9	81.4	1.77	0.52	6
16.	1.7	5.1	93.2	1.92	0.33	0
17.	3.4	5.1	91.5	1.88	0.41	2

Missing data are shown before logical imputation, and frequencies are shown as calculated following imputation. For Q14 a to e, a missing response was assumed to mean not applicable. For all other questions, this was treated as missing data.

Table 4.

Means, SDs, and Interquartiles for the NREQ and All Other Measures at Time 1

			Interquartiles
	Mean	SD	25	50	75
NREQ	33.5	2.6	32.2	34.4	36
The Delighted-Terrible FACES Scale	1.4	0.6	1	1	2
National Health Service of Great Britain and Northern Ireland (NHS) Adult In-patient Survey²⁷	87.5	11.9	83	92	100
PANAS: positive affect	34.8	9.2	29	35	42.2
PANAS: negative affect	16.7	7.5	11	15	19

Abbreviations: SD, standard deviation; NREQ, Neurorehabilitation Experience Questionnaire; PANAS, Positive and Negative Affect Scale.

Internal reliability

An α value of .7 or greater is taken to indicate good internal reliability.³⁷ At both initial testing (time 1; n = 59) and postdischarge (time 2; n = 53), α exceeded this criterion, indicating good internal reliability: time 1 α = .76; time 2 α = .80.

Test-retest reliability (n = 49)

There is no agreed guideline as to what is considered acceptable in terms of test-retest reliability coefficients, with the literature presenting significant coefficients ranging from 0.4 to above 0.8.^38-40 The current study classified 0.7 or above as acceptable in consideration of previous recommendations.^41,42 The results of the current study indicate good test-retest reliability, with a highly significant correlation between the average experience score at time 1 compared with that at time 2, with the correlation coefficient exceeding 0.7: r = 0.78; P = .001. Nonparametric statistics failed to identify any significant differences between individual items from time 1 to time 2.

Concurrent validity (n = 57)

The results indicate that the experimental questionnaire has good concurrent validity as compared with both comparison measures, with highly significant relationships demonstrated via Pearson correlations: NREQ versus FACES scale, r = 0.32, P = .004; NREQ versus NHS adult inpatient survey, r = 0.56, P = .001.

Subsidiary Analyses

Influence of mood

The positive mood subscale of the PANAS was significantly correlated with the average experience score at time 1, r = 0.30, P = .03, but not the negative mood subscale, r = −0.15, P = .29, indicating that those with more positive mood reported a better experience, and negative mood was unrelated to experience.

Relationship between demographic information, length of stay, functional independence, being assisted to complete the questionnaire, and experience

As is seen in Table 5, respondents’ views of their experience were not related to their age, education level, or how long they were on the unit at time 1 or time 2.

Table 5.

Relationship Between Age, Education, and Length of Stay and NREQ

		Age	Education	Length of Stay
NREQ, time 1	r	−0.29	−0.075	0.04
	P	.83	.62	.75
NREQ, time 2	r	0.16	−0.171	0.02
	P	.27	.33	.86

Abbreviations: NREQ, Neurorehabilitation Experience Questionnaire.

There was no significant difference in mean NREQ scores between men and women at time 1 (men, 32.53, standard deviation [SD] = 3.87, vs women, 32.55, SD = 4.36; t = 0.015; NS) or at time 2 (men, 32.42, SD = 4.95, vs women, 32.80, SD = 3.77; t = 0.301; NS). There was also no significant difference in NREQ scores at time 1 in regard to whether it was the patient’s first time on the unit (first time ‘yes’, 32.07, SD = 4.45, vs first time ‘no’, 34.01, SD = 2.35; t = 0.015; NS). Neither FIM scores prerehabilitation (r = .043, NS) or postrehabilitation (r = .099, ns) nor FIM change scores (r = .049, NS) were associated with NREQ scores at time 1. Thus, neither sex, previous experience of rehabilitation, functional independence, or change in functional independence appear to be associated with NREQ ratings. Too few patients (5 at time 1 and 13 at time 2) were assisted in their completion of the questionnaire to consider the effect of this on scores.

Discussion

A questionnaire to assess patient experience with neurological rehabilitation was developed based on the qualitative expansion of themes established from interviews with neurological rehabilitation patients, the adoption of questions from general patient satisfaction questionnaires, and a literature review. This questionnaire was refined following face validation via focus groups and interviews with patients and staff. An experienced qualitative researcher validated the changes. The questionnaire’s reliability (internal and test-retest) and its concurrent validity were then examined: good reliability and validity were established. The questionnaire has potential for identifying, via individual items, specific areas for improvement in a service or, using the full score, monitoring overall patient experience.

Limitations

It is acknowledged that more staff than patients participated in the face validity phase of the project. This may have subtly biased the face-validated NREQ. However, this is unlikely because at all stages, patients’ opinions were given priority over those of staff when modifying the NREQ.

An additional potential bias is identified when the effect of the inclusion/exclusion criteria on patient participation is considered. Approximately one third of the patients on each of the units were not eligible to participate, usually because of cognitive or communication difficulties. Some compensation for this exclusion, and the overall small sample of patients in the focus groups, may have been provided by the inclusion of staff in the face validity stage. Staff familiarity with the whole patient population meant that they may have been able to give an indication of how the questionnaire would be received by those not directly represented by patient participants. Given the nature of the patient group, however, the fact remains that some patients will be unable to complete the questionnaire even with assistance, and this is a limitation of the NREQ. Other strategies to consider patient experience will need to be developed for these individuals.

The study found participants indicating a high level of experience of what is important to them. This is not uncommon in such evaluations⁴³ but does raise concern as to the ability of the questionnaire to detect if patients are not experiencing what is important to them.

One of the challenges of developing the questionnaire was to cover all the relevant themes in sufficient detail and yet keep the questionnaire short enough so as not to place too much burden on respondents. Inevitably, there have been some themes, or aspects of themes, that were not addressed in the questionnaire (eg, dietary requirements and night staff). However, this study has established preliminary reliability and validity for the NREQ. It shows promise, subject to further validation, for use in clinical settings.

Future Work

Given the sample size and focused locality of this project, a larger trial involving different neurological rehabilitation units would help establish the NREQ’s broad application to this service setting. A larger sample would also provide the power to conduct factor analysis statistics that could provide further validation of the questionnaire by matching it against the thematic structure identified in prior qualitative research.¹³ Its utility with different ethnic groups should also be considered. Future work in units where concerns have been raised might better establish the NREQ’s ability to detect when experiences important to patients are not occurring and, via repeated administration, its ability to detect change.

Conclusions

Patient-centered care has been recommended as the gold standard for health care services⁴⁴ and is specifically important within neurological rehabilitation services. A successful rehabilitation environment involves fostering independence, choice, and control. One way to contribute to achieving this is the use of a standardized, reliable, and valid patient experience questionnaire to assess patients’ views on these issues.

The NREQ has been successfully developed. Future work with a larger sample might enable its factor structure to be identified, providing validation of the mapping onto the themes established in prior qualitative work.¹³ Its use with discrete patient groups (eg, stroke, acquired brain injury, and multiple sclerosis) may lead to further refinement.

Footnotes

Acknowledgements

We gratefully acknowledge the assistance of the following people: Jenny Billings, Sophie Dewar, Damian Dewhurst, Sue Guerrier, Celia Leck, Jan Nielsen, Penny Sharp, Beth Taverner, Helen Wain, Jan Wheatcroft, and the staff and patients of the Godwin, Runfold, and Bradley Units who took part in the study.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The projects reported here were funded through the UK National Health Service, Research and Development Fund, Budget One.

References

Department of Health. The NHS Plan: A Plan for Investment, a Plan for Reform. London, UK: Stationary Office; 2000.

Commission for Health Improvement. Sharing the learning on patient and public involvement from CHI’s work: i2i involvement to improvement. http://webarchive.nationalarchives.gov.uk/20080814090418/http://www.healthcarecommission.org.uk/_db/_documents/04010188.pdf. Accessed January 18, 2012.

Keith

. Patient satisfaction and rehabilitation services. Arch Phys Med Rehabil. 1998;79:1122-1128.

Greenwood

Barnes

McMillan

Ward

. Handbook of Neurological Rehabilitation. Hove, UK: Psychology Press; 2003.

Fitzpatrick

Hopkins

. Problems in the conceptual framework of patient satisfaction research: an empirical exploration. Sociol Health Illn. 1983;5:297-311.

Petterson

Veenstra

Guldvog

Kolstad

. The patient experiences questionnaire: development, validity and reliability. Int J Qual Health Care. 2004;16:453-463.

Sitzia

Wood

. Patient satisfaction: a review of issues and concepts. Soc Sci Med. 1997;45:1829-1843.

Jenkinson

Coulter

Bruster

Richards

Chandola

. Patients’ experiences and satisfaction with health care: results of a questionnaire study of specific aspects of care. Qual Saf Health Care. 2002;11:335-339.

Williams

. Patient satisfaction: a valid concept? Soc Sci Med. 1994; 38:509-516.

10.

Picker Institute Europe. Trends in Patients’ Experience of the NHS. http://www.pickereurope.org/Filestore/Publications/Trends_with_ISBN_web_(3).pdf Accessed April 22, 2005.

11.

Heinemann

Bode

Cichowski

Kan

. Measuring patient satisfaction with medical rehabilitation. J Rehabil Outcomes Measure. 1997;1:52-65.

12.

Pound

Gompertz

Ebrahim

. Patients’ satisfaction with stroke services. Clin Rehabil. 1994;8:7-17.

13.

Wain

Kneebone

Billings

. Patient experience of neurologic rehabilitation: a qualitative investigation. Arch Phys Med Rehabil. 2008;89:1366-1371.

14.

Picker Institute Europe. Measuring patients’ experience of health care: the Picker approach. http://www.pickereurope.org/about/approach.htm. Accessed December 26, 2011.

15.

Price

CIM

Curless

Rodgers

. Can stroke patients use visual analogue scales? Stroke. 1999;30:1357-1361.

16.

Thomas

McColl

Priest

Bond

. Open-ended questions: do they add anything to a quantitative patient satisfaction scale? Soc Sci Health. 1996;2:23-35.

17.

Arnetz

. The development and application of a patient satisfaction measurement system for hospital-wide quality improvement. Int J Qual Health Care. 1996;8:555-566.

18.

Jackson

Furnham

. Designing and Analysing Questionnaires and Surveys: A Manual for Health Professionals and Administrators. London, UK: Whurr; 2000.

19.

Coyle

Williams

. An exploration of the epistemological intricacies of using qualitative data to develop a quantitative measure of user views of health care. J Adv Nurs. 2000;31: 1235-1243.

20.

Winter

Keith

. A model of outpatient satisfaction in rehabilitation. Rehabil Psychol. 1988;33:131-142.

21.

Hodkinson

. Evaluation of a mental test score for assessment of mental impairment in the elderly. Age Ageing. 1972; 1:233-238.

22.

Folstein

McHugh

. Mini Mental State: a practical guide for grading the cognitive state of patients for the clinician. J Psychol Res. 1975;12:189-198.

23.

American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4th ed. Washington, DC: American Psychiatric Association; 1994.

24.

World Health Organisation. International Classification of Diseases: Version 10. Geneva, Switzerland: World Health Organization; 1999.

25.

Kitzinger

. Qualitative research: introducing focus groups. BMJ. 1995;311:299-302.

26.

Andrews

Withey

. Social Indicators of Well-being: Americans’ Perceptions of Life Quality. New York, NY: Plenum Press; 1976.

27.

Picker Institute Europe. The NHS Adult Inpatient Survey 2005: Core Questions. Oxford, UK: Picker Institute Europe; 2004.

28.

Pound

Gompertz

. A patient-centred study of the consequences of stroke. Clin Rehabil. 1998;12:338-347.

29.

Pound

Tilling

Rudd

Wolfe

. Does patient satisfaction reflect differences in care received after stroke? Stroke. 1999;30:49-55.

30.

Watson

Clarke

Tellegen

. Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol. 1998;54:1063-1070.

31.

Commission for Healthcare Audit and Inspection. Survey of patients 2005: stroke. London, UK: Commission for Healthcare Audit and Inspection; 2005.

32.

Vroman

Arnsberger

Stevens

Williams

. Rehabilitation clients’ recovery expectations and their relationship to functional outcomes. J Allied Health. 2005;34:51-55.

33.

Rahmqvist

. Patient satisfaction in relation to age, health status and other background factors: a model for comparisons of care units. Int J Qual Health Care. 2001;13:385-390.

34.

Turner-Stokes

Nyein

Turner-Stokes

Gatehouse

. The UK FIM+FAM: development and evaluation. Clin Rehabil. 1999;13:277-287.

35.

Faul

Erdfelder

. GPOWER: A Prior, Post-hoc, and Compromise Power Analysis for MS-DOS (Computer Program). Bonn, Germany: Bonn University, Department of Psychology; 1992.

36.

Jackson

Chamberlin

Kroenke

. Predictors of patient satisfaction. Soc Sci Med. 2001;52:609-620.

37.

Field

. Discovering Statistics Using SPSS. 2nd ed. London, UK: Sage; 2005.

38.

Hersen

. Comprehensive Handbook of Psychological Assessment. New York, NY: John Wiley; 2004.

39.

Campbell

Rohlman

Storzbach

. Test-retest reliability of psychological and neurobehavioral tests self-administered by computer. Assessment. 1999;6: 21-23.

40.

McDaniel

. Criterion-related diagnostic validity and test-retest reliability of the MMPI-168(L) in mentally retarded adolescents and adults. J Clin Psychol. 1997;52:485-489.

41.

Kline

. The Handbook of Psychological Testing. 2nd ed. London, UK: Routledge; 2000.

42.

Streiner

Norman

. Health Measurement Scales: A Practical Guide to Their Development and Use. 2nd ed. Oxford, UK: Oxford University Press; 1995.

43.

Wadell

Palmer

. Provider empathy for a quality network: merit behavioral care’s new focus on a key to patient satisfaction. Behav Health Manag. 1996;16:14-15.

44.

Department of Health. Standards for Better Health. London, UK: Stationary Office; 2004.

Reliability and Validity of the Neurorehabilitation Experience Questionnaire for Inpatients

Abstract

Keywords

Introduction

Methods

Questionnaire Content and Design

Face Validity Testing

Participants

Procedure

Reliability and Concurrent Validity

Participants

Procedure

Measures

Draft Neurorehabilitation Experience Questionnaire

Demographic information

The Delighted-Terrible FACES Scale 26

A summary section of the National Health Service of Great Britain and Northern Ireland (NHS) Adult Inpatient Survey 27

Positive and Negative Affect Scale 30

Analyses

Missing data

Ethical Approval

Results

Face Validity

Participants

Neurorehabilitation Experience Questionnaire

Internal Reliability, Test-Retest Reliability, and Concurrent Validity

Participants

Internal reliability

Test-retest reliability (n = 49)

Concurrent validity (n = 57)

Subsidiary Analyses

Influence of mood

Relationship between demographic information, length of stay, functional independence, being assisted to complete the questionnaire, and experience

Discussion

Limitations

Future Work

Conclusions

Footnotes

Acknowledgements

Declaration of Conflicting Interests

Funding

References

The Delighted-Terrible FACES Scale²⁶

A summary section of the National Health Service of Great Britain and Northern Ireland (NHS) Adult Inpatient Survey²⁷

Positive and Negative Affect Scale³⁰