Sage Journals: Discover world-class research

Abstract

Objective:

Determine the psychometric properties of PRECiS (Patient Reported Evaluation of Cognitive State): A new patient-centred, patient reported outcome measure for perceived impact of cognitive problems, developed through qualitative work, systematic review and service user consultation.

Design:

An observational study exploring acceptability, internal consistency, construct validity, inter-rater reliability and test-retest reliability, with opportunistic qualitative data on sensitivity to change.

Setting:

Home visits in the community.

Participants:

Stroke survivors with self-reported cognitive difficulties and informal carers.

Measures:

The 27 item PRECiS was self-completed with support, and proxy completed by informal carers. We collected descriptive cognitive screening test data, and measures of overall stroke impact, mood and activities of daily living to explore construct validity.

Results:

Data were collected from 159 (visit 1) and 66 (visit 2) stroke survivors and 86 informal carers. PRECiS showed good acceptability (no missing values or floor/ceiling effects, minimal skewness); high internal consistency (α = 0.94, indicative of potential redundancy); with moderate to strong construct correlations in the directions hypothesised (0.40 to 0.74). An intraclass correlation coefficient of 0.85 indicated good test-retest reliability. Where self-reported change had occurred from visit 1 to 2, PRECiS appeared sensitive. Using carers as proxy respondents is not supported by this analysis (inter-rater ICC = 0.43).

Conclusions:

PRECiS is a patient-centred, practical and reliable measure assessing perceived impact of cognitive problems from the unique perspective of stroke survivors.

Keywords

Psychometry outcome assessment (health care)stroke cognitive impairment patient-centred outcome measure

Introduction

Stroke survivors commonly experience a variety of cognitive difficulties that have an adverse impact on confidence, mood and long-term functional recovery.^1–3 Improving cognition is the number one priority for research into life after stroke according to health professionals, carers and stroke survivors themselves.⁴ Recent reviews conclude that - as well as significant gaps in the evidence base regarding how to rehabilitate these difficulties - interventions rarely assess outcomes that are of ‘real life’ importance to patients or take their perspective on outcome i.e. through patient reported outcome measures. ^5–6 Individuals with cognitive difficulties have been systematically excluded from the development and use of patient reported outcome measures due to their difficulties with comprehension and communication.^7–8 A commonly used patient reported outcome measure for cognition in stroke trials is the Cognitive Failures Questionnaire⁹ but this is geared towards memory difficulties and was developed without input from service users. Overall, there is a lack of patient-centred outcome measures available for stroke trials.¹⁰

To address this gap, we set out to develop a patient-centred, patient reported outcome measure that would be suitable for cognitive rehabilitation trials in stroke. Qualitative work,¹¹ systematic review of existing measures (paper in preparation) and pilot testing including patient and carer public involvement with service users, led to the development of the Patient-Reported Evaluation of Cognitive State (PRECiS).

PRECiS measures the perceived impact of persisting problems with cognition. Impact is conceptualised as ‘bother’ as opposed to frequency or amount of problem; a concept recommended by service users¹¹ and used elsewhere in stroke.¹² Its format and content are designed to facilitate use with cognitively-impaired stroke survivors. It includes large-print text that is bolded and coloured for emphasis, as well as lay-friendly definitions of concepts such as ‘cognition’. PRECiS includes 27 core items asking respondents about the impact of cognition on four conceptual dimensions: skills (12 items); family and life (six items); mood (six items); and sense of self (three items). For each core item, respondents first indicate whether they experience a problem and if so, they rate the bother associated with that problem on a Likert scale from 0 (not bothered) to 4 (extremely bothered). Five additional PRECiS questions ask respondents to rate agreement with statements that we hypothesised would influence rated bother. These statements include: whether respondents try to hide their problems with cognition; how supported they feel; how far they have accepted their problems with cognition; whether others tell them they have problems that they don’t see themselves; and if they feel that others understand the effects of their problems with cognition. An example of a PRECiS core item is shown in Figure 1. All 27 core items are listed in Table 3 and Figure 3. PRECiS documents (measure and administration/scoring guide) are protected by copyright and freely available for download and non-commercial use on the University of Manchester’s click2go licensing website at: http://www.click2go.umip.com/i/coa/precis.html

Figure 1.

Example item from PRECiS.

The study aimed to assess the psychometric properties of PRECiS including: acceptability to respondents; internal and external validity. Reliability was explored including test-retest and inter-rater (using informal carers as proxy respondents).

Methods

A cross-sectional, interview-based psychometric study was completed. Participants were adults living in the community across two areas in England. Eligibility also comprised: being at least six months post-stroke (no upper limit) with self-reported ongoing difficulty due to cognition; an ability to provide informed consent and communicate in English through any communicative medium. Aphasia-friendly versions of the information and consent materials were available and the use of communication aids was encouraged. People were excluded if they had a co-morbid diagnosis likely to cause ongoing cognitive decline (e.g. dementia) as that might affect test-retest reliability in this validation study. Adult English-speaking informal carers of recruited stroke survivors were also invited to participate in the study.

Participants were recruited through four routes between November 2013 and August 2014:

Primary care physicians sent letters of invitation to potentially eligible stroke register patients.

Healthcare professionals invited eligible stroke survivors.

The study was advertised using posters at participant identification centres.

Community stroke support groups were visited.

Appropriate ethics and governance approvals were obtained from the National Research Ethics Service, participating hospitals and primary care practices.

Participants were visited in their homes by one of two researchers: one researcher was based in the North, one in the South of England. Demographic details and an indicator of disability (a modified Barthel Index,)¹³ were collected for stroke survivors and available carers. Stroke survivors also completed cognitive screening tests: the Montreal Cognitive Assessment;¹⁴ the Apraxia Screen of TULIA;¹⁵ the Star Cancellation Test;¹⁶ and the Frenchay Aphasia Screening Test.¹⁷

Stroke survivors then completed PRECiS with as much support from the researcher as required. Support included introducing the questionnaire and its purpose; providing examples to support interpretation; using an alternative version of the questionnaire with one question per page. An administrator guide maximised consistency in PRECiS delivery. Administrators were asked to record concerns about respondents’ insight or comprehension.

Carers completed PRECiS as proxy respondents independently, although support was available if required. Participants gave their views on the coverage and acceptability of PRECiS following completion.

Comparison with other measures explored aspects of construct validity. To reduce burden, not all participants were asked to complete all comparison measures; we sought a minimum of N=50 respondents on each of the scales to be compared.^18–19 Respondents either completed measures of mood (the Patient Health Quesionnaire-9 for depression²⁰ and the Generalised Anxiety Disorder Assessment-7 for anxiety)²¹ plus the Nottingham Extended Activities of Daily Living; ²² or they completed a stroke-specific quality of life tool, the Stroke Impact Scale. ²³ If participants were unable to complete the full Stroke Impact Scale (typically due to fatigue or understanding) a short form composite version was available.²⁴

A sub-sample completed PRECiS a second time within two weeks. The sub-sample was purposively selected to include participants achieving a diverse range of scores at visit one to maximise variability. Qualitative data were collected to explore the assumption of stability across visits: by asking participants about experiences since visit one and whether they felt their mood and/or cognition were the same, better or worse.

If the visit(s) revealed significant distress and/or unmet needs, researchers provided information about local services available. Patients could be advised to make an appointment with their General Practitioner or referred to community services, such as those provided by the Stoke Association.

Data analysis

PRECiS scores were computed by adding the ratings from relevant items e.g. total score was all 27 items; ‘family and life’ conceptual dimension was six items. Each item was scored from 0 to 4, with a ‘No’ response (indicating no perceived difficulty) transformed to a 0 being equivalent to a ‘Yes+0’ response (indicating that a perceived difficulty did not bother the participant at all). Maximum possible score on the 27 item scale was 108, indicating highest impact. Standard psychometric methods²⁵ were used to evaluate PRECiS, as follows:

Acceptability and practicality

Missing data for each item should be <10%, with remaining missing data imputed as the mean of non-missing responses. Skewness values should be <±1 for at least 75% of items. For floor/ceiling effects, no items should have >80% endorsement at the top/bottom extreme. Time to complete and qualitative feedback were also analysed.

Internal consistency

The extent to which items measure the same construct was explored using Cronbach’s alpha (criteria of > 0.8) and item total correlations (criteria ⩾ 0.2) across scale as a whole and conceptual dimensions.

Construct validity – internal

To explore validity of the four conceptual dimensions of PRECiS (skills; life; mood; and self), item convergence and item discrimination was assessed using Pearson product moment correlations between each item and the dimension total score. Item correlation with its proposed dimension should be 2 standard errors (2/√n) greater than correlations between the item and non-proposed dimensions.²⁶ Factor analysis with varimax rotation was performed.

Construct validity – external

Correlations between PRECiS and comparator measures were calculated. Hypotheses were: PRECiS would correlate most strongly with mood measures (e.g. the emotion subscale of the Stroke Impact Scale), given the well-documented relationship between cognition and mood.^27–29 Consistent with the World Health Organisation’s framework for understanding the differential impact of disease,³⁰ we predicted that perceived impact of cognitive difficulties would not necessarily map to cognitive impairment given the many variables that moderate this relationship, including pre-morbid levels of cognition, and external support. As such, we did not specify a priori that a relationship would exist between the cognitive screen data and PRECiS data. To examine the value of the five additional PRECiS questions described above, a linear regression used them as predictor variables for PRECiS score. We also included age, sex and time post-stroke as possible predictors.

Carer as proxy respondent (inter-rater reliability)

The reliability of this approach was explored using intraclass correlation coefficients (criteria of >0.8). A linear regression explored whether “trying to hide problems with cognition” (PRECiS additional question) was a predictor for discrepancy between patient and carer scores.

Test-retest reliability and sensitivity to change

Intraclass correlation coefficients were computed (criteria of >0.8). Bland-Altman plots¹⁹ highlighted participants with scores varying by ⩾ 10% (> 10 points on the 108-point scale). Qualitative data collected at visit two categorised participants into groups self-reporting positive, negative or no change in cognition and/or mood. Change scores across groups were compared using Analysis of Variance (ANOVA).

Results

Of the 235 stroke survivors referred to the research team, 164 (70%) were eligible and agreed to participate. 159 (97%) of these provided usable data for psychometric analysis. The five excluded from analysis chose to withdraw after being unable to complete PRECiS due to severe receptive aphasia and/or cognitive impairment. The 159 included had a wide range of scores on cognitive screening tests and measures of stroke severity (see Methods and Table 1). We recruited stroke survivors with a range of ages (from 34 to 93 years) and times post stroke (from 6 months to 25 years).

Table 1.

Characteristics of stroke survivors with usable PRECiS data (N=159).

Variable	N (%)/Mean (SD)
MALE	89 (56%)
Age (years)	65 (12)
Years since stroke	3.1 (3.7)
Ethnicity: White British	156 (98%)
Asian	1 (0.6%)
Black	2 (1%)
Employment: Retired	93 (59%)
Full/part/self/voluntary	28 (18%)
Unemployed / sick leave	28 (18%)
Other	10 (6%)
At least 12 years education	82 (52%)
Live with others	110 (69%)
Modified Barthel	16.8 (4.4)
MoCA	19.6 (5.5)
Cognitive Impairment Indicated (<26)	138 (87%)
FAST*	24.6 (5.4)
Aphasia indicated (score & age dependent)	67 (43%)
AST TULIA*	11.3 (1.3)
Apraxia indicated (<9)	7 (4%)
Star Cancellation ^†	51.1 (7.5)
Neglect indicated (⩽44)	14 (9%)

1 participant refused to complete this test.^†2 participants could not complete this test due to visual processing difficulties.

MoCA: Montreal Cognitive Assessment; FAST: Frenchay Aphasia Screening Test; AST TULIA: Apraxia Screen of TULIA (Test for Upper Limb Apraxia).

Carer-as-proxy data were available for 86 of the 159 stroke survivors participants (54%). For the 73 without carer-as-proxy data, 38 (24%) lived alone, 34 (21%) lived with others who were either unavailable during the visit (e.g. during working hours) or chose not to take part (e.g. using the visit as an opportunity to do independent errands) and one carer agreed to take part but withdrew after finding it too difficult to answer PRECiS as a proxy. The 86 participating carers were mostly female (N=59, 67%) and aged between 33 and 84 years old (mean = 61 (SD = 12)). The majority (N=70, 81%) were partners of the participating stroke survivor and 35 (41%) were in paid employment. Carers mostly had good functional abilities, with all carers achieving scores of at least 15 out of 20 on the modified Barthel.

The flowchart in Figure 2 summarises the number of participants and the measures they completed.

Figure 2.

Flowchart showing number of participants and the measures they completed.

Acceptability and practicality

Table 2 summarises the psychometric properties of PRECiS. For the 159 (97%) completing PRECiS, there was good acceptability with very few missing items and a median time to complete of 13 minutes. Adapted versions of material were used with 15 (9%) stroke survivors. Adapted materials were not sufficiently accessible for the five participants who were unable to complete PRECiS and withdrew. Researchers noted concerns about comprehension or insight for 31 (19%) participants. Concerns about comprehension were often raised when participants appeared to contradict themselves throughout the visit. For example: expressing that they had difficulties with their memory in the early stages of the visit, then selecting that they had ‘no problems’ with memory in PRECiS. The administrator guide encouraged researchers to prompt participants to reflect on their answers in these cases but ultimately, answers were recorded as given by participants.

Table 2.

Psychometric properties of PRECiS.

Variable	Result
Missing data (> 10% missing items)	0
Skewness (> ±1. n of items affected)	4 (15%)
Floor/Ceiling effects	0
Minutes to complete	Mean (SD) = 14.9 (7.4)
Cronbach’s α	0.94
Item-total correlation (Pearsons r). PRECiS (27 items)	0.33 to 0.75
For each conceptual domain:
Skills (12 items)	0.35 to 0.72
Family and Life (6 items)	0.29 to 0.58
Mood (6 items)	0.49 to 0.72
Sense of Self (3 items)	0.55 to 0.71
Construct correlations (Pearsons r)
Depression (PHQ9) (N=81)	0.72
Anxiety (GAD7) (N=81)	0.47
Activities of Daily Living (NEADL) (N=83)	−0.40
Quality of Life - Stroke Impact Scale (SIS):
SIS Physical (N=65)	−0.48
SIS Communication (N=65)	−0.56
SIS Emotion (N=65)	−0.74
SIS Memory (N=65)	−0.66
SIS Social Relationships (N=65)	−0.67
Short Form SIS Composite(N=10)	−0.24
Disability measure (mBI) (N=159)	−0.29
Cognitive impairment screening tests:
MoCA (N=159)	−0.18
FAST (N=158)	−0.21
AST TULIA (N=158)	−0.20
Star (N=157)	−0.11

PHQ9: Patient Health Questionnaire-9; NEADL: Nottingham Extended Activities of Daily Living; GAD7: Generalized Anxiety Disorder-7; SIS: Stroke Impact Scale; mBI: modified Barthel Index; MoCA: Montreal Cognitive Assessment; FAST: Frenchay Aphasia Screening Test; AST TULIA: Apraxia Screen of TULIA; Star : Star Cancellation.

Missing data were minimal and there were no floor/ceiling effects; respondents made use of the full range of responses, showing minimal end aversion (see Figure 3). Skewness was an issue for 15% of items (N=4): items nine, 10, 11 and 16 (finding new places; using objects; ordering actions; and friendships). All were positively skewed, indicating that they were highly endorsed and highly bothersome.

Figure 3.

Frequency (%) of response rating endorsement across each item.

Qualitative feedback from the 159 participants who completed PRECiS was positive. Participants mostly found the questionnaire content clear and easy to understand and respond to. They felt it was a good length and format, with good coverage of the topic; enjoying rating ‘bother’ as a relatable and endorsable concept. There was some critique as well. Thirty four participants (21%) commented on difficulties with the clarity of PRECiS; reporting that it was challenging to consider cognition outside of motor function or understand the term cognition, despite the pictorial memory aid with definition. Some found it difficult to consider the impact of chronic cognitive problems over the “last week or so” given how they had adapted to the condition but still felt bothered if considering a pre-stroke self. A small number (N=15, 9%) felt that “bothered” wasn’t an appropriate word as it alluded to caring; suggesting that “stressed” or “frustrated” might better capture impact.

Internal consistency

Cronbach’s alpha was high at 0.94, suggesting possible redundant items. Candidate items were 2 (thinking clearly), 24 (feeling negative about the future) and 26 (feeling capable) due to high correlations (>0.6) with other conceptually similar items (respectively: concentration; feeling sad; feeling like a burden). Item-total correlations were all > 0.2 (ranging from 0.29 to 0.75).

Construct validity – internal

A total of 108 item/dimension correlations were explored for item convergence and discrimination relating to the four proposed dimensions (skills; life; mood; and self). Our items did not meet pre-specified criteria so the structure of the scale does not statistically map the four dimensions. The format and content of the measure was supported by participants in qualitative feedback; they found the different ‘sections’ intuitive.

Factor analysis demonstrated substantial loading (>0.4) of all items except items 9 (finding places, 0.38) and 14 (work, 0.35) on the first unrotated factor that explained 38.6% of variance. A 6-factor solution explained 61% of the variance but the content of factors on the rotated varimax solution did not map onto four conceptual dimensions (see Table 3).

Table 3.

Factor loadings for PRECiS based on principal component analysis with varimax rotation N=159.

			Factor loadings (N=159)
			1	2	3	4	5	6
Conceptual domains in questionnaire generation	SKILLS	1 - think quickly	.28	.76
		2 - think clearly		.71
		3 - concentrate	.35	.63
		4 - remember		.65		.35
		5 - communicate		.30	.71
		6 - problem solve	.43	.46	.32
		7 - decisions		.56	.40		.39
		8 - find things	.44			.47
		9 - find places					.85
		10 - objects	.39		.31	.33	.32
		11 - order actions	.32	.32		.47	.35
		12 - move safely			.36	.30	.28	.31
	LIFE	13 - family	.27		.63	.31
		14 - work				.78
		15 - hobbies		.31				.72
		16 – friends	.56
		17 – importance	.32					.66
		18 – energy		.51		.43		.27
	MOOD	19 – sad	.63	.36				.30
		20 – worry	.53	.51
		21 - embarrass	.67		.27
		22 – angry	.56	.28	.26
		23 - emotionalism	.52	.39
		24 - future	.61	.27			.31
	SELF	25 - burden	.69		.29
		26 – capable	.74					.32
		27 - confidence	.42	.28	.34		.51

Highest loading for each item bolded.

Factor loadings <.25 are suppressed.

Construct validity - external

The target for a minimum of N=50 for each tool to be compared was exceeded (see Table 2). However, ten of the 75 participants (13%) selected to complete the Stroke Impact Scale had difficulties completing the full version.

PRECiS correlated strongly with depression (r= 0.72), less strongly with anxiety (r=0.47) and lower still with extended activities of daily living (r=0.4). Correlations with the Stroke Impact Scale varied but, of all the subscales, the strongest correlation was emotion (-.74) and the lowest was physical (-.48). Relationships between PRECiS and baseline severity / cognitive screen data were weakest (r=0.11 to 0.29).

A regression model using age, sex, time post stroke and the five additional exploratory PRECiS questions explained 55% of the variation in total PRECiS score. The strongest predictors in the model were acceptance of cognitive difficulties and the perception that others understood the effects of problems with cognition (P⩽.01). Younger stroke survivors rated more bother (P=0.01).

Carer as proxy respondent (inter-rater reliability)

A total of 86 informal carers completed PRECiS as proxy respondents. There was low-to-moderate agreement between stroke survivors and carers (ICC = 0.43, 95%CI = 0.24 to 0.59). The responses to the PRECiS additional question asking for agreement with the statement “I try to hide my problems with cognition” predicted 58% of the variance in difference between stroke survivor and carer scores; higher agreement on this question meant larger discrepancy. Qualitative carer feedback suggested that whilst many completed the measure as a proxy with ease, they were not confident rating ‘bother’ as it was not directly observable.

Test-retest reliability and sensitivity to change

Sixty-six participants completed PRECiS a second time, within two weeks of visit one. PRECiS had high test-retest reliability (ICC = 0.85, 95%CI = 0.76 to 0.9). The mean difference in scores from visits one to two was 3 (SD=12.1). Bland-Altman plots (see Figure 4) revealed that, whilst the majority of participants (N=40, 61%), were within acceptable limits of change (maximum of 10 points difference on the 108 point scale), changes varied from −25 to +44 points on PRECiS across the sample as a whole.

Figure 4.

Bland-Altman plot of the N=66 with test-retest data. Points within the red line are within acceptable limits of change (10 points difference on the 108 point scale).

Qualitative data collected at visit two related to changes in perceived impact of cognitive difficulties and/or mood. Participants were categorised into three groups for implying positive change (N=15), negative change (N=15), or no change (N=36). For example, P001 reported that visit one had prompted the re-uptake of taught strategies to overcome her cognitive difficulties and she had been pleased with the results (positive change). Major life changes had occurred for some participants between visits e.g. P002 workplace mentoring (positive change), P005 return to work (positive change), P057 dealing with house repossession (negative change). Figure 5 shows change scores according to these three groups, including means and standard deviations. There was a statistically significant difference between the groups (F(2,63)=7.65, P=0.001), consistent with the subjective classification.

Figure 5.

Scatterplot of raw changes in score against ‘perceived changes’ categories. On the Y-axis a positive number = a positive change / reduced bother ratings.

Discussion

PRECiS is a new patient-centred outcome developed to measure perceived impact of cognitive difficulties. It has been evaluated in a population of community-dwelling adults in the chronic phase post-stroke with a wide variety of cognitive impairment and self-reported cognitive difficulties, including those with and without aphasia. The analysis demonstrates that PRECiS is acceptable to potential users, is practical to use (interviewer-administered), has good reliability and construct validity, with some supportive evidence of sensitivity to change. PRECiS provides an important insight into the stroke survivor perspective; one that may help guide rehabilitation approaches as well as evaluate their effectiveness. This perspective may not be available to others, including carers with a close relationship; given our finding that carers were not reliable proxy respondents.

The quantitative analysis proposed three items (number 2, 24, and 26) that may be candidates for exclusion. However, participants qualitatively reported positive face validity for all items. Thus the items have been retained but future work will include service user consultation around removal of these items as well as testing with a new sample to explore impact on psychometric properties.

We included people with a wide range of cognitive and communication difficulties who may otherwise be excluded from development and completion of these measures.⁷ The face to face completion – including administrator guide and alternative easy access format – allowed flexible delivery reflecting the varying needs of participants. The aim was to maximise motivation and ability to respond without affecting reliability. A more standardised mode of administration, such as by phone or mail, may not have been appropriate for this patient group. Future work will be required to generate guidelines for the minimum levels of cognition and insight necessary to complete PRECiS, since researchers reported concerns about understanding that may have affected the reliability of scores for 31 (19%) included participants. In addition, five of the 164 (3%) initially recruited were excluded due to difficulties completing PRECiS. However, this compares favourably with 10 of 75 (13%) participants who were unable to complete the full Stroke Impact Scale; a well-used scale in stroke.

The notion of insight and comprehension is a potential issue for any patient reported outcome measure that seeks the perspective of those with cognitive difficulties. It is a strength of this study that we collected qualitative feedback from stroke survivors, carers and researchers to support analysis of the data as a whole and we would recommend this approach for future use. Whilst the majority fed back that they found PRECiS simple to understand and use, a minority of participants highlighted potential ambiguities or difficulties. Some found it difficult to consider the impact of cognition outside of motor function. For example, severely physically impaired/house-bound participants might be bothered by difficulty maintaining relationships with friends, regardless of the effect of cognitive abilities on their friendships. However, the majority of respondents were positive about the approach and research in the field of cancer³¹ and Parkinson’s disease³² has endorsed the viewpoint that cognitive difficulties impact domains of life that should be explored explicitly in rehabilitation approaches and measurement tools.

Our cross-sectional study was not designed to detect change. However, we wished to test the assumption that perceived impact would remain stable between two closely-timed visits. This might be particularly relevant in our population that were in the chronic stages post-stroke; rarely receiving community rehabilitation for their difficulties. For these individuals, visit one could have negative implications by focusing attention on negative impacts of cognition. Conversely the process of measurement and discussion could be therapeutic in itself. Or, by chance, important life events could occur between visits that could legitimately have an effect on perceived impact. By asking simple questions at the beginning of visit two, we were able to opportunistically explore this in more detail and our findings suggest that PRECiS may be sensitive to change. However, changes were not always in the direction expected and participants would sometimes focus heavily on the timescale implications (rating “in the last week or so”) such that an incident in the short time between visits could be prominent in the memory and affect levels of bother. Future work must test sensitivity to change.

Previous work highlights the limitations of rating frequency of a problem; particularly for cognitively-impaired individuals for whom accurate recall might be an issue.^{11, 25} PRECiS uses ‘bother’ as an alternative, which may be susceptible to fluctuations due to variable emotional state. However, emotional state has been shown to significantly contribute to perceived recovery³³ and it would arguably affect any self-report measurement tool, given the level of psychological engagement required to complete a questionnaire. Improving emotional state and outlook would therefore be an important aspect of any rehabilitation intervention and PRECiS is equipped to detect change in these cases.

The sample was almost exclusively of white British ethnicity. This has implications for generalisability and future work would usefully include testing within a more ethnically diverse population. The mean age of 65 years old is relatively young for a stroke population.³⁴ However, a broad age range of individuals was invited to participate. Age was a significant factor influencing PRECiS scores (with younger stroke survivors reporting more bother) and it is possible that younger stroke survivors overall are more bothered by their cognitive difficulties – for example, if still of working age and unable to work. Younger stroke survivors may therefore have been more driven to participate in this study since they are the individuals who seek cognitive rehabilitation and, by extension, on whom PRECiS would eventually be used.

Clinical messages

PRECiS is a patient-centred, practical, reliable measure of the perceived impact of cognitive problems on skills, family, life, mood and sense of self. It assesses the unique perspective of the person with cognitive difficulties.

PRECiS has been designed with and for people with post-stroke cognitive difficulties including aphasia.

Footnotes

Acknowledgements

We would like to thank participants of this study as well as the service user consultees on the project. We are grateful to the following for supporting recruitment to this study: the stroke support groups, NHS community teams and Primary Care practices in Greater Manchester; Jacki Mundi and Sue Patterson at the North East London NHS Foundation Trust. Finally thank you to Stroke Association for funding this research.

Declaration of Conflicting Interests

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: AB and EP’s salaries are partly funded by the National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care (NIHR CLAHRC) Greater Manchester. The funder had no role in the design of the study, data collection and analysis, decision to publish, or preparation of the manuscript. However, the project outlined in this article may be considered to be affiliated to the work of the NIHR CLAHRC Greater Manchester. The views expressed in this article are those of the author(s) and not necessarily those of the NHS, NIHR or the Department of Health.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by the Stroke Association (reference TSA_JRTF_2010/01). The Primary Care Clinical Research Network (CRN) of the Department of Health National Institute for Health Research (DH NIHR) funded Service Support Costs (SSCs) for primary care practices involved in recruitment (reference UKCRN ID: 15113)

References

Cumming

Marshall

Lazar

. Stroke, cognitive deficits, and rehabilitation: still an incomplete picture. Int J Stroke. 2013; 8: 38–45.

Patel

Coshall

Rudd

Wolfe

. Cognitive impairment after stroke: clinical determinants and its associations with long-term stroke outcomes. J Am Geriatr Soc. 2002; 50: 700–706.

Barker-Collo

Feigin

Parag

Lawes

Senior

. Auckland Stroke Outcomes Study. Part 2: Cognition and functional outcomes 5 years poststroke. Neurology. 2010; 75: 1608–1616.

Pollock

St George

Fenton

Firkins

. Top ten research priorities relating to life after stroke. Lancet Neurol. 2012; 11: 209.

Gillespie

Bowen

Chung

Cockburn

Knapp

Pollock

. Rehabilitation for post-stroke cognitive impairment: an overview of recommendations arising from systematic reviews of current evidence. Clin Rehabil. 2014; 29: 120–128.

Quinn

Dawson

Walters

Lees

. Functional outcome measures in contemporary stroke trials. Int J Stroke. 2009; 4: 200–205.

Dawson

Doll

Fitzpatrick

Jenkinson

Carr

. The routine use of patient reported outcome measures in healthcare settings. BMJ. 2010; 340: c186.

Kroll

Wyke

Jahagirdar

Ritchie

. If patient-reported outcome measures are considered key health-care quality indicators, who is excluded from participation? Health Expectations. 2012; 17: 605–607.

Broadbent

Cooper

FitzGerald

Parkes

. The Cognitive Failures Questionnaire (CFQ) and its correlates. Br J Clin Psychol. 1982; 21: 1–16.

10.

Lawrence

Kinn

. Defining and measuring patient-centred care: an example from a mixed-methods systematic review of the stroke literature. Health Expectations. 2012; 15: 295–326.

11.

Patchick

Horne

Woodward-Nutt

Vail

Bowen

. Development of a patient-centred, patient-reported outcome measure (PROM) for post-stroke cognitive rehabilitation: qualitative interviews with stroke survivors to inform design and content. Health Expectations. 2015; 18 (6): 3213–3224.

12.

Tibaek

Dehlendorff

. Validity of the Danish Prostate Symptom Score questionnaire in stroke. Acta Neurologica Scandinavica. 2009; 120: 411–417.

13.

Collin

Wade

Davies

Horne

. The Barthel ADL Index: a reliability study. Int Disabil Stud. 1988; 10: 61–63.

14.

Nasreddine

Phillips

Bedirian

. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc. 2005; 53: 695–699.

15.

Vanbellingen

Kersten

Van de Winckel

. A new bedside test of gestures in stroke: the apraxia screen of TULIA (AST). J Neurol Neurosurg Psychiatry. 2011; 82: 389–392.

16.

Wilson

Cockburn

Halligan

. Development of a behavioural test of visuo-spatial neglect Arch Phys Med Rehabil. 1987; 68: 98–102.

17.

Enderby

Wood

Wade

Hewer

. The Frenchay Aphasia Screening Test: a short, simple test for aphasia appropriate for non-specialists. Disability & Rehabilitation. 1986; 8: 166–170.

18.

Altman

. Practical statistics for medical research. London: Chapman & Hall, 1991,pp. 1365–1395.

19.

Bland

Altman

. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986; 1: 307–310.

20.

Spitzer

Kroenke

Williams

. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary Care Evaluation of Mental Disorders. Patient Health Questionnaire. JAMA. 1999; 282: 1737–1744.

21.

Spitzer

Kroenke

Williams

Lowe

. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006; 166: 1092–1097.

22.

Nouri

Lincoln

. An extended ADL scale for use with stroke patients. Clinical Rehabilitation. 1987; 1: 301–305.

23.

Jenkinson

Gibbons

Fitzpatrick

. A structred review of patient-reported outcome measures in relation to stroke. Oxford: Department of Public Health, 2009.

24.

Jenkinson

Fitzpatrick

Crocker

Peters

. The Stroke Impact Scale: validation in a UK setting and development of a SIS short form and SIS index. Stroke. 2013; 44: 2532–2535.

25.

Streiner

Norman

. Health measurement scales : a practical guide to their development and use. 4th ed. Oxford: Oxford University Press, 2008.

26.

McHorney

Ware

Jr Lu

Sherbourne

. The MOS 36-item Short-Form Health Survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse patient groups. Med Care. 1994; 32: 40–66.

27.

Horner

Harvey

Denier

. Self-report and objective measures of cognitive deficit in patients entering substance abuse treatment. Psychiatry Res. 1999; 86: 155–161.

28.

Kauhanen

Korpelainen

Hiltunen

. Aphasia, depression, and non-verbal cognitive impairment in ischaemic stroke. Cerebrovasc Dis. 2000; 10: 455–461.

29.

Hackett

Anderson

. Predictors of depression after stroke: a systematic review of observational studies. Stroke. 2005; 36: 2296–2301.

30.

WHO. Towards a Common Language for Functioning, Disability and Health. Geneva: World Health Organization, 2002.

31.

Habermann

Carpenter

Schneider

. Impact of perceived cognitive impairment in breast cancer survivors. European Journal of Oncology Nursing. 2013; 17: 236–241.

32.

Lawson

Yarnall

Duncan

. Severity of mild cognitive impairment in early Parkinson’s disease contributes to poorer quality of life. Parkinsonism & Related Disorders. 2014; 20: 1071–1075.

33.

Duncan

Wallace

Lai

Johnson

Embretson

Laster

. The stroke impact scale version 2.0. Evaluation of reliability, validity, and sensitivity to change. Stroke. 1999; 30: 2131–2140.

34.

Lee

Shafe

Cowie

. UK stroke incidence, mortality and cardiovascular risk management 1999–2008: time-trend analysis from the General Practice Research Database. BMJ open. 2011; 1: e000269.

PRECiS (Patient Reported Evaluation of Cognitive State): psychometric evaluation of a new patient reported outcome measure of the impact of stroke

Abstract

Objective:

Design:

Setting:

Participants:

Measures:

Results:

Conclusions:

Keywords

Introduction

Methods

Data analysis

Acceptability and practicality

Internal consistency

Construct validity – internal

Construct validity – external

Carer as proxy respondent (inter-rater reliability)

Test-retest reliability and sensitivity to change

Results

Acceptability and practicality

Internal consistency

Construct validity – internal

Construct validity - external

Carer as proxy respondent (inter-rater reliability)

Test-retest reliability and sensitivity to change

Discussion

Clinical messages

Footnotes

Acknowledgements

Declaration of Conflicting Interests

Funding

References