Abstract
Introduction
Impairments in executive functioning are common following acquired brain injury (ABI) involving the prefrontal cortex1,2 and are associated with poorer functional and social outcomes.3,4 Executive processes include breaking down a complex goal into a series of ordered subgoals that determine behavior, holding the steps and overarching goal in mind, constraining attention and behavior to the main goal, and weighing its priority against competing demands that may arise.1,5-7 When a goal cannot be executed immediately it becomes a prospective memory (PM) 8 that does not remain at the forefront of consciousness but remains latent, to be recalled at the appropriate time (time-based PM), when the appropriate opportunity arises (event-based PM), or at some future stage (step PM 9 ). PM failure can result from memory difficulties (forgetting the plan) and executive difficulties 8 (failure to act despite memory of one’s intention, also known as goal neglect 10 ). Rehabilitation of executive functioning is, therefore, inherently challenging because the capacities that maximize adaptive change, including ability to transfer rehabilitation from clinic to everyday life, are compromised, resulting in reduced effectiveness of rehabilitation11-13 and poorer emotional outcomes. 14
Interventions for executive deficits such as Goal Management Training (GMT5,15) emphasize effective implementation of intentions to varying degrees. Typically run in groups over 8 or more sessions, GMT includes education to develop awareness and structured practice of goal setting, self-monitoring, and managing competing distractions. 16 Reviews of intervention studies favor metacognitive strategy training (incorporating self-monitoring and self-regulation) 17 and approaches combining GMT with other strategies such as supports for transfer into daily life 16 over stand-alone goal management. The latter review concluded that proof of principle was demonstrated for studies of content-free cues provided at random intervals for improving goal-directed behavior during brief (10-15 minute), complex office-based tasks.18,19 However, whereas the international INCOG guideline for rehabilitation of executive impairment supports the use of metacognitive strategy training, 20 the INCOG guideline for rehabilitation of attention deficits 11 states that evidence for periodic content-free cues is conflicting and further clinical outcome studies are required. A functional imaging study failed to find beneficial effects of periodic alerts on the Sustained Attention to Response Test (SART) but did show reduced right dorsolateral prefrontal activation during provision of alerts. This was interpreted by the authors as indicating that cues assisted the maintenance of intentions by reducing reliance on specific endogenous control processes underpinned by the right frontoparietal control and attention networks involved in sustaining attention to task goals. 21 A recent trial 22 found that GMT incorporating text message reminders resulted in gains on self-report and neuropsychological measures, although the independent contribution of cueing was not evaluated. Previous trials have used questionnaires or neuropsychological tests rather than real-world behavioral measures to evaluate outcome. In one exception to this, Fish et al 23 evaluated transfer of training on a naturalistic task of remembering to make phone calls at set times each day over a 2-week period. Participants with ABI learned specified times to call the study’s answerphone, then received very brief (30-minute) GMT in which the process of pausing current activity to mentally review one’s intentions was linked with a cue phrase (“STOP”; Stop, Think, Organize, Plan). STOP cues were provided on randomly selected days at random intervals. Cued days were associated with significantly more, and more accurately timed, calls than noncued days. Although promising for potential application in rehabilitation, the effectiveness in terms of participants’ own everyday intentions and potential effect on emotional outcomes were not evaluated. Further evaluation of the effect of combined brief GMT and cueing on everyday goals is, therefore, required.
Here we report a trial examining the efficacy of assisted intention monitoring (AIM) comprising brief GMT followed by randomly timed SMS text messages for improving achievement of everyday intentions. The broad aim was to extend prior research using GMT plus periodic alerts to evaluate potential efficacy in improving achievement of everyday intentions. The primary outcome was a composite score of proportion of all intentions achieved, made up of different types of intention and an objectively scored proxy task (the phone task). The primary study hypotheses were the following:
the proportion of all intentions achieved will be significantly greater during AIM than control phases, and
the proportion of all intentions achieved excluding the phone call task will be significantly greater during AIM than control phases.
A subsidiary hypothesis was that increased goal attainment would be associated with improved self-rated mood. Exploratory analyses were planned to identify factors that might influence response to intervention, a necessary process in the development of complex health care interventions. 24
Method
Ethics
Ethical approval to conduct the study was provided by a National Health Service Research Ethics Committee (study reference 08/H0306/45) and the relevant Research and Development Department for each of the health services involved in recruitment of participants. All participants provided written informed consent to participate.
Trial Design
The study used a randomized controlled, parallel group crossover design with 3 phases (baseline phase, intervention phase 1, intervention phase 2), each of which lasted 3 weeks, with a 1-week break between phases for completion of measures (phases shown in Figure 1). Assessments and primary analyses were conducted blind to group allocation. Following consent, participants completed initial assessment questionnaires and neuropsychological tests and were supported in identifying daily intentions to be monitored for the study duration. They were then randomized to either AIM or control for intervention phase 1 (equal numbers in each), after which they crossed over to phase 2, during which AIM-first participants received no intervention or usual care and control-first participants received AIM. A conceptually symmetrical crossover was not possible for the AIM-first group because messages from the study had already been associated with reviewing intentions. The cessation of messages to the AIM-first group in phase 2, therefore, allowed examination of whether their receipt was relevant to efficacy of goal management. This design also ensured that all participants had access to an intervention hypothesized to be useful, minimized the possible confounding effect of group differences on treatment effects, provided increased power to detect effects, and allowed examination of the maintenance of any gains in the AIM-first group.
Trial flow chart showing numbers of participants referred, excluded, randomized to intervention, completed, and analyzed.
A Steering Group comprising researchers, the local NHS Research and Development manager, and a person who had sustained a brain injury oversaw study management. The trial was conducted in accordance with National Institute for Health Research (NIHR) Good Clinical Practice in research guidelines, was adopted by the United Kingdom Clinical Research Network (UKCRN), and registered onto their research portfolio (Study ID: 5368).
Participants
Participants were recruited from UK community services in the East Anglia region, the Cambridge Cognitive Neuroscience Research Panel (CCNRP; a group of people with ABI who have agreed to be approached for relevant research studies) between February 2009 and August 2011. Health care professionals working with ABI patients were asked to provide potential participants with information about the study and seek their consent to be contacted by the research team. Members of the CCNRP were contacted directly by the researcher.
Inclusion criteria were as follows:
age ≥18 years;
nonprogressive brain injury, acquired in adulthood;
more than 1 year postinjury;
clinician, carer, or self-reported everyday organization and memory problems; and
able to use a mobile phone.
Exclusion criteria were as follows:
memory impairment of sufficient severity to limit retention of intentions and training information (clinical judgment and neuropsychological assessment);
patient or carer participant with severe and enduring mental health problem, or substance misuse or dependence, as identified by referring clinician; and
participation in a rehabilitation intervention with significant overlap with the study intervention.
Interventions
Interventions were delivered by a member of the research team (EG), a qualified occupational therapist with significant experience in providing cognitive rehabilitation interventions in both clinical and research settings with people with stroke and ABI. TM, a coauthor of the GMT materials, provided supervision.
Assisted Intention Monitoring
A brief GMT was provided by EG in participants’ homes or a community setting on a one-to-one basis over 2 sessions, not more than 5 days apart, each lasting between 90 and 120 minutes. Training materials were selected from the full GMT program (as described by Levine and colleagues5,6,15) and presented on a laptop as a PowerPoint presentation with an accompanying workbook. The slides selected covered the following topics supported with discussion of examples drawn from the workbook or provided by the participant:
utility of setting goals and breaking goals into steps (module 1)—for example, breaking a large goal or problem such as planning a trip away into doable steps;
absentmindedness and slip-ups (module 2)—for example, walking into a room and forgetting what you went there for and discussion of factors that can increase slips, such as fatigue;
using the “mental blackboard” to take note of goals and steps (module 5)—for example, rehearsing the mental visualization of written or pictorial checklist of steps on a “blackboard”; and
checking the status of one’s intentions (module 9), which was linked with the acronym “STOP”—for example, discussing how periodically stopping and thinking about our intentions can help us stay on track.
The training was provided to the point where the trainer was confident that the participant understood the material and the STOP acronym, so the training period varied depending on the knowledge and abilities of the participant. Participants were told that after training they would receive 8 “STOP” texts each day, designed to increase the frequency of such reviews. These occurred at random points between 08:00 and 18:00 hours on each working day. They did not occur within 30 minutes of each other or a set phone call time (see below). Messaging was provided via a reminding service 25 with the capacity to send SMS text messages.
Control Intervention
This involved one-to-one sessions (also provided by EG) of the same duration as AIM consisting of brain injury information 26 (excluding reference to executive functioning) presented using Powerpoint and a computerized visuospatial game involving increasingly speeded mental rotation (Tetris) plausibly linked to improving cognitive skills but not hypothesized to improve PM. Participants in the control phase also received 8 daily SMS text messages reading: “AIM research study. Please ignore.”
Measures
Assessment and Screening Measures
Standardized neuropsychological assessments were completed and demographic and injury-related data collected. The National Adult Reading Test 27 was used to derive an estimate of premorbid general intelligence. The Speed and Capacity of Language Processing 28 was used to assess speed of processing. Nonverbal reasoning abilities were assessed with the Matrix Reasoning subtest of the Wechsler Adult Intelligence Scale, Third Edition. 29 Immediate and delayed verbal recall was assessed using the Logical Memory subtest of the Wechsler Memory Scales. 30 Executive functioning and attention were assessed using the Letter Fluency part of the Verbal Fluency subtest (Delis-Kaplan Executive Functioning System), 31 the SART,32,33 and the multipart Hotel Test 18 (similar to the 6 elements 34 ). The Coping Inventory for Stressful Situations, 35 which has been validated for use with ABI,36,37 was included to identify possible moderators of treatment response.
Primary Outcome
The primary outcome was the mean daily proportion of intentions achieved by a participant averaged over the final 2 weeks of each 3-week study phase (consistent with previous studies,23,25 data from the first week were excluded because of novelty effects). The primary outcome measure was a composite of participants’ own, ongoing “set” intentions, established at initial assessment with the researcher and set for the study duration; participants’ ad hoc intentions, one-off tasks that might arise during the course of the study; 7 fixed intentions set to ensure compliance with study procedures (eg, make sure mobile phone is with you, charged, and switched on); and the phone task 23 described below. With the exception of the phone task, participants recorded success or otherwise in a structured diary and relayed this information to the research team in a daily phone call initiated by the researcher (according to preference, this could be via less-frequent phone calls, no fewer than 3 per week, or via email). This was also used to determine if goals were irrelevant (eg, “remembering keys and wallet when going out” would be irrelevant on a day intentionally spent indoors).
At initial assessment, participants were asked to nominate 3 times of the day when it would be convenient to make a brief call to the study’s answerphone. These had to be at least 30 minutes from a previous phone call and not set to coincide with a memorable time of day. Participants were asked to make their calls as close to the set time as possible over the 9 weeks of the study phases (ie, time-based PM) in addition to 1 further phone call at an unscheduled time each day (ie, step PM). Participants were simply asked to state their name on connection. Attainment and timing accuracy were scored from answerphone records. Scheduled calls made within 5 minutes (±) of the target time scored 6. This decreased by 1 for each additional 5-minute discrepancy down to 1 (±25 out) and 0 (call missed completely). Unscheduled calls gained 1 point if they were made at all, a further point if they were more than 30 minutes from another call, and a final point if they were made at a different time to the unscheduled call on previous days of the study. Not all calls were possible on all days because of phone malfunction, poor signal, or clash with important activity, and accordingly, the score was based on the proportion of the score achieved out of the total score attainable that day.
For each day, the total number of relevant intentions for each participant in each intention type (set, ad hoc, fixed, and phone calls) was summed and the daily proportion attained calculated. These values were then averaged across each 2-week assessment period.
Secondary Outcome
Given expectations that the phone call task would benefit from AIM, our second planned comparison considered attainment of all goals excluding the phone call task.
Subsidiary Measures
Subsidiary measures were administered after each baseline and intervention phase. The Profile of Mood States 38 total mood disturbance (POMS MD) score was used to evaluate the impact of AIM on overall emotional functioning. The Hotel Task and Verbal Fluency were used to evaluate effect of AIM on executive functioning in the absence of cues.
Randomization
The randomization procedure was administered by the academic department of one of the authors (JJE) at a site remote from the main research site. Blocked sequences (6 and 4, via http://www.randomization.com) enabled equal numbers of participants to be allocated to each group. Only 1 investigator (JJE) was able to access the sequence and allocation, which remained concealed until the researcher delivering the interventions (EG) requested the next participant allocation code, which was provided via email. Allocations were not revealed to any other member of the study team, clinical staff in recruitment sites, or participants.
Analysis and Sample Size Calculation
Hypotheses 1 and 2 were tested with crossover analyses conducted on the complete data set on a per protocol (PP) basis using repeated-measures ANOVA, the within-subject factor being study phase (postintervention 1 vs postintervention 2) and between-subject factor being group (control-first vs AIM-first), with baseline scores as a covariate. Significant Group × Phase interaction effects were taken as indicating relative efficacy of the AIM intervention. A power calculation for this design carried out using G Power,
39
with α = .05, 80% power, 2 groups, and 1 covariate based on detection of a medium-large effect size (as previously found
24
and to identify potentially clinically meaningful response), indicated that a sample size between 52 (
Results
Participant Characteristics
Enrolment and allocation information is provided in Figure 1. Eligibility screening was carried out for 93 people, 74 proceeded to randomization, and 60 participants completed the study, with 58 participants completing the trial and all outcome measures; 1 further person completed only the daily intention diary, and another completed only the POMS. In the PP group, cause of injury was predominantly traumatic brain injury (TBI; 27, 46%) or stroke (21, 35%). Severity of injury was obtainable for 15 (55%) TBI participants (severe: 11, 41%; moderate: 2, 7%; mild: 2, 7%). Notable differences (PP and ITT) were found in preinjury employment and time since injury and (ITT only) work hours (see Table 1).
Demographic Information and Neuropsychological Test Performance at Initial Assessment for Intention To Treat (ITT) and Per Protocol (PP) Groups.
Abbreviations: AIM, assisted intention monitoring; D-KEFS, Delis-Kaplan Executive Functioning System; NART, National Adult Reading Test; SCOLP, Speed and Capacity of Language Processing; TBI, traumatic brain unjury; WAIS-III, Wechsler Adult Intelligence Scale, Third Edition; WMS-III LM, Wechsler Memory Scales 3rd Edition Logical Memory; CVA, cerebro-vascular accident.aMean of standardised score (standard deviation). **Control first and AIM first groups significantly different at
Hypotheses 1 and 2: Crossover Analyses
Hypotheses 1 and 2 were tested with repeated-measures ANOVA to identify the presence of Group × Time interaction effects in favor of AIM, as planned. Mauchly’s test of sphericity for equality of variances was not significant, and missing data were excluded. Figure 2 shows changes in performance for AIM-first and control-first groups across all phases, for all intentions, and also all intentions excluding the phone call task. For hypothesis 1, the repeated-measures ANOVA yielded a statistically significant Group × Time interaction [
The proportion of intentions achieved for AIM-first and control-first groups at baseline, end of intervention phase 1, and end of intervention phase 2 for all intentions, all intentions minus phone task, and phone task.
In terms of subsidiary analyses, no significant Group × Time interaction effect was found for the POMS MD score [
Group Differences Postintervention Phase 1
Data summarizing group differences postintervention phase 1 are provided in Table 2. For analysis, missing data were excluded, and Levene’s test for equality of variances was not significant. No significant differences on all intentions were identified with ITT (
Comparison of Changes in Primary and Secondary Outcome Measures for Placebo First and AIM First Groups Between Baseline and Intervention Phase One on Intention to Treat and Per Protocol Basis.
Abbreviations: AIM, assisted intention monitoring; POMS MD, Profile of Mood States total mood disturbance.
Exploratory Analyses
To examine factors that may have influenced response to treatment, simple correlations between possible predictor variables (age, time since injury, avoidant coping style, POMS MD) and change (difference between AIM and Control for all intentions and the phone task) were evaluated. The only near-significant correlation (at corrected
Discussion
Interpretation
This study examined whether AIM intervention was associated with enhanced attainment of daily intentions for people with self- or clinician-reported everyday organizational problems and objective executive impairment following ABI. The results show that participants achieved their everyday intentions at a significantly higher frequency during the AIM phases of the study than the control conditions. The findings build on the body of work that shows that randomly occurring periodic cues to prompt “mental review” of intentions may contribute to improved performance on tasks requiring attentive control of goal-directed behavior.18,19,23 The results suggest that any benefit of the training offered in AIM was only detectable when participants were receiving cues. Although this comparison has a confound of the extra time since training, it forms some indication that generalization from training is likely to be enhanced when participants are reminded about it in everyday life. There were no training effects on executive neuropsychological tests (during which cues were not present), suggesting that treatment effects are a result of compensatory management of, rather than improvement in, executive difficulties. A recent trial 22 found that combined group GMT and reminders resulted in improvements to neuropsychological functioning sustained at the 6-month follow-up, suggesting potential benefits of increased intervention time. Fish et al 40 reported independent maintenance of routines after prolonged experience of timed specific reminders, which was evident for TBI participants but not those with stroke; this was attributed to better executive functions in the former group. In the current study, we did not find such group differences in executive functioning, although it is important to note the smaller group sizes, participant selection on the basis of poor organizational skills rather than memory, and the use of cues that occurred at random rather than fixed times each day. Further investigation of the treatment duration and intensity required for internalization of metacognitive or mnemonic cues over time is, thus, warranted.
Comparing groups postintervention phase 1, there was no evidence of significant benefit of the AIM intervention versus placebo on achievement of intentions or mood (ITT and PP analyses) or performance on the phone task (ITT analyses only), although PP analysis found a benefit of AIM for the phone task. At the most conservative level, this result indicates rejection of the study hypotheses. However, the study was not designed with this analysis in mind, and hence, these comparisons were underpowered to detect anything other than large effects. The PP analysis of the effect of training on the phone task at the end of phase 1 did yield favorable results, as did the adequately powered primary crossover analysis. We have, therefore, cautiously rejected the null hypothesis, bearing in mind the study limitations and, in particular, threats to the comparability of groups after crossover.
There were no significant effects of AIM on POMS mood disturbance scores, suggesting that a simple model of enhanced attainment of intentions leading to improved mood may be wrong.
Limitations
At 20%, drop-out rates were high, contributing to selection bias and limiting generalizability of results. It is likely that this attrition is attributable to aspects of the protocol (daily goal-attainment recording, daily phone calls, and long assessment sessions), not the intervention itself. The crossover design was justified to provide an opportunity for both groups to receive the AIM intervention, for the AIM-first group to have a meaningful control phase, for withdrawal of alerts to be monitored in 1 arm, and to provide increased power to detect effects of undergoing the intervention. However, this design combined data from the different control phases, compromising the comparability of arms after the point of crossover. Furthermore, it was not possible to examine efficacy of the intervention at follow-up.
Randomization produced groups that were well matched on primary and secondary outcome measures, neuropsychological functioning, or other demographic variables but that differed on time postinjury and employment. Although any effect is less problematic for the within-subjects crossover analysis, it may have influenced postintervention phase 1 analyses. Regarding precision of measurement, the evaluation of real-world impact of the intervention relied on participants’ own ratings in contrast to the phone task, which provided an objective metric of attainment and, therefore, may have been a more sensitive measure. Although the study was appropriately powered for the analysis of the crossover data, the subsidiary and exploratory analyses should be interpreted with caution. Finally, a number of statistical analyses were used to address main and subsidiary hypotheses and exploratory analyses. To reduce the likelihood of false-positive results, we limited the number of analyses used to test the primary hypotheses and specified the directions of predicted relationships. The exploratory findings are reported as tentative.
Generalizability
The current study included elements of evaluation of effectiveness, such as referral on the basis of clinician, carer, or self-identified problems; intervention deliverable within health services; and evaluation of “real-world” outcomes. However, the delivery of intervention was not tailored to each individual on the basis of specific needs or ongoing response to intervention, and a placebo control condition was included, limiting clinical generalization. Many participants had difficulty with identifying and articulating intentions in precise terms, and results suggested differences in effects depending on etiology. Therefore, careful thought is needed in clinical application. The relatively brief 2-session GMT adopted here (in comparison with the 14 or more hours of face-to-face GMT training typically reported 16 ) might be considered insufficient for many with ABI. Future evaluation of clinical effectiveness should consider a more extended and tailored period of strategy and self-regulation training16,17,22 and inclusion in the intervention of additional components that enhance likelihood of transfer of strategies.16,22,41-43
Conclusions
The results of this trial show some support for the efficacy of combining a brief goal management intervention and cueing. Findings are consistent with previous proof-of-principle studies and have been extended to show some improvement in subjective reports of goal attainment in everyday life. However, when only the initial training period was considered and when ITT was taken into account, effect sizes were small or negligible and not supportive of the efficacy of AIM. The challenge of identifying intentions that are both easy to measure and meaningful to participants may have made detection of effects more difficult. Given the potential effectiveness of AIM, the costliness of neuropsychological rehabilitation interventions, and difficulty transferring skills from rehabilitation to everyday life, further investigation of periodic cues to enhance realization of intentions in everyday life following rehabilitation is warranted.
Footnotes
Acknowledgements
We are grateful to Dr Adam Wagner for additional statistical advice, Prof Barbara Wilson for consultation, all the participants and the staff who took part and supported recruitment, and Alyssa Hewson for formatting and proofing the manuscript. This article presents independent research funded by the National Institute for Health Research (NIHR) under its Research for Patient Benefit (RfPB) Programme (Grant Reference Number PB-PG-0107-12011). FG and DM were also supported by the NIHR Collaborations for Leadership in Applied Health Research and Care (CLAHRC) for Cambridgeshire and Peterborough. TM and JEF were supported by the Medical Research Council (MC-A060-5PQ20). The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, or the Department of Health.
Declaration of Conflicting Interests
The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The authors declare that there are no conflicts of interest. TM is a contributing author to Goal Management Training but receives no income from its commercialization. AB is the manager of the NeuroPage reminding service but receives no personal income from the service.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This article presents independent research funded by the National Institute for Health Research (NIHR) under its Research for Patient Benefit (RfPB) Programme (Grant Reference Number PB-PG-0107-12011). FG and DM were also supported by the National Institute for Health Research (NIHR) Collaborations for Leadership in Applied Health Research and Care (CLAHRC) for Cambridgeshire and Peterborough. TM and JEF were supported by the Medical Research Council (MC-A060-5PQ20).