Abstract
Background. Feedback is used in rehabilitation to improve self-awareness in people with traumatic brain injury (TBI), but there have been no comparisons of the different methods of providing feedback. Objective. To compare the effect of different methods of feedback on impaired self-awareness after TBI. Method. This was a randomized, assessor-blinded trial with concealed allocation. A total of 54 participants with TBI and impaired self-awareness (85% male) were recruited from inpatient and community rehabilitation settings. Participants performed a meal preparation task on 4 occasions and were randomly assigned to 1 of 3 feedback intervention groups: video plus verbal feedback, verbal feedback, or experiential feedback. The primary outcome was improvement in online awareness measured by the number of errors made during task completion. Secondary outcomes included level of intellectual awareness, self-perception of rehabilitation, and emotional status. Results. Receiving video plus verbal feedback reduced the number of errors more than verbal feedback alone (mean difference = 19.7 errors; 95% confidence interval [CI] = 9.2-30.1) and experiential feedback alone (mean difference = 12.4 errors; 95% CI = 1.8-23.0). Conclusion. The results suggest that the video plus verbal feedback approach used in this study was effective in improving self-awareness in people with TBI. The results also provide evidence that improvement in self-awareness was not accompanied by deterioration in emotional status.
Introduction
Impaired self-awareness is common after traumatic brain injury (TBI). 1 Lack of understanding of postinjury impairments can reduce participation in rehabilitation, decrease compliance with compensatory strategies, and impede the ability to set realistic goals. In this way, impaired self-awareness may reduce independence and occupational performance after TBI.2-4 Self-awareness is described as a person’s understanding of their own strengths and limitations and how these affect everyday functioning. 5 It is also defined as the ability to objectively perceive oneself while maintaining a sense of subjectivity. 6 Self-awareness is conceptualized as having 2 components: (1) intellectual awareness, involving self-knowledge and beliefs about one’s abilities, and (2) online awareness, involving the capacity to self-monitor and adjust one’s behavior (eg, self-correction of errors) during task performance. 7
Two single case studies by Ownsworth et al 8 suggested that metacognitive training in the context of meaningful occupation may be effective in improving online awareness after TBI. 8 This study concluded that by targeting error self-regulation, metacognitive training promotes independence in complex everyday tasks. The provision of feedback is a fundamental component of metacognitive training. Feedback is most often provided verbally by a therapist, but it may also be provided using videos or using peers.2,9,10 However, the findings of case studies and clinical observations have not yet been verified by high-quality clinical trials.
A systematic review by Schmidt et al 11 identified 3 randomized controlled trials (RCTs) designed to investigate the effects of feedback interventions (alone or in combination with other interventions) on improvement in self-awareness following TBI. This review concluded that feedback produces moderate improvement in self-awareness (Hedges adjusted g = 0.64; 95% confidence interval [CI] = 0.11-1.16). 11 However, all the studies provided feedback as only 1 component of an intervention program. None of the reviewed trials directly compared different methods of feedback, and all the included trials were of “moderate” or “low” methodological quality. Further high-quality research is required to determine the most effective feedback intervention to improve self-awareness. 11
The primary aim of this study, therefore, was to evaluate the effectiveness of feedback interventions for improving online awareness in individuals with TBI. Comparisons were made with 3 types of feedback interventions—namely, (1) video plus verbal feedback, (2) verbal feedback, and (3) experiential feedback. It was hypothesized that the group receiving the combined video and verbal feedback would demonstrate greater improvement in online awareness, as measured by the reduction in errors during task performance, compared with the other 2 groups. It was also hypothesized that the group receiving verbal feedback would demonstrate greater improvement in online awareness than the experiential feedback group. A secondary aim of this RCT was to compare the effects of the 3 feedback interventions on intellectual awareness, emotional status, and self-perceptions of rehabilitation.
Methods
Design
This study was an assessor- and participant-blinded RCT with 3 intervention groups. The trial was registered with the Australian New Zealand Clinical Trials Registry, identifier number ACTRN12609000378224. 12
Participants
Consecutive admissions to inpatient and community rehabilitation at a brain injury unit in Sydney were screened to recruit 54 participants. The trial recruitment was undertaken between November 2009 and March 2012. To be eligible for inclusion, participants had to have sustained a TBI, have emerged from posttraumatic amnesia (PTA), be older than 16 years, have functional English language, and demonstrate impaired self-awareness, defined as having a positive discrepancy of at least 2 points between participant and therapist ratings on the Awareness Questionnaire (AQ). 13 Although there are no guidelines concerning the amount of discrepancy on the AQ that indicates impairment of self-awareness for treatment purposes, a 2-point positive discrepancy score was used in conjunction with observation of errors in performance of a cooking task. These 2 criteria in combination demonstrated intellectual and online awareness deficits. Therefore, the participant had the potential to benefit from awareness training. Participants were excluded if they had a severe communication disability or behavioral changes that would compromise their ability to participate in a meal preparation task and/or feedback intervention, had an expected length of rehabilitation less than the intervention study period (ie, 2 weeks), or demonstrated an error-free meal preparation during the baseline assessment.
The primary outcome measure was the number of errors identified during a meal preparation task. A sample size calculation performed prior to the commencement of the study indicated that a sample of 54 participants (18 per group) would provide an 80% probability of detecting a 30% reduction in errors, assuming a loss to follow-up of 10% and α of .05. The 30% reduction in errors was based on the investigators’ estimate of a clinically meaningful change. 9 The study received ethical approval from the appropriate institutions, and informed consent was obtained from all participants.
Outcomes
The primary outcome variable of online awareness was measured by counting the number of errors made during the meal preparation sessions from a videotape. An error was counted if the participant performed any component of the task inappropriately or inaccurately, took too much time to complete the step, or demonstrated compromised safety (eg, if the participant forgot to check the toast in the toaster). 8 Assessors (n = 3) were trained to recognize errors in meal preparation, and a sample of videos was rated by all assessors to ensure consistency.
Secondary outcomes were intellectual awareness, emotional distress, and perceptions of rehabilitation. The AQ 13 was used to assess intellectual awareness across sensory, physical, cognitive, and behavioral domains, with positive discrepancy scores (participant minus therapist ratings) indicating that the participant overestimated his or her abilities. 13 The Depression Anxiety Stress Scales (DASS-21) were used to measure symptoms of distress. 14 Self-perception of rehabilitation was measured using the Self-perceptions in Rehabilitation Questionnaire (SPIRQ). 15 This 20-item self-rated questionnaire measured self-perceptions on 4 scales, including (1) changes in self and life plans, (2) emotional reactions, (3) self in rehabilitation, and (4) perceptions of recovery and returning to normal.
Participants also completed a battery of neuropsychological function tests. These included the Behavioural Assessment of the Dysexecutive Syndrome, 16 the Wisconsin Card Sorting Test 17 and the Wechsler Memory Scale, third edition. 18
Study Protocol
After completion of baseline measurements on the secondary outcomes, participants were randomly allocated to 1 of 3 groups: video feedback, verbal feedback, or experiential feedback condition. A computer-generated random allocation schedule was determined prior to the study by one of the authors who was not otherwise involved in group allocation. To ensure concealment, the allocations were placed in sealed, opaque, sequentially numbered envelopes and kept off-site. Following the recruitment and baseline assessment (secondary outcomes only), the investigator contacted an independent researcher to obtain group allocation for each participant. Participants were considered to have entered the trial at this stage.
All participants completed a meal preparation task on 4 occasions, with 2 to 4 days between tasks. All meal preparation tasks were videotaped and were completed in the kitchen of the rehabilitation hospital for inpatients and in the kitchen of participants’ homes for those that lived in the community. Participants chose the meal to prepare from 3 options: spaghetti bolognaise, ham and cheese omelette with toast, or sausages and mashed potatoes. These options were assessed to have equal levels of difficulty, with a similar number of steps, ingredients, and time required to prepare the meal. A choice was provided to theoretically increase motivation and engagement in sessions.19,20 Participants were given a written recipe prior to the first meal preparation session and were permitted to make modifications to suit their personal preference or habits. Any modifications to the recipe then became the standard recipe for all subsequent sessions.
During the participant’s meal preparation, a therapist provided appropriately timed prompts and on-the-spot feedback using the “pause, prompt, praise” technique.3,8,21 Using this technique, the therapist did not provide any prompts initially when the participant made an error, which allowed the participant time to self-correct. However, if the participant did not correct the error within a reasonable period of time (eg, between 2 and 5 s), the therapist provided a nonspecific prompt (eg, “Is there something you should check?”). If the participant continued to make the error, the therapist provided a specific prompt to enable the participant to correct the error (eg, “You should check the toast in the toaster”). The therapist provided positive verbal reinforcement when the participant spontaneously used strategies to enhance the performance of the task or if the participant responded to a prompt and corrected an error. A blinded assessor counted all errors from the videotaped performances (not live ratings) and the error count from the first session became the baseline error score for online awareness (primary outcome). The error count from the final session was used as the postintervention error score.
To structure feedback provided, the treating therapist and participant individually rated task completion at the end of the meal preparation task using the Meal Independence Rating Scale (MIRS). The MIRS was developed as a treatment activity for this study and was based on the Executive Functional Performance Task. 22 It rated the task components of initiation, execution (including organization, sequencing, judgment, and safety), and completion on a 6-point scale according to the amount of assistance required (ie, 0 = independent and 6 = completely dependent).
The participants received a feedback intervention, based on their group allocation, after each of the 4 meal preparation sessions. Feedback sessions for inpatient participants were conducted in a separate treatment room for the purposes of participant blinding. Participants in the video feedback group watched their videotaped performance of the meal preparation task with the therapist. While viewing, the therapist encouraged the participant to retrospectively identify errors in task performance, observe areas of strength, and suggest compensatory strategies that could be used in future sessions. The therapist and the participant then verbally discussed any discrepancies in their ratings of the task performance on the MIRS. Therapist feedback was nonconfrontational and encouraged the participant to disclose areas of strength and limitation (guidelines for the provision of feedback available on request). For participants in the verbal feedback group, the therapist followed the same guidelines for verbal discussion of discrepancies between ratings on the MIRS without viewing the videotape. For participants in the experiential feedback group, no direct feedback was provided following the meal preparation task. The participant and treating therapist separately completed the MIRS, and discrepancies and ratings were not discussed.
Meal preparation sessions and feedback interventions were provided by 3 occupational therapists with at least 1 year of experience working in a specialist unit with people with TBI. Training was provided to all therapists using a feedback protocol, and written guidelines for delivery of feedback were provided. All therapists were trained in each feedback intervention and were equally likely to deliver any of the 3 feedback interventions.
Measures were taken at 2 time points. Baseline measures were completed during the first intervention session for the primary measure of error count and, following recruitment, for all secondary questionnaire measures. Postintervention measures were conducted during the last session for error count and following completion of all 4 of the intervention sessions for secondary variables.
Data Analysis
Between-group differences were calculated, accounting for baseline scores for all 3 groups on all outcomes. Negative binomial regression was used to test for group differences on the primary outcome (number of errors postintervention), with the baseline scores entered as a covariate. Secondary outcomes were analyzed using a series of mixed 2 × 3 analyses of variance (ANOVAs), with group allocation as the between-subjects variable and time (baseline vs postintervention) as the within-subjects or repeated-measures variable. Covariates were included into the analysis if they were associated with the primary outcome and unevenly distributed between the groups. 23 The primary outcome variable (number of errors postintervention) was tested to ensure that assumptions of homoscedasticity and linearity were met. Group means were used to replace missing values where the number of missing values was <5% 23 ; therefore, all participants were included in the analysis. Groups were compared on demographic, injury, neuropsychological, and baseline variables. The size of the effect of intervention was identified using a 95% CI. Data were analyzed using an intention-to-treat analysis, 24 meaning that participants were analyzed in the group to which they had been allocated, irrespective of compliance. 25
Results
The flow of participants through the trial is given in Figure 1. In all, 167 patients with a TBI were admitted and/or involved in inpatient or community brain injury services over the duration of the study. Of these, 101 patients did not meet the inclusion criteria, and 12 declined to be involved. In total, 54 participants enrolled in the study, with the majority recruited from the inpatient service (n = 41, 76%). The mean (standard deviation) age of participants was 40 (13) years, and time since injury was 4 (8) years; 46 participants were men (85%), and most participants had sustained their injuries in a road traffic accident (n = 40, 74%). There was no difference in meal choice between groups (P = .50). There were significant differences between groups on the Functional Independence Measure (P < .01), number of days in PTA (P = .01), the Logical Memory I Age Scale score of the Wechsler Memory Scale III (P = .04), and the failure to maintain set score of the Wisconsin Card Sorting Test (P < .01). However, these were not significantly associated with the primary outcome measure and therefore were not included as covariates in the analysis (Table 1).
Design and flow of participants through the trial. Abbreviations: TBI, traumatic brain injury; AQ, Awareness Questionnaire; DASS, Depression Anxiety Stress Scales; SPIRQ, Self-perceptions in Rehabilitation Questionnaire.
Group Baseline Characteristics on Demographic, Injury-Related, and Baseline Variables
Abbreviations: SD, standard deviation; GCS, Glasgow Coma Scale; PTA, posttraumatic amnesia; BADS, Behavioural Assessment of the Dysexecutive Syndrome.
Effect of Feedback Intervention on Online Awareness (Primary Outcome)
Video plus verbal feedback improved online awareness, as measured by the number of errors, more than either of the comparison interventions (Table 2; Wald χ2 = 152.71; P < .001). The video feedback group displayed significantly fewer errors postintervention compared with the verbal feedback group (mean difference = 19.7; 95% CI = 9.2-30.1) and compared with the experiential feedback group (mean difference = 12.4; 95% CI = 1.8-23.0). There were no statistically significant differences in error counts between the verbal feedback and the experiential feedback groups (mean difference = 7.3; 95% CI = −3.4 to 17.9).
Mean (SD) of Groups, Mean Difference (95% CI) Within Groups, and Mean Difference (95% CI) Between Groups
Abbreviations: SD, standard deviation; CI, confidence interval; AQ, Awareness Questionnaire; DASS, Depression Anxiety Stress Scales; SPIRQ, Self-perceptions in Rehabilitation Questionnaire.
Denotes a statistically significant difference, P < .05.
Denotes a statistically significant difference, P < .01.
Effect of Feedback Intervention on Secondary Outcomes
Video plus verbal feedback also led to greater intellectual awareness as measured by the AQ discrepancy score, demonstrated by a significant Group × Time interaction effect: F(2, 51) = 5.47; P < .01 (Table 2). Participants allocated to the video feedback group had a significantly greater improvement in intellectual awareness compared with the verbal feedback group (mean difference = 4.9; 95% CI = 0.3-9.5) and experiential feedback group (mean difference = 7.3; 95% CI = 2.8-11.9). There was no between-group difference with the verbal feedback and experiential feedback groups (mean difference = −2.4; 95% CI = −7.0 to 2.1) or between any groups for emotional distress, as measured by the DASS-21, or for self-perceptions of rehabilitation, as measured by the SPIRQ (Table 2).
Compliance With Trial Method
Compliance with the intervention protocol was considered successful. Most participants in each group completed all 4 intervention sessions (78% in the video feedback group, 72% in the verbal feedback group, and 61% in the experiential feedback group). Noncompliance was not statistically different between the groups and was considered evenly distributed (P = .14). There were no adverse events for any participant during the study.
Discussion
Providing video feedback in addition to verbal feedback significantly improved online awareness in TBI participants compared with verbal feedback and experiential feedback alone. Contrary to the second hypothesis, the provision of verbal feedback did not improve online awareness over and above experiential feedback. The video feedback group also had significantly greater improvement in intellectual awareness compared with the other groups receiving verbal and experiential feedback.
Although the video feedback was specifically designed to reduce the number of errors made in a functional task (ie, online awareness), it had broader effects on self-knowledge about personal abilities (ie, improvement in intellectual awareness). The multimodal combination of video and verbal feedback involved self-observation of performance and therapist-guided reflection on errors and strategies. In addition to facilitating error monitoring, this approach may help participants process the meaning of their experiences and modify self-knowledge and beliefs about their abilities. Agnew and Morris describe this process as updating one’s “personal database.” 26 In contrast, participants in the verbal feedback or experiential feedback groups did not demonstrate the same degree of improvement in both online and intellectual awareness.
An early model of self-awareness (ie, the Pyramid Model) suggested that levels of self-awareness are hierarchical and that individuals need to have adequate intellectual awareness before improvements in online awareness can occur. 27 In contrast, the experiential feedback group had a significant improvement in online awareness without an improvement in intellectual awareness over the intervention period. This finding concurs with previous single-case-study findings that improvements in online awareness can occur independent of changes in intellectual awareness. 3
Previous research has identified a positive association between self-awareness and emotional distress. 28 We found that improvement in self-awareness after receiving the video feedback was not accompanied by a significant increase in emotional distress. Providing a supportive and nonconfrontational therapeutic context may buffer the potentially distressing effects of learning about postinjury impairments. Providing feedback that emphasizes personal strengths (eg, spontaneous strategy use) and gives suggestions on how to compensate for difficulties may alleviate emotional distress. 29
Clinical Implications
The video feedback group had a 70.7% mean reduction in errors over the intervention period, which far exceeded the 30% reduction in errors identified a priori to be clinically meaningful. 12 The verbal feedback group had a 20.7% mean reduction, and the experiential feedback group had a 37.6% mean reduction in errors over the intervention period. Therapists should therefore consider incorporating video feedback for people with TBI and impaired self-awareness. Clinical judgment is required to consider likely contraindications to this approach (eg, severe psychiatric disturbance).
The experiential feedback group demonstrated significant improvement in online awareness over the intervention period. This shows that task-based metacognitive training can be effective as a therapy technique. Previous single-case studies, which also incorporated the use of the “pause, prompt, praise” technique and posttask self-reflection, showed similar gains in online awareness.3,8 These results led Ownsworth et al 8 to recommend metacognitive skills training as an alternative approach to continual instruction or therapist correction of errors, which may increase reliance on external support and limit independence. However, unlike these previous studies, the participants in the experiential feedback group in our study did not receive additional feedback from the therapist after performing the task. Yet their gains in online awareness were not significantly different from the group that received verbal feedback after the task. Therefore, the on-task metacognitive training may be the relevant component facilitating the development of independent self-regulatory behavior. Further research is recommended to explain this finding.
Limitations of the Study
Assessors were blinded on the primary outcome measure, and participants were blinded throughout the study. A limitation was the lack of blinded assessment on the measure of intellectual awareness (AQ discrepancy score). The participant’s occupational therapist, who provided the feedback intervention, also provided the proxy ratings for the AQ. An alternative approach would have been to collect the proxy ratings from significant others. However, this approach can be problematic, particularly in inpatient settings where significant others may have little opportunity to observe the individual’s functional impairments. 30
The groups differed with respect to Glasgow Coma Scale, PTA, Functional Independence Measure, and 2 neuropsychological test scores. The experiential feedback group appeared to be more severely injured, which may be seen as a limitation to the study. However, this failure of randomization is alleviated to some degree because these variables were not associated with the primary outcome.
The participants were recruited from both community and inpatient settings, resulting in a varied sample in terms of length of time postinjury. Although time since injury was not significantly related to online awareness, it is unclear whether TBI chronicity may affect response to intervention. Further research is needed to determine whether the efficacy of a feedback intervention varies as a function of time postinjury.
Further Research
Video feedback interventions are more time intensive because the therapist is not only involved in the functional task but also in the viewing of the video and guiding participant reflection. Further research is needed to determine how many sessions are required to facilitate gains in online and intellectual awareness and to determine the optimal timing of this feedback. The impact of improved self-awareness on long-term outcomes, such as independence, role participation, and self-concept is also unclear. We did not assess changes in participation and self-concept and recommend that this be addressed in future research.
In conclusion, this RCT demonstrated that the combination of video plus verbal feedback is most effective in enhancing both online and intellectual awareness compared with other feedback methods. A reassuring finding is that the intervention was not accompanied by a significant increase in emotional distress.
Footnotes
Authors’ Note
Trial registration: Australian New Zealand Clinical Trials Registry identifier number ACTRN12609000378224.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This trial has been partially funded through a grant awarded by the Occupational Therapists Board of Queensland. At the time of this study, JS was supported by a fellowship provided by the Wenkart Foundation, Australia, and NAL was supported by the Balnaves Fellowship provided by the Cerebral Palsy Alliance, Australia. There are no industrial links or affiliations of authors involved in this study, and there are no conflicts of interest.