Abstract
PURPOSE:
Students with TBI enter college with strategies that they have used prior to being injured yet often without knowing which ones will be effective in helping them to be successful. The purpose here is to describe how semi-structured interviews were used to identify self-regulated learning strategies, to demonstrate the utility and reliability of coding self-regulated learning strategies, and to provide examples of student-centered goals derived from survey and interview responses.
METHODS:
College students completed the College Survey for Students with Brain Injury (CSS-BI) and were interviewed before and after coaching support that focused on teaching self-regulated learning. Responses to interview questions about strategies were coded using a modified version of Zimmerman and Martinez-Pons’s (1986) schema. Coders also rated strategies for specificity.
RESULTS:
Strategies were reliably coded into 16 categories of self-regulation. Inter and intra-reliability were strong. Four of the five students reported using a larger variety of self-regulation strategies and strategies that were more specific after coaching support.
DISCUSSION AND CONCLUSIONS:
It is possible to reliably code self-regulation learning strategies reported by college students with TBI. These measures have potential as functional ‘outcomes’ for students who are transitioning to college. Interview responses can be used to collaboratively create student-centered goals.
Keywords
Introduction
Students with traumatic brain injury (TBI) may be transitioning back to college after being injured or may be going to college for the first time after having been injured. Additionally, some students are returning to college to retool after their injury if they are unable to return to their prior job. Regardless, going to college is often an achievable goal for individuals recovering from TBI. Although 91% of institutions of higher education serve students with TBI (Raue & Lewis, 2011), this includes only those students who self-report a history of TBI and register themselves as students with disability through campus offices of accessibility. Krause and Richards (2014) found that 16.9% of undergraduate students reported a history of TBI.
Cognitive deficits after TBI have been well-characterized to include deficits in memory, attention, and executive function though it is only recently that researchers have begun to understand the impact these deficits have on college performance (Willmott, Ponsford, Downing, & Carty, 2014). For professionals who are working to support these students as they transition to college, a starting place is to identify the strategies students have used in the past to manage their academic performance; discussing what is and is not being used effectively allows the professional and student to work towards student-centered goals and to determine which strategies could assist in reaching these goals.
Self-regulated learning in college students
Self-regulated learning has long been recognized as a feature of the most successful college students (e.g., Nandagopal & Ericsson, 2012). Research into self-regulated learning has revealed that learners across a variety of contexts and academic subjects perform better when using self-regulation strategies effectively (Tang, 2015; Tuckman & Kennedy, 2011). For example, high achieving students use more strategies and a greater variety of strategies, engage in more self-evaluation, and use more elaborative encoding strategies than low achieving students (Ruban & Reis, 2006). Research has also focused on how self-regulation supports students with disabilities in meeting academic goals. For example, university students with learning disabilities experienced a greater positive effect from the use of self-regulated learning strategies than students without learning disabilities (Ruban, McCoach, McGuire, & Reis, 2003).
Kennedy and Coelho (2005) developed a model of self-regulation (SR) for clinical populations that describes a cyclical process of monitoring, goal-setting, and strategy deployment that can be explicitly taught. This model has since then been expanded (Kennedy, 2017). Individuals with TBI may experience difficulty at any one of these steps: assessing their skills accurately, identifying achievable goals, selecting a strategy, using a strategy, and making adjustments in the goal or strategy based on experience and feedback. Intervention that explicitly teaches this self-regulation process provides students with a skill set that could be used across contexts and academic courses.
Kennedy and Krause (2011) first described this as a self-regulatory coaching program designed to support two students returning to college after TBI. The intervention support entailed explicitly teaching students to regulate their learning, develop and employ appropriate strategies, and monitor progress. To evaluate strategy use, students completed The College Survey for Students with Brain Injury (CSS-BI; Kennedy & Krause, 2009) and were interviewed about strategies based on their responses to the academic challenges they endorsed. Results revealed that both students developed more detailed strategies to cope with academic challenges over the course of intervention. However, the type of strategies students reported was not examined.
Zimmerman and Martinez-Pons (1986, 1988), developed a scale called the Self-Regulated Learning Interview Schedule (SRLIS) that identifies self-regulation learning strategies. The 14 categories of self-regulated learning include self-evaluation, goal-setting, rehearsing and memorizing, and self-consequences. In high school students, the frequency of reported use of such strategies predicted performance in achievement scores beyond demographic measures alone. In university students, use of this scale indicated that students who were required to take remedial courses upon college admission reported using fewer self-regulation strategies as compared to students who were not remediating (Ley & Young, 1998).
The purpose of the current study was to: 1) describe how semi-structured interview responses can be used to identify the type and specificity of self-regulation strategies in college students with TBI, and; 2) to demonstrate how interview responses lead to student-centered goals that are created collaboratively between students and professionals.
Methods
Participants
All research activities were conducted under the oversight of the University of Minnesota Institutional Review Board. Five adults with history of TBI (2 males/3 females) were all participating in a pilot study on coaching support while transitioning into or back to college (Kennedy & Krause, 2011; Kennedy, 2017). Students ranged from 19 to 24 years in age and included a college freshman, a sophomore, two juniors, and a senior. All were traditional college students. None had other neurological disorders, learning disabilities, or attention deficit disorder. With the exception of Student 1, all others were transitioning to college after being injured. Student 1 had sustained multiple concussions during high school. She was a Junior in college and had recently experienced exacerbated post-concussion symptoms. A counselor referred her. Student 2 had sustained multiple concussions in high school and college, with the most recent one being one month prior to being referred for support by the office of accessibility. He was a junior in college. Student 3 had sustained a moderate TBI one year prior to returning to college as a senior. The office of accessibility referred her. Student 4 had sustained a severe TBI as a graduating high school senior. She attempted college one year later but switched colleges where she could receive more direct support. She was a freshman and referred by a community-based clinician. Student 5 was a freshman who sustained a severe TBI two years prior while in high school. He was referred by the Department of Vocational Rehabilitation.
Procedures
Students participated in a study in which the goal was to develop a collaborative coaching approach that explicitly taught them to engage in self-regulated learning with functional outcome measures. This approach is student-focused. It consists of the clinician ‘coach’ and student collaboratively identifying student strengths and weakness, creating student-centered goals and plans for reaching their goals, and explicitly instructing students in self-regulation strategies in three areas: self-learning and -studying, self-management and -organization, and self-advocacy. Each student’s coaching program was highly individualized, based on their specific needs, abilities and disabilities. Students addressed both proximal (immediate) and distal (long term) goals. For a further discussion of this approach, readers are referred to Kennedy and Krause (2011) and Kennedy (2017).
As a part of the evaluation process, student participants completed a semi-structured interview based on the academic challenges section of the CSS-BI (Kennedy & Krause, 2009) before and after two semesters of coaching support.1
Some supplemental testing was conducted initially. Reports providing detailed test results (e.g., neuropsychological test results) were available to coaches. These test results were not considered outcomes and dare not reported here
After completing the survey, participants were interviewed and asked to elaborate on their responses to each statement, explain why they selected the rating, and identify strategies they used to help with the problem. Interviews were conducted by the last author and followed a script that listed the academic item. The interviewer stated, “I see that you agree with ‘I have to study more than I used to.’ Can you tell me more about that?” Participants described why that rating was selected and examples of when it would occur. The interviewer subsequently asked, “What do you do when that happens? Do you use any strategies to manage that?”
Two research assistants (RAs) transcribed the semi-structured interviews verbatim. From the transcripts, RAs identified 321 strategies reported by students across both time points and randomized the strategies for coding. Two other RAs coded the strategies by strategy category and strategy specificity.
To code self-regulation strategy categories, we used a modified version of the SRLIS self-regulated learning strategy coding schema developed by Zimmerman and Martinez-Pons (1986) and further described by Purdie and Hattie (1996). Minor adaptations were necessary to tailor the categories to college students with TBI, since it was originally used with high school students. Adaptations were as follows: The category “Seeking Assistance from Adults” was expanded to include Disability Service personnel rather than examples such as parents. To account for the use of technology, categories were expanded accordingly. For example, “Environmental Structuring” included setting alarms or reminders using a smartphone or other technology to assist in completing tasks. The ‘uncodable’ category was replaced with an ‘other’ category (as done by Purdie and Hattie, 1996) which encompassed strategies that were non-strategic, non-academic, or strategies that were initiated by someone else. These included strategies of willpower (e.g., “I force myself to remember.”), cheating (e.g., “I got it from a friend.”), coping (e.g., “I just let it go.”), and self-advocacy (e.g., “No, you don’t understand; I have to sleep that much.”). A category of “Too Vague” was added for descriptions that did not contain enough information to be coded (e.g. “Yeah, my notes.”) and were not included in the “Other” category.
The adapted version consisted of 16 categories of self-regulated learning strategies appropriate for college students with TBI, of which 11 were original, 1 was an addition (“Too Vague”), and 4 were adapted or substantially revised. Detailed explanations and examples for each strategy category were provided to the research assistants to improve coder reliability. The complete rubric is available in Appendix A.
Strategies were also coded for specificity using a 4-point scale ranging from very vague to very specific (Kennedy & Krause, 2011). Very specific and specific strategies were those that could be easily replicated using only the information given, while vague or very vague strategies did not include enough detail for the coder to know how or in what context(s) they were being used. Ratings were collapsed into two categories: specific and vague. Disagreements in specificity coding were discussed between the raters, and consensus for specificity on 99.7% of strategies was reached. One strategy for which they could not achieve consensus was eliminated from the analysis. Appendix B is the rubric used for rating strategy specificity.
Results
Self-regulation strategy descriptions
Reliability of coding the type of self-regulation strategy was evaluated first by examining the agreement between the two trained coders of randomly selecting 20% of the strategies for comparison. Cohen’s kappa was acceptable for this subset of items (K = 0.66). Because the second rater (RA2) had less experience with the coding schema (i.e, only coded 20% of the strategies), we hypothesized that inter-rater reliability might increase with additional practice for RA2. Thus, we instructed RA2 to code the remaining 80% of the strategies. As predicted, agreement for the entire set of strategies increased substantially (K = 0.83).
As a preliminary measure of clinical change over time within a single rater, intra-rater reliability was measured by having each coder re-code 10% of randomly selected strategies. Intra-rater reliability varied between the two coders, revealing acceptable substantial to strong agreement (KRA1 = 0.90, KRA2 = 0.68). Thus, this finding suggests that this modified coding schema for self-regulation strategies could be an effective and valid measure of change in self-regulated learning strategies over time.
Having established the reliability and utility of the coding of self-regulation strategies, a number of measures were used to document the amount and type of strategy used by college students with TBI. These included: 1) the total number of self-regulation strategies reported, 2) the number of different self-regulation strategy categories reported and, 3) the percentage of strategies coded as “specific.”
Table 1 includes these measures for students at baseline and after two semesters of coaching support. Across students, the total number of self-reported strategies increased after two semesters of coaching for three of the five students; the other two reported fewer strategies than at baseline. The average number of different self-regulation strategy categories increased after two semesters of coaching, with the exception of one student. Likewise, the percentage of strategies that were coded as specific increased after two semesters of coaching for 4 of the 5 students, from an average of 36.10% to 48.84%.
Number and types of strategies reported by students with TBI from the CSS-BI survey and interview
Number and types of strategies reported by students with TBI from the CSS-BI survey and interview
The primary purpose of this study was to describe how interview responses can be used to reliably identify self-regulated learning strategies, and to demonstrate how these responses can be used to help create self-regulation goals with students who are transitioning to college. To this end, students with TBI were interviewed based on their responses from the CSS-BI (Kennedy & Krause, 2009) and specifically asked about strategy usage.
Self-regulation strategy measures
The adapted SRLIS coding schema of 16 types of self-regulation strategies appears to be a clinically feasible and useful tool, as demonstrated by its adequate inter- and intra-rater reliability. Our results indicated that student-reported strategies fell into a range of strategy categories using this coding schema. Furthermore, strategies could be reliably classified into strategy types both within and across coders. This aligns with high coding reliability found by previous studies that investigated the reliability of the original schema with typical learners (e.g., Zimmerman & Martinez-Pons, 1986; Purdie & Hattie, 1996). Nevertheless, we did observe that adequate experience using the coding schema is required to achieve optimal reliability; intra-rater reliability increased between our two RAs after the second RA coded the entire set of samples (as opposed to the initial random sample of 20%).
While counting the number of strategies that students self-report may be easy to do, it may not be the most informative measure when trying to determine what students are using; it does not provide information about the quality of the strategies. In other words, the use of ineffective strategies included in a total count may not serve students as well as fewer effective strategies. Furthermore, the number of strategies that students use depends heavily on their academic coursework at the time of each interview (i.e., taking easier or harder courses).
Measuring the variability of strategy categories avoids some of the limitations mentioned above. This measure also demonstrates better validity with regard to the goal of the intervention program which was to help students develop a wide array of specific strategies for their learning “toolbox.” The students could then select and employ the most appropriate strategy given the unique demands of the situation. Thus, having a wider variety of strategy categories could help them achieve their goals, given the ongoing change in academic situations, environments, and coursework. Using our adapted version of Zimmerman and Martinez-Pons’s (1986) coding schema, the number of strategy categories increased for four of the five students. Prior to coaching support, students reported using an average of 8.2 different strategy categories to manage their academic challenges. After coaching support, they reported using an average of 10.4 categories and strategies fell into 15 of the 16 self-regulation categories; only “Reviewing Tests” was not represented in the data set.
Specificity of strategies, a measure of strategy detail presented here replicates Kennedy and Krause’s (2011) findings that strategy specificity can be coded with high agreement using a consensus procedure. Across the five students, 42.6% of the strategies were coded as “specific” with a range of 0%–53.8% prior to intervention. Post-intervention, 50.4% of strategies were coded as “specific,” with a range of 26.3% to 67.9%. This change was particularly dramatic for Student 4, who initially did not describe any specific strategies. Student 1 was again the only student who did not demonstrate improvement in strategy specificity. The reason for her lack of change is not clear, but may be related to the fact that she described strategies with relatively high specificity at baseline. It should be noted that she was the only student not ‘transitioning’ to college, and had lived with her disability for some time.
Using strategy information to create student-centered goals
The findings suggest that strategy type (as coded using the adapted SRLIS schema) and specificity may be useful in launching discussions that lead to goal formation. By following the survey and interview procedures, professionals can engage students in discussions in order to determine which strategies were already being used effectively or ineffectively, and can explore the use of more effective strategies with students. The following are examples of how interviews were used to identify needs and strategies currently in place, so that coaches could collaboratively create goals with students. Progress toward these goals could also then be re-assessed using the semi-structured interview format around strategy usage. These are not meant to be all-inclusive, but only serve of as examples of self-regulation, student-centered goals, areas of need that emerged from interview responses, and how progress toward those goals could be monitored.
For Student 1, the number of strategies, self-regulation categories, and percent of specific strategies did not change after coaching support. This was a student who had lived with her symptoms for years and had already developed some useful strategies for managing her post-concussion symptoms. In this sense, she was not truly transitioning to college, but had recently experienced an exacerbation of her symptoms. What emerged from her interview responses was the need to develop and use more effective time-management strategies in order to minimize her fatigue and anxiety. By the end of coaching support, she stated, “Everything’s on my planner. I’m planning more breaks…and I’m a lot more realistic about time frames.” Related to this, she was better at identifying triggers of headaches, stating “…I could probably look at my schedule and say if I’ll have a headache that day.”
Student 2’s self-report of self-regulation strategies and the specificity of his strategies had increased by the end of coaching support. At baseline he reported vague learning strategies, e.g., “just tried to study a little more.” His goal was to develop and implement effective learning strategies to recall large amounts of rote information. After coaching support, he reported “I’ve started underlining key information and picking out what I need and then always having examples of both ways to do the problem. It helps me feel more comfortable coming into it I guess.”
Student 3’s self-regulation strategies and the specificity of her strategies increased after coaching support. Before intervention, many of the strategies Student 3 described were not effective, such as, “…I use small note pads like post-its for my scheduling, but it doesn’t really work when you have to memorize a lot of terms and equations.” Her goal was to develop and implement effective strategies to increase recall of information presented in class (e.g., using the SmartPen to record and review). The interview also revealed her lack of self-advocacy when she did not understand instructions or the textbook. This was developed into a goal, so that after coaching support she reported, “I would email the TA asking, what do you mean or can you explain what this question is really asking? Or I would write what I understood from reading it and then ask, is this correct?”
Student 4 initially reported using 11 non-specific strategies that revealed a non-strategic reliance on willpower to learn academic content (e.g. “Study a lot. Make sure I know everything. Everything.”) at baseline. One of her goals was to develop and implement a combination of effective internal and external memory strategies to increase efficiency of learning/recall. After coaching support, she used more strategies and a wider range of strategy types. Furthermore, 26.3% of these strategies were coded as specific.
Student 5 described 22 strategies across 9 self-regulation categories initially. He described managing his assignments vaguely as, “Take notes and keep up with it.” He and his coach developed a goal to learn to use a smart phone to support recalling past events and remembering/planning to turn in assignments and complete tasks. After intervention, he described 28 strategies across 13 self-regulation categories and 67.9% were specific. He reported, “I write down all the assignments in my planner. I have an assignment sheet on one side and a regular calendar sheet for the day on the other side. So, then I write down all my assignments on one side after every single class both semesters this year. Usually I take as many notes as I can now for each assignment in my planner. So, like let’s say it’s for reading class, I’d write down that it was in my reading binder in the to-do section, because my folder has sections for homework not complete and homework completed.” Following intervention, Student 5 could describe specifics of strategies and systems in place for him to manage both when his assignments were due and details about what was required on the assignments.
Conclusions
This preliminary study provides clinicians and researchers with an evaluation schema and numerical measures that have potential to become outcome measures for those who are receiving support as they transition to college after TBI. Having established the inter- and intra-rater reliability of identifying self-regulated learning strategies, professionals now have reliable measures with which to describe change. Although these measures represent only one aspect of change, the face validity of specific strategies and a wider variety of strategies implies that students are better equipped to manage their own transitions across the varying demands of courses, teaching styles of different professors, and course loads within semesters. Future work will benefit from determining the clinical feasibility and utility of coding strategy use in real time using the CSS-BI interview procedures. Finally, these measures may be useful as benchmarks that indicate general changes in self-regulation strategies across time, although students and professionals will continue to benefit from rich discussions about what is working and what is not. Indeed, regular feedback, particularly self-generated feedback during real-life experiences such as college, can result in more realistic self-evaluation, with the potential for students to become their own expert in the strategies that are most useful (Tate et al., 2014).
Conflict of interest
None to report.
Footnotes
Appendices
As best as possible, code the behavior not your interpretation of their purpose in using the strategy. For example, if the student is taking notes as a means of relieving anxiety versus keeping track of information imparted in class, both behaviors include taking notes and therefore would fall under “Keeping Records and Monitoring.”
Rate each strategy from “very vague” to “very specific.” For example, if you wanted to save money, a vague strategy would be “to spend less,” while a specific strategy would be “to use coupons from the newspaper for groceries.” One way to think about specificity is to imagine whether or not you would be able to carry out the strategy exactly the way she or he described it.
When coding for specificity, be sure to think about the strategy for which the item was coded. For example, if the student mentions taking notes, how specifically does the student describe the note-taking strategy? Similarly, if the student mentions planning and scheduling time for studying, consider how specifically the execution of the strategy is described.
Acknowledgments
This work was supported in part by the University of Minnesota – Twin Cities, Department of Speech-Language-Hearing Sciences and a grant with the Minnesota Department of Employment and Economic Development (DEED). A special acknowledgment is extended to Lauren O’Connor and Alissa Allison who served as research assistant coders.