Abstract
Traumatic brain injury (TBI) is common among legally-impacted adults and has also been linked to negative outcomes throughout the criminal legal system. Despite this, relatively limited TBI research has focused on or even included legally-impacted adults. Existing literature in this population has used heterogeneous TBI definitions, populations, and measures when studying outcomes. This study therefore investigates the feasibility and acceptability of telephone-administered TBI common data elements (CDEs) and the Rehabilitation Needs Survey (RNS) in 85 legally-impacted community-dwelling adults. Regarding feasibility, completion rates across measures were high (88–100%), and noncompletion was most commonly due to participants declining to continue. Regarding acceptability, data collectors were able to administer and code most measures with relative ease. Reported difficulties related to measures requiring detailed data collector training to administer and code, such as the Brain Injury Screening Questionnaire (BISQ) and Brief Test of Adult Cognition by Telephone, or challenges inherent to self-report tests in general. In addition, data collectors recommended adding specific questions to the BISQ to query head injuries experienced during or as a result of their exposure to the criminal legal system. Overall, results support the use of telephone-administered TBI CDEs and the RNS in legally-impacted adults, and underscore the need for culturally-responsive training and technical assistance for TBI researchers engaging with this population.
Introduction
Adults living with traumatic brain injury (TBI) and who have been exposed to the criminal legal system (hereinafter “legally-impacted adults”) represent a large and underserved group. The estimated lifetime prevalence of TBI in legally-impacted adults is 45.8% (95% confidence interval 37.8 − 54.1%), 1 compared to 12.1% in the general population. 2 With over 5 million people incarcerated in prisons and jails or under supervision in parole or probation in the United States, 3 these estimates suggest there are several million people living with TBI and related needs in the United States criminal legal system alone.
Unfortunately, relatively limited systematic TBI research has centered on legally-impacted adults. This is likely due in part to the various ethical, clinical, and practical challenges to conducting clinical research with legally-impacted adults,4,5 the limited evidence base directly supporting the use of common TBI research methods in this population,6,7 and unfamiliarity among TBI researchers with population-relevant research considerations. This is an important gap, particularly given the high rates of TBI in legally-impacted adults, 1 and preliminary evidence that TBI is a risk factor for poorer outcomes throughout the criminal legal system, including arrest,8,9 conviction, 10 and incarceration; 11 behavioral health difficulties and higher service utilization in correctional institutions;12–17 and lower treatment seeking/completion, 16 and higher recidivism,13,18,19 following release into the community. Furthermore, this preliminary research has been heterogeneous in the methods used to measure and define TBI itself, relevant covariates, and related outcomes. As a result, fundamental questions about the characteristics and needs of legally-impacted adults with TBI remain unanswered.
This study takes significant steps toward addressing these concerns by investigating the feasibility (completion rates) and acceptability (data collector perceptions) 20 of several telephone-administered screening tools selected from the TBI common data elements (CDEs) 21 and the Rehabilitation Needs Survey (RNS) 22 with legally-impacted adults living in the community. The TBI CDEs were developed to standardize the definition and collection of TBI data and support scientific rigor, data sharing, and collaboration in TBI research. 21 The recently developed RNS is a validated TBI needs assessment tool 22 that addresses an important gap in the current TBI CDEs of direct relevance to historically underserved populations.
To the best of our knowledge, this study is the first systematic investigation of the selected TBI CDEs and RNS in legally-impacted adults. Establishing their feasibility and acceptability will advance TBI research with this population by promoting the use of these rigorous methods in future studies centering on legally-impacted adults, and by supporting the inclusion of legally-impacted adults in ongoing systematic research already using these methods. This, in turn, will enable meaningful comparisons of findings across studies. This study also appears to be the first to leverage the benefits of telephone administration of rigorous TBI screening tools (especially TBI history/severity 23 and cognition24,25) in a sample of community-dwelling adults with various exposures to the United States criminal legal system. These benefits include accessibility, scalability, and efficient use of resources, which may be particularly useful in research with legally-impacted adults given the large size and geographic diversity of this group, and the restricted access and resource limitations inherent to those incarcerated in prisons or jails. If successful, this study will support the implementation of these rigorous TBI screening methods to better understand and address the needs of this large, diverse, and underserved TBI group.
Materials and Methods
Participants
Participants were adults served by The Fortune Society (Fortune) in New York City, NY, USA. Founded in 1967, Fortune (fortunesociety.org) serves thousands of individuals each year using a holistic one-stop model to provide a broad range of diversion programs (i.e., diverting people from the criminal legal system and into appropriate services), re-entry programs (i.e., providing services to people re-entering society from the criminal legal system), medical and behavioral health care, education and employment training, and other services. Exclusion criteria were being under age 18, self-reported physical or sensory difficulties that interfered with using a telephone, self-reported or observed difficulties with English that precluded them from engaging with study procedures, or a self-reported or observed medical status (e.g., psychiatric crisis, intoxication, and acute withdrawal) that precluded them from safely engaging in research.
Procedures
This study was reviewed and approved by the City University of New York Human Research Protection Program (#2021-0337). Data for this study were collected between November 2021 and May 2022. Participants were recruited in person during their regular engagement in services at Fortune, during which they were introduced to the study and completed a screening of exclusion criteria. Eligible participants then completed scheduling, informed consent, data collection, debriefing, and compensation by telephone. All procedures were completed in English. Data collection was estimated to take 90 min and participants were compensated $75 for their participation.
Measures were generally administered and scored according to their standardized instructions (see Measures section for relevant modifications). Participants were asked to sit in a quiet environment and not use writing or other recording materials. Data collectors were instructed to provide passive encouragement and discontinue any study procedure as they deemed necessary. Potentially invalid results were flagged by data collectors for further review by the first author (C.L.). Measure completion (i.e., administered in full), validity (i.e., results valid or invalid), and reasons for noncompletion (i.e., attempted but not completed in full) were recorded (see Table 1 for a summary of codes and relevant definitions). These data will be used to assess feasibility, defined here as the extent to which the materials and methods can be successfully used or carried out with consumers (i.e., participants) in this setting. 20
Test Completion Codes
Used for internal purposes only; reviewed and updated to relevant code in the final dataset.
Symptom severity that precluded measurement completion.
Data collection being discontinued due to complications with participants’ communication or behavior.
Technological interruptions, scheduling conflicts, or similar logistical reasons without re-engaging in data collection.
One data collector (C.L.) earned a doctoral degree in clinical psychology and had extensive training and experience in clinical interviewing and cognitive testing, including with legally-impacted adults. Two data collectors (including K.T.) were recruited from doctoral programs in clinical psychology, had previously earned master’s degrees in forensic psychology or criminal justice (respectively), and had prior training and experience with clinical interviewing and cognitive testing, including legally-impacted adults. The remaining four data collectors were recruited from master’s programs in forensic psychology or forensic mental health counseling and had training in clinical interviewing and cognitive testing (including with legally-impacted adults) but little to no experience.
The first author (C.L.) oversaw the initial training of all other data collectors, including asynchronous review of study materials and supporting literature followed by live or recorded trainings addressing study procedures and measures. The first and second authors (C.L. and K.T.) oversaw further practical training of all other data collectors, including the review, coding, and entry of three recorded data collection sessions (one mock and two actual) followed by individual feedback and retrainings as needed.
Several quality assurance methods were instituted to ensure methodological fidelity. These included de-identified recording of data collection sessions (with consent) to allow for independent reviews as needed and double-coding whenever possible, 100% double entry, and periodic retrainings as needed. In addition, data dictionaries and standard operating procedures were developed to guide data collection, entry, and cleaning, and were further supported by regular teleconferences, an email listserv, and external technical support as needed.
At the end of their involvement, all data collectors (N = 7) were invited to complete an online survey developed for this study to investigate their experiences. These data will be used to assess acceptability, defined here as the perceptions among implementation stakeholders (i.e., data collectors) that the materials and methods were agreeable, palatable, or satisfactory. 20
Measures
All standardized measures and related subtests are summarized and described in Table 2. Several standardized and other measures requiring further description are presented below.
Summary and Description of Study Measures/Subtests
TBI Common Data Element.
Included Core and Supplemental items.
Developed specifically for the TBI Common Data Elements.
EF, executive functioning; FITBIR, Federal Interagency Traumatic Brain Injury Research; PTSD, Post-Traumatic Stress Disorder; RAVLT, Rey Auditory Verbal Learning Test; TBI, traumatic brain injury; WAIS-III, Weschler Adult Intelligence Scale–Third Edition.
Brief Test of Adult Cognition by Telephone
The Brief Test of Adult Cognition by Telephone (BTACT) is a telephone-administered cognitive test battery developed from several common neuropsychological tests encompassing a wide range of cognitive domains and ability levels. 24 The subtests included in this study are summarized in Table 2. The Stop and Go Switch Task was not included as it is not required for composite scores similar to those used in prior BTACT validation research in TBI populations, 25 and due to potential confounds when measuring reaction time by telephone. 26
Brain Injury Screening Questionnaire
The Brain Injury Screening Questionnaire (BISQ) is a structured screening tool for the lifetime history and severity of head trauma exposures and TBI. 23 Participants are asked whether they have ever experienced a blow to the head in 19 specific situations, and an open-ended item to report head trauma in situations not queried. Supplemental items query whether they have ever been hospitalized or treated in an emergency department for eight specific medical events (plus any other self-reported event) to document alternative contributors to clinical symptoms. Each endorsed item is followed by additional questioning to determine the number of blows to the head sustained, the presence and length of any periods of loss of consciousness and alterations of consciousness, and the date of each occurrence. The version used in this study also included three modules to assess head injury exposure due to sports, military activity, and intimate partner violence.
Post-Traumatic Stress Disorder Checklist—Civilian Version
The Post-Traumatic Stress Disorder (PTSD) Checklist—Civilian Version (PCL-C) is a standardized self-report rating scale for PTSD). 27 Participants were presented with 17 items corresponding to key PTSD symptoms and asked how much they have been bothered by each item in the last month on a scale of 1 (not at all) to 5 (extremely).
Satisfaction with Life Scale
The Satisfaction with Life Scale (SWLS) is a standardized self-report rating scale for overall satisfaction with life. 28 Participants are presented with five statements and asked to indicate their agreement on a scale of 1 (strongly disagree) to 7 (strongly agree).
Rehabilitation Needs Survey
The Rehabilitation Needs Survey (RNS) is a standardized self-report measure to identify, quantify, and characterize rehabilitation service needs following TBI. 22 Participants are presented with 21 rehabilitation needs common following TBI, and asked (a) whether they have ever received help in that area and (b) whether they need or want (more) help. Each item is then categorized as a met need, partially met need, unmet need, or not a need. RNS administration was modified by adapting Item 21 to “Adjusting to life in the community” (vs. “Transitioning from military to civilian life”) to approximate the underlying need in this sample.
Data Collectors Survey
A survey was developed to investigate data collectors’ experiences with this study (available in Supplementary Data S1). Core items surveyed perceptions about the administration and coding of 44 aspects of the standard assessment protocol, on a scale of 1 (extremely difficult) to 5 (extremely easy). Supplemental items surveyed perceptions about each measure/subtest and other study procedures.
Statistical analysis
Descriptive analyses were performed for sample demographics and completion rates. Responses to the data collector survey were tabulated and grouped by study measure/subtest. Quantitative analyses were performed using IBM SPSS 29. Qualitative responses to the data collector survey were reviewed by the first author (C.L.) using an open coding framework, which is an exploratory and inductive method designed to find meaningful results close to the data itself (i.e., versus existing theories).
Results
Eighty-five individuals participated in this study. Sample demographics and criminal legal exposures are summarized in Table 3.
Summary of Sample Demographics and Criminal Legal Exposures
Demographic variable labels and response options defined by Federal Interagency Traumatic Brain Injury Research (FITBIR) Demographics Form. Percentages are rounded to the nearest integer.
Included responses commonly defined as racial backgrounds (e.g., Black, Native American, White; n = 9), specific countries (n = 7), and other self-defined ethnic backgrounds (n = 1).
IQR, interquartile range.
Feasibility
Rates of completion and reasons for noncompletion are shown in Tables 4 and 5. Overall, a large majority of participants completed and provided valid results for all TBI CDEs (88–100%) and the RNS (92%). Invalid results were not observed for any measure (0%). Non-completion was rare, and mostly due to participants declining to continue following an individual measure or item (1–5% on 6/10 TBI CDEs; 0% on RNS) followed by interactive complexity, interrupted administration, or other logistical reasons (all 1% on 2/10 TBI CDEs; 0% on RNS). Non-completion due to cognitive/neurological or other medical reasons was not observed in this sample (0%). Non-attempt was also rare (5–9% on 7/10 TBI CDEs; 8% on RNS) and due exclusively to data collection having been discontinued prior to the attempt rather than anything related to these measures themselves. Other codes were also rare (1–2% on 5/10 TBI CDEs). Similar results were found for the BTACT subtests in terms of completion (95–100% complete and valid; 1% complete but invalid on 4/6 subtests), non-completion (1% of the participants declined to continue on 3/6 subtests), and non-attempt (2% limited only to the last subtest administered).
Sample Completion Codes
N = 85. Measures organized by standardized order of administration. Completion codes without responses were excluded, specifically: test administered in full—results invalid; test attempted but not completed, due to cognitive/neurological reasons; and test attempted but not completed, due to other medical reasons (all n = 0, 0%). Cells without responses (0%) were left blank for clarity. Percentages are rounded to the nearest integer.
Core items.
Supplemental items.
BTACT, Brief Test of Adult Cognition by Telephone; BISQ, Brain Injury Screening Questionnaire; FITBIR, Federal Interagency Traumatic Brain Injury Research; PCL-C, Post-Traumatic Stress Disorder (PTSD) Checklist—Civilian Version; RNS, Rehabilitation Needs Survey; SWLS, Satisfaction with Life Scale; TBI, traumatic brain injury.
Brief Test of Adult Cognition by Telephone Subtest Completion Codes
N = 85. Subtests are organized by standardized order of administration. Completion codes without responses were excluded, specifically: Test attempted but not completed, due to cognitive/neurological reasons, other medical reasons, interactive complexity, interrupted administration, or other logistical reasons; and other (all n = 0, 0%). Cells without responses (0%) were left blank for clarity. Percentages are rounded to the nearest integer.
Acceptability
Five of seven data collectors participated in the acceptability survey (71% response rate). This included data collectors who had experience with the study measures and procedures via direct administration and coding (n = 2), review and data cleaning (n = 1), or both (n = 2 including the first and second authors). A summary of data collectors’ perceptions regarding the acceptability of the measures and subtests is shown in Table 6. Overall, little to no difficulties were reported for nearly all of the TBI CDEs or the RNS. Areas of relative difficulty noted by data collectors are summarized below.
Data Collectors’ Perceptions Regarding Acceptability of Measures and Subtests
N = 5. Measures/subtests are organized by standardized order of administration. Ratings are summarized by median (Med) and mode, based on a scale of 1 (extremely difficult) to 5 (extremely easy).
Percent of data collectors reporting difficulty, based on combined ratings of 1 (extremely difficult) or 2 (somewhat difficult).
BTACT, Brief Test of Adult Cognition by Telephone; BISQ, Brain Injury Screening Questionnaire; FITBIR, Federal Interagency Traumatic Brain Injury Research; HIE, head injury exposure module; IPV, intimate partner violence; PCL-C, Post-Traumatic Stress Disorder (PTSD) Checklist—Civilian Version; RNS, Rehabilitation Needs Survey; SWLS, Satisfaction with Life Scale; TBI, traumatic brain injury.
On the BISQ, data collectors reported that some participants struggled to recall dates of injury, number of blows to the head sustained, and details required to ascertain the duration of both loss and alteration of consciousness. Qualitatively, data collectors reported that some participants found the BISQ somewhat long and tedious, especially those with extensive TBI history. In addition, data collectors reported that some participants had particular difficulty reporting event-specific information for events that occurred in childhood. Last, all data collectors suggested that it would be useful to add item(s) specific to hits to the head directly related to their exposure to the criminal legal system when working with this population to better capture TBI history, severity, and related context.
Some administration challenges were reported for three BTACT subtests, especially Backward Counting but also Category Fluency and Number Series. In particular, data collectors reported difficulty balancing the dual demands of timing and recording responses and therefore found audio recording helpful to ensure the accuracy of scoring.
Last, some administration challenges were reported for the Behavioral History measure. Qualitatively, data collectors reported some participants had difficulty quantifying substance use (especially alcohol) on this measure, and reported that item wording was occasionally confusing to participants and may have led to inaccurate responses (e.g., licit versus illicit use of medications). Data collectors noted that the inclusion of marijuana in illicit drugs posed challenges in the context of jurisdictional trends toward decriminalization.
Discussion
This article presented a protocol for implementing telephone-administered TBI CDEs and the RNS in 85 community-dwelling adults with prior and ongoing exposures to the United States criminal legal system, and provided evidence for their feasibility and acceptability in this important TBI group. Participants were generally able to complete and provide valid results for the selected measures. In addition, data collectors were generally able to administer and code the selected measures with relative ease, with difficulties mostly reported for measures requiring relatively higher levels of training to administer and code (e.g., BISQ and BTACT) or self-report of details people commonly find challenging (e.g., dates of head injuries sustained in groups with high levels of violence exposure, and quantitative amounts of alcohol consumption). Last, data collectors unanimously recommended that the BISQ (and similar TBI screening measures) include items specific to head injuries experienced during or as a result of their exposures to the criminal legal system to more fully capture TBI history in this population.
TBI history/severity and cognitive functioning are foundational to contemporary TBI research, and any heightened efforts related to training, technical assistance, and quality assurance related to such measures (e.g., the BISQ and BTACT) are clearly worthwhile. This appears particularly warranted with data collection conducted by groups less specialized in TBI research, such as the university-based data collectors involved in this study. Conversely, the high completion rates and general ease reported by data collectors in this study underscores the benefits of their training and experiences working with legally-impacted adults, including effective clinical interviewing in the context of the structure and practical realities of the criminal legal system, this population’s historical marginalization and oppression in and by clinical research, and related barriers in personal, social, and structural determinants of health. This finding highlights the need for culturally responsive training and technical assistance for TBI research groups engaging with legally-impacted adults.
There are several limitations that merit further discussion. The sample of legally-impacted adults was relatively small (N = 85), mostly male (80%), recruited from a single community site with a relatively high level of resources, and all resided in a large urban area in the northeastern United States, all of which may limit the generalizability of the presented feasibility evidence. Relatedly, all participants resided in the community and used their personal phones to complete the research protocol, limiting our ability to understand the feasibility and acceptability of these methods and measures with legally-impacted adults in correctional institutions. In addition, all data collectors were drawn from relevant graduate training programs, and none had prior research experience in TBI or with the selected measures. While this underscores the benefits and successful implementation of these standardized research methods in real-world settings, it may also limit the generalizability of the acceptability evidence (and possibly the feasibility evidence) to those better versed in TBI research but with less experience with this novel TBI population. Relatedly, several procedures were implemented to increase the quality of the data, though these may not be reasonably efficient (e.g., audio recording, double-coding, 100% double entry) or accessible in all settings (e.g., technical assistance from researchers specializing in people with TBI and legally-impacted adults, respectively). Last, the sample of data collectors providing their perceptions of the study was relatively small (n = 5), which may limit the interpretation of the related quantitative data and the generalizability of the acceptability evidence in general.
Feasibility and acceptability are foundational to the use of rigorous TBI research methods in the large and underserved population of legally-impacted adults. Further cross-validation of the telephone-administered TBI CDEs and recently developed standardized screen of related rehabilitation needs described here is still needed in this important TBI group. This is particularly true for the BTACT, as prior evidence suggests the cultural identities and clinical characteristics of legally-involved adults are poorly captured in the development and standardization of measures of cognition.7,29 Nonetheless, the selected TBI CDEs and RNS hold great promise to better understand the characteristics and needs of individuals with TBI in the United States criminal legal system, and the future development of related intervention strategies that are likely to have significant impacts on public health and safety.
Footnotes
Acknowledgments
The authors would like to acknowledge Marcus Aviles, Raquel Borg, Zoë Feygin, Janna Jensen, Mahathi Kosuri, Agnieszka Nogalska-Parra, John Riley, Nicolas Rodriguez, and Sandra Secerio from the City University of New York for their contributions to data collection, entry, and quality assurance, and Colleen Gleason from The Fortune Society for their contribution to participant recruitment. The authors would also like to acknowledge Annell Ovalles, Ariel Pruyser, Madison Seger, Dr. Raj Kumar, and Dr. Lisa Spielman from the Brain Injury Research Center at Mouth Sinai for providing training, materials, and technical assistance for this study.
Authors’ Contributions
C.L.: Conceptualization, data curation, formal analysis, funding acquisition, investigation, methodology, project administration, supervision, visualization, writing—original draft, writing—review, and editing; K.T.: Investigation, project administration, and writing—review and editing. M.L.: Conceptualization, investigation, methodology, writing—review and editing. R.D.: supervision and writing—review and editing. K.D.O.C.: conceptualization, methodology, resources, supervision, and writing —review and editing.
Author Disclosure Statement
No competing financial interests exist.
Funding Information
This work was funded by a grant from the Professional Staff Congress of the City University of New York Research Award Program (ENHC-51-78), Seed Funding and Open-Access Publication Funding from the Office for the Advancement of Research of John Jay College of Criminal Justice, and startup funding from the John Jay College of Criminal Justice. C.L. was supported by an extramural award from the National Institutes of Health Loan Repayment Program sponsored by the National Institute of Neurological Disorders and Stroke while completing this work.
Abbreviations Used
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
