Technology and concussion: A scoping review

Abstract

Background

Technology for concussion identification and management is rapidly expanding across the continuum of care. Although many technologies offer a range of services around concussion, there is an absence of a non-commercial online location for medical providers to access regarding the functionality of the various technologies used in concussion identification and management.

Objective

The purpose of this review is to present research findings on technology for concussion identification and management.

Methods

Searches for eligible studies were conducted using the PubMed, EMBASE, and Scopus databases with specific search criteria. Through a stepwise process, full-text articles were selected for inclusion if they described clinically useful electronic technologies (i.e. electronics able to be used in standard clinical environments including telehealth) by healthcare providers or end users (i.e. parents or athletes).

Results

A total of 29 articles were included in this review and described technology used to measure symptoms (3), neurocognitive performance (7), the visual system (4), and balance or dual task performance (18). Within the results, various technologies demonstrated increased utility for concussion identification, often detecting subtle deficits not possible with current low-tech clinical methods, differentiating those with concussion from those without concussion, with strong reliability and validity.

Conclusion

Innovative technologies included in this review demonstrate enhanced ability to identify and manage symptoms of concussion, neurocognitive deficits, visual deficits, and balance and dual-task deficits.

Introduction

Among civilians, athletes, and service members, concussion has gained attention as a concerning diagnosis with potential to impact health status for a prolonged period of time. Interestingly, although millions of research dollars have been spent to better understand concussion, there still seem to be more questions than answers. In fact, to date there is still no “gold standard” for diagnosis of concussion. ¹ No routinely available biomarker or imaging technique has been identified to definitively diagnose concussion. Rather, concussion emerges as a clinical determination by coupling a reported traumatic biomechanical event with common signs and symptoms.^1,2 Because concussion can potentially impact multiple brain functions, but does not always impact any one brain function, it can be a difficult diagnosis to make. Similarly, because concussion presents differently across the populations affected, treatment (or management) does not include one formal or typical path.^2,3 This makes concussion one of the most difficult diagnoses to identify and treat, especially when associated with sports and military environments. ¹

Technology use for concussion identification and management is a burgeoning avenue for research and development. As an adjunct to traditional clinical tests and patient report, technological advances could serve to increase the accuracy of diagnosis and aid clinical decision making by providing enhanced objective measurements. Technology may also assist clinicians in managing concussion symptoms by treating resultant functional disturbances. From the use of phone-based applications for education, ⁴ to computerized testing for baseline assessment (pre-concussion), to eye movement video tracking and analysis for diagnosis of concussion, the use of technology is rapidly expanding across the continuum of concussion care. ⁵

At this time, although there are many technologies and start-up companies offering a range of services around concussion, ⁶ there is an absence of a non-commercial online location for medical providers to access regarding the functionality of the various technologies used in concussion management. This makes it difficult for medical providers to navigate between biased claims versus research evidence supporting the best technologies to enhance concussion care. Therefore, the purpose of this review is to present research findings on technology for concussion identification and management. This will include a description of the technology, its purpose, how it is used, and the research findings. Following the description of current technologies, future directions and opportunities for technological development in this area will be discussed.

Methods

This project was completed as a scoping review of existing literature. A systematic search was used to identify relevant publications within the aim of this project.

Data sources and search strategy

Searches for eligible studies were conducted using the PubMed, EMBASE, and Scopus databases. The search was developed with assistance from a medical reference librarian at the University of Mississippi Medical Center using the search string ((“electronic technology” or “wearable electronic devices” or technology or telemedicine)) AND (“brain concussion” or concussion).

Selection criteria

Studies included in the review met the following inclusion criteria: a full text article written in English describing clinically useful electronic technologies (i.e. electronics able to be used in standard clinical environments including telehealth) by healthcare providers (i.e. physicians, neuropsychologists, physical therapists, athletic trainers, etc.) or end users (i.e. parents or athletes) in affiliation with a concussion diagnosis (for our purposes, concussion was synonymous with mild traumatic brain injury or mTBI). Diagnostic accuracy, psychometrics (validity, reliability, etc.), user interaction/feelings/etc., and intervention designs were also included in this review in order to search for all evidence on electronic technology and its usefulness in diagnosing and managing/treating concussion. Studies were excluded if the technology consisted of on-field sensors to detect head acceleration or impacts only (where diagnostic accuracy for concussion is not reported), or if data was obtained via instrumented lab-based biometric measurements that would not be available for a typical clinician to use in an environment where concussions are typically diagnosed, managed, or treated.

Study selection and management

A stepwise process was used to select articles for inclusion. Title and abstracts were screened by two authors, with discrepancies decided by a third author. Two authors reviewed the full text articles with discrepancies decided by a third author. After selection, the included studies were grouped into categories based on the type of neurological function being studied or described. Furthermore, within each category, studies were classified based on what type of evidence was presented on the usefulness of the technology for a given purpose.

Data extraction

Data extraction was performed by all authors, with articles separated into categories. A second author checked the accuracy of the original author’s data extraction. The extracted items included the description of participants, the technology, the technology’s purpose, and how the technology is used. The results of each study were extracted based on the study type and purpose.

Results

Study selection

From the database search, 818 titles were identified and screened (after removal of duplicates); 201 abstracts were screened, 27 of which were abstracts only from conference proceedings and were discarded, leaving 75 full text articles. After review, a total of 29 articles were included in this review.

Study characteristics

Upon review of the 29 included studies, the following primary categories of function emerged: balance/dual task; neurocognition; symptoms; and vision. Studies that described results in more than one category were included in both categories. Within these categories, the studies presented evidence on specific technology and its usefulness for differentiation of concussion from healthy; normative values of performance as measured by the technology; reliability and validity of the technology; and usefulness of the technology in the management of a concussion. Results for each category are presented in Tables 1 to 4.

Table 1.

Technology for balance and dual-task performance.

Study Author and year	Concussion Participants	Healthy Participants	Study Purpose	Technology Description	Outcomes
Alberts et al. 22	n = 181 with a concussion; divided into two groups: n = 92 (typical) recovered within 3 weeks and n = 89 (delayed) had symptoms >3 weeks post-injury	None	Determine responsiveness of the Cleveland Clinic Concussion Application (C3 app) in detecting postural sway while performing the BESS. Used to monitor recovery, identify those at risk for prolonged recovery, and expedite referral to specialty services.	The C3 App functions on a tablet device and assesses a range of potentially disrupted brain functions including balance, static and dynamic vision, reaction time, cognitive processing, executive function and set switching	Between group: significant differences in two of the six BESS stances quantifying postural sway: double limb stance on foam (p = 0.02); tandem stance on foam (p = 0.04); but BESS errors were not significantly different between the two groups (p = 0.26).
Brett et al. 7	None	n = 3763 (9-21 y/o) who completed the Sway Balance Sports Protocol	Designed to provide normative data for the Sway Balance System for children and adolescents	Sway Balance System measures postural sway using mobile devices built in accelerometer. The balance portion of the Sway includes 5 stances under different test parameters. The app measures balance for 10 seconds in each stance	Sway balance scores differed by age (p < 0.001); with older age groups performing better than younger. Sway balance scores also differed by sex (p < 0.001); with girls performing better than boys. Normative data was then constructed by age and sex.
Chang et al. 10	None	n = 30 healthy young adults (18-35 y/o).	Using a force plate to determine if the Wii Balance Board could provide a low-cost improvement to the reliability/validity of concussion balance assessments compared to the BESS.	Participants perform balance postures in six conditions. Each trial would last 20 seconds, and data collected. Wii balance board measurements were compared to force plate data.	The Wii Balance Board demonstrated a reliability of r = 0.88 and validity of r = 0.99 compared to the BESS reliability of r = 0.61-0.78 and validity of r = 0.10-0.52.
Engelson et al. 14	None	n = 20 Division 1 women soccer players.	Determine the validity of the OptoGait motion analysis technology compared to the BESS and Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT).	The OptoGait is a floor-based photocell system that assesses movement using LED modules and cameras. The participants completed eight trials of marching in place with: marching in place, head looking left, head looking right, head looking up, blindfolded, earplugs, blindfolded and earplugs, cognitive challenge.	Out of the 64 OptoGait mean scores, 30 were significantly related to the double-leg foam BESS scores; 17 of the 64 OptoGait standard deviations scores were statistically significant related to balance tandem hard surface BESS scores. There were no statistically significant correlations between the OptoGait and ImPACT.
Furman et al. 17	n = 43 high school students	n = 27 high school controls	Determine the ability to detect differences in postural sway between concussed and non-concussed patients using the National Institute of Health’s Balance Accelerometer Measure (BAM) and comparison of the BAM with the BESS in their ability to detect differences in postural sway between adolescents with sports concussion and same age controls.	The BAM consists of six standing balance conditions. Each condition is performed for 45 seconds with three attempts. The BESS was scored by experts using video recordings.	There were no significant differences between males and females or between concussion groups across all conditions using the BAM. The BAM did not demonstrate significant differences with identification of acute concussions versus healthy participants. The BESS, especially tandem stance, adequately discriminated between healthy and concussed adolescents. Total BESS score of ≥21 discriminated between the acute concussion group and healthy adolescents; 60% sensitivity; 82% specificity.
Goble et al. 8	None	n = 10,045 male and female athletes that performed the BBT postural sway.	Objective was to determine normative data for BTrackS™ Balance Test Concussion- Measurement Tool (BBT).	BBT measures postural sway through the use of a balance plate connected to a computer system. Each participant performs three 20 second trials, cued through auditory tones	There were significant differences sex (p < 0.001); age (p < 0.001); sex and age interaction (p= .04). Females had better BBT results than males (p=.05). Percentile rankings were stratified by sex and age for understanding performance.
Houston et al. 9	None	n = 440 physically active college-aged adults	Determine normative balance data for the BESS using the Tuscan MobileMat based on sex, concussion history, and competition level.	The BTrackS™ MobileMat uses a portable sensor platform and computer application to score balance errors.	Males performed significantly better than females for the BESS single-limb foam stance (p = 0.032). No differences were detected between males and females for other stances (p = 0.067). There were no significant differences for concussion history (p = 0578) or level of competitive sport (p = 0.524).
King et al.	n = 52 college athletes with an acute concussion (1-4 days)	n = 76	Determine which portions of the mBESS and postural sway measurements obtained with inertial sensors could discriminate athletes with acute concussions.	mBESS tests were completed having participants stand with eyes closed and as still as possible for 30 seconds in 3 stance positions. Postural sway was recorded with an Opal inertial sensor placed at vertebral segment L5 using a belt.	Inertial sensor measures were significant in discriminating acutely concussed vs healthy (p<.001); clinical mBESS measures were not able to discriminate acutely concussed vs healthy (p = 0.06). Double leg stance best discriminated those with acute concussion vs healthy population (p<.001) using the inertial sensor.
Linder et al. 23	None	N = 50 (18-24 yrs.)	Develop a dual-task model to evaluate cognitive-motor function using mobile device technology	Inertial sensors (gyroscope and accelerometer) within the iPad to measure postural sway.	Based on changes in postural sway and cognitive performance under dual-task conditions, the intermediate level (60 stimuli/min) in tandem stance-eyes closed was the only dual-task condition in which the Cleveland Clinic-Postural Stability Index (CC-PSI) was significantly improved compared to the single task condition (p = 0.02, corrected) revealing improved postural stability under dual-task conditions without decrease in cognitive performance. The optimal dual-task model is 60 stimuli/min delivered in tandem stance with eyes closed.
Napoli et al. 12	None	n = 15	Determine the effectiveness of the Automated Assessment of Postural Stability (AAPS) system to determine balance errors while performing the BESS.	The AAPS system uses a motion capture sensor combined with software to track movements during the BESS testing. The system utilizes this tracking software to detect errors in balance.	There were no significant differences between balance scoring on the AAPS and human scoring (p > 0.05). Agreement between human scoring of BESS and AAPS scoring ranged from 87.9 to 99.8% The AAPS was able to measure multiple balance errors with a high resolution, which provided additional consistency and accuracy.
Parrington et al.	N = 5; individuals with mTBI; mean age = 36.4 years	N = 5 healthy adults; mean age = 24.0 years	To determine the validity of wearable inertial measurement units (IMU) to estimate head and trunk ROM and peak rotational velocity during head movements made in standing and walking in comparison with 3-D motion capture data. To be used to characterize head and trunk movements during standard vestibular rehab.	Wearable IMUs attached to sternum and forehead. Each IMU contained a tri-axial accelerometer(± 6 g), a gyroscope (± 2000˚/s), and a magnetometer (± 6 gauss) that measured at a sampling frequency of 128 Hz. Moveo application was used to record the IMU data using wireless synchronization. 3-D motion capture-(120 Hz, Raptor-E, Motion Analysis Co., processed using Cortex v6.2.3 (Motion Analysis Co.). Motion capture was synchronized with the IMU recording using an APDM synchronization box.	Between group comparison (IMUs and motion capture data) Head: ROM and peak rotational velocity ICC (A,1) >0.9; RMSE across all conditions were low for head ROM (1.29-3.78) but increased in the walking Left/Right (20.26) and tandem walking Left/Right (17.04) conditions for peak rotational head velocity. % Error (<5%) across all conditions Trunk: Left/Right direction stronger-(ROM ICC(A,1) > 0.9) and (peak rotational velocity ICC(A,1) >0.9); RMSE and % Error stronger. Up/Down direction- ROM ICC (A,1)=0.580 to 0.907 and peak rotational velocity ICC(A,1)=0.436 to 0.787. RMSE and % Error= reduced strength also. Excellent validity for the IMU system for head movements in both Left/Right and Up/Down directions; trunk movements in Left/Right directions; moderate to excellent trunk ROM validity in Up/Down directions; moderate to good validity in trunk peak rotational velocity in Up/Down; poor validity in trunk during tandem walking in which it was poor.
Ràbago and Wilken20	31 y/o male who sustained an mTBI and had delayed recovery post-concussion	Comparison made to normative lab values.	To describe an mTBI specific clinical assessment and rehab intervention administered in a virtual reality environment	Virtual reality environment CAREN system (Motek, Amsterdam, The Netherlands). Composed of a 7-m diameter dome with 300-degrees of visual field projection and a 6-degrees-of-freedom motion platform. Subject stood and walked in the center of the platform on an instrumented treadmill. Operator manipulated the environment. Kinematic data were collected at 60 Hz using a 24-camera infrared motion capture system (Vi-con Motion Systems, Oxford, United Kingdom) within the dome.	After 6 treatments using the CAREN system: Static and Dynamic Balance: increased all single leg stance (SLS) times (max of 30 on all but “eyes closed on foam” Normalized composite Y-balance score: increased by 25.5% to 114.7 ± 6.0 (normal controls: 93.5 ± 7.0 above minimum detectable change of 3.8%) 4 square step test (dynamic agility): 21.9% reduction in time to 5.6 ± 0.6 sec. above minimum detectable change of 7.8% Executive function and dual-task tests: Symbol recognition reaction time during DLS with no optic flow; decreased by 231 ms to 360 ± 44 ms (w/in normal range for non-concussed) Symbol recognition reaction time during DLS with optic flow-decreased by 151 ms; with gait-decreased by 77 ms. Symbol matching reaction time during DLS -with no optic flow; decreased by 125 ms to 873 ± 174 ms.; with optic flow-decreased by 133 ms; with gait; decreased 126 ms. (not within nonconcussed range) Stroop- accuracy rate across all conditions- improved from 8.5 ± 10% for combined average of 98 ± 4%.
Rhine et al. 19	n = 13 children presenting to the Emergency Department with one or more: Loss of Consciousness <30 mins, amnesia, any alteration of mental state at the time of the injury.	n = 26 children presenting to the ED with minor complaints and otherwise healthy.	Determine if the Wii Balance Board is a feasible tool to assess balance in the pediatric population between concussed and control.	The Wii Balance Board was connected with specialized software. The Wii Balance Board measurement was zeroed before each measurement and the patient was measured using four balance stances. Single limb trials were 30 seconds and 60 seconds for double limb trials. Total of three attempts were performed for each stance.	Performance on double limb eyes open test was significantly different between the two groups (p = 0.04), differentiating concussed from controls. No other stances yielded significantly different measurements between groups.
Salisbury et al. 13	None	n = 42 (23.8 y/o)	Determine the validity of using smart-glass balance accelerometer measurements compared to waist-based accelerometer measurements using the National Institute of Health’s Balance Accelerometer Measure (BAM) protocol.	Glass Explorer Edition (Glass) are smart glasses that use an inertial measurement tool, accelerometer, gyroscope and magnetometer to determine quantitative balance assessment data. Subjects performed the BAM protocol with the glasses donned and a smartphone donned to the waist. Data was collected from each device and compared using Android based software.	Balance measurements taken using the smart-glasses accelerometer were highly correlated with the waist-based accelerometer (=.85). Test-Retest reliability measurements with the smart-glasses: ICC = 0.85, 95% CI 0.81-0.88). Correlation between devices improved further on normalized path length magnitude using all 3 axes: (p=.90); ICC=.87, 95% CI 0.083-0.90.
Slobounov et al. 20	n = 36 with concussion; 12 were assessed prior to 30 days post- concussion and were presented in this study. All 12 subjects were clinically asymptomatic at day 30 of testing.	None	Determine postural control in subjects pre- and post-concussion using center of pressure and virtual time-to-contact (VTC) measures through a force-plate.	VTC was used to estimate postural stability in standing. A force platform was used with integrative software to collect balance data. Subjects performed three trials per each condition: standing eyes open, standing eyes closed, and a dynamic task. Prior to testing, subjects performed individual “maximum functional stability boundaries” in each plane.	There were no significant differences for any of the centers of pressure measurements of postural control prior to and 30 days post-concussion (p > 0.05). There were no significant differences in VTC shape, distribution or nominal values prior to and 30 days post- concussion (p > 0.05). There were significant differences in VTC at the deflection points and mode at 30 days post-injury.
Teel et al. 15	n = 27 Division I varsity college athletes who were tested 7-10 days after concussion	n = 94 Division I varsity college athletes	Determine the validity of Virtual Reality (VR) balance assessment in clinical concussion care. To establish sensitivity and specificity of a VR balance module that can be used to identify unresolved balance deficits post- concussion.	An inertial tracking technology was attached to the subject’s head to detect position. A projector screen was used to display VR animations. The subject viewed these projections through liquid crystal shutter glasses and then performed various balance testing with different virtually projected scenarios.	VR balance module was found to have a high sensitivity (85.7% ) and specificity (87.8% )
Wright et al. 16	n = 11 (average age of 20.4 y/o)	n = 56 (average age of 22.6 y/o)	Determine the validity of Virtual Environment TBI Screen (VETS) compared to the Neurocom Sensory Organization Test (SOT).	VETS is a virtual reality technology that simulates movement in a visual environment while measuring center of pressure data through the use of a Wii Balance Board. Participants perform testing in a dark room and must remain upright under 6 different conditions.	VETS demonstrated a sensitivity of 81.8% and specificity of 85.7% with an overall accuracy of 91.0% accuracy through all six VETS posture conditions. SOT demonstrated 84.8% accuracy for all six stance conditions. With a 54.5% sensitivity and a specificity of 83.6%. The forward stepwise condition was found to be significant (p = 0.015).
Zhu et al. 11	None	n = 30 (22 and 31 y/o)	Determine the validity and reliability using two Kinect sensors to score errors committed during BESS and mBESS balance testing in healthy individuals.	The Microsoft Kinect V2 sensors measure 3 D coordinates of skeletons and joints. Nine of the twenty-five Kinect measureable joints were used in conjunction with algorithms to measure balance. These measurements were used with custom software to detect balance errors to score the BESS.	Validity of Kinect V2 sensors when compared to a manual rater: BESS: validity-r = 0.93 (p < 0.05); reliability-ICC 0.81 (p < 0.001) mBESS: validity-r = 0.92 (p < 0.05); reliability-ICC 0.84 (p < 0.001)

Balance Accelerometer Measure (BAM); Balance Error Scoring System (BESS); Inertial Measurement Units (IMU); Intra-class Correlation Coefficients (ICC); modified BESS (mBESS); Range of Motion (ROM); Root Mean Square Error (RMSE); Sensory Organization Test (SOT); Virtual Environment TBI Screen (VETS); Virtual Reality (VR); virtual time-to-contact (VTC); years old (y/o);

Results of individual studies

Balance and dual-task

Eighteen studies examined how concussion adversely influences balance and the performance of dual tasks (Table 1). Three of these studies established normative values for technologies that measure postural sway and balance errors after a suspected concussion. These technologies included the Sway Balance System for children and adolescents, ⁶ BTracts Balance Test Concussion Measurement Tool, ⁷ and the Tekscan Mobile Mat. ⁸

Seven studies investigated reliability and/or validity for new technologies used for identification or management of concussion. Five of these studies investigated specific technology compared to the Balance Error Scoring System (BESS) or modified BESS (mBESS). The Wii Balance Board displayed good reliability and excellent validity. ⁹ Kinect V2 sensors exhibited good test-retest reliability and excellent validity with total BESS score. The total mBESS scores were also compared in this study and found to exhibit good reliability and excellent validity. ¹⁰ The effectiveness of the Automated Assessment of Postural Stability (AAPS) system was evaluated for sideline identification of concussion. Good correlation was found with the AAPS algorithm and human scoring of the BESS. The AAPS also measured multiple balance errors with high resolution providing additional consistency and accuracy. ¹¹ Another study examined the validity of the Glass Explorer Edition smart glasses compared to the well-described smartphone waist accelerometer using the National Institute of Health’s Balance Accelerometer Measure (BAM). The smart glasses were highly correlated with the waist accelerometer, and reliability was high. When all three axes were used in the normalized path magnitude, the correlation improved even further. ¹² Validation of the OptoGait motion analysis technology was attempted by comparing it to the BESS and the Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT). Thirty of the 64 OptoGait mean scores were correlated with the double-leg foam BESS scores. Seventeen of the 64 OptoGait standard deviation scores were correlated with the balance tandem hard surface BESS scores, but none of the OptoGait variables correlated with the ImPACT. ¹³

Further, in the diagnosis of concussion, two studies described new virtual reality (VR) technologies. One of these used an inertial tracking technology attached to the head to detect position while watching VR animations through liquid crystal shutter glasses and then performed balance testing with different virtually projected scenarios. VR balance module was found to have high sensitivity and specificity. ¹⁴ Another study examined the Virtual Environment TBI scan (VETS) which uses a Wii Balance Board, compared to the Neurocom Sensory Organization Test (SOT). The VETS was found to have high sensitivity and specificity. The overall accuracy through all six VETS conditions was 91.0%. ¹⁵

Three additional studies described sensor technology in the measurement of balance for diagnosis of concussion. One of these compared the ability of the BESS to the ability of the BAM to detect the differences in postural sway between concussed and healthy individuals. The BAM was not able to delineate between them. However, the BESS, especially tandem stance, was able to discriminate between adolescents with acute concussion and healthy individuals with a low sensitivity and high specificity. ¹⁶ Another study examined which portions of the mBESS and postural sway measurements obtained with an Opal inertial sensor placed at L5 could detect acute concussion. The measures taken with the inertial sensor successfully discriminated acutely concussed individuals, and the double leg stance portion of the mBESS best discriminated those with acute concussion. Clinical mBESS measures were not able to discriminate between acutely concussed and healthy adolescents. ¹⁷ The third study examined the ability of the Wii Balance Board to assess balance in the pediatric population to identify concussion. Only the double limb eyes open test successfully identified concussion. ¹⁸

Three studies were identified which described technology in the management of concussion. One study examined postural sway in standing using the virtual time-to-contact (VTC) measures through a force-plate. The only significant differences seen from prior to and 30 days post-concussion in the VTC were at the deflection points and mode at 30-days post injury. ¹⁹ A case study examined treatment of an individual who had delayed concussion recovery using the virtual Computer Assisted Rehabilitation Environment (CAREN) system. At the conclusion of 6 treatments, the patient improved in postural and gait balance with almost complete resolution of concussion symptoms. Improvement was also seen in the executive function and dual tasks tests. ²⁰ The third study determined the responsiveness of the C3 App in detecting postural sway while performing the BESS protocol by comparing scores of concussed individuals who had typical recovery with those who had prolonged recovery. There were significant differences in two of the six BESS stances as measured by the C3 App including double limb stance on foam and tandem stance on foam. ²¹

Finally, one additional study explored the use of a wearable inertial measurement unit (IMU) to quantify head movements in response to vestibular rehabilitation. Here, it was found that this technolology had good agreement with a room-based motion capture system. ²² One study developed a dual-task model to evaluate cognitive-motor function using the inertial sensors in an iPad to measure postural sway. The optimal dual-task model was found to be the model where 60 stimuli/min was delivered in tandem stance with eyes closed. ²³

Neurocognition

Seven studies described the use of technology for identifying, managing, or developing normative data in relation to neurocognitive symptoms commonly seen with concussion (Table 2). A variety of technology was used including the C3 App,^21,24 the Display Enhanced Testing for Concussions mTBI(DETECT System), ²⁵ the CogSport, ²⁶ the ImPACT, ²⁷ the Cambridge Neuropsychological Test Automated Battery (CANTAB), ²⁸ and the BrainScope Ahead 300. ²⁹ The C3 App was successful at detecting significant differences for simple reaction time, choice reaction time, and Trail Making Test B when comparing individuals who recovered within three weeks of a concussion to individuals who remained symptomatic after three weeks. ²¹ When the C3 App was combined with a dual task activity, a significant difference was found in simple reaction time, choice reaction time, gait speed, and stride length when comparing concussed and healthy individuals. ²³ One study evaluated the effectiveness of a portable neuropsychological assessment device with a distraction-free, immersive visual and auditory environment, the DETECT system, in eliminating external audio and visual stimuli in healthy college students. No significant difference was found in performance in a noisy versus quiet environment. ²⁴

Table 2.

Technology for neurocognitive performance.

Study Author and year	Concussion Participants	Healthy Participants	Study Purpose	Technology Description	Outcomes
Alberts et al. 22	n = 181 with a confirmed concussion; n = 92 (typical) recovered within 3 weeks of the injury and n = 89 (delayed) continued to have symptoms >3 weeks post-injury.	None	Determine responsiveness of the Cleveland Clinic Concussion Application (C3 App) in detecting change in neurological function post-concussion	C3 App functions on a tablet device and assesses a range of potentially disturbed brain functions including balance, static and dynamic vision, reaction time, cognitive processing, executive function and set switching	There were significant differences for simple reaction time (p<.001), choice reaction time (p<.001), and Trail Making Test B (p=.01). Other measurements within the C3 App did not demonstrate a difference between groups.
Barker et al. 25	None	n = 42 healthy collegiate students (no history of concussion previous 6 months)	To evaluate a portable system for rapid screening of mTBI by creating an immersive environment to eliminate external audiovisual stimuli. It adapted (reduced length) existing neuropsychological tests and integrated them into the software.	Display Enhanced Testing for concussions and mTBI (DETECT System). Head- mount to present the software: Sony Glasstron LDf-100BE personal LCD monitor. Light reduction: custom made cardstock visor. Noise reduction: Bose Aviation Headset X active noise reduction headphones. Yes/No buttons for responses: 2 Jelly Bean switches by AbleNet, Inc. connected to X-keys USB Switch Interface. Computer: Dell Latitude C840 notebook computer. Neuropsychological tests included the N-Back; reaction time; and selective reminding.	Neuropsychological tests were performed with the DETECT in noisy and quiet environments. No significant differences found in performance on any of the tests by environment (p-values between 0.13 and 0.93); therefore, the DETECT system was successful in blocking external audio and visual stimuli. Post-task questionnaire: Subject’s self-reported experience with the DETECT did not correlate with overall test performance or response time. Clear instructions of the testing process were shown to be vital for success.
Collie et al. 26	None	n = 300 (240 elite Australian Football League players and 60 volunteers)	Describe normative performance scores of the CogSport test battery and the stability of the test over time through test-retest (TRT). The CogSport could be used to monitor recovery from concussion and aid in return to play decisions.	CogSport test includes 8 tasks delivered via a standard personal or laptop computer. The tests assess speed and accuracy of the following domains: psychomotor function, decision making, working memory and learning	Normative scores for Psychomotor function: Speed (ms) 294.8 (55.6) Accuracy of 98.9 (3.2) Decision making: Speed (ms) 631.9 (139.4) Accuracy of 88.4 (10.56) Working Memory: Speed (ms) 655.7 (160.9) Accuracy of 92.5 (7.1) Learning: Speed (ms) 1121.7(241.6) Accuracy of 79.3(11.4). TRT: speed: 1 hour and 1 week in each domain was strong (ICC = 0.69-0.90); accuracy: 1 hour and 1 week in each domain was weak to moderate (ICC = -0.08-0.51)
Howell et al. 24	n = 19 NCAA Division I collegiate athletes within 5 days of concussion	n = 41 NCAA Division I collegiate athletes at baseline	Examine difference in performance on instrumented dual-task gait (walking while completion of a cognitive activity) and tablet-based neurocognitive outcomes in participants with acute concussion compared to participants without concussion	Neurocognitive: Tablet-based neurocognitive test (C3 App) including the Trails A and Trails B test, Processing Speed Task, Simple and Choice Reaction Time test Dual-Task: Inertial measurement sensors attached to the lumbar spine and dorsum of each foot, sampling at 128 HZ synchronized to a laptop computer. Gait characteristics measured by Mobility Lab software	Trails A, Trails B, and Processing Speed, cadence and cognitive task accuracy were not significantly different between groups. Simple RT (p < 0.001), Choice RT (p = 0.015), gait speed (p = 0.010), stride length (p = 0.024) were significantly different between healthy and concussed.
Iverson et al. 27	n = 72 athletes within 21 days of a sports-related concussion	None	Examine construct validity of the Immediate Post-concussion Assessment and Cognitive Testing (ImPACT) by correlating each component to the Symbol Digit Modalities Test (SDMT).	Version 2.0 of ImPACT is a computer administered neuropsychological test which includes 6 test modules measuring different aspects of cognitive function: attention, verbal and visual memory, reaction time, and processing speed	Correlation measures to the SDMT: Verbal Memory (0.46) Visual Memory (0.37) Processing Speed (0.70) Reaction time (-0.60)
Lunter et al. 28	n = 36 with a mTBI who presented to the ED who presented within 24 hours after injury	n = 20 with other trauma (not mTBI) within 24 hours after injury n = 20 healthy controls	Evaluate the feasibility and role for computerized cognitive testing in the emergency department as a component of a head injury examination; compare performance in those with head injury, other trauma and healthy controls.	Testing was completed on an Apple iPad 2 with three tasks from the Cambridge Neuropsychological Test Automated Battery (CANTAB) and a simple reaction time test. Tests were self-administered using a voiceover for instructions. The tests included Paired Associates Learning (PAL) of visual episodic memory and new learning; Spatial Working Memory (SWM) for executive functioning and working memory; and Match to Sample (MTS) for attention, speed, and accuracy on visual search.	mTBI vs. controls found significant differences in: RT (p = 0.011), MTS (ms) (p = 0.018), SWM errors (p < 0.001) and SWM strategy (p < 0.001). No difference between mTBI and controls on PAL and MTS (% correct) mTBI vs. trauma found significant differences in RT (p = 0.015), MTS (ms) (p = 0.0313), SWM strategy (p = 0.021). No difference between mTBI and controls on PAL, MTS (% correct) and SWM errors
Vincent et al. 29	None	n = 624 (mean age 47.3 years)	Establish normative data for Android version of Automated Neuropsychological Assessment Metrics (ANAM) match to sample (M2S) and procedural reaction time (PRO) tests; determine consistency of for same tests on a computer.	BrainScope Ahead 300 is a device with multiple test modules. These can be selected by user and represent multiple dimensions. For this study, cognitive performance was scored using procedural reaction time and matching to sample tests to measure reaction time, processing speed, working memory, attention and visual-spatial processing	There was no difference in throughout score normative data when administered on a personal computer (M2S p = 0.65 and PRO p = 0.09). Normative scores: Mean M2S = 33.8(12.2) and mean PRO = 99.8(23.8)

Two studies enrolled only healthy individuals in order to determine normative data. The CogSport’s measure of speed and accuracy for neurocognitive functions was found to be highly reliable in serial testing and ideal for repeated testing of cognition. ²⁶ Normative data for two cognitive performances on the Automated Neuropsychological Assessment Metrics (ANAM) was obtained via The BrainScope Ahead 300 android mobile device with multiple test modules to aid in assessment of mTBI, and then compared to data collected on a personal computer platform. There were no significant differences found between the two, indicating that this mobile device can be used clinically with comparable reliability as a personal computer. ²⁹

Lunter et al., ²⁷ compared individuals with a mTIB to a healthy control group and a trauma group (not mTBI) in order to determine the feasibility and role for computerized cognitive testing (CANTAB) in the emergency department as a component of head injury examination. When comparing mTBI to the control group, there was a significant difference in reaction time, match to sample, spatial working memory errors, and spatial working memory strategies. There was also a significant difference in reaction time, match to sample, and spatial working memory strategy when comparing mTBI to the other trauma group. No significant differences were found on paired associates learning or match to sample percent correct between the mTBI individuals and the healthy or trauma groups. Lastly, one study enrolled only concussed individuals in order to determine construct validity for the ImPACT, which developed correlation measures to the Symbol Digit Modalities Test (SDMT). ²⁶

Symptoms

Three studies described the use of technology for symptom management or reporting post-concussion (Table 3). The technology used in each of these studies include the following: Self-Management Activity Restriction and Relaxation Training (SMART), ³⁰ Ecological Momentary Assessment (EMA), ³¹ and SuperBetter. ³² These applications monitor symptoms, provide education in managing current symptoms, and provide reminders to track occurrence and intensity of symptoms. Babcock et al., ³⁰ established that using a web-based intervention such as the SMART program soon after injury is safe because there were no worsening symptoms or disability observed with its use. Significant improvements were reported in functional disability and executive functioning by the parents, but not the adolescents. In another study, participants demonstrated high utilization and satisfaction of the SuperBetter mobile app to track and self-report symptoms following concussion. Greater improvement in symptoms and optimism were reported with the app use; however, no significant differences were found with app use for depression. ³² The EMA methodology using the Palm Pilot device was found to be easy to train adolescents to use, feasible for a school setting, and had a high compliance rate. ³¹

Table 3.

Technology for symptom measurement.

Study Author and year	Concussion Participants	Healthy Participants	Study Purpose	Technology Description	Outcomes
Babcock et al. 30	n = 21 adolescent and parent pairs; adolescents; with a Glascow Coma Scale of 13-15 in emergency department with mTBI.	None	To evaluate the effectiveness of the Self-Management Activity Restriction and Relaxation Training (SMART) to improve symptom burden, functional disability, and executive functioning during the initial month following mTBI.	SMART is a Web-based intervention to promote concussion recovery for adolescents through education, and training in self-management and effective coping. It utilized two components: 1) symptom and activity monitoring and self- management 2) educational modules	Symptom burden: measured by Health and Behavior Inventory; improvement reported: parent p = 0.004; adolescent p = 0.0005. Functional Disability: measured by Functional Disability Inventory; improvement reported by mean scores: parent (p = 0.009); adolescent (p = 0.07) Executive Functioning: measured by abbreviated Behavior Rating Inventory of Executive Functioning; improvement reported by mean scores: parent (p = 0.03); adolescent (p = 0.67). Initiation of the SMART program early after injury was not associated with worsening of symptoms or increased disability.
Lewandowski et al. 31	n = 3 reported concussion within the past 6 months and were still symptomatic	n = 3 no history of concussion and no current physical or psychological illness	A pilot study to examine a momentary data collection system, ecological momentary assessment (EMA) Feasibility of use in a school setting and the sensitivity of EMA in capturing symptom fluctuation in adolescents in concussion was assessed.	Participants recorded the occurrence and intensity of the target concussion symptoms on a personal digital assistant (Palm Pilot 100) when signaled by the device,-5 times a day for 5 days. Symptom Severity Scale (SSS) and the Functional Status Scale (FSS) were measured.	Concussed group reported a higher level of symptom severity on all 12 symptoms (mean rating 4.79 compared to the control 2.96). Average differences between groups ranged from 0.3 SD on some symptoms to 2-3 SD on symptoms such as headache, concentration, irritability, and confusion. Pattern of symptoms were compared via mean score differences; between group: p<.01 with symptoms getting worse as the day progressed. Symptoms resolved with the end of the school day for the control group but persisted into the evening (8 pm) in the concussed group. Impairment was assessed with the FSS; Between group: greater impairment in concussed group (p<.0001; d = 0.86) The EMA was easy to train in use, usable in a school setting, and had a high rate of compliance (93.3%)
Worthen-Chaudhari et al. 32	Phase I (Experimental): n = 20 with unresolved concussion symptoms at 3 weeks to 12 months post injury; received standard medical care plus app use; assessed satisfaction and feasibility of the mobile app. Phase II (Control): n = 22; also had unresolved concussion symptoms; received only standard medical care.	None	To assess whether recovery profiles differed between youth who augmented medical care with a mobile health app and those who received medical care alone. To evaluate the feasibility for use by youth as a complement to standard medical care in managing concussion. Phase I: Feasibility: Satisfaction: assessed by satisfaction survey using a 7-point Likert scale. Barriers to compliance Phase II: control group received standard medical care. Both the intervention and control group assessed symptoms using the SCAT-3, optimism measured by the Life Orientation Test-Revised (LOT_R) and depression measured by the Center for Epidemiological Studies- Depression Child (CES_DC).	Mobile application, SuperBetter, with the Battle Royal Power Pack, which guided participants to track the frequency and severity of 22 concussion symptoms. Symptoms were represented as bad guys and medical recommendations were represented as power ups. Logged activity consisted on in-app use, reporting if a bad guy was battled, how severe the battle was, and if a power up was completed. Experimental group was asked to log into app 1 time a day, 5 days a week for 3 weeks.	Phase I: 14 of 20 completed the intervention. Feasibility: 70%; high app utilization Satisfaction: high app utilization-p < 0.0001. Barriers to compliance included: discontinuation of medical care, Internet access issues, extracurricular activity conflicts, and illness during enrollment. Phase II: n = 19; Between group comparisons: More responders in experimental group for symptoms (p = 0.0006) and optimism (p = 0.012). Greater improvement in symptoms and optimism for the experimental group (U = 18.5; p = 0.028; r = 0.50) than for the active control cohort (U = 18.5; p = 0.028; r = 0.51). No significant differences found between cohorts for depression responders (p = 0.906) or change in depression scores (p = 0.156). Participants who used the app to complement medical care saw more relief from concussion symptoms than those experiencing traditional medical care alone.

Table 4.

Technology for visual system performance.

Study Author and year	Concussion Participants	Healthy Participants	Study Purpose	Technology Description	Outcomes
Alberts et al. 22	n = 181 with concussion; n = 92 (typical) recovered within 3 weeks and n = 89 (delayed) continued to have symptoms >3 weeks post-injury	None	Determine responsiveness of the Cleveland Clinic Concussion Application (C3 App) in detecting change in neurological function post-concussion	C3 App functions on a tablet device and assesses a range of potentially disturbed brain functions including static and dynamic vision.	No difference at examination, post-concussion between the typical and delayed recovery groups on static or dynamic vision tests
Fine et al. 33	n = 16 (20.9) days since injury with persistent symptoms	n = 14 controls matched on age	Examine whether deficits in visual-motor target tracking could improve concussion detection and the feedback response used to detect and correct errors in performance	A hand held dynamometer connected to a laptop, displaying a horizontal and vertical bar. Participants are to match the height of the variably positioned target bar by generating a specific force to change the height of the second bar	Velocity error was greater in those with concussion for the unpredictable and predictable condition (p = 0.019-0.048). Positional errors were not significantly different between concussed and controls for either condition.
Howell et al. 24	n = 19 NCAA Division I collegiate athletes within 5 days of concussion	n = 40 NCAA Division I collegiate athletes at baseline	Examine performance on tablet-based vision outcomes in participants with acute concussion compared to participants without concussion	Tablet-based neurocognitive test via the C3 App including Static and Dynamic Visual Acuity Tests	Visual acuity (static to dynamic line difference) demonstrated no difference between healthy and concussed (p = 0.75) and effect size of 0.10.
Mansur et al. 34	n = 29; 16 – 80 y/o with Post-Concussion Syndrome (>3 months since concussion)	None	Assess feasibility of a non-Liquid Crystal Display (LCD) screen to decrease computer screen intolerance in patients with PCS	A 15” non-LCD computer monitor which is non backlit and does not refresh (similar to a Kindle reader)	Reading on the LCD screen produced a significant increase in symptom score (p < 0.01) and total number of symptoms (p < 0.01). There was no significant increase in the non-LCD screen (p = 0.063 for symptom score and 0.47 for number of symptoms). Participants read longer on the non-LCD screen (p = 0.009).

Visual system

Four studies described the use of technology for components of the visual system related to concussion (Table 4). Three of these explored technology in identification of visual deficits post-concussion.^21,24,33 Within these studies, the technology described included the C3 App; a computer based program to examine deficits in visuomotor target tracking; and the use of a non-Liquid Crystal Display to decrease visual sensitivity. Across the three studies, one found that the technology was useful to detect velocity error, which was greater for the concussion group when compared to healthy controls. ³³ The C3 App did not detect significant differences in static or dynamic visual acuity between concussion and healthy participants. ²⁴ The second study using the C3 App found that the vision tests were unable to discriminate between the typical and delayed recovery groups at examination. ²¹ One of the included studies described technology in concussion management. ³⁴ This study found that a non-LCD display produced less screen intolerance in participants with post-concussion syndrome.

Discussion

The 29 articles included in this review described technology to measure balance, neurocognition, vision/oculomotor function, and symptoms associated with concussion. An interesting point is that 27 articles were unable to be included in this review because they were only available as an abstract, having no corresponding manuscript. Typically, this occurs when research is presented as a poster or platform at a scientific conference or meeting but does not go on to become a published, peer-reviewed, full-length manuscript. It is not known why so many studies describing technology and concussion are not being published, but it is disappointing because the results of these additional 27 studies could have been very informative to this review and to the general research base supporting technology development. We would like to encourage those researching in this area to bring their research findings to publication so that it is able to be incorporated more completely into the evidence base.

A majority of the studies included in this review reported on balance and within this, most focused on technology’s ability to identify concussed individuals from non-concussed individuals. Several studies explored technology’s measurement of balance in comparison to the BESS or mBESS (no-tech clinically based outcome measures), or used sensor measurement to provide enhanced measurement of balance during BESS/mBESS performance.^{8,10,11,13,16,17,21} Overall, the additional information provided through sensor measurement increased the usefulness of the BESS/mBESS for concussion identification. A second finding within the research on balance is that quantification of postural sway with VR technologies provided sensitivities and specificities of >80% to differentiate concussed from non-concussed individuals.^14,15 Other sensor based technologies of postural sway identified specific postures that were the most discriminant for concussion diagnosis, two of which showed double leg stance, eyes open, ¹⁸ and closed, ¹⁹ were the most discriminant positions for concussion identification.

It is understandable why technology in concussion has an abundance of research exploring balance. Standing balance is quick and easy to assess, can be examined almost anywhere, and if advanced sensor measurements can demonstrate high diagnostic utility, balance could be an ideal biologic marker of concussion. According to the results presented here, although technology’s ability to measure balance is superior to human measurement, produces good to excellent validity and reliability, and provides an enhanced ability to identify balance impairments, there are still 13–19% false positives and false negatives in differentiating concussed from non-concussed,^14,15 making these technologies very useful, but not stand-alone tools in concussion identification.

Tests of neurocognition have been widely used to screen for concussion. Initially, these tests were completed with paper and pencil but have moved to computer-based platforms. Of the seven studies included in this review, various types of tests on tablets, other mobile devices, and immersive environments produced significant differences in performance between those with and those without a concussion.^21,24,25,29 The sensitivity and specificity of these tests was not described in any of these studies. We believe that new technologies to administer valid tests of neurocognition hold much promise, particularly as they are moved in to emergency departments and are able to obtain reliable results in non-optimal (noisy) environments,^25,28 these technologies hold promise for more immediate concussion identification. Future directions should continue to work towards technology-enabling tests of neurocognition with measurement capacities that do not require baseline testing for optimal usefulness.

The experience of symptoms by individuals with concussion is variable based on provocation of movement, ³⁵ environment, and cognitive load. The studies included in this review explored technology for symptom monitoring, enabled enhanced methods of tracking daily symptom reports, and provided medically based suggestions for self-management.^30,32 We see these tools as being very useful as mobile applications, accessible from a smart-phone, for monitoring response to increased mental and physical demands, including gradual return to learn, and providing real-time solutions to de-escalate increased symptoms. We suggest that further advances in this area would include the use of these types of tools for collection of symptom metrics each day post-concussion to present to medical providers through a telehealth platform for asynchronous remote patient monitoring. This would permit directed management, including change of management strategies. For example, remote symptom monitoring would enable a provider to assess if symptomatic recovery is progressing faster than anticipated. This may enable a progressive resumption of typical activities. Conversely, if unanticipated spikes in symptoms occur, this could indicate neurological overload and warrant an in-person visit or scaling back of activities.

Within the recent past, vision, oculomotor function, visual attention, and perception have all become areas of intense interest in the study of concussion.^36–38 In the absence of concussion, research on oculomotor function has been conducted using instrumented laboratories with eye tracking capabilities, specialized computer software, data filters, and advanced statistical analyses. These are difficult things to translate from a lab-based environment to a mobile platform. The studies included here did not utilize instrumented labs but rather laptop computers and iPads. From the results provided, there is limited evidence that static and dynamic vision, as measured on an iPad, can differentiate concussions from non-concussed or those with typical or delayed recovery.21,24 There was some evidence that visuomotor target tracking could improve concussion detection using a hand-held dynamometer and a laptop. ³³ One study also demonstrated that visual sensitivity could be decreased with a non-LCD computer screen, ³⁴ which is helpful as a management strategy as student athletes try to return to the learning environment. Given that eye-related metrics have gained considerable ground as biomarkers for concussion,³⁸ we believe that research to detect eye-related impairments with mobile technology is just beginning. We expect advances in this area to accelerate in the near future.

Technological advances offer the opportunity to enhance concussion identification by detecting minute impairments undetectable with regular clinical measurement capacities, thereby increasing the objectivity of examination. Most of the studies included here explored technologies for concussion identification. We believe that technologies also hold promise for the provision of unique opportunities for rehabilitative treatment of functional impairments. Unfortunately, only one case study was identified in this review describing high-end technology (a fully immersive virtual reality system) as a treatment modality. ²⁰

We hope that innovative ideas using advanced technologies continue to be studied and are able to add enhanced diagnostic accuracy for concussion as well as effective treatment mechanisms.

It is likely that many of the commercially available technologies for concussion are not represented in the literature included in this review. Some of these may be represented in the 27 abstracts which were unable to be included. However, it is also possible that these were not identified by the search strategy utilized. Because this review sought to include all available technologies which are able to be used in a clinical environment for concussion, many research designs and study objectives were included, making strong generalizations across studies difficult.

Conclusion

There are many technologies which have been and are being investigated to assist with concussion care. To date, these technologies have predominantly focused on the ability to identify those with concussion from those without concussion. While the research included in this review demonstrates some enhancement over clinical tests and measures, the ability to definitively detect concussion remains obscure. Additionally, there are many unexplored opportunities where technology could be investigated as an intervention modality. It is likely that this broad and diverse area of research will continue to evolve as new sensors and software are developed, exposing new tools to enable greater accuracy in identification and usefulness in management.