Abstract
People with disabilities encounter many difficulties, especially when a diagnosis of more than one dysfunction is made, as is the case for visually impaired wheelchair users. In fact, this scenario generates a degree of incapacity in terms of the performance of basic activities on the part of the wheelchair user. The treatment of disabled patients is performed in an individualized manner according to their particular clinical aspects. People with visual and motor disabilities are restricted in independent navigation. In this navigation scenario, there is a requirement for interaction that justifies the use of virtual reality (VR). In addition, locomotion needs to be under natural control to be successfully incorporated. Based on such a condition, electroencephalography (EEG) has shown great advances in the area of health by employing spontaneous brain signals. This research demonstrates, through an experiment, the use of a wheelchair adapted to have the support of VR and EEG for training of locomotion and individualized interaction of wheelchair users with visual impairment. The objective was to provide efficient interactions, thus allowing the social inclusion of patients who are considered otherwise incapacitated. This project was based on the following criteria: natural control, feedback, stimuli, and safety. A multilayer computer rehabilitation system was developed that incorporated natural interaction supported by EEG, which activated the movements in the virtual environment and real wheelchair through adequately performed experiments. This research consisted of elaborating a suitable approach for blind wheelchair user patients. The results demonstrated that the use of VR with EEG signals has the potential for improving the quality of life and independence of blind wheelchair users.
Introduction
People with disabilities face a daily battle, as the lack of accessibility and social inclusion is still a big problem. 1,2 There currently exist a significant number of people with disabilities; therefore, solutions that will help these people are constantly sought, whether for mobility, rehabilitation, communication, or digital inclusion. In Brazil, there are still many structural issues in the area of therapy for the disabled. For example, the 2010 Census 3 shows that 45,606,048 of Brazilians, 23.9% of the total population, have some kind of disability—visual, auditory, motor, mental, or intellectual. Visual impairment had the highest incidence rate, affecting 18.6% of the population. Second is motor impairment, occurring in 7% of the population, followed by auditory in 5.10% and mental disabilities in 1.40%. 3
The disability knowledge area offers categorical challenges in helping people with severe needs, and their impact on the right to come and go. This constitutional right is often violated due to the lack of accessibility, 4 especially when dealing with patients who have more than one significant disability. Those patients are classified as incapacitated for independent living. Motivated by this scenario, this work is about the difficulty of locomotion of the blind wheelchair user. This class of patients, by far, depends on the use of wheelchairs for the rest of their lives. In fact, individuals with multiple disabilities, on average, are shown to suffer greater exclusion from new technologies. In this case, these computational systems need to be supported by a type of interface that provides the user with a more cognitive form of training. This emphasis is placed upon virtual reality. 5 VR can be visualized as a system used to create an artificial environment, in which the user has the impression not only of being inside this environment but also enabled, with the ability to navigate the same, interacting with its objects in an intuitive and natural way. 6 In addition, studies demonstrate that creating a functional space for the tetraplegic individual through a computer system is favorable for the training of the wheelchair user. 7 Accordingly, through the already mentioned studies and the long search for social inclusion, newly assisted technologies are being sought and developed. 8 One of the main functional interfaces is the brain computer interface (BCI), which is a technique that aims at interpreting electrical signals from the cortical surface of the brain without going through nerves and muscles. This technique called electroencephalography (EEG) is used to acquire brain signals to interact with the external environment through devices, seeking to interpret thoughts toward movements without the need for real movement. 9
Currently, the major VR systems for wheelchair training through brain waves do not support visually impaired wheelchair users. 10 Therefore, the synchronization between acquisition of the signal and the virtual environment (VE) presents itself as a challenge to be overcome. 11 The proposal behind this study is to, therefore, approach the development of a tool for training blind wheelchair users. 12
Overview
Each deficiency has prerequisites for implementation of computational systems and experimental approaches for training and treatment solutions. One notes here that all projects are focused only on one deficiency, leaving aside patients with more than one dysfunction, which could be treated together, being necessary only to redefine the appropriate requirements for both disabilities. Table 1 presents the list of studies categorized with specific criteria required for each impairment, these projects presented important attention points.
Describe Requirement for Each Deficiency
3D, three-dimensional.
Case Study
The patient is 33-years-old, suffered a car accident on returning home after a day's work. He was diagnosed with cranial polytrauma in the presence of a loss of encephalic mass in part of the parietal lobe. Consequently, speech loss, neck movements down, and vision completely compromised. The only means of communication with the patient is through blinking.
System Prototype
A VE for locomotion training for blind wheelchair users was developed, using hardware, software, and computational techniques to improve interaction and inclusion of the patient. For this adaptation, it is necessary to investigate the requirements for the construction of the prototype; according to Grant (2004), the architectures are derived from the main components as visual simulation, physical simulation, and control 13 (Figure 1).
Overview of virtual brainy chair.
The proposal of this work promotes a pretraining on the bed, and as this advances, it changes to being performed in the wheelchair. The visual simulation consists of providing a simple or complex model depending on what is most appropriate, thus providing criteria and applying the necessary techniques to the prototype.
Architecture
The architecture is divided into three layers: hardware layer, Emotiv Epoc Application Programming Interface (API) and user interface. In the first layer, hardware architecture, are the devices responsible for capturing, acquiring, and executing commands for movements. The second Emotiv Epoc API layer performs the processing of captured raw EEG signals (Figure 2).
Solution architecture.
There is a sublevel with a wrapper for communication with the Arduino microcontroller, which has the logic to feed (force) and perform the locomotion activities next to the motor for wheelchair movements to occur. The connector receives the information generated by the user to drive the motors located simultaneously in the lower right and left hand side of the wheelchair, defining the speed and direction of rotation of the motor. The user interface layer is where the control for movements and application of the techniques of audio three-dimensional (3D) is implemented, so that the actions in the VE are reproduced. The integrations of these three layers formed a single block, the project is composed of three primary axes: (1) EEG capture, (2) classification and sending of commands, and (3) replicate movement in the VE and in the wheelchair. In addition, the Logitech surround sound speakers Z506 delivers 75 W of power with 3D stereo, as shown in Figure 3.
Functional architecture.
Virtual Environment
The VE was designed for the training of blind wheelchair users in the process of locomotion, and navigation takes place first through 3D sound orientation to indicate what will be the course and the movement to follow, so the user must perform a facial expression, in order that the required movement occurs (Figure 4).
General bird's eye view of the VE. VE, virtual environment.
The training sessions were designed to be conducted on public roads or sidewalks around the house, with obstacles and interactions to simulate locomotion. The movements are guided by 3D sound instructions to the patient and triggered by the brain signals based on the facial expressions and then by means of a wheelchair avatar using the wheelchair in the VE, as shown in Figure 5.
Three-dimensional model of patient in the VE.
Facial Expression and Movements
The solution developed will translate Expressiv Emotiv Epoc API results into commands to move the virtual avatar and wheelchair. Table 2 lists the commands based on facial expressions.
Commands Based on Facial Expressions
Experiments
The brainy virtual chair project aims at enabling the user to move independently in the VE, with the purpose of going around the house along the sidewalk, as shown in Figure 6.
Route in VE.
The experiment was performed with a blind patient in two stages: No. 1, in bed (Fig. 7); No. 2, wheelchair (Fig. 8). The VE is used in both situations, using 3D sound techniques. To evaluate the performance of the system and verify whether the patient is able to perform the training, sessions are performed in the VE, accompanied by a health professional.
Simulation in progress, with patient on bed.
Patient preparing for starting a trial session.
The patient will move through the VE using Emotiv Epoc, making the facial expressions as described in Table 2. In this first part of the session, the circuit is activated with the patient on the bed, and soon after completion, the user initiates phase 2 of the training with the wheelchair. In Figure 8, the patient is shown prepared to begin the training with the wheelchair, and then beginning the use of the adapted wheelchair.
Results and Discussion
This work is qualitative, characterized as a descriptive study with empirical basis, systematic observation being used as a data collection instrument, based on Clemente et al. 14 The experiment began by measuring the communication time between the collections of brain waves to the movement represented in the VE.
EEG–Wheelchair Interactions
The integration of EEGs in wheelchair control aims to reach a high level of data related to usability, this to compare the BCI interaction allowing for satisfactory control and experience. Table 3 demonstrates that Emotiv presents a false positive for detection (error 5%) for detections of facial representation, against 2% for Emotiv API; this result, however, does not compromise the experiment, as each movement can be validated by a visually validated facial expression.
Electroencephalography–Wheelchair Interactions
API, application programming interface.
Virtual Environment
The VE presented several modifications during the pre-established conceptions, but for such a comparison, two tests were carried out contemplating the VE and the wheelchair, according to Table 4.
Results of Sessions
EEG, electroencephalography; VE, virtual environment.
In the first session, we noted that the patient had a low level of correct EEG classification due to the lack of step-by-step instructions to guide each movement in a more detailed way. For this reason, it was evidenced that they had ∼62% accuracy level for the required level of assertiveness. Trial No. 2 presented better results especially due to a better implementation of audio instruction commands in the VE. The training was conducted with timeframe of 30 days for these testing and duration of ∼60 minutes per session during 2 months. Before beginning actual test sessions, 12 training sessions were conducted relative to trial periods of 30 days. This level of practice could be improved with sessions one time per week that will improve accuracy and memory of the user.
Patient and Doctor Results
The vital signs were considered, wherein it was possible to use these as an instrument to measure the impact on the patient's emotional and physical factors. According to the results of the physiological data, those readings taken after the training sessions show an increase in the vital signs of the patient as given in Table 5.
Vital Signs Collected During Trials
During the experiment, AF3, F3, F7, F4, T7, and T8 channels were those with the highest activity and represented that these are responsible for the reasoning and control of movements, as given in Table 5.
Conclusion
In this work, a computational system was developed involving hardware, software, and computational techniques to improve integration in the training of visually impaired wheelchair users. In the case study evaluated, it was demonstrated that the patient was able to test the tool and perform clinical practices. The test subject was still able to walk through predefined actions through use of the wheelchair and reproduce the actions trained in the VE. This research demonstrates, through an experiment, the use of a wheelchair adapted to have the support of VR and EEG for training of locomotion and individualized interaction of wheelchair users with visual impairment. The objective was to provide efficient interactions, thus allowing the social inclusion of patients who are considered otherwise incapacitated. This project was based on the following criteria: natural control, feedback, stimuli and safety. A multi-layer computer rehabilitation system was developed that incorporated natural interaction supported by EEG, which activated the movements in the Virtual Environment and real wheelchair through adequately performed experiments. This research consisted of elaborating a suitable approach for blind wheelchair user patients. The results demonstrated that the use of Virtual Reality with EEG signals has the potential for improving the quality of life and independence of blind wheelchair users.
Footnotes
Disclosure Statement
No competing financial interests exist.