Alertness-based subject-dependent and subject-independent filter optimization for improving classification efficiency of SSVEP detection

Abstract

BACKGROUND:

Mental task-based brain computer interface (BCI) systems are usually developed for neural prostheses technologies and medical rehabilitation. The mental workload was too heavy for the user to manipulate BCI effectively. Fortunately, electroencephalography (EEG) signal is not only used for BCI control but also relates to the changes of mental states.

OBJECTIVE:

We proposed a novel method for identifying non-effective trials of Steady State Visual Evoked Potential (SSVEP)-based BCI.

METHODS:

We used the subject-dependent and subject-independent alertness models identifying non-effective trials of SSVEP-BCI systems.

RESULTS:

The result implied that the subject-dependent alertness model was most useful for improving the classification accuracy in the task. However, the subject-independent alertness model could enhance the prediction ability of SSVEP-based BCI system.

CONCLUSION:

In comparison to the conventional canonical correlation analysis (CCA) method without alertness-model filtering, the raise of precision was valuable for the technical development of BCI works. It demonstrated the effectiveness of our proposed subject-dependent and subject-independent methods.

1. Introduction

EEG-based brain computer interface (BCI) technology had been developed for medical equipment and artificial intelligence applications [1, 2]. EEG signals are commonly applied to BCI tasks including the P300 potential [3, 4], SSVEPs [5, 6, 7] and event-related desynchronization/synchronization (ERD/ERS) produced by motor imagery tasks [8, 9].

Conventionally, users of BCI have heavy mental workloads which manipulates relevant applications [10, 11]. In the task requiring sustained attention, human alertness was difficult or impossible to retain for a constant level for a long time [12]. Fortunately, EEG signal was not only used for BCI control but was also related to the changes of mental status [13, 14]. During the past half century, many studies of vigilance demonstrated that EEG rhythm was an effective indicator to reflect the fluctuation at the level of alertness [15].

As known, SSVEP signal was typically related to the EEG spectrum [16]. Canonical correlation analysis (CCA), using several harmonics of the stimuli frequencies, was the most popular method for SSVEP recognition [17]. Bin et al. [18] conducted the experiments of character spelling to verify the high efficiency of this algorithm in comparison with other methods. However, there were no effective methods that could be used for evaluating mental states in the task of SSVEP-BCI.

Traditional alertness supervising made use of subject-dependent pilot EEG data which combined features of spectral power and machine learning methods [19, 20]. Subject-dependent manner was described as constructing the classifier based on prior data. In contrast, the subject-independent model was trained by irrelevant data from other subjects and tested by his/her own data [19, 21]. The normalization was applied to unify the evaluation rule at a standard level of data dimensions. On this basis, the features of EEG rhythms would be extracted for constructing the subject-independent alert-model. Furthermore, the sleepiness propensity was a transient period occurring after a long period of monotonous or heavy attention-demanding tasks [22]. It could be used for vigilance assessment reflected the process from wakefulness to sleepiness [23]. In this paper, we used two subject-independent models and one subject-dependent model for alertness evaluation in the task of SSVEP-BCI.

2. Methodology

2.1 Data process

The alertness predictor was developed for identifying non-effective trials from SSVEP task. As shown in Fig. 1, the predictor was used for assessing the subjects’ mental state by SVM classifier.

Figure 1.

Flowchart of the data process for classification. The subject-dependent method utilized 80% of task data for training and 20% for testing. Two subject-independent models were developed for classifying all task trials into fatigue state and wakeful state, which represented non-effective and effective recognition. After alertness-model filtering, the CCA algorithm was employed for detecting SSVEP signal for labeling these effective trials.

We compared two subject-independent methods with subject-dependent alert-model to construct the alertness predictor. Moreover, the non-model classification method (i.e., CCA) without the alertness-model filtering was used for contrasting with the above three methods. The subject-dependent approach utilized 80% of task data for training and 20% for testing. The 10-fold cross validation was applied to verify the efficiency of this model.

On the other hand, two subject-independent models trained by task data except for his/her own data and Sleep-EDF dataset for training, respectively. We made a reasonable assumption for the subject to be fatigued in the experiment of error detection. Thus, all trials could be divided into two portions which represented wakeful and fatigue states. Sleeping-EDF dataset which was obtained from the PhysioBank online resource is a standard public data which recorded subjects from wakeful state to sleeping state [24]. We only made use of four recordings: sc4002e0, sc4012e0, sc4102e0 and sc4112e0 from healthy subjects recorded in 1989. Also a typical SVM classifier was applied to classification tasks. Similarly, both subject-independent models were developed for classifying all task trials into fatigue state and wakeful state, which represented non-effective and effective recognition. Then, CCA algorithm was employed for detecting SSVEP signal for labeling these effective trials.

2.2 CCA algorithm

CCA was a well-known algorithm for two datasets, which may have an underlying correlation. It assumed that the raw EEG signal was correlated with the stimulus signal which was represented by the Fourier series of its harmonics. The coefficient of canonical correlation could be calculated by solving an optimization problem for maximizing the above signals combination. The detailed introduction has been described in [17].

2.3 Feature extraction

The goal of this step was to extract features from EEG signal to distinguish between wakeful state and fatigue state optimally. Firstly, raw data were preprocessed by Gaussian normalization for all models. Specifically, Sleep-EDF data were consecutive signals without artificial separation. Hence, we segmented them into several epochs within 30 seconds time window, and task data was separated into several trials according to the experimental setup. It was conventional to describe EEG in terms of its frequency components, including the typical band power composed by delta (0.5–4 Hz), theta (4–8 Hz), alpha (8–12 Hz), and beta (12–35 Hz). The power spectral density analysis was implemented for calculating these spectral powers. Moreover, the ratios of four spectral powers and total power, theta/beta, (theta $+$ alpha)/(alpha $+$ beta), (theta $+$ alpha)/beta, as well as beta/alpha were also computed for training the alertness-model classifier. Especially for Sleep-EDF data, only one channel of EEG (Fpz-Oz) was selected for extracting features. Subsequently, we just used the task data acquired at OZ for alertness-model detection.

2.3.1 SVM classifier

SVM is a kind of supervised machine learning algorithm based on statistical learning theory. It mapped raw data into high-dimension space for extracting enough features. Also, kernel function selection is an essential step in SVM design. We used radial basis function support vector machine (RBF-SVM) algorithms for constructing three alertness models. Finally, the effectiveness of the alertness models was evaluated by corresponding testing data.

3. Experiment and result analysis

3.1 Experimental setup

In our study, 10 healthy BCI-naive subjects (8 males and 2 females), aged from 22 to 29, participated in our task after obtaining their IRB-approved consent forms. They had normal or corrected to normal vision. A 16-channel bio-signal amplifier (g.tec Medical Engineering GmbH, Austria) was used for acquiring scalp EEG signals. SSVEP signals were typically related to the neural activities on the visual cortex. Hence, the signal was collected from four passive gel-based electrodes (i.e., POz, O1, Oz, O2). Electrodes impedances were kept below 5 k $\mathrm{\Omega }$ . The signals were amplified, digitalized with a sample rate of 256 Hz and bandpass-filtered between 0.1 and 35 Hz. A notch filter was used to suppress the 50-Hz power line interference. The stimuli frequencies were 6, 7, 8, 9 Hz.

The SSVEP-based BCI tasks were conducted in a normal room without electromagnetic shield- ing at Tongji University, China. At the beginning, each subject was requested to pay attention to the center of the computer screen. Then, he/she performed the task of five runs and each run contained three trials per class. In a trial, one random digital of four classes (i.e., 1, 2, 3, 4) was shown in the screen. The subject was instructed to gaze at the corresponding flickering button in the stimulus panel for 10 seconds. An interval feedback of about 0.5 seconds was given for the shift of visual attention.

3.2 Result

In our study, an offline analysis of SSVEP detection was implemented for performance evaluation. Three alertness models were used for identifying non-effective trials. Table 1 shows the classification accuracy of SSVEP detection by non-model (NM), subject-dependent model (SM) and two subject-independent models trained by all other nine subjects’ data (AOSD-SM) and Sleep-EDF data (SD-SM). It was implied that the efficiencies of classification methods based on SM (paired $t$ -test: $t=$ 1.959, $p<$ 0.05) and SD-SM (paired $t$ -test: $t=$ 2.632, $p<$ 0.05) models were better than the conventional NM method. The AOSD-SM algorithm was useless for improving the classification precision.

Table 1

The classification accuracy of SSVEP detection by non-model (NM), subject-dependent model (SM) and subject-independent model based on the subjects’ data (AOSD-SM) and sleep-EDF data (SD-SM)

Subject	1	2	3	4	5	6	7	8	9	10	Mean
NM (%)	73.3	68.3	80.0	98.3	83.3	79.3	93.3	91.8	74.4	90.0	83.2
SM (%)	92.0	75.0	87.0	82.0	100	86.5	100	86.0	96.5	97.5	90.3
AOSD-SM (%)	73.3	33.3	68.3	71.7	95.0	73.3	98.3	78.3	93.3	91.6	77.6
SD-SM (%)	90.0	72.7	81.8	100	85.7	85.7	100	90.5	77.8	90.0	87.4

For the SD-SM model, the validity of the method was proven by quantitative statistics of trials under fatigue and wakeful conditions, which represented non-effective and effective task recognitions, respectively. Table 2 shows the numbers of trials under fatigue condition (FC) and wakeful condition (WC). It was suggested that the qualities of trials under both conditions were statistically reasonable for all subjects.

Table 2

The numbers of trials in fatigue and wakeful conditions, which represent non-effective and effective task recognitions, respectively

Subject	1	2	3	4	5	6	7	8	9	10
The number of trials in fatigue condition	29	30	27	33	16	29	28	25	31	23
The number of trials in wakeful condition	31	30	33	27	44	31	32	35	29	37

4. Discussion

In our study, CCA algorithm was used for SSVEP detection. It was reported that the method obtained good results in SSVEP-based BCIs [18, 17]. In order to verify the algorithm validity, we analyzed the spectral feature of subject 4’s EEG signals. Figure 2 shows the power spectra of channel Oz separated for each evoked frequency. It was indicated that this algorithm was effective for frequency recognition of SSVEP signals.

Figure 2.

The power spectra of channel Oz separated for each evoked frequency by subject 4. Sequentially, the target frequencies were 6 Hz, 7 Hz, 8 Hz and 9 Hz.

Previously, the recognition strategy (feature extraction and classification algorithm) was considered as the only one factor that influenced the efficient of BCI systems. Nevertheless, the physiological status of the subject directly influenced the performance of BCI tasks. The control effect of BCI went worse while the subject could not afford the heavy load in spirit. This is why we applied alertness assessment for detecting the subject’s mental status to exclude the subjective disturbance.

In this paper, three alertness models were presented for improving the efficiency of our BCI system. From the viewpoint of classification, the subject-dependent model was most suitable for detecting error-sorted trials. The method of AOSD-SM was inferior to the non-model method. Besides, the outstanding performance of the SD-SM model was a novel finding for alertness model-based SSVEP detection. Except for the SD-SM model, other models conformed to the experimental expectation. Previous studies had shown that subject-dependent models were feasible for identifying the low-alert state [12]. For most subjects, group statistics could not be utilized to precisely predict changes in mental states [25]. However, the result of the SD-SM model-based approach suggested that low-alert cognitive states and sleepy mental states have similar spectral features.

Figure 3.

Comparison of the mean values of delta, theta, the ratio of delta and total power, the ratio of theta and total power and theta/theta between fatigue condition and wakeful condition.

For clarifying the theoretical principle of the SD-SM model, the means of all features were statistically analyzed between fatigue condition (FC) and wakeful condition (WC). We found that the means of delta, theta, the ratio of delta and total power, the ratio of theta and total power as well as theta/theta were less in FC for all subjects (Fig. 3). Meanwhile, Fig. 4 shows the statistical result of the numbers of correct trials between the first half and the second one of the task. The classification accuracies of first half trials were not less than those of last half trials for most subjects (8 of 10 subjects). It was indicated that the performance of the mental task seemed to be worse over time. This phenomenon was consistent with prior conclusions about the spectral power change from the wakeful state to the sleepy state [22, 26]. It was implied that work difficulty was crucial to maintain the high efficiency of the task performance.

Figure 4.

The numbers of correct trials between the first half and second half of the SSVEP task. The classification accuracies of the first 30 trials were no more than those of the second half for most subjects.

The subject-dependent and subject-independent alertness models proved to play a positive role in SSVEP detection. They could be applied for different monitor conditions with or without pilot data. The subject-dependent model was more suitable for intra-session vigilance monitoring at the basis of beforehand data acquisition, whereas the subject-independent model was more suitable for application to cross-session alertness evaluation without detailed information about the user’s cognitive state. However, there are several issues that need to be resolved to improve the performance. Firstly, the quantity of data must be increased to verify the universality to avoid overfitting. Therefore, thorough analyses would require more studies with a larger amount of subjects. Secondly, the process control needs to be considered for online BCI tests. The balance between transmission speed and systematical efficiency should be kept for meeting actual demands.

5. Conclusions

This paper introduced a novel method for identifying non-effective trials of SSVEP-based BCI by subject-dependent and subject-independent alertness models. These models were used for filtering ineffective trials. The result implied that the subject-dependent alertness model was most useful for improving classification trials. However, the subject-independent alertness model could raise the classification accuracy.