Abstract
Auditory selective attention is a crucial top-down cognitive mechanism for understanding speech in noise. Cochlear implant (CI) users display great variability in speech-in-noise performance that is not easily explained by peripheral auditory profile or demographic factors. Thus, it is imperative to understand if auditory cognitive processes such as selective attention explain such variability. The presented study directly addressed this question by quantifying attentional modulation of cortical auditory responses during an attention task and comparing its individual differences with speech-in-noise performance. In our attention experiment, participants with CI were given a pre-stimulus visual cue that directed their attention to either of two speech streams and were asked to select a deviant syllable in the target stream. The two speech streams consisted of the female voice saying “Up” five times every 800 ms and the male voice saying “Down” four times every 1 s. The onset of each syllable elicited distinct event-related potentials (ERPs). At each syllable onset, the difference in the amplitudes of ERPs between the two attentional conditions (attended - ignored) was computed. This ERP amplitude difference served as a proxy for attentional modulation strength. Our group-level analysis showed that the amplitude of ERPs was greater when the syllable was attended than ignored, exhibiting that attention modulated cortical auditory responses. Moreover, the strength of attentional modulation showed a significant correlation with speech-in-noise performance. These results suggest that the attentional modulation of cortical auditory responses may provide a neural marker for predicting CI users’ success in clinical tests of speech-in-noise listening.
Keywords
Introduction
Because the world poses complex living environments with many sources of sound, it can be challenging to comprehend speech in noise. The term “cocktail party problem (Cherry, 1953)” can explain this difficulty. To resolve the cocktail party problem, a person needs to be able to separate the mixed sounds into auditory objects or streams (McDermott, 2009) and focus on the target object.
Hearing loss can hinder the process of sound segregation, and consequentially, have a negative impact on the performance of speech in noise (Mackersie et al., 2001; Oxenham, 2008; Paredes-Gallardo et al., 2018), even in those who wear hearing aids that can amplify the sound signals (Nelson et al., 2003). Cochlear implants (CI) may even further exacerbate sound segregation due to the lack of critical acoustic cues such as pitch (Galvin et al., 2009; Gfeller et al., 2007).
Although hearing remediation through CIs has evolved in the recent decade to combine amplified acoustic hearing (Gantz & Turner, 2004; von Ilberg et al., 1999), such “bimodal” CI users still exhibit large variability in speech-in-noise performance [e.g., (Berger et al., 2021)]. Understanding the factors that contribute to this variability has clinical value. Such information can suggest potential targets for diagnostic tests and rehabilitative strategies.
The ability to segregate sound is correlated with speech-in-noise performance both in normal hearing (Holmes & Griffiths, 2019) and in bimodal CI users (Choi et al., 2022). However, it is not the only contributing factor to speech-in-noise performance. Selective attention (Posner & Driver, 1992; Treisman, 1969), which can function as a neural attentional filter (Bester et al., 2016; da Costa et al., 2013; Lakatos et al., 2013) or as a form of sensory gain control (Hillyard et al., 1998), must be considered as well because it occurs simultaneously with auditory scene analysis (Alain & Arnott, 2000; Shinn-Cunningham et al., 2020). Selective attention results in suppressed neural representations of undesired auditory objects and enhanced neural responses to the attended sound (Mesgarani & Chang, 2012; Shinn-Cunningham et al., 2020; Woldorff & Hillyard, 1991). Such attentional modulation of neural responses can be measured using event-related potentials (ERPs) [(Choi et al., 2013; Hillyard et al., 1973)]. The strength of attentional modulation has been shown to have a relationship with how an individual performs in speech-in-noise or similar tasks (Choi et al., 2014; Dai & Shinn-Cunningham, 2016; Kim, Schwalje, et al., 2021b). It has been also reported that speech-in-noise performance can be improved by perceptual training that enhances the attentional modulation of ERPs (Kim, Emory, et al., 2021a), implying that stronger auditory selective attention predicts better speech-in-noise performance.
A few studies have explored whether attention modulates ERPs in CI users (Bester et al., 2016; Nogueira & Dolhopiatenko, 2022; Paredes-Gallardo et al., 2018; Paul et al., 2020). Among those studies, Paul et al. (2020) showed that attentional modulation observed in bilateral CI users’ neural responses to a mixture of two competing continuous speech streams correlated with their behavioral attention performance. Recently, Nogueira and Dolhopiatenko (2022) reported that the attentional modulation of neural responses to continuous speech directly presented to the CI processor was associated with speech-in-noise performance.
The present study was designed to show whether 1) CI users exhibit attentional modulation and 2) the strength of attentional modulation is correlated with clinical outcomes in the emerging population of bimodal CI users when they perform speech-in-noise tasks in the sound field using their usual combination of hearing devices. We used relatively shorter (i.e., 4 s) and repetitive speech streams that have been used in our previous study about perceptual training of selective attention (Kim, Emory, et al., 2021a), expecting that findings from this study can evolve to post-CI rehabilitation using the similar training paradigm in (Kim, Emory, et al., 2021a).
Method
Participant Information
Thirteen CI users, between 37 and 77 years of age (mean = 62.2 years, SD = 13.5 years; median = 70.0 years; 7 (53.8%) female), were recruited from the University of Iowa Cochlear Implant Research Center. They gave written consent according to a written protocol that followed the guidelines provided by the Institutional Review Board at the University of Iowa, and they were provided $50 U.S. dollars for their 2-h participation in the EEG session. Table 1 shows the list of subjects with their demographic information, hearing profiles, and the type of CI devices.
List of Subjects with Their Demographic & Audiometric Information.
List of Subjects with Their Demographic & Audiometric Information.
Both selective attention and speech-in-noise experiments were implemented in custom-written Matlab scripts (R2016b, Mathworks) using the Psychtoolbox 3 toolbox (Brainard, 1997). Tests were conducted in an acoustically-treated, electrically-shielded booth with a single loudspeaker (model LOFT40, JBL) positioned at a 0° azimuth angle at a distance of 1.2 m. Sound levels were the same across subjects at 70 dB SPL.
The auditory stimuli contained two clearly separated speech streams: the “Up” and the “Down” streams. The “Up” stream consisted of a female voice repeating the word “up” every 800 ms five times while the “Down” stream had the word “down” spoken by a male voice every 1 s for four times. Thus, by design, the timing of the words was misaligned except for the very first utterance of each stream so that there is no physical overlap between the competing streams.
In each stream, one of the utterances had a 26% higher pitch than the others so that an oddball detection task can be given. The high pitch utterance's duration was maintained the same. The same-duration higher pitch deviant was made by down-sampling recorded audio waveform by 26% and time-stretching it at the same rate. The 26% pitch increase was equal to four semitones, which have been reported to be distinguishable even by CI users without any acoustic hearing (Gfeller et al., 2007). All the subjects easily distinguished deviant from standard utterances when a single-stream voice (either “Up”s or “Down”s) was given. See Figure 1 for the stimulus (single utterance) spectrograms.
Stimulus (single utterance) spectrograms.
The top part of Figure 2 showed example waveforms of the two competing streams of “Up”s and “Down”s.
Grand average ERPs calculated in global field power (GFP), along with stimulus waveforms, is graphed against time in seconds. The red curve is from the “Attend-up” condition while the blue is from the “Attend-down.” Shaded ranges around the ERPs represent standard error across subjects. Red and blue dashed boxes represent the expected timings of peaks of ERPs for each syllable with red corresponding to the utterance “up” and blue corresponding to the utterance “down” (i.e., 50 – 250ms after each utterance onset). For the attentional modulation index (AMI) calculation, the peak amplitude of ERP was found within those boxes. The positions of EEG electrodes that were used for the GFP computation are represented as filled circles at the rightmost corner (the top view).
At the beginning of each trial, the subjects fixated on the stationary sign ( + ) at the center of the blank screen of a computer for a second. Then, a written instruction (i.e., a visual cue) was provided on the screen to direct subjects to attend to either the “Up” or “Down” streams. While a deviant utterance (i.e., the utterance with a higher pitch) existed in each of the “Up” and “Down” streams, subjects were instructed to choose a deviant occurrence from the attended stream only.
The participants were asked to fixate on the screen again at the sign ( + ) before and while the auditory stimulus was provided. After the auditory stimulus, the participants could report their answer (the deviant occurrence number: 3, 4, or 5 for the attend “up” condition and 3 or 4 for the attend “down” condition) by pressing a number into the keypad. 60 attend-up and 60 attend-down trials were randomly intermixed in a single block. Subjects were given a break after every 30 trials.
Speech-in-Noise Test: California Consonant Test (CCT) Stimulus and Design
In every trial, the participants were shown a white cross sign on the black screen and were instructed to fix their gaze upon it so that eye movement artifacts were reduced. The auditory cue phrase “Check the word” (spoken by the same person whose voice was used to introduce the target words) was presented to the listeners for 800 ms so that the participants could be primed for the beginning of the babble. Then, there was a 700 ms silence, which was followed by the eight-talker babble that lasted for 2 s. Target words, which were 100 monosyllabic words chosen from the California Consonant Test (CCT) (Owens & Schubert, 1977), were presented 1 s after the onset of the babble at 70 dB SPL. The signal-to-noise ratio was fixed to + 7 dB for all the trials. The sound levels were selected using the same process as described by Berger et al. (2021) and Kim et al. (2021b). Every participant was provided with 100 words in random order. After 100 ms following the offset of the target word stimulus, participants were given four options, which consisted of four words, displayed on the center of the screen. The participants then used the keypad to choose the word they thought they heard. They were not informed about the correctness of their answer. The task did not move on to the next trial until 1 s after the participants gave their responses. Although the participants were not given time to be trained for CCT, verbally explained instructions were provided by the audiologists (Berger et al., 2021).
Speech-in-Noise Test: AzBio Stimulus and Design
The same participants were also tested with a sentence-based speech-in-noise test, AzBio (Spahr et al., 2012). The sentences were presented in 10-talker babble at 5 dB SNR, at 70 dB SPL presentation level. The total number of words repeated correctly was scored.
EEG Data Acquisition and Analysis
The EEG data were recorded during the selective attention task using a BioSemi ActiveTwo system with a 64-electrode cap with a standard 10–20 position at a 2048 Hz sampling rate. For each subject, 4–5 EEG electrodes were excluded (i.e., not inserted into the cap) because they were situated on top of the CI coil and magnet. For the EEG data analysis, first, the raw EEG data were band-pass filtered offline between 1 to 50 Hz with a 1024-point zero-phase non-causal FIR filter. Then we epoched data from 0.5 s before the sound onset to 0.5 s after the sound offset. Baseline correction was applied by subtracting the mean voltage between −200 and 0 ms before stimulus onset for each channel. Then epoched data were down-sampled to 256 Hz.
Next, electric field-evoked artifacts introduced by CIs and eye-blink artifacts were removed using independent component analysis based on the principles suggested by (Gilley et al., 2006) implemented in the Matlab EEGLab toolbox (Delorme & Makeig, 2004). The artifact-related components were determined by visual inspection that observed temporal and spatial patterns. Out of ∼60 independent components in each subject, the number of removed independent components (because contaminated by CI and eye-blink artifacts) ranged from 3 to 19. The mean number of removed components was 8.8.
Then epochs with maximum absolute amplitude exceeding 100 μV were discarded from the following analyses. After this preprocessing, the number of remaining artifact-free epochs ranged from 51 to 59 in each condition in each subject. The mean numbers of epochs were 53.6 and 54.0 for Attend-Up and Attend-Down conditions, respectively. There was no significant difference in the number of epochs between conditions.
Results
Behavioral Results
Participants’ performance in the selective attention test ranged from 91.67% to 100% correct (mean accuracy 96.67%, standard deviation 3.07%). This accuracy from the selective attention task was ceiling as expected and not used for any further analyses.
The same participants’ performance at the CCT speech-in-noise test ranged from 30% to 80% (mean accuracy 56.47%, standard deviation 16.82%).
The performance at the AzBio test ranged from 3% to 87% (mean 52.15, standard deviation 30.51%).
There was a significant correlation between the two speech-in-noise tests (i.e., CCT and AzBio); a Pearson correlation coefficient R = 0.74, p = 0.0039.
EEG Results: Quantification of Attentional Modulation
For each subject, at least 50 epochs in each attentional condition (i.e., Attend-Up or Attend-Down) were averaged in the time domain at each EEG channel to yield evoked responses. Then, global field power (GFP) was computed for each condition at each time sample in each subject by taking the standard deviation of the evoked responses across 50 electrodes (out of 64) that were commonly used for all the 13 subjects. The filled circles in the rightmost panel of Figure 2 show the positions of the 50 electrodes that were commonly used for each subject's GFP computation. Open circles represent the positions of excluded electrodes as they were situated near CI coils and magnets.
The expected timings of the amplitude peaks of evoked potentials in GFP were hypothesized to occur between 50–250 ms after the onset of each utterance, which are represented by dashed boxes in Figure 2. The red dashed boxes correspond to the utterance “up” while the blue dashed boxes correspond to the utterance “down.” As seen in Figure 2, the evoked peaks following “up” line up within the red dashed boxes show greater amplitude in the “Attend-Up” condition while the evoked peaks following “down” located within the blue dashed boxes show greater amplitude in the “Attend-Down,” which implies graphically that attentional modulation exists at the group level.
Each participant's attentional modulation was quantified through the following procedures.
For each of the “Attend-Up” and “Attend-Down” conditions, the peak evoked amplitude at each utterance was measured by finding the maximum GFP value in the time window of 50–250 ms after the onset of each utterance (i.e., the dashed boxes in Figure 2). The “attended” evoked amplitude was computed by averaging peak evoked amplitudes across “Up” windows (i.e., red dashed boxes in Figure 2) for the “Attend-Up” condition and “Down” windows (i.e., blue dashed boxes) for the “Attend-Down” condition. The “ignored” evoked amplitude was computed by averaging peak evoked amplitudes across “Up” windows (i.e., red dashed boxes in Figure 2) for the “Attend-Down” condition and “Down” windows (i.e., blue dashed boxes) for the “Attend-Up” condition. The attentional modulation index (AMI) was calculated by dividing the difference between the “attended” and the “ignored” evoked amplitudes with the sum of the “attended” and the “ignored” evoked amplitudes [i.e., (attended – ignored) / (attended + ignored)] (Dai & Shinn-Cunningham, 2016; O'Sullivan et al., 2019).
When computed this way, a positive AMI indicates that evoked response is stronger when the sound is attended, which implies a successful sensory gain control (Kim et al., 2021b). A one-sample t-test against the mean of zero proved that attentional modulation was positive at the group level: T = 3.26, p = 0.0069. To note, this main AMI result is from the combination of both evoked time windows following the “Up” and “Down” streams (i.e., “All” in Figure 3).
Comparisons of stimulus-specific attentional modulation indices (AMI). All: Computed over the time windows following both “up” and “down” utterances (i.e., both red and blue dashed boxes in Figure 2). “Up”: AMI computed over the time windows following “up” utterances (i.e., red dashed boxes in Figure 2) only. “Down”: AMI computed over the time windows following “down” utterances (i.e., blue dashed boxes in Figure 2) only. Gray dots represent the AMIs of individual subjects. The top and bottom edges of blue boxes represent the 75 and 25 percentile, respectively. Red bars depict the median. *: p < 0.05, **: p < 0.01, N.S.: Non-significant (from two-sided one-sample t-tests, after Bonferroni corrections of multiple comparisons).
In this section, to observe the stimulus effect on attentional modulation, we computed AMI for the “Up” and “Down” streams separately (i.e., “Up” and “Down” columns in Figure 3, respectively).
When computed in the “Up” time windows only, AMIs did not show an “above-zero” distribution (two-sided one-sample t-test, p = 0.118). However, when computed in the “Down” time windows only, AMI distribution was above zero (p < 0.01 after Bonferroni correction), indicating significant attentional modulation of evoked responses at the group level (Figure 3).
Correlation between AMI and accuracy in the speech-in-noise tasks.
Then, we observed the relationship between AMI and speech-in-noise performance. As Figure 4 shows, significant correlations were found between AMI and participants’ accuracy in the speech-in-noise tasks. Pearson correlation coefficients (R) were 0.71 (uncorrected p = 0.0068) against CCT and 0.71 (uncorrected p = 0.0063) against AzBio scores. These results indicate that AMI and CI users’ performance in speech-in-noise tasks are associated.
None of the observed demographic and audiometric factors (i.e., age, device experience, and unaided hearing threshold) showed a significant correlation with any speech-in-noise performance or with the AMIs. See Figure 5 for the scatter plots and Pearson correlation coefficients. In Figure 5, “Speech-in-noise performance” represented mean percent-correct values averaged across CCT and AzBio scores. All the P values were uncorrected.
Relationship between device experience (i.e., post-implantation time in months), unaided pure-tone average (PTA) hearing thresholds (in dB HL), age (years) and AMI (the top row) and speech-in-noise performance (the bottom row). R: Pearson correlation coefficient. P-values were uncorrected.
The present study has shown that attentional modulation of cortical auditory evoked responses exists at a group level in CI listeners. This result is consistent with the study by Paredes-Gallardo et al. (2018). The current study also showed that an individual's performance at a speech-in-noise task was correlated with her/his strength of attentional modulation, also consistent with previous studies that used different stimulus presentation methods and EEG analyses (Nogueira & Dolhopiatenko, 2022; Paul et al., 2020).
Previous studies found that the variance of speech-in-noise performance in CI users is not well predicted by demographic factors (Berger et al., 2021). Rather, it has been found that auditory encoding fidelity (Friesen et al., 2001; Won et al., 2007) or auditory streaming (Choi et al., 2022; Hong & Turner, 2006) showed correlations with speech-in-noise performance in CI users. Although it is unclear whether our suggested measure of selective attention is redundant with the measures of auditory encoding fidelity and streaming, our results still imply that cognitive factors such as selective attention can work as a neural marker of speech-in-noise performance. The results of this study suggest including an analysis of auditory cognitive functions in the assessment of CI benefits and outcomes. Potential strategies for compensating for hearing loss can be devised through rehabilitation of auditory cognitive functions in the future.
There are certain limitations to the study. First, since all the subjects had some degree of amplified acoustic hearing through contralateral hearing aids, it is unclear to what extent the finding of this study is related to CI-only listening. This is because we focused on the emerging population of bimodal CI users and their outcomes measured using their usual everyday hearing devices in the sound field. A future within-subject study may investigate the relative contributions of acoustic and electric hearing to the neural processing of selective attention and speech-in-noise performance.
Second, although a multiple linear regression analysis could be a good option for this study to investigate the relative contributions of various predictors, we did not conduct it because the sample size (i.e., 13 subjects) was too small. Instead, we showed a series of bivariate relationships of demographic and audiometric factors with AMI and speech-in-noise performance (see Figure 5). Future studies with larger sample sizes will provide a better interpretation of the preliminary result of the current study by conducting a multiple linear regression analysis.
Third, behavioral performance in the selective attention task exhibited a strong ceiling effect, which prevented observing correlations between the attentional modulation of neural responses and attention performance. We intentionally made the attention task easy so that we can obtain a similar number of correctly-answered trials across subjects for a consistent number of EEG epochs. A future study may add a more difficult attentional condition to investigate the neural vs. behavioral relationship of attention.
Forth, by our stimulus design, there were two distinct features that differentiated competing streams: the voice identity (i.e., male vs. female) and the word content ("up" vs. "down"). Either or both features could be used to engage attention, meaning that we cannot disentangle relative contributions of different acoustic and/or linguistic features for engaging attention. This can be controlled in a future study.
Lastly, the deviations by pitch modulations in the speech streams were somewhat predictable (the deviations were always fourth or fifth syllables). It is possible that the participants may have been less engaged or may have been paying attention to both speech streams simultaneously. Thus, it must be acknowledged that the task may not have measured the participants’ capability of employing selective attention to the full desired effect.
Conclusion
The current study has demonstrated that CI listeners exhibit attentional modulation of cortical evoked responses at the group level and that the strength of attentional modulation correlates with performance in SiN tasks.
Footnotes
Acknowledgment
The authors appreciate Mallory Orr, Ann Holmes, Jason Geller, and Jean Hong for their help with data collection and initial analyses. This work was supported by NIDCD P50 (DC000242 31) awarded to Gantz, Griffiths, and McMurray, NIH T32 (DC000040-24) awarded to Hansen, and the Department of Defense Hearing Restoration and Rehabilitation Program grant awarded to Choi (W81XWH1910637). The authors declare no competing financial interests. All the data will be shared publicly.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by American Otological Society Research Grant (awarded to Choi), the U.S. Department of Defense (W81XWH1910637), and National Institute of Deafness and other Communication Disorders (DC000242 31).
