Sage Journals: Discover world-class research

Abstract

Cochlear implants (CIs) are optimized for speech perception but poor in conveying musical sound features such as pitch, melody, and timbre. Here, we investigated the early development of discrimination of musical sound features after cochlear implantation. Nine recently implanted CI users (CIre) were tested shortly after switch-on (T1) and approximately 3 months later (T2), using a musical multifeature mismatch negativity (MMN) paradigm, presenting four deviant features (intensity, pitch, timbre, and rhythm), and a three-alternative forced-choice behavioral test. For reference, groups of experienced CI users (CIex; n = 13) and normally hearing (NH) controls (n = 14) underwent the same tests once. We found significant improvement in CIre's neural discrimination of pitch and timbre as marked by increased MMN amplitudes. This was not reflected in the behavioral results. Behaviorally, CIre scored well above chance level at both time points for all features except intensity, but significantly below NH controls for all features except rhythm. Both CI groups scored significantly below NH in behavioral pitch discrimination. No significant difference was found in MMN amplitude between CIex and NH. The results indicate that development of musical discrimination can be detected neurophysiologically early after switch-on. However, to fully take advantage of the sparse information from the implant, a prolonged adaptation period may be required. Behavioral discrimination accuracy was notably high already shortly after implant switch-on, although well below that of NH listeners. This study provides new insight into the early development of music-discrimination abilities in CI users and may have clinical and therapeutic relevance.

Keywords

cochlear implants mismatch negativity auditory discrimination music perception neural plasticity

Introduction

With the significant advancements achieved in cochlear implant (CI) technology in terms of speech perception, it is understandable that many current CI users wish to be able to listen to music. Furthermore, because music has played a vital role in many of these patients’ cultural and social lives prior to deafness, the promise of regaining music enjoyment is a major motivator for CI candidates (Fuller et al., 2021; Gfeller et al., 2000). This is confirmed by self-reports stating that music is the second most significant stimulus for CI users after speech (Gfeller et al., 2002).

Given the high temporal resolution and transmission accuracy of the implant, CI users, in general, perform on par with normally hearing (NH) controls in rhythmic tasks (Brockmeier et al., 2011; Jiam et al., 2017). Several studies, however, have shown that due to inadequate spectral resolution and compromised temporal fine-structure information, discrimination of fundamental features in music such as pitch, melody, and timbre is significantly poorer in CI users than in NH listeners (Bruns et al., 2016; Cooper et al., 2008; Petersen et al., 2012; Timm et al., 2012). Another reported deficit is the reduced ability to discern dynamics or intensity in music, which is ascribed to the high level of compression in the CI signal (Limb & Roy, 2014). Thus, for most CI users the music experience is unsatisfying, leading to little or no music enjoyment (Fuller et al., 2021; Mirza et al., 2003).

Nevertheless, some CI users seem to overcome the technological limitations and engage in a variety of music-listening activities (Riley et al., 2018), often reporting increased appreciation after repeated listening practice (Looi et al., 2012). Moreover, a few studies have indicated that targeted music training may significantly enhance the discrimination of specific features such as melodic contour, musical timbre, and rhythm (Gfeller, 2016; Gfeller et al., 2015; Jiam et al., 2019). Such improvements are an indication of the auditory system's capacity to reactivate brain function in response to new stimuli (Carcea & Froemke, 2013; Kolb et al., 2003) and the important role of neural plasticity in determining patient outcomes. It also suggests that the information delivered from the CI signal may not be fully utilized by CI users (Moore & Shannon, 2009).

These neuroplastic changes are due to the central nervous system's ability to reorganize itself by forming new synapses and rewiring neuronal circuits. The plastic changes, which can be both functional and structural, can happen throughout the lifespan and may help people recover from, for example, stroke (Johnston, 2009). In the case of cochlear implantation, plastic changes may occur at two levels. First, deprivation of auditory input may result in significant maladaptive changes in the central auditory system and potential cross-modal reorganization of the auditory cortex in support of other sensory processes, including visual and somatosensory input (Rettenbach et al., 1999). Second, after switch-on of the implant, the novel auditory stimulus itself induces plasticity, thereby allowing for the essential experience-dependent adaptation to the input from the CI (Glennon et al., 2020; Ito et al., 2004; Moore & Shannon, 2009).

The initial rehabilitation process following cochlear implantation has been the focus of several studies. In a study with 48 bilaterally implanted CI recipients, Chang et al. (2010) found that the strongest benefit on word recognition occurs within the first month, with performance continuing to improve up to 48 months after implantation. Using positron emission tomography and a speech perception test, our group measured 15 recently implanted CI users 3 and 6 months after switch-on (Petersen et al., 2013). In addition to a significant gain in speech perception scores achieved within the first 3 months, activation of Broca's area increased over time. Employing electroencephalography (EEG) to measure auditory event-related potentials, Sandmann et al. (2015) assessed the ability to discriminate 1, 12, and 18 semitones frequency changes in 11 postlingually deafened CI users <1, 8, 15, and 59 weeks postimplantation. The authors found that the most pronounced improvement occurred over the first 8 weeks of CI experience, as reflected in increased N1 amplitudes and decreased latencies. Taken together, these findings suggest that the most dramatic progress in outcomes happens in the early stages after the initial CI switch-on. We hypothesize that this may also be the case for music-discrimination skills.

In addition to the N1, EEG offers the possibility to record the mismatch negativity (MMN) response which has been successfully applied as a reliable objective marker for CI users’ discrimination ability (Cai et al., 2020; Näätänen et al., 2017; Petersen et al., 2020). The MMN is a preattentive component of the event-related potential. It is elicited by a violation of expectancy, or a deviation, in a sequence of stimuli, that is, the sequence of regular stimuli (standard) is infrequently interrupted by a deviant stimulus. Stimuli are often presented in oddball paradigms, though newer developments within the MMN paradigm include multifeature paradigms with multiple deviants that deviate on independent features (e.g., intensity, pitch, timbre, and rhythm for musical tones) (Pakarinen et al., 2007; Petersen et al., 2020; Vuust et al., 2011) and multifeature paradigms without any standard stimuli (Kliuchko et al., 2016; Petersen et al., 2020). Typically, the negative peak occurs around 100–250 ms from deviation onset (Kujala et al., 2007; Näätänen et al., 2007). In a study involving 11 experienced CI users and 14 NH controls, our group recently validated a novel musical multifeature (MuMuFe) no-standard MMN paradigm that presents deviants in intensity, pitch, timbre, and rhythm at four levels of magnitude. The results showed that for all deviants, the CI users displayed MMN responses that were not significantly different from those of NH controls, neither in terms of amplitude nor latency. By contrast, measurements of behavioral discrimination of the same deviants showed that CI users scored significantly below NH controls in intensity and pitch (Petersen et al., 2020). The results suggest that preattentive neural discrimination is established before behavioral discrimination abilities, which require higher-order processing; a distinction that has also been established in studies concerning, for example, amusia (Moreau et al., 2013). To investigate the reliability of this hypothesis, a longitudinal study of the adaptation process in CI users is warranted.

So far, no studies have examined the early development of music-related discrimination skills in recently implanted CI users. Hence, the study aimed to investigate recently implanted CI users’ development of music-discrimination abilities in the early phase following implant switch-on applying both neurophysiological and behavioral measures. Considering previous findings, we hypothesized that recently implanted CI users would show significant development in their neural and behavioral discrimination of changes in intensity, pitch, and timbre after a period of approximately 3 months. By contrast, we expected that detection of changes in rhythm would be strong even shortly after CI activation and thus remain unaffected by the adaptation process. Moreover, based on previous reports of continued adaptation to the CI signal, we expected to see significant differences between recently implanted and experienced CI users, as reflected through smaller MMN amplitudes, longer latencies, and poorer behavioral discrimination accuracy for recently implanted CI users. The study may contribute to the general knowledge about music perception in CI users. It is critical to expand the research in this area since ameliorating CI users’ experience of music might have beneficial implications for their quality of life. Finally, gaining insight into the early development of music-discrimination abilities in CI users with the use of both neurophysiological and behavioral measures may have both clinical and therapeutic relevance.

Materials and Methods

Participants

Twelve recently implanted CI users (CIre) were recruited for the study. Due to technical problems and a coronavirus disease 2019 (COVID-19)-related cancelation, three participants were excluded, leading to a total number of nine (median age: 61 years; range: 30–85; two women). CIre participants were tested twice: within 6 weeks after switch-on of the implant (T1) and approximately 3 months after T1 (T2). However, because of logistical and individual issues, three participants had to postpone their T2 tests. CIre users had a median CI experience of 21 days at T1 (range: 3–42) and 118 days at T2 (range: 105–188). One CIre user was bilaterally implanted and eight were bimodally aided, that is, using a hearing aid on the side contralateral to their CI. The CIre users had a median duration of severe-profound deafness (defined as the period from the point of time at which an ipsilateral hearing aid was no longer beneficial to the time of CI switch-on) prior to CI of 2 years (range: 0–16). Seven participants reported being able to communicate over the phone (see Table 1 for details).

Table 1.

Demographic and Clinical Characteristics of CIre and CIex Participants.

a: Recently implanted cochlear implant (CI) users
Group ID	Age (T1)	Duration of deafness (years)	CI experience—T1 (days)	CI experience—T2 (days)	Days between T1 and T2	Side of implant	Hearing aid	Telephone ability	Device
CIre01	60–65	8	18	116	98	L	+	+	Oticon Medical
CIre02	70–75	3	19	111	92	R	+	+	Oticon Medical
CIre03	50–55	2	21	147	126	L	+	+	Cochlear
CIre04	60–65	16	10	105	95	L	+	−	Cochlear
CIre05	35–40	0	35	188	153	R	+	+	Cochlear
CIre06	30–35	1	22	112	90	R + L	−	+	Advanced Bionics
CIre07	80–85	9	3	177	174	L	+	−	Cochlear
CIre08	60–65	1	42	131	89	R	+	+	Cochlear
CIre09	50–55	2	35	118	83	R	+	+	Cochlear
Median Range	61 30–85	2 0–16	21 3–42	118 105–188	95 83–174
b: Experienced CI users
Group ID	Age	Duration of deafness (years)			CI experience (Years)	Side of implant	Hearing aid	Telephone ability	Device
CIex01	65–69	4			7	L	+	+	Cochlear
CIex02	45–50	0			10	L	−	+	Cochlear
CIex03	50–55	0			5	R	−	+	Cochlear
CIex04	55–60	2			9	R + L	−	+	Cochlear
CIex05	60–65	0			4	L	−	+	Cochlear
CIex06	60–65	3			14	R + L	−	+	Cochlear
CIex07	45–50	0			7	R	+	+	Cochlear
CIex08	65–70	0			8	R	−	−	Cochlear
CIex09	75–80	0			4	L	−	−	Cochlear
CIex10	35–40	1			1	R	+	+	Cochlear
CIex11	30–35	1			8	L	+	+	Cochlear
CIex12	18–25	0			1	R + L	−	+	Oticon Medical
CIex13	65–70	0			1	L	+	+	Oticon Medical
Median Range	56 18–77	0 0–4			7 1–14

Note. Duration of severe-profound deafness prior to CI is defined as the period from the point of time at which an ipsilateral hearing aid was no longer beneficial to the time of CI switch-on. Hearing aid: whether the participant wears a hearing aid on the ear contralateral to the implanted ear. Telephone ability: the participant reports ability to communicate via telephone.

Additionally, 13 experienced CI users (CIex; median age: 56 years; range: 18–77; nine women) and 14 adults with normal hearing (NH; median age: 62 years; range: 55–77; seven women) were included for reference and only tested once. Participants in the CIex group had a median CI experience of 7 years (range: 1–14). The CIex users had a median duration of severe-profound deafness prior to CI of 0 years (range: 0–4). Prior to the study, NH participants were required to pass an online hearing test that adaptively estimated a threshold for the perception of words and numbers in background noise¹.

To be included in the study, CI users had to meet the following criteria: >18 years of age, postlingual hearing loss (i.e., acquired profound hearing loss on the implanted ear developed after language acquisition), absence of neurological and severe psychological disorders, and no use of medication affecting brain function. The CI users were recruited in cooperation with the Department of Otorhinolaryngology, Head and Neck Surgery, Aarhus University Hospital and through the Danish CI users’ organization and their online platform.

Participants were informed about the study orally and in print before giving consent to participate. Participants were not compensated monetarily. Each participant was anonymized using a participant code (ID). The study was conducted in accordance with the Helsinki declaration, and it was approved by the Research Ethics Committee of the Central Denmark Region (#55018).

The study was part of a broader project which tested and validated the CI MuMuFe MMN paradigm, reported in Petersen et al. (2020). Importantly, data from all NH controls and 11 of the CIex participants are the same as those presented in the previous study (see Petersen et al., 2020). However, as the main focus in the present study is on the early development of CIre, the data from NH and CIex are merely included here as a reference for this study's CIre data. Participants were also presented with a free-listening EEG paradigm, presenting real music, which will be reported in a separate paper.

Stimuli

The CI MuMuFe MMN paradigm (Petersen et al., 2020) consists of an Alberti bass melody in which deviant notes are inserted to elicit MMN responses (Figure 1). It is a no-standard paradigm, meaning that it contains no melodic patterns without a deviant note. The deviant note always occurs on the third out of four notes in the melodic pattern, and it involves a change in either the intensity, pitch, timbre, or rhythm feature. On each of these four features the degree of deviation alternates between small (S), medium (M), large (L), and extra-large (XL) deviant magnitudes. For intensity deviants, the intensity of the note is reduced by 3 dB (S), 6 dB (M), 9 dB (L), or 12 dB (XL). Pitch deviants involve a lowering of the pitch by one (S), two (M), three (L), or eight (XL) semitones. The standard timbre is a grand-piano sound, whereas the deviant timbres are bright piano (S), blues piano (M), trumpet (L), or guitar (XL) sounds. For rhythm deviants, the second note is shortened by 26 ms (S), 52 ms (M), 103 ms (L), or 155 ms (XL) and the third note prolonged accordingly. The order of these deviants is pseudo-randomized. All notes in the Alberti bass melody have a duration of 200 ms (with 18-ms rise and fall) and a 5-ms silent interstimulus interval. The Alberti bass melody is played in four different keys (C, Eb, Gb, and A) with resulting notes spanning the range between Ab3 (208 Hz) and E5 (659 Hz). For further details regarding the CI MuMuFe MMN paradigm see Petersen et al. (2020).

Figure 1.

The CI MuMuFe no standards MMN paradigm with four deviant features at four levels of magnitude. Note. The paradigm is presented randomly in four different keys: C, Eb, Gb, and A major. Lowest note: Ab3 (208 Hz); highest note: E5 (659 Hz). S: small; M: medium; L: large; XL: extra-large. Reprinted from Petersen et al. (2020). MMN = mismatch negativity; MuMuFe = musical multifeature; CI = cochlear implant.

Procedure

The experiment was conducted at Aarhus University Hospital, Denmark. Participants were assessed using both EEG and a behavioral three-alternative forced-choice (3-AFC) task. For each session and for each participant, the sound level was individually adjusted to a comfortable level from a set starting point of 65 dB SPL. The sound was delivered bilaterally to NH participants through in-ear Shure headphones. For CI users, sound was transmitted unilaterally and directly to the implant through an audio connection with microphones switched off. This step was made to prevent the participants from using their residual hearing. Bilateral CI users were instructed to use their preferred implant, whereas bimodal CI users were instructed to remove their hearing aid. CI users were tested with their everyday processor settings. If the CI speech processors did not contain a direct audio input, the participant's personal maps were programmed onto a spare processor, which was used instead.

Electroencephalography

EEG was recorded in an acoustically shielded room with a BrainAmp amplifier system (Brain Products, Gilching, Germany) at a sampling rate of 1000 Hz. A 32-electrode cap was used, and electrodes were placed in accordance with the international 10/20 system. All electrode impedances were ensured to be <25 kΩ. To record the electrooculogram, electrodes 28 and 32 were placed beside and above the left eye. For online recording, the FCz was used as a reference electrode. The audio stimuli were presented at a 44.1 kHz sample rate during the EEG recording using the Presentation software (Neurobehavioral System, Berkeley, CA). All participants were instructed to ignore the audio stimuli and watch a subtitled movie with muted audio (more details available in Petersen et al., 2020). The duration of the EEG recording was approximately 35 min.

Behavioral Test

In addition to the EEG measurements, all participants were assessed with a 3-AFC task. Similar to the MMN paradigm, this test presented deviants in intensity, pitch, timbre, and rhythm embedded in a four-note Alberti bass pattern, thus making comparisons between the responses from both measures possible. In this test, participants were instructed to choose the odd one out of three melodic patterns. Thus, one of the three melodic patterns in each trial contained a deviant note whereas the two others did not contain a deviant and were the corresponding “standard” version of the deviant pattern. Each of the 16 deviant variants was presented six times (total n trials = 96). The order of the deviant variants as well as the location of the deviant (first, second, or third pattern) were randomized. The 3-AFC task was presented using the MACarena software (Lai & Dillier, 2002). The duration of the test was approximately 15 min.

Data from both measures were stored and handled in compliance with the European general data protection regulations.

EEG Preprocessing

The EEG data were preprocessed by following the same procedures as in Petersen et al. (2020) by applying the FieldTrip Toolbox for MATLAB (Oostenveld et al., 2010). Data were downsampled to 250 Hz and subsequently, a 1 Hz highpass filter was applied followed by a 25 Hz lowpass filter. Bad channels (range: 0–4 channels, showing excessive noise or no signal) were reconstructed by the standard method for weighted interpolation of neighboring channels implemented in FieldTrip. Eye and CI artifacts were suppressed with the infomax independent component analysis method (Delorme et al., 2007; Makeig et al., 1996), where independent components reflecting the artifacts (range: 1–10 components) were identified and subtracted. Subsequently, the continuous EEG was re-referenced to the average across all electrodes and segmented into standard trials (n = 6288) and deviant trials (n = 144 for each of the 16 deviant variants). For the intensity, pitch, and timbre deviants, a baseline correction was applied by subtracting the average signal between −100 and 0 ms prior to the note onset. For the rhythm deviants, a baseline correction was applied by subtracting the average signal between −100 and 0 ms prior to the onset of the preceding note, that is, the non-shortened note occurring before the actual rhythm deviant. Trials exceeding −/+100 μV were automatically rejected (range: 0%–5% trials), and average waveforms across trials were obtained.

Statistical Analyses

The MMN was analyzed at the Fz electrode. Due to the high variance in the MMN latency across the heterogeneous participant groups (spanning from approximately 100–250 ms), the MMN amplitude was measured as the mean amplitude across the entire 100–250 ms time window, which, as already mentioned, reflects the typical latency range for the MMN. This enabled more valid group comparisons than would have been achieved by calculating the MMN amplitude over a narrower time window. The individual MMN latency was automatically estimated as the most negative peak between 100 and 250 ms, and for the few cases where no negative peak could be estimated within 100–250 ms, latency was estimated as the most negative value between 100 and 250 ms (3 observations: 2 CIex and 1 NH, all pertaining to the XL level in the rhythm deviant). For each group, we tested whether each level of each deviant elicited significant MMN responses. As the amplitude data for some contrasts were not normally distributed, we conducted one-sample Wilcoxon signed-rank tests against 0 (see Supplemental Material for full reporting of these tests, including the Shapiro-Wilk tests).

Longitudinal Analysis

We used mixed-effects modeling to statistically assess the potential development in both behavioral performance in the 3-AFC task and the amplitude and latency of MMNs. This was done in the RStudio software (RStudio Team, 2021), an IDE for R (R Core Team, 2021), using the glmer and lmer functions, respectively, from the ‘lme4’ package (Bates et al., 2015). A hierarchical approach was used, and random intercepts for participants were included in all models to account for individual differences. Starting with a null model, independent variables were added incrementally, all as within-subject factors. These comprised deviant features (intensity, pitch, timbre, rhythm), deviant level (S, M, L, XL), and time (T1, T2), and all interactions between these (all three 2-way interactions and the full 3-way interaction). Models were fitted using maximum likelihood. Improvement of model fit was evaluated using likelihood ratio tests. Additionally, R² values are provided (using the “StatisticalModels” package; Newbold, 2015) as a tentative proxy for the variance explained by the model with the best fit, though we urge the reader to interpret these values with caution as R² values depend on several factors and not merely goodness of fit. For the MMN data, the window-averaged amplitudes and the peak latencies were used as the dependent variables. For the behavioral data, participants’ correct and incorrect responses were used as the dependent variable, incorporating a binomial response function.

Following any significant interaction effects involving time, pairwise contrasts were used to test whether the result of the time contrast between T1 and T2 differed between either deviant features or levels, depending on the nature of the interaction. Contrasts were carried out using the “emmeans” package (Lenth et al., 2021) in R. Confidence intervals were calculated using the Kenward–Roger approximated degrees of freedom and was used as they serve as both precision indicators and significance tests. For contrasts of the linear models, that is, those with MMN amplitude and latency as the dependent variables, contrasts are based on comparing the beta (β) estimates to 0, with intervals excluding 0 indicating a statistically significant result. For contrasts of the nonlinear models, that is, those concerning the performance of the 3-AFC task, contrasts are based on comparing the odds ratio to 1, with intervals excluding 1 indicating a statistically significant result. These post-hoc analyses were solely intended to explore any relevant significant main or interaction effects. Since we report on relatively subtle effects in this study, not all post-hoc analyses would be useful as explorations of the observed effects if we were to correct for multiple comparisons. Hence, we report uncorrected confidence intervals for these post-hoc tests.

Group Comparisons

In addition to the main analysis, a group analysis was carried out to compare CIre to the CIex and NH groups. As the reference data from CIex and NH did not include a longitudinal contrast (i.e., they were only tested once), we constrained the comparisons with CIre to the last of the two time points, namely T2. The logic behind this choice was that the contrast between CIre and the reference groups for both T1 and T2 would only be relevant if the corresponding contrast was significant between T1 and T2 in the longitudinal analysis for CIre.

The same modeling approach as described above was used for assessing potential group differences. However, the independent variables in these models comprised deviant feature, deviant level, and group (CIre, CIex, NH), and all interactions between these (all three 2-way interactions and the full 3-way interaction). Following any significant main or interaction effects involving group, pairwise contrasts were used to explore what group contrasts between CIre, CIex, and NH drove an effect and for interactions to test whether these group contrasts differed between either deviant features or levels, depending on the nature of the interaction. Hence, any significant interactions between deviant feature and deviant level were not explored any further, as this contrast was not of relevance to our hypotheses, but nonetheless modeled for full hierarchical transparency. The same approach was used to calculate the confidence intervals as in the longitudinal analysis.

For the linear mixed effects analyses (i.e., MMN amplitude and latency for both within- and between-group analyses), plots of all final model residuals were used to inspect whether models violated the assumptions of normality and homoscedasticity. No violations were detected.

For visualization, pirate plots were made using the “yarrr” package (Phillips, 2018) in R. MMN topographies and waveform plots were made with FieldTrip functions.

Results

Only significant results are reported in-text (all post-hoc contrasts in Tables 2–5); alpha level was set to 0.05. For a full list of model evaluation metrics see Supplemental Material. Results for MMN amplitudes are visualized in Figures 2–4. Results for MMN latencies and behavioral performance scores are visualized in Figures 5 and 6, respectively.

Figure 2.

Averaged MMN amplitudes within each feature for each group. Note. Averaged MMN amplitudes in μV within each deviant for all groups, including CIre at both T1 (CIre_T1; green) and T2 (CIre_T2; gray), CIex (orange), and NH (red). As the MMN elicits a negative peak, a stronger MMN response is reflected by a more negative μV value. MMN = mismatch negativity; NH = normally hearing; CIre = recently implanted cochlear implant users; CIex = experienced cochlear implant users.

Figure 3.

MMN topographies within each feature for each group. Note. Measured across the 100–250 ms measurement time window. MMN = mismatch negativity; NH = normally hearing; CIre = recently implanted cochlear implant users; CIex = experienced cochlear implant users.

Figure 4.

MMN waveforms within each feature for each group. Note. Showing deviant (colored) and standard (gray) waveforms at the Fz electrode. The shaded error bars show bootstrap 95% confidence intervals. The gray box indicates the 100–250 ms measurement time window. CIre_T1: green; CIre_T2: gray; CIex: orange; NH: red; MMN = mismatch negativity; NH = normally hearing; CIre = recently implanted cochlear implant users; CIex = experienced cochlear implant users.

Figure 5.

Averaged MMN latencies within each feature for each group. Note. Averaged MMN latencies in ms within each deviant for all groups, including CIre at both T1 (CIre_T1; green) and T2 (CIre_T2; gray), CIex (orange), and NH (red). Shorter latency indicates faster neural processing. MMN = mismatch negativity; NH = normally hearing; CIre = recently implanted cochlear implant users; CIex = experienced cochlear implant users.

Figure 6.

Averaged behavioral performance scores within each feature for each group. Note. Averaged correct in percentage within each deviant for all groups, including CIre at both T1 (CIre_T1; green) and T2 (CIre_T2; gray), CIex (orange), and NH (red). Chance level (33.33%) is marked with the dotted line. NH = normally hearing; CIre = recently implanted cochlear implant users; CIex = experienced cochlear implant users.

Table 2.

Longitudinal Analysis — Post-hoc Contrasts for MMN Amplitude.

MMN amplitude (µV)
Deviant (T1-T2)	β estimates	Standard error	Degrees of freedom	95% Confidence intervals
Intensity	−0.118	0.074	299	−0.263, 0.028
Pitch	0.161	0.074	299	0.016, 0.306	*
Timbre	0.161	0.074	299	0.016, 0.307	*
Rhythm	0.092	0.074	299	−0.054, 0.237

Note. Post-hoc contrasts for MMN amplitude between T1 and T2 for the CIre group. Significant results are marked with ‘*’. MMN = mismatch negativity.

Table 3.

Group Comparisons – Post-hoc Contrasts for MMN Amplitude.

MMN amplitude (µV)
Group contrast	β estimates	Standard error	Degrees of freedom	95% Confidence intervals
CIre – CIex	0.413	0.132	39.3	0.146, 0.680	*
CIre – NH	0.429	0.130	39.3	0.166, 0.692	*
CIex – NH	0.016	0.117	39.3	−0.221, 0.253
Within deviant (T1)
Pitch
CIre – CIex	0.548	0.159	59	0.231, 0.866	*
CIre – NH	0.567	0.156	59	0.254, 0.880	*
CIex – NH	0.019	0.141	59	−0.263, 0.301
Timbre			59
CIre – CIex	0.588	0.159	59	0.271, 0.906	*
CIre – NH	0.568	0.156	59	0.255, 0.881	*
CIex – NH	−0.020	0.141	59	−0.302, 0.261
Within deviant (T2)
Pitch
CIre – CIex	0.387	0.148	63.1	0.091, 0.683	*
CIre – NH	0.406	0.146	63.1	0.115, 0.698	*
CIex – NH	0.019	0.132	63.1	−0.244, 0.282
Timbre
CIre – CIex	0.427	0.148	63.1	0.131, 0.723	*
CIre – NH	0.407	0.146	63.1	0.115, 0.698	*
CIex – NH	−0.020	0.132	63.1	−0.283, 0.242

Note. Post-hoc contrasts for MMN amplitude between CIre, CIex and NH groups. First, the main effect is explored, that is, across deviants. Then, the findings from the longitudinal analysis are explored further, that is, group contrasts within pitch and timbre between CIre at T1, CIex, and NH as well as CIre at T2, CIex, and NH. Significant results are marked with ‘*’. MMN = mismatch negativity; NH = normally hearing; CIre = recently implanted cochlear implant users; CIex = experienced cochlear implant users.

Table 4.

Group Comparisons — Post-hoc Contrasts for MMN Latency.

MMN latency (ms)
Group contrast within deviant	β estimates	Standard error	Degrees of freedom	95% Confidence intervals
Intensity
CIre – CIex	18.56	7.96	168.23	2.84, 34.29	*
CIre – NH	13.33	7.85	168.23	−2.16, 28.82
CIex – NH	−5.23	7.07	168.23	−19.20, 8.73
Pitch
CIre – CIex	3.38	7.96	168.23	−12.34, 19.11
CIre – NH	24.43	7.85	168.23	8.94, 39.92	*
CIex – NH	21.04	7.07	168.23	7.08, 35.01	*
Timbre
CIre – CIex	19.78	7.96	168.23	4.06, 35.50	*
CIre – NH	23.42	7.85	168.23	7.93, 38.91	*
CIex – NH	3.64	7.07	168.23	−10.32, 17.61
Rhythm
CIre – CIex	18.93	7.96	168.23	3.21, 34.65	*
CIre – NH	44.21	7.85	168.23	28.72, 59.70	*
CIex – NH	25.27	7.07	168.23	11.31, 39.24	*

Note. Post-hoc contrasts for MMN latency between CIre, CIex, and NH groups within each deviant. Significant results are marked with “*.” MMN = mismatch negativity; NH = normally hearing; CIre = recently implanted cochlear implant users; CIex = experienced cochlear implant users.

Table 5.

Group Comparisons — Post-hoc Contrasts for Behavioral Performance Scores.

Behavioral performance (odds ratio)
Group contrast within deviant	Odds ratio	Standard error	95% Confidence intervals
Intensity
CIre \| CIex	0.700	0.072	0.571, 0.857	*
CIre \| NH	0.571	0.060	0.466, 0.701	*
CIex \| NH	0.817	0.074	0.683, 0.97	*
Pitch
CIre \| CIex	1.017	0.105	0.831, 1.244
CIre \| NH	0.747	0.078	0.609, 0.916	*
CIex \| NH	0.735	0.067	0.616, 0.878	*
Timbre
CIre \| CIex	0.889	0.092	0.727, 1.086
CIre \| NH	0.762	0.079	0.622, 0.935	*
CIex \| NH	0.858	0.078	0.719, 1.024
Rhythm
CIre \| CIex	0.950	0.097	0.777, 1.162
CIre \| NH	0.953	0.099	0.778, 1.168
CIex \| NH	1.003	0.091	0.840, 1.197

Note. Post-hoc contrasts for behavioral performance between CIre, CIex, and NH groups within each deviant. As nonlinear models were used, contrasts are reflected through the odds ratio and confidence intervals are asymptotic. Significant results are marked with “*.” NH = normally hearing; CIre = recently implanted cochlear implant users; CIex = experienced cochlear implant users.

Wilcoxon signed-rank tests revealed significant MMN responses for CIre at T2 for pitch (S, M, and L) and timbre (S, M, L, and XL). For CIex and NH, MMN responses were significant for all levels within all deviants (see the Supplemental Material for full reporting of these results).

Longitudinal Analysis

MMN Amplitude

With outset in the null model, the likelihood ratio tests revealed a significant improvement of model fit by adding deviant feature (χ²(3) = 10.3, p = .016) and time (χ²(1) = 3.93, p = .047). The likelihood ratio tests further revealed that the model including the interaction between deviant feature and time was the model that fitted the data the best (χ²(3) = 10.13, p = .018), explaining 50% of the variance (R² conditional = .501) with the fixed effects explaining 8% of the variance (R² marginal = .080). Follow-up contrasts of the interaction between deviant feature and time based on the estimated marginal means of the best model projected the MMN amplitude to decrease (i.e., grow larger) by 0.161 µV (SE = 0.074) from T1 to T2 for pitch and by 0.161 µV (SE = 0.074) for timbre (Table 2 and Figure 2).

MMN Latency

None of the models explained the data better than the null model, indicating no significant differences in latencies between the different deviant features, between the different deviant levels, or between the two time points.

Behavioral Performance Scores

With outset in the null model, the likelihood ratio tests revealed a significant improvement of model fit by adding deviant feature (χ²(3) = 160.2, p < .001) and deviant level (χ²(3) = 66.97, p < .001). The model that fitted the data the best, including deviant feature and deviant level, explained 15% of the variance (R² conditional = .145) with the fixed effects explaining 13% of the variance (R² marginal = .125). However, no effect of time was identified, indicating no significant development in performance over time in the behavioral 3-AFC task.

Group Comparisons

All group contrasts involving CIre were based on measurements from T2, except for separate follow-up analyses of MMN amplitude for pitch and timbre at both T1 and T2 given the results from the longitudinal analysis. All contrasts for all three dependent variables are reported in Tables 3–5.

MMN Amplitude

With outset in the null model, the likelihood ratio tests revealed a significant improvement of model fit by adding the deviant feature (χ²(3) = 40.72, p = .001), deviant level (χ²(3) = 13.26, p = .004), and group (χ²(2) = 11.90, p = .003). The likelihood ratio tests further revealed that the model including the interaction between deviant feature and deviant level was the model that fitted the data the best (χ²(9) = 37.53, p < .001), explaining 52% of the variance (R² conditional = .515) with the fixed effects explaining 21% of the variance (R² marginal = .214). However, as mentioned in the “Methods” section, only interactions involving group were further explored. Follow-up contrasts based on the estimated marginal means of the best model projected the CIex' MMN amplitudes to be 0.413 µV (SE = 0.132) more negative than CIre's and the NH's MMN amplitudes to be 0.429 µV (SE = 0.130) more negative than CIre's; Table 3).

As the longitudinal analysis showed significant development in size of the MMN amplitude to pitch and timbre from T1 to T2, we performed a follow-up analysis to test these two deviant features specifically between CIre and the reference groups (CIex and NH) at both T1 and T2. These follow-up contrasts revealed significant differences between CIre and the reference groups for pitch and timbre at both T1 and T2 (Table 3 and Figure 2).

MMN Latency

With outset in the null model, the likelihood ratio tests revealed a significant improvement of model fit by adding deviant feature (χ²(3) = 9.14, p = .027) and group (χ²(2) = 19.58, p < .001). The likelihood ratio test further revealed a significant improvement in model fit by adding the interaction between the deviant feature and deviant level (χ²(9) = 24.28, p = .004) and the interaction between the deviant feature and group (χ²(6) = 23.65, p < .001). The model that fitted the data the best, including both aforementioned two-way interactions, explained 25% of the variance (R² conditional = .247) with the fixed effects explaining 17% of the variance (R² marginal = .173). Since we focused on the interactions involving group, we only report follow-up contrasts for the interaction between deviant feature and group. These follow-up contrasts based on the estimated marginal means of the best model projected the latencies to be significantly longer for CIre compared to CIex in intensity (18.56 ms, SE = 7.96), timbre (19.78 ms, SE = 7.96), and rhythm (18.93 ms, SE = 7.96). Additionally, latencies were projected to be significantly longer for CIre compared to NH in pitch (24.43 ms, SE = 7.85), timbre (23.42 ms, SE = 7.85), and rhythm (44.21 ms, SE = 7.85). Finally, latencies were projected to be significantly longer for CIex compared to NH in pitch (21.04 ms, SE = 7.07) and rhythm (25.27 ms, SE = 7.07; for an overview, see Table 4 and Figure 5)

Behavioral Performance Scores

With the outset in the null model, the likelihood ratio tests revealed a significant improvement of model fit by adding the deviant feature (χ²(3) = 142.50, p<.001), deviant level (χ²(3) = 131.51, p < .001), and group (χ²(2) = 14.30, p < .001). The likelihood ratio test further revealed a significant improvement in model fit by adding the interaction between deviant feature and deviant level (χ²(9) = 30.57, p < .001) and the interaction between deviant feature and group (χ²(6) = 22.71, <.001). The model that fitted the data the best, including both aforementioned two-way interactions, explained 12% of the variance (R² conditional = .117) with the fixed effects explaining 10% of the variance (R² marginal = .102). Since we focused on the interactions involving group, we only report follow-up contrasts for the interaction between deviant feature and group. These follow-up contrasts based on the estimated marginal means of the best model projected CIre to be 0.70 (SE = 0.07) times as likely to answer correctly for intensity as CIex and compared to NH 0.57 (SE = 0.06) as likely for intensity, 0.75 (SE = 0.08) as likely for pitch and 0.76 (SE = 0.08) as likely for timbre. Additionally, CIex were projected to be 0.82 (SE = 0.07) times as likely as NH to answer correctly for intensity and 0.74 (SE = 0.07) as likely for pitch (see Table 5 and Figure 6).

Discussion

This study investigated the early development in discrimination of prominent features of music in recently implanted CI users, as measured both neurophysiologically (using the MMN response) and behaviorally. In line with our hypotheses, the neurophysiological results showed a significant improvement in the discrimination of pitch and timbre, as indicated by increased MMN amplitudes. This development was further underpinned by significant MMN responses elicited by these features at the T2 endpoint. Contrary to our hypotheses, the progress was not reflected in the behavioral results. The neurophysiological results align with previous findings suggesting that improvement in postlingually deafened adult CI users’ auditory discrimination ability takes place in the very early stages after switch-on of the implant (Chang et al., 2010; Petersen et al., 2013; Sandmann et al., 2015). Furthermore, it reflects the hearing-deprived brain's ability to “learn” to decode the artificial and novel sounds transmitted by the implant, even when presented in a musically complex context.

As hypothesized, the CIre group showed significantly lower amplitudes in the MMN response across deviants compared to the CIex and NH groups. At a more specific level, amplitudes for CIre were significantly lower than those of CIex and NH for pitch and timbre at both T1 and T2, reflecting that, despite the marked development, the T2 amplitudes still did not match those of the reference groups. Both the CIex and the NH groups exhibited significant MMN responses to all deviants. Furthermore, no significant difference was found in amplitude between CIex and NH, suggesting that the neural adaptation not only occurs in the initial phase after switch-on but is a long-term process.

As expected, CIre performed on par with both CIex and NH in their behavioral discrimination accuracy of the rhythm deviant. By contrast, the CIre group scored significantly lower in intensity compared to CIex and in intensity, pitch and timbre compared to NH. Interestingly, and notwithstanding expectations, no significant differences were found between CIre and CIex in the discrimination of pitch and timbre.

Neural vs Behavioral Responses

The significant change over time in the CIre's neural discrimination of pitch and timbre did not reflect itself in the behavioral results. Already at T1, the mean CIre performance rate was well above the chance level (Figure 6), and at T2 it was not significantly different from the mean performance rate of CIex (Table 5). In other words, the CIre group exhibited a general behavioral discrimination accuracy at the early stage that did not leave much room for improvement. This is quite surprising and evidence of both the efficacy of the present-day speech-processing technology and the speed with which behavioral adaptation to the CI takes place after switch-on.

When considering the apparent inconsistency between MMN responses and behavioral performances, it is important to note that the MMN response constitutes a preattentive and early part of the discrimination process, whereas the behavioral discrimination accuracy constitutes the final product of said discrimination process and involves higher-order processing (Bishop & Hardiman, 2010). Moreover, the 3-AFC task involves a deliberate choice which implies that the participant may be able to detect “a change” when forced to make a decision. Thereby, the level of sensitivity reflected by these two measures may not be directly comparable.

Generally, the MMN has been associated with successful behavioral discrimination in NH listeners and some clinical populations (Näätänen et al., 2007). On a related point, Mathew et al. (2017), found correspondence between behavioral electrode discrimination and objective auditory change complex responses in recently implanted CI users. However, several papers have also reported on MMN and behavioral results that fail to correspond (Dalebout & Fox, 2000; Kurtzberg et al., 1995; Ortmann et al., 2017; Uwer & von Suchodoletz, 2000). Thus, significant MMN responses do not preclude poor behavioral performances, and likewise, nonsignificant MMN responses do not preclude good behavioral performances (Bishop & Hardiman, 2010).

The relative inconsistency between MMN and behavioral results was also reflected in the results of the CIex participants, showing MMN amplitudes that were not significantly different from those of NH, while performing significantly poorer in their behavioral discrimination of intensity and pitch. This finding was also reported in Petersen et al. (2020), which gave rise to the idea of a neural precursor to behavioral discrimination ability. However, findings from the present study do not corroborate this hypothesis. In fact, our results indicate that the ability to behaviorally discriminate between features of music is established at an earlier stage of the CI adaptation process than is the case for neural discrimination.

At a more speculative point, the apparent inconsistency between MMN and behavioral performance might also be a result of the differences in the applied measurements. A person suffering from years of hearing deprivation is trained in making sense of the small amounts of sounds that are available. This demands a huge effort and therefore may create a high degree of “sensory” alertness (e.g., facilitated by attentional gain, Scolari & Serences, 2009). Thus, when faced with the task of being forced to choose which pattern is different from the other two patterns, as required in the behavioral test, this alertness may be advantageous—even when partly guessing. These subtle cues in a degraded sound input that may be detected behaviorally are likely paralleled by relatively weak neuronal signals that are in the process of adapting to the novel sound input and thus hard to detect at the scalp level, potentially leading to the observed discrepancy. However, more studies following larger pools of recently implanted CI users longitudinally is warranted to establish a more refined understanding of the time course and the nature of the development of musical discrimination abilities after implantation.

Adapting to Spectral Complexity

CIre's development was exclusively observed in the neural discrimination of pitch and timbre features. This might imply that the auditory training associated with the adaptation process is exhibited most strongly in the discrimination of spectrally complex characteristics of music. For pitch, the reason could lie in the nature of the paradigm. The MMN indexes an error in the predictive coding of the environment, for example, when a deviation occurs in a regular pattern of standard stimuli (Näätänen et al., 2001). Thus, according to the predictive coding theory (Friston, 2005; Vuust & Witek, 2014; Vuust et al., 2022), if one cannot neurally generate the predictive model, one is unable to neurally detect the deviations from that model. The Alberti bass pattern is based on alternating pitches. Because the sound from the implant has a degraded representation of temporal fine-structure information, the brain may at first be unable to distinguish the spectral signals embedded in the pattern. However, after a short period of acclimatization, the auditory system may have grown so accustomed to the sound from the CI that the changes in pitch are more easily detected. Therefore, the results may be a testimony to the fact that the CIre group is becoming increasingly able to neurally discriminate the pattern and therefore also the deviations (Quiroga-Martinez et al., 2019).

Unexpectedly, T2 MMN responses were significant for the S, M, and L but not for the XL 8- semitone-change. A similar phenomenon was observed in our previous study, in which the experienced CI users showed a large variance in their behavioral identification of the XL deviant, exhibiting floor as well as ceiling effects (Petersen et al., 2020). One explanation for this apparent inconsistency could be a poor representation of the low frequency of the deviant note. In the key of C, the XL deviant is a sounding Ab which corresponds to an F0 of 208 Hz. Apart from the single participant who used a device from Advanced Bionics, which has a low-frequency cut-off at 250 Hz, this tone is not outside of the frequency range of the devices used by the CIre participants. Both the Oticon Medical and the Cochlear devices transmit as low as 188 Hz. Nonetheless, since the tone consists of a range of frequencies, some of which may range lower than the cut-off, this low-frequency energy may not be represented properly. This could lead to the loss of cues that may be important for perception. Also, the standard discrimination tests rarely measure below 250 Hz, indicating that stimulation below this threshold is not commonly adjusted, leading to uncertainty in how well these low frequencies are represented. Taken together, the potential variance in low-frequency representation, as well as the general sensitivity of the MMN response, may have caused the unexpected null result.

Timbre deviants typically show robust MMN responses in experienced CI users (Hahne et al., 2016; Petersen et al., 2015; Timm et al., 2014), indicating that detection of changes in spectral energy is possible with a CI. Hence, the strong progress observed in the recently implanted CI users’ automatic detection of the timbre deviants could be due to a gradual acclimatization to the richer representation of higher frequencies in the activation of electrodes coupled with a gradual normalization of the presumed “frequency mismatch.” The latter phenomenon is caused by the discrepancy between the stimulated frequencies and the location in the cochlea, for example, low frequencies stimulating neurons that were naturally more responsive to higher frequencies before cochlear implantation (Glennon et al., 2020).

Intensity

Contrary to our hypotheses, no significant development was seen in the CIre group's discrimination of the intensity feature. The simple explanation for this may be that they are unable to detect the changes, as indicated by nonsignificant MMNs as well as near chance level behavioral performances (33.3%) at both T1 and T2 (Figure 6). We speculate that heavy automatic gain control at this early stage levels out these changes and thereby impedes possible distinction.

That being said, it is also important to note that across groups, both behavioral and neurophysiological measures indicate that the intensity deviants were more difficult to discriminate as compared to the other deviants. Inspection of the MMN waveforms (Figure 4) reveals that the peak for NH participants is less prominent in the intensity deviant compared to the other deviants. Additionally, the behavioral data show that the mean NH performance score across levels is only around 60% and at near chance level performance for CIre (Figure 6). Increasing the magnitudes of the levels of the intensity feature in a revision of the paradigm may deem more useful, especially when it comes to tracking the early adaptation process following cochlear implantation.

Rhythm

As predicted, we found no development in the discrimination of the rhythm feature. There seems, however, to be some incongruence between the neurophysiological and behavioral results. While CIre performed on par with both CIex and NH in the behavioral discrimination task, their MMN responses were weak in amplitude at both T1 and T2. However, looking at the topographies (Figure 3), the averaged MMN amplitudes for the rhythm deviant are also weaker for the CIex and NH groups in comparison to the other three deviants. This may be explained by the wide window of 100–250 ms used for the detection of negative peaks, adding some noise to the signal, as compared to using a narrower time window. However, because the main focus of the study was on the potential progress between T1 and T2 in the recently implanted CI users, this wide window was necessary in order to capture the peaks from all groups, acknowledging that this might be suboptimal for NH and CIex.

On a more general note, as also discussed in the work of Petersen et al. (2020), the rhythm deviant differs as it contains two deviants in one (i.e., shortening the second note and presenting the third note earlier than expected). Consequently, the P50 response occurs closer to the onset of the third note, contaminating the baseline correction (see Petersen et al., 2020 for a more elaborated discussion). Taking these different shortcomings of the MMN paradigm with regard to the rhythm deviant into account, we might consider testing alternative electrophysiological measures of rhythm perception. In that respect, the frequency tagging approach may provide interesting information (Nozaradan, 2014). To the best of our knowledge, this approach has not yet been applied to CI users.

Latency

We found no significant change over time in the latencies of the MMN responses. Since latency measures for nonsignificant MMNs should be interpreted with caution, we focus on the results pertaining to those deviant features that elicited significant MMNs for CIre at T2. CIre showed significantly longer latencies in MMN responses to timbre compared to CIex and to both pitch and timbre compared to NH. Longer latencies have been coupled to weaker and less precise neural responses (Kujala & Näätänen, 2010), which again supports the notion of a reorganization process in CIre extending beyond the initial phase.

Limitations

Group Size

This study included a relatively small cohort of CI users, which was mainly caused by the difficulty associated with recruitment within a patient population limited in numbers. Moreover, the duration of the experiment (+2 h) may have deterred some from signing up. Finally, CIre participants were required to come in for two rounds of testing, the first being very shortly after switch-on, which for some patients may have been too overwhelming. A larger cohort of CI users would have increased the statistical power and thereby the robustness of our findings.

Maps

Another important consideration is that the programming of the CI users’ speech processor settings is configured at an individual level, making it difficult to account for. Typically, in the period following the initial processor set-up, the patient will have the option to select between programs with increasing levels of loudness, as the patient is accommodating to the novel sound sensations. This, of course, might again influence perception. However, basic psychophysics such as spectral and temporal resolution are relatively stable from the day of activation to later (Drennan et al., 2016), while speech perception continues to improve, as it relies more on central processes. The primary effect it would have would be in intensity, which does not seem to have been affected in the current case. It is, however, important to note that during testing, the sound level was adjusted according to each participant's subjective liking, which again may have evened out some of the differences in loudness imposed by the programming.

To which degree this factor might influence MMN responses is unclear. Yet, Sandmann et al. (2015) found no significant correlation between stimulation levels and N1 amplitude and latency, indicating that the individual mappings did not explain their findings concerning early neural development following cochlear implantation.

Individual Variance

Both the recently implanted and the experienced CI users’ behavioral performances were characterized by a large amount of variation, spanning from 100% correct scores in some cases to near or below chance level performance in others. This high individual variation in performance is often seen in CI-research and may reflect differences in the patients’ history of hearing loss, such as duration and etiology of deafness, age at hearing loss, and residual hearing (Blamey et al., 2013).

Contralateral Hearing Aids

Eight of nine CIre participants used a hearing aid on the contralateral ear, indicating the existence of varying degrees of residual hearing. This could mean that fewer maladaptive changes have taken place prior to receiving the cochlear implant (CI), or at least these would primarily have happened in the auditory cortical area contralateral to the implant. Thus, as opposed to profoundly deaf CI recipients, the involved participants might have benefitted from a reduced requirement of reorganization after implantation.

Direct Audio Input vs. Free Field

In the current investigation, CI users were exposed to sound stimuli via a direct audio input cable rather than loudspeakers. This provides for more control over which sound sources are delivered to participants, as well as the elimination of confounding factors such as residual hearing. Some challenges, however, are associated with this method. Some of the CI users had bilateral CIs and were required to do the tests using their highest functioning ear. Both for them and the bimodal listeners, the monaural stimulation constituted a less satisfying and unfamiliar listening condition, meaning that the brain has not adapted to this specific way of listening. We can only speculate on how much this influenced their performance. We would suggest that, while there is a trade-off in directly presenting the stimuli, it offers the most appropriate base for a fair comparison and standardizes one component in a highly heterogeneous population.

Summary and Conclusion

Our findings confirm and expand the previous reporting on postlingually deafened CI-users’ music discrimination abilities, showing a marked improvement in the neural discrimination of music in the very early stages after switch-on of the implant. Furthermore, the significant differences between the experienced and the recently implanted CI users illustrate the long-term functional cortical changes taking place, following experience and everyday exposure to sounds. The findings also imply that the auditory training associated with the adaptation process is exhibited most strongly in the discrimination of spectrally complex characteristics of music. Finally, the results indicate that ability to behaviorally discriminate changes in pitch, timbre, and rhythm is notable already short after initial processor set-up.

Given the limited research including recently implanted CI users, this study is an original contribution to the area of music perception in CI users. Increasing the body of knowledge in this area is important as improving music perception for CI users can have implications for their quality of life. With the early development of discrimination abilities in CI users measured both behaviorally and using EEG, the study provides insights into just how early the cortical changes occur, which may be of clinical and therapeutic relevance.

Supplemental Material

sj-pdf-1-tia-10.1177_23312165221148035 - Supplemental material for Adapting to the Sound of Music — Development of Music Discrimination Skills in Recently Implanted CI Users

Supplemental material, sj-pdf-1-tia-10.1177_23312165221148035 for Adapting to the Sound of Music — Development of Music Discrimination Skills in Recently Implanted CI Users by Alberte B. Seeberg, Niels T. Haumann, Andreas Højlund, Anne S. F. Andersen, Kathleen F. Faulkner, Elvira Brattico, Peter Vuust and Bjørn Petersen in Trends in Hearing

Supplemental Material

sj-pdf-2-tia-10.1177_23312165221148035 - Supplemental material for Adapting to the Sound of Music — Development of Music Discrimination Skills in Recently Implanted CI Users

Supplemental material, sj-pdf-2-tia-10.1177_23312165221148035 for Adapting to the Sound of Music — Development of Music Discrimination Skills in Recently Implanted CI Users by Alberte B. Seeberg, Niels T. Haumann, Andreas Højlund, Anne S. F. Andersen, Kathleen F. Faulkner, Elvira Brattico, Peter Vuust and Bjørn Petersen in Trends in Hearing

Footnotes

Acknowledgments

The authors wish to thank all the participants for their unrestricted commitment. They also thank Franck Michel for providing the programing of replacement sound processors, Monica Ipsen for her great effort in the recruitment of participants, organizing tests, and collection of data. Special thanks to Dora Grauballe, Center for Functionally Integrative Neuroscience, for invaluable and unconditioned help with bookings and management of laboratory facilities and to Christopher Bailey for technical assistance.

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Kathleen F. Faulkner was employed by company Oticon Medical. The authors declare that this study received funding from Oticon Medical. The funder had the following involvement in the study: idea for and contribution to the design of the study.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Alberte B. Seeberg's, Anne Sofie Friis Andersen's and Niels Trusbak Haumann's contributions to the project were partly funded by Oticon Medical, Denmark, and partly by the Center for Music in the Brain. The Center for Music in the Brain is funded by the Danish National Research Foundation (DNRF117).

Data Availability

The codes for the statistical analyses are available on GitHub (). Anonymized data can be made available upon reasonable request. Pseudonymized data will require a data-sharing agreement. For inquiries or requests contact corresponding author Alberte B. Seeberg (abseeberg@clin.au.dk).

Ethical Approval

The study was conducted in accordance with the Helsinki declaration, and it was approved by the Research Ethics Committee of the Central Denmark Region (#55018).

ORCID iDs

Alberte B. Seeberg

Andreas Højlund

Supplemental Material

Supplemental material for this article is available online.

Notes

References

Bates

Maechler

Bolker

Walker

(2015). Fitting linear mixed-effects models using {lme4}. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01

Bishop

D. V. M.

Hardiman

M. J.

(2010). Measurement of mismatch negativity in individuals: A study using single-trial analysis. Psychophysiology, 47(4), 697–705. https://doi.org/10.1111/j.1469-8986.2009.00970.x

Blamey

Artieres

Başkent

Bergeron

Beynon

Burke

Dillier

Dowell

Fraysse

Gallégo

Govaerts

P. J.

Green

Huber

A. M.

Kleine-Punte

Maat

Marx

Mawman

Mosnier

O’Connor

A. F.

, … Lazard

D. S

. (2013). Factors affecting auditory performance of postlinguistically deaf adults using cochlear implants: An update with 2251 patients. Audiology & Neuro-Otology, 18(1), 36–47. https://doi.org/10.1159/000343189

Brockmeier

S. J.

Fitzgerald

Searle

Fitzgerald

Grasmeder

Hilbig

Vermiere

Peterreins

Heydner

Arnold

(2011). The MuSIC perception test: A novel battery for testing music perception of cochlear implant users. Cochlear Implants International, 12(1), 10–20. https://doi.org/10.1179/146701010X12677899497236

Bruns

Mürbe

Hahne

(2016). Understanding music with cochlear implants. Scientific Reports, 6, 32026. https://doi.org/10.1038/srep32026

Cai

Liu

Yao

Zhang

(2020). A neurophysiological study of musical pitch identification in Mandarin-speaking cochlear implant users. Neural Plasticity, 2020, e4576729. https://doi.org/10.1155/2020/4576729

Carcea

Froemke

R. C.

(2013). Chapter 3—cortical plasticity, excitatory–inhibitory balance, and sensory perception. In Merzenich

M. M.

Nahum

Van Vleet

T. M.

(Eds.), Progress in brain research (Vol. 207, pp. 65–90). Elsevier. https://doi.org/10.1016/B978-0444-63327-9.00003-5

Chang

S.-A.

Tyler

Dunn

Witt

Gantz

Hansen

(2010). Performance overtime on adults with simultaneous bilateral cochlear implants. Journal of the American Academy of Audiology, 21(1), 35–43. https://doi.org/10.3109/14992020903095791

Cooper

W. B.

Tobey

Loizou

P. C.

(2008). Music perception by cochlear implant and normal hearing listeners as measured by the Montreal battery for evaluation of Amusia. Ear and Hearing, 29(4), 618–626. https://doi.org/10.1097/AUD.0b013e318174e787

10.

Dalebout

S. D.

Fox

L. G.

(2000). Identification of the mismatch negativity in the responses of individual listeners. Journal of the American Academy of Audiology, 11(1), 12–22. https://doi.org/10.1055/s-0042-1748004

11.

Delorme

Palmer

Oostenveld

Onton

Makeig

(2007). Comparing Results of Algorithms Implementing Blind Source Separation of EEG Data. 5.

12.

Drennan

W. R.

Won

J. H.

Timme

A. O.

Rubinstein

J. T.

(2016). Non-linguistic outcome measures in adult cochlear implant users over the first year of implantation. Ear and Hearing, 37(3), 354–364. https://doi.org/10.1097/AUD.0000000000000261

13.

Friston

(2005). A theory of cortical responses. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 360(1456), 815–836. https://doi.org/10.1098/rstb.2005.1622

14.

Fuller

Free

Maat

Başkent

(2021). Self-reported music perception is related to quality of life and self-reported hearing abilities in cochlear implant users. Cochlear Implants International, 1–10. https://doi.org/10.1080/14670100.2021.1948716

15.

Gfeller

(2016). Music-based training for pediatric CI recipients: A systematic analysis of published studies. European Annals of Otorhinolaryngology, Head and Neck Diseases, 133(Suppl 1), S50–S56. https://doi.org/10.1016/j.anorl.2016.01.010

16.

Gfeller

Christ

Knutson

J. F.

Witt

Murray

K. T.

Tyler

R. S.

(2000). Musical backgrounds, listening habits, and aesthetic enjoyment of adult cochlear implant recipients. Journal of the American Academy of Audiology, 11, 390–406. https://doi.org/10.1055/s-0042-1748126

17.

Gfeller

Guthe

Driscoll

Brown

C. J.

(2015). A preliminary report of music-based training for adult cochlear implant users: Rationales and development. Cochlear Implants International, 16(Suppl 3), S22–S31. https://doi.org/10.1179/1467010015Z.000000000269

18.

Gfeller

Witt

Adamek

Mehr

Rogers

Stordahl

Ringgenberg

(2002). Effects of training on timbre recognition and appraisal by postlingually deafened cochlear implant recipients. Journal of the American Academy of Audiology, 13(3), 132–145. https://doi.org/10.1055/s-0040-1715955

19.

Glennon

Svirsky

M. A.

Froemke

R. C.

(2020). Auditory cortical plasticity in cochlear implant users. Current Opinion in Neurobiology, 60, 108–114. https://doi.org/10.1016/j.conb.2019.11.003

20.

Hahne

Mainka

Leuner

Mürbe

(2016). Adult Cochlear Implant Users Are Able to Discriminate Basic Tonal Features in Musical Patterns. Otology & Neurotology, 37(9), e360–e368. http://doi.org/10.1097/MAO.0000000000001067

21.

Ito

Momose

Oku

Ishimoto

Yamasoba

Sugasawa

Kaga

(2004). Cortical activation shortly after cochlear implantation. Audiology and Neurotology, 9(5), 282–293. https://doi.org/10.1159/000080228

22.

Jiam

N. T.

Caldwell

M. T.

Limb

C. J.

(2017). What does music sound like for a cochlear implant user? Otology & Neurotology, 38(8), e240. https://doi.org/10.1097/MAO.0000000000001448

23.

Jiam

N. T.

Deroche

M. L.

Jiradejvong

Limb

C. J.

(2019). A randomized controlled crossover study of the impact of online music training on pitch and timbre perception in cochlear implant users. JARO: Journal of the Association for Research in Otolaryngology, 20(3), 247–262. https://doi.org/10.1007/s10162-018-00704-0

24.

Johnston

M. V.

(2009). Plasticity in the developing brain: Implications for rehabilitation. Developmental Disabilities Research Reviews, 15(2), 94–101. https://doi.org/10.1002/ddrr.64

25.

Kliuchko

Heinonen-Guzejev

Vuust

Tervaniemi

Brattico

(2016). A window into the brain mechanisms associated with noise sensitivity. Scientific Reports, 6(1), 39236. https://doi.org/10.1038/srep39236

26.

Kolb

Gibb

Robinson

T. E.

(2003). Brain plasticity and behavior. Current Directions in Psychological Science, 12(1), 1–5. https://doi.org/10.1111/1467-8721.01210

27.

Kujala

Näätänen

(2010). The adaptive brain: A neurophysiological perspective. Progress in Neurobiology, 91(1), 55–67. https://doi.org/10.1016/j.pneurobio.2010.01.006

28.

Kujala

Tervaniemi

Schröger

(2007). The mismatch negativity in cognitive and clinical neuroscience: Theoretical and methodological considerations. Biological Psychology, 74(1), 1–19. https://doi.org/10.1016/j.biopsycho.2006.06.001

29.

Kurtzberg

Vaughan

H. G.

Kreuzer

J. A.

Fliegler

K. Z.

(1995). Developmental studies and clinical application of mismatch negativity: Problems and prospects. Ear and Hearing, 16(1), 105–117. https://doi.org/10.1097/00003446-199502000-00008

30.

Lai

Dillier

(2002). MACarena: A flexible computer-based speech testing environment. 7th International Cochlear Implant Conference, Manchester. http://www.uzh.ch/orl/projects/speechtests/speechtests.html

31.

Lenth

R. V.

Buerkner

Herve

Love

Riebl

Singmann

(2021). emmeans: Estimated Marginal Means, aka Least-Squares Means (1.7.0) [Computer software]. https://CRAN.R-project.org/package=emmeans

32.

Limb

C. J.

Roy

A. T.

(2014). Technological, biological, and acoustical constraints to music perception in cochlear implant users. Hearing Research, 308, 13–26. https://doi.org/10.1016/j.heares.2013.04.009

33.

Looi

Gfeller

Driscoll

V. D.

(2012). Music appreciation and training for cochlear implant recipients: A review. Seminars in Hearing, 33(4), 307–334. https://doi.org/10.1055/s-0032-1329222

34.

Makeig

Bell

Jung

T.-P.

Sejnowski

T. J.

(1996). Independent Component Analysis of Electroencephalographic Data. Advances in Neural Information Processing Systems, 8. https://proceedings.neurips.cc/paper/1995/hash/754dda4b1ba34c6fa89716b85d68532b Abstract.html

35.

Mathew

Undurraga

Meerton

Boyle

Shaida

Selvadurai

Jiang

Vickers

(2017). Objective assessment of electrode discrimination with the auditory change complex in adult cochlear implant users. Hearing Research, 354, 86–101. https://doi.org/10.1016/j.heares.2017.07.008

36.

Mirza

Douglas

S. A.

Lindsey

Hildreth

Hawthorne

(2003). Appreciation of music in adult patients with cochlear implants: A patient questionnaire. Cochlear Implants International, 4(2), 85–95. https://doi.org/10.1179/cim.2003.4.2.85

37.

Moore

D. R.

Shannon

R. V.

(2009). Beyond cochlear implants: Awakening the deafened brain. Nature Neuroscience, 12(6), 686–691. https://doi.org/10.1038/nn.2326

38.

Moreau

Jolicœur

Peretz

(2013). Pitch discrimination without awareness in congenital amusia: Evidence from event-related potentials. Brain and Cognition, 81(3), 337–344. https://doi.org/10.1016/j.bandc.2013.01.004

39.

Näätänen

Paavilainen

Rinne

Alho

(2007). The mismatch negativity (MMN) in basic research of central auditory processing: A review. Clinical Neurophysiology: Official Journal of the International Federation of Clinical Neurophysiology, 118(12), 2544–2590. https://doi.org/10.1016/j.clinph.2007.04.026

40.

Näätänen

Petersen

Torppa

Lonka

Vuust

(2017). The MMN as a viable and objective marker of auditory development in CI users. Hearing Research, 353, 57–75. https://doi.org/10.1016/j.heares.2017.07.007

41.

Näätänen

Tervaniemi

Sussman

Paavilainen

Winkler

(2001). ‘Primitive intelligence’ in the auditory cortex. Trends in Neurosciences, 24(5), 283–288. https://doi.org/10.1016/S0166-2236(00)01790-2

42.

Newbold

(2015). StatisticalModels (0.1) [Computer software]. https://rdrr.io/github/timnewbold/StatisticalModels/

43.

Nozaradan

(2014). Exploring how musical rhythm entrains brain activity with electroencephalogram frequency-tagging. Philosophical Transactions of the Royal Society B: Biological Sciences, 369(1658). https://doi.org/10.1098/rstb.2013.0393

44.

Oostenveld

Fries

Maris

Schoffelen

J.-M.

(2010). Fieldtrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Computational Intelligence and Neuroscience, 2011, e156869. https://doi.org/10.1155/2011/156869

45.

Ortmann

Zwitserlood

Knief

Baare

Brinkheetker

am Zehnhoff-Dinnesen

Dobel

(2017). When hearing is tricky: Speech processing strategies in prelingually deafened children and adolescents with cochlear implants having good and poor speech performance. PLOS ONE, 12(1), e0168655. https://doi.org/10.1371/journal.pone.0168655

46.

Pakarinen

Takegata

Rinne

Huotilainen

Näätänen

(2007). Measurement of extensive auditory discrimination profiles using the mismatch negativity (MMN) of the auditory event-related potential (ERP). Clinical Neurophysiology, 118(1), 177–185. https://doi.org/10.1016/j.clinph.2006.09.001

47.

Petersen

Andersen

A. S. F.

Haumann

N. T.

Højlund

Dietz

M. J.

Michel

Riis

S. K.

Brattico

Vuust

(2020). The CI MuMuFe – a new MMN paradigm for measuring music discrimination in electric hearing. Frontiers in Neuroscience, 14. https://doi.org/10.3389/fnins.2020.00002

48.

Petersen

Gjedde

Wallentin

Vuust

(2013). Cortical plasticity after cochlear implantation. Neural Plasticity. https://doi.org/10.1155/2013/318521

49.

Petersen

Mortensen

M. V.

Hansen

Vuust

(2012). Singing in the key of life: A study on effects of musical ear training after cochlear implantation. Psychomusicology: Music, Mind, and Brain, 22(2), 134–151. https://doi.org/10.1037/a0031140

50.

Petersen

Weed

Sandmann

Brattico

Hansen

Sørensen

S. D.

Vuust

(2015). Brain Responses to Musical Feature Changes in Adolescent Cochlear Implant Users. Frontiers in Human Neuroscience, 9, 229. http://doi.org/10.3389/fnhum.2015.00007

51.

Phillips

(2018). YaRrr! The Pirate’s Guide to R . https://bookdown.org/ndphillips/YaRrr/

52.

Quiroga-Martinez

D. R.

Hansen

N. C.

Højlund

Pearce

M. T.

Brattico

Vuust

(2019). Reduced prediction error responses in high-as compared to low-uncertainty musical contexts. Cortex, 120, 181–200. https://doi.org/10.1016/j.cortex.2019.06.010

53.

R Core Team (2021). R: A language and environment for statistical computing (4.1.1) [Computer software]. R Foundation for Statistical Computing. http://www.R-project.org

54.

Rettenbach

Diller

Sireteanu

(1999). Do deaf people see better? Texture segmentation and visual search compensate in adult but not in juvenile subjects. Journal of Cognitive Neuroscience, 11(5), 560–583. https://doi.org/10.1162/089892999563616

55.

Riley

P. E.

Ruhl

D. S.

Camacho

Tolisano

A. M.

(2018). Music appreciation after cochlear implantation in adult patients: A systematic review. Otolaryngology–Head and Neck Surgery, 158(6), 1002–1010. https://doi.org/10.1177/0194599818760559

56.

RStudio Team (2021). RStudio: Integrated Development for R (1.4.1717) [Computer software]. http://www.rstudio.com/

57.

Sandmann

Plotz

Hauthal

de Vos

Schönfeld

Debener

(2015). Rapid bilateral improvement in auditory cortex activity in postlingually deafened adults following cochlear implantation. Clinical Neurophysiology, 126(3), 594–607. https://doi.org/10.1016/j.clinph.2014.06.029

58.

Scolari

Serences

J. T.

(2009). Adaptive allocation of attentional gain. The Journal of Neuroscience, 29(38), 11933–11942. https://doi.org/10.1523/JNEUROSCI.5642-08.2009

59.

Timm

Agrawal

Viola

F. C.

Sandmann

Debener

Büchner

Dengler

Wittfoth

(2012). Temporal feature perception in cochlear implant users. PLOS ONE, 7(9), e45375. https://doi.org/10.1371/journal.pone.0045375

60.

Timm

Vuust

Brattico

Agrawal

Debener

Büchner

Dengler

Wittfoth

(2014). Residual Neural Processing of Musical Sound Features in Adult Cochlear Implant Users. Frontiers in Human Neuroscience, 8, 211. http://doi.org/10.3389/fnhum.2014.00181

61.

Uwer

von Suchodoletz

(2000). Stability of mismatch negativities in children. Clinical Neurophysiology: Official Journal of the International Federation of Clinical Neurophysiology, 111(1), 45–52. https://doi.org/10.1016/s1388-2457(99)00204-7

62.

Vuust

Brattico

Glerean

Seppänen

Pakarinen

Tervaniemi

Näätänen

(2011). New fast mismatch negativity paradigm for determining the neural prerequisites for musical ability. Cortex, 47(9), 1091–1098. https://doi.org/10.1016/j.cortex.2011.04.026

63.

Vuust

Heggli

O. A.

Friston

K. J.

Kringelbach

M. L.

(2022). Music in the brain. Nature Reviews Neuroscience, 23(5), 287–305. https://doi.org/10.1038/s41583-022-00578-5

64.

Vuust

Witek

M. A. G.

(2014). Rhythmic complexity and predictive coding: A novel approach to modeling rhythm and meter perception in music. Frontiers in Psychology, 5. https://doi.org/10.3389/fpsyg.2014.01111

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.13 MB