Sage Journals: Discover world-class research

Abstract

Dynamic range compression is a crucial component in hearing aids, aiming to restore audibility for hearing-impaired listeners. However, determining suitable compression parameters, such as the time constants for the level estimation stage, remains a topic of debate, as the perceptual benefit of different parameter configurations varies depending on the acoustic conditions. In this study, a data-driven distance metric based on physical metrics was developed to evaluate and compare the performance of various compression systems. This analysis encompassed fast-acting and slow-acting compression, as well as a “scene-aware” compression that dynamically adjusted the release time constant based on the presence of the target. A reference system called “source-independent compression” was also considered, which had access to individual speech and noise signals. Multiple physical metrics were employed to assess the effects of these different compression systems under diverse acoustic conditions, including varying levels of interfering noise and degrees of room reverberation. Factor analysis was applied to derive a concise set of interpretable features representing the impact of compression, expressed as linear combinations of carefully selected objective metrics. The reduced dimensional representation enabled the use of the Manhattan distance to measure the similarity between the compression systems. Results demonstrated that the scene-aware compressor generally exhibited a smaller distance to the reference system compared to both the fast-acting and slow-acting compressors. This finding suggests that an adaptive compression system holds potential benefits across a range of acoustic conditions.

Keywords

wide dynamic range compression objective metrics hearing aids

Introduction

People with sensorineural hearing loss typically experience a phenomenon called loudness recruitment, which is characterized by an elevated threshold of hearing while the threshold of discomfort at high sound pressure levels remains unchanged (Fowler, 1936). Consequently, soft sounds often become difficult to perceive whereas intense sounds remain equally loud compared to normal-hearing people. This reduced dynamic range (DR) of hearing poses challenges for speech recognition, particularly for people with moderate-to-severe hearing loss, as many of the subtle speech components, such as consonants, may fall near or below their threshold of hearing (Walden & Montgomery, 1975). To address the diminished DR of hearing, modern hearing aids employ a technique known as wide dynamic range compression (WDRC). WDRC applies level-dependent amplification in multiple frequency channels to compensate for the reduced audibility of low-intensity sounds while maintaining comfortable loudness levels for high-intensity sounds (Villchur, 1973). By providing a stronger amplification for lower-input levels and adjusting amplification levels for higher input levels, typically in multiple frequency channels, WDRC aims to restore audibility and improve speech intelligibility for hearing-impaired people.

WDRC, commonly divided into fast-acting and slow-acting compressors based on their release times, remains the subject of ongoing debate regarding the appropriate compression parameters (Kates, 2010; Souza, 2002). Fast-acting compressors are characterized by release times below 200 ms which can provide amplification on a syllabic time scale while slow-acting compressors have release times above 200 ms similar to automatic gain control. Listening tests have indicated that fast-acting compression systems can lead to increased intelligibility while slow-acting compression systems tend to enhance listening comfort (Gatehouse et al., 2006; Alexander & Masterson, 2015; Moore, 2008). However, the benefits of different compression systems in terms of intelligibility and listening comfort are contingent upon specific acoustic conditions, such as reverberation and the spectro-temporal characteristics of background noise, which makes the evaluation of compression systems challenging (Rallapalli & Alexander, 2019; Reinhart et al., 2017; Rhebergen et al., 2017).

While compression is employed to address audibility loss associated with sensorineural hearing loss, its application to speech in the presence of noise and reverberation can lead to adverse effects. These effects include reduced effective compression and amplification of background noise, resulting in a reduced signal-to-noise ratio (SNR) (Reinhart et al., 2016; Souza et al., 2006; Naylor & Johannesson, 2009). Moreover, compression can affect the direct-to-reverberant energy ratio (DRR) and the spatial characteristics of the sound through the amplification of low-level reverberant components (Hassager et al., 2017). Additionally, compression causes negative correlations in the envelopes of multiple sources as the applied gain is determined by the loudest signal (Stone & Moore, 2007). The severity of these effects of compression is influenced by the release time, input SNR, amount of reverberation, and spectro-temporal characteristics of the sources in the mixture. For example, fast-acting compression can enhance the output SNR for speech signals in very low SNR conditions with fluctuating maskers, as it can amplify speech components during noise gaps (Reinhart et al., 2016). However, this benefit diminishes in conditions with stationary maskers.

Despite the detrimental effects, hearing aid compression is still considered necessary to restore audibility caused by the loss of compression in the cochlea. However, there are differences between hearing aid compression and cochlear compression. Hearing aid compression applies amplification in the ear canal instead of directly in the cochlea, leading to differences in neural coding between the two (Armstrong et al., 2021). This implies that hearing aid compression should be selective, focusing on compressing the target sound while avoiding excessive amplification of background noise and reverberation. Several studies have proposed a similar approach, suggesting that an “ideal” compression system should primarily apply compression to the target signal of interest while applying minimal or no compression to the background noise or the reverberant portions of the sound (Corey & Singer, 2021; Llave et al., 2020; Hassager et al., 2017; Edwards, 2014). Such a system, here referred to as “source-independent” compression, has been shown to improve speech intelligibility and preserve spatial cues (Stone & Moore, 2007; Rhebergen et al., 2008; Hassager et al., 2017). However, implementing such a system is not feasible as it requires a priori access to the independent source signals.

Instead, the properties of source-independent compression can be approximated by combining source separation and compression (Corey & Singer, 2017; Ngo et al., 2008; Kortlang et al., 2018; Llave et al., 2020) or by using an adaptive compression system that adjusts the release time of the compressor based on the detection of the target signal (May et al., 2018; Hassager et al., 2017; May et al., 2020). Listening tests have shown that combining source separation and WDRC can reduce noise “annoyance” but does not improve speech intelligibility compared to a system with linear gain (Brons et al., 2015; Kortlang et al., 2018). Listening tests with an adaptive compression system have demonstrated preserved spatial cues in reverberant environments (Hassager et al., 2017), as well as increased speech intelligibility and increased subjective preference compared to conventional compression systems (Kowalewski et al., 2020). However, the adaptive compression system has only been evaluated in either noisy or reverberant conditions, but not in realistic conditions with both interfering noise and room reverberation.

Conducting listening tests in a wide variety of acoustic environments to evaluate and compare multiple compression systems would provide the most accurate assessment. However, such tests are time-consuming, and optimizing the parameters of a novel compression system based on these tests would be challenging. Instead, it would be beneficial to rely on a set of physical metrics for analyzing the effects of compression and adjusting the compression parameters in a given environment. Previous studies have considered certain physical metrics to estimate the beneficial and detrimental effects of compression, although the selection of metrics has varied greatly across studies. Some studies have shown a trade-off between improved audibility and detrimental effects such as co-modulation distortion and a reduced SNR when applying compression to a mixture (Souza et al., 2006; Corey & Singer, 2021). Kates (2010) considered perceptually inspired metrics, including I3 and HASPI as speech intelligibility measures, and HASQI as an indicator of speech quality, to evaluate compression systems. The author argued that determining the “optimal” choice of the compression system is difficult since the compression parameters that maximize intelligibility may not be the same parameters that maximize quality. However, the perceptually inspired metrics used for evaluating WDRC systems only apply to monaural signals and do not account for the spatial distortion typically associated with fast-acting compression systems. Additionally, interpreting the implications of “high-level” metrics like HASPI and HASQI in terms of their signal-level effects can be challenging compared to “low-level” physical metrics. The latter include metrics such as SNR, DR, and co-modulation between the target signal and background noise, which offer a more direct interpretation of the effects of compression.

In the present study, a data-driven approach was employed to explore the effects of compression. A comprehensive set of physical metrics was considered to derive a latent space where each dimension explains the properties of how compression affects a noisy and reverberant mixture signal. The similarity between different compression systems was evaluated based on their relative distance within the latent space. This approach allowed the distance metric to be interpreted in terms of the contributions of the individual dimensions. Conventional fast- and slow-acting compression, as well as an adaptive system based on the SNR-aware approach described in May et al. (2018), were evaluated in terms of their similarity to the source-independent compression systems that the adaptive system aimed to approximate.

Methods

Figure 1 shows the framework for computing the data-driven distance metric, consisting of the following four steps: (1) acoustic scene generation, (2) dynamic range compression, (3) objective metrics, and (4) a latent space analysis. The first three steps can be considered a data generation stage, while the last step aims to produce a latent space with interpretable features that are used to explain and compare the effects of the different compression systems.

Figure 1.

Framework for evaluating the compression systems in adverse conditions. A large set of acoustic scenes was generated and processed by four different dynamic range compression systems. The effect of compression was measured using objective metrics which were then transformed into the feature dimensions of a latent space using factor analysis. The similarity of compression systems was measured as the distance to the source-independent compression system which served as the reference.

Acoustic Scene Generation

The effect of compressing noisy and reverberant speech is typically measured by comparing how the target speech is processed in comparison to the background. For this reason, the acoustic scene generation stage was designed such that both the target speech and the background could be isolated from the mixture. “Shadow-filtering” was used to apply the same compression gain function that was estimated from the mixture separately to the target speech and the background.

The noisy and reverberant mixtures, referred to as the acoustic scenes, were generated as illustrated in Figure 2. An acoustic scene consisted of a foreground, a background, and a mixture of component. The foreground was the target speech signal convolved with the direct sound of a binaural room impulse response (BRIR), which was considered to be the initial 2.5 ms following the largest peak. The background contained both the reverberant noise and the late reflections of the target speech signal. The mixture was the sum of the foreground and the background.

Figure 2.

Schematic representation for generating an acoustic scene. The speech is adjusted to have a level of 65 dB RMS, and the noise level is adjusted according to a desired SNR. The speech and noise signals are then spatialized using binaural room impulse responses (BRIRs) that were separated into early reflections and late reverberation by applying a windowing procedure. The spatialized signals were then used to create a foreground signal, a background signal and a mixture signal. RMS = root mean square; SNR = signal-to-noise ratio; BRIR = binaural room impulse response.

The acoustic scene signal was generated by first randomly selecting and concatenating five speech sentences from a speech database. The level of the speech signals was normalized to a root mean square (RMS) level of 65 dB. The noise was then selected from a noise database and its level was adjusted relative to that of the speech to achieve SNRs between $- 12$ dB and 12 dB. Both the speech and the noise signals were spatialized using a BRIR.

The data-driven distance metric was implemented and validated using different sets of speech, noise, and BRIR databases. The acoustic scenes used for implementing the data-driven distance metric employed a separate collection of speech, noise, and BRIR compared to the acoustic scenes used for validating the metric. Details on the datasets can be found in the “Dataset generation” section, with a summary provided in Table 2.

Table 2.

Summary of the training and test datasets.

	Training set	Test set
Speech	TIMIT	PTDB-TUG
Noise	ICRA	NOISEX
	1,5,6,7	car, factory, babble
		speech-shaped noise
BRIR	Surrey	ASH
	A, B, C, D	R06, R07, R08,
	Anechoic	Anechoic

Compression Systems

Four compression systems were implemented. Three of them represented the systems under test and one represented the source-independent compression system which served as the reference. The systems under test were a fast-acting compressor with a short release time, a slow-acting compressor with a long release time, and a scene-aware compressor with an adaptive release time. All systems were implemented similar to May et al. (2018), using seven frequency channels and compression parameters according to the NAL-NL2 prescription for an N4 standard audiogram (Bisgaard et al., 2010). The parameters of the compression systems are indicated in Table 1 and signal processing details are shown in Figure 3. All systems operated in seven frequency channels using a 1-octave filterbank ranging from a center frequency of 125 Hz to 8 kHz. The systems had a common attack time of 5 ms and the release time was 50 ms for the fast-acting compressor, and 2000 ms for the slow-acting compressor. The scene-aware compressor adaptively changed the release time, such that it was 50 ms in time-frequency (T-F) units of the filterbank output with a foreground-to-background ratio (FBR) above 0 dB, and 2000 ms when the FBR was below 0 dB. All of the compressors utilized a single knee point in each frequency channel, applying compression above the knee point and maintaining linear gain below the knee point. The scene-aware compressor had a priori information about the FBR in each T-F unit. The reference source-independent compression system had access to individual speech and noise signals of the mixture signal and applied fast-acting compression to the foreground and slow-acting compression to the background before mixing.

Table 1.

Compression ratio ( $C_{R}$ ), compression threshold ( $C_{T}$ ) and linear gain ( $G$ ) for seven octave frequency bands.

	125 Hz	250 Hz	500 Hz	1 kHz	2 kHz	4 kHz	8 kHz
$C_{R}$	2.2:1	2.2:1	2.2:1	3.0:1	3.5:1	3.3:1	2.5:1
$C_{T}$ [dB]	46	46	48	40	35	32	30
$G$ [dB]	22.1	22.1	24.4	34.5	39.4	43.4	42.5

Figure 3.

Signal processing diagram and configuration of the WDRC systems. The input signal is first processed by an STFT (based on the SciPy implementation (Virtanen et al., 2020)), then the level is estimated in seven frequency bands, followed by the gain calculation stage based on the NAL-NL2 prescription with a 50 ms release time for the fast-acting compressor and 2000 ms release time for the slow-acting compressor. The gain is interpolated from the seven frequency bands to match the number of frequency bins in the STFT. The original STFT representation of the input signal is multiplied by the interpolated gain prior to being transformed into the time-domain using an ISTFT. WDRC = wide dynamic range compression; STFT = short-time Fourier transform; ISTFT = inverse short-time Fourier transform.

Physical Metrics

To evaluate the effects of compression, an extensive set of physical metrics was used to identify which metrics would be most relevant for the data-driven distance metric. The selected metrics have either been considered in previous studies or were assumed here to be affected by compression. The physical metrics were all computed for the foreground, the background and the mixture signals in individual frequency bands, identical to the frequency bands of the compressors. Some specific physical metrics were only computed for the foreground, such as the unvoiced-to-voiced ratio (UVR) and the binaural source modulation correlation (BSMC). Other metrics were calculated for combinations of the foreground and the background, such as the SNR and the across source modulation correlation (ASMC). The effect of compression on a given physical metric was evaluated by comparing the metric before and after compression.

The ASMC measures the co-modulation between the envelope of the foreground and the background signal (Stone & Moore, 2008). An increased amount of compression has been shown to cause the foreground and background in a mixture to become negatively correlated. The envelopes were using the same STFT, filterbank, and level estimation approach as the fast-acting compressor, followed by applying a threshold to the level estimates such that the lowest level was maximally 13 dB below the average level according to Stone & Moore (2008).

The BSMC is similar to interaural coherence but uses the same pre-processing as the ASMC. Compression has been shown to distort spatial cues by causing the left and right channels to be less correlated in the presence of reverberation due to the increased amplification of reverberant components in the mixture (Hassager et al., 2017). This metric was computed only for the mixture.

The dynamic range (DR) represents the long-term DR of the sound, representing the difference between the 99th and 50th level percentile in dB. The DR was calculated for the foreground, the background, and the mixture. The level estimation of the signals used the same front-end as the fast-acting compressor. Increased amount of compression is associated with a reduction of DR.

The effective compression ratio (ECR) reflects the ratio between the DR before and after compression (Souza et al., 2006). The ECR indicates how much compression is effectively applied and was calculated for the foreground, background, and mixture signal as the ratio of the DR before and after processing. Increased amount of compression is related to an increase in ECR.

The fidelity of envelope shape (FES) reflects the correlation between the envelope of the same sound before and after compression and can be considered as a measure of envelope distortion (Stone & Moore, 2008). It uses the same pre-processing as the ASMC and the BSMC and was calculated here for the foreground, the background, and the mixture. Distortion caused by compression would lower the FES.

The FBR reflects the level of the foreground signal in relation to the background. This is equivalent to the SNR in the absence of reverberation, and equivalent to the DRR in the absence of noise. Compression typically lowers the SNR and DRR by the increased amplification of noise and reverberation relative to the target speech signal (Souza et al., 2006).

The UVR measures the change in the power ratio between the unvoiced and the voiced speech components. Fast-acting compression may provide an increased amplification of the soft unvoiced speech components, such as some consonants, relative to the louder voiced speech components. The UVR has been hypothesized to be related to speech intelligibility (Kowalewski et al., 2018). Computing this metric requires labeled speech data, where the voiced and unvoiced speech components can be separated. The UVR was measured here only for the foreground signal.

Dimensionality Reduction

The seven physical metrics (ASMC, BSMC, DR, ECR, FES, FBR, and UVR) were calculated in seven frequency channels and some of the metrics were calculated for both foreground, background, and mixture, resulting in a total number of 91 metrics. To reduce the dimensionality and determine the underlying effects of compression on speech in adverse conditions, factor analysis was used. To ensure comparability across metrics, each metric was standardized by subtracting the mean and dividing it by the standard deviation of the training set. This process guarantees that all metrics are operating on a consistent scale. Traditionally, principal component analysis (PCA) has been used for dimensionality reduction. However, factor analysis was considered more beneficial here, as it allows for interpretable latent space dimensions (Costello & Osborne, 2005). Similar to PCA, factor analysis is based on eigenvalue decomposition and a factor with a higher eigenvalue describes more variance of the data. Following the recommendation of Hair et al. (2018), the number of retained factors was controlled by selecting the number of factors that lied above the knee point of the scree plot showing the eigenvalues for each factor. Each factor consists of factor loadings for each given metric. The factor loadings reflect how much the metric contributes to the factor. The factor loadings are commonly neglected if they do not exceed a threshold of an absolute value of 0.7 (Costello & Osborne, 2005).

Dataset Generation

Both a training set and a test set were created to assess the generalization of the latent space representation. The training set and the test set used disjoint speech, noise, and reverberation data, as summarized in Table 2. The latent space was created using factor analysis of the training set. The generalizability of the latent space was then evaluated by projecting the physical metrics calculated from the test set onto the latent variables derived from the factor analysis. A similar distribution within the latent space of the training set and the test set would suggest that the latent space generalizes to previously unseen acoustic conditions.

The TIMIT database (Garofolo et al., 1993) was used to create the training data while the PTDB-TUG database (Pirker et al., 2011) was used to create the test set. Both datasets allowed to calculate the UVR metric as the supplemented labels allowed them to identify unvoiced and voiced components. The noise databases consisted of ICRA noise (Dreschler et al., 2001) for training and NOISEX (Varga & Steeneken, 1993) for testing. The ICRA noise contained speech-shaped noise with either a static level or with a fluctuating level according to 1-talker, 2-talkers, or 6-talkers scenarios. The NOISEX stimuli were factory noise, car noise and babble noise. The test dataset had additional noise stimuli, which was a stationary noise with a long-term average spectrum of the PTDB-TUG speech data. These noise types were chosen to contain realistic background sounds with varying degree of amplitude envelope fluctuations. The BRIRs from the SURREY database (Hummersone et al., 2010) were used to create the reverberant training data, while BRIRs from the ASH-IR database (Pearce, 2021) were used for testing. The selected rooms and respective characteristics are summarized in Table 3. The compression systems and physical metrics were the same for both the training and the test sets.

Table 3.

Acoustical properties of the BRIRs used for acoustic scene generation.

Name	Room type	RT60 [s]	DRR[dB]
Anechoic [S]	Anechoic room	0.00	inf
A [S]	Office	0.32	6.09
B [S]	Class room	0.47	5.31
C [S]	Cinema-style lecture theater	0.68	8.82
D [S]	Medium-large sized auditorium	0.89	6.12
R06 [ASH]	Seminar room	0.47	−1.25
R07 [ASH]	Small conference room	0.41	6.53
R08 [ASH]	Small meeting room	0.79	3.51

R06, R07, R08 were used for the test set while A,B,C,D were used for the training set. The anechoic room was used for both sets. BRIRs marked with [S] are from the Surrey database, while [ASH] are from the ASH database. DRR = direct-to-reverberant energy ratio; BRIR = binaural room impulse response

Distance to Source-independent Compression

Within the low-dimensional space as determined by the factor analysis, it can be assumed that data points that are close to each other are similar. Each data point in the latent space corresponds to an acoustic scene processed by a compression system. The relative distance between the points represents a measure of similarity between the compression systems. The distance metric was chosen to be represented by the Manhattan distance since this metric can easily be decomposed into the contributions from the different dimensions and can as such provide an interpretable distance metric. The distance of each of the considered compression systems to the source-independent compression system was calculated in each acoustic condition to evaluate the respective compression system’s similarity to the source-independent system.

Results

Latent Space

The factor analysis of the training data resulted in a latent space, where each factor represents one dimension in the space. This latent space describes the effects of compression on speech in noise and reverberation. The influence of each factor is captured by the scree plot in Figure 4 showing the eigenvalue for each factor. The number of retained factors was chosen following the recommendations of Hair et al. (2018), according to which eigenvalues above the knee point in the scree plot, shown in Figure 4, should be retained. Based on this principle, the factors 1–3 were selected in this analysis.

Figure 4.

Scree plot showing the eigenvalue for each individual factor calculated for the training set. The knee point was selected at the third component, as indicated by the red dot, after which the change in eigenvalue was less pronounced.

Each factor contained a set of coefficients for each objective metric referred to as loadings. The loadings are shown in Figure 5. Loadings of the retained factors with an absolute value above the threshold (0.7) are indicated by the darker bars. In total, 47 out of 91 objective metrics were retained after the factor analysis. The first factor (top panel) mainly corresponded to a change in the DR of the foreground, reflected by a decrease in DR of the foreground, a change in UVR, and an increase of ECR of the foreground. The second factor (middle panel) essentially corresponded to a decrease in ECR of the background, an increase in ECR of both the background and the mixture, as well as decreases in BSMC and ASMC. The third factor (bottom panel) represented a change in the FBR, which was the only metric above the threshold for this factor.

Figure 5.

The factor loadings for each selected factor. Each metric is grouped according to the output of the 1-octave filterbank, where the leftmost loading is calculated for the 125 Hz band and the rightmost for the 8 kHz band. The factor loadings that were retained are highlighted with a darker color. The first factor can be interpreted as representing the amount of compression of the foreground, the second factor describes the effects on the background, and the third factor represents the change in the foreground-to-background ratio (FBR). The physical metrics that were calculated for the different components in the noisy mixture signals include a subscript referring to F (for foreground), B (for background), or M (for mixture) to indicate the signal they were calculated for.

Generalization Assessment

The latent space was created using a training set of noisy and reverberant mixtures and was assessed on noisy and reverberant mixtures generated from a disjoint set of speech, noise, and BRIRs to ensure that this space could explain the effects of compression on previously unseen data. The comparison of the two datasets in Figure 6 indicates that the latent space can be used to describe previously unseen data, as the distributions of the compression systems are comparable between the training set (left panel) and test set (right-panel).

Figure 6.

Comparison of the latent space of the training set (left panel) and the test set (right panel) for factor 1 and factor 2. Both datasets show a similar distribution within the latent space. High signal-to-noise ratio (SNR) correspond to the acoustic conditions with an SNR above 0, and Low SNR corresponds to acoustic conditions with an SNR of 0 or below.

The two-dimensional projection of the latent space in Figure 6 shows that the source-independent compressor (red) performs as designed, by providing the highest amount of foreground compression (high factor 1) and a relatively low degree of background and mixture compression (low factor 2). The slow-acting compressor (orange) provides the smallest amount of foreground compression but also does not compress the background and the mixture. The fast-acting compressor (blue) provides the largest amount of compression to the background and the mixture. The scene-aware compressor (green) is located in between the fast- and the slow-acting compressor.

The Influence of Noise and Reverberation

To analyze the influence of noise and reverberation, the data were grouped by the noise type, the input SNR, and the presence of reverberation. The resulting distributions are shown in Figure 7 and visualized in terms of factor 1 and factor 2. In Figure 7, the upper row shows the influence of stationary noise and the bottom row shows the influence of modulated noise. The left column shows the distributions in anechoic conditions for input SNRs of 0 dB and below, the middle column shows the distributions in anechoic conditions for input SNRs above 0 dB, and the right column shows the distributions for reverberant conditions with an input SNR above 0 dB.

Figure 7.

Distributions of processed acoustic scenes for the test set in both stationary noise (top row) and modulated noise (bottom row) in the low signal-to-noise ratio (SNR) condition (left column), the high SNR condition (middle column), and the reverberant condition (right column).

The effects of the input SNR on the amount of foreground compression (factor 1) is consistent with the findings from previous studies (Souza et al., 2006), where a lower input SNR (left column) results in a reduced amount of foreground compression compared to acoustic conditions with a higher input SNR (middle column). This is reflected by the shift towards the right for the distributions for the slow-acting (orange), the fast-acting (blue), and the scene-aware (green) compressors. The presence of reverberation reduces the amount of foreground compression, as can be seen when comparing the high SNR conditions (middle column) with the high SNR and reverberant conditions (right column).

The influence of the noise type is primarily represented by factor 2, which is reduced for the stationary noise (upper panels) compared to the modulated noise (lower panels), especially for the fast-acting (blue) and the scene-aware (green) compressors at low SNRs. The reduction of factor 2 for stationary noise can be explained by the expansion due to the co-modulation effects when the compressor is driven by the foreground signal. In addition, the increase of factor 2 in the modulated noise conditions can also be explained by the reduction of the DR of the background after compression.

The influence of reverberation (right column) is especially represented by an increase in factor 2, relative to the anechoic and high SNR conditions (middle column). The distributions are, however, similar across noise types. The increase in factor 2 might thus be explained by the binaural distortion reflected in the BSMC.

The scene-aware compressor appears in all acoustic conditions as an interpolation between the fast- and slow-acting compressors. In extreme conditions, the scene-aware-compressor becomes equivalent to either the slow- or the fast-acting compressor. For example, in the case of stationary noise at a low SNR (upper left panel), it can be assumed that most of the target-related components are masked by the background, such that the scene-aware compressor primarily uses a long release time. The same degree of masking is, however, not seen in the case of modulated noise, as the scene-aware compressor can switch the release time from long to short in the noise dips, such that it behaves more similarly to the fast-acting compressor in the presence of speech glimpses. The overlap between the fast-acting, scene-aware and ideal compressors in the high SNR conditions (middle panels) is considered to stem from the noise-free conditions where all three compressors provide an equivalent fast-acting compression to the foreground only.

Distance to Source-independent Compression

The similarity to the source-independent compressor was measured as the Manhattan distance between each acoustic scene processed by any compression system to each of the similar acoustic scenes processed by the source-independent compressor. The acoustic conditions were grouped by the input SNR, the noise type, and the presence of reverberation. These distances are summarized in Figure 8 as a function of the input SNR, where the left panel shows the distances for the stationary noise conditions, the middle panel shows the distances for the modulated noise conditions and the right panel shows the distances for both noise types in reverberant conditions. The distance is visualized as stacked bars where the lower bars (dark colors) represent the contribution from factor 1, the middle bars (medium colors) represent the contribution from factor 2 and the upper bars (light colors) represent the contribution from factor 3. The error bars in the figure represent the standard deviation from the distances between groups with similar configurations. The non-zero distance for the source-independent compressor is attributed to the variance within the group. At infinite SNR (most-right bars in each panel), the fast-acting (blue) and the scene-aware (green) compressors are equivalent to the source-independent compressor (red) because they apply the exact same processing. The only exception is in the condition with reverberation, where all compressors differ from the source-independent compressor. The fast-acting compressor (blue) generally provided a higher amount of foreground compression (indicated by a lower factor 1) than the slow-acting (orange) and scene-aware (green) compressors. In stationary noise and in reverberant conditions, the fast-acting compression system showed the largest distance to the source-independent compressor, due to the increased amount of background compression (factor 2) and the substantial reduction of the FBR (factor 3). In modulated noise, the distance to the source-independent compressor was reduced for both the fast-acting compressor and the scene-aware compressor, which according to the factor contributions shown in Figure 8 might be explained by the reduction of factor 1 and 3 corresponding to increased foreground compression and less influence on the FBR.

Figure 8.

Distance to source-independent compression across acoustic conditions with either noise only or both noise and reverberation. Each distance is decomposed into the contributions from each dimension in the latent space, where the lowest bar corresponds to factor 1, the middle to factor 2, and the uppermost to factor 3. The error bars indicate the standard deviation of the distances between the groups with similar configurations.

Discussion

This study evaluated a set of dynamic range compression systems applied to speech signals in the presence of both noise and reverberation using a novel data-driven distance metric. The distance metric was based on the similarity within a latent space between each compression system and a reference system, which was a source-independent compressor. The latent space was created through factor analysis, selecting and combining the most relevant physical metrics from a larger set. The latent space aimed to capture the primary effects of compression on noisy and reverberant speech mixtures and consisted of the following three dimensions: the change in foreground DR, the amount of compression and distortion of the background, and the change in FBR.

The distance between data points in the latent space reflected the similarity of the processing effects applied to the input signals, with a shorter distance indicating higher similarity. In this study, the distance metric was used to evaluate the effects of compression. However, it could also be potentially used to optimize compression parameters in order to approximate a reference system, such as the source-independent compressor, which requires access to individual speech and noise signals and thus cannot be realized in practical applications. For example, optimizing a compressor to maximize SNR would result in either linear gain or slow-acting compression while optimizing it to maximize the ECR of the foreground would result in fast-acting compression. Using a reference system as an optimization target could be beneficial when optimizing a system on multiple attributes.

Based on the analysis of the distance metric across various acoustic conditions, the recommended compression system in this study leaned towards the scene-aware compressor, particularly in anechoic conditions with SNRs below 12 dB, as it had the shortest distance to the reference system. However, in the presence of reverberation, both the scene-aware compressor and the slow-acting compressor exhibited a similar distance to the reference system. Nevertheless, there is a trade-off between the slow-acting compressor and the scene-aware compressor, where the former is more similar to the reference system in terms of factor 2, while the latter is more similar in terms of factor 1. This trade-off demonstrates that the challenge of balancing foreground compression and preserving the naturalness of the background persists for adaptive compression systems, including the scene-aware compressor. The distance to the reference system was only zero in an anechoic and noise-free condition where both the fast-acting and scene-aware compressors were equivalent to the source-independent system. In the highest SNR condition, the distance increased, indicating that additional signal processing would be required to further reduce the distance and approximate the reference system. Considering that the source-independent compression essentially employs ideal source-separation before compression, incorporating a speech enhancement stage, such as noise reduction or a beamformer, to isolate the target speech signal prior to compression might lead to a better approximation of the reference system. In such cases, the latent space would still be suitable for evaluating combined speech enhancement and compression systems since speech enhancement can also be assessed using the same physical metrics as compression.

One limitation of the distance metric is its dependence on a reference system. In this study, the selected reference system was the source-independent system, assuming that hearing-impaired listeners would benefit from applying fast-acting compression to the target speech and slow-acting compression to the background. When utilizing the distance metric for optimization, the reference system should be evaluated perceptually, and the compression parameters could potentially be adjusted to ensure suitability for individual hearing-impaired listener, such as adapting the level of fast-acting compression based on the working memory capabilities of the hearing aid user (Souza & Sirow, 2014).

The distance metric relied solely on physical metrics and was derived through a data-driven approach. Consequently, the perceptual consequence of changes in distance or changes with respect to each factor was not evaluated. The contribution of each factor was equally weighted when calculating the distance to the reference, implying that a change in factor 1 has the same impact on the distance as a change in factor 2 or factor 3. A possible future direction would be to investigate how each factor influences perceptual outcomes, such as speech intelligibility, listening comfort, or binaural perceptual attributes. Additionally, it would be beneficial to determine if changes in the data-driven distance metric correspond to perceived changes towards or away from the reference system.

Conclusion

This study proposed a data-driven distance metric to evaluate the impact of dynamic range compression on speech in adverse acoustic conditions. The distance metric was derived from a set of physical metrics and transformed into a latent space with three dimensions. The dimensions represented the compression applied to the foreground, the compression and distortion of the background, and the change in the FBR. By utilizing the latent space, the effects of compression and also the similarity between different compression systems could also be analyzed. The scene-aware compressor consistently exhibited a smaller distance to the reference system compared to both the fast-acting and slow-acting compressors across various conditions. This suggests that an adaptive compression system may offer benefits in diverse acoustic environments. In future research, the data-driven distance metric could be utilized to develop novel compression techniques, using the distance as a means of optimization.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication ofthis article: This research was supported by the Technical University of Denmark and funding from Sonova AG (Stäfa, Switzerland).

ORCID iDs

Niels Overby

Torsten Dau

Tobias May

References

Alexander

J. M.

Masterson

(2015). Effects of WDRC release time and number of channels on output SNR and speech recognition. Ear and Hearing, 36(2), e35–e49. https://doi.org/10.1097/AUD.0000000000000115

Armstrong

A. G.

Lam

C. C.

Sabesan

Lesica

N. A.

(2021). Compression and amplification algorithms in hearing aids impair the selectivity of neural responses to speech. Nature Biomedical Engineering 2021 6:6, 6(6), 717–730. https://doi.org/10.1038/s41551-021-00707-y. https://www.nature.com/articles/s41551-021-00707-y

Bisgaard

Vlaming

M. S.

Dahlquist

(2010). Standard audiograms for the IEC 60118-15 measurement procedure. Trends in Amplification, 14(2), 113–120. https://doi.org/10.1177/1084713810379609

Brons

Houben

Dreschler

W. A.

(2015). Acoustical and perceptual comparison of noise reduction and compression in hearing aids. Journal of Speech, Language, and Hearing Research, 58(4), 1363–1376. https://doi.org/10.1044/2015_JSLHR-H-14-0347

Corey

R. M.

Singer

A. C.

(2017). Dynamic range compression for noisy mixtures using source separation and beamforming. In: IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, volume 2017-Octob. https://doi.org/10.1109/WASPAA.2017.8170041.

Corey

R. M.

Singer

A. C.

(2021). Modeling the effects of dynamic range compression on signals in noise. The Journal of the Acoustical Society of America, 150(1), 159–170. https://doi.org/10.1121/10.0005314

Costello

A. B.

Osborne

J. W.

(2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research and Evaluation, 10(7).https://doi.org/10.7275/jyj1-4868

Dreschler

W. A.

Verschuure

Ludvigsen

Westermann

(2001). ICRA noises: Artificial noise signals with speech-like spectral and temporal properties for hearing instrument assessment. International Journal of Audiology, 40(3), 148–157. https://doi.org/10.3109/00206090109073110

Edwards

(2014, April). Compression and mixing for hearing assistance devices. US 9,332,360 B2.

10.

Fowler

E. P.

(1936). a method for the early detection of otosclerosis: A study of sounds well above threshold. Archives of Otolaryngology - Head and Neck Surgery, 24(6), 731–741. https://doi.org/10.1001/archotol.1936.00640050746005

11.

Garofolo

J. S.

Lamel

L. F.

Fisher

W. M.

Fiscus

J. G.

Pallett

D. S.

Dahlgren

N. L.

Zue

(1993). TIMIT acoustic-phonetic continuous speech corpus LDC93S1. Web download. Linguistic Data Consortium.

12.

Gatehouse

Naylor

Elberling

(2006). Linear and nonlinear hearing aid fittings - 1. Patterns of benefit. International Journal of Audiology, 45(3), 130–152. https://doi.org/10.1080/14992020500429518. https://www.tandfonline.com/action/journalInformation?journalCode=iija20

13.

Hair

Anderson

Babin

Black

(2018). Multivariate Data Analysis, volume 50. 8 edition. Cengange Learning EMEA. ISBN 9781473756540.

14.

Hassager

H. G.

May

Wiinberg

Dau

(2017). Preserving spatial perception in rooms using direct-sound driven dynamic range compression. The Journal of the Acoustical Society of America, 141(6), 4556–4566. https://doi.org/10.1121/1.4984040. http://dx.doi.org/10.1121/1.4984040

15.

Hassager

H. G.

Wiinberg

Dau

(2017). Effects of hearing-aid dynamic range compression on spatial perception in a reverberant environment. The Journal of the Acoustical Society of America, 141(4), 2556–2568. https://doi.org/10.1121/1.4979783. http://dx.doi.org/10.1121/1.4979783

16.

Hummersone

Mason

Brookes

(2010). Dynamic precedence effect modeling for source separation in reverberant environments. IEEE Transactions on Audio, Speech and Language Processing, 18(7), 1867–1871. https://doi.org/10.1109/TASL.2010.2051354

17.

Kates

J. M.

(2010). Understanding compression: Modeling the effects of dynamic-range compression in hearing aids. International Journal of Audiology, 49(6), 395–409. https://doi.org/10.3109/14992020903426256. https://www.tandfonline.com/action/journalInformation?journalCode=iija20

18.

Kortlang

Chen

Gerkmann

Kollmeier

Hohmann

Ewert

S. D.

(2018). Evaluation of combined dynamic compression and single channel noise reduction for hearing aid applications. International Journal of Audiology, 57, S43–S54. https://doi.org/10.1080/14992027.2017.1300695

19.

Kowalewski

Dau

May

(2020). Perceptual evaluation of signal-to-noise-Ratio-Aware dynamic range compression in hearing aids. Trends in Hearing, 24, 1–14. https://doi.org/10.1177/2331216520930531

20.

Kowalewski

Zaar

Fereczkowski

MacDonald

E. N.

Strelcyk

May

Dau

(2018). Effects of slow- and fast-acting compression on hearing-impaired listeners’ consonant-vowel identification in interrupted noise. Trends in Hearing, 22, 233121651880087. https://doi.org/10.1177/2331216518800870. http://journals.sagepub.com/doi/10.1177/2331216518800870

21.

Llave

Leglaive

Seguier

(2020). Localization Cues Preservation in Hearing Aids by Combining Noise Reduction and Dynamic Range Compression. 2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2020 - Proceedings : 686–693.

22.

May

Kowalewski

Dau

(2018). Signal-to-noise-ratio-aware dynamic range compression in hearing aids. Trends in Hearing, 22, 1–12. https://doi.org/10.1177/2331216518790903

23.

May

Kowalewski

Dau

(2020). Scene-Aware Dynamic-Range Compression in Hearing Aids. In: Blauert, J., Braasch, J. (Eds.). The Technology of Binaural Understanding. Modern Acoustics and Signal Processing. Springer International Publishing. ISBN 978-3-030-00386-9, (pp. 763–799). https://doi.org/10.1007/978-3-030-00386-9_25.

24.

Moore

B. C.

(2008). The choice of compression speed in hearing aids: Theoretical and practical considerations and the role of individual differences. Trends in Amplification, 12(2), 103–112. https://doi.org/10.1177/1084713808317819

25.

Naylor

Johannesson

R. B.

(2009). Long-term signal-to-noise ratio at the input and output of amplitude-compression systems. Journal of the American Academy of Audiology, 20(3), 161–171. https://doi.org/10.3766/jaaa.20.3.2

26.

Ngo

Doclo

Spriet

Moonen

Wouters

Jensen

(2008). An integrated approach for noise reduction and dynamic range compression in hearing aids. European Signal Processing Conference, 10(Eusipco), 2–6.

27.

Pearce

(2021). Audio Spatialisation for Headphones - Impulse Response Dataset. https://doi.org/10.5281/zenodo.4780815.

28.

Pirker

Wohlmayr

Petrik

Pernkopf

(2011). A pitch tracking corpus with evaluation on multipitch tracking scenario. Twelfth annual conference of the international speech communication association, 1509–1512. https://doi.org/10.21437/interspeech.2011-317

29.

Rallapalli

V. H.

Alexander

J. M.

(2019). Effects of noise and reverberation on speech recognition with variants of a multichannel adaptive dynamic range compression scheme. International Journal of Audiology, 58(10), 661–669. https://doi.org/10.1080/14992027.2019.1617902. https://www.tandfonline.com/action/journalInformation?journalCode=iija20

30.

Reinhart

Zahorik

Souza

(2017). Effects of reverberation, background talker number, and compression release time on signal-to-noise ratio. The Journal of the Acoustical Society of America, 142(1), EL130–EL135. https://doi.org/10.1121/1.4994683

31.

Reinhart

P. N.

Souza

P. E.

Srinivasan

N. K.

Gallun

F. J.

(2016). Effects of reverberation and compression on consonant identification in individuals with hearing impairment conclusions-reverberation and WDRC release time both affect aided consonant identification for individuals with hearing impairment, and these condi. Ear Hear, 37(2), 144–152. https://doi.org/10.1097/AUD.0000000000000229

32.

Rhebergen

K. S.

Maalderink

T. H.

Dreschler

W. A.

(2017). Characterizing speech intelligibility in noise after wide dynamic range compression. Ear and Hearing, 38(2), 194–204. https://doi.org/10.1097/AUD.0000000000000369

33.

Rhebergen

K. S.

Versfeld

N. J.

Dreschler

W. A.

(2008). Quantifying and modeling the acoustic effects of compression on speech in noise. The Journal of the Acoustical Society of America, 123(5), 3167–3167. https://doi.org/10.1121/1.2933226

34.

Souza

P. E.

(2002). Effects of compression on speech acoustics, intelligibility, and sound quality. Trends in Amplification, 6(4), 131–165. https://doi.org/10.1177/108471380200600402

35.

Souza

P. E.

Jenstad

L. M.

Boike

K. T.

(2006). Measuring the acoustic effects of compression amplification on speech in noise. The Journal of the Acoustical Society of America, 119(1), 41–44. https://doi.org/10.1121/1.2108861. http://asa.scitation.org/doi/10.1121/1.2108861

36.

Souza

P. E.

Sirow

(2014). Relating working memory to compression parameters in clinically fit hearing aids. American Journal of Audiology, 23(4), 394–401. https://doi.org/10.1044/2014_AJA-14-0006. https://pubs.asha.org

37.

Stone

M. A.

Moore

B. C. J.

(2007). Quantifying the effects of fast-acting compression on the envelope of speech. The Journal of the Acoustical Society of America, 121(3), 1654–1664. https://doi.org/10.1121/1.2434754

38.

Stone

M. A.

Moore

B. C. J.

(2008). Effects of spectro-temporal modulation changes produced by multi-channel compression on intelligibility in a competing-speech task. The Journal of the Acoustical Society of America, 123(2), 1063–1076. https://doi.org/10.1121/1.2821969

39.

Varga

Steeneken

H. J.

(1993). Assessment for automatic speech recognition: II. NOISEX-92: A database and an experiment to study the effect of additive noise on speech recognition systems. Speech Communication, 12(3), 247–251. https://doi.org/10.1016/0167-6393(93)90095-3

40.

Villchur

(1973). Signal processing to improve speech intelligibility in perceptive deafness. The Journal of the Acoustical Society of America, 53(6), 1646–1657. https://doi.org/10.1121/1.1913514

41.

Virtanen

Gommers

Oliphant

T. E.

Haberland

Reddy

Cournapeau

Burovski

Peterson

Weckesser

Bright

van der Walt

S. J.

Brett

Wilson

Millman

K. J.

Mayorov

Nelson

A. R. J.

Jones

Kern

Larson

Carey

C. J.

Polat

Feng

Moore

E. W.

Van der Plas

Laxalde

Perktold

Cimrman

Henriksen

Quintero

E. A.

Harris

C. R.

Archibald

A. M.

Ribeiro

A. H.

Pedregosa

van Mulbregt

, SciPy 10 Contributors. (2020). SciPy 1.0: Fundamental algorithms for scientific computing in python. Nature Methods, 17, 261–272. https://doi.org/10.1038/s41592-019-0686-2

42.

Walden

B. E.

Montgomery

A. A.

(1975). Dimensions of consonant perception in normal and hearing impaired listeners. Journal of Speech and Hearing Research, 18(3), 444–455. https://doi.org/10.1044/jshr.1803.444

A Data-driven Distance Metric for Evaluating the effects of Dynamic Range Compression in Adverse Conditions

Abstract

Keywords

Introduction

Methods

Acoustic Scene Generation

Compression Systems

Physical Metrics

Dimensionality Reduction

Dataset Generation

Distance to Source-independent Compression

Results

Latent Space

Generalization Assessment

The Influence of Noise and Reverberation

Distance to Source-independent Compression

Discussion

Conclusion

Footnotes

Declaration of Conflicting Interests

Funding

ORCID iDs

References