Rethinking Batch Composition: Promoting Feature Diversity for Speech Enhancement

1. INTRODUCTION

• Existing enhancement approaches address variability through acoustic-condition or speaker-aware methods, as well as changes to losses, targets, architectures, and training data.Source: §1 · p. 1
• Clean-speech formant structure and loudness can correlate with enhancement performance even when noise type, SNR, and speaker are controlled.Source: §1 · p. 1
• Feature-stratified sampling partitions training examples into quantiles of a clean-speech characteristic and gives each quantile equal representation in every mini-batch.The characteristic determines batch composition, rather than being used as model input.Source: §1 · p. 1
• Performance improves across two enhancement paradigms, two languages, and two speaker groups, with gains on four objective metrics.The improvements are consistent across the reported evaluation scope.Source: §1 · pp. 1–2
• Gains are robust to stochastic batch-sampling variation, and their magnitude depends on the stratified characteristic, with greater benefits from features strongly associated with enhancement performance.Source: §1 · p. 2

2. SPEECH ENHANCEMENT

• Speech enhancement aims to recover clean speech from noisy observations, with objectives formulated as direct regression or generative mapping.Source: §2 · p. 2

2.1. Speech enhancement models

• CR predicts the real and imaginary components of clean-speech STFT coefficients, whereas SB learns a stochastic path between noisy and clean representations.CR is optimized using time-domain mean squared error; SB uses a weighted prediction objective in time and STFT domains.Source: §2.1 · p. 2

2.2. Training procedure

• The random-sampling baseline forms each mini-batch by randomly selecting B paired clean-noisy training examples.Source: §2.2 · p. 2

3. PROPOSED METHOD

• Because clean-speech properties correlate with enhancement gains, the method investigates whether controlling their distribution during training can improve generalization.It changes how examples are sampled, independently of the chosen characteristic.Source: §3 · p. 2

3.1. Speech characteristics

• The characteristic is extracted from each clean utterance and used only to assign its sampling group, not as an enhancement-model input.Experiments use mean third formant frequency F3 and standard deviation of fundamental frequency f0.Source: §3.1 · p. 2

3.2. Stratified mini-batch sampling

• Each batch samples B/J utterances from each of J quantiles, with sampling without replacement within an epoch and reshuffling between epochs.J is chosen so B is divisible by J, producing equal representation across the selected characteristic's range.Source: §3.2 · p. 2
• Quantile boundaries remain fixed, and the sampler can be applied to different SE models without changing their architectures, objectives, or optimization procedures.Source: §3.2 · p. 2

4.1. Datasets

• Experiments use WSJ0 English and CROWD Spanish speech, plus PC-GITA speech from people with Parkinson’s disease and neurotypical controls.PC-GITA evaluation uses speaker-independent 10-fold cross-validation.Source: §4.1 · pp. 2–3
• Noisy mixtures use bus, cafe, pedestrian-area, and street noises, with mixture SNR sampled uniformly from −6 dB to 14 dB.Source: §4.1 · p. 3

4.2. Model architectures and training

• Signals are transformed to the STFT domain using a 510-sample window, 128-sample hop, and Hann window, then compressed with α = 0.5 and β = 0.33.Source: §4.2 · p. 3
• CR and SB models use NCSN++M U-Nets, with 22.1 million and 25.2 million trainable parameters, respectively.Models use Adam with a 10^-4 learning rate and a maximum of 1000 epochs; training stops after 20 consecutive epochs without validation-loss improvement.Source: §4.2 · p. 3

4.3. Evaluation

• Evaluation reports changes in ESTOI, PESQ, fwSSNR, and SI-SDR relative to the noisy mixture, using clean speech as the reference.Source: §4.3 · p. 3

5.1. SE performance using standard and stratified mini-batch sampling

• With B=8 and J=4 F3 quantiles, CR and SB were tested on WSJ0, CROWD, and PC-GITA, with PC-GITA split by speaker group.The main experiments stratified training utterances by F3.Source: §5.1 · p. 3, §5 · p. 3, Table 1 · p. 4
• Across three datasets and both speaker groups, stratification maintains or improves performance; CR gains extend to pathological speech, while SB is near random on PC-GITA.The magnitude of gains varies by dataset, model architecture, and evaluation metric.Source: §5.1 · p. 3, Table 1 · p. 4

5.2. Effect of the acoustic characteristic used for stratification

• On WSJ0, mean F3 stratification yields larger E-STOI and fwSSNR gains than f0 standard-deviation stratification, while PESQ and SI-SDR remain comparable.The comparison suggests that diversity is more beneficial along a dimension reflecting enhancement difficulty.Source: Table 2 · p. 4, §5.2 · p. 4

5.3. Robustness to stochastic mini-batch sampling

• Across three WSJ0 training runs, F3-stratified sampling outperforms random sampling on all four metrics, with small standard deviations.The runs use the same random seeds for both sampling strategies.Source: Table 3 · p. 4, §5.3 · p. 4

• The strategy partitions training data by a clean-speech characteristic, such as mean F3, and enforces equal representation of its quantiles in each mini-batch.Source: §6 · p. 4
• The strategy improves performance across multiple models, datasets, and metrics, with gains varying across experimental conditions.Source: §6 · p. 4
• Future work will examine joint diversification, curriculum learning, batch size and quantile choices, and subjective evaluation of enhanced speech.The method requires no modification to the enhancement architecture, loss function, or training procedure.Source: §6 · p. 4

• The study retrospectively used human-subject data, and the attached license confirmed ethical approval was not required.Source: §7 · p. 4