Extracting Domain Invariant Features by Unsupervised Learning for Robust Automatic Speech Recognition

• ASR performance can degrade when testing domains mismatch training because speech contains nuisance variability from speakers, noise, and rooms.Source: §1 · p. 1
• Domain-invariant features offer an alternative to parallel-data enhancement and exhaustive labeled multi-domain training.Source: §1 · p. 1
• FHVAE factorizes sequence-level and segment-level attributes, and the paper uses segment-level latent variables as unsupervised ASR features.Source: §1 · p. 1
• The proposed features reduce absolute WER by 27% to 41% versus filter bank features and by 14% to 16% versus VAE-based features on Aurora-4 and CHiME-4.Source: §1 · p. 1

2.1. Modeling a Generative Process of Speech Segments

• A hierarchical generative model represents each sequence with sequence-level and segment-level latent variables under different priors.Source: §2.2 · p. 2

2.2. Extracting Domain Invariant Features from FHVAEs

• FHVAE inference makes z2 capture sequence attributes while z1 captures residual segment attributes needed with z2 to generate speech segments.Source: §2.2 · p. 2
• Because speaker, room, and noise attributes are consistent within utterances, treating utterances as sequences makes z1 the desired domain-invariant linguistic feature.Source: §2.2 · pp. 1–2

2.3. Training FHVAE and Preventing S-Vector Collapsing

• FHVAEs optimize a variational lower bound, but identical sequence vectors would reduce the model to a vanilla VAE.Source: §2.3 · p. 2
• A discriminative objective weighted by α encourages z2 to identify its source utterance and prevents sequence-vector collapse.Source: §2.3 · p. 2

3. EXPERIMENT SETUP

• The evaluation trains ASR on clean data and tests both clean and noisy data, using domain discrepancy as a measure of feature invariance.Source: §3 · p. 2

3.1. Dataset

• Aurora-4 provides clean, channel, noisy, and channel-plus-noisy conditions with artificially added noise, while CHiME-4 supplies real distant-talking noisy recordings.Source: §3.1 · p. 2

3.2. VAE/FHVAE Setup and Training

• VAE and FHVAE models use 20-frame FBank chunks, and extracted posterior means and variances form ASR features aligned to the original utterance length by padding.Source: §3.2 · p. 3

3.3. ASR Setup and Training

• The ASR experiments use fixed LSTM acoustic-model and decoding settings across conditions, with Kaldi for feature extraction and conventional ASR preparation.Source: §3.3 · p. 3

4.1. Baseline

• 14% absolute WER reduction separates FHVAE from VAE features across all mismatched Aurora-4 conditions, with only slight matched-domain degradation.FBank instead shows a 49% to 79% absolute WER increase in mismatched conditions.Source: §4.1 · pp. 3–4
• FHVAE features retain domain-invariant linguistic information in z1 while encoding domain-related information in z2, unlike VAE features concentrated in z.The experiment supports factorization as the mechanism behind the robustness difference.Source: §4.1 · p. 4

4.2. Comparing Model Architectures

• 24.30% averaged WER is obtained with three layers and 256 hidden units per layer, although higher capacity helps noisy condition B but harms channel-mismatch condition C beyond selected capacities.Architecture capacity therefore affects conditions differently rather than improving every mismatch uniformly.Source: §4.2 · p. 4

4.3. Effect of FHVAE Discriminative Training

• Increasing discriminative training weight from 0 to 10 consistently improves all four conditions, whereas weight 20 degrades performance by constraining latent-sequence modeling capacity.The study attributes the initial improvement to better factorization of segment and sequence information.Source: §4.3 · p. 4

4.4. Choice of Sequence Label

• Supervised sequence labels do not improve performance because concatenating utterances can move noise or speaker information into latent segment variables.The intended invariance depends on which attributes remain consistent within each training sequence.Source: §4.4 · p. 4

4.5. Use of S-Vector

• S-vector augmentation significantly degrades WER because z1 and µ2 together resemble VAE's z, while µ2 distributions mismatch between training and testing.The s-vector summarizes sequence-level attributes and is concatenated with each frame representation.Source: §4.5 · p. 4

4.6. Verifying Results on CHiME4

• 7% absolute WER decrease is achieved on CHiME-4 by increasing FHVAE encoder/decoder layers from 1 to 3, while latent segment variables outperform VAE and FBank features.This reproduces the Aurora-4 trends on real distant-talking noisy recordings.Source: §4.6 · p. 4, Table 2 · p. 4

5. CONCLUSION AND FUTURE WORK

• 41% and 27% absolute WER reductions are achieved on Aurora-4 and CHiME-4, respectively, compared with FBank and VAE-based features in mismatched domains.The paper concludes that FHVAE-derived features provide superior robustness across both datasets.Source: §5 · p. 4