Source-linked AI summary
Large-Scale Domain Adaptation via Teacher-Student Learning
Jinyu Li, Michael L. Seltzer, Xi Wang, Rui Zhao, Yifan Gong
TL;DR
Target-domain acoustic adaptation often still depends on substantial transcribed data, despite transcription costs and the limits of unsupervised adaptation. This paper trains a student from teacher-generated posteriors using unlabeled parallel source–target speech. It reports up to 44% relative WER reduction on CHiME-3, with gains varying across tasks and children’s speech conditions.
Problem
Domain adaptation can bootstrap a system from a trained model, but supervised methods still require target-domain transcriptions, which are costly to obtain in large quantities.
Source: §1 · p. 1Method
The student minimizes KL divergence from teacher posteriors computed on source inputs, using paired target inputs without transcriptions.
Source: §2 · p. 2Results
On CHiME-3, the student achieved up to 44% relative WER reduction over the source model; gains on children’s speech were significant for girls but challenging for boys.
Source: §5 · p. 4Takeaways & Limitations
Increasing unlabeled parallel-data volume improved model robustness, particularly when target-domain training data was simulated.
Source: §5 · p. 4Improving boys’ speech WER was very challenging because the employed voice-conversion method had limitations.
Source: §4.3 · p. 4, §5 · p. 4
Abstract
from arXiv · showhide
High accuracy speech recognition requires a large amount of transcribed data for supervised training. In the absence of such data, domain adaptation of a well-trained acoustic model can be performed, but even here, high accuracy usually requires significant labeled data from the target domain. In this work, we propose an approach to domain adaptation that does not require transcriptions but instead uses a corpus of unlabeled parallel data, consisting of pairs of samples from the source domain of the well-trained model and the desired target domain. To perform adaptation, we employ teacher/student (T/S) learning, in which the posterior probabilities generated by the source-domain model can be used in lieu of labels to train the target-domain model. We evaluate the proposed approach in two scenarios, adapting a clean acoustic model to noisy speech and adapting an adults speech acoustic model to children speech. Significant improvements in accuracy are obtained, with reductions in word error rate of up to 44% over the original source model without the need for transcribed data in the target domain. Moreover, we show that increasing the amount of unlabeled data results in additional model robustness, which is particularly beneficial when using simulated training data in the target-domain.
1. Introduction
The paper addresses the limits of target-domain transcription by adapting with unlabeled parallel source–target data. It uses teacher/student learning and reports improvements in noisy-speech and children’s-speech recognition.
1. Introduction
- Supervised domain-adaptation methods still require target-domain transcriptions, while decoder-generated labels leave a large performance gap to supervised adaptation..Source: §1 · p. 1
- Parallel speech pairs can be collected simultaneously with close-talking and distant microphones or simulated by replaying or digitally transforming existing speech.The examples include noisy or reverberant environments.Source: §1 · p. 1
- Teacher/student learning uses source-model posterior probabilities as soft labels to train a target-domain student on paired data.The source and target samples are paired but unlabeled.Source: §1 · p. 1
- The approach reports improved noisy-speech recognition and reduced WER for children’s speech without requiring target-domain transcriptions.The noisy model also showed better robustness to unseen noise conditions than multi-condition training.Source: §1 · p. 1
- Earlier distillation approaches retained hard-label supervision, whereas this approach relies solely on teacher soft labels and can use substantially more adaptation data.The paper reports that its unlabeled training data is an order of magnitude larger than the data used to create the source model.Source: §1 · p. 1
2. T/S learning for domain adaptation
The framework adapts a student network by minimizing the KL divergence between teacher outputs on source inputs and student outputs on paired target inputs. Teacher-generated posteriors provide supervision without transcriptions, and larger parallel datasets can broaden acoustic coverage.
2. T/S learning for domain adaptation
- The method minimizes KL divergence between teacher and student output distributions using unlabeled parallel data.Only the student network’s parameters are optimized.Source: §2 · p. 2
- Unlike the original T/S formulation, the teacher and student process different paired inputs from the source and target domains.The teacher receives source data and the student receives target data.Source: §2 · p. 2
- For each mini-batch, the teacher and student compute posteriors from their respective inputs, then the student receives the error signal and is updated until convergence..Source: §2 · p. 2
- Teacher posteriors supervise student training without transcriptions, while more parallel data can improve acoustic-space coverage and make target behavior resemble the teacher’s.The supervision signal is obtained by passing source data through the teacher network.Source: §2 · p. 2
3. Parallel corpus generation
The study considers parallel-corpus generation for clean-to-noisy and adult-to-child adaptation. Noisy pairs can be recorded or simulated, while child-speech pairs are generated by transforming adult speech.
3. Parallel corpus generation
- The study explores clean-to-noisy and adult-to-child adaptation using unlabeled real or simulated source–target training pairs..Source: §3 · p. 2
- Clean/noisy pairs can be recorded by replaying clean speech in noise or simulated by digitally mixing noise into clean speech..Source: §3 · p. 2
- Because natural adult and child speech is hard to synchronize, the study simulates children’s speech by warping adult-speech frequencies and raising pitch.It selects a warping factor of 0.1, which moves formant frequencies higher, rather than targeting a specific child voice.Source: §3 · p. 2
- For adult-to-child adaptation, the student is trained against adult-speech teacher posteriors using child-speech inputs..Source: §3 · p. 2
4. Experimental evaluation
Evaluation uses a 4-layer LSTM-RNN baseline trained on 375 hours of US-English Cortana audio, testing adaptation on noisy speech and children’s speech. Results vary with domain mismatch and the amount and quality of parallel training data.
4.1 Noisy Cortana task
- On original Cortana speech, student models beat the teacher below 35 dB SNR, while the gap above 35 dB was small.The parallel training used simulated noisy utterances at 5–20 dB SNR.Source: Table 2 · p. 3, §4.1 · p. 3
4.2 CHiME-3 task
- 19.89% WER on CHiME-3, a 14% relative WER reduction from 23.16%, required 3400h of parallel Cortana data; 375h adaptation did not improve performance.Both the noisy transcribed-data model and the 375h parallel-data student failed to improve on the baseline under this task mismatch.Source: §4.2 · p. 3, Table 3 · p. 4
- 12.99% WER on CHiME-3 with clean CHiME-3 source data and noisy targets corresponds to about 44% relative WER reduction over the original source model.The reported comparison uses the final student trained with real and simulated data from all channels.Source: §4.2 · p. 3, Table 4 · p. 4
4.3 Children’s speech
- From the male-only baseline, adaptation improved girls’ WER from 39.05% to 25.03%, then 21.19% with more parallel data; boys’ WER moved from 34.16% to 32.32%, then 31.89%.Only transformed utterances classified as children’s speech were used as target data.Source: Table 5 · p. 4, §4.3 · p. 4
- For the all-speaker model, 3400h adaptation changed girls’ WER from 18.38% to 16.65% but degraded boys’ WER; 375h adaptation did not improve performance.The authors attribute boys’ weaker results to the difficulty of creating accurate boys’ speech through voice transformation.Source: Table 6 · p. 4, §4.3 · p. 4
5. Conclusions
The teacher/student approach adapts models using unlabeled parallel data, with the student’s behavior approaching the source teacher as training data grows. Results include up to 44% relative WER reduction on CHiME-3, while improvements on children’s speech vary by gender and voice-conversion limits.
5. Conclusions
- Increasing unlabeled parallel training data brings the target-domain student closer to the source-domain teacher and adds robustness, especially with simulated target-domain data.Source: §5 · p. 4
- On CHiME-3, the student model achieves up to 44% relative WER reduction over the source model.Source: §5 · p. 4
- On noisy Cortana, the student model achieves 16.11% WER, close to the source model’s 15.62% WER on original Cortana.Source: §5 · p. 4
- For children’s speech, the student significantly improves girls’ WER, while boys’ WER remains challenging to improve because of voice-conversion limitations.Source: §5 · p. 4