Source-linked AI summary
Continuous Learning for Children's ASR: Overcoming Catastrophic Forgetting with Elastic Weight Consolidation and Synaptic Intelligence
Edem Ahadzi, Vishwanath Pratap Singh, Tomi Kinnunen, Ville Hautamaki
TL;DR
Children’s ASR remains underexplored in online learning, where sequential data and minors’ privacy constraints make retaining earlier data impractical. This study evaluates EWC and SI with a MyST protocol that simulates speaker-population drift. Both methods mitigate forgetting relative to standard fine-tuning while adapting to new data.
Problem
The utility of continual-learning methods for children’s ASR remains unanswered, despite the relevance of sequential learning under minors’ data-deletion constraints.
Source: §1 · p. 1Method
The study evaluates EWC and SI using a custom MyST protocol that simulates gradual drift in speaker population.
Source: §1 · p. 1Results
Takeaways & Limitations
The MyST results show that either parameter-regularization technique can mitigate catastrophic forgetting in children’s online speech recognition.
Source: §5 · p. 4The MyST data are relatively homogeneous and drawn from a narrow age group, grades 3–5, constraining performance to a tight range.
Source: §4 · p. 4
Abstract
from arXiv · showhide
In this work, we present the first study addressing automatic speech recognition (ASR) for children in an online learning setting. This is particularly important for both child-centric applications and the privacy protection of minors, where training models with sequentially arriving data is critical. The conventional approach of model fine-tuning often suffers from catastrophic forgetting. To tackle this issue, we explore two established techniques: elastic weight consolidation (EWC) and synaptic intelligence (SI). Using a custom protocol on the MyST corpus, tailored to the online learning setting, we achieve relative word error rate (WER) reductions of 5.21% with EWC and 4.36% with SI, compared to the fine-tuning baseline.
1. Introduction
Children’s ASR remains challenging despite its importance, while sequential data and minors’ privacy constraints make online learning especially relevant. This study addresses the open question of whether EWC and SI can mitigate forgetting in children’s ASR.
1. Introduction
- Children’s ASR performance remains low, with distinct acoustic and linguistic challenges and relatively scarce training data.The study notes children’s rapid developmental changes, shorter vocal tracts, higher pitch, different formants, and inconsistent articulation.Source: §1 · p. 1
- Sequential data and minors’ data-deletion requirements make storing and revisiting earlier datasets impractical for later training.The paper cites GDPR and COPPA requirements to delete minors’ data once the immediate processing purpose is met.Source: §1 · p. 1
- Fine-tuning and retraining can cause catastrophic forgetting as models focus on new data and lose previously acquired knowledge.Retraining is also computationally demanding.Source: §1 · p. 1
- Prior mitigation approaches include replay, parameter regularization, and dynamic architectures; this study tests EWC and SI for children’s ASR using a MyST speaker-drift protocol.The experiments also address which and how many ASR models to retain during online learning.Source: §1 · p. 1
2. Methods
EWC and SI address catastrophic forgetting by penalizing changes to parameters important for prior tasks. The methods differ in how they estimate parameter importance: EWC uses Fisher information, while SI accumulates each parameter’s contribution to loss reduction during training.
2.1. Overview of Parameter-Based Continual Learning
- Both methods add a regularization penalty to the new-task loss, constraining changes to parameters identified as important for previous tasks.The objective uses θ* for parameters learned from earlier tasks and β to control regularization strength.Source: §2.1 · p. 2
2.2. Elastic Weight Consolidation
- EWC penalizes deviations from the most recently learned task parameters, weighting the quadratic penalty by importance measured with the Fisher information matrix.Its formulation approximates the previous-task posterior as a Gaussian using a Laplace approximation.Source: §2.2 · p. 2
2.3. Synaptic Intelligence
- SI tracks how parameter updates contribute to loss reduction and uses accumulated per-parameter importance to penalize changes during subsequent tasks.The accumulated contribution is computed along the training trajectory and decomposed by parameter.Source: §2.3 · pp. 2–3
3. Experimental Setup
The setup uses MyST data organized into sequential utterance batches with changing speaker overlap, and evaluates Whisper small under different continual-learning and model-selection strategies.
3.1. Dataset
- MyST provides approximately 473 hours of children’s conversational English from 1,371 students, with speaker-disjoint training, development, and test partitions.After filtering, the splits contain 71,939 training, 11,592 development, and 12,578 test utterances.Source: §3.1 · p. 3
- Ten sequential utterance batches model an evolving dataset with partially overlapping speakers and exponentially decaying retention from earlier batches.The retained-speaker count scales by (1/2)^(i-j); Table 1 summarizes each batch’s new and total speakers, utterances, and hours.Source: §3.1 · p. 3, Table 1 · p. 3
3.2. Model
- Whisper small, pretrained on around 680,000 hours of audio, serves as the ASR model for the continual-learning framework.The paper describes its Transformer encoder-decoder as a tradeoff between accuracy and computational efficiency for resource-constrained lifelong learning.Source: §3.2 · p. 3
3.3. Experiments
- Experiments compare incremental fine-tuning without forgetting controls against EWC and SI, paired with no selection, rolling-window-of-three, or best-of-all-so-far model caching.The selection strategies trade lower overhead or storage against validation and the ability to revisit earlier models.Source: §3.3 · p. 3
- Models train for three epochs, with hyperparameters selected by development-set WER and performance reported as WER with 95% confidence intervals.The tested learning rates range from 1.25 × 10^-5 to 1 × 10^-10; EWC and SI regularization strengths are 0.1 and 0.01.Source: §3.3 · p. 3
4. Results and Discussion
Without continual-learning techniques, WER drifts upward; EWC and SI show relatively stable batch-wise performance. Model selection can cause temporary spikes, and WER eventually plateaus across the approaches amid homogeneous MyST data.
4. Results and Discussion
- Without continual-learning techniques, WER drifts upward through batch 4 before stabilizing at a marginally higher level.The authors link the upward trend to the absence of mechanisms protecting important parameters.Source: §4 · pp. 3–4
- 18.94% to 18.75% development WER and 21.42% to 21.11% test WER for EWC by batch 4; SI moves 18.94% to 18.80% and 21.42% to 21.30%.These are the reported changes from utterance batch 1 to batch 4 under the no-selection strategy.Source: Table 2 · p. 4, §4 · p. 4
- Model selection can recover from detrimental updates, but a speaker shift can briefly raise WER when the fallback model lacks the latest speaker data.For example, EWC with RW3 development WER rises to 21.07% at batch 5 from 19.79% in the previous batch before recovery.Source: §4 · p. 4
- WER eventually plateaus across approaches; similar batches and MyST’s grades 3–5 age range contribute to data homogeneity and constrain performance to a tight range.Source: §4 · p. 4
- EWC and SI have confidence-interval ranges of 0.28–0.31 and 0.29–0.31, respectively; fine-tuning without continual learning has average WER about 0.29–0.32, with occasional intervals up to 0.40.The authors associate the narrower intervals with consistency across batches and the wider range with greater variability.Source: Figure 1 · p. 4, §4 · p. 4
5. Conclusion
On MyST, EWC and SI mitigate catastrophic forgetting while preserving earlier knowledge and capturing new information; SI is faster than EWC. The evaluation is constrained by MyST’s homogeneity, with varied data quality left for future study.
5. Conclusion
- EWC and SI outperform standard fine-tuning in preserving earlier knowledge while capturing new information on MyST.Both techniques mitigate catastrophic forgetting in this online children’s ASR setting.Source: §5 · p. 4
- SI and EWC have comparable WER, but SI is faster and recommended as the first choice for practitioners.Source: §5 · p. 4
- MyST’s homogeneous data constrain the evaluation; future work may test noise and reverberation to assess robustness to varied data quality.Source: §5 · p. 4