Source-linked AI summary
Mitigating Accent-Language Confusion in Self-Supervised Speech Representations for Language Identification
Minu Kim, Jihwan Lee, David R. Mortensen, Shrikanth Narayanan
TL;DR
LID should recognize the language being spoken, yet L2-accented speech can be misclassified as the speaker’s L1, and existing mitigations require extra processing, components, or adaptation. The paper estimates an L1-bias direction solely from native speech and projects it out before the frozen LID head. Across models and corpora, this restores L2 English identification while preserving native predictions.
Problem
L2-accented speech can be misclassified as the speaker’s L1, while existing mitigation approaches require extra processing, components, or parameter adaptation.
Source: §1 · p. 1Method
The method estimates an L1-bias direction using native target-language and L1 speech, then projects it out before the frozen LID head.
Source: §1 · p. 1, §2 · p. 2, §2.1 · p. 2Results
Takeaways & Limitations
Accent-induced L1 bias can be corrected directly in representation space without non-native training data or model adaptation.
Source: §5 · p. 4The approach assumes non-native English features are displaced along the geometric axis separating native English and native L1 speech.
Source: §2.1 · p. 2
Abstract
from arXiv · showhide
Spoken language identification (LID) aims to recognize the target language regardless of accent. In practice, however, LID models fine-tuned from self-supervised speech representations frequently confuse accents with languages, misclassifying non-native (L2) speech as the speaker's first language (L1). We show that non-native speech representations lie between native target-language and native L1 poles, causing systematic misclassification. To address this, we introduce a geometric projection that estimates an L1-bias direction solely from native speech and removes it before the frozen LID head. Across five MMS-LID models and non-native corpora, this projection substantially improves target language identification for L2-accented speech while preserving predictions for native speech. These results show that accent-induced L1 bias can be corrected directly within the representation space without L2 training data or model adaptation.
1. INTRODUCTION
LID systems can mistake L2-accented speech for the speaker’s L1, reflecting accent–language confusion. The paper proposes removing an L1-biased direction estimated from native speech and reports improved L2 English identification while preserving native predictions.
1. INTRODUCTION
- LID should identify the language being spoken, but stronger accents degrade accuracy and errors often shift toward the speaker’s L1.The passage links this confusion to interlanguage phonology transfer, in which L2 speakers realize unfamiliar sounds through L1 categories,.Source: §1 · p. 1
- Existing mitigations combine acoustic predictions with ASR, aggregate chunks, incorporate phonetic sequences, or fine-tune on code-switched speech, requiring extra processing, components, or adaptation.The described approaches include lexicon-free ASR, phonetic sequences [2], and code-switched speech.Source: §1 · p. 1
- The method estimates an L1-bias direction from native target-language and L1 speech, then projects out that component before classification without additional data or training.The approach operates directly within representation geometry.Source: §1 · p. 1
- Across five MMS-LID models and three corpora, the projection restores misclassified L2 English while leaving native and correctly classified L2 speech intact.Errors disproportionately shift toward speakers’ L1 or related languages; random-direction removal yields no such gain.Source: §1 · p. 1
2. METHOD
The method estimates an L1-biased direction from native speech, removes that single component from representations, and sends corrected vectors to the unchanged LID head.
2. METHOD
- The method targets accent–language confusion directly within representation geometry without additional data or training.Source: §2 · p. 1
2.1. Native-only direction estimation
- A native-only reference set labels English 0 and non-English L1 speech 1; an ℓ2-regularized separator estimates their axis without L2 samples.The auxiliary separator is discarded after estimating the direction.Source: §2.1 · p. 2
2.2. L1 bias removal via geometric projection
- The projection removes only the one-dimensional component along the estimated L1-biased direction, then passes the corrected representation to the frozen LID head.The correction requires neither L2-accented speech nor model retraining.Source: §2.2 · p. 2
3. EXPERIMENTAL SETUP
Experiments use three English corpora with diverse speaker L1 backgrounds, estimate the direction from native FLEURS speech, and evaluate native-language predictions separately.
3.1. Data
- 3,035, 2,250, and 3,214 clips are retained from Svarah, L2-ARCTIC, and ALLSSTAR, respectively, covering 14, 6, and 17 L1 groups.Source: §3.1 · p. 3
- Native English and corresponding native L1 speech from the FLEURS dev split estimate the direction; no accented-English speech is used for estimation.Source: §3.1 · p. 3
- The effect on native-language predictions is evaluated separately using the FLEURS test split.Source: §3.1 · p. 3
4. RESULTS
Across MMS-LID variants, projection improves non-native English identification on Svarah and L2-ARCTIC while leaving native-group predictions largely unchanged. The estimated direction tracks the native English–L1 separation, unlike random directions.
4.1. L1-dependent accent-language confusion
- MMS-LID-4017 identifies 14%, 57%, and 93% of non-native English clips as English in Svarah, L2-ARCTIC, and ALLSSTAR, respectively; most errors favor speakers’ L1 or related languages.This pattern is consistent with contact-driven phonetic convergence and language relationships captured by MMS-LID.Source: §4.1 · p. 3
- Mean Whisper NLL falls from 1.80 on Svarah to 1.51 on L2-ARCTIC and 1.03 on ALLSSTAR, versus 0.90 for native English, tracking English identification inversely.Across L1 groups, performance gain strongly correlates with Whisper NLL (r = +0.77).Source: §4.1 · p. 3, §4.2 · p. 3
4.2. Results across models and corpora
- English identification for non-native English improves 51–71%p on Svarah and 17–24%p on L2-ARCTIC across MMS-LID variants.The effect is minimal on ALLSSTAR, where non-native English is already correctly identified; native English and native L1 rates change negligibly.Source: §4.2 · p. 3, Figure 3 · p. 4
- Projecting out the FLEURS-dev-estimated direction redirects most non-native English representations toward English while keeping native English and native L1 on their respective sides.Non-native representations lie between the native poles; removing the component isolates L1 bias without collapsing their distinction.Source: §4.2 · p. 3, Figure 4 · p. 4
4.3. Validity of the estimated direction
- The estimated direction aligns with the native English–L1 separation, while removing 20 random directions yields no baseline improvement and removing the estimated direction improves Svarah and L2-ARCTIC.The comparison indicates that gains depend on targeting the specific L1-bias axis rather than arbitrary subspace modification.Source: §4.3 · pp. 3–4, Figure 5 · p. 4
5. CONCLUSION
The paper concludes that L1 bias in non-native English can be corrected by projecting out a direction estimated from native speech, restoring English identification while preserving native predictions.
5. CONCLUSION
- Across models and corpora, projecting out a native-speech-estimated English–L1 direction restores L2-accented English identification while leaving native English and native L1 speech unaffected.Removing random directions yields no gains; the correction requires neither non-native training data nor model adaptation.Source: §5 · p. 4