PaperlayerView exact revision on arXiv

Source-linked AI summary

Phonological Interference in Multilingual Speech Models

Moran Yanuka, Raja Giryes, Moris Alper

arXiv:2610.11275v1cs.CLcs.LG

TL;DR

Phoneme-level multilingual speech models can impose one language’s sound system on speech that switches languages or is absent from training. This paper measures and traces that interference to a low-dimensional language subspace, then tests a local-window repair. The models lose many language-specific phonemes on code-switched input, while the repair substantially reduces interference across three models.

  • Problem

    Phoneme-level models often fail on speech that matches no single training language, including code-switched speech and languages unseen during training.

    Source: §1 · p. 1

  • Method

    The paper measures interference through unshared-phoneme recall, traces it to a language subspace, and replaces its coordinates with short-window estimates at inference time.

    Source: §1 · p. 2, §2 · p. 3

  • Results

    On code-switched input, models lose 32% to 79% of unshared phoneme recall, while WLE reduces interference by 34%-69% across all three models.

    Source: §1 · p. 2, §2.3 · p. 5

  • Takeaways & Limitations

    WLE needs no language label or external language identifier and leaves phone recognizers’ monolingual error rates essentially unchanged.

    Source: §1 · p. 2

    Code-switched experiments use utterances that switch once between two languages; broader switching patterns remain future work.

    Source: §6 · p. 10

Abstract

from arXiv · show

Phoneme-level models transcribe or generate speech as a sequence of phonemes, the smallest sound units that distinguish words. These models enable fine-grained pronunciation control and understanding, yet often fail on input that does not match any single training language, such as speech alternating between two languages, known as code-switching, or low-resource languages absent from training. We identify a systematic failure mode behind this, phonological interference: models assume the input is in a single language and impose its phonology, overriding local phoneme-level decisions that conflict with the assumed language. We measure interference by how often a model retains phonemes that one language has but the other lacks. On code-switched input, two phone recognizers (speech-to-phoneme models) and a phoneme-conditioned text-to-speech model lose 32% to 79% of these phonemes, but lose far fewer of the phonemes both languages share. On unseen languages, we find that phone recognizers impose the phonology of the training language they assign to the speech, and the more confident the assignment, the more they lose phonemes the unseen language has but the assigned language lacks. We probe the models' language estimate from their internal activations, and trace interference to a low dimensional subspace. On monolingual speech, steering this subspace toward another language makes the model lose the phonemes that only the original language uses and produce phonemes that only the target language has. We introduce windowed language estimation (WLE), an inference time repair that replaces the model's language estimate in this subspace with one computed from a short window around each position. On code-switched input, WLE removes 34% to 69% of the interference in all three models, and in the recognizers it leaves monolingual performance essentially unchanged.

1 INTRODUCTION

Phoneme-level multilingual speech models impose an inferred language’s phonology on local phoneme decisions, causing interference on code-switched and unseen-language speech. The paper traces this behavior to a low-dimensional language subspace and introduces windowed language estimation (WLE) as an inference-time repair.

1 INTRODUCTION

  • Phonological interference occurs when a model infers a language from context and imposes its phonology, overriding local phoneme decisions such as English [ô] becoming French [K].The paper examines this failure in code-switched speech and languages absent from training.Source: §1 · pp. 1–2
  • The paper measures interference through unshared-phoneme recall and evaluates two phone recognizers and a phoneme-conditioned TTS model on code-switching, plus recognizers on 43 unseen languages.The contributions span measurement, subspace analysis, and an inference-time repair.Source: §1 · p. 2
  • Manipulating a low-dimensional language subspace makes models drop phonemes unique to the original language and replace them with target-language phonemes, while mostly preserving shared phonemes.An analogous subspace in Whisper also changes the output transcript’s language.Source: §1 · p. 2
  • WLE reduces phonological interference by 34%-69% across all three models while leaving phone recognizers’ monolingual error rates essentially unchanged.It replaces language-subspace coordinates at each position with estimates computed from a short window, without requiring a language label or external language identifier.Source: §1 · p. 2

2 MEASURING PHONOLOGICAL INTERFERENCE

The section defines phonological interference as a loss of language-specific phonemes in favor of another language’s phonemes and measures it by comparing unshared with shared phoneme recall. It finds this selective loss both in code-switched speech and in unseen-language recordings when recognizers confidently assign them to a training language.

2 MEASURING PHONOLOGICAL INTERFERENCE

  • Interference is defined as losing phonemes one language has but another lacks in favor of the other language’s phonemes, measured by unshared versus shared recall.An unshared phoneme counts as lost only when deleted or replaced by a phoneme of the other language; for code-switching, the drop is measured against processing the worse span alone.Source: §2 · p. 3

2.1 MODELS

  • The study tests two phone recognizers and a phoneme-conditioned TTS model trained on monolingual speech, using seven seen test languages and 43 unseen DoReCo languages.POWSM and PhoneticXeus are the recognizers; the TTS model is fine-tuned on IPA input across 16 languages without language tags or embeddings.Source: §2.1 · p. 3, §2.2 · p. 4

2.2 EVALUATION

  • Evaluation uses natural SwitchLingua code-switches and controlled Spliced FLEURS pairs, with the latter balancing language position and duration.SwitchLingua contains 288 recordings across six pairings with English; Spliced FLEURS contains 300 concatenated samples.Source: §2.2 · p. 4

2.3 RESULTS: CODE-SWITCHED INPUT

  • 32% to 79% of unshared phoneme recall is lost across all three models on code-switched input, 1.9 to 8.2 times the drop in shared phonemes.Most losses are replaced by phonemes of the other language; on SwitchLingua, this accounts for 65% of TTS, 70% of PhoneticXeus, and 78% of POWSM’s CTC-head exact-match losses.Source: Figure 3 · p. 5, §2.3 · p. 5
  • On Spliced FLEURS, when the other language lacks a phoneme, recall odds fall an extra factor of 2.2 in POWSM and 1.3 in PhoneticXeus; the TTS effect is not significant.The test holds phoneme and language pair fixed while comparing recall drops next to languages that lack versus have the phoneme.Source: §2.3 · p. 5

2.4 RESULTS: UNSEEN LANGUAGES

  • From the lowest to highest probe-confidence third, unshared recall falls by 0.053 for POWSM and 0.060 for PhoneticXeus, while shared recall rises slightly.The lost phonemes are mostly replaced by phonemes of the training language assigned to the unseen-language recording.Source: §2.4 · pp. 5–6, Figure 4 · p. 6
  • The unseen-language evaluation covers phone recognizers only because scoring TTS would require PhoneticXeus, which itself shows interference on unseen languages.Splitting audio at the language boundary does not solve this scoring problem for monolingual input.Source: §2.4 · p. 6

3 WHAT CAUSES PHONOLOGICAL INTERFERENCE?

The section tests whether interference is driven by a low-dimensional language subspace by editing model representations toward another language. These edits reproduce selective phoneme loss, while controls and a Whisper analysis probe the specificity and broader reach of the mechanism.

3 WHAT CAUSES PHONOLOGICAL INTERFERENCE?

  • The language subspace spans 15 dimensions, under 2% of each model’s hidden dimensions, and is constructed from whitened language centroids.The edit replaces the representation’s projection onto this subspace with the target language centroid while leaving the rest unchanged.Source: §3 · pp. 6–7
  • Steering the language subspace toward another language makes recognizers lose 86–88% of unshared recall and TTS lose about half, with shared-phoneme drops only a quarter to a third as large.A single direction between the two languages produces 97% of the full 15-dimensional edit’s unshared-recall drop.Source: §3 · p. 7
  • Random-subspace edits barely change unshared recall, while language-information erasure causes about half the targeted loss in recognizers and 66% in TTS.An edit toward B adds 7.3 phonemes per 100 used only by B in POWSM, 4.6 in PhoneticXeus, and 0.9 in TTS; an edit toward C adds essentially none of B’s.Source: §3 · p. 7
  • In Whisper, editing the encoder’s language subspace toward B raises the probability of B’s language token by 0.20 and shifts the transcript toward B.Editing toward a third language or through a random subspace does neither.Source: §3 · p. 7

4 WINDOWED LANGUAGE ESTIMATION MITIGATES INTERFERENCE

Windowed language estimation (WLE) replaces a model’s utterance-wide language estimate with a local one to mitigate phonological interference. It reduces interference across the three models while largely preserving recognizer performance on monolingual speech.

4 WINDOWED LANGUAGE ESTIMATION MITIGATES INTERFERENCE

  • WLE replaces each frame’s language-subspace coordinates with estimates from a local window; TTS estimates those coordinates from phoneme windows rather than audio windows.The recognizers use sliding audio windows, while TTS tests fixed windows and language-span windows.Source: §4 · pp. 7–8
  • Evaluation compares WLE on a whole recording against unedited spans decoded alone, and measures its monolingual recognizer cost with PER and PFER.Controls include an equal-rank random subspace and a sliding window that replaces every coordinate.Source: §4 · p. 7
  • WLE removes 69% of interference in TTS with the span window, 34% with the fixed window, 58% in PhoneticXeus, and 36% in POWSM.Table 1 reports POWSM with its CTC head.Source: Table 1 · p. 8
  • WLE reduces interference with windows from 1 to 4 seconds, using the model’s own representations without an external language-identification system.The TTS fixed window spans about 2 seconds and requires no language label.Source: §4 · p. 8
  • Recognizer WLE stays within ±0.35 pp of baseline monolingual PER and ±0.15 pp of PFER, while naive sliding windows degrade both more, except POWSM’s DoReCo PFER.The sliding window raises PER by 1–3 pp and PFER by up to 1.1 pp; a random subspace produces no meaningful change.Source: §4 · pp. 8–9, Table 2 · p. 8
  • In recognizers, the Oracle removes more interference than WLE, indicating room for better local estimates; it requires every frame’s language and is impractical for unseen languages.In TTS, the span window matches the Oracle.Source: §4 · p. 9

5 RELATED WORK

The paper situates phoneme-level interference among work on multilingual language confusion, code-switching, and speech-model interpretability. It distinguishes its phoneme-level analysis, causal subspace intervention, and inference-time repair from prior approaches.

Language confusion in text LLMs

  • Prior work documents wrong-language generation and language anchoring in text LLMs; this paper finds an analogous language subspace in speech and edits it using local estimates from the same input.Earlier text-model fixes steer toward a supplied target language or an input-derived source-language anchor.Source: §5 · p. 9

Code-switching in speech models

  • Prior code-switching work measures word-level confusion, documents phoneme confusions without identifying their cause, or improves systems through data and model design; this work measures and repairs phoneme-level interference.The paper also studies unseen languages and both recognition and synthesis.Source: §5 · p. 9

Interpretability of speech models

  • Earlier speech interpretability studies characterize represented information without intervening; this work edits a language subspace to change output and uses same-input windows for inference-time repair.Unlike optimization-based distributed alignment search, WLE uses language-centroid subspaces without optimization.Source: §5 · p. 9

6 CONCLUSION

The paper concludes that multilingual phoneme-level models impose an estimated language’s phonology on code-switched and unseen-language speech, and that WLE reduces code-switch interference. The experiments have boundaries in code-switch patterns, unseen-language interventions, and causal attribution.

6 CONCLUSION

  • Phoneme-level models impose the phonology of the language they estimate, replacing phonemes that language lacks, on code-switched and unseen-language speech.The study traces interference to a low-dimensional language subspace that also sets Whisper’s transcript language.Source: §6 · p. 9
  • WLE substantially reduces code-switch interference without a language label and leaves phone recognizers’ monolingual performance unchanged.The subspace manipulation also reproduces interference on monolingual speech.Source: §6 · p. 9
  • Code-switch experiments use utterances with one switch between two languages; broader patterns remain future work, and unseen-language evidence is observational rather than interventional.Separating architecture, training-data, and labeling-convention contributions remains future work.Source: §6 · p. 10

A ADDITIONAL EXPERIMENTAL DETAILS

The appendix details model settings, data construction, language probing, and phoneme scoring used in the experiments. It also describes example unshared-phoneme replacements.

Phone Recognizers

  • Phone recognizers use unchanged public checkpoints on 16 kHz mono audio; POWSM receives an unknown-language tag, and encoder frames cover 40 ms or 20 ms depending on model.POWSM reports attention-decoder and greedy CTC outputs; PhoneticXeus uses greedy CTC.Source: A ADDITIONAL EXPERIMENTAL DETAILS · p. 14

TTS training and inference

  • TTS training freezes Fun-CosyVoice3-0.5B-2512 and trains LoRA adapters on 30,388 utterances across 16 languages for one epoch.Inference samples four seeds and caps output at 20 speech tokens per input phoneme.Source: A ADDITIONAL EXPERIMENTAL DETAILS · p. 14

Language estimate

  • Language estimation uses multinomial logistic regression on standardized frames, trained on up to 160,000 frames and tested on 5,800 utterances from disjoint speakers.The probe reads selected layers of POWSM, PhoneticXeus, and the TTS language model.Source: A ADDITIONAL EXPERIMENTAL DETAILS · p. 14

SwitchLingua

  • SwitchLingua uses 288 recordings with exactly one language change; word references are phonemized with each word in its own language’s voice, and cuts fall midway between adjacent words.The sample comes from 239 conversations, with at most three recordings per conversation.Source: A ADDITIONAL EXPERIMENTAL DETAILS · p. 14

Spliced FLEURS

  • Spliced FLEURS uses utterance halves joined at language boundaries, with 10 clips for each of 30 ordered language pairs and same-language control splices.Halves come from utterances with at least four seconds of speech; clips have no gap or crossfade.Source: A ADDITIONAL EXPERIMENTAL DETAILS · pp. 14–15

Additional experimental details

DoReCo

  • DoReCo analysis excludes French and covers the remaining 43 languages; clips are split into thirds by assigned-language probe-score rank within each language.An unshared phoneme occurs at least three times in the DoReCo language references and fewer than three times in the assigned language’s training references.Source: A ADDITIONAL EXPERIMENTAL DETAILS · p. 15

Scoring

  • Scoring aligns references and outputs by edit distance; normalized exact matches count as recall, while deletions and substitutions by the other language’s phonemes count as losses.The analysis scores 266 of 288 SwitchLingua recordings and 282 of 300 spliced clips, with 95% percentile bootstrap intervals from 10,000 recording resamples.Source: A ADDITIONAL EXPERIMENTAL DETAILS · p. 15

B DETAILED INTERFERENCE RESULTS

The detailed analyses test whether interference depends on the other language rather than phoneme difficulty or baseline choice, and find that added substitutions are often far from the closest phoneme in that language.

B DETAILED INTERFERENCE RESULTS

  • On Spliced FLEURS, a same-language-half baseline changes recall by at most 0.012, and every interval includes zero, leaving the reported drop nearly unchanged across baselines.Table 6 reports the two baselines for worst-span recall of unshared phonemes.Source: B DETAILED INTERFERENCE RESULTS · p. 15, Table 6 · p. 17
  • The matched-phoneme comparison tests whether recall drops more when the neighboring language lacks the same phoneme, separating that effect from phoneme-specific difficulty.Only phonemes unshared in at least one language pair and shared in another contribute; 56 meet this criterion on Spliced FLEURS.Source: B DETAILED INTERFERENCE RESULTS · pp. 15–16
  • The regression coefficient γ measures the additional log-odds recall drop when the other language lacks a phoneme, controlling for phoneme-specific drops and ordered language-pair differences.γ is zero if the drop depends only on the phoneme and not on whether the other language has it.Source: B DETAILED INTERFERENCE RESULTS · pp. 16–17
  • For the edit experiment, each language is edited toward the other 15 languages, with 262–300 phonemes contributing to the comparison.The model compares unedited output with output edited toward the target language; 95% bootstrap intervals are refit on each resample.Source: B DETAILED INTERFERENCE RESULTS · pp. 17–18
  • Added substitutions are 3.0–4.3 times farther than the closest phoneme of the other language, and only 19%–42% are closest matches.The distance analysis considers substitutions added by the other-language condition; deletions have no distance and are excluded.Source: Table 8 · p. 18, B DETAILED INTERFERENCE RESULTS · p. 18

C UNSEEN LANGUAGES: DETAILED RESULTS

In unseen languages, stronger language assignments accompany lower unshared-phoneme recall and more replacements from the assigned language’s inventory. Probe behavior also varies by model and layer, while WLE does not repair interference when its basis lacks the unseen languages.

C UNSEEN LANGUAGES: DETAILED RESULTS

  • Unshared phoneme recall falls by 0.053 on POWSM and 0.060 on PhoneticXeus from the lowest to highest language-estimate confidence tercile.Shared recall rises slightly, and the unshared-recall pattern also holds for consonants only.Source: C UNSEEN LANGUAGES: DETAILED RESULTS · p. 19
  • Assigned-language inventory replacements rise from 0.57 to 0.63 on POWSM (+0.054) and from 0.58 to 0.61 on PhoneticXeus (+0.033) across confidence terciles.Source: C UNSEEN LANGUAGES: DETAILED RESULTS · p. 19
  • WLE does not reduce interference on DoReCo when its basis contains none of that dataset’s languages; the sliding window also lowers shared recall by 0.02–0.03.On monolingual input, the windowed and full-clip language estimates largely agree, leaving little to correct.Source: C UNSEEN LANGUAGES: DETAILED RESULTS · p. 19

D LANGUAGE PROBE BY LAYER

  • The TTS language probe peaks at block 7 accuracy 0.956, up from 0.694 at block 6, and is more sensitive to an unseen corpus than either recognizer’s probe.Accuracy drops by 0.129 for TTS, versus 0.011 for POWSM and 0.066 for PhoneticXeus; the selected readout layers are 8, 17, and block 7, respectively.Source: D LANGUAGE PROBE BY LAYER · p. 19

E LANGUAGE SUBSPACE ADDITIONAL DETAILS

The additional analyses describe how the language subspace and edit settings are constructed, and show that steering effects vary with rank and depth while controls distinguish targeted edits from generic perturbations.

E LANGUAGE SUBSPACE ADDITIONAL DETAILS

  • The language subspace uses centered language centroids whitened by within-language covariance before selecting the top-k singular directions.Whitening aims to separate languages after accounting for within-language variance; the unwhitened variant removes slightly more unshared phonemes but also damages shared phonemes more.Source: E LANGUAGE SUBSPACE ADDITIONAL DETAILS · p. 19
  • The edit evaluation uses held-out monolingual speech across 16 languages and reduces notation differences in phoneme scoring; reported edits retain 96–98% of unedited transcript length.Four TTS target languages are reported separately after edited outputs degenerated, with longer audio and higher PER and PFER than other targets.Source: E LANGUAGE SUBSPACE ADDITIONAL DETAILS · pp. 19–22, Table 9 · p. 22
  • The edit starts at the earliest block whose cumulative effect reaches 90% of the maximum, selecting block 6 for POWSM and block 14 for PhoneticXeus; rank is 15.Edits with PER above 0.5 relative to unedited output are excluded as destructive.Source: E LANGUAGE SUBSPACE ADDITIONAL DETAILS · p. 20
  • Rank 8 reaches 72% of the rank-15 effect, while a probe inside the subspace reads the correct language at rank 7 with score at least 0.94.The probe requires fewer directions for classification than steering requires to reproduce interference.Source: E LANGUAGE SUBSPACE ADDITIONAL DETAILS · p. 20, Figure 7 · p. 21
  • Editing before block 5 in POWSM or block 11 in PhoneticXeus changes recall by at most 0.03; no single block carries more than 75% of the full-stack effect.Starting at block 0 adds only 0.05–0.07 to unshared-phoneme loss while increasing shared-phoneme damage by a similar amount.Source: E LANGUAGE SUBSPACE ADDITIONAL DETAILS · p. 21, Figure 7 · p. 21
  • A random equal-rank subspace removes at most 0.06 unshared phonemes, while erasing language information removes roughly half without steering toward a particular language.Editing toward B raises B-specific output phonemes by 7.3 for POWSM, 4.6 for PhoneticXeus, and 0.9 for TTS; editing toward C changes this count by at most 0.06.Source: E LANGUAGE SUBSPACE ADDITIONAL DETAILS · p. 21

F LANGUAGE CONFUSION IN WHISPER

Whisper’s whole-clip language choice is linked to code-switched transcript errors: the span outside the picked language is translated or dropped. Editing the encoder’s language subspace can change that choice and the transcript language.

F LANGUAGE CONFUSION IN WHISPER

  • WER rises from 0.16 to 0.26 on SwitchLingua and from 0.15 to 0.54 on Spliced FLEURS when the other language is present.The rise appears in both Spliced FLEURS span positions and all 15 language pairs.Source: Table 10 · p. 23, F LANGUAGE CONFUSION IN WHISPER · p. 23
  • Almost all of the WER increase comes from the span outside Whisper’s picked language, which it usually translates on SwitchLingua or drops on Spliced FLEURS.Whisper transcribes the picked-language span almost as well as when it is alone; it writes nothing for the other span in 218 of 286 Spliced FLEURS clips.Source: F LANGUAGE CONFUSION IN WHISPER · p. 23
  • Editing toward B lowers P(A) by 0.45, raises P(B) by 0.20, and raises the transcript-in-B identifier confidence by 0.22.Whisper picks B in 28% of utterances after the edit, and writes those transcripts in B.Source: Table 11 · p. 24, F LANGUAGE CONFUSION IN WHISPER · p. 24
  • An edit toward C raises C by 0.19 without changing B, while supplying A’s language token after an edit toward B keeps the transcript in A with character error rate rising by 0.02.A same-rank random-subspace edit has almost no effect.Source: F LANGUAGE CONFUSION IN WHISPER · p. 24
  • On Spliced FLEURS, the encoder’s probe still assigns most frames of both halves to their own language (0.88), but the decoder picks one language for the clip.Giving the decoder the correct language token makes the transcript nearly correct even after the edit.Source: F LANGUAGE CONFUSION IN WHISPER · p. 24

G WLE: CONFIDENCE INTERVALS AND RANDOM SUBSPACE CONTROL

WLE’s gains depend on editing the language subspace, and local span windows can outperform fixed windows. Window width affects interference and monolingual cost, while examples show WLE restoring phonemes lost in whole-utterance processing.

G WLE: CONFIDENCE INTERVALS AND RANDOM SUBSPACE CONTROL

  • The random-subspace control does not reduce interference; WLE leaves 0.16 to 0.28 less interference than that control in every model and window.The edits perturb the model by the same amount, but only steering through the language subspace removes interference.Source: G WLE: CONFIDENCE INTERVALS AND RANDOM… · p. 24, Table 12 · p. 25
  • In TTS, the edit removes a further 0.162 [0.131, 0.194] interference beyond chunked decoding without the edit.Chunked decoding without the edit slightly increases interference by 0.059 [0.025, 0.093], and the random-subspace edit does not undo that increase.Source: G WLE: CONFIDENCE INTERVALS AND RANDOM… · p. 25
  • The span window leaves 0.106 [0.074, 0.137] less interference than the fixed window and has smaller shared-recall cost: 0.086 versus 0.122.Its target matches the chunk’s language on 88% of chunks, versus 79% for fixed windows staying within one language and 65% for windows crossing a switch.Source: G WLE: CONFIDENCE INTERVALS AND RANDOM… · p. 25
  • In selected examples, WLE restores phonemes replaced by the other language’s counterparts; it recovers at least half the interference drop in 79% of TTS, 57% of PhoneticXeus, and 43% of POWSM recordings.In TTS, the substitutions appear in three or four sampling seeds without the edit and none with WLE.Source: G WLE: CONFIDENCE INTERVALS AND RANDOM… · p. 25, Table 14 · p. 26

H WINDOW WIDTH

  • At 1 and 3 seconds, WLE’s monolingual PER and PFER changes stay smaller than those of sliding-window decoding; narrow windows are usable only when restricted to the language subspace.At 1 second, sliding windows degrade PER by 4.9 to 13.5 pp and PFER by 1.9 to 7.0 pp; at 3 seconds, the respective PER degradation is 0.4 to 2.7 pp.Source: Table 13 · p. 26, H WINDOW WIDTH · pp. 26–27
  • Across 1-, 2-, 3-, and 4-second windows, WLE reduces interference at every width, but no single width is best everywhere.PhoneticXeus on SwitchLingua does best at 1 second, POWSM on Spliced FLEURS at 3–4 seconds, and the other settings change little between 2 and 3 seconds.Source: H WINDOW WIDTH · p. 26, Table 15 · p. 27

I WLE WITH DIFFERENT LANGUAGE SUBSETS

WLE’s language basis works best when it includes a language from the evaluated pair; bases without either relevant language perform no better than random. Languages outside the evaluation pairs contribute little to the repair.

I WLE WITH DIFFERENT LANGUAGE SUBSETS

  • The full basis and the seven code-switched languages perform similarly, while removing one evaluated language partly degrades repair and removing both degrades it further.On SwitchLingua, a basis missing both languages recovers 0.24–0.34 of the window’s reduction in PhoneticXeus and 0.00–0.05 in POWSM.Source: I WLE WITH DIFFERENT LANGUAGE SUBSETS · pp. 27–28
  • Bases containing none of the evaluation languages perform no better than random, despite comparable representation displacement.The results support needing relevant language directions in the subspace for WLE to be effective.Source: I WLE WITH DIFFERENT LANGUAGE SUBSETS · pp. 27–28
  • Removing both languages of a pair leaves only 4–15% of their pairwise direction in the remaining centroid span, near the 1–3% level of a same-rank random subspace.The 16 language centroids are described as lying in near-general position.Source: I WLE WITH DIFFERENT LANGUAGE SUBSETS · p. 28

J ORACLE REPAIR (KNOWN LANGUAGE)

An oracle using known language labels removes more interference than WLE in recognizers, while the TTS span-window estimate performs similarly to the oracle. The oracle has practical and comparison limits, and model interference is larger and more categorical than the small, gradient shifts described for bilingual speech.

J ORACLE REPAIR (KNOWN LANGUAGE)

  • The oracle removes more interference than WLE in all four recognizer cells, but residual interference remains; in TTS, oracle and span-window WLE leave similar interference, 0.106 versus 0.093.For POWSM on Spliced FLEURS, the 15 edited dimensions do not explain the whole effect.Source: J ORACLE REPAIR (KNOWN LANGUAGE) · p. 28
  • The oracle is a loose upper bound rather than a WLE target: it requires known language boundaries and shifts test frames toward training-corpus centroids.The TTS oracle also uses real-audio centroids, unlike the span window’s targets from generated states.Source: J ORACLE REPAIR (KNOWN LANGUAGE) · pp. 28–29
  • Giving POWSM each span’s own language tag lowers interference by 0.312 [0.263, 0.360] on SwitchLingua and 0.197 [0.160, 0.235] on Spliced FLEURS.The tag-based approach requires knowing each span’s language and the switch point.Source: Table 18 · p. 29, J ORACLE REPAIR (KNOWN LANGUAGE) · p. 29

K INTERFERENCE IN BILINGUAL SPEECH

  • Unlike the small, gradient shifts described in bilingual speech, the studied models replace whole phonemes in recognition and synthesis at inference.The paper distinguishes this inference-time effect from multilingual training interference.Source: K INTERFERENCE IN BILINGUAL SPEECH · p. 29
Loading 2610.11275v1…