PaperlayerView exact revision on arXiv

Source-linked AI summary

Itgan at NADI 2026 shared task: Parameter-Efficient Whisper Adaptation for Robust, Mixed-Dialect and Code-Switched Arabic ASR

Ibrahim Almajai

arXiv:2610.09934v1cs.CLcs.AIcs.SD

TL;DR

NADI 2026 evaluates Arabic ASR under noisy national-dialect, unlabeled mixed-dialect, and Tunisian code-switching conditions. Itgan adapts Whisper with LoRA and adds subtask-specific interventions; the submitted systems used different interventions across tasks, with 57.1% country-average WER on Subtask 1.1.

  • Problem

    The NADI 2026 ASR subtasks span noisy national dialects, unlabeled mixed varieties, and Tunisian speech with frequent French and English switching.

    Source: §1 · p. 1

  • Method

    The systems share a Whisper-LoRA recipe, with pooled-adapter specialists, base-model selection, or training-free combination added for different subtasks.

    Source: §1 · p. 1, §8 · p. 5

  • Results

    Different interventions carried the three subtasks, and the submitted Subtask 1.1 system reached 57.1% country-average WER.

    Source: §1 · p. 1, §8 · p. 5

  • Takeaways & Limitations

    The authors’ practical lesson is to diagnose the binding constraint before spending GPU-hours, using checks such as a loss gap, zero-shot comparison, or disagreement rate.

    Source: §7 · p. 5

    Run-to-run variance is not separated from the effects of design changes, and the submitted Subtask 1.2 model has only a blind-test measurement because it used the full development split for training.

    Source: Limitations · p. 5

Abstract

from arXiv · show

We describe the Itgan systems for the three ASR subtasks of NADI 2026, namely robust country-level ASR (1.1), mixed-dialect ASR (1.2), and Tunisian code-switched ASR (1.3). All three share one recipe, Whisper adapted with LoRA on consumer GPUs, and each was carried by a different addition to it. On 1.1, where the dialect label is given at test time, per-dialect specialists continued from a pooled adapter gave the largest gain, and the submitted system reached 57.1% country-average WER. A post-evaluation linear probe on frozen encoder features routes utterances without the label and recovers 44% of what oracle routing gives. On 1.2 the choice of base model mattered more than adapter capacity, and system combination helped only once we added a decorrelated member, reaching 46.7% WER. On 1.3 our system placed second at 14.49% WER with the lowest CER among the leading submissions, 5.38%. Its last 0.60 WER points came without further training, mostly from an exact weight-space average of independently trained runs, with ROVER voting adding the remainder. Every comparison carries a paired-bootstrap test, and we report eight directions that did not work.

1 Introduction

The paper presents three NADI 2026 ASR systems built from a shared parameter-efficient Whisper recipe, with a different intervention carrying each subtask. Its contribution is in the measurements and system design rather than new components.

1 Introduction

  • Per-dialect specialists, base-model choice, and training-free combination carried Subtasks 1.1, 1.2, and 1.3, respectively.The authors report that the subtasks reward different interventions more than additional training.Source: §1 · p. 1
  • On Subtask 1.2, base-model choice was worth 3.76 points while widening the adapter was a non-significant 1.02; training-free combination helped Subtask 1.3 by 0.60 points but not Subtask 1.2 until ensemble decorrelation.These are the reported contrasts motivating the paper’s distinct interventions.Source: §1 · p. 1
  • The contribution is a pooled-adapter continuation recipe, a frozen-feature probe for routing without gold dialect labels, base-model comparisons, and cross-run weight averaging.The paper also reports paired-bootstrap tests and eight rejected directions.Source: §1 · p. 1

2 Data

The three ASR subtasks use different speech and supervision conditions: noisy country-level dialect data, unlabeled mixed varieties, and Tunisian speech with frequent French and English switching. The section describes their available training, development, and test data.

2 Data

  • Subtask 1.1 covers eight national dialects, each with 1,600 training utterances, a 10,808-utterance development set, and a 4,000-utterance blind test set.Audio is sharply band-limited to about 2.1 kHz effective bandwidth, with under 0.01% of energy above 4 kHz.Source: §2 · pp. 1–2
  • Subtask 1.2 provides 3,152 development utterances from more than fifteen unlabeled sub-regional varieties and a blind test set, with no other in-domain data.Source: §2 · p. 2
  • Subtask 1.3 provides 38 hours of Tunisian training speech, 731 validation utterances, 842 practice-test utterances, and a 2,123-utterance blind test set.After dropping six unusable training examples, 24,887 utterances remain; targets and scoring use the organizers’ clean_transcription normalization.Source: §2 · p. 2
  • Code-switching is pervasive in the Subtask 1.3 practice test, where 33.3% of reference words are in Latin script.Source: §2 · p. 2

3 Common Setup

All systems fine-tune Whisper with LoRA adapters while keeping the base model frozen. The setup also specifies hardware, evaluation metrics, uncertainty testing, and differences in decoding across subtasks.

3 Common Setup

  • The shared setup applies rank-32, α-64 LoRA adapters to attention and feed-forward projections in every encoder and decoder block, keeping the base frozen in bfloat16 and adapters in float32.The authors report that this supports training a 1.54B-parameter checkpoint within 12 GB without quantization.Source: §3 · p. 2
  • Training and decoding used two consumer GPUs, and models were publicly released Whisper checkpoints trained only on their subtask data, except one pooled Subtask 1.2 ensemble member.The exception pooled Subtask 1.2 data with Subtask 1.1 audio.Source: §3 · p. 2
  • WER and CER are percentages, system differences are absolute WER points unless labeled as relative reductions, and paired-bootstrap comparisons use utterance resampling.P is the fraction of 4,000 resamples in which the second system has lower WER, not a p-value.Source: §3 · p. 2
  • Decoding is not matched across subtasks: Subtasks 1.1 and 1.3 use different beam settings and toolchains, while beam search on 1.1 was tested only after evaluation.Source: §3 · p. 2

4 Subtask 1.1: Robust Country-Level ASR

The submitted Subtask 1.1 system combines pooled training, per-dialect specialists, backbone choices, and a length guard. It reached 57.1% country-average WER on the blind test set, seventh of nine teams.

4 Subtask 1.1: Robust Country-Level ASR

  • Four dialect specialists continued the pooled turbo adapter for three epochs, and a step-matched pooled control attributes their gain to specialization rather than extra epochs.The pooled adapter reached 57.6% country-average development WER, compared with the organizers’ 80% zero-shot baseline.Source: §4 · p. 2
  • The six-words-per-second length cap affected about 12 of 10,808 development utterances but improved country-average WER by 1.0 point.Runaway outputs concentrated in the weakest country; the guard also made beam search safe.Source: §4 · p. 3
  • 57.1% country-average WER and 28% CER placed the submitted system seventh of nine teams on the blind test set, versus 43.2% WER for the leader.The paper says it cannot identify what separated the leader’s system from theirs.Source: §4 · p. 3
  • Specialists removed 4–8% of each dialect’s remaining error without the guard and 3.6–4.5% under it; every specialist gain was significant except Jordan’s.The table caption specifies that its reported deltas are unguarded and that the guard overlaps with specialist gains for Mauritania and Yemen.Source: §4 · p. 3, Table 1 · p. 3
  • After evaluation, beam search with the guard added 1.1 points over all eight specialists, reaching 53.55% development WER.Source: §4 · p. 3

5 Subtask 1.2: Mixed-Dialect ASR

Subtask 1.2 rewarded choosing the base model over expanding adapter capacity, while ensemble gains appeared only after adding a decorrelated member. The final five-system submission scored 46.7% WER and 19% CER, placing third of four teams.

5 Subtask 1.2: Mixed-Dialect ASR

  • Design used a 2,537/615 split of the 3,152 development utterances; retraining on all data changed 86% of test transcripts and prevented rescoring the held-out IDs.The final model had seen every held-out ID.Source: §5 · p. 3
  • 3.76 WER points separated turbo from full large-v3; at matched adapter size, large-v3 still led by 2.65 points, while extra rank added 1.11.The 2.65-point comparison had P = 100% and 95% interval [−3.85, −1.39]; the extra-rank gain had P = 99%.Source: Table 2 · p. 3, §5 · p. 3
  • The fourth member, trained with pooled Subtask 1.1 audio, disagreed with scorable large-v3 on 91% of held-out lines while scoring within 0.33 points; a fifth zero-shot member added nothing.The four-system ensemble scored 46.7%.Source: §5 · p. 4
  • 46.7% WER and 19% CER placed the submitted ensemble third of four teams.The submitted archive combined five systems by word-level ROVER voting.Source: §5 · p. 4
  • Roughly 5 WER points of reported error were unwinnable orthographic variation.Source: §5 · p. 4

6 Subtask 1.3: Code-Switched ASR

For Tunisian code-switched ASR, regularization improved held-out loss, while training-free adapter averaging and ROVER voting supplied the final system’s last 0.60 WER points. The system placed second of six, with the lowest CER among leading submissions.

6 Subtask 1.3: Code-Switched ASR

  • At matched step 3,000, dropout plus SpecAugment raised training loss from 0.229 to 0.293 while lowering held-out loss from 0.340 to 0.322, shrinking the generalization gap 3.9-fold.The initial wide run had training loss of 0.04–0.05 while held-out loss stagnated near 0.31.Source: §6 · p. 4

6.1 Cross-run adapter souping

  • Concatenating LoRA factors, rather than averaging them elementwise, gives the exact mean update as a rank-Nr adapter that can be merged before decoding.The construction was verified on the submitted soup with a maximum float32 deviation of 9 × 10^-8.Source: §6.1 · p. 4
  • 16.91% WER for averaging two seeds was 2.38 points below the same-run pair and 1.16 below the cross-recipe pair; it was indistinguishable from the submitted soup at 17.19%.The authors attribute the cross-run gain to seed decorrelation alone.Source: §6.1 · p. 4

6.2 ROVER and results

  • ROVER used word-level majority voting across the soup and two trained models, modifying 63 utterances; its −0.06 blind-set WER effect was described as noise-level.The effect was not uniformly positive: it was +0.01 on the practice test split.Source: §6.2 · pp. 4–5
  • The final system placed second of six, 0.08 WER points behind first, and had the lowest CER among leading submissions at 5.38%; training-free steps added 0.60 WER points over the best single trained model.Table 3 reports the Subtask 1.3 results and the 0.60-point blind-test WER reduction.Source: Table 3 · p. 5, §6.2 · p. 5
  • Embedded French and English words were recognized better than surrounding Tunisian Arabic, identifying the dialect—not code-switching—as the bottleneck.Source: §6.2 · p. 5

7 Discussion and Future Work

The discussion finds that routing quality, not specialist quality, limits the benefit of dialect specialization. It argues that spoken dialect identification and ASR are complementary tasks.

7 Discussion and Future Work

  • A linear probe recovered 0.8 of the 1.9 country-average WER points gained by oracle routing, and misrouting costs about seven tenths of correct routing’s gain.The probe achieved 65% identification accuracy; the authors identify the identifier, rather than the specialists, as the limit.Source: §7 · p. 5
  • The recoverable share is likely lower for 1.2, which has more than fifteen unlabeled varieties rather than 1.1’s eight known classes; the authors call dialect identification and ASR complementary.They say an identifier built for NADI’s separate spoken dialect identification track could serve as the ASR front-end.Source: §7 · p. 5

8 Conclusion

The three NADI 2026 ASR systems shared one consumer-hardware recipe, with a different intervention carrying each subtask. The systems ranked seventh of nine, third of four, and second of six, respectively.

8 Conclusion

  • One recipe on consumer hardware reached 57.1% country-average WER on 1.1 (seventh of nine), 46.7% on 1.2 (third of four), and 14.49% on 1.3 (second of six).Per-dialect specialists, base-model choice, and cross-run averaging carried the respective systems.Source: §8 · p. 5

Limitations

The authors qualify their Subtask 1.3 design decisions with the limits of a single development split and one seed per configuration. For Subtask 1.1, within-country noise and limited backbone-selection data constrain interpretation of acoustic and model comparisons.

Limitations

  • 731 development utterances cannot resolve differences below roughly 1.5 WER points, and one seed per configuration leaves run-to-run variance confounded with the effects of changes.Some development gains may therefore reflect selection noise; second-seed checks bound but do not remove this concern.Source: Limitations · p. 5
  • Backbone selection used only 200 utterances per country, with margins under 3 WER points for three of four countries; greedy decoding also left 1.1–1.3 country-average WER points on the table.The Subtask 1.3 script-based language proxy counts French and English words written in Arabic script as Arabic.Source: Limitations · p. 6

A Acoustic Character of Subtask 1.1

  • The noisier half of each country’s Subtask 1.1 data decodes 8.6 WER points worse, while eight-country correlations cannot establish bandwidth or mean SNR as explanations for cross-country spread.Per-country WER spans 39 points, and the reported correlation intervals include zero.Source: A Acoustic Character of Subtask 1.1 · p. 7
  • Training-time subset scoring understated WER by about 2.4 points and reversed the ranking of two configurations that full-development scoring found equivalent.This makes the subset an unsafe basis for model selection in this comparison.Source: A Acoustic Character of Subtask 1.1 · p. 7

B Post-Evaluation

Post-evaluation analyses examine additional Subtask 1.1 specialists and guarded beam decoding, then assess statistical comparisons and transfer to the Subtask 1.3 blind test. The analyses show that some development gains persist, but their size can shrink on the blind test.

B Post-Evaluation

  • The completed eight-specialist set was reachable in under 3 GPU-hours with the one-epoch recipe, but its four post-evaluation gains were measured against the pooled adapter and do not establish submission-time improvements.The gains were Yemen 3.9, UAE 1.6, Palestine 1.1, and Jordan 0.4 WER points; the test references were not released.Source: B Post-Evaluation · pp. 7–8
  • A length guard makes beam search improve all eight Subtask 1.1 countries; guarded beam adds 0.3 points on Mauritania to 1.7 on Morocco, while guard and decoder default together yield 2.3 points on the pooled adapter.Table 4’s guard applies from its second row onward, unlike the unguarded per-country deltas in Table 1; the guard and beam require no additional training.Source: B Post-Evaluation · p. 8, Table 4 · p. 8

C Statistical Significance

  • With 731 development utterances, differences below roughly 1.5 WER points cannot be resolved, and the regularization step is not significant on its own whereas cross-run soup is unambiguous.Thirty-six bootstrap comparisons were reported without multiplicity correction, so the authors caution that marginal comparisons are weaker than their numbers suggest.Source: C Statistical Significance · p. 8
  • Seven of eight Subtask 1.1 specialists show unambiguous gains against the pooled adapter on their own countries’ development splits; Jordan’s comparison is not unambiguous.Jordan is identified as the country with the least headroom.Source: C Statistical Significance · p. 8
  • On 615 held-out Subtask 1.2 utterances, switching the base model is decisive (−3.76 WER points), while widening turbo’s target set is not (−1.02 WER points, P = 88%).The matched-capacity control retains 2.65 points of the base-model gain; extra rank on large-v3 is worth 1.11 points.Source: C Statistical Significance · pp. 8–9

D How Much Transfers (Subtask 1.3)

  • The Subtask 1.3 soup’s 2.26-point development gain transferred as 0.54 points (24%) on the blind test, while ROVER’s 0.10-point gain transferred as 0.06 points (60%).The authors caution that the ROVER transfer ratio is unstable because the same step is neutral on the practice test split; the blind-test set was easier than development for the same model.Source: D How Much Transfers (Subtask 1.3) · p. 9

E Error Analysis by Script (Subtask 1.3)

The analyses show script-dependent errors and a substantial orthographic noise floor, while failed interventions contrast with specialization and routing gains. Specialist routing remains identifier-limited, and some apparent gains shrink under controls or decoding safeguards.

E Error Analysis by Script (Subtask 1.3)

F The Metric Noise Floor (Subtask 1.2)

  • After folding common variants, 844 word types remain inconsistent across 21.9% of tokens; folding both sides lowers zero-shot turbo WER from 62.2% to 57.2%.The authors recommend reporting a folded score alongside the raw score when judging changes on this data.Source: F The Metric Noise Floor (Subtask 1.2) · p. 9

G What Did Not Work

  • Enhancement and noise augmentation failed on 1.1, while 1.1-to-1.2 transfer, corpus-frequency spelling remapping, and rank-64 adaptation also underperformed or stalled.Source: G What Did Not Work · pp. 9–10
  • A 4-gram constraint gained 0.5 versus the guard’s 1.0 country-average WER points; central-hypothesis selection scored 18.22%, above the soup’s 17.19%, and matched-step schedule comparisons misled.Source: G What Did Not Work · p. 10

H What the Specialists Learn

I Routing Without the Dialect Label

Related papers

Loading 2610.09934v1…