PaperlayerView exact revision on arXiv

Source-linked AI summary

Polyglot-Lion: Efficient Multilingual ASR for Singapore via Balanced Fine-Tuning of Qwen3-ASR

Quy-Anh Dang, Chris Ngo

arXiv:2603.16184v1cs.CL

TL;DR

Singapore multilingual ASR must handle four official languages, including low-resource varieties and code-switching, while existing systems can be costly to fine-tune and deploy. Polyglot-Lion fine-tunes compact Qwen3-ASR models on balanced public data without language tags, achieving near-specialist accuracy with substantially lower cost and faster inference. Its evaluation also leaves accuracy gaps on English-NSC and Tamil and excludes code-switched test sets.

  • Problem

    Singapore’s four-language, code-switching environment lacks broadly accessible open-source ASR covering all official languages, while existing systems have accuracy or deployment-cost limitations.

  • Method

    Polyglot-Lion fine-tunes Qwen3-ASR-0.6B and Qwen3-ASR-1.7B on public speech using balanced per-language sampling and no language-tag conditioning.

  • Results

    Polyglot-Lion-1.7B achieves 14.85 average error rate across 12 benchmarks versus 14.32 for MERaLiON-2-10B-ASR, with 20× faster inference and 233× lower training cost.

  • Takeaways & Limitations

    Balanced data and lightweight fine-tuning yield near state-of-the-art multilingual ASR performance at substantially reduced computational expense.

  • Takeaways & Limitations

    The evaluation excludes code-switched test sets, and Polyglot-Lion-1.7B trails MERaLiON-2-10B-ASR on English-NSC and Tamil.

Abstract

from arXiv · show

We present Polyglot-Lion, a family of compact multilingual automatic speech recognition (ASR) models tailored for the linguistic landscape of Singapore, covering English, Mandarin, Tamil, and Malay. Our models are obtained by fine-tuning Qwen3-ASR-0.6B and Qwen3-ASR-1.7B exclusively on publicly available speech corpora, using a balanced sampling strategy that equalizes the number of training utterances per language and deliberately omits language-tag conditioning so that the model learns to identify languages implicitly from audio. On 12 benchmarks spanning the four target languages, Polyglot-Lion-1.7B achieves an average error rate of 14.85, competitive with MERaLiON-2-10B-ASR (14.32) - a model 6x larger - while incurring a training cost of \$81 on a single RTX PRO 6000 GPU compared to \$18,862 for the 128-GPU baseline. Inference throughput is approximately 20x faster than MERaLiON at 0.10 s/sample versus 2.02 s/sample. These results demonstrate that linguistically balanced fine-tuning of moderate-scale pretrained models can yield deployment-ready multilingual ASR at a fraction of the cost of larger specialist systems.

1 Introduction

Polyglot-Lion targets Singapore’s challenging four-language ASR setting with compact models, balanced fine-tuning, and implicit language identification. Its 1.7B model approaches specialist accuracy while reducing inference and training costs.

  • Singapore’s ASR setting combines four official languages with Singlish, speaker variation, accents, and code-switching.
  • High-quality open-source systems covering all four languages remain scarce, while existing general-purpose or specialist models face accuracy or deployment-cost limitations.
  • 14.8 average error rate across 12 benchmarks closely matches MERaLiON-2-10B-ASR’s 14.3 while Polyglot-Lion-1.7B runs nearly 20× faster.
  • Balanced sampling equalises per-language training coverage and improves recognition of low-resource Tamil and Malay without proprietary data.
  • Removing language tags makes the model identify languages implicitly from acoustic features, supporting code-switching robustness.
  • Polyglot-Lion is benchmarked on 12 datasets against eight published baselines spanning general-purpose and specialist systems.
  • $81 on one GPU versus $18,862 on 128 GPUs represents over 233× lower estimated training cost, alongside approximately 20× faster inference.

2 Related Work

Prior multilingual ASR work establishes the benefits and limitations of scale, language-specific resources, balanced sampling, and language conditioning. Polyglot-Lion combines balanced training with implicit language identification for Singapore’s four-language setting.

  • Large-scale multilingual ASR: Whisper demonstrated robust multilingual recognition from 680,000 hours of weakly supervised audio spanning 99 languages, while wav2vec 2.0 and HuBERT advanced self-supervised speech learning.
  • Audio-language models: Audio-language models combine speech encoders with LLM decoders, using linguistic priors to support ASR alongside broader audio understanding.
  • Southeast Asian and Singapore ASR: SEA-LION and Singapore-focused systems highlight region-specific data and benchmarks, with MERaLiON providing unified coverage of English, Mandarin, Tamil, and Malay.
  • Multilingual training balance: Language-balanced batching reduces low-resource WER without degrading high-resource languages, while explicit upsampling guarantees exact per-language epoch parity.
  • Language identification in ASR: Language-token conditioning can improve recognition when the input language is known but creates dependence on language identification and can fail in code-switched settings.
  • Language identification in ASR: Polyglot-Lion revisits language-agnostic decoding and reports reliable implicit identification across four typologically diverse languages using balanced data.

3 Datasets

Polyglot-Lion trains and evaluates on public speech corpora covering Singapore’s four official languages. The resulting 607,839 utterances and 968.83 hours remain substantially imbalanced before balanced sampling, especially for Malay.

  • The study uses only publicly available speech corpora covering English, Mandarin Chinese, Tamil, and Malay.
  • Table 1 reports dataset statistics by language, split, and duration, with S denoting samples and H denoting hours.
  • English: English includes 100.59 hours from Librispeech and 147.97 hours from Singapore’s National Speech Corpus, covering varied speaking styles and demographics.
  • Mandarin: Mandarin combines AISHELL-1, AISHELL-3, Common Voice 23, and Fleurs, spanning standard, crowdsourced, and read speech.
  • Tamil: Tamil combines SLR127, Common Voice 23, SLR65, and Fleurs, totaling 215.58 training hours across read and crowdsourced speech.
  • Malay: Malay uses Mesolitica4 and Fleurs, totaling 58.98 training hours across multiple domains and clean read speech.
  • Data statistics and imbalance: The combined corpus contains 607,839 utterances and 968.83 hours, but English and Mandarin provide about 65% of training hours while Malay contributes under 8%.
  • Preprocessing: Audio longer than 30 seconds is discarded, and transcripts are lowercased with punctuation removed.

4 Method

Polyglot-Lion fine-tunes compact Qwen3-ASR checkpoints using balanced multilingual sampling and language-agnostic decoding. The method equalizes language coverage and omits explicit language tags to support implicit identification in Singapore’s multilingual speech.

  • Model Configuration: Polyglot-Lion fine-tunes Qwen3-ASR-0.6B and Qwen3-ASR-1.7B with identical procedures, varying only model capacity.Both variants use publicly available checkpoints and share the same architecture design and training procedures.
  • Balanced Multilingual Sampling: Two-stage upsampling first balances datasets within each language, then equalizes the resulting language corpora.The procedure replicates and randomly subsamples datasets before applying the same process across languages.
  • Balanced Multilingual Sampling: The final training corpus contains four languages with precisely 25% of utterances contributed by each language.This guarantees exact per-language parity regardless of the original corpus distribution.
  • Language-Agnostic Transcription: Language-agnostic decoding omits language tags during training and inference, so the model identifies the spoken language implicitly from audio.The design targets conversational and code-switched settings where the language may be unknown in advance.
  • Training Configuration: Both variants are fine-tuned for 48 hours on a single NVIDIA RTX PRO 6000 GPU using AdamW and cosine annealing.The peak learning rate is 2 × 10^-5, with an effective batch size of 32 utterances.

5 Experimental Setup

The experiments evaluate multilingual ASR with language-appropriate error metrics and compare Polyglot-Lion against eight diverse systems. All models use released checkpoints and run on the same hardware for latency comparisons.

  • Evaluation Metrics: WER evaluates English, Tamil, and Malay using normalized word-level edit distance.WER combines substitutions, deletions, and insertions divided by the number of reference words.
  • Evaluation Metrics: CER evaluates Mandarin Chinese because character-level scoring is appropriate when explicit word boundaries are absent.Character-level evaluation is described as widely adopted for Mandarin Chinese.
  • Evaluation Metrics: Lower values indicate better performance for both WER and CER after lowercasing and punctuation removal.Scoring uses the asr-evalkit library.

6 Results

Polyglot-Lion-1.7B ranks second across 12 benchmarks with an average error rate close to the 6× larger MERaLiON-2-10B-ASR, while delivering strong language-specific results and substantially lower inference latency and training cost.

  • Overall results: 14.85 average error rate places Polyglot-Lion-1.7B second across 12 benchmarks, close to MERaLiON-2-10B-ASR at 14.32 despite being 6× larger.Polyglot-Lion-0.6B averages 16.52, leading models at or below 1B parameters; the next best is Whisper-large-v3-turbo at 33.04.
  • English: 2.10 WER on Librispeech is the best evaluated result, while 5.28 WER on Singapore English NSC substantially improves over Whisper-large-v3-turbo at 32.02.MERaLiON remains best on NSC with 4.62 WER.
  • Mandarin: Polyglot-Lion-1.7B achieves the lowest CER on all four Mandarin benchmarks, including 1.45 on AISHELL-1 and 8.00 on Fleurs.It outperforms MERaLiON-2-10B-ASR on each reported Mandarin benchmark.
  • Tamil: 139.96 to 39.19 Tamil CV WER is a 72% relative reduction after fine-tuning, although MERaLiON remains best on all four Tamil benchmarks.Polyglot-Lion-1.7B also reports 19.75, 26.83, and 37.28 on SLR65, SLR127, and Fleurs respectively.
  • Malay: 21.51 WER on Mesolitica is the best evaluated Malay result, while 9.98 WER on Malay Fleurs is competitive with MERaLiON at 8.55.Mesolitica covers conversational and domain-diverse Malay speech, making it more representative of deployment conditions than read-speech Fleurs.
  • Effect of fine-tuning: 65% and 72% Tamil CV WER reductions for the 0.6B and 1.7B models, respectively, show the largest fine-tuning benefits on under-represented languages.Malay Mesolitica WER falls by 49% and 45%; English and Mandarin are preserved or improved.
  • Inference speed: 0.10 s/sample inference latency for both Polyglot-Lion variants is approximately 20× faster than MERaLiON at 2.02 s/sample.The measurements were collected on a single NVIDIA RTX PRO 4500 GPU; Whisper-large-v3-turbo runs at 0.28 s/sample.
  • Training cost: $81 estimated training cost is a 233× reduction from MERaLiON-2-10B-ASR’s $18,862, with comparable average error rates of 14.85 and 14.32.Polyglot-Lion uses one RTX PRO 6000 GPU, whereas MERaLiON used 128 GPUs.

7 Analysis

Balanced upsampling substantially improves Tamil recognition while preserving English and Mandarin performance, and language-agnostic decoding supports competitive recognition across all four target languages. Results also show that balanced fine-tuning matters more than parameter count for accuracy and enables a smaller deployment option.

  • Effect of Language Balancing: 39.19 Tamil CV WER follows balanced upsampling, a 72% relative reduction from the unmodified base models’ error exceeding 120%.English and Mandarin performance did not degrade, and no additional hyper-parameter tuning was required beyond monolingual fine-tuning.
  • Language-Agnostic Decoding: Language-agnostic decoding omits language tags during training and inference, requiring implicit identification from acoustic and linguistic patterns.Competitive results across Mandarin, Tamil, and the other target languages provide empirical support for this design.
  • Parameter Efficiency: 14.85 average error rate makes Polyglot-Lion-1.7B comparable to MERaLiON-2-10B-ASR at 14.32 despite using six times fewer parameters.Qwen3-ASR-1.7B without fine-tuning scores 53.76 average WER, while the 0.6B variant reaches 16.52 average error rate with 63% fewer parameters.
  • Parameter Efficiency: 16.52 average error rate for Polyglot-Lion-0.6B incurs a 1.67-point penalty relative to the 1.7B model, supporting edge deployment when memory is constrained.The smaller model uses 63% fewer parameters.

8 Conclusion

Polyglot-Lion is a compact multilingual ASR family for Singapore’s four target languages, trained through balanced fine-tuning and implicit language identification. The 1.7B model closely matches a much larger specialist system while using fewer parameters, faster inference, and lower training cost.

  • Conclusion: Polyglot-Lion fine-tunes Qwen3-ASR base models on publicly available speech corpora for Singapore English, Mandarin, Tamil, and Malay.The models remove language-tag conditioning to enable implicit language identification.
  • Conclusion: 14.85 average error rate across 12 benchmarks closely matches MERaLiON-2-10B-ASR at 14.32.Polyglot-Lion-1.7B uses six times fewer parameters, runs 20× faster at inference, and has 233× lower training cost.

Limitations

The remaining limitations concern accuracy gaps on Singapore English and Tamil, and the absence of code-switched evaluation despite widespread multilingual mixing in Singapore speech.

  • Remaining accuracy gaps: Polyglot-Lion-1.7B trails MERaLiON-2-10B-ASR on English-NSC, with 5.28 versus 4.62 WER.The authors associate this gap with Singapore-specific pronunciation, prosody, and code-mixed Singlish constructions at the 1.7B scale.
  • Remaining accuracy gaps: 39.19 versus 31.78 WER on Common Voice marks a larger Tamil gap against MERaLiON-2-10B-ASR.The authors identify Tamil morphology, phonology, dialectal variation, and limited Tamil pretraining exposure as contributing difficulties.
  • Code-switching and intra-sentential mixing: The current evaluation excludes code-switched test sets, although code-switching is described as common in everyday Singapore speech.Future work proposes code-switch-aware training objectives and evaluation on SEAME or CS-Singlish.

A Dataset Details

The work uses publicly available datasets under their respective licences, including a limited 100,000-sample draw from the National Speech Corpus.

  • Dataset Details: All datasets used in the work are publicly available and used according to their respective licences.
  • Dataset Details: 100,000 National Speech Corpus samples from Part 1 were drawn for training.
Loading 2603.16184v1…