Source-linked AI summary
Typhoon ASR Real-time: FastConformer-Transducer for Thai Automatic Speech Recognition
Warit Sirichotedumrong, Adisai Na-Thalang, Potsawee Manakul, Pittawat Taveekitworachai, Sittipong Sripaisarnmongkol, Kunat Pipatanakul
TL;DR
Thai ASR lacks efficient streaming solutions because dominant Whisper-like systems are optimized for offline transcription and Thai text contains substantial orthographic ambiguity. The paper combines a 115M-parameter FastConformer-Transducer with canonical normalization and two-stage Isan adaptation, achieving comparable accuracy with substantially lower computation while releasing standardized evaluation resources.
Problem
Thai ASR is dominated by offline architectures that are poorly suited to low-latency streaming, while ambiguous transcription conventions create unreliable training and evaluation targets.
Method
The paper uses a FastConformer-Transducer, canonical text normalization, a 303-hour gold-standard Isan adaptation set, and standardized Thai ASR evaluation resources.
Results
115M parameters deliver performance that outperforms offline baselines 13× its size on standard benchmarks, while normalization yields 5.69% CER on FLEURS against normalized references.
Takeaways & Limitations
Rigorous normalization and architectural efficiency can provide Thai ASR with real-time capability and strong benchmark performance without relying on large offline models.
Takeaways & Limitations
Strict phonetic normalization can produce outputs unsuitable for direct display, and heavy Thai-English code-switching is limited because speech is mapped to Thai characters.
Abstract
from arXiv · showhide
Large encoder-decoder models like Whisper achieve strong offline transcription but remain impractical for streaming applications due to high latency. However, due to the accessibility of pre-trained checkpoints, the open Thai ASR landscape remains dominated by these offline architectures, leaving a critical gap in efficient streaming solutions. We present Typhoon ASR Real-time, a 115M-parameter FastConformer-Transducer model for low-latency Thai speech recognition. We demonstrate that rigorous text normalization can match the impact of model scaling: our compact model achieves a 45x reduction in computational cost compared to Whisper Large-v3 while delivering comparable accuracy. Our normalization pipeline resolves systemic ambiguities in Thai transcription --including context-dependent number verbalization and repetition markers (mai yamok) --creating consistent training targets. We further introduce a two-stage curriculum learning approach for Isan (north-eastern) dialect adaptation that preserves Central Thai performance. To address reproducibility challenges in Thai ASR, we release the Typhoon ASR Benchmark, a gold-standard human-labeled datasets with transcriptions following established Thai linguistic conventions, providing standardized evaluation protocols for the research community.
1 Introduction
Thai ASR faces streaming, data, and evaluation challenges that architecture alone does not resolve. The paper introduces a streaming FastConformer-Transducer and data-centered solutions spanning normalization, dialect adaptation, scalable curation, and standardized benchmarks.
- Motivation: Whisper-like encoder-decoder systems deliver strong offline accuracy but suffer unpredictable latency and hallucinations in low-latency streaming applications.Thai-specific difficulties include ambiguous numbers, repetition markers, loanwords, and the absence of explicit word boundaries.
- Motivation: Systematic text normalization creates canonical training targets that resolve Thai transcription ambiguities and support robust, reliable ASR.The paper frames data quality and consistency as essential alongside model architecture, especially in low-resource settings.
- Approach: FastConformer-Transducer is selected for low-latency streaming inference instead of a Whisper-based architecture.The contribution emphasizes real-time inference and consistent, hallucination-free outputs.
- Approach: 11,000 hours of training data are curated through semi-supervised consensus from multiple teacher models with minimal human intervention.This pipeline addresses the scarcity of large-scale labeled Thai corpora.
- Approach: A multi-stage adaptation strategy extends the model to Isan while mitigating catastrophic forgetting of Central Thai.The paper combines dialect adaptation with continued robustness on Central Thai data.
- Approach: Typhoon ASR Benchmark and TVSpeech introduce standardized evaluation resources for Thai ASR reproducibility and real-world acoustic conditions.Gigaspeech2-Typhoon provides canonically normalized targets, while TVSpeech is manually curated for production-like environments.
2 Data Curation and Normalization
The paper builds Thai ASR training data around consensus transcription, targeted human verification, canonical normalization, and mixtures combining broad public coverage with curated datasets. Separate Isan and General Thai sources support dialect learning while preserving formatting consistency.
- Consensus-Based Transcription Pipeline: Three Thai Whisper-Large models process audio in parallel, with majority voting selecting consensus transcriptions and Pathumma-Whisper-Large as the fallback.Selected outputs then undergo automated complexity verification.
- Consensus-Based Transcription Pipeline: Complex transcriptions containing Arabic numerals or special punctuation receive human review, while clean outputs proceed directly to storage.This hybrid workflow maintains normalization consistency while minimizing manual annotation overhead.
- Text Normalization Pipeline: The normalization pipeline transforms transcripts into canonical representations matching spoken forms and follows established Thai transcription guidelines.It targets orthographic ambiguities that otherwise introduce training noise.
- General Training Data: 11,000 hours of Thai audio are curated through a composite dataset balancing large-scale public corpora with targeted internal datasets.The mixture is designed to provide broad acoustic coverage and precise output formatting.
- Isan Dialect Adaptation Data: 303 hours of gold-standard adaptation data combine Isan sources for dialect signals with General Thai sources for regularization and formatting consistency.The mixture includes numeric-sequence and repetition-marker data alongside Isan acoustic and domain coverage.
3 Model Architecture and Training Strategy
The model uses a streaming-optimized FastConformer-Transducer architecture and trains on large-scale Thai data, then adapts to Isan through a two-stage curriculum. The curriculum first adjusts the full model conservatively and then specializes linguistic components while freezing the encoder.
- Model Architecture: FastConformer-Transducer addresses Whisper’s latency and computational bottlenecks with streaming-optimized encoding and aggressive downsampling.The architecture uses an 8× depthwise convolutional subsampling layer with 256 channels and reduced kernel sizes.
- General Thai Model Training: The Thai model is initialized from a pre-trained English FastConformer-Transducer Large model and fine-tuned on 11,000-hour General Training Data.Training uses all model parameters and completes in approximately 17 hours on 2× NVIDIA H100 GPUs.
- Curriculum Learning for Dialect Adaptation: Stage 1 fine-tunes the full model for 10 epochs at η = 10^-5 to gently adapt acoustic representations to Isan.The conservative rate is intended to avoid overwriting robust Central Thai features.
- Curriculum Learning for Dialect Adaptation: Stage 2 freezes the encoder and trains the decoder and joint network for 15 epochs at η = 10^-3 for linguistic specialization.This stage targets Isan lexical structures and dialect-specific particles while relying on stable acoustic representations.
4 General Thai Evaluation
The Typhoon ASR Benchmark standardizes Thai evaluation across clean academic speech and real-world robustness, using canonical normalization to focus metrics on phonetic accuracy. Typhoon ASR Realtime combines streaming efficiency with competitive accuracy, while data quality substantially improves robustness and exposes orthographic artifacts in comparisons.
- 4.1 Typhoon ASR Benchmark: The Typhoon ASR Benchmark evaluates clean-read accuracy and real-world robustness through standardized, publicly released tracks.Ground-truth transcripts are renormalized to established Thai transcription guidelines for deterministic evaluation.
- 4.1 Typhoon ASR Benchmark: Canonical normalization shifts CER evaluation from formatting inconsistencies toward phonetic accuracy, addressing variants such as digits versus spoken number forms.Without canonical targets, valid orthographic choices can be penalized as errors.
- 4.2 Main Results: Typhoon Isan2 achieves the best performance across both general Thai datasets, supporting general robustness from the curriculum learning strategy.The evaluation isolates data-pipeline effects by comparing offline baselines trained on the paper’s dataset with an architecture-matched external baseline.
- 4.2 Main Results: 5.84% to 4.69% CER on the Standard Track represents an approximately 20% relative error reduction on the general academic baseline.This result is reported for Gigaspeech2 evaluation.
- 4.2 Main Results: 4.04% absolute CER reduction on TVSpeech lowers error from 10.36% to 6.32% in challenging in-the-wild conditions.The reported improvement is attributed to consensus-based labeling and strict normalization, independent of the underlying neural architecture.
- 4.2 Main Results: On FLEURS, Typhoon Whisper Large-v3 improves from an apparent 9.98% CER to 5.69% after references are normalized canonically, indicating an orthographic rather than phonetic mismatch.The normalized result is reported as outperforming the listed open-source baselines and Gemini 3 Pro at 6.91%.
5 Isan Dialect Evaluation
The Isan evaluation combines CER benchmarking with human A/B testing to assess dialect adaptation in both automated and perceptual terms. Two-stage curriculum learning substantially improves streaming performance, although human preference and CER diverge between foundation and dedicated ASR systems.
- 5.1 Quantitative Evaluation: The Isan evaluation uses quantitative CER benchmarking and qualitative human A/B testing on a held-out dialect test set.The comparison includes external baselines and Gemini 2.5 Pro, with two native Isan speakers providing 2,000 blind pairwise judgments over 500 samples.
- 5.1 Quantitative Evaluation: 10.65% CER for Typhoon Isan ASR Realtime substantially outperforms Whisper-Medium-Dialect at 17.72%, despite the offline Typhoon model achieving 8.85%.The result demonstrates stronger streaming performance than the public-dataset baseline while retaining an offline-versus-streaming gap.
- 5.2 Curriculum Learning Ablation: 5.57% absolute CER reduction, from 16.22% to 10.65%, occurs when Stage 2 freezes the encoder and fine-tunes the decoder and joint network.Stage 1 acoustic adaptation captures Isan tonal distinctiveness but lacks dialect-specific vocabulary.
- 5.2 Qualitative Human Evaluation: CER does not always correlate perfectly with usability in Isan because of high morphological variance.The paper therefore supplements automated evaluation with human A/B testing.
- 5.2 Qualitative Human Evaluation: Gemini 2.5 Pro exceeds the 50% win threshold against every competitor in the human comparisons, while Typhoon Isan models lead among dedicated ASR systems in combined wins and ties.The external SLSCU baseline records negligible wins, consistent with its 70.08% CER in Table 7.
- 5.2 Qualitative Human Evaluation: Gemini 2.5 Pro and Typhoon Isan Realtime have nearly identical CER, 10.20% versus 10.65%, yet Gemini wins more often in human preference.The paper associates this divergence with semantic coherence versus phonetic fidelity objectives.
6 Limitations and Future Work
The paper identifies scope boundaries involving readability, bilingual code-switching, semantic ambiguity, and production scenarios. It proposes future work spanning inverse text normalization, domain adaptation, multi-speaker handling, dialect coverage, and on-device deployment.
- 6.1 Limitations: Strict normalization prioritizes phonetic fidelity over stylistic readability, producing outputs that may require post-processing for end-user display.Examples include Thai transliterations for English terms and spoken words instead of written numbers.
- 6.1 Limitations: Heavy Thai-English code-switching remains constrained because the model maps English content to Thai characters instead of preserving Latin-script orthography.This limits immediate utility in bilingual technical and corporate settings.
- 6.1 Limitations: The 115M-parameter model lacks the world knowledge needed to resolve complex semantic ambiguities or context-dependent homophones when acoustics are unclear.The limitation is identified in the Isan evaluation relative to massive foundation models.
- 6.2 Future Work: Future work should develop contextual inverse text normalization, runtime domain biasing, speaker diarization, and overlapping-speech handling for production applications.These directions target conversion to user-facing formats, underrepresented vocabulary, multi-party attribution, and overlapping speech.
- 6.2 Future Work: The Isan curriculum suggests extending dialect coverage to Northern and Southern Thai and investigating unified models for zero-shot dialect identification and adaptation.The proposed expansion is framed as a way to broaden regional access to voice technology.
- 6.2 Future Work: The model’s 115M parameters motivate exploring INT8 or INT4 quantization and ONNX or CoreML optimization for privacy-preserving offline deployment on mobile and IoT devices.This is presented as a future deployment direction rather than a demonstrated result.
7 Conclusion
The paper presents Typhoon ASR Realtime as a compact streaming system that prioritizes data quality and architectural efficiency over parameter scaling. Its results support rigorous normalization for low-resource Thai ASR, while the Isan analysis distinguishes phonetic precision from semantic coherence in foundation models.
- Conclusion: Typhoon ASR Realtime prioritizes data quality and architectural efficiency rather than parameter scaling.The conclusion frames the system as a streaming ASR approach built around these principles.
- Conclusion: 115M parameters enable the model to outperform offline baselines 13× its size on standard benchmarks.The conclusion presents this as validation that rigorous normalization is essential for low-resource languages.
- Conclusion: Isan evaluation reveals a divergence between automated metrics and human preference: foundation models favor semantic coherence, while Typhoon ASR provides phonetic fidelity for technical ASR.The paper connects Typhoon ASR’s phonetic fidelity with the speed required for real-time deployment.