Source-linked AI summary
Bolbosh: Script-Aware Flow Matching for Kashmiri Text-to-Speech
Tajamul Ashraf, Burhaan Rasheed Zargar, Saeed Abdul Muizz, Ifrah Mushtaq, Nazima Mehdi, Iqra Altaf Gillani, Aadil Amin Kak, Janibul Bashir
TL;DR
Kashmiri lacks robust neural TTS despite its large speaker community, and zero-shot multilingual systems struggle with its diacritic-rich script. Bolbosh adapts an OT-CFM Matcha-TTS model through supervised cross-lingual training, acoustic enhancement, and script-aware encoding. It achieves substantially better Kashmiri synthesis quality than multilingual baselines, while evaluation remains constrained by high baseline ASR error.
Problem
Kashmiri is severely under-resourced in speech technology, and multilingual TTS systems struggle with its diacritic-rich Perso-Arabic script and language-specific phonotactics.
Method
Bolbosh uses supervised cross-lingual adaptation of Matcha-TTS with OT-CFM, enhanced heterogeneous speech data, and an expanded Kashmiri grapheme vocabulary.
Results
3.63 MOS and 3.73 MCD substantially outperform multilingual baselines, establishing a new benchmark for Kashmiri speech synthesis.
Takeaways & Limitations
The findings demonstrate the importance of script-aware encoding and supervised flow-based adaptation for low-resource TTS in diacritic-sensitive languages.
Takeaways & Limitations
WER is treated as a supplementary metric because the ground-truth recordings have a high baseline ASR error rate.
Abstract
from arXiv · showhide
Kashmiri is spoken by around 7 million people but remains critically underserved in speech technology, despite its official status and rich linguistic heritage. The lack of robust Text-to-Speech (TTS) systems limits digital accessibility and inclusive human-computer interaction for native speakers. In this work, we present the first dedicated open-source neural TTS system designed for Kashmiri. We show that zero-shot multilingual baselines trained for Indic languages fail to produce intelligible speech, achieving a Mean Opinion Score (MOS) of only 1.86, largely due to inadequate modeling of Perso-Arabic diacritics and language-specific phonotactics. To address these limitations, we propose Bolbosh, a supervised cross-lingual adaptation strategy based on Optimal Transport Conditional Flow Matching (OT-CFM) within the Matcha-TTS framework. This enables stable alignment under limited paired data. We further introduce a three-stage acoustic enhancement pipeline consisting of dereverberation, silence trimming, and loudness normalization to unify heterogeneous speech sources and stabilize alignment learning. The model vocabulary is expanded to explicitly encode Kashmiri graphemes, preserving fine-grained vowel distinctions. Our system achieves a MOS of 3.63 and a Mel-Cepstral Distortion (MCD) of 3.73, substantially outperforming multilingual baselines and establishing a new benchmark for Kashmiri speech synthesis. Our results demonstrate that script-aware and supervised flow-based adaptation are critical for low-resource TTS in diacritic-sensitive languages. Code and data are available at: https://github.com/gaash-lab/Bolbosh.
1. Introduction
Kashmiri is a widely spoken but severely under-resourced language whose script and linguistic complexity challenge speech technology. Bolbosh addresses this gap with a script-aware system that substantially outperforms multilingual baselines.
- Motivation: Kashmiri is spoken by approximately 7 million people but remains severely under-resourced in speech technology.The lack of high-quality TTS limits accessibility, digital participation, and inclusive human-computer interaction.
- Challenges: Scarce paired corpora, multiple writing systems, diacritic-dependent vowel distinctions, and dialectal variation complicate Kashmiri TTS development.These factors can substantially degrade zero-shot multilingual TTS performance.
- Contributions: Bolbosh introduces a script-aware, flow-matching-based Kashmiri TTS system with acoustic-domain integration and supervised cross-lingual adaptation.The framework unifies heterogeneous speech corpora and leverages a pretrained English multi-speaker checkpoint.
- Results: 3.63 MOS and 3.73 MCD substantially outperform the multilingual baseline’s 1.86 MOS, establishing a new benchmark for Kashmiri speech synthesis.The reported results support the system’s state-of-the-art performance claim.
- Implications: The findings highlight limitations of multilingual TTS in modeling diacritic-sensitive scripts and the importance of script-aware encoding for low-resource synthesis.The broader conclusion extends beyond Kashmiri to diacritic-sensitive languages.
2. Related Work
Existing Indic speech systems provide multilingual baselines but leave Kashmiri largely unsupported. Flow matching offers the architectural basis for Bolbosh’s low-resource cross-lingual adaptation.
- Existing Systems: Kashmiri lacks a publicly available neural TTS system, while multilingual systems that unofficially support it perform poorly.These systems are not explicitly adapted to Kashmiri phonotactics or its diacritic-rich Perso-Arabic script.
- Flow-Matching TTS: OT-CFM transports a simple prior distribution to the target acoustic distribution through a direct continuous-time vector field.Unlike diffusion models, this formulation avoids iterative Markov chains while preserving stable training dynamics.
- Flow-Matching TTS: Flow-matching models are presented as suitable for low-resource TTS because they combine stable training, sample-efficient learning, faster inference, and end-to-end alignment.The paper adopts Matcha-TTS as its OT-CFM backbone.
3. Kashmiri TTS Dataset
The 79.9-hour Kashmiri corpus combines clean studio speech with diverse spontaneous recordings. Acoustic enhancement and script-preserving normalization are used to support stable alignment and fine-grained pronunciation modeling.
- Corpus: 79.9 hours of Kashmiri speech are split into training, validation, and test sets.Training combines RASA and IndicVoices-R, whereas validation and test data come exclusively from RASA.
- Data Sources: RASA provides clean studio speech, while IndicVoices-R contributes multi-speaker and prosodic diversity from predominantly spontaneous recordings.IndicVoices-R contains noise, reverberation, and amplitude variation that can destabilize alignment and degrade synthesis quality.
- Acoustic Enhancement: A three-stage enhancement pipeline is applied to heterogeneous recordings because MAS-based flow architectures require clean, temporally consistent audio.The supplied passage identifies dereverberation and denoising as part of this pipeline.
- Text Processing: The normalization pipeline canonicalizes Unicode, expands numbers, filters characters, and preserves pronunciation-critical diacritics.The model vocabulary is expanded to 272 letters and language-specific text cleaners are disabled.
- Text Processing: The expanded vocabulary enables end-to-end grapheme-to-acoustic mapping while preserving fine-grained vowel distinctions.This design directly targets Kashmiri’s diacritic-sensitive orthography.
4. Proposed Framework
Bolbosh extends Matcha-TTS with OT-CFM, auxiliary acoustic supervision, cross-lingual initialization, and multi-speaker regularization. These components target stable alignment and generalization in low-resource Kashmiri adaptation.
- Architecture: Bolbosh uses a text encoder, duration, pitch, and energy predictors, an OT-CFM decoder, and a neural vocoder.The system extends Matcha-TTS for script-aware low-resource cross-lingual adaptation.
- Training Objective: The total objective combines mel-spectrogram reconstruction with duration, pitch, and energy supervision.Mel loss uses L1 reconstruction, while the auxiliary objectives use grapheme-level regression targets.
- Training Objective: The flow-matching objective regularizes the learned velocity field toward the optimal-transport trajectory between a Gaussian prior and target mel distributions.This provides the flow-based component of the training formulation.
- Cross-Lingual Adaptation: A pretrained English multi-speaker Matcha-TTS checkpoint supplies transferable acoustic representations and accelerates MAS convergence on Kashmiri data.The strategy avoids training the flow-based model entirely from scratch.
- Multi-Speaker Regularization: Enhanced IndicVoices-R data and learned speaker embeddings regularize training, while RASA speaker embeddings are used exclusively during inference.IndicVoices-R supplies acoustic diversity and RASA anchors high-fidelity synthesis.
5. Results and Discussion
Bolbosh substantially improves Kashmiri TTS objective and subjective quality over multilingual baselines, while spectrograms show clearer acoustic structure and the evaluation highlights limitations in Kashmiri ASR.
- Evaluation Metrics: 41.20% WER: the diacritics-removed IndicConformer RNN-T has the lowest Kashmiri ASR error among the compared systems.Retaining diacritics increases WER to 66.59%, so WER is treated as supplementary for proxy intelligibility evaluation.
- Objective TTS Evaluation: 3.73 MCD: Bolbosh outperforms IndicParler’s 4.73, indicating lower spectral distortion.Under diacritic-preserving input, Bolbosh also achieves 4.14% rWER versus 46.75% for IndicParler.
- Objective TTS Evaluation: 13.23% rWER without diacritics versus 4.14% with them, showing that explicit diacritic modeling improves intelligibility.IndicParler’s rWER rises to 100.32% when diacritics are removed.
- Subjective Human Evaluation: 3.634 ±0.061 MOS: Bolbosh significantly outperforms IndicParler at 1.864 ±0.065.Ground-truth recordings score 4.614 ±0.059, leaving a remaining gap from natural speech.
- Subjective Human Evaluation: The baseline frequently produces unintelligible or prosodically distorted outputs, while supervised flow-matching adaptation yields near-natural speech quality.This qualitative finding accompanies the large MOS difference between Bolbosh and IndicParler.
- Spectral Analysis: Bolbosh preserves clear harmonic structures and formant trajectories, whereas IndicParler shows oversmoothing, blurred formants, and temporal instability.The qualitative contrast is consistent with Bolbosh’s lower MCD and higher MOS.
6. Conclusion
The paper introduces Bolbosh as an open-source Kashmiri TTS system and reports improved synthesis through script-aware modeling and supervised flow-based adaptation. It establishes benchmark results while identifying future directions including dialect modeling, prosody control, and transfer to other under-resourced languages.
- 6. Conclusion: Bolbosh is presented as the first dedicated, open-source TTS system optimized for Kashmiri.The system targets synthesis in a low-resource setting.
- 6. Conclusion: The system combines a pretrained English multi-speaker checkpoint, Kashmiri characters and diacritics, and structured multi-speaker regularization.These components are described as supporting stable alignment and high-fidelity synthesis.
- 6. Conclusion: 3.73 MCD and 3.63 MOS: Bolbosh substantially outperforms multilingual baselines and establishes a new Kashmiri TTS benchmark.The conclusion presents these as the system’s principal objective and subjective results.
- 6. Conclusion: The findings emphasize script-aware modeling and flow-based supervised adaptation for scalable low-resource speech synthesis.The conclusion frames this as the broader significance of the work.
- 6. Conclusion: Future work will explore multi-dialect modeling, enhanced prosody control, and cross-lingual transfer to other under-resourced languages.These directions define the stated scope for extending the system.