Source-linked AI summary
Beyond Unified Models: A Service-Oriented Approach to Low Latency, Context Aware Phonemization for Real Time TTS
Mahta Fetrat, Donya Navabi, Zahra Dehghanian, Morteza Abolghasemi, Hamid R. Rabiee
TL;DR
Lightweight real-time TTS faces difficulty handling context-dependent pronunciation without sacrificing speed, especially for ambiguous languages. The paper combines lightweight context-aware phonemization with a service-oriented architecture, and reports improved pronunciation soundness while maintaining low-latency real-time performance. It identifies limited prosodic and expressive capacity in lightweight speech synthesis as a remaining constraint on naturalness.
Problem
Lightweight TTS phonemizers struggle with context-dependent pronunciation, while larger context-aware models can compromise speed and efficiency.
Source: §1 · pp. 1–2Method
Results
The framework improves phonemization accuracy and synthesized-speech soundness while maintaining low latency and real-time performance.
Source: §5 · p. 6Takeaways & Limitations
The architecture is particularly suitable for offline, end-device, and low-latency applications such as screen readers.
Source: §5 · p. 6Even with corrected phoneme sequences, lightweight TTS models' limited capacity for prosodic and expressive features keeps complete naturalness out of reach.
Source: Limitations · p. 7
Abstract
from arXiv · showhide
Lightweight, real-time text-to-speech systems are crucial for accessibility. However, the most efficient TTS models often rely on lightweight phonemizers that struggle with context-dependent challenges. In contrast, more advanced phonemizers with a deeper linguistic understanding typically incur high computational costs, which prevents real-time performance. This paper examines the trade-off between phonemization quality and inference speed in G2P-aided TTS systems, introducing a practical framework to bridge this gap. We propose lightweight strategies for context-aware phonemization and a service-oriented TTS architecture that executes these modules as independent services. This design decouples heavy context-aware components from the core TTS engine, effectively breaking the latency barrier and enabling real-time use of high-quality phonemization models. Experimental results confirm that the proposed system improves pronunciation soundness and linguistic accuracy while maintaining real-time responsiveness, making it well-suited for offline and end-device TTS applications.
1 Introduction
Lightweight TTS must balance real-time operation and naturalness, while context-dependent phonemization challenges its lightweight G2P modules. The paper proposes lightweight context-aware components and a service-oriented design for integrating heavier phonemizers without embedding them in the TTS runtime.
1 Introduction
- Lightweight TTS must balance real-time performance and naturalness, while highly non-phonetic and ambiguous languages challenge context-dependent pronunciation.Screen readers are one motivating use case, with requirements for lightweightness, real-time performance, and naturalness.Source: §1 · pp. 1–2
- Decoupling G2P from P2S supports smaller, faster neural synthesizers, but output naturalness and intelligibility then depend heavily on G2P accuracy.Persian homographs and the Ezafe phoneme are cited as context-sensitive challenges.Source: §1 · pp. 1–2
- The paper combines lightweight, statistically driven context-aware modules with a service-oriented architecture that lets the TTS engine query heavier neural phonemizers independently.The design avoids embedding those components in the TTS runtime, reducing their computational and loading overhead.Source: §1 · p. 2
- The work proposes integrating neural components into real-time TTS through a service-oriented approach and presents a service-oriented adaptation of PiperTTS.These are listed among the paper's key contributions.Source: §1 · p. 2
- The paper introduces a lightweight, fast, context-aware phonemizer for Persian as an enhanced version of eSpeak.It is tailored to Persian phonemization challenges.Source: §1 · p. 2
- The work provides a new Persian Piper voice trained on the largest publicly available Persian TTS dataset to date.This is listed as a key contribution.Source: §1 · p. 2
2 Related Works
The related work traces TTS architectures and G2P methods across trade-offs in granularity, linguistic capability, and compute. It distinguishes this work’s internal service decomposition from systems that expose a complete TTS system as an API.
2.1 TTS approaches
- TTS architectures range from end-to-end models to modular pipelines; this study focuses on CPU-first, low-latency models and uses Piper’s modular VITS-based G2P/P2S design as its baseline.The surveyed progression runs from rule-based and concatenative methods through neural architectures and large language models; the paper also reviews VITS, FastSpeech, flow-based, and diffusion/consistency families.Source: §2.1 · pp. 2–3
2.2 G2P Tools
- Persian G2P tools performed poorly in a benchmark (Qharabagh et al., 2025b); Homo-GE2PE offers high quality, while fast non-neural HomoFast eSpeak has limited Ezafe detection (Qharabagh et al., 2025a).Statistical and rule-based methods are relatively lightweight, whereas neural models capture longer-range dependencies but typically require more compute and memory.Source: §2.2 · pp. 3–4
2.3 Decoupled TTSs
- Unlike API-based systems that expose an entire synthesizer, this work decomposes the TTS pipeline internally, separating heavy context-aware modules from the lightweight inference core.The authors state that this design improves responsiveness and enables real-time performance.Source: §2.3 · p. 4
3 Methodology
The methodology strengthens Piper’s G2P stage with complementary lightweight statistical and distilled neural approaches. It runs context-aware modules as independent services, refines the initial phoneme sequence, and fine-tunes the P2S model for enhanced phonemes.
3 Methodology
- The method retains Piper’s text-to-phoneme then neural P2S pipeline while targeting context-aware phonemization, where eSpeak’s rules and dictionary lookups are weak on Persian homographs and Ezafe.The P2S model is fine-tuned on enhanced phoneme sequences to handle Ezafe and distinguish homographs differing by only a few phonemes.Source: §3 · p. 4, §3.3 · p. 5
3.1 Statistical Context-Awareness
- For homograph disambiguation, the method adopts co-occurrence statistics that select pronunciations by contextual overlap, improving accuracy by up to 30 percentage points without added computational overhead or latency (Qharabagh et al., 2025a).The approach uses shallow contextual statistics rather than a heavy neural model.Source: §3.1 · p. 4
3.2 Distilled Linguistic Knowledge
- For Ezafe detection, the authors distill SpaCy’s Persian tagging knowledge into a smaller ALBERT-based model, reported to perform nearly comparably to its teacher.Ezafe detection is treated as a part-of-speech tagging subtask; the teacher is reported to achieve an F1 score of 0.99298 but to be relatively heavy and slow.Source: §3.2 · p. 4
3.3 Service-Based Integration
- Independent persistent services communicate with the core TTS module through file pipes; the service refines Piper’s initial phonemes by correcting homographs, inserting Ezafe, then returning them for P2S synthesis.The paper describes this decomposition as reducing latency by separating module initialization and inference.Source: §3.3 · pp. 4–5, Figure 3 · p. 5
4 Experiments
The experiments assess phonemization quality and speed, MOS, and the distilled Ezafe detector. They describe CPU-based evaluation settings and report that the distilled detector reduces size and computational overhead while retaining strong teacher-model performance.
4 Experiments
- The evaluation compares phonemization quality and inference speed, including Piper with lightweight context-aware components and a 300M-parameter neural G2P against Piper’s 15–20M parameters.The real-time-factor experiments used CPU-only inference on a Windows system with a 12th Gen Intel Core i7-1255U and 16 GB RAM.Source: §4 · p. 5, Table 1 · p. 6
4.1 Mean Opinion Score
- MOS was evaluated for baseline and enhanced TTS against natural speech using ratings from 16 native Persian speakers across seven utterances.The supplied passages identify Table 3 as reporting the MOS results but provide no scores.Source: §4.1 · pp. 5–6, Table 3 · p. 6
4.2 Ezafe Detection Module Evaluation
- The distilled Ezafe detector is reported to substantially reduce model size and computational overhead while retaining strong teacher-model performance.The evaluation was conducted on CPU in Google Colab; the supplied passages do not provide the table’s values.Source: §4.2 · p. 6, Table 2 · p. 6
5 Conclusion
The study addresses the trade-off between context-aware phonemization and lightweight, real-time G2P-aided TTS. Its framework combines lightweight auxiliary modules with a service-based architecture, improving phonemization accuracy without compromising real-time performance.
5 Conclusion
- The proposed approach combines lightweight, statistically driven context-aware modules with a service-based architecture for integrating heavier neural phonemizers.The independent services can be queried by the main TTS engine rather than embedded in its runtime.Source: §5 · p. 6
- The framework improves phonemization accuracy without compromising real-time performance by decoupling heavy context-aware components from the core runtime.The components run as independent services, maintaining low latency and improving synthesized-speech soundness.Source: §5 · p. 6
- The authors report that source code, models, and experimental results are publicly available.The passage points to a project repository.Source: §5 · p. 6
Limitations
The framework does not eliminate naturalness limits: lightweight phoneme-to-speech models may not reproduce higher-level prosodic and expressive features. The authors also identify perceptual-evaluation and service-layer improvements as areas for further work.
Limitations
- Even fully corrected phoneme sequences cannot deliver complete naturalness when lightweight phoneme-to-speech models lack capacity for higher-level prosody and expression.Further work is needed to improve naturalness while retaining speed and lightweight design.Source: Limitations · p. 7
- Naturalness depends more directly on smoothness, noiselessness, intonation, and stress than on phonemization, which primarily affects pronunciation soundness.The authors suggest separating phonemization accuracy from prosody and fluency in subjective evaluation.Source: Limitations · p. 7
- The service layer could be further optimized with request-level parallelism or asynchronous processing to reduce latency and improve scalability.These are proposed optimization opportunities for the decoupled setup.Source: Limitations · p. 7
A Selection Criteria for PiperTTS
PiperTTS was selected as the base architecture because its stated characteristics align with the study’s requirements. The selection criteria include deployment on low-end hardware, speed, perceptual quality, accessibility integration, open-source development, and Persian support.
A Selection Criteria for PiperTTS
- PiperTTS was selected as the base TTS architecture because the authors found its characteristics suitable for the study’s requirements.The listed considerations include lightweightness, speed, naturalness, accessibility integration, open-source development, and Persian support.Source: A Selection Criteria for PiperTTS · p. 9
- PiperTTS is described as running efficiently on CPUs and low-end devices such as Raspberry Pi, with an ONNX runtime export for deployment across hardware.The authors present this as a lightweightness strength.Source: A Selection Criteria for PiperTTS · p. 9
- PiperTTS is reported to have a real-time factor of approximately 0.2 and medium-to-high perceptual quality in publicly available checkpoints.These are separate speed and naturalness criteria in the selection rationale.Source: A Selection Criteria for PiperTTS · p. 9
- The selection rationale also cites PiperTTS’s NVDA screen-reader integration, open-source status, and active Persian-speaking community.The authors note an established synthesizer driver and available Persian checkpoints and training setups.Source: A Selection Criteria for PiperTTS · p. 9
- Figure 4 reports overall average MOS across seven utterances for each TTS system and natural speech, and its caption highlights improved naturalness for enhanced Piper.The supplied caption does not provide MOS values.Source: Figure 4 · p. 10
- Table 4 is described as reporting per-utterance MOS mean and standard deviation for each source, with source presentation order randomized per utterance.The supplied caption does not provide the table’s cell values.Source: Table 4 · p. 10
B Mean Opinion Score Details
The authors assessed perceived naturalness using seven utterances and ratings from native Persian speakers. The section describes the sample sources and rating procedure, while pointing to aggregate and per-utterance MOS results without supplying their values here.
B Mean Opinion Score Details
- The evaluation used seven utterances from a Nasl-e-Mana issue whose content was not included in the ManaTTS corpus used for fine-tuning.The publication serves the blind community and is the source of the publicly available ManaTTS dataset.Source: B Mean Opinion Score Details · p. 10
- Each utterance was generated by five sources, and source order was independently shuffled for each utterance to avoid bias from model reputation or perceived quality.The sources were GlowTTS, MatchaTTS, baseline PiperTTS, enhanced Piper, and natural speech.Source: B Mean Opinion Score Details · p. 11
- Sixteen native Persian speakers rated each audio clip’s naturalness on a 1–5 MOS scale, where 5 indicated the most natural pronunciation.Participants were asked to lower ratings for unnatural intonation, mispronunciation, or mechanical quality.Source: B Mean Opinion Score Details · p. 11
- Overall average MOS across utterances is shown in Figure 4, while Table 4 is described as giving per-utterance means and standard deviations.Figure 5 is identified as showing the distribution of participant MOS ratings for enhanced Piper; the supplied passages do not give numeric results.Source: B Mean Opinion Score Details · p. 11, Figure 5 · p. 11
Additional Figures
Figure 6 compares inference speed and phonemization quality, with Piper + LCA-G2P in the fast, high-quality region. The paper presents its service-based architecture as mitigating the speed–quality trade-off.
Additional Figures
- Figure 6 plots evaluated models by speed and quality, with inference speed measured using real-time factor (RTF) on a logarithmic scale.Source: Additional Figures · p. 12
- The quality metric rewards higher Ezafe F1 and homograph accuracy while penalizing higher phoneme error rate.The displayed formulation gives G2P Quality = Ezafe F1 + Homograph Acc.; quality values are schematically arranged while maintaining relative correctness.Source: Additional Figures · p. 12
- Piper + LCA-G2P occupies Figure 6’s top-right position, indicating strong inference speed alongside substantially improved phonemization quality.Source: Additional Figures · p. 12