Source-linked AI summary
Deterministic Prompting for Speaker-Stable Low-Resource Greek TTS
Georgios Syllas, Efthymios Georgiou, Kosmas Kritsis, Alexandros Potamianos
TL;DR
Low-resource Modern Greek lacks the clean, consistent single-speaker data that supports high-quality TTS. The paper combines curated audiobook-derived training data with multilingual Parler-TTS adaptation, deterministic prompting, and speaker-specific LoRA, achieving near-human intelligibility and intra-system speaker consistency under limited data and compute.
Problem
Modern Greek TTS is constrained by scarce clean single-speaker data, heterogeneous recordings, inconsistent segmentation, and imperfect transcriptions.
Method
The paper curates Greek audio into standardized TTS clips, adapts the 880M-parameter multilingual Parler-TTS model, replaces stochastic LLM prompts with deterministic prompts, and applies speaker-specific LoRA.
Results
WER = 10.7% vs. 7.8% for human audio, and MOS-C = 4.24 vs. 4.30 for human recordings, with deterministic prompting and LoRA producing stable speaker identity.
Takeaways & Limitations
Multilingual transfer, deterministic prompt conditioning, and speaker-specific LoRA provide a practical recipe for Greek TTS under severe data and compute constraints.
Takeaways & Limitations
Evaluation is limited to Modern Greek and one male LoRA speaker in a reading style, while the source recordings cannot be redistributed.
Abstract
from arXiv · showhide
Modern TTS systems approach human quality for high-resource languages but degrade when clean speech data is scarce. Modern Greek exemplifies this, lacking the curated corpora behind state-of-the-art synthesis. We propose a data curation recipe that transforms audiobook recordings into TTS-ready data via WhisperX alignment and filtering. Then we fine-tune Parler-TTS (880M), a prompt-based multilingual model whose pre-training encodes phonetic priors transferable to Greek. During development, we find that LLM-generated style prompts introduce speaker drift at inference. Replacing them with deterministic prompts resolves this, and a speaker-specific LoRA stage trained on 3.5 h of single-speaker data anchors identity while updating ~5% of parameters. Our system achieves WER 10.7% (2.9 above the ASR floor), MOS-I 4.00 (vs. 4.36 human speech), and near-human speaker consistency (MOS-C 4.24 vs. 4.30), showing that robust single-speaker Greek TTS is achievable with limited curated data.
1. Introduction
Modern Greek TTS is constrained by scarce, noisy, and heterogeneous speech data, especially for stable single-speaker synthesis. The paper addresses this gap with curated audiobook data, multilingual Parler-TTS adaptation, deterministic prompts, and speaker-specific LoRA.
- Motivation: Modern Greek lacks the large, clean, stylistically consistent single-speaker corpora supporting near-human TTS in high-resource settings.Available resources are either limited in clean single-speaker coverage or larger but heterogeneous and noisy.
- Contribution: The curation recipe aligns long recordings with WhisperX, segments them into TTS-ready clips, and filters alignment, duration, and acoustic quality.It transforms community recordings and audiobooks into standardized examples for adaptation.
- Contribution: Two audiobook-derived single-speaker Greek datasets compensate for the shortage of clean, long-duration speaker data.The datasets extend the available material beyond fragmented multi-speaker supervision.
- Model and adaptation: Parler-TTS is selected because multilingual pretraining supplies phonetic and prosodic priors transferable to Greek, including through its Spanish training data.The model is first fully fine-tuned on curated Greek data before speaker specialization.
- Key finding: LLM-generated style prompts cause noticeable speaker drift, whereas deterministic human-designed prompts resolve the instability when combined with speaker-specific LoRA trained on 3.5 h.The resulting configuration provides stable identity and high intelligibility.
- Contribution: The paper contributes a reusable curation pipeline and a two-stage adaptation recipe for anchoring single-speaker identity with limited speaker-specific data.The recipe combines full fine-tuning with deterministic prompting and speaker-specific LoRA.
2. Data Curation Pipeline
The data pipeline combines public Greek corpora with audiobook-derived speech, applying filtering and verification to produce training material suitable for low-resource TTS. Transcription quality is prioritized over raw volume, while source licensing limits redistribution.
- Public corpora: Common Voice is reduced from approximately 32 h across 412 speakers to 17.5 h after validation and manual cleaning, then to 15.5 h after TTS preprocessing.Cleaning removes clipping, background music, and transcription mismatches.
- Audiobook data: Approximately 3.5 h of manually verified male audiobook speech is reserved for speaker-specific LoRA adaptation.A separate automatically filtered 7.5 h corpus is excluded from the final pipeline.
- Quality control: The excluded 7.5 h audiobook corpus increased hallucinated syllables because residual ASR errors propagated into training.This supports prioritizing transcription accuracy over raw data volume in low-resource TTS.
- Reproducibility: The source recordings are privately licensed and cannot be redistributed, although preparation templates are released for researchers with access to them.This constrains direct data sharing while supporting reproducibility of the preparation process.
- Augmentation: Voice-conversion augmentation produces artifacts and does not improve performance, so it is omitted from evaluation.The experiment attempted to convert Common Voice samples toward the CSS10 speaker timbre.
3. Models and Adaptation
The study compares a Greek-only baseline with multilingual Parler-TTS and uses staged adaptation to improve Greek synthesis while preserving pretrained multilingual knowledge. Deterministic prompts and LoRA specifically target timbre drift and speaker specialization.
- Baseline: The Greek-only VITS baseline exhibits monotonic prosody, timbre inconsistency, and audible artifacts, preventing formal quality evaluation.These results motivate using a multilingual foundation model instead.
- Model: Parler-TTS generates discrete audio tokens autoregressively from transcript and natural-language style conditioning, with a DAC decoder reconstructing the waveform.Its architecture includes a frozen style-text encoder, Transformer decoder, and residual vector quantization codebooks.
- Model: The multilingual checkpoint is chosen because Spanish pretraining is treated as a favorable, but not guaranteed, cross-lingual prior for Greek.Its prompt-conditioned and modular design also supports explicit style control and parameter-efficient adaptation.
- Full fine-tuning: Full fine-tuning uses multi-speaker Greek data, while deterministic prompts replace stochastic LLM-generated descriptions that destabilize optimization and generation.Extended training improves prosody and naturalness but can introduce hallucinated syllables and timbre drift.
- LoRA adaptation: LoRA updates approximately 25 M parameters, or 5% of the 500 M-parameter decoder, to anchor one speaker without overwriting multilingual phonetic knowledge.The adapters are inserted into Transformer decoder attention projections.
4. Evaluation
Evaluation combines objective intelligibility and speaker-similarity measures with a native-speaker listening study. The design separates held-out Common Voice testing from audiobook-based speaker evaluation and measures naturalness and intelligibility subjectively.
- Objective evaluation: WER and CER are measured with WhisperX v3 on 50 held-out Common Voice utterances after text normalization.The same evaluation also computes MCD and ECAPA-TDNN embedding similarity on 20 held-out audiobook utterances.
- Subjective evaluation: A listening study with 29 native Greek speakers rates naturalness and intelligibility on 5-point Likert scales.The reported MOS-N and MOS-I values use the 25 participants who completed all ratings.
5. Results
Deterministic prompting paired with speaker-specific LoRA improves objective intelligibility and supports stable voice consistency, while subjective differences require cautious interpretation under corrected statistical tests.
- 10.7% WER with deterministic prompts plus LoRA substantially outperforms 21.1% with LLM prompts after LoRA adaptation.The best configuration is 2.9 percentage points above the 7.8% ASR floor on human audio.
- LoRA reduces hallucinated syllables and punctuation–prosody mismatches, but lexical-stress errors persist at low rates across configurations.
- Deterministic prompts without LoRA achieve MOS-N = 3.76 and MOS-I = 4.11, the highest naturalness and intelligibility means among synthetic systems.The objective WER/CER benefit emerges after LoRA adaptation.
- Det. + LoRA reaches MOS-C = 4.24 versus 4.30 for a human speaker, whereas LLM + LoRA reaches 3.56.
- The intelligibility omnibus test is significant, but only the LLM baseline differs significantly from human recordings after Holm correction.For naturalness, χ2 = 3.83 and p = 0.43; for intelligibility, χ2 = 10.26 and p = 0.036.
6. Discussion
Multilingual transfer, deterministic conditioning, and speaker-specific LoRA form a practical recipe for low-resource Greek TTS, with near-human intra-system consistency but limited scope and unresolved causal questions.
- Multilingual pretraining provided cross-lingual phonetic priors that made Greek adaptation feasible under limited data.A Greek-only baseline did not reach sufficient quality for formal evaluation.
- Deterministic prompts reduced conditioning variance and yielded the highest mean MOS among synthetic systems, especially with LoRA.
- Speaker-specific LoRA yielded MOS-C = 4.24, with trends favoring Det. + LoRA over LLM + LoRA at padj = 0.076.
- The experiments are limited to Modern Greek and one male LoRA speaker in a reading style, with broader multilingual and speaker/style coverage left for future work.
- The best configuration approaches human recordings at WER = 10.7% versus 7.8% and MOS-C = 4.24 versus 4.30 despite limited data and compute.
8. Generative AI Use Disclosure
The authors disclose that a large language model assisted with language editing, while the technical content and experimental results are based on the authors’ work.
- A large language model assisted with language editing, but the authors’ work provides the technical content and experimental results.