PaperlayerView exact revision on arXiv

Source-linked AI summary

Balalaika-Longform: A Russian Speech Corpus for Continuous Long-Form Text-to-Speech

Nikita Vasiliev, Kirill Borodin, Vasilii Kudryavtsev, Maxim Maslov, Grach Mkrtchian

arXiv:2610.00658v1cs.SD

TL;DR

Short-output evaluation does not test whether a model can sustain a requested reading in one pass. Balalaika-Longform pairs long Russian recordings with matched short windows and evaluates continuous synthesis across four backbones. Long targets favor the three autoregressive models and, by a small margin, F5-TTS, whose reading remains unreliable.

  • Problem

    Short-output evaluation can conceal omissions and early stops, and chunked synthesis does not test whether a model can sustain a single-pass reading.

    Source: §1 · p. 1

  • Method

    Balalaika-Longform pairs continuous Russian speech units with short windows from the same recordings and evaluates continuous fine-tuned synthesis while retaining every attempt.

    Source: §1 · p. 1

  • Results

    At about 1,200 words, Long-SFT reduced CosyVoice3 WER by 52.7 pp [−62.7, −42.2]; paired intervals favor long targets for the three autoregressive backbones and slightly for F5-TTS.

    Source: §4.2 · p. 4, §6 · p. 4

  • Takeaways & Limitations

    The corpus supports studying sustained reading while keeping content fidelity and perceptual quality distinct.

    Source: §6 · p. 4

    The 50 text–voice pairs were selected from a planned 120 after preliminary results had been seen, no listening study was run, and corpus transcripts and speaker labels are automatic.

    Source: §5 · p. 4

Abstract

from arXiv · show

Long-form text-to-speech must retain requested words over extended generations, yet sentence-level training and evaluation can hide omissions and early stops. We introduce Balalaika-Longform, an open Russian corpus of 189 hours in continuous units of 30 seconds to 15 minutes. Long units and matched short windows support fine-tuning comparisons on the same source recordings, and the accompanying evaluation retains every synthesis attempt. We fine-tune CosyVoice3, Qwen3-TTS, VoxCPM2 and F5-TTS on either view and test continuous synthesis on 50 text-voice pairs with voices unseen in fine-tuning, at about 75, 300 and 1,200 words. At 1,200 words, CosyVoice3 WER falls from 99.9% after short-window fine-tuning to 47.3% with long targets, and to 16.6% with punctuated, format-matched transcripts; paired intervals favor long targets for Qwen3-TTS and VoxCPM2, and by a small margin for F5-TTS, whose absolute WER stays above 90%. The results isolate the effect of training sequence length under a fixed continuous-generation protocol.

1. INTRODUCTION

Sentence-level speech synthesis can conceal omissions and early stops, while chunked pipelines do not test sustained single-pass reading. Balalaika-Longform pairs long and short Russian training views and evaluates their use across generation families.

1. INTRODUCTION

  • Sentence-level training and evaluation can conceal omissions and early stops, while chunking cannot test whether a model sustains a reading in one pass.Chunked synthesis also discards cross-sentence prosodic and discourse context.Source: §1 · p. 1
  • Balalaika-Longform provides continuous Russian speech units up to 15 minutes and paired short-window and long-sequence views from the same source recordings.Channel-disjoint splits support source-controlled comparisons of sequence length.Source: §1 · p. 1
  • The evaluation compares four TTS backbones on fixed text–voice pairs and retains every failure in word-error and correct-word-recall estimates.A controlled two-voice CosyVoice3 experiment tests duration under matched training exposure.Source: §1 · p. 1
  • The study separates the training-data question from continuous-versus-chunked generation and states transcript-source, formatting, and corpus-composition confounds.Its contributions are the long-form corpus, paired training views, and all-attempt evaluation across generation families.Source: §1 · p. 1

2. RELATED WORK

Prior work addresses long-form speech through corpus construction, inference and model design, and multidimensional evaluation. Balalaika-Longform focuses instead on training data and continuous Russian synthesis evaluation.

2. RELATED WORK

  • Balalaika-Longform builds on Balalaika’s context-preserving segmentation, multi-ASR transcription, and speaker-purity screening to organize Russian speech into long continuous units.Related corpora include English audiobook resources and Russian single-speaker or uncurated audio collections.Source: §2 · p. 1
  • Long-form methods include state carried across chunks, duration modeling, multiminute sampling, and non-autoregressive flow matching.The paper studies whether long training targets help existing models under uninterrupted inference, without assuming long-form training is necessary for every method.Source: §2 · p. 1
  • The paper claims Balalaika-Longform is the first openly released Russian long-form TTS corpus and its all-attempt evaluation the first public Russian long-form TTS evaluation.Its evaluation controls input duration, retains failed attempts, and tests whether one generation reads the requested text.Source: §2 · p. 1

3. CORPUS AND EVALUATION DESIGN

The design pairs continuous Russian recordings with short windows, compares fine-tuning views under continuous inference, and evaluates every attempt with paired uncertainty estimates.

3.1. Corpus, data splits, and training arms

  • The corpus contains 6,380 Russian single-speaker segments totaling 189.13 hours, split by YouTube channel before windowing and training.Units span 30–900 seconds; transcripts and speaker labels are automatic and uncorrected, and held-out channels supply reference voices without gradient updates.Source: §3.1 · p. 2

3.2. Fine-tuning to assess corpus utility

  • Short-SFT trains on windows that tile the same recordings, while Long-SFT concatenates them; Long-SFT-Punct keeps long audio tokens but changes transcript source and format.Qwen3-TTS and VoxCPM2 use punctuated text in both arms. VoxCPM2 drops long units exceeding its 8,192-position budget, so its comparison also differs in corpus composition; F5-TTS trains on intact recordings or windows.Source: §3.1 · p. 2, §3.2 · p. 2

3.3. State-continuous inference and evaluation sets

  • Continuous inference uses one target text and one voice-conditioning event, with splitting, state resets, best-of-N selection, and regeneration disabled.The published-text set uses 50 fixed pairs from 22 works and 21 reference speakers unseen in fine-tuning; Table 2 covers six texts, five duration buckets, two voices, and 60 attempts per arm. Windowed diagnostics use 5-second windows with a 2.5-second hop.Source: §3.3 · p. 2, Table 2 · p. 3, Figure 1 · p. 3

3.4. Evaluator and paired uncertainty

  • Published-text WER counts substitutions, deletions, and insertions, and paired 95% intervals resample 22 work clusters while retaining associated prefixes and voices.The intervals condition on fitted checkpoints and inference seed and do not adjust for multiple comparisons; the two-voice set instead resamples six source texts.Source: §3.4 · p. 3

4. EXPERIMENTS

Across continuous synthesis, long-sequence fine-tuning improved content fidelity over short-window training, while the experiments distinguish duration effects from transcript formatting and add perceptual-quality checks.

4.1. Earlier controlled duration comparison

  • Long-SFT gained 45.0 completion points [33.3, 56.7] over Short-SFT under matched exposure, while neither Base nor Short-SFT completed items beyond 90 seconds.The two-voice results are descriptive: repeated decoding shifted bucket percentages by up to about 20 points.Source: §4.1 · p. 3
  • Long-SFT-Punct reached 83.3% completion, adding 11.7 completion points and reducing WER by 6.11 points versus Long-SFT, but transcript source and formatting both changed.Format-matched lowercased, unpunctuated input reached 88.3% completion and 10.07% WER, so a separate punctuation-content benefit remains unestablished.Source: §4.1 · p. 3

4.2. Content fidelity across backbones

  • At about 1,200 words, Long-minus-Short WER changes were −52.7 pp for CosyVoice3 [−62.7, −42.2], −52.7 pp for Qwen3-TTS [−61.7, −43.4], and −41.5 pp for VoxCPM2 [−47.7, −35.4].These are paired effects on the same 22 work passages; Table 3 evaluates all attempts across input lengths.Source: Table 3 · p. 4, §4.2 · p. 4

4.3. Flow matching and inference protocol

  • At about 75 words, F5-TTS Long-SFT-Punct lowered WER from 44.2% to 27.4%, its largest gain, while its longest-input relative improvement did not establish reliable reading.The duration schedule is shared across checkpoints, making this a content-realization rather than EOS-decision comparison.Source: §4.3 · p. 4
  • On a separate earlier recorded-text set, F5-TTS Base WER was 3.58% with sentence chunking and 73.35% with continuous synthesis; these texts do not establish a Table 3 ranking.The two values use different texts from the published-text comparison.Source: §4.3 · p. 4

4.4. Evaluation checks and practical limits

  • F5-TTS Long-SFT-Punct had higher DistillMOS than Base despite high WER, while speaker similarity declined over time for Qwen3-TTS and VoxCPM2 but stayed flat for CosyVoice3.The perceptual proxies condition on scorable windows; truncated outputs can score well, so they complement rather than replace all-attempt WER and listening tests.Source: §4.4 · p. 4

5. LIMITATIONS

The published-text evaluation and several experimental factors constrain how broadly to interpret the reported results.

5. LIMITATIONS

  • The published-text evaluation covers 22 literary works and 21 voices, with 50 pairs selected from a planned 120 by a metadata-only rule after preliminary results were seen.The rule was metadata-only, but selection followed preliminary results.Source: §5 · p. 4
  • No listening study was run, and GigaAM supplied the training transcripts and scored the synthesized outputs.Corpus transcripts and speaker labels are automatic.Source: §5 · p. 4
  • Transcript-source, formatting, and VoxCPM2 corpus-composition effects are not isolated, and one F5-TTS model does not represent all non-autoregressive systems.Source: §5 · p. 4

6. CONCLUSION

Balalaika-Longform pairs continuous Russian speech with short-window training views, and the experiments report long-target advantages of differing size across four backbones.

6. CONCLUSION

  • Balalaika-Longform preserves long continuous Russian speech and provides paired short-window training views.Source: §6 · p. 4
  • Paired intervals favor long targets for CosyVoice3, Qwen3-TTS, and VoxCPM2, and favor them by a small margin for F5-TTS, whose reading remains unreliable.Source: §6 · p. 4
  • The corpus supports studying sustained reading while keeping content fidelity distinct from perceptual quality.Source: §6 · p. 4

7. COMPLIANCE WITH ETHICAL STANDARDS

The study uses retrospectively collected recordings and describes the limits of the license evidence and release.

7. COMPLIANCE WITH ETHICAL STANDARDS

  • The study used speech recordings retrospectively; uploaders had published them on YouTube with a Creative Commons Attribution declaration.Source: §7 · p. 5
  • The uploader’s declaration is license evidence, not an independent audit of rights in underlying texts or performances or proof of speaker consent for synthesis.Source: §7 · p. 5
  • No recordings were made and no experiment involved human participants; the release does not relicense third-party material, which remains under its source licenses.Attribution corrections and removal requests are handled through the repository discussion page and the stated contact address.Source: §7 · p. 5
Loading 2610.00658v1…