PaperlayerView exact revision on arXiv

Source-linked AI summary

Ready to Speak: Aligning LLMs for TTS-Friendly Text Generation

Thibaut Thonet, Jos Rozen, Laurent Besacier

arXiv:2609.01246v1cs.CL

TL;DR

LLMs optimized for written text may generate outputs that are poorly suited to TTS, motivating direct generation of TTS-friendly text. The paper frames this as preference alignment, introduces domain-spanning datasets and evaluations, and finds that FaST offers the best overall tradeoff between TTS-friendliness and helpfulness. Its heuristic metric also strongly correlates with TTS→ASR results and human judgments.

  • Problem

    LLMs can produce grammatically correct and helpful text that is not optimized for spoken delivery by TTS systems.

  • Method

    The paper directly aligns LLMs for TTS-friendly generation using CORA and Recipe preference datasets, interpretable features, and heuristic, TTS→ASR, and human evaluations.

  • Results

    FaST achieves the best overall tradeoff between TTS-friendliness and helpfulness across datasets and settings, with particularly strong gains from few training samples.

  • Takeaways & Limitations

    TTS-friendliness can be assessed efficiently with a heuristic metric that strongly correlates with TTS→ASR metrics and human judgments.

  • Takeaways & Limitations

    The empirical study is limited to English, two domains, and 3B/4B-parameter models, leaving broader-domain, multilingual, and larger-scale generalization open.

Abstract

from arXiv · show

Current Large Language Models (LLMs) are primarily optimized for written text, often producing outputs that are grammatically correct and helpful yet poorly suited for spoken delivery via Text-to-Speech (TTS). In this work, we study how to make LLMs natively generate TTS-friendly text, which we frame as a preference alignment problem: instead of relying on downstream rewriting modules, we directly align LLMs to generate text optimized for spoken delivery. We introduce two preference datasets spanning different target domains, CORA and Recipe, which contain paired TTS-friendly and TTS-unfriendly responses. We further propose an evaluation suite combining a pattern-based heuristic metric, a TTS$\to$ASR evaluation pipeline, and a MUSHRA listening study with human judges. Our experiments compare the recently proposed Feature-aware Sampling and Tuning (FaST) framework -- leveraging interpretable features instead of a black-box reward model -- against an array of alignment baselines on the TTS-friendly generation task. Notably, we found that FaST achieves the best overall tradeoff between TTS-friendliness and helpfulness across various settings. We also identified a strong correlation between our different metrics, highlighting the ability to reliably assess TTS-friendliness via an efficient heuristic.

1. Introduction

The paper frames TTS-friendly generation as directly aligning LLMs to produce text optimized for spoken delivery while preserving helpfulness. It introduces datasets, metrics, and FaST-based comparisons, finding a strong TTS-friendliness/helpfulness tradeoff even with limited data.

  • Motivation: LLMs trained mainly on textual data can produce grammatically correct outputs that are not optimized for TTS spoken delivery.TTS-friendly text must support semantic correctness and fluency while also satisfying spoken-delivery characteristics.
  • Approach: The paper directly aligns LLMs to generate TTS-ready outputs instead of relying on post-hoc text normalization.Direct generation can simplify architecture, reduce latency and inference costs, and support any TTS system.
  • Approach: TTS-friendliness is defined through interpretable rules and measurable objectives covering symbols, abbreviations, numeric notation, factual correctness, and helpfulness.The challenge is explicitly multi-objective rather than purely stylistic.
  • Contributions: The study introduces CORA and Recipe preference datasets and evaluates alignment methods using heuristic, TTS→ASR, and human-listening measures.FaST is compared with prompting, SFT, DPO, GRPO, and RFT across two datasets and two data regimes.
  • Results: FaST achieves the best TTS-friendliness/helpfulness tradeoff, outperforming baselines with as few as 10 examples.The remaining introduction text describes the paper’s sections rather than a substantive contribution.

2. Related Work

Prior work studies speech-oriented generation, controllable speech characteristics, domain-specific speech challenges, and constrained generation. The paper identifies gaps in upstream text alignment and efficient, disentangled evaluation of TTS-friendliness.

  • Speech-oriented alignment: Earlier work tunes LLMs for speechworthy text, while other preference-alignment studies target TTS acoustic outputs rather than upstream text.The paper positions its focus on text generation before synthesis.
  • Related approaches: Other approaches control spoken characteristics continuously or address domain-specific speech challenges, but may require architectural modifications or specialized coverage.The cited examples span controllable generation, mathematical documents, and broader speech-synthesis surveys.
  • Constrained generation: Constrained-generation research offers interpretable ways to satisfy explicit form requirements while preserving content, but had not been applied to TTS-friendliness.The paper connects this direction to low-data, constraint-aware generation.
  • Identified gaps: The paper addresses two gaps: disentangling TTS-friendliness from helpfulness upstream of TTS, and replacing costly subjective judgments with interpretable text-level features and metrics.These features also guide construction of synthetic preference datasets.

3. Datasets & Evaluation Metrics

The study uses paired preference data from conversational and procedural domains, then evaluates both task helpfulness and TTS-friendliness. Its metrics combine interpretable pattern scoring with synthesis-transcription and rubric-based helpfulness assessment.

  • Datasets: CORA covers coffee-shop conversational responses, while Recipe uses sampled cooking recipes for procedural descriptions.Each instance contains a TTS-friendly chosen response and a TTS-unfriendly rejected response.
  • Datasets: Both datasets use five reshuffled train/validation/test splits from shared pools designed for low-data preference alignment.The dataset sizes target deployment scenarios with only a few preference tuples per domain.
  • Metrics: The heuristic TTS-friendliness metric scores responses from 1–5 and penalizes symbols, abbreviations, URLs, email addresses, and poorly formatted quantities.Higher scores indicate greater TTS-friendliness.
  • Metrics: The TTS→ASR metric synthesizes responses and transcribes them back, using CER and WER against the original text; lower scores indicate greater TTS-friendliness.The pipeline uses Kyutai Pocket TTS and Wav2Vec2-large.
  • Metrics: Helpfulness is independently rated from 1–5 for factual correctness and request fulfillment, with Recipe judgments anchored to the preferred response.The judge first evaluates atomic requirements before assigning an overall score.

4. Method

FaST aligns generation through interpretable feature discovery, scoring, weighting, and iterative sampling-and-tuning. It is evaluated against prompting, supervised, preference-based, and traditional reward-model baselines.

  • Method: FaST targets responses that are optimized for spoken delivery while preserving helpfulness through interpretable feature-based preference alignment.The method avoids relying on traditional monolithic reward models.
  • Method: FaST proceeds through feature discovery, feature-wise response scoring, feature-weight learning, and sampling-and-tuning alignment.These four stages define the framework’s pipeline.
  • Feature discovery: Feature discovery prompts an LLM to identify high-level stylistic and structural properties that distinguish preference options.The resulting features are designed to be interpretable.
  • Feature scoring: Each response is independently scored on every feature using frozen LLM-based feature functions that require no parameter learning.Scores range from 1 to 5 for feature prevalence.
  • Feature weighting: FaRM combines feature scores linearly, learning one scalar weight per feature from preference annotations to capture objective trade-offs.In CORA, learned weights favor conversational phrasing and spelled-out numbers while penalizing abbreviations and symbols.
  • Alignment: Iterative alignment samples candidates, ranks them with FaRM, and fine-tunes the generation model on top-ranked responses using DPO or SFT.This sampling-and-tuning procedure is the final FaST stage.
  • Baselines: The comparison includes prompting, SFT, DPO, GRPO-RM, and RFT-RM baselines, spanning reward-model-free and traditional reward-model approaches.FaST is evaluated alongside multiple alignment strategies rather than a single baseline.
  • Baselines: The Oracle returns dataset-provided TTS-friendly chosen responses as an approximate upper bound on achievable performance.It is defined for comparison against generated outputs.

5. Experimental Setup

Experiments evaluate alignment approaches on CORA and Recipe using heuristic TTS-friendliness and Helpfulness, under full-data and 10-sample training regimes. Qwen3-4B is the primary base model, with FaST implemented using discovered features.

  • Evaluation: Experiments compare approaches on CORA and Recipe using heuristic TTS-friendliness and LLM-judge Helpfulness.The datasets represent conversational and procedural text domains.
  • Training regimes: The study evaluates both 10-sample and full-data settings, corresponding to 10 and 100 training samples.These regimes approximate low-data scenarios from prior work.
  • Evaluation: The heuristic metric is evaluated alongside a TTS→ASR pipeline and human judgments through reported validation of metric correlation.The heuristic metric is described as correlating well with both evaluation approaches.
  • Models: Qwen3-4B is the primary base model, with reasoning abilities disabled.Results for SmolLM3-3B are reported separately, and Qwen3-4B is also used to implement FaST feature functions.

6. Results

FaST generally offers the strongest balance between TTS-friendliness and Helpfulness, especially with limited data, while SFT becomes highly competitive with 100 samples. Feature analysis links spoken-friendly outputs to conversational and expanded-number phrasing and disfavors compact written conventions.

  • Generation results: FaST achieves the best overall tradeoff between TTS-friendliness and Helpfulness across nearly all experimental conditions.It lies on or near the Pareto frontier and remains competitive across CORA and Recipe.
  • Baseline comparisons: FaST consistently outperforms GRPO-RM and RFT-RM while preserving or improving Helpfulness alongside higher TTS-friendliness.The comparison suggests feature-based reward signals better capture properties desirable for TTS-friendliness than traditional reward-model weights.
  • Baseline comparisons: Prompting alone provides gains over Zeroshot but underperforms FaST in most settings, particularly on CORA.Explicit alignment or fine-tuning, especially FaST and SFT, shows clearer benefits.
  • Low-data regime: In the 10-sample setting, FaST consistently outperforms SFT, DPO, GRPO-RM, and RFT-RM.The advantage is especially clear on Recipe, where SFT degrades in both dimensions while FaST remains comparatively stable.
  • Full-data regime: With 100 training samples, standard supervised fine-tuning becomes highly competitive and provides a strong TTS-friendliness–Helpfulness tradeoff.This pattern contrasts with FaST’s stronger advantage when preference data are limited.
  • Feature analysis: FaST’s feature analysis assigns negative weights to compact written conventions and positive weights to conversational tone and spelled-out numbers.Examples include use_of_numeric_formatting (−0.50), abbreviation_density (−0.32), natural_conversational_tone (+0.33), and use_of_spelled_out_numbers (+0.33).
  • Direct versus post-hoc generation: FaST achieves competitive TTS-friendliness in one inference step, whereas PolyNorm improves Zeroshot TTS-friendliness at the cost of doubled inference time.PolyNorm requires initial generation followed by rewriting; FaST avoids this two-step procedure.

7. Validating our Heuristic Metric as a Reliable TTS-friendliness Proxy

The paper validates its low-cost Heuristic TTS-friendliness metric against TTS→ASR errors and human listenability judgments. It closely tracks system rankings and correlates strongly with both automatic speech-recognition metrics and MUSHRA ratings.

  • Metric validation: The Heuristic metric is evaluated as a low-cost proxy for TTS→ASR metrics and human judgments.The evaluation compares outputs from representative systems across CORA and Recipe, supplemented by a MUSHRA-style listening study.
  • TTS→ASR validation: −1.00 and −0.90: system-level Spearman correlations show identical CORA rankings and only one DPO/Prompting swap on Recipe.CORA ranks Oracle > FaST > DPO > Prompting > Zeroshot; Recipe differs only by exchanging DPO and Prompting.
  • TTS→ASR validation: −0.71 to −0.80: utterance-level Heuristic–CER correlations are strong and negative across CORA and Recipe.The corresponding WER correlations are −0.68 on CORA and −0.71 on Recipe, with all reported p-values much smaller than 0.001.
  • Human validation: Human ratings confirm the automatic-metric pattern, with FaST preferred over both DPO and Prompting for spoken-content listenability.The listening test holds the TTS engine and voice constant and asks participants to judge how naturally responses are rendered when read aloud.
  • Human validation: 68.3: FaST receives the highest human MUSHRA rating, ahead of DPO at 55.9 and Prompting at 51.4.FaST exceeds DPO by +12.4 points (p=0.003) and Prompting by +16.9 points (p=0.001).

8. Conclusion

The paper frames TTS-friendly generation as LLM preference alignment and evaluates it with datasets, heuristic, TTS→ASR, and human-judgment methods. FaST achieves the best overall tradeoff between TTS-friendliness and helpfulness, while limitations include restricted coverage and architecture assumptions.

  • The work targets LLM outputs optimized for spoken delivery rather than written text alone.
  • The human listening study used 14 participants and included only three systems to limit cognitive load and fit an approximate £300 budget.
  • The study introduces CORA and Recipe plus an evaluation suite combining heuristic, TTS→ASR, and human-judge assessments.
  • FaST achieves the best overall tradeoff between TTS-friendliness and helpfulness, with particularly strong gains in low-data settings.
  • FaST tends to associate longer responses with TTS-friendliness, creating a potentially undesirable verbosity bias that can be adjusted through feature weights.
  • The experiments cover English and two domains, while generalization to larger models, other TTS systems, and end-to-end speech LLMs remains unresolved.

A. Details on MUSHRA Tests with Prolific

The adapted MUSHRA study evaluated how naturally and intelligibly different generated texts were spoken, using a fixed TTS voice and controlled participant protocol. Human ratings strongly correlated with the heuristic metric across systems and utterances.

  • Stimuli and protocol: The study synthesized 20 CORA questions with Kyutai Pocket TTS using one fixed voice and compared Oracle, Prompting, DPO, and FaST outputs.
  • Evaluation target: The protocol held the voice and TTS engine constant, so ratings targeted the spoken message rather than the synthesizer.
  • Quality control: The hidden reference served as an attention check, with submissions excluded when reference ratings fell below 80 on more than 20% of trials.
  • Stimuli and protocol: Participants rated each clip from 0 to 100 for naturalness and intelligibility after hearing the labeled and hidden Oracle reference.
  • Participants and quality control: The final sample contained 14 participants who passed automatic quality-control checks.
  • Correlation: ρ = +0.84 (p ≪ 0.001) across 80 utterance-system pairs linked heuristic scores with mean human MUSHRA ratings.

B. Dataset Examples

Table 7 presents one illustrative preference tuple from each dataset together with TTS-friendliness and helpfulness scores.

  • Table 7 pairs one context question, chosen response, and rejected response from each dataset with both evaluation scores.

C. Details on the Heuristic TTS-Friendliness Metric

The heuristic metric detects surface patterns that can hinder TTS, normalizes their frequency and coverage, and maps aggregate risk to a 1–5 friendliness score. Its design uses interpretable regex classes and calibrated weights.

  • Metric design: Each regex class contributes a weighted score based on match count and the fraction of text it covers.
  • Normalization: The normalization length is L = max(|t|, 20), preventing very short texts from being over-penalized.
  • Score mapping: The 1 + L/200 denominator limits risk accumulation in long answers, while the final mapping saturates at 5 when no risky pattern appears.
  • Calibration: Weights and normalization constants were selected by inspection on a held-out sample, without using the paper’s evaluation sets.

F.1. Generation Results with SmolLM3-3B

With SmolLM3-3B, FaST broadly matches the Qwen3-4B trends and remains among the strongest methods for balancing TTS-friendliness with helpfulness. Its performance is strong on CORA, while Recipe shows degradation in the 10-sample setting, and FaST tends to generate longer responses.

  • Generation results with SmolLM3-3B: FaST provides one of the strongest overall tradeoffs between TTS-friendliness and helpfulness across CORA and Recipe.The SmolLM3-3B trends are broadly aligned with those observed using Qwen3-4B.
  • Generation results with SmolLM3-3B: On CORA, FaST remains near the Pareto frontier in both the 10-sample and full-data settings.It achieves among the highest TTS-friendliness scores while preserving strong helpfulness.
  • Generation results with SmolLM3-3B: On Recipe, FaST remains highly competitive with full training but degrades when trained on 10 samples.The authors hypothesize that the random 10-sample subset may contain weak preference contrasts.
  • Generation results with SmolLM3-3B: FaST improves its TTS-friendliness score over SFT, suggesting its features help produce samples more appropriate for spoken delivery.This comparison isolates the improvement relative to supervised fine-tuning.
  • Generation results with SmolLM3-3B: FaST tends to generate longer responses than other approaches because its learned features associate conciseness and brevity negatively with TTS-friendliness.Response length is measured by average generated characters over validation and test responses.

G. Discovered Features

FaST discovers interpretable, dataset-specific features whose learned weights distinguish compact written conventions from more natural spoken delivery. These features explain both shared TTS-related patterns and domain-specific differences between CORA and Recipe.

  • Shared feature patterns: FaST assigns negative weights to compact written conventions and positive weights to features associated with natural spoken delivery.Examples include use_of_numeric_formatting at −0.50 on CORA, numeral_symbol_usage at −0.45 on Recipe, natural_conversational_tone at +0.33, and narrative_prose_style at +0.76 on Recipe.
  • CORA-specific features: On CORA, external_referral and technical_code_reference receive negative weights because URLs, email addresses, and identifiers challenge TTS systems.The learned weights are −0.24 and −0.18, respectively.
  • Response-length analysis: Table 10 measures average response length in characters across the combined validation and test sets.The length comparison contextualizes FaST’s tendency toward longer outputs.
  • Recipe-specific features: On Recipe, abbreviation_density and parenthetical_annotation dominate negatively, reflecting compressed shorthand such as 1 Tbsp. and 1/2 tsp.Their learned weights are −0.58 and −0.35, respectively.
  • Interpretation: FaST recovers domain-specific TTS-related patterns automatically without domain-specific engineering.The discovered feature sets and weights are reported separately for CORA and Recipe.
Loading 2609.01246v1…