Mask, Sample, Revise: A Revisable CTMC Inference Stack for Guided Discrete Flow Matching Text-to-Speech

I. INTRODUCTION

• Alignment-free infilling removes explicit duration modules, but conditional control can weaken at low sampling budgets and early errors may persist.Such errors can include deletions, substitutions, or speaker drift.Source: §I · p. 1
• DFM offers a CTMC framework suited to discrete codec-token speech, but necessary inference-time controls for stable conditional sampling remain unclear.Source: §I · p. 1
• SC-ReMask adds explicit token-to-mask CTMC transitions, making discrete infilling revisable during generation.The mechanism is adapted from masked discrete diffusion.Source: §I · p. 2
• The stack combines predictor-free guidance, prompt-matched conditional coupling, and schedule-constrained remasking in a single inference framework.The components require no post-hoc fine-tuning.Source: §I · pp. 1–2
• Controlled ablations identify inference-time control, rather than merely increasing sampling steps, as the main factor behind improved intelligibility in prompted low-NFE settings.Source: §I · p. 2

II. RELATED WORK

• Alignment-free infilling methods reduce explicit duration modeling, but robustness may depend on the inference procedure, especially at small sampling budgets.E2-TTS uses text-filler infilling; F5-TTS uses continuous acoustic features, while MaskGCT uses discrete token infilling.Source: §II · p. 2
• Discrete speech-token generation has also been studied with diffusion samplers and remasking, but not specifically guided, revisable CTMC inference for alignment-free codec-token TTS.Ku et al. study text-aligned speech-token reconstruction with discrete diffusion models.Source: §II · p. 2
• Prior DFM-TTS work emphasizes codec representations, RVQ hierarchy, probability paths, or solver correction rather than inference-time control.Examples include DiFlow-TTS [5], H-DFM, and GibbsTTS [7].Source: §II · p. 2
• The paper targets sampling-time control for mask-source codec-token generation, combining CTMC guidance, conditional coupling, and SC-ReMask.SC-ReMask adapts remasking as explicit CTMC transitions inside tau-leaping.Source: §II · p. 2

III. G-DFLOW-TTS: DFM FORMULATION AND REVISABLE INFERENCE STACK

• G-DFlow-TTS keeps the DFM-TTS architecture fixed and enhances its CTMC sampler with PFG, prompt-matched conditional coupling, and SC-ReMask.It uses a DiT backbone and text-filler conditioning without explicit duration predictors or external aligners.Source: §III · p. 2
• Training samples along a convex-mixture path from an all-mask source toward the ground-truth codec-token sequence and predicts the target tokens from the sampled sequence and text.The path uses a monotone schedule κ_t from 0 to 1, with a linear schedule in this work.Source: §III · pp. 2–3
• Conditional coupling copies a random-length prefix of the target into the source and masks the remaining positions before applying the same probability path.The prefix duration is sampled uniformly from 0.25 to 12.0 seconds, with at least 0.5 seconds left to generate.Source: §III · p. 3
• At inference, CTMC tau-leaping generates tokens in parallel only at masked suffix positions while keeping the prompt pinned.The sampler discretizes time into K steps of size Δt = 1/K.Source: §III · p. 3
• PFG geometrically mixes conditional and unconditional CTMC transition rates, with γ controlling guidance strength.Source: §III · p. 3

A. SC-ReMask

• SC-ReMask adapts ReMDM by implementing remasking as explicit token-to-mask CTMC transitions, making discrete infilling revisable.Source: A. SC-ReMask · p. 3
• A switch time can postpone remasking, while a capped and rescaled schedule sets its probability after the switch.Source: A. SC-ReMask · p. 3
• The per-step remasking probability is converted to a CTMC rate matched to the tau-leap jump probability, contributing to total hazard and jump decisions.Source: A. SC-ReMask · pp. 3–4
• Remasking rates apply only to generated suffix positions, excluding the acoustic prompt and already masked positions; remasking is skipped when no suffix tokens are eligible.Source: A. SC-ReMask · pp. 3–4

IV. METHODS

• Training uses the English portion of Emilia-YODAS, selected for reproducibility and artifact sharing under CC BY 4.0.The original Emilia split is CC BY-NC 4.0, which can complicate redistribution and reuse.Source: §IV · p. 4
• NeuCodec provides single-codebook acoustic tokens, avoiding the multistream prediction problem of RVQ-based codecs; text uses GPT-2 BPE.Source: §IV · p. 4
• Evaluation follows a voice-prompted protocol on a 2.2-hour LibriSpeech test-clean subset, pairing each target with a random same-speaker prompt.Systems share prompt selection and preprocessing.Source: §IV · p. 4
• The 232M-parameter model trains for 1M iterations, using text dropout on 10% of examples to provide conditional and unconditional predictions for PFG.Source: §IV · p. 4
• Intelligibility uses WER and CER, while speaker similarity and perceptual quality use SIM-o, UTMOS, and listener MOS ratings.The MOS study uses 19 listeners and 20 clips per system; per-utterance metrics use paired differences with bootstrap confidence intervals and sign-flip tests.Source: §IV · p. 4

PFG is necessary for conditional control at low NFE

• At low NFE, stronger predictor-free guidance improves intelligibility, but overly large γ can worsen WER; γ = 1.5 is selected for later experiments.Figure 2 grids guidance strength and sampling budget, with colors encoding WER on the prompted LibriSpeech test-clean subset.Source: §V · p. 4, Figure 2 · p. 4

Conditional coupling helps when paired with guided sampling

• At NFE=32, conditional coupling consistently improves intelligibility over U-coupling across γ, but coupling alone degrades performance without sampling-time control.The reported interpretation is complementary roles: conditional coupling matches the prompted training path, while PFG strengthens conditional CTMC rates during sampling.Source: §V · p. 4, Figure 3 · p. 5

SC-ReMask benefits from always-on schedule-constrained revision

• Among tested schedules, always-on remasking with tswitch = 0 and (ηrescale, ηcap) = (0.5, 0.5) gives the best low-NFE behavior.The dev-clean ablation reports CER against NFE while varying switch time, rescaling factor, and cap; the no-remasking baseline uses C-coupling with PFG.Source: §V · p. 5, Figure 4 · p. 6

SC-ReMask makes generation revisable and improves robustness

• WER 8.39% and CER 3.56% are the best reported results for G-DFlow-TTS, with SC-ReMask also improving SIM-o and UTMOS.The full configuration combines conditional coupling, predictor-free guidance, and SC-ReMask; reported NFE=32 gains over baseline are statistically significant.Source: §V · p. 5, Table I · p. 5, Table II · p. 5

Inference-time control matters more than additional sampling steps

• At 8 NFE, the full stack reaches 15.92% CER, outperforming the unguided baseline at 128 NFE.Increasing the unguided baseline from 32 to 128 NFE reduces CER only from 47.25% to 40.39%.Source: §V · pp. 5–6

• The Mask, Sample, Revise stack significantly improves intelligibility and robustness across objective metrics and human listening tests.SC-ReMask provides the largest gains by allowing early token decisions to be revised.Source: §VI · p. 6
• The system combines predictor-free guidance, prompt-matched conditional coupling, and SC-ReMask within a single tau-leaping sampler.Source: §VI · p. 6
• Table III compares quality-speed trade-offs across sampling budgets on LibriSpeech Test-Clean for the baseline, PFG + CC, and Full systems.Baseline uses U-coupling; PFG + CC adds PFG with C-coupling; Full also includes SC-ReMask.Source: Table III · p. 6

• ChatGPT (OpenAI, GPT-5.5 Thinking) was used only to polish portions of the manuscript.Source: GENERATIVE AI USE DISCLOSURE · p. 7
• The tools supported minor editing tasks, including improving grammar, clarity, and phrasing.Source: GENERATIVE AI USE DISCLOSURE · p. 7
• The authors say the tools were not used for scientific ideas, experiments, analysis, figures, experimental code, or major manuscript sections.Source: GENERATIVE AI USE DISCLOSURE · p. 7
• The authors reviewed and edited all text and take responsibility for the final content.Source: GENERATIVE AI USE DISCLOSURE · p. 7