X-Pred MeanFlow for Streaming Token-to-Mel Speech Decoding

1. Introduction

• Discrete speech representations separate high-level content modeling from acoustic reconstruction, leaving the downstream decoder to recover speaker characteristics, prosody, and fine spectral structure.That decoder also affects perceptual quality, output-packet computation, and incremental generation.Source: §1 · p. 1
• Flow matching offers high-quality generation, but numerical integration requires multiple neural function evaluations, and preserving acoustic detail under extremely few-step token-to-mel generation remains challenging.MeanFlow models average velocity over a time interval to reduce the sampling budget.Source: §1 · p. 1
• A streaming decoder must coordinate sampling and context budgets while balancing acoustic quality, bounded per-packet computation, smooth chunk transitions, look-ahead delay, and the real-time cost of extra steps.Shorter chunks reduce waiting time but provide less context; additional look-ahead can help locally while increasing startup delay.Source: §1 · pp. 1–2
• X-Pred MeanFlow improves few-step token-to-mel quality over Direct-u MeanFlow at Small and Base scales, while layer-selective bounded context supports stable streaming within the real-time budget.The design also provides a favorable quality-efficiency trade-off for packet-level generation.Source: §1 · p. 2

2.1. Conditional Flow Matching

• Conditional flow matching trains a neural vector field to transport samples from a prior distribution toward the data distribution along an interpolation path.The conditional velocity is the path derivative, and inference solves an ODE to recover the data sample from noise, typically using multiple function evaluations.Source: §2.1 · p. 2

2.2. MeanFlow

• MeanFlow models average velocity over a time interval and uses a derived target for training, enabling one-NFE synthesis with endpoint recovery.Its training target substitutes conditional velocity for marginal velocity and includes a total-derivative term; at t = r, the objective reduces to standard CFM loss.Source: §2.2 · p. 2

3.1. Overview

• The decoder maps discrete speech tokens to target mel spectrograms with a DiT backbone; the upstream token generator and downstream vocoder are fixed and excluded from optimization.The framework’s two components respectively control the sampling budget and context budget.Source: §3.1 · p. 2

3.2. X-Pred MeanFlow

• X-Pred MeanFlow predicts a generalized mel field and analytically derives the corresponding average velocity, keeping MeanFlow dynamics while making the network output a mel-space prediction.The generalized field is interval-conditioned and is the clean mel endpoint when r = 0.Source: §3.2 · p. 3
• At each sampling step, the decoder predicts the generalized mel field and converts it into a MeanFlow update; with 1 NFE, its output is the predicted clean mel endpoint.Inference starts from noise and proceeds through a descending sequence of time points.Source: §3.2 · p. 3

3.3. Layer-Selective Block-Masked Attention

• Layer-selective block masks assign past- and future-block budgets across DiT layers, allowing selected layers to exchange context across block boundaries.Packet inference uses a fixed sliding window, so frames outside it are excluded and context, memory, and per-packet computation have a strict upper bound.Source: §3.3 · pp. 3–4

4.1. Experimental Setup

• Direct-u and X-Pred share each scale’s backbone, data, block-mask configuration, optimizer, and 2.0M-update training budget.Evaluation uses 100 sentences from unseen speakers and reports UTMOS, WER, and SIM.Source: §4.1 · p. 4

4.2. Few-Step Acoustic Quality

• At 2–3 NFEs, X-Pred consistently outperforms Direct-u on all reported metrics at both scales; Small X-Pred exceeds Base Direct-u in overall speech quality.The evaluation reports UTMOS, WER, and SIM; 10 NFEs serves as a quality reference.Source: §4.2 · p. 4, Table 1 · p. 4, §4.1 · p. 4

4.3. Streaming Quality and Runtime

• Independent Local decoding has the lowest computational overhead but degrades acoustic quality and boundary continuity; Uniform bounded masks recover quality but add processing latency.Table 2 compares quality and runtime across context schedules; Full is the non-streaming B-CMOS reference.Source: §4.3 · p. 4, Table 2 · p. 4
• LS-Bounded improves speech quality and boundary smoothness over Uniform, reduces real-time factor and estimated startup latency, and has zero deadline misses at 2 and 3 NFEs.Runtime evaluations over hundreds of seconds of audio report stable packet latencies.Source: §4.3 · p. 4, Table 2 · p. 4

• X-Pred predicts a generalized mel field and analytically derives average velocity, preserving MeanFlow sampling with a direct acoustic prediction target.Source: §5 · p. 4
• Layer-selective block-masked attention supplies bounded context for streaming inference.Source: §5 · p. 4
• At both Small and Base scales, X-Pred improves few-step UTMOS, SIM, and WER over Direct-u MeanFlow.Source: §5 · p. 4
• Small X-Pred exceeds Base Direct-u in low-NFE naturalness while maintaining comparable speaker similarity and intelligibility.Source: §5 · p. 4
• Layer-selective bounded context provides the best evaluated quality-efficiency trade-off and stable real-time decoding at 2 and 3 NFEs.Source: §5 · p. 4