Source-linked AI summary
ConvergeFlow: Language Flow with Provable Convergence to Token Embeddings
Na Li, Yuchen Jiao, Changxiao Cai, Gen Li
TL;DR
Existing continuous flow-based language models still rely on CE-supervised decoders because their trajectories are not guaranteed to reach valid token embeddings. ConvergeFlow uses an embedding-space predictor trained solely with flow-matching MSE, proves convergence to token embeddings, and achieves competitive performance while controlling the Gen. PPL–entropy trade-off.
Problem
Continuous flow-based language models lack guaranteed convergence to valid token embeddings, requiring CE-supervised decoders for direct token prediction.
Method
ConvergeFlow constrains its data predictor to vocabulary-embedding combinations and trains it solely with the MSE objective induced by flow matching.
Results
ConvergeFlow's flow converges to valid token embeddings despite data-predictor errors, and its MSE objective steadily reduces Gen. PPL while outperforming CE during training.
Takeaways & Limitations
The method enables direct token prediction without a separately trained CE decoder and provides sampling mechanisms for controlling generation quality and diversity.
Takeaways & Limitations
The FM objective can produce degenerate embedding-collapse solutions, so ConvergeFlow keeps a pretrained embedding matrix fixed during training and inference.
Abstract
from arXiv · showhide
Recent advances in continuous diffusion and flow-based language models (LMs) have achieved performance competitive with discrete LMs. However, existing continuous frameworks still rely on decoders supervised with cross entropy (CE) because the flow trajectories are not guaranteed to terminate at valid token embeddings. Motivated by this limitation, we introduce \textbf{ConvergeFlow}, an embedding-space flow-based LM, which constrains the data predictor to the convex hull of token embeddings and trains it solely with the mean squared error objective induced by flow matching. Under suitable regularity conditions, we prove that the resulting flow converges to valid token embeddings despite errors in the data predictor, enabling direct token prediction without a CE-supervised decoder. We further develop three sampling mechanisms for controlling the trade-off between the generative perplexity and entropy. Experiments on OpenWebText demonstrate that ConvergeFlow achieves performance competitive with existing continuous and discrete diffusion LMs. These findings demonstrate the potential of the flow-based paradigm for language modeling. Our code is available at https://github.com/Na-Li66/ConvergeFlow.
1 Introduction
Continuous and flow-based language models enable parallel, bidirectional refinement but still commonly require CE-supervised decoding because trajectories may end between token embeddings. ConvergeFlow addresses this gap with provable convergence, direct token prediction, and controllable quality–diversity trade-offs.
- Autoregressive language models limit bidirectional reasoning and sampling parallelism through fixed left-to-right, sequential generation.
- Diffusion and flow-based language models iteratively refine all token positions using bidirectional context, supporting parallel updates, global planning, and iterative revision.
- Existing continuous models may terminate between vocabulary embeddings, so they retain CE supervision to decode off-embedding terminal states into tokens.
- ConvergeFlow constrains its data predictor to vocabulary-embedding combinations and trains it with the MSE objective induced by flow matching.
- Under suitable regularity conditions, the learned flow converges to valid token embeddings despite predictor errors, enabling direct token prediction without a CE-supervised decoder.
- At dataset entropy 5.44, ConvergeFlow achieves Gen. PPL 33.17, compared with approximately 60 for the lowest continuous flow-based LM result.
2 Background
Flow matching learns a continuous transport from noise to data through a probability path and velocity field, with tractable conditional training objectives. The section also defines generative perplexity and entropy as complementary quality and diversity measures.
- 2.1 Flow matching and diffusion models: Flow matching transports samples from a source distribution, typically standard Gaussian noise, to the target data distribution.
- 2.1 Flow matching and diffusion models: The framework specifies an interpolating probability path between source and target distributions, then learns a time-dependent velocity field whose ODE realizes that path.
- 2.1 Flow matching and diffusion models: Conditional flow matching provides a tractable training objective because the posterior expectation of conditional velocities yields the marginal velocity generating the probability path.
- 2.1 Flow matching and diffusion models: The velocity can be parameterized through either a data predictor or a noise predictor, making flow matching equivalent to predicting clean data or noise.
- 2.1 Flow matching and diffusion models: Inference draws an initial sample from the source distribution and numerically solves the learned ODE, often using forward Euler updates.
- 2.2 Evaluation metrics for language modeling: Generative perplexity reflects reference-model likelihood and concentration, while entropy is evaluated alongside it to measure diversity.
- 2.2 Evaluation metrics for language modeling: Unigram entropy approximates normalized sequence entropy and provides a simple diagnostic of within-sequence diversity.
3 ConvergeFlow
ConvergeFlow constrains its data predictor to convex combinations of token embeddings and trains it with the flow-matching MSE objective. Under regularity conditions, this structure guarantees convergence to valid token embeddings, while parameter-free decoding and three sampling mechanisms support direct prediction and quality–diversity control.
- Training caveat: The FM objective alone permits degenerate embedding-collapse solutions, so ConvergeFlow fixes a pre-trained LangFlow embedding matrix during training and inference.Jointly learning the embedding matrix and predictor is therefore not sufficiently supervised by the FM objective alone.
- Framework: ConvergeFlow represents each predicted token embedding as a convex combination of vocabulary embeddings and trains the predictor with flow-matching MSE.The learned coefficients are not supervised or interpreted as token posteriors; the parameterization incorporates the known Gaussian corruption kernel.
- Convergence theory: Theorem 1 proves that, under positivity, Lipschitz, and time-grid assumptions, each token-level sampling state converges to a valid vocabulary embedding.The result holds despite errors in the learned data predictor and relies on the structured parameterization.
- Convergence theory: Nearest-neighbor decoding yields token predictions without a separately trained terminal decoder when vocabulary embeddings are distinct.The paper also reports accurate direct recovery of discrete tokens from terminal flow states.
- Convergence theory: An unconstrained predictor can be smooth and asymptotically accurate yet still induce a flow that fails to converge to any vocabulary embedding.This counterexample motivates imposing convex structure rather than relying on data-prediction accuracy alone.
- Sampling: Three sampling mechanisms explicitly control the generative-perplexity–entropy trade-off, with iterative self-conditioning refinement offering depth-based control and reduced sensitivity to solver-step count.Both token prediction rules are parameter-free and require no separately trained terminal decoder.
4 Experiments
Experiments on OpenWebText show that embedding-weighted prediction improves convergence to token embeddings, while MSE training improves Gen. PPL and sampling controls expose quality–diversity trade-offs.
- Empirical flow convergence: Embedding-weighted prediction separates the closest token embedding from the second-closest as SNR increases, unlike the unconstrained direct predictor.The closest distance approaches one and remains concentrated for the embedding-weighted predictor, while both distances increase with greater variation for the direct predictor.
- Empirical flow convergence: Weight-based and distance-based token prediction agree at 99.16%–99.82% of token positions as sampling steps increase from 32 to 512.Agreement improves with finer sampling discretization, supporting practical equivalence between the two prediction rules.
- Training objective: MSE training steadily reduces Gen. PPL and maintains a clear advantage over CE training, which provides no consistent improvement over the initial checkpoint.Both objectives maintain entropy above 5.5 throughout training.
- Training objective: Switching a CE-trained checkpoint to MSE consistently reduces Gen. PPL, whereas switching an MSE-trained checkpoint to CE degrades it.The crossover experiment attributes the improvement to the MSE objective rather than favorable initialization.
- Quality–diversity control: Time-adaptive sampling variants generally improve the Gen. PPL–entropy frontier relative to constant-strength variants.Self-conditioning guidance covers the broadest range and performs best at low and medium entropy, while iterative refinement is particularly effective at high entropy.
- Combined sampling techniques: Increasing iterative self-conditioning refinement generally shifts the frontier toward lower Gen. PPL at comparable entropy levels.Combining sufficiently large Kiscr with appropriate self-conditioning guidance substantially improves the trade-off.
- Further improvement: Unconditional guidance mainly moves operating points along a similar frontier rather than shifting it outward when combined with the other techniques.Self-conditioning guidance and iterative refinement capture most of the gain, while unconditional guidance remains useful without a conditioning signal.
- Token decoding: Nearest-neighbor projection supports direct mapping from generated continuous flow states to token sequences without a separately trained terminal decoder.Gen. PPL–entropy curves for the compared decoding rules nearly overlap, with distance-based decoding slightly better in the high-entropy, high-Gen. PPL regime.
5 Discussion
ConvergeFlow’s formulation exposes several limitations and open directions, including fixed embeddings, asymptotic theory, scale, evaluation scope, and acceleration.
- The formulation keeps token embeddings fixed to avoid degenerate solutions.
- Future theory should extend convergence analysis under weaker assumptions and provide non-asymptotic guarantees.
- Scaling to larger models and evaluating conditional generation, instruction following, and reasoning would clarify broader applicability.
- Distillation and higher-order acceleration are potential applications whose effectiveness for language generation remains unexplored.
A.1 Proof of Theorem 1
The proof constructs an auxiliary sequence whose updates match the original trajectory, then bounds density ratios to show concentration on vocabulary embeddings at the terminal time.
- The proof is organized into four steps, beginning with construction of an auxiliary sequence.
- Auxiliary weights replace learned base weights while preserving the embedding-weighted data predictor and its flow updates.
- Induction shows the auxiliary and original sampling trajectories coincide at every update.
- As t_N approaches 1, Gaussian variances vanish and component centers approach vocabulary embeddings, so the terminal distribution is supported on those embeddings.
- Density-ratio estimates compare reference and auxiliary distributions using change-of-variable formulas, bounded embeddings, and telescoping products.
A.3 Proof of Proposition 2
The proposition constructs a smooth unconstrained predictor that can be asymptotically accurate while its induced flow still fails to converge to a token embedding with constant probability.
- For schedules with α_t → 1 and σ_t → 0, the paper defines a smooth, unconstrained data predictor.
- The predictor is asymptotically accurate, but the induced flow can fail to converge to any token embedding with constant probability.
- Solving the ODE shows that the trajectory evolves with the data predictor remaining constant along the flow.
- Because the limiting Gaussian noise has a continuous distribution, the final iterate misses every finite set of token embeddings with constant probability.
A.4 Proof of Proposition 3
The proof uses normalized embeddings and probability weights to identify the limiting embedding associated with each token position.
- The argument normalizes each vocabulary embedding and fixes an arbitrary token position.
- Probability weights over embeddings are used to establish the limiting relationship for each token position.
- When the predictor converges to an embedding, the corresponding index is identified with the token index.
B Further experimental results
This section reports additional numerical analyses and compares LangFlow’s original sampling schedule with the proposed adaptive schedule across different sampling-step budgets.
- Figure 9 compares LangFlow’s original sampling schedule with the adaptive schedule t_i = (i + 0.5)/N across different sampling steps N.The adaptive schedule uses a uniform grid over the sampling interval.
B.1 Detailed results for quality-diversity control
The detailed results show consistent quality–diversity trade-offs across NFE budgets of 64 and 128, with time-adaptive variants generally retaining more entropy at comparable generative perplexity.
- Across NFE budgets of 64 and 128, increasing each control parameter lowers Gen. PPL at the cost of entropy.
- Time-adaptive variants generally retain more entropy at comparable Gen. PPL than standard variants.
- The consistent trade-off behavior supports concentrating sampling control near the relevant parts of the trajectory.
B.2 Detailed results for combinations of sampling techniques
Joint sweeps show that iterative self-conditioning refinement and guidance can improve the quality–diversity frontier, while unconditional guidance adds only marginal improvement after their combination.
- A sufficiently large K_iscr with appropriate w_scg shifts the trade-off frontier toward lower Gen. PPL at comparable entropy.
- Varying w_ug mainly moves the operating point along a similar frontier and provides only marginal additional improvement after combining the other techniques.
B.3 Detailed results for guidance allocation
The configuration results distinguish operating regimes across NFE budgets of 64 and 128: Configuration A favors low entropy and low Gen. PPL, B offers intermediate points, and C preserves the greatest entropy.
- Detailed sweeps: Tables 3 and 4 compare standard and time-adaptive variants under fixed NFE budgets of 64 and 128.
- Detailed sweeps: Tables 5 and 6 vary iterative self-conditioning refinement and self-conditioning guidance jointly under NFE budgets of 64 and 128.
- Detailed sweeps: Table 7 reports three-way combinations of unconditional guidance, iterative refinement, and self-conditioning guidance under NFE = 64.
- Configuration comparison: Configuration A reaches the low-entropy, low-Gen. PPL regime.
- Configuration comparison: Configuration B provides strong intermediate operating points, while Configuration C preserves the greatest entropy.
- Refinement allocation: In overlapping regions, a larger K_iscr generally yields a more favorable trade-off.
- Configuration definitions: Configuration A uses a constant refinement count K and self-conditioning guidance strength w_scg.
- Configuration definitions: Configuration B scales both the refinement count and self-conditioning guidance strength by 1/(1 + ...).
B.4 Qualitative Samples
The section presents qualitative generation samples from ConvergeFlow at NFE=64, using fixed sequence lengths of 1024 tokens. Reported samples have Gen. PPL values of 33.09 or 33.17, with entropy 5.44.
- NFE=64 qualitative samples are generated with a fixed sequence length of 1024 tokens.
- Gen. PPL: 33.09 and Entropy: 5.44 are reported for one sample configuration.