What Matters for Latent Reasoning with Flow Matching

1 Introduction

• Explicit CoT costs grow with chain length, can spend tokens on prose that adds little reasoning, and commits to early tokens within a rollout.Source: §1 · p. 1
• FLaRe compresses CoTs into learned latent codes, refines them with a question-conditioned flow, trains the answer reader on generated thoughts, and self-trains on verified rollouts.Source: §1 · pp. 1–2, Figure 1 · p. 2
• FLaRe improves on prior latent methods across all five requirement probes, compares favorably on arithmetic benchmarks, and reaches 97% of explicit CoT accuracy at a quarter of its latency.Source: §1 · p. 2

2.1 Explicit and Latent Reasoning

• Explicit CoT is an autoregressive token sequence, whereas latent reasoning keeps the thought continuous and generates the answer from that latent variable.Source: §2.1 · pp. 2–3

2.2 Latent Reasoning with Flow Matching

• A VAE compresses explicit CoTs into latent codes, and a question-conditioned flow model denoises Gaussian noise into a thought used by the same model to generate an answer.Source: §2.2 · p. 3

2.3 Setup

• The default uses eight latent slots of dimension 512, trains on GSM8K-Aug, and evaluates both direct answers and answers extracted from decoded thoughts.Source: §2.3 · pp. 3–4

3.1.1 Reconstruction Alone Is Not Sufficient

• At d = 512, expanding from 8 to 32 slots raises reconstruction from 96.8% to 99.4% but lowers direct accuracy from 47.1% to 25.4% and decoded accuracy from 53.2% to 32.3%.Across 66 VAEs, reconstruction and decoded accuracy correlate at −0.12.Source: §3.1.1 · p. 5

3.1.2 Smoothness Is Necessary

• All three VAE corruptions improve direct accuracy; token substitution has the largest decoded-accuracy effect, despite lower reconstruction than no corruption.The corruptions shape each CoT into a smooth region, so nearby codes can still decode to it.Source: §3.1.2 · p. 5

3.1.3 The Target Must Be Information Dense

• Symbolic CoTs are more compressible: eight slots reconstruct nearly every held-out symbolic CoT exactly but fewer than half of natural-language CoTs.Encoding natural-language CoTs loses about 10 direct and 17 decoded points against the default; dual routes also let one frozen decoder render both forms.Source: §3.1.3 · pp. 5–6

3.1.4 A Stronger Decoder Improves Downstream Accuracy

• With the 1B encoder fixed, a fine-tuned 3B decoder gives the strongest result among the tested decoder sizes and training choices.The authors suggest a stronger decoder can recover details omitted by the encoder and handle approximate generated thoughts more robustly.Source: §3.1.4 · p. 6

3.2.1 The Flow Is Trained Mostly Near Noise

• Uniform time performs worst; flow learning works well only when training is strongly shifted toward noise, where inference starts.The chosen distribution is logit-normal, with t = sigmoid(s) and s ∼ N(−2, 0.82).Source: §3.2.1 · p. 6

3.2.2 The Answer Pass Must Read Imperfect Thoughts

• A 50/50 mixture of predicted endpoints and noised codes works best for the answer pass, but endpoints must stay detached from its loss.Training only on exact or noised VAE codes misreads generated thoughts; backpropagating through noisy one-step endpoints conflicts with the flow loss.Source: §3.2.2 · p. 7

3.2.3 Several CoTs per Question Help the Flow Model

• Several CoTs per question give the largest gain in both readings; same-style new questions help less, while different-style questions lower accuracy.Multiple target modes encourage the flow to cover valid solutions rather than memorize one code per question.Source: §3.2.3 · p. 7

3.2.4 End-to-End Training through the Full Rollout Is Required

• Stage 2 improves direct reading by 3.8 points and decoded reading by 1.3, with gains coming from answer loss backpropagated through the full rollout.It self-trains on verified thoughts generated from noise; removing verification, full-rollout loss, or multiple rollout steps loses most or all gains.Source: §3.2.4 · pp. 7–8

4.1 Useful Latent Thoughts

• On twin pairs, following rates are 40% (stage 1) and 37% (stage 2), versus 32% CODI, 28% PCCoT, and 7% Coconut.Among pairs each model answers correctly on both questions before injection, rates rise to 91% in stage 1 and 76% in stage 2.Source: §4.1 · p. 9

4.2 Diverse Latent Thoughts

• With 16 samples, decoded stage 1 gains 16.6 coverage points and 3.3 through voting; stage 2 gains 14.7 and 1.8, respectively.Baselines gain at most 5 coverage points, and majority voting never improves on their greedy answer.Source: §4.2 · p. 9

4.3 Explainable Latent Thoughts

• Decoded thoughts recover all reference intermediate results in order on about half the questions, more often than any baseline.On twin pairs with changed intermediate values, decoded thoughts usually recover the twin’s value, and answers follow that reasoning more often than for baselines.Source: §4.3 · p. 9

4.4 Refinable Latent Thoughts

• From 0.1 to 0.5× explicit-CoT compute, decoded accuracy gains 18.2 points in stage 1 and 24.2 in stage 2; direct gains at most 1.8.Baselines add at most 6.1 points over the same budget range, and PCCoT loses accuracy.Source: §4.4 · p. 9

4.5 Efficient Latent Thoughts

• At two Euler steps, stage 2 direct reading reaches 97% of explicit-CoT accuracy at 3.9× speedup.The comparison measures accuracy and time to answer on one GPU, one question at a time.Source: §4.5 · p. 10

• 6.9 and 2.6 points over KaVa on GSM8K at 0.5B and 1B, respectively; FLaRe is comparable at 3B and leads or ties on GSM8K-Hard at every scale.SVAMP is the main exception, likely because nearly half its questions add distractor numbers, which are rare in GSM8K-Aug.Source: §5 · p. 10
• 2.7 to 3.8 points on GSM8K: stage 2 improves every evaluated column at every scale.Source: §5 · p. 10
• 10.7 to 21.0 points on GSM8K: stage 1 gains over LaDiR despite using the same GSM8K-Aug and Diverse CoT rows.The passage attributes this gap to design choices rather than data.Source: §5 · p. 10

• Earlier latent methods generally use embeddings or hidden states, produce thoughts slot-by-slot or in fixed iterations, and often decode through the LM head at a training-fixed budget.The design axes include method family, thought representation and production, supervision, decoding, and inference-budget flexibility.Source: §6 · p. 10, Table 9 · p. 11
• Implicit-CoT methods move reasoning into hidden states, including fixed input slots or recurrent computation, rather than generating an intermediate text sequence.Source: §6 · pp. 10–11
• Filler tokens widen parallel computation before the answer without adding sequential feedback steps.Source: §6 · p. 11
• Continuous-feedback methods feed each step’s continuous representation back as the next input, retaining explicit CoT’s autoregressive step structure.Source: §6 · p. 11
• Looped methods reuse the same layers over fixed slots; PCCoT and KaVa instead loop over a fixed block of slots in parallel.Source: §6 · p. 11
• Most methods still train using explicit CoT, through curricula, stepwise internalization, or distillation.Source: §6 · p. 11
• Diffusion methods denoise a Gaussian-noise latent conditioned on the input into a reasoning state, then generate the answer autoregressively.The latent space can be induced by a VAE trained on explicit CoT.Source: §6 · p. 11

7 Conclusion

• FLaRe’s thoughts improve on prior latent methods across all five requirements, and it compares favorably on arithmetic benchmarks while reaching 97% of explicit CoT accuracy at a quarter of its latency.The paper reports controlled ablations across the pipeline and distills their findings into a recipe.Source: §7 · p. 11

A.1 Implementation Details

• Table 10 lists implementation details and hyperparameters for the VAE, both flow-model stages, inference, and evaluation.Appendix B provides the algorithms.Source: §A.1 · p. 12, Table 10 · p. 13

A.2.1 Answer Pass

• Each answer pass uses the question and, with probability 0.5, takes the model’s one-step endpoint rather than the noised code as its latent input.The endpoint is computed without gradient from the first flow draw using classifier-free guidance at scale 4; this requires two extra forward passes per row.Source: §A.2.1 · p. 12
• When the answer pass uses the cached code, it is pulled toward noise by a uniformly sampled amount up to 0.3, simulating imperfect inference thoughts.The corruption uses the first noise draw for that row.Source: §A.2.1 · p. 12
• The null-token replacement for classifier-free guidance applies only to the flow pass; the answer pass always sees the question.Source: §A.2.1 · p. 12

A.2.2 Noise Draws per Code

• With EMA weights, final accuracies remain within about one point across draw counts on both readings, while multiple draws mainly improve convergence speed.Eight draws cost about 1.6 times the runtime of four, so the authors use four draws.Source: §A.2.2 · p. 13

A.3 Flow Model, Stage 2

• Stage 2 proposes multiple thoughts, verifies decoded CoTs against reference answers, re-encodes accepted distinct CoTs, and trains on them with replay and rollout answer loss.Verification checks only the final answer, not intermediate reasoning steps.Source: §A.3 · pp. 14–15
• Stage 2 improves direct accuracy by 3.8 points and decoded accuracy by 1.3 over stage 1; these gains come from answer loss through the full rollout.Without answer loss, performance stays at stage 1 level; detaching the rollout endpoint loses 2.3 direct and 2.6 decoded points, while a one-step rollout loses 2.8 and 1.9 points.Source: §A.3 · p. 16

A.4 Variable-Length Latent Thoughts

• On GSM8K-Aug, the best variable-width model matches the fixed width of 8 at 60.1 decoded accuracy; eight slots already reconstruct 98.7% of test CoTs exactly.The authors expect variable widths to help on datasets with more varied CoT lengths, where width can be chosen as a thinking budget at inference.Source: §A.4 · p. 16

A.5 Baseline Numbers

• Baseline reproductions can differ by up to nine points, so the appendix combines original-paper results, authors’ runs, or evaluated released weights rather than copying one reproduction.For example, explicit CoT on Qwen2.5-0.5B-Instruct scores 50.6 in Kuzina et al. (2026) and 59.7 in the authors’ run.Source: §A.5 · p. 16

A.6 LaDiR Baseline

• At 20 steps, LaDiR needs about 130 sequential passes versus FLaRe’s 40 and is 3.2 to 4.8 times slower than FLaRe and 5.5 times slower than explicit CoT.FLaRe’s direct reading reaches 57.4 in 68 ms, 11 to 21 times faster than LaDiR at 10 to 20 steps per block.Source: §A.6 · pp. 17–18

B Algorithms

• The algorithms use a frozen VAE, flow-model training, verified-thought self-training, and inference that denoises noise before producing the answer and optional decoded CoTs.Stage 1 trains from encoded CoTs; Stage 2 trains on verified re-encoded targets and replay while backpropagating answer loss through a fresh rollout.Source: B Algorithms · pp. 18–19

C Datasets

• GSM8K-Aug contains 385K training rows, 500 validation rows, and 1,319 GSM8K test questions, with arithmetic CoTs represented as symbolic steps; GSM8K-Aug-NL writes those same steps as sentences.The VAE uses 367K training pairs meeting the stated answer and intermediate-value consistency checks.Source: C Datasets · p. 19, Table 13 · p. 19
• Evaluation uses 1,319 GSM8K test questions plus zero-shot tests on GSM8K-Hard, SVAMP, and MultiArith.The additional sets contain 1,319, 1,000, and 180 questions, respectively.Source: C Datasets · p. 19
• Diverse CoT adds 216K distinct verified solutions over 148K questions, while IID and OOD corpora add converted OMI-2 symbolic CoTs from GSM8K-like and MATH-like data.The IID and OOD question sets do not overlap any GSM8K-Aug split.Source: C Datasets · p. 19
• Adding more VAE training data helps reconstruction only to a very limited degree because reconstruction is already saturated on the default corpus.The comparison varies VAE data while holding flow data fixed.Source: C Datasets · pp. 19–20, Table 14 · p. 20
• The Stage 2 pool contains 100K OMI-2 questions with reference answers but no CoTs.It includes 73K augmented_gsm8k and 26K augmented_math questions.Source: C Datasets · p. 20

D Symbolic Conversion Pipeline

• The converter maps natural-language CoTs into a small-step symbolic language whose moves include equations, inequalities, declarations, definitions, and guarded cases.Names must come from the question or earlier declarations, and the final marker contains only the answer.Source: D Symbolic Conversion Pipeline · p. 20
• Qwen3.8-27B compiles numbered CoT sentences into symbolic steps or rejects the trace, while leaving the natural-language CoT unchanged.The conversion prompt includes the language rules, question, CoT sentences, and reference answer.Source: D Symbolic Conversion Pipeline · p. 20
• After filtering, blind auditing, review, deduplication, and a leakage check, the pipeline retains 629,824 pairs over 119,835 questions.The review accepts 90.5% of 700,131 distinct traces; the blind audit retains questions when the converter’s answer matches the label.Source: D Symbolic Conversion Pipeline · p. 20
• The authors leave execution-feedback training of the converter and inducing symbolic structure directly in latent space from task feedback for future work.The current conversion is described as a simple offline LLM-based step.Source: D Symbolic Conversion Pipeline · p. 20

E The Requirements of Latent Reasoning: Details

• The probes use a Llama-3.2-1B-Instruct model fine-tuned on GSM8K-Aug, generally evaluated on GSM8K test questions, with one fixed noise draw per question.Table 16 reports the shares of twin pairs whose answer follows the injected thought, keeps the original answer, or is neither; clean follow counts only pairs answered correctly both before and after injection.Source: E The Requirements of Latent… · p. 20, Table 16 · p. 21

E.1 Useful: The Twin Test

• The twin test swaps in a thought generated for a one-number question variant, then checks whether the original-question answer follows that thought.The diversity probe compares 16 flow noise draws with baseline thoughts perturbed at a calibrated noise scale; a committed error is repeated by every sample.Source: §E.1 · p. 21, §E.2 · p. 22
• On the twin pairs, stage 1 and stage 2 follow injected thoughts on 40% and 37% of all pairs, and 91% and 76% of clean pairs.Explicit CoT follows injected steps on 97% of pairs; CODI follows on 32%, PCCoT on 28%, and Coconut on 7%.Source: §E.1 · p. 21
• Replacing the thought reduces both stages’ accuracy, with the other-question thought lowering stage 1 to 5.9 and stage 2 to 21.6.For stage 1, mean, no, and other-question thoughts yield 47.4, 40.6, and 5.9; for stage 2, they yield 47.3, 39.3, and 21.6.Source: §E.1 · p. 22

E.2 Diverse: Sampling by Noise

• 77.8 pass@16 versus 61.2 greedy, 3.4 distinct answers per question, and only 6% of greedy errors repeated for stage 1; stage 2 retains most coverage.From one to 32 samples, stage 2 gains 17.5 points, compared with 3.5 to 9.0 for baselines and 10.8 for explicit CoT.Source: §E.2 · p. 22, Figure 6 · p. 22, Table 17 · p. 22

E.3 Explainable: Reading the Thoughts

• 46% and 49% of questions have reference values decoded in order by stage 1 and stage 2, versus 39% for CODI, 19% for Coconut, and 8% for PCCoT.The match requires exact intermediate values in order and counts a correct CoT taking another route as a miss; all models exceed the 3.8% No-CoT control and at most 0.3% chance level.Source: §E.3 · p. 23, Table 18 · p. 23
• On twin pairs, decoded FLaRe thoughts show the twin’s values on 81%, and show them while the answer follows them on 41% and 42% for stages 1 and 2.When FLaRe’s answer follows the twin, the decoded thought shows it in over 90% of cases; alignment remains partial on about 40% of pairs.Source: §E.3 · p. 23, Table 19 · p. 23

E.4 Refinable: Accuracy against Inference Budget

• Refinement tests run models below and above their training budgets, including 1 to 40 Euler steps for FLaRe, trained with 10 steps in stage 2.Baselines are evaluated with varied thought counts or Jacobi iterations, and a depth-recurrent model is evaluated with varied loop counts.Source: §E.4 · p. 23
• Stage 2 improves from one to twenty Euler steps, rising from 36.6 to 62.6 decoded accuracy and from 56.7 to 59.1 direct accuracy.It was trained on a 10-step rollout; neither stage loses more than a point at 40 steps.Source: §E.4 · p. 23, Figure 7 · p. 24

E.5 Efficient: Latency Measurement

• At two Euler steps, FLaRe’s direct reading reaches 57.4 accuracy in 68 ms, half CODI’s latency and a quarter of explicit CoT’s.At five steps, it is within a point of explicit CoT at half its latency; at twenty steps, it nearly matches explicit CoT at 1.7 times its latency.Source: §E.5 · p. 24, Figure 8 · p. 24
• One Euler step costs 22 ms, while one explicit CoT token costs 9.7 ms; decoded reading adds about half a second of decoding.Timing uses one question at a time on an RTX 3090 in bf16, with medians over 100 test questions.Source: §E.5 · pp. 24–25