Source-linked AI summary
Aha-Flow Distillation: Flow Markers Matter in LLM Reasoning
Xiaodong Wang, Peixi Peng
TL;DR
Reasoning traces can contain either revision-oriented Aha Moments or sustained Flow Moments, but how reasoning style affects post-training is less explored. The paper introduces Flow-CoT and Aha-Flow Distillation, pairing distinct privileged information with corresponding instructions. AFD improves Avg@12 over reproduced OPSD baselines on Qwen3-8B and Qwen3-4B.
Problem
The paper asks whether reasoning style used during post-training affects the resulting model, noting that Flow Markers have received less attention than Aha Markers.
Source: §1 · p. 2Method
AFD rewrites discourse markers into Flow-CoT while preserving reasoning trajectories, then pairs Flow-CoT and solution-based privileged information with corresponding instructions in dual-mode self-distillation.
Source: §1 · p. 2Results
AFD raises Avg@12 from 60.8 to 61.3 on Qwen3-8B and from 57.5 to 58.6 on Qwen3-4B over reproduced OPSD baselines.
Source: §1 · pp. 2–3Takeaways & Limitations
With the same 50/50 Flow-CoT and Aha-CoT mixture, dual-mode training improves Avg@12 from 59.5 to 60.1 on Qwen3-8B.
Source: §1 · p. 2
Abstract
from arXiv · showhide
We identify the Flow Moment, a reasoning pattern characterized by sustained, process-confirming verbalizations such as I'm doing, in contrast to the revision- and backtracking-oriented Aha Moment. We refer to their corresponding linguistic expressions as Flow Markers and Aha Markers, respectively. Based on this observation, we construct Flow-CoT by rewriting the discourse markers of original reasoning traces while preserving their underlying reasoning content, and use it as auxiliary supervision for on-policy self-distillation (OPSD). We further propose \textbf{Aha-Flow Distillation (AFD)}, a dual-mode extension of OPSD that pairs different forms of privileged information with corresponding reasoning instructions. The Aha branch retains concise solution-based supervision, while the Flow branch introduces rewritten Flow-CoT under a direct and confident reasoning instruction. At inference time, the model uses only the standard reflective instruction, so Flow-style reasoning serves purely as a training signal. Experiments on AIME25 and HMMT25 show consistent improvements across Qwen3-8B and Qwen3-4B: AFD improves Avg@12 from 60.8 to 61.3 on Qwen3-8B and from 57.5 to 58.6 on Qwen3-4B over our reproduced OPSD baselines. Controlled ablations further show that, with the same Flow-CoT/Aha-CoT composition, dual-mode training improves Avg@12 from 59.5 to 60.1, indicating that the benefit comes not only from introducing heterogeneous reasoning supervision, but also from how it is organized during self-distillation. The code is available at https://github.com/Wang-Xiaodong1899/Aha-Flow-Distillation.
1 Introduction
The paper introduces Flow-style reasoning as a complement to revision-oriented Aha reasoning and uses it as auxiliary supervision in Aha-Flow Distillation (AFD). AFD pairs distinct privileged information with matching reasoning instructions and improves Avg@12 over reproduced OPSD baselines on both model scales.
1 Introduction
- The Flow Moment verbalizes sustained progression along a reasoning trajectory, complementing the revision- and redirection-oriented Aha Moment.The paper calls their characteristic linguistic expressions Flow Markers and Aha Markers, respectively.Source: §1 · pp. 1–2
- AFD rewrites discourse markers into Flow-CoT while preserving reasoning content, then pairs it with direct reasoning instructions in a Flow branch alongside the solution-based Aha branch.Within each branch, student and teacher share the same reasoning instruction.Source: §1 · pp. 2–3
- Flow-CoT is auxiliary training supervision; at inference, the model uses only the standard reflective instruction.Source: §1 · p. 2
- AFD raises Avg@12 from 60.8 to 61.3 on Qwen3-8B and from 57.5 to 58.6 on Qwen3-4B over reproduced OPSD baselines.The paper reports the same trend across AIME25 and HMMT25.Source: §1 · pp. 2–3
2 Related Work
Related work discusses reflective Aha behavior and on-policy distillation, then distinguishes Flow-style verbalization as an alternative supervision signal. The paper’s focus is how reasoning markers and training modes shape self-distillation.
2.1 Reasoning Traces and the Aha Moment
- Prior work treats reasoning traces largely as sequences, while Aha Markers signal revision or redirection; Flow Markers instead signal trajectory continuation and motivate complementary Flow-CoT supervision.The paper says Flow-style verbalization has received less attention and, to its knowledge, has not been investigated as an auxiliary reasoning-distillation signal.Source: §2.1 · p. 3
2.2 On-policy distillation for reasoning
- On-policy distillation trains on student-generated trajectories; OPSD removes a separate teacher by conditioning its teacher pass on privileged reference information, and AFD organizes supervision into two instruction-matched branches.Figure 2 depicts randomly rewritten Flow-CoT examples and remaining Aha examples, with both branches optimized by token-level JSD.Source: §2.2 · pp. 3–4, Figure 2 · p. 4
3 Method
The method distinguishes revision-oriented Aha markers from progression-oriented Flow markers, then uses on-policy self-distillation to pair different privileged references with matching reasoning instructions. Flow-CoT changes discourse markers while preserving reasoning content, and Aha-Flow Distillation (AFD) combines it with solution-based supervision.
3.1 Aha Markers and Flow Markers
- Aha Markers signal reasoning discontinuities such as re-evaluation or strategy changes, whereas Flow Markers signal that the current path continues without redirection.The paper describes the corresponding Aha Moment as revision or redirection and the Flow Moment as sustained progression.Source: §3.1 · pp. 4–5
3.2 OPSD Preliminaries
- OPSD trains on student-sampled responses while conditioning the teacher on privileged reference information; the teacher and student share the generated prefix.The teacher sees the reference solution, unlike the student, so the form of privileged information affects the teacher distribution and resulting supervision; the objective uses JSD with per-token divergence clipping at δclip.Source: §3.2 · p. 5
3.3 Constructing Flow-CoTs
- Flow-CoT replaces Aha-style discourse expressions with first-person action statements while preserving the original reasoning trajectory, mathematical content, and step order.The rewrite also preserves dead ends, self-corrections, verification steps, paragraph structure, and the final answer.Source: §3.3 · p. 5
3.4 Aha-Flow Distillation
- AFD pairs concise solution references with standard step-by-step instructions in its Aha branch, and rewritten Flow-CoT with direct, confident instructions in its Flow branch.The same instruction is given to student and teacher, while the teacher additionally observes the privileged reference.Source: §3.4 · pp. 5–6
4 Experiments
Experiments compare AFD with OPSD on Qwen3-8B and Qwen3-4B using AIME25 and HMMT25, then ablate privileged information and training mode. AFD improves over reproduced OPSD at both scales, and controlled comparisons examine the effects of supervision composition and dual-mode organization.
4.1 Experimental Setup
- Evaluation uses Qwen3-8B and Qwen3-4B on AIME25 and HMMT25, reporting Avg@12 across 12 sampled completions per problem.Training follows the OPSD recipe with OpenThoughts-30K; Flow-CoT traces are rewritten from original Aha-style traces.Source: §4.1 · p. 6
4.2 Main Results
- 61.3 vs 60.8 Avg@12 on Qwen3-8B and 58.6 vs 57.5 on Qwen3-4B: AFD outperforms reproduced OPSD† across both benchmarks and model scales.The corresponding AIME25/HMMT25 scores are 73.6/48.9 vs 73.1/48.6 for Qwen3-8B, and 69.7/47.5 vs 69.2/45.8 for Qwen3-4B.Source: §4.2 · pp. 6–7, Table 1 · p. 7
4.3 Ablation Study
- 60.1 vs 59.5 Avg@12: AFD improves on standard OPSD with the same 50/50 Flow-CoT/Aha-CoT mixture on Qwen3-8B.The table holds other training and evaluation settings fixed; the comparison attributes the difference to pairing heterogeneous privileged information with corresponding reasoning instructions.Source: Table 2 · p. 7, §4.3 · pp. 7–8
4.2 Main Results
- All models use the same reflective instruction at evaluation, and the reported gains occur on both benchmarks for both model scales.The authors attribute the gains to the training procedure rather than additional test-time prompting.Source: §4.1 · p. 6, §4.2 · p. 7
4.3 Ablation Study
- 60.8 for solution-only OPSD exceeds 59.2 for Aha-CoT, while mixing Flow-CoT and Aha-CoT under standard OPSD gives 59.5 Avg@12.The comparison suggests concise solution supervision is generally more effective as privileged information than a lengthy CoT trace.Source: §4.3 · p. 7
- 61.3 Avg@12: Flow-CoT/Solution is the strongest configuration, exceeding Flow-CoT/Aha-CoT under AFD at 60.1 and solution-only OPSD at 60.8.Replacing Aha-CoT with the refined solution improves both benchmarks in this comparison.Source: §4.3 · p. 8
5 Conclusion
The paper identifies Flow Moments as a counterpart to correction-oriented Aha Moments and uses rewritten Flow-CoT as auxiliary supervision in AFD. Experiments report improvements over OPSD on two benchmarks while retaining standard reflective reasoning at inference.
5 Conclusion
- The Flow Moment signals sustained reasoning progression and complements the correction-oriented Aha Moment.The paper describes Aha Moments as correction-oriented and Flow Moments as signaling sustained progression.Source: §5 · p. 8
- Flow-CoT rewrites discourse markers while preserving the underlying reasoning trajectory, providing auxiliary supervision for on-policy self-distillation.AFD pairs different forms of privileged information with corresponding reasoning instructions.Source: §5 · p. 8
- AFD improves over OPSD on AIME25 and HMMT25 for Qwen3-8B and Qwen3-4B, while inference retains standard reflective reasoning.The paper concludes that the form and organization of privileged reasoning information are important design choices for self-distillation.Source: §5 · p. 8
A Flow-CoT Rewriting Prompt
The rewriting prompt changes how reasoning is verbalized, not its mathematical content or trajectory. It prescribes explicit first-person action markers while retaining the transcript’s original steps and formatting.
A Flow-CoT Rewriting Prompt
- The prompt rewrites discourse markers into explicit first-person action statements, typically using “I’m <verb>ing ...” phrasing.Examples convert fillers or transitions into statements such as “I’m checking the constraints” or “I’m trying another approach.”Source: A Flow-CoT Rewriting Prompt · p. 11
- The rewrite must preserve the reasoning trajectory, including dead ends, self-corrections, and verification, without adding, removing, merging, or reordering steps.The prompt also requires all mathematics to remain verbatim.Source: A Flow-CoT Rewriting Prompt · p. 11
- The rewritten transcript retains paragraph structure and the final-answer line, and the prompt requests the full transcript without commentary.The user template asks for the full rewritten transcript only.Source: A Flow-CoT Rewriting Prompt · p. 11