TraceForge: Distilling with Your Own Thinking

Yan Wang

Source-linked AI summary

TraceForge: Distilling with Your Own Thinking

Yan Wang

arXiv:2610.29680v1Oct 21, 2026cs.AIcs.LG

TL;DR

Frontier models may withhold detailed reasoning traces, while recovering them by replaying encrypted blocks violates providers’ terms. TraceForge generates traces in the student family’s style from context and a teacher’s answer, then trains the generator adversarially. On coding benchmarks, family-native traces outperform Fable 5’s reasoning; results on general-knowledge tasks are about the same.

Problem
Some frontier models provide only an answer or short reasoning summary, and replaying encrypted reasoning blocks to recover traces violates providers’ terms.
Source: §1 · p. 2
Method
The method trains a generator on the student family’s reasoning traces and uses a critic to refine traces generated from a teacher’s context and answer.
Source: §1 · p. 2, §2.2 · p. 3
Results
3.7 points on HumanEval+ and 7.1 on MBPP+ are the final-checkpoint gains over Fable 5’s reasoning, while results on three general-knowledge tasks are about the same.
Source: §1 · p. 2, §3.2 · p. 5
Takeaways & Limitations
The authors attribute the coding gains to family-style traces being easier for the student to absorb, with lower adaptation costs and less catastrophic forgetting.
Source: §1 · p. 2
The critic must be at least as large as the generator because a smaller critic fitted shortcuts and provided no useful learning signal.
Source: §2.3 · p. 3

Abstract

from arXiv · show

Using an LLM to generate training data for another is among the most general ways to train one, and it remains common practice. Some frontier teams have closed that route for their own models: what comes back is an answer, or a short summary, without the detailed reasoning trace behind it. That is a little ironic: they were themselves built on distilling human knowledge, and distilling an open-source model is still entirely routine; the restriction applies to their own outputs alone. Recent work recovers the hidden reasoning by replaying encrypted blocks across models. It somehow works, but it breaks the providers' terms of service, so it is not something a product can be built on long-term. We built the alternative. We take the context and the teacher's answer as input, and generate the reasoning trace ourselves, in the style of the student's own model family. Training is two-stage, supervised and then adversarial: a generator learns to regenerate the family's own reasoning traces from their context and answer, and a critic learns how real a reasoning trace looks. Applied to a teacher's answer, the generator writes one in the family's style. An independent judge rates how likely traces are to be genuine on a scale from 0 to 1: ours average 0.68, real traces 0.66, and traces selected by the critic from four candidates 0.82. These are authenticity scores, not correctness rates. We test the idea on five benchmarks with everything else held fixed. On general tasks the five training corpora are about the same; on coding, the generated reasoning beats Fable 5's own thinking content. With the same 5,000 questions and the same teacher answers, HumanEval+ improves by 3.7 points and MBPP+ by 7.1 at the final checkpoint, and the MBPP+ lead holds at every checkpoint. We suspect that reasoning written in the student's family style is easier for it to learn from.

1 Introduction

The paper addresses restricted access to frontier models’ reasoning traces and proposes generating missing traces in the student family’s style instead. Its coding results favor family-native traces over Fable 5’s reasoning, while general-task results are about the same.

Motivation

Frontier models may return only answers or short reasoning summaries, while replaying encrypted reasoning blocks can recover traces but violates providers’ terms.The paper frames lawful access to useful training traces as the gap its approach addresses.Source: §1 · p. 2

Approach

The proposed alternative generates missing reasoning from context and the teacher’s answer in the student model family’s style.A critic helps make generated traces resemble real traces from that family.Source: §1 · p. 2

Results

3.7 points on HumanEval+ and 7.1 on MBPP+ are the final-checkpoint gains over Fable 5’s reasoning, while general-knowledge results are about the same.The MBPP+ lead holds at every measured checkpoint.Source: §1 · p. 2

Interpretation

The authors attribute the coding advantage to family-style reasoning being easier for the student to absorb, with lower adaptation costs and less catastrophic forgetting.They contrast this with capability delivered in a foreign model’s style.Source: §1 · p. 2

Contribution

The paper presents legal trace generation as a way to distill another model’s output while staying within the line it identifies as legally important.It characterizes training on another model’s output as an established technique.Source: §1 · p. 2

2 Method

The method learns the student family’s reasoning-trace distribution from its own conversations, then applies it to a closed-source teacher’s context and answer. A critic distinguishes real from generated traces, and adversarial co-training improves the generator’s judged authenticity score.

2.1 Problem

Training uses conversations from the student’s family with reasoning traces removed from the input, while closed-source teacher context and answers supply inference inputs.The generator models the reasoning trace given the system prompt, user turns, answer, and tool exchanges; the teacher enters only at inference.Source: §2.1 · pp. 2–3

2.2 Supervised training of the generator

The Qwen-family generator is fine-tuned on family-written traces, using the answer as input and the reasoning trace alone as its target.This trains a derivation ending at the answer rather than a restatement, and the generated trace for a teacher’s answer is not copied from a seen trace.Source: §2.2 · p. 3

2.3 Supervised training of the critic

The critic compares real traces with matched generated negatives, and training stops when the positive’s probability is at least 0.2 higher.A smaller critic fitted shortcuts and gave the generator nothing to learn from, so the critic must be at least as large as the generator.Source: §2.3 · p. 3

2.4 Adversarial co-training

In each online round, the generator produces 5,000 traces; traces that fool the critic become critic-training data, while those that do not become generator preference pairs.The process is run for five rounds, with a held-out set rescored after each round.Source: §2.4 · p. 4
After five co-training rounds, the generator alone scores 0.68, above 0.66 for real reasoning traces; selecting the best of four candidates with the critic scores 0.82.The score stops improving after five rounds.Source: §2.4 · p. 4

3 Evaluation

The evaluation holds the student and training setup fixed while comparing five data corpora. Family-native reasoning improves coding results over Fable 5’s own traces, while the further analysis shows distinct training trajectories across corpora.

3.1 Setup

Students share the Qwen3.5-9B-Base initialization and use the same 5,000 prompts and Fable 5 answers; only the data source varies across corpora.Evaluation covers three non-reasoning knowledge tasks and two coding benchmarks.Source: §3.1 · p. 4

3.2 Main comparison

Replacing Fable 5’s reasoning with family-native traces adds 3.7 points on HumanEval+ and 7.1 on MBPP+, with the MBPP+ lead holding at every checkpoint.Table 2 reports the checkpoint results for both coding benchmarks.Source: §3.2 · p. 5, Table 2 · p. 6
Qwen-Max traces trail Fable 5 by 18 points on HumanEval+ and 28 on MBPP+, showing teacher capability is not sufficient for transfer.The comparison uses Qwen-Max’s own reasoning traces and answers.Source: §3.1 · p. 4, §3.2 · p. 5
TraceInversion finishes weakest at 3.66 and 6.61; its traces use generic narration and numbered steps that the family does not write.The authors report that these traces often restate answers rather than derive them.Source: §3.1 · p. 4, §3.2 · p. 5
With GPT-6.1-SOL answers fixed, family-native traces score 59.15 and 60.05 versus TraceInversion’s 3.66 and 6.61.Compared with S2, S5 is 9 points behind on HumanEval+ and level on MBPP+.Source: §3.1 · p. 4, §3.2 · p. 5

3.3 Further analysis

S1 falls from 66.46 on HumanEval+ at step 600 to 33.54 at step 1,200, then recovers to 64.63 by the end.The authors interpret the decline and recovery as a possible overshoot from fitting a reasoning style distant from the family’s own.Source: §3.3 · p. 5
S5 rises at every checkpoint on both benchmarks, while S3 and S4 show peaks followed by declines and increasingly frequent dead loops or repetition.The authors report that family-native traces do not produce the dead-loop pattern; S1 also does not show it.Source: §3.3 · pp. 5–6
The authors conclude that family-native traces outperform real traces in the comparison holding questions and answers fixed.The two tables together support this conclusion.Source: §3.3 · p. 6

4 Discussion and Conclusion

Family-native traces outperform the teacher’s own reasoning traces in the reported comparisons. A variant that avoids backpropagating Claude or OpenAI tokens performs essentially the same as S2.

With questions and answers held fixed, family-native traces edge the teacher’s by 3.7 and 7.1 points, while beating Qwen-Max’s own traces and answers by 21 and 35 points.Source: §4 · p. 6
The pipeline uses Qwen3.8-Max examples under terms that permit this use, and neither the generator nor critic has seen a Claude or OpenAI token.The authors state that the pipeline therefore violates no provider agreement.Source: §4 · p. 6
A risk-averse variant generates a family-native trace, then has the strongest legally distillable model in that family write the answer; it performs essentially the same as S2.No Claude or OpenAI token is backpropagated in this variant.Source: §4 · pp. 6–7

5 Related Work

Related work studies reasoning supervision, hidden-trace recovery and synthesis, teacher–student distribution differences, verification, and adversarial training. TraceForge focuses on synthesizing reasoning in the student family’s style while retaining a teacher’s answer.

Learning from reasoning traces

Prior work uses teacher-generated rationales or explanations and curated reasoning data to supervise smaller models; this work asks whether a student-family-style trace teaches better when the teacher’s answer is fixed.Distilling Step-by-Step [4], Orca, and DeepSeek-R1 motivate training on intermediate reasoning.Source: §5 · p. 7

Recovering and synthesizing hidden reasoning

TraceForge trains a generator on the student family’s traces and uses a family-trace critic, while its comparisons show downstream differences but do not isolate family style from every pipeline difference.Prior approaches include encrypted-block recovery, Trace Inversion, and REER [6].Source: §5 · p. 7

Teacher–student distribution differences

TraceForge retains a strong teacher’s answer while synthesizing reasoning in a family-native form, whereas S3 is a distinct baseline rather than a fixed-answer trace-style ablation.MiniLLM changes the distillation objective using reverse KL.Source: §5 · p. 7

Selection, verification and rewriting

Unlike correctness-based selection or step-level verification, the family-trace critic distinguishes genuine from generated traces and enables best-of-four selection without step-level correctness labels.The critic and independent judge score resemblance to family traces, while benchmark performance is evaluated separately.Source: §5 · p. 7

Adversarial training and the reconstruction target

The critic constructs online preference data for the generator, which reconstructs a reasoning trace from conversation and answer rather than the teacher’s hidden computation.The paper’s goal is a useful family-native training trace, not verbatim reconstruction of hidden reasoning.Source: §5 · p. 7