PaperlayerView exact revision on arXiv

Source-linked AI summary

How to Steal Reasoning Without Reasoning Traces

Tingwei Zhang, John X. Morris, Vitaly Shmatikov

arXiv:2603.07267v2cs.CR

TL;DR

When reasoning-model APIs hide full traces, it remains unclear whether exposing only answers and brief summaries protects the model’s reasoning capabilities. Trace Inversion synthesizes traces from those observable outputs, and fine-tuning on them improves student performance across math, scientific reasoning, and coding benchmarks. The paper concludes that hiding chains of thought may not prevent capability theft, though true-trace fidelity for closed-source victims cannot be verified.

  • Problem

    Reasoning traces can support capability transfer, but exposing full chains of thought raises intellectual-property and sensitive-information concerns, motivating APIs that reveal only summaries and answers.

    Source: §1 · p. 1

  • Method

    Trace Inversion trains an inversion model to generate detailed traces from a black-box victim’s inputs, answers, and optionally exposed reasoning summaries.

    Source: §1 · p. 1, §4 · p. 3

  • Results

    Across math, scientific reasoning, and coding benchmarks, students fine-tuned on inverted traces outperform surrogate-trace or answer-and-summary baselines; for example, JEEBench reaches 31.6% versus 19.7% with R1-Distill traces.

    Source: §1 · pp. 1–2

  • Takeaways & Limitations

    Hiding or obfuscating exposed reasoning may reduce transparency without preventing users from stealing reasoning capability through trace inversion.

    Source: §7 · p. 9

    For closed-source victims, the authors cannot determine whether inverted traces accurately match the models’ true reasoning traces.

    Source: §7 · p. 9

Abstract

from arXiv · show

Many large language models (LLMs) use reasoning to generate responses but do not reveal their full reasoning traces (a.k.a. chains of thought), instead outputting only final answers and brief reasoning summaries. To demonstrate that hiding reasoning traces does not prevent users from "stealing" a model's reasoning capabilities, we introduce trace inversion models that, given only the inputs, answers, and (optionally) reasoning summaries exposed by a target model, generate detailed, synthetic reasoning traces. We show that (1) traces synthesized by trace inversion have high overlap with the ground-truth reasoning traces (when available), and (2) fine-tuning student models on inverted traces substantially improves their reasoning and enables distillation from proprietary, black-box LLMs.

1 Introduction

Reasoning traces can transfer capabilities beyond final answers alone, but exposing full chains of thought raises intellectual-property and information-leakage concerns. Trace Inversion reconstructs synthetic traces from a black-box model’s inputs, answers, and optionally summaries, and the paper reports both trace overlap and student gains from those traces.

1 Introduction

  • Step-by-step teacher traces can transfer reasoning capabilities to students and substantially outperform fine-tuning on final answers alone.The introduction frames full-trace exposure as an intellectual-property risk and notes that traces may also encode sensitive prompts, safety policies, or private context.Source: §1 · p. 1
  • Trace Inversion synthesizes detailed reasoning traces from a target model’s final answers and, optionally, compressed summaries, rather than requiring its hidden traces.The paper reports 81% token-length recovery and 52.76 token-overlap F1 for DeepSeek-R1 traces, despite training inversion on a much weaker surrogate.Source: §1 · p. 1, Figure 1 · p. 2
  • Fine-tuning on inverted traces reached 31.6% on JEEBench versus 19.7% with R1-Distill’s reasoning traces, and 52.4% on MATH500 versus 16.4% with GPT-5.4 mini reasoning summaries and answers.Source: §1 · p. 2
  • The authors release their code to support research on model-stealing attacks and defenses.Source: §1 · p. 2

2 Background and Related Work

This section situates Trace Inversion among work on reasoning models, output-based information leakage, and model distillation. It distinguishes inference-input and prompt inversion from capability stealing, which transfers model abilities under limited black-box access.

2 Background and Related Work

  • Chain-of-thought prompting and reasoning-oriented tuning elicit intermediate steps that can improve arithmetic, logical, symbolic, and other benchmark performance.The cited work also reports that reinforcement learning can bring smaller models closer to large base models’ reasoning performance.Source: §2 · p. 2
  • Black-box leakage research includes extracting memorized training data and inferring whether specific data appeared in training, including in-context settings.Source: §2 · p. 2
  • Model inversion infers information about inputs from black-box outputs, while prompt inversion targets secret portions of inputs such as system prompts.Source: §2 · pp. 2–3
  • Capability stealing transfers a teacher’s capabilities to a student under limited access; in black-box settings, attackers observe outputs without internal states, parameters, or training data.Prior work finds that reasoning-trace supervision can improve reasoning-task distillation, while this paper focuses on APIs that do not provide those traces.Source: §2 · p. 3

3 Threat Model

The threat model gives an attacker black-box access to a stronger victim whose internal trace is hidden and whose API returns an answer, optionally with a short summary. The attacker uses available auxiliary models and public reasoning data to synthesize traces for student fine-tuning.

3 Threat Model

  • The victim internally generates a trace leading to its answer, but exposes only the answer and optionally a much shorter summary through its API.Source: §3 · p. 3
  • The attacker seeks to improve a student by synthesizing traces consistent with observed input-output tuples, whether or not those traces match the victim’s true reasoning.The traces serve as supervision for fine-tuning the student.Source: §3 · p. 3
  • The attacker may use surrogate, compression, inversion, and student models, and must be able to fine-tune the inversion and student models.The surrogate supplies complete reasoning traces and is typically weaker than the victim.Source: §3 · p. 3
  • The attacker need not know the victim’s summary-compression method, instead using a prompted compression model on surrogate traces before training the inversion model.The inversion model then synthesizes traces from victim inputs and answers, plus summaries when available.Source: §3 · p. 3
  • Public reasoning datasets provide inputs for querying both the surrogate for traces and the victim for answers and summaries.Source: §3 · p. 3

4 Inversion Methodology

Trace Inversion is a three-stage attack with separate inversion models for summary and no-summary settings. It trains on surrogate traces, applies the inversion model to victim outputs, then distills the resulting traces into a student, optionally adding surrogate data.

4 Inversion Methodology

  • Trace Inversion uses three stages and separate inversion models depending on whether the victim exposes a reasoning summary.Figure 2 depicts the pipeline.Source: §4 · p. 3, Figure 2 · p. 4

4.1 Stage 1: Training the Inversion Model

  • Stage 1 trains on surrogate inputs, traces, and answers, optionally adding compressed summaries to emulate commercial APIs whose summarization method is unknown.The summary and no-summary training datasets differ in whether the compressed summary is included.Source: §4.1 · pp. 3–4
  • Stage 1 trains the summary-setting inversion model to predict the surrogate trace from the input, answer, and summary using token-level cross-entropy.The no-summary setting trains a separate model without the summary condition.Source: §4.1 · p. 4

4.2 Stage 2: Inverting Victim’s Outputs

  • Stage 2 queries the victim for inputs and outputs, then applies the trained inversion model to synthesize traces compatible with those observations.The victim tuple includes its summary when available.Source: §4.2 · p. 4

4.3 Stage 3: Student Distillation

  • Stage 3 fine-tunes the student on each input with the synthetic trace concatenated with its answer as the supervision target.The attacker may optionally augment this set with already-generated surrogate examples, either mixed in or used first as a curriculum.Source: §4.3 · p. 4

5 Evaluation

The evaluation tests trace fidelity and student reasoning across open-weight and black-box victims. Inverted traces improve student performance over accessible baselines, including when the victim is GPT-5.4 mini, though gains depend on the surrogate and task.

5.1 Experimental Setup

  • The setup uses disjoint 10k-example surrogate and victim splits, with R1 or R1-Weak as surrogates and Qwen and Llama as student models.Students are evaluated on MATH500, JEEBench, and LiveCodeBench; trace overlap metrics are auxiliary to downstream reasoning performance.Source: §5.1 · p. 5

5.2 Fidelity of Trace Inversion

  • Matching GPT-5.4 mini summaries on median length and dominant style features is the rationale for the surrogate compression prompt.The victim summaries have median length ≈592 tokens and median compression ≈4.2×; matching summary distributions supports inversion generalization.Source: §5.2 · p. 5, Table 1 · p. 5
  • With R1-Weak inversion, synthesized traces are 4,972 / 5,434 tokens long and reach TF1 52.76 / 49.01, versus zero-shot TF1 35.36 / 37.58.For R1 fidelity, the weaker-surrogate rows are a realistic attack setting; Table 2 reports trace length and overlap with ground truth.Source: §5.2 · p. 6, Table 2 · p. 6

5.3 Stealing Reasoning Capabilities of an Open-Weight Model (R1)

  • On MATH500, R1-Weak inverted traces raise Qwen from 63.0% with Summary+Answer to 71.8%; on JEEBench, they raise it from 24.0% to 36.3%.Synthesized-Trace beats attacker-accessible baselines in almost every experiment, including for Llama; Table 3 reports downstream accuracy when attacking R1.Source: §5.3 · p. 6, Table 3 · p. 7
  • Without summaries, Synthesized-Trace improves over Answer-only and Surrogate-Trace (Qwen JEEBench 19.7%→29.7%); with a strong R1 surrogate, Qwen reaches 74.1% MATH500 and 33.2% JEEBench, close to the oracle on both.Source: §5.3 · p. 6

5.4 Stealing Reasoning Capabilities of a Black-Box, Commercial Model (GPT-5.4 mini)

  • With R1-based inversion of GPT-5.4 mini outputs, Qwen reaches 76.0% on MATH500 and 43.7% on JEEBench, while Llama reaches 52.4% and 19.9%, respectively.These results exceed Answer-only baselines of 68.4% and 27.6% for Qwen, and 13.0% and 6.3% for Llama.Source: §5.4 · p. 7, Table 4 · p. 8
  • For GPT-5.4 mini, R1-Weak inversion beats Surrogate-Trace on Qwen and Llama JEEBench, while R1-based inversion is competitive but can fall below it.When victim and surrogate are comparable, direct surrogate traces can be as effective as or better than inversion.Source: §5.4 · pp. 7–8
  • From 5k to 15k victim queries, Qwen accuracy rises from 67.0% to 80.8% on MATH500 and from 38.8% to 44.5% on JEEBench.The reported 10k-query collection cost is $173.28; inversion and student fine-tuning require no further GPT-5.4 mini queries.Source: §5.4 · p. 8, Figure 3 · p. 8

6 Defenses

Trace inversion threatens defenses that rely on hiding or obfuscating reasoning because it can work from answers and summaries instead. Related defenses target exposed reasoning, while watermarking may aid attribution but its survival through inversion is unknown.

6 Defenses

  • Prior defenses include access control, query limits and truncation [37], and output perturbations such as undistillable teachers, modified logits, and adaptive noise [23] [6].Most cited perturbation methods apply to image-classifier internals, such as logits or intermediate representations.Source: §6 · pp. 8–9
  • Trace inversion can synthesize useful traces from answers and summaries without access to internal reasoning, unlike defenses that target exposed reasoning [33] [21].The framework is agnostic to how answers and summaries were obtained, as long as they are correct.Source: §6 · p. 9
  • Watermarking generated text may support post hoc attribution [18] [11] [32], but whether watermarks survive inversion and subsequent fine-tuning remains open [32].Table 5 is described as covering further improvements for Synthesized-Trace with a Llama student and GPT-5.4 mini victim.Source: §6 · p. 9, Table 5 · p. 9

7 Discussion and Future Research

The discussion concludes that hiding chains of thought may not protect reasoning capability from theft: answer-only or summary-based access can still support trace inversion. It identifies uncertainty about trace fidelity for closed-source victims and calls for stronger defenses and further scaling and verification work.

7 Discussion and Future Research

  • Hiding chains of thought may not protect reasoning capability: inverted traces can teach students more effectively than answers or surrogate traces.For closed-source victims, the authors cannot determine whether inverted traces match the models’ true reasoning.Source: §7 · p. 9
  • Obfuscating exposed reasoning can reduce user transparency without preventing capability stealing, since attackers can apply trace inversion to outputs alone.Source: §7 · p. 9
  • Future work could scale query budgets and inversion models, broaden reasoning tasks, train across summary styles, and verify traces before student fine-tuning.Suggested checks include self-consistency for reasoning, execution for code, and tool-based checking for math.Source: §7 · p. 9

Impact Statement

The impact statement frames the paper as an evaluation of whether hiding chains of thought prevents reasoning-capability theft, intended to motivate more robust defenses.

Impact Statement

A Benchmark Accuracy of Victim and Surrogate Models

The appendix identifies a benchmark-accuracy comparison for victims and surrogates, then provides prompts for compression and zero-shot trace inversion. The prompts describe how to expand summaries or input-answer pairs into detailed traces.

A Benchmark Accuracy of Victim and Surrogate Models

B Prompts for Compression and Zero-shot Trace Inversion

  • The compression prompt asks for a first-person inner-monologue recap of a long chain-of-thought trace.It specifies three to six short sections, a tentative present-tense voice, and roughly 600–900 tokens overall.Source: B Prompts for Compression and… · p. 13
  • With summaries, the inversion prompt supplies the problem, answer, and reasoning bubbles, then asks for a detailed trace that fleshes out the bubbles without adding new reasoning steps.Source: B Prompts for Compression and… · p. 14
  • Without summaries, the inversion prompt uses only the input and final output to reconstruct a plausible, detailed reasoning process.Its guidance calls for intermediate steps and alternatives while keeping the trace consistent with the input and output.Source: B Prompts for Compression and… · p. 15

C Examples of Synthesized Reasoning Traces

The examples compare zero-shot inversion with trace inversion using R1-Distill as the surrogate, and compare ground-truth traces with synthesized traces from that surrogate.

C Examples of Synthesized Reasoning Traces

  • Figure 4 contrasts zero-shot inversion with trace inversion using R1-Distill as the surrogate model.Source: Figure 4 · p. 16
  • Figure 5 compares a ground-truth reasoning trace with one synthesized using R1-Distill as the surrogate model.Source: Figure 5 · p. 17

D Synthesized Traces Can Be Better Than Ground-Truth Traces

The example compares R1’s ground-truth trace with a synthesized trace; the synthesized excerpt uses more direct reasoning than the ground-truth excerpt, which tries and abandons several approaches.

D Synthesized Traces Can Be Better Than Ground-Truth Traces

  • The synthesized excerpt uses more direct reasoning than R1’s ground-truth excerpt, which tries and abandons several approaches.Figure 6 shows the query above the two matching excerpts.Source: Figure 6 · p. 18
Loading 2603.07267v2…