PaperlayerView exact revision on arXiv

Source-linked AI summary

The Distillation Game: Adaptive Attacks & Efficient Defenses

Youssef Allouah, Mahdi Haghifam, Sanmi Koyejo, Reza Shokri

arXiv:2605.22737v3cs.LGcs.AI

TL;DR

Distillation defenses must be evaluated against students that adaptively emphasize the most valuable released examples, not only passive students. This paper develops a minimax framework with adaptive evaluation and derives PoE, finding that adaptive evaluation reveals substantially more leakage, narrows ADS’s apparent advantage, and leaves PoE cheaper while preserving higher-quality reasoning traces.

  • Problem

    Existing evaluations often test defenses against passive distillers, leaving adaptive attacker behavior and a common framework for comparing defenses insufficiently addressed.

  • Method

    The paper formulates a minimax teacher–student game, derives adaptive evaluation and teacher-side defense responses, and instantiates a cheap proxy-based Product-of-Experts defense.

  • Results

    Across GSM8K and MATH, defenses that appear strong against passive students leak substantially more under adaptive evaluation, while PoE narrows ADS’s advantage, costs less to run, and preserves higher-quality reasoning traces.

  • Takeaways & Limitations

    Antidistillation progress should be judged against adaptive rather than passive students, with PoE offering a cheaper defense that retains higher-quality reasoning traces.

  • Takeaways & Limitations

    The framework focuses on adaptive reweighting and leaves richer attacks, broader model and task families, and stronger proxy choices for future work.

Abstract

from arXiv · show

Distillation attacks create a deployment trade-off for model providers: the same outputs that make a model more useful can also make it easier to imitate. We study this trade-off through a minimax game between a utility-constrained teacher and an adaptive student. Our framework yields tractable one-sided response rules: an adaptive evaluation rule in which the student reweights high-value examples, and a teacher-side defense template that suppresses outputs most useful for distillation. From a cheap proxy for example value, we derive Product-of-Experts (PoE), a simple forward-pass-only defense that combines the teacher with a proxy student during generation. Empirically, adaptive evaluation reveals a large passive--adaptive gap: on state-of-the-art defenses, adaptive students recover substantially more capability than passive evaluation suggests on GSM8K and MATH. Under this stronger evaluation, the apparent robustness gap between expensive defenses and PoE narrows considerably, while PoE remains substantially cheaper and preserves higher-quality reasoning traces. Overall, our results suggest that strong distillation remains difficult to stop, and that progress on antidistillation should be judged against adaptive students rather than passive ones. Our code is available at: https://github.com/ysfalh/distillation-game.

1 Introduction

Richer model outputs improve usefulness but expose reusable signals for distillation, creating a deployment trade-off that requires evaluating defenses against adaptive students. The paper frames this challenge as a minimax game yielding adaptive evaluation and teacher-side defense rules.

  • Motivation: Richer answers, reasoning traces, tool-use sequences, corrections, and feedback can be aggregated into training data for imitation attacks.The same output richness that improves usefulness, transparency, and interactivity can also make models easier to distill.
  • Motivation: Existing defenses are typically evaluated against distillers that train uniformly, risking a cat-and-mouse cycle with attackers that filter or reweight released data.A realistic attacker can concentrate on examples carrying the greatest learning value after observing defended outputs.
  • Motivation: Frontier providers already limit exposed reasoning information, but hiding full traces or replacing them with summaries does not resolve the underlying distillation problem.Public reporting characterizes distillation attacks as an operational concern rather than a hypothetical one.
  • Game formulation: The paper models distillation defense as a minimax game between a utility-constrained teacher and an adaptive student.The teacher selects a released model near a reference teacher, while the student selects an effective distribution near the released model but able to concentrate on valuable outputs.
  • Contributions: The framework yields explicit student best responses for adaptive evaluation and corresponding teacher-side defense templates once output value and fidelity or adaptation budgets are specified.These responses provide a common foundation for designing and evaluating defenses against adaptive students.

2 Problem Statement: Distillation Game

The distillation game models a teacher choosing a high-fidelity released distribution and an adaptive student choosing a nearby effective training distribution. A scalar value function measures output usefulness, yielding a minimax formulation with teacher fidelity and student adaptation budgets.

  • Setup and models: The setup distinguishes a reference teacher, a released teacher, and a student operating over context-output examples.Contexts x are drawn from D, outputs y belong to Y, and language models are conditional distributions over outputs.
  • Teacher fidelity: The teacher’s fidelity budget ε constrains the released model’s KL divergence from the reference teacher.This KL direction penalizes released models for placing probability mass on outputs considered unlikely by the reference teacher.
  • Student adaptation: The student’s adaptation budget ρ permits filtering, reweighting, or subsampling released outputs while restricting the effective training distribution’s deviation from the released teacher.When ρ = 0, the student is passive; larger ρ permits progressively more selective concentration on high-value outputs without inventing a new data source.
  • Example value: A scalar value function v(x, y) measures how useful each released example is for downstream distillation and is independent of any particular student update rule.A gradient-based first-order proxy illustrates how upweighting an example can affect a downstream objective, while later defenses use a cheaper log-likelihood-ratio proxy.
  • Minimax formulation: The minimax objective has the student maximize expected value within its adaptation budget and the teacher minimize that attainable value within its fidelity budget.The budgets (ε, ρ) define the threat model, while different choices of v produce different attack and defense rules.

3 Best Responses and Implications

Theorem 3.1 yields exponential-tilt best responses for both adaptive students and defending teachers. These responses motivate value-based reweighting for evaluation and the forward-pass-only Product-of-Experts defense.

  • Adaptive student evaluation: The student’s best response exponentially reweights released traces toward examples with higher value.The sharpness parameter η controls selectivity: η = 0 recovers passive training, while larger η concentrates mass on high-value traces.
  • Adaptive student evaluation: Adaptive evaluation trains on traces reweighted by value rather than on released traces uniformly.Algorithm 1 implements the reweighting as a stochastic minibatch procedure.
  • Teacher-side defenses: The teacher’s best response tilts away from outputs with high value to the student, with λ controlling defense strength.Choosing a value function therefore produces a corresponding teacher-side defense.
  • Teacher-side defenses: Gradient-based value recovers the principle behind ADS: suppress outputs whose student gradients are most aligned with reducing downstream risk.The framework provides a game-theoretic interpretation of the objective that ADS approximates under a first-order gradient-based value notion.
  • Product-of-Experts defense: PoE is forward-pass-only and uses token-level interpolation during decoding because exact sequence-level sampling is intractable.It requires no gradient estimates through the student and combines teacher and proxy-student log-probabilities at each decoding step.
  • Product-of-Experts defense: PoE combines teacher and proxy-student probabilities geometrically, downweighting outputs where the teacher has a large likelihood advantage.It arises by substituting a cheaper proxy value for gradient-based value into the teacher best-response rule.

4 Empirical Results

Adaptive evaluation exposes substantially more distillation leakage than passive evaluation, narrowing the apparent robustness advantage of ADS over PoE. Although both defenses improve the utility–distillability frontier, PoE also offers lower reported runtime overhead and higher-quality reasoning traces than ADS.

  • Adaptive evaluation: On GSM8K, ADS student accuracy rises from 34% under passive evaluation to 52% under adaptive evaluation, while PoE rises from 39% to 49%.On MATH, both ADS and PoE rise from 9% to 13% under adaptive evaluation, ignoring decimals.
  • Adaptive evaluation: Under adaptive evaluation, the GSM8K ADS–PoE student-accuracy gap narrows from 5 points under passive evaluation to 3 points.ADS reaches 34% versus PoE’s 39% passively, compared with 52% versus 49% adaptively.
  • Runtime and trace quality: On GSM8K, PoE increases teacher generation time by 64% (1.64×), compared with ADS’s 193% increase.Runtime changes the practical comparison beyond teacher and student accuracy alone.
  • Runtime and trace quality: PoE produces more high-scoring reasoning traces than ADS on both datasets and remains closer to the standard teacher under the Claude Sonnet 4.6 rubric.The rubric assesses whether reasoning steps are identifiable, relevant, and checkable on a 1–5 scale.
  • Runtime and trace quality: The trace-quality judge has 0.76 calibration and 0.40 rubric-point mean absolute error relative to human ratings, with mean signed error +0.13.These results indicate reasonable calibration without a large systematic offset.
  • Utility–distillability frontier: Both ADS and PoE improve the utility–distillability trade-off relative to the standard teacher, but their gains are much smaller against adaptive than passive evaluation.For ADS on GSM8K, sacrificing about 5 teacher-accuracy points reduces student accuracy by 23 points passively but only 5 points adaptively.

5 Conclusion and Future Work · Appendix

The paper frames distillation as a game between a utility-constrained teacher and adaptive student, yielding adaptive evaluation and the forward-pass-only PoE defense. Results show passive evaluation overestimates robustness, motivating adaptive assessment while highlighting richer attacks as future work.

  • 5 Conclusion and Future Work: The framework models interactions between a utility-constrained teacher and an adaptive student as a game-theoretic distillation problem.Specifying an example-value function yields both student-side adaptive evaluation and a teacher-side defense template.
  • 5 Conclusion and Future Work: A cheap proxy for example value produces Product-of-Experts (PoE), a simple defense combining the teacher with a proxy student during generation.PoE requires only forward passes.
  • 5 Conclusion and Future Work: Across GSM8K and MATH, defenses that appear strong against passive students leak substantially more capability under adaptive evaluation.The findings therefore favor adaptive rather than passive students for judging antidistillation progress.
  • 5 Conclusion and Future Work: Under adaptive evaluation, ADS’s apparent advantage narrows substantially relative to passive evaluation.This changes the practical comparison between defenses.
  • 5 Conclusion and Future Work: PoE is cheaper to run and preserves higher-quality reasoning traces under the stronger adaptive evaluation.The comparison concerns PoE and defenses whose robustness appears stronger under passive evaluation.
  • 5 Conclusion and Future Work: The framework focuses on adaptive reweighting because it captures realistic distillers’ ability to emphasize selected released examples.This focus keeps response rules tractable and evaluation protocols implementable while linking defense design and evaluation.
  • 5 Conclusion and Future Work: Natural extensions include richer adaptive attacks and broader model and task families.These directions extend the framework beyond its deliberately simple attacker-adaptation setting.

A Proof of Theorem 3.1

Theorem 3.1 characterizes the student’s and teacher’s best responses under finite-output assumptions as exponential tilts governed by dual variables. When KL constraints are active, the dual variables uniquely enforce tight expected budgets, with limiting policies defined as the tilt parameters diverge.

  • Best responses: Theorem 3.1 gives the student’s best response to a fixed released policy and the teacher’s best response to a fixed student.The theorem assumes that Y is finite.
  • KL constraints: When global KL constraints are active, η and λ are unique nonnegative constants chosen so the expected KL budgets are tight.The passage identifies η and λ as dual variables and states that the expected budgets bind.
  • Existence conditions: Finite Y makes each per-context policy a simplex vector, ensuring finite expectations and attainment of the optimization optima on compact feasible sets.The finite-space restriction avoids measurability and existence issues addressed by broader assumptions.
  • Exponential tilts: The student-side policy is an exponential tilt of πrel by ηv(x,y), while the teacher-side policy tilts πref by −λv(x,y).The displayed formulas normalize over y′∈Y for every y∈Y and x∈X.
  • Limiting policies: As η or λ tends to +∞, the corresponding policy is interpreted as an arbitrary policy supported on the limiting maximizer or minimizer set.These interpretations match the limiting policies of πeff,η and πrel,λ.

A.1 Student best response · A.2 Teacher best response

The student and teacher optimization problems each admit optimizers characterized by KL-regularized response families. Strong duality and KKT conditions establish finite parameter values when the corresponding KL constraint is active, while inactive constraints reduce to support on value-extremizing actions.

  • A.1 Student best response: Student best response optimizers have the form πeff,η for η ∈ [0, +∞], with finite η when the KL constraint is active.Existence follows from compactness of the feasible set and continuity of the objective.
  • A.1 Student best response: For finite η, the student solution is obtained by uniquely maximizing a penalized objective for each η ∈ (0, +∞).The constrained characterization uses strong duality because the feasible set has nonempty relative interior.
  • A.1 Student best response: With an inactive KL constraint, any student optimizer is supported on actions maximizing v(x, y) among those with πrel(y|x) > 0.This is the unconstrained linear-objective solution over policies absolutely continuous with respect to πrel.
  • A.2 Teacher best response: Teacher best response optimizers have the form πrel,λ for λ ∈ [0, +∞], with finite λ when the KL constraint is active.Existence follows from compactness and continuity, using the same argument as for the student problem.
  • A.2 Teacher best response: When the teacher KL constraint is active, strong duality and KKT conditions imply that every optimizer equals πrel,λ for some finite λ > 0.Complementary slackness supplies the corresponding constraint relation.
  • A.2 Teacher best response: With an inactive KL constraint, the teacher optimum reduces to minimizing the expected value functional over policies absolutely continuous with respect to πref.The resulting optimizer is supported on the minimizing actions permitted by πref.

A.3 Proof of the theorem … B.4 Existence and coupled form of an optimal pair

The theorem’s policies are identified with the student and teacher propositions, and the Stackelberg analysis reduces the game to variational optimizations over the relevant policies and η. In the finite-Z setting, an optimal teacher–student pair exists, with a coupled form when the outer infimum and both KL constraints are active.

  • A.3 Proof of the theorem: Theorem 3.1 follows directly from the student and teacher propositions, with η = ∞ and λ = ∞ interpreted through limiting policies.The displayed theorem formulas are exactly the definitions of π_eff,η and π_rel,λ.
  • B Stackelberg equilibrium of the distillation game: The Stackelberg analysis is conducted in the finite-Z setting, equivalent to working with the policies’ induced joint laws because D is fixed.Finite-dimensional compactness arguments rely on this finite-Z formulation.
  • B.1 Variational form of the student’s problem: The student’s variational optimizer has a finite η⋆ when the KL constraint is active, while an inactive constraint yields support on maximizers of v over supp(π_rel).The inactive case is also obtained as the appropriate limiting value.
  • B.2 Teacher optimization for fixed η: For fixed η, the teacher optimization admits an optimizer of the stated λ-parameterized form, with finite λ when its KL constraint is active.This follows by applying Proposition 2 to the bounded function g_η in the finite-Z setting.
  • B.3 Reduction of the game: The full game reduces to an optimization over π_rel and η, and for each fixed η the inner minimization over π_rel has the stated policy form.The reduction uses the equality of infima over Πε(π_ref) × (0, +∞) and the exponential-tilting characterization from Lemma B.2.
  • B.4 Existence and coupled form of an optimal pair: An optimal teacher–student pair exists in the finite-Z setting.Compact feasibility and continuity establish attainment of the relevant optimization.
  • B.4 Existence and coupled form of an optimal pair: When the outer infimum is attained at η⋆ > 0 and both KL constraints are active, the teacher optimizer and student best response take the stated coupled forms.The teacher form follows from Lemmas B.3 and B.2, while the student form follows from Lemma B.1 and constraint activity.

C Additional experiments and experimental details

This section supplies experimental details and reports additional experiments.

  • The section provides details of the experiments.
  • It also presents additional experiments.
  • Together, these contents expand the paper’s experimental material.

C.1 Experimental details

Experiments use GSM8K and MATH with fixed train/holdout/test splits, a DeepSeek-R1-Distill-Qwen-7B teacher, and mismatched proxy and final student architectures. Teacher generation, defense hyperparameters, distillation training, evaluation metrics, seeds, and hardware are specified consistently across runs.

  • Datasets and splits: Experiments evaluate GSM8K and MATH using fixed train/holdout/test splits, including 5,238/2,246/1,319 GSM8K prompts and 5,000/2,500/5,000 MATH examples.The GSM8K split is standard-training-derived; MATH pools standard subject-area subsets.
  • Teacher, proxy, and student models: The reference teacher is DeepSeek-R1-Distill-Qwen-7B, the internal proxy is Qwen2.5-3B, and the final distilled student is Llama-3.2-3B.The proxy/final-student architecture mismatch is intentional.
  • Teacher generation: Teacher traces use temperature 0.6, top-p 0.95, 512-token prompt truncation, 1,024-token generation caps, and a 32-token answer-forcing suffix.Representative tables use GSM8K ADS λ = 0.052 and PoE γ = 0.65, and MATH ADS λ = 0.08 and PoE γ = 0.75.
  • Defense sweeps: ADS and PoE are swept across separate λ and γ grids for GSM8K and MATH, with ranges spanning λ = 0.043–0.060 and γ = 0.50–0.75 on GSM8K.For MATH, ADS uses λ = 0.07–0.09 and PoE uses γ = 0.70–0.90.
  • Student distillation and reporting: Students share one supervised fine-tuning pipeline and differ only in trace weighting; training uses LoRA for 3 epochs with rank 128 and learning rate 5 × 10−4.The passive student uniformly weights released traces; experiments report teacher and downstream student accuracy under passive and adaptive evaluation, averaged over three seeds.

C.2 Distillation using frontier models

Commercial frontier-model traces provide effective supervision for a smaller open-weight student, producing dramatic gains over both the untuned base model and question–answer-only training. On GSM8K, fine-tuning Llama-3.2-3B reaches 51.75%, 53.57%, and 54.78%, versus 0.76% for the base model and 3.43% with Q&A-only supervision.

  • C.2 Distillation using frontier models: Commercial frontier-model traces are evaluated as supervision for a smaller open-weight student.The frontier models are GPT-5.4 mini, Claude Sonnet 4.6, and Gemini 3 Flash Preview.
  • C.2 Distillation using frontier models: 51.75%, 53.57%, and 54.78% are the reported GSM8K results after fine-tuning Llama-3.2-3B on frontier-model traces.These results correspond to the three commercial frontier models listed in the passage.
  • C.2 Distillation using frontier models: The GSM8K results improve over 0.76% in the base setting and 3.43% with Q&A-only supervision.The passage characterizes the gains over both comparison settings as dramatic.

C.3 LLM-as-a-Judge prompt for trace auditability

The paper audits generated reasoning traces with a blinded Claude Sonnet 4.6 judge using a 1–5 rubric for human-checkable reasoning. On 30 human-rated traces, the judge shows strong agreement with only a small upward rating offset.

  • Evaluation procedure: Claude Sonnet 4.6 independently rates each trace’s auditability from the math problem, final answer, and reasoning trace without seeing the defense type.The judge is instructed to evaluate traces strictly on auditability.
  • Auditability rubric: The rubric assesses whether each reasoning step is distinct, relevant, and checkable against the preceding step or problem statement.Verbose coherent traces and concise traces receive equal treatment when both are auditable.
  • Auditability rubric: Scores range from 1 for no auditable reasoning to 5 for fully auditable reasoning in which every step is identifiable, relevant, and independently checkable.Intermediate scores capture few, partial, or mostly auditable reasoning, while allowing traces to be long or short.
  • Human validation: κ = 0.76, mean absolute error 0.40, and mean signed error +0.13 on 30 human-rated traces indicate strong judge agreement with a small upward offset.The sample contains 10 traces each from Standard, PoE, and ADS conditions, and human raters use the same 1–5 rubric.

C.4 Comparison of the traces

PoE produces shorter, more polished reasoning traces than the unperturbed teacher while generally preserving correctness and logical structure. It can also change the reasoning route itself, whereas the strategic student favors clean traces over faulty ones.

  • Semantic comparison: The unperturbed teacher is the most verbose and self-reflective, while PoE compresses traces by removing hesitation, backtracking, and repetition.PoE usually preserves the full logical structure of the solution while making the trace more polished.
  • Style of reasoning comparison: PoE sometimes reaches the same correct answer through a different mathematically valid route rather than merely shortening the original trace.Examples include direct substitution in the functional-equation problem and compact directional relationships in the countries problem.
  • Example of weighting chosen by the strategic student: The strategic student assigns higher preference to clean traces than to faulty traces.Figure 7 illustrates this weighting pattern with two examples.

C.5 Adaptive student scores

Adaptive student scoring filters traces according to downstream gradient alignment, as illustrated in Figure 7.

  • Adaptive student scores: Adaptive student scores filter traces based on downstream gradient alignment.Figure 7 depicts this filtering rule.
  • Adaptive student scores: The student uses downstream gradient alignment to select which traces to retain.
  • Adaptive student scores: Figure 7 visualizes adaptive trace filtering by the student.
Loading 2605.22737v3…