PaperlayerView exact revision on arXiv

Source-linked AI summary

Embarrassingly Simple Self-Distillation Improves Code Generation

Ruixiang Zhang, Richard He Bai, Huangjie Zheng, Navdeep Jaitly, Ronan Collobert, Yizhe Zhang

arXiv:2604.01193v2cs.CL

TL;DR

The paper asks whether an LLM can improve code generation from its own raw outputs without a teacher, verifier, or reinforcement learning. It introduces simple self-distillation, which fine-tunes on temperature- and truncation-controlled samples, and reports broad gains, including a rise from 42.4% to 55.3% pass@1 for Qwen3-30B-Instruct on LiveCodeBench v6.

  • Problem

    High-quality coding supervision is expensive, while synthetic pipelines often require stronger teachers or execution-based verification.

  • Method

    SSD samples solutions from the base model with specified temperature and truncation, then fine-tunes on those raw, unverified outputs using standard supervised learning.

  • Results

    42.4% to 55.3% pass@1 on LiveCodeBench v6 for Qwen3-30B-Instruct, with gains concentrating on harder problems and improvements generalizing across six models.

  • Takeaways & Limitations

    SSD suggests that code models’ latent capability can be unlocked by reshaping token distributions to suppress distractors while preserving useful exploration, without a teacher, verifier, or reinforcement learning.

  • Takeaways & Limitations

    Because SSD is trained only on competitive-programming data, its transfer beyond that domain is a concern, although 30B-model performance remains broadly stable on several other benchmarks.

Abstract

from arXiv · show

Can a large language model (LLM) improve at code generation using only its own raw outputs, without a verifier, a teacher model, or reinforcement learning? We answer in the affirmative with simple self-distillation (SSD): sample solutions from the model with certain temperature and truncation configurations, then fine-tune on those samples with standard supervised fine-tuning. SSD improves Qwen3-30B-Instruct from 42.4% to 55.3% pass@1 on LiveCodeBench v6, with gains concentrating on harder problems, and it generalizes across Qwen and Llama models at 4B, 8B, and 30B scale, including both instruct and thinking variants. To understand why such a simple method can work, we trace these gains to a precision-exploration conflict in LLM decoding and show that SSD reshapes token distributions in a context-dependent way, suppressing distractor tails where precision matters while preserving useful diversity where exploration matters. Taken together, SSD offers a complementary post-training direction for improving LLM code generation. Our code is available at https://github.com/apple/ml-ssd

1 Introduction

SSD addresses limited access to high-quality coding supervision by fine-tuning models on their own raw outputs, improving code generation without external teachers, verifiers, or reinforcement learning. The paper attributes these gains to resolving a precision-exploration conflict through context-dependent distribution reshaping.

  • Expensive human solutions and reliance on teachers or execution-based verification constrain scalable supervised signal for difficult coding tasks.
  • SSD samples solutions with temperature and truncation, then fine-tunes on raw unverified outputs using standard cross-entropy loss.The method requires only problem prompts and the model itself.
  • 42.4% to 55.3% pass@1 on LiveCodeBench v6 for Qwen3-30B-Instruct, with especially large gains on hard problems.Hard-problem pass@5 rises from 31.1% to 54.1%.
  • Code generation alternates between fork positions needing diversity and lock positions needing precision, creating a precision-exploration conflict for global decoding temperature.Lower temperatures suppress distractors but starve forks; higher temperatures enable exploration but destabilize locks.
  • SSD suppresses distractors at locks while preserving useful diversity at forks through context-dependent support compression and within-support reshaping.The account is supported by controlled simulation, real-model analysis, and theory.

2 Embarrassingly Simple Self-Distillation (SSD)

SSD constructs a self-distillation dataset from model-generated solutions under chosen decoding settings, then applies ordinary supervised fine-tuning and evaluates the resulting model with separate decoding settings.

  • Data synthesis: SSD samples candidate solutions from a frozen language model using training temperature Ttrain and truncation configuration ρtrain.
  • Data synthesis: The generated solutions are entirely unverified, with no execution, test cases, or correctness filtering; a single sample per prompt already suffices in practice.The resulting raw outputs form DSSD.
  • Training: The model is fine-tuned on DSSD with standard supervised fine-tuning.
  • Inference: The fine-tuned model is deployed using an evaluation-time temperature and truncation configuration, (Teval, ρeval).

3 Experiments

Across six models and LiveCodeBench evaluations, SSD improves code-generation performance, with gains concentrated on medium and hard problems and remaining beyond decode-only temperature tuning. Training and evaluation temperatures compose broadly, while training-time truncation raises the achievable performance ceiling.

  • Experimental setup: Six models span three families, scales from 4B to 30B, and instruct and thinking reasoning styles.
  • Experimental setup: The experiments use roughly 10K competitive-programming problems and one minimally filtered, unverified solution per prompt for SSD training.Evaluation uses LiveCodeBench v6 as the primary benchmark, with pass@1, pass@5, and difficulty breakdowns.
  • Results: SSD improves every evaluated model on LiveCodeBench, including Qwen3-30B-Instruct from 42.4% to 55.3% pass@1 on LCB v6.The same model improves from 45.8% to 54.3% pass@1 on LCB v5.
  • Results: Qwen3-30B-Instruct gains +15.3pp on hard versus +6.5pp on easy LCB v6 pass@1, with the largest gains generally occurring on medium and hard problems.Its pass@5 gains are +23.0pp on hard and +6.6pp on easy problems.
  • Results: Across Qwen models, pass@5 gains exceed pass@1 gains, consistent with preserved or improved diversity across generated solutions.For Qwen3-30B-Instruct, the gains are +18.1pp versus +12.9pp on LCB v6.
  • Out-of-domain evaluation: Performance remains broadly stable on math reasoning, general code generation, and code understanding benchmarks for the trained 30B models.
  • Decode-only comparison: For Qwen3-30B-Instruct, base-model pass@1 ranges only from 41.3% to 43.5% across evaluated temperatures, whereas SSD retains a +11.8pp advantage over the best-tuned base model.The advantage reaches +13.3pp on hard pass@1 and +19.4pp on hard pass@5.
  • Hyperparameter sweeps: Without truncation, performance is broadly organized by effective temperature Teff = Ttrain · Teval; training-time truncation lifts the ceiling and widens the above-baseline region.The no-truncation temperature composition has R2=0.75.

4 Why SSD Works

SSD addresses a precision-exploration conflict in code generation by reshaping token distributions differently at fork and lock positions. Evidence from toy simulations, real models, theory, and stress tests links its gains to cleaner distractor tails and more usable exploration.

  • The Precision-Exploration Conflict: A single global evaluation temperature forces a tradeoff: lower values protect locks but starve forks, while higher values restore fork exploration but revive lock distractors.Locks require commitment to dominant tokens; forks require diversity among plausible continuations.
  • Real-Model Evidence: Real-model analysis finds that SSD concentrates cumulative probability mass earlier, leaving weaker diffuse tails before decoding begins.This cleaner head is the real-model analogue of the toy simulation’s lock effect.
  • Real-Model Evidence: SSD makes evaluation temperature more effective by retaining multiple viable top continuations and spreading probability across them as temperature rises.The advantage persists even when the models assign similar mass to their top 20 tokens.
  • How SSD Works: SSD reshapes distributions context-dependently, suppressing distractor tails at locks while preserving or increasing useful diversity among fork continuations.The paper describes this as support compression and within-support reshaping rather than uniform sharpening.
  • Bad Data, Good Results: In a high-temperature stress test, roughly 62% of synthesized outputs lacked extractable code, yet SSD reached 48.1% pass@1 and 64.0% pass@5.The result supports learning from token-quality reshaping rather than requiring mostly correct training programs.
  • Bad Data, Good Results: The stress-test gain depends on evaluation-time truncation and degrades when evaluation temperature becomes too high.The authors report a bounded operating region rather than uniformly beneficial decoding settings.

5 Related Work

SSD belongs to research on self-generated supervision, synthetic code data, decoding control, and self-improvement without external rewards. It differs by training directly on temperature-shifted, truncated self-samples without correctness filtering, external feedback, or reinforcement learning.

  • Self-training and self-distillation: Self-training and distillation provide the broader precedent for learning from model-generated targets.The paper situates SSD among classical self-training, knowledge distillation, sequence-level distillation, and newer self-distillation variants.
  • Code generation and synthetic data: Code-generation synthetic-data pipelines commonly add filtering, clustering, verification, or execution feedback to self-generated outputs.Related approaches such as STaR and ReSTEM use correctness-based filtering or external feedback.
  • Reasoning and RL: Reasoning and coding research also uses prompting, self-consistent sampling, self-bootstrapping, and reinforcement-learning-based post-training.These approaches form a complementary context for SSD rather than defining its training procedure.
  • Decoding and truncation: Unlike new decoding rules, SSD trains the model on samples generated under shifted decoding so fixed test-time decoding becomes more effective.Its contribution is changing the model itself rather than proposing another inference-time sampler.
  • Self-improvement without external reward: SSD differs from self-improvement methods using critique, judging, filtering, internal evaluation, or intrinsic rewards because it uses raw self-samples directly.The paper also distinguishes SSD from RL procedures that directly optimize scalar entropy objectives.
  • Self-improvement without external reward: Mechanistically, SSD suppresses diffuse tails while preserving useful entropy within the retained head, including increased entropy at fork-like contexts.This can lower total entropy while retaining exploration where it matters.

6 Conclusion

The paper concludes that simple self-distillation improves code generation using only a model’s own raw outputs, and that its gains extend across models with the largest improvements on harder problems. The proposed explanation is context-dependent distribution reshaping that supports exploration without reopening distractor tails.

  • Conclusion: SSD consistently improves LiveCodeBench performance across six models spanning three model families, with the largest gains on harder problems.The conclusion presents this as the paper’s broad empirical pattern.
  • Conclusion: For Qwen3-30B-Instruct, pass@1 rises from 42.4% to 55.3% on LiveCodeBench v6.This is the headline benchmark comparison reported in the conclusion.
  • Conclusion: The evidence attributes the gains to reshaping token distributions so decoding can explore useful branches without reopening distractor tails.The mechanism links precision-bound locks with exploration-bound forks.
  • Conclusion: The authors suggest that strong code models contain latent capability that can be unlocked without a verifier, teacher, or reinforcement learning.This is the paper’s broader supported implication for post-training.

A Decoding Pipeline: From Notation to Implementation

The paper’s decoding pipeline applies temperature scaling, top-k filtering, top-p filtering, and sampling in sequence, and its theory defines SSD from the corresponding retained support and renormalized target. The formal treatment emphasizes that temperature and truncation change the target before ordinary cross-entropy optimization.

  • Decoding pipeline: Temperature scaling divides logits by T, with low temperature sharpening distributions and high temperature flattening them.Temperatures below 10^-5 trigger greedy decoding in the documented implementation.
  • Decoding pipeline: Top-k retains the k largest temperature-scaled logits, after which top-p removes low-mass survivors until the chosen cumulative threshold is reached.Top-p always retains at least the highest-probability token.
  • Decoding pipeline: Gumbel-max sampling draws from the post-truncation distribution without CPU-GPU synchronization and is mathematically equivalent to multinomial sampling.The implementation uses independent exponential noise and an argmax operation.
  • Decoding pipeline: The implementation order is temperature scaling, top-k filtering, top-p filtering, then Gumbel-max sampling.Temperature acts on logits first; top-k and top-p operate on the resulting distribution before sampling.
  • SSD formalization: SSD defines a retained support by applying training-time temperature, top-k, and top-p to the frozen pre-SSD model.The target is the truncated, renormalized tempered distribution on that support.
  • SSD formalization: The SSD objective still uses ordinary cross-entropy, but temperature and truncation alter the target before optimization begins.When the retained support is a proper subset of the vocabulary, the target assigns zero probability outside it.

B.2 Understanding the SSD Objective and Its Learning Signal

SSD avoids naive self-training’s fixed point by altering the model-generated target with temperature and truncation. Truncation suppresses probability outside a retained support, while temperature reshapes probabilities within that support, producing a learning signal without correctness labels.

  • Why SSD learns: Naive self-training produces no signal when the model trains on samples from its own unmodified distribution at unit temperature.The expected update direction vanishes because the model’s probabilities sum to one.
  • Truncation: Truncation creates a support gate that maximizes the student’s probability mass inside the retained token set.The loss separates into a gate-level objective and a conditional objective over the retained support.
  • Truncation: A truncated target assigns zero probability to outside-support tokens, maintaining pressure during training to suppress their logits.The optimum is approached only as outside-support logits tend toward −∞, so the gate penalty is not fully attained at finite logits.
  • Temperature: Temperature reshapes the distribution within the retained support, smoothing it when T > 1 and sharpening it when T < 1.At T = 1, the Rényi-shaping term vanishes and the fixed-point symmetry returns; temperature alone can also lift harmful tail tokens.
  • Full SSD: Full SSD combines support compression, within-support reshaping, and a KL anchor that preserves alignment with the teacher-induced target.The entropy term is constant in the student parameters and does not contribute to optimization.
  • Full SSD: The student approaches the base model only after its distribution has been transformed by training-time temperature and truncation, rather than matching the raw distribution.This structured target is the objective-level basis for SSD’s potential improvement over the base model.

B.3 How SSD Reshapes Locks and Forks

SSD reshapes token distributions differently at lock-like and fork-like contexts: it suppresses distractor tails while preserving useful diversity within retained supports. This context-dependent change lowers broad uncertainty without eliminating exploration where multiple continuations remain plausible.

  • Locks: At lock-like contexts, SSD removes diffuse tail mass, making the dominant token more robust to evaluation-time decoding.When the retained support is effectively singleton, the effect reduces to pure support compression.
  • Forks: At fork-like contexts, SSD smooths probabilities among surviving alternatives while preventing discarded tail tokens from returning.The KL anchor keeps this reshaping aligned with the teacher’s within-support preferences.
  • Mechanism: The student’s local gain separates exactly into support-compression and within-support reshaping channels under the ideal-fit assumption.Support compression recovers mass leaked outside the retained support, while reshaping redistributes probability among retained tokens.
  • Mechanism: Without truncation there is no support-compression gain, while without a non-unit training temperature there is no within-support reshaping gain.These regimes isolate the two channels algebraically.
  • Evaluation-time behavior: After SSD, lock-like contexts are nearly temperature-inert, whereas fork-like contexts retain a nontrivial head that keeps evaluation-time temperature effective.The variance criterion is small for singleton or nearly uniform heads and larger for uneven multi-token heads.
  • Entropy: SSD can lower full-vocabulary entropy while preserving or increasing conditional head entropy where several plausible continuations survive truncation.Evaluation-time temperature acts on the retained conditional head rather than the discarded tail.
  • Decode-only limitation: Decode-only tuning cannot create this context-dependent transformation because it only reorders or applies one global power transform to the frozen distribution.SSD instead changes the base distribution itself, altering the cumulative geometry presented to the decoder.

C.1 Full Experimental Setup

SSD training uses unlabeled competitive-programming prompts and the model’s own sampled solutions, without verification or external supervision. The experiments evaluate multiple decoding settings, model-specific baselines, and LiveCodeBench performance using pass@1 and pass@5.

  • Data generation: SSD training data consists only of competitive-programming prompts and the model’s own sampled solutions, with no verifier, execution filter, or external teacher.The prompts come from an unlabeled pool of competitive-programming problems.
  • Data generation: Approximately 10,168 unique problems remain after exact string de-duplication of whitespace-normalized problem statements.Each prompt is used to synthesize training data from the frozen base model.
  • Prompting: All models use official chat templates, with instruct models prompted for Python code blocks and thinking models relying on their native reasoning templates.The task presentation remains a single-turn chat format.
  • Decoding: Generation and baseline comparisons use 128K maximum sequence length, with N=1 synthetic sample per prompt during data generation.The generation-time and evaluation-time configurations are summarized in Tables 3 and 4.
  • Optimization: Training uses AdamW with cosine learning-rate decay, global batch size 32, and sequence length 65,536.Instruct models train for 2,500 iterations, while thinking models train for 300 iterations.
  • Evaluation: LiveCodeBench v6 contains 131 problems, with pass@1 as the primary metric and pass@5 and difficulty breakdowns reported additionally.Pass@k estimates use 10 independent samples per problem.

C.2 How SSD Hyperparameters Interact: Full Sweeps

Full sweeps show a broad, stable operating band for SSD configurations rather than a single fragile optimum. Without training-time truncation, performance is largely organized by effective temperature, while truncation raises the pass@1 ceiling.

  • Full sweeps: Successful configurations occupy a broad operating band rather than a single fragile optimum.This supports a relatively stable interaction between training-time and evaluation-time temperatures.
  • Full sweeps: Truncated runs achieve a higher pass@1 ceiling than no-truncation runs across the Qwen3-30B-Instruct sweep.The comparison uses representative configurations plotted against effective temperature.
  • Effective temperature: Without training-time truncation, pass@1 and pass@5 are largely organized by Teff = TtrainTeval.Configurations with similar temperature products achieve similar performance even when the factors differ.
  • Effective temperature: The no-truncation sweep has a broad performance peak near Teff ≈1.2.This pattern is consistent with the paper’s temperature-composition analysis.
  • Cross-model confirmation: Qwen3-4B-Thinking exhibits the same moderate diagonal-band structure across training-time and evaluation-time temperatures.The pattern is not confined to instruct-style models.

C.3 Out-of-Domain Transfer

Programming-only SSD transfers unevenly across capabilities and scales: the 30B models remain broadly stable, while smaller models show benchmark-specific tradeoffs. The transfer evaluation covers math reasoning, general code generation, code understanding, and general knowledge.

  • Evaluation scope: The transfer evaluation probes AIME, HumanEval, CruxEval, and MMLU as adjacent but non-identical tests of retained capabilities.These benchmarks cover math reasoning, general code generation, code understanding, and general knowledge.
  • 30B models: The 30B models remain broadly stable after programming-only SSD, with benchmark changes within roughly ±2 percentage points.Both 30B models maintain MMLU scores within 0.3 percentage points.
  • Smaller models: Smaller models show uneven benchmark-specific tradeoffs rather than a uniform transfer pattern.Qwen3-4B-Instruct regresses on some tasks, Qwen3-4B-Thinking has a mixed profile, and Llama-3.1-8B-Instruct trades AIME losses for HumanEval and CruxEval gains.
  • Smaller models: Qwen3-4B-Thinking shows substantial gains on CruxEval despite small declines on some other benchmarks.This is one example of the scale-dependent transfer profile.

C.4 Toy Simulation: Full Specification and Additional Analyses

The toy finite-state machine isolates SSD’s proposed precision–exploration conflict: successful paths require both broad fork exploration and precise lock decisions. SSD reshapes these contexts asymmetrically, improving the globally optimized decoding regime.

  • FSM specification: The toy FSM combines a fail-dominated root, a broad fork, and three sharply peaked locks with distractor tails.Each successful path traverses one fork and three locks before reaching PASS, while success probability remains exactly computable under decoding settings.
  • SSD-induced distributions: With Ttrain=0.9 and top-p=0.85, SSD compresses locks to two tokens but retains five fork tokens, demonstrating context-dependent reshaping.The correct lock token receives 94.8% mass; at forks, the correct token receives 16.9% alongside three near-tied alternatives.
  • Global decoding optimum: The student reaches 13.77% optimal success versus the teacher’s 8.32%, while its optimal temperature shifts from T=0.639 to T=2.091.The toy’s optimal temperature therefore moves roughly 3× upward after SSD.
  • Fork policy: At their respective optima, the student’s fork nucleus is flatter than the teacher’s and allocates more mass to the correct lower-ranked continuation.The teacher’s four-token nucleus is steeply descending, whereas the student’s is a near-uniform plateau.
  • Robustness: Across top-p values from 0.65 to 0.90, the student remains ahead, with gaps ranging from +1.4 pp to +5.4 pp.This robustness indicates the toy advantage is not tied to one selected truncation threshold.

C.5 High-Temperature Case Study: Full Details and Additional Analyses

The high-temperature stress test deliberately trains SSD on mostly unusable, unfiltered outputs. Even in this setting, the student improves across a bounded evaluation-temperature region, although the gains are weaker and more fragile than with training-time truncation.

  • Experimental purpose: The stress test targets the explanation that SSD succeeds mainly by training on already-good sampled programs.Its poor training distribution is intended to isolate distributional reshaping from superficial sample quality.
  • Training corpus: At Ttrain=2.0 without truncation, approximately 62% of synthesized outputs contain no extractable code.Only about 37% contain a chain-of-thought followed by an extractable code block, and all outputs are retained without filtering.
  • Performance: The best configuration reaches 48.1% pass@1 and 64.0% pass@5, with gains concentrated on hard problems.Across 100 checkpoint–temperature configurations, 62 exceed the 42.4% frozen-base pass@1 baseline, and the optimum lies in a contiguous late-training ridge.
  • Evaluation-temperature range: Performance remains competitive for Teval approximately in [0.6, 1.1] but falls below baseline at Teval=1.3 and declines further at Teval=1.5.The successful operating region is therefore bounded rather than uniformly improved at higher evaluation temperatures.
  • Comparison with truncated SSD: Compared with standard truncated SSD, the no-truncation stress test produces real but smaller and more fragile gains.Without training-time support compression, distractor tails must be removed during evaluation by top-k/top-p truncation.
Loading 2604.01193v2…