PaperlayerView exact revision on arXiv

Source-linked AI summary

BoundaryEvo: Data-Model Co-Evolution at the Capability Boundary for Long-Chain Reasoning Distillation

Jiaming Su, Suorong Yang, Xudong Li, Tengchao Yang, Peng Ye

arXiv:2610.13285v1cs.AIcs.CL

TL;DR

Fixed training sets overlook how reasoning examples’ usefulness changes as a student evolves, while reliable selection is difficult when the student is weak. BoundaryEvo primes capability through direct teacher distillation, then adapts supervision to the student–expert capability boundary. With 1,000 examples, it outperforms static and dynamic selection baselines across 11 reasoning benchmarks.

  • Problem

    Fixed training sets overlook changing example usefulness as the student evolves, and reliable data selection is difficult when the student is initially weak.

    Source: §1 · p. 2

  • Method

    BoundaryEvo first directly distills teacher reasoning trajectories to prime capability, then repeatedly selects questions unresolved by the student but reliably solved by a fixed expert.

    Source: §1 · p. 2, §3 · p. 3

  • Results

    With 1,000 training examples, BoundaryEvo outperforms static and dynamic data-selection baselines across 11 reasoning benchmarks, improving Qwen3.5-4B from 62.9 to 75.3 on IMO-AnswerBench.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Ablations support complementary roles for capability priming, boundary-guided evolution, and replay in establishing capability, expanding the training frontier, and stabilizing improvement.

    Source: §4.6 · p. 9

Abstract

from arXiv · show

Distilling long-chain reasoning trajectories from large teacher models provides an effective way to improve the reasoning ability of small language models. However, existing methods typically select a fixed training set before distillation, overlooking that the usefulness of a reasoning example changes as the student evolves. This mismatch becomes especially costly when only a small number of teacher trajectories can be used for training. Besides, examples already mastered provide little new supervision, while examples far beyond the capability of student may be difficult to learn from. Training data should therefore adapt as model capability evolves, yet reliable data selection is difficult when the student is initially weak. To address these challenges, our key idea is to first establish the basic reasoning capability and then adapt the training data to the evolving capability boundary of the student. We propose BoundaryEvo, a two-stage data--model co-evolution framework. In Capability Priming, we directly distill a small set of teacher-generated long reasoning trajectories without using a separate model to judge and filter them, allowing the student to acquire an initial reasoning capability before capability-aware data selection. In Capability-Boundary Co-Evolution, we repeatedly select questions that the current student has not mastered but the expert can reliably solve, and update the training data as the student evolves. With only 1,000 training examples, BoundaryEvo consistently outperforms static and dynamic data selection baselines across 11 reasoning benchmarks, raising the original Qwen3.5-4B accuracy from 62.9 to 75.3 on IMO-AnswerBench and from 26.8 to 46.0 on AMO-Bench. The code and data will be made publicly available.

1 INTRODUCTION

Long-chain reasoning distillation can improve small models, but fixed supervision ignores how example usefulness changes as the student learns. BoundaryEvo first primes student capability, then adapts supervision to questions the student has not mastered but a fixed expert can solve.

1 INTRODUCTION

  • Fixed supervision overlooks changing example utility: mastered examples become redundant, while initially unreachable examples may later become useful.The paper argues that supervision should adapt to the student's evolving capability boundary.Source: §1 · p. 2
  • Error-driven selection remains reactive because student failures include both near-boundary questions and questions far beyond the student's reach.Student difficulty alone does not guarantee useful supervision; selection should align unresolved student needs with reliable teacher supervision.Source: §1 · p. 2
  • With 1,000 training examples, BoundaryEvo outperforms static and dynamic data-selection baselines across 11 reasoning benchmarks.The introduction reports accuracy increases from 62.9 to 75.3 on IMO-AnswerBench and from 26.8 to 46.0 on AMO-Bench versus the original Qwen3.5-4B.Source: §1 · p. 2
  • BoundaryEvo uses judge-free capability priming before repeatedly selecting questions unresolved by the student but reliably solved by a fixed expert.After student updates, it re-estimates the boundary and refreshes supervision; the framework is evaluated across multiple model backbones.Source: §1 · p. 2
  • Early capability priming retains diverse teacher reasoning, including exploration and unsuccessful attempts, while later training uses capability-boundary-guided selection.The paper presents this as a stage-dependent principle for reasoning distillation.Source: §1 · p. 2

2 RELATED WORK

Related work covers reasoning distillation, self-evolution and co-evolution, and data selection. The paper positions BoundaryEvo as adapting supervision to the student–expert capability boundary rather than relying on static distributions or student errors alone.

Reasoning distillation

  • Prior reasoning-distillation methods use long-chain supervision and small curated datasets, but largely assume a static supervision distribution as model capabilities evolve.The review also discusses chain-of-thought, self-consistency, and large-scale mathematical reasoning data construction.Source: §2 · p. 3

Self-evolution and co-evolution

  • Self-improvement methods use feedback, self-play, or synthetic data, while LLM2LLM selects teacher-generated data from student errors; BoundaryEvo instead tracks the capability boundary.The review also situates multimodal data evolution and agentic task-generation pipelines in this area.Source: §2 · p. 3

Data selection

  • Data selection spans curriculum learning, active learning, quality filtering, replay, and outcome or process verification; BoundaryEvo selects by the student–expert boundary.The review notes that model-based judging can be imperfect.Source: §2 · p. 3

3 THE PROPOSED BOUNDARYEVO

BoundaryEvo first directly distills teacher trajectories to establish student capability, then iteratively selects and verifies supervision near the evolving student–expert capability boundary. Both stages train the student on teacher reasoning trajectories through supervised fine-tuning.

3.2 STAGE 2: DYNAMIC CAPABILITY-BOUNDARY CO-EVOLUTION

  • At each evolution round, the current student and fixed expert solve candidate questions, and an analyzer selects valid, high-quality questions the expert can solve but the student has not mastered.The selected examples are combined with revalidated historical data, then verified teacher trajectories are distilled before the student is updated.Source: §3 · p. 3, §3.2 · pp. 4–5

3.1 STAGE 1: JUDGE-FREE DIRECT DISTILLATION FOR CAPABILITY PRIMING

  • Stage 1 directly distills teacher-generated reasoning trajectories from seed questions without additional model-based judging or capability-gap filtering.This stage aims to transfer teacher reasoning behaviors and establish initial student capability before boundary-aware selection.Source: §3.1 · pp. 3–4

3.2 STAGE 2: DYNAMIC CAPABILITY-BOUNDARY CO-EVOLUTION

  • Stage 2 reuses eligible replay examples and fills the remaining fixed per-turn budget with quality-ranked novel boundary examples.Fresh teacher rollouts are generated for selected questions, and only trajectories whose final answers pass reference verification supervise the next student.Source: §3.2 · p. 5

3.3 TRAINING OBJECTIVE

  • Both stages use standard supervised fine-tuning on teacher trajectories containing full reasoning and final answers; only response tokens contribute to the loss.Stage 1 produces the primed checkpoint, while each Stage 2 turn updates the student using verified capability-boundary trajectories.Source: §3.3 · p. 5

4 EXPERIMENTS

The experiments assess BoundaryEvo under a 1,000-example budget against data-selection baselines, across reasoning benchmarks and model backbones. Results report gains over selection and fixed-data alternatives, alongside training-dynamics evidence and component ablations.

4.1 EXPERIMENTAL SETUP

  • Experiments use 1,000 total examples (100 priming plus three rounds of 300) on Qwen3.5-4B, comparing random, error-based LLM2LLM, and static 1,000-example SFT baselines.Evaluation covers eight mathematical benchmarks and three general-reasoning benchmarks.Source: §4.1 · pp. 6–7

4.2 DOWNSTREAM EFFECTIVENESS

  • BoundaryEvo-4B leads five named math benchmarks, beats DASD-4B-Thinking on all shared benchmarks (including 19.5 points on AMO-Bench), and scores 75.3 on IMO-AnswerBench.It also reaches 96.0 on ZebraLogicBench, 81.3 on GPQA-Diamond, and 55.0 on SuperGPQA.Source: §4.2 · p. 7

4.3 MAIN RESULTS

  • BoundaryEvo beats random selection and LLM2LLM on all 11 benchmarks, with Math-average gains of 8.7 and 6.4 points and General-average gains of 2.5 and 1.2 points, respectively.The table reports Qwen3.5-4B results across three evolution turns.Source: Table 2 · p. 7, §4.3 · p. 8

4.4 GENERALIZATION ACROSS MODEL BACKBONES

  • Across backbones, Qwen3.5-2B Turn 2 reaches AIME25 46.7, while Gemma-4-E4B-it Turn 3 averages 70.4 Math and 80.5 General; the 2B model saturates earlier.The smaller model also shows less stable gains across turns.Source: §4.4 · p. 8, Table 3 · p. 8

4.5 TRAINING DYNAMICS

  • Across three turns, SFT loss falls from 0.4362 to 0.3936 and 0.3388 while student scores rise from 0.233 to 0.265 and 0.271 against high expert scores.Mean expert scores are 0.846, 0.879, and 0.884; Figure 3 plots loss and student/expert scores by turn.Source: §4.5 · p. 8, Figure 3 · p. 9

4.6 DISCUSSION

  • Removing Stage 1 lowers final Math/General averages from 76.4/77.4 to 73.5/74.0; removing replay drops final IMO-AnswerBench from 75.3 to 71.1.The ablations assign complementary roles to priming, boundary-guided evolution, and replay.Source: §4.6 · p. 9

5 CONCLUSION

BoundaryEvo adapts long-chain reasoning supervision as student capability evolves, and the conclusion presents co-evolution as a way to discover useful supervision with 1K examples.

5 CONCLUSION

  • BoundaryEvo combines judge-free capability priming with capability-boundary-guided selection and verified trajectory distillation, achieving strong reasoning performance with 1K examples.The conclusion says ablations validate both capability priming and capability-boundary-guided evolution.Source: §5 · p. 9

AI USE STATEMENT

  • The authors report limited generative-AI assistance for language refinement and writing improvement, with all assisted content reviewed by the authors.They state that AI tools were not used for research ideas, experiment design, evaluations, or scientific claims.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

  • The study uses public training and evaluation datasets under their respective terms, recruits no human participants, and collects no private user data.The authors also state that they do not intentionally introduce personally identifiable information into the data.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • The authors say implementation and evaluation details are provided in the paper and supplement, and plan to release code and processed data.Source: REPRODUCIBILITY STATEMENT · p. 10

A HYPERPARAMETER AND SELECTION SENSITIVITY

The sensitivity study varies three choices independently to test optimization, evolution, and boundary-selection behavior.

A HYPERPARAMETER AND SELECTION SENSITIVITY

B TRAINING DETAILS

Training uses response-only LoRA supervised fine-tuning, with Stage 1 priming followed by sequential evolution turns and backbone-specific preprocessing.

B TRAINING DETAILS

  • Students use response-only LoRA SFT: question and system-prompt tokens are masked, and LoRA targets differ between Qwen3.5 and Gemma-4-E4B-it.Source: B TRAINING DETAILS · p. 21
  • Training preserves an effective batch size of eight across one- or four-H200-GPU jobs, using gradient accumulation of eight or two, respectively.A 100-example, five-epoch Stage 1 run has 65 optimizer updates; a standard 300-example Stage 2 run has 190.Source: B TRAINING DETAILS · p. 21, Table 6 · p. 22
  • Stage 1 starts from the pretrained backbone with 100 direct teacher trajectories; three 300-example Stage 2 turns bring the reported budget to 1,000 examples.Replayed examples count again when reused in a later turn.Source: B TRAINING DETAILS · p. 21
  • Each turn starts from the previous merged student checkpoint, so selection uses the fully updated model rather than an unmerged adapter or base model.Random-selection and LLM2LLM baselines use the same SFT objective, optimizer family, epoch count, and backbone-specific formatting.Source: B TRAINING DETAILS · pp. 21–22
  • Preprocessing uses a 40,960-token cutoff for Qwen3.5 and retains rendered Gemma examples up to 32,768 tokens to avoid implicit truncation.Malformed records and examples without a usable assistant response are discarded.Source: B TRAINING DETAILS · p. 22

C REASONING-TRAJECTORY LENGTH ANALYSIS

On matched questions, BoundaryEvo has higher accuracy and shorter median reasoning trajectories across seven benchmarks, while upper-tail changes vary by task.

C REASONING-TRAJECTORY LENGTH ANALYSIS

  • The comparison retains exact normalized question matches, summarizes repeated rollouts by per-question median, and measures reasoning tokens after removing final answers and boundary tokens.Source: C REASONING-TRAJECTORY LENGTH ANALYSIS · p. 22
  • 7.4%–52.9%: median reasoning length falls across all seven matched benchmarks while accuracy rises on each; AIME26 improves from 84.0 to 93.3 as median length falls from 36.9K to 17.4K tokens.Figure 5 plots smoothed per-question token-count densities with medians; Table 7 reports matched-question length statistics.Source: C REASONING-TRAJECTORY LENGTH ANALYSIS · p. 22, Table 7 · p. 23, Figure 5 · p. 23
  • Upper-tail behavior is task dependent: the 90th percentile falls on five tasks, changes little on HMMT26, and rises on GPQA-D.The analysis is observational and does not establish that shorter typical trajectories cause the accuracy gains.Source: C REASONING-TRAJECTORY LENGTH ANALYSIS · p. 23

D GPQA-D SCALING AND DATA EFFICIENCY

BoundaryEvo-4B combines strong GPQA-D performance with a small disclosed training set, while comparisons with larger models and different data budgets are limited to the reported systems and are not controlled compute comparisons.

D GPQA-D SCALING AND DATA EFFICIENCY

  • BoundaryEvo-4B scores 81.3 on GPQA-D, 5.1 points above its Qwen3.5-4B base and above every reported model up to 12B parameters.It is competitive with Qwen3-235B at 81.1 and LongCat-560B at 81.5; Phi-4-Reasoning at 14B leads the displayed scores with 81.9.Source: D GPQA-D SCALING AND DATA EFFICIENCY · p. 23, Figure 6 · p. 24
  • The plots include only systems with reported parameter counts and GPQA-D scores, or disclosed data amounts and GPQA-D scores, as applicable.Right-pointing arrows in the data-budget plot denote lower bounds, not exact dataset sizes.Source: D GPQA-D SCALING AND DATA EFFICIENCY · p. 23, Figure 6 · p. 24, Figure 7 · p. 24
  • With 1K training examples, BoundaryEvo scores 81.3 on GPQA-D, the best result with the smallest disclosed training set among the reported systems.DASD-4B uses 448K reported examples and scores 68.4; three 7B comparison models use at least 102K to 6M examples and score from 46.2 to 61.1.Source: Figure 7 · p. 24, D GPQA-D SCALING AND DATA EFFICIENCY · p. 24
  • The data-budget comparison measures example efficiency, not total computational cost, because dataset definitions, trajectory lengths, and filtering policies differ.BoundaryEvo’s 1K count includes repeated exposure but excludes student and expert inference, analyzer judging, and teacher-rollout generation.Source: D GPQA-D SCALING AND DATA EFFICIENCY · p. 25

E BOUNDARYEVO ALGORITHM AND IMPLEMENTATION DETAILS

The implementation primes the student with teacher trajectories, then repeatedly builds fixed-budget supervision from eligible replay and novel questions; prompts specify rollouts, analyzer judgments, and final-answer verification.

E BOUNDARYEVO ALGORITHM AND IMPLEMENTATION DETAILS

  • At each turn, the algorithm combines eligible replay questions with ranked novel candidates while keeping the training budget fixed.Replay examples come from the preceding turn and remain eligible under the current screening protocol; novel candidates were not in that preceding set.Source: E BOUNDARYEVO ALGORITHM AND… · pp. 25–26

F PROMPT TEMPLATES

  • The prompt templates use no model-based judge in Stage 1, and the teacher trajectory is used after basic response-validity checks.The Stage 1 prompt asks the teacher to reason step by step and place its final answer in boxed notation.Source: F PROMPT TEMPLATES · p. 26, §F.1 · p. 26

F.1 STAGE 1 TEACHER TRAJECTORY DISTILLATION PROMPT

  • Stage 1 asks the teacher for a step-by-step reasoning trajectory ending with a boxed final answer.The resulting trajectory is used directly after basic response-validity checks.Source: §F.1 · p. 26

F.2 STAGE 2 STUDENT AND EXPERT ROLLOUT PROMPT

  • In Stage 2, the current student and fixed expert independently solve the same question using concise reasoning and a final-answer line.The prompt allows optional context.Source: §F.2 · p. 26

F.3 CAPABILITY-BOUNDARY ANALYZER PROMPT

  • The analyzer judges the question and both responses using clarity, answerability, reference correctness, context support, and relative answer quality.Reference-correctness and leakage judgments are auxiliary diagnostics, and their hard rejection switches are disabled in the reported route.Source: §F.3 · pp. 26–27

F.4 FINAL-ANSWER VERIFICATION PROMPT

  • Teacher trajectories are verified by final-answer equivalence, allowing algebraically equivalent forms and reasonable numerical precision.The verification prompt compares each candidate teacher answer with the reference answer.Source: §F.4 · p. 28

G.1 COMMON EVALUATION PROTOCOL

The evaluation protocol combines publicly reported scores with local evaluations when needed and fixes a common follow-up setup; the paper distinguishes its contribution from LLM2LLM-style synthesis and general ZPD-style filtering.

G.1 COMMON EVALUATION PROTOCOL

  • When public model–benchmark scores are unavailable, the study locally evaluates primarily smaller open-weight models using EvalScope and the official split.Local runs use default top-p and temperature with a maximum of 102,400 generated tokens.Source: §G.1 · p. 28
  • Follow-up methods share evaluation settings and a three-turn final checkpoint, and exact or near-duplicate training–evaluation questions are manually checked and removed.Competition-math benchmarks use at least four completions per question and other benchmarks use four; unparseable or truncated answers count as incorrect.Source: §G.1 · pp. 28–29

G.2 RELATIONSHIP TO LLM2LLM AND AGENTFRONTIER

  • The paper’s claimed contribution is priming followed by repeated boundary re-estimation for tool-free, fixed-pool long-trajectory distillation, not ZPD-style filtering alone.The fixed-pool baseline rescans questions the student answers incorrectly, whereas LLM2LLM also synthesizes new error-targeted questions.Source: §G.2 · p. 29

Related papers

Loading 2610.13285v1…