PaperlayerView exact revision on arXiv

Source-linked AI summary

Toward Skill-Native LLMs: Skill Entropy for Benchmarking and Training Long-Horizon Reasoning

Yinghui He, Ling Yang, Jiarui Liu, Yongjin Yang, Lechen Zhang, Yingcheng Wu, Zhenfei Yin, Mengdi Wang, Sanjeev Arora

arXiv:2608.05139v1cs.CLcs.LG

TL;DR

Cross-skill long-horizon reasoning requires models to switch skills across dependent steps, but existing evaluations lack a principled measure of switching difficulty. The paper introduces Skill Entropy, Skill2-Bench, and Skill-Entropy RL, finding that accuracy declines with higher entropy while Qwen3-4B-Instruct improves from 34.4% to 68.4%.

  • Problem

    Existing evaluations lack a task-level measure of skill-switching difficulty in dependent, cross-skill reasoning chains.

  • Method

    The paper defines directed Skill Entropy, builds Skill2-Bench over 558 skills across nine domains, and uses predicted skill sequences as an RL training signal.

  • Results

    Accuracy decreases nearly monotonically as task-level skill entropy rises, while Skill-Entropy RL raises Qwen3-4B-Instruct’s Skill2-Bench score from 34.4% to 68.4%.

  • Takeaways & Limitations

    Skill entropy serves as both a benchmark difficulty scale and a reusable training signal for cross-skill long-horizon reasoning.

  • Takeaways & Limitations

    Skill entropy is calibrated with a fixed reference model, keeping the difficulty scale constant across evaluated models.

Abstract

from arXiv · show

Long-horizon reasoning in recent LLMs demands that the model switch between distinct skills inside a reasoning chain, such as first doing a math derivation, then using the result to plan a schedule. We call such problems cross-skill long-horizon tasks: multi-step tasks whose steps require different reasoning skills and depend on earlier outputs. Existing benchmarks often evaluate individual skills, lacking a principled way to measure how well a model switches between skills. We address this gap from both the evaluation and training sides. We introduce Skill Entropy, a measure of the difficulty of switching from one skill to another. We then propose Skill^2-Bench, a benchmark of cross-skill long-horizon tasks built over 558 skills across 9 verifiable and open-ended domains. Each task is assigned a task-level skill-entropy score and grouped into three difficulty levels. Evaluating 8 frontier and 4 open-source models on Skill^2-Bench reveals a skill-switching gap: accuracy decreases on higher-entropy tasks. We then turn skill entropy from a benchmark scale into a training signal. We propose Skill-Entropy RL, an RL framework where the model predicts not only the answer at each step but also the skill used to produce it. The reward combines step-level correctness with a skill-entropy reward that measures the alignment between the model-predicted skill sequence and the gold skill sequence. On Qwen3-4B-Instruct and Qwen3-1.7B, Skill-Entropy RL improves the Skill^2-Bench score from 34.4% to 68.4% and from 14.6% to 40.1%, respectively, outperforming competitive baselines. The same pipeline can be applied to off-the-shelf training data such as OpenR1-Math, indicating that skill entropy is a reusable training signal. Code available at: https://github.com/Gen-Verse/Skill-Entropy-RL

1. Introduction

The paper introduces Skill Entropy to quantify cross-skill switching difficulty and uses it to build Skill2-Bench and guide Skill-Entropy RL training. Evaluations reveal that model accuracy declines as task-level skill entropy increases, while the proposed training signal substantially improves benchmark performance.

  • Task formulation: Cross-skill long-horizon tasks require sequential steps that invoke different reasoning skills and depend on earlier answers.The paper distinguishes skill switching from isolated per-skill performance because different skills have different reasoning styles and transferability.
  • Skill Entropy: Skill Entropy is a directed pairwise measure of switching difficulty, aggregated across a task’s skill sequence into task-level skill entropy.The resulting scalar quantifies each task’s skill-switching difficulty.
  • Benchmark: Skill2-Bench covers 558 skills across 9 domains and sorts tasks into low-, medium-, and high-entropy difficulty levels.The domains include math, coding, science, planning, logic, information extraction, instruction following, context retrieval, and creative writing.
  • Evaluation: −4 ∼−13% accuracy drops when the same skill appears inside a cross-skill task rather than a single-skill question.Across 8 frontier and 4 open-source models, accuracy decreases nearly monotonically as task-level skill entropy rises, including for skills models otherwise handle well.
  • Training: 34.4% to 68.4% Skill2-Bench score improvement is achieved by Qwen3-4B-Instruct with Skill-Entropy RL.The framework combines step-level correctness with a reward comparing predicted skill chains against gold skill sequences, after requiring a skill label before each answer.

2. Preliminaries

The paper formalizes skills as domain-specific capabilities and defines cross-skill long-horizon tasks as dependent multi-step sequences that alternate domains. Task performance is measured by averaging domain-specific per-step scores.

  • Skills: Skills are organized across verifiable and open-ended domains, with each domain backed by seed question–answer pairs and problems labeled by domain skills.Verifiable domains provide ground-truth answers, whereas open-ended domains provide grading rubrics.
  • Cross-Skill Long-Horizon Task: A cross-skill long-horizon task is a length-L sequence of question–answer pairs paired with a skill sequence, with consecutive steps drawn from different source domains.Each step is adapted from a seed dataset and uses a skill from the full skill set.
  • Cross-Skill Long-Horizon Task: An LLM proposer rewrites each question within a unifying scenario so that it depends on the preceding answer while preserving the seed question’s underlying logic and ground-truth answer.This construction links otherwise separate skill-specific questions into a long-horizon task.
  • Skills: Skill^2-Bench spans 9 domains and 558 labeled skills, using seed-derived question–answer pairs for verifiable domains and LLM-generated question–rubric pairs for open-ended domains.Open-ended tasks are scored by LLM judges.
  • Evaluation: Task scores average per-step scores from domain-specific scorers, including symbolic equivalence for mathematics and sandboxed unit-test execution for coding.Each scorer maps a model response, reference answer, and question to a value in [0, 1].

3. Skill2-Bench: Benchmarking Cross-Skill Long-Horizon Reasoning

Skill2-Bench measures cross-skill long-horizon reasoning by using directional and task-level skill entropy to quantify switching difficulty and organize tasks by controlled entropy levels. Evaluations show that accuracy generally declines as entropy rises, revealing cross-skill failures that single-skill tests miss.

  • Benchmark motivation: Skill2-Bench evaluates how reliably LLMs switch between skills inside long-horizon tasks.It targets cross-skill tasks whose later steps depend on earlier outputs.
  • Skill entropy: Skill Entropy SkE(s_a, s_b) quantifies the directional difficulty of switching from source skill s_a to target skill s_b.Higher entropy corresponds to a sharper accuracy drop on the target skill after a preceding source-skill answer, and switch order matters.
  • Skill entropy: Task-level skill entropy averages pairwise entropy across directed switches and partitions tasks into low, medium, and high entropy levels.Thresholds are taken from the empirical distribution of pairwise scores.
  • Benchmark construction: Skill2-Bench construction combines per-domain skill banks, reference-model entropy estimates, target-level skill-sequence sampling, scenario rewriting, and verifier filtering.Tasks use sampled lengths between 2 and 10 and span verifiable and open-ended domains.
  • Evaluation findings: −4% to −10%: every frontier model loses accuracy in cross-skill settings, with the largest drops on Planning skills.Skill2-Bench performance also decreases from low to high task-level skill entropy for nearly all models, while wrong skill selection at a step roughly halves accuracy.

4. Training Skill-Native LLMs via Skill Entropy

Section 4 introduces Skill-Entropy RL, which trains models to predict structured skill plans alongside answers and rewards alignment with gold skill sequences. The method improves cross-skill performance, enables cleaner skill switching, and transfers to existing reasoning data such as OpenR1-Math.

  • Method: Skill-Entropy RL uses a two-stage SFT-to-GRPO pipeline that teaches structured skill-and-answer outputs before optimizing them with answer and skill-entropy rewards.The reward combines per-step answer accuracy with alignment between predicted and gold skill-entropy ranks, allowing semantically similar skill substitutions.
  • Training setup: 9K synthesized cross-skill tasks support training: 3K teacher-labeled traces are used for SFT warm-up and 6K tasks for RL, while open-ended domains are evaluation-only.Training uses six verifiable domains, whereas evaluation covers the same 300-task test pool across all nine domains.
  • Main results: +9.6% and +7.9% are Skill-Entropy RL’s gains over GRPO on Qwen3-4B-Instruct and Qwen3-1.7B, respectively; gains over STAT are +7.0% and +7.1%.GRPO is the ablation without the skill-entropy reward, isolating the contribution of grading predicted skill structure beyond final-answer correctness.
  • Main results: Seven of nine Skill2-Bench domains show the best per-domain accuracy under both model sizes, with the largest gain in Creative Writing.The method also achieves the highest average score across MuSR, LongBench-MuSiQue, GPQA-Diamond, MMLU, and IFEval.
  • Off-the-shelf data: OpenR1-Math training improves over GRPO by +1.9% on average and over the base model by +7.7% across six math benchmarks.After vanilla GRPO plateaus, adding the skill-entropy reward continues improving without saturation and requires no data-pipeline change.

5. Discussion … B.4. Factorizing pairwise skill entropy through domains

The paper positions Skill Entropy as a reward-compatible signal for cross-skill reasoning and details the Skill2-Bench synthesis pipeline, including skill-bank construction, entropy estimation, and domain-based factorization. The factorization replaces infeasible all-pairs evaluation with directional skill→domain and domain→skill estimates while preserving multiplicative transition costs.

  • A. Related Works: Prior benchmarks largely evaluate isolated skills or long-horizon tasks without a principled skill-switching measure, while prior skill-aware methods inject skills into data or routing rather than RL rewards.The related-work discussion distinguishes benchmark design, skill-aware training, and reward-based reinforcement learning.
  • 5. Discussion: Skill Entropy measures directed switching difficulty, supports Skill2-Bench over 558 skills across nine domains, and improves Qwen3-4B-Instruct from 34.4% to 68.4%.The signal also transfers to open-ended domains and off-the-shelf training data.
  • B. Synthesis pipeline details: The synthesis pipeline covers seed datasets and judges, per-domain skill banks, entropy estimation, skill→domain factorization, reference-model selection, skill-sequence sampling, generation prompts, and verification.These stages are expanded across Appendix B and its subsections.
  • B.1. Seed datasets and rubric prompts: Open-ended steps receive questions and per-skill rubrics generated from the scenario, task plan, domain, skill, and previous steps, with judges returning [0, 1] scores and justifications.Verifiable domains use deterministic scoring, whereas open-ended domains use rubric-based evaluation.
  • B.2. Skill bank: Verifiable skill banks begin with representative skills, expand through LLM labeling, cluster near-duplicates with embeddings, and undergo manual review; open-ended banks are generated directly and processed similarly.The resulting per-domain bank sizes match Table 1, with complete lists released with the project.
  • B.3. Skill entropy estimation procedure: The reference model Claude-opus-4.7 estimates skill-level, domain-level, and directional two-step accuracies through Monte Carlo sampling, using N_s = 5, N_d = 5, and N_c = 5 samples.Smoothed accuracy ratios with α = 0.1 produce directional entropies before pairwise factorization.
  • B.4. Factorizing pairwise skill entropy through domains: With |S| = 558, direct ordered-pair evaluation requires |S|^2 ≈ 3.1 × 10^5 pairs, so pairwise entropy is approximated by multiplying skill→domain and domain→skill entropies.The factors use directional accuracies Accuracy(s, d) and Accuracy(d, s), with domains averaging over target or source skills.
  • B.4. Factorizing pairwise skill entropy through domains: The multiplicative form models leaving and landing costs as separable, and reduces computation from O(|S|^2) to O(|S| · |D|), requiring ∼5k evaluations per direction for |D| = 9.A hard transition in either factor increases the product, while two no-interference transitions yield ratios approximately 1.

B.5. Choice of reference model … B.9. Filtering and verification

The appendix establishes that Skill2-Bench’s entropy-based difficulty is robust to the reference model, then details task sampling, cross-skill synthesis, worked examples, and verifier-based filtering. These procedures constrain skill sequences, preserve dependencies and answers, and reject tasks that violate validity checks.

  • B.5. Choice of reference model: Claude-opus-4.7 is the fixed reference model for task-level skill entropy, keeping the independent-baseline numerator near ceiling so SkE(s_a, s_b) > 1 reflects cross-context difficulty.The reference choice is intended to distinguish switching difficulty from weak single-skill ability.
  • B.5. Choice of reference model: Above 80% partition overlap and approximately 2-point cross-skill-average differences under Gemini-3.1-pro and GPT-5.5 preserve the headline qualitative trends.The reproduced trends include cross-skill drop, planning fragility, and GPT-5.4-mini’s non-monotonic behavior.
  • B.5. Choice of reference model: Single-skill accuracy rank correlations of 0.75–0.90 and cross-skill accuracy rank correlations of 0.67–0.81 show substantial agreement on difficult skills and tasks across reference models.Additional checks find stable entropy rankings, difficulty stratification, and alignment with blinded human judgments.
  • B.6. Skill-sequence sampling: Each sampled task has length L∼Uniform{2, 3, . . . , 10}, a low, medium, or high target entropy level, and consecutive skills drawn from different domains.Rejection sampling accepts a sequence only when its task-level SkE(μ) falls within the selected level’s empirical boundaries.
  • B.7. Proposer and open-ended generation prompts: Verifiable-domain synthesis rewrites seed questions into one scenario while preserving each answer and linking every step to the preceding answer.Open-ended skills instead generate question–rubric pairs conditioned on the scenario and prior-step answer, scored on [0, 1].
  • B.7. Proposer and open-ended generation prompts: The proposer prompt requires a coherent one-sentence scenario and a brief description under 300 words explaining how each sampled skill applies.Verifiable rewrites must preserve core logic, answers, and multiple-choice options; open-ended prompts request both a question and grading rubric.
  • B.8. Worked example: A length-5 worked example combines geometry, deductive reasoning, planning, number words, and brainstorming, with later steps reusing earlier puzzle results.Its planning step explicitly combines house-placement constraints with triangle-area conditions to prune the search space.
  • B.9. Filtering and verification: The verifier checks answer preservation, scenario consistency, dependency presence, and absence of answer leakage; failed tasks regenerate and are dropped after three attempts.A task is accepted only when all four checks pass, with later steps required to depend on preceding answers.

C. Full evaluation results · C.1. Inference settings

The appendix reports full evaluation coverage, validates LLM judging on open-ended domains, and analyzes cross-skill failures. It also specifies inference settings and shows that wrong skill-family selection is a major failure mode.

  • C. Full evaluation results: Four omitted domains—information extraction, Creative Writing, Context Retrieval, and instruction following—receive appendix results alongside the main evaluation.The added domains include one verifiable domain and three open-ended domains.
  • C. Full evaluation results: Open-ended domains omit Single-Skill accuracy because their questions and per-step rubrics are generated conditionally rather than drawn from fixed isolated datasets.Generation conditions include the scenario, task plan, and previous-step questions and answers.
  • C. Full evaluation results: 200 cross-skill steps per open-ended domain are rescored by three human annotators using the same rubric provided to the LLM judge.Table 10 compares correlations, mean absolute error, binary agreement, and Cohen’s κ between LLM and human scores.
  • C. Full evaluation results: Across all three open-ended domains, the LLM judge tracks human ratings closely, supporting its use as the scorer in Table 2.The agreement analysis uses Claude-opus-4.7 as the LLM judge.
  • C. Full evaluation results: 9–17% of steps solved by GPT-5.5, Gemini-3.1-pro, Claude-opus-4.7, and O4-mini in Single-Skill mode fail in Cross-Skill mode.Among these new failures, 31–62% involve selecting a skill from the wrong domain.
  • C. Full evaluation results: 32–57% per-step accuracy on wrong-domain steps is roughly half the 63–79% accuracy on right-domain steps.This comparison concerns GPT-5.5, Gemini-3.1-pro, Claude-opus-4.7, and O4-mini.
  • C.1. Inference settings: Open-source models use vLLM with temperature 0.7, top-p 0.9, and a 16K-token generation cap, while frontier models use providers’ default APIs.Cross-Skill prompts contain the full scenario and all per-step questions; Single-Skill queries each step independently without the surrounding scenario.
  • C.1. Inference settings: Figure 5 matches verifiable Cross-Skill steps with their Single-Skill counterparts and separates errors by predicted skill-family correctness.It distinguishes wrong-skill-family errors from right-family-but-wrong-answer errors and compares their per-step accuracy.

D. Skill-Entropy RL training details

The appendix details the Skill-Entropy RL method’s training setup, covering response formatting, reward definitions, skill matching, and RL hyperparameters. It also includes an end-to-end SFT trace.

  • Response format and SFT trace: The appendix presents the response format and an end-to-end supervised fine-tuning trace.These materials are provided in Section D.2.
  • Reward definitions: It formally defines the rewards used by Skill-Entropy RL.The formal reward definitions appear in Section D.3.
  • Skill matching and RL hyperparameters: It describes embedding-based skill matching for looking up entropies under lexical variation and reports the full RL hyperparameters.These components are covered in Sections D.4 and D.6, respectively.

D.1. Off-the-shelf training results on OpenR1-Math … D.4. Embedding-based skill matching

The appendix extends Skill-Entropy RL to off-the-shelf OpenR1-Math training and specifies its skill-annotated trace format, rewards, normalization, and embedding-based label matching. These details connect benchmark gains to an end-to-end training pipeline that scores both answers and skill transitions.

  • D.1. Off-the-shelf training results on OpenR1-Math: D.1. OpenR1-Math training evaluates a 6K OpenR1-Math subset across six math benchmarks, including AIME24, AIME25, HMMT25, AMOBench, OpenR1-Math, and MATH.Skill labels are obtained through the Section 4.4 labeling pipeline.
  • D.1. Off-the-shelf training results on OpenR1-Math: D.1. +1.9% average improvement over GRPO and +7.7% over the base model are achieved by Skill-Entropy RL on Qwen3-4B-Instruct, with the best score on every benchmark.The trend mirrors the results on Skill2-Bench in Table 3.
  • D.2. Response format and example SFT trace: D.2. The teacher produces one <think> block followed by alternating <skill> and <answer> tags, with one pair for each task step.Each skill tag contains a comma-separated domain, skill pair from the Skill2-Bench skill bank.
  • D.2. Response format and example SFT trace: D.2. A length-3 SFT trace demonstrates the skill sequence [information_extraction, coding, math] through extraction, Fenwick-tree coding, and number-theory answers.The example includes the answers “Usain Bolt” and 142857.
  • D.2. Response format and example SFT trace: D.2. Malformed responses receive zero answer reward and zero skill-entropy reward when each step lacks the required adjacent <skill> and <answer> tags.Parsing is performed with a regex requiring every step to carry a <skill> tag immediately followed by an <answer> tag.
  • D.3. Reward definitions: D.3. The answer reward averages per-step accuracy using the per-domain scorer eval𝑑𝑖, while the skill-entropy reward compares predicted and gold skill sequences through their task-level entropies.The predicted sequence is ˆ𝜇(𝜏) and the gold sequence is 𝜇(𝜏).
  • D.3. Reward definitions: D.3. CDF rank normalization makes the skill-entropy reward scale-free across tasks with different difficulty levels.A fixed raw-entropy gap is penalized more when it spans many training tasks and less when it spans few.
  • D.4. Embedding-based skill matching: D.4. Emitted skill labels are embedded with Qwen3-Embedding-0.6B and matched to the nearest canonical key by cosine similarity, with scores below 0.5 treated as out-of-bank.Out-of-bank labels contribute no skill-entropy reward for the affected switch, reducing noisy matches for hallucinated skills.

D.5. Training data and evaluation protocol · D.6. RL hyperparameters

Training uses skill-annotated cross-skill tasks from verifiable domains, while evaluation tests the same protocol on longer chains and open-ended steps. RL additionally applies decoupled clipping and variance-based dynamic sampling.

  • D.5. Training data and evaluation protocol: The training-data subsection complements the shortened Training data paragraph in Section 4.2.This appendix subsection provides additional training-data and evaluation-protocol detail.
  • D.5. Training data and evaluation protocol: Training uses only six verifiable Skill2-Bench domains, reserving three open-ended domains for evaluation.This enables automatic verification during training.
  • D.5. Training data and evaluation protocol: 9K synthesized cross-skill tasks are held out from testing, balanced across three skill-entropy levels, and restricted to lengths 3 to 5 for tractable RL rollouts.The task pool is generated with the Skill2-Bench pipeline.
  • D.5. Training data and evaluation protocol: 3K tasks provide SFT warm-up data with skill-annotated traces generated by Qwen3-8B.The passage identifies Qwen3-8B as the stronger teacher.
  • D.5. Training data and evaluation protocol: Evaluation follows the Cross-Skill protocol on the same 300-task test pool, with the evaluated model judging open-ended steps.The model itself serves as the LLM judge for open-ended evaluation steps.
  • D.5. Training data and evaluation protocol: Test-task lengths range from 2 to 10, exceeding the 3-to-5 training range to assess generalization to longer skill chains.The wider evaluation range directly differs from the training-task length restriction.
  • D.6. RL hyperparameters: RL uses DAPO-style decoupled clipping with both low and high clip ratios set to 0.2.These hyperparameters apply to both Qwen3-4B-Instruct and Qwen3-1.7B.
  • D.6. RL hyperparameters: Dynamic sampling admits rollouts only when group reward variance is non-zero, accelerating convergence when all rollouts succeed or fail.The hyperparameter configuration is reported for both Qwen3-4B-Instruct and Qwen3-1.7B.

D.7. Reward weight ablation · D.8. Comparison against the single-skill oracle

Reward-weight ablation identifies (0.7, 0.3) as the best answer-to-skill-entropy split, with performance falling when either signal is weakened or entropy dominates. Skill-Entropy RL also exceeds the base model’s single-skill-oracle accuracy on both evaluated models in the cross-skill setting.

  • D.7. Reward weight ablation: Four reward splits sweep the simplex λ_ans + λ_ent = 1 from answer-dominated to entropy-dominated supervision.The ablation is conducted on Qwen3-4B-Instruct with all other settings fixed to Table 12’s configuration.
  • D.7. Reward weight ablation: The default (0.7, 0.3) split is the peak across all four reward-weight sweeps.This split is the configuration marked as default in Table 3.
  • D.7. Reward weight ablation: 7.6 points: setting λ_ent = 0.1 lowers the Skill2-Bench score relative to the default split.The result supports retaining a substantial skill-entropy contribution rather than shrinking it excessively.
  • D.7. Reward weight ablation: 12.8 to 19.8 points: letting λ_ent ≥0.5 lowers the Skill2-Bench score relative to the default split.The larger decline on the entropy-dominated side indicates that answer supervision should remain dominant.
  • D.7. Reward weight ablation: The asymmetric ablation pattern treats skill entropy as structural shaping layered on top of a dominant answer signal.The passage states that this matches the design intent of the reward.
  • D.8. Comparison against the single-skill oracle: The single-skill oracle evaluates each step independently without the surrounding scenario, unlike the cross-skill evaluation using the full task and all per-step questions.This comparison tests whether Skill-Entropy RL improves underlying skill ability rather than only cross-skill-format handling.
  • D.8. Comparison against the single-skill oracle: Skill-Entropy RL achieves a higher overall Skill2-Bench score in the cross-skill setting than the base model achieves under the Single-Skill setting on both Qwen3-4B-Instruct and Qwen3-1.7B.This comparison is reported in Table 14.

D.9. Additional base models

Table 15 extends the evaluation to Llama-3.2-3B-Instruct and Olmo3-7B-Instruct under the same training setup, where Skill-Entropy RL achieves the highest overall Skill2-Bench score on both models.

  • Additional base models: Skill-Entropy RL attains the highest overall Skill2-Bench score on both additional base models.This matches the pattern observed for Qwen3-4B-Instruct and Qwen3-1.7B in Table 3.
  • Additional base models: Table 15 evaluates Llama-3.2-3B-Instruct and Olmo3-7B-Instruct using the same training data and configuration as Section 4.2.The results complement Table 3’s evaluation of Qwen3-4B-Instruct and Qwen3-1.7B.
  • Additional base models: Table 15 reports per-domain accuracy and overall Skill2-Bench scores for both Llama-3.2-3B-Instruct and Olmo3-7B-Instruct.The table uses the same training data and configuration as Table 3.

D.10. Generalization to external benchmarks · D.11. Worked example: labeling an off-the-shelf problem

Skill-Entropy RL generalizes beyond the authors’ pipeline, achieving the strongest external-benchmark results across model sizes while preserving general reasoning. A worked OpenR1-Math example shows how traces are segmented, skill-labeled, and converted into task-level skill entropy for reusable training.

  • D.10. Generalization to external benchmarks: Skill-Entropy RL matches or outperforms GRPO on all but one external-benchmark entry, indicating no compromise in general reasoning performance.The comparison covers the five external benchmarks evaluated under the same inference-time settings.
  • D.10. Generalization to external benchmarks: Skill-Entropy RL attains the highest overall average across five external benchmarks at both model sizes.The benchmarks include MuSR, LongBench-MuSiQue, GPQA-Diamond, MMLU, and IFEval.
  • D.10. Generalization to external benchmarks: Skill-Entropy RL achieves the best MuSR score at both scales and the best LongBench-MuSiQue score on Qwen3-4B-Instruct.This supports transfer of the skill-entropy reward to external multi-step reasoning benchmarks.
  • D.11. Worked example: labeling an off-the-shelf problem: The labeling pipeline segments a gold trace into reasoning steps, assigns each step a (domain, skill) label, and emits its intermediate conclusion as the gold answer.Labels are drawn from the Skill2-Bench skill bank.
  • D.11. Worked example: labeling an off-the-shelf problem: For the worked polynomial problem, the trace derives roots of f, solves f(x) = k for k in {1, 2, 3}, and obtains the final root sum 18.Each shifted cubic has root sum 6 by Vieta, yielding 3 * 6 = 18.
  • D.11. Worked example: labeling an off-the-shelf problem: The example produces the skill sequence factoring → polynomial_equations → vieta_formulas, whose task-level skill entropy is computed from the pairwise skill table.The sequence uses the labels (math, factoring), (math, polynomial_equations), and (math, vieta_formulas).
  • D.11. Worked example: labeling an off-the-shelf problem: The same labeling procedure runs over a balanced 6K OpenR1-Math subset matched in size to the Skill2-Bench RL split.Out-of-bank labels are routed to the nearest canonical key by embedding lookup, preserving the entropy-table vocabulary.
Loading 2608.05139v1…