PaperlayerView exact revision on arXiv

Source-linked AI summary

From Context to Skills: Can Language Models Learn from Context Skillfully?

Shuzheng Si, Haozhe Zhao, Yu Lei, Qingyi Wang, Dingwei Chen, Zhitong Wang, Zhenhailong Wang, Kangyang Luo, Zheng Wang, Gang Chen, Fanchao Qi, Minjia Zhang, Maosong Sun

arXiv:2604.27660v4cs.AI

TL;DR

Context learning requires language models to reason from complex, previously unseen contexts, but constructing useful skills without costly annotation or external feedback remains difficult. Ctx2Skill uses self-evolving multi-agent self-play and Cross-Time Replay to discover, refine, and select context-specific skills, consistently improving solving rates across CL-Bench tasks and backbone models.

  • Problem

    Context learning requires language models to learn from diverse, complex contexts, yet effective skill construction for this setting remains limited.

  • Method

    Ctx2Skill uses skill-optimized multi-agent self-play and Cross-Time Replay to autonomously discover, refine, and select context-specific skills without human annotation or external feedback.

  • Results

    Ctx2Skill consistently improves solving rates across four CL-Bench task categories and multiple backbone models, raising GPT-4.1 from 11.1% to 16.5%.

  • Takeaways & Limitations

    The resulting context-specific skills improve context learning across backbone models and task categories and are transferable across models.

  • Takeaways & Limitations

    The framework targets inference-time skill augmentation without parameter updates or external feedback and keeps the Challenger and Reasoner strictly adversarial.

Abstract

from arXiv · show

Many real-world tasks require language models (LMs) to reason over complex contexts that exceed their parametric knowledge. This calls for context learning, where LMs directly learn relevant knowledge from the given context. An intuitive solution is inference-time skill augmentation: extracting the rules and procedures from context into natural-language skills. However, constructing such skills for context learning scenarios faces two challenges: the prohibitive cost of manual skill annotation for long, technically dense contexts, and the lack of external feedback for automated skill construction. In this paper, we propose Ctx2Skill, a self-evolving framework that autonomously discovers, refines, and selects context-specific skills without human supervision or external feedback. At its core, a multi-agent self-play loop has a Challenger that generates probing tasks and rubrics, a Reasoner that attempts to solve them guided by an evolving skill set, and a neutral Judge that provides binary feedback. Crucially, both the Challenger and the Reasoner evolve through accumulated skills: dedicated Proposer and Generator agents analyze failure cases and synthesize them into targeted skill updates for both sides, enabling automated skill discovery and refinement. To prevent adversarial collapse caused by increasingly extreme task generation and over-specialized skill accumulation, we further introduce a Cross-time Replay mechanism that identifies the skill set achieving the best balance across representative cases for the Reasoner side, ensuring robust and generalizable skill evolution. The resulting skills can be plugged into any language model to obtain better context learning capability. Evaluated on four context learning tasks from CL-bench, Ctx2Skill consistently improves solving rates across backbone models.

1 Introduction

Ctx2Skill addresses the difficulty of learning from diverse, complex contexts by autonomously discovering, refining, and selecting natural-language skills without human annotation or external feedback. Its self-play framework improves context-learning solving rates across four CL-Bench tasks and multiple backbone models.

  • Motivation: Real-world tasks require language models to learn new knowledge from complex contexts rather than rely solely on parametric knowledge.Contextual knowledge may come from books, experimental data, and search results, making context learning fundamentally difficult.
  • Challenges: Context-specific skill construction is hindered by the prohibitive cost of manual annotation and the absence of external feedback.Without feedback, generated skills cannot be evaluated for usefulness, fidelity, completeness, or omitted critical knowledge.
  • Method: Ctx2Skill autonomously discovers, refines, and selects skills directly from complex contexts through skill-optimized self-play, requiring neither human annotation nor external feedback.The framework uses a multi-agent loop with competing, co-evolving roles, including a Challenger that generates probing tasks and rubrics.
  • Evaluation: Ctx2Skill consistently improves solving rates across four CL-Bench context-learning categories and multiple backbone models.The evaluated categories are Domain Knowledge Reasoning, Rule System Application, Procedural Task Execution, and Empirical Discovery & Simulation.
  • Results: 11.1% to 16.5%: GPT-4.1 solving rate; 21.1% to 25.8%: GPT-5.1; 18.2% to 21.4%: GPT-5.2.These results demonstrate gains from autonomously discovered context-specific skills across the reported backbone models.

2 Related Work

Context learning enables language models to acquire and use relevant knowledge from complex contexts for effective reasoning. Natural-language skills augment models with reusable procedural knowledge, but human-annotated skill libraries do not scale to long, technically dense, domain-specific contexts.

  • Context Learning: Context learning concerns acquiring relevant knowledge from complex contexts and using it to reason effectively.It is needed for tasks such as applying newly released clinical guidelines or following technical documentation.
  • Skills for LMs: Natural-language skills encode reusable procedural knowledge that augments language models at inference time.Skill augmentation has been validated across agent tasks including coding and web navigation.
  • Skills for LMs: Human annotation dominates early skill-library construction but does not scale to long, technically dense, domain-specific contexts.This limitation is especially important for context learning scenarios.

3 Methodology

Ctx2Skill autonomously constructs context-specific skills without human annotation, external feedback, or parameter updates by co-evolving Reasoner and Challenger skills through multi-agent self-play. Cross-time replay selects a balanced Reasoner skill set from historical candidates for robust, generalizable inference-time use.

  • Motivation: Ctx2Skill addresses costly manual annotation and missing external feedback by discovering and refining context-specific skills from the context alone.The framework targets unseen contexts where contextual knowledge must be extracted without human supervision or an external feedback signal.
  • Self-play framework: The framework co-evolves Reasoner skill set SR and Challenger skill set SC through failure-driven textual edits across N iterations.At inference time, only SR is deployed with a language model for context-specific task evaluation.
  • Self-play framework: Each self-play iteration generates probing tasks and rubrics, solves them with SR, judges every rubric, and routes failed or solved cases to update the corresponding side.Failed cases inform Reasoner skill updates, while solved cases tighten Challenger skills to sustain adversarial pressure.
  • Agent roles: Proposer–Generator pairs diagnose shared failure or success patterns and materialize them into complete replacement skill sets while preserving unrelated entries.The Reasoner pair incorporates missing contextual knowledge, whereas the Challenger pair identifies gaps in task and rubric generation.
  • Cross-time replay: Cross-time replay selects the most generalizable Reasoner skill set from historical candidates by re-evaluating them on representative probe tasks and balancing performance across cases.The resulting skills can be prepended to an LM’s system prompt at inference time without parameter updates.

4 Experiments

Experiments on CL-bench show that Ctx2Skill improves context-learning solving rates across backbone models and produces higher-quality skills than AutoSkill4Doc. Ablations further identify Challenger evolution and Cross-Time Replay as key components, while generated skills can transfer across backbones asymmetrically.

  • Evaluation: Ctx2Skill is evaluated on CL-bench, which tests learning unseen contextual knowledge for reasoning beyond pre-training, rather than only retrieval or simple task-pattern learning.The evaluation uses CL-bench as the context-learning benchmark.
  • Main Results: 5.4%, 4.7%, and 3.2% are the overall solving-rate gains for GPT-4.1, GPT-5.1, and GPT-5.2, respectively, with Ctx2Skill outperforming Prompting and AutoSkill4Doc across all four categories.Ctx2Skill raises GPT-4.1 from 11.1% to 16.5%, GPT-5.1 from 21.1% to 25.8%, and GPT-5.2 from 18.2% to 21.4%.
  • Skill Quality: Ctx2Skill achieves the highest average generated-skill quality across all three backbones, exceeding AutoSkill4Doc by 3.6, 2.1, and 2.3 points on GPT-4.1, GPT-5.1, and GPT-5.2.GPT-4.1 judges skills on conciseness, faithfulness, clarity, effectiveness, and reusability.
  • Ablation Study: Removing Challenger skill evolution causes the largest ablation drop, while removing Cross-Time Replay produces adversarial collapse in later-iteration skill sets.Cross-Time Replay is the second most impactful component, and its hard probe set contributes more than the alternative within that mechanism.
  • Effect of Cross-Time Replay: 16.5% from Cross-Time Replay exceeds every fixed iteration on GPT-4.1, including Iter-1 at 15.9%, while fixed-iteration performance declines to 14.7% by Iter-5.Cross-Time Replay improves over the best fixed iteration by 0.6% through adaptive selection of a balanced skill set.
  • Skill Transferability Testing: GPT-5.1-generated skills transfer to GPT-4.1 at 16.1%, whereas GPT-4.1-generated skills transfer to GPT-5.1 at 23.1%, below the 4.6% gain from GPT-5.1’s own skills.The reported transfer results indicate stronger-model skills transfer well to weaker models, while the reverse is less effective.

5 Conclusion

Ctx2Skill is a self-evolving framework that autonomously discovers, refines, and selects context-specific skills from complex contexts without human annotation or external feedback. Its skill-optimized self-play loop co-evolves Challenger and Reasoner skills, while Cross-Time Replay selects generalizable skill sets to prevent adversarial collapse.

  • Conclusion: Ctx2Skill autonomously discovers, refines, and selects context-specific skills from complex contexts without human annotation or external feedback.The framework is designed for skill evolution without either manual skill annotation or external feedback.
  • Conclusion: A skill-optimized self-play loop co-evolves Challenger and Reasoner skill sets through failure-driven textual edits.Both agents update their skills based on failure cases during self-play.
  • Conclusion: Cross-Time Replay prevents adversarial collapse by selecting the most generalizable skill set across iterations.The mechanism evaluates skill sets over time and retains the one with the strongest generalizability.

Appendix · A Statistics

The appendix reports CL-bench statistics and shows that Ctx2Skill sustains adversarial self-play while progressively increasing task, answer, rubric, and skill-set complexity. These dynamics motivate Cross-Time Replay’s selection of a generalizable skill set rather than simply using the final iteration.

  • A Statistics: CL-bench evaluates whether language models learn knowledge from provided contexts and apply it to solve tasks involving diverse, often niche or newly created information.The contexts include books, journalism, transcripts, research papers, documents, reports, experimental data, and code repositories.
  • A Statistics: On GPT-4.1, the average solved rate rises from 18.2% at Iter-1 to 23.3% at Iter-5, while Cross-Time Replay avoids defaulting to the last iteration.Across 500 contexts, self-play reports average solved and failed tasks per iteration; the observed pattern motivates selecting the most generalizable skill set.
  • A Statistics: Failed rates remain 51%–82% across all three backbones and iterations, indicating that self-play avoids trivial convergence and maintains adversarial pressure.The sustained failures support continuous skill discovery rather than collapse into universally easy tasks.
  • A Statistics: Rubric pass rates of 79%–91% substantially exceed task solved rates of 18%–49%, showing that all-or-nothing scoring can penalize one or two missed requirements.On GPT-4.1, the rubric pass rate steadily improves across iterations, despite lower task-level solved rates.
  • A Statistics: Median rubric counts stay at 11–12, while mean counts rise from 11.7 to 12.3 on GPT-4.1 and from 11.2 to 12.0 on GPT-5.2.Tasks with zero rubrics, comprising less than 1%, are excluded; the trend indicates increasingly fine-grained Challenger verification criteria.
  • A Statistics: GPT-5.1’s maximum rubric count is substantially higher while its rubric pass rate remains above 90%, highlighting its strength as a Reasoner backbone.On GPT-4.1 and GPT-5.2, increasing mean rubric counts indicate harder tasks with expanding requirement sets.
  • A Statistics: Mean Challenger task length increases monotonically: GPT-4.1 grows from 46.3 to 59.4 words, while GPT-5.2 nearly doubles from 69.1 to 139.1 words.The GPT-4.1 increase is described as a moderate rise in task specificity, whereas GPT-5.2 shows more dramatic growth.
  • A Statistics: Mean Reasoner output length increases monotonically from 209.4 to 322.2 words on GPT-4.1, 216.9 to 312.3 on GPT-5.2, and 337.6 to 399.6 on GPT-5.1.GPT-5.1 starts with the longest outputs and has the smallest relative growth, while shorter-starting backbones exhibit larger growth as skills evolve.

B Implementation Details

Ctx2Skill is implemented as an N-step multi-agent self-play pipeline in which task generation, solving, judging, and skill updates are coordinated across agents. Cross-Time Replay selects the final Reasoner skill set, while the evaluation uses constrained iteration and task budgets.

  • Key Prompts: The Challenger generates M evaluation tasks with binary rubrics, while the Judge applies strict all-or-nothing grading through requirement analysis, rubric verification, and self-reflection.The Judge does not participate in skill evolution.
  • Algorithm: Each of N self-play iterations routes failed tasks to the Reasoner Proposer–Generator and passed tasks to the Challenger Proposer–Generator, producing updated skill sets.The Challenger generates tasks, the Reasoner solves them, and the Judge evaluates the rubrics before routing outcomes.
  • Algorithm: Cross-Time Replay re-evaluates every Reasoner skill set on accumulated hardest-failure and easiest-success probe sets, selecting the best iteration as the final skill set.The hardest failure and easiest success from each iteration are accumulated into probe sets Qh and Qe before final selection.
  • Limitations and Future Work: N=5 iterations and M=5 tasks per iteration are used because of API budget constraints, while evaluation covers all 500 CL-bench contexts without error bars or confidence intervals.Larger budgets and multiple independent runs remain unexplored.

C Case Study

The case study compares GPT-5.1 responses with and without Ctx2Skill skills on a CL-bench task involving a heterogeneous context exceeding 172K characters. The governing rulebook occupies roughly 33–48% of the context, while the remaining documents serve as distractors.

  • C Case Study: The case study evaluates GPT-5.1 with Ctx2Skill skills against GPT-5.1 without skills.The comparison is presented in Figure 5.
  • C Case Study: The CL-bench context is a heterogeneous collection exceeding 172K characters.The paper describes this as the context used in the case study.
  • C Case Study: The governing rulebook occupies roughly 33–48% of the context, with the remaining documents acting as distractors.The retained figure shows only the system prompt, user turn, and opening context portion, which is distractor material.

D Discussion

Ctx2Skill’s gains are not explained by reward hacking, test-set contamination, or generic answering strategies. Analyses instead indicate that its improvements arise from genuinely learned, context-specific knowledge.

  • Conclusion: The combined analyses confirm that Ctx2Skill’s gains come from genuinely learned, context-specific knowledge rather than judge exploitation, test contamination, or generic answering strategies.The discussion explicitly evaluates all three concerns and reports that none holds.
  • Reward Hacking: Ctx2Skill’s improvements remain nearly identical when GPT-5.2 replaces GPT-5.1 as the self-play Judge, indicating they do not depend on matching the evaluation judge.GPT-5.1 remains the CL-bench evaluation judge in this analysis.
  • Test Set Contamination: Challenger-generated tasks share no 50-gram or 100-gram with CL-bench test tasks within the same context, indicating no task-generation leakage.The analysis uses the GPT-5.1-based setting and measures word-level n-gram overlap.
  • Context-Specificity: 16.5% matched-skill performance falls to 12.5% with mismatched skills, while the base model achieves 11.1%, showing that most improvement is context-specific.Mismatched performance is close to the 12.3% single-pass Prompting baseline, suggesting only a small residual contribution from generic strategies.
Loading 2604.27660v4…