PaperlayerView exact revision on arXiv

Source-linked AI summary

SkillRise: Agentic Reinforcement Learning for Cross-Task Skill Evolution

Zhiyuan Yao, Yuxin Chen, Zhengxi Lu, Zishan Xu, Yueqing Sun, Yifu Guo, Yuquan Lu, Zhengzhou Cai, Kangning Zhang, Zhuowen Han, Zi-Han Wang, Ziang Ye, Qi Gu, Xunliang Cai, Weiwen Liu, Yongliang Shen

arXiv:2607.26784v1cs.LGcs.AI

TL;DR

Agentic RL often treats related tasks independently, while skill-learning approaches leave cross-task reuse insufficiently addressed. SkillRise trains one policy to solve tasks and curate evolving skills across sequences, achieving the strongest Pass@1 across three benchmarks and improving with longer test-time sequences.

  • Problem

    Standard agentic RL treats tasks as independent episodes, while multi-stage skill pipelines entangle extraction, retrieval, and execution, leaving cross-task skill learning insufficiently addressed.

    Source: §1 · pp. 1–2

  • Method

    SkillRise uses one reinforcement-learning policy to solve related tasks and curate an evolving skill document for subsequent tasks, with separate credit assignment for solving and curation.

    Source: §1 · p. 2, §3.2 · p. 4, §3.3 · p. 4

  • Results

    SkillRise achieves strongest overall Pass@1 across ALFWorld, WebShop, and ScienceWorld, beating GiGPO by 2.3, 7.1, and 8.5 percentage points, respectively, and improves with longer test-time task sequences.

    Source: §1 · p. 2, §4.2 · p. 5

  • Takeaways & Limitations

    The results support cross-task self-improvement, with skills acquired on earlier tasks systematically improving performance on later tasks.

    Source: §1 · p. 2

    Experiments are limited to models up to 4B parameters and three text-based benchmarks with verifiable rewards, leaving broader multimodal and real-world tasks untested.

    Source: §8 · p. 9

Abstract

from arXiv · show

Large language model agents often encounter related yet distinct tasks that share reusable solution patterns. Yet standard agentic reinforcement learning treats tasks as independent episodes, while existing approaches to skill learning either focus on repeated attempts of one task or use pipelines with multiple stages that entangle extraction, retrieval, and execution. We introduce SkillRise, a unified reinforcement learning framework for learning skills across tasks. SkillRise organizes related instances into progressively challenging sequences and uses a single policy to alternate between task solving and curating an evolving skill document passed directly to the next task. Decoupled credit assignment across tasks supervises solving with the current task outcome and curation with discounted downstream outcomes. Experiments on ALFWorld, WebShop, and ScienceWorld show that SkillRise achieves the strongest Pass@1 performance among the compared methods, with gains over the strongest baseline ranging from 2.3 to 8.5 percentage points. Although trained across distinct tasks, its learned curation policy remains effective for repeated attempts on the same task. Further analysis reveals scaling at test time across tasks: performance improves with longer sequences of related tasks even when each task is attempted only once. This trend suggests that SkillRise reuses transferable skills across tasks rather than benefiting from repeated sampling of the same task. SkillRise further retains strong performance while substantially reducing the runtime overhead of skill learning pipelines with multiple stages. Together, these results provide a simple and efficient training paradigm for LLM agents to extract, refine, and reuse transferable skills across tasks.

1 INTRODUCTION

SkillRise addresses repeated rediscovery across related tasks by training one policy to solve tasks and curate an evolving skill document, using later-task outcomes to assess transfer.

1 INTRODUCTION

  • Standard agentic RL discards experience across task episodes, while repeated-task learning can yield instance-specific skills and skill-bank pipelines obscure whether extraction caused downstream success.Repeated attempts also require additional interactions and are not directly optimized for cross-task transfer.Source: §1 · pp. 1–2
  • Prior approaches improve repeated attempts on one task (Jiang et al., 2025) or maintain skill banks for future tasks (Xia et al., 2026; Wu et al., 2025), but have the respective limitations described above.The skill-bank approach makes it difficult to attribute downstream success or failure to skill-extraction quality.Source: §1 · pp. 1–2
  • SkillRise orders similar but distinct tasks by difficulty and uses one policy to alternate between solving each task and curating a skill document for the next.The document evolves from interaction trajectories throughout the task sequence.Source: §1 · p. 2
  • Its decoupled credit assignment uses the current task reward for solving and discounted subsequent-task rewards for curation, with group-relative advantages computed by task group, sequence stage, and behavioral phase..Source: §1 · p. 2
  • Across ALFWorld, WebShop, and ScienceWorld, SkillRise achieves the best overall results among compared methods, with within-task generalization and cross-task scaling absent in competing methods.It also offers a better effectiveness-efficiency balance than multi-stage skill-learning pipelines.Source: §1 · p. 2

2 PRELIMINARIES

The preliminaries model an LLM agent solving tasks through environment interactions and define standard reinforcement learning as maximizing expected task reward, treating tasks independently.

2 PRELIMINARIES

  • At each step, an LLM agent selects an action from its policy using the interaction history, receives an observation, and then observes the environment’s response.The task instruction is sampled from a task distribution D.Source: §2 · p. 2
  • An episode yields a trajectory that ends upon task completion or exhaustion of the interaction budget and is evaluated by a verifiable outcome reward.Source: §2 · p. 3
  • The standard objective maximizes expected task reward over tasks and policy-generated trajectories, treating each task independently.The following section extends this objective to related task sequences connected through an evolving skill state.Source: §2 · p. 3

3 METHOD

SkillRise sequences related tasks and uses one policy to alternate between solving each task and curating an evolving skill document. Its learning signals separately optimize task solving and cross-task skill transfer.

3.1 CROSS-TASK SEQUENCE CONSTRUCTION

  • Tasks are grouped into sequences of distinct instances from one family and ordered from simpler to more complex, allowing later-task performance to test transfer of shared routines.Sequences are constructed by grouping diverse instances within each task family; for example, WebShop requests are ordered by the number of required attributes and options.Source: §3.1 · pp. 3–4

3.2 CROSS-TASK ROLLOUT WITH SKILL EVOLUTION

  • For each task, the policy solves using the current skill document, then rewrites it to retain useful skills, consolidate successes or failures, and remove instance-specific details.Only the revised document passes to the next task; earlier trajectories are not carried forward, making the document the sole cross-task information channel.Source: §3.2 · p. 4

3.3 DECOUPLED CREDIT ASSIGNMENT AND OPTIMIZATION

  • Solving receives the current task reward, while curation receives discounted rewards from subsequent tasks, with nearer tasks weighted more heavily.The same policy jointly optimizes solving and curation responses while keeping their phase-specific learning signals separate.Source: §3.3 · p. 4

4 EXPERIMENTS

SkillRise achieves the strongest overall Pass@1 across ALFWorld, WebShop, and ScienceWorld, and its cross-task-learned skill curation also improves repeated attempts on the same task.

4.1 EXPERIMENTAL SETUP

  • Experiments evaluate 128 held-out instances per benchmark across ALFWorld, WebShop, and ScienceWorld, against prompting, task-independent RL, and LaMer baselines.Baselines include Zero-shot, ReAct, Reflexion, PPO, RLOO, GRPO, GiGPO, and LaMer; each trainable method processes 384 task plays per update.Source: §4.1 · p. 5

4.2 MAIN RESULTS

  • 85.9% ALFWorld, 84.4% WebShop, and 54.6% ScienceWorld Pass@1 beat GiGPO by 2.3, 7.1, and 8.5 percentage points, respectively.On ALFWorld, SkillRise ranks first or second in five of six task families. The authors attribute its gains to cross-task training that exposes the policy to different realizations of shared routines, reinforcing transferable skills in its parameters.Source: §4.2 · p. 5
  • 92.2% ALFWorld, 96.1% WebShop, and 61.0% ScienceWorld Pass@3 exceed the strongest baseline by 0.8, 1.6, and 5.5 points, respectively.SkillRise also achieves the best Pass@2 and Pass@3 on all three benchmarks, and its Pass@3 results exceed LaMer by 0.8, 1.6, and 14.2 points, respectively. This tests carrying an evolving skill document across retries of the same held-out task.Source: §4.2 · p. 6, Table 2 · p. 6

5 ANALYSIS

SkillRise’s analysis shows that performance improves with longer sequences of related tasks and larger model backbones, that explicit skill curation contributes to its gains, and that its compact pipeline retains strong performance with lower runtime.

5.1 CROSS-TASK TEST-TIME SCALING

  • SkillRise improved from 83.6% at K = 2 to 87.5% at K = 6, widening its margin over the strongest baseline from 3.9 to 8.6 percentage points.Each task was attempted once; the passage attributes the trend to transferable skills from earlier tasks and progressive refinement of the skill document, while LaMer, GiGPO, and GRPO showed no sustained improvement as K grew.Source: Figure 2 · p. 6, §5.1 · p. 6

5.2 PERFORMANCE ACROSS MODEL SIZES

  • At Qwen3-1.7B, SkillRise reached 78.1%, 3.1 percentage points above the strongest baseline; at Qwen3-4B, it reached 85.9%, a 6.2-point margin.SkillRise gained 7.8 points from 1.7B to 4B, compared with 4.7 for GRPO and 3.3 for LaMer; the passage suggests added capacity helps with task execution and cross-task abstraction.Source: Table 3 · p. 7, §5.2 · p. 7

5.3 ABLATION STUDIES

  • At the end of training, SkillRise led its no-curation variant by roughly three percentage points and GRPO by more than six points.The no-curation variant kept K = 3 task sequences and the same interaction budget but removed inter-task curation and skill transfer. Across γ ∈ {0.3, 0.4, 0.6, 0.7}, learning trajectories were closely aligned and converged within approximately one percentage point, indicating robustness to discount-factor choice.Source: §5.3 · p. 7, Figure 3 · p. 8

5.4 PIPELINE COMPLEXITY AND TRAINING EFFICIENCY

  • 85.9% average success rate on ALFWorld matched RetroAgent and exceeded SkillRL by 12.5 percentage points, while RetroAgent and SkillRL took 6.0× and 4.3× SkillRise’s runtime.SkillRise uses one policy to solve tasks, curate a sequence-local skill document, and apply it to the next task, without an external teacher, separate memory maintenance, or retrieval module.Source: Figure 4 · p. 8, §5.4 · p. 8

6 RELATED WORK

Prior work advances agentic reinforcement learning and develops increasingly reusable experience and skill representations, but learning procedural knowledge across related episodes remains underexplored. Multi-stage skill pipelines also make it difficult to attribute downstream performance to individual components.

Reinforcement Learning for LLM Agents

  • RL for LLMs has expanded from preference alignment to verifiable rewards and multi-turn agentic decisions, where long horizons complicate exploration and credit assignment (Shao et al., 2024; Guo et al., 2025; Yao et al., 2026; Chen et al., 2026; Zhang et al., 2025).Related approaches develop hierarchical, step-level, or milestone-guided credit assignment and improve exploration through interaction and curricula.Source: §6 · pp. 8–9
  • Most methods still optimize independently sampled task episodes, leaving how agents can learn from episode relationships and convert earlier interactions into reusable procedural knowledge underexplored.Outcome-based supervision can improve long-horizon interaction, but does not by itself address this cross-episode learning gap.Source: §6 · p. 9

Experience and Skill Learning in LLM Agents

7 CONCLUSION

SkillRise trains a single policy to solve related tasks and curate transferable skills across ordered task sequences. It achieves the strongest overall performance among compared methods and retains strong performance with one sixth of the running time.

  • SkillRise achieves the strongest overall performance among compared methods across ALFWorld, WebShop, and ScienceWorld.Source: §7 · p. 9
  • The pipeline comparison finds that SkillRise maintains strong performance while requiring only one sixth of the running time.Source: §7 · p. 9
  • A single policy alternates between solving ordered, related tasks and curating an evolving skill document, with credit assigned to current-task outcomes for solving and downstream outcomes for curation.Source: §7 · p. 9
  • SkillRise generalizes from cross-task training to repeated attempts and improves at test time with longer sequences of related tasks.Source: §7 · p. 9

8 LIMITATION

SkillRise’s evaluation is limited by its reliance on task-family metadata, experiments with models up to 4B parameters, and a focus on three text-based benchmarks with verifiable rewards.

  • The framework assumes task-family metadata to construct related-instance sequences; automatically discovering task relationships in open-ended streams remains future work.Source: §8 · p. 9
  • Experiments cover models only up to 4B parameters because of computational constraints, leaving larger-scale evaluation for future work.Source: §8 · p. 9
  • Evaluation focuses on three text-based agent benchmarks and verifiable outcome rewards; broader multimodal, real-world, and less readily verifiable tasks remain to be tested.Source: §8 · p. 9

A PROMPTS

The appendix reproduces runtime-filled user-message templates for task solving and skill curation, with prompts for ALFWorld, WebShop, and ScienceWorld. At the start of each task sequence, the skill document is empty.

A PROMPTS

B CASE STUDY

The ALFWorld cases illustrate how SkillRise carries an evolving skill document across tasks, distilling a general lesson from failure and incrementally revising a workflow after success. They demonstrate update processes, not causal effects.

B CASE STUDY

  • In both rollouts, the skill document starts empty, and only its updated version—not earlier task trajectories—is carried between tasks.The examples show the first two tasks of ALFWorld training rollouts with sequence length K = 3.Source: B CASE STUDY · p. 17
  • After failing to place two spraybottles, the agent adds a rule to repeat the procedure for each instance; the next task uses it to place two peppershakers successfully.The next task involves a different object, and the case shows downstream use of the failure-derived rule.Source: B CASE STUDY · p. 17
  • After a successful task, curation adds an “open the destination if closed” condition to the existing placement workflow.The next task required opening a closed microwave, but this case documents the revision rather than downstream use of the new condition.Source: B CASE STUDY · p. 18, Figure 12 · p. 18

Related papers

Loading 2607.26784v1…