PaperlayerView exact revision on arXiv

Source-linked AI summary

EmbodiedCLBench: Evaluating Continual Learning for Self-Evolving Embodied Agents

Jie Huang, Yanan Chen, Ruixun Liu, Kaichen He, Xinyi He, Rui Huang, Zilong Zheng, Zhenliang Zhang, Yiwu Zhong

arXiv:2610.13021v1cs.AIcs.CLcs.CV

TL;DR

Continual learning in embodied environments lacks a dedicated benchmark, despite agents needing to transfer experience to new tasks. EmbodiedCLBench tests that capability by having agents learn from basic tasks and recombine skills on unseen advanced tasks. The experiments find that basic-task experience improves advanced-task performance, but gains do not consistently grow with more samples.

  • Problem

    Existing continual-learning benchmarks largely focus on non-embodied settings, while embodied benchmarks are not specifically designed to evaluate continual learning.

    Source: §1 · p. 1

  • Method

    EmbodiedCLBench evaluates continual learning through compositional generalization, testing whether agents can transfer skills learned on basic tasks to unseen advanced tasks.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    Experience from basic tasks substantially improves performance on unseen advanced tasks, but the benefit does not consistently increase with the amount of experience.

    Source: §1 · p. 2, §6 · p. 10

  • Takeaways & Limitations

    How to enable agents to accumulate and exploit additional interactions remains an open problem for self-evolving embodied agents.

    Source: §6 · p. 10

    Performance across harnesses is close yet far from saturation, and neither In-Context Learning nor Skill Learning has a consistent advantage.

    Source: §4.2 · p. 7

Abstract

from arXiv · show

Continual learning requires models to accumulate past experience and transfer it to new tasks. Despite substantial research on continual learning, existing work remains largely confined to non-embodied settings, without extending to embodied environments. To address this gap, we introduce EmbodiedCLBench, the first benchmark in embodied simulation for evaluating the continual learning capability of self-evolving agents. With compositional generalization as the core evaluation principle, our benchmark evaluates whether agents can first learn from basic tasks and then recombine the acquired skills to solve complex, unseen advanced tasks. Based on EmbodiedCLBench, we conduct extensive experiments across multiple agent harnesses and learning methods, revealing consistent limitations and distinctive behaviors. Our results show that while advanced tasks can be hardly solved under zero-shot setting, the experience from basic tasks enables consistent improvement. Such benefit from the experience remains stable even if the task complexity increases. However, this improvement does not scale accordingly with the number of learning samples, and current agents cannot extract meaningful information from additional experience. Collectively, our findings highlight continual learning as a fundamental yet underexplored capability, and our benchmark offers valuable resources for designing effective self-evolving agents in embodied environments. Our code and dataset are available at https://pku-value-lab.github.io/EmbodiedCLBench-homepage.

1 INTRODUCTION

The paper addresses the lack of a dedicated benchmark for evaluating continual learning in embodied environments. EmbodiedCLBench uses compositional generalization to test whether experience from basic tasks transfers to unseen advanced tasks, and experiments find that it helps while gains do not consistently grow with more experience.

1 INTRODUCTION

  • Continual-learning benchmarks largely omit embodied settings, while embodied benchmarks are not designed to measure learning and transfer across tasks.Embodied tasks add spatial complexity and physical constraints that non-embodied platforms do not capture.Source: §1 · p. 1
  • EmbodiedCLBench tests whether agents recombine skills learned from basic tasks to solve unseen advanced tasks, rather than learning each task from scratch.Its three levels are atomic actions, basic household tasks, and advanced tasks combining elements from multiple basic-task categories.Source: §1 · p. 2, Figure 1 · p. 2
  • Experience from basic tasks substantially improves success on unseen advanced tasks, but gains do not consistently increase with more learning samples.The benefit remains stable as task complexity increases, and experience also supports more meaningful action choices with fewer unexecutable ones.Source: §1 · p. 2
  • The paper evaluates multiple agent harnesses and learning methods to identify current limitations and inform future agent design.Its stated contributions include using compositional generalization to construct tasks, design evaluation protocols, and assess continual learning.Source: §1 · p. 2

2 RELATED WORK

Prior continual-learning benchmarks chiefly cover digital or symbolic domains, while embodied benchmarks assess capabilities such as perception, navigation, manipulation, and planning. Compositional generalization is studied across domains, but this work evaluates continual learning in embodied simulation without benchmark-specific training.

2 RELATED WORK

  • Continual-learning benchmarks cover knowledge-intensive tasks, interactive digital environments, long-term simulations, and skill-centric evolution, but chiefly use digital or symbolic settings.They leave continual learning in embodied scenarios largely unexplored.Source: §2 · p. 3
  • Embodied benchmarks cover perception, navigation, manipulation, and long-horizon planning, but none is designed to evaluate continual learning of skills in long-horizon tasks.Recent work also examines general-purpose embodied-agent evaluation and realistic simulation environments.Source: §2 · p. 3
  • Compositional generalization has been studied in digital agents, video understanding, reasoning, and embodied manipulation; EmbodiedCLBench evaluates it for continual learning in simulation without benchmark-specific training.Source: §2 · p. 3

3 BENCHMARK CONSTRUCTION

EmbodiedCLBench is constructed around compositional generalization: agents learn from basic tasks before being tested on unseen, more complex tasks. The benchmark uses household simulation tasks, state-transition graphs, and an interactive evaluation protocol.

3.1 ENVIRONMENT SETUP: EMBODIED SIMULATION

  • TongSim provides indoor household scenes with diverse objects and actions, supporting combinatorial task diversity and grounding tasks in household activities.The authors report it has not been widely used as training data for existing foundation models, reducing data-contamination risk.Source: §3.1 · pp. 3–4

3 BENCHMARK CONSTRUCTION

  • Compositional generalization operationalizes continual learning by testing whether skills acquired on basic tasks help solve novel tasks instead of requiring learning from scratch.The benchmark construction is organized into environment setup, task generation, graph construction, and evaluation protocol.Source: §3 · p. 3, §3.2 · p. 4

3.2 TASK GENERATION: COMPOSITIONAL GENERALIZATION

  • Task levels build from single object interactions to practical household operations and then more complex tasks combining more atomic actions.Task descriptions are generated by Qwen3.7-Plus and manually verified for correctness.Source: §3.2 · p. 4

3.3 GRAPH CONSTRUCTION: ACTION TRAJECTORY ANNOTATION

  • State-transition graphs pair each environment state with an image and verified action candidates; shortest paths define task routes, while executable detours form distractor paths.Because the closed-source simulation makes online evaluation impractical, the authors use offline graphs that allow parallel evaluation, complexity control, and fine-grained behavior analysis.Source: §3.3 · pp. 4–5

3.4 EVALUATION PROTOCOL

  • Agents first complete basic tasks to accumulate experience, then attempt unseen advanced tasks; each task is evaluated through interactive, multi-turn decisions.At each step, agents choose among optimal, executable-but-suboptimal, and invalid actions, continuing until success or the step budget is exhausted.Source: §3.4 · p. 5

4 MAIN EXPERIMENTS

EmbodiedCLBench evaluates continual learning through agents’ transfer of experience from basic tasks to unseen advanced tasks. Across harnesses, learning from basic tasks raises advanced-task success, though neither learning method consistently outperforms the other.

4.1 EXPERIMENTAL SETUP

  • EmbodiedCLBench contains 268 tasks: 168 basic tasks across six categories and 100 advanced tasks combining actions from two to four basic-task categories.Basic tasks have two atomic actions and six shortest-path steps; advanced tasks have four atomic actions and 12 shortest-path steps, with three steps per atomic action.Source: §4.1 · pp. 5–6, Figure 3 · p. 5
  • ICL retains full trajectories from two related basic tasks, while Skill Learning provides a Markdown skill summary from those same tasks for advanced-task testing.Zero-shot agents attempt basic and advanced tasks without a learning phase.Source: §4.1 · p. 6
  • Four harnesses are paired with two backbone models, and success rate measures task completion within the allotted step budget as an indicator of continual learning.Testing allows three extra steps beyond the shortest path; learning allows six extra steps for exploration.Source: §4.1 · p. 6

4.2 RESULTS

  • 10% to above 30%: ICL and Skill Learning raise advanced-task success rate consistently across harnesses; zero-shot basic-task success is around 50%.The results also characterize zero-shot advanced tasks as difficult to solve.Source: Table 1 · p. 6, §4.2 · pp. 6–7
  • Neither ICL nor Skill Learning has a consistent advantage, and performance across harnesses remains far from saturation.The authors identify continual-learning harness design as a future direction.Source: §4.2 · p. 7

5 ANALYSIS

The analysis finds that basic-task experience improves performance on harder tasks, but gains do not keep increasing with more samples. It also examines how complexity, decision behavior, and step budgets affect success.

5.1 EFFECTS OF COMPOSITIONAL COMPLEXITY

  • At 6 atomic actions, zero-shot success is 4.0%, while ICL and Skill Learning reach 32.0% and 26.0%; the gain over zero shot stabilizes around 30%.Success declines as complexity increases, with the largest drop between 3 and 4 atomic actions; learning gains remain comparatively stable.Source: §5.1 · p. 7, Figure 4 · p. 7

5.2 SCALING WITH THE NUMBER OF LEARNING SAMPLES

  • A single learning sample raises success from 8.0%–10.0% to 28.0%–40.0%, but none of four harness-method curves improves consistently with additional samples.Most of the total improvement occurs between k=0 and k=1; scalable continual learning remains an open challenge.Source: §5.2 · p. 8, Figure 5 · p. 8

5.3 ERROR ANALYSIS

  • Experience reduces invalid-action selections by 13%–20%, increases optimal actions, and slightly increases suboptimal actions by 3%–8%.Skill Learning produces smaller decision-behavior shifts than ICL, although the methods show no consistent success-rate advantage over one another.Source: Figure 6 · p. 8, §5.3 · p. 9

5.4 STEP BUDGET SENSITIVITY

  • At 0 extra steps, zero-shot success is 0.0%, versus 15.0% for ICL and 7.0% for Skill Learning; ICL ranks above Skill Learning and zero shot at every budget.Success rises with extra steps in all settings, and experienced agents increase more steeply and retain a substantial advantage even at 6 extra steps.Source: §5.4 · p. 9, Figure 7 · p. 9

6 CONCLUSION

EmbodiedCLBench evaluates continual learning in embodied simulation by testing whether agents can recombine experience from basic tasks to solve unseen advanced tasks. Evaluations find transfer benefits alongside limits in extracting further gains from additional experience.

6 CONCLUSION

  • EmbodiedCLBench organizes atomic actions, basic tasks, and advanced tasks into a protocol for measuring whether agents acquire, retain, and transfer experience.The benchmark uses compositional generalization: agents accumulate experience from basic tasks and recombine skills for unseen, more complex tasks.Source: §6 · p. 9
  • Experience from basic tasks substantially improves performance on unseen advanced tasks over zero-shot execution, but benefits do not consistently increase with more experience.The conclusion identifies exploiting additional interactions as an open problem for current agents.Source: §6 · p. 10
  • The benchmark provides a testbed for studying self-evolution and highlights the need for stronger mechanisms to consolidate, generalize, and reuse experience across tasks.Source: §6 · p. 10

A BASIC TASK CATEGORIES

Table 2 covers six capability categories of basic tasks and identifies a core evaluation question for each category.

A BASIC TASK CATEGORIES

B GRAPH CONSTRUCTION PIPELINE

The graph-construction pipeline builds task graphs from TongSim atomic actions, generates task descriptions and per-state action options, and retains samples only after unanimous human approval.

B GRAPH CONSTRUCTION PIPELINE

  • Given TongSim atomic actions, the pipeline first constructs a state-transition graph and then generates task descriptions and per-state action options.Source: B GRAPH CONSTRUCTION PIPELINE · p. 15
  • A three-stage Qwen3.7-Plus pipeline turns atomic actions into household scenarios and prevents action options from revealing the correct answer through shortcuts.Source: B GRAPH CONSTRUCTION PIPELINE · p. 15
  • Three annotators independently check images, task descriptions, and action options for factual errors, logical consistency, and realistic household activity; all must approve a sample for retention.Source: B GRAPH CONSTRUCTION PIPELINE · p. 15

B.1 PROMPT SPECIFICATIONS

The prompt specifications rewrite household tasks and action options to test whether agents use scene information rather than linguistic shortcuts. They harden options within each state and across a task graph while preserving the underlying task structure.

Stage 1: Task Description Rewrite

  • Task descriptions combine atomic L1 goals into coherent L2 basic or L4 advanced scenarios without listing objects, exposing order, or changing the task structure.The rewrite replaces specific object names with functional or contextual references.Source: §B.1 · pp. 15–16

Stage 2: Per-State Option Rewrite

  • Each state's four options should sound like plausible next steps, so choosing the optimal action requires the scene image and current state rather than linguistic cues.Options represent an optimal success path, an executable task-relevant detour, or an action invalid in the current state.Source: §B.1 · p. 16
  • Options are constrained to fit both the sub-goal and high-level scenario while concealing action phases and keeping wording, style, and length similar.Invalid options should still read as plausible and must not explain why they cannot be executed.Source: §B.1 · p. 16

Stage 3: Cross-State Option Hardening

  • Cross-state hardening targets recurring phase rhythms, style biases, and repeated phrasing that an agent could exploit using its interaction history.Rewrites blur phase boundaries, mix phase styles in distractors, vary expression, and modify Stage 2 text without changing option metadata.Source: §B.1 · p. 17

C TASK EXAMPLES

The examples show agents receiving a natural-language task, observing the current-state image, and choosing among candidate actions. They illustrate this format for basic and advanced tasks, including options with different hidden graph roles.

C TASK EXAMPLES

  • The evaluation format gives two basic-task examples (Figures 8 and 9; Tables 3 and 4) and one advanced-task example (Figure 10; Table 5).Source: C TASK EXAMPLES · p. 17
  • In one basic-task example, the shortest-path choices include approaching the desk and opening its cabinet, alongside invalid options and an executable detour.The example labels these hidden roles across states 4 and 5.Source: C TASK EXAMPLES · p. 21
  • Later basic-task states illustrate shortest-path actions alongside invalid choices and executable detours, including closing a bedside cabinet and moving a remote control toward a trash can.The options span multiple steps rather than presenting only a single decision.Source: C TASK EXAMPLES · p. 21
Loading 2610.13021v1…