PaperlayerView exact revision on arXiv

Source-linked AI summary

EmbodiExp: Compositional Experience Transfer for Embodied Agents

Zhifeng Gu, Ying Chen, Rui Jiang, Lei Yi, Bing Wang

arXiv:2610.39180v1cs.ROcs.AIcs.CVcs.LG

TL;DR

Embodied agents may plan what to do without knowing how to ground those plans under current environmental conditions, and experience transfer remains challenging. EmbodiExp represents experience at the subgoal level and composes applicable experiences online using verified outcomes. On EB-Habitat and EB-ALFRED, it exceeds the strongest baselines by 10.0 and 9.0 percentage points, respectively.

  • Problem

    Embodied agents can plan what to achieve, but grounded execution requires selecting actions suited to current conditions and verifying their outcomes.

    Source: §1 · p. 1, §3.1 · p. 3

  • Method

    EmbodiExp represents experiences as subgoals with applicability conditions, adaptable policies, and intended outcomes, composing them online when verified outcomes establish subsequent conditions.

    Source: §1 · p. 2

  • Results

    EmbodiExp achieves 68.0% success on EB-Habitat and 68.3% on EB-ALFRED, exceeding the strongest baselines by 10.0 and 9.0 percentage points, respectively.

    Source: §4.1 · p. 7

  • Takeaways & Limitations

    Subgoal-level attribution allows failed trajectories to contribute verified local successes and condition-specific guidance on operations to avoid.

    Source: §3.2.2 · p. 6

    The evaluation is simulation-based; real-world robotic interaction is a further direction.

    Source: D LIMITATIONS AND FUTURE WORK · p. 21

Abstract

from arXiv · show

Experience can help embodied agents turn high-level plans into grounded actions, but transferring it across tasks remains challenging: previously effective strategies may be inapplicable to the current state or fail to produce their intended outcomes. We present EmbodiExp, a framework for constructing transferable experiences and composing them online during embodied task execution. Our key observation is that tasks often share subgoals whose completion can be assessed independently of overall task success. We therefore introduce an outcome-grounded subgoal experience that specifies a subgoal, applicability conditions, an adaptable policy, and a verifiable outcome. During execution, applicable experiences guide action generation, while verified outcomes can establish the conditions required by subsequent experiences, enabling online composition. Subgoal-level outcome attribution further enables failure utilization: failed trajectories can still contribute verified local successes and condition-specific guidance on actions to avoid. Experiments on multiple embodied benchmarks under a fold-based protocol with frozen experience banks demonstrate the effectiveness of EmbodiExp.

1 INTRODUCTION

EmbodiExp treats subgoals as transferable units, grounding each experience in when it applies and what outcome verifies completion. Verified outcomes support online composition across trajectories, while local outcome attribution preserves useful guidance from failed tasks.

1 INTRODUCTION

  • The framework addresses transferability: experience must extend beyond its source interaction while remaining grounded in the new task’s conditions and outcomes.Tasks can share local goals despite different overall goals and execution sequences, but a previously effective strategy may not apply or achieve its intended outcome in new conditions.Source: §1 · p. 1
  • EmbodiExp represents each experience with a subgoal, applicability conditions, an adaptable policy, and an intended local outcome.Experiences are retrieved for the current subgoal and applied only when their conditions are satisfied; completion requires interaction evidence for the intended outcome.Source: §1 · p. 2
  • Verified outcomes from one experience can satisfy another’s applicability conditions, connecting experiences from different trajectories through realized state transitions.Retrieval alone does not determine when an experience becomes applicable as execution unfolds.Source: §1 · p. 2
  • Subgoal-level attribution retains locally successful behavior from failed trajectories and derives condition-specific guidance on actions to avoid.This uses local action-outcome evidence rather than overall task outcomes, avoiding both the loss of useful local knowledge and indiscriminate transfer of unsuccessful behavior.Source: §1 · p. 2
  • The paper reports evaluations on multiple benchmarks using a fold-based protocol with frozen experience banks.The motivating example illustrates composing experiences from different previous tasks to complete a new task with shared subgoals.Source: §1 · p. 2, Figure 1 · p. 2

2 RELATED WORK

Related work covers embodied reasoning and control, agent memory, and experience or skill learning. EmbodiExp focuses on extracting outcome-grounded subgoal experiences and composing them across tasks according to task state.

2 RELATED WORK

3 METHOD

EmbodiExp represents experience through subgoals, applicability conditions, adaptable policies, and verifiable outcomes, then composes locally supported experience online as execution state changes. Its failure-utilization mechanism retains evidence-backed local successes and condition-specific avoidance guidance from failed trajectories.

3.1 PROBLEM FORMULATION

  • Each subgoal experience couples a goal with applicability conditions, execution guidance, and a locally verifiable outcome, addressing the gap between semantic planning and grounded execution.Completion is credited only when interaction evidence supports the intended outcome; an unverified attempt is not necessarily a failure.Source: §3.1 · pp. 3–4

3.2 EMBODIEXP

  • Experience construction segments source trajectories into subgoal attempts and admits guidance supported by local evidence, while execution retrieves applicable entries from a frozen bank to guide actions.The executor updates task state with newly supported facts after checking outcomes.Source: §3.2 · p. 4

3.2.1 EXPERIENCE COMPOSITION

  • Online composition uses verified outcomes to update task state, which may establish conditions for subsequent experiences without requiring a direct PRODUCES-to-REQUIRES match.An experience is used only when its requirements are supported and its intended outcome is not already established; other supported state changes can still be incorporated when completion is unverified.Source: §3.2.1 · pp. 5–6

3.2.2 FAILURE UTILIZATION

  • Subgoal-level attribution lets failed trajectories contribute verified effective behavior and condition-specific avoidance guidance, with entries grounded in evidence for the relevant operations.Unverified completion alone is not evidence of failure, and a completed attempt may include both successful behavior and failed operations.Source: §3.2.2 · p. 6

4 EXPERIMENTS

The experiments evaluate EmbodiExp against memory and experience-learning baselines on EmbodiedBench and VIGIL, using frozen experience banks in a three-fold protocol. Results cover benchmark performance, experience granularity and composition, failure utilization, and additional transfer analyses.

4.1 MAIN RESULTS

  • 68.0% on EB-Habitat and 68.3% on EB-ALFRED, exceeding the strongest baselines by 10.0 and 9.0 percentage points, respectively.The qualitative cases illustrate verified outcomes enabling dependent subgoals and applicability allowing independent subgoals to be reordered.Source: Table 1 · p. 7, §4.1 · p. 7, Figure 3 · p. 8
  • 40.8% overall world-state completion on VIGIL exceeded NoExp by 9.2 percentage points and ReMe by 4.4 percentage points.The results support outcome-grounded subgoals beyond high-level skill interfaces.Source: Table 2 · p. 7, §4.1 · pp. 7–8

4.2 ABLATION STUDY

  • Subgoal-level experience achieved 58.3% success, versus 54.0% for trajectory-level and 54.7% for action-level experience.On Long-Horizon tasks, subgoal-level experience raised success from 40% to 46%, while trajectory- and action-level experience provided no gain over NoExp.Source: §4.2 · p. 8
  • Combining applicability checks and outcome verification achieved 68.0% success, compared with 58.3% for similarity-based composition, 64.3% for applicability only, and 56.7% for verification only.The combined components had their largest gain on Long-Horizon tasks.Source: §4.2 · pp. 8–9
  • Including failed source trajectories raised success from 62.7% to 68.0%, with the largest Long-Horizon gain rising from 54% to 70%.Subgoal-level attribution retains verified local successes and condition-specific guidance on actions to avoid.Source: §4.2 · p. 9

4.3 ADDITIONAL ANALYSIS

  • Both experience banks improved both executors, with the 8B executor reaching 52.3–55.0% success, above the 35B NoExp baseline.Constructor rankings reversed across executors: the Qwen bank performed better with 35B, while the GPT bank performed better with 8B.Source: §4.3 · p. 9, Figure 4 · p. 9
  • Across 300 paired episodes, EmbodiExp converted 62 NoExp failures into successes (20.7%) and regressed on 8 successes (2.7%), for a net gain of 18.0 percentage points.It recovered 41.3% of baseline failures while preserving 94.7% of baseline successes, though transfer still reduced performance on some tasks.Source: §4.3 · p. 9
  • State-grounded cross-source composition occurred in 28.0% of Long-Horizon episodes versus 11.7% overall.This criterion requires verified local facts from earlier experience applications to remain valid in the Task State and support later prerequisites.Source: §4.3 · p. 9, Figure 4 · p. 9

5 CONCLUSION

EmbodiExp represents experience as outcome-grounded subgoals and composes experiences online through verified outcomes. Subgoal-level attribution also extracts execution and avoidance guidance from local successes and failures, and fold-based experiments on multiple benchmarks demonstrate effectiveness.

5 CONCLUSION

  • EmbodiExp represents each experience with a subgoal, applicability conditions, an adaptable policy, and an intended local outcome.Source: §5 · p. 9
  • Verified outcomes can make subsequent experiences applicable, while subgoal-level attribution derives execution and avoidance guidance from local successes and failures.Source: §5 · p. 10
  • Fold-based experiments on multiple embodied benchmarks with frozen experience banks demonstrate EmbodiExp’s effectiveness.Source: §5 · p. 10

A.1 BENCHMARKS AND TASK CATEGORIES

The evaluation spans EmbodiedBench’s high-level skill environments and VIGIL’s more atomic native-action tasks. Their agent interfaces differ in action controls, observations, and feedback, while experience decisions rely on accessible interaction evidence rather than evaluator-side success labels.

A.1 BENCHMARKS AND TASK CATEGORIES

  • Evaluation covers EB-Habitat and EB-ALFRED, plus VIGIL’s Search and Interact and Sequential Manipulation task families.The settings cover high-level skill interfaces and more atomic native actions.Source: §A.1 · p. 15
  • EB-Habitat tests navigation and object rearrangement, while EB-ALFRED tests household tasks; each has 300 episodes across six capability categories.VIGIL uses 250 episodes: 125 Search and Interact and 125 Sequential Manipulation episodes.Source: §A.1 · p. 15

A.2 OBSERVATION AND ACTION INTERFACES

  • EmbodiedBench provides visual observations, high-level skills, and textual action feedback, whereas VIGIL uses RGB observations and task-specific native controls without action-success feedback.Source: §A.2 · p. 15
  • Task State St is agent-maintained from accessible interaction evidence; evaluator goal predicates and success labels are not online completion signals.Source: §A.2 · p. 15

A.3 FOLD-BASED EVALUATION PROTOCOL

The protocol separates experience construction from evaluation using three rotating folds, with each episode evaluated once. Experience banks are built only from construction-fold trajectories and frozen during held-out evaluation.

A.3 FOLD-BASED EVALUATION PROTOCOL

  • Three disjoint folds rotate the held-out evaluation set, so each episode is evaluated once and other folds supply source trajectories.Fold assignment is outcome-blind and keeps predefined episode groups intact.Source: §A.3 · p. 15
  • Table 4 describes evaluated environments and reports counts for complete pools partitioned into construction and held-out evaluation folds.Source: Table 4 · p. 16
  • EB-Habitat and EB-ALFRED use grouped episode folds, while VIGIL keeps scenes intact and uses scene-disjoint folds.EB-Habitat and EB-ALFRED are not fully scene-disjoint or task-template-disjoint.Source: §A.3 · p. 16
  • Each run’s experience bank excludes its held-out fold’s trajectories and outcomes; the held-out unit is the episode.Source: §A.3 · p. 16
  • The bank remains frozen throughout evaluation, preventing evaluation interactions from changing cross-episode experience entries.Episode-local state and plans may still change, and are reset between episodes.Source: §A.3 · p. 16
  • All compared methods use the same fold assignments and held-out episodes.Source: §A.3 · p. 16

A.4 EVALUATION METRICS

EmbodiedBench scores complete task success, while VIGIL reports terminal world-state goal satisfaction. Results pool held-out outcomes across folds, and benchmark scoring is independent of the agent’s internal outcome verification.

A.4 EVALUATION METRICS

  • EmbodiedBench task success requires satisfying the complete task condition; individual subgoal completion alone does not count.Source: §A.4 · p. 16
  • VIGIL’s world-state completion W measures whether the semantic goal holds at episode closure, without requiring a correct terminal report.It does not measure whether the goal was reached earlier in the episode.Source: §A.4 · p. 16
  • Success rates pool held-out episode outcomes across folds, with each episode counted once.Avg. is the arithmetic mean of the six EmbodiedBench category scores or the SI and SM scores for VIGIL.Source: §A.4 · p. 17
  • Benchmark success is determined by the environment evaluator, independently of the agent’s internal outcome-verification decisions.Source: §A.4 · p. 17

B IMPLEMENTATION DETAILS

EmbodiExp builds experiences from locally assessed outcomes, then retrieves and applies them only when current evidence supports their requirements. Verified outcomes update task state for later experience selection, while unsupported or unavailable guidance leaves the agent free to replan.

B.1 SOURCE TRAJECTORY COLLECTION

  • Source trajectories are collected once per construction-fold episode under NoExp, recording instructions, observations, actions, available feedback, and final outcomes.Both successful and unsuccessful trajectories are used, with local outcomes attributed from interaction evidence so verified local successes can survive overall task failure.Source: §B.1 · p. 17

B.2 EXPERIENCE CONSTRUCTION

  • Construction segments trajectories into locally assessable attempts, verifies completion from local evidence, and abstracts supported attempts into experiences separating subgoal, applicability conditions, policy, and intended outcome.An attempted action sequence alone does not verify completion; locally successful behavior remains usable even if a later task part fails.Source: §B.2 · p. 17

B.3 EXPERIENCE REPRESENTATION

  • The experience representation distinguishes REQUIRES, which supports applying a policy, from PRODUCES, the local outcome assessed after execution.The four fields are subgoal, requires, policy, and produces.Source: §B.3 · p. 18, Table 6 · p. 18

B.4 RETRIEVAL AND EXPERIENCE SELECTION

  • At each subgoal decision, retrieval ranks experiences by semantic similarity, then selects the highest-ranked candidate whose requirements are supported and whose intended outcome is not already established.Retrieval and applicability are separate checks, and selection is repeated as interaction evidence changes.Source: §B.4 · p. 18

B.5 STATE MAINTENANCE, VERIFICATION, AND FALLBACK

  • The executor maintains an episode-local evidence-supported task state, verifies outcomes after adaptable policy execution, and uses supported updates to enable later experience selection or replanning.If no experience is usable, or an attempt is unsuccessful or unverified, the agent can continue with native planning and select a different experience as execution proceeds.Source: §B.5 · pp. 18–19

C.1 DETAILED QUANTITATIVE RESULTS

  • EmbodiExp outperforms all baselines in every fold on all three benchmarks under the grouped three-fold evaluation protocol.Each episode is evaluated once, using an experience bank constructed from the other two folds and frozen during evaluation.Source: Table 7 · p. 19, §C.1 · p. 19

C.2 EFFICIENCY ANALYSIS

On EB-Habitat, EmbodiExp improves task success while reducing environment steps relative to NoExp, at the cost of more model usage. Qualitative EB-ALFRED examples show progress through locally verified outcomes where NoExp repeats failed actions.

C.2 EFFICIENCY ANALYSIS

  • Task success rises from 50.0% to 68.0% and average environment steps fall from 15.96 to 13.67 with EmbodiExp versus NoExp.These changes accompany approximately 2.38× as many logical model calls and 2.04× as many reported tokens.Source: §C.2 · p. 19
  • The efficiency trade-off is higher model-side usage alongside higher task completion and fewer environment interactions.The reported model-call and token increases include additional experience-selection and local-outcome-verification calls.Source: §C.2 · p. 19
  • On EB-ALFRED, EmbodiExp uses verified local outcomes to continue toward placement, while NoExp repeats slicing or invalid pickups after failures.The examples include retrieving heated food and acquiring a knife before proceeding through later subgoals.Source: Figure 5 · p. 20

C.3 ADDITIONAL QUALITATIVE RESULTS

Selected EB-ALFRED and VIGIL examples show EmbodiExp using verified intermediate outcomes to continue across subgoals or recover from ineffective interactions, while NoExp often repeats actions or stalls.

C.3 ADDITIONAL QUALITATIVE RESULTS

  • On EB-ALFRED, EmbodiExp proceeds from verified slicing or heating toward placement, while NoExp repeats slicing or invalid pickups.The examples illustrate intermediate state transitions across long-horizon manipulation tasks.Source: §C.3 · p. 20
  • On VIGIL, EmbodiExp connects possession or target visibility to later pickup or placement and retries fridge opening after moving within reach.NoExp stalls during navigation or interaction, or repeats unsuccessful opening attempts.Source: §C.3 · p. 20
  • In a VIGIL box example, both agents open the box, but EmbodiExp completes card pickup while NoExp reports completion after an unverified pickup.The example ties subsequent action to verification of the open-container prerequisite.Source: §C.3 · p. 20

D LIMITATIONS AND FUTURE WORK

The paper identifies boundaries in bank evolution, control level, and source-experience diversity, and presents prompt examples for offline construction and online execution. The current evaluation uses fixed experience banks in simulated task-level settings.

D LIMITATIONS AND FUTURE WORK

  • Evaluation freezes the experience bank, leaving continual acquisition, refinement, and application by self-evolving agents for future work.Experience selection and composition remain online, but bank evolution through subsequent interactions is not examined.Source: D LIMITATIONS AND FUTURE WORK · p. 20
  • The experiments cover simulated task-level execution, not experience transfer within low-level VLA policies or real-world robotic interaction.Extending evaluation to finer-grained control interfaces is also identified as future work.Source: D LIMITATIONS AND FUTURE WORK · pp. 20–21
  • Benchmark-specific source-trajectory pools under a controlled collection protocol do not characterize transfer from substantially larger, more heterogeneous interaction collections.The paper proposes expanding trajectory scale and diversity, including interactions collected by different agents and policies.Source: D LIMITATIONS AND FUTURE WORK · p. 21

E.1 OFFLINE EXPERIENCE CONSTRUCTION

  • Offline construction prompts segment attempts, extract candidate experiences, and review evidence for admission, retaining supported effective behavior or condition-specific avoidance guidance.The examples instruct the constructor to use local evidence regardless of trajectory-level success or failure and to preserve necessary applicability conditions.Source: §E.1 · pp. 21, 23

E.2 ONLINE EXPERIENCE-GUIDED EXECUTION

  • Online execution prompts maintain task state, select the highest-ranked applicable experience, adapt its policy to current observations, and verify its intended outcome before updating state.If no candidate applies, the actor proceeds without experience guidance; an unverified outcome does not by itself establish failure.Source: §E.2 · pp. 24, 26, 28

Related papers

Loading 2610.39180v1…