PaperlayerView exact revision on arXiv

Source-linked AI summary

Experience-Guided Initiation Search for Learned Skills in Skill Composition

Qixuan Li, Yanhong Zhao, Jincheng Yu

arXiv:2610.11418v1cs.RO

TL;DR

In new environments, the initiation state of a frozen skill can affect both its own success and whether later skills can proceed, while target-specific testing is costly and historical experience may transfer unreliably. EVIS uses historical experience to prioritize initiation candidates and target rollouts to validate them. It reduces target interaction against a fixed-budget variant and improves confirmed reliable success over direct history reuse.

  • Problem

    Finding reliable initiation configurations for frozen skills is difficult because target capability is costly to estimate with extensive rollouts and historical candidates may not transfer reliably across environment changes.

    Source: §I · p. 1

  • Method

    EVIS uses historical execution experience to prioritize promising initiation configurations, then uses target-environment behavioral evidence to validate candidates and guide further search.

    Source: §I · p. 1, §VI · p. 8

  • Results

    EVIS uses 8.125 queries on average, reducing target interaction by 59.4% versus EVIS-Fixed B20 with identical confirmation outcomes; Confirmed SR rises from 70.8% to 76.7% versus History Reuse B5.

    Source: B. Initiation Search on Unseen Target… · pp. 6–7

  • Takeaways & Limitations

    Historical experience can be reused effectively without assuming perfect transfer, reducing target rollouts while improving reliable candidate discovery under limited interaction budgets.

    Source: §VI · p. 8

Abstract

from arXiv · show

Deploying frozen learned skills, such as Vision-Language-Action (VLA) policies, in new environments requires identifying initiation configurations that support reliable execution. In skill composition, an initiation configuration affects not only the current skill but also the physical state passed to subsequent skills, so successful execution of an individual skill does not necessarily imply successful completion of the composed task. Estimating target-specific capability through extensive rollouts is costly in real-world deployment, while directly reusing historical experience can be unreliable under environment changes. We propose EVIS, an Experience-Guided and Behavior-Validated Initiation Search framework for discovering reliable initiation configurations under limited target interaction. EVIS uses historical execution experience to prioritize promising candidates and target-environment behavior to validate whether they remain effective. We evaluate EVIS on single-skill and two-stage manipulation tasks with frozen VLA policies. EVIS reduces mean target-environment queries and improves reliable candidate discovery under small interaction budgets. These results show that combining historical guidance with target-side behavioral validation can reduce the interaction cost of deploying frozen learned skills in new environments.

I. INTRODUCTION

For frozen learned skills, initiation configurations matter because upstream execution determines both local success and the physical state handed to later skills. EVIS uses historical experience to prioritize candidates, then validates them in the target environment; the paper evaluates this approach on single-skill and two-stage manipulation tasks.

I. INTRODUCTION

  • An upstream skill can succeed locally yet leave a physical state that prevents the next skill from completing, so composition requires choosing its initiation configuration.Opening a drawer, for example, does not guarantee successful retrieval or placement afterward.Source: §I · p. 1
  • EVIS treats historical execution experience as candidate guidance and uses target-environment behavior to decide whether a candidate is reliable or search should continue.The framework is evaluated with a frozen pretrained VLA policy on RoboCasa365 single-skill and two-stage manipulation tasks.Source: §I · pp. 1–2
  • On unseen environments, historical guidance improves early candidate discovery under small budgets, while behavioral validation filters unreliable transfers and continued search recovers some lost coverage.Two-stage tasks isolate handoff effects: one downstream skill is sufficient to test whether the upstream initiation state supports subsequent execution.Source: §I · p. 2

A. Skill Applicability and Composition

  • Skill-applicability and planning methods model applicability or feasibility, whereas EVIS searches for initiation configurations that support successful execution of a composed sequence under limited target interaction.The related work discusses TAMP, learned precondition models, skill chaining, and PDDLStream [5],, [7], [8],.Source: A. Skill Applicability and Composition · p. 2

B. Policy Orchestration and Execution Memory

  • Execution-memory systems guide primitive composition or policy routing, while EVIS transfers experience across environments specifically to guide initiation search and validate candidates with target rollouts.The related work identifies HarnessVLA [3] and RoboHarness [2] as examples of execution-memory systems.Source: B. Policy Orchestration and Execution… · p. 2

C. Experience-Guided Search and Candidate Discovery

EVIS seeks a reliable initiation configuration using historical experience and limited target feedback, rather than reconstructing the full target-specific success function. The formulation covers frozen skills in single- and multi-stage manipulation, where success for a composed task means completing the sequence from the states produced by preceding skills.

III. PROBLEM FORMULATION

  • The formulation searches initiation configurations for frozen learned skills in single-skill and multi-stage manipulation, with context describing environment conditions and configuration specifying the robot–environment starting state.A mobile-manipulation configuration may parameterize the robot base pose relative to a target object or fixture.Source: §III · p. 2
  • For skill composition, success means completing the full sequence from physical states produced by preceding skills, not merely completing the current skill.The success predicate G is binary over an execution trajectory.Source: §III · p. 2
  • Historical records contain evaluated configurations and binary outcomes from prior contexts, but they need not transfer reliably to a new target context.The target success probability is unknown, and each rollout provides a stochastic observation.Source: §III · pp. 2–3
  • The objective is to return a configuration in the latent reliable initiation set with as few target executions as possible, rather than reconstructing the full target-specific success function.The search is related to good-arm identification, but historical execution data can guide early queries while target observations validate transferred candidates.Source: §III · p. 3

IV. METHODS

EVIS divides initiation search into historical proposal, behavioral validation, and behavior-guided recovery. History orders where to search first, while target-environment evidence determines whether to return a candidate or keep searching.

IV. METHODS

  • EVIS separates historical guidance from target-side decision-making through experience-guided proposal, behavioral validation, and behavior-guided recovery.Previous executions prioritize candidates; target evidence supports returning a provisional candidate, while unsuccessful or insufficiently supported proposals trigger continued search.Source: §IV · p. 3
  • Historical experience guides the ordering of candidate evaluations, but target-environment behavior determines whether a candidate is trusted or further search is required.Source: §IV · p. 3
  • Behavior-guided recovery continues the search when transferred proposals are unsuccessful or lack sufficient behavioral support.Source: §IV · p. 3

A. Experience-Guided Proposal

EVIS uses multiple historical scoring hypotheses to prioritize initiation configurations, transferring their relative rankings rather than trusting their predictions as target-environment decisions. For multi-stage execution, it keeps local feasibility, conditional continuation, and full-sequence completion as distinct proposal signals.

A. Experience-Guided Proposal

  • EVIS parameterizes candidates relative to the target fixture and normalizes each dimension by its search range so historical experience can be compared across environments.Historical predictors are evaluated on a finite candidate set.Source: A. Experience-Guided Proposal · p. 3
  • Historical scoring hypotheses provide different views of candidate promise, and EVIS transfers their ordering rather than treating their predictions as target-environment decisions.Because hypotheses can differ in numerical scale and calibration, EVIS compares their relative preferences using normalized percentile ranks.Source: A. Experience-Guided Proposal · p. 4
  • Context-specific, pooled, and retrieval-weighted predictors offer complementary hypotheses, emphasizing individual-context structure, shared structure, or historically similar contexts.Neither pooled nor retrieval-based predictors are assumed to transfer reliably to every target environment.Source: A. Experience-Guided Proposal · p. 4
  • For multi-stage execution, EVIS keeps local feasibility, conditional continuation, and full-sequence completion as separate proposal signals because upstream suitability need not imply composed-task suitability.Continuation scores use only trajectories that reach the relevant intermediate event.Source: A. Experience-Guided Proposal · p. 4
  • The proposal set retains high-ranking, non-duplicate candidates from multiple hypotheses and interleaves ranked lists so no single hypothesis dominates query order.Any strictly increasing transformation of an individual score leaves its candidate ranking unchanged.Source: A. Experience-Guided Proposal · p. 4

B. Behavioral Validation and Budgeted Return

A successful discovery rollout makes an initiation configuration provisional, not immediately returnable. EVIS uses fresh target executions to qualify it, within the same interaction budget used for discovery.

B. Behavioral Validation and Budgeted Return

  • A successful target rollout produces a provisional candidate; EVIS requires additional fresh executions when online behavioral validation is used.The discovery rollout is excluded from qualification statistics.Source: B. Behavioral Validation and Budgeted… · p. 4
  • EVIS accepts after mq qualification successes, rejects when the remaining trials cannot reach mq, and otherwise continues while budget remains.The early stopping boundaries avoid executions that cannot change the final mq-of-nq decision.Source: B. Behavioral Validation and Budgeted… · p. 4
  • Discovery and online qualification share one target-interaction budget; EVIS returns an accepted candidate, recovers after rejection, or abstains if required support is not obtained before exhaustion.If a candidate is rejected, the remaining budget goes to recovery search.Source: B. Behavioral Validation and Budgeted… · pp. 4–5

C. Behavior-Guided Recovery Search

When historical proposals are insufficient or rejected, EVIS redirects search using target-side observations. Its recovery operators expand or refine the search while leaving the frozen execution policies unchanged.

C. Behavior-Guided Recovery Search

  • Target-side observations redirect the remaining search when historical proposals are insufficient or rejected.Recovery uses accumulated discovery history to change where subsequent queries go.Source: C. Behavior-Guided Recovery Search · p. 5
  • Hypothesis disagreement prioritizes candidates whose historical ranks differ while retaining candidates with plausible mean ranks, as an exploration heuristic rather than calibrated uncertainty.A positive ϵ prevents candidates with moderate mean rank from being completely suppressed.Source: C. Behavior-Guided Recovery Search · p. 5
  • Coverage selects the untested configuration farthest from the discovery set, while local exploration can query an untested neighbor of an observed success.Coordinates are normalized; nearby initiation states are selected to exploit their empirical tendency to share execution behavior.Source: C. Behavior-Guided Recovery Search · p. 5
  • Recovery can revisit historical rankings, explore disagreement, expand geometric coverage, exploit local success structure, or use regularized target correction.The corrected score can affect candidate ranking but is not a calibrated estimate of target success probability.Source: C. Behavior-Guided Recovery Search · p. 5
  • The procedure builds a historical proposal order, evaluates proposals with target executions, and triggers behavior-guided recovery after unsuccessful or rejected proposals until acceptance or budget exhaustion.The search can return abstention if no candidate is accepted within the available budget.Source: C. Behavior-Guided Recovery Search · pp. 5–6

V. EXPERIMENTS

The experiments assess initiation search on unseen contexts and separately analyze the mechanisms that make up EVIS. They measure target interaction alongside reliable candidate discovery for single-skill and multi-stage manipulation.

V. EXPERIMENTS

  • The evaluation tests whether historical guidance reduces target-environment interaction while maintaining reliable candidate discovery on unseen contexts.It covers both single-skill and multi-stage manipulation.Source: §V · p. 6
  • The experimental analysis measures initiation search across single-skill and multi-stage manipulation tasks.The stated focus is interaction use and reliable candidate discovery.Source: §V · p. 6
  • Controlled mechanism analyses isolate the roles of experience-guided proposal, behavioral validation, and recovery search.Source: §V · p. 6

A. Experimental Setup

The setup tests frozen VLA skills in RoboCasa365 across single-skill and two-stage tasks, with unseen target contexts held apart from historical experience. It defines initiation poses, task success, and measures of query cost and reliability.

A. Experimental Setup

  • Experiments use the frozen pretrained RLDX-1-FT-RC365 VLA policy in RoboCasa365 contexts specified by kitchen layout, visual style, and initialization.The policy parameters remain frozen throughout evaluation.Source: A. Experimental Setup · p. 6
  • Initiation configurations are robot-base pose offsets from the fixture, searched over bounded x, y, and yaw ranges for OpenDrawer and two OpenDrawer-plus-downstream-skill tasks.The downstream skill begins from the physical state produced at the drawer-accessibility condition; the environment is not reset, and success requires the full sequence.Source: A. Experimental Setup · p. 6
  • Historical contexts provide experience and determine predictors, proposal sources, and hyperparameters, while unseen target contexts are reserved for evaluation.The contexts vary in visual appearance and kitchen layout.Source: A. Experimental Setup · p. 6
  • In the representative single-skill example, EVIS observes a successful initiation at query 5, versus 12 queries for Scratch-Sobol.The figure illustrates historical guidance prioritizing a promising region in an unseen target context.Source: Figure 4 · p. 6
  • Mean Queries counts target rollouts through termination, while Confirmed SR and Reliable Rate measure confirmation success and contexts returning reliable candidates.Selective-return measures include False Accepts and Reliable Precision; Recovery Rate measures reliable returns after entering recovery.Source: A. Experimental Setup · p. 6

B. Initiation Search on Unseen Target Contexts

EVIS is evaluated on single-skill and multi-stage initiation search in unseen target contexts. Historical guidance improves discovery under limited interaction, while target-side outcomes assess candidate reliability and composed-task completion.

Single-skill discovery

Multi-stage initiation search

  • At B = 10, EVIS achieved Reliable Rates of 41.7% and 58.3% on the two skill sequences, versus 8.3% and 16.7% for Scratch-Sobol.The gap narrowed as additional target interaction became available.Source: B. Initiation Search on Unseen Target… · p. 7
  • For OpenDrawer→ChooseMeasuringCup, 37.5% of executions that completed OpenDrawer still failed downstream, so upstream success alone did not establish composed-task success.Multi-stage search compared EVIS with Scratch-Sobol under matched budgets, qualification rules, candidate pools, and independent confirmation; Reliable Rate measured successful candidate discovery, with Confirmed SR as a complementary execution-quality measure.Source: B. Initiation Search on Unseen Target… · p. 7

C. Mechanism Analysis: Propose, Validate, and Recover

The analysis separates proposal, validation, and recovery: historical experience prioritizes where to search, target-side behavior determines what to trust, and continued search restores coverage after failed transfer.

Experience-guided proposal

Behavioral validation

  • Reliable precision rose from 41.7% and 8.3% to 100% with 5-of-6 qualification, eliminating all false accepts in both tasks.This validation ablation did not replace rejected candidates, so the precision gain came with reduced return coverage.Source: C. Mechanism Analysis: Propose,… · p. 7, Table III · p. 8

Recovery after rejected reuse

VI. CONCLUSION

EVIS uses historical experience to guide initiation search and target-side behavioral evidence to assess candidates, improving discovery while reducing rollout requirements.

VI. CONCLUSION

  • EVIS discovers reliable initiation configurations for frozen learned skills under limited target-environment interaction.The framework uses historical execution experience to guide search and target-side evidence to decide whether candidates should be trusted and whether search should continue.Source: §VI · p. 8
  • Experiments show that EVIS improves reliable candidate discovery under limited interaction budgets and reduces target-environment rollouts required for deployment.Source: §VI · p. 8
  • The experiments support reusing historical experience without assuming perfect transfer to a new environment.Source: §VI · p. 8
Loading 2610.11418v1…