PaperlayerView exact revision on arXiv

Source-linked AI summary

Where Success Breaks: Failure-Boundary Learning for Robust Vision-Language-Action Models

Yanzhe Chen, Zhijun Cao, Mike Zheng Shou

arXiv:2609.06114v1cs.RO

TL;DR

SFT teaches successful behavior but offers little guidance about where closed-loop execution becomes unreliable. DLS uses digital-twin rollouts and progress-aware labels to shape flow policies, improving real-robot robustness over SFT and online RL baselines.

  • Problem

    SFT provides little signal about where successful behavior ceases to be reliable, leaving a gap in guidance on closed-loop breakdowns.

    Source: §1 · pp. 1–2

  • Method

    DLS builds a real-grounded prior, discovers failures through on-policy digital-twin rollouts, localizes progress with privileged-state labels, and shapes flow dynamics without action likelihoods or critics.

    Source: §1 · p. 2, §3.2.1 · p. 5

  • Results

    Across three real-robot tasks and two VLA backbones, DLS exceeds Real-only SFT trained with N=100 demonstrations by +17.8% on unseen conditions using half the real data.

    Source: §6 · p. 8

  • Takeaways & Limitations

    The paper concludes that a digital twin’s privileged state can make a policy’s own failure boundary a more informative generalization signal than additional expert demonstrations.

    Source: §6 · p. 8

    DLS assumes a sufficiently physically faithful digital twin and pre-existing simulation assets for each task object, limiting out-of-the-box use on novel objects.

    Source: §5 · p. 8

Abstract

from arXiv · show

Vision-language-action (VLA) models adapted through supervised fine-tuning (SFT) inherit a structural asymmetry: expert demonstrations teach the policy where success behavior lies, but provide no signal about where it ceases to be reliable. We argue that robust VLA adaptation should therefore be viewed not as further demonstration fitting, but as **Failure-Boundary Learning**---the problem of *Discovering*, *Localizing*, and *Shaping* the boundary between recoverable deviations and task failure. To instantiate this view, we propose **DLS**: built on a **real-grounded behavioral prior** from few real demonstrations and simulated co-training, DLS *discovers* failure boundaries at scale through on-policy digital twin rollouts. Rather than reducing each rollout to a binary label, **semantic progress localization** uses privileged simulator states to assign progress-aware signals that capture *where* the failure boundary is crossed, not merely *whether*. These signals drive **directional boundary shaping** in the flow dynamics---reinforcing success-producing denoising directions and suppressing failure-producing ones, without action likelihoods or auxiliary critics. Across real-robot manipulation tasks, DLS improves robustness over SFT and online RL baselines, especially under randomized initial states and unseen visual conditions.

1 Introduction

SFT teaches successful behavior but gives little guidance about where closed-loop actions become unreliable. The paper frames this gap as Failure-Boundary Learning and introduces DLS to discover, localize, and shape that boundary.

1 Introduction

  • SFT pulls policies toward expert actions but provides little signal about breakdowns or recovery after small errors move execution outside the demonstration distribution.The paper identifies the missing signal as where successful behavior ceases to be reliable, rather than insufficient exposure to successful behavior.Source: §1 · pp. 1–2
  • Failure-Boundary Learning targets where a policy’s closed-loop behavior shifts from recoverable deviation to task failure, including how far it progressed and which partial behaviors remain worth preserving.The framework calls for discovering self-generated failures, localizing breakdown by task progress, and shaping the policy away from failure-producing directions.Source: §1 · p. 2
  • Robust adaptation requires scalable on-policy failures, progress-localized supervision, and generation-aware updates that avoid unavailable action likelihoods and value functions.Manually curated failures may miss current-policy errors, while flow-based VLAs generate actions through multi-step denoising.Source: §1 · p. 2
  • DLS combines a few-real-demonstration, simulation-co-trained behavioral prior with digital-twin rollouts, privileged-state progress labels, and directional shaping of flow dynamics.Its signals distinguish clean successes, inefficient successes, and partial failures; shaping reinforces success-producing denoising directions and suppresses failure-producing ones.Source: §1 · p. 2

2 Related Work

Prior work addresses flow-policy optimization and simulation transfer through several distinct approaches. The paper positions DLS as combining progress-aware supervision with online co-training and simulator-provided state information.

2.1 Post-SFT Optimization for Flow-Based VLAs

2.2 Sim-to-Real Transfer for Manipulation Policies

  • Simulation transfer uses domain alignment [51] [53] [54], supervised co-training [55] [56] [58], or online co-training, but simulator privileged state remains underused as structured supervision.DLS builds on online co-training and uses the digital twin to generate labels, while the deployed policy relies on RGB and proprioception.Source: §2.2 · p. 3

3 Approach: Discover–Localize–Shape Pipeline

DLS first establishes a real-grounded prior, then uses on-policy simulation rollouts to discover failures, localize progress, and shape the flow policy. Its training signals use privileged simulator state while its shaping updates remain anchored to real demonstrations.

3.1 Stage 1: Real-Grounded Behavioral Prior

  • Stage 1 co-trains on limited real demonstrations and simulated demonstrations synthesized from five real seed trajectories with randomized initial object positions.The resulting real-grounded prior teaches nominal behavior and supports downstream exploration.Source: §3 · p. 3, §3.1 · p. 3

3 Approach: Discover–Localize–Shape Pipeline

  • The pipeline first establishes a real-grounded prior, then uses online exploration, semantic progress localization, and directional boundary shaping to move flow dynamics away from errors and toward robust success.The approach frames these two stages as an implementation of Failure-Boundary Learning.Source: §3 · p. 3

3.2.2 Directional Boundary Shaping (DBS)

  • Directional Boundary Shaping compares mirrored flow-velocity branches using Mahalanobis errors, steering toward success-producing directions and away from failure-producing ones without likelihoods or critics.Real-demonstration SFT regularization anchors the updates against forgetting foundational skills.Source: Figure 2 · p. 4, §3.2.2 · p. 5

3.2 Stage 2: Failure-Boundary Learning via Online Exploration

  • The prior policy explores on-policy in parallel simulation environments with Flow-SDE stochastic exploration, and stores transition tuples with SPL-assigned rewards.The rollouts run for at most K steps across N environments.Source: §3.2 · p. 4

3.2.1 Semantic Progress Localization (SPL)

  • SPL assigns phase-based credit from privileged simulator predicates, using a discrete gate for completed phases and continuous progress within each phase.Near-miss rollouts retain partial supervision, while the failure locus receives zero credit and later unvisited stages contribute no tuples.Source: §3.2.1 · pp. 4–5

4 Experiment

The experiments evaluate DLS across real-robot tasks, backbones, and conditions, including tests of progress localization, regularization, and demonstration scale. DLS reports higher average success across both backbones, with additional gains under unseen conditions and ablations favoring SPL.

4.1 Experimental Setup

  • The evaluation covers Pick & Place, Sweep, and Plugin, with 30 real-robot rollouts per policy under standard and unseen visual and position conditions.Baselines include Real-only SFT at N=50 or N=100 and few-shot SFT + RL at N=50.Source: §4.1 · p. 6

4.2 Main Results

  • DLS posts 84.4% / 72.2% (ID. / Unseen.) on π0.5 and 78.9% / 71.1% on π0, the highest average success across both backbones.Table 1 reports success rates across three tasks and ID. and Unseen. conditions, with each cell based on 30 real-world rollouts.Source: §4.2 · p. 6, Table 1 · p. 7
  • With 50 demonstrations, DLS exceeds Real-only SFT with 100 demonstrations by +17.8% absolute on π0.5 under unseen conditions.The reported comparison spans all tasks under unseen conditions.Source: §4.2 · p. 6
  • On Plugin with π0.5, DLS reaches 90.0% / 70.0% (ID. / Unseen.), gaining +33.3% / +36.7% over the Sim-Real SFT prior.Across most task–backbone combinations, unseen-condition gains exceed in-distribution gains.Source: §4.2 · p. 7

4.3 Ablation Studies

  • SPL produces monotonically increasing simulation success rates, converging faster and finishing higher than binary reward across all tasks.Binary reward treats barely successful and efficiently successful trajectories alike, while SPL localizes where failures occur.Source: §4.3 · p. 7, Figure 4 · p. 7
  • SPL improves success rate by over 6.7% while reducing average episode length comparably to the step-count penalty.Source: §4.3 · p. 8
  • On π0.5, unseen success stabilizes for λ in [0.5, 1.0] and drops sharply when λ≤0.2.Removing the SFT anchor causes catastrophic forgetting of the real-grounded prior; over-regularization mildly lowers performance.Source: §4.3 · p. 8, Figure 5 · p. 8
  • On Plugin, DLS exceeds Real-only SFT in unseen success at every tested real-demonstration count across both backbones.The reported ablation associates simulation-based on-policy failure discovery with compensation for sparse real-world coverage.Source: §4.3 · p. 8, Figure 6 · p. 8

5 Limitation

DLS depends on a sufficiently physically faithful digital twin, and its current object coverage requires simulation assets prepared in advance.

5 Limitation

  • Residual sim-to-real gaps may reduce SPL label quality, particularly for tasks with tight contact tolerances.The limitation is specifically tied to fidelity differences between the twin and the physical task.Source: §5 · p. 8
  • The pipeline requires pre-existing simulation assets for each task object.Extending it to novel objects would require 3D scanning or reconstruction, adding engineering overhead that limits out-of-the-box applicability.Source: §5 · p. 8
  • Novel-object use would require additional asset-construction work rather than out-of-the-box application.The passage identifies scanning or reconstruction as the needed extension.Source: §5 · p. 8

6 Conclusion

DLS combines on-policy digital-twin boundary discovery, progress localization, and flow shaping without likelihoods or critics. Across three real-robot tasks and two backbones, it reports higher success under unseen conditions than Real-only SFT, while its twin uses calibrated and perturbed camera views.

6 Conclusion

  • DLS reaches 84.4% / 72.2% (ID. / Unseen.) on π0.5 and 78.9% / 71.1% on π0, exceeding Real-only SFT with N=100 demonstrations by +17.8% on unseen conditions using half the real data.The conclusion reports these outcomes across three real-robot tasks and two VLA backbones.Source: §6 · p. 8
  • DLS discovers failures through on-policy digital-twin rollouts, localizes progress with SPL, and shapes flow dynamics with DBS without action likelihoods or critics.The pipeline is built on a real-grounded behavioral prior.Source: §6 · p. 8

A.1 Construction of Digital Twin

  • Figure 7 juxtaposes real observations, representative simulated rollouts, and task prompts for each task.The real and simulated visual observations are described as strongly aligned.Source: Figure 7 · p. 13, §A.1 · p. 13
  • The simulated camera uses real-world hand-eye calibration, with minor angle and position perturbations of ±1° and ±0.02 m.The visual alignment is reported to keep privileged-state predicates semantically consistent with deployment observations.Source: §A.1 · p. 13

A.2 SPL Phase Specifications Per Task

SPL assigns progress across ordered task phases and weights phase-level progress; across training, failure labels migrate toward later subgoals as success increases. The appendix also specifies control-frequency alignment and stage-two update settings.

A.2 SPL Phase Specifications Per Task

  • Pick & Place and Plugin use four phases, while Sweep uses five, including a Contact stage that checks broom–cube velocity alignment and sweep direction before Push.Phase entry is triggered by a gate predicate on privileged simulator state; within-phase progress measures movement toward the next gate.Source: §A.2 · p. 13, Table 3 · p. 14
  • SPL combines the current phase and within-phase progress into a stage-weighted potential, using cumulative prior-phase weights as the baseline.The Grasp phase receives the highest weight among non-final phases; near-miss trajectories entering the final phase without completing it receive only partial credit.Source: §A.2 · p. 14

A.3 Training Dynamics: Label Distribution over Training

  • As training proceeds, failure labels shift from early Grasp and Reach failures toward terminal Place, Insert, and Push subgoals, while the success band expands.Sweep retains a Transit component associated with resolving broom–cube alignment before the push boundary.Source: §A.3 · pp. 14–15, Figure 8 · p. 14

A.4 Hyperparameter

  • Stage-one demonstrations recorded at 20 Hz are temporally strided to 10 Hz, matching the digital-twin control frequency in stage two.DBS uses a global/micro batch of 6400/320 with five update epochs per rollout, yielding 40 optimizer steps per training step.Source: §A.4 · p. 15, Table 4 · p. 15

B.1 Qualitative Analysis on Real-Robot Rollouts

Real-robot rollouts under altered visual conditions and randomized initial positions show ordered task execution, broad spatial generalization, and preserved precision at task completion.

B.1 Qualitative Analysis on Real-Robot Rollouts

  • Under unseen conditions, DLS largely avoids intermediate breakdowns and maintains the correct phase order across Pick & Place, Sweep, and Plugin.The protocol changes the tablecloth and background, uses 560 Lux single-side illumination, and randomizes initial object positions.Source: §B.1 · pp. 15–16, Figure 9 · p. 16
  • Successful Pick & Place, Sweep, and Plugin rollouts span the randomized initialization space, including a 30×30 cm workspace for Pick & Place.The described behaviors include adapting broom–cube contact to varied offsets and aligning plug orientation before insertion from far-from-nominal poses.Source: §B.1 · p. 17
  • Final interactions remain precise under unseen conditions, including coaster placement, entry into a 12×10 cm target region, and plug seating within ∼3 mm clearance.The passage contrasts this with SFT-prior near-miss failures at the final phase under distribution shift.Source: §B.1 · p. 17

B.2 Failure Mode Analysis

Under unseen conditions, the SFT prior fails at grasp, pre-contact positioning, and terminal task phases. Full DLS training eliminates the dominant failure mechanism, though some depth-ambiguous failures remain.

B.2 Failure Mode Analysis

  • Unseen lighting and background changes produce positional offsets that cross distinct phase boundaries, from early grasp failures to terminal near-misses.The reported failures include stalled grasp approaches, blocked pre-contact positioning, push-direction misalignment, and insert offsets beyond the sub-3 mm tolerance.Source: §B.2 · p. 17
  • Terminal near-misses reach phase Z but fail its gate, making them the highest-credit near-miss regime in SPL.Examples include contact with a misdirected push and a plug aligned over the socket but outside the Insert tolerance.Source: §B.2 · p. 17
  • Full DLS training effectively eliminates the dominant unseen-condition failure mechanism, while residual failures occur where one RGB camera cannot resolve fore-aft depth.The residual ambiguity is described as outside the failure manifold reachable through simulation exploration.Source: §B.2 · p. 17

B.3 Additional Ablation

The ablations show that the behavioral prior needs both simulation coverage and real-world grounding, with co-training performance peaking near an even mix. The theoretical analysis links localized trajectory credits and signed labels to phase-specific directional updates.

B.3 Additional Ablation

  • All three non-full behavioral priors underperform Sim-Real SFT (70.0% Unseen. SR), because removing SFT, real grounding, or simulation coverage leaves a distinct deficit.Without SFT results collapse to near-zero; Real-only SFT reaches 46.7%, and Sim-only SFT reaches 60.0%.Source: §B.3 · p. 17, Figure 11 · p. 18
  • Performance peaks at α=0.5 across tasks, while a real-dominated ratio degrades more sharply than a sim-dominated ratio.With 50 real and 1,000 simulated demonstrations fixed, sim-dominated training retains broad exploration diversity and is partly re-anchored by stage-2 real-data regularization.Source: §B.3 · p. 18, Figure 12 · p. 18

C Theoretical Analysis of Failure-Boundary Learning

  • Under the stated assumption, SPL trajectory credit falls in a unique interval that identifies the oracle failure phase without per-step annotations.The theoretical analysis presents this as Failure Boundary Localization.Source: C Theoretical Analysis of… · p. 18
  • Negative SPL labels suppress denoising perturbations associated with crossing a failure boundary, while positive labels reinforce perturbations associated with successful execution.These signed signals support directional shaping in the flow dynamics.Source: C Theoretical Analysis of… · p. 18

C.1 Failure Boundary Localization

Failure Boundary Localization uses ordered task phases and simulator-state gates to convert trajectory progress into a credit that identifies where a failed trajectory breaks down. Oracle-Aligned Denoising Perturbation Shaping uses signed phase labels to suppress failure-producing denoising directions and reinforce success-producing ones.

C.1 Failure Boundary Localization

  • The task is modeled as an ordered sequence of phases, each with a gate predicate over privileged simulator states and an abstraction assigning the current phase.The SPL potential is defined over this phase structure.Source: §C.1 · p. 18
  • The oracle failure phase is the first phase whose gate is never satisfied during an unsuccessful trajectory.The phase index ranges from 1 to Z.Source: §C.1 · p. 18
  • Gate–metric alignment requires each phase’s intra-phase metric to reach its threshold exactly when that phase’s gate is satisfied.The paper states this holds by construction for the geometric and force gate predicates used in the SPL specifications.Source: §C.1 · pp. 18–19
  • For a failed trajectory, its deepest reached phase equals the oracle failure phase, and its SPL credit uniquely identifies that phase.The credit lies in the interval indexed by the deepest phase reached, so a scalar trajectory credit recovers the breakdown phase without per-step outcome labels.Source: §C.1 · p. 19

C.2 Oracle-Aligned Denoising Perturbation Shaping

  • Negative labels direct DBS to suppress denoising perturbations associated with crossing the failure boundary; positive labels reinforce perturbations associated with successful execution.Theoretical results characterize failure-phase updates as opposite to the gradient of the failure-inducing action’s log-probability under the stated Gaussian Flow-SDE model.Source: §C.2 · p. 20
  • DLS converts sparse trajectory outcomes into phase-specific gradient updates that provably suppress failure-inducing perturbations without per-step annotations or a learned critic.The update operates in the velocity space of the flow-based policy.Source: §C.2 · p. 20
Loading 2609.06114v1…