PaperlayerView exact revision on arXiv

Source-linked AI summary

F4R: Failure-Driven Recognition, Reconstruction, Refinement, and Redeployment for Continual Robot Self-Improvement

Zhuoyuan Yu, Jiacheng Wang, Tianle Liu, Yihua Ren, Peng Yu, Chen Bai, Ziheng Zhang, Yufei Jia, Jindou Jia, Yuhang Zhang, Xinrui Zhang, Shang Yujing, Yuxiang Chen, Chuhao Zhou, Tiancai Wang, Jianfei Yang

arXiv:2609.35575v2cs.ROcs.AI

TL;DR

Real-world robot policies remain brittle in long-tail conditions, and collecting corrective demonstrations is costly. F4R reconstructs deployment failures in simulation and refines policies through sim-real co-training and targeted reinforcement learning. It reaches 90.0% OOD success, 18.75 percentage points above budget-matched Targeted BC, without additional real-world corrective demonstrations.

  • Problem

    Policies remain brittle in OOD conditions, while broad simulation diversity may not target the deployment failures that expose gaps in training coverage.

    Source: §1 · pp. 1–2

  • Method

    F4R diagnoses deployment failures, reconstructs their task-relevant conditions in interactive simulation, then applies failure-conditioned sim-real co-training and targeted reinforcement learning.

    Source: §1 · p. 2

  • Results

    90.0% OOD success, 18.75 percentage points above budget-matched Targeted BC, without additional real-world corrective demonstrations.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Deployment failures can serve as practical signals for directing simulation generation and policy optimization.

    Source: §5 · p. 9

    Future work will scale F4R to broader manipulation domains and investigate more efficient ways to prioritize and reuse accumulated failures.

    Source: §5 · p. 9

Abstract

from arXiv · show

The real-world performance of current vision-language-action models is fundamentally constrained by the limited coverage of expert demonstrations and their insufficient understanding of physical interactions. A common remedy is to collect additional real-world demonstrations of newly encountered failures. However, this process is costly, inefficient, potentially unsafe, and difficult to scale. To address this challenge, we propose Failure for Rising (F4R), a failure-driven real-to-sim-to-real closed-loop learning framework that converts real-world failures into targeted policy improvement. F4R first uses an agent to automatically identify and diagnose failures from rollouts. It reconstructs each failure as an interactive, object-centric table-top environment that preserves the task-relevant spatial and physical conditions. The policy is then refined through failure-conditioned sim-real co-training followed by targeted reinforcement learning in the reconstructed environments. The improved policy is subsequently redeployed, while newly observed failures are continuously fed back into the next reconstruction and learning cycle. Real-world evaluations on four manipulation tasks show that F4R achieves 93.75% In-Distribution and 90.0% Out-of-Distribution (OOD) success, outperforming the budget-matched Targeted BC baseline by 18.75 percentage points under OOD conditions without collecting additional real-world corrective demonstrations.

1 Introduction

Robot policies remain brittle in long-tail OOD conditions, while collecting real-world demonstrations is costly. F4R uses deployment failures to guide reconstruction and policy refinement in simulation.

1 Introduction

  • Policies remain brittle under OOD variations, and long-tail failure conditions are underrepresented or absent from original training data.Source: §1 · p. 1
  • Unlike targeted BC’s open-loop pipeline of manual diagnosis and repeated data recollection, F4R autonomously recognizes failures, reconstructs them in simulation, refines the policy, and redeploys it.Source: Figure 1 · p. 2
  • Broad randomization may increase diversity without targeting deployment failures; reconstructing their conditions creates reusable scenarios for corrective exploration without costly or unsafe real-world demonstrations.Additional real-world data collection also requires substantial human effort, hardware resources, and operational coordination.Source: §1 · pp. 1–2
  • F4R’s VLM-based agent identifies and diagnoses failed executions, then reconstructs their objects and spatial configurations in interactive simulation.Failure-aware randomization expands each reconstructed scene; MimicGen adapts object-centric motion segments from successful real demonstrations, followed by sim-real co-training and targeted reinforcement learning.Source: §1 · p. 2
  • 90.0% OOD success, 18.75 percentage points above budget-matched Targeted BC, without additional real-world corrective demonstrations.Source: §1 · p. 2

2 Related Work

Prior work reconstructs real environments for robot learning and adapts VLA policies through sim-real co-training or reinforcement learning. F4R distinguishes itself by using deployment failures to select what to reconstruct and where to concentrate training.

2.1 Real-to-Sim Reconstruction for Robot Learning

  • Real-to-sim reconstruction work uses real-world scans and 3D Gaussian Splatting for visual and physical alignment, scene editing, and recovery of simulation-ready geometry and properties.Examples include RialTo (Torne Villasevil et al., 2024), RL-GSBridge (Wu et al., 2025), and RoboSplat (Yang et al., 2025).Source: §2.1 · pp. 2–3
  • Existing real-to-sim pipelines are generally scene- or task-driven, whereas F4R uses deployment failures to select reconstructions and expand scenes into a failure-centered training distribution.Source: §2.1 · p. 3

2.2 VLA Adaptation via Sim-Real Co-Training and RL

  • Sim-real co-training broadens demonstration coverage, but still depends on simulated demonstrations covering deployment-relevant conditions; RL approaches generally optimize over predefined distributions.Source: §2.2 · p. 3
  • ASPIRE diagnoses failures in code-as-policy systems, while TwinRL reconstructs a workspace digital twin; F4R instead uses failures to determine both what to reconstruct and where to focus training.Source: §2.2 · p. 3

3 Method

F4R iteratively turns deployment failures into reusable training assets for real-to-sim-to-real policy refinement. It diagnoses failures, reconstructs and expands relevant conditions, refines the policy through sim-real co-training and targeted RL, then redeploys it to encounter further failures.

3.1 Problem Formulation

  • F4R seeks to improve the policy around observed failure scenarios while preserving capabilities supported by the original real-world demonstrations.The policy context includes visual observation, robot state, and language instruction.Source: §3.1 · p. 3

3.2 Failure Recognition and Diagnosis

  • A VLM-based agent identifies the earliest visually supported failure from synchronized third-person and wrist-view evidence, then records relevant objects, failure factors, evidence, and confidence.Chronologically ordered intervals and initial/terminal state boards support diagnosis; the resulting record determines what to reconstruct and randomize.Source: §3.2 · pp. 3–4

Manipulation scene reconstruction.

Manipulated object reconstruction.

  • Object reconstruction combines refined point clouds and completed watertight meshes with optional articulation and synthesized PBR materials, then aligns Gaussians for rendering.The reconstructed assets and diagnosed factors specify scene conditions for physical interaction, appearance, and optional articulation.Source: Manipulated object reconstruction · p. 5

Failure-aware domain randomization.

  • Failure-aware randomization samples failure-relevant variables around the recovered configuration while keeping unrelated variables fixed, creating local variations around each observed failure.This expands a single reconstructed realization into a failure-centered distribution used for corrective demonstrations and RL interaction.Source: Failure-aware domain randomization · p. 5, §3.4 · p. 5
  • F4R adapts successful historical real-world trajectories to randomized failure environments with MimicGen, then co-trains on simulated trajectories and original real-world data.When successful seeds are insufficient, AnyGrasp supplies additional grasp proposals; simulated data provide failure-specific corrections while real data preserve capabilities and stabilize RL initialization.Source: Failure-aware domain randomization · p. 5
  • Starting from the co-trained policy, PPO uses the original task reward without failure-specific shaping and an auxiliary SFT objective on original real-world data.The passage attributes improved closed-loop and contact-sensitive behavior to RL and a mitigating role against catastrophic forgetting to real-data supervision.Source: Failure-aware domain randomization · p. 5

3.5 Closed-Loop Redeployment

  • After refinement, F4R redeploys the policy, evaluates previous failures and new configurations, and feeds persistent or newly emerging failures into the next reconstruction and refinement round.This iterative process converts observed failures into reusable training assets and aligns improvement with encountered operating conditions.Source: §3.5 · p. 6

4 Experiments

Experiments assess F4R on deployment failures, sim–real consistency, robustness across failure conditions, closed-loop refinement, and the effects of its components.

4.1 Experimental Setup

  • Evaluations cover eight tabletop tasks; four receive refinement tests, with 20 real trials per ID/OOD setting and simulation rollouts across 100 parallel environments.The setup uses a UR5 with two cameras; policy refinement uses PPO with trainable LoRA adapters.Source: §4.1 · p. 6

4.2 Real-World Refinement on Deployment Failures

  • F4R achieves 93.75% success on the original distribution and 90.0% on the failure distribution, exceeding Targeted BC and RLinf-Co by 18.75 and 12.5 points on failures.Targeted BC uses one hour of real-world corrective-data collection; F4R concentrates randomization around diagnosed failures.Source: §4.2 · pp. 6–7

4.3 Reliability of Autonomous Failure Diagnosis

  • F4R diagnoses 69 of 80 trajectories correctly on the first attempt (86.25% accuracy); a second attempt raises accuracy to 92.5%.Six cases remain unresolved: four involve camera-viewpoint shifts and two involve illumination changes, exposing difficulty reasoning over fine-grained temporal evidence.Source: §4.3 · p. 7, Table 2 · p. 7

4.4 Sim–Real Consistency and Data Efficiency

  • Across 24 checkpoints, simulation and real-world success rates correlate strongly (Pearson r = 0.7625, p = 1.49 × 10−5), though simulation is not an exact performance estimate.The reconstructed environments preserve broad task-difficulty rankings; deviations from the diagonal reflect appearance, contact-dynamics, and execution-sensitivity gaps.Source: §4.4 · pp. 7–8, Figure 3 · p. 7

4.5 Robustness Across Failure Conditions

  • On Pick Fruits, F4R raises average success across four failure variations from 46.25% to 90.0%; co-training alone reaches 83.75%.Targeted RL adds gains under illumination and distractor variations.Source: §4.5 · p. 8, Table 3 · p. 8

4.6 Multi-Round Closed-Loop Refinement.

  • Across three cycles, Stack Bowls success rises from 30% to 95%; Pick Fruits reaches 100% after the first cycle and maintains it for two more.Newly observed failures guide subsequent reconstruction and training, supporting gradual correction or rapid saturation depending on task difficulty.Source: §4.6 · p. 8, Figure 4 · p. 8

4.7 Ablation Study

  • Removing failure diagnosis cuts ID/OOD success by 6.25/12.5 points; removing real-data supervision cuts it by 8.75/3.75, while co-training alone reaches 90.0% ID and 83.75% OOD.Co-training supplies an effective initialization and alleviates exploration difficulty; targeted RL further improves the co-trained policy by 3.75 ID and 6.25 OOD points.Source: §4.7 · pp. 8–9, Table 4 · p. 8, Table 5 · p. 9

5 Conclusion

F4R turns deployment failures into targeted simulation-based policy refinement and redeploys the updated policy for continued learning. Real-world experiments show improvements across ID and OOD conditions, while multi-round refinement uses newly observed failures to guide later updates.

5 Conclusion

  • F4R improves real-world performance under both ID and OOD conditions, outperforming budget-matched Targeted BC while reducing reliance on additional real-world demonstrations.Multi-round refinement further shows that post-redeployment failures can guide subsequent updates.Source: §5 · p. 9
  • F4R uses a failure-discovery and diagnosis agent, reconstructs task-relevant conditions in simulation, then refines the policy through failure-conditioned sim-real co-training and targeted RL.The updated policy is redeployed so newly observed failures can guide later refinement.Source: §5 · p. 9
  • Future work will scale F4R to broader manipulation domains and develop more efficient ways to prioritize and reuse accumulated failure cases.Source: §5 · p. 9

Supplementary Overview

The supplementary material documents F4R’s failure recognition and diagnosis, reconstruction, policy-refinement, and evaluation procedures. It also includes qualitative analyses, limitations, safety considerations, and reproducibility information.

Supplementary Overview

  • Section S1 details failure recognition and diagnosis, from temporal evidence proposal and evidence-board construction to prompting, validation, and additional VLM evaluations.Source: Supplementary Overview · p. 13
  • Section S2 describes manipulation-scene and object reconstruction and the human effort required by the system.Source: Supplementary Overview · p. 13
  • Section S3 provides implementation details for failure-aware randomization, MimicGen data generation, co-training, and PPO.Source: Supplementary Overview · p. 13
  • Section S4 specifies the robot platform, tasks, distributions, success criteria, baselines, and budget matching.Source: Supplementary Overview · p. 13
  • Sections S5 and S6 provide additional quantitative results, qualitative examples, failure cases, limitations, and safety considerations.Source: Supplementary Overview · p. 13

S1 Failure Recognition and Diagnosis Details

F4R diagnoses failed rollouts from dual-view video, using temporal proposals and structured evidence boards to identify and attribute the earliest failure. Its diagnosis benchmarks report strong task-specific accuracy alongside unresolved visual-change errors and below-70% best-backend cause accuracy on ViFailback without task-specific adaptation.

S1.1 Input and output

  • A task-specific binary checker separates successful from failed rollouts; diagnosis then uses synchronized wrist and external-camera frames and records evidence, confidence, objects, and relations.Source: S1.1 Input and output · p. 13, Table S1 · p. 14

S1.2 Temporal evidence proposal

  • The proposal stage scans sparse frames, smooths and standardizes visual-change signals from both views, then fuses them to select separated peaks and expand them with temporal context.Videos are scanned every three frames; peaks receive 120 preceding and 180 subsequent frames, with nearby intervals merged.Source: S1.2 Temporal evidence proposal · p. 14

S1.3 Dual-view evidence-board construction

S1.4 Diagnostic prompting

  • A fixed pretrained VLM controller analyzes event and state boards, focuses on the earliest positive event, and aggregates structured evidence; simulation validation can reject reconstructions and eventually fall back to broad randomization.The focused pass seeks the earliest visible anomaly, while the validation loop accepts a reconstruction when the initial policy’s success rate is at most 0.70; after three rejections it switches to broad randomization.Source: S1.4 Diagnostic prompting · pp. 16–18

S1.5 Autonomous diagnosis benchmark

  • 69 of 80 trajectories are correctly diagnosed on the first attempt (86.25% accuracy); a second attempt raises cumulative accuracy to 92.5%.The six unresolved cases involve viewpoint shifts or illumination changes, exposing limits in temporal evidence retention and reasoning.Source: S1.5 Autonomous diagnosis benchmark · p. 20

S1.6 Complete ViFailback evaluation

S2 Failure-Case Reconstruction Details

F4R reconstructs manipulation scenes and objects as simulation assets, coupling Gaussian rendering with simulator-driven robot motion. The supplementary material also describes the reconstruction workflow and its human-involvement boundary.

S2.1 Manipulation-scene reconstruction

  • Aligned robot Gaussians follow simulated robot motion through link-level labels, while the reconstructed workspace and rendering preserve task-relevant conditions with a small visual gap to real observations.The scene pipeline prunes reconstruction outliers and aligns Gaussian primitives to the simulated robot; rendered comparisons are shown in Figure S6.Source: Figure S5 · p. 21, S2.1 Manipulation-scene reconstruction · p. 21, Figure S6 · p. 22

S2.2 Manipulated-object reconstruction

  • Object assets are reconstructed from wrist-camera RGB-D views through pose optimization, masked TSDF fusion, mesh completion and regularization, then texturing and Gaussian-rendering optimization.For articulated objects, an additional module infers movable parts and kinematic relationships from multi-view renderings.Source: S2.2 Manipulated-object reconstruction · pp. 22–23

S2.3 Automation and human Effort

  • Workspace capture and scene setup require one-time human involvement; later failure detection, reconstruction, simulation refinement, and corrective-data generation are automated, and assets can be reused in unchanged environments.Source: S2.3 Automation and human Effort · p. 23

S3 Failure-Aware Randomization and Policy Refinement

F4R builds failure-centered simulation distributions, generates corrective trajectories, and refines the policy through sim–real co-training followed by targeted PPO.

S3.1 Failure-aware domain randomization

  • Failure-relevant variables are sampled locally around recovered failure configurations, while invalid or infeasible scenes are rejected and accepted failures are sampled uniformly.Unrelated variables are fixed or weakly perturbed; each reconstructed failure distribution contributes equally during training.Source: S3.1 Failure-aware domain randomization · pp. 24–25

S3.2 Corrective trajectory generation

  • Historical successful rollouts seed MimicGen object-centric motion segments for randomized scenes, and only successful simulated trajectories enter supervised co-training.When seed rollouts lack grasp coverage, AnyGrasp supplies candidate poses; unsuccessful rollouts remain available as PPO interaction experience.Source: S3.2 Corrective trajectory generation · p. 25

S3.3 Stage I: sim–real co-training

S3.4 Stage II: failure-targeted reinforcement learning

  • PPO starts from the co-trained policy and uses 256 parallel failure-distribution environments, while original-data SFT regularizes training without failure-specific reward shaping.Co-training helps reach meaningful states in severe failure regions; PPO improves closed-loop and contact-sensitive behavior.Source: S3.4 Stage II: failure-targeted… · p. 26

S4 Complete Experimental Setup

The experiments use a dual-view UR5 platform and an eight-task benchmark, with matched ID/OOD evaluations, shared success criteria, and time-matched baselines.

S4.1 Robot platform and calibration

S4.2 Eight-task benchmark

  • All eight tasks evaluate sim–real behavioral consistency, while four representative tasks are used for policy-refinement experiments.ID conditions follow collected demonstrations; OOD conditions are deployment-derived failures not covered by those demonstrations.Source: S4.2 Eight-task benchmark · p. 27

S4.3 ID and OOD distributions

  • OOD evaluations expand object-pose and distractor-placement ranges while preserving task semantics and physical feasibility; other settings are held fixed unless specified.Initializations are pre-generated with fixed seeds and reused across methods within each task and setting.Source: S4.3 ID and OOD distributions · pp. 27, 29

S4.4 Success criteria and evaluation protocol

  • Real-world performance is measured over 20 trials per task and setting, with success requiring a stable task goal after robot release and blinded video scoring.Ambiguous outcomes count as failures; simulation uses one rollout in each of 100 parallel environments.Source: S4.4 Success criteria and evaluation… · p. 29

S4.5 Baselines and budget matching

  • Methods are matched by end-to-end data-preparation time: one hour for Targeted BC, RLinf-Co, and F4R, rather than by trajectory count.Targeted BC collects corrective real-world demonstrations; F4R and RLinf-Co use the hour for simulation preparation and rollout collection.Source: S4.5 Baselines and budget matching · pp. 29–30

S5 Additional Quantitative Results

F4R outperforms the budget-matched baselines in OOD success, while reconstructed simulations track real-world performance and repeated refinement improves task success across cycles.

S5.1 Complete real-world results

  • F4R improves OOD success by 18.75 percentage points over Targeted BC and 12.5 points over RLinf-Co.The comparison uses the complete real-world results; RLinf-Co has the same optimization and rollout budget but uses broad variations.Source: Table S19 · p. 31, S5.1 Complete real-world results · p. 31

S5.2 Full sim–real consistency results

S5.3 Multi-round closed-loop refinement

  • Stack Bowls success rises from 30% to 80%, 90%, and 95% over three refinement cycles; Pick Fruits rises from 40% to 100% after one cycle and maintains that level.After each cycle, the policy is redeployed and newly observed failures guide the next cycle.Source: S5.3 Multi-round closed-loop refinement · p. 32, Table S21 · p. 32

S5.4 Cross-task capability retention

S6 Qualitative Results and Limitations

The paper reports a tabletop, predominantly rigid-object evaluation and describes a continuing human safety role during robot deployment.

S6.1 Failure cases

S6.2 Scope and safety

  • Evaluation is limited to tabletop manipulation with predominantly rigid objects; deformable objects, dynamic scenes, mobile manipulation, and long-horizon assembly remain unvalidated.Deployment still relies on a human operator for emergency stops and hardware safety, although the operator does not diagnose failures or provide corrective demonstrations.Source: S6.2 Scope and safety · p. 33

S6.3 Reproducibility

  • The reported software stack uses π0.5, GPT-5.5, Isaac Sim 5.1, Isaac Lab 2.3.2, and a fixed random seed of 42.Source: S6.3 Reproducibility · p. 33
Loading 2609.35575v2…