PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning from Fleet Experience: Collective Experience Learning Across 96 Robots

Chao Yu, Shu'ang Yu, Hao Lin, Hongzhi Zang, Yuxiang Yan, Chenxi Jiang, Zhen Guo, Chuankang Li, Yihao Chen, Xin Xu, Xu Fu, Jiakai Zhou, Liangzhi Shi, Hongyi Ren, Yiwei Liu, Jing Wu, Yuanqing Wang, Chen Wu, Bo Dai, Chunyu Yang, Hongliang Huang, Shuoyu Yue, Weiqi Yin, Yinuo Chen, Ruihua Chen, Chunyang Zhu, Han Xiao, Yanghao Li, Ruipu Wu, Zhikai Cai, Xiu Chen, Xu Tan, Si Liu, Yuchen Yan, Jialin Zhang, Wen Zeng, Yiming Zhou, Zixiao Huang, Xinlei Chen, Guanghui Ren, Quanlu Zhang, Boxun Li, Maoqing Yao, Yu Wang

arXiv:2610.25736v2cs.RO

TL;DR

Robot-policy improvement during deployment can depend on expert action supervision, which is difficult to scale with growing fleets. The paper introduces Collective Experience Learning, using fleet executions for shared policy updates through failure-guided retrieval and asynchronous training support. Across five real-world tasks, larger fleets generally improved performance, with success rates of 93.3–99.2% at the largest evaluated fleet sizes.

  • Problem

    Robot deployment generates experience for policy improvement, but continued reliance on expert corrective actions makes learning difficult to scale across growing fleets.

    Source: §1 · p. 5

  • Method

    Collective Experience Learning pools deployed robots’ executions to improve a shared policy, using FERT to retrieve successful segments relevant to failures and prioritize their recorded actions for supervised updates.

    Source: §1 · pp. 5–6

  • Results

    Across five real-world manipulation tasks, larger fleets generally improved performance, with success rates of 93.3–99.2% at the largest evaluated fleet sizes.

    Source: §6 · p. 29

  • Takeaways & Limitations

    The results support CEL as a way to improve shared policy reliability through deployment-generated experience without a corresponding expansion of expert action supervision.

    Source: §1 · p. 5, §6 · p. 29

    HotSwarm recovery restores software only; repairing hardware and resetting the scene remain operator tasks.

    Source: §3.2.3 · p. 17

Abstract

from arXiv · show

Robot deployment generates experience that can support policy improvement, but continued reliance on expert corrective actions makes learning difficult to scale across growing fleets. We propose learning from fleet experience: using deployed robots' own executions as a source of supervision for further policy improvement. We call this paradigm Collective Experience Learning (CEL), in which robots pool execution experience to update a shared policy that returns to the fleet. Following expert-supervised initialization, we realize CEL through failure-guided experience retrieval fine-tuning (FERT). FERT uses failure contexts to retrieve relevant segments from successful trajectories and prioritizes their recorded actions for supervised updates, without collecting additional expert demonstrations or corrective actions. Operators provide binary episode-outcome labels and environment resets. A fault-tolerant asynchronous training system (FATS) coordinates robot execution, cloud inference and policy optimization despite local interruptions. Across five real-world manipulation tasks, the resulting policies achieve success rates of 93.3-99.2%, evaluated over 120 trials per task on robots excluded from training data collection. With collection time per robot held fixed, larger fleets provide more experience and generally improve performance across the evaluated sizes. On multi-stage phone packaging, learning with 96 robots raises success from 79.2% to 98.3%. These results support CEL as a scalable approach to improving robot policy reliability through experience generated during deployment.

1 Introduction

Robot execution produces experience that could improve policies, but scaling action supervision remains constrained by human effort. The paper proposes CEL, using fleet experience for shared policy improvement, and develops FERT and FATS to support learning during deployment.

1 Introduction

  • Robot foundation models can provide initial capabilities, but reliable execution remains challenging amid variation and accumulated errors in multi-stage tasks.Expert demonstrations and corrective interventions help improve policies, yet ongoing reliance on new expert actions is difficult to scale alongside robot interaction.Source: §1 · p. 5
  • CEL shifts continued action supervision from newly collected expert actions to deployed robots’ executions, pooling experience to improve a shared policy returned to the fleet.Following expert-supervised initialization, this realization requires no additional expert demonstrations or corrective actions.Source: §1 · p. 5
  • FERT retrieves successful-trajectory segments relevant to failure contexts, prioritizes them for sampling, and uses their recorded actions as supervised update targets.Operators provide binary episode-outcome labels and reset environments; FERT update actions come from autonomous robot execution.Source: §1 · p. 6
  • FATS coordinates robot execution, cloud inference, and policy optimization asynchronously, without requiring a common collection schedule.SwarmLink pools inference requests and streams trajectory fields, while HotSwarm restarts failed robot workers as available robots continue collecting.Source: §1 · p. 6
  • Across five real-world manipulation tasks, policies trained with fleets of up to 96 robots achieved 93.3–99.2% success; phone-packaging success rose from 79.2% to 98.3% with 96 robots.Evaluation used 120 trials per task on robots excluded from training data collection; with per-robot collection time fixed, larger fleets generally improved performance.Source: §1 · p. 6

2 Preliminaries

The preliminaries specify the vision-language-action policy and deployment setting: robots share task experience and optimize task success using execution trajectories and outcomes.

2.1 Robot policy

  • The policy conditions on visual observations, proprioceptive measurements, and a language instruction, and generates continuous action chunks.The implementation uses π0, a flow-based vision-language-action model.Source: §2.1 · pp. 6–7

2.2 Problem setting

  • For a task instruction, robots operate during deployment with a shared policy, though individual robots may execute different policy versions as learning proceeds.Each execution supplies a trajectory and binary outcome; operators provide outcome labels and reset environments, while subsequent learning uses robot-executed actions.Source: §2.2 · p. 7
  • The objective is to maximize successful execution, while the learning algorithm determines which fleet trajectories are retained and how they contribute to updates.CEL uses fleet-generated experience after task initialization without additional expert demonstrations or corrective actions; the objective does not prescribe a specific update method.Source: §2.2 · p. 7

3 Collective Experience Learning

This section presents Collective Experience Learning (CEL), which pools deployment-generated robot experience to improve a shared policy returned to the fleet. It covers FERT’s use of successful actions and failure contexts for supervision without further expert corrections, and FATS’s coordination of asynchronous execution, cloud inference and training, and recovery despite local interruptions.

3.1 Algorithm: Failure-guided Experience Retrieval Fine-Tuning (FERT)

  • FERT uses successful robot actions as targets and failure contexts to prioritize related experience, so policy updates need no additional expert demonstrations or corrective actions.Humans provide binary episode-outcome labels and reset environments; subsequent action supervision comes from robot executions.Source: §3 · p. 7, §3.1 · p. 9

3.1.1 Step 1: Asynchronous experience collection

  • Successful trajectories populate a rolling experience buffer as action targets, while failed trajectories serve only as temporary retrieval queries, not action supervision.Asynchronous trajectory arrival lets robots contribute at different rates while policy optimization continues.Source: §3.1.1 · p. 9

3.1.2 Step 2: Experience encoding

  • FERT combines visual features with normalized proprioceptive state, using a frozen visual encoder to keep failure queries and cached successes in a consistent representation space.The representation preserves scene and robot-configuration context across robots.Source: §3.1.2 · p. 9

3.1.3 Step 3: Failure-guided retrieval

  • FERT retrieves successful segments using short pre-endpoint failure-context queries, avoiding frame-level failure annotations and enabling cross-robot experience matching.Slope-constrained subsequence dynamic time warping ranks candidate intervals; typically, the 10 lowest-cost candidates are selected and expanded by 30 frames on each side.Source: §3.1.3 · pp. 10–11

3.1.4 Step 4: Prioritized fine-tuning

  • Retrieved-interval overlap raises sampling priority for related successful action chunks, while repeated retrieval reinforces them and exponential decay reduces older events’ influence.Binary outcomes guide updates through successful-data selection and failure-conditioned prioritization, without a learned critic, value targets, or advantage estimates.Source: §3.1.4 · p. 11

3.1.5 Understanding FERT

  • Under stated replay and retrieval-alignment conditions, Theorem 1 guarantees a local benefit from retrieval-weighted likelihood updates for a sufficiently small positive step size.The result is a conditional local analysis; the paper evaluates retrieval’s practical benefit in a later ablation.Source: §3.1.5 · p. 12

3.2 Fault-tolerant asynchronous training system

  • FATS connects robot execution, cloud inference, and policy optimization across 96 robots and six cloud servers, accommodating the distinct timing needs of interaction and episode-record transfer.In the 96-robot run, fault incidents affected 22 robots, with an interruption every 22 minutes on average; the system separates experience-worker recovery from learning-state recovery.Source: §3.2 · p. 13

3.2.1 Architecture

  • FATS separates environment workers that operate robots, inference workers that compute actions, and training workers that update the policy from collected experience.Environment workers run at the robot site, while inference and training workers run on cloud servers.Source: §3.2.1 · p. 13
  • FATS’s data and control planes coordinate worker identity, failure reporting, and repair, linking robot-side execution with shared cloud inference and training.Worker identities accompany data and replies; failures flow to HotSwarm, which rebuilds connections and channel replicas after a worker fails.Source: Figure 4 · p. 14, §3.2.1 · p. 14

3.2.2 SwarmLink data plane

  • SwarmLink uses distributed channels, shared inference, and cloud-side trajectory assembly to let robots progress independently while complete episodes become available for training.Inference and execution fields are joined by worker, episode, and step identifiers; an episode is released only after required data and its outcome arrive.Source: §3.2.2 · p. 15

3.2.3 HotSwarm control plane

  • HotSwarm restores failed robots in place while other workers continue, with recovery tailored to environment, inference, and training workers.Environment workers can restart at the cost of at most one episode; inference workers reload the current policy, while training workers resume from periodic checkpoints and queued experience waits for training to resume. Hardware repair and scene resets remain operator tasks.Source: §3.2.3 · pp. 15–17

3.2.4 A robot fault across both planes

  • When a robot worker fails, FATS discards its partial episode and stale action, preserves queued complete experience, and resumes collection after in-place recovery and reset.In the 96-robot run, robot 20 returned to collection after an 11-min restart while the rest of the fleet continued.Source: §3.2.4 · p. 17

4 Experimental Results

The experiments assess FERT’s held-out-robot performance across five manipulation tasks, including fleet-size effects, ablations, and a 96-robot run. They also examine how collection, recovery, and communication affect fleet experience and system operation.

4.1.1 Tasks and robot setup

  • The evaluation spans five real-world tabletop manipulation tasks, uses a shared expert-supervised initialization for FERT and its baselines, and tests learned policies autonomously on held-out robots.Each policy is evaluated over 120 trials per task across two robots excluded from training data collection; FERT training uses binary task-outcome labels and resets, without additional corrective actions.Source: §4.1.1 · p. 18, §4.1.2 · p. 19, §4.1.4 · pp. 19, 21

4.2 Policy improvement from fleet experience

  • FERT raises mean success from 81.0% with (M)HG-DAgger to 97.3%, a 16.3-percentage-point gain across five tasks.This corresponds to an 86.0% relative reduction in average failure rate; FERT uses fleet experience without additional expert demonstrations or corrective actions.Source: Figure 6 · p. 21, §4.2 · p. 21, Table 3 · p. 22
  • FERT exceeds human teleoperation by 16.7 percentage points on average and beats RLT and RECAP by 12.3 and 11.5 points, respectively, across five tasks.Its largest stated gains over human teleoperation are 25.8 points on Phone Charging and 23.3 points on Phone Packaging; over RLT, Phone Charging rises from 75.8% to 97.5%, and over RECAP, Phone Packaging rises from 75.0% to 98.3%.Source: §4.2 · p. 21

4.3 Scaling with fleet size

  • With collection time per robot fixed, FERT success generally rises with fleet size, including Phone Packaging’s increase from 86.7% with 16 robots to 98.3% with 96.At smaller fleet sizes, FERT can underperform initialization; (M)HG-DAgger also shows single-robot drops on three challenging tasks, later saturation on Phone Charging and Phone Packaging, and a decline on Stool Assembly.Source: Figure 7 · p. 22, §4.3 · p. 23

4.4 Ablation studies

  • On Phone Packaging with 16 robots, disabling failure-guided retrieval lowers success from 86.7% to 64.2%, while removing retrieved-segment extensions lowers it to 57.5%.The extension comparison holds the maximum query length at L = 90 frames; the full configuration uses hpre = hpost = 30 frames.Source: Figure 8 · p. 23, §4.4 · pp. 23–24
  • On Phone Packaging with 16 robots, synchronization every 500 optimizer steps achieves 86.7% success, versus 80.0% at 1,500 steps and 75.0% at 100 steps.Priority retention ρ = 0.88 also achieves 86.7%, compared with 70.8% at ρ = 0.8 and 82.5% at ρ = 0.95.Source: Figure 8 · p. 23, §4.4 · p. 24

4.5.1 Faults and their containment

  • In a 96-robot run, FATS collected 9,834 episodes, and fleet-level collection continued through local faults, completing 99% as many episodes after a median fault event as before.Outside the lunch break, an average of 65 robots were executing and the fleet completed 24 episodes per minute; incidents accounted for 1.3% of 660 robot-hours.Source: §4.5.1 · pp. 24–25, Figure 9 · p. 25

4.5.2 Decoupled progress

  • Robot progress varied even without faults: median episode duration was 3.2 minutes, per-robot medians ranged from 0.6 to 4.2 minutes, and robots completed 45 to 222 episodes each.Resets and faults add further pauses between episodes.Source: §4.5.2 · p. 26
  • Synchronous rounds collect only 33% of the episodes in the trace replay, as robots wait for the slowest episode or pause.The median round lasts 9.3 min versus a typical 4.1 min robot cycle; replacing pauses overlapping faults raises collection to 42%, and replacing all pauses raises it to 78%.Source: §4.5.2 · p. 26
  • Full-restart replay preserves 92%, 86% and 74% of episodes for restart durations of 2, 5 and 10 min, respectively.The analytical model predicts collection-time fractions of 91.6%, 81.3% and 68.5% for those durations; both approaches show greater losses with longer restarts.Source: §4.5.2 · p. 26
  • Local recovery confines recorded collection loss to 1.3% in this run, while the model predicts whole-system restart losses grow with fleet size.Under fixed per-robot fault-rate and downtime assumptions, the model predicts a fleet-size-independent expected loss fraction for local recovery.Source: §4.5.2 · p. 26

4.5.3 Edge-cloud communication

  • 95.18% of Env-to-Rollout operations finish within 20 ms, while approximately 97.6% of Rollout-to-Env operations finish within 250 ms.Only 14.44% of action-channel operations finish within 50 ms, and fewer than 1% exceed 1 s; Figure 10 summarizes measured durations across the three communication channels.Source: §4.5.3 · p. 26, Figure 10 · p. 27
  • More than 99% of Env-to-Train operations finish within two minutes, motivating inference requests to be served independently of experience delivery.Trajectory streaming uploads training data during robot execution and reuses inference observations in the cloud, reducing post-episode transfer.Source: §4.5.3 · p. 27

5 Related Works

Prior work studies real-world policy improvement, fleet-scale learning, experience retrieval, and distributed learning systems. This work distinguishes itself by using deployed robots’ experience without additional expert action supervision, with FERT and FATS supporting that learning loop.

5.1 Robot Learning from Real-world Experience

  • Real-world learning methods address distribution mismatch and limited interventions in different ways, while this work improves policies from deployed-robot experience without additional expert actions.The related methods include DAgger, HG-DAgger, DART, ThriftyDAgger, HIL-SERL, ConRFT and RECAP.Source: §5.1 · pp. 27–28

5.2 Fleet-scale Robot Learning

  • CEL connects fleet-generated experience with shared policy updates during deployment, evaluating learning without additional expert action supervision on fleets of up to 96 robots.Collection time per robot is held fixed to examine performance as fleet size and total experience increase.Source: §5.2 · p. 28

5.3 Experience Retrieval and Prioritized Learning

  • FERT queries successful fleet experience with failed-execution contexts, prioritizing retrieved action segments for supervised updates without value estimates or target demonstrations.Successful executions supply imitation targets, while failures guide which targets receive emphasis.Source: §5.3 · p. 28

5.4 Distributed Systems for Robot Learning

  • FATS supports CEL by sustaining fleet learning through asynchronous execution and interruptions, while FERT determines how successful and failed executions contribute to policy updates.In the reported experiments, operators supply binary outcome feedback while robot executions provide subsequent action supervision.Source: §5.4 · pp. 28–29

6 Conclusion

The conclusion presents CEL as a way to improve shared robot policies using deployment experience after expert-supervised initialization. Across the evaluated tasks and fleet sizes, results support this approach while identifying limits on the study’s scope.

6 Conclusion

  • 93.3–99.2% success rates were achieved across five real-world manipulation tasks on robots excluded from training data collection.The study evaluates 120 trials per task, as stated in the paper context.Source: §6 · p. 29
  • With collection time per robot held fixed, larger fleets provided more experience and generally improved performance across the evaluated sizes.Source: §6 · p. 29
  • On phone packaging, learning with 96 robots raised success from 79.2% to 98.3%.Source: §6 · p. 29
  • Local fault events occurred on average every 22 minutes, while fault-related downtime accounted for 1.3% of total robot time and fleet-level collection continued.Source: §6 · p. 29
  • The study evaluates task-specific policies on one robot platform and retains human-provided outcome labels and environment resets; broader task distributions, heterogeneous robots, and longer deployments remain future work.Source: §6 · p. 29

B Theoretical Analysis of FERT

This appendix supplies the setting and proofs for Section 3.1.5. Its analysis models successful-experience fitting and retrieval prioritization with a local likelihood-based mean-gradient update.

B Theoretical Analysis of FERT

B.1 Proof of Lemma 1: Successful-Replay Alignment

The proof models successful replay by sampling from the current policy conditioned on success, then uniformly selecting an action chunk. It relates trajectory likelihood gradients to policy factors and concludes by proving Lemma 1.

B.1 Proof of Lemma 1: Successful-Replay Alignment

  • The proof treats episodes as Tc action chunks, allows early termination through parameter-independent absorbing steps, and concludes by proving Lemma 1.Source: §B.1 · p. 36
  • The proof models a successful replay sample as an episode from πθ0 conditioned on y(τ) = 1, followed by uniform selection of one of its Tc chunks.It assumes J(θ0) > 0 and v̸ = 0.Source: §B.1 · p. 36
  • Because initial-state distributions and environment dynamics are independent of θ, only policy factors contribute to the trajectory likelihood’s logarithmic derivative.Source: §B.1 · p. 36

B.2 Proof of Lemma 2: Retrieval Alignment

The analysis models sequential task stages with stage-specific failure rates and retrieval weights. Under its stated ordering condition, the proof establishes positive retrieval alignment, while clarifying that nonuniform priorities alone do not guarantee the required retrieval quality.

B.2 Proof of Lemma 2: Retrieval Alignment

  • The proof assumes successful chunks’ relative likelihoods are fixed within each stage and uses nonnegative retrieval weights with at least one pair where higher failure and retrieval weight coincide.Source: §B.2 · p. 37
  • In the stage-wise model, a successful chunk from a stage with a higher local failure rate has a larger first-order contribution to expected return.Each stage has an independent trainable logit, and local failure rates are conditional on reaching that stage.Source: §B.2 · pp. 36–37
  • Under the ordering condition, the covariance argument yields κ > 0 and proves Lemma 2.Source: §B.2 · p. 37
  • The required retrieval quality concerns local failure rates conditional on reaching a stage; nonuniform priorities alone do not guarantee it.Source: §B.2 · p. 37

B.3 Proof of Theorem 1: Lower bound on the FERT improvement

The proof establishes a positive local comparison for FERT under smooth-neighborhood and step-size conditions. The appendix also details expert-supervised initialization and multi-robot HG-DAgger procedures used before FERT.

B.3 Proof of Theorem 1: Lower bound on the FERT improvement

  • Under the stated neighborhood and step-size conditions, the FERT update has a strictly positive improvement comparison over uniform replay.The guarantee concerns one mean-gradient update under the current-policy replay approximation and stage-wise retrieval model; repeated historical replay and flow matching are evaluated experimentally.Source: §B.3 · p. 38

C Policy Initialization

  • Each task's selected expert-supervised checkpoint initializes FERT, RECAP, and RLT, with FERT's later action targets coming from autonomous fleet executions.The initialization adapts π0 through supervised fine-tuning and human-gated dataset aggregation.Source: C Policy Initialization · p. 39

C.1 Supervised fine-tuning on human demonstrations

  • Supervised fine-tuning trains the base policy on human demonstrations using conditional flow matching, then provides the checkpoint for HG-DAgger.Samples contain observations, task instructions, and demonstrated action chunks.Source: §C.1 · p. 39

C.2 Multi-robot HG-DAgger with human interventions

  • Multi-robot HG-DAgger collects expert actions in policy-visited states and trains a shared task-specific policy from human intervention chunks.Successful trajectories are stored with per-frame indicators distinguishing policy-generated actions from human-supplied actions.Source: §C.2 · p. 39

C.2.1 Selecting Intervention Chunks

  • A chunk is eligible only when every valid action in its horizon was supplied by a human, and the first Kint candidate starts in each intervention segment are discarded.The typical horizon is H = 30, or about one second at 30 Hz; the discard filter targets transient actions near control handoffs.Source: §C.2.1 · p. 39

C.2.2 Supervised Fine-tuning with Loss Filtering

  • A batch-level loss filter retains lower-loss samples for gradient updates, using a threshold that combines a loss quantile and an absolute floor.It filters sampled batches rather than stored frames or trajectories, and high loss alone is not treated as evidence of an incorrect human action.Source: §C.2.2 · p. 40

C.2.3 Online Rollout and Policy Synchronization

  • FATS runs environment, rollout, and training workers concurrently, while data balancing and non-blocking weight synchronization accommodate uneven collection.Updated parameters reach rollout workers every Ksync optimizer steps; policy inference is skipped during human control.Source: §C.2.3 · p. 41

D Training System Deployment

The deployment separates robot-side execution from cloud inference and optimization, with asynchronous workers supporting collection and recovery. The appendix also specifies the shared observation and action interface and models how recovery strategies affect collection availability.

D Training System Deployment

D.1 Robot and computing resources

  • The site supports 100 robots, with 96 in the largest training fleet, while six cloud servers provide 48 NVIDIA A800 GPUs.The robot site and cloud connect through a dedicated 30 Gbps link.Source: §D.1 · p. 41, Figure 12 · p. 41, §D.2 · p. 42

D.2 Network topology

  • The robot site and cloud are linked by a dedicated 30 Gbps connection, with tiered site switches and dual 100 Gbps links between interconnected switches.Robots operate at one collection site, and cloud servers are in the provider's data center.Source: §D.2 · p. 42

D.3 Worker roles and placement

  • Environment workers execute actions at the robot site, while cloud inference and training workers exchange observations, actions, and trajectory data across the link.Placement specifications bind workers to resource groups and support either co-located or separate-cluster inference.Source: §D.3 · p. 42

D.4 Recovery cost and fleet size

  • The recovery model contrasts fleet-wide downtime after any fault with per-robot downtime while unaffected robots continue collecting.Under fixed per-robot incident-rate and downtime assumptions, expected Arobot does not depend on fleet size.Source: §D.4 · p. 42
  • For 96 robots, modeled Asys is 91.6%, 81.3%, and 68.5% at 2-, 5-, and 10-minute restart times; at 1,024 robots it is 50.4%, 28.9%, and 16.9%, while Arobot remains 98.7%.These values compare whole-system restarts with per-robot recovery under the stated model.Source: §D.4 · p. 43
  • The recovery estimates assume fleet-invariant per-robot fault rates and downtime, independent robot faults aside from grouped events, and exclude shared cloud or link failures.The model also excludes environment resets and favors whole-system restart by omitting failed-robot repair time and in-progress episodes lost on other robots.Source: §D.4 · p. 43

E.1 Observation preprocessing and action representation

  • The shared policy input comprises three RGB images and a 16-dimensional proprioceptive state, and the VLA predicts 30-step action chunks with 16 dimensions.Each action contains seven joint commands per arm and two gripper commands.Source: §E.1 · p. 43

E.2 Task success and failure criteria

Evaluation scores autonomous trials against ordered task completion and task-specific failure conditions. The criteria include limits on stalled progress, grasp errors, and object placement or assembly.

Common evaluation rules

  • Trials fail if required steps are skipped or out of order, or if progress stalls beyond the task-specific limit; three consecutive permitted empty grasps also cause failure.The no-progress limit is 8 seconds for Flower Arrangement, Phone Charging, and Phone Packaging, and 25 seconds for Stool Assembly.Source: §E.2 · p. 44

Flower Arrangement

  • Flower Arrangement requires grasping the vase horizontally and inserting two roses and a leaf; specified grasp, insertion, displacement, or tipping errors constitute failure.The vase must not be displaced by more than 2 cm or knocked over.Source: §E.2 · p. 44

Clothes Folding

  • Clothes Folding requires five ordered actions, including folding both sides inward, folding the garment in half, and smoothing it.Evaluation uses three garment colors, with 20 consecutive trials per color on each held-out robot.Source: §E.2 · p. 44

Phone Charging

  • Phone Charging requires picking up the cable and adapter, inserting the adapter plug, and completing the specified cable and phone handling steps.The sequence ends with placing the phone on the box and pushing it into position.Source: §E.2 · p. 45

Stool Assembly

  • Stool Assembly requires positioning an arm, seating two screws, grasping a screwdriver, and tightening both screws; table poking or repeated attempts can fail the trial.Attempting the same action more than three times constitutes failure.Source: §E.2 · p. 45

Phone Packaging

  • Phone Packaging requires completing ten ordered steps, including transferring an adapter between grippers, arranging the contents, and fully closing the box.Every specified grasp, transfer, and placement must be completed; the third consecutive empty grasp ends the trial.Source: §E.2 · p. 45

F FERT Implementation Details

FERT uses stored successful autonomous trajectories and failed trajectories as retrieval queries, then fits prioritized successful chunks with the base policy's flow-matching objective. The appendix also gives training duration and baseline initialization details.

F FERT Implementation Details

  • FERT stores successful autonomous trajectories in a rolling buffer and uses failed trajectories as temporary retrieval queries, not as action supervision.This separation guides learning with failure contexts without imitating unsuccessful actions.Source: F FERT Implementation Details · p. 46
  • FERT uses conditional flow matching as a tractable surrogate for likelihood-based fitting of priority-sampled successful chunks.The appendix motivates this choice because evaluating action likelihoods for a flow-based policy requires tracking density changes along the generative flow.Source: F FERT Implementation Details · p. 46
  • FERT runs for approximately 7 to 8 hours of online training, and the checkpoint at convergence is used for final evaluation.The appendix summarizes the key training parameters in Table 5.Source: F FERT Implementation Details · p. 46, Table 5 · p. 46

G Baseline Details

  • RECAP and RLT start from the same selected expert-supervised checkpoint as FERT but differ in how they use experience, human corrections, and policy updates.These configurations define the comparisons used in Section 4.2.Source: G Baseline Details · p. 47

G.1 RECAP

G.1 describes a RECAP offline reinforcement-learning baseline adapted to evaluate fleet-data policy improvement with human corrections; G.2 describes RLT, an online reinforcement-learning baseline trained on one robot with task rewards and human supervision.

G.1 RECAP

  • RECAP provides an offline fleet-data baseline that retains advantage labeling and advantage-conditioned policy training while continuing to use human corrective actions.The implementation performs one round of offline policy improvement from an initialized (M)HG-DAgger checkpoint.Source: §G.1 · p. 47
  • RECAP replaces its broad-data multitask value model with a separate vision-and-state value network for each task, trained on collected task data.It retains distributional value prediction with two-hot targets and undiscounted Monte Carlo returns.Source: §G.1 · p. 47
  • RECAP labels frames above the dataset’s 70th advantage percentile as positive, while human-intervention frames are always positive.Non-intervention samples use indicator dropout with probability 0.3; evaluation uses the positive indicator at guidance scale 1.Source: §G.1 · p. 47

G.2 RLT

  • RLT compares individual-robot online policy improvement using task rewards and human corrections, retaining a two-stage RL-token and actor–critic procedure.The RL token is learned by reconstructing frozen VLA image-prefix features from the SFT dataset.Source: §G.2 · p. 48
  • RLT’s actor predicts residuals around VLA reference actions, with a trust-region mapping that bounds deviations in normalized action coordinates.For Phone Charging, residual corrections are enabled only for the right arm; the left arm and grippers follow the VLA reference.Source: §G.2 · p. 48
  • RLT collects one-robot experience with sparse success rewards and operator interventions, then optimizes with twin critics and behavior-cloning regularization.Operators intervene during difficult segments for 5–10 trajectories as needed; evaluation switches between VLA and RLT automatically, without operator phase selection or corrective actions.Source: §G.2 · p. 48

H Extended Experimental Results

The appendix details evaluation results, training-run traces, and examples of failure-guided retrieval. It reports the held-out evaluation design, variable teleoperation performance, system activity and incidents, and visual examples of retrieved successful segments.

H.1 Detailed Success Rate

  • Table 11 compares SFT, (M)HG-DAgger, RLT, RECAP, and FERT across five tasks and evaluated fleet sizes using two held-out robots and 120 trials per checkpoint.Average success rates pool the two robots’ 60 trials each; the caption identifies the highest average for each task but does not provide the values here.Source: Table 11 · p. 50, §H.1 · p. 50
  • Table 12 reports FERT ablations on Phone Packaging with 16 training robots, comparing configurations over 120 held-out trials each.The configurations include full FERT, removal of failure-guided retrieval, and no extension; the caption provides no success-rate values.Source: §H.1 · p. 50, Table 12 · p. 51
  • Phone Packaging teleoperation success varied from 25% (5/20) to 95% (19/20) across operators, with an overall average of 75%.Each of six operators completed 20 trials per task.Source: §H.1 · p. 51, Table 13 · p. 51

H.2 Detailed System Traces

  • The 96-robot run contains 9,834 episodes, with success and failure patterns differing across robots; blank intervals include resets and operator breaks.Robots 3 and 4 succeed in most episodes, whereas Robots 6 and 7 fail in most.Source: §H.2 · p. 51, Figure 14 · p. 53
  • Outside the lunch break, an average of 65 robots were executing actions at a time during the 96-robot run.Figure 15 distinguishes raw counts from counts smoothed with a 60-second Gaussian standard deviation.Source: §H.2 · p. 51, Figure 15 · p. 54
  • The run records 30 fault incidents affecting 22 robots, with Figure 16 encoding incident timing, robot identities, and error types.The trace intervals support the fault-event grouping and recovery statistics reported in the main text.Source: §H.2 · p. 51, Figure 16 · p. 54

H.3 Visualization of failure-guided retrieval

  • Figures 17–21 pair failed execution contexts with retrieved successful segments across five tasks, illustrating the examples used to visualize FERT retrieval.In Phone Packaging, the lid-placement example contrasts repeated failed angle adjustments with three successful attempts despite limited visibility and high precision demands.Source: §H.3 · p. 51, Figure 21 · p. 57
  • In Clothes Folding, FERT retrieves the same critical step across differently colored shirts when the feature representation is appropriate.The example concerns failure to grasp a T-shirt’s right corner with the right gripper.Source: Figure 18 · p. 55

Related papers

Loading 2610.25736v2…