PaperlayerView exact revision on arXiv

Source-linked AI summary

ARS: Agentic Reward System for Robot Learning

Sheng Hu, Weiyi Lu, Lingbing Zeng, Gan Weng, Weiwei Zhang, Kai Xie, Xiaofeng Mou, Yi Xu

arXiv:2609.34484v1cs.ROcs.LG

TL;DR

Robot progress rewards must distinguish meaningful task changes from failed attempts and irrelevant actions. ARS uses adaptive visual inspection and cross-agent event verification to estimate progress without additional reward-model training. It better suppresses wrong-object errors and improves policy learning in simulation and on a real robot.

  • Problem

    Estimating robot task progress requires distinguishing meaningful state changes from failed attempts and task-irrelevant actions in visual trajectories.

    Source: §1 · p. 1, §3.2 · p. 4

  • Method

    ARS uses adaptive visual inspection for event proposals and verification, then estimates per-frame progress from verified events without additional reward-model training.

    Source: §1 · p. 2, §3.5 · p. 5

  • Results

    ARS better suppresses wrong-object errors; LIBERO-Long policy success improves by 13.7 percentage points over unfiltered behavior cloning and 10.3 points over the strongest compared reward baseline.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results suggest structured inference and verification can improve the usefulness of general-purpose VLMs for robot reward modeling.

    Source: §1 · p. 2

    Iterative visual inspection and generative verification add inference overhead relative to single-forward-pass reward models, making the current workflow particularly suited to offline data curation.

    Source: §5 · p. 9

Abstract

from arXiv · show

Progress reward modeling is the problem of estimating how a robot's behavior changes task progress over time. Reliable estimation requires distinguishing meaningful state changes from failed attempts and task-irrelevant actions. We introduce the Agentic Reward System (ARS), an inference framework for progress reward modeling with general-purpose vision-language models (VLMs), without additional reward-model training. Given an offline trajectory and a task instruction, ARS uses adaptive visual inspection for both event proposal and verification. A subagent proposes a task-relevant event timeline, which a primary agent verifies and revises before estimating per-frame progress. ARS can incorporate optional terminal outcome labels and visual references to inform its judgments. It can also audit progress estimates from external reward models. We evaluate ARS with a 27B VLM on a controlled semantic-mismatch benchmark and downstream policy learning in simulation and on a real robot. The benchmark reveals that several evaluated reward baselines assign spurious progress to wrong-object manipulation even in simple pick-and-place scenes. ARS better suppresses these errors and outperforms these baselines in simulation policy learning. We further demonstrate that ARS supports long-horizon policy learning from mixed-quality offline experience on real-robot multi-screw fastening in a full-scale laboratory replica of an industrial washing-machine assembly line. These results suggest that structured inference and verification can improve the usefulness of general-purpose VLMs for robot reward modeling. Code is at https://github.com/midea-ai/ars

1 INTRODUCTION

Reliable robot rewards require estimating task progress from visual evidence while distinguishing meaningful state changes from failed attempts and irrelevant actions. ARS uses adaptive inspection and cross-agent verification to build progress estimates, and its evaluations show improved suppression of wrong-object errors and downstream policy performance.

Motivation

  • Progress estimation must distinguish task-relevant state changes from failed attempts and irrelevant actions because mixed-quality trajectories can contain useful actions even when their overall outcome fails.Visual progress estimates can identify local differences and provide training supervision from mixed-quality experience.Source: §1 · p. 1
  • ARS evaluates how inference procedure affects general-purpose VLM reward judgments against prior approaches that elicit progress from pretrained models or train dedicated reward models.Prior studies report that specialized robotic reward models can outperform directly prompted frontier flagship VLMs of their time.Source: §1 · p. 1

Contribution

  • ARS builds on structured VLM reasoning by using unresolved event claims to guide further visual inspection and revisions to an explicit event timeline.The workflow can also incorporate terminal outcomes and visual references or audit external reward predictions.Source: §1 · p. 2
  • ARS uses a subagent to propose task-relevant events and a primary agent to verify and revise them before estimating per-frame progress.Agents adaptively inspect temporal windows, sampling densities, and spatial crops; progress can reflect failed attempts and regressions after setbacks without additional reward-model training.Source: §1 · p. 2, Figure 1 · p. 2

Evaluation

  • ARS-selected action chunks improve LIBERO-Long policy success by 13.7 percentage points over unfiltered behavior cloning and 10.3 points over the strongest compared reward baseline.On real-robot multi-screw fastening, ARS selection with visual references raises success from 58.0% for demonstration-only behavior cloning to 81.5%.Source: §1 · p. 2
  • On a controlled semantic-mismatch benchmark, several evaluated reward systems assign spurious progress to wrong-object manipulation, while ARS better suppresses these errors.The benchmark includes simple pick-and-place scenes with only two candidate objects.Source: §1 · p. 2

2 RELATED WORK

Prior work spans learned and training-free robotic reward estimation, structured reasoning, and agentic evidence gathering. ARS combines active visual inspection with separate event proposal and verification for executed robot trajectories.

Robotic reward and progress estimation

  • Robotic reward methods include pretrained visual representations, video-language similarity, temporal supervision, VLM-generated preferences, dedicated progress models, and training-free approaches such as GVL and TOPReward.Other lines use stage structure, relative progress, label correction, online refinement, confidence gating, or prompt optimization.Source: §2 · p. 3
  • Structured reasoning approaches include recursive subtask reasoning in ROVER, spatiotemporal reasoning in SOLE-R1, procedural reasoning in ProcVLM, retrieval and mental simulation in ProgressLM, and recurrent task decomposition in R2VLM.ARS builds on these directions by letting unresolved event claims guide further visual inspection and event-timeline revision.Source: §2 · p. 3

Agentic inference and reward verification

  • Agentic inference methods gather or check evidence through tool use, visual retrieval, multimodal tools, evaluator agents, or environment probing.ARS applies active verification to executed robot trajectories, coupling event proposals with verification in separate reasoning contexts and requiring evidence coverage and renewed confirmation after revisions.Source: §2 · p. 3

3 AGENTIC REWARD SYSTEM

ARS estimates progress from offline robot trajectories by grounding task understanding in visually inspected events. Its workflow combines adaptive inspection, event coverage and verification, progress construction, optional auxiliary information, and progress-guided policy learning.

3.1 PROBLEM SETUP

  • ARS frames progress estimation as mapping an offline trajectory’s observation sequence and task instruction to per-frame task progress.Evidence for an event may lie beyond the current frame, such as a later lift clarifying whether an apparent grasp succeeded.Source: §3.1 · p. 3

3.2 ADAPTIVE VISUAL INSPECTION

  • Complete video access does not ensure close examination: brief transitions can be missed and failed attempts conflated with later successful interactions.ARS adaptively requests temporal windows, sampling density, and optional spatial crops, including later observations for retrospective verification.Source: §3.2 · p. 4

3.3 EVENT-GROUNDED REASONING

  • ARS represents a trajectory as task-relevant events with temporal intervals and descriptions grounded in visual evidence, requiring full-trajectory event coverage before proposal generation finishes.When evidence does not cover an event interval, the harness returns missing intervals for correction.Source: §3.3 · p. 4

3.4 CROSS-AGENT VERIFICATION

  • A subagent proposes the event timeline, while a primary agent in a separate reasoning context gathers evidence to verify and revise proposals before estimating per-frame progress.Revisions invalidate prior confirmation; separating contexts also preserves primary-agent capacity by avoiding transfer of the subagent’s full interaction history.Source: §3.4 · pp. 4–5

3.5 PROGRESS ESTIMATION

  • The primary agent constructs and revises a progress curve from verified events, using interpolation or pointwise edits and readbacks to check jumps against visual evidence.The illustrated LIBERO trajectory captures regression when the stove is switched off while retaining subsequent progress toward pot placement.Source: Figure 2 · p. 4, §3.5 · p. 5

3.6 FLEXIBLE INTEGRATION OF AUXILIARY INFORMATION

  • ARS can anchor terminal completion with trajectory-level success labels, use unannotated or annotated multimodal references to clarify task criteria, and audit external reward curves against verified events.The corrected curves and audit traces could also be explored as potential supervision for reward-model training.Source: §3.6 · p. 5

3.7 PROGRESS-GUIDED POLICY LEARNING

  • Progress-guided policy learning scores chunks by progress change and uses a binary-weighted behavior-cloning objective to select training chunks.The threshold can be zero, retaining positive-progress chunks, or stage-dependent, retaining a top fraction ranked within progress bins.Source: §3.7 · pp. 5–6

4 EXPERIMENTS

The experiments evaluate task alignment on a controlled semantic-mismatch benchmark and progress-guided policy learning in simulation and on a real robot, with ablations of model choice and inference workflow.

4.1 SEMANTIC-MISMATCH BENCHMARK

  • On the semantic-mismatch benchmark, ARS leads all three metrics; Robometer combines 0.983 VOC with only 0.240 FPSR.R2VLM assigns higher mean peak progress to mismatched instructions than mean terminal progress to correct ones, yielding negative CMPG.Source: Table 1 · p. 7, §4.1 · p. 7
  • Auditing Robometer raises FPSR from 0.240 to 0.915 and CMPG from 0.366 to 0.891 while maintaining VOC of 0.988.These results approach ARS without proposals.Source: §4.1 · p. 7

4.2 POLICY LEARNING IN SIMULATION

  • With terminal outcome labels, ARS reaches 62.2% success, 9.6 points above success-only BC and 4.3 points below the 66.5% privileged-state reference.Successful trajectories can still contain flat or decreasing progress, while unsuccessful ones can contain increasing segments.Source: §4.2 · pp. 7–8, Table 2 · p. 8
  • ARS reaches 57.9% success on LIBERO-Long, highest among trajectory-and-instruction systems, exceeding Vanilla BC by 13.7 percentage points.ARS and TOPReward-32B retain similar chunk proportions, yet ARS is 10.3 points higher in success.Source: §4.2 · p. 7, Table 2 · p. 8

4.3 ABLATION STUDY

  • Qwen3.6-27B improves FPSR and CMPG over Qwen3.5-4B, while ARS with Qwen3.5-4B exceeds Robometer-4B on all three metrics.With Qwen3.6-27B fixed, single-prompt inference has lower VOC and CMPG than ARS.Source: §4.3 · p. 8, Table 3 · p. 8
  • Against TOPReward-32B, full ARS raises success from 47.6% to 57.9%, using 93.6% fewer queries and 65.8% fewer input tokens.Separate-context verification reaches 57.9%, 6.1 points above the same-context control.Source: §4.3 · p. 8, Table 4 · p. 9

4.4 REAL-WORLD POLICY LEARNING

  • On real-robot screw fastening, adding all rollouts lowers success from 58.0% to 50.5%, while ARS-selected rollouts reach 72.0%.Visual references further raise observed success to 81.5%, 23.5 points above demonstration-only BC.Source: Table 5 · p. 9, §4.4 · p. 9

5 DISCUSSION

ARS improves downstream performance with fewer model queries and cumulative input tokens than frame-wise TOPReward, but its iterative inspection and verification add overhead over single-pass reward models. The appendices describe its staged prompts, adaptive inspection, and tool requirements.

5 DISCUSSION

  • ARS improves downstream performance while using fewer model queries and cumulative input tokens than frame-wise TOPReward, but adds inference overhead compared with single-forward-pass models such as Robometer.The authors say this tradeoff makes the workflow particularly suited to offline data curation; causal online reward estimation remains open.Source: §5 · p. 9

A AGENT PROMPTS AND TOOL SET

  • A subagent proposes events, while a primary agent independently verifies them and estimates progress in a separate reasoning context.The proposal stage uses the full video when it fits the configured input limit and otherwise inspects segments.Source: A AGENT PROMPTS AND TOOL SET · p. 15

A.1 PROMPT SUMMARIES

  • The prompt-summary table organizes ARS instructions by component, retaining core reasoning and evidence requirements while omitting repetitive instructions and interface examples.The default workflow consists of event proposal, independent verification, and progress estimation.Source: §A.1 · p. 15, Table 6 · p. 15, A AGENT PROMPTS AND TOOL SET · p. 15

B ARS REASONING TRACE ON A LIBERO TRAJECTORY

In a partially successful LIBERO trajectory, independent verification revises a proposed event boundary closer to the manually identified transition, though it still places control acquisition too early. The verified timeline also captures a setback and later task progress.

Subagent inspection and event proposal

  • The subagent made nine additional visual inspections and proposed eight events; the primary agent made eight inspections before revising the timeline.The trajectory has 520 frames and follows the instruction to turn on the stove and put the moka pot on it.Source: B ARS REASONING TRACE ON A LIBERO… · p. 17, Table 8 · p. 17

Primary-agent verification and boundary refinement

  • Verification shortened the approach event to frames 0–45 and moved the start of knob manipulation to frame 46 after inspecting frames 0–100.The primary agent separately assessed the proposed timeline and revised its event boundary.Source: B ARS REASONING TRACE ON A LIBERO… · p. 17
  • Verification moved the knob-manipulation boundary from frame 95 to frame 46, reducing its error from 39 frames late to 10 frames early relative to the manually identified onset at frame 56.The revised boundary remained early, illustrating that subtle manipulation events can still be difficult to localize precisely.Source: B ARS REASONING TRACE ON A LIBERO… · p. 17

Progress estimation

C IMPLEMENTATION DETAILS OF COMPARED METHODS

The compared reward systems use different input protocols and output conventions, so their native outputs are not assumed to share calibration. The ManiSkill exposure summary labels documented benchmark-family training exposure without establishing overlap with evaluation trajectories.

C IMPLEMENTATION DETAILS OF COMPARED METHODS

  • Compared methods differ in how they estimate progress: LRM emits discrete frame-level values, R2VLM carries forward subtask reasoning, SOLE-R1 uses per-timestep reasoning, Robometer predicts from sampled frames, and TOPReward scores video prefixes.For Robometer, sliding-window inference and larger frame counts did not improve on the original eight-frame setting.Source: C IMPLEMENTATION DETAILS OF COMPARED… · p. 18
  • The evaluation uses method-specific output conventions rather than assuming common calibration, including rescaling R2VLM and preserving SOLE-R1’s protocol-defined initial zero.ARS integer progress points are divided by 1000; TOPReward’s primary convention linearly normalizes log-probabilities.Source: C IMPLEMENTATION DETAILS OF COMPARED… · p. 18

D.1 TRAINING-DATA EXPOSURE TO MANISKILL

  • ManiSkill exposure labels indicate documented benchmark-family use during training or no evidence of such use, but do not establish overlap with evaluation trajectories or rule out underlying-VLM exposure.Source: §D.1 · pp. 18–19, Table 9 · p. 19

D.2 FALSE PROGRESS SUPPRESSION RATIO (FPSR)

On the semantic-mismatch benchmark, ARS combines strong suppression of false progress with broad coverage of intended expert progress, while tolerance increases can also remove useful signal. CMPG provides a complementary measure of separation between correct-task completion and peak mismatch progress.

D.2 FALSE PROGRESS SUPPRESSION RATIO (FPSR)

  • TOPReward-8B probability outputs raise FPSR from 0.368 to 0.853 as η increases from zero to 10^-4, while EPCR falls from 0.656 to 0.354.ARS EPCR also falls to 0.588 at η = 0.01, so higher suppression can come with reduced useful coverage.Source: §D.2 · pp. 19–20
  • For 0 ≤ η ≤ 10^-3, ARS has FPSR 0.918–0.921 and EPCR 0.902–0.908; LRM has FPSR 0.893 but covers only 0.159 of expert-trajectory duration.The table reports FPSR / EPCR on native predictions; these results pair mismatch suppression with coverage under correct instructions.Source: §D.2 · p. 19, Table 10 · p. 20

D.3 CORRECT–MISMATCH PROGRESS GAP (CMPG)

  • Table 1 reports TOPReward’s probability-based CMPG values as more favorable among the two evaluated conventions; both remain below ARS’s CMPG of 0.904.CMPG measures separation between correct terminal progress and peak mismatch progress and is sensitive to output scale.Source: §D.3 · pp. 20–21

E.1 TRAINING-DATA EXPOSURE TO LIBERO

  • The LIBERO exposure labels report documented use or no found evidence of use, but do not establish overlap with the specific policy-training trajectories or exclude underlying-VLM exposure.Source: §E.1 · p. 21, Table 11 · p. 21

E.2 TASK INSTRUCTION CLARIFICATIONS

  • For basket-placement tasks, brief object color, shape, and packaging descriptions address difficult visual matching at 256 × 256 resolution; the same instruction clarifications go to ARS and all baselines.The mug-placement clarification specifies robot-relative left and right directions and their corresponding image directions.Source: §E.2 · p. 21, Table 12 · p. 22

E.3 HAND-DESIGNED PRIVILEGED PROGRESS

  • The privileged placement signal combines reaching, lifting, and target-distance progress, while articulated-object progress uses changes in joint distance to the target range.The signal uses simulator state and privileged task information during offline selection; scoring parameters were fixed before downstream policy training.Source: §E.3 · pp. 21–22

E.4 POLICY TRAINING AND EVALUATION

Under a shared policy-training protocol, ARS exceeds the strongest compared baseline at the final checkpoint; diagnostics also show that progress curves can yield sparse chunk selection and retain local changes under outcome conditioning.

E.4 POLICY TRAINING AND EVALUATION

  • All reward systems share the rollout dataset, policy initialization, training budget, and evaluation procedure, with progress changes determining action-chunk contributions.This protocol connects progress estimates to policy supervision without an additional learned critic.Source: §E.4 · p. 22
  • The simulation comparison retains chunks with ∆t > 0 at equal weight, avoiding shared weighting hyperparameters but discarding progress magnitude.The authors note that this coarse rule reduces dependence on numerical calibration across reward systems.Source: §E.4 · p. 23
  • Simulation uses a zero threshold; real-robot experiments use stage-dependent thresholds while retaining all demonstration chunks.The zero threshold retains slow, potentially necessary transitions; selecting only high-gain chunks may reduce their coverage.Source: §E.4 · p. 23
  • Policies train for 30k steps with three seeds and are evaluated every 2k steps on all ten LIBERO-Long tasks, using 1,000 rollouts per checkpoint.For each seed, the reported peak is the checkpoint with the highest mean success across tasks; the same evaluation set is used for selection and reporting.Source: §E.4 · p. 23
  • Because retained datasets differ in size, training fixes gradient updates rather than epochs; all methods plateau, and final-checkpoint results are additionally reported.Methods retaining fewer chunks revisit their samples more frequently.Source: §E.4 · p. 23
  • At the final checkpoint, ARS achieves 51.6% success, 11.5 percentage points above Vanilla BC and 9.5 points above Robometer.Table 13 reports final-checkpoint LIBERO-Long results as mean success rate ± standard deviation over three training runs per method.Source: §E.4 · p. 23, Table 13 · p. 23

E.5 PROGRESS-CURVE DIAGNOSTICS

  • LRM retains 13.7% of chunks and yields 38.6% success, while R2VLM retains 13.6% and yields 41.5%.Zero progress differences affect 76.7% of LRM candidate chunks; R2VLM often repeats an early estimate, producing zero changes for 79.5%.Source: §E.5 · pp. 23–24
  • Among 3,098 trajectories with matching terminal-outcome predictions, conditioning leaves non-increasing intervals on success-predicted trajectories at 21.45% and increasing intervals on failure-predicted trajectories at 23.38%.Without conditioning, these figures are 21.70% and 27.03%, respectively; the authors interpret the pattern as endpoint anchoring with intermediate progress still guided by visual evidence.Source: §E.5 · p. 24

F.1.1 ADDITIONAL BASE-MODEL EVALUATION FOR TOPREWARD

The appendix evaluates TOPReward under a different base model, small-model execution on longer trajectories, and ARS workflow variants. Results show lower downstream success for workflow ablations on LIBERO-Long, while context separation reduces primary-agent input tokens.

F.1.1 ADDITIONAL BASE-MODEL EVALUATION FOR TOPREWARD

  • TOPReward with Qwen3.6-27B achieves 0.452 VOC, 0.471 FPSR, and −0.049 CMPG; tested prompt and chat-template alternatives do not yield competitive results in this setup.The authors report reasonable results for TOPReward with Qwen3-VL-8B-Instruct and Qwen3-VL-32B-Instruct under the public inference implementation.Source: §F.1.1 · p. 24
  • The authors hypothesize prompt-format sensitivity may partly explain TOPReward’s difficulty, but their experiments do not isolate it as the cause of degradation.They retain the original Qwen3-VL family in the main comparisons and report the Qwen3.6-27B adaptation separately.Source: §F.1.1 · p. 24

F.1.2 BASE-MODEL EXECUTION REQUIREMENTS

  • Qwen3-VL-4B-Instruct could not complete the ARS workflow in the attempted runs, so the Robometer comparison uses Qwen3.5-4B, a different base model at the same nominal parameter scale.The attempted failures involved tool use and harness constraints.Source: §F.1.2 · p. 24

F.1.3 SMALL-MODEL EXECUTION ON LONG-HORIZON TRAJECTORIES

  • On LIBERO-Long, Qwen3.5-4B produces valid ARS outputs for 4,063 of 4,096 trajectories, a 99.19% completion rate.The remaining 33 runs were marked as failures after extended attempts, including repeated harness-constraint violations.Source: §F.1.3 · p. 24

F.2.1 ARS WORKFLOW VARIANTS

  • The ablations omit inspection and verification, omit verification alone, or combine proposal, verification, and progress estimation in one context rather than separating proposal from later stages.All variants use Qwen3.6-27B with the same inference settings and progress-scoring criteria.Source: §F.2.1 · pp. 24–25
  • During verification and progress estimation, the primary agent averages 41.4K input tokens per query, 28.1% below the single-context control’s 57.6K.The authors relate this to retaining event proposals without carrying the subagent’s full interaction history.Source: §F.2.1 · p. 25, Table 14 · p. 25
  • All three ARS workflow ablations yield lower LIBERO-Long policy success than full ARS, although they score close to it on the short semantic-mismatch benchmark.The short trajectories average 31.7 frames, so that benchmark provides limited separation among the workflows.Source: §F.2.1 · pp. 24–25, Table 15 · p. 25

F.2.2 GVL-STYLE SINGLE-PROMPT EVALUATION

Single-prompt GVL-style progress estimates depend strongly on frame ordering and perform poorly for LIBERO-Long chunk selection. ARS’s event timeline and evidence links also support intermediate reasoning without requiring standalone language CoT.

F.2.2 GVL-STYLE SINGLE-PROMPT EVALUATION

  • The evaluation gives the instruction and all recorded frames in one response, using the initial frame as reference; it compares chronological and shuffled frame presentation.Shuffled predictions are mapped back to original temporal order before evaluation and chunk selection.Source: §F.2.2 · p. 25
  • On the semantic-mismatch benchmark, chronological GVL-style inference scores 0.637 VOC, 0.900 FPSR, and 0.354 CMPG, versus ARS’s 0.918 FPSR.The authors say the lower VOC and CMPG indicate weaker progress ordering and separation between correct and mismatched instructions.Source: §F.2.2 · p. 26
  • On LIBERO-Long, shuffling raises GVL-style policy success to 34.1%, still 10.1 points below Vanilla BC and 23.8 points below ARS with the same base model.Chronological presentation retains 3.58% of chunks and yields 0.0% success; shuffling retains 14.80%, while Table 16 reports retained chunk percentages and policy success.Source: §F.2.2 · p. 26, Table 16 · p. 26

F.3 THE FORM OF INTERMEDIATE REASONING

  • R2VLM and SOLE-R1 organize intermediate reasoning through explicit language CoT, using recurrent task decomposition or spatiotemporal CoT, respectively.Removing CoT from SOLE-R1 progress training produces flatter, noisier rewards and weakens downstream learning.Source: §F.3 · p. 26
  • ARS without thinking mode reaches 55.4% downstream success, 11.2 points above Vanilla BC; enabling thinking adds 2.5 points to 57.9%.Table 17 reports the effect of thinking mode under the same ARS workflow with Qwen3.6-27B.Source: §F.3 · p. 26, Table 17 · p. 26
  • The authors conclude standalone language CoT is not the only effective way to organize intermediate reasoning for reward estimation.The ARS harness maintains and revises reasoning state through an event timeline, evidence links, and a progress curve.Source: §F.3 · p. 26

G.1 TASK AND EVALUATION PROTOCOL

The real-robot study evaluates multi-screw fastening with mixed demonstration and rollout data, visual references, and optional outcome conditioning. Its reported conditioning gain is not statistically significant, and the evaluation uses one training seed per configuration.

G.1 TASK AND EVALUATION PROTOCOL

  • The task requires fastening all three screws on the washing machine’s rear panel, with success confirmed by a human evaluator.Timeouts are automatically marked as failures, but avoiding timeout alone does not establish success.Source: §G.1 · p. 27
  • Because real-robot evaluation is time-consuming, each configuration uses one training seed and 200 complete-task trials, leaving between-seed variability unquantified.Source: §G.1 · p. 27

G.2 POLICY TRAINING AND DATA SAMPLING

  • The dataset contains 2,574 demonstration trajectories and 3,092 offline rollout trajectories; ARS configurations combine demonstrations with selected rollout chunks.Policies share a pretrained checkpoint, use an action-chunk horizon of 50, and are evaluated at 200,000 gradient steps.Source: §G.2 · p. 27
  • Rollout-containing configurations sample demonstration and rollout chunks at a fixed 1:1 ratio, with uniform sampling within each source.For ARS configurations the rollout pool is the retained subset, and the BC loss is applied separately to each source.Source: §G.2 · p. 27
  • ARS receives two positive visual references for screw engagement and full tightening, plus one negative reference showing stalled fastening.Each reference is a single image with a brief description and no numerical progress annotation.Source: §G.2 · p. 27

G.3 OUTCOME CONDITIONING ON THE REAL ROBOT

  • Adding terminal outcome labels to ARS with visual references raises observed success from 81.5% to 84.0%, but the gain is not statistically significant (two-sided Fisher’s exact test, p = 0.597).Source: §G.3 · p. 27

Related papers

Loading 2609.34484v1…