PaperlayerView exact revision on arXiv

Source-linked AI summary

ReWeight: Leveraging Human Data for VLA Post-Training via Demonstration Retrieval and Sample Weighting

Chenwei Wang, Dianye Huang, Match W. L. Ko, Chenjia Bai, Zhongliang Jiang

arXiv:2609.13851v1cs.ROcs.AI

TL;DR

Robot data collection is costly, and indiscriminate human–robot data mixing can introduce interference. ReWeight retrieves relevant human demonstrations and weights samples by cross-embodiment relevance; it reports higher success rates than robot-only and random-mix baselines across simulation and real-world evaluations.

  • Problem

    Robot data collection requires task-specific hardware and repeated interaction, while indiscriminate cross-embodiment mixing can introduce interference and degrade VLA performance.

    Source: §I · p. 1

  • Method

    ReWeight learns a cross-embodiment visuomotor representation to retrieve relevant human demonstrations and assign selected samples continuous relevance-based weights.

    Source: §I · p. 2

  • Results

    57% average simulation success (vs. 39% robot-only and 44% random-mix) and 68.8% real-world success (28.8 and 13.8 percentage points above the respective baselines).

    Source: §V · p. 9

  • Takeaways & Limitations

    ReWeight offers a way to use egocentric human demonstrations as selectively retrieved and sample-weighted supervision for VLA post-training.

    Source: §V · p. 8

Abstract

from arXiv · show

Post-training vision-language-action (VLA) models for specific robots and tasks requires in-domain demonstrations, yet collecting diverse robot data is costly. Egocentric human demonstrations provide a scalable alternative, but directly mixing human and robot data can introduce cross-embodiment discrepancies and degrade policy performance. To address this challenge, we introduce ReWeight, a framework that incorporates human data into VLA post-training through demonstration-level retrieval and sample-level weighting. ReWeight learns a cross-embodiment visuomotor representation that combines visual observations with future actions to measure behavioral similarity between human and robot demonstrations. Based on optimal transport, it retrieves human demonstrations relevant to the target robot data and assigns larger weights to samples with smaller cross-embodiment discrepancies. We evaluate ReWeight using $\pi_{0.5}$ across eight simulation tasks and four real-world tasks under both clean and randomized settings. In simulation, ReWeight improves the average success rate of post-trained $\pi_{0.5}$ from 39% with only robot data and 44% with randomly mixed human-robot data to 57%. In the physical experimental setting, it achieves an average success rate of 68.8%, outperforming the baselines by 28.8% and 13.8%, respectively. Overall, ReWeight provides an effective paradigm for transforming abundant egocentric human experience into transferable supervision for robot learning. (Project webpage: https://reweight-vla.github.io/)

I. INTRODUCTION

ReWeight addresses the cost of collecting diverse robot demonstrations and the risks of indiscriminately mixing human and robot data. It combines demonstration retrieval with sample-level weighting for VLA post-training.

I. INTRODUCTION

  • Robot data collection requires task-specific hardware setup and repeated interaction, making diverse data costly and difficult to acquire at scale.Source: §I · p. 1
  • Indiscriminate cross-embodiment mixing can degrade VLA performance, while retrieval alone misses variation in transferability among selected demonstrations.The paper frames human-data use as deciding both which demonstrations to transfer and how strongly each should influence training.Source: §I · pp. 1–2
  • ReWeight retrieves human demonstrations relevant to target robot data and assigns continuous weights according to cross-embodiment relevance.The framework learns a shared visuomotor representation to compare human and robot demonstrations.Source: §I · p. 2
  • 57% average simulation success and 68.8% average real-world success are reported for ReWeight, compared with 39% robot-only simulation success and 40.0% robot-only real-world success.The evaluation covers 8 simulation tasks and 4 real-world tasks.Source: §I · p. 2

A. Problem Description

The problem description frames human-data use as selecting a fixed-size subset relevant to the target robot data, then post-training on that subset together with robot demonstrations.

A. Problem Description

  • The objective is to select M human demonstrations most similar to the target robot data and combine them with the robot demonstrations for VLA post-training.The subset size is constrained to M, the number of robot demonstrations.Source: A. Problem Description · p. 2
  • The formulation seeks to leverage relevant human manipulation diversity while improving downstream task performance.Source: A. Problem Description · p. 2
  • Robot and human datasets include visual observations, robot control commands or estimated human hand kinematics, and corresponding textual commands.Source: A. Problem Description · p. 2

B. Optimal Transport for Data Discrepancy Measurement

ReWeight uses optimal transport to quantify discrepancies between human and robot demonstrations and between individual human samples and robot demonstrations. These measures guide retrieval and sample weighting.

B. Optimal Transport for Data Discrepancy Measurement

  • Optimal transport measures both demonstration-level and sample-level human–robot discrepancies to guide retrieval and sample weighting.Source: B. Optimal Transport for Data… · p. 2
  • The transport plan matches human and robot timesteps using pairwise costs between their observations, with entropy regularization controlled by ϵ.The plan has uniform marginal distributions and is solved using the Sinkhorn algorithm [21].Source: B. Optimal Transport for Data… · pp. 2–3
  • The optimal transport plan yields a demonstration-level discrepancy and a separate discrepancy for each human timestep.Source: B. Optimal Transport for Data… · p. 3

III. METHOD

ReWeight refines human actions for the robot embodiment and learns a cross-embodiment visuomotor representation to compare human and robot demonstrations. Visual pretraining first captures task-relevant features, then aligns representations across embodiments.

A. Embodiment-Aware Human Action Refinement

  • A residual policy refines retargeted human actions to address robot dynamic constraints, producing robot-embodiment-aware motion.The policy adds a gated residual correction and is trained with pose-tracking, velocity-matching, and regularization rewards.Source: A. Embodiment-Aware Human Action… · p. 3

B. Cross-Embodiment Visuomotor Encoder

1) Visual Encoder Pretraining:

  • Alignment combines domain-adversarial learning with OT correspondence loss to suppress embodiment-specific cues and bring task-consistent observations closer in latent space.The OT cost combines visual-feature similarity with a temporal-mismatch penalty based on a soft-DTW alignment path.Source: 1) Visual Encoder Pretraining · p. 4

2) Visuomotor Representation Learning:

ReWeight encodes visual observations and future action chunks in a shared latent representation. Its training objectives reconstruct visual and motion information while aligning representations by task stage across embodiments.

2) Visuomotor Representation Learning:

  • An InfoVAE-based visuomotor encoder separately encodes visual features and future action chunks, then fuses them into latent representation z.The future action chunk spans 20 steps, is represented relative to its initial pose, and preserves gripper state.Source: 2) Visuomotor Representation Learning · p. 4
  • A stage-prediction head preserves task-stage information, while an auxiliary loss aligns human and robot representations from the same stage and separates different stages.The different-stage similarity penalty applies when similarity exceeds margin μ = 0.3.Source: 2) Visuomotor Representation Learning · p. 4
  • The VAE objective combines reconstruction, regularization, and latent losses to retain visual and motion information, suppress embodiment variation, and preserve task progression.The reconstruction loss covers visual features and relative motion trajectories; regularization combines KL divergence and maximum mean discrepancy.Source: 2) Visuomotor Representation Learning · p. 4

C. Human Demonstration Retrieval and Sample Weighting

ReWeight selects human demonstrations by comparing each candidate with robot demonstrations of the same task, then retaining the lowest-scoring candidates for post-training.

1) Demonstration-level Retrieval:

  • Each candidate human demonstration is scored by its average discrepancy to its K nearest robot demonstrations, using latent features and cosine distance in the retrieval cost matrix.K is empirically set to 5 in all experiments.Source: 1) Demonstration-level Retrieval · p. 4
  • ReWeight selects the M lowest-scoring human demonstrations, yielding a 1:1 human-to-robot data ratio for VLA post-training.M is the number of robot demonstrations in each task.Source: 1) Demonstration-level Retrieval · p. 4

2) Sample-level Weighting:

After retrieval, ReWeight assigns sample-level weights according to how closely each retained human sample matches robot demonstrations, giving greater weight to smaller discrepancies.

2) Sample-level Weighting:

  • Each retained human sample receives a discrepancy-aware weight because frames within a retrieved demonstration can vary in relevance to robot data.The sample-level discrepancy compares the human sample with its K nearest robot demonstrations; smaller discrepancy indicates greater consistency.Source: 2) Sample-level Weighting · p. 5
  • ReWeight converts sample discrepancies into similarity weights, then normalizes the weights using calibration parameter α to ensure sufficient influence of human samples.Source: 2) Sample-level Weighting · p. 5

3) Loss for Post-Training:

Post-training optimizes a mixed robot and retrieved-human dataset using per-sample VLA losses and human-sample weights. Human trajectories are refined with a residual RL policy, and the visuomotor encoder is trained in stages.

3) Loss for Post-Training:

  • The post-training loss uses mixed robotic and retrieved human data, with a per-sample VLA loss and human weights specified for the objective.The cited passages do not provide the full displayed loss equation.Source: 3) Loss for Post-Training · p. 5

1) Residual RL Policy:

  • A separate actor–critic residual policy is trained with PPO for 50k environment steps per task, then frozen to refine retargeted human trajectories.The policy uses learning rate 1×10−4, batch size 64, and discount factor 0.95.Source: 1) Residual RL Policy · p. 5

2) Visuomotor Encoder:

  • The visuomotor encoder is warmed up for 20 epochs, aligned across domains for 260 epochs, then trained with the latent objective for 500 epochs.Encoder training uses AdamW with learning rate 1 × 10−4 and batch size 64.Source: 2) Visuomotor Encoder · p. 5

3) VLA Post-Training:

Across simulation tasks, retrieval and sample-specific weighting outperform indiscriminate human-data use, with gains reported in both clean and randomized settings. The experiments compare post-trained π0.5 models across eight simulated tasks and four real-world tasks.

3) VLA Post-Training:

  • All methods use the same pretrained π0.5 checkpoint and optimization settings, with 20k post-training steps in simulation and 40k in real-world experiments.Training uses AdamW, batch size 64, and learning rate 2.5×10−5 on two NVIDIA H100 GPUs.Source: 3) VLA Post-Training · p. 5

IV. EXPERIMENTS AND RESULTS

  • The experiments assess post-trained π0.5 on eight RoboTwin 2.0 simulation tasks and four real-world DoBot tasks under clean and randomized settings.Simulation success rates are computed over 100 trials per task; the simulation setup groups tasks into stacking and other manipulation tasks.Source: §IV · p. 5, A. Simulation Settings · pp. 5–6, Table I · p. 6

1) Comparison Study:

  • All1.0 and Random1.0 raise clean-setting average success rates over Robot-Only by 4% and 5%, but lower randomized-setting performance by 5% and 2% percentage points.The authors suggest this contrast may reflect irrelevant or mismatched samples and overfitting to the clean training distribution.Source: 1) Comparison Study · p. 6
  • ReWeight gains 5% and 2% over Retrieval1.0 in clean and randomized simulation, respectively; Retrieval0.4 beats Random1.0 by 6% in randomized settings.Retrieved-data variants also consistently outperform All1.0 and Random1.0 across both settings.Source: 1) Comparison Study · p. 6

2) Ablation Study:

Ablations show that action information and a temporal penalty improve cross-embodiment matching, while α=0.5 performs best among tested sample-weight calibration settings. The evaluation also describes alignment metrics and real-world perturbation scope.

2) Ablation Study:

  • The encoder ablation evaluates action input and temporal mismatch penalties using Stage Accuracy and Keyframe Distance across four stacking tasks.For each human demonstration, its smallest Wasserstein-distance robot match is selected; the metrics are averaged over human demonstrations across the tasks.Source: 2) Ablation Study · pp. 6–7, Table II · p. 7
  • Removing action input lowers average Stage Accuracy from 0.93 to 0.83 and raises average Keyframe Distance from 4.1 to 10.0 frames; removing the temporal penalty drops Stage Accuracy by 7% and adds 3.9 frames.The authors conclude that action information captures behaviorally meaningful discrepancies and the temporal prior promotes temporally consistent correspondences.Source: 2) Ablation Study · p. 7
  • Among tested calibration settings, α=0.5 achieves the best downstream performance while retaining a minimum contribution from each retrieved human sample.A smaller α more strongly suppresses samples with large cross-embodiment discrepancies.Source: 2) Ablation Study · p. 7, Table III · p. 7
  • The real-world perturbation comparison covers lighting changes and visual distractors.The supplied table caption gives the evaluation scope but not its outcomes.Source: Table IV · p. 8

C. Realworld Evaluation

ReWeight is evaluated on four real-world manipulation tasks under clean and randomized conditions. It outperforms robot-only training and naive human-data mixing, with gains in both settings.

C. Realworld Evaluation

  • The evaluation uses a dual-arm DoBot platform and four tasks, comparing ReWeight with robot-only training and randomly sampled human-data mixing.Clean trials varied initial object positions; randomized trials tested lighting changes and unseen visual distractors.Source: C. Realworld Evaluation · p. 8

D. Real-world Results

  • 28.8 and 13.8 percentage points: ReWeight improves average success over Robot Only and Naive Mixing, respectively, across four clean-setting real-world tasks.Each task was evaluated over 20 trials.Source: Figure 7 · p. 8, D. Real-world Results · p. 8
  • 60.0% overall success rate (48/80) under randomized conditions, versus 41.3% (33/80) for Naive Mixing and 28.8% (23/80) for Robot Only.The randomized tests used lighting variations and unseen visual distractors.Source: D. Real-world Results · p. 8, C. Realworld Evaluation · p. 8

V. CONCLUSION

  • ReWeight learns a cross-embodiment visuomotor representation to retrieve relevant demonstrations and assign discrepancy-aware sample weights.The conclusion frames the approach as choosing which human demonstrations to transfer and how strongly each sample contributes to optimization.Source: §V · pp. 8–9
  • In simulation, ReWeight improves the average success rate of post-trained π0.5 to 57%, from 39% with robot-only data and 44% with randomly mixed human–robot data.Source: §V · p. 9

Related papers

Loading 2609.13851v1…