Source-linked AI summary
RobotAPO: Adversarial Physics Preference Optimization for Robotic Manipulation Video Generation
Kerui Li, Zhe Jing, Chenyi Huang, Xiaofeng Wang, Zheng Zhu, Haoming Cui, Huaibo Huang
TL;DR
Generated manipulation videos can look plausible while violating contact and object-state physics, and SFT does not directly penalize these localized failures. The paper introduces condition-matched preferences and adversarial optimization in denoising space; in real-robot replay, it reports a 37.4% relative task-success improvement over the strongest controlled internal baseline.
Problem
SFT lacks direct training pressure to distinguish localized interaction-boundary physics violations from harmless visual variation, although such errors can invalidate execution timing and pose.
Source: §1 · p. 1Method
RobotAPO trains on condition-matched preference pairs that isolate physical failures and uses an adversarial counterfactual proposer to explore failure-biased directions in flow-matching denoising space.
Source: §1 · p. 2Results
37.4% relative improvement in task success over the strongest controlled internal baseline, with a 64.7% real-robot task success rate.
Source: §1 · p. 2Takeaways & Limitations
Real-robot evaluations show improved end-to-end video-to-action execution under a fixed action-extraction interface when localized physics violations are reduced.
Source: §5 · p. 11The study does not use explicit 3D geometry, force sensing, or rigid-body simulation priors, and does not establish zero-shot transfer across arbitrary embodiments or task families.
Source: §B.9 · pp. 18–19
Abstract
from arXiv · showhide
Robotic manipulation videos are increasingly used as visual plans for embodied agents, but optimizing purely for visual plausibility often fails to capture the fragile physical manifold of real-world interactions. Even minor physics-violating errors at the interaction boundary, such as interpenetration or premature object motion, can completely invalidate the inferred timing and pose needed for downstream execution. Because standard supervised fine-tuning lacks the direct pressure to penalize these localized failures, we introduce AgiBot-PhysPref. This rigorously curated 10,000-sample preference dataset isolates condition-matched physics violations, turning the generator's own failure distribution into a foundational signal for physical consistency. Building upon this, we propose RobotAPO, an adversarial physics preference optimization framework operating in the continuous flow-matching denoising space. To prevent the policy from merely memorizing static curated failures, RobotAPO employs a lightweight adversarial counterfactual proposer that learns a condition-dependent, physical-failure-biased direction in denoising space. This encourages the model to explore and better respect the physical interaction boundary, all while maintaining a pure prompt-and-reference inference interface without requiring external structural conditioning. Comprehensive evaluations demonstrate that explicitly correcting these localized physics violations improves downstream robot execution from generated videos. On held-out AgiBot conditions, RobotAPO outperforms the strongest controlled internal baseline in physical consistency by 6.8% hard score and 10.0% soft score. Crucially, in real-robot replay, it translates these physical-consistency gains into a 37.4% relative improvement in task success over the strongest controlled internal baseline.
1. Introduction
Visual plausibility alone does not ensure that generated manipulation videos respect the physical interaction boundaries needed for execution. RobotAPO combines condition-matched failure preferences with adversarial denoising-space optimization, and reports improved real-robot task success over a controlled internal baseline.
1. Introduction
- SFT captures broad visual patterns but lacks direct pressure to distinguish action-critical physics violations from harmless perceptual variation.Errors such as premature object motion can invalidate timing and pose needed for downstream execution.Source: §1 · pp. 1–2
- AgiBot-PhysPref provides 10,000 curated preference pairs that isolate local physics violations under matched visual and semantic conditions.The pairs are drawn from 40,000 SFT rollouts across 200 AgiBot task IDs and are screened with VLMs and double human annotation.Source: §1 · p. 2
- RobotAPO learns a condition-dependent, physical-failure-biased direction in flow-matching denoising space to explore hard negatives near interaction boundaries.It retains prompt-and-reference-image inference without external structural conditioning.Source: §1 · p. 2
- 64.7% real-robot task success, a 37.4% relative improvement over the strongest controlled internal baseline, is the reported downstream result.The introduction also reports a 31.4 absolute percentage point improvement over SFT in AnyPos-driven replay.Source: §1 · p. 2
2. Related Work
Prior work uses generated robot videos for plans, world models, and demonstration transfer, while common training objectives emphasize broad visual patterns. This work targets localized physics violations that matter for embodied execution.
2. Related Work
- Generated robot videos support visual plans, embodied world models, and demonstration transfer, building on open-domain generative models.The cited applications include robot learning and cross-domain experience adaptation.Source: §2 · p. 2
- Reconstruction- and prediction-based training capture broad scene appearance and coarse dynamics, but embodied execution also requires respecting precise contact timing and grasp maintenance.VLM evaluators diagnose general visual and commonsense consistency; real-world execution tests localized physical requirements.Source: §2 · p. 3
- Existing preference and video-generation objectives emphasize visual quality, semantic alignment, and global plausibility, while rarely targeting action-critical localized physics violations.RobotAPO aligns generation with physical causality at the interaction boundary rather than generic visual preferences.Source: §2 · p. 3
3. Method
RobotAPO post-trains an SFT video generator with condition-matched physical-failure preferences and adversarial counterfactuals in flow-matching denoising space. Its training design targets local interaction failures while retaining a prompt-and-reference inference interface.
3.1. Problem and Overview
- RobotAPO adds trainable policy and adversarial LoRA modules to a frozen SFT generator; only the policy generator is deployed with the standard prompt-and-reference interface.No external structural conditioning such as action, trajectory, masks, or depth is required at inference.Source: §3.1 · p. 3
3.2. AgiBot-PhysPref Construction
- Qwen3-VL screened 40,000 SFT rollouts; annotators reviewed 12,935 candidates, excluded 2,935, and accepted 10,000 human-verified failures paired with matching demonstrations.The real demonstration and generated failure share the same manipulation condition, focusing the preference on physical interaction validity.Source: §3.1 · p. 3, §3.2 · p. 4
3.3. Denoising-Space Physical-Failure Preference Learning
- Preference learning compares condition-matched successful and failed videos in flow-matching denoising space, using SFT-centered rewards, a successful-sample anchor, and a LoRA drift penalty.The pair uses the same prompt, first-frame reference, noise level, and target construction.Source: §3.3 · pp. 5–6
3.4. Adversarial Counterfactual Hard-Negative Generation
- Alternating adversary-policy updates search continuous denoising space for plausible, condition-preserving negatives near the interaction boundary, without external simulators.The proposer is a training-only generator-side module; the policy compares successful targets against both curated failures and adversarial counterfactuals.Source: §3.4 · pp. 6–7
4. Experiments
Experiments test whether optimizing for physical causality transfers across automatic scoring, real-robot execution, and failure-focused inspection of interaction boundaries.
4.1. Experimental Setup
- Internal methods share the Wan2.2-TI2V backbone, SFT initialization, preference pairs, and inference settings; automatic scoring uses 200 held-out conditions with five videos each.The controlled comparison evaluates 1,000 videos per method.Source: §4.1 · p. 7, §4.2 · p. 7
4.2. Main Quantitative Comparison
- RobotAPO sets the best mark on every hard and soft metric among internal Wan2.2-TI2V methods, while visual quality improves alongside physical consistency.All internal methods are controlled 5B-parameter models.Source: Table 2 · p. 8, §4.2 · p. 8
- The external 14B LVP baseline has the highest hard visual-quality score and ties RobotAPO on Phys-H; RobotAPO has the same observed Phys-H at roughly one-third the parameter scale.RobotAPO also has higher observed soft scores and Align-H in this evaluation; LVP is an unmatched external reference.Source: §4.2 · p. 8
4.3. Robot Execution from Generated Videos
- RobotAPO reaches 64.7% task success (33/51), improving 31.4 absolute percentage points over SFT under matched adaptation and AnyPos extraction.Against external LVP, the observed +2.0-point difference has a 95% CI of [−9.8, 13.7], so the results do not establish an advantage over LVP.Source: Table 3 · p. 9, §4.3 · p. 9
4.4. Ablation Study
- Removing the adversarial counterfactual proposer drops task success from 64.7% to 49.0%, while static failures alone still improve generic VideoScore2 metrics.Removing the anchor and regularization also collapses alignment and physical-consistency scores.Source: Table 4 · p. 9, §4.4 · p. 9
4.5. Failure-Focused Qualitative Comparison
- At action-critical phases, RobotAPO reduces local interaction artifacts and better preserves gripper–object geometry and contact causality than baselines.The comparisons align frames to the same action-critical phase and use identical frame lengths and inference settings.Source: Figure 2 · p. 10, §4.5 · p. 10
4.4. Ablation Study
- The default λadv = 0.05 balances boundary exploration with visual-manifold stability; 0.02 is too weak, while 0.10 harms alignment and physical scores.The coefficient controls the intensity of the counterfactual physical-boundary perturbation.Source: §4.4 · pp. 9–10
5. Conclusion
RobotAPO uses condition-matched preferences and an adversarial proposer to improve physical consistency in robotic video generation. Real-robot evaluations show improved video-to-action execution under a fixed action-extraction interface.
5. Conclusion
- RobotAPO optimizes physical preferences in continuous flow-matching denoising space using condition-matched AgiBot-PhysPref.Its adversarial proposer learns a condition-dependent, physical-failure-biased denoising direction to avoid memorizing static artifacts.Source: §5 · p. 11
- The adversarial proposer explores interaction failures without relying on external structural constraints.At inference, the approach retains a prompt-and-reference interface.Source: §5 · p. 11
- Real-robot evaluations show that reducing localized physics violations improves end-to-end video-to-action execution under a fixed action-extraction interface.Source: §5 · p. 11
A.1. Source-Matched Generated-vs.-Generated Preferences
This section tests whether RobotAPO's advantage persists when preferences compare generated rollouts from the same source, rather than real demonstrations against generated failures. Under this source-matched supervision, RobotAPO retains gains over generic preference baselines.
A.1. Source-Matched Generated-vs.-Generated Preferences
- The source-matched control pairs generated rollouts from the same Wan2.2-TI2V SFT generator under the same AgiBot condition.The original dataset pairs real successful demonstrations with generated failures, leaving a real-versus-generated source difference.Source: §A.1 · p. 14
- For each condition, the least-flawed and most-flawed of ten rollouts form preferences, yielding 10,000 retained pairs after filtering 12,935 candidates.Independent human checking on 100 sampled pairs showed 84.5% aggregate agreement with classifier ordering.Source: §A.1 · p. 14
- The source-matched comparison retrains Diffusion-DPO, RealDPO, and RobotAPO from the same SFT initialization with identical data and training budgets.Evaluation uses 200 held-out conditions and five generations per condition.Source: §A.1 · p. 14
- RobotAPO improves Phys-S over Diffusion-DPO by +0.232 and over RealDPO by +0.201, with corresponding Phys-H gains of +0.125 and +0.130.All four gains are reported with condition-level paired bootstrap intervals excluding zero.Source: §A.1 · p. 14
A.2. Learned Direction vs. Matched Gaussian Perturbations
The appendix tests whether RobotAPO's learned denoising direction produces targeted physical failures beyond matched Gaussian perturbations and whether its gains extend to CogVideoX-5B. Results support physical-failure bias at the evaluated operating point and portability across the two evaluated backbones.
A.2. Learned Direction vs. Matched Gaussian Perturbations
- The perturbation test pairs learned-adversarial and equal-norm Gaussian-control videos across 64 held-out conditions while holding prompt, reference, noisy latent, and decoding settings fixed.All 128 videos were assessed by two blinded annotators followed by blinded adjudication.Source: §A.2 · p. 14
- The learned direction produces condition-preserving local failures 1.85 times as often as matched Gaussian control (95% CI [1.14, 3.38]; exact McNemar p=0.0266).Among adversarial outputs with any failure or unusability, 75.0% are targeted local physical failures; this does not imply explicit spatial localization.Source: §A.2 · p. 14, Table 7 · p. 15
A.3. Transfer to CogVideoX-5B
- Results on CogVideoX-5B, together with Wan2.2-TI2V results, support portability across the two evaluated video backbones.The authors explicitly keep the scope to these two evaluated backbones.Source: §A.3 · p. 15
A.4. Uncertainty and Execution-Relevant Metric Analysis
This section quantifies RobotAPO's Phys-S gains, relates physical-consistency metrics to real-robot task success, and clarifies that the metric complements rather than replaces physical evaluation.
A.4. Uncertainty and Execution-Relevant Metric Analysis
- Phys-S gains are +0.336 over SFT, +0.243 over Diffusion-DPO, and +0.370 over RealDPO in the Wan2.2-TI2V comparison.The cited condition-clustered paired bootstrap intervals exclude zero for all three comparisons.Source: §A.4 · p. 15
- Across 255 execution videos, Phys-S correlates with full-task success at 0.775 ([0.721, 0.822]), versus 0.467 for Visual-S and 0.452 for Align-S.Source: §A.4 · p. 15
- Under the reported conditions, RobotAPO retains adjusted Phys-S gains of +0.160 [0.099, 0.227], +0.101 [0.042, 0.161], and +0.223 [0.161, 0.291] over SFT, Diffusion-DPO, and RealDPO, respectively.Phys-S is a complementary indicator alongside complete-video inspection and robot execution, rather than a substitute for physical evaluation.Source: §A.4 · p. 15
- Task-success differences are +31.4 percentage points over SFT ([15.7, 49.0]), while intervals include zero for Diffusion-DPO (+19.6; [−2.0, 41.2]), RealDPO (+17.6; [−2.0, 37.3]), and LVP (+2.0; [−9.8, 13.7]).The authors state that the current trial budget supports the improvement over SFT; the other intervals include zero.Source: §A.4 · p. 15
B. Dataset, Implementation, and Reproducibility Details
AgiBot-PhysPref uses internal manifests to document dataset construction, preference pairs, and evaluation outputs for auditability and reproducibility.
B.1. Dataset Documentation
- The condition manifest records each video identifier, task instruction, visual-condition path, split, and prompt hash.The generated-candidate manifest tracks generator checkpoints, sampling settings, outputs, and conditions for the 40,000-video pool.Source: §B.1 · p. 16
- Screening and human-review manifests preserve model responses, parsed scores, filtering decisions, annotator judgments, and review status.The final preference-pair manifest records condition-matched tuples containing a condition, successful demonstration, failed rollout, and primary failure label.Source: §B.1 · p. 16
- The evaluation manifest records held-out prompts, outputs from compared methods, the selected LoRA scale, VideoScore2 results, and robot replay records when applicable.Source: §B.1 · p. 16
B.2. Dataset Summary and Annotation Flow
This appendix documents how AgiBot-PhysPref was screened, verified, and labeled, alongside implementation, compute, and evaluation protocols.
B.2. Dataset Summary and Annotation Flow
- A 40,000-video pool from 200 AgiBot task conditions was screened until 10,000 negatives were accepted, requiring review of 12,935 candidates.The dataset is a training preference set, not a standalone benchmark; evaluation uses disjoint held-out conditions paired with real first-frame references and task instructions.Source: §B.2 · p. 16
B.3. Qwen3-VL-30B-Instruct Screening Protocol
- Qwen3-VL-30B-Instruct ranks candidates using interaction quality, 3D consistency, and instruction-following scores from 16 sampled frames.Candidates are ranked by the mean of the three parsed scores and shortlisted for human annotation.Source: §B.3 · pp. 16–17
B.4. Human Verification Protocol
- Two annotators independently verify each candidate, with disagreements sent for adjudication before condition-matched pairs enter training.Only human-verified pairs are used as preference supervision; ambiguous, mismatched, or non-physical cases are rejected or marked uncertain.Source: §B.2 · p. 16, §B.4 · p. 17
B.5. Failure Type Taxonomy
- The unified taxonomy has five failure categories, including contact-causality failures such as premature object motion or state changes without effective contact.The taxonomy is used for curation, training metadata, and failure-focused evaluation.Source: Table 9 · p. 17, §B.5 · p. 17
- The 10,000 primary labels comprise 2,936 contact-causality samples (29.36%), the largest category, and four other mutually exclusive categories.Those other counts are 2,072 interpenetration (20.72%), 1,632 entity-consistency (16.32%), 1,286 unsupported-floating (12.86%), and 2,074 invalid-interaction/other (20.74%); secondary tags are retained for auditing, while training uses binary preference.Source: §B.5 · pp. 17–18
B.6. Implementation Details
- RobotAPO post-training freezes the SFT Wan2.2-TI2V generator, uses policy and adversary LoRA adapters, and deploys only the policy adapter.Training alternates policy and adversary phases after a policy-only warm-up; preference pairs use synchronized frames and shared resize-crop transforms.Source: §B.6 · p. 18
B.7. Compute Resources
- The main policy and adversary runs used eight NVIDIA RTX PRO 6000 GPUs for about 200 wall-clock hours, or 1,600 GPU-hours.Source: §B.7 · p. 18
B.8. Official VideoScore2 Evaluation Protocol
- Official VideoScore2 evaluation reports visual quality, text-to-video alignment, and physical/common-sense consistency, each with hard and soft scores on a 1–5 scale.H is the parsed hard integer score; S is the official logit-based soft score.Source: §B.8 · p. 18
B.9. Limitations and Broader Impacts
The study’s approach centers on visible interaction failures under matched conditions, while its stated scope excludes several physical signals and complex settings. Its broader-impact discussion cautions that generated plans still require task-specific checks and oversight.
B.9. Limitations and Broader Impacts
- The method does not use explicit 3D geometry, force sensing, or rigid-body simulation priors, and complex multi-object, occluded, deformable, and closed-loop settings remain outside its focus.Source: §B.9 · p. 18
- The original preference pairs match conditions but compare a real successful demonstration with a generated failure; a source-matched G+G control reduces, but does not change, this dataset construction concern.Source: §B.9 · p. 18
- Demonstrated generalization is limited to two video backbones, held-out AgiBot conditions, and matched limited-data adaptation to AgileX, not zero-shot transfer across arbitrary embodiments or task families.Source: §B.9 · pp. 18–19
- Generated plans may reduce wasted trials and aid failure diagnosis, but should not be treated as verified plans without task-specific validation, safety checks, and human oversight.Source: §B.9 · p. 19
B.10. Reproducibility Checklist
The reproducibility checklist lays out a staged pipeline from candidate generation and curation through training and controlled evaluation.
B.10. Reproducibility Checklist
- Generate 40,000 SFT Wan2.2-TI2V videos across 200 AgiBot task-ID conditions, recording seeds and sampling parameters.Source: §B.10 · p. 19
- Screen candidates with Qwen3-VL-30B-Instruct, double-annotate shortlisted videos, review disagreements, and build 10,000 condition-matched samples without condition leakage.Source: §B.10 · p. 19
- Precompute prompt embeddings and train RobotAPO with policy warm-up followed by alternating policy and adversary phases.Source: §B.10 · p. 19
- Evaluate internal methods at a shared LoRA scale of 0.4 using VideoScore2, AnyPos-based robot replay, and failure-focused qualitative comparisons.LVP is evaluated only from its released checkpoint as an external embodied-video reference.Source: §B.10 · p. 19