PaperlayerView exact revision on arXiv

Source-linked AI summary

RoboEvolve: Co-Evolving Planner-Simulator for Robotic Manipulation with Limited Data

Harold Haodong Chen, Sirui Chen, Yingjie Xu, Wenhang Ge, Ying-Cong Chen

arXiv:2605.13775v1cs.ROcs.CV

TL;DR

Robotic manipulation lacks scalable, task-aligned interactive data, while VLM planners and VGM simulators face semantic-spatial misalignment and physical hallucination. RoboEvolve couples them in a dual-phase co-evolutionary loop and outperforms static paradigms, elevating base planners by 30 absolute points.

  • Problem

    Robotic manipulation remains constrained by scarce scalable, task-aligned interactive data, while VGM-generated trajectories can be physically infeasible.

  • Method

    RoboEvolve couples a VLM planner and VGM simulator in a dual-phase co-evolutionary loop using unlabeled seed images.

  • Results

    RoboEvolve outperforms static paradigms across metrics, elevating base planners by 30 absolute points on EB-ALFRED and EB-Habitat.

  • Takeaways & Limitations

    The framework supports robust, open-ended skill acquisition with 500 unlabeled seeds and continual improvement without catastrophic forgetting.

  • Takeaways & Limitations

    The framework operates only in the generative visual domain and has not been deployed on physical robotic hardware.

Abstract

from arXiv · show

The scalability of robotic manipulation is fundamentally bottlenecked by the scarcity of task-aligned physical interaction data. While vision-language models (VLMs) and video generation models (VGMs) hold promise for autonomous data synthesis, they suffer from semantic-spatial misalignment and physical hallucinations, respectively. To bridge this gap, we introduce RoboEvolve, a novel framework that couples a VLM planner and a VGM simulator into a mutually reinforcing co-evolutionary loop. Operating purely on unlabeled seed images, RoboEvolve leverages a cognitive-inspired dual-phase mechanism: (i) daytime exploration fosters physically grounded behavioral discovery through a semantic-controlled multi-granular reward, and (ii) nighttime consolidation mines "near-miss" failures to stabilize policy optimization. Guided by an autonomous progressive curriculum, the system naturally scales from simple atomic actions to complex tasks. Extensive experiments demonstrate that RoboEvolve (I) achieves superior effectiveness, elevating base planners by 30 absolute points and amplifying simulator success by 48% on average; (II) exhibits extreme data efficiency, surpassing fully supervised baselines with merely 500 unlabeled seeds--a 50x reduction; and (III) demonstrates robust continual learning without catastrophic forgetting.

1. Introduction

RoboEvolve addresses the scarcity of scalable, task-aligned robotic interaction data by coupling a VLM planner with a VGM simulator in a self-evolving, dual-phase loop. Using unlabeled images, it combines physically grounded exploration with failure-driven consolidation to scale manipulation from simple to complex tasks.

  • Motivation: Scalable, task-aligned interactive data remains a fundamental bottleneck for transferring foundation-model capabilities to robotic manipulation.High-quality robot trajectories are expensive and time-consuming to collect, particularly when precise annotations or human demonstrations are required.
  • Motivation: VLMs and VGMs offer complementary capabilities but remain limited by semantic-spatial misalignment and physically hallucinated, infeasible trajectories.RoboEvolve is motivated by using VLMs for task prompts and judgments and VGMs for physical-feasibility simulation and feedback.
  • RoboEvolve Framework: RoboEvolve couples a VLM planner and VGM simulator into a self-evolving framework that learns continually from simple to complex manipulation using only unlabeled images.Scene-grounded atomic-action difficulty modeling progressively increases task complexity while preserving executability, without external annotations or rewards.
  • Dual-Phase Evolution Loop: RoboEvolve alternates daytime exploration, guided by semantic-controlled multi-granular rewards, with nighttime consolidation that mines failures through hierarchical preference optimization.Daytime expands behavioral coverage and physical grounding, while nighttime uses unsuccessful attempts to refine and stabilize both planner and simulator policies.
  • Empirical Evaluation: 48% relative simulator success gains on BridgeData V2 and 30 absolute-point improvements for base planners on EB-ALFRED and EB-Habitat demonstrate superior effectiveness.RoboEvolve also surpasses fully supervised baselines with merely 500 unlabeled seeds, a 50× reduction in annotations, while maintaining monotonic improvements without catastrophic forgetting.

2. Related Work

Related work positions VLMs as planners and VGMs as scalable neural simulators for embodied AI, while highlighting limitations from spatial reasoning, scarce expert labels, and task-specific data. RoboEvolve addresses these constraints through a co-evolving VLM–VGM loop that aligns simulation with tasks using sparse, unlabeled images.

  • Vision-Language Models as Planners: VLMs have emerged as the “brain” for embodied agents, with conventional methods fine-tuning them to map observations into textual instructions.These approaches rely on VLMs to internalize complex spatial and physical reasoning.
  • Video Generation Models as Simulators: VGMs increasingly serve as scalable neural-world-model simulators that capture physical plausibility and reduce the cost of manual data collection.Their embodied-AI use is motivated by the expense of collecting physical interaction data.
  • Video Generation Models as Simulators: VGM methods primarily use SFT trajectory fitting or RL exploration, but remain constrained by scarce high-quality labels or pre-annotated task-specific datasets.SFT trains on expert demonstrations, whereas RL can uncover deeper physical insights but still depends heavily on task-specific data.
  • Video Generation Models as Simulators: RoboEvolve instead uses a co-evolving loop in which a VLM planner semantically anchors a VGM that evolves into a task-aligned simulator from sparse, unlabeled images.This differs from static or data-hungry paradigms that treat the VGM as a fixed oracle.
  • Self-Evolving System: Self-evolving systems have emerged as mechanisms for lifelong learning, with existing language-model approaches commonly accumulating reasoning trajectories to improve future problem solving.The passage identifies experience accumulation as one of two prevailing paradigms.

3. Preliminary

RoboEvolve learns complex robotic manipulation from unlabeled seed images by co-evolving a planner and simulator in a closed loop. It structures tasks as atomic-action trajectories and uses difficulty-driven curriculum evolution with alternating exploration and consolidation.

  • Problem Formulation: RoboEvolve represents each manipulation task as a trajectory video transforming an initial state into a goal state, synthesized from a limited set of unlabeled seed images.The seed set is denoted D = {𝐼1, 𝐼2, . . . , 𝐼𝑁}, and each video frame corresponds to an intermediate state.
  • Problem Formulation: The framework co-evolves planner P and simulator S so plans become physically feasible and simulations become high-fidelity and physically consistent without expert demonstrations or ground-truth reward functions.This self-evolving environment replaces predefined simulators and extensive manual annotations.
  • Atomic Action and Difficulty Space: Plans decompose into visually identifiable atomic actions such as pick(X) and place(X, target), providing building blocks for complex manipulation tasks.Each plan is represented as 𝜋=⟨𝑎1, 𝑎2, . . . , 𝑎𝑛⟩ with 𝑎𝑖∈A, and each action corresponds to a motion segment in the generated video.
  • Atomic Action and Difficulty Space: RoboEvolve uses execution-cost difficulty as the curriculum state, guiding learning from simple single-stage manipulations toward complex multi-stage tasks.The difficulty function D(𝜏|𝐼) incorporates the unit cost c(𝑎𝑖) of each atomic action.
  • Complementary Learning System: Its complementary learning system alternates daytime GRPO exploration for relative-advantage discovery with nighttime DPO consolidation from success-failure preference pairs.This division fosters exploration breadth while mitigating physical hallucinations in S and logical fallacies in P.

4. Methodology

RoboEvolve builds a self-evolving planner–simulator loop from unlabeled seed images by grounding tasks in structured scene affordances, jointly refining physical execution and abstract planning, and consolidating near-miss failures. Its progressive curriculum scales capability from atomic actions toward increasingly complex tasks without manual intervention.

  • Scene-Grounded Task Initialization: RoboEvolve converts unlabeled seed images into actionable task repositories grounded in observed physical affordances.The planner extracts objects, spatial relations, and affordance priors; eight independent parsing samples retain only majority-consistent entities and relations.
  • Scene-Grounded Task Initialization: The planner maps scenes into 13 fundamental task templates and composes atomic actions into feasible, potentially high-difficulty hierarchical plans.Plans are decomposed into atomic actions whose cumulative costs define task difficulty for curriculum binning.
  • Daytime Exploration: RoboEvolve alternates simulator and planner training, using GRPO and selective segment-wise simulation to align abstract reasoning with physical execution limits.The simulator first improves trajectories at base difficulty, after which the planner tackles tasks with complexity (D, 2D] while constraining simulated segments to difficulty ≤D.
  • Daytime Exploration: The semantic-controlled multi-granular reward uses planner-generated prompt revisions to suppress physical scores when trajectories deviate from task semantics.Binary frame-, segment-, and episode-level signals provide stable supervision and explicit failure criteria.
  • Nighttime Consolidation: Nighttime learning converts daytime experiences into preference pairs, using near-miss failures to suppress simulator hallucinations and hierarchical preferences to refine the planner.Positive trajectories are physically consistent and task-aligned, while hard negatives expose critical failure modes; planner optimization extracts signals across three cognitive dimensions.
  • Progressive Curriculum: As simpler-task success saturates, progressive sampling shifts budget toward higher-complexity frontiers, enabling continuous manual-free capability scaling.The mechanism uses sampling counts and an exploration weight to redirect effort as P_k(b) →0 on simpler tasks.

5. Experiments

Experiments evaluate RoboEvolve’s co-evolutionary framework on simulator and planner benchmarks against static and decoupled variants. Results show stronger complex-task performance, stabilized optimization through interleaved dual phases, continual learning across difficulty stages, and effective scaling from unlabeled seeds.

  • Experimental setup: RoboEvolve is evaluated with Wan2.2-TI2V-5B as simulator S and Qwen3-VL-4B as planner P against vanilla, daytime-only, nighttime-only, and decoupled variants.Simulator evaluation uses BridgeData V2 across atomic and compositional manipulation tasks, while planner evaluation uses EB-ALFRED and EB-Habitat.
  • RQ1: Static-paradigm comparison: 40.0% to 49.6% relative Task Success gains over the static base model amplify from Level-1 to harder tasks, where static and decoupled variants struggle with physical grounding.The mutually reinforcing loop yields substantial improvements across all reported metrics as task complexity increases.
  • RQ2: Dual-phase evolution: Nighttime consolidation stabilizes policy optimization, while isolated phases saturate and sequential “D+N” scheduling underperforms interleaved evolution.Daytime-only exploration accumulates uncorrected physical hallucinations, motivating nighttime correction during the evolution loop.
  • RQ2: Dual-phase evolution: RoboEvolve sustains continual learning from difficulty D=1 through D=4 by autonomously advancing its progressive curriculum after foundational skills are acquired.The curriculum avoids collapse under the reward sparsity encountered when directly tackling complex tasks.
  • RQ3: Unlabeled-data scaling: 300 seeds produce ∼7.6K trajectories—less than a third of raw BridgeData’s ∼25K—while surpassing its Level-3 simulator Task Success and planner EB-ALFRED performance.Scaling unlabeled seeds from 300 to 1000 consistently drives task instantiation from D=1 to D=3 and monotonic gains for both components.

6. Conclusion

RoboEvolve addresses robotic manipulation’s data-scarcity bottleneck by mutually refining a VLM planner and VGM simulator using only unlabeled seed images. Its dual-phase loop supports robust, open-ended skill acquisition and extreme data efficiency, with 300 seeds outperforming massive manually annotated datasets.

  • 6. Conclusion: RoboEvolve mutually refines a VLM planner and VGM simulator to address the critical data scarcity bottleneck in robotic manipulation.The framework is described as co-evolutionary.
  • 6. Conclusion: Operating entirely on unlabeled seed images, RoboEvolve interleaves daytime physical exploration with nighttime failure consolidation for robust, open-ended skill acquisition.This cognitive-inspired dual-phase loop combines exploration and failure consolidation.
  • 6. Conclusion: 300 unlabeled seeds enable autonomously synthesized, high-density supervision that completely outperforms massive, manually annotated datasets.The conclusion highlights this as evidence of extreme data efficiency.

A. More Details of Scene-Grounded Task Initialization … B.2. More Details of Hierarchical Preference Optimization

RoboEvolve initializes manipulation tasks from scene-grounded symbolic representations and scales them through compositional atomic actions. Its optimization details combine multi-granular trajectory evaluation with hierarchical preference pairs spanning planning, understanding, and transitions.

  • A.1. Taxonomy of Task Templates and Atomic Actions: 13 fundamental task templates decompose diverse manipulation skills into a comprehensive atomic-action space grounded in physical execution.The taxonomy is inspired by BridgeData V2 and bridges high-level reasoning with low-level execution.
  • A.1. Taxonomy of Task Templates and Atomic Actions: Task difficulty equals compositional horizon length because each atomic action has base execution cost c(a_i) = 1.Chaining templates yields D=1 for opening a drawer and D=2 for opening it and picking up an apple.
  • A.2. More Details of Self-Consistency Voting: 54.00% human agreement was achieved by a single direct VLM pass, motivating self-consistency voting to reduce hallucinated or missed scene constraints.The study evaluated 50 randomly sampled seed images for object, affordance, and spatial-relation correctness.
  • A.2. More Details of Self-Consistency Voting: Self-consistency voting decomposes images into object attributes, pair-wise relations, and actionable freespace regions for physically viable trajectory synthesis.Examples include graspable and openable attributes, with the resulting structured representation supporting subsequent curriculum generation.
  • B.1. More Details of Semantic-Controlled Multi-Granular Reward: The semantic-controlled multi-granular reward separates whole-video fit, frame continuity, action-step completion, and episode-level physics and integrity checks.This decomposition supplies dense feedback while targeting semantic deviations, discontinuities, missing actions, and severe physical violations.
  • B.2. More Details of Hierarchical Preference Optimization: Planning-level preferences pair each seed image with a consensus natural-language plan and hard negatives from runner-up or randomly clipped plans.These negatives represent plausible alternatives or incomplete logical steps.
  • B.2. More Details of Hierarchical Preference Optimization: Understanding-level preferences use validated, highly rewarded videos to contrast the successful task prompt with a visually similar but factually incorrect runner-up.The contrast is intended to mitigate perceptual hallucinations in dynamic visual grounding.
  • B.2. More Details of Hierarchical Preference Optimization: Transition-level preferences condition on initial and final frames, contrasting the true executed prompt with a runner-up to teach causal state-transition reasoning.The state-pair context requires distinguishing similar intents from before-and-after physical states without continuous temporal cues.

B.3. More Details of Evaluation Benchmarks · C. More Results & Sensitivity Analysis · C.1. RL Training Curve

RoboEvolve’s evaluation combines multi-level simulator benchmarks, granular task scoring, user preference studies, and planner tests across household reasoning capabilities. Its daytime reinforcement-learning curves show steady reward convergence for both the simulator and planner across progressively harder tasks.

  • B.3. More Details of Evaluation Benchmarks: 214 prompts per level define a balanced simulator benchmark spanning atomic, two-stage, and three-stage compositional tasks.Level-2 and Level-3 instructions are logically composed from basic atomic skills using Gemini-2.5-Pro.
  • B.3. More Details of Evaluation Benchmarks: VBench evaluates six video-generation dimensions, while Gemini-2.5-Pro assigns step-wise task-success scores with partial credit for completed subtasks.For example, completing one of two subtasks yields a 50% success rate.
  • B.3. More Details of Evaluation Benchmarks: The evaluation protocol verifies physical completion sequentially and propagates failure to dependent subtasks when assigning granular success scores.Scoring checks object interactions, spatial relations, and physical state changes.
  • B.3. More Details of Evaluation Benchmarks: 50 videos per difficulty level and 10 expert evaluators support a blind user-preference study comparing all eight evaluated methods.Evaluators independently select the single best execution for each task group.
  • B.3. More Details of Evaluation Benchmarks: EB-ALFRED evaluates semantic macro-action planning across 300 episodes covering six capability dimensions, while EB-Habitat extends planner evaluation through EmbodiedBench.The six EB-ALFRED dimensions are Base, Common Sense, Complex Instruction, Spatial Awareness, Visual Appearance, and Long Horizon.
  • C.1. RL Training Curve: 300 training iterations show steady, monotonic daytime reward convergence for both the simulator and planner across difficulty levels D=1, 2, and 3.The curves remain stable despite escalating task complexity and the combinatorial nature of complex tasks.

C.2. Analysis of Semantic-Controlled Multi-Granular Reward · C.3. Analysis of Selective Simulation Strategy

Ablations show that semantic alignment is essential for preserving task fidelity, while selective simulation protects planner optimization from simulator hallucinations. Removing either component causes substantial performance degradation, especially on challenging benchmarks and capability dimensions.

  • C.2. Analysis of Semantic-Controlled Multi-Granular Reward: The reward ablation isolates each daytime component while still computing omitted components to form nighttime near-miss preference pairs.This design separates daytime reward shaping from nighttime consolidation data construction.
  • C.2. Analysis of Semantic-Controlled Multi-Granular Reward: Removing the semantic-alignment indicator 𝕀𝑠𝑒𝑚 causes the most severe degradation across all reported metrics.The indicator is identified as the most critical factor in the semantic-controlled multi-granular reward.
  • C.2. Analysis of Semantic-Controlled Multi-Granular Reward: 11.0 and 11.6 absolute points are lost on EB-ALFRED and EB-Habitat, respectively, when 𝕀𝑠𝑒𝑚 is removed.These are reasoning-heavy benchmarks in which semantic alignment is particularly important.
  • C.2. Analysis of Semantic-Controlled Multi-Granular Reward: The semantic-alignment indicator gates optimization so physical realism does not displace semantic task fidelity.Its role is to prevent the system from blindly optimizing physical realism at the expense of the task’s semantics.
  • C.3. Analysis of Selective Simulation Strategy: The selective simulation ablation evaluates daytime exploration of Planner P on the EB-ALFRED benchmark.The strategy is assessed specifically during exploration rather than as a general evaluation of the full framework.
  • C.3. Analysis of Selective Simulation Strategy: 6.0 absolute points are lost in average task success rate when selective simulation is removed.The performance decline is most pronounced in the Visual and Long capability dimensions.
  • C.3. Analysis of Selective Simulation Strategy: Without self-consistency filtering, Planner P risks over-fitting to or internalizing Simulator S’s occasional physical hallucinations during exhaustive rollouts.The degradation is strongest in highly demanding scenarios, including the Visual and Long capability dimensions.

C.4. Analysis of Hierarchical Preference Optimization

The nighttime hierarchical preference optimization strategy benefits from all three cognitive dimensions, whose signals provide orthogonal and synergistic gains. Planning- and understanding-level optimization are foundational, while transition-level optimization supports complex and visual reasoning by promoting state-transition causality.

  • Ablation findings: Ablating any preference level strictly degrades P’s performance on EB-ALFRED, showing that the hierarchical signals provide orthogonal and synergistic benefits.The analysis examines the three cognitive dimensions during nighttime consolidation.
  • Ablation findings: 7.0 points: Omitting either planning-level (D𝑃) or understanding-level (D𝑈) optimization causes a severe drop in average s….Both dimensions are described as particularly foundational to performance.
  • Ablation findings: Transition-level (D𝑇) ablation causes milder overall degradation but still impairs Complex and Visual reasoning.This optimization helps P internalize state-transition causality.

C.5. Analysis of Curriculum Hyperparameter · D. Exhibition Board

RoboEvolve’s curriculum is highly sensitive to λ: conservative pacing traps training at simple tasks, whereas λ = 0.10 aligns each day-night cycle with a difficulty increase. Qualitative comparisons show that removing nighttime consolidation causes temporal and physical hallucinations, including disappearing manipulated objects or tools during sub-task transitions.

  • C.5. Analysis of Curriculum Hyperparameter: λ governs the pacing of curriculum difficulty progression in Equation (7).The analysis evaluates sensitivity to this exploration hyperparameter.
  • C.5. Analysis of Curriculum Hyperparameter: λ = 0.01 traps all three dual-phase iterations at D = 1 and yields 0.621 on Level-1 evaluation.The prolonged focus on atomic tasks is reported as inferior to the progressive setting and as trapping models in local optima.
  • C.5. Analysis of Curriculum Hyperparameter: λ = 0.05 slightly accelerates learning but delays transition to complex tasks.This moderate setting remains slower to advance the curriculum than the optimal setting.
  • C.5. Analysis of Curriculum Hyperparameter: λ = 0.10 maps one day-night cycle to one difficulty ascension, D = 1 → 2 → 3.The passage identifies this as the optimal setting and links its scaling to unlocking multi-stage capabilities and positive backward transfer.
  • D. Exhibition Board: The exhibition board compares RoboEvolve qualitatively with the Daytime-Only baseline.The extended comparisons are presented to illustrate differences between the two variants.
  • D. Exhibition Board: The Daytime-Only variant can initiate complex, multi-step instructions but suffers severe temporal and physical hallucinations.The comparisons identify hallucinations as a characteristic failure of the variant without nighttime consolidation.
  • D. Exhibition Board: A recurrent failure is the sudden disappearance of manipulated objects or tools during transitions between sub-tasks.The disappearing entities are indicated by pink bounding boxes in the qualitative comparisons.
  • D. Exhibition Board: The comparisons visually demonstrate the importance of nighttime consolidation for learning from “near-miss” negatives.Without this phase, the Daytime-Only variant exhibits the described transition failures.

E. Limitations and Future Work

RoboEvolve remains limited to generative visual-domain evaluation and has not yet been deployed on physical robotic hardware. Future work targets bridging this gap with world action models and improving robustness through trainable rewards and stronger scene-grounding initialization.

  • Limitations and Future Work: RoboEvolve has not yet deployed its evolved vision-language Planner on physical robotic hardware, remaining within the generative visual domain.The authors identify bridging the generative-to-real gap as a key future direction.
  • Limitations and Future Work: Integrating the co-evolutionary pipeline with world action models could translate synthesized high-level semantic trajectories into continuous actions.This proposed direction is intended to connect generative outputs with physical robotic execution.
  • Limitations and Future Work: RoboEvolve currently relies heavily on the base VLM’s zero-shot evaluation capabilities to assess multi-granular physics and semantics.The framework is described as a preliminary, purely self-contained co-evolutionary pipeline.
  • Limitations and Future Work: Future iterations could improve robustness with a dedicated trainable external reward model and stronger scene-grounding data initialization.Self-consistency voting already mitigates initial hallucinations, while the passage identifies further initialization improvements.
Loading 2605.13775v1…