PaperlayerView exact revision on arXiv

Source-linked AI summary

EvoVLA: Self-Evolving Vision-Language-Action Model

Zeting Liu, Zida Yang, Zeyu Zhang, Hao Tang

arXiv:2511.16166v1cs.CV

TL;DR

Long-horizon manipulation remains difficult for VLAs because policies can exploit coarse progress signals and memory can fail to retain useful context. EvoVLA combines stage-aligned rewards, pose-based exploration, and selective gated memory in a self-supervised framework. On Discoverse-L, it reports 69.2% average success, 10.2 points above the strongest baseline, while reducing hallucination to 14.8%.

  • Problem

    Long-horizon VLA policies can exploit superficial visual cues as progress, while sparse feedback, brittle pixel-based curiosity, and fragile memory complicate multi-stage learning and evaluation.

    Source: §1 · pp. 1–2

  • Method

    EvoVLA combines stage-aligned contrastive rewards, pose-grounded curiosity, and selective context memory with gated fusion for self-supervised VLA fine-tuning.

    Source: §1 · p. 2, §3.2 · pp. 3–4

  • Results

    69.2% average success on Discoverse-L, 10.2 points above the strongest baseline, with hallucination reduced to 14.8%.

    Source: §5.2 · p. 6, §6 · p. 9

  • Takeaways & Limitations

    Discoverse-L and EvoVLA together provide a method and measurement suite for studying memory-heavy VLA training and evaluation in long-horizon manipulation.

    Source: §1 · p. 2

    Systematic real-world evaluation remains future work, and the method depends on accurate 6D object poses from simulator ground truth or AprilTag markers.

    Source: §7 · p. 12

Abstract

from arXiv · show

Long-horizon robotic manipulation remains challenging for Vision-Language-Action (VLA) models despite recent progress in zero-shot generalization and simulation-to-real-world transfer. Current VLA models suffer from stage hallucination, where agents exploit coarse evaluation signals to shortcut multi-step tasks, reporting high progress without truly completing them. We present EvoVLA, a self-supervised VLA framework that addresses this issue through three complementary components: Stage-Aligned Reward (SAR), which uses triplet contrastive learning with Gemini-generated hard negatives to prevent visual shortcuts; Pose-Based Object Exploration (POE), which grounds curiosity in relative object-gripper pose instead of raw pixels; and Long-Horizon Memory, which uses selective context retention and gated fusion to stabilize intrinsic shaping during extended rollouts. Extensive evaluations on Discoverse-L, a long-horizon manipulation benchmark with three multi-stage tasks, show that EvoVLA improves average task success by 10.2 percentage points over the strongest baseline (OpenVLA-OFT), reaching 69.2 percent. EvoVLA also achieves one-and-a-half times better sample efficiency and reduces stage hallucination from 38.5 percent to 14.8 percent. Real-world deployment on physical robots reaches an average success rate of 54.6 percent across four manipulation tasks, outperforming OpenVLA-OFT by 11 points, demonstrating effective sim-to-real transfer and strong generalization. Code: https://github.com/AIGeeksGroup/EvoVLA. Website: https://aigeeksgroup.github.io/EvoVLA.

1. Introduction

EvoVLA targets stage hallucination and fragile memory in long-horizon manipulation with stage-aware rewards, pose-grounded exploration, selective memory, and the Discoverse-L benchmark. Its method combines these components in a self-supervised VLA training framework.

1. Introduction

  • Long-horizon VLA policies face stage hallucination from superficial cues, while fragile memory and the lack of suitable benchmarks hinder reliable evaluation of multi-stage tasks.The introduction motivates stage-aware feedback and a benchmark with ground-truth stage events as responses to these gaps.Source: §1 · pp. 1–2

2. Related Work

  • Related work positions EvoVLA across VLA fine-tuning, self-supervised exploration, counterfactual shaping, and memory, against persistent challenges in stage hallucination and brittle pixel curiosity.The discussion also notes sparse reward brittleness and limitations of short memory windows.Source: §2 · p. 2

3.1. Overview

  • EvoVLA combines Discoverse-L data and generated stage dictionaries with SAR, POE, and Long-Horizon Memory modules on the OpenVLA-OFT backbone.Training applies PPO with task-aligned normalization while the modules remain active.Source: §3.1 · p. 3

3.2. Self-Supervised Reinforcement Learning

  • SAR scores image-text triplets with hard negatives and temporal smoothing, while POE uses relative gripper-object pose and model-based rewards to target manipulation dynamics over visual novelty.SAR's near-miss negatives are designed to distinguish actual stage completion from shortcut cues; POE targets nuisance changes such as lighting and camera motion.Source: §3.2 · pp. 3–4

3.3. Long-Horizon Memory

  • Long-Horizon Memory selects timestamp-aware Top-K history tokens, gates their fusion with current latents, writes back selected items, and evicts low-utility entries.The text reports standalone memory gains of +2.4 success points and a 3.9-point hallucination reduction, with POE integration adding +3.1 success points.Source: §3.3 · p. 5

3.4. Training Objective

  • The training objective combines policy-gradient, value, and world-model losses, with PPO dual critics separating extrinsic and intrinsic returns.Separate optimizers and stop-gradient are used for world models and memory-conditioned training to prevent reward exploitation.Source: §3.4 · p. 5

4. Discoverse-L Benchmark

Discoverse-L uses the DISCOVERSE simulator with the AIRBOT-Play platform to evaluate three multi-stage manipulation tasks of varying difficulty. Its stage dictionaries are validated against simulator annotations, and evaluation uses randomized initial conditions.

4. Discoverse-L Benchmark

  • Discoverse-L covers Block Bridge, Stack, and Jujube-Cup, with 74, 18, and 19 stages, respectively.The tasks range from multiobject bridge construction to block stacking and placing a jujube in a cup before moving it onto a plate.Source: §4 · p. 5
  • Block Bridge is the most complex task, requiring precise multiobject coordination; Stack demands accurate alignment, while Jujube-Cup has fewer contact-critical stages.The benchmark runs on the DISCOVERSE simulator with the AIRBOT-Play platform.Source: §4 · p. 5
  • The task dictionaries are Gemini-prompted, and the simulator’s fine-grained stage annotations are used only to validate stage-discovery coverage.For Bridge, the 74-stage dictionary merges five simulator micro-adjustments into semantically equivalent steps, with 93.7% recall; 50 simulator rollouts per task use randomized initial conditions.Source: §4 · p. 5

5. Experiments

EvoVLA is evaluated on Discoverse-L and physical AIRBOT-Play deployments, with comparisons covering benchmark performance, sim-to-real transfer, and component ablations. The experiments report gains over OpenVLA-OFT in success, sample efficiency, and hallucination rate.

5.1. Experimental Design and Evaluation

  • Evaluation uses task-aligned normalization, three simulation seeds, and a 400-step episode cap that accommodates Bridge’s 74 stages.Policies are trained with PPO for 2M timesteps per seed across eight parallel environments; evaluation uses 50 episodes per task.Source: §5.1 · p. 6

5.2. Simulation Results

  • On the three-task Discoverse-L benchmark (mean of 3 seeds, 50 evaluation episodes), EvoVLA achieved 69.2% average success, a +10.2 absolute-point improvement over the strongest baseline, OpenVLA-OFT (59.0%).Source: §5.2 · p. 6, Table 1 · p. 6
  • EvoVLA reaches 50% average success in approximately 6 × 10^5 environment steps, versus approximately 9 × 10^5 for OpenVLA-OFT, and lowers hallucination rate from 38.5% to 14.8%.The hallucination-rate reduction is 23.7 points; the reported sample-efficiency improvement is 1.5×.Source: §5.2 · p. 7

5.3. Real-World Evaluation

  • 54.6% average success across three Sim2Real tasks and an on-robot insertion task, surpassing OpenVLA-OFT by +11.0 points overall.The three simulation-trained policies transfer without real-world retraining; the unseen insertion policy is warm-started with demonstrations and fine-tuned on-robot.Source: §5.3 · p. 7, Figure 6 · p. 8
  • In the Stack comparison, OpenVLA-OFT opens before contact and drops the block, while EvoVLA delays opening until contact and leaves a stable stack.EvoVLA aligns within a few corrections across the two sub-tasks.Source: Figure 5 · p. 8, §5.3 · p. 8

5.4. Ablation Studies

  • In a progressive ablation averaged across 3 tasks and 3 seeds from OpenVLA-OFT, hard negatives, temporal smoothing, Long-Horizon Memory, and POE add +2.8, +1.9, +2.4, and +3.1 SR points, respectively.Their corresponding hallucination-rate changes are -7.3, -7.8, -3.9, and -4.7 points.Source: §5.4 · p. 8, Table 2 · p. 8
  • The CLIP threshold remains effective from 0.65 to 0.75, while intrinsic weight ρ yields stable performance within ±2.4 points from 0.3 to 0.9.The selected settings are θ = 0.7 and ρ = 0.6.Source: §5.4 · p. 8

6. Conclusion

EvoVLA combines stage-aligned rewards, pose-based exploration, and long-horizon memory to address stage hallucination and forgetting in long-horizon manipulation. Across Discoverse-L tasks, it reports higher success, better sample efficiency, and a lower hallucination rate than the strongest baseline.

6. Conclusion

  • EvoVLA combines stage-aligned rewards with triplet contrastive learning and temporal smoothing, pose-based exploration, and long-horizon memory to address stage hallucination and forgetting.The framework grounds curiosity in relative object-gripper poses and gates intrinsic shaping using retained context.Source: §6 · p. 8
  • The conclusion reports that each component contributes significantly to performance and that the framework substantially reduces spurious exploration.The reported evaluation scope is the three Discoverse-L tasks.Source: §6 · pp. 8–9
  • Across the three Discoverse-L tasks, the method achieved 69.2% average success, 10.2 points above the strongest baseline, and 1.5× improved sample efficiency, reaching 50% success with fewer environment steps.The hallucination rate was reduced to 14.8%.Source: §6 · p. 9

7. Limitations and Discussion

The discussion identifies limits in real-world evaluation, pose tracking, stage discovery, and compute, alongside threats from changing datasets, simulators, implementations, or distributions. It also describes the real-robot rollout scope and formalizes the long-horizon manipulation setting.

7. Limitations and Discussion

  • Systematic real-world evaluation remains future work; the approach requires accurate 6D object tracking, and stage discovery depends on demonstration diversity and prompt quality.Rare corner cases may require additional data curation; training uses 4×H20 GPUs for 24 hours per seed.Source: §7 · p. 12
  • Results may vary with dataset or simulator changes, baseline implementation choices, and unseen distribution shifts.The authors mitigate these threats with matched 2M-step budgets, identical backbones and input resolutions, and three random seeds.Source: §7 · p. 12

8. Reproducibility Statement

  • The real-world evaluation covers four tasks with at least 20 valid physical trials per model–task pair; insertion is trained on-robot, unlike the three simulation-trained tasks.Experiments used deterministically seeded runs on four NVIDIA H20 GPUs, and training and evaluation code, stage dictionaries, and trajectories are slated for release.Source: Table 3 · p. 12, §8 · p. 12

10. Additional Rollout Visualizations

  • Full Discoverse-L and AIRBOT-Play rollout sequences show stable gripper-object contact across place, bridge, stack, and insert executions in synchronized third-person and wrist views.The rollout descriptions characterize these traces as consistent Sim2Real behavior.Source: §10 · p. 12, Figure 7 · p. 13, Figure 8 · p. 13

11. Problem Formulation

  • Manipulation is modeled as a POMDP with latent object and robot states, continuous 7-D actions, multi-view RGB and proprioception observations, and sparse task-success reward.The policy is initialized from OpenVLA-OFT and fine-tuned with PPO for 2M environment steps across eight simulators.Source: §11 · pp. 12–13
  • Tasks decompose into sequential stages from video-driven demonstrations; a learned VLM stage tracker advances stages using recent image-text similarity scores, without simulator ground-truth stage labels.For the three real-world deployments, the simulation-derived stage dictionaries are not reestimated; an unseen on-robot task receives a dictionary from 50 teleoperated demonstrations.Source: §11 · pp. 12–13

12. Benchmark Technical Details

Discoverse-L provides interactive evaluation using held-out simulator seeds and reports success, hallucination, and sample efficiency. Sensitivity analyses support the selected CLIP threshold and intrinsic reward weight across tested ranges.

12. Benchmark Technical Details

  • Discoverse-L supports interactive evaluation with held-out seeds and reports Success Rate, Hallucination Rate, and Sample Efficiency.The suite includes demonstration rollouts and task-aligned normalization files.Source: §12 · p. 14

13.1. CLIP Threshold θ Sensitivity

  • 69.2% Success Rate occurs at θ = 0.7, remaining within ±0.5 points over [0.65, 0.75]; lower thresholds raise hallucination, while θ > 0.8 lowers success to 63–66%.The threshold balances false-positive and false-negative rates.Source: §13.1 · p. 14, Figure 9 · p. 14

13.2. Intrinsic Weight ρ Sensitivity

  • ρ = 0.6 reaches peak Success Rate (69.2%) and converges to 50% success in 6.0 × 10^5 steps, versus 7.2 × 10^5 at ρ = 0.3 and 7.5 × 10^5 at ρ = 0.9.Success Rate is 67.1% at ρ = 0.3 and 66.8% at ρ = 0.9; performance varies within ±2.4 points.Source: §13.2 · p. 14, Figure 10 · p. 14

14. Complete Hyperparameter Table

The section identifies a hyperparameter table for DISCOVERSE simulation experiments and lists several PPO defaults held constant across experiments.

14. Complete Hyperparameter Table

  • Table 4 covers key hyperparameters for DISCOVERSE simulation experiments.Its caption does not provide the table’s parameter values.Source: Table 4 · p. 14
  • PPO batch size is 256, clip ratio 0.2, entropy coefficient 0.01, and maximum gradient norm 0.5; these settings remain fixed across experiments.Source: §14 · p. 14

15. Video-driven Stage Discovery: Real Example

The example describes a prompted, video-driven workflow for discovering shared task stages and constructing geometric positive, hard-negative, and negative predicates. Validation checks the resulting dictionary, whose reported stage matches include 74 of 79 simulator ground-truth stages.

15. Video-driven Stage Discovery: Real Example

  • Gemini prompts align ten block-bridge demonstrations with varied end-effector poses, object placements, and camera viewpoints to discover stage semantics and supply SAR triplets.The workflow emphasizes cross-video alignment, minimal-sufficient stage segmentation, and unified stage IDs.Source: §15 · p. 15
  • Stage semantics use mutually exclusive geometric and contact predicates: positives specify success, hard negatives encode near-misses, and negatives describe clear failures.The prompt avoids color and texture cues and requires a self-check for conflicts, temporal gaps, and appearance-based descriptions.Source: §15 · p. 15
  • 74 of 79 simulator ground-truth stages are discovered (93.7% recall); the five unmatched annotations are micro-adjustments merged with adjacent stages.The predicates emphasize geometric and contact cues, and hard negatives capture near-miss states for SAR.Source: §15 · p. 16
  • Before accepting a dictionary, automated checks enforce mutual exclusivity and a six-frame minimum stage length, and reject unresolved TODOs or appearance-only predicates.Ten percent of stages are manually inspected; ambiguous predicates trigger a Gemini re-prompt.Source: §15 · p. 16

16. Additional Implementation Details

The implementation details specify how EvoVLA extracts and retains memory context, gates stage transitions, and creates and validates counterfactual hard negatives.

Memory latent extraction

  • The memory module mean-pools visual-token hidden states and projects them to a 128-D latent for memory attention and storage after gated fusion.The pooling and projection layers train end-to-end with the policy via PPO.Source: §16 · p. 16

Stage gating

  • A sliding window of 8 recent stage rewards advances the current stage when its average exceeds a threshold, capturing sustained progress and preventing premature transitions.The reward measures change in smoothed scores rather than raw scores.Source: §16 · p. 16

Hard negative generation

  • Gemini generates hard negatives as near-miss descriptions that differ from stage-positive descriptions by one spatial or contact predicate, avoiding appearance-based cues.Examples distinguish a gripper near a cup from touching it, or grasping a non-target object from grasping the target cup.Source: §16 · p. 16

Automated validation

  • Validation checks predicate exclusivity, minimum stage length, appearance terms, and triplet completeness; annotators spot-check 10% of stages and refine ambiguous predicates.Stages shorter than 6 frames are rejected, and failed outputs trigger automatic re-prompting.Source: §16 · pp. 16–17

17. Network Architecture Details

The architecture details describe EvoVLA’s visual and language inputs, chunked action prediction, auxiliary world models, and trainable components.

Visual Encoding Pipeline

  • Two camera views produce 1152-D and 1024-D visual features, projected together to the 4096-D Llama-2-7B token space; the vision encoders remain frozen during fine-tuning.FiLM layers inject language embeddings into visual features before the combined sequence enters the language model.Source: §17 · p. 17

Action Decoding

  • The action head predicts 8-step chunks of 7-D actions, but inference executes only the first action before recomputing.The 7-D action comprises 6-DoF end-effector twist deltas and a gripper command; training uses L1 regression loss.Source: §17 · p. 17

World Model Specifics

  • The forward model maps 13-D state-action inputs to 6-D outputs, while the inverse model maps consecutive 6-D states to 7-D actions; both use MSE loss and update each PPO epoch.Each uses a lightweight 256-D hidden-layer architecture and a separate Adam optimizer.Source: §17 · p. 17

Model Size and Efficiency

  • EvoVLA uses a 7B-parameter Llama-2-7B backbone with pretrained vision encoders; fine-tuning freezes the vision encoders while training the language model, projections, and task-specific modules.The passage estimates approximately 7B trainable parameters.Source: §17 · p. 17

18. Computational Cost

The computational-cost estimate reports training duration, hardware, parallel simulation, and peak GPU memory for Discoverse-L.

Training cost

  • Training takes approximately 24 hours per seed for 2M environment steps on 4 NVIDIA H20 GPUs using 8 parallel DISCOVERSE simulators.Each GPU has 96 GB VRAM.Source: §18 · p. 17
  • Across 3 random seeds, total training time is approximately 72 hours (3 days).Source: §18 · p. 17

Memory footprint

  • Peak memory reaches approximately 80 GB per GPU during policy updates.Source: §18 · p. 17
Loading 2511.16166v1…