PaperlayerView exact revision on arXiv

Source-linked AI summary

Evo-Depth: A Lightweight Depth-Enhanced Vision-Language-Action Model

Tao Lin, Yuxin Du, Jiting Liu, Nuobei Zhu, Yunhe Li, Yuqian Fu, Yinxinyu Chen, Hongyi Cai, Zewei Ye, Bing Cheng, Kai Ye, Yiran Mao, Yilei Zhong, MingKang Dong, Junchi Yan, Gen Li, Bo Zhao

arXiv:2605.14950v1cs.CVcs.RO

TL;DR

VLA models struggle with precise spatial manipulation because 2D representations provide limited depth information, while explicit 3D solutions add sensing and complexity. Evo-Depth uses lightweight implicit depth features and spatial enhancement to improve action learning, achieving strong simulation and real-world results with efficient deployment.

  • Problem

    Current VLA models rely primarily on 2D visual representations, limiting depth information and spatial relationships needed for precise manipulation, while explicit 3D approaches add sensors, complexity, and sensitivity to noise.

  • Method

    Evo-Depth extracts compact implicit depth features from multi-view RGB observations and uses spatial enhancement and progressive alignment to integrate them with vision-language action learning.

  • Results

    Evo-Depth achieves superior simulation performance across four benchmarks and the highest real-world average success rate of 90%, with 3.2 GB GPU memory usage and 12.3 Hz inference frequency.

  • Takeaways & Limitations

    The results support lightweight implicit depth enhancement as an effective direction for practical, deployment-efficient VLA systems.

  • Takeaways & Limitations

    Real-world evaluation focuses on representative fixed-base tabletop manipulation tasks rather than broader mobile, whole-body, or varied embodied settings.

Abstract

from arXiv · show

Vision-Language-Action models have emerged as a promising paradigm for robotic manipulation by unifying perception, language grounding, and action generation. However, they often struggle in scenarios requiring precise spatial understanding, as current VLA models primarily rely on 2D visual representations that lack depth information and detailed spatial relationships. While recent approaches incorporate explicit 3D inputs such as depth maps or point clouds to address this issue, they often increase system complexity, require additional sensors, and remain vulnerable to sensing noise and reconstruction errors. Another line of work explores implicit 3D-aware spatial modeling directly from RGB observations without extra sensors, but it often relies on large geometry foundation models, resulting in higher training and deployment costs. To address these challenges, we propose Evo-Depth, a lightweight depth-enhanced VLA framework that enhances spatially grounded manipulation without relying on additional sensing hardware or compromising deployment efficiency. Evo-Depth employs a lightweight Implicit Depth Encoding Module to extract compact depth features from multi-view RGB images. These features are incorporated into vision-language representations through a Spatial Enhancement Module via depth-aware modulation, enabling efficient spatial-semantic enhancement. A Progressive Alignment Training strategy is further introduced to align the resulting depth-enhanced representations with downstream action learning. With only 0.9B parameters, Evo-Depth achieves superior performance across four simulation benchmarks. In real-world experiments, Evo-Depth attains the highest average success rate while also exhibiting the smallest model size, lowest GPU memory usage, and highest inference frequency among compared methods.

1 Introduction

Evo-Depth addresses the spatial limitations of 2D-based VLAs with lightweight implicit depth enhancement from multi-view RGB observations. Its modules preserve deployment efficiency while improving manipulation performance across simulation and real-world settings.

  • Motivation: Current VLAs often degrade on manipulation tasks requiring precise spatial understanding because 2D representations provide limited depth and spatial relationships.This affects accurate localization, fine-grained placement, and spatially consistent manipulation.
  • Motivation: Explicit 3D approaches improve spatial understanding but can require additional sensors and geometry-processing pipelines, increasing complexity and sensitivity to noise and reconstruction errors.
  • Approach: Evo-Depth extracts compact latent depth features from multi-view RGB observations and injects them into vision-language representations without explicit 3D inputs or additional sensing hardware.The IDEM captures relative spatial relationships and object layout, while the SEM uses depth features as lightweight modulation signals.
  • Results: With 0.9B parameters, Evo-Depth reaches 84.4% on Meta-World, 41.1% on VLA-Arena, 95.4% on LIBERO, and 69.6% on LIBERO-Plus.In real-world experiments, it achieves a 90% average success rate, 3.2 GB GPU memory usage, and 12.3 Hz inference frequency, while outperforming the baseline under four disturbance settings.
  • Approach: The framework combines implicit depth encoding, spatial-semantic enhancement, and progressive alignment to support downstream action learning while preserving a lightweight deployable design.

2 Related Work

Related work spans generalist and lightweight VLAs, explicit 3D perception, and the staged architecture used by Evo-Depth. These approaches frame the trade-off between spatial information, model complexity, and deployment efficiency.

  • VLA Models: Generalist VLAs unify language grounding, visual observations, and robot action generation through multimodal pretraining and end-to-end policy learning.
  • Efficient VLA Models: Lightweight VLA models such as TinyVLA, SmolVLA, and Evo-1 target faster inference, data efficiency, reduced computation, or preserved semantic alignment.
  • 3D Perception: Explicit 3D methods including 3D-VLA, SpatialVLA, and PointVLA incorporate 3D perception, spatial context, or point-cloud features into robotic policy learning.
  • Evo-Depth: Evo-Depth combines an Implicit Depth Encoding Module, vision-language backbone, Spatial Enhancement Module, and action expert, with staged training for cross-module coordination.

3 Method

Evo-Depth combines implicit depth extraction, vision-language encoding, spatial modulation, and action generation in a lightweight VLA architecture. Its multi-view RGB pipeline progressively aligns depth-aware representations with downstream policy learning.

  • Architecture: Evo-Depth comprises IDEM, a vision-language backbone, SEM, and an action expert for depth-enhanced policy learning without explicit 3D inputs or additional sensors.The model conditions action prediction on spatially enhanced vision-language representations and robot state.
  • Implicit Depth Encoding Module: IDEM extracts compact implicit depth features from multi-view RGB observations using a 0.13B-parameter Vision Transformer initialized from a pretrained multi-view depth encoder.The encoder is designed to capture spatial relationships across observations and outputs latent depth-aware representations.
  • Implicit Depth Encoding Module: IDEM progressively alternates within-view and cross-view attention so spatial cues from different observations are aggregated into a unified latent feature space.Early layers preserve local image structure, while later layers enable information exchange across views.
  • Vision-Language Backbone: The vision-language backbone jointly encodes multi-view RGB observations and language instructions into 2D representations using a pretrained InternVL3-1B model.The implementation retains the first 14 language-branch layers to leverage intermediate cross-modal alignment.
  • Spatial Enhancement Module: SEM transforms depth features into the vision-language hidden space, averages them into a global descriptor, and uses FiLM-style scaling and shifting to produce spatially enhanced representations.The resulting representations are passed to the action expert for action generation.
  • Progressive Alignment Training: Progressive alignment training first optimizes SEM and the action expert, then unfreezes IDEM while keeping the vision-language backbone frozen to align depth and vision-language features.The strategy gradually aligns newly introduced modules with pretrained representations before broader optimization.

4 Simulation Experiments

Evo-Depth is evaluated on four simulation benchmarks spanning varying task difficulty, long-horizon execution, and distribution shifts. It achieves strong or state-of-the-art performance across these settings, including robust results on challenging spatial tasks.

  • The evaluation covers Meta-World, VLA-Arena, LIBERO, and LIBERO-Plus under varying difficulty, long-horizon execution, and distribution shifts.
  • 84.4 average success rate on Meta-World, with the best results on medium, hard, and very hard splits.
  • 95.4 average success rate on LIBERO, with balanced performance across its task suites.
  • Evo-Depth delivers strong overall performance on VLA-Arena, particularly in distractor and long-horizon settings.
  • Evo-Depth demonstrates strong robustness on LIBERO-Plus across diverse perturbation dimensions.

5 Real-World Experiments

Real-world evaluation uses three manipulation tasks of increasing difficulty and compares task success with deployment efficiency. Evo-Depth achieves the strongest overall performance while offering a favorable efficiency profile.

  • The physical-robot setup includes partially occluded third-view observations during critical manipulation stages.
  • Evo-Depth achieves the strongest overall performance across Orange Placement, Tennis Deposit, and Cup Stacking.These tasks impose progressively stronger spatial-perception and fine-grained-manipulation requirements.
  • Performance gaps become more evident as tasks require greater spatial complexity, especially target motion and precise object-object interaction.
  • 90.0% average success, 0.9B parameters, 3.2 GB GPU memory, and 12.3 Hz inference establish Evo-Depth’s best overall efficiency-performance trade-off.The comparison covers parameter size, GPU memory usage, inference frequency, and average success rate.

6 Generalization Experiments

Evo-Depth is tested under four real-world disturbance settings covering appearance, distractors, and horizontal or vertical spatial perturbations. It consistently outperforms the baseline, especially when accurate spatial reasoning is required.

  • The generalization evaluation covers appearance changes, distractor interference, and horizontal and vertical spatial perturbations.Each task-disturbance combination uses 20 trials, with success rate computed from successful trials over total trials.
  • Evo-Depth achieves higher success rates than the baseline across all four disturbance settings and three real-world tasks.
  • Its advantage is more significant under position perturbations and precise object-object interaction, which demand accurate spatial reasoning.
  • Figure 5 summarizes generalization performance across perturbations using Pi0 and Evo-Depth.

7 Ablation Studies

Ablations examine implicit depth perception, progressive alignment training, and fusion strategies. The results support the depth module, three-stage alignment, and asymmetric fusion of depth with 2D vision-language representations.

  • Implicit Depth Perception Analysis: Evo-Depth outperforms the variant without implicit depth perception across all four LIBERO-Plus evaluation dimensions.The gains are particularly clear on camera, robot, and layout.
  • Multi-Stage Fusion Training: The three-stage alignment variant outperforms both one-stage and two-stage variants on LIBERO Long.The two-stage variant improves over one-stage training, while the additional intermediate stage further strengthens multimodal integration.
  • Fusion Strategy Analysis: Cross-attention and SEM-based modulation achieve much stronger results than concatenation on LIBERO Long.The comparison evaluates concatenation, cross-attention, and SEM-based modulation for integrating implicit depth with 2D vision-language representations.

8 Conclusion

Evo-Depth is a lightweight depth-enhanced VLA framework that improves spatially grounded manipulation while preserving deployment efficiency without additional sensing hardware. Its implicit depth encoding, spatial enhancement, and progressive alignment support depth-aware action learning across simulation and real-world settings.

  • 8 Conclusion: Evo-Depth combines an Implicit Depth Encoding Module, Spatial Enhancement Module, and Progressive Alignment Training for depth-aware spatial understanding.These components align enhanced spatial representations with downstream action learning.
  • 8 Conclusion: Experiments in simulation and real-world settings demonstrate Evo-Depth’s effectiveness, robustness, and deployment efficiency.
  • 8 Conclusion: Lightweight implicit depth enhancement is presented as an effective direction for practical VLA systems.

A.1 Simulation Benchmark Setup

The simulation evaluation uses four complementary benchmarks to test manipulation competence, VLA generalization, cross-task transfer, and robustness under perturbations. Each benchmark targets distinct spatial, language-conditioned, or disturbance-related demands.

  • Simulation Benchmark Scope: Four simulation benchmarks assess base manipulation competence, VLA-oriented generalization, cross-task transfer, and robustness under perturbations.Success rates are compared with representative VLA baselines under aligned evaluation settings.
  • Meta-World: Meta-World contains 50 manipulation environments grouped into easy, medium, hard, and very hard splits to examine performance as spatial demands increase.Its tasks cover localization, spatial coordination, and fine-grained object placement.
  • VLA-Arena: VLA-Arena evaluates 170 tasks across safety, distractor, extrapolation, and long-horizon task-structure dimensions.The benchmark also varies language commands and visual observations.
  • LIBERO: LIBERO covers spatial, object, goal, and long task categories spanning spatial reasoning, object understanding, goal-conditioned execution, and long-horizon composition.
  • LIBERO-Plus: LIBERO-Plus introduces perturbations involving object layout, camera viewpoints, robot initial states, language, lighting, backgrounds, and sensor noise.It is designed to expose robustness failures that standard benchmark scores may hide.

B.1 Real-World Experiment Setup

The real-world evaluation uses three progressively difficult tabletop manipulation tasks on a fixed-base xArm6 with complementary external and wrist-mounted RGB views. Success rates are measured under aligned conditions alongside model size, GPU memory, and inference frequency, while failure analysis identifies remaining precision challenges.

  • Tasks: Orange Placement, Tennis Deposit, and Cup Stacking impose progressively stronger requirements for spatial perception and manipulation precision.Cup Stacking has the tightest geometric tolerance, while Tennis Deposit involves a ball that can roll away.
  • Hardware Setup: The platform uses a fixed-base UFACTORY xArm6 with external and wrist-mounted Intel RealSense D435i RGB cameras.The two views provide complementary observations of the workspace and manipulation area.
  • Camera Configuration: Partial occlusion of the external camera during critical stages creates incomplete observations when accurate grasping and placement are needed.
  • Evaluation Protocol: Task success is reported under aligned observation and control settings, with parameter size, GPU memory, and inference frequency additionally measured for deployment efficiency.Efficiency measurements use an NVIDIA RTX 4090D platform.
  • Failure Case Analysis: Real-world failures remain concentrated in grasping under occlusion, securing a moving tennis ball, and cup insertion under tight geometric tolerances.These cases indicate that contact-rich interactions with occlusion, target motion, and precise alignment remain challenging.
  • Spatial Attention Visualization: Progressive alignment produces more concentrated IDEM attention around gripper-object interaction and target placement regions than the initial IDEM.

C.1 Generalization Experiment Setup

The broader evaluation examines robustness, training choices, implementation settings, external assets, and scope boundaries. It combines disturbance tests, ablations, deployment details, and explicit limitations to contextualize Evo-Depth’s generalization claims.

  • Generalization Tests: Real-world generalization is tested on three tasks under background, distractor, horizontal-position, and height-position disturbances.The task objectives and success criteria remain unchanged across settings.
  • Ablation Studies: Removing implicit depth perception tests its contribution on LIBERO-Plus camera, robot, light, and layout dimensions.
  • Ablation Studies: Progressive Alignment Training is compared through one-stage, two-stage, and three-stage variants on LIBERO Long.The variants differ in when SEM, the Action Expert, IDEM, and the vision-language backbone are optimized.
  • Ablation Studies: Concat, Crossatten, and SEM are compared as strategies for integrating implicit depth features with 2D vision-language representations.
  • Training Configuration: The default three-stage pipeline progressively trains SEM and the Action Expert, then IDEM, and finally all modules end-to-end.
  • Scope and Limitations: The study focuses on representative tabletop tasks and fixed-base manipulation rather than broader mobile or whole-body embodied settings.Future work is proposed for wider task, scene, and platform coverage.
  • Broader Impacts: Deployment should include safety constraints and human oversight because perception or action errors may cause failed grasps or object damage.
Loading 2605.14950v1…