PaperlayerView exact revision on arXiv

Source-linked AI summary

StrataVLA: Hierarchical and Efficient 3D Geometric Grounding for Vision-Language-Action Models

Jin Cui, Zhaoyu Pu, Botao Cai, Jun Ye, Xinyue Long, Boran Zhao, Pengju Ren

arXiv:2609.26071v1cs.RO

TL;DR

VLA models have strong semantic priors but remain limited in 3D spatial awareness, while existing approaches do not provide persistent task-relevant geometry throughout the backbone. StrataVLA uses layer-specific adapters to retrieve shared geometry and combines routing with caching; it reports consistent gains across evaluations, including 98.53% average LIBERO success and up to 88% fewer VGGT invocations.

  • Problem

    VLA models remain limited in reliable 3D spatial perception, while existing methods leave gaps in persistent backbone access to task-relevant geometry.

    Source: Introduction · p. 1

  • Method

    StrataVLA uses layer-specific Geometry Adapters to retrieve evidence from shared geometric features, with task-aware routing and temporal feature reuse.

    Source: Introduction · pp. 1–2

  • Results

    Across LIBERO, SimplerEnv, and real-world tasks, StrataVLA reports consistent gains, 98.53% average success across four LIBERO suites, and up to 88% fewer VGGT invocations.

    Source: Introduction · p. 2, Main results · p. 5

  • Takeaways & Limitations

    The results support hierarchical geometry injection as an effective and practical approach to spatially grounded robotic control.

    Source: Conclusion · p. 7

Abstract

from arXiv · show

Vision-Language-Action (VLA) models inherit strong semantic priors from large-scale vision-language pretraining, yet remain limited in robotic manipulation by insufficient 3D spatial awareness. Existing approaches either require explicit depth or point-cloud inputs, compress geometry into training-time supervision, or inject it only at the model input or action expert, leaving the vision-language backbone without persistent access to task-relevant spatial information. We introduce StrataVLA, a plug-and-play framework for hierarchical geometric grounding. A frozen geometry foundation model extracts shared geometric features from RGB observations, while sparse, layer-specific Geometry Adapters allow visual representations at selected backbone depths to retrieve relevant geometric evidence through cross-attention. To make inference-time geometry practical, StrataVLA further combines task-aware routing with an LRU feature cache that exploits temporal redundancy during task manipulation. Experiments on LIBERO, SimplerEnv, and real-world manipulation demonstrate consistent gains over strong VLA baselines. StrataVLA achieves 98.53% average success on LIBERO suites while reducing geometry-model invocations by up to 88%, establishing hierarchical geometry injection as an effective and efficient way to achieve spatially grounded robotic control.

Introduction

VLA models have strong semantic priors but limited reliable 3D spatial perception, while existing geometry-integration strategies leave gaps in persistent backbone access or inference efficiency. StrataVLA introduces layer-specific geometric retrieval and task-aware reuse to address those gaps.

Introduction

  • Reliable 3D spatial perception remains a VLA limitation, especially when occlusion, depth-dependent configurations, or precision manipulation make projection ambiguous.The cited passage notes that these are scenarios in which accurate geometry, depth, and spatial relations are essential.Source: Introduction · p. 1
  • Existing approaches trade off additional 3D processing, persistent access to geometry, preservation of pretrained semantic priors, or geometric awareness in the VLM backbone.The passage distinguishes explicit 3D inputs, training-time distillation, inference-time fusion, and action-expert injection.Source: Introduction · p. 1
  • StrataVLA connects a frozen VGGT geometry model to the VLM backbone with plug-and-play Geometry Adapters that retrieve task-relevant evidence at selected layers.Each selected layer uses its current image tokens to query shared geometric features through cross-attention.Source: Introduction · pp. 1–2
  • StrataVLA reports consistent gains across simulation and real-world manipulation while task-aware routing and an LRU cache reduce VGGT invocation frequency by up to 88%.The sparse adapters add 3.8% latency overhead when geometry is available, and the routing-and-cache system limits amortized end-to-end overhead to 23.0%.Source: Introduction · p. 2
  • The paper's stated contributions are persistent hierarchical grounding, sparse layer-specific adapters with zero-initialized residual injection, and efficient routing with feature reuse.The authors report consistent improvements across simulation and real-world manipulation with modest amortized latency overhead.Source: Introduction · p. 2

Related Work

Prior 3D-enhanced VLA methods differ in where and how geometry enters the policy, with trade-offs in sensing, inference access, and preservation of pretrained representations. StrataVLA instead keeps a shared geometric representation available for repeated retrieval at selected backbone layers.

3D-Enhanced Vision-Language-Action Models

Hierarchical and Stable Backbone Adaptation

  • VLM layers specialize from local visual structure to cross-modal grounding and task-level semantics, motivating distributed geometric injection; zero-initialized residual projections let grounding emerge gradually [Kaduri, Bagon, and Dekel 2024; Jiang et al. 2024; Li et al. 2025c; Abouzeid et al. 2025; Goodfellow et al. 2013; Biderman et al. 2024].The passage connects gradual adaptation to preserving pretrained capabilities amid the risk of catastrophic forgetting.Source: Related Work · p. 2

Method

StrataVLA adds a shared geometric memory to a VLA and lets selected backbone layers retrieve geometry relevant to their current visual representations. Task-aware routing, temporal feature reuse, and cached prefix information are used to make geometry practical during action generation.

Method

  • StrataVLA builds on π0, combining a vision-language backbone and Action Expert with a frozen 3D expert's shared geometric memory and a task-aware gate.The gate controls residual injection strength and whether the geometry branch executes.Source: Overview · p. 2

Geometric Memory Extraction

  • VGGT processes selected camera views resized to 518 × 518, producing camera and register tokens plus a 37 × 37 patch grid; the five special tokens are removed before spatial reshaping and stride-2 subsampling.The geometric sequence is reduced by 73.6% while preserving spatial layout; VGGT is frozen and its output detached during optimization.Source: Geometric Memory Extraction · p. 2

Hierarchical Geometric Injection

  • At selected depths, independent Geometry Adapters let current visual states query shared geometric memory, then scatter a gated residual update back only to visual-token positions.The residual projection is initialized so the added branch initially produces zero output, preserving the pretrained backbone's behavior while geometric information emerges during optimization.Source: Figure 1 · p. 3, Hierarchical Geometric Injection · p. 3

Task-Aware Geometry Routing and Reuse

  • A frame-level geometry route is predicted from instruction semantics and scene complexity, while an LRU cache exploits temporal redundancy in fixed-camera manipulation.The route can bypass VGGT and all Geometry Adapters; the cache is cleared at episode boundaries and disabled during training.Source: Task-Aware Geometry Routing and Reuse · pp. 3–4

Flow-Matching Action Generation and Inference

  • The Action Expert predicts conditional velocity for flow matching, and inference reuses the geometry-enhanced prefix key-value cache through ten Euler denoising steps without recomputing VGGT or Geometry Adapters.VGGT remains frozen during training; the model jointly optimizes the vision components, Action Expert, adapters, and routing heads, with a gate loss when task-level geometry labels are available.Source: Flow-Matching Action Generation and… · p. 4

Experiments

StrataVLA is evaluated across simulation and real-world manipulation, with analyses of geometric information across backbone layers and inference-time efficiency. The results report strong benchmark performance and substantial reductions in geometry-model computation.

Experimental Setup

  • Evaluation covers LIBERO’s four ten-task suites, SimplerEnv’s Google Robot tasks, and an in-house real-world suite, using π0 and OpenVLA-OFT policy families.The LIBERO policy is trained on a mixture of the four suites; the real-world evaluation uses 400 demonstrations across five tasks.Source: Experimental Setup · p. 4

Main results

  • 98.53% average success across four LIBERO suites, including 100.0% on LIBERO-Spatial, with best results on LIBERO-Object and LIBERO-Goal.The paper also reports consistent gains on SimplerEnv and state-of-the-art performance across simulation and real-world evaluations.Source: Table 1 · p. 5, Main results · p. 5

Layer-wise Geometric Probing

  • Sparse three-layer injection matches all-layer injection on LIBERO and Google Robot, performs slightly better in real-world evaluation, and optimizes more stably.Performance is largely insensitive to the exact layer within each shallow, middle, or deep range; LIBERO-Spatial has the strongest injection magnitude.Source: Main results · p. 5
  • Layer-wise probes and injection ratios identify shallow, middle, and deep geometric response stages, motivating injection at Layers 2, 8, and 13.Probe maxima can emerge downstream of injection: normal decoding peaks at Layer 4, depth-gradient decoding at Layer 9, and object-center error reaches 2.64 cm at Layer 15.Source: Layer-wise Geometric Probing · p. 6

Inference Efficiency

  • 80.8% of VGGT invocations are suppressed on average and up to 88.0% on LIBERO-Long; amortized VGGT cost falls from 120.0 ms to 23.0 ms on average.Three sparse GAs add 4.8 ms, or 3.8% overhead, when geometric memory is already available; caching has negligible accuracy effect and uses about 11.3 MiB.Source: Inference Efficiency · p. 7

Conclusion

StrataVLA grounds VLA models hierarchically by letting layer-specific adapters retrieve task-relevant evidence from shared geometric features. Task-aware routing and temporal reuse reduce geometry-model calls with negligible performance loss.

Conclusion

  • Layer-specific Geometry Adapters let visual tokens at different backbone depths retrieve task-relevant evidence from shared geometric memory.The framework uses hierarchical 3D grounding in VLA models.Source: Conclusion · p. 7
  • Task-aware routing and temporal reuse reduce geometry-model invocations while preserving performance with negligible loss.Source: Conclusion · p. 7
  • StrataVLA yields consistent gains across simulation and real-world tasks, supporting its use for spatially grounded robotic control.Source: Conclusion · p. 7

Additional Training and Data Details

The appendix describes data mixtures, sampling, optimization, and training protocols. It distinguishes LIBERO and Fractal-based SimplerEnv training setups and details two-stage optimization and router supervision.

Dataset Construction and Sampling

  • LIBERO sampling gives greater weight to Spatial and Long for their metric-spatial reasoning and long-horizon geometric-consistency demands.Sampling proceeds by suite, then task, demonstration, and valid anchor timestep, avoiding implicit overrepresentation of longer LIBERO-Long trajectories.Source: Dataset Construction and Sampling · p. 9
  • For Google Robot experiments, the pretrained policy is fine-tuned on real-robot Fractal trajectories, while SimplerEnv is used only for real-to-sim evaluation.Source: Dataset Construction and Sampling · p. 9
  • Fractal episodes are grouped by normalized instruction for temperature sampling, with an episode-disjoint 2% development split.An episode and valid action window are sampled uniformly within the selected instruction group.Source: Dataset Construction and Sampling · p. 9

Two-Stage Optimization

  • Stage I freezes the pretrained policy while optimizing adapters and routing heads; Stage II jointly optimizes policy components while keeping VGGT frozen.Source: Two-Stage Optimization · p. 10
  • LIBERO and SimplerEnv use 80K and 70K updates, while real-world training uses 30K updates and lower Stage II learning rates to reduce overfitting.Source: Two-Stage Optimization · p. 10
  • The backbones retain their native action objectives: flow matching for π0 and continuous ℓ1 regression for OpenVLA-OFT.No additional action loss is introduced for either backbone.Source: Two-Stage Optimization · p. 10

Router Warm-up and Weak Supervision

  • Geometry routing is forced during warm-up before the router’s continuous coefficient scales the injected residual end-to-end.At inference, coefficients below τroute = 0.10 are set to zero, bypassing VGGT and all Geometry Adapters.Source: Router Warm-up and Weak Supervision · pp. 10–11
  • Router supervision combines instruction-based geometry cues with task-level development-loss gains; intermediate cases remain unlabeled for the auxiliary router loss.Low-quality frames are excluded from that loss when the geometry branch is disabled by the image-quality constraint.Source: Router Warm-up and Weak Supervision · p. 10

Additional OpenVLA-OFT Results

On LIBERO, StrataVLA improves OpenVLA-OFT’s average success rate and shows that hierarchical geometric injection transfers to a decoder-only VLA with continuous regression actions.

Additional OpenVLA-OFT Results

  • 2.55 percentage points: hierarchical geometric injection improves average success rate relative to OpenVLA-OFT on LIBERO.The result supports transfer to a decoder-only VLA with a continuous regression action head.Source: Additional OpenVLA-OFT Results · p. 11

Related papers

Loading 2609.26071v1…