GaussVLA: Geometry-Aware Spatial Reasoning for Vision-Language-Action Model

• Conventional VLA visual tokens lack explicit metric 3D geometry and treat patches equally despite differences in depth, surface orientation, and geometric reliability.Source: §1 · p. 2
• Per-pixel monocular depth primarily encodes distance, leaving surface orientation and geometric reliability to be inferred implicitly from potentially ambiguous depth patterns.Source: §1 · p. 2
• Current action pipelines encode spatial relationships implicitly, while autoregressive 2D reasoning adds sequential latency and remains decoupled from action decoding.Source: §1 · p. 2
• GST converts visual and depth features into compact 3D Gaussian primitives whose means encode position, covariances encode local geometry, and opacity represents depth-aware confidence.Source: §1 · p. 2
• DA-CoT uses learnable queries to cross-attend to GST tokens under language and flow-time conditioning, anchoring non-autoregressive reasoning to action prediction.Source: §1 · p. 2
• GaussVLA integrates GST and DA-CoT in an efficient sequence-modeling backbone and reports comparable state-of-the-art performance with greater computational efficiency.Source: §1 · pp. 2–3

• Autoregressive action-token methods preserve language grounding but accumulate prediction errors over long sequences and become computationally costly as chunk length increases.Source: §2 · p. 3
• Diffusion-based policies capture multimodal, smooth trajectories but require multiple denoising steps, slowing execution.Source: §2 · p. 3
• Decoupled diffusion or flow-matching action heads improve continuous control, while adding computational overhead through extra heads or repeated denoising.Source: §2 · p. 3
• RGB-D and point-cloud approaches improve spatial awareness but can lose fine-grained morphology and remain sensitive to depth-estimation errors and sensor noise.Source: §2 · p. 3
• Textual 2D Chain-of-Thought reasoning is interpretable but incurs autoregressive latency, whereas recurrent latent reasoning sacrifices auditability and can degrade under excessive unrolling.Source: §2 · pp. 3–4
• Mamba-style selective state-space models provide linear-time sequence processing as an efficient alternative to transformers' quadratic attention and growing KV cache.Source: §2 · p. 4
• GaussVLA addresses the representation and reasoning trade-off by supervising short structured CoT in metric 3D coordinates and coupling its Mamba core to a Gaussian field.Source: §2 · p. 4

3 Proposed Method: GaussVLA

• GaussVLA processes RGB observations and language instructions through dual-stream visual encoding, structured tokenization, reasoning, and flow-matching action prediction.Semantic and depth features are transformed into Gaussian spatial tokens, refined with reasoning tokens, and decoded by a Mamba-based policy.Source: §3 · p. 4, §3.1 · p. 5

3.3 Gaussian Spatial Tokenization (GST)

• GST converts dense 2D patch features and predicted depth into compact 3D Gaussian tokens encoding semantic content, spatial position, and confidence-aware local geometry.Patch-center coordinates and corrected depth support back-projection, while Gaussian parameters represent position, geometric structure, and reliability.Source: §3 · p. 4, §3.3 · p. 6
• Learned spatial queries aggregate geometry-aware patch tokens into a compact representation passed to DA-CoT and the action decoder.The pooling module uses learnable queries and spatial attention to produce the final GST features.Source: §3.3 · p. 7

3.4 Depth-Aware Chain-of-Thought (DA-CoT)

• DA-CoT uses learnable reasoning queries to attend over GST tokens under joint language and flow-time conditioning, producing structured non-autoregressive spatial summaries.Each reasoning token summarizes a distinct GST subset conditioned on the task and flow time.Source: §3.4 · p. 8

3.5 GaussVLA

• GaussVLA aligns geometry-aware observation tokens, DA-CoT reasoning tokens, and flow-state action tokens within a single conditional policy backbone.The integrated design refines action states using contextual states, DA-CoT tokens, and time embeddings before predicting conditional flow velocity.Source: §3.5 · p. 9

LIBERO

• 93.5% average success rate is achieved on LIBERO, including 100.0% on Spatial, 95.8% on Object, and 95.3% on Goal.This is the best overall average and exceeds QueST by 4.9 points.Source: §4.2 · p. 11

Meta-World and CALVIN

• GaussVLA remains competitive across Meta-World difficulty splits and ranks second behind GR-1 on CALVIN for one-to-four consecutive tasks.Its average CALVIN sequence length is 1.474, while very long-horizon recovery remains challenging.Source: §4.2 · pp. 11–12

Robustness

• 0.33 average SR is obtained on LIBERO-PRO despite larger shifts in appearance, position, semantics, instructions, and environments.The benchmark remains highly challenging for large pretrained VLM-based VLAs as well.Source: §4.3 · p. 13

Real-world evaluation

• 58.8% average success rate is achieved on real-world SO-101 multi-task evaluation, compared with 46.3% for SpatialVLA and 35.7% for ACT.The largest cited category gain is on Sorting, where GaussVLA exceeds SpatialVLA by 15.7 points.Source: §4.3 · p. 13

Ablations

• GST raises LIBERO performance from 78.1% to 90.5% and LIBERO-PRO performance from 11.2% to 29.0% when added alone.DA-CoT alone improves the vanilla model to 82.1% on LIBERO and 16.7% on LIBERO-PRO; combining both reaches 93.5% and 33.3%.Source: §4.4 · pp. 13–14

Efficiency and sensitivity

• 12.97 ms latency, 179M trainable parameters, and 4.83 GFLOPs are reported for full GaussVLA versus 10.85 ms, 158M, and 3.50 GFLOPs for vanilla.Increasing GST queries improves performance up to Ng = 128, while DA-CoT benefits saturate at Nr = 4.Source: §4.4 · p. 14

5 Conclusion

• GaussVLA lifts frozen semantic and depth features into compact anisotropic 3D Gaussian tokens that provide structured spatial cues and task-relevant geometry.DA-CoT supplies intermediate geometric supervision before action prediction.Source: §5 · p. 14
• 8.7% relative improvement over π0 and 13.5% over CoT-VLA are reported on LIBERO with only 200M parameters.This corresponds to about 97% fewer parameters than the cited 7B-scale VLA baselines.Source: §5 · p. 14
• GST provides the dominant performance gain, while DA-CoT contributes complementary improvements in reasoning and robustness.Results on Meta-World, CALVIN, and real-world SO-101 further support simulation generalization and physical robot execution.Source: §5 · p. 14

A.2.1 Architecture Hyperparameter

• Table 6 lists the architecture and training hyperparameters used for all reported GaussVLA runs.The defaults apply unless otherwise stated.Source: Table 6 · p. 19, §A.1 · p. 19

A.2.1 Hardware

• A single SO-101 6-DOF arm with a parallel-jaw gripper is evaluated using a Realsense RGB-D camera at 640×480 and 60 Hz.Control runs at 15 Hz with action-chunk replanning every H=10 steps.Source: §A.2.1 · p. 19
• Inference runs on one GeForce 1080 Ti external workstation, demonstrating deployment viability on consumer-grade hardware.This differs from the RTX Pro 6000 Blackwell used for main-paper latency benchmarking.Source: §A.2.1 · p. 19

• Pick-and-Place requires lifting a 3-cm cube from a 30×30 cm workspace and placing it in a box.Source: §A.2.2 · p. 19
• Stacking requires ordering cubes and a small wheel according to language-specified object instructions.Source: §A.2.2 · p. 19
• Sorting separates a cube and small wheel into left and right bins.Source: §A.2.2 · p. 19
• Pick-Place ID uses the training camera pose, whereas OOD shifts the camera by ±5 cm and ±10° rotation.Source: §A.2.2 · p. 19

• 50 demonstrations per task are collected via teleoperation.Source: §A.2.3 · p. 19
• SigLIP and Depth-Anything-V2 visual and depth encoders remain fully frozen during training.Source: §A.2.3 · p. 19

Per-Component Latency Breakdown

• 12.97 ms is the total end-to-end inference latency per step under batch size 1 and BF16 precision on an NVIDIA RTX Pro 6000 Blackwell.Source: §A.3 · p. 19, Table 7 · p. 20
• 8.50 ms comes from frozen encoders, while trainable modules add 4.47 ms; with H = 10, amortization yields approximately 5.32 ms per control step, or about 188 Hz.Source: §A.3 · p. 20

Geometric Representation Probes

• DA-CoT tokens encode richer 3D geometric structure than flat 2D patches or pooled GST representations, according to LIBERO linear-probe regression R2.Source: Table 8 · p. 20, §A.4 · p. 20

Backbone and Action Decoder Ablation

• 73.36 LIBERO average with Mamba improves over 67.49 while reducing computation from 9.82 to 3.28 GFLOPs; the action-token query decoder reaches 78.1 at 3.50 GFLOPs.Source: §A.5 · p. 21, Table 9 · p. 21

GST Ablation

• 90.5 LIBERO average and 100.0 spatial performance with full GST exceed scalar-depth concatenation, which reduces the average from 78.1 to 73.3; LIBERO-PRO reaches 29.0.Source: §A.6 · pp. 21–22, Table 10 · p. 21

Confidence-Aware Pooling

• Removing confidence bias lowers LIBERO average from 93.5 to 91.6 and LIBERO-PRO average from 33.3 to 29.7, isolating a measurable robustness contribution from opacity-weighted pooling.Source: §A.8 · pp. 22–23

Meta-World Successful Frames

• A Very Hard Meta-World example demonstrates object localization, grasping, and precise placement during a spatially demanding manipulation task.Source: §A.11 · p. 24