VOTE: Vision-Language-Action Optimization with Trajectory Ensemble Voting

1 Introduction

• Existing VLAs face two stated problems: massive token generation raises inference latency and training cost, while underused generated actions can reduce performance.The introduction also identifies slow inference as a barrier to practical deployment.Source: §1 · p. 2
• VOTE improves OpenVLA’s average success rate by over 20% across four LIBERO suites and achieves 39× action-generation throughput on NVIDIA Jetson Orin.It also surpasses CogACT by 7% ASR on SimplerEnv WidowX Robot.Source: §1 · p. 2
• A special <ACT> token represents an entire action chunk, replacing multiple generated action tokens and avoiding sequential decoder passes.The authors say this also reduces training cost by requiring fewer input-output tokens and simplifies decoding.Source: §1 · p. 2
• The ensemble committee combines current and previous action predictions, using accumulated tickets to vote on the current action.Figure 1 depicts parallel generation of the next n actions alongside the ensemble voting strategy.Source: §1 · p. 2, Figure 1 · p. 2
• VOTE combines a training framework for generating one special token with an inference optimization that selects actions through ensemble voting.The authors summarize the framework and voting strategy as improving action success rate while reducing training costs and inference time.Source: §1 · p. 3

2 Related works and motivation

• CogACT uses a diffusion-transformer action module [14], SpatialVLA adds Ego3D position encoding [23], and OpenVLA fine-tunes the Prismatic VLM on OXE [12].The section situates these methods among VLA policies fine-tuned from pretrained VLMs on robot data [3] [14] [23] [2] [15] [18] [33].Source: §2 · p. 3

2.1 Massive Computational Overhead

• VLM decoding accounts for at least 50% of action-prediction latency across three models; CogACT adds diffusion overhead and SpatialVLA adds visual-token latency.Figure 2 presents latency for SpatialVLA, CogACT, and OpenVLA.Source: §2.1 · p. 3, Figure 2 · p. 4
• Padding action embeddings adds training input tokens, while diffusion action heads require 1.67×–2× more gradient steps than MLP heads [11].VLA fine-tuning also uses downstream robot datasets to adapt models to new tasks and embodiments.Source: §2.1 · pp. 3–4

2.2 Insufficient Utilization of Generated Actions

• Robots typically discard past action predictions despite their availability; prior aggregators [43] [14] can be ineffective or combine predictions from different action modes.This underuse of historical predictions is linked in the passage to less stable trajectories and potential performance degradation.Source: §2.2 · p. 4

3.1 Problem Statement

• The action sequence uses seven degrees of freedom: relative end-effector translation, rotation changes, and a binary gripper state.The model predicts an action sequence from an image and language instruction.Source: §3.1 · p. 5

3.2 Training Framework

• The bottleneck action-head design outperforms the isotropic architecture while using fewer parameters.The ablation reports this comparison for the action-head designs.Source: §3.2 · p. 6
• A single <ACT> token represents an action chunk, whose hidden state feeds an action head to predict multiple consecutive actions.This avoids generating multiple action tokens and supports parallel action prediction.Source: §3.2 · pp. 5–6

3.3 Ensemble Voting

• The ensemble compares adjacent-time-step predictions with the current prediction, then averages the actions in the more-voted similarity group.Similarity is measured by cosine similarity with an empirically set threshold of 0.5.Source: §3.3 · p. 7
• The ensemble gives the current prediction a definite vote and filters inconsistent mode predictions rather than simply averaging all predictions.Similarity is computed relative to the current action, which the authors associate with the most critical real-time information.Source: §3.3 · pp. 7–8

4.1 Experimental Setup and Baselines

• Evaluation uses LIBERO and SimplerEnv; OpenVLA is fine-tuned with LoRA, and action chunk size and ensemble horizon are both set to 8.Source: §4.1 · p. 8

4.2 Performance Evaluation

• On WidowX, VOTE reaches 58.3% average success and over 3× OpenVLA speedup; on LIBERO, it performs best in every sub-task.Source: §4.2 · pp. 8–9

4.3 Cross-Platform Inference Evaluation

• On Jetson Orin, VOTE reaches 46 Hz and 38.6× OpenVLA speedup; on A6000, chunk 16 reaches 48.8× OpenVLA speedup.Source: §4.3 · pp. 9–10, Table 2 · p. 9

4.4 Ablation Study

• The bottleneck architecture beats isotropic at chunk sizes 8 and 16; chunk 16 reduces average success by 1.5%, while voting peaks at 58.3% for τ = 0.5 and leads across τ ∈ [0.0, 0.8].Source: §4.4 · p. 10, Table 4 · p. 10

• VOTE predicts actions in a hidden latent space with an action-tokenizer-free method and bottleneck action head, reducing training and inference computation.Source: §5 · p. 10
• The framework simultaneously predicts multiple actions using a bottleneck action head.Source: §5 · p. 10
• An action ensemble algorithm optimizes action sampling; experiments report superior inference speedups and generative performance.Source: §5 · p. 10