PaperlayerView exact revision on arXiv

Source-linked AI summary

Text-Vision Synergistic Token Caching: A Training-Free Framework for Efficient Vision-Language-Action Inference

Qianer Li, Chengjie Zhang, Jingwen Chen, Zanjia Tong, Jiyuan Zhang, Hong Zhang

arXiv:2609.34319v1cs.CVcs.RO

TL;DR

Existing VLA caching does not fully use text-vision synergy to account for attention-head reliability and layer stability, limiting how it selects visual tokens and reuse locations. TVCache filters attention heads and selects reuse layers using text-vision signals. Across the reported evaluations it improves task success at matched retention ratios; on OpenVLA-OFT at 12.5% retention, average success improves by 14.5 percentage points over VLA-Cache.

  • Problem

    Existing VLA caching does not fully exploit text-vision synergy or explicitly account for head-wise reliability and layer-wise stability.

    Source: §I · p. 1

  • Method

    TVCache is a training-free framework that filters attention heads by text-vision information focus and selects reuse layers using text-vision entropy differences.

    Source: §I · p. 1

  • Results

    At 12.5% retention, OpenVLA-OFT average success improves by 14.5 percentage points over VLA-Cache, from 69.5% to 84.0%, while FLOPs are 2.45× lower than full-token inference.

    Source: §I · p. 2

  • Takeaways & Limitations

    Across four VLA architectures, two simulation benchmarks, and real-world robotic tasks, TVCache consistently improves task success at matched token-retention ratios.

    Source: §VI · p. 8

Abstract

from arXiv · show

Vision-Language-Action (VLA) models enable generalizable robotic control but remain computationally expensive. Token caching provides a training-free, plug-and-play acceleration alternative. However, existing VLA caching does not fully exploit a key inductive bias of VLA models: text-vision synergy, wherein textual semantics guide the precise visual grounding of task-relevant regions. In particular, existing designs insufficiently account for head-wise reliability in attention aggregation and layer-wise stability in cache reuse. To address this, we propose Text-Vision Synergistic Token Caching (TVCache), a training-free framework for efficient VLA inference. TVCache filters attention heads based on text-vision information focus to improve task-relevant and physically consistent visual grounding. Concurrently, we introduce a reuse-layer selection mechanism guided by text-vision entropy differences to avoid caching unstable representations and improve cache resource allocation. Extensive experiments across four representative VLA models, two simulation benchmarks, and real-world robotic tasks demonstrate the effectiveness and generality of TVCache. At matched token-retention ratios, TVCache consistently improves task success over existing VLA caching with comparable computational cost. On OpenVLA-OFT, it improves average success by up to 14.5 percentage points over VLA-Cache at 12.5% retention while reducing FLOPs by 2.45x relative to full-token inference.

I. INTRODUCTION

Existing VLA caching does not fully use text-vision synergy to select relevant visual tokens and stable reuse layers. TVCache addresses these gaps, and experiments report higher task success than existing caching at matched retention ratios.

I. INTRODUCTION

  • Repeated processing of dense visual tokens makes real-time VLA inference costly, while token caching offers a training-free alternative that reuses prior KV representations.Token caching exploits temporal redundancy across consecutive observations and is described as complementary to model-level acceleration.Source: §I · p. 1
  • Existing VLA caching insufficiently accounts for head-wise reliability in attention aggregation and layer-wise stability during cache reuse.Indiscriminate head aggregation can bias token relevance toward background regions, while reuse at unstable layers or after stabilization can propagate unreliable features or waste computation.Source: §I · p. 1
  • TVCache filters attention heads using text-vision information focus and selects reuse layers using text-vision entropy differences.The mechanisms target task-relevant, physically consistent visual grounding and avoid unstable caching locations while reducing redundant computation.Source: §I · p. 1
  • At 12.5% retention, OpenVLA-OFT average success rises from 69.5% to 84.0% over VLA-Cache, with 2.45× fewer FLOPs than full-token inference.Experiments span four representative VLA models, two simulation benchmarks, and real-world robotic tasks; TVCache consistently improves task success at matched token-retention ratios with comparable computational cost.Source: §I · p. 2

II. RELATED WORK

Prior VLA acceleration methods include model changes and token caching, but caching approaches do not explicitly select reliable attention heads and stable reuse layers. TVCache adds those choices to temporal KV-cache reuse.

II. RELATED WORK

  • Existing VLA caching reuses temporal information but does not explicitly model head-wise reliability or select layers by representation stability; TVCache targets both gaps while remaining training-free.Other acceleration approaches include lightweighting, quantization, and early exits, which typically require retraining, calibration, or architectural modifications.Source: §II · p. 2

A. Overview of the TVCache Framework

  • Token caching reuses historical KV representations for stable visual regions and recomputes dynamic ones to reduce redundant Transformer computation.At timestep T, the framework figure distinguishes static and dynamic visual tokens and shows reuse of historical caches for invariant instructions and static tokens.Source: A. Overview of the TVCache Framework · p. 2, Figure 2 · p. 3
  • TVCache evaluates individual attention heads before estimating token relevance and uses text-vision entropy differences to identify stable reuse locations across network depth.These token- and layer-level mechanisms target spatial misalignment and depth mismatch, respectively.Source: A. Overview of the TVCache Framework · p. 3

B. Temporal Patch Similarity

TVCache first identifies temporally similar visual patches as cache candidates, then uses selected attention heads to protect task-relevant patches from reuse. Head selection combines text-conditioned response strength with spatial attention concentration.

B. Temporal Patch Similarity

  • Top-k patches with the highest temporal similarity above threshold τ form initial cache-reuse candidates, with k set by the token-retention budget.The similarity is cosine similarity; task relevance is applied afterward.Source: B. Temporal Patch Similarity · p. 3

C. Task-Relevant Token Selection via Attention-Head Filtering

  • Attention heads differ in grounding reliability, so averaging them can dilute informative responses and distort visual-token relevance.Some heads focus on interaction-relevant regions, while others spread attention over background or task-irrelevant content.Source: C. Task-Relevant Token Selection via… · p. 3
  • TVCache scores heads using semantic response intensity and attention entropy, selecting heads with strong text-conditioned responses and concentrated spatial attention.The attention weights are normalized over the visual dimension; α = β = 0.5 across experiments without model- or benchmark-specific tuning.Source: C. Task-Relevant Token Selection via… · p. 3
  • Task-relevant visual-token scores are estimated from the selected heads, and interaction-critical tokens are excluded from the temporally stable candidates to form the reusable set.The reusable set is Preuse = Pstatic \ Ptask.Source: C. Task-Relevant Token Selection via… · p. 3

D. Cache Resource Allocation via Reuse-Layer Selection

TVCache uses text-to-vision attention entropy to assess layer stability and select reuse locations under a cache budget. It then varies token retention by depth and reuses selected KV states during inference.

D. Cache Resource Allocation via Reuse-Layer Selection

  • Lower entropy indicates more concentrated text-to-vision attention and serves as a layer-stability cue for choosing reuse locations.The layer stability score is one minus min-max-normalized entropy across layers.Source: D. Cache Resource Allocation via… · p. 4
  • Given a budget of K layers, reuse-layer selection favors high stability while avoiding excessive concentration in a narrow depth range.An interval-constrained greedy algorithm appends layers with the highest stability score while satisfying dmin.Source: D. Cache Resource Allocation via… · p. 4
  • Retention increases monotonically with depth, from aggressive token reduction in shallow layers to full retention at the terminal reuse layer.During inference, selected KV states for reusable tokens are loaded according to each layer’s retention proportion, bypassing their self-attention computation.Source: D. Cache Resource Allocation via… · p. 4

IV. EXPERIMENT

The evaluation covers four VLA architectures in simulated and real-world settings, including a physical-robot deployment of OpenVLA-OFT.

IV. EXPERIMENT

  • TVCache is evaluated with OpenVLA-OFT, BitVLA, π0.5, and VLA-Adapter across simulated and real-world environments.The experiments use a workstation with four NVIDIA RTX 4090 GPUs.Source: §IV · p. 4
  • The real-world evaluation deploys OpenVLA-OFT on a physical robotic platform.The passage identifies this as validation of real-world robustness.Source: §IV · p. 4
  • The evaluation scope combines simulation with physical-robot testing rather than relying on simulation alone.The named architectures include OpenVLA-OFT, BitVLA, π0.5, and VLA-Adapter.Source: §IV · p. 4

A. Experimental Setup

The setup compares training-free accelerators under matched compute budgets and evaluates VLA performance across simulation and physical tasks. It also specifies measurement protocols and reports latency and task-performance metrics.

A. Experimental Setup

  • TVCache is compared with FastV, SparseVLM, DivPrune, and VLA-Cache under compute budgets, using fixed α = β = 0.5, K = 4, dmin = 2, and γ(l) = [0.5, 0.65, 0.8, 1.0].These settings are held across models and benchmarks.Source: A. Experimental Setup · p. 4
  • Table I compares OpenVLA-OFT and BitVLA on LIBERO, with task-success retention ratios and speedups calculated against each model’s vanilla 100%-token baseline.The table caption identifies latency and performance as comparison measures.Source: Table I · p. 5
  • Methods are evaluated at 50%, 25%, and 12.5% retention using task success, FLOPs, control frequency, and end-to-end latency.Latency is measured on one RTX 4090 at batch size 1 in BF16, excluding 10 warm-up runs and averaging 500 measured runs.Source: A. Experimental Setup · pp. 4–5
  • Evaluation spans four LIBERO task suites for four VLA architectures, VLA-Adapter on CALVIN Split ABC →D, and four real-robot manipulation tasks.LIBERO uses 500 episodes per suite; the real-robot evaluation uses 100 demonstrations and 50 trials per task.Source: A. Experimental Setup · p. 5

B. Results and Analysis

TVCache improves task success across simulated and real-world VLA evaluations at matched retention, with gains especially notable under tighter token budgets. Its ablations and attention visualizations support the contributions of head filtering and reuse-layer selection.

LIBERO Benchmark

  • At 12.5% retention, OpenVLA-OFT success rises from 69.50% to 84.00% over VLA-Cache, with 2.45× lower FLOPs than full-token inference.The LIBERO results also report control frequency rising from 10.43 Hz to 15.04 Hz and latency falling from 95.87 ms to 66.51 ms.Source: B. Results and Analysis · pp. 5–6
  • At 25% retention, TVCache improves over VLA-Cache from 86.85% to 93.90% on π0.5 and from 48.35% to 54.40% on VLA-Adapter.Gains are also reported at 50% retention, and are larger under tighter token budgets.Source: B. Results and Analysis · p. 6, Table II · p. 6

CALVIN Benchmark

  • At 25% retention on CALVIN Split ABC →D, VLA-Adapter success rises from 48.06% to 58.24% over VLA-Cache, with a 1.92× FLOPs reduction versus full-token inference.Latency falls from 46.62 ms to 43.72 ms and control frequency rises from 21.45 Hz to 22.87 Hz; the passage attributes the modest latency gain to routing overhead being more significant on the lightweight model.Source: B. Results and Analysis · p. 6, Table III · p. 6

Runtime Overhead

  • TVCache adds 2.84–2.89 ms of nearly constant runtime overhead, with reuse-layer selection the main contributor and token selection plus bookkeeping adding less than 0.4 ms.The fixed cost is more noticeable on lightweight models and less significant on larger backbones.Source: B. Results and Analysis · p. 6, Table V · p. 6

Real Robot Deployment

  • In physical deployments, TVCache surpasses VLA-Cache across retention ratios and matches full-token performance at 50% retention on SetKettle (94%) and HangCup (82%).At 12.5% retention, it records 87% success on SetKettle.Source: B. Results and Analysis · p. 7, Figure 4 · p. 7

Ablation Study

  • At 12.5% retention on OpenVLA-OFT, reuse-layer selection alone raises average success from 69.50% to 82.20%, and adding attention-head filtering reaches 84.00%.The Object suite has the largest reported gain: 62.00% to 84.80% (↑22.80 pp); improvements are also reported on BitVLA.Source: B. Results and Analysis · p. 7, Table VI · p. 7

Qualitative Analysis of Attention Distributions

  • TVCache filters degraded attention heads to produce more concentrated attention on task-relevant regions than VLA-Cache’s diffuse, spatially misaligned responses.The comparison is presented as qualitative evidence of improved visual grounding.Source: B. Results and Analysis · p. 7, Figure 5 · p. 7

A. Definitions and Baseline Complexity

The baseline analysis defines the sequence dimensions and isolates attention-related computation. It attributes that cost to projection and attention operations.

A. Definitions and Baseline Complexity

B. TVCache Computational Savings

At selected reuse layers, TVCache uses cached K/V states for reused visual tokens to avoid redundant attention-related computation.

B. TVCache Computational Savings

  • At selected reuse layers, Lr reused visual tokens reuse cached K/V states, avoiding redundant projections and query-side attention computation.The per-layer savings comprise projection savings and reduced attention computation.Source: B. TVCache Computational Savings · p. 7
  • The per-layer savings account separately for reduced projection cost and reduced attention computation.The first term represents projection savings; the second represents reduced attention computation.Source: B. TVCache Computational Savings · p. 7

C. Total Computational Complexity

TVCache reduces attention-related costs through token reuse, while unchanged components and routing overhead limit end-to-end speedup. Experiments report improved task success at matched retention ratios across multiple VLA settings.

C. Total Computational Complexity

  • At 12.5% retention on OpenVLA-OFT, TVCache achieves a 2.45× FLOPs reduction and a 1.44× latency speedup.Routing overhead remains 2.84–2.89 ms across retention ratios.Source: C. Total Computational Complexity · p. 8
  • Token reuse reduces projection and attention costs, but unchanged model components and routing overhead constrain the corresponding end-to-end speedup.The per-layer complexity is derived by subtracting caching savings from baseline cost.Source: C. Total Computational Complexity · pp. 7–8

VI. CONCLUSION

  • TVCache consistently improves task success at matched token-retention ratios across four VLA architectures, two simulation benchmarks, and real-world robotic tasks.The conclusion identifies text-vision-aware cache allocation as effective for training-free VLA acceleration.Source: §VI · p. 8

Related papers

Loading 2609.34319v1…