PaperlayerView exact revision on arXiv

Source-linked AI summary

ComVLA: Communication-Aware Split Inference for VLA Models in 6G-Connected Robotics

Boliang Liu, Wint Yi Poe, Jingyun Di, Riccardo Trivisonno, Giuseppe Caire

arXiv:2609.07838v1cs.RO

TL;DR

VLA models are too large for edge deployment, while cloud offloading strains wireless bandwidth and existing approaches leave a channel-adaptation gap. ComVLA uses language-guided token selection with a channel-capacity-matched budget; with 32 rather than 512 tokens, it reduces compute and latency at a small task-success cost.

  • Problem

    VLA edge inference is impractical and cloud offloading strains wireless bandwidth, while semantic codecs require channel-specific retraining and VLA token pruners use static budgets.

    Source: §I · p. 1

  • Method

    ComVLA uses language instructions to identify task-relevant visual tokens and dynamically sets the transmitted-token budget to match instantaneous channel capacity.

    Source: §I · pp. 1–2

  • Results

    Transmitting 32 instead of 512 tokens cuts cloud compute by 3.8× and inference latency by 22%, at a cost of 1.5 pp in task success rate.

    Source: §I · p. 2

  • Takeaways & Limitations

    ComVLA frames token pruning as capacity-constrained co-design between VLA inference and wireless communication for 6G-connected robotics.

    Source: §I · p. 2

Abstract

from arXiv · show

Connected robotics is an emerging 6G application where mobile robots follow natural-language instructions to manipulate physical objects. The Vision-Language-Action (VLA) models that enable this are too large to run on the robot; a common trend is to offload inference to the cloud. The wireless link, however, limits how much sensing data the edge can transmit per control step. Two recent lines address this constraint: semantic communication codecs compress sensor data but require channel-specific retraining, and VLA token pruners select tokens from image but ignore the channel. Our insight is that the dense semantic information contained in the language already indicates which visual tokens matter. We propose ComVLA, a framework that uses this language guidance to adapt the VLA token budget to the channel capacity. Transmitting 32 tokens instead of 512 on the LIBERO benchmark, ComVLA cuts inference compute by 74% and inference latency by 22% versus the original OpenVLA-OFT baseline, at a cost of 1.5 pp in average task success (95.4% vs. 96.9%), and it stays within the capacity budget under Rayleigh and Rician fading. These results demonstrate that co-designing VLA inference and wireless communication is a practical direction for 6G-connected robotics.

I. Introduction

ComVLA addresses the tension between impractical on-robot VLA inference and bandwidth-limited cloud offloading by selecting task-relevant visual tokens and matching their budget to channel capacity.

I. Introduction

  • VLA models are too large for practical edge deployment, cloud offloading strains wireless bandwidth, and action decisions depend on sparse task-relevant visual regions.The paper describes visual tokens as highly redundant in large vision-language models.Source: §I · p. 1
  • Semantic communication codecs require channel-specific retraining, while VLA token pruners use static budgets that do not adapt to dynamic wireless conditions.These approaches address bandwidth or compute, respectively, but leave a channel-adaptation gap.Source: §I · p. 1
  • ComVLA uses language instructions to identify task-relevant visual tokens and dynamically matches the transmitted-token budget to instantaneous channel capacity.Only tokens identified as task-critical by cross-attention voting are transmitted.Source: §I · pp. 1–2, Figure 1 · p. 1
  • The framework frames VLA token pruning as capacity-constrained selection, establishing a co-design surface between inference and wireless links for 6G-connected robotics.The paper presents ComVLA as a split-inference framework for VLA-based robotic manipulation.Source: §I · pp. 1–2
  • Transmitting 32 instead of 512 tokens cuts cloud compute by 3.8× and inference latency by 22%, at a cost of 1.5 pp in task success rate.The method also maintains robust performance under dynamic fading channels.Source: §I · p. 2

II. Related Work

Prior split-inference systems and token pruners leave the wireless link insufficiently integrated into VLA inference. ComVLA frames token selection as capacity-constrained, using language-guided importance scores to choose what the edge sends to the cloud.

Semantic and Task-Oriented Communication

  • Semantic and task-oriented communication codecs target semantic objectives rather than pixel fidelity, but require channel-specific retraining.DeepJSCC and WITT are examples of codec-based approaches.Source: §II · p. 2

VLA Models in Edge-Cloud Systems; Visual Token Pruning

  • ComVLA’s token-pruning framework co-designs split inference with channel capacity, unlike prior systems that transmit raw visual features over bandwidth-limited links or treat the link as fixed.Existing visual-token pruners target compute reduction and leave token count blind to channel state.Source: §II · p. 2

System Model

  • The system sends the capacity-selected visual-token subset from the edge to the cloud, where the VLA backbone predicts the robotic action.The selected tokens retain position encodings, and the paper states that visual tokens are spatially redundant.Source: §III · p. 2

Information Bottleneck Formulation

  • The paper motivates this selection objective through an information bottleneck view, using language-conditioned action relevance under a strict link-budget constraint.Direct mutual-information estimation is described as provably hard in high dimensions and impractical at robot control rates.Source: §III · p. 2

Capacity-Constrained Token Pruning

  • The edge selects the top-K* visual tokens by task-importance score, with K* set by channel capacity and each token’s bit overhead.Equal token payloads make sorting by importance an exact solution to the stated budgeted selection problem.Source: §III · p. 2

Attention-Based Semantic Importance Estimation

B. Capacity-Constrained Token Pruning

ComVLA matches the number of transmitted tokens to the link budget, then ranks tokens by task importance. Its selected tokens concentrate on manipulation-relevant regions across bandwidth constraints, unlike random pruning.

B. Capacity-Constrained Token Pruning

  • The capacity-aligned budget determines how many tokens can be sent, while the token-priority mechanism determines which tokens are retained.These are the budget and ranking components of the pruning procedure.Source: B. Capacity-Constrained Token Pruning · p. 3
  • K* is the per-step token budget supported by effective goodput, aligning the selected token count with the link budget.The number of retained tokens is set from channel capacity.Source: B. Capacity-Constrained Token Pruning · p. 3
  • For a representative LIBERO task, at K* values 128, 64, 48, and 32, ComVLA concentrates its limited budget on the end-effector, target object, and goal container, while random pruning scatters kept tokens uniformly and loses spatial continuity around the manipulated object.Source: B. Capacity-Constrained Token Pruning · p. 3

V. Performance Evaluation

The evaluation uses LIBERO with OpenVLA-OFT and compares ComVLA against token-pruning, codec, and random-selection baselines. The supplied passages describe the evaluation setup, pruning procedure, qualitative token selections, and compute-side latency accounting.

Evaluation Setup

  • ComVLA is evaluated on four LIBERO suites, using OpenVLA-OFT fine-tuned on LIBERO and SigLIP and DINOv2 visual encoders on the edge.Each suite has 10 tasks evaluated over 50 episodes; experiments run on an H100 GPU.Source: §V · p. 3

ComVLA Procedure

  • The edge embeds the instruction, scores visual tokens with cross-attention, calculates K* from SNR, and transmits the highest-scoring tokens.The algorithm forms the payload from the top-K* tokens.Source: §V · p. 3

Qualitative Token Selection

  • Across K* values of 128, 64, 48, and 32, ComVLA retains task-critical regions while random pruning scatters selections and loses spatial continuity.The figure passage reports this pattern for a representative LIBERO task and says it reflects behavior across all four suites.Source: Figure 3 · p. 4

FLOPs and Latency Accounting

  • Reported latencies measure compute-side inference only; the paper states wireless transmission contributes negligibly relative to that compute latency under its cited submillisecond air-interface targets.The quadratic 4s^2d attention term means reducing token count has a disproportionate effect on VLM cost.Source: §V · p. 4

A. Task Success and Compute Efficiency

ComVLA retains high task success at tight token budgets while reducing transmission and cloud compute. The largest reported task drop occurs on Long Horizon, and reducing the token budget does not reduce cloud-side latency.

A. Task Success and Compute Efficiency

  • LightVLA and VLA-ADP use content-adaptive budgets but cannot adjust K at runtime, while forcing their fixed-budget heads below their trained budget reduces token dropping to random selection.Source: A. Task Success and Compute Efficiency · p. 4
  • At K*=32, ComVLA reaches 95.4% success, reduces transmission by 16×, and cuts cloud compute from 8.8 to 2.3 TFLOPs.Random Pruning falls to 55.7% at K*=32 and to 32.6% on Object, versus ComVLA’s 97.0% on Object.Source: A. Task Success and Compute Efficiency · p. 4
  • Long Horizon drops by 4.4 pp at K*=32, compared with a 95.2% baseline versus 97–98% on the other suites; K=16 is beyond the practical operating range for both methods.Source: A. Task Success and Compute Efficiency · p. 4
  • On-device compute falls from 8.80 TFLOPs to 0.83 TFLOPs (90.6%), while cloud-side latency remains 42.3–42.7 ms across token budgets.Transmission and cloud compute fall near-linearly with K*, but the smaller budgets do not translate into cloud-side latency savings.Source: A. Task Success and Compute Efficiency · p. 4

B. Performance under Fading-Aware Token-Budget Emulation

ComVLA’s success degrades more gracefully than Random Pruning as SNR falls, with a large advantage in a low-SNR Rician example. Its success rate also drops only marginally under delayed CSI feedback.

B. Performance under Fading-Aware Token-Budget Emulation

  • The fading evaluation considers Rayleigh and Rician channels, while fixed-budget pruners and SemCom codecs are excluded.The stated reasons are that fixed-budget pruners cannot adapt to tight capacities and SemCom codecs require SNR-specific retraining.Source: B. Performance under Fading-Aware… · pp. 4–5
  • At 5 dB on Object under Rician fading, ComVLA reaches 88.0% versus 56.2% for Random Pruning, a 31.8 pp gap.At the same average SNR, Rician outperforms Rayleigh because its smaller SNR fluctuations avoid deep troughs that can push K* into steep low-budget degradation.Source: B. Performance under Fading-Aware… · p. 5
  • ComVLA’s success degrades more gracefully than Random Pruning as SNR drops.Source: B. Performance under Fading-Aware… · p. 5
  • On LIBERO-Spatial, delaying the SNR estimate by up to four control periods (200 ms) yields only a marginal success-rate drop.Source: B. Performance under Fading-Aware… · p. 5

C. Effect of Score Aggregation

Hard voting concentrates token-importance scores on foreground tokens, unlike soft voting, which spreads weight across background tokens. It yields a 1.7 pp average gain, with the largest reported increase on Long Horizon.

C. Effect of Score Aggregation

  • Under a tight K* budget, importance assigned to task-irrelevant patches displaces task-relevant tokens from the top-K* selection.The passage connects hard voting’s argmax commitment to the Information Bottleneck preference for concentrated importance scores.Source: C. Effect of Score Aggregation · pp. 5–6
  • Hard voting gains 1.7 pp on average over soft voting, including +5.0 pp on Long Horizon and −0.6 pp on Spatial.The effect is largest where the K*=32 budget is tightest relative to task complexity.Source: C. Effect of Score Aggregation · p. 6, Table III · p. 6
  • Hard voting concentrates each query on its single foreground argmax, while soft voting spreads importance across background tokens.Source: C. Effect of Score Aggregation · pp. 5–6

D. Effect of Token Quantization

INT8 halves payload relative to FP16 with negligible performance impact, while INT4 substantially degrades task success. The authors identify INT8 as the balance between bandwidth and accuracy.

D. Effect of Token Quantization

VI. Conclusion

ComVLA uses language-guided vision-token pruning to adapt split VLA inference to bandwidth constraints. Its 32-of-512-token setting reduces transmission, cloud compute, and latency with a small task-success cost, while tolerating fading and stale CSI.

VI. Conclusion

  • 32 of 512 tokens cut per-step TX data by 16× (1088 KB to 68 KB), cloud compute by 3.8×, and inference latency by 22%, at a task-success cost of 1.5 pp.Language-guided vision token pruning provides the transmission-priority signal for selecting task-critical tokens at the edge.Source: §VI · p. 6
  • ComVLA remains effective under Rayleigh and Rician fading and tolerates stale CSI feedback up to 200 ms.Source: §VI · p. 6
  • Future work targets real-channel evaluation, real-world deployment, and uncertainty-guided token scoring.Source: §VI · p. 6

Related papers

Loading 2609.07838v1…