PaperlayerView exact revision on arXiv

Source-linked AI summary

Efficient Vision-Language-Action Management and Serving for Robot Factories

Dionysios Adamopoulos, Nattapol Chanpaisit, Basel Fakhri, Christina Giannoula

arXiv:2609.12075v1cs.DCcs.ARcs.LGcs.PFcs.RO

TL;DR

Strict latency and safety demands make VLA inference difficult to run on robots that cannot accommodate high-end GPUs, while existing serving systems are not suited to multi-model, multi-GPU SLO-aware execution. Robion combines concurrent VLM–ADiT serving with flexible placement and traffic control; it serves more robot load than vLLM-Omni and Monolithic, including up to 64 robots across 8 models on 4 GPUs.

  • Problem

    Robots need low-latency VLA inference but cannot practically integrate high-end GPUs, and existing systems do not support multi-model, multi-request execution across GPUs under SLOs.

    Source: §1 · p. 1, §2.5 · p. 4

  • Method

    Robion combines intra-GPU VLM–ADiT stream execution and SLO-aware scheduling with configurable model placement and traffic assignment that increases batching while bounding GPU load.

    Source: §1 · p. 2

  • Results

    Robion serves 6.7× and 1.5× more robot load within 98% SLO attainment than vLLM-Omni and Monolithic, respectively, and serves up to 64 robots across 8 models on 4 GPUs.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Local edge servers are a practical deployment target for serving VLA inference to many robots.

    Source: §1 · p. 1

    Throughput-oriented stage-disaggregated systems are poorly suited to millisecond-scale VLA stages, with buffer inspection accounting for 30–35% of vLLM-Omni inference latency.

    Source: §2.5 · p. 4

Abstract

from arXiv · show

Vision-Language-Action (VLA) models show high robotic manipulation capabilities via a two-stage design: a Vision-Language Model (VLM) stage followed by an Action Diffusion Transformer (ADiT) stage. Since robots must meet strict Service-Level Objectives (SLOs) for safety, VLA inference is inherently latency-critical. Meeting these SLOs requires high-end GPUs, yet weight, cost, and power constraints preclude integrating such GPUs on-robot. Prior works offload VLA inference to edge servers that serve many robots on VLA models. However, current VLA systems lack support for multi-request, multi-model execution on a multi-GPU server under SLOs, while existing serving systems for multi-stage models are optimized for throughput and stage disaggregation across separate GPUs, which are ill-suited for the millisecond-scale stages of VLA models. We design Robion, the first VLA serving and management system for multi-robot, multi-model requests on multi-GPU edge servers that meets SLOs. Our serving engine disaggregates the VLM and ADiT stages within a GPU via two streams, dynamically restricting the SMs on VLM stream so ADiT always finds SMs to run alongside it, and co-locates multiple models by sharing these streams across them, prioritizing requests by least remaining SLO time. Our management engine enables flexible model placements on multi-GPU servers, and integrates an intelligent traffic controller that maximizes per-model batching under the chosen placement while bounding each GPU's load to meet SLOs. For individual models, Robion serves on average 6.7$\times$ and 1.5$\times$ higher robot load within 98% SLO attainment over vLLM-Omni, the most widely used multi-stage serving system, and Monolithic, which runs VLM and ADiT as a single pipeline, respectively. In a large-scale experiment of serving 8 different models on a 4-GPU server, Robion can serve up to 64 robots within 98% SLO attainment.

1 Introduction

VLA inference is latency-critical, but current systems do not support multi-request, multi-model execution across GPUs under SLOs. Robion combines concurrent stage execution and flexible model placement with SLO-aware serving, achieving higher robot loads than existing systems.

Motivation

  • High-end GPUs help meet VLA latency demands, but robot weight, cost, and power constraints make on-robot deployment impractical; existing serving systems also lack SLO-aware multi-model execution.Prior multi-stage systems target throughput and incur costly buffer inspection for millisecond-scale VLA stages.Source: §1 · pp. 1–2

Serving engine

  • Robion overlaps VLM and ADiT work on separate GPU streams, dynamically restricts VLM SM use, and prioritizes requests with the least remaining SLO time.The VLM and ADiT streams are shared across co-located models.Source: §1 · p. 2

Management engine

  • Its management engine supports configurable model placement and assigns traffic to increase per-model batching while bounding GPU load to prevent SLO violations.Models can be co-located, replicated, or served on disjoint GPU groups.Source: §1 · p. 2

Results

  • Robion sustains 6.7× and 1.5× more robot load within 98% SLO attainment than vLLM-Omni and Monolithic, and serves up to 64 robots across 8 models on 4 GPUs.The 64-robot result is from a large-scale 4-GPU experiment.Source: §1 · p. 2

2 Background & Motivation

VLA models produce robot actions through two lightweight stages, while safety deadlines motivate edge inference. Their size and distinct compute profiles shape multi-robot serving objectives, and throughput-oriented multi-stage systems incur inefficiencies for VLA workloads.

2.1 Vision-Language-Action (VLA) Models

  • A VLA takes camera images and a text prompt through a VLM and ADiT to produce an action chunk; both stages typically take only milliseconds on high-end GPUs.ADiT typically runs 5–10 diffusion steps to produce 10–50 future actions.Source: §2.1 · p. 3, Figure 1 · p. 3

2.2 The Edge Server for VLA Inference

  • High-end edge GPUs can yield up to ∼3× lower latency than on-robot inference despite network cost, while on-robot inference can reduce battery life by up to ∼45%.Robots typically cannot integrate high-end GPUs because of weight, cost, and power constraints.Source: §2.2 · p. 3, Figure 2 · p. 3

2.3 Multi-Robot Serving Scenario

  • A robot factory assigns robots to tasks and models, which may be fine-tuned variants or different architectures, and can host them on a shared multi-GPU edge server.Robots send requests over a low-latency wireless network and receive action chunks.Source: §2.3 · p. 3

2.4 Key Objectives for Efficient VLA Serving

  • Efficient VLA serving should meet per-request latency SLOs, disaggregate compute-intensive VLM and memory-intensive ADiT stages to overlap concurrent requests, and co-locate multiple models to use high-end GPU memory.VLA models have 1–5 billion parameters and require ∼10GB for weights plus a few MB for inference tensors; cited examples are 96GB RTX 6000 Pro and 80GB H100 GPUs.Source: §2.4 · pp. 3–4

2.5 Limitations of Existing Systems

  • Throughput-oriented stage-disaggregated systems poorly fit millisecond-scale VLA stages: they underutilize separate GPUs and incur buffer inspection overhead, reaching 30–35% of vLLM-Omni inference latency.At batch size 4, the π0 VLM and ADiT tensor-core utilization averages 52% and 15%, respectively.Source: §2.5 · p. 4, Figure 3 · p. 4

3 Robion: Design Overview

Robion combines intra-GPU stage disaggregation and SLO-aware request scheduling with configurable model placement and adaptive traffic assignment. These mechanisms are designed for multi-robot, multi-model serving on multi-GPU edge servers.

Serving engine

  • Robion runs VLM and ADiT stages concurrently on separate GPU streams, using fixed-size batches and avoiding inspection and packing during millisecond-scale stages.It restricts VLM SM use in locksteps so ADiT can run alongside it, dynamically choosing the restriction by batch-size combination.Source: §3 · p. 4
  • A shared VLM stream and ADiT stream serve co-located models, prioritizing requests with the least remaining time before SLO expiration.This enables stages from different models to run concurrently on a GPU.Source: §3 · p. 4

Management engine

  • Programmer-configurable placement and an integer-program traffic controller support co-location, replication, or GPU partitioning while increasing per-model batching and limiting GPU load.GPU groups can be reserved for fine-tuned variants of one VLA architecture.Source: §3 · pp. 4–5

4 Robion: Detailed Design

Robion’s design combines intra-GPU stage execution and SLO-aware multi-model scheduling with flexible GPU placement and traffic assignment. Its initialization profiles execution and establishes robot limits and placements for the configured models.

4.1.1 Fixed-Batch Intra-GPU Stage Disaggregation.

  • Robion runs VLM and ADiT stages on separate GPU streams with fixed-size stage batches, allowing different batch sizes and overlapping independent request batches.Fixed-size batching avoids per-ADiT-step inspection and tensor-packing costs associated with admitting requests into running batches.Source: §4.1.1 · p. 5

4.1.2 Guarded & Dynamic Spatial SM Partitioning

  • Profiling selects VLM stream SM restrictions for co-running batch pairs, which Robion applies at lockstep starts to avoid stage preemption and shorten lockstep execution.The profiler searches candidate SM counts for each batch-size combination and stores the restriction that minimizes lockstep time.Source: §4.1.2 · pp. 5–6

4.1.3 SLO-Aware Dual-Stream Multi-Model Serving.

  • Robion shares one VLM stream and one ADiT stream across co-located models, selecting each lockstep’s queue by the head request’s least remaining SLO time.Offline profiling covers cross-model stage pairs; fine-tuned variants of the same architecture can reuse profiling results for matching batch sizes.Source: §4.1.3 · pp. 6–7

4.2 Robion Management Engine

  • Programmers can configure model placement by co-locating architectures or fine-tuned variants, replicating models across GPUs, and partitioning GPUs into disjoint groups.A group can serve fine-tuned variants of one VLA architecture.Source: §4.2.1 · p. 7
  • The traffic adapter concentrates requests for batching while splitting robot populations across GPUs when needed to keep each GPU within its supported limit.In the example, GPU 0 serves four Model A robots, while GPU 1 serves the remaining two Model A robots and two Model B robots.Source: §4.2.2 · p. 7, Figure 5 · p. 7
  • Robion benchmarks each model to find the maximum robot population a GPU can serve at the target SLO attainment, then assigns each model’s robots across GPUs hosting it.The per-model assignments sum to that model’s total robot population.Source: §4.2.2 · p. 7

4.3 Initialization of Robion

  • Initialization captures CUDA graphs, profiles stage restrictions, benchmarks per-model robot limits, and solves robot-to-GPU assignments for the configured placement.For eight models on a four-GPU server, the reported times for these four parts are approximately 4 minutes, 3 minutes, 1 hour, and 10 seconds, respectively.Source: §4.3 · p. 8

5 Evaluation

The evaluation spans four VLA models, heterogeneous robot populations, varying SLO deadlines, placement configurations, and two hardware platforms. Results include higher single-model robot loads, large-scale multi-model service, and lower SLO attainment over cloud.

5.1 Evaluation Methodology

  • The evaluation uses four VLA architectures, LIBERO inputs, two four-GPU servers, and model-specific deadlines based on Jetson AGX Thor inference latency.Unless stated otherwise, experiments use 2 Hz robot traffic, five epochs with resampled offsets, and WiFi 7 network latency.Source: §5.1 · p. 8

5.2 Single Model Serving on a GPU

  • Robion sustains 6.7× and 1.5× more robot load within 98% SLO-AT than vLLM-Omni and Monolithic, respectively.Across models and robot loads, Robion also achieves 1.4× lower p99 latency on average than Monolithic at its highest load before saturation.Source: Figure 6 · p. 8, §5.2 · p. 8
  • Guarded SM partitioning reaches 82% SLO-AT for PI0 at 20 robots and 89% for XR0 at 24, versus 0% for Monolithic.With max_batch=2, PI0 reaches 98% SLO-AT at 18 robots; statically restricting VLM to 96 of 188 SMs reduces supported load by about 18% on average.Source: Figure 7 · p. 9, Figure 8 · p. 9, §5.2 · p. 9

5.3 Multi-Model Co-Location and Serving on a GPU

  • Robion achieves 2.1× lower SLO-MR than Monolithic; ESF yields 1.06× and 1.07× higher average SLO-AT than RR and LQF.ESF has little advantage when co-located models share architectures and SLOs, but outperforms RR and LQF when architectures or SLOs vary.Source: Figure 9 · p. 9, §5.3 · pp. 9–10, Figure 10 · p. 9

5.4 Multi-GPU Model Serving & Traffic Control

  • Robion’s traffic adapter achieves 8.2× and 4.4× fewer SLO misses than SMG and EPS, respectively, on average.At a PI0 distribution of (8, 8, 8, 24), SMG has 70% SLO-MR; at (28,16,16,8), EPS has 4.1× more SLO-MR than Robion.Source: Figure 11 · p. 10, §5.4 · p. 10
  • Robion serves 8 VLA variants for up to 64 robots within 98% SLO-AT on one 4-GPU server.Mix placement consistently has lower SLO-AT than the best disjoint-group placement; the best group split varies by robot population.Source: Figure 12 · p. 10, §5.4 · p. 10
  • Robion’s intra-GPU stage disaggregation serves 72 robots within 98% SLO-AT, versus 60 for the best inter-GPU scheme, Dis-3:1.Robion replicates the full model on all four GPUs, while the inter-GPU schemes replicate only one stage across at most three GPUs.Source: §5.4 · pp. 10–11, Figure 13 · p. 11

5.5 Robion in an Enterprise Scenario on Cloud

  • At 24 robots per model, cloud deployment has 23.6% and 18.4% lower SLO-AT than edge on 4×RTX 6000 Pro and 4×H100, respectively.At 98% SLO-AT on edge, 4×H100 serves 24 robots at about 2× the cost of 4×RTX 6000 Pro, which serves 16.Source: Figure 14 · p. 11, §5.5 · p. 11

6 Related Work

Prior VLA work addresses model placement, task-level efficiency, or request scheduling, but not concurrent execution and GPU sharing across requests and models under SLOs; Robion targets that serving gap.

6 Related Work

  • Prior VLA systems address placement, task-level efficiency, or request scheduling, but treat VLA stages as one pipeline and lack concurrent execution and GPU sharing across requests or models.Robion instead operates at the serving layer for concurrent requests across multiple VLA models.Source: §6 · p. 11
  • VLA optimization work covers action-chunking, compression, caching, and fused kernels, while Robion's serving layer can support these optimizations for further serving-level gains.Source: §6 · p. 11
  • Existing multi-stage serving systems target separate-GPU disaggregation and larger models, while VLA stages run at millisecond scale and incur high inter-stage communication costs.Robion uses intra-GPU stage disaggregation for independent stage scheduling.Source: §6 · p. 11
  • DiT serving systems parallelize a single dominant DiT stage, whereas VLA models fit on one GPU and have two stages with distinct compute and memory characteristics.Robion serves VLA stages through intra-GPU disaggregation.Source: §6 · p. 11
  • Kernel-level GPU-sharing schedulers add dispatch overhead comparable to VLA kernels' 8–10 μs runtimes, while Robion co-locates models at stage granularity.Source: §6 · p. 11
  • LLM serving systems are not suited to VLA models, which have small KV caches, fixed-length requests, and fit on one GPU.Source: §6 · p. 12

7 Conclusion

Robion is an SLO-aware serving and management system for multi-robot requests across multiple VLA models on multi-GPU edge servers. It serves higher robot loads than two baselines and supports up to 64 robots in an eight-model, four-GPU experiment.

7 Conclusion

  • 6.7× and 1.5× higher robot load within 98% SLO attainment on average over vLLM-Omni and Monolithic, respectively.The comparison is for individual models.Source: §7 · p. 12
  • Up to 64 robots were served within 98% SLO attainment in a large-scale experiment with 8 models on a 4-GPU server.Source: §7 · p. 12
  • Robion provides SLO-aware serving and management for multi-robot requests across multiple VLA models on multi-GPU local edge servers.Source: §7 · p. 12
Loading 2609.12075v1…