PaperlayerView exact revision on arXiv

Source-linked AI summary

Efficient Block-Layer Parallel Inference for Vision-Language-Action on Hybrid Architectures

Haibo HU, Lianming Huang, Qiao Li, Nan Guan, Chun Jason Xue

arXiv:2608.14586v1cs.DCcs.AI

TL;DR

Deploying driving VLA models on existing vehicles is constrained by high latency and GPU resource pressure. This paper introduces block-layer CPU–GPU hybrid inference with cross-frame asynchronous execution, reducing latency for Orion and MindDrive while easing deployment constraints.

  • Problem

    Existing heterogeneous-inference methods do not specifically address autonomous-driving VLA workloads under dynamically changing onboard resource demands.

  • Method

    The framework partitions VLA blocks between GPU and CPU and overlaps cross-frame execution to balance computation, communication, and resource use.

  • Results

    Latency falls from 521ms to 408.0ms for Orion and from 443ms to 306.2ms for MindDrive, while driving performance is largely preserved.

  • Takeaways & Limitations

    The framework reduces inference latency and GPU memory pressure while making large driving VLA models more feasible on realistic vehicle platforms.

Abstract

from arXiv · show

Vision-Language-Action (VLA) models are becoming a promising paradigm for autonomous driving, but their deployment on existing vehicle platforms remains difficult because they introduce both high inference latency and strong GPU-side resource pressure. In a full autonomous driving stack, this problem is even more pronounced: legacy vehicle platforms were provisioned for modular pipelines, yet after several planning-related functions are absorbed into a unified VLA model, part of the original CPU budget becomes underutilized, while the visual encoder and the main reasoning path still concentrate most computation and memory demand on the GPU. As a result, directly deploying VLA together with the rest of the onboard system can be hard under realistic GPU memory constraints. To address this issue, we present a hybrid CPU--GPU inference framework with flexible resource scheduling for autonomous driving. Our design partitions the VLA backbone at the block-layer granularity, executes the visual encoder and LLM prefix on the GPU, and offloads the LLM suffix to the CPU through a cross-frame asynchronous pipeline, thereby exposing a schedulable boundary for redistributing compute and memory pressure across heterogeneous processors. We evaluate the proposed framework on two representative driving VLA models, Orion and MindDrive. On Bench2Drive, our method reduces average latency from 521ms to 408.0ms for Orion and from 443ms to 306.2ms for MindDrive, corresponding to 21.7% and 30.9% reduction, respectively. For Orion, the estimated peak GPU memory is further reduced from 45GB to 29GB. In real-vehicle deployment under coexistence with Autoware.Universe, native Orion cannot run because the onboard GPU memory budget is insufficient, whereas the hybrid version runs successfully together with the full vehicle stack.

I. INTRODUCTION · II. RELATED WORK · A. Vision-Language-Action Models for Autonomous Driving

Autonomous driving is shifting from modular pipelines toward integrated VLA systems that combine visual observations, language-level reasoning, and driving actions. The paper addresses VLA deployment challenges through block-layer CPU–GPU collaboration, cross-frame asynchronous execution, and flexible scheduling.

  • I. INTRODUCTION: VLA models unify visual observations, language-level instructions or semantic reasoning, and driving actions for end-to-end embodied intelligence.This shift positions VLA-like driving systems as an important target for next-generation intelligent driving stacks.
  • I. INTRODUCTION: Higher-frequency trajectory updates support smoother, more stable vehicle behavior by continuously refining future actions as the driving scene evolves.VLA inference efficiency is therefore important for practical autonomous driving systems.
  • I. INTRODUCTION: GPU execution can become the main bottleneck while the CPU remains partially idle, motivating CPU–GPU collaborative inference for imbalanced onboard workloads.Other system phases can instead activate CPU-intensive modules while GPU pressure drops.
  • I. INTRODUCTION: Naively offloading the ViT or substantial LLM portions to the CPU can create a new CPU bottleneck, while single-frame VLA inference remains largely serial.These limitations reduce the benefit of direct CPU participation.
  • I. INTRODUCTION: Block-layer partitioning preserves the large model’s original structure while exposing an execution boundary for heterogeneous CPU–GPU deployment.The approach controls how much computation is assigned to each processor.
  • I. INTRODUCTION: The framework offloads a suitable VLA suffix to the CPU, improving CPU utilization and reducing GPU memory pressure under GPU-dominant execution.It exploits the layered structure of large multimodal backbones for collaborative inference.
  • I. INTRODUCTION: Cross-frame asynchronous inference overlaps CPU execution on the current frame with GPU execution on the next frame, converting serialized inference into a heterogeneous pipeline.The design leverages continuous multi-frame streaming to improve sustained inference efficiency and effective planning frequency.
  • A. Vision-Language-Action Models for Autonomous Driving: Recent VLA driving research incorporates language reasoning, data scaling, and unified autoregressive action generation into closed-loop and end-to-end decision making.Examples include LMDrive, DriveLM, CoVLA, and AutoVLA.

B. CPU-GPU Collaborative, Heterogeneous, and CPU-Optimized Inference Architectures · III. PRELIMINARIES · A. Observation 1: Resource Mismatch after Introducing VLA into Autonomous Driving Systems

Prior heterogeneous-inference and CPU-optimization studies improve efficiency but do not specifically address autonomous-driving VLA workloads. Real-vehicle observations show that VLA creates a GPU-heavy execution pattern while leaving CPU capacity underused, motivating CPU-GPU collaboration.

  • B. CPU-GPU Collaborative, Heterogeneous, and CPU-Optimized Inference Architectures: µLayer, CoDL, BAND, and HiDP explore heterogeneous inference through cooperative acceleration, CPU-GPU concurrency, operator partitioning, scheduling, or coordinated multi-DNN execution.These studies target diverse processors across on-device, mobile, and multi-DNN settings.
  • B. CPU-GPU Collaborative, Heterogeneous, and CPU-Optimized Inference Architectures: AMX accelerates matrix-oriented BF16 and INT8 workloads on modern Xeon CPUs, while IPEX exposes AMX, AVX-512, and related optimizations with limited code modification.The direction combines CPU hardware support with software backend optimization.
  • B. CPU-GPU Collaborative, Heterogeneous, and CPU-Optimized Inference Architectures: Prior work mainly targets generic DNN, mobile AI, or distributed edge workloads rather than autonomous-driving VLA inference under changing onboard workloads.VLA-specific constraints include visual encoding, multimodal fusion, and autoregressive decoding.
  • III. PRELIMINARIES: The preliminaries introduce two empirical observations motivating the proposed design.These observations concern resource behavior in practical autonomous-driving systems.
  • A. Observation 1: Resource Mismatch after Introducing VLA into Autonomous Driving Systems: CPU and GPU utilization fluctuate over time in real driving, with peaks that do not always coincide across scenario-dependent workloads.Perception-intensive or emergency-related modules raise GPU usage, while map-related and localization-intensive operations raise CPU usage.
  • A. Observation 1: Resource Mismatch after Introducing VLA into Autonomous Driving Systems: Introducing VLA absorbs functions previously handled by separate modular-stack components, leaving part of the originally provisioned CPU-side computation underutilized.Conventional hardware reserves CPU resources for perception, localization, HD map processing, and planning.
  • A. Observation 1: Resource Mismatch after Introducing VLA into Autonomous Driving Systems: 88% GPU utilization versus 20% CPU utilization was observed during open-loop Orion inference over 500 frames.The measurements come from real-vehicle deployment under the same driving environment.
  • A. Observation 1: Resource Mismatch after Introducing VLA into Autonomous Driving Systems: VLA visual encoding and Transformer-based reasoning remain GPU-concentrated, making the GPU the dominant bottleneck while CPU resources remain underused.The resulting opportunity is to migrate part of model execution onto available CPU resources to improve hardware utilization.

B. Observation 2: Temporal Continuity of Driving VLA Workloads · IV. METHODOLOGY · A. Problem Formulation and Framework Overview

Driving VLA inference processes temporally correlated frame streams continuously rather than isolated queries, creating sustained GPU-centered execution and dynamic resource pressure. The framework therefore formulates resource-aware streaming latency optimization and partitions execution across CPU and GPU using block-layer splitting and asynchronous processing.

  • B. Observation 2: Temporal Continuity of Driving VLA Workloads: Driving VLA workloads process correlated multi-frame observations continuously, unlike generic one-shot multimodal inference over isolated inputs.The workload repeatedly generates reasoning and driving actions for consecutive frames describing an evolving traffic scene.
  • B. Observation 2: Temporal Continuity of Driving VLA Workloads: The default streaming pattern executes each frame’s ViT and full LLM stack sequentially on the GPU while leaving the CPU largely idle.This produces persistent GPU-centered execution over time rather than isolated bursts of computation.
  • B. Observation 2: Temporal Continuity of Driving VLA Workloads: Driving VLA deployment occurs within a multi-module vehicle system with dynamic resource pressure, requiring computation to be distributed across CPU and GPU.The workload should not be treated as a fixed single-device task.
  • A. Problem Formulation and Framework Overview: The problem targets average online streaming inference latency under practical heterogeneous CPU–GPU resource budgets while preserving the VLA model’s original driving capability.The execution policy minimizes end-to-end latency across time steps rather than optimizing one isolated query.
  • A. Problem Formulation and Framework Overview: Block-Layer Partitioning assigns the ViT module and LLM prefix to the GPU and the LLM suffix to the CPU.This partition exploits the layered structure of the VLA backbone to distribute execution across heterogeneous processors.
  • A. Problem Formulation and Framework Overview: The framework uses a cross-frame asynchronous pipeline to leverage continuous multi-frame inference and overlap CPU-side execution across frames.The cited methodology passage introduces this component as part of the framework’s three core designs; the supplied text truncates its detailed operation.
  • A. Problem Formulation and Framework Overview: The framework overview identifies three partition granularities for hybrid VLA inference.Figure 6 presents the available granularities, while the supplied passage does not enumerate them.

B. Block-Layer Partitioning for Hybrid VLA Inference

The framework adopts Transformer block-layer partitioning as the basic unit for hybrid VLA inference, balancing structural integrity with scheduling flexibility. A partition point divides the backbone into a GPU-executed prefix and CPU-executed suffix, exposing a computation–communication tradeoff for resource scheduling.

  • Partitioning Granularity: Intra-block partitioning exposes finer granularity but disrupts Transformer blocks’ sequential dependencies, residual connections, and normalized feature transformations.The passage characterizes Transformer blocks as tightly coupled computation units rather than independent operator collections.
  • Partitioning Granularity: Block-layer partitioning treats each structurally complete Transformer block as an independent scheduling unit while retaining flexibility across the stacked backbone.This granularity preserves the VLA hierarchy and avoids excessive coordination overhead from finer-grained splitting.
  • Execution Boundary: Each partition point p divides the backbone after p block layers into a GPU prefix and CPU suffix, with boundary activations transferred between devices.The CPU suffix consumes the transferred hidden state through the remaining blocks and final action head.
  • Resource Scheduling: Partitioning is presented as the structural basis for adaptive CPU–GPU resource scheduling rather than as a one-time static placement decision.This enables the system to exploit heterogeneous resources under different runtime budgets.

C. Cross-Frame Asynchronous Pipeline on Hybrid Architectures

The section shows that single-frame hybrid execution remains largely serial, limiting offloading benefits, and introduces cross-frame asynchronous pipelining to overlap GPU prefixes with CPU suffixes. This pipeline keeps both processors active while transferring one boundary hidden state per frame, with measured communication overhead far below compute-stage times.

  • Cross-Frame Asynchronous Pipeline: The pipeline runs the current frame’s ViT and LLM prefix on the GPU while the CPU concurrently executes previous frames’ LLM suffixes.After warm-up, different frames occupy heterogeneous stages simultaneously.
  • Motivation: Single-frame offloading remains largely serial because the CPU suffix waits for the same frame’s GPU prefix and intermediate-state handoff.The key opportunity is exploiting VLA inference over a temporally continuous frame stream.
  • Communication: Around 0.41 ms, measured transfer overhead remains small compared with GPU and CPU stages on the order of hundreds of milliseconds.Only one boundary tensor is transferred per frame, avoiding repeated synchronization of multiple intra-block intermediates.
  • Cross-Frame Asynchronous Pipeline: The asynchronous schedule converts intra-frame serial dependence into inter-frame overlap by advancing GPU prefix computation for frame t + 1 while the CPU processes frame t.This organizes inference over the streaming input sequence rather than serializing each frame end to end.
  • Runtime Organization: The runtime uses an asynchronous producer–consumer flow in which the GPU produces boundary states, the transfer interface forwards them, and the CPU consumes them in frame order.Frame identities are preserved across the handoff while generating final driving outputs.

D. Flexible Resource Scheduling and Real-World Deployment

The section introduces boundary-adaptive resource scheduling for dynamic driving conditions, allowing limited online layer migration while preserving a stable hybrid pipeline. It also evaluates coexistence with Autoware.Universe on a heterogeneous real-vehicle platform.

  • Flexible Resource Scheduling: Dynamic driving conditions and shared onboard resources make a fully fixed VLA partition insufficient, because a policy optimal in one runtime condition may become suboptimal in another.The scheduler addresses changing hardware load and competition with other autonomous-driving modules.
  • Flexible Resource Scheduling: The backbone uses GPU-fixed, CPU-fixed, and boundary-centered dynamic zones, with only boundary-adjacent layers eligible for migration across frames.Fixed zones preserve pipeline stability and the dynamic zone avoids large-scale layer movement while enabling runtime adaptation.
  • Flexible Resource Scheduling: The scheduler selects among candidate layer-allocation policies using current GPU utilization, CPU utilization, and pipeline backlog or queue status.A lightweight controller maps runtime signals to policies that trade VLA inference speed against compute capacity reserved for other onboard modules.
  • Real-World Deployment: Real-vehicle experiments run Autoware.Universe and a separate VLA planner concurrently on an external heterogeneous unit with an Intel Xeon CPU and NVIDIA L20 GPU.Autoware.Universe supplies representative perception, localization, control, and system-management modules while its original planning module is disabled.

E. Hardware-Aware CPU Acceleration

Section E explains that CPU-side acceleration is necessary to prevent the CPU-executed VLA suffix from becoming the bottleneck in hybrid inference. It reduces suffix execution time using Intel AMX and IPEX, which target matrix-heavy LLM operations through hardware-specialized computation and optimized CPU kernels.

  • Motivation: CPU-side acceleration is necessary because assigning VLA suffix blocks to the CPU can otherwise create a new end-to-end bottleneck.The cross-frame asynchronous pipeline enables CPU participation but does not guarantee acceleration unless the CPU suffix executes fast enough.
  • Optimization objective: The optimization goal is to reduce CPU suffix execution time so the CPU stage better matches the GPU stage in the pipeline.CPU-stage latency includes both transfer cost and execution time for the CPU-assigned suffix blocks.
  • Implementation: Intel AMX and IPEX form a lightweight hardware-aware optimization stack for CPU-executed suffix LLM layers.AMX improves dense tensor computation, while IPEX supplies optimized CPU kernels and reduces PyTorch execution-path overhead.
  • Implementation: The stack targets matrix-heavy operations, including linear projections and feed-forward layers, with architecture-specialized acceleration.These operations dominate the CPU-executed suffix LLM layers.

V. EXPERIMENT · A. Experimental Setup

The study evaluates a block-layer hybrid VLA inference framework for real-time autonomous driving on Bench2Drive using Orion and MindDrive2. Experiments preserve the original model architectures while comparing native and hybrid execution on a heterogeneous CPU–GPU platform under continuous streaming workloads.

  • A. Experimental Setup: Bench2Drive provides a realistic urban-scene benchmark for assessing practical end-to-end VLA driving performance and real-time heterogeneous inference.The benchmark is designed to evaluate autonomous driving across diverse urban scenarios.
  • A. Experimental Setup: Experiments use two representative driving VLA models, Orion and MindDrive2, whose differing architectures and computational characteristics test framework generality.The comparison examines whether block-layer hybrid execution consistently improves efficiency across distinct VLA backbones.
  • A. Experimental Setup: The hardware platform combines one NVIDIA L20 GPU with an Intel(R) Xeon(R) Platinum 8470Q CPU.The CPU supports Intel AMX and IPEX for efficient execution of CPU-resident VLA blocks.
  • A. Experimental Setup: The implementation preserves the original Orion and MindDrive2 architectures and output heads for fair comparison while partitioning their backbones at block-layer granularity.Different stages execute on heterogeneous devices through an asynchronous streaming pipeline.
  • A. Experimental Setup: CPU-resident blocks use a hardware-aware optimization stack to reduce CPU-side latency during hybrid execution.The optimization is applied within the asynchronous heterogeneous execution pipeline.
  • A. Experimental Setup: Evaluation uses continuously arriving input frames and sequential online outputs to measure single-frame latency, steady-state inference frequency, and device utilization.For each model, original and hybrid execution modes are compared under the same benchmark setting.

B. Evaluation Metrics · C. Main Results

The evaluation measures system efficiency, driving effectiveness, and stage-level behavior. Results show that block-layer hybrid inference substantially lowers VLA latency while largely preserving driving performance and enabling practical CPU participation.

  • B. Evaluation Metrics: System efficiency is evaluated using average latency, inference frequency, GPU/CPU memory usage, and GPU/CPU utilization.These metrics assess efficiency, hardware utilization, and resource balance on the heterogeneous platform.
  • B. Evaluation Metrics: Driving effectiveness is assessed with L2 trajectory error and Coll. collision behavior during closed-loop execution.These metrics verify runtime gains without sacrificing trajectory quality or driving safety.
  • B. Evaluation Metrics: Stage-level profiling measures GPU-stage latency, CPU-stage latency, inter-device transfer overhead, and pipeline overlap behavior.The analysis attributes efficiency gains to block-layer partitioning, asynchronous execution, and hardware-aware CPU optimization.
  • C. Main Results: Compared with conventional non-VLA baselines, ORION and MindDrive achieve substantially better trajectory accuracy and safety through lower L2 and collision metrics.Table I compares conventional driving baselines, original VLA models, and hybrid execution variants on the heterogeneous platform.
  • C. Main Results: 21.7%: ORION average latency decreases from 521 ms to 408.0 ms, while 30.9%: MindDrive latency decreases from 443 ms to 306.2 ms.The framework shortens the critical inference path without modifying the original model architecture or output head.
  • C. Main Results: ORION average L2 changes from 0.68 to 0.69 and average collision from 0.37 to 0.38 after hybrid execution.The reported changes are small relative to the latency improvement.
  • C. Main Results: MindDrive average L2 changes from 0.76 to 0.77 and average collision from 0.42 to 0.43 after hybrid execution.These results indicate that the original VLA driving effectiveness is well preserved.
  • C. Main Results: The optimized CPU stage becomes practically usable, actively cooperating with the GPU to process model blocks in the hybrid variants.This validates hardware-aware CPU optimization as a way to involve CPU-side execution without becoming the dominant bottleneck.

D. Resource Scheduling in Real-Vehicle Deployment · E. Impact of Block-Layer Partition · F. Impact of Cross-Frame Asynchronous Pipeline

The hybrid framework enables resource-feasible real-vehicle deployment by shifting workload from GPU to CPU, while block-layer partitioning and cross-frame asynchronous execution jointly improve inference efficiency. Experiments identify k=16 as the best Orion split and show that asynchronous overlap is essential for converting partitioning into practical speedup.

  • D. Resource Scheduling in Real-Vehicle Deployment: Native Orion fails with Autoware.Universe because its 45 GB GPU requirement exceeds the 40.2 GB available after 7.8 GB is occupied.The vehicle platform has 48 GB total GPU memory, leaving insufficient capacity for native Orion.
  • D. Resource Scheduling in Real-Vehicle Deployment: Hybrid Orion runs successfully alongside the full Autoware.Universe stack despite native Orion’s GPU memory overflow.This demonstrates deployment feasibility under realistic resource contention.
  • D. Resource Scheduling in Real-Vehicle Deployment: 546 ms to 390 ms latency and 1.83 Hz to 2.56 Hz Inference frequency improvements for MindDrive accompany GPU memory reduction from 21 GB to 14 GB.CPU memory increases from 24 GB to 33 GB, indicating workload reallocation to CPU resources.
  • E. Impact of Block-Layer Partition: At k=16, Orion achieves the best partitioning trade-off, with GPU latency of 408 ms, CPU latency of 375 ms, and average latency of 408 ms.This split also yields the highest inference frequency of 2.45 FPS.
  • F. Impact of Cross-Frame Asynchronous Pipeline: Without asynchronous overlap, hybrid Orion reaches 783 ms single-frame and steady-state latency, while inference frequency drops to 1.28 FPS.Strictly sequencing the CPU suffix after the same frame’s GPU prefix leaves execution largely serialized.
  • F. Impact of Cross-Frame Asynchronous Pipeline: With cross-frame asynchronous execution, Orion’s steady-state latency falls to 404 ms and inference frequency rises to 2.48 FPS, despite 760 ms single-frame latency.Compared with hybrid execution without async, this is a 48.4% steady-state latency reduction and a 93.8% inference-frequency increase.
  • F. Impact of Cross-Frame Asynchronous Pipeline: Asynchronous execution improves performance without increasing memory footprint: both hybrid configurations use 29 GB GPU memory and 22 GB CPU memory.The gain comes from overlapping current-frame CPU execution with subsequent-frame GPU execution rather than additional allocation.

G. Robustness under Constrained CPU Budget · H. Memory Footprint Analysis

Under constrained CPU budgets, the hybrid framework remains stable through moderate reductions and degrades when CPU threads become insufficient for the CPU-assigned suffix. Its heterogeneous partitioning also reduces GPU memory pressure for Orion and MindDrive by shifting workload from VRAM to DRAM.

  • G. Robustness under Constrained CPU Budget: The evaluation emulates CPU budgets from 100% of 102 threads down to smaller fractions, including 10% with only 10 enabled threads.This setup models deployment scenarios with limited CPU resources.
  • G. Robustness under Constrained CPU Budget: From 100% down to about 40%, Orion and MindDrive latency changes only slightly while inference frequency remains nearly flat.The hybrid design therefore does not require full CPU capacity for efficient execution.
  • G. Robustness under Constrained CPU Budget: Below 30% CPU budget, Orion latency rises rapidly and inference frequency decreases, while MindDrive shows the same but less severe degradation.The CPU-assigned suffix blocks cannot keep pace with the GPU stage, making the pipeline CPU-bounded.
  • G. Robustness under Constrained CPU Budget: MindDrive maintains relatively stable inference frequency across a wider CPU-budget range, whereas Orion is more sensitive to aggressive CPU restriction.The difference reflects their different CPU-side workloads after partitioning.
  • G. Robustness under Constrained CPU Budget: The framework exploits abundant CPU resources while degrading gracefully when only a limited fraction of CPU threads is available for VLA inference.These results demonstrate flexible resource scheduling under varying CPU availability.
  • H. Memory Footprint Analysis: For ORION, hybrid execution reduces estimated peak VRAM usage from around 45 GB to around 29 GB by dividing 32 LLM layers evenly between GPU and CPU, while CPU DRAM rises to around 22 GB.GPU-resident layers decrease from 32 to 16.
  • H. Memory Footprint Analysis: For MindDrive, hybrid execution lowers estimated peak VRAM usage from around 13 GB to around 6 GB while increasing CPU DRAM usage to around 10 GB.All LLM layers move to the CPU side from native deployment’s GPU placement.
  • H. Memory Footprint Analysis: Hybrid execution redistributes model blocks across heterogeneous devices, shifting part of VLA deployment’s memory burden from VRAM to DRAM when GPU memory is constrained.This complements latency improvement in systems running concurrent perception, planning, and multi-sensor workloads.

I. Impact of Hardware-Aware CPU Optimization · VI. CONCLUSION

The full CPU optimization stack reduces CPU-side latency from an impractical fallback cost to 718 ms, enabling effective hybrid execution. The framework combines block-layer partitioning and cross-frame overlap to reduce latency and GPU pressure for VLA deployment on heterogeneous vehicle platforms.

  • I. Impact of Hardware-Aware CPU Optimization: 11,008 ms CPU-side latency in the PyTorch eager baseline falls to 4,530 ms with IPEX and AVX-512 fallback, a 2.43× speedup.Memory footprint also decreases from 28.5 GB to 24.1 GB.
  • I. Impact of Hardware-Aware CPU Optimization: 2,184 ms latency with AMX achieves a 5.04× speedup, demonstrating the importance of architecture-specific matrix acceleration.This improvement follows IPEX with AVX-512 fallback.
  • I. Impact of Hardware-Aware CPU Optimization: 718 ms latency after NUMA-aware scheduling and thread binding corresponds to a 15.3× speedup over the eager baseline.torch.compile and CPU KV-cache optimization reduce latency to 2,015 ms and 1,250 ms before the final runtime optimizations.
  • I. Impact of Hardware-Aware CPU Optimization: The full optimization stack transforms the CPU from an impractically slow fallback device into an effective execution component of the heterogeneous pipeline.CPU participation is beneficial only when CPU-resident VLA blocks execute efficiently.
  • VI. CONCLUSION: The framework restructures monolithic VLA execution into a heterogeneous pipeline for autonomous driving by addressing GPU-dominant resource mismatch and streaming workload continuity.Its design is based on block-layer partitioning and cross-frame execution across CPU–GPU architectures.
  • VI. CONCLUSION: Bench2Drive experiments with Orion and MindDrive consistently reduce inference latency, improve inference frequency, and relieve GPU memory pressure while largely preserving original driving performance.Further analysis identifies balanced block-layer splitting, cross-frame overlap, and CPU-side optimization as necessary for practical hybrid acceleration.
  • VI. CONCLUSION: System-level hybrid execution is presented as a practical path toward real-time deployment of large VLA models under realistic autonomous-driving vehicle resource limits.The paper proposes heterogeneous large-model inference as a direction for further research.
Loading 2608.14586v1…