PaperlayerView exact revision on arXiv

Source-linked AI summary

VkVIO: Cross-platform GPU Acceleration for Visual-Inertial Odometry with Vulkan

Ole Hoffmann, Mateo de Mayo, Daniel Cremers

arXiv:2609.30459v1cs.ROcs.CV

TL;DR

GPU acceleration can support efficient pose estimation, and VkVIO addresses cross-platform deployment with a Vulkan-based VIO frontend. Across Nvidia, Apple, and Radxa devices, it reports speedups over Basalt’s CPU implementation and also outperforms CUDA-only frontends on Nvidia hardware.

  • Problem

    VkVIO addresses the limited cross-platform availability of GPU-accelerated VIO by presenting a cross-platform GPU-accelerated frontend.

    Source: §I · p. 2

  • Method

    VkVIO reformulates Basalt’s CPU frontend as Vulkan compute shaders, including subgroup and shared-memory fallback variants.

    Source: §I · p. 2

  • Results

    Across three devices, VkVIO reports frontend speedups of up to 11× over the CPU baseline without accuracy loss, and surpasses CUDA-only frontends on the same Nvidia hardware.

    Source: §I · p. 2

  • Takeaways & Limitations

    VkVIO runs on an extremely inexpensive, low-power SBC that CUDA baselines cannot run on, while allowing deployment across GPU vendors without rewriting the frontend.

    Source: §V · p. 8

Abstract

from arXiv · show

Perception in robotics and XR fundamentally relies on good state estimation. Visual-inertial odometry (VIO) and Simultaneous Localization and Mapping (VI-SLAM) are proven ways of achieving this goal in a cost-effective and accurate manner. Efficiency in these systems allows for smaller, cooler, and lighter devices. GPU acceleration is a natural approach for reducing latency, thanks to their wide availability in platforms like embedded computers, mobile phones, and XR headsets. However, previous works in the literature have limited themselves to the use of CUDA for this task, significantly reducing deployment options to a single vendor. We instead leverage the vendor-agnostic Vulkan API, originally designed for the strict performance requirements of 3D graphics applications. In this work, we present VkVIO, the first, to the best of our knowledge, cross-platform GPU-accelerated VIO method. We provide state-of-the-art accuracy with causal estimates required for real-time operation. We deploy VkVIO on a diverse range of devices spanning a workstation, a laptop, and an extremely inexpensive single-board computer, while outperforming CUDA-based systems on the same hardware. VkVIO enables possibilities for low-latency, low-power, and low-cost VIO in robotics and XR.

I. INTRODUCTION

VIO and VI-SLAM offer cost-effective, accurate pose estimation, but CUDA-based GPU acceleration limits deployment across vendors and can favor higher-cost, higher-power devices. VkVIO uses Vulkan to bring GPU-accelerated VIO to diverse platforms, with reported speed and efficiency gains.

Motivation

  • The authors motivate GPU acceleration as a way to reduce latency, power consumption, and heat while improving battery life, weight, and device size.They identify the Radxa Cubie A7Z as at least 20× cheaper and lower-power than the closest CUDA-capable Jetson alternative.Source: §I · p. 1
  • CUDA-based GPU approaches exclude other vendors, limiting deployment on many phones, single-board computers, and laptops without CUDA GPUs.CUDA-capable devices also tend toward higher power envelopes and price ranges than alternatives.Source: §I · p. 1

Approach

  • Vulkan is proposed as a cross-platform graphics and compute API with support across GPU vendors and embedded devices, phones, and laptops.The authors cite its performance focus and continued relevance on Nvidia hardware as reasons for choosing it over OpenCL or OpenVX.Source: §I · p. 1

Contributions

  • VkVIO is presented as the first, to the authors’ knowledge, cross-platform GPU-accelerated VIO frontend, built on Vulkan/SPIR-V and demonstrated across Nvidia, Apple, and ARM-class GPUs.The implementation reformulates Basalt’s CPU frontend as Vulkan compute shaders, with subgroup-based and shared-memory fallback variants.Source: §I · p. 2

Evaluation

  • VkVIO reports frontend speedups of up to 11× over the CPU baseline across three devices, with more stable per-frame latency, lower power and thermal load, and no accuracy loss.The evaluated devices are an Nvidia RTX 3070 workstation, Apple M3 laptop, and Radxa Cubie A7Z SBC.Source: §I · p. 2
  • On the same Nvidia hardware, VkVIO’s vendor-agnostic approach surpasses the CUDA-only Jetson-SLAM and FastTrack frontends.The figure also compares progress on EuRoC MH02 after equal wall-clock time and end-to-end speedups over Basalt across three devices and six sequences.Source: Figure 1 · p. 1, §I · p. 2

II. RELATED WORK

Prior work accelerates VIO and SLAM frontends on GPUs, but many approaches depend on CUDA and Nvidia hardware. VkVIO applies Vulkan compute to Basalt’s frontend for cross-platform stereo VIO.

VIO and VI-SLAM pipeline

  • VIO and VI-SLAM pipelines pair a visual frontend that converts images into keypoints with a backend that fuses them with inertial data to estimate position.The paper builds on Basalt, an optimization-based system, with modifications from.Source: §II · p. 2

GPU acceleration for visual frontends

  • The visual frontend processes every incoming frame and its workload grows with image resolution, camera count, and tracked-feature count, making it a GPU-acceleration target.Its data-parallel operations apply the same instructions across pixels or features.Source: §II · p. 2

CUDA-based methods

  • Most existing GPU-accelerated frontend methods rely on CUDA, tying deployment to Nvidia hardware and limiting use across GPU vendors.Prior systems include GPU feature-extraction pipelines and CUDA-based detection, tracking, stereo matching, and local-map tracking.Source: §II · p. 2

Cross-platform alternatives

  • GLidE-SLAM uses OpenGL compute for direct photometric tracking on intermediate frames, but is monocular and not directly comparable to stereo VkVIO.The authors argue Vulkan is a better choice given its growing importance on modern platforms and continued vendor investment.Source: §II · p. 2

VkVIO

  • VkVIO introduces a Vulkan GPU implementation of Basalt’s frontend and, to the authors’ knowledge, is the first use of Vulkan for cross-platform GPU acceleration of stereo VIO.Other Vulkan projects demonstrate cross-vendor high-throughput pipelines, but do not address VIO directly.Source: §II · p. 2

III. METHOD

VkVIO uses Vulkan’s explicit compute model to dispatch shader work across GPU workgroups and subgroups. Its shader representation is compiled ahead of time and translated by the target driver into native instructions.

A. Architecture

  • Vulkan compute work is recorded as dispatch commands in command buffers and submitted to queues, with each dispatch launching a specified number of workgroups.Workgroups contain shader invocations that execute the bound compute pipeline.Source: A. Architecture · p. 2
  • Within workgroups, hardware issues subgroups together on a SIMD unit, and subgroup operations let members exchange values without shared memory and barriers.Subgroup size is a device property.Source: A. Architecture · pp. 2–3
  • VkVIO shaders are written in GLSL and compiled ahead of time to hardware-independent SPIR-V, which the driver translates into native instructions when creating the pipeline.Source: A. Architecture · p. 3

B. Pipeline

VkVIO implements the Basalt frontend as four Vulkan compute-shader stages submitted once per frame. Its image pyramid uses separable binomial filtering and device-configured GPU tiles, while feature detection maps image cells onto workgroups.

B. Pipeline

  • The four frontend stages each use a Vulkan compute shader, run from one command buffer per frame, and reuse resources allocated at initialization.Pipeline barriers separate stages; after submission, the host waits for GPU completion before reading results.Source: B. Pipeline · p. 3

C. Pyramid

  • Each pyramid level applies a separable 5 × 5 binomial blur followed by 2:1 subsampling, with horizontal and vertical passes.The horizontal pass produces an intermediate image used by the vertical pass.Source: C. Pyramid · p. 3
  • One invocation computes each output pixel; workgroups process tiles sized for the device’s subgroup, and successive pyramid levels run in sequence.On the reported hardware, S = 32 and R = 8; all levels for one camera are stored in a two-dimensional texture atlas.Source: C. Pyramid · p. 3
  • Feature detection assigns each 24×24 cell to a workgroup, which runs FAST-9 testing and 3 × 3 NMS before sorting its strongest survivors.Each pixel maps to an invocation, and the selected keypoints are appended to DETt.Source: Figure 3 · p. 3

D. Detection

Detection partitions images into cells with per-cell feature limits and evaluates FAST-9 corners. Threshold retries, non-maximum suppression, and GPU workgroup processing select keypoints, with a reduced-resource variant for embedded GPUs.

D. Detection

  • FAST-9 marks a pixel as a corner when at least nine contiguous points on a radius-three circle are all brighter or all darker than the center.The cell-based selection strategy encourages spatially uniform keypoint distribution.Source: D. Detection · p. 3
  • Cells test thresholds 40, 20, 10, and 5 in sequence, applying NMS and selecting up to k strongest survivors before retrying at a lower threshold.The evaluation uses k = 1, Basalt’s default setting.Source: D. Detection · p. 3
  • On the GPU, each cell maps to a workgroup that evaluates FAST responses, retains strict 3×3 local maxima, and selects up to k highest-scoring keypoints.The workgroup loads its image tile into shared memory; a bitonic k-sort selects the survivors.Source: D. Detection · p. 3
  • For embedded GPUs with limited execution resources, a single-keypoint variant replaces bitonic sorting with an atomic maximum reduction over NMS survivors.It selects the highest-scoring candidate per cell without the sorting network’s repeated compare-exchange operations and barriers.Source: D. Detection · p. 4

E. Tracking

VkVIO tracks and matches keypoints with patch-based inverse-compositional KLT using planar rigid warps and photometric residuals. Forward and backward passes refine each feature across pyramid levels, while the GPU distributes patch work across invocations.

E. Tracking

  • Patch-based inverse-compositional KLT models feature displacement as a planar rigid transform and minimizes photometric error over 52 sample locations.Each patch is divided by its mean intensity to remove global brightness offset between frames.Source: E. Tracking · p. 4
  • The inverse-compositional Jacobian is evaluated on the source image and remains fixed across iterations; its Hessian and gain are computed once.The Jacobian rows use source-image gradients, while the Hessian is formed as JᵀJ.Source: E. Tracking · p. 4
  • Each tracking call builds a source-image template, initializes the warp from pose and inverse depth, and refines it coarse-to-fine with five Gauss–Newton iterations per level.A backward pass exchanges the image roles, and the feature is discarded if the returned position exceeds a fixed distance threshold from its original position.Source: E. Tracking · p. 4
  • One workgroup of 32 invocations handles each keypoint, distributing the 52-point sums across invocations and combining them with subgroup reductions.Devices without subgroup arithmetic use shared-memory accumulation instead, at the cost of additional barriers.Source: E. Tracking · p. 4

IV. RESULTS

The evaluation prioritizes device diversity and input-resolution scaling over sequence count, drawing on visual-inertial datasets for robotics and XR. It selects representative sequences across devices and reports trajectory-error and per-frame timing metrics.

IV. RESULTS

  • Dataset selection prioritizes a broad range of devices over sequence count to demonstrate VkVIO’s versatility and performance scaling with input resolution.The datasets are drawn from the SLAM literature and center on real-time robotics and XR.Source: A. Dataset Selection · p. 4
  • The selected datasets cover indoor micro-aerial-vehicle imagery, handheld imagery, VR headsets with differing resolutions and camera counts, and opposite-facing wide-FoV cameras.The sources are EuRoC, TUM-VI, Monado SLAM, and Hilti Challenge 2026.Source: A. Dataset Selection · p. 4
  • Table I is framed as maintaining accuracy from the Basalt baseline.The supplied caption gives no metric values or further comparison details.Source: Table I · p. 5
  • Reported measures include SE(3)-aligned absolute and relative trajectory error and mean ± standard deviation of frontend and end-to-end time per frame.The trajectory errors use a six-frame delta.Source: A. Dataset Selection · p. 5
  • The evaluation uses one representative sequence per capture device: EMH02, TR2, MOO02, MGO02, MIO02, and HF151.The passage states that per-frame timing metrics vary little across sequences from the same device.Source: A. Dataset Selection · pp. 4–5

B. Improvements over CPU-Only Baseline

VkVIO speeds up Basalt’s frontend across devices while preserving accuracy, and its timing is more stable. Long-run operation on the M3 also uses less energy per frame.

B. Improvements over CPU-Only Baseline

C. Improvements over CUDA-Accelerated Systems

On the evaluated Nvidia hardware, VkVIO outperforms the CUDA-based frontends in end-to-end tracking time while retaining Basalt-level accuracy. The comparison excludes sequences the CUDA systems cannot complete or support.

C. Improvements over CUDA-Accelerated Systems

  • The CUDA systems do not support four-camera MGO02 and neither completes HF151; comparisons use online poses and tracking-path end-to-end times.GLidE-SLAM is excluded because it is monocular-only and no public implementation was available at the time.Source: C. Improvements over CUDA-Accelerated… · pp. 6–7
  • VkVIO has the highest frame rate on every sequence, with end-to-end time 1.6–3.1× faster than FastTrack and 1.7–3.9× lower than Jetson-SLAM’s.Table III compares the systems on one desktop with an Intel Core i5-13600 CPU and Nvidia RTX 3070 GPU.Source: C. Improvements over CUDA-Accelerated… · p. 6, Table III · p. 7
  • Against CUDA baselines, VkVIO has lower ATE on three of five completed sequences and lower RTE on all five; each CUDA system has lower ATE on one sequence.FastTrack has lower ATE on TR2, while Jetson-SLAM has lower ATE on MIO02.Source: C. Improvements over CUDA-Accelerated… · pp. 6–7

D. Deployment on a Low-Cost and Low-Power Platform

VkVIO-lite brings three Radxa sequences above camera rate on average, while other sequences remain below rate and some accuracy trade-offs persist. On the bare board, VkVIO also runs cooler than Basalt during the long sequence.

D. Deployment on a Low-Cost and Low-Power Platform

  • On the bare Radxa without cooling, Basalt exceeds 80°C within 80 s and holds at 84°C, while VkVIO reaches 80°C after 13 minutes and is cooler thereafter.VkVIO is cooler by 5°C on average after reaching 80°C and by 2°C at the end.Source: D. Deployment on a Low-Cost and… · p. 7
  • VkVIO-lite runs TR2, MOO02, and EMH02 above camera rate on average, with p99 frame times 2.6–4.6× shorter than Basalt’s; MGO02, MIO02, and HF151 remain below rate.Lite raises the minimum FAST threshold from 5 to 20, increases cell size from 50 to 82, and reduces Gauss-Newton iterations and the optimization window.Source: D. Deployment on a Low-Cost and… · p. 7, Table IV · p. 7
  • Lite keeps ATE within 15 mm of the standard configuration on the three real-time sequences, but increases ATE by 0.5 m on MIO02 and 0.8 m on HF151.RTE increases noticeably only on MIO02, by 21 mm.Source: D. Deployment on a Low-Cost and… · p. 7

V. CONCLUSIONS

VkVIO uses Vulkan for cross-platform GPU acceleration and improves Basalt’s runtime across Apple, Nvidia, and Radxa GPUs while preserving tracking accuracy. Its results also support deployment on inexpensive, low-power hardware.

V. CONCLUSIONS

  • Across Apple, Nvidia, and Radxa GPUs, VkVIO delivers frontend and end-to-end speedups over Basalt while preserving tracking accuracy and improving frame-time stability.The conclusion also reports better power efficiency and thermals over long operation.Source: §V · pp. 7–8
  • VkVIO beats the CUDA-only Jetson-SLAM and FastTrack frontends on the same Nvidia hardware despite using a vendor-agnostic API.The conclusion cites Table III and Fig. 9 for this comparison.Source: §V · p. 8
  • VkVIO makes real-time VIO viable on a low-power SBC about 20× cheaper than the closest CUDA-capable device Nvidia maintains.The paper presents cross-vendor deployment without rewriting the frontend as a practical flexibility.Source: §V · p. 8
  • The authors hope the work encourages wider adoption of Vulkan as a vendor-neutral compute target for real-time robotics and XR perception.Source: §V · p. 8

Related papers

Loading 2609.30459v1…