Source-linked AI summary
HexVIO: Towards All-Day Stereo-Inertial Tracking Through Commodity DSPs
Patrick Wolf, Mateo de Mayo, Daniel Cremers
TL;DR
VIO offers a cost-effective, accurate way for devices to localize, but accuracy per watt remains a limitation and stereo-inertial DSP tracking has seen little prior use. HexVIO moves the visual frontend to a Hexagon DSP and reports substantial power and throughput gains over CPU execution, including a 30 fps configuration at 0.83 W.
Problem
VIO has accuracy-per-watt limitations, and few works exploit DSPs for visual-inertial tracking, with none known to the authors for stereo-inertial tracking.
Source: §I · p. 1Method
HexVIO offloads Basalt’s visual frontend to a Hexagon DSP while keeping the backend on the CPU and restructuring the frontend for DSP constraints.
Source: §I · p. 1Results
HexVIO reports 67% lower power or 86% higher throughput than the CPU baseline, and sustains 30 fps tracking at 0.83 W.
Source: §I · pp. 1–2Takeaways & Limitations
At 0.83 W, tracking corresponds to about 18 hours on the 15 Wh testing-device battery, a step toward all-day tracking on commodity devices.
Source: §IV · p. 7
Abstract
from arXiv · showhide
The ability of a device to localize itself within its surroundings is a fundamental prerequisite for spatial computing. Visual-inertial odometry (VIO) has proven to be a cost-effective and accurate solution for this task. Robots, wearables, XR devices, and drones can benefit significantly from efficient implementations of VIO since they allow for cooler, lighter, and cheaper devices with longer battery life and a better user experience. In this work, we propose to enhance the efficiency of a VIO system by leveraging the Hexagon DSP, a commodity co-processor present in many modern smartphones and XR devices. Our approach offloads the visual frontend of a stereo-inertial odometry system to the DSP while keeping the backend on the main CPU. By optimizing the implementation for the DSP architecture, we achieve significant reductions in power consumption and latency compared to CPU-only execution. Our system, HexVIO, demonstrates a 67% reduction in power consumption or an 86% increase in throughput on a commodity smartphone, with the ability to sustain long-term real-time 30 fps tracking for 0.83 W, corresponding to ~18 hours of tracking on the testing device. These results highlight the potential of commodity DSPs for enabling all-day visual-inertial tracking in robotics and mobile devices.
I. INTRODUCTION
HexVIO targets energy-efficient stereo-inertial tracking by moving Basalt’s visual frontend to a widely deployed Hexagon DSP. Its optimized implementation reports lower power or higher throughput than CPU execution, including a real-time 30 fps configuration.
I. INTRODUCTION
- VIO and VI-SLAM use cameras and IMUs to localize devices, supporting applications from drones and robotics to wearables and VR headsets.The paper identifies accuracy per watt as an ongoing limitation in the field.Source: §I · p. 1
- The authors identify limited prior use of DSPs for visual-inertial tracking and, to their knowledge, no prior stereo-inertial example.They position HexVIO as the first stereo-inertial odometry system leveraging the Hexagon DSP.Source: §I · p. 1
- The system offloads Basalt’s visual frontend to a Samsung Galaxy S23 Hexagon DSP while leaving the backend on the CPU.A direct port did not outperform the CPU; the work restructures the frontend around vector execution, memory, and DSP-call costs.Source: §I · p. 1
- 86% higher throughput or 67% lower power than the CPU baseline are the reported HexVIO gains.The comparison is shown in Fig. 1.Source: §I · p. 1, Figure 1 · p. 2
- An energy-focused configuration sustains real-time 30 fps tracking at 0.83 W, corresponding to about 18 hours on the testing device.The 18-hour estimate is for the testing device.Source: §I · p. 2
II. RELATED WORK
Prior VIO work spans estimator designs and hardware acceleration, but trade-offs remain among energy efficiency, accuracy, platform availability, and flexibility. HexVIO instead accelerates a stereo-inertial frontend on a commodity Hexagon DSP while preserving Basalt’s pipeline.
II. RELATED WORK
- LEVIO reaches 20 fps below 100 mW by redesigning the estimator, whereas HexVIO preserves Basalt’s complete pipeline and accelerates its execution.The paper describes LEVIO’s redesign as sacrificing accuracy.Source: §II · p. 2
- GPU frontend accelerators such as vilib, Jetson-SLAM, FastTrack, and GLidE-SLAM use higher-power GPUs, while HexVIO leaves the GPU available for concurrent workloads, –[31].The paper contrasts their thermal constraints and power draw with its DSP approach.Source: §II · p. 2
- Compared with the Hexagon DSP, custom accelerators lack broad availability on commodity platforms; ASICs offer no post-deployment reprogrammability, while FPGA examples accelerate ORB-SLAM components or compute block-matching optical flow –.Hexagon DSPs are programmable, widely deployed, and optimized for energy-efficient SIMD vector processing.Source: §II · p. 3
- HexVIO is presented as the first multicamera VIO pipeline accelerated on a Hexagon DSP in a stock consumer smartphone.It is open-source and runs on unmodified consumer hardware.Source: §II · p. 3
- Earlier Hexagon tracking efforts lacked stereo-inertial support, were closed-source, or required rooted devices.The closest cited effort offloaded OpenVINS frontend routines but ran slower than CPU equivalents,.Source: §II · p. 3
A. Architecture
The Hexagon V73 combines scalar, vector, and matrix units with a shared memory hierarchy, while its vector execution and memory constraints shape how the frontend is implemented.
A. Architecture
- The V73 combines a scalar core, HVX vector unit, and HMX matrix unit; HexVIO uses HVX for wide SIMD on 1024-bit registers.The units share one memory hierarchy, and the DSP has four vector contexts.Source: A. Architecture · p. 3, Figure 2 · p. 3
- Each hardware thread has scalar registers, while each vector context has its own vector registers and six functional units.Peak vector throughput requires all four vector contexts active.Source: Figure 2 · p. 3, A. Architecture · p. 3
- The frontend uses pixel-parallel vectors for image stages and feature-parallel vectors for tracking.For tracking, 32 features advance together through the shared 52-point sampling pattern.Source: Figure 3 · p. 3
- HVX reads from L2 or software-controlled VTCM, and its gather instruction can address only VTCM.Bulk transfers from DDR to VTCM can use DMA.Source: A. Architecture · p. 3, Figure 2 · p. 3
B. Pipeline
HexVIO runs Basalt’s three visual frontend stages on the DSP in one fused call per stereo frame. The pipeline keeps image pyramids in VTCM and uses different vectorization layouts for image processing and feature tracking.
B. Pipeline
- Pyramid construction, FAST-9 detection, and sparse inverse-compositional KLT tracking run on the DSP in one fused FastRPC call per frame.Inertial integration and the estimator remain on the CPU.Source: B. Pipeline · p. 3, Figure 4 · p. 4
- Each call DMA-fetches the stereo pair into VTCM, where current and previous pyramids remain resident for tracking gathers.Other arguments are sparse feature data passed by shared-memory mapping rather than copied.Source: Figure 4 · p. 4, B. Pipeline · p. 4
- Fusing the stages amortizes fixed call costs that would otherwise recur across seven calls per frame.The paper reports a no-work call taking 0.25 ms at top clock under polling, 0.35 ms at the low-voltage corner under polling, and 0.65 ms under driver defaults.Source: B. Pipeline · p. 4
- A coarse occupancy grid links temporal tracking, new left-camera detections, stereo matching, and right-camera detection.Right-camera detection skips cells projecting into the left camera’s view, as in.Source: B. Pipeline · p. 4
- Image stages process adjacent pixels across lanes, while tracking assigns one feature to each lane and gathers its samples.Each tracking batch advances 32 features in lockstep through 52 patch points.Source: B. Pipeline · p. 4
C. Pyramid
The pyramid stage builds four image levels per camera using separable blur and subsampling, with DSP memory layout and row-parallel workers limiting image traffic.
C. Pyramid
- Four-level per-camera pyramids use a separable 5 × 5 binomial blur followed by 2:1 subsampling.The vertical pass writes a 32-bit temporary because its value range can grow by 16×.Source: C. Pyramid · p. 4
- The horizontal pass deinterleaves even and odd columns so filtering runs only at retained subsampled columns, using three one-lane alignments.The five filter terms require no other cross-lane traffic.Source: C. Pyramid · p. 4
- Four workers build the pyramids in VTCM, with aligned rows, disjoint row bands, and one synchronization between levels.Levels 1–3 remain in VTCM; apart from the host copy, level-0 DMA is the only image-sized DDR traffic.Source: C. Pyramid · p. 4
D. Detection
Detection preserves Basalt’s per-cell highest-response corner selection while replacing its threshold passes with a single vectorized sweep and occupancy-based skipping.
D. Detection
- One full-image pass at τ = 5 with non-maximum suppression keeps the per-cell maximum, yielding the same corners as Basalt’s threshold sequence.Basalt runs thresholds ⟨40, 20, 10, 5⟩ from strict to lenient and retains the highest-response candidate at the first threshold yielding one.Source: D. Detection · p. 4
- Each 128-candidate row block tests fixed-offset FAST-9 ring points in parallel, with one vector comparison per ring point.The candidates occupy an 8-bit working image in VTCM.Source: D. Detection · p. 4
- Nine consecutive bright-point masks determine the bright-arc test; occupancy bitsets let workers skip fully occupied rows or vector blocks.Four workers partition cell rows and merge private results when finished.Source: D. Detection · p. 4
E. Tracking
Tracking uses inverse-compositional KLT over fixed feature patches, then reorganizes samples and computation across DSP vector lanes to avoid branch-heavy, scalar feature processing.
E. Tracking
- Patch-based inverse-compositional KLT estimates an SE(2) warp by minimizing mean-normalized SSD over a fixed 52-point pattern.The warp parameters are translation in x and y and rotation θ.Source: E. Tracking · p. 5
- Each feature is pose-and-depth reprojected, iterated coarse-to-fine with precomputed pseudoinverses, then rejected if forward-backward tracking fails.The tracker runs five Gauss-Newton iterations per pyramid level and rebuilds reference patches only for surviving features.Source: E. Tracking · p. 5
- One feature per lane and 32 features per batch turn residuals and pseudoinverses into 52 × 32 matrices for row-wise vector updates.The Hessian inverse uses pivoted LDL⊤ with predicated swaps, avoiding branching across the 32 inversions.Source: E. Tracking · p. 5
- Target pixels are fetched by predicated vector gathers, while gathers for the next pattern point overlap interpolation of the previous point.Two gathers supply each 2 × 2 bilinear neighborhood; backward tracking compacts survivors into dense batches.Source: E. Tracking · p. 5
IV. RESULTS
HexVIO accelerates Basalt’s frontend on the DSP while retaining the CPU backend, and the results show higher throughput alongside lower power and sustained real-time tracking.
Performance
- HexVIO is 1.65–2.38× faster across unpaced sequences; on MOO05 it reaches 180 fps at 4.5 W.The maximum-throughput comparison is reported in Table I; the MOO05 scaling result is reported separately.Source: Table I · p. 6, §IV · pp. 6–7
Measurement caveat
- The on-device fuel gauge resolves power only at 100 ms–1 s, precluding per-stage energy measurements and leaving absolute power figures subject to systematic bias.On MOO12, the two measurement channels differ by 8–12% in absolute power but agree within 2.4% on improvements.Source: §IV · p. 6
Fused dispatch
- Fusing six DSP calls into one saves 2.3 ms per frame under adaptive QoS and 0.7 ms under polling.The adaptive-policy saving is almost entirely RPC round-trip overhead, while on-DSP time falls only slightly.Source: §IV · p. 6, Figure 6 · p. 6
Accuracy
- Across six sequences with ground truth, CPU and DSP ATE differs by at most 0.4% on four, while RTE is identical to two decimals on five.Frontend timing standard deviations shrink by a factor of five to fifteen; on XBIKE they change from 9.84 ± 2.53 to 1.95 ± 0.17 ms.Source: §IV · p. 6
Configurations
- DSPL uses less work rather than a lower clock to reach a lower operating point, while DSPB preserves baseline accuracy at a lower clock.DSPL raises the FAST detection threshold floor from 5 to 20 and uses six keyframes instead of seven.Source: §IV · pp. 5–7
Efficiency
- At matched accuracy, DSPB draws 35% less whole-device power than the CPU baseline on average, with reductions of 28–43% across sequences.Board temperature falls by 2–5 °C on every sequence, and the pinned mid-cluster clock drops by up to 44%.Source: §IV · pp. 6–7
Real-time behavior
- Across eight sequences, the CPU baseline misses 342 deadlines, compared with 21 for DSPB and one for DSPL.Most XWALK late frames follow delivery gaps; these frames average a 0.8 GHz host clock versus 1.9 GHz on schedule.Source: §IV · p. 7
All-day tracking
- In a 40-minute MOO12 run, DSP configurations maintain lower, flatter power and at most 0.05% late frames, versus 0.25% for CPU.The DSP configurations keep SoC die temperature at 34–35 °C, compared with 42 °C for CPU.Source: §IV · p. 7, Figure 7 · p. 7
V. CONCLUSIONS
HexVIO runs a stereo VIO frontend on an unmodified consumer phone’s DSP while preserving CPU-reference accuracy. It meets the frame deadline at 0.83 W, with future work targeting live sensor data and the backend optimizer.
V. CONCLUSIONS
- 0.83 W meets the frame deadline on a 40-minute sequence, corresponding to about 18 hours on a 15 Wh battery while preserving CPU-reference accuracy.The system held a flat mid-cluster clock and a 34–35 °C SoC die temperature, against 42 °C for the CPU.Source: §V · p. 7
- Fusing dispatches into one DSP call per frame lets the frontend retain the powersaving RPC policy and still meet the deadline.Source: §V · p. 7
- Future work includes direct camera streaming to DSP L2, which requires signed vendor integration, and backend optimization, now the largest remaining frame-time target.Hexagon also appears in robotics, XR, and automotive platforms, extending the potential device scope beyond smartphones.Source: §V · p. 7