PaperlayerView exact revision on arXiv

Source-linked AI summary

SHIFT: Surface-aware High-speed Integration For TSDFs

Ayaan Choudhury, Lokender Tiwari

arXiv:2609.06547v1cs.RO

TL;DR

Conventional integrators repeatedly fuse depth pixels after corresponding voxels have converged, making planar scenes a source of redundant computation. SHIFT classifies depth geometry, bundles flat regions into weighted super-rays, and skips selected flat-voxel updates. Across RGB-D and LiDAR sequences, it reduces TSDF cost while maintaining or improving reported accuracy and lowering ESDF-layer memory.

  • Problem

    Conventional integrators re-fuse depth pixels every frame even after voxel distances and gradients have converged, wasting computation on planar regions.

    Source: §1 · p. 1

  • Method

    SHIFT classifies pixels from 3D depth geometry, bundles flat local regions into weighted super-rays, freezes converged flat-voxel gradients, and uses a compact ESDF voxel layout.

    Source: §1 · pp. 1–2

  • Results

    SHIFT cuts TSDF integration cost by 1.42 to 4.07×, holds RGB-D mesh error within 0.19 cm, improves real-LiDAR TSDF RMSE, and reduces ESDF-layer memory by up to 28%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    TSDF savings are largest on the most planar scene and smallest where planar structure is scarce.

    Source: §4.1 · p. 5

    The fixed per-frame front-end cost narrows gains on less planar or coarse-voxel scenes.

    Source: §5 · p. 8

Abstract

from arXiv · show

Real-time 3D mapping is fundamental for autonomous robotic navigation, with Euclidean Signed Distance Fields (ESDFs) serving as the standard representation for online motion planning. While recent advancements in non- projective distance fields yield highly accurate maps, their computational overhead remains a severe bottleneck. Conventional integrators redundantly re-fuse millions of depth pixels every frame, even long after the corresponding voxels have converged, wasting significant computational resources in environments dominated by large planar surfaces. In this paper, we present SHIFT (Surface-aware High-speed Integration For TSDFs), an efficient mapping framework designed to reduce this per-frame update cost. By exploiting structural redundancy directly from 3D depth geometry, SHIFT compresses flat local regions into weighted super-rays and freezes flat-voxel gradients. A compact ESDF voxel layout further reduces the memory footprint of the remaining wavefront. Extensive evaluations across various RGB-D and LiDAR sequences show that SHIFT cuts TSDF cost by 1.42 to 4.07 times, while holding mesh error within millimeters, and reduces ESDF-layer memory by up to 28%

1. Introduction

Online robotic mapping needs accurate, dense maps while operating under tight computational budgets. SHIFT reduces repeated work on planar surfaces by adapting TSDF integration to surface geometry.

1. Introduction

  • ESDFs support constant-time obstacle-distance and available free-space-gradient queries, while onboard mapping must fuse incoming depth frames at sensor rate.Explicit voxel-grid pipelines remain a practical foundation for onboard, hard-real-time mapping; continuous-field training or inference costs make sustained MAV operation difficult.Source: §1 · p. 1
  • Conventional integrators repeatedly re-project, re-weigh, and re-fuse pixels even after voxel distances and gradients have converged, wasting work on planar regions.The TSDF update, gradient re-estimation, and downstream bookkeeping are repeated although the result does not change.Source: §1 · p. 1
  • SHIFT reduces TSDF integration cost by 1.42 to 4.07× over its baseline, keeps RGB-D mesh error within 0.19 cm, improves real-LiDAR TSDF RMSE, and cuts ESDF-layer memory by up to 28%.Source: §1 · p. 2
  • SHIFT classifies organized depth-frame pixels from 3D geometry and compresses flat pixels in local tiles into weighted super-rays.Its lightweight classifier uses neither RGB nor learned semantics and supports RGB-D and spinning-LiDAR inputs.Source: §1 · p. 2
  • After a flat voxel's accumulated weight crosses a threshold, SHIFT freezes its gradient and skips averaging when a new distance agrees within a tolerance.Disagreeing measurements still update the stored distance, and pixels failing planarity tests are fused as in Voxfield.Source: §1 · p. 2

2. Related work

Prior work spans explicit voxel-grid fusion, continuous distance fields, pipeline acceleration, and plane-aware geometry processing. SHIFT targets redundant fusion work left by these approaches.

2.1. Explicit voxel-grid TSDF/ESDF mapping

  • Voxblox, FIESTA, and Voxfield develop explicit TSDF/ESDF mapping, with Voxfield replacing projective distance approximation with a gradient-driven non-projective correction.Some integrators merge rays ending in the same voxel, but per-pixel projection and weighting still run in full.Source: §2.1 · p. 2

2.2. Continuous and learned distance fields

  • Continuous neural, point-based, hierarchical, and Gaussian-process fields can complete sparse observations, but training or inference overhead challenges sustained embedded operation.The cited approaches include neural implicit maps [8] [14] [26] [27], point-based implicit maps [16] [17] [21], HIO-SDF [22], and Gaussian-process fields [1] [24] [25].Source: §2.2 · p. 2

2.3. Accelerating volumetric mapping

  • GPU-accelerated TSDF/ESDF fusion and a unified linear parametric map are alternatives for reducing explicit-pipeline or dense-grid costs.Source: §2.3 · p. 2

2.4. Plane extraction and structure-aware mapping

  • Organized-cloud plane extraction provides a precedent for SHIFT's depth-geometry classifier, while AiSDF uses structural cues to improve completion rather than reduce per-frame cost.PEAC extracts planar patches from organized point clouds in a single pass.Source: §2.4 · p. 2

3. Method

SHIFT extends Voxfield’s non-projective pipeline by identifying planar structure in organized depth data, bundling flat pixels into weighted super-rays, and freezing converged flat-voxel gradients. It also uses a compact ESDF layout to reduce wavefront memory.

3. Method

  • SHIFT processes RGB-D and projected LiDAR through classification, flat-pixel bundling, non-projective TSDF fusion with freezing and skipping, and a compact ESDF layout.LiDAR scans are projected into a spherical range image so both modalities share the downstream pipeline.Source: §3 · pp. 2–3

3.1. Depth-geometry surface classification

  • The classifier labels organized-cloud pixels FLAT, EDGE, or CURVED using depth geometry, fitting and merging local planes while stopping growth when accumulated curvature raises the merged error.Creases between neighboring segments are kept separate, preventing bundling or freezing across corners.Source: §3.1 · p. 3

3.2. Coplanar bundling into weighted super-rays

  • Flat pixels are grouped by tile and plane segment, then represented by one residual-weighted ray projected onto the fitted plane and carrying the group’s summed weight.Edge and curved pixels remain individual unit-weight rays; grouping by segment prevents tiles spanning two planes from mixing points.Source: §3.2 · pp. 3–4

3.3. TSDF fusion with freezing and skipping

  • A flat voxel freezes its gradient after its accumulated weight reaches Wf=5; later distance updates are skipped only within δskip, while larger changes still update distance and weight.Frozen gradients retain the first observing frame’s segment normal without refinement; 32-bit wavefront indices and packed voxel fields reduce cache-line use without changing stored distances.Source: §3.3 · p. 4

4. Experiments

SHIFT is evaluated on four public datasets spanning synthetic and real capture in RGB-D and LiDAR, with runtime and field accuracy compared against four mapping baselines. The experiments find lower TSDF integration cost while examining accuracy, the sources of savings, component contributions, and parameter sensitivity.

4. Experiments

  • The evaluation uses four public datasets spanning synthetic and real capture in RGB-D and LiDAR, with Voxfield as the primary baseline and three projective mapping systems as additional baselines.All methods share the data loader, pose source, mesh extractor, and evaluation code; runs use one CPU thread and no GPU.Source: §4 · pp. 4–5, Table 1 · p. 5

4.1. Integration cost and accuracy

  • SHIFT cuts per-frame TSDF time by 1.53 to 3.28× at representative resolutions and has the lowest TSDF curve across datasets and voxel sizes; ESDF time tracks Voxfield.End-to-end gains are smaller because the classifier adds fixed per-frame cost.Source: §4.1 · p. 5, Table 3 · p. 5, Figure 2 · p. 6
  • Accuracy remains close to Voxfield: on N. College, mesh error is 6.43 cm for both while SHIFT's TSDF RMSE is 12.82 vs. 14.74 cm; on Flat, TSDF RMSE is 3.87 vs. 3.31 cm.Across sequences, ESDF-GT RMSE stays within 0.73 cm of Voxfield and coverage drops by at most 0.3 points.Source: §4.1 · p. 5, Table 2 · p. 5, Figure 3 · p. 7

4.2. Where the saving comes from: ray compression

  • At representative resolution, bundling compresses Flat's 307 k input pixels to 62.8 k rays (4.9×), while the ESDF layer shrinks by 8 to 28% against Voxfield.The compression ratio exceeds the caster ratio on Flat because Voxfield's merged caster already collapses some coplanar pixels; ray counts do not translate one-for-one into TSDF speedups.Source: §4.2 · pp. 5–6, Table 4 · p. 6

4.3. Component ablation

  • Ablation attributes the TSDF speedup to super-ray bundling: it yields 3.28× on Flat and 1.87× on N. College, while skipping costs 1.6% on N. College and nothing on Flat.Adding classification without bundling is not a speedup and is slower than Voxfield on N. College.Source: §4.3 · p. 8, Table 5 · p. 8

4.4. Hyperparameter sensitivity

  • On LiDAR, increasing classifier cell size P from 8 to 16 costs 8.4 s and raises TSDF RMSE from 12.82 to 13.20 cm; the same sweep leaves dense RGB-D essentially unchanged.Skip tolerance and freeze-threshold settings do not materially change TSDF RMSE or the total-time ranking against Voxfield.Source: §4.4 · p. 8, Table 6 · p. 8

5. Conclusion and Future Work

SHIFT reports TSDF savings alongside reduced ESDF-layer memory, then identifies fixed front-end cost, dynamic-scene handling, and plane projection as areas for improvement. The supplement also describes additional materials and a planned public release.

5. Conclusion and Future Work

  • 1.42 to 4.07× TSDF savings accompany millimeter-level RGB-D accuracy, improved LiDAR RMSE, and up to 28% less ESDF-layer memory.The ablation attributes the gain to super-ray bundling rather than the classifier.Source: §5 · p. 8
  • The fixed per-frame classifier cost narrows gains on less planar or coarse-voxel scenes; N. College reaches 1.12× end-to-end speedup at 40 cm despite a 1.76× faster TSDF stage.Sharing classification with normal estimation is proposed as a way to close that gap.Source: §5 · p. 8
  • Freezing has no residual-motion unfreeze, making an unfreeze trigger for dynamic scenes a proposed next step.Source: §5 · p. 8
  • Eq. (9)’s plane projection raises Flat mesh error from 1.75 to 1.94 cm; adaptive projection is proposed to recover accuracy.Source: §5 · p. 8
  • The supplement adds classifier hyperparameters, dataset identifiers, ground-truth procedures, and results; code, converted dataset bags, and evaluation scripts are slated for public release upon publication.Source: §5 · p. 9

6. Classifier Hyperparameters

The classifier and fusion settings are compiled into fixed launch-time parameters, with selected values differing by sensor modality. The voxel size ν determines how truncation distance scales.

6. Classifier Hyperparameters

  • The classifier uses compiled-in constants κcell, κm, κreg, and γ, identical across datasets and modalities.Source: §6 · p. 9, Table 7 · p. 9
  • P is 12 for RGB-D and 8 for LiDAR; δskip is 1 cm and Wf is 5.Source: §6 · p. 9
  • Truncation distance scales with voxel size ν rather than remaining fixed, and εedge=0 for RGB-D means dense-depth cells are not rejected by the edge-fraction test.Source: §6 · p. 9

7. Dataset Sequences

The evaluation uses four RGB-D and LiDAR sequences with differing acquisition and ground-truth sources. The converted N. College bag and associated data-preparation and evaluation code are slated for public release.

7. Dataset Sequences

  • Flat is a 399-frame synthetic RGB-D room sequence with planar walls, floor, and furniture; its reference is the simulator’s 10,000-point analytic surface sample.Source: §7 · p. 9
  • Cow & Lady is the Voxblox RGB-D sequence played from its distributed ROS bag, with the release’s laser-scanned reference mesh.Source: §7 · p. 9
  • TUM uses the handheld fr1/room Kinect sequence; its 655k-point reference cloud is accumulated from 84 depth clouds and voxel-downsampled at 0.02 m.The benchmark provides trajectory ground truth rather than a surface reference.Source: §7 · p. 9
  • N. College uses the authors’ pre-processed dataset at half playback rate, with a 2.42 M-point accumulated reference cloud downsampled at 0.1 m.Source: §7 · p. 9
  • The converted N. College bag, data-preparation materials, and evaluation code are slated for inclusion in a public repository upon publication.Source: §7 · p. 9

8. Ground-Truth Generation

For TUM and N. College, which lack surface-reference ground truth, the authors generate accumulated reference clouds from sensor data. A utility transforms scans into a fixed world frame and merges them, with optional controls for range, stride, and downsampling.

8. Ground-Truth Generation

  • TUM provides trajectory ground truth only, and the used N. College version provides LiDAR odometry ground truth only, so neither supplies a surface reference.Source: §8 · p. 9
  • The utility transforms each point cloud into a fixed world frame using its pose, then merges the clouds into one accumulated reference cloud.Source: §8 · p. 9
  • Optional controls set a maximum sensor range, retain every N-th cloud, and voxel-downsample the accumulation; TUM yielded 655k points and N. College 2.42 M points.The utility and ground-truth mesh-generation script are slated for release upon publication.Source: §8 · p. 9

9. Additional Results

Supplementary results report classifier labels and per-frame timing, document reproducibility settings, and extend metric and sensitivity comparisons across voxel sizes.

9.1. Classifier label distribution

  • 78.9% Flat, 47.3% TUM, 42.1% Cow & Lady, and 70.6% N. College pixels were classified as flat at representative voxel sizes.Table 8 averages the logged frames; LiDAR classifies more edge pixels than any RGB-D sequence.Source: §9.1 · p. 10, Table 8 · p. 10

9.2. Per-frame timing distribution

  • SHIFT’s per-frame timing spread is tighter than Voxfield’s on Flat and TUM, comparable on Cow & Lady, and unavailable for N. College.The reported distributions are for representative-voxel runs.Source: §9.2 · p. 10, Table 9 · p. 10

9.3. Reproducibility

  • All baselines share the build, data loader, pose source, mesh extractor, and evaluation code; no GPU is used, and only the mapping back-end differs.The experiments use Ubuntu 22.04, ROS 2 Humble, C++14, and a single CMake/ament workspace.Source: §9.3 · p. 10

9.4. Full voxel sweeps of supplementary metrics

9.5. Ablation and sensitivity figures

  • The ablation figures show classifier front-end cost on N. College repaid by bundling’s TSDF savings; skip tolerance and freeze threshold barely move results, while LiDAR cell size has a clear effect.The stage breakdown separates front-end, TSDF, and ESDF time across the ablation configurations.Source: §9.5 · p. 10, Figure 5 · p. 11, Figure 6 · p. 11

9.4. Full voxel sweeps of supplementary metrics

  • SHIFT uses less ESDF-layer memory than Voxfield at every voxel size and dataset, while Chamfer-L1, coverage, reconstruction coverage, and ESDF ground-truth RMSE are near-ties.These sweeps extend the representative-voxel comparison.Source: §9.4 · p. 10, Figure 7 · p. 12, Figure 8 · p. 12
Loading 2609.06547v1…