PaperlayerView exact revision on arXiv

Source-linked AI summary

TravSUITE: Traversability via Self-Supervised, Uncertainty-Aware IRL and Terrain Estimation

Samuel Triest, Amirreza Shaban, David D. Fan, Wenshan Wang, Sebastian Scherer

arXiv:2610.32130v1cs.RO

TL;DR

Off-road robots need reliable maps and traversability estimates despite noisy or incomplete observations. TravSUITE combines self-supervised map inpainting with learned traversability and risk estimation; its experiments report steadier predictions with inpainting and safer hardware navigation with risk awareness.

  • Problem

    Off-road observation noise and occlusion can degrade local maps, making reliable traversability estimates difficult in complex environments.

    Source: §I · pp. 1–2

  • Method

    TravSUITE jointly learns map representations and traversability from hindsight supervision using inpainting and inverse reinforcement learning.

    Source: §I · p. 2, C. Optimizing Behavior · p. 4

  • Results

    Inpainting improved cost- and speed-map stability, while Parallel Trav. + CVaR had the best intervention metrics and avoided all man-made obstacles in hardware tests.

    Source: D. RQ3: Comparison to Other Methods · p. 7, F. Analysis of Hardware Results · p. 8

  • Takeaways & Limitations

    Hindsight supervision can generate large-scale geometric-semantic inpainting labels without additional engineering overhead.

    Source: §VI · p. 9

    The method is evaluated on too few biomes and platforms to substantiate its claimed generality, uses rudimentary uncertainty estimation, and cannot reason beyond its mapping horizon.

    Source: §VI · p. 9

Abstract

from arXiv · show

Traversability analysis in off-road settings remains a fundamental challenge for mobile robots. Key difficulties include constructing an accurate and expressive local map from multimodal sensor data, and using the map to design traversability rules that yield desirable navigation behavior. Importantly, this system must be resilient to the limited sensing regime brought about by complex environments and high speeds. In this paper, we present TravSUITE, a traversability system suitable for high-speed navigation in off-road environments. TravSUITE consists of two major components: 1) an efficient voxel mapper that leverages visual foundation models (VFMs) to build a rich geometric-semantic local map from streams of on-board sensor data, and 2) a unified neural network that jointly predicts traversability-relevant quantities in bird's eye view (BEV), including geometry, semantics, speed and cost. Our training strategy is entirely annotation-free and self-supervised, leveraging tasks such as map inpainting and inverse reinforcement learning (IRL) to learn both map representations and traversability. We perform a thorough ablation study and comparison to state-of-the-art approaches, and the results indicate that cost learning and auxiliary inpainting each contribute significantly to planning quality, and their combination is critical for achieving state-of-the-art performance in off-road path planning. We also design a simple risk adaptation mechanism to leverage our method's uncertainty estimates at deploy-time, and demonstrate that a combination of inpainting and risk estimation can result in 80% fewer navigation errors and 5% faster autonomous traversal speeds in real-world hardware experiments.

I. INTRODUCTION

Off-road navigation depends on reliable local maps and traversability estimates, yet sensing uncertainty and the diversity of robot and environment requirements complicate both. TravSUITE combines flexible mapping, annotation-free inpainting, and learned traversability to study these design choices.

I. INTRODUCTION

  • Off-road mapping and traversability are difficult because observation noise and occlusion can degrade maps, while uncertainty can leave them erroneous or incomplete.Traversability rules also depend on robot capability, operator preference, and environmental geometry and semantics.Source: §I · pp. 1–2
  • Prior mapping approaches can be robust to occlusion and sparse sensing, but may depend on robot-specific data or environment-specific semantics and annotations.The cited methods include approaches [40] [27], with robustness at high speeds and long ranges reported for [40] [27].Source: §I · p. 2
  • Prior learned cost functions lack consensus on input representation, and uncertainty, large-scale datasets, and dense supervision remain underexplored or limited.The passage identifies these issues across prior approaches [26] [63] [15] [74].Source: §I · p. 2
  • TravSUITE combines flexible geometric-semantic mapping, annotation-free map inpainting, and uncertainty-aware traversability learning to study their effects on path planning.The paper reports that inpainting improves traversability estimation across mapping representations and that cost distributions support safer deployment.Source: §I · p. 2

II. RELATED WORK

  • BEV maps are computationally efficient and align with terrestrial robot motion, whereas volumetric maps represent the environment more fully but are harder to maintain.BEV can lose overhang and multiple-support-surface detail; volumetric approaches often need additional support-surface estimation.Source: §II · p. 2

A. Problem Setup

  • The problem setup models traversal as finding a cost function whose minimum-cost trajectory matches an expert trajectory, combining goal distance with behavior-map cost.The behavior map is defined in BEV with per-cell cost and maximum speed.Source: A. Problem Setup · p. 3

B. Decomposing Mapping and Traversability

TravSUITE decomposes traversability into mapping and behavior functions, then optimizes trajectory matching alongside map learning. Behavior supervision uses expert demonstrations to learn both costs and safe speeds.

B. Decomposing Mapping and Traversability

  • The unified objective optimizes expert-trajectory matching and auxiliary map prediction, coupling the mapping function with the traversability function that produces the behavior map.The behavior map contains cost and maximum-speed values in BEV; auxiliary inpainting adds structure to the map representation.Source: B. Decomposing Mapping and… · p. 3

C. Optimizing Behavior

  • TravSUITE uses MaxEntIRL and a LEARCH-like update based on the difference between expert and learner state visitations to train the cost map.The sampled learner trajectory is generated under the current cost map and goal, and the gradient updates both mapping and traversability parameters.Source: C. Optimizing Behavior · p. 4
  • Expert trajectories supervise a per-cell speed distribution trained toward the demonstrated speed, while negative samples encourage zero speed on untraversed cells.The speed target is an upper bound, so the model predicts categorical distributions over fixed bins and minimizes squared earth mover’s distance.Source: C. Optimizing Behavior · p. 4
  • Training samples use 50-meter expert trajectory segments and MPPI learner rollouts; the training-time speed penalty is set to zero for stability.Fixed-length segments avoid reducing path complexity for demonstrations in slower, more challenging scenarios.Source: C. Optimizing Behavior · p. 4

D. Optimizing the Perception Representation

TravSUITE learns map representations from hindsight-derived targets that include future observations, encouraging inpainting. Its voxel mapper aggregates visual and geometric features in a fixed local volume for learned mapping and traversability.

A. Aggregating Observations with VFM Voxel Mapping

  • The voxel mapper aggregates visual and geometric features in a fixed volume around the robot, producing voxel or BEV maps from prior observations as learned-model inputs.The map covers 100m × 100m × 20m at 0.4m × 0.4m × 0.1m per cell; the authors report competitive processing time against elevation mapping and real-time operation.Source: A. Aggregating Observations with VFM… · p. 4
  • The listed mapping features are provided in Table I, which the paper identifies as the feature inventory for its maps.The supplied table passage gives its title but not the feature entries.Source: Table I · p. 4, A. Aggregating Observations with VFM… · p. 4

D. Optimizing the Perception Representation

  • The network uses a modality-dependent ResNet-style encoder, followed by BEV-space inpainting and parallel BEV decoder heads for navigation-relevant predictions.The individual heads are lightweight, leaving most feature extraction to the backbone and encoder.Source: Figure 3 · p. 5

1) Generating Hindsight Supervision from Voxel Maps:

  • Hindsight map targets combine past and future sensor measurements, so predicting them encourages the mapping network to inpaint beyond current observations.The authors aggregate voxel grids intersecting the current map volume rather than using a fixed time window, making aggregation quality independent of robot speed.Source: 1) Generating Hindsight Supervision… · pp. 4–5, D. Optimizing the Perception… · p. 4
  • Hindsight supervision masks BEV cells observed only from distances of at least 10m, while future poses can still provide labels at any distance from the current state.The target is an oracular voxel map and corresponding BEV map for each dataset timestep.Source: 1) Generating Hindsight Supervision… · pp. 4–5

B. Network Architecture

TravSUITE uses an encoder and shared BEV backbone to produce features for multiple navigation-relevant prediction heads. Separate heads decode map modalities and categorical cost and speed distributions, enabling risk-aware estimates.

B. Network Architecture

1) Encoder and Backbone:

  • For voxel inputs, sparse 3D convolutions are splatted into BEV; BEV inputs use a ResNet-style encoder, and a UNet backbone models longer-range dependencies.Source: 1) Encoder and Backbone · p. 5

2) Prediction Heads:

  • At CVaR=0.8, the high-entropy cyan cell’s cost is inflated significantly, while the low-entropy red cell stays at a similar value across CVaR settings.Source: Figure 4 · p. 5
  • Separate lightweight decoders predict geometry and semantics, while cost and speed heads output categorical distributions over fixed bins for risk-measure selection.Source: 2) Prediction Heads · p. 5

C. Leveraging Uncertainty

Cost and speed are represented as distributions so the system can select risk measures at test time. The experiments use expert-driving data and evaluate behavior prediction and map inpainting.

C. Leveraging Uncertainty

  • At test time, CVaR of each costmap cell provides a risk-aware cost, while speedmap quantiles tune speed aggressiveness.Source: C. Leveraging Uncertainty · p. 5
  • A running risk tolerance is adjusted with vehicle speed: it decreases above a threshold and increases below another, encouraging safer behavior at speed and riskier choices when progress is limited.Source: C. Leveraging Uncertainty · p. 5

D. Training Details

  • Training uses roughly four hours of expert driving, about 100,000 frames across two sites, and five epochs; modality-specific losses supervise continuous and segmentation targets.Source: D. Training Details · pp. 5–6

A. Offline Experiments

  • Offline evaluation measures cost prediction with modified Hausdorff distance, speed prediction with expert-speed probability, and inpainting with modality-specific errors.Geometry uses L1 error, semantics uses L1 distance between distributions, and VFM features use L2 error.Source: A. Offline Experiments · p. 6

B. RQ1: The Effect of Inpainting on Behavior Learning

The experiments examine inpainting, input representation, comparisons with other methods, and hardware navigation. Across offline tests, inpainting improves planning, while representation and aggregation affect cost, speed, and map-prediction performance.

B. RQ1: The Effect of Inpainting on Behavior Learning

  • Inpainting improves both costmap and speedmap performance across datasets, with joint geometry, VFM, and segmentation inpainting performing best.Ablations used the same number of behavioral examples.Source: B. RQ1: The Effect of Inpainting on… · p. 6

C. RQ2: Choice of Input Representation

D. RQ3: Comparison to Other Methods

  • Parallel and cascaded traversability outperform TerrainNet, while the FCN without inpainting performs worst across both planning metrics.Parallel produces better speedmaps, and cascaded produces better costmaps.Source: D. RQ3: Comparison to Other Methods · p. 7, Table V · p. 7
  • Qualitative comparisons show more stable and accurate cost and speed maps with inpainting than with either network without inpainting.The figure depicts dense-vegetation navigation and a high-speed open-trail case with sparse LiDAR measurements.Source: D. RQ3: Comparison to Other Methods · p. 7, Figure 5 · p. 8

E. Hardware Experiments

  • Hardware trials used a full-scale ATV on a roughly 1.4 km waypoint course with on-trail and off-trail terrain, vegetation, obstacles, and winter snow cover.Source: E. Hardware Experiments · p. 7

F. Analysis of Hardware Results

Hardware trials show that inpainting and risk estimation shape both route choices and intervention outcomes. The conclusion presents TravSUITE as a self-supervised framework and identifies limits in validation scope, uncertainty treatment, and mapping horizon.

F. Analysis of Hardware Results

  • Parallel Trav. + CVaR was the only method to avoid all man-made obstacles; Parallel Trav. and TerrainNet Cfn. each had a collision.The authors associate risk-awareness with safer navigation; Table VI presents the run metrics, but its values are not included here.Source: F. Analysis of Hardware Results · p. 8, Table VI · p. 8
  • Inpainting helped the planner avoid unseen vegetation, while the FCN no Inpaint method underestimated its cost and became stuck in a cul-de-sac.The inpainting models inferred high cost from surrounding geometric and semantic cues and took a longer, safer trail route.Source: F. Analysis of Hardware Results · p. 8, Figure 7 · p. 9
  • All methods with inpainting had higher average autonomous speed, while adding CVaR reduced speed and interventions.The authors suggest inpainting stabilized speed predictions; CVaR reduced speed because the robot avoided more objects.Source: F. Analysis of Hardware Results · p. 8

VI. CONCLUSION

  • TravSUITE is a fully self-supervised framework for traversability estimation and map inpainting, with geometric-semantic inpainting supervision requiring no additional engineering overhead.The conclusion says inpainting and risk estimation matter in both offline and hardware results.Source: §VI · pp. 8–9
  • Further experiments across biomes and platforms are needed to substantiate generality; the method estimates local traversability and cannot reason beyond its mapping horizon.The authors also describe their uncertainty estimation as rudimentary and suggest more principled treatment may improve OOD performance.Source: §VI · p. 9
  • Future work includes additional self-supervised data, multi-floor and legged-robot tests, full 3D planning and segmentation, and non-metric representations for long-range planning.Source: §VI · p. 9

APPENDIX

The appendix describes VFM feature extraction and lifting, efficient voxel-grid mapping, and examples illustrating risk adaptation and the value of a volumetric representation.

APPENDIX

  • For most of the course, maximum risk aversion (ν = 0.8) was sufficient, while denser vegetation required more aggressive settings (ν = 0.2 −0.5).Source: APPENDIX · p. 13
  • Under a low-hanging canopy, the voxel network predicted low trail cost, whereas the BEV network classified the trail as an obstacle.The authors attribute the BEV error to limited vertical representativeness.Source: APPENDIX · p. 13, Figure 10 · p. 14

A. Semantic Feature Extraction and Lifting

  • Talk2Dino open-set segmentation uses Rellis3d prompts with an added snow class, while the raw-feature option uses Radiov3 features reduced to 32 PCA components.The authors report that sharpening cosine similarities before softmax improved segmentation empirically.Source: A. Semantic Feature Extraction and… · p. 14
  • Image features are lifted into 3D by projecting point-cloud points into the image frame using camera intrinsics and camera-lidar extrinsics.Bilinear interpolation and per-pixel depth binning help preserve feature sharpness, especially near object edges.Source: A. Semantic Feature Extraction and… · p. 14

B. Voxel Local Mapping with Features

  • The mapper maintains a sparse voxel grid by discretizing current measurements, shifting the previous grid into the current frame, and merging the grids.The sparse representation is described as compact and efficient for real-time online computation; the mapping extent is fixed locally.Source: B. Voxel Local Mapping with Features · pp. 14–15

1) EXTENT:

The mapping operations set a pose-centered local extent, discretize sensor measurements into voxels, and shift the prior grid into the current frame.

1) EXTENT:

  • EXTENT sets the local map origin to a multiple of the voxel resolution while retaining the configured local length and resolution.Source: 1) EXTENT · p. 15

2) DISCRETIZE:

  • DISCRETIZE assigns measurements to voxel indices and uses the point closest to the robot pose as each voxel's feature.Averaging is noted as a possible alternative aggregation function.Source: 2) DISCRETIZE · p. 15

3) SHIFT:

  • SHIFT moves the previous voxel grid into the current frame, discarding indices and features outside the mapping volume.Source: 3) SHIFT · p. 15

4) MERGE:

The appendix explains voxel-grid merging, runtime measurements, behavior-label generation, and terrain-surface estimation for BEV features.

4) MERGE:

  • When voxel indices overlap, MERGE combines their features with an exponential moving average; indices present in only one grid retain their feature.The mapper can optionally raycast to track hits and misses, discarding voxels whose passthrough ratio exceeds 0.75.Source: 4) MERGE · p. 15

C. Runtime Experiments

  • The voxel and elevation mapping pipelines differed by fewer than 2 ms per timestep on average, and both ran at around 8–10 Hz.Visual and BEV feature extraction accounted for 70–80% of runtime; the voxel-based network was considerably slower and may scale less effectively to larger maps.Source: C. Runtime Experiments · p. 15, Table VII · p. 16

D. Computing Behavior Labels

  • Behavior labels are generated by scattering trajectory visits into a normalized state-visitation map and averaging trajectory speeds per map cell.Optional footprint offsets account for the robot occupying multiple BEV cells.Source: D. Computing Behavior Labels · p. 15

E. Estimating the Terrain Surface

  • A Markov random field estimates the terrain surface because noise and occlusions make the minimum-height voxel unreliable.The estimated surface determines the diff feature and overhang threshold for each BEV cell.Source: E. Estimating the Terrain Surface · p. 15

F. Computing Hindsight Labels

TravSUITE computes hindsight voxel supervision with a two-pass, robot-speed-invariant algorithm, then derives BEV labels from the resulting voxel maps. The method uses future mapping-volume overlap to select supervision while discounting backtracks.

F. Computing Hindsight Labels

  • A two-pass algorithm avoids fixed time-window supervision, which can provide too few in-volume frames at high speeds or repeated observations when stationary.The method is described as invariant to robot speed.Source: F. Computing Hindsight Labels · p. 16
  • For each timestep, the target is the last future mapping volume that intersects the current one; the next volume shares no voxels with it.The contiguous interval up to that target supplies the voxel contributions, while backtracks are discounted because arbitrary-length backtracking requires perfect state estimation.Source: F. Computing Hindsight Labels · p. 16
  • The target-time map is cropped to the current mapping extent, using a volume with triple the side length to contain that extent.The larger volume is centered at the target-time state.Source: F. Computing Hindsight Labels · p. 16
  • The algorithm uses O(T^2) volume-intersection operations but O(T) voxel-processing operations, matching prior methods because voxel mapping dominates computation.The comparison is specifically with prior methods’ voxel-mapping operations.Source: F. Computing Hindsight Labels · p. 16
  • BEV labels are generated by running the BEV feature extractor on each resulting voxel map.The method derives BEV labels from voxel maps.Source: F. Computing Hindsight Labels · p. 16

G. Hyperparameters

The section lists parameter tables for local mapping, the network, training, and hardware experiments. It also states that voxel and BEV metadata match in the x and y dimensions.

G. Hyperparameters

Related papers

Loading 2610.32130v1…