PaperlayerView exact revision on arXiv

Source-linked AI summary

FLINT: Fast Lightweight Inference for Traversability

William Bonilla, Maxime Boisvert, David-Alexandre Poissant, David Meger, Louis Petit

arXiv:2609.26857v1cs.ROcs.AI

TL;DR

Off-road traversability depends on both terrain and robot capability, and self-supervised labels inherit the collecting robot’s capabilities. FLINT compares corridor-derived label strategies with a lightweight single-camera estimator; its heads outperform a human-labeled baseline live and typically beat WildOS on replayed logs.

  • Problem

    Off-road traversability depends on terrain and platform capability, while self-supervised labels encode different definitions and inherit the capabilities of the robot that collected them.

    Source: §I · p. 1

  • Method

    FLINT uses a frozen backbone and lightweight head to produce dense costmaps from one camera, comparing three corridor-derived self-supervision strategies with a human-labeled baseline.

    Source: §I · p. 1

  • Results

    On replayed logs, traj and stego produced cheaper driven-path costmaps than WildOS on 23/24 logs, while sam3 reached 99.0% autonomy in live deployment against HLTM’s 96.3%.

    Source: A. Offline Comparison Against… · p. 5, B. Self-Supervised Heads vs.… · p. 5

  • Takeaways & Limitations

    The reported field results show that CPU-only, single-camera inference can outperform a human-labeled baseline in live deployment and compare favorably with WildOS on replayed logs.

    Source: §I · p. 1, A. Offline Comparison Against… · p. 5

Abstract

from arXiv · show

Navigation in off-road conditions is challenging due to the lack of structure. There is no fixed vocabulary for what is traversable. The traversability depends on both the environment and the embodiment's dynamics. Neither of these two variables can be hand-labeled at scale. Thus, traversability has to be learned by the embodiment's own experience. Modern platforms tend to use multiple sensors to estimate traversability and navigate: RGBD cameras, lidar, radar, IMU, with computationally intensive platforms to run inference on neural networks. Against this trend, we propose FLINT, a lightweight traversability estimator: a 21.6M-parameter backbone, 38\times smaller than a comparable foundation-model backbone, that scores higher on held-out terrain probes and runs at 14.7 FPS on CPU alone using a RGB camera has the only sensor. Despite that gap in scale, FLINT produces a cheaper, more accurate costmap than a deployed foundation-model system (WildOS) on 23 of 24 replayed field logs. We compare different self-supervised learning signals and deploy the resulting models on a real platform in closed-loop field trials: the best self-supervised head reaches 99% autonomy over the route, outperforming a human-label-trained baseline deployed live on the same course. Our results show that heavy sensing and computing are not necessary for traversability estimation.

I. INTRODUCTION

Off-road traversability depends on both terrain and robot capability, while self-supervised labels inherit the capabilities of the robot that generated them. FLINT pairs a lightweight single-camera estimator with controlled self-supervision comparisons and field evaluation.

I. INTRODUCTION

  • Self-supervised traversability labels vary in meaning and inherit the collecting platform’s capabilities, so a label learned by one robot may not transfer to a less capable robot.Footholds, collision-free travel, and visual similarity encode different notions of traversability.Source: §I · p. 1
  • FLINT’s frozen-backbone, lightweight-head estimator produces dense costmaps from one camera without a GPU, lidar, or depth sensor.The paper presents this as a smaller sensing and compute footprint than comparable field systems.Source: §I · p. 1
  • The paper compares three corridor-derived self-supervision methods on identical data and backbone against a human-labeled baseline, using zero-shot, replay, and live-field evaluation.The comparison is paired with real-robot experiments on a Clearpath Husky A200 using one RGB camera in mixed forest and gravel terrain.Source: §I · p. 1
  • FLINT’s best self-supervised heads required fewer interventions than a human-labeled baseline in controlled field deployment, and the gain tracked propagation beyond the driven corridor.The same ranking appeared on an independent offline zero-shot benchmark, pointing to propagation rather than a particular label generator as the source of the gain.Source: §I · p. 1

II. RELATED WORK

Related work is organized around the self-supervision signal and the sensing and compute required for field use. Prior approaches use varied signals and modalities, but the cited work had not directly compared them on identical data, backbone, and protocol.

A. Self-supervised labels for traversability

  • Traversability self-supervision uses diverse signals, including safe footholds, predicted collision and bumpiness, traction or footprint estimates, terrain sound, and proprioceptive traces.Approaches differ in both the experience signal and how it is processed into labels.Source: A. Self-supervised labels for… · p. 2
  • WVN propagates driven labels to visually similar regions, whereas V-STRONG uses SAM-segmented instances to derive positive and negative samples.Even camera-based observations of the driven path yield different information under these processing strategies.Source: A. Self-supervised labels for… · p. 2

II. RELATED WORK

  • The prior approaches reviewed had not been compared directly on identical data, backbone, and protocol.The related-work section frames the comparison around label-generation signals and field sensing and compute demands.Source: A. Self-supervised labels for… · p. 2, §II · p. 2

B. Computing and sensing

Prior systems trade sensing and compute in different ways, while FLINT uses a single RGB stream without a GPU. The paper also positions frozen vision backbones and datasets as tools for comparing label strategies and evaluating generalization.

B. Computing and sensing

  • V-STRONG and WVN use a single camera but require a GPU, while WayFAST and WayFASTER run on embedded GPUs and still depend on depth.WildOS combines lidar, IMU, three RGB-D cameras, and a foundation-model backbone on a Jetson AGX Orin.Source: B. Computing and sensing · p. 2
  • Boxi uses a substantially larger sensor payload, including two lidars, ten RGB cameras, an RGB-D camera, seven IMUs, and RTK-GNSS.The passage uses the system to illustrate the sensing demands some off-road platforms incur.Source: B. Computing and sensing · p. 2

C. Datasets and backbones

  • DINOv3 supplies dense patch features, SAM3 instance masks, and STEGO visual-similarity clusters; TartanDrive 2.0 supports label generation and RELLIS-3D provides dense human-annotated evaluation data.These backbones and datasets support comparing downstream strategies and evaluating generalization across label sources.Source: C. Datasets and backbones · p. 2

III. METHOD

  • The method turns one driven corridor into training labels through trajectory reprojection, SAM3 instance propagation, or STEGO cluster propagation, then trains a lightweight head and converts its output to a deployment costmap.The three strategies define positive and negative regions differently from the same underlying signal.Source: §III · p. 2

A. Self-Supervised Label Generation

The method derives three distinct supervision signals from the same driven corridor and trains separate heads on a shared frozen DINOv3 backbone. The signals label the corridor, instances overlapping it, or feature-space clusters.

A. Self-Supervised Label Generation

  • Trajectory reprojection labels the projected future 2.0-second wheel corridor positive and samples negatives outside a dilated margin.The path is reprojected using each run’s super-odometry and camera calibration, with negatives excluding sky and an automatically fitted hood mask.Source: A. Self-Supervised Label Generation · p. 2
  • STEGO labels feature-space clusters by corridor overlap, while the three strategies supervise different regions or groupings from the same drive.The figure contrasts the corridor band alone, the full coherent surface, and clusters on the patch grid.Source: Figure 1 · p. 3
  • A 384-dimensional frozen DINOv3 patch feature passes through a 512-unit hidden layer to produce a scalar traversability score.The MLP head is applied to patch-level features from the shared backbone.Source: Figure 2 · p. 3
  • SAM3 marks an instance positive in full when at least 5% of its area overlaps the corridor, labels sky negative, and leaves other below-threshold instances unlabelled.This propagates the sparse trajectory signal to the coherent surface containing the overlapping instance.Source: A. Self-Supervised Label Generation · p. 3

B. Head Training

Head Training uses shared run-level splits and a common lightweight MLP setup to compare propagation strategies. In deployment, a single RGB stream feeds frozen DINOv3 features through one active head and IPM into a Nav2 costmap, with CPU-only inference at 14.7 FPS.

B. Head Training

  • The training split uses 49 TartanDrive 2.0 runs and an 85/15 split by run, preventing adjacent frames from the same drive from leaking across train and validation.The split contains 104,083 valid frames; stationary segments are excluded.Source: B. Head Training · p. 3
  • Each head is the same MLP with 197,633 trainable parameters, trained with BCEWithLogitsLoss and AdamW for 20 epochs; SAM3 and STEGO use pre-extracted features, while TRAJ recomputes the frozen backbone.Training uses a single 12 GB desktop GPU; the optimizer uses a 10^-4 learning rate and 0.01 weight decay.Source: B. Head Training · p. 3

C. System Overview

  • A single RGB stream feeds frozen DINOv3 patch features to an active propagation head, whose per-patch probabilities are projected by IPM into a ground-plane occupancy grid for Nav2.The camera’s depth output is not used; Nav2 handles costmap construction, planning, and low-level control.Source: C. System Overview · p. 3
  • 14.7 FPS: the frozen backbone and one active head run on CPU at 68 ms per frame, with the shared backbone cost paid once across offline head comparisons.Live field deployment runs one head at a time; additional heads add negligible overhead only in offline side-by-side comparisons.Source: C. System Overview · p. 4

D. Human-Labeled Baseline

HLTM provides a human-annotation baseline for measuring how closely self-supervised labels approach human-labeled performance. It shares the frozen DINOv3 backbone and MLP head, but is trained on pooled dense masks from three datasets, with RELLIS-3D held out.

D. Human-Labeled Baseline

  • HLTM uses the same frozen DINOv3 backbone and MLP head as the self-supervised heads, but trains on pooled dense human-annotated masks from Yamaha, CAVS, and Great Outdoors.The masks are collapsed to binary traversable/non-traversable labels under the convention used for RELLIS-3D evaluation ground truth.Source: D. Human-Labeled Baseline · p. 4
  • RELLIS-3D is held out entirely from HLTM’s training pool, so its evaluation is genuinely zero-shot, exactly like evaluations of the other three heads.The pool includes 677 CAVS images and 1,199 Great Outdoors images.Source: D. Human-Labeled Baseline · p. 4

IV. EXPERIMENTS

The experiments first compare FLINT with foundation-model baselines, then assess self-supervised heads against human-labeled training and examine their compute cost. In replayed field logs, FLINT heads produce cheaper driven-path costmaps than WildOS in most logs while using substantially less latency and GPU memory.

IV. EXPERIMENTS

  • The section’s broader evaluation covers foundation-model comparisons, live closed-loop comparison with a human-labeled baseline, mechanisms distinguishing self-supervised heads, and CPU inference cost.The experiments vary the traversability head while keeping other system variables fixed for the live comparison.Source: §IV · p. 4

A. Offline Comparison Against Foundation-Model Baselines

  • The comparison replays 24 Husky field logs across five collection days and scores each system’s costmap with the same threshold-free driven-cost percentile.WildOS inference is run against RGB alone without its lidar and elevation-mapping stages; WVN is reimplemented in ROS-independent shadow mode and still takes gradient steps from logged velocity-tracking error.Source: A. Offline Comparison Against… · p. 5
  • 23/24 logs: traj and stego produce cheaper driven-path costmaps than WildOS, while sam3 does so in 22/24 logs.On the deployed Jetson Xavier, FLINT uses 5.7× lower latency and roughly a third of WildOS’s GPU memory by sharing one 70.6 ms backbone pass across three heads; WildOS takes 405.3 ms for one.Source: A. Offline Comparison Against… · p. 5, Table I · p. 5

B. Self-Supervised Heads vs. Human-Labeled Baseline

In a controlled live deployment, SAM3 and STEGO outperform the human-labeled baseline HLTM, with SAM3 reaching the highest autonomy and STEGO the longest intervention-free stretch. The paper also compares their driven-cost percentile against random feasible alternatives as an independent measure.

B. Self-Supervised Heads vs. Human-Labeled Baseline

  • An intervention is any interval in which the operator disengages autonomous mode with the planner’s deadman switch, regardless of cause.Minor and major interventions are manually distinguished by reviewing the costmap and driven context; major events involve a traversability misclassification that routes the planner toward a hazard.Source: B. Self-Supervised Heads vs.… · p. 5
  • The four heads ran on the same route and day with the same A* planner and costmap interface, so only the traversability head changed across runs.Across runs the robot covered 4.47 km during 119.1 minutes of continuous CPU-only autonomous inference without dropped frames.Source: B. Self-Supervised Heads vs.… · p. 5
  • 99.0% autonomy: SAM3 reached the highest rate and fewest interventions relative to distance, while STEGO logged the longest intervention-free stretch at 809.9 m (21.9 min).Both self-supervised heads outperformed HLTM, which reached 96.3% autonomy with 7 interventions.Source: B. Self-Supervised Heads vs.… · p. 5
  • 13.9%: SAM3 ranked ahead of HLTM at 15.0% and STEGO at 16.6% on driven-cost percentile, matching the offline zero-shot RELLIS-3D ordering among those heads.This measure compares the driven path’s cost with 100 random, kinematically feasible alternatives sampled from the same start over the same horizon.Source: B. Self-Supervised Heads vs.… · pp. 5–6

C. Head-to-Head Among Self-Supervised Signals

Among the self-supervised heads, traj is weakest because its labels cover only the driven corridor, while sam3 and stego propagate that signal beyond the band. In one successful turn, all heads chose different physical paths, showing that their training signals can shape behavior without changing whether the terrain is traversable.

C. Head-to-Head Among Self-Supervised Signals

  • 87.5% autonomy, 8 live interventions, and 22.1% driven-cost percentile made traj the weakest head across both live and offline evaluations.Its labels cover only the fixed-width driven corridor and do not propagate to surrounding coherent traversable surface.Source: C. Head-to-Head Among Self-Supervised… · p. 6
  • sam3 propagates the driven-corridor signal through instance masks, while stego propagates it through feature clusters, extending labels beyond the vehicle’s path.This added structure, rather than a different label source, distinguishes them from traj and lets them close the gap to and exceed HLTM.Source: C. Head-to-Head Among Self-Supervised… · p. 7
  • All four heads crossed the sharp turn without intervention or with only one small correction, yet each chose a different physical line through it.The passage states there is no single correct line; the training signal leaves a physical fingerprint on behavior even when every trajectory succeeds.Source: C. Head-to-Head Among Self-Supervised… · p. 7, Figure 5 · p. 7

D. CPU-Only Compute Benchmark

FLINT sustains CPU-only inference in the live control loop and is substantially faster than the separately benchmarked WVN feature-extraction pipeline on the same CPU. The same inference code also runs faster on a desktop GPU than CPU.

D. CPU-Only Compute Benchmark

  • 119.1 min of continuous autonomous-mode inference over 4.47 km ran live with no dropped frames or fallback to a lower rate.FLINT sustained CPU-only inference in the robot’s real-time control loop across all four field deployments.Source: D. CPU-Only Compute Benchmark · p. 7
  • 68.2 ms/frame (14.7 FPS) on CPU alone, versus 12.3 ms/frame (81.5 FPS) on an RTX 5070 GPU.The CPU-only speed is a live option on the robot’s real-time control computer, not just a desktop result.Source: D. CPU-Only Compute Benchmark · p. 7
  • 832.7 ms/frame (1.20 Hz) for WVN feature extraction was 12.2× slower than FLINT’s 68.2 ms/frame on the same CPU hardware.The WVN measurement is a lower bound because its online-learning step was not confirmed active.Source: D. CPU-Only Compute Benchmark · p. 7

V. CONCLUSION

FLINT combines a small frozen backbone with self-supervised traversability heads and runs CPU-only in real-time field deployments. In a controlled live comparison, its best self-supervised head exceeded the human-labeled baseline despite the baseline leading on zero-shot RELLIS-3D; the study was limited to one route and one day.

V. CONCLUSION

  • 99.0% autonomy was reached by the best self-supervised head, which outperformed the human-labeled baseline in live deployment despite that baseline leading on zero-shot RELLIS-3D.The comparison held route and planner fixed while varying the traversability head.Source: §V · p. 7
  • FLINT uses a frozen 21.6M-parameter backbone, 38× smaller than a comparable foundation-model backbone, and runs at 14.7 FPS on CPU alone.The system sustained live CPU-only inference for 119.1 min of autonomous driving across 4.47 km.Source: §V · p. 7
  • An independent driven-cost-percentile analysis agreed with the offline zero-shot RELLIS-3D ranking, corroborating the live comparison across two evaluations.At a shared zero-intervention waypoint, heads supported safe outcomes while differing in their traversability estimates.Source: §V · p. 7
  • The study’s scope was limited to a single test route and single-day conditions.The authors identify broader terrain and conditions, and online adaptation during deployment, as future work.Source: §V · p. 7
Loading 2609.26857v1…