Continual Learning for Traversability Prediction with Uncertainty-Aware Adaptation

Hojin Lee; Yunho Lee; Daniel A Duecker; Cheolhyeon Kwon

Source-linked AI summary

Continual Learning for Traversability Prediction with Uncertainty-Aware Adaptation

Hojin Lee, Yunho Lee, Daniel A Duecker, Cheolhyeon Kwon

arXiv:2609.17141v1Sep 15, 2026cs.ROcs.AIcs.LG

TL;DR

Learning-based traversability prediction struggles to adapt to novel terrains without catastrophic forgetting. The paper proposes generative experience recall with uncertainty-aware adaptation, and real-world experiments demonstrate adaptation across diverse environments while retaining prior experience.

Problem
Learning-based traversability predictors often generalize poorly to novel terrains, while adaptation can cause catastrophic forgetting of previously learned environments.
Method
The framework uses generative experience recall and uncertainty estimates to update traversability and recall models without storing past data.
Results
Real-world experiments with a skid-steering robot demonstrate adaptation across diverse environments while retaining performance in previously trained environments.
Takeaways & Limitations
Uncertainty-aware recalled samples support continual traversability learning with prior-experience retention and reliable navigation performance.

Abstract

from arXiv · show

Traversability prediction is a critical component of autonomous navigation in unstructured environments, where complex and uncertain robot-terrain interactions pose significant challenges such as traction loss and dynamic instability. Despite recent progress in learning-based traversability prediction, these methods often fail to adapt to novel terrains. Even when adaptation is achieved, retaining experience from previously trained environments remains a challenge, a problem known as catastrophic forgetting. To address this challenge, we propose a continual learning framework for traversability prediction that incrementally adapts to new terrains using a generative experience recall model. A key virtue of the proposed framework is two folds: i) retain prior experience without storing past data; and ii) incorporate the uncertainty of the generated samples from the recall model, enabling uncertainty-aware adaptation. Real-world experiments with a skid-steering robot validate the effectiveness of the proposed framework, demonstrating its ability to adapt across a series of diverse environments while mitigating catastrophic forgetting.

I. INTRODUCTION

Learning-based traversability predictors often generalize poorly to novel terrains, while adaptation can overwrite prior experience. The proposed framework addresses both challenges through generative experience recall and uncertainty-aware updates without storing past data.

I. INTRODUCTION: Learning-based methods perform well near their training conditions but often generalize poorly to novel terrains.Adaptation methods address novel terrain conditions, yet can produce catastrophic forgetting in previously trained environments.
I. INTRODUCTION: The framework combines generative experience recall with uncertainty-aware adaptation to retain prior experience while adapting to novel environments.It selectively uses trustworthy recalled samples to update the traversability predictor and recall model.
I. INTRODUCTION: The framework avoids memory-demanding replay by recalling prior experience generatively rather than storing past data.This design is presented as an efficient update strategy for continual traversability learning.
I. INTRODUCTION: Real-world experiments with a skid-steering robot show adaptation across diverse environments while retaining performance in previously trained environments.The trained predictor is integrated with model predictive control for navigation.
I. INTRODUCTION: The method predicts traversability from terrain features and robot dynamics, capturing complex robot-terrain interactions.Its inputs combine sensory environmental information with robot state information.

A. Learning-based Traversability Prediction

Learning-based traversability prediction captures robot-terrain interactions beyond handcrafted rules, but novel environments expose generalization, retention, and uncertainty-modeling challenges.

A. Learning-based Traversability Prediction: Learning-based methods model complex robot-terrain interactions with data, while geometry-only features may miss semantic terrain characteristics.
A. Learning-based Traversability Prediction: Novel terrains challenge generalization, and adaptation can degrade performance in previously trained environments through catastrophic forgetting.
A. Learning-based Traversability Prediction: Experience replay reuses past data for continual updates but trades memory scalability against representational capacity.
A. Learning-based Traversability Prediction: Generative continual learning synthesizes past data without storing it, but generated-sample uncertainty must be modeled for reliable adaptation and retention.
A. Learning-based Traversability Prediction: The framework overview combines uncertainty-aware prediction and navigation during deployment with uncertainty-aware adaptation through generative experience recall.
A. Robot Dynamics with Traction Parameters: The robot dynamics model uses traction parameters to represent linear and angular discrepancies between commanded and actual velocities across ground-robot platforms.

B. Planning with Uncertain Traversability Prediction

Navigation is formulated as a receding-horizon stochastic optimal-control problem whose dynamics depend on predicted traction distributions and is solved with sampling-based MPC.

B. Planning with Uncertain Traversability Prediction: The controller minimizes navigation cost over a receding horizon by selecting an optimal sequence of control inputs.
B. Planning with Uncertain Traversability Prediction: The dynamics propagate predicted traction samples, with ˆξ_t drawn from the traction distribution predicted by the traversability model.
B. Planning with Uncertain Traversability Prediction: Sampling-based MPC supports real-time solution of the stochastic control problem by exploiting GPU parallelizability.

C. Continual Learning Problem Formulation

The framework defines domain-incremental learning for probabilistic traversability prediction while addressing data-access constraints through uncertainty-aware adaptation and compact terrain-feature processing.

C. Continual Learning Problem Formulation: The model maps robot state and terrain features to Gaussian traction means and variances, defining probabilistic traversability prediction across deployments.
C. Continual Learning Problem Formulation: Each deployment introduces a new environment whose input distribution may shift, while the model must retain experience from earlier deployments.
C. Continual Learning Problem Formulation: The framework updates the previous model using only current-environment data, without direct access to earlier datasets because storing them is impractical on resource-constrained platforms.
IV. ALGORITHM DEVELOPMENT: The proposed algorithm performs uncertainty-aware adaptation as depicted in Fig. 2.
A. Self-Supervised Traversability Labeling: Traction labels are estimated from odometry with an Extended Kalman Filter modeling traction parameters as a random walk during navigation over varied terrain.
B. Terrain Feature Processing: RGB images and 3D point clouds provide visual and geometric terrain information, but dense high-dimensional visual grids are costly for real-time mobile-robot processing.
B. Terrain Feature Processing: The terrain feature grid represents geometric and visual information from terrain patches beneath the robot’s four wheels over a fixed pose window.
B. Terrain Feature Processing: An unsupervised autoencoder compresses DINOv2 embeddings into low-dimensional visual features for integration with geometric information in the terrain grid map.

C. Robot-terrain Interaction Dataset

The interaction dataset pairs terrain features and robot velocity with traction labels, and an ensemble predicts traction distributions with separate uncertainty components.

C. Robot-terrain Interaction Dataset: Each dataset sample contains terrain features, robot velocity state, and a labeled traction parameter at one time step.
C. Robot-terrain Interaction Dataset: For a four-wheeled robot, terrain features are extracted at individual wheel contact points along trajectory histories within local terrain maps.
D. Uncertainty-aware Traversability Prediction: The prediction model receives concatenated terrain features and velocity state and outputs traction parameters as Gaussian distributions.
D. Uncertainty-aware Traversability Prediction: An independently trained probabilistic ensemble captures both aleatoric and epistemic uncertainty from the terrain and velocity inputs.
D. Uncertainty-aware Traversability Prediction: The final prediction and its mean and variance are computed from the ensemble members’ outputs.

E. Uncertainty-Aware Adaptation with Experience Recalling

The framework adapts traversability prediction across environments using generative recall without storing raw past data, while modeling uncertainty in recalled samples.

E. Uncertainty-Aware Adaptation with Experience Recalling: Generative recall synthesizes past input–output samples for experience replay without storing raw data.The recalled inputs combine terrain features with robot dynamic states, including velocity.
E. Uncertainty-Aware Adaptation with Experience Recalling: The prediction model jointly uses terrain features and robot dynamics to estimate traversability-related outputs and their uncertainty.The initial model is trained with negative log-likelihood over terrain features and robot velocity.
E. Uncertainty-Aware Adaptation with Experience Recalling: Velocity is sampled within hardware limits, while terrain features are generated conditionally through a CVAE using a Gaussian latent prior.The recall process samples velocity first and then generates corresponding terrain features conditioned on it.
E. Uncertainty-Aware Adaptation with Experience Recalling: The recalled output includes uncertainty σrecall, with higher values indicating environments poorly represented in the initial training dataset.This uncertainty measures the prediction model’s confidence in generated samples.

3) Uncertainty-aware traversability prediction model update:

The prediction model update combines newly collected data with recalled samples and aligns their output distributions to retain prior experience while limiting unreliable updates.

3) Uncertainty-aware traversability prediction model update:: The update loss combines new-data NLL with a recalled-sample adaptation term, weighted by λ.The first term represents newly acquired experience, while the second retains prior experience through recalled samples.
3) Uncertainty-aware traversability prediction model update:: Jensen–Shannon divergence aligns predicted and recalled output distributions during adaptation.The distributional alignment provides the distance measure for the recalled-sample adaptation term.
3) Uncertainty-aware traversability prediction model update:: Distributional alignment alleviates catastrophic forgetting and reduces overfitting to recalled samples with high σrecall.This reduces the risk of erroneous updates from unreliable recalled samples.

4) Uncertainty-aware generative experience recall model update:

The recall model is updated without explicit access to past data by filtering generated samples using predictive uncertainty and training on new data plus retained recalls.

4) Uncertainty-aware generative experience recall model update:: The generative recall model is updated from Ri−1 to Ri using newly collected and recalled data without explicit access to prior datasets.The procedure samples recall inputs with Ri−1 and evaluates them using the preceding prediction model.
4) Uncertainty-aware generative experience recall model update:: A variance threshold τ filters recalled samples whose predicted uncertainty indicates deviation from the real-data distribution.Uncertainty estimates serve as proxies for distributional similarity.
4) Uncertainty-aware generative experience recall model update:: The augmented dataset combines newly acquired data with filtered recalled samples for recall-model training.Only recalled samples with sufficiently low uncertainty contribute to retaining prior experience.
4) Uncertainty-aware generative experience recall model update:: Algorithm 1 initializes T1 and R1, then repeatedly generates recalls, updates Ti, and updates Ri as new datasets arrive.The framework starts from D1 and iterates whenever a new dataset is acquired.

F. Navigation with Traversability Prediction

Navigation uses the updated traversability model within an uncertainty-aware MPC controller that plans with predicted traction and penalizes uncertainty across diverse real-world terrains.

F. Navigation with Traversability Prediction: The MPC objective penalizes predictive uncertainty and slip while encouraging progress toward the goal.The slip term drives predicted traction toward 1, and uncertainty reflects limited environment knowledge or irreducible measurement noise.
F. Navigation with Traversability Prediction: Robot dynamics are propagated using mean estimates of uncertain traction parameters to approximate MPC costs.The propagated states are used to evaluate the cost function and optimize control inputs.
F. Navigation with Traversability Prediction: Only the first optimized control is applied, while remaining controls provide fallback actions if a later MPC computation misses its control-frequency target.The remaining actions are temporarily stored in a buffer.
F. Navigation with Traversability Prediction: Experiments use a Clearpath Jackal equipped with LiDAR, stereo vision, onboard computation, and lidar-inertial odometry across five distinct terrain environments.The environments include asphalt, gravel, sand, a bike path, and dense forest vegetation.

B. Evaluation Metrics

The evaluation measures continual learning through adaptation and memory retention, using Forgetting Measure (FM) to quantify performance degradation on earlier datasets and comparing the proposed framework with five baselines.

B. Evaluation Metrics: Continual learning performance is evaluated by adaptation and memory retention using prediction performance of the traction model.The model is trained sequentially on datasets D1 through D5, corresponding to Environments 1 through 5.
B. Evaluation Metrics: Forgetting Measure (FM) captures the maximum performance degradation on dataset Dk after training on Dj, with lower values indicating better experience retention.FM is defined as fk,j = max(0, nk,j − mini<j nk,i).
B. Evaluation Metrics: The comparison includes Complete-Memory, Incremental-Memory, Fast Adaptation, LwF-fashion, and Naive Generative Rehearsal baselines.The baselines respectively use full-data retraining, dynamic memory, recent data only, distillation, or recalled samples without uncertainty-aware adaptation.
B. Evaluation Metrics: All methods receive terrain and robot-state inputs and output probabilistic traction, with hyperparameters selected from the Pareto front using equal weighting of adaptation and memory retention.This setup is intended to make the comparison consistent across methods.

D. Field Experiment Results and Discussion

Experiments evaluate recalled-sample validity, quantitative adaptation and retention, and navigation after sequential training across five environments. The proposed method aligns recalled samples with real terrain features, achieves the lowest forgetting among methods without direct access to past data, and supports navigation performance comparable to Complete-Memory.

1) Evaluation of experience recalling:: The proposed recall model produces samples closely aligned with real terrain-feature distributions, unlike the dispersed samples generated by NGR.The distributions are visualized with PCA using real and recalled features from Environments 1–5.
1) Evaluation of experience recalling:: Uncertainty-aware adaptation helps the recall model produce samples that better reflect collected experience, supporting more accurate continual-learning updates.This comparison evaluates the proposed method against the NGR ablation without uncertainty-aware adaptation.
2) Evaluation of adaptation performance:: The proposed method yields the lowest FM among methods without direct access to past data, while retaining prior experience without storing it.Complete-Memory has the lowest forgetting overall, whereas IMOST can achieve comparable or occasionally better retention with a continually growing memory requirement.
2) Evaluation of adaptation performance:: After sequential training from D1 through D5, the proposed method reliably reaches goals while avoiding OoD grass areas across navigation trials.Other baselines except Complete-Memory and the proposed method show failures in retaining prior experience or estimating OoD uncertainty correctly.
2) Evaluation of adaptation performance:: The proposed method performs comparably to Complete-Memory in navigation despite being trained without access to past data.The navigation evaluation uses three trials per method and reports successful goal completion and OoD avoidance.