PaperlayerView exact revision on arXiv

Source-linked AI summary

NeSAM: Neuro-Symbolic Kinodynamics with Soil Adaptation for Off-Road Mobility

Chenhui Pan, Tong Xu, Francesco Cancelliere, Xuesu Xiao

arXiv:2608.21330v1cs.RO

TL;DR

Off-road motion prediction is difficult because deformable-soil effects vary with local conditions, while existing learning models do not explicitly represent soil mechanics. NeSAM combines learned terrain representations with differentiable Bekker-Wong terramechanics, Transformer residual dynamics, and EKF-based soil adaptation, improving prediction and closed-loop tracking across simulation and physical experiments.

  • Problem

    Existing learning-based kinodynamic models do not explicitly represent soil mechanics, while deformable terrain causes sinkage, slip, and terrain-dependent motion that must be modeled for off-road prediction.

  • Method

    NeSAM combines learned terrain representations, differentiable Bekker-Wong terramechanics, Transformer-based residual dynamics, and EKF-based online updates of interpretable soil parameters.

  • Results

    69.4% reduction in geometric tracking error is achieved with online soil adaptation, while NeSAM attains the lowest error across all six reported simulated state dimensions.

  • Takeaways & Limitations

    NeSAM improves long-horizon vehicle-motion prediction and closed-loop trajectory tracking in simulation and physical experiments over deformable terrain.

  • Takeaways & Limitations

    NeSAM assumes Bekker-Wong terramechanics adequately represents wheel-terrain interaction, so substantially different terrain behaviors may remain difficult to model after parameter adaptation.

Abstract

from arXiv · show

Accurate prediction of off-road vehicle motion over deformable terrain remains challenging because sinkage, slip, and traction vary with local soil conditions. Existing learning-based kinodynamic models directly approximate vehicle-terrain interactions from data but do not explicitly represent soil mechanics and offer limited physical interpretability. To address these limitations, we present NeSAM, a neuro-symbolic framework that combines differentiable Bekker-Wong terramechanics with learned terrain representations and a Transformer-based residual dynamics model for long-horizon, six degree-of-freedom kinodynamic prediction. The terramechanics component models soil-dependent interaction forces, while the residual model corrects discrepancies between the analytical prediction and the observed vehicle dynamics. NeSAM further estimates physically meaningful soil parameters from terrain observations and updates them online using an extended Kalman filter. We evaluate NeSAM in Verti-Bench, a simulator built on the Chrono multiphysics engine, and validate its performance on a physical Verti-4-Wheeler platform. NeSAM improves prediction accuracy by up to 30% in simulation and 29% on real-world data relative to the strongest compared baselines. When integrated with a close-loop navigation controller, NeSAM further improves traversal success rate through online soil adaptation while reduces Hausdorff distance to the reference trajectory by 69.4%, indicating improved trajectory tracking accuracy.

I. INTRODUCTION

Off-road motion prediction must account for deformable-terrain effects that alter vehicle translation and attitude. NeSAM combines differentiable terramechanics, learned terrain representations, residual dynamics, and online soil adaptation to address this need.

  • Deformable terrain induces sinkage, shear deformation, and wheel slip, making conventional planar kinematic models inadequate for six-degree-of-freedom prediction.
  • Classical terramechanics provides interpretable soil parameters but may miss vehicle-dependent effects, while parameter estimation and repeated simulator queries remain challenging.
  • Learning-based kinodynamics models capture nonlinear interactions from data but generally treat terrain as rigid and lack explicit deformable-soil variables for adaptation.
  • NeSAM integrates differentiable Bekker-Wong terramechanics, learned terrain representations, and Transformer-based residual kinodynamics for long-horizon six-degree-of-freedom prediction.
  • The framework updates physically interpretable terramechanics parameters online through an EKF using differences between predicted and measured vehicle motion.
  • The evaluation covers long-horizon prediction in Chrono-based simulation and real-world trajectories, plus closed-loop navigation on a physical Verti-4-Wheeler platform.

A. Terramechanics and Soil Identification

Prior approaches either model deformable-soil mechanics analytically or learn terrain-conditioned vehicle dynamics without explicit soil variables. NeSAM couples learned interaction modeling with differentiable terramechanics and online parameter adaptation.

  • Foundational terramechanics models describe wheel-soil forces using Bekker pressure-sinkage and Janosi-Hanamoto shear relationships.
  • Learning-based forward models support predictive planning, but recent terrain-conditioned methods generally represent terrain as rigid surfaces.
  • Existing learning-based methods do not explicitly model load-dependent soil variables governing sinkage and shear or provide physical parameters for online soil adaptation.
  • Differentiable simulators have supported system identification, control, and residual modeling, but prior work had not jointly combined deformable-soil mechanics, six-degree-of-freedom prediction, and interpretable soil adaptation.
  • NeSAM uses learned terrain-conditioned interaction modeling, predicted wheel sinkage, differentiable terramechanics, and a learned state residual to predict vehicle motion.
  • During deployment, measured motion updates the soil correction through a buffered EKF while learned model components remain fixed.

A. Problem Formulation

NeSAM formulates off-road motion as a terrain-conditioned transition from vehicle state, controls, and multimodal terrain observations. The transition recursively predicts a future trajectory while maintaining an adaptable soil estimate.

  • The vehicle state includes position, roll-pitch-yaw angles, linear velocity, and angular velocity, while controls correspond to speed and steering angle.
  • Vehicle-aligned elevation and semantic terrain patches are encoded into latent representations that form the terrain context.
  • A semantic soil-prior head produces nominal soil parameters for the differentiable terramechanics model, which are later refined online during deployment.
  • The one-step terrain-conditioned transition maps the current state, control, terrain observations, and soil estimate to the next predicted vehicle state.
  • Applying the transition recursively for K steps predicts the vehicle trajectory over the planning horizon.

B. Neuro-Symbolic Wheel-Terrain Interaction

The wheel-terrain branch uses recent interaction history to infer physical wheel variables, computes contact forces through differentiable Bekker-Wong terramechanics, and passes them to Newton-Euler dynamics.

  • Interaction tokens concatenate recent vehicle states, controls, and terrain contexts across a history of length L.
  • The sinkage head predicts terrain-dependent wheel sinkages from recent interaction history rather than from a single terrain observation.
  • For each wheel, predicted sinkage, angular velocity, vehicle state, and soil estimate instantiate the differentiable Bekker-Wong model.
  • The model computes wheel-terrain contact forces through differentiable pressure-sinkage and shear-force calculations.
  • The wheel branch predicts physical variables required by terramechanics rather than a learned force residual, and the resulting forces feed the Newton-Euler model.

C. Newton-Euler Kinodynamics Model Prediction

The Newton-Euler model aggregates wheel-level terramechanics forces into vehicle-level force and moment, then integrates the resulting accelerations into a physics-based state increment. A learned residual compensates for vehicle-level transition errors not explicitly represented by the analytical model.

  • Wheel-level Bekker-Wong contact forces are aggregated into the net force and moment acting on the vehicle.
  • The resulting net force and moment determine linear and angular accelerations through the Newton-Euler equations.
  • Numerical integration over the sampling interval converts the accelerations into changes in position, orientation, and linear and angular velocity.
  • The learned state residual compensates for discrepancies caused by effects such as suspension compliance, load transfer, simplified wheel-contact geometry, and integration error.

D. NeSAM Learning and Adaptation

NeSAM trains its learned components with multi-step autoregressive prediction and adapts soil parameters online while the vehicle operates. The semantic soil prior supplies nominal parameters, which measured motion periodically refines through buffered EKF updates.

  • Offline Autoregressive Training: Multi-step autoregressive training recursively feeds each predicted state into the next model input without future ground-truth states.
  • Offline Autoregressive Training: The rollout objective weights vehicle-state prediction errors across multiple steps, with gradients passing through the learned and differentiable physics components.
  • Online Soil Adaptation: During deployment, learned components remain fixed while only the terramechanics soil parameters are adapted.
  • Online Soil Adaptation: The semantic soil-prior head provides the nominal estimate before an online correction becomes available.
  • Online Soil Adaptation: The posterior correction remains fixed during the next buffer, while clipping enforces physically admissible soil-parameter bounds and initialization relies entirely on the semantic prior.
  • Online Soil Adaptation: After each buffer of transitions, differences between predicted and measured velocities, together with parameter sensitivities, drive one EKF update.

IV. IMPLEMENTATION

NeSAM is trained and evaluated on diverse simulated and physical vehicle-terrain interaction datasets using trajectory-level splits. Data collection uses randomized steering and speed exploration, while simulation varies soil parameters within semantic terrain classes.

  • Datasets and Preprocessing: The Verti-Bench simulation dataset contains 500 trajectories of approximately 25 seconds, totaling about 3.5 hours and 125,000 transitions.
  • Datasets and Preprocessing: The physical dataset contains 50 trajectories of approximately 30 seconds, totaling about 25 minutes and 15,000 transitions.
  • Datasets and Preprocessing: Both datasets use an 80%/20% train/test split at the trajectory level to prevent transitions from one trajectory appearing in both sets.
  • Datasets and Preprocessing: Randomized sinusoidal steering and speed commands expose the model to varying velocities, turning rates, and wheel slip.
  • Datasets and Preprocessing: Each simulated semantic terrain class samples bounded terramechanics parameters independently by region, allowing identical labels to produce different soil responses.
  • Datasets and Preprocessing: Each transition records vehicle state, control input, next state, and aligned 128 × 128 elevation and RGB semantic patches at 10 Hz.

B. Terrain Representation and Soil Prediction

NeSAM uses separate pretrained U-Net autoencoders for elevation and semantic observations, then maps semantic features to a six-dimensional nominal soil estimate for differentiable terramechanics.

  • Separate U-Net autoencoders encode elevation and RGB semantic inputs into matching 18×16×16 spatial feature maps.
  • The semantic autoencoder receives three-channel RGB input, whereas the elevation autoencoder receives a single channel.
  • A soil predictor processes semantic features and outputs the six-dimensional nominal soil estimate used by the differentiable terramechanics model.
  • The soil predictor is optimized jointly with the kinodynamic predictor and contains 28,262 trainable parameters.

C. Neuro-Symbolic Predictor and Training

NeSAM combines a causal Transformer with differentiable terramechanics and learned residual correction for autoregressive six-degree-of-freedom prediction. Evaluation uses 32-step rollouts in simulation, with ablations isolating residual and sinkage contributions.

  • Architecture: A causal Transformer processes recent vehicle-terrain interaction history to predict wheel-interaction quantities and a 12-dimensional chassis state residual.The wheel branch supplies quantities required by differentiable terramechanics, while the residual head corrects the propagated state.
  • Evaluation: NeSAM is evaluated using fully autoregressive 32-step rollouts over a 3.2 s horizon at 10 Hz on held-out trajectories.Prediction error is measured with position and orientation mean absolute error.
  • Results: NeSAM achieves the lowest error across all six reported state dimensions in Verti-Bench.Compared with the Transformer, x-, y-, and z-errors decrease from 0.89, 0.90, and 0.09 m to 0.80, 0.78, and 0.06 m, while yaw decreases from 6.76° to 4.18°.
  • Ablations: Removing the state-residual branch increases x- and y-errors from 0.80 and 0.78 m to 0.99 and 0.98 m.Suppressing wheel sinkage increases error across all six dimensions; relative to Sinkage = 0, full NeSAM reduces x-, y-, and z-errors by approximately 7.1%, 7.8%, and 18.3%.

2) Trajectory Tracking with Online Soil Adaptation:

Online EKF-based soil adaptation is tested for closed-loop tracking under out-of-distribution soil conditions, using the same NeSAM predictor and MPPI controller with fixed or adapted soil estimates. Adaptation improves completion and geometric tracking on simulation and supports the same deformation-aware benefits on physical trajectories.

  • Simulation tracking: The closed-loop comparison changes only EKF-based soil adaptation: fixed soil retains the nominal estimate, whereas adapted soil refines parameters online.Each configuration is evaluated over five runs under out-of-distribution soil conditions using completion, traversal time, Hausdorff distance, and roll and pitch error.
  • Simulation tracking: 4/5 adapted-soil runs complete the reference trajectory, compared with 0/5 fixed-soil runs.Traversal time is not meaningfully comparable because no fixed-soil trial completes the trajectory.
  • Simulation tracking: 69.4% reduction in geometric tracking error accompanies online adaptation, with Hausdorff distance decreasing from 6.87 to 2.10 m.Mean absolute roll and pitch also decrease when soil parameters are refined online.
  • Physical validation: On physical trajectories, NeSAM achieves the lowest error in five of six state dimensions, while TAL Elev.+Sem. has the lowest pitch error at 3.61°.Relative to the Transformer, NeSAM reduces physical x-, y-, and z-errors by 17.1%, 15.7%, and 36.8%, respectively.
  • Physical validation: Physical ablations reproduce the simulation trend: removing the residual or suppressing sinkage increases prediction error.Relative to Sinkage = 0, full NeSAM reduces physical x-, y-, and z-errors by 11.7%, 13.2%, and 20.0%.

2) Trajectory Tracking with Online Soil Adaptation:

Online EKF soil adaptation improves NeSAM’s closed-loop trajectory tracking by updating soil parameters from measured vehicle motion, while physical experiments impose a clear limitation on the model’s constitutive assumptions.

  • Online soil adaptation: Online soil adaptation compares predicted and measured vehicle motion to update soil parameters during closed-loop navigation.The Fixed Soil configuration retains nominal parameters, whereas Adapted Soil updates them online.
  • Physical evaluation: Trajectory completion increases from 1/5 to 3/5 with online adaptation.This comparison uses five trials per configuration.
  • Physical evaluation: Mean absolute roll decreases from 9.23° to 7.27°, while mean absolute pitch decreases from 10.40° to 8.01%.The reported reductions are 21.2% for roll and 23.0% for pitch across five trials.
  • Interpretation: The experiments indicate that adapting interpretable soil parameters helps accommodate mismatch between nominal terramechanics and encountered wheel-terrain interactions.The reported consequence is more reliable closed-loop trajectory tracking.
  • Limitation: NeSAM assumes wheel-terrain interaction is adequately represented by Bekker-Wong terramechanics, so adaptation cannot change the underlying constitutive relationships.Terrain behaviors that substantially deviate from those pressure-sinkage and shear assumptions may remain difficult to model.
Loading 2608.21330v1…