Source-linked AI summary
Explore Where It Matters: Targeted Exploration for Efficient Rover Adaptation
Jacob Levy, Arthur Bouton, Tristan D. Hassler, Hari Nayar, David Fridovich-Keil
TL;DR
Steep, deformable terrain creates mobility risks that simulation-trained rover models may not capture, while rover operations constrain trial-and-error. TERRA targets reward-relevant model errors during online adaptation and exploration, reaching high simulated slope-climbing success within five trials and adapting on ERNEST within three to five.
Problem
Simulation-trained rover strategies face unmodeled soil deformation and other terrain dynamics, while limited onboard operations leave little room for extensive trial-and-error.
Source: §1 · p. 2Method
TERRA adapts a simulation-trained world model with reward-weighted prediction loss and uses a loss predictor to guide MPC exploration toward performance-relevant errors.
Source: §1 · pp. 2–3, §4.1 · p. 5, §4.2 · p. 5Results
TERRA reaches approximately 90% success within five trials in No-Slope and Hard-Ground, with higher reward, shorter completion times, and lower energy to first success than baselines; ERNEST reaches the top within three to five trials.
Source: §5.2 · p. 7, §5.3 · p. 8Takeaways & Limitations
Targeting errors that matter for the task supports efficient adaptation from failed zero-shot slope traversal to successful active-suspension locomotion within a few trials.
Source: §6 · p. 8TERRA assumes task rewards are available during deployment, although reward functions may depend on privileged training-time information unavailable onboard.
Source: §6 · p. 8
Abstract
from arXiv · showhide
Future planetary rovers will traverse steep, deformable terrain where slip, sinkage, and terrain deformation can threaten mobility. Although simulation can capture some wheel-soil effects, unmodeled deformable-terrain phenomena create sim-to-real gaps requiring deployment-time adaptation. Yet, rover operations leave little room for extensive trial-and-error. We introduce TERRA (Targeted Exploration for Reward-Relevant Adaptation), an online adaptation framework that efficiently seeks deployment data exposing task-relevant model errors and corrects them. TERRA adapts a simulation-trained world model with a reward-weighted prediction loss and trains a loss predictor to estimate where future performance-relevant errors are likely to occur, using this signal to guide exploration during MPC. In simulation, TERRA reaches approximately 90% slope-climbing success within five trials under severe out-of-distribution pretraining conditions, outperforming exploration baselines in reward, completion time, and energy to first success. On the ERNEST rover, TERRA recovers coordinated active-suspension locomotion on a deformable Mars-simulant slope within three to five real-world trials. Project website: https://terra-explore.github.io
1 Introduction
Steep, deformable terrain makes rover mobility hazardous, while simulation-trained strategies face wheel–soil dynamics that simulators do not capture. TERRA targets rapid deployment-time adaptation by directing scarce exploration toward model errors relevant to task performance.
1 Introduction
- Steep granular slopes amplify wheel slip and sinkage, and past rovers have experienced severe mobility consequences on sandy or weak soil.Opportunity suffered substantial slip and sinkage; Spirit became immobilized at Troy.Source: §1 · p. 2
- ERNEST’s Active Gimbal Suspension coordinates roll-based load shifting and yaw-based bogie repositioning to support crawling-like travel on soft, steep slopes.The 75 kg rover’s two actively controlled joints also enable steering, wheel reconfiguration, load redistribution, and wheel lifting.Source: §1 · p. 2
- Simulation-trained locomotion faces a sim-to-real gap because classical terramechanics omits soil yielding, flow, excavation, and terrain deformation that can alter mobility.High-fidelity simulators resolving these effects remain computationally prohibitive for RL-scale training.Source: §1 · p. 2
- Rover operations leave little room for prolonged trial-and-error, so adaptation must collect deployment data that matters for progress or avoiding entrapment.The challenge is coordinating online adaptation and exploration so limited real-world interaction improves mobility on unseen terrain.Source: §1 · p. 2
- TERRA adapts a simulation-trained world model using reward-weighted dynamics loss and predicts where performance-relevant errors may arise to target exploration.The framework guides data collection toward interactions where reducing model error is expected to improve task performance.Source: §1 · pp. 2–3
2 Related Work
TERRA builds on model-based adaptation and self-modeling exploration, while targeting prediction errors by their relevance to downstream MPC performance. Its rover-mobility focus is online adaptation from very few real trials when simulation misses deployment dynamics.
Model-Based Adaptation
- TERRA adapts a learned model during deployment and selects interactions expected to improve downstream MPC performance by focusing on reward-relevant prediction errors.This extends pretrained-model and sampling-based MPC adaptation to unseen conditions.Source: §2 · p. 3
Exploration for Model Learning
- Unlike intrinsic-exploration signals such as novelty or model disagreement, TERRA predicts reward-weighted dynamics loss over MPC’s candidate sequences to guide task-relevant exploration.The predictor’s signal is described as planner-aligned and self-regulating as adaptation corrects errors.Source: §2 · p. 3
Rover Mobility
- TERRA addresses the regime where simulation misses deployment terrain dynamics by adapting a world model online from very few real trials.This complements terramechanics control, terrain-cue prediction, and domain-randomized active-locomotion strategies.Source: §2 · p. 3
3 Preliminaries and Overview
The setup uses ERNEST observations and actions, simulation data, and a simulation-pretrained world model for deployment-time sampling-based planning. The planner executes a selected action and the dynamics model is finetuned online as deployment data arrive.
Platform Setup and Simulation Data
- ERNEST observations combine proprioception, odometry, wheel forces, and a local terrain height map, while actions specify forward velocity and AGS joint-angle targets.A low-level controller generates wheel and AGS torques from these commands.Source: §3 · p. 3
- The task reward encourages progress along a predefined path while penalizing path deviation and excessive AGS use; simulation policy experience supplies the pretraining dataset.The simulation policy is trained in the DARTS environment using the procedure from [10].Source: §3 · p. 3
Model Pretraining and Deployment
- A simulation-pretrained world model predicts future observations from history and candidate actions, and MPPI scores sampled rollouts by predicted return before executing the selected sequence’s first action.Deployment data finetune the dynamics parameters online; the overview also shows both models updated from resulting data.Source: §3 · pp. 3–4, Figure 3 · p. 4
4 Targeted Exploration for Reward-Relevant Adaptation
TERRA adapts a simulation-trained world model by prioritizing prediction errors that matter for reward, then uses predicted future errors to guide exploration during MPC. Its closed loop couples informative planning with online model updates.
4.1 Performance-Aware Online Dynamics Adaptation
- Reward-weighted prediction loss and lightweight low-rank updates adapt the dynamics model while keeping its simulation-trained backbone fixed.The loss weights errors by local task-reward sensitivity while retaining ordinary prediction error to constrain the full observation space.Source: §4.1 · pp. 4–5
4.2 Loss Predictor for Estimating Performance-Aware Loss
- A loss predictor forecasts discretized performance-aware losses over the same candidate action sequences evaluated by the planner.Class-weighted training emphasizes high-loss bins, which can arise from poorly modeled soil deformation or rut formation.Source: §4.2 · p. 5
4.3 Exploration-Augmented MPC and Online Adaptation
- MPPI uses predicted performance-aware loss as an exploration reward alongside predicted task reward, directing candidates toward interactions likely to expose task-relevant errors.The exploration bonus decreases as those errors are corrected, so planning shifts toward exploitation.Source: §4.3 · p. 5
- After each planning step, TERRA adds the new transition to deployment data and updates both models on trajectory segments, always including the latest segment.The joint objective includes LoRA regularization.Source: §4.3 · pp. 5–6
5 Experiments
The experiments test whether TERRA can adapt a simulation-trained rover model to severe terrain shifts through targeted exploration. It reaches about 90% simulated slope-climbing success within five trials and succeeds on the real deformable slope within three to five trials.
5.1 Experimental Setup
- TERRA’s simulated tests use two severe pretraining-to-deployment shifts: no slope during No-Slope pretraining and hard-ground-like dynamics during Hard-Ground pretraining.The evaluation slope is 20° with internal friction angle reduced to 25°, increasing slip; adaptation carries forward across repeated trials.Source: §5.1 · p. 6
5.2 Simulated Experiments
- Approximately 90% success within five trials in both No-Slope and Hard-Ground; TERRA also achieves higher reward, shorter completion times, and lower energy to first success than exploration baselines.The simulated comparison covers 100 random seeds; several baselines approach TERRA’s final success rate in Hard-Ground, but not its other reported metrics.Source: Figure 4 · p. 7, §5.2 · p. 7
- Ablations show that the full method performs best when exploration is coupled with reward-weighted loss.The loss predictor directs interaction toward control-relevant errors, while reward weighting turns those interactions into task-relevant model updates.Source: §5.2 · pp. 7–8
- Figure 5 analysis indicates that TERRA targets errors where reward is sensitive: it achieves lower error on forward travel, while RND lowers aggregate error partly on reward-insensitive groups.The authors conclude that planning-relevant errors, rather than overall prediction loss, should be prioritized.Source: §5.2 · p. 7, Figure 5 · p. 8
5.3 Real-World Validation
- On the real 15° damp Mars-simulant slope, TERRA reaches the top by Trial 3 for Hard-Ground and Trial 5 for No-Slope.In Hard-Ground, the successful climb includes large alternating AGS yaw motions, unlike the near-zero yaw of Trial 1.Source: §5.3 · p. 8
6 Limitations and Future Work
TERRA’s scope depends on deployment-time reward availability and on selected loss-bin thresholds. The authors identify learned reward models and adaptive bins as future directions.
6 Limitations and Future Work
- TERRA assumes task rewards are available during deployment to compute reward-weighted prediction errors.The authors note that robot-learning rewards may rely on privileged training-time information unavailable onboard.Source: §6 · p. 8
- The loss predictor requires chosen loss-bin thresholds, which affect its supervision target and exploration-reward scale.The authors suggest adaptive bins that track the online loss distribution.Source: §6 · p. 8
- The authors propose learning reward models from partial observations or latent states as a direction for addressing reward availability.The proposed models would provide sensitivities for reward-weighted adaptation and exploration.Source: §6 · p. 8
A Platform and Task Reward Function
The appendix describes the platform’s observation and action spaces and defines the task reward as a weighted sum of terms. The supplied passages give additional detail on height-map sampling and reward quantities.
A Platform and Task Reward Function
- The modified height-map sampling uses three longitudinal columns ahead of each wheel, up to 1.185 m from the AGS, while excluding the centerline.Samples span −0.515 m to 1.185 m longitudinally and −0.6 m to 0.6 m laterally.Source: A Platform and Task Reward Function · p. 12
- The appendix identifies separate observation-space and action-space tables, but the supplied passages do not enumerate their entries.Source: Table 2 · p. 12, Table 3 · p. 12
- The task reward is a weighted sum of terms, with forward and lateral displacement, bearing deviation, and AGS joint angles identified in the reward-function table.The table identifies x and y as forward and lateral displacements, β as deviation from initial bearing, and qyaw and qroll as AGS joint angles.Source: A Platform and Task Reward Function · p. 12, Table 4 · p. 12
B Reinforcement Learning, Model, and Pretraining Details
The appendix describes RL pretraining data, world-model and loss-predictor architecture, and training and implementation details. The model uses a simulation-trained backbone with a separate loss predictor and policy architecture.
B Reinforcement Learning, Model, and Pretraining Details
- Pretraining logs observation–action pairs across independent policies in parallel environments and aggregates them into one simulation dataset per condition.RL follows the cited procedure using TD3; condition-specific environment counts and randomization parameters are listed in appendix tables.Source: B Reinforcement Learning, Model, and… · p. 12, Table 5 · p. 13, Table 6 · p. 13
- The world-model and loss-predictor transformers receive history encodings and candidate future-action embeddings, while the base-policy transformer generates warm-start action sequences for MPPI.The dynamics transformer produces future observation predictions; the loss-prediction transformer outputs logits over discrete loss bins.Source: B Reinforcement Learning, Model, and… · pp. 12–13, Figure 7 · p. 13
- The model is pretrained for 1.5 × 10^6 gradient steps with batch size 512 and takes approximately 24 hours on one NVIDIA A100 GPU.Training uses Adam, learning-rate decay, warmup, and Gaussian input-observation noise augmentation.Source: B Reinforcement Learning, Model, and… · p. 13
C Implementation Details
- The appendix points to separate parameter tables for MPPI and TERRA but does not provide their values in the supplied passages.Source: C Implementation Details · p. 14
D Baselines
The baselines use differing exploration signals, with exploration incentives tuned for comparability. When they share TERRA’s reward-weighted adaptation, TERRA still leads across all reported metrics and has the lowest energy to first success.
D Baselines
- The listed baselines include RND’s predictor of a frozen network, an ensemble of five LoRA modules for Disagreement, latent predictors for Plan2Explore, and an ensemble of reward predictors for RUNE.The passages provide these baseline components but do not fully specify each exploration signal.Source: D Baselines · pp. 14–15
- Exploration weights were tuned to match TERRA’s Trial-1 exploration reward, except Disagreement, which was matched on mean reward over 10 trials.The authors use this procedure to make exploration incentives comparable.Source: D Baselines · p. 15
- With reward-weighted adaptation shared, TERRA still outperforms baselines across all metrics and has the lowest energy to first success.This comparison tests whether targeted exploration remains more efficient when baselines also use TERRA’s adaptation mechanism.Source: D Baselines · p. 15
E Additional Experiments
Additional experiments examine TERRA under noise, a safety constraint, and a ripple task. The reported results indicate robustness to small/moderate noise, reduced roll-limit violations under constraints, and stronger ripple-task performance than baselines.
E Additional Experiments
- When baselines also use reward-weighted adaptation, TERRA still outperforms across all metrics and has the lowest energy to first success.This additional comparison isolates the benefit of targeted exploration when the adaptation loss is shared.Source: E Additional Experiments · p. 15
- TERRA remains robust to small/moderate noise and degrades only at large noise.The authors attribute robustness in part to pretraining noise augmentation and multi-step history and prediction, which favor temporally consistent structure over instantaneous sensor noise.Source: E Additional Experiments · p. 16
- With a 10° soft body-roll penalty and a 15° MPPI hard limit, roll-limit violations fall from 14.2% to 2.4% without sacrificing performance.The hard constraint rejects trajectories predicted to exceed the limit, keeping exploration within the safety-constrained feasible set.Source: E Additional Experiments · p. 16
- On the soft-soil ripple task, TERRA outperforms baselines while learning a front-wheel-lifting gait absent from Hard-Ground pretraining.Pretraining used forward velocity alone; deployment required placing a front wheel on the backside of the next ripple and pulling forward.Source: E Additional Experiments · p. 16
F Additional Ablations
The ablations indicate that TERRA’s exploration, reward-weighted loss, and lightweight adaptation design contribute to performance. They also show sensitivity to adaptation capacity and data selection, while some alternatives remain competitive on selected outcomes.
F Additional Ablations
- Removing either exploration or reward-weighted loss degrades performance.The ablation removes each component individually and/or together.Source: F Additional Ablations · p. 16
- Reducing LoRA rank from 64 to 32 lowers success and reward in both settings.The result indicates benefit from the larger low-rank adaptation space.Source: F Additional Ablations · p. 16
- Removing LoRA regularization remains competitive but worsens average rank; equal class weighting remains competitive but increases energy expenditure.Both alternatives are compared with TERRA.Source: F Additional Ablations · pp. 16–17
- Full dynamics adaptation substantially degrades performance relative to lightweight LoRA adaptation.The authors characterize unconstrained online updates as less effective.Source: F Additional Ablations · p. 17
- Removing the ordinary prediction term degrades performance, supporting its role in grounding predictions beyond reward-sensitive dimensions..Source: F Additional Ablations · p. 17
- Excluding the latest deployment segment from each batch degrades average rank.The result suggests that immediately training on the newest interaction helps online adaptation.Source: F Additional Ablations · p. 17
- Replacing discretized classification with direct regression of the loss predictor’s target yields worse performance.The stated rationale for binning is reduced sensitivity to extreme loss magnitudes, while class weighting emphasizes rare high-loss regimes.Source: F Additional Ablations · p. 17
- Scaling TERRA’s hyperparameters to 0.5× or 2.0× their Table 12 values yields similar performance..Source: F Additional Ablations · p. 17