PaperlayerView exact revision on arXiv

Source-linked AI summary

DELTA: Deformable Elevation-Based Local Terrain Attention Encoder for Sparse-Terrain Quadrupedal Locomotion

Sanghyun Park, Moonkyu Jung, Jemin Hwangbo

arXiv:2608.22033v1cs.RO

TL;DR

Sparse-terrain quadrupedal locomotion needs precise, state-relevant foothold evidence, but dense AME encoding becomes costly as map resolution increases. DELTA replaces full-map processing with fixed-size, proprioception-conditioned local sampling and attention. It preserves standard-resolution traversal performance while improving efficiency, supports better fine-grained traversal and mixed-course generalization, and transfers successfully to RAIBO2.

  • Problem

    Sparse terrain requires selecting limited valid footholds precisely, while dense AME encoding becomes more expensive at higher map resolutions.

  • Method

    DELTA predicts proprioception-conditioned deformable sampling locations, encodes adaptive local elevation patches as terrain tokens, and attends over a fixed-size token set without foothold or attention supervision.

  • Results

    DELTA matched AME’s standard-resolution final traversal performance while improving learning efficiency, fine-grained traversal, unseen mixed-course performance, and sim-to-real transfer on RAIBO2.

  • Takeaways & Limitations

    Fixed encoder cost enables higher-resolution terrain maps while preserving efficient learning and supporting generalization across mixed terrain compositions.

Abstract

from arXiv · show

Stable quadrupedal locomotion on sparse terrain requires selecting state-relevant terrain evidence for precise foot placement. Model-based foothold planners provide precise foothold selection but rely heavily on explicit model assumptions. Recent attention-based map encoding (AME) studies show that end-to-end reinforcement learning (RL) can learn implicit foothold guidance. However, the computational cost of dense AME encoding grows with map resolution, limiting its scalability to fine-grained sparse terrain. We propose DELTA, a Deformable Elevation-Based Local Terrain Attention encoder. DELTA predicts state-conditioned sampling locations, forms terrain evidence tokens from adaptive local elevation patches, and attends only to a fixed-size token set. With fixed sampling and patch settings, DELTA's encoder cost is independent of map resolution. Experiments show that DELTA achieves final traversal performance comparable to AME at the standard resolution while improving learning efficiency. This fixed encoder cost enables the use of higher-resolution terrain maps, improving traversal on fine-grained sparse terrain. DELTA also demonstrates strong generalization to unseen mixed evaluation courses composed of continuous and discrete terrain elements. Beyond simulation, DELTA demonstrates successful sim-to-real transfer on RAIBO2. Analysis of the learned sampling offsets and attention weights shows that DELTA samples steppable regions and attends to terrain evidence relevant to future touchdowns without foothold labels or attention supervision.

I. INTRODUCTION

Sparse-terrain locomotion requires precise selection of limited steppable regions, but dense attention encoding makes higher-resolution maps computationally expensive. DELTA addresses this trade-off with fixed-cost deformable terrain sampling and improves efficiency, fine-grained traversal, generalization, and real-robot transfer.

  • Motivation: Sparse terrain offers few valid contact regions, so foothold errors or gap-edge contacts can cause immediate locomotion failure.Policies must remain robust to map noise, state-estimation error, contact uncertainty, and disturbance while generalizing to unseen terrain compositions.
  • Limitations of prior approaches: Model-based and hybrid foothold methods depend on explicit assumptions and online planning, which can be affected by contact mismatch, unmodeled dynamics, and optimization cost.
  • Limitations of prior approaches: Dense AME encoding couples terrain precision to computation because higher map resolution increases dense feature extraction, projections, attention, and training time.Finer resolution is desirable for representing small footholds and narrow terrain features.
  • DELTA: DELTA predicts proprioception-conditioned deformable sampling locations, refines them with scout patches, encodes local elevation patches, and attends over terrain evidence tokens.It replaces full-map candidate construction with a fixed number of local samples.
  • Results: With fixed sampling and patch settings, DELTA’s encoder cost is independent of map resolution while preserving standard-resolution AME-level performance and improving learning efficiency.
  • Results: DELTA improves fine-grained stepping-stone traversal at higher resolution, generalizes strongly to four unseen mixed courses, and transfers successfully to the RAIBO2 platform.Attention analysis indicates sampling of steppable regions and terrain evidence relevant to future touchdowns without foothold labels or attention supervision.

II. METHOD

The policy uses proprioception and a robot-centered local elevation map as inputs, encodes the map into a compact terrain representation, and learns control end-to-end with PPO without foothold or attention supervision.

  • Policy overview: DELTA trains an end-to-end actor-critic locomotion policy from robot proprioception and a robot-centered local elevation map.
  • Inputs: The proprioceptive observation includes projected gravity, base angular and linear velocity, joint state, previous desired joint-position target, and velocity command.The map contains three channels storing base-frame x, y, and z coordinates.
  • Policy overview: DELTA maps proprioception and terrain input to a compact representation et, which is fused with proprioception for the policy.
  • Action generation: The multilayer perceptron predicts action means that are sampled during training and used directly during evaluation and deployment.
  • Training and control: The encoder and policy are optimized end-to-end with PPO and temporal action-mean smoothness regularization without foothold labels or attention supervision.A joint-space PD controller converts desired joint positions into clipped torque commands.

B. Deformable Elevation-Based Local Terrain Attention

DELTA builds terrain representations through sequential deformable attention layers that condition sampling on robot state, refine locations, encode local patches, and update a query through cross-attention.

  • Layered encoder: DELTA comprises L sequential deformable terrain attention layers that begin by converting proprioception into an initial query.
  • Sampling and tokens: Each layer determines sampling locations through state-conditioned sampling and scout-guided refinement before encoding local elevation patches into terrain evidence tokens.
  • Query update: The encoder updates the query through multi-head cross-attention and residual, normalization, and feed-forward operations, producing the final terrain representation qL = et.
  • Reference locations: Sampling uses K points per attention head, with first-layer references initialized from a perturbed base grid and later references passed forward from the preceding layer.

1) State-Conditioned Sampling:

DELTA generates state-conditioned sampling locations and bounds them to valid patch-center regions. A scout patch then refines each candidate using local terrain evidence and the current query.

  • State-conditioned sampling: Raw reference locations receive query-conditioned deformable offsets before being bounded to valid patch-center regions.The bounded locations serve as scout locations for subsequent local terrain inspection.
  • State-conditioned sampling: The scout patch tests whether a query-proposed candidate lies on a locally steppable region.DELTA extracts the patch around each scout location by bilinear sampling.
  • State-conditioned sampling: Scout-patch elevations are flattened and encoded with a shared MLP, then combined with the sampled center’s normalized base-frame coordinates.This produces the scout token used for location refinement.
  • State-conditioned sampling: The current query and scout token jointly predict a refinement offset for each sampling location.The refined reference is passed to the next layer, while its bounded version is used as the refined sampling location.

3) Adaptive Patch Encoding:

DELTA encodes adaptive local elevation patches into terrain evidence tokens. Its final layer separates center geometry from surrounding context and gates how much context is incorporated.

  • Adaptive patch encoding: Each refined sampling location yields a bilinearly sampled local elevation patch represented with center-relative elevations.The resulting terrain evidence token has dimension D.
  • Adaptive patch encoding: The first L −1 layers use Ps × Ps patches, while the final layer uses a wider adaptive P × P center–context patch.The wider final patch supplies additional context for edges and gaps.
  • Adaptive patch encoding: A wider final patch can mix irrelevant surroundings with center geometry, creating spurious correlations in unseen terrain compositions.This motivates separating center and context regions.
  • Adaptive patch encoding: At the final layer, DELTA independently encodes center and context features for every sampled point.The operations are applied at the final deformable layer for each indexed head-and-sample pair.
  • Adaptive patch encoding: A sigmoid gate controls the amount of projected context added to the center feature.The resulting feature preserves center geometry while incorporating only the necessary surrounding context.
  • Adaptive patch encoding: The final local terrain feature is concatenated with normalized three-dimensional base-frame coordinates to form the final-layer evidence token.The coordinate features retain the sampled center’s spatial information.

4) Multi-Head Cross-Attention:

DELTA applies single-query multi-head cross-attention to a fixed set of sampled terrain tokens. Its post-map encoder operations depend on fixed architectural settings rather than map height and width.

  • Multi-head cross-attention: Each deformable attention layer reshapes the query into head-specific subvectors and projects terrain tokens into keys and values.Projected values receive nonaffine RMS normalization before aggregation.
  • Multi-head cross-attention: Value-only normalization prevents low-attention tokens with large value norms from dominating aggregated context.This aligns token contribution magnitudes with learned attention weights.
  • Multi-head cross-attention: Residual, layer-normalization, and feed-forward updates produce the query for the next deformable attention layer.The head outputs are concatenated and linearly projected before the query update.
  • Multi-head cross-attention: Each DELTA head attends to K sampled candidates, whereas dense attention attends to all HW map-cell candidates.This replaces resolution-dependent candidate enumeration with fixed candidate sampling.
  • Multi-head cross-attention: For fixed sampling and patch settings, patch sampling, token encoding, projections, attention, and query updates depend on L, Nh, K, Ps, P, and D rather than H and W.Full-map preprocessing and data movement are excluded from this encoder-cost definition.

C. Policy Learning, Reward, and Curriculum

DELTA’s policy, encoder, and critic are trained jointly with PPO under randomized quadrupedal locomotion settings. Sparse-terrain rewards and a staged curriculum support training across continuous and discrete terrains.

  • Policy learning and control: The encoder, actor, and critic are trained jointly with PPO and temporal action-mean smoothness regularization.Actor observations are noisy, while the critic receives corresponding clean observations during randomized training.
  • Policy learning and control: Training uses 200 parallel environments, 4.0-second maximum episodes, 400 Hz simulation, and 100 Hz policy control.Joint-friction and slip randomization are also applied.
  • Reward design: The sparse-terrain reward relaxes posture regularization and strengthens penalties for prolonged stance and undesired contacts.Foot-related terms are modified for terrain edges, gaps, and limited footholds.
  • Reward design: A terminal penalty of −50 is applied upon locomotion failure.
  • Curriculum: The curriculum trains on five continuous and three discrete terrain classes using independent environment stages from 1 to 10.Stages promote after two consecutive successes and demote after three consecutive failures.
  • Curriculum: The mixed evaluation courses combine continuous and discrete terrain elements, including evaluation-only variants in Courses 3 and 4.These variants include irregular elliptical holes and curved gaps.

III. EXPERIMENTS AND RESULTS

The experiments compare MLP, AME, and DELTA terrain encoders under standard and fine-grained settings, using matched downstream policies and fixed training budgets. Evaluation covers traversal, learning, computation, and generalization-related performance.

  • Experimental design: Each policy was trained for 30,000 iterations and evaluated from the corresponding final checkpoint across 1,000 episodes and five fixed evaluation seeds.The same evaluation seeds were used for all compared policies.

B. Simulation Traversal Performance and Learning Efficiency

DELTA preserves AME-level standard-resolution traversal while learning faster and scaling better to fine-grained terrain. Its fixed encoder cost also produces larger computational gains as map resolution increases.

  • Standard resolution: 96.4% versus 96.3% overall mean SR: DELTA matches AME at 26 × 16 without losing final traversal performance.Both policies were evaluated at standard resolution under the same comparison.
  • Learning efficiency: 2,210 ± 160 iterations: DELTA reaches the sustained 90% criterion using 75.1% fewer iterations than AME.DELTA also shows lower variability across training seeds.
  • Fine-grained terrain: 95.9% versus 68.0% fine-grained SR at 41 × 25: DELTA substantially outperforms AME under the fine-grained curriculum.At 26 × 16, DELTA also reaches 84.0% compared with AME’s 65.7%.
  • Computational efficiency: 4.496 M encoder FLOPs remain fixed for DELTA across resolutions, while AME rises from 27.570 M to 67.910 M.DELTA reduces encoder FLOPs by 83.7% at 26 × 16 and 93.4% at 41 × 25.
  • Computational efficiency: 2.46× and 4.42× mean iteration-time speedups occur for DELTA at 26 × 16 and 41 × 25, respectively.DELTA reaches the sustained 90% criterion in 5.06 ± 0.33 hours, corresponding to a 9.9× speedup over AME.

D. Generalization to Mixed Evaluation Courses

DELTA generalizes substantially better than AME to unseen courses combining continuous and discrete terrain elements. Ablations associate this advantage with adaptive patch encoding and deformable, scout-guided sampling.

  • Generalization: 96.6% versus 26.0% average SR: DELTA substantially outperforms AME across four unseen mixed evaluation courses.Average progress is 98.4% for DELTA versus 62.7% for AME.
  • Failure analysis: AME failures emerge at terrain transitions and co-occurring continuous-discrete sections, where attention did not consistently shift toward upcoming foothold-relevant regions.These cases contain outgoing terrain, boundaries, upcoming footholds, and surrounding irrelevant terrain simultaneously.
  • Attention analysis: Scout-guided refinement raises steppable-surface sampling from 68.7% to 85.4%, while DELTA assigns 2.11× uniform attention at subsequent touchdown locations.Fixed Sampling produces the weakest sampling and attention metrics and the lowest mixed-course SR.
  • Ablation analysis: Removing Adaptive patch encoding reduces mixed-course average SR from 96.6% to 56.1% while leaving stepping-stone SR high at 97.3%.The result suggests that center–context control in sampled evidence primarily supports generalization.
  • Attention analysis: Without foothold labels or attention supervision, DELTA’s final-layer samples tend to lie on steppable regions and attention emphasizes evidence associated with subsequent touchdowns.Figure 4 presents sampling locations and head-averaged attention over the elevation map.

F. Real-Robot Experiments

The high-resolution DELTA policy transfers directly from simulation to the real RAIBO2 platform without additional real-world training or policy adaptation. It completes all evaluated real-world trials without an explicit foothold planner or online trajectory optimization.

  • Transfer setup: The deployed policy transferred directly from simulation without real-world training, policy adaptation, an additional foothold planner, or online trajectory optimization.Only the high-resolution DELTA policy was deployed on hardware; other policies were evaluated only in simulation.
  • System setup: Real-robot terrain observations came solely from front and rear Intel RealSense D430 depth cameras mapped onto a 2.0 × 1.2 m, 41 × 25 local elevation grid.The TensorRT policy ran at 100 Hz, with only NaN filling and sensor-spike removal applied.
  • Real-world performance: 100% SR: high-resolution DELTA completed all 21 real-world trials across seven terrain courses.The courses include gaps, stepping stones, stairs, and composite combinations.
  • Overall findings: The conclusion reports 95.9% fine-grained stepping-stone SR, 93.4% fewer encoder FLOPs than AME, and a 70.6-percentage-point mixed-course SR improvement.It also reports 75.1% fewer iterations and a 9.9× wall-clock speedup at standard resolution.
Loading 2608.22033v1…