PaperlayerView exact revision on arXiv

Source-linked AI summary

TERRA: Terrain-Aware Reconstruction, Retargeting and Control for Musculoskeletal Locomotion

Merkourios Simos, Chengkun Li, Bianca Ziliotto, Alexander Mathis

arXiv:2609.38653v1cs.ROcs.AIcs.LGq-bio.NC

TL;DR

Motion datasets rarely include aligned terrain geometry, and retargeting locomotion to musculoskeletal bodies must preserve contact and anatomical feasibility. TERRA reconstructs support geometry from kinematics and trains a muscle-actuated controller on motion–terrain pairs. It improves reconstruction and retargeting benchmarks and achieves the highest held-out completion rate across supported terrain families.

  • Problem

    Musculoskeletal imitation remains largely confined to flat ground, while motion datasets rarely provide aligned terrain geometry and retargeting must preserve contacts and anatomical and musculotendon limits.

    Source: §I · p. 1

  • Method

    TERRA reconstructs motion-relevant support geometry from kinematics using terrain priors, contact cues, and free-space evidence, then retargets motion under terrain-aware constraints.

    Source: §I · p. 1

  • Results

    TERRA achieved 99.9% terrain-family accuracy, reduced retargeting violations, and produced the highest held-out completion rate across supported terrain families.

    Source: A. Terrain Reconstruction · p. 7, B. Motion Retargeting · p. 7, C. Tracking Policy Performance · p. 8

  • Takeaways & Limitations

    Motion kinematics can provide enough environmental evidence to recover task-relevant collision geometry for muscle-actuated control across stairs, ramps, isolated supports, and chairs.

    Source: §VI · p. 8

Abstract

from arXiv · show

Recent advances in musculoskeletal modeling and reinforcement learning have enabled muscle-actuated agents to reproduce increasingly complex human motions. Yet these capabilities remain largely confined to flat ground, in part because motion datasets rarely include aligned terrain geometry and because retargeting terrain interactions to complex musculoskeletal bodies is challenging. We present TERRA, an end-to-end pipeline for terrain-aware retargeting and control of musculoskeletal locomotion. From kinematic trajectories alone, TERRA combines terrain priors, estimated contacts, and negative free-space evidence to recover task-relevant support geometry. TERRA further considers anatomical, tendon-continuity, and contact constraints during retargeting. Using the resulting motion-terrain pairs from five datasets, we successfully train a single muscle-actuated control policy on 9.4 hours of diverse locomotion. Across reconstruction, retargeting, and held-out tracking benchmarks, TERRA improves terrain accuracy, sharply reduces anatomical and interaction violations, and achieves the highest observed completion rate over supported terrain families. Overall, TERRA provides a practical route from scene-less motion data to muscle-actuated locomotion over diverse non-flat terrain. Project website: https://cnai.epfl.ch/terra/

I. INTRODUCTION

Musculoskeletal imitation has largely remained on flat ground because motion datasets rarely include aligned terrain geometry and retargeting must preserve contacts while respecting anatomical and musculotendon limits. TERRA reconstructs support geometry from motion and trains a muscle-actuated controller on paired motion–terrain scenes, extending this work to non-flat locomotion.

I. INTRODUCTION

  • Motion-capture datasets rarely provide aligned terrain geometry, and retargeting must preserve contacts while avoiding penetration, skating, floating, and anatomical or musculotendon-limit violations.These challenges are amplified by non-flat terrain interaction and can make reference motions infeasible.Source: §I · p. 1
  • TERRA reconstructs motion-relevant support geometry from kinematics and retargets motion with anatomical, contact, clearance, and collision constraints for a muscle-actuated model.The resulting motion–terrain pairs are used to train a single reference-conditioned controller.Source: §I · pp. 1–2
  • TERRA achieved the highest held-out completion rate across supported terrain families while reducing anatomical and interaction violations.The motion–terrain library spans five sources and 9.4 hours; the paper also compares muscle activity and vertical ground-reaction forces with held-out data.Source: §I · pp. 1–2

II. RELATED WORK

  • TERRA addresses an upstream gap left by methods that assume scene geometry: it recovers task-relevant support geometry from kinematics and evaluates reconstructed geometry against paired ground truth.Its interaction-mesh approach adapts OmniRetarget and adds target-specific anatomical and terrain-interaction terms for a muscle-driven embodiment.Source: §II · p. 2
  • TERRA extends musculoskeletal imitation, previously largely confined to flat ground, to three-dimensional locomotion using motions paired with reconstructed terrain.Prior work established reusable locomotor repertoires and empirical validation of simulated muscle activity.Source: §II · p. 2

III. METHOD

The method converts heterogeneous motion data into a musculoskeletal-compatible representation, reconstructs compact collision terrain, and retargets motion under terrain-aware constraints. The resulting paired scenes train a muscle-actuated tracking policy.

A. Problem Setup

  • The target is the finger-disabled MyoFullBody configuration, with 354 Hill-type musculotendon actuators, 83 MuJoCo joints, and 49 active polynomial couplings.The model has one floating-root joint and 82 scalar articulated joints.Source: A. Problem Setup · p. 2
  • Seventeen SMPL-H landmarks are paired with MyoFullBody body origins to align the model with reference motion data.Alignment optimizes SMPL-H shape and global scale in corresponding neutral poses, then applies residual position offsets to source motions.Source: A. Problem Setup · p. 2
  • TERRA accepts AMASS-compatible SMPL-H motion and marker-based C3D, MAT, or TRC data, and represents terrain with support-box, ramp, staircase, and seat priors.Terrain priors are assembled from static MuJoCo box geometries; staircases use ordered horizontal boxes and ramps use a pitched box.Source: A. Problem Setup · pp. 2–3

III. METHOD

  • TERRA’s pipeline converts motion data to SMPL-H, reconstructs compact collision terrain, retargets motion to a musculoskeletal model, and trains a muscle-actuated tracking policy.The overview also shows biomechanical outputs compared with subject-matched force-plate and EMG data on held-out motions.Source: §III · p. 2, Figure 2 · p. 3

B. Terrain Reconstruction

TERRA infers support surfaces from stationary, low ankle and toe landmarks while using the moving body as free-space evidence. It distinguishes ramps from discrete surfaces using stance-foot orientation and swing-foot clearance, then fits terrain geometry to the motion evidence.

B. Terrain Reconstruction

  • Stationary, low ankle and toe intervals provide positive surface evidence, while the remaining body trajectory bounds admissible terrain through free-space evidence.Candidate intervals are contiguous runs below a landmark-speed threshold; intervals overlapping detected seated periods are excluded from surface-height estimation.Source: B. Terrain Reconstruction · p. 3
  • TERRA distinguishes continuous ramps from discrete horizontal surfaces using ankle–toe orientation during stance and swing-foot clearance between stance intervals.When orientation errors are within 2°, a normalized clearance score favors ramps for near-linear trajectories and discrete surfaces for early ascent or delayed descent; the cue must exceed 5% of the observed support-level change.Source: B. Terrain Reconstruction · p. 3
  • For motions with at least three raised height levels, TERRA tests staircase and independent-box candidates, selecting stairs only when treads fit raised foot contacts and body penetration stays within 5 cm.Contact envelopes are expanded by a foot-sized margin and trimmed where they intersect free-space body trajectories.Source: B. Terrain Reconstruction · p. 3
  • TERRA detects a seat when pelvis speed stays below 0.15 m/s for at least 0.4 s and the pelvis is sufficiently outside the grounded-limb hull.It estimates seat height from posterior pelvis/hip surface heights and discards candidates outside 0.04–0.60 m.Source: B. Terrain Reconstruction · pp. 3–4

C. Terrain-Aware Motion Retargeting

TERRA retargets each motion–terrain pair by extending an interaction-mesh objective with anatomical and terrain-interaction terms, then solving framewise updates with sequential quadratic programming. Its residuals preserve motion and contact while enforcing terrain clearance and collision constraints.

C. Terrain-Aware Motion Retargeting

  • TERRA adapts OmniRetarget’s interaction-mesh objective with anatomical and terrain-interaction terms, then solves the retargeting problem by sequential quadratic programming.The framewise update minimizes residuals subject to penetration and trust-region constraints.Source: C. Terrain-Aware Motion Retargeting · p. 4
  • The interaction mesh pairs source landmarks with robot positions and samples reconstructed terrain, including box tops and a coarse floor grid.A Delaunay tetrahedralization defines neighbor sets for the mesh’s uniform Laplacian.Source: C. Terrain-Aware Motion Retargeting · p. 4
  • Added residuals match pelvis and limb orientations, anchor calcanei during stance, discourage ankle and toe motion, and enforce swing clearance and collision separation.Near terrain faces, ankle and toe targets add at most 0.06 m of terrain-clearing lift; seated-rest residuals bring pelvis collision geometry to the reconstructed seat.Source: C. Terrain-Aware Motion Retargeting · p. 4
  • The SQP step combines interaction-mesh, temporal-smoothing, trunk-angle, and active TERRA residuals while limiting penetration recovery to 10 mm per iteration.After optimization, short collision outliers and tendon discontinuities are fixed by bounded interpolation.Source: C. Terrain-Aware Motion Retargeting · p. 4

D. Policy Training

TERRA trains terrain-aware muscle-actuated policies by motion imitation, conditioning tracking on body state, terrain, and reference motion. PPO training uses pose and velocity tracking with muscle-activity regularization.

D. Policy Training

  • The tracking policy observes joint and root state, muscle commands and activations, foot and toe touch sensors, and an 11×11 terrain height map.The map is pelvis-centered with 0.1 m spacing.Source: D. Policy Training · pp. 4–5
  • One-step targets encode heading-frame root and mimic-site errors, while future targets at 0.2, 0.4, 0.6, and 0.8 s encode root motion and pelvis-relative site positions.Future targets do not encode motion phase.Source: D. Policy Training · p. 5
  • Policies are trained with PPO in 8192 parallel environments at 100 Hz for 4 billion steps, using adaptive sampling to emphasize hard motions.The reward combines pose and velocity tracking with a terminal quality bonus and muscle-activity regularization.Source: D. Policy Training · p. 5

IV. EXPERIMENTAL PROTOCOL

The protocol evaluates terrain reconstruction, retargeting validity, held-out policy tracking, and physiological profiles across five motion-capture sources. It includes known-terrain benchmarks, scene-less motion–terrain checks, and robustness tests.

IV. EXPERIMENTAL PROTOCOL

  • Five sources contribute non-flat motions, including 993 AMASS, 31 PRISM, 3,476 Gait120, 1,282 Darmstadt, and 716 Vielemeyer clips; policy experiments add 595 Gait120 and 975 AMASS flat clips.PRISM provides ground-truth object meshes, while Gait120, Darmstadt, and Vielemeyer provide known terrain geometry.Source: §IV · p. 6
  • Reconstruction is compared against contact least squares and Voronoi, with terrain-family accuracy and dimensional MAE on known-terrain datasets and additional support-height, coverage, and IoU measures on PRISM.Ablations test swing-clearance cues; robustness tests vary landmark noise, support intervals, and selected thresholds.Source: §IV · p. 6
  • For scene-less AMASS motions, terrain consistency is evaluated using contact-height error, contact and raised-support measures, and non-support penetration criteria.The event height uses a source-only offset from each probe’s lowest contact cluster.Source: §IV · p. 6
  • Retargeting comparisons include OmniRetarget, GMR, and MM-MoCap-Body, with a shared TERRA scene and ablations of TERRA residual terms.TERRA+Voronoi combines Voronoi reconstruction with TERRA motion retargeting.Source: §IV · p. 6
  • Held-out evaluation measures policy completion and tracking, while physiological comparisons visualize policy-generated EMG and vertical-GRF waveforms against human traces.The physiological cases cover level walking, ramp descent, stair descent, and sit/stand; traces are independently min–max normalized to compare waveform shape.Source: §IV · pp. 6–7, Figure 4 · p. 7, Figure 5 · p. 7

A. Terrain Reconstruction

TERRA accurately classifies and reconstructs terrain, with flexible priors and robustness to several perturbations. Support removal causes gradual, terrain-dependent degradation, most notably for stools.

A. Terrain Reconstruction

  • Terrain priors enabled flexible inference of slanted or piecewise-flat surfaces, while baseline methods were limited to more rigid representations.Source: A. Terrain Reconstruction · p. 7
  • TERRA achieved 99.9% terrain-family accuracy, sub-centimeter stair-height accuracy, and sub-degree ramp accuracy, outperforming baselines across all terrain categories.The reported family accuracy was 99.9%.Source: A. Terrain Reconstruction · p. 7
  • Across independent captures, grade MAE was 0.544°/0.529° at 7.5°/10°, and riser MAE was 7.36/4.34/3.61 mm at 100/170/240 mm.The subject-clustered 95% CI for pooled family accuracy was 99.853–99.982% across 144 dataset–subject clusters.Source: A. Terrain Reconstruction · p. 7
  • TERRA remained robust to threshold perturbations and landmark noise, while substantial support removal caused gradual, terrain-dependent degradation, especially for stools.Source: A. Terrain Reconstruction · p. 7

B. Motion Retargeting

TERRA’s retargeting improves physical validity and contact timing, and its reconstructed terrain further reduces several interaction violations compared with TERRA+Voronoi.

B. Motion Retargeting

  • TERRA achieved the best results on all five physical violation metrics and contact-timing F1.Source: B. Motion Retargeting · p. 7
  • Residual optimization raised contact-timing F1 from 88.49% to 93.80% and reduced terrain penetration from 33.45% to 0.16%.It also lowered all five violation metrics.Source: B. Motion Retargeting · p. 7
  • Compared with TERRA+Voronoi, TERRA terrain further reduced penetration, skating, and floating while increasing Contact F1.Source: B. Motion Retargeting · p. 8

C. Tracking Policy Performance

In held-out tracking, TERRA-retargeted references yielded the highest completion rate overall, with performance differences across terrain families and a trade-off against MPJPE.

C. Tracking Policy Performance

D. Physiological Evaluation

TERRA supports physiological comparisons between policy-generated and measured signals in matched locomotion examples, while its conclusion reports terrain-spanning control results and identifies open challenges.

D. Physiological Evaluation

  • Held-out flat, ramp, stair, and sit/stand examples showed similarities and visible mismatches between policy and human EMG timing and shape.Vertical-GRF panels compare stance timing and normalized waveform shape; the authors suggest these examples may support future study of physiological processes.Source: D. Physiological Evaluation · p. 8

VI. CONCLUSION

  • TERRA demonstrates that motion kinematics can provide enough environmental evidence to recover task-relevant collision geometry for muscle-actuated control.Source: §VI · p. 8
  • Across stairs, ramps, isolated supports, and chairs, TERRA reduced anatomical and interaction violations and yielded the highest held-out completion rate.Source: §VI · p. 8
  • Dynamic primitive geometry, online retargeting, and moving surfaces remain open challenges.The authors suggest the terrain-navigation repertoire may offer a route to studying affordance representation in embodied agents.Source: §VI · p. 8
Loading 2609.38653v1…