PaperlayerView exact revision on arXiv

Source-linked AI summary

Differentiable Contact Dynamics for Legged Locomotion: Policy Learning and Contact-Implicit Trajectory Optimization

Gijeong Kim, Seungbin Oh, Dongyun Kang, Hae-Won Park

arXiv:2610.35852v1cs.RO

TL;DR

Exact contact sensitivities provide derivatives only within the current mode, limiting first-order guidance toward neighboring modes. This paper relaxes the full frictional contact condition in the backward pass while preserving hard-contact rollouts. The resulting gradient supports policy learning and trajectory optimization, with reported hardware transfer and improved contact-rich optimization outcomes.

  • Problem

    Exact contact sensitivity holds the active contact mode fixed, providing no first-order direction toward neighboring modes and limiting contact-rich optimization.

    Source: §I · p. 1

  • Method

    The method applies a central-path relaxation of the full de Saxc´e-corrected frictional contact condition only in the backward pass, retaining the hard-contact forward solve.

    Source: §I · pp. 1–2

  • Results

    BPTT trained trotting and pacing policies from 100,000 simulator samples each in 108 s, transferred policies to hardware without fine-tuning, and reduced CITO costs across references while increasing recovered-pull limits.

    Source: §IX · p. 16

  • Takeaways & Limitations

    The proposed gradient supports hardware-transferred legged-locomotion policies and contact-rich trajectory optimization with contact-mode changes.

    Source: §IX · p. 16

Abstract

from arXiv · show

This paper proposes a smooth analytic gradient for frictional contact and applies it to policy learning and contact-implicit trajectory optimization for legged locomotion. The exact contact sensitivity is available in closed form but describes changes within the current contact mode, providing no first-order information toward neighboring contact modes. The proposed gradient replaces this sensitivity only in the backward pass, leaving the hard-contact forward solve unchanged and the realized rollout feasible. A central-path relaxation of the de Saxc\'e-corrected second-order-cone complementarity problem for Coulomb friction smooths the conditions defining the separating, clamping, and sliding modes. The relaxed sensitivity is evaluated at a point on the central path obtained by a Newton iteration initialized from the hard-contact solution. The sensitivity remains continuous through contact-mode transitions and converges to the exact sensitivity as the relaxation vanishes. Using 100,000 samples and less than 2 min of training on a desktop PC, policy learning with the proposed gradient produces a policy that transfers to hardware without fine-tuning. On hardware, the robot under a trotting policy withstands pushes and traverses debris and slippery ground. In trajectory optimization, the proposed gradient reaches contact-rich solutions missed by exact and Signorini-relaxed gradients. Project page: https://diffcontact.github.io

I. INTRODUCTION

Exact contact sensitivities describe changes within the current contact mode, limiting gradient-based learning and trajectory optimization when neighboring modes are needed. The paper relaxes the full frictional contact condition only in the backward pass, retaining hard-contact rollouts while guiding mode changes.

I. INTRODUCTION

  • The exact sensitivity is mode-restricted and discontinuous at active-set changes, limiting first-order guidance toward other modes in policy learning and contact-implicit optimization.This can keep trajectory search near the nominal contact sequence and make BPTT optimization fragile or ineffective.Source: §I · p. 1
  • The proposed gradient relaxes the full de Saxc´e-corrected frictional contact condition in the backward pass while retaining the hard-contact forward solve.The central-path relaxation couples normal and tangential sensitivities across separating, clamping, and sliding contacts.Source: §I · pp. 1–2
  • The method is tested in first-order policy learning and contact-implicit trajectory optimization against exact and Signorini-relaxed sensitivities with the forward model, objective, and initialization fixed.The experiments examine hardware transfer and contact-rich motion generation and recovery from external pulls.Source: §I · p. 2
  • Trotting policies trained from 100,000 samples in under 2 min are tested on hardware without fine-tuning under pushes, debris, and low-friction surfaces.The paper also demonstrates pacing and gait-free locomotion.Source: §I · p. 2

B. Differentiable simulation for policy learning

  • Unlike approaches that smooth forward contact or use a reduced-order surrogate, this method differentiates hard-contact articulated rollouts and reports hardware transfer with one rollout per update.The comparison includes prior approaches using 128 and 64 parallel simulation environments.Source: B. Differentiable simulation for… · p. 3

C. Contact-implicit trajectory optimization

Contact-implicit trajectory optimization jointly determines contact timing and location with the state and control trajectories. The section contrasts formulations that encode contact in different places and situates shooting-based methods within a shared finite-horizon optimal-control framework.

C. Contact-implicit trajectory optimization

  • Contact-implicit trajectory optimization determines contact timing and location jointly with state and control trajectories, resolving the discrete contact-mode structure embedded in its constraints.Direct transcription includes contact forces or impulses as decision variables, while shooting methods resolve contact at each simulation step and provide transition Jacobians.Source: C. Contact-implicit trajectory… · p. 3
  • Smoothed sensitivities can modify the contact model or its differentiation, but Signorini-only relaxation leaves the friction cone unrelaxed and provides no tangential direction from clamping toward slip.The proposed approach instead relaxes the full de Saxc´e-corrected frictional cone, with a separate Newton iteration setting the backward-pass relaxation point.Source: C. Contact-implicit trajectory… · p. 3

A. Optimal-control formulation

  • Trajectory optimization and first-order policy learning are finite-horizon optimal-control problems that both rely on transition Jacobians for backward updates.In shooting-based DDP/iLQR, controls form the decision variable and a forward rollout alternates with a backward Riccati sweep; BPTT instead differentiates a feedback policy through a closed-loop rollout.Source: A. Optimal-control formulation · pp. 3–4

B. Dynamics model and the frictional contact problem

The dynamics model represents intermittent legged-robot contact through impulses and a time-stepped rigid-body model. Contact-space velocity is related to impulse through a Delassus pair, and frictional point contact admits separating, clamping, and sliding modes.

B. Dynamics model and the frictional contact problem

  • The robot state comprises configuration and generalized velocity, with dynamics integrated at the velocity level using a semi-implicit Euler time step.The model includes inertia, nonlinear forces, actuation, contact Jacobians, and contact impulses.Source: B. Dynamics model and the frictional… · p. 4
  • The contact-space relation maps impulses to contact velocity through the Delassus matrix and free-contact term, with a gap correction added to the normal component to suppress contact drift.The normal contact velocity times the step size approximates the next-step signed gap.Source: B. Dynamics model and the frictional… · p. 4
  • The contact solution determines whether a contact is separating, clamping with an interior friction impulse, or sliding with the impulse on the friction-cone boundary.The active mode is not known in advance, giving the contact problem its combinatorial structure.Source: B. Dynamics model and the frictional… · p. 4

C. The de Saxc´e SOCCP and its Jordan-algebra form

The de Saxc´e correction recasts frictional contact as a conic complementarity condition. The section expresses that condition in a self-dual second-order cone form suitable for Jordan-algebra and path-following methods.

C. The de Saxc´e SOCCP and its Jordan-algebra form

  • The de Saxc´e correction offsets normal contact velocity by the dissipation prescribed by the maximum-dissipation principle, combining frictional contact laws into one conic complementarity condition.The complementarity pairs an admissible impulse with a corrected contact velocity.Source: C. The de Saxc´e SOCCP and its… · pp. 4–5
  • The formulation rescales the friction cone to a self-dual second-order cone so standard interior-point treatment can use the Jordan algebra.The rescaling is needed because the friction cone is self-dual only at unit friction.Source: C. The de Saxc´e SOCCP and its… · p. 5
  • The resulting SOCCP has separating, clamping, and sliding solution branches and can be solved by Jordan-based interior-point and path-following methods.The Jordan-product form preserves the primal–dual pairing of the original condition.Source: C. The de Saxc´e SOCCP and its… · p. 5

D. Contact impulse sensitivity

Mode-fixed contact derivatives can leave optimizers without directions toward neighboring modes, while the three point-mass tasks expose distinct normal and tangential transition requirements. The proposed full-cone gradient reaches the target in all three tasks, unlike exact and Signorini-relaxed gradients.

D. Contact impulse sensitivity

IV. PROBLEM SETTING

  • The point-mass examples compare contact relaxations using identical objectives, zero-control initialization, iLQR, and hard-contact forward dynamics.The MIQP optimizes the contact-mode sequence to zero optimality gap, and its cost normalizes the reported costs.Source: §IV · p. 5, A. Requirements on the backward… · p. 6

A. Requirements on the backward contact gradient

  • The proposed gradient reaches the target in all three CITO tasks; exact sensitivity reaches none, and Signorini-relaxed sensitivity succeeds only on lift-off.The tasks include lift-off, lateral traverse, and friction braking; the figure compares final normalized costs and contact-mode sequences against an MIQP reference.Source: Figure 1 · p. 5, A. Requirements on the backward… · p. 6
  • Across the tasks, covering normal and tangential contact-mode transitions requires relaxing the full frictional contact condition.Lift-off needs clamping-to-separation variation, traverse needs tangential variation, and braking needs coupled normal and tangential variation.Source: A. Requirements on the backward… · p. 6

V. ANALYTIC CONTACT SENSITIVITY VIA RELAXED DE SAXC´E CONIC COMPLEMENTARITY

The method relaxes de Saxcé-corrected frictional conic complementarity along a central path and differentiates that relation without fixing the contact mode. A Newton iteration initialized at the hard-contact solution supplies the evaluation point for the unified sensitivity.

V. ANALYTIC CONTACT SENSITIVITY VIA RELAXED DE SAXC´E CONIC COMPLEMENTARITY

  • The proposed sensitivity relaxes de Saxcé-corrected conic complementarity at ρ > 0, coupling normal and tangential derivatives without fixing the contact mode.The backward-pass relaxation applies across clamping, sliding, and separating contacts; the forward contact solve remains hard-contact.Source: §V · p. 6, A. Central-path relaxation and the… · p. 6
  • A Newton iteration initialized from the hard-contact solution computes a relaxed impulse–velocity pair satisfying relaxed complementarity and the Delassus relation.A normal free-velocity offset additionally adjusts clamping–separating sensitivity, and both modifications are restricted to backward sensitivity evaluation.Source: §V · p. 6

A. Central-path relaxation and the unified contact sensitivity

  • The central-path condition ˜λ ◦ ˜σ = ρe places both scaled variables inside their cones, and differentiating it yields a unified sensitivity equation.The equation combines the differentiated complementarity relation with dσ = A dλ + (dA)λ + db.Source: A. Central-path relaxation and the… · pp. 6–7
  • For multiple contacts, per-contact 3 × 3 blocks form a joint 3nc-dimensional system whose solution supplies contact-impulse sensitivities for computing fx and fu.The system is solved jointly across contacts.Source: A. Central-path relaxation and the… · p. 7

B. Evaluating the sensitivity at ρ > 0

Evaluating the contact sensitivity at ρ > 0 replaces fixed-mode boundary conditions with a unified relaxed system. This allows sensitivity components to remain nonzero across contact-mode transitions.

  • At ρ = 0, fixed-mode differentiation imposes conditions such as dσ = 0 in clamping and dλ = 0 in separating, giving no direction toward a mode change.Source: B. Evaluating the sensitivity at ρ > 0 · p. 8
  • For ρ > 0, the relaxed pair lies in the cone interiors and the unified linearization permits sensitivity components that are zero under exact mode constraints to become nonzero.Source: B. Evaluating the sensitivity at ρ > 0 · pp. 7–8
  • Across the plotted contact-mode transitions, proposed sensitivities vary continuously while exact sensitivities change discontinuously, with formerly constrained components becoming nonzero.The sweep also shows nonzero tangential sensitivities for a normal perturbation.Source: B. Evaluating the sensitivity at ρ > 0 · p. 8
  • A Newton iteration initialized from the hard-contact solution computes a pair satisfying both the relaxed complementarity and the Delassus relation.The Delassus relation is maintained at every iterate.Source: B. Evaluating the sensitivity at ρ > 0 · p. 8

C. Normal free-velocity offset

The normal free-velocity offset modifies the relaxed evaluation point and can improve Newton conditioning while increasing sensitivity toward contact release. As ρ vanishes with δ = 0, the proposed sensitivity recovers the exact fixed-mode sensitivity.

C. Normal free-velocity offset

  • The offset adds a positive diagonal contribution to the Newton matrix and is observed to lower convergence to spurious roots.Source: C. Normal free-velocity offset · p. 9
  • Increasing δ shifts the normal sensitivity curve toward more negative unshifted bN, increasing sensitivity toward contact release for clamped forward states.This sensitivity can guide trajectory optimization toward solutions involving liftoff.Source: C. Normal free-velocity offset · p. 9

D. Recovery of the fixed-mode sensitivity as ρ →0

  • With δ = 0, the proposed sensitivity converges to the exact fixed-mode sensitivity as ρ → 0+, recovering the separating and clamping conditions.Source: D. Recovery of the fixed-mode… · p. 9
  • For sliding contact, the limiting conditions coincide with the differentiated Signorini and maximum-dissipation conditions, recovering the exact sliding sensitivity.Source: D. Recovery of the fixed-mode… · p. 9

VI. IMPLEMENTATION

The implementation uses relaxed contact sensitivities to assemble transition Jacobians for BPTT and DDP while retaining the hard-contact forward solve. The experiments use a quadruped setup, with policy learning trained on short rollout windows.

A. Transition-Jacobian assembly

  • The proposed sensitivity supplies contact derivatives for transition Jacobians, while other rigid-body and contact-geometry derivatives are computed analytically.The latter include derivatives of the contact Jacobian and the position-level correction.Source: A. Transition-Jacobian assembly · p. 9

B. Optimal-control setup

  • The 45-kg quadruped uses a 37-dimensional state, 12 torque inputs, four point feet, Coulomb friction, and a 25-ms control step.The hard-contact forward solve is retained; relaxation applies only to backward contact sensitivity.Source: B. Optimal-control setup · p. 9
  • The regulation reference does not prescribe a contact mode or sequence, and policy-learning costs include foot-position, gait, and action-smoothness terms.Gait tracking is weighted zero in gait-free tasks.Source: B. Optimal-control setup · p. 10

C. Policy-learning implementation

  • Policy learning uses 20-control-step differentiation windows and continues each training window from the preceding terminal state until episode termination.The policy is a [256, 128] MLP trained on a desktop CPU.Source: C. Policy-learning implementation · p. 10

D. Trajectory optimization implementation

The policy-learning experiments compare backward contact gradients across four locomotion tasks and test whether single-environment BPTT policies transfer to hardware. The proposed gradient learns gait-free, trotting, and pacing policies, which are deployed without fine-tuning.

D. Trajectory optimization implementation

  • CITO uses a 20-step, 0.5-s Box-FDDP horizon and receding-horizon solves; the reference specifies base pose while nominal joint configuration is retained.Contact modes, impulses, and foothold placements are determined by the optimized trajectory rather than preplanned.Source: D. Trajectory optimization… · p. 10, §VII · p. 10

A. Training across contact gradients

  • The proposed gradient alone learns both gait-free tasks; exact and Signorini-relaxed gradients plateau with high costs and near-universal falls.Neither baseline provides tangential contact-velocity variation at clamped contacts, which is needed for these tasks.Source: A. Training across contact gradients · p. 11, Figure 6 · p. 11
  • For gait-free locomotion at low friction, more frequent sliding in the rollout does not make either baseline learn locomotion.The proposed gradient supplies sensitivity across contact-mode transitions, enabling gait-free locomotion.Source: A. Training across contact gradients · p. 11, Figure 6 · p. 11
  • The proposed gradient learns trot and pace with the lowest median evaluation costs; Signorini relaxation learns both gaits but has higher slippery-ground fall rates.The prescribed phase and clearance objective supplies liftoff timing, while the proposed gradient also provides a direction toward clamping-to-sliding transitions.Source: A. Training across contact gradients · pp. 11–12, Figure 6 · p. 11

B. Hardware demonstration

  • Trotting and pacing policies trained with 100,000 samples in 108 s, and the gait-free policy trained with 160,000 samples in 172 s, transfer to hardware without fine-tuning.The hardware trials track commanded velocities for trot and pace, and demonstrate gait-free locomotion without a prescribed gait.Source: Figure 7 · p. 11, B. Hardware demonstration · p. 12
  • On hardware, the trotting policy traverses scattered debris, recovers from pushes across 14 trials, and continues locomotion after slips on a surface with µ ≈ 0.25.The low-friction test surface was below the policy's training range of µ ∈ [0.4, 1.0].Source: B. Hardware demonstration · p. 12, Figure 8 · p. 12

C. Comparison with Model-Free RL

On trot tracking, BPTT with the proposed gradient learns with fewer environments and less training time than PPO in the reported comparisons. The section also reports Newton-solver behavior and lower CITO costs for translation and yaw reaching with the proposed gradient.

C. Comparison with Model-Free RL

D. Numerical Behavior of the Newton Iteration

  • 99.95% of backward contact evaluations meet the residual tolerance within eight Newton iterations; with both regularization and offset enabled, spurious convergence is 0.0058%.Spurious convergence means residual convergence that fails the cone-feasibility check; such a failure blocks gradient propagation through that transition.Source: D. Numerical Behavior of the Newton… · pp. 13–14, Figure 11 · p. 14

A. Base-Pose Reaching Without Prescribed Contact Modes

  • The proposed gradient attains the lowest costs throughout the Translation and Yaw rotation tasks.The comparison evaluates successive finite-horizon objectives over reference translations and yaw rotations.Source: A. Base-Pose Reaching Without… · p. 14, Figure 12 · p. 15
  • For a 1.2-rad pitch reference, the proposed trajectory keeps pitch within 0.91–1.34 rad versus 0.77–1.50 rad for Signorini relaxation, and reduces peak-to-peak base displacement from 1.19 to 0.60 m.The proposed sensitivity guides rear-foot separation and front-foot adjustment for balance.Source: A. Base-Pose Reaching Without… · p. 14, Figure 12 · p. 15

B. Analysis of a Single CITO Problem

In a shared CITO initialization, the proposed gradient repositions a loaded foot and reduces base roll, unlike the exact and Signorini-relaxed gradients. In pull-recovery simulations, it tolerates larger pulls, and the conclusion identifies extensions still needed.

B. Analysis of a Single CITO Problem

  • After 25 DDP iterations, the proposed gradient lowers cost from 116 to 39 as the loaded rear-right foot slides about 0.4 m, repositions, and the planned base roll approaches zero.With the exact and Signorini-relaxed gradients, the foot remains clamped, lateral displacement stays zero, and costs reach only 98 and 95.Source: B. Analysis of a Single CITO Problem · p. 15

C. Push Recovery under Unmodeled Pulls

  • Pull limits with the proposed gradient range from 240 to 350 N and average 272 N, 3.4 times the exact-gradient average and 2.5 times the Signorini-relaxed average.For a 250-N forward pull, planned front-foot sliding lets the feet advance with the base; the rollout completes without failure.Source: C. Push Recovery under Unmodeled Pulls · pp. 15–16, Figure 14 · p. 16

IX. CONCLUSION

  • The method's demonstrated scope leaves non-point feet, uneven terrain, and contacts between arbitrary shapes for future work; those extensions require differentiating through collision detection.The conclusion also notes that the present 0.5-s BPTT rollouts could be augmented with a learned terminal value for longer-horizon policy learning.Source: §IX · p. 16

APPENDIX A JORDAN PRODUCT AND ARROW MATRIX

The appendix introduces Jordan-product notation for point contacts, gives the Signorini-relaxed contact-impulse sensitivity, and points to the PPO settings table. The supplied passages provide only limited detail about the table's contents.

APPENDIX A JORDAN PRODUCT AND ARROW MATRIX

  • For a point contact, the appendix sets d = 3 and introduces the Jordan product and its symmetric arrow-matrix representation.The displayed product is expressed as w ◦ z = L(w)z = L(z)w.Source: Appendix A · p. 16

APPENDIX B SIGNORINI-RELAXED CONTACT-IMPULSE SENSITIVITY

  • The Signorini-relaxed sensitivity relaxes normal complementarity while retaining unrelaxed tangential relations, yielding the contact-impulse sensitivity in [15, Eq. 19].Its differential system has Mλ = ρD and Mσ = I.Source: Appendix B · p. 16

APPENDIX C PPO CONFIGURATION

  • The PPO configuration is summarized in Table II; PPO bootstraps each 20-step rollout segment's return using a learned critic.The environment, observation, action, per-step cost, and episode handling follow the BPTT setup.Source: Appendix C · p. 17, Table II · p. 17

Related papers

Loading 2610.35852v1…