PaperlayerView exact revision on arXiv

Source-linked AI summary

Feedback MPC for Torque-Controlled Legged Robots

Ruben Grandia, Farbod Farshidian, René Ranftl, Marco Hutter

arXiv:1905.06144v2cs.RO

TL;DR

Torque-level whole-body MPC for legged robots cannot update quickly enough on computationally limited mobile platforms, motivating separate fast tracking controllers. The paper instead deploys a DDP-based MPC feedback policy directly, augmenting it with relaxed friction-cone barriers and frequency-aware costs. The approach produces stable locomotion policies for torque-controlled ANYmal in simulation and on hardware, including at 15 Hz MPC updates.

  • Problem

    Mobile robots lack sufficient computational power for torque-level whole-body MPC at update rates needed by complex legged systems.

    Source: §I · p. 1

  • Method

    The paper deploys a DDP-based MPC feedback policy directly, extending SLQ with relaxed barrier functions for friction-cone constraints and frequency-aware costs for actuator bandwidth limits.

    Source: §I · p. 1, §V · p. 8

  • Results

    The approach finds stable locomotion policies for torque-controlled ANYmal in simulation and on hardware, with stable locomotion demonstrated at 15 Hz MPC updates.

    Source: §I · p. 1

  • Takeaways & Limitations

    The optimized feedback policy can bridge low-rate MPC updates and high-rate torque execution without a separate motion controller.

    Source: §I · p. 1

Abstract

from arXiv · show

The computational power of mobile robots is currently insufficient to achieve torque level whole-body Model Predictive Control (MPC) at the update rates required for complex dynamic systems such as legged robots. This problem is commonly circumvented by using a fast tracking controller to compensate for model errors between updates. In this work, we show that the feedback policy from a Differential Dynamic Programming (DDP) based MPC algorithm is a viable alternative to bridge the gap between the low MPC update rate and the actuation command rate. We propose to augment the DDP approach with a relaxed barrier function to address inequality constraints arising from the friction cone. A frequency-dependent cost function is used to reduce the sensitivity to high-frequency model errors and actuator bandwidth limits. We demonstrate that our approach can find stable locomotion policies for the torque-controlled quadruped, ANYmal, both in simulation and on hardware.

I. INTRODUCTION

Torque-level whole-body MPC for legged robots is limited by mobile platforms’ computational power and low update rates. The paper proposes deploying the MPC’s optimized feedback policy directly, using constrained SLQ, frequency-aware costs, and relaxed barriers to achieve stable locomotion at 15 Hz.

  • High-dimensional legged systems cannot update MPC fast enough on mobile platforms to handle model uncertainty and external disturbances effectively.Source: §I · p. 1
  • A separately designed motion tracker is commonly used to provide higher-rate feedback corrections, but it cannot anticipate contact-configuration changes.Source: §I · p. 1
  • The proposed feedback MPC directly deploys its optimized policy on hardware, eliminating the need for a separate motion controller and maintaining consistency across updates.Source: §I · p. 1
  • The constrained SLQ-based policy combines a time-varying LQR correction with relaxed barrier functions for locomotion constraints, including friction-cone constraints.Source: §I · pp. 1–2
  • 15 Hz update rates yielded robust and stable locomotion policies in simulation and on a real legged system, supporting onboard execution on computationally limited mobile platforms.Source: §I · p. 2

A. Related Work

Prior approaches either use open-loop or separately designed feedback policies that can be conservative, poorly scalable, or inconsistent with MPC trajectories. This work applies feedback MPC directly to whole-body legged-robot control and validates constraint-aware, bandwidth-aware gains on ANYmal.

  • Min-max MPC can be overly conservative, while optimizing feedback policies for all disturbance realizations does not scale to legged-robot dimensions.Source: A. Related Work · p. 2
  • DDP-based SLQ computes feedback policies efficiently using a Gauss-Newton approximation that considers linearized rather than second-order dynamics.Source: A. Related Work · p. 2
  • Directly using DDP-derived LQR gains can fail on real torque-controlled hardware when low-level controller bandwidth limitations are neglected.Source: A. Related Work · p. 2
  • The proposed whole-body MPC computes actuation commands directly from the MPC feedback policy and is empirically validated on ANYmal in simulation and hardware.Source: B. Contributions · p. 2
  • A barrier-function extension incorporates friction-cone inequality constraints and designs constraint-satisfactory LQR gains without additional computational cost.Source: B. Contributions · p. 2
  • Frequency-domain design incorporates actuator bandwidth limits to avoid overly stiff gains, allowing feedback gains to be applied directly to the robot.Source: B. Contributions · p. 2
  • The feedback MPC bridges low-rate MPC updates and high-rate torque-command execution using only an onboard computer with moderate computational power.Source: B. Contributions · p. 2

II. METHOD

The method formulates nonlinear constrained optimal control and solves it with an SLQ feedback-policy procedure. Forward rollouts and backward Riccati-based updates iteratively improve a state-affine policy, extended here with relaxed barrier functions.

  • The nonlinear optimal-control problem minimizes running and terminal costs subject to system dynamics, initial conditions, and equality and inequality constraints.Source: A. Problem Definition · p. 2
  • The DDP-based SLQ method computes a time-varying state-affine control policy from a quadratic approximation of the optimal value function.Source: A. Problem Definition · pp. 2–3
  • The method handles equality constraints with a Lagrangian approach and pure state constraints with a penalty method.Source: A. Problem Definition · p. 3
  • SLQ alternates forward simulation with backward optimization, constructing a local linear-quadratic model and solving its Riccati differential equation.Source: A. Problem Definition · p. 3
  • SLQ is extended with a relaxed barrier function to incorporate inequality constraints while accommodating rollout behavior outside the feasible region.Source: A. Problem Definition · p. 3

B. Relaxed Barrier Functions

The paper replaces the standard logarithmic barrier with a relaxed barrier that remains usable outside the feasible region and limits curvature near constraint boundaries. This preserves a tractable LQ approximation while retaining the log-barrier behavior farther from the boundary.

  • Barrier terms absorb inequality constraints into the cost, allowing constrained optimization within the SLQ formulation.Source: B. Relaxed Barrier Functions · p. 3
  • The logarithmic barrier is undefined outside the feasible region and becomes ill-conditioned near the boundary because its Hessian diverges.Source: B. Relaxed Barrier Functions · p. 3
  • The relaxed barrier switches from a logarithmic form to another function at distance δ from the constraint boundary.Source: B. Relaxed Barrier Functions · p. 3
  • The quadratic extension is continuous and twice differentiable, and its curvature is upper-bounded to prevent ill-conditioning of the LQ approximation.Source: B. Relaxed Barrier Functions · p. 3
  • As δ →0, the relaxed barrier recovers the standard logarithmic barrier; for h < δ it acts as an augmented-Lagrangian penalty, while for h ≥δ it acts as a log barrier.Source: B. Relaxed Barrier Functions · p. 3

C. LQ Approximation

The method linearizes the dynamics and constraints around the previous trajectory, forms a quadratic cost approximation, and obtains a constrained feedback policy through a Riccati backward pass. Frequency-dependent input shaping is incorporated so the feedback structure penalizes high-frequency actuation.

LQ Approximation

  • The dynamics and state-input constraints are linearized around a previous state and input trajectory using deviations δx and δu.Source: C. LQ Approximation · p. 3
  • The quadratic cost approximation requires second-order approximations of the barrier terms and inequality constraints.Source: C. LQ Approximation · p. 3
  • A constrained Riccati backward pass produces a quadratic value function that induces the optimal feedback policy.Source: C. LQ Approximation · p. 3
  • The feedback gains satisfy equality constraints by projecting corrections into the constraint nullspace and adding a pseudoinverse term for state deviations.Source: C. LQ Approximation · p. 3

Frequency Shaping

  • The frequency-dependent cost is evaluated on auxiliary inputs generated by shaping functions, with high-pass filters increasing costs at higher input frequencies.Source: D. Frequency Shaping · p. 4
  • The original system is augmented with filter states, optimization is performed over auxiliary inputs, and the original input is recovered through the filter output policy.Source: D. Frequency Shaping · p. 4

III. IMPLEMENTATION

The implementation applies the approach to a quadruped kinodynamic model representing a free-floating body together with the kinematics of each leg. Its equations of motion use base orientation, center-of-mass position, angular velocity, linear velocity, and contact forces.

  • The robot is modeled using quadruped kinodynamics: a single free-floating body coupled with the kinematics of each leg.Source: §III · p. 4
  • The equations of motion describe base orientation, center-of-mass position, angular and linear velocity, and leg contact-force contributions.Source: §III · p. 4
  • The model uses the base rotation matrix, angular-velocity transformation, gravity, inertia, total mass, and foot positions relative to the center of mass.Source: §III · p. 4

A. Equality Constraints

The locomotion formulation combines time-dependent equality constraints for stance and swing legs with a relaxed friction-cone inequality constraint. A perturbed cone is used because it is differentiable at the origin, excludes negative normal forces, and conservatively lies inside the original feasible cone.

Equality Constraints

  • Stance legs are constrained to remain on the ground, while swing legs follow a prescribed surface-normal trajectory and exert zero contact force.Source: A. Equality Constraints · p. 4

Inequality Constraints

  • The friction cone is represented directly as a second-order cone in local contact forces rather than through a polytope approximation.Source: B. Inequality Constraints · p. 4
  • The original cone constraint has an undefined gradient at zero contact force, creating numerical problems near the origin.Source: B. Inequality Constraints · p. 4
  • Squaring the cone constraint is unsuitable with the relaxed barrier because it creates a saddle point at the origin and makes the negative cone reflection feasible.Source: B. Inequality Constraints · pp. 4–5
  • Figure 3 compares the perturbed-cone level sets with the original cone and shows a convex constraint whose zero crossing lies strictly inside the feasible region.Source: Figure 3 · p. 5, B. Inequality Constraints · p. 5
  • The perturbed cone is differentiable at the origin, infeasible for negative Fz, and a conservative lower bound for the original cone.Source: B. Inequality Constraints · p. 5

C. Torque Computation

The torque-computation pipeline translates feedback-policy commands into desired torques, while experiments assess the proposed method and its implementation for onboard execution.

Torque computation

  • The control inputs are contact forces and joint velocities that must be translated into torques.Feedforward trajectories are tracked through desired accelerations and contact forces, whereas feedback policies are briefly forward-simulated before extracting desired accelerations.Source: C. Torque Computation · p. 5

Motor control

  • Each motor uses an embedded local control loop, with torque control for stance legs and position-velocity feedback for swing legs.Source: C. Torque Computation · p. 5

Experiments

  • The experiments compare feedback and feedforward policies under low update rates and disturbances, then examine inequality constraints and frequency shaping.Source: §IV · p. 5

Hardware execution

  • The combined method is evaluated for successful execution on the onboard hardware of a torque-controlled robot.Source: §IV · p. 5

Cost settings

  • Frequency shaping uses α_i = 0.01 with β_i = 0.2 for contact-force inputs and β_i = 0.1 for joint-velocity inputs.A diagonal cost on state and control inputs is used in all experiments.Source: §IV · p. 5

A. Feedback MPC

Feedback MPC reduces update-induced discontinuities at low rates, respects friction-cone constraints through gradual gain changes, and shapes feedback to preserve constraints while limiting direct high-frequency gains.

A. Feedback MPC

  • At 20 Hz, SLQ feedback MPC produces smooth desired accelerations comparable to 400 Hz feedforward MPC, unlike low-rate feedforward MPC's visible update discontinuities.The low-rate feedforward trajectory resets accumulated deviation at each update, whereas feedback MPC significantly reduces those discontinuities.Source: A. Feedback MPC · p. 5
  • Feedback from MPC recovers some performance lost at lower update rates by using a policy consistent with future updates.The reduction in discontinuities holds within the validity of the linear quadratic approximation.Source: A. Feedback MPC · p. 5

B. Feedback Gains Near Inequality Constraints

  • With inequality constraints, contact forces remain positive and feedback gains decrease as the right-hind contact force approaches zero.The experiment uses barrier parameters μ = 0.5 and δ = 0.1, under which all constraints are strictly satisfied.Source: B. Feedback Gains Near Inequality… · pp. 5–6
  • Barrier-based gains decrease gradually near active constraints, unlike clamping's unchanged gains or active-set methods' instantaneous drop to zero.Source: B. Feedback Gains Near Inequality… · p. 6
  • Before lifting the left-front leg, its feedback gains increase because the body position becomes important while right-hind feedback is reduced for the upcoming phase.Source: B. Feedback Gains Near Inequality… · p. 6
  • Finite barrier scaling μ trades some optimality for a larger stability margin by regulating distance to the constraint boundary.Source: B. Feedback Gains Near Inequality… · p. 6

C. Feedback Structure

  • The feedback matrix encodes zero end-effector velocity through strong dependence of joint-velocity commands on base linear and angular velocities.This empirically verifies that the feedback matrices are consistent with the equality constraints.Source: C. Feedback Structure · p. 6
  • Frequency shaping routes feedback mainly from system states through auxiliary inputs and filter states, reducing direct gains while preserving equality-constraint structure.Joint-velocity feedback from filter states is zero, while the matrix retains terms needed to enforce zero end-effector velocity.Source: C. Feedback Structure · pp. 6–7

D. Hardware Experiments: Disturbance Rejection

Hardware disturbance experiments compare conventional tracking feedback with frequency-shaped SLQ feedback under a constant 5.7 kg load, showing the practical difference between the policies.

Disturbance setup

  • A 5.7 kg mass, approximately 15% of ANYmal's total mass, is placed on the standing robot as a constant disturbance.The comparison examines desired and measured left-front-knee torques after the system reaches equilibrium.Source: D. Hardware Experiments: Disturbance… · p. 7

Disturbance rejection

E. Hardware Experiments: Dynamic Walking

The proposed controller achieves stable dynamic walking on ANYmal using onboard computation at approximately 15 Hz, while respecting contact and actuator constraints.

Dynamic walking

Hardware results

  • Approximately 15 Hz MPC updates are achieved with a 1.0 s receding horizon, and desired and measured torque and joint-velocity signals remain close throughout three walking cycles.The results show that the feedback policy respects actuator bandwidth limits; pace and trot motions are also feasible.Source: E. Hardware Experiments: Dynamic Walking · p. 7, Figure 9 · p. 8

Conclusion

  • Stable walking is possible with low update rates despite mobile-platform computational restrictions.Source: §V · pp. 7–8
  • The relaxed barrier method extends SLQ to inequality-constrained optimization, implementing the friction cone through a perturbed second-order cone constraint.It adds a convex cost penalty and avoids numerical ill-conditioning at the cone origin.Source: §V · p. 8
  • Frequency-aware MPC attenuates high-frequency gains while preserving low-frequency gains, supporting closed-loop hardware stability and constraint-consistent feedback.The policy also reduces gains near friction-cone boundaries, sets zero swing-leg contact-force gains, and encodes zero stance-leg end-effector velocity.Source: §V · p. 8

Related papers

Loading 1905.06144v2…