Source-linked AI summary
Feedback MPC for Torque-Controlled Legged Robots
Ruben Grandia, Farbod Farshidian, René Ranftl, Marco Hutter
TL;DR
Torque-level whole-body MPC for legged robots cannot update quickly enough on computationally limited mobile platforms, motivating separate fast tracking controllers. The paper instead deploys a DDP-based MPC feedback policy directly, augmenting it with relaxed friction-cone barriers and frequency-aware costs. The approach produces stable locomotion policies for torque-controlled ANYmal in simulation and on hardware, including at 15 Hz MPC updates.
Problem
Mobile robots lack sufficient computational power for torque-level whole-body MPC at update rates needed by complex legged systems.
Method
The paper deploys a DDP-based MPC feedback policy directly, extending SLQ with relaxed barrier functions for friction-cone constraints and frequency-aware costs for actuator bandwidth limits.
Results
The approach finds stable locomotion policies for torque-controlled ANYmal in simulation and on hardware, with stable locomotion demonstrated at 15 Hz MPC updates.
Takeaways & Limitations
The optimized feedback policy can bridge low-rate MPC updates and high-rate torque execution without a separate motion controller.
Abstract
from arXiv · showhide
The computational power of mobile robots is currently insufficient to achieve torque level whole-body Model Predictive Control (MPC) at the update rates required for complex dynamic systems such as legged robots. This problem is commonly circumvented by using a fast tracking controller to compensate for model errors between updates. In this work, we show that the feedback policy from a Differential Dynamic Programming (DDP) based MPC algorithm is a viable alternative to bridge the gap between the low MPC update rate and the actuation command rate. We propose to augment the DDP approach with a relaxed barrier function to address inequality constraints arising from the friction cone. A frequency-dependent cost function is used to reduce the sensitivity to high-frequency model errors and actuator bandwidth limits. We demonstrate that our approach can find stable locomotion policies for the torque-controlled quadruped, ANYmal, both in simulation and on hardware.
I. INTRODUCTION
Torque-level whole-body MPC for legged robots is limited by mobile platforms’ computational power and low update rates. The paper proposes deploying the MPC’s optimized feedback policy directly, using constrained SLQ, frequency-aware costs, and relaxed barriers to achieve stable locomotion at 15 Hz.
- High-dimensional legged systems cannot update MPC fast enough on mobile platforms to handle model uncertainty and external disturbances effectively.
- A separately designed motion tracker is commonly used to provide higher-rate feedback corrections, but it cannot anticipate contact-configuration changes.
- The proposed feedback MPC directly deploys its optimized policy on hardware, eliminating the need for a separate motion controller and maintaining consistency across updates.
- The constrained SLQ-based policy combines a time-varying LQR correction with relaxed barrier functions for locomotion constraints, including friction-cone constraints.
- 15 Hz update rates yielded robust and stable locomotion policies in simulation and on a real legged system, supporting onboard execution on computationally limited mobile platforms.
A. Related Work
Prior approaches either use open-loop or separately designed feedback policies that can be conservative, poorly scalable, or inconsistent with MPC trajectories. This work applies feedback MPC directly to whole-body legged-robot control and validates constraint-aware, bandwidth-aware gains on ANYmal.
- Min-max MPC can be overly conservative, while optimizing feedback policies for all disturbance realizations does not scale to legged-robot dimensions.
- DDP-based SLQ computes feedback policies efficiently using a Gauss-Newton approximation that considers linearized rather than second-order dynamics.
- Directly using DDP-derived LQR gains can fail on real torque-controlled hardware when low-level controller bandwidth limitations are neglected.
- The proposed whole-body MPC computes actuation commands directly from the MPC feedback policy and is empirically validated on ANYmal in simulation and hardware.
- A barrier-function extension incorporates friction-cone inequality constraints and designs constraint-satisfactory LQR gains without additional computational cost.
- Frequency-domain design incorporates actuator bandwidth limits to avoid overly stiff gains, allowing feedback gains to be applied directly to the robot.
- The feedback MPC bridges low-rate MPC updates and high-rate torque-command execution using only an onboard computer with moderate computational power.
II. METHOD
The method formulates nonlinear constrained optimal control and solves it with an SLQ feedback-policy procedure. Forward rollouts and backward Riccati-based updates iteratively improve a state-affine policy, extended here with relaxed barrier functions.
- The nonlinear optimal-control problem minimizes running and terminal costs subject to system dynamics, initial conditions, and equality and inequality constraints.
- The DDP-based SLQ method computes a time-varying state-affine control policy from a quadratic approximation of the optimal value function.
- The method handles equality constraints with a Lagrangian approach and pure state constraints with a penalty method.
- SLQ alternates forward simulation with backward optimization, constructing a local linear-quadratic model and solving its Riccati differential equation.
- SLQ is extended with a relaxed barrier function to incorporate inequality constraints while accommodating rollout behavior outside the feasible region.
B. Relaxed Barrier Functions
The paper replaces the standard logarithmic barrier with a relaxed barrier that remains usable outside the feasible region and limits curvature near constraint boundaries. This preserves a tractable LQ approximation while retaining the log-barrier behavior farther from the boundary.
- Barrier terms absorb inequality constraints into the cost, allowing constrained optimization within the SLQ formulation.
- The logarithmic barrier is undefined outside the feasible region and becomes ill-conditioned near the boundary because its Hessian diverges.
- The relaxed barrier switches from a logarithmic form to another function at distance δ from the constraint boundary.
- The quadratic extension is continuous and twice differentiable, and its curvature is upper-bounded to prevent ill-conditioning of the LQ approximation.
- As δ →0, the relaxed barrier recovers the standard logarithmic barrier; for h < δ it acts as an augmented-Lagrangian penalty, while for h ≥δ it acts as a log barrier.
C. LQ Approximation
The method linearizes the dynamics and constraints around the previous trajectory, forms a quadratic cost approximation, and obtains a constrained feedback policy through a Riccati backward pass. Frequency-dependent input shaping is incorporated so the feedback structure penalizes high-frequency actuation.
- LQ Approximation: The dynamics and state-input constraints are linearized around a previous state and input trajectory using deviations δx and δu.
- LQ Approximation: The quadratic cost approximation requires second-order approximations of the barrier terms and inequality constraints.
- LQ Approximation: A constrained Riccati backward pass produces a quadratic value function that induces the optimal feedback policy.
- LQ Approximation: The feedback gains satisfy equality constraints by projecting corrections into the constraint nullspace and adding a pseudoinverse term for state deviations.
- Frequency Shaping: The frequency-dependent cost is evaluated on auxiliary inputs generated by shaping functions, with high-pass filters increasing costs at higher input frequencies.
- Frequency Shaping: The original system is augmented with filter states, optimization is performed over auxiliary inputs, and the original input is recovered through the filter output policy.
III. IMPLEMENTATION
The implementation applies the approach to a quadruped kinodynamic model representing a free-floating body together with the kinematics of each leg. Its equations of motion use base orientation, center-of-mass position, angular velocity, linear velocity, and contact forces.
- The robot is modeled using quadruped kinodynamics: a single free-floating body coupled with the kinematics of each leg.
- The equations of motion describe base orientation, center-of-mass position, angular and linear velocity, and leg contact-force contributions.
- The model uses the base rotation matrix, angular-velocity transformation, gravity, inertia, total mass, and foot positions relative to the center of mass.
A. Equality Constraints
The locomotion formulation combines time-dependent equality constraints for stance and swing legs with a relaxed friction-cone inequality constraint. A perturbed cone is used because it is differentiable at the origin, excludes negative normal forces, and conservatively lies inside the original feasible cone.
- Equality Constraints: Stance legs are constrained to remain on the ground, while swing legs follow a prescribed surface-normal trajectory and exert zero contact force.
- Inequality Constraints: The friction cone is represented directly as a second-order cone in local contact forces rather than through a polytope approximation.
- Inequality Constraints: The original cone constraint has an undefined gradient at zero contact force, creating numerical problems near the origin.
- Inequality Constraints: Squaring the cone constraint is unsuitable with the relaxed barrier because it creates a saddle point at the origin and makes the negative cone reflection feasible.
- Inequality Constraints: Figure 3 compares the perturbed-cone level sets with the original cone and shows a convex constraint whose zero crossing lies strictly inside the feasible region.
- Inequality Constraints: The perturbed cone is differentiable at the origin, infeasible for negative Fz, and a conservative lower bound for the original cone.
C. Torque Computation
The torque-computation pipeline translates feedback-policy commands into desired torques, while experiments assess the proposed method and its implementation for onboard execution.
- Torque computation: The control inputs are contact forces and joint velocities that must be translated into torques.Feedforward trajectories are tracked through desired accelerations and contact forces, whereas feedback policies are briefly forward-simulated before extracting desired accelerations.
- Motor control: Each motor uses an embedded local control loop, with torque control for stance legs and position-velocity feedback for swing legs.
- Experiments: The experiments compare feedback and feedforward policies under low update rates and disturbances, then examine inequality constraints and frequency shaping.
- Hardware execution: The combined method is evaluated for successful execution on the onboard hardware of a torque-controlled robot.
- Cost settings: Frequency shaping uses α_i = 0.01 with β_i = 0.2 for contact-force inputs and β_i = 0.1 for joint-velocity inputs.A diagonal cost on state and control inputs is used in all experiments.
A. Feedback MPC
Feedback MPC reduces update-induced discontinuities at low rates, respects friction-cone constraints through gradual gain changes, and shapes feedback to preserve constraints while limiting direct high-frequency gains.
- A. Feedback MPC: At 20 Hz, SLQ feedback MPC produces smooth desired accelerations comparable to 400 Hz feedforward MPC, unlike low-rate feedforward MPC's visible update discontinuities.The low-rate feedforward trajectory resets accumulated deviation at each update, whereas feedback MPC significantly reduces those discontinuities.
- A. Feedback MPC: Feedback from MPC recovers some performance lost at lower update rates by using a policy consistent with future updates.The reduction in discontinuities holds within the validity of the linear quadratic approximation.
- B. Feedback Gains Near Inequality Constraints: With inequality constraints, contact forces remain positive and feedback gains decrease as the right-hind contact force approaches zero.The experiment uses barrier parameters μ = 0.5 and δ = 0.1, under which all constraints are strictly satisfied.
- B. Feedback Gains Near Inequality Constraints: Barrier-based gains decrease gradually near active constraints, unlike clamping's unchanged gains or active-set methods' instantaneous drop to zero.
- B. Feedback Gains Near Inequality Constraints: Before lifting the left-front leg, its feedback gains increase because the body position becomes important while right-hind feedback is reduced for the upcoming phase.
- B. Feedback Gains Near Inequality Constraints: Finite barrier scaling μ trades some optimality for a larger stability margin by regulating distance to the constraint boundary.
- C. Feedback Structure: The feedback matrix encodes zero end-effector velocity through strong dependence of joint-velocity commands on base linear and angular velocities.This empirically verifies that the feedback matrices are consistent with the equality constraints.
- C. Feedback Structure: Frequency shaping routes feedback mainly from system states through auxiliary inputs and filter states, reducing direct gains while preserving equality-constraint structure.Joint-velocity feedback from filter states is zero, while the matrix retains terms needed to enforce zero end-effector velocity.
D. Hardware Experiments: Disturbance Rejection
Hardware disturbance experiments compare conventional tracking feedback with frequency-shaped SLQ feedback under a constant 5.7 kg load, showing the practical difference between the policies.
- Disturbance setup: A 5.7 kg mass, approximately 15% of ANYmal's total mass, is placed on the standing robot as a constant disturbance.The comparison examines desired and measured left-front-knee torques after the system reaches equilibrium.
- Disturbance rejection: Under feedforward MPC, each update resets the reference to the measured state, nullifying tracking-controller feedback and allowing disturbance-induced deviation to accumulate.
E. Hardware Experiments: Dynamic Walking
The proposed controller achieves stable dynamic walking on ANYmal using onboard computation at approximately 15 Hz, while respecting contact and actuator constraints.
- Dynamic walking: Dynamic walk combines underactuated two-foot and overactuated three-foot contact configurations.The gait pattern marks the intervals during which each leg is in contact.
- Dynamic walking: The friction-cone constraint prevents negative contact forces while the robot navigates changing support polygons.
- Hardware results: Approximately 15 Hz MPC updates are achieved with a 1.0 s receding horizon, and desired and measured torque and joint-velocity signals remain close throughout three walking cycles.The results show that the feedback policy respects actuator bandwidth limits; pace and trot motions are also feasible.
- Conclusion: Stable walking is possible with low update rates despite mobile-platform computational restrictions.
- Conclusion: The relaxed barrier method extends SLQ to inequality-constrained optimization, implementing the friction cone through a perturbed second-order cone constraint.It adds a convex cost penalty and avoids numerical ill-conditioning at the cone origin.
- Conclusion: Frequency-aware MPC attenuates high-frequency gains while preserving low-frequency gains, supporting closed-loop hardware stability and constraint-consistent feedback.The policy also reduces gains near friction-cone boundaries, sets zero swing-leg contact-force gains, and encodes zero stance-leg end-effector velocity.