PaperlayerView exact revision on arXiv

Source-linked AI summary

CARO: Contact-Agnostic Residual Observation for Zero-Shot Robust Quadruped Locomotion

Zihan Yang, Shixuan Han, Kexin Guo, Xiang Yu

arXiv:2608.24217v1cs.RO

TL;DR

Quadruped locomotion policies can degrade outside their training distribution, while contact-rich dynamics complicate explicit disturbance estimation. CARO embeds a fixed-base Euler–Lagrange model in the RL loop and feeds a sensorless joint-level residual to the policy for adaptation. It improves zero-shot robustness across simulated and real-world payload, terrain, center-of-mass, dynamics-change, and elevated-landing scenarios.

  • Problem

    Learned quadruped locomotion policies can degrade under out-of-distribution conditions, while robust adaptation lacks an explicit physically structured feedback channel that avoids difficult sensing requirements.

    Source: §I · pp. 1–2

  • Method

    CARO embeds a fixed-base Euler–Lagrange internal model in the RL loop and supplies its joint-level dynamics-mismatch residual to the policy as an adaptation signal without torque sensing or explicit contact estimation.

    Source: §I · pp. 1–2, B. Fixed-Base Euler–Lagrange Internal… · p. 3

  • Results

    CARO consistently improves locomotion robustness under matched training conditions, including out-of-distribution payloads, terrain geometries, center-of-mass shifts, abrupt dynamics changes, and real-world elevated-platform landings.

    Source: §I · pp. 1–2

  • Takeaways & Limitations

    A deliberately simplified internal model can provide useful adaptation feedback for zero-shot robust quadruped locomotion across simulation and physical experiments.

    Source: §V · p. 8

    The bounded residual-estimation result does not establish stability of the complete learned closed-loop system, and higher observer gain amplifies measurement noise.

    Source: §VI · p. 8

Abstract

from arXiv · show

We propose CARO, a contact-agnostic residual observation framework for policy adaptation. CARO embeds a fixed-base Euler--Lagrange model into the reinforcement learning control loop and constructs a torque-level residual observation without requiring torque sensors, explicit contact estimation, or vision-based measurements of the floating-base position and linear velocity. A disturbance observer extracts a structured signal representing dynamics mismatch, while the policy learns to exploit this feedback for online adaptation. CARO is trained under the same terrain, command, and domain-randomization conditions as the nominal policy, without specialized disturbance curricula or additional adaptation supervision. Nevertheless, it achieves substantially improved zero-shot robustness in simulation and sim-to-real transfer tasks involving out-of-distribution payloads, center-of-mass shifts, terrain geometries, abrupt dynamics changes, and elevated-platform landings.

I. INTRODUCTION

CARO addresses robustness degradation under out-of-distribution conditions by embedding a fixed-base internal model in the RL loop and feeding structured residuals to the policy. It reports zero-shot robustness gains without specialized robustness training.

Motivation

  • CARO targets deployment robustness when learned quadruped policies encounter conditions outside their training distribution.Existing approaches primarily rely on simulation, domain randomization, or latent adaptation modules.Source: §I · p. 1, A. Robustness in RL-Based Legged… · p. 2

Framework

  • CARO embeds a fixed-base Euler–Lagrange model in the RL loop to construct a lightweight joint-level residual observation.The framework captures joint-space effects of payload and contact changes without explicitly estimating contact states, locations, or wrenches.Source: §I · p. 1, Figure 1 · p. 1
  • CARO avoids explicit contact-force estimation, force/torque sensing, and floating-base position or linear-velocity measurements.These requirements are challenging for quadruped locomotion because of hybrid, nonlinear, contact-sensitive dynamics and large body motions.Source: §I · pp. 1–2
  • The residual is supplied to the RL policy as an adaptation input rather than used for direct joint-level compensation.This lets the policy determine responses under contact-rich and out-of-distribution conditions.Source: §I · p. 2

Results

  • CARO improves robustness to out-of-distribution payloads, center-of-mass shifts, terrain geometries, and abrupt dynamics changes in simulation and real-world experiments.It is trained under the same terrain, command, and domain-randomization conditions as the nominal policy, without specialized disturbance curricula or additional adaptation supervision.Source: §I · pp. 1–2

B. Disturbance Estimation for Legged Robots

The paper motivates sensorless disturbance estimation for legged robots through floating-base and contact-coupled dynamics. CARO uses a joint-level residual as policy adaptation feedback rather than requiring physically accurate external-torque identification.

Challenges

  • Contact-rich legged dynamics make disturbance estimation difficult without contact-state or force/torque sensing.Classical and recent observers infer disturbances from discrepancies between nominal dynamics and measured motion, but unseen payloads and contact conditions remain challenging.Source: B. Disturbance Estimation for Legged… · p. 2

CARO design

  • CARO uses the observed joint-level residual as policy adaptation feedback instead of recovering physically accurate external torques for force control or contact localization.This relaxes the need for precise disturbance identification under contact-rich and out-of-distribution conditions.Source: B. Disturbance Estimation for Legged… · p. 2

Dynamics

  • The full floating-base model represents inertia, nonlinear bias, actuation, contact wrenches, and additional generalized disturbances.The generalized velocity includes floating-base spatial velocity and actuated joint positions and velocities.Source: A. Preliminaries · pp. 2–3
  • Floating-base disturbances affect actuated joint dynamics through base-joint coupling and changes in contact wrenches.The actuated dynamics include the coupling term Mjb ˙vb and joint-space contact loading.Source: A. Preliminaries · p. 3

Control interface

  • A low-level PD controller converts the policy's reference joint-position increment into a commanded torque.The default joint position is q0, while Kp and Kd are positive-definite diagonal gain matrices.Source: A. Preliminaries · p. 3

B. Fixed-Base Euler–Lagrange Internal Model

CARO uses a fixed-base Euler–Lagrange model as a lightweight internal model while the physical robot remains floating-base. Model mismatch and contact effects are retained as a structured joint-space residual for policy adaptation.

B. Fixed-Base Euler–Lagrange Internal Model

  • CARO locks the base only inside the internal model, avoiding the need for a fully calibrated floating-base model and floating-base linear-velocity estimates.The physical robot remains a floating-base system.Source: B. Fixed-Base Euler–Lagrange Internal… · p. 3
  • The model computes fixed-base inertia, Coriolis, and gravity terms, with their combination forming the nonlinear bias and an internal-model residual.The residual is denoted r, while hf = Cf(q, ˙q) ˙q + Gf(q).Source: B. Fixed-Base Euler–Lagrange Internal… · p. 3
  • CARO uses commanded torque rather than measured torque during residual estimation, eliminating the need for torque sensing.The residual includes combined effects rather than separately identifying each discrepancy term.Source: B. Fixed-Base Euler–Lagrange Internal… · p. 3
  • Payloads, pushes, and terrain changes alter support-force distributions and joint-space loading, causing their effects to appear in the residual.CARO does not explicitly recover contact states, locations, or wrenches; it retains their joint-space effects.Source: B. Fixed-Base Euler–Lagrange Internal… · p. 3

C. Residual Observer

CARO estimates the residual with a momentum-based disturbance observer that avoids directly measuring joint acceleration. Using joint measurements and commanded torque, the observer produces an effective residual for policy adaptation.

C. Residual Observer

  • A momentum-based disturbance-observer structure estimates r without explicitly measuring joint acceleration.This avoids relying on potentially noisy numerical differentiation of joint velocity.Source: C. Residual Observer · p. 3
  • The observer defines first-order residual-estimation dynamics for the estimated residual ˆr with a positive-definite diagonal gain matrix L.The gain matrix is specified as L ≻ 0.Source: C. Residual Observer · p. 3
  • Fixed-base momentum pf := Mf(q) ˙q is differentiated and combined with the model equations to remove explicit dependence on joint acceleration.An auxiliary state is introduced before obtaining the implementable observer.Source: C. Residual Observer · p. 3
  • The implementable observer uses measured joint positions, joint velocities, commanded torque, and an estimated momentum bias to output effective residual ˆreff.It requires no force/torque sensors, explicit contact estimation, or floating-base position and linear-velocity measurements.Source: C. Residual Observer · p. 4

D. Boundedness of Residual Estimation

Theorem 1 bounds the sampled residual observer’s estimation error under forward-Euler integration and a bounded-increment assumption. The stated result is limited to residual-estimation error rather than complete-system stability.

D. Boundedness of Residual Estimation

E. Integration with the RL Policy

CARO augments the nominal policy observation with an estimated fixed-base residual, while privileged environment information is restricted to the critic during training. The residual observer is active from the beginning of training.

E. Integration with the RL Policy

  • CARO augments the nominal locomotion observation with the estimated fixed-base residual as an adaptation input.The actor then generates a reference joint-position increment.Source: E. Integration with the RL Policy · p. 4
  • In simulation, the nominal observation includes joint states, previous action, projected gravity, commands, and base velocities, although base-state quantities are excluded from the residual observer.Base linear velocity is omitted from real-world observations because it is not directly measurable without an estimator.Source: E. Integration with the RL Policy · p. 4
  • During training, the critic receives privileged contact, friction, payload, center-of-mass, motor-strength, and PD-gain information unavailable to the actor at deployment.This preserves the actor’s deployment-time information boundary.Source: E. Integration with the RL Policy · p. 4
  • The residual observer is activated from the beginning of policy training so the policy can learn how residuals relate to its closed-loop response.No separate activation phase is described.Source: E. Integration with the RL Policy · p. 4

IV. EMPIRICAL STUDY

The empirical study evaluates CARO’s zero-shot robustness under payload and terrain shifts in simulation, using shared training conditions and comparisons against three baselines. CARO achieves the highest aggregate success rate and shows particularly strong task-completion robustness under large simultaneous changes in dynamics and contact conditions.

Experimental setup

  • CARO is evaluated through payload-terrain sweeps and abrupt payload changes, followed by real-world tests on unseen terrains, center-of-mass shifts, increased payloads, and elevated-platform landings.The simulation studies assess generalization and transient response, while the physical experiments test deployment robustness across several out-of-distribution conditions.Source: §IV · p. 4
  • All methods share the same terrain, command, domain-randomization, reward, and privileged-variable conditions during training; CARO adds only its residual observation.Training uses flat terrain, while the evaluation includes unseen terrain geometries and payload conditions.Source: §IV · p. 4, A. Zero-Shot Payload-Terrain… · p. 5

Payload-terrain robustness

  • 88.6% mean success rate is achieved by CARO, the highest among the compared methods across the payload-terrain sweep.The aggregate metrics are averaged over all 25 payload-terrain combinations.Source: A. Zero-Shot Payload-Terrain… · pp. 5–6
  • At 3.0× nominal base mass, CARO retains 64.5% success versus 36.4% for RMA, 0.2% for RL2AC, and 0% for Vanilla.At 2.5× mass, CARO achieves 87.4% versus 84.6% for RMA; CARO ranks first on four of five terrains, while RMA leads on rough slope.Source: A. Zero-Shot Payload-Terrain… · p. 5

Tracking and transient response

  • CARO travels 13.189 m on average and has the lowest yaw-rate error at 0.080 rad/s, while RMA has slightly better reward per step and linear-velocity error.The results indicate that CARO’s residual observation particularly improves task-completion robustness under large simultaneous shifts in dynamics and contact conditions.Source: A. Zero-Shot Payload-Terrain… · pp. 5–6

B. Reaction to Sudden Payload Changes

CARO responds to an abrupt payload-induced dynamics change by preserving command tracking more effectively than the baselines. Its structured residuals reflect joint-specific loading changes and support rapid adaptation without explicit payload estimation.

Experiment setup

  • A 5.6 kg payload addition doubled torso mass from 5.6 kg to 11.2 kg, exceeding the 3.0 kg maximum added mass used during training.The payload equaled 50% of the robot’s nominal total mass.Source: B. Reaction to Sudden Payload Changes · p. 6

Transient response

  • After the payload step, CARO’s forward velocity stayed closer to the 0.8 m/s command and recovered quickly with comparatively narrow variability.Vanilla and especially RMA showed larger speed drops and wider trial-to-trial spread.Source: B. Reaction to Sudden Payload Changes · p. 6
  • CARO’s mean yaw rate quickly returned to the zero command after the transient payload-induced response.All methods exhibited a transient yaw-rate response.Source: B. Reaction to Sudden Payload Changes · p. 6

Residual response

  • The residual change was spatially structured: knee residuals increased on every leg, with the largest sustained changes on the rear legs.Post-step variability was most visible in rear-knee channels and reflected trial-dependent transient loading.Source: B. Reaction to Sudden Payload Changes · p. 6

Tracking errors

  • CARO had the lowest plotted post-step linear-velocity and yaw-rate errors, with yaw-rate error increasing from approximately 0.05 to 0.07 rad/s.The corresponding post-step yaw-rate means were about 0.18 rad/s for Vanilla, 0.13 rad/s for RMA, and 0.22 rad/s for RL2AC.Source: B. Reaction to Sudden Payload Changes · pp. 6–7

Real-world deployment

  • The physical deployment used a five-observation history, and this modification was critical for reducing vibration on the real robot.CARO was deployed without hardware-specific policy fine-tuning.Source: C. Real-World Experiments · p. 7

2) Center-of-Mass Shifts:

CARO maintains locomotion under asymmetric loading and demanding real-world transitions where the Vanilla policy fails. These tests extend beyond nominal flat-terrain training to lateral center-of-mass shifts, heavier payloads, and elevated landings.

Center-of-Mass Shifts

  • With a 2.5 kg lateral payload causing a substantial center-of-mass shift, CARO rapidly restored a near-level body attitude and maintained the commanded forward heading.The test commanded 1 m/s forward locomotion on flat ground.Source: 2) Center-of-Mass Shifts · p. 7

Maximum Payload Capacity

  • CARO maintained stable forward locomotion with an 8.5 kg payload, whereas Vanilla failed to complete the trial with a 6.5 kg payload.The increased payload capacity extended the tested operating range beyond the training payload distribution.Source: 3) Maximum Payload Capacity · pp. 7–8

Elevated-Platform Landings

  • CARO achieved 3/3 successful landings from a 0.2 m-high platform, while Vanilla fell in all three trials.The policy was trained only for flat-terrain locomotion, and CARO absorbed the landing impact and continued forward.Source: 4) Elevated-Platform Landings · p. 8

Method interpretation

  • CARO converts modeled joint-dynamics mismatch into a structured adaptation input rather than using it for direct torque compensation.The residual retains joint-space consequences of payload and contact changes without explicitly estimating contact states or wrenches.Source: §V · p. 8

Overall outcome

  • Across the reported evaluations, CARO achieved an 88.6% mean success rate and retained 64.5% success at 3.0× nominal base mass.The conclusion also reports four unseen terrains with a 2.5 kg payload and stable forward locomotion with an 8.5 kg payload.Source: §V · p. 8

Limitations

  • The theoretical residual bound applies under bounded Euler-consistent step-to-step increments and does not establish stability of the complete learned closed-loop system.This is the paper’s stated theoretical scope boundary.Source: §VI · p. 8

APPENDIX I PROOF OF THE RESIDUAL-ESTIMATION BOUND

The appendix derives a bound for the residual observer’s estimation error under forward-Euler sampling and bounded residual increments. The proof uses the observer’s discrete update, matrix properties, norm bounds, and an asymptotic vanishing term.

Discrete update

  • Forward Euler is applied to the auxiliary-state dynamics to obtain the implemented discrete observer update.The derivation begins from the auxiliary-state dynamics and the sampled residual definition.Source: Appendix I · p. 8

Causality

  • Equation (26) exactly describes the implemented Euler update, while the online update remains causal and does not require a momentum difference.The use of p_f,k+1 is restricted to the analysis.Source: Appendix I · p. 8

Matrix condition

  • Positive-definite diagonal L makes A symmetric, and 0 < T_sλ_i(L) < 2 supplies the stated discrete-time condition.These matrix properties are used in the stability argument for the observer error dynamics.Source: Appendix I · p. 8

Error bound

  • Taking Euclidean norms, using submultiplicativity, and applying the bounded-increment assumption yields the intermediate inequality needed to prove the residual bound.The final asymptotic step uses α_k−1 → 0 and a limit superior.Source: Appendix I · p. 8

APPENDIX II REWARD TERMS

All policies share the reward terms, network architecture, domain randomization, and core training objectives, while their deployable actors receive method-specific observations. The appendix also documents baseline-specific adaptation modules and the residual observer implementation.

Reward terms

Network architecture

  • Actors and critics use MLPs with hidden-layer dimensions [512, 256, 128], while critics additionally receive privileged information.Each deployable actor receives its method-specific observation.Source: A. Baseline Implementations · p. 9

Baseline configuration

  • Domain-randomization parameters and reward terms are shared across methods, and the baselines are configured with their respective observation or adaptation mechanisms.The Vanilla policy has no adaptation mechanism; RMA and RL2AC use history-based modules.Source: A. Baseline Implementations · p. 9

Residual observer

  • In simulation, the fixed-base residual observer computes M_f and G_f using Isaac Gym Jacobians and approximates C_f with diag(C_f) = 0.1 · diag(M_f).The observer is integrated with forward Euler for batched training.Source: B. Realization of the Residual Observer · p. 9

Deployment implementation

  • For real-world deployment, Pinocchio computes observer-model Jacobians and rl_sar runs the observer with a LibTorch policy at 50 Hz.This describes the real-time inference implementation.Source: B. Realization of the Residual Observer · p. 9
Loading 2608.24217v1…