PaperlayerView exact revision on arXiv

Source-linked AI summary

CARO: Contact-Agnostic Residual Observation for Zero-Shot Robust Quadruped Locomotion

Zihan Yang, Shixuan Han, Kexin Guo, Xiang Yu

arXiv:2608.24217v1cs.RO

TL;DR

Quadruped locomotion policies can degrade outside their training distribution, while contact-rich dynamics complicate explicit disturbance estimation. CARO embeds a fixed-base Euler–Lagrange model in the RL loop and feeds a sensorless joint-level residual to the policy for adaptation. It improves zero-shot robustness across simulated and real-world payload, terrain, center-of-mass, dynamics-change, and elevated-landing scenarios.

  • Problem

    Learned quadruped locomotion policies can degrade under out-of-distribution conditions, while robust adaptation lacks an explicit physically structured feedback channel that avoids difficult sensing requirements.

  • Method

    CARO embeds a fixed-base Euler–Lagrange internal model in the RL loop and supplies its joint-level dynamics-mismatch residual to the policy as an adaptation signal without torque sensing or explicit contact estimation.

  • Results

    CARO consistently improves locomotion robustness under matched training conditions, including out-of-distribution payloads, terrain geometries, center-of-mass shifts, abrupt dynamics changes, and real-world elevated-platform landings.

  • Takeaways & Limitations

    A deliberately simplified internal model can provide useful adaptation feedback for zero-shot robust quadruped locomotion across simulation and physical experiments.

  • Takeaways & Limitations

    The bounded residual-estimation result does not establish stability of the complete learned closed-loop system, and higher observer gain amplifies measurement noise.

Abstract

from arXiv · show

We propose CARO, a contact-agnostic residual observation framework for policy adaptation. CARO embeds a fixed-base Euler--Lagrange model into the reinforcement learning control loop and constructs a torque-level residual observation without requiring torque sensors, explicit contact estimation, or vision-based measurements of the floating-base position and linear velocity. A disturbance observer extracts a structured signal representing dynamics mismatch, while the policy learns to exploit this feedback for online adaptation. CARO is trained under the same terrain, command, and domain-randomization conditions as the nominal policy, without specialized disturbance curricula or additional adaptation supervision. Nevertheless, it achieves substantially improved zero-shot robustness in simulation and sim-to-real transfer tasks involving out-of-distribution payloads, center-of-mass shifts, terrain geometries, abrupt dynamics changes, and elevated-platform landings.

I. INTRODUCTION

CARO addresses robustness degradation under out-of-distribution conditions by embedding a fixed-base internal model in the RL loop and feeding structured residuals to the policy. It reports zero-shot robustness gains without specialized robustness training.

  • Motivation: CARO targets deployment robustness when learned quadruped policies encounter conditions outside their training distribution.Existing approaches primarily rely on simulation, domain randomization, or latent adaptation modules.
  • Framework: CARO embeds a fixed-base Euler–Lagrange model in the RL loop to construct a lightweight joint-level residual observation.The framework captures joint-space effects of payload and contact changes without explicitly estimating contact states, locations, or wrenches.
  • Framework: CARO avoids explicit contact-force estimation, force/torque sensing, and floating-base position or linear-velocity measurements.These requirements are challenging for quadruped locomotion because of hybrid, nonlinear, contact-sensitive dynamics and large body motions.
  • Framework: The residual is supplied to the RL policy as an adaptation input rather than used for direct joint-level compensation.This lets the policy determine responses under contact-rich and out-of-distribution conditions.
  • Results: CARO improves robustness to out-of-distribution payloads, center-of-mass shifts, terrain geometries, and abrupt dynamics changes in simulation and real-world experiments.It is trained under the same terrain, command, and domain-randomization conditions as the nominal policy, without specialized disturbance curricula or additional adaptation supervision.

B. Disturbance Estimation for Legged Robots

The paper motivates sensorless disturbance estimation for legged robots through floating-base and contact-coupled dynamics. CARO uses a joint-level residual as policy adaptation feedback rather than requiring physically accurate external-torque identification.

  • Challenges: Contact-rich legged dynamics make disturbance estimation difficult without contact-state or force/torque sensing.Classical and recent observers infer disturbances from discrepancies between nominal dynamics and measured motion, but unseen payloads and contact conditions remain challenging.
  • CARO design: CARO uses the observed joint-level residual as policy adaptation feedback instead of recovering physically accurate external torques for force control or contact localization.This relaxes the need for precise disturbance identification under contact-rich and out-of-distribution conditions.
  • Dynamics: The full floating-base model represents inertia, nonlinear bias, actuation, contact wrenches, and additional generalized disturbances.The generalized velocity includes floating-base spatial velocity and actuated joint positions and velocities.
  • Dynamics: Floating-base disturbances affect actuated joint dynamics through base-joint coupling and changes in contact wrenches.The actuated dynamics include the coupling term Mjb ˙vb and joint-space contact loading.
  • Control interface: A low-level PD controller converts the policy's reference joint-position increment into a commanded torque.The default joint position is q0, while Kp and Kd are positive-definite diagonal gain matrices.

B. Fixed-Base Euler–Lagrange Internal Model

CARO uses a fixed-base Euler–Lagrange model as a lightweight internal model while the physical robot remains floating-base. Model mismatch and contact effects are retained as a structured joint-space residual for policy adaptation.

  • B. Fixed-Base Euler–Lagrange Internal Model: CARO locks the base only inside the internal model, avoiding the need for a fully calibrated floating-base model and floating-base linear-velocity estimates.The physical robot remains a floating-base system.
  • B. Fixed-Base Euler–Lagrange Internal Model: The model computes fixed-base inertia, Coriolis, and gravity terms, with their combination forming the nonlinear bias and an internal-model residual.The residual is denoted r, while hf = Cf(q, ˙q) ˙q + Gf(q).
  • B. Fixed-Base Euler–Lagrange Internal Model: CARO uses commanded torque rather than measured torque during residual estimation, eliminating the need for torque sensing.The residual includes combined effects rather than separately identifying each discrepancy term.
  • B. Fixed-Base Euler–Lagrange Internal Model: Payloads, pushes, and terrain changes alter support-force distributions and joint-space loading, causing their effects to appear in the residual.CARO does not explicitly recover contact states, locations, or wrenches; it retains their joint-space effects.

C. Residual Observer

CARO estimates the residual with a momentum-based disturbance observer that avoids directly measuring joint acceleration. Using joint measurements and commanded torque, the observer produces an effective residual for policy adaptation.

  • C. Residual Observer: A momentum-based disturbance-observer structure estimates r without explicitly measuring joint acceleration.This avoids relying on potentially noisy numerical differentiation of joint velocity.
  • C. Residual Observer: The observer defines first-order residual-estimation dynamics for the estimated residual ˆr with a positive-definite diagonal gain matrix L.The gain matrix is specified as L ≻ 0.
  • C. Residual Observer: Fixed-base momentum pf := Mf(q) ˙q is differentiated and combined with the model equations to remove explicit dependence on joint acceleration.An auxiliary state is introduced before obtaining the implementable observer.
  • C. Residual Observer: The implementable observer uses measured joint positions, joint velocities, commanded torque, and an estimated momentum bias to output effective residual ˆreff.It requires no force/torque sensors, explicit contact estimation, or floating-base position and linear-velocity measurements.

D. Boundedness of Residual Estimation

Theorem 1 bounds the sampled residual observer’s estimation error under forward-Euler integration and a bounded-increment assumption. The stated result is limited to residual-estimation error rather than complete-system stability.

  • D. Boundedness of Residual Estimation: The theorem establishes a bound on the proposed residual observer’s estimation error under its stated sampling and bounded-step assumptions.The proof is provided in Appendix I.
  • D. Boundedness of Residual Estimation: Theorem 1 analyzes the residual observer at sampling instants tk = kTs when equation (16) is integrated with forward Euler.The analysis uses an Euler-consistent sampled residual.
  • D. Boundedness of Residual Estimation: The observer-gain and sampling condition 0 < Tsλi(L) < 2 yields A := I − TsL with α := ∥A∥2 < 1.This condition is imposed for every eigenvalue of L.

E. Integration with the RL Policy

CARO augments the nominal policy observation with an estimated fixed-base residual, while privileged environment information is restricted to the critic during training. The residual observer is active from the beginning of training.

  • E. Integration with the RL Policy: CARO augments the nominal locomotion observation with the estimated fixed-base residual as an adaptation input.The actor then generates a reference joint-position increment.
  • E. Integration with the RL Policy: In simulation, the nominal observation includes joint states, previous action, projected gravity, commands, and base velocities, although base-state quantities are excluded from the residual observer.Base linear velocity is omitted from real-world observations because it is not directly measurable without an estimator.
  • E. Integration with the RL Policy: During training, the critic receives privileged contact, friction, payload, center-of-mass, motor-strength, and PD-gain information unavailable to the actor at deployment.This preserves the actor’s deployment-time information boundary.
  • E. Integration with the RL Policy: The residual observer is activated from the beginning of policy training so the policy can learn how residuals relate to its closed-loop response.No separate activation phase is described.

IV. EMPIRICAL STUDY

The empirical study evaluates CARO’s zero-shot robustness under payload and terrain shifts in simulation, using shared training conditions and comparisons against three baselines. CARO achieves the highest aggregate success rate and shows particularly strong task-completion robustness under large simultaneous changes in dynamics and contact conditions.

  • Experimental setup: CARO is evaluated through payload-terrain sweeps and abrupt payload changes, followed by real-world tests on unseen terrains, center-of-mass shifts, increased payloads, and elevated-platform landings.The simulation studies assess generalization and transient response, while the physical experiments test deployment robustness across several out-of-distribution conditions.
  • Experimental setup: All methods share the same terrain, command, domain-randomization, reward, and privileged-variable conditions during training; CARO adds only its residual observation.Training uses flat terrain, while the evaluation includes unseen terrain geometries and payload conditions.
  • Payload-terrain robustness: 88.6% mean success rate is achieved by CARO, the highest among the compared methods across the payload-terrain sweep.The aggregate metrics are averaged over all 25 payload-terrain combinations.
  • Payload-terrain robustness: At 3.0× nominal base mass, CARO retains 64.5% success versus 36.4% for RMA, 0.2% for RL2AC, and 0% for Vanilla.At 2.5× mass, CARO achieves 87.4% versus 84.6% for RMA; CARO ranks first on four of five terrains, while RMA leads on rough slope.
  • Tracking and transient response: CARO travels 13.189 m on average and has the lowest yaw-rate error at 0.080 rad/s, while RMA has slightly better reward per step and linear-velocity error.The results indicate that CARO’s residual observation particularly improves task-completion robustness under large simultaneous shifts in dynamics and contact conditions.

B. Reaction to Sudden Payload Changes

CARO responds to an abrupt payload-induced dynamics change by preserving command tracking more effectively than the baselines. Its structured residuals reflect joint-specific loading changes and support rapid adaptation without explicit payload estimation.

  • Experiment setup: A 5.6 kg payload addition doubled torso mass from 5.6 kg to 11.2 kg, exceeding the 3.0 kg maximum added mass used during training.The payload equaled 50% of the robot’s nominal total mass.
  • Transient response: After the payload step, CARO’s forward velocity stayed closer to the 0.8 m/s command and recovered quickly with comparatively narrow variability.Vanilla and especially RMA showed larger speed drops and wider trial-to-trial spread.
  • Transient response: CARO’s mean yaw rate quickly returned to the zero command after the transient payload-induced response.All methods exhibited a transient yaw-rate response.
  • Residual response: The residual change was spatially structured: knee residuals increased on every leg, with the largest sustained changes on the rear legs.Post-step variability was most visible in rear-knee channels and reflected trial-dependent transient loading.
  • Tracking errors: CARO had the lowest plotted post-step linear-velocity and yaw-rate errors, with yaw-rate error increasing from approximately 0.05 to 0.07 rad/s.The corresponding post-step yaw-rate means were about 0.18 rad/s for Vanilla, 0.13 rad/s for RMA, and 0.22 rad/s for RL2AC.
  • Real-world deployment: The physical deployment used a five-observation history, and this modification was critical for reducing vibration on the real robot.CARO was deployed without hardware-specific policy fine-tuning.

2) Center-of-Mass Shifts:

CARO maintains locomotion under asymmetric loading and demanding real-world transitions where the Vanilla policy fails. These tests extend beyond nominal flat-terrain training to lateral center-of-mass shifts, heavier payloads, and elevated landings.

  • Center-of-Mass Shifts: With a 2.5 kg lateral payload causing a substantial center-of-mass shift, CARO rapidly restored a near-level body attitude and maintained the commanded forward heading.The test commanded 1 m/s forward locomotion on flat ground.
  • Maximum Payload Capacity: CARO maintained stable forward locomotion with an 8.5 kg payload, whereas Vanilla failed to complete the trial with a 6.5 kg payload.The increased payload capacity extended the tested operating range beyond the training payload distribution.
  • Elevated-Platform Landings: CARO achieved 3/3 successful landings from a 0.2 m-high platform, while Vanilla fell in all three trials.The policy was trained only for flat-terrain locomotion, and CARO absorbed the landing impact and continued forward.
  • Method interpretation: CARO converts modeled joint-dynamics mismatch into a structured adaptation input rather than using it for direct torque compensation.The residual retains joint-space consequences of payload and contact changes without explicitly estimating contact states or wrenches.
  • Overall outcome: Across the reported evaluations, CARO achieved an 88.6% mean success rate and retained 64.5% success at 3.0× nominal base mass.The conclusion also reports four unseen terrains with a 2.5 kg payload and stable forward locomotion with an 8.5 kg payload.
  • Limitations: The theoretical residual bound applies under bounded Euler-consistent step-to-step increments and does not establish stability of the complete learned closed-loop system.This is the paper’s stated theoretical scope boundary.

APPENDIX I PROOF OF THE RESIDUAL-ESTIMATION BOUND

The appendix derives a bound for the residual observer’s estimation error under forward-Euler sampling and bounded residual increments. The proof uses the observer’s discrete update, matrix properties, norm bounds, and an asymptotic vanishing term.

  • Discrete update: Forward Euler is applied to the auxiliary-state dynamics to obtain the implemented discrete observer update.The derivation begins from the auxiliary-state dynamics and the sampled residual definition.
  • Causality: Equation (26) exactly describes the implemented Euler update, while the online update remains causal and does not require a momentum difference.The use of p_f,k+1 is restricted to the analysis.
  • Matrix condition: Positive-definite diagonal L makes A symmetric, and 0 < T_sλ_i(L) < 2 supplies the stated discrete-time condition.These matrix properties are used in the stability argument for the observer error dynamics.
  • Error bound: Taking Euclidean norms, using submultiplicativity, and applying the bounded-increment assumption yields the intermediate inequality needed to prove the residual bound.The final asymptotic step uses α_k−1 → 0 and a limit superior.

APPENDIX II REWARD TERMS

All policies share the reward terms, network architecture, domain randomization, and core training objectives, while their deployable actors receive method-specific observations. The appendix also documents baseline-specific adaptation modules and the residual observer implementation.

  • Reward terms: All methods use the reward terms and weights listed in Table III.The shared reward design supports comparisons under common training objectives.
  • Network architecture: Actors and critics use MLPs with hidden-layer dimensions [512, 256, 128], while critics additionally receive privileged information.Each deployable actor receives its method-specific observation.
  • Baseline configuration: Domain-randomization parameters and reward terms are shared across methods, and the baselines are configured with their respective observation or adaptation mechanisms.The Vanilla policy has no adaptation mechanism; RMA and RL2AC use history-based modules.
  • Residual observer: In simulation, the fixed-base residual observer computes M_f and G_f using Isaac Gym Jacobians and approximates C_f with diag(C_f) = 0.1 · diag(M_f).The observer is integrated with forward Euler for batched training.
  • Deployment implementation: For real-world deployment, Pinocchio computes observer-model Jacobians and rl_sar runs the observer with a LibTorch policy at 50 Hz.This describes the real-time inference implementation.
Loading 2608.24217v1…