PaperlayerView exact revision on arXiv

Source-linked AI summary

Higher-Order Morphology Priors for Quadruped Reinforcement Learning Under Actuator Degradation

Derek You, Zafir Shamsi, Keqin Wang, Christine Allen-Blanchette

arXiv:2610.10934v1cs.ROcs.LG

TL;DR

Actuator degradation reduces control authority at specific joints, requiring the rest of the quadruped to compensate, while morphology-aware policies have largely represented structure through pairwise relations. This paper models the Go1 with limb- and body-level rank-2 cells and Hodge-based message passing. Under degradation training, the Hodge-F policy has the advantage, with higher return, survival, and lower velocity error in degraded-regime evaluation.

  • Problem

    Actuator degradation reduces control authority at particular joints and requires compensation by the remaining body, while morphology-aware policies generally encode pairwise relations rather than higher-order mechanical structures.

    Source: §I · p. 1

  • Method

    The Unitree Go1 is represented as a cell complex with rank-2 limb and body cells, and a Hodge actor passes messages over nodes, edges, and faces.

    Source: §I · p. 1

  • Results

    Under degraded-regime training and evaluation, Hodge-F achieves the highest return, survival rate, and lowest velocity error; healthy-regime training instead favors MLP-C.

    Source: C. In-Regime Training and Evaluation · p. 4

  • Takeaways & Limitations

    The results indicate that rank-2 limb and body structure matters most when policies must learn compensatory coordination rather than nominal locomotion.

    Source: §VI · p. 4

Abstract

from arXiv · show

Actuator degradation turns quadruped locomotion into a coordination problem requiring joints to compensate for lost actuation. Prior work suggests that morphology-aware graph policies improve learning and generalization under body perturbations. We ask whether these benefits can be strengthened by explicitly modeling higher-order mechanical structure. We represent the Unitree Go1 as a cell complex with limb- and body-level rank-2 cells and apply Hodge-based message passing. Under degradation training, the node-edge-face Hodge actor achieves the highest return on unseen actuator degradations, with higher survival and lower velocity-tracking error. These results support higher-order morphology as a useful inductive bias for whole-body compensation under actuator degradation.

I. INTRODUCTION

Actuator degradation removes control authority at specific joints, making compensation by the remaining body important; the paper asks whether morphology representation alone can support that coordination. It introduces a Go1 cell-complex policy and evaluates it against competitive baselines on held-out compound actuator degradation data.

I. INTRODUCTION

  • Actuator degradation reduces control authority at specific joints, requiring the remaining body to compensate; the paper asks whether morphology representation can support this without dedicated fault-handling mechanisms [6] [7].Source: §I · p. 1
  • The Go1 representation adds rank-2 limb and body cells to joint and edge structure, and the Hodge actor passes messages across nodes, edges, and faces.The five faces comprise one for each limb and one body cell spanned by the four coplanar hips.Source: §I · p. 1, Figure 1 · p. 1
  • On held-out compound actuator degradation data, the approach outperforms competitive baselines in a capacity-matched training-evaluation study.The comparison includes healthy and degradation training-evaluation settings.Source: §I · p. 1

II. RELATED WORK

  • Morphology-aware policies establish morphology as a useful prior, but typically encode pairwise relations; this work instead uses mechanically meaningful higher-order relations [17–19].Source: §II · p. 2

III. PRELIMINARIES

The problem setting uses the Unitree Go1 and velocity-commanded locomotion, with actuator degradation modeled by scaling each joint’s maximum available torque. The robot’s limb and hip structure motivates the higher-order representation.

A. Unitree Go1

  • Each Go1 leg is a mechanical unit whose three joints jointly determine foot placement, while its four rigidly mounted hips form a fixed coplanar ring.Source: A. Unitree Go1 · p. 2

B. Problem Setting

  • The velocity-commanded MuJoCo Playground task supplies base, command, and per-joint observations, and the policy outputs one normalized position offset per actuator.Policies use PPO with the task’s provided locomotion reward.Source: B. Problem Setting · p. 2
  • Degradation scales actuator i’s maximum available torque by ρ_i: ρ_i = 1 is nominal actuation and ρ_i = 0 is complete torque loss.Source: B. Problem Setting · p. 2

C. Cell Complexes and Incidence Operators

Cell complexes retain graph nodes and edges while adding selected closed edge cycles as explicit faces. Incidence operators connect adjacent ranks for feature exchange.

C. Cell Complexes and Incidence Operators

A. Higher-Order Go1 Representation

The Go1 cell complex encodes its joints, kinematic and hip connections, and limb and body faces. The actor initializes cell features from joint observations and static structural information.

A. Higher-Order Go1 Representation

  • The Go1 complex has 12 joint nodes, 16 edges, and five faces: four limb cycles and one body face, connected by fixed incidence matrices.The edges include eight along the legs, four around the hips, and four hip-calf closures.Source: A. Higher-Order Go1 Representation · p. 2

B. Feature Initialization

  • The actor reorganizes the observation into joint features containing shared base and command quantities, joint-specific state, and static limb and joint-type indicators.Each joint feature is in R^20.Source: B. Feature Initialization · p. 2
  • Node features are projected into the hidden dimension; edge and face features use incident-joint embeddings plus static edge or face information.Source: B. Feature Initialization · p. 2

C. Multi-Rank Hodge Message Passing

The actor mixes messages within each rank and across adjacent ranks, then applies a gated, normalized residual update. Hodge operators and incidence-based routes let cells exchange information through shared boundaries and cofaces.

C. Multi-Rank Hodge Message Passing

  • Each propagation layer combines same-rank messages with cross-rank messages, then applies a learned norm gate, RMSNorm, and a residual update.Same-rank messages use Hodge-Laplacian propagation.Source: C. Multi-Rank Hodge Message Passing · pp. 2–3
  • The actor initializes node, edge, and face states from joint-wise features, applies residual Hodge blocks, and decodes rank-0 states into joint actions.The figure specifies 12 node, 16 edge, and 5 face states.Source: Figure 2 · p. 3
  • Lower-rank propagation lets rank-k cells sharing a boundary exchange information, while upper-rank propagation couples cells sharing a coface.Source: C. Multi-Rank Hodge Message Passing · p. 3

D. Policy and Value Networks

The actor independently decodes each joint embedding into action-distribution parameters. A privileged-observation critic is shared across actor variants to isolate architectural differences.

D. Policy and Value Networks

  • After four propagation layers, a shared two-layer decoder maps each final joint embedding to its action-distribution parameters for a tanh-transformed Gaussian factorized policy.Source: D. Policy and Value Networks · p. 3
  • The critic receives privileged simulator state in addition to the actor observation, and the same standard MuJoCo Playground critic is used for every actor variant.Source: D. Policy and Value Networks · p. 3

A. Experimental Setup and Baselines

The study compares parameter-matched actor architectures across healthy and degraded training-evaluation regimes, with shared training conditions and held-out actuator-degradation evaluation.

A. Experimental Setup and Baselines

B. Training and Evaluation Regimes

  • Evaluation covers healthy and degraded regimes, including H→H, D→D, H→D, and D→H training-evaluation combinations.Source: B. Training and Evaluation Regimes · pp. 3–4
  • MLP-C performs best in the healthy regime, while Hodge-F achieves the strongest performance under degradation.Source: Figure 3 · p. 4
  • Degraded training samples two-actuator conditions across 44 training pairs, while evaluation averages over 22 held-out pairs and specified actuator strengths.Training episodes are 30% healthy, 50% weak, and 20% dead.Source: B. Training and Evaluation Regimes · p. 4

C. In-Regime Training and Evaluation

In matched training and evaluation, Hodge-F leads under degradation while MLP-C leads under healthy conditions. The ablations and Hodge-L comparison point to higher-order Hodge structure as most useful when compensation is required.

C. In-Regime Training and Evaluation

  • In D →D, Hodge-F achieves the highest return, survival rate, and lowest velocity error.Table I reports the in-regime results; the prose identifies Hodge-F as best on all three measures under degradation.Source: Table I · p. 4, C. In-Regime Training and Evaluation · p. 4
  • In H →H, MLP-C achieves the highest return and lowest velocity error, while Hodge-F eventually leads episode return under D→D.MLP-C improves rapidly early under both curricula and retains its lead in healthy training.Source: C. In-Regime Training and Evaluation · p. 4
  • Under D →D, MLP-D’s lower return and higher velocity error show that factorization alone does not explain Hodge-F’s gain, while GCN’s weaker results indicate an advantage beyond a conventional graph.The reported comparisons cover return and velocity error; no numeric values are supplied.Source: C. In-Regime Training and Evaluation · p. 4
  • Under D →D, Hodge-F improves survival and reduces velocity RMSE relative to Hodge-L, with its clear learning-curve advantage appearing only in the degraded regime.The passage attributes the face-based advantage primarily to settings requiring learned compensation rather than nominal locomotion.Source: C. In-Regime Training and Evaluation · p. 4

D. Cross-Regime Evaluation

Cross-regime results depend on training exposure: nominal training does not make Hodge-F best under unseen degradation, whereas degradation training gives it an advantage across evaluation regimes. The conclusion locates the higher-order structure’s value in learned compensatory coordination.

D. Cross-Regime Evaluation

  • With no retraining, MLP-C has the highest return in H →D, while Hodge-F has the highest return and lowest velocity RMSE in D →H.The latter advantage also appears under nominal evaluation after training with actuator degradation.Source: Table II · p. 4, D. Cross-Regime Evaluation · p. 4

VI. CONCLUSION

  • The conclusion reports that degraded-regime training shifts the advantage from the centralized MLP to Hodge-F, where rank-2 limb and body structure matters most for compensatory coordination.The contrast is with nominal locomotion, where the centralized MLP remains strongest after healthy-regime training.Source: §VI · p. 4
Loading 2610.10934v1…