PaperlayerView exact revision on arXiv

Source-linked AI summary

Coupled State-Space Modelling, Control, and Policy Distillation for Hybrid Rigid-Pneumatic Manipulators

Alan Royce Gabriel Samuel, Pulkit Verma

arXiv:2609.29424v1cs.RO

TL;DR

Published hybrid arms use decoupled per-DOF loops, leaving the cost of that approximation unmeasured without a coupled model. This paper derives the model, measures coupling, and distills MPC into a real-time neural policy that tracks more tightly than decoupled PID.

  • Problem

    Published hybrid rigid-pneumatic arms use decoupled per-DOF control, and the cost of that approximation had not been quantified because a coupled model was unavailable.

    Source: §I · p. 1

  • Method

    The paper derives a coupled state-space model and distills MPC into a neural policy using behavior cloning and DAgger.

    Source: §I · p. 1, §VI · p. 4

  • Results

    Coupled LQR and MPC track 2.5× tighter than decoupled PID with 3.7× less torque; the distilled policy runs within the 5 ms control step with zero collisions and within 3 points of its teacher.

    Source: §I · p. 1

  • Takeaways & Limitations

    Inertial coupling predicts where decoupled control fails, while distillation provides a policy that runs within the control step where MPC does not.

    Source: §I · p. 1

    At N=2, MPC needs 34–118 ms per outer solve, against a 5 ms simulation step and its own 20 ms cadence.

    Source: §VI · p. 4

Abstract

from arXiv · show

Hybrid manipulators combine motorized rigid joints with pressure-actuated origami segments. Published arms of this kind are controlled with decoupled per-DOF loops, and the cost of this approximation has not been quantified, because the coupled model needed to measure it has not been built. This paper derives such a model for a chain of $N$ alternating revolute joints and Kresling origami segments, including pneumatic chamber dynamics and crease hysteresis. Using the model, we measure the coupling directly and show that its strength varies joint by joint, and that decoupled control loses precisely on the strongly coupled joints while remaining competitive on the one nearly decoupled joint. Coupled model-based controllers track $2.5\times$ tighter than a decoupled PID baseline at lower torque. However, the model predictive controller (MPC) is too slow for real time, and model-free reinforcement learning stalls far below acceptable success rates on a strict settling metric. We therefore distill the MPC into a small neural policy with behavior cloning and DAgger. The distilled policy settles 93-94$\%$ of goals with zero collisions, within a few points of its teacher, and runs inside the 5 ms control step where the MPC does not. Where the teacher itself fails, we trace the failure to a limit cycle with the bellows' lightly damped mode, and we remove it by selecting goal postures holdable at low pressure.

I. INTRODUCTION

The paper builds a coupled model to quantify the cost of decoupled control in hybrid rigid-pneumatic arms. Coupled control improves tracking, while a distilled MPC policy addresses the teacher’s runtime limit.

I. INTRODUCTION

  • Published hybrid arms use decoupled per-DOF loops, but the cost of that approximation had not been quantified because a coupled model was unavailable.The two actuator classes interact mechanically, so the model treats those couplings explicitly rather than as disturbances.Source: §I · p. 1
  • Pitch angle switches joint-to-bellows coupling on and off, correctly predicting the one joint where decoupled control remains competitive.Across collision-free configurations, normalized inertial coupling between bellows has a median of 0.71 and never falls below 0.49.Source: §I · p. 1
  • The coupled state-space model combines alternating motorized revolute joints and pressure-actuated Kresling segments, covering joint, extension, pressure, and crease-hysteresis dynamics.It has 6N states and is calibrated against published measurements.Source: §I · p. 1
  • Coupled LQR and MPC track 2.5× tighter than decoupled PID with 3.7× less torque.The comparison uses the coupled model to measure the cost of treating interactions as disturbances.Source: §I · p. 1
  • The paper distills MPC into a neural policy with behavior cloning and DAgger that runs within the 5 ms control step, with zero collisions and success within 3 points of its teacher.MPC exceeds the control period at N=3, while model-free reinforcement learning fails to reach acceptable success rates.Source: §I · p. 1
  • At N=3, selecting redundant goal postures holdable at low pressure removes the MPC teacher’s limit cycle with the bellows’ lightly damped mode, lifting success from 90.1% to 99.7%.The limit cycle is traced to the bellows’ lightly damped mode.Source: §I · pp. 1–2

II. RELATED WORK

Prior work spans hybrid arms controlled with decoupled loops, kinematic origami arms, and model-based control of individual soft or origami actuators. This paper addresses the missing multisegment coupled state space for hybrid arms and uses it to measure posture-dependent coupling.

II. RELATED WORK

  • Prior hybrid arms use decoupled per-joint PID, while other origami-arm work is kinematic or models a single actuator [5].Soft-arm dynamics and pressure-aware trajectory optimization address non-hybrid morphologies.Source: §II · p. 2
  • The paper identifies a gap in multisegment coupled state-space models for hybrid arms, despite mature single-unit origami control and continuum-arm dynamics.Its analytic model is intended to provide collision constraints, physical input limits, and posture-dependent coupling measurements.Source: §II · p. 2
  • Related learning work includes SAC with hindsight experience replay and neural approximations of MPC,, while DAgger addresses behavior cloning’s compounding covariate shift,.Guided policy search and PLATO are described as reaching similar ends.Source: §II · p. 2
  • Unlike approaches that condition the network on optimizer information, this work canonicalizes a redundant joint-space target so the teacher’s labels depend on the student’s task-space observation.The teacher is a QP that can be queried at states visited by the student; offline TD3+BC instead learns from a static dataset.Source: §II · p. 2

III. COUPLED STATE-SPACE MODEL

The model links Kresling segment geometry and crease hysteresis to pneumatic pressure dynamics and multibody motion. Its state-space formulation includes cross-segment mechanical coupling and pressure as a dynamic state.

III. COUPLED STATE-SPACE MODEL

  • Under a symmetric rigid-facet ansatz, the Kresling fold state reduces to twist angle ϕ, with plate separation determined by the fabrication-fixed diagonal crease length b.The segment’s crease stiffness supplies a restoring force, and pressure extends the single-acting bellows.Source: §III · pp. 2–3
  • A Bouc–Wen state z represents path-dependent crease hysteresis, with α specifying the elastic fraction of crease torque and 1−α the hysteretic fraction.The calibrated fit reproduces 4.95% hysteresis loss against the reported 4.7%; the fit varies by calibration window, so the experiments sweep α.Source: §III · p. 3
  • Chamber pressure evolves with valve mass flow and piston-volume change, so piston motion can change pressure even when the valve is shut.The chamber’s linear volume approximation keeps the same piston area for pneumatic force and pressure dynamics.Source: §III · p. 3
  • The multibody equations include inertia, Coriolis and centrifugal terms, gravity, crease forces, damping, and generalized joint torques and bellows forces.Cross-segment coupling resides in M(q); pressure enters the generalized forces algebraically, and the pure-integrator pressure dynamics support the MPC’s two-timescale reduction.Source: §III · p. 3
  • Each segment’s state contains joint position and velocity, extension and velocity, pressure, and hysteresis state; the input contains joint torque and valve mass flow.The state x has 6N components and the input u has 2N components.Source: §III · p. 3

IV. PROBLEM SETTING

The evaluation uses a simulated hybrid arm with bounded torque, pressure, and mass-flow inputs, testing trajectory tracking and Cartesian goal reaching. It reports tracking error, torque, settled success, collision rate, and decision time.

IV. PROBLEM SETTING

  • The simulated plant runs at a 5 ms step with limits of 5 N m torque, 50 kPa pressure, and 5 g/s mass flow.Tasks are joint-space reference trajectories or Cartesian goals drawn from reachable targets.Source: §IV · p. 3
  • Tracking is measured by per-joint RMS error and worst-joint RMS torque on a single-target ramp and a five-leg benchmark with sign reversals and near-limit legs.Goal-reaching evaluations also report collision rate and per-decision wall time.Source: §IV · p. 3
  • A goal-reaching episode succeeds only when the end effector stays within 10 mm of the goal for 20 consecutive steps, the settled-success criterion.Episodes start from home and run for at most 500 steps; goals are reachable by construction and results are in simulation.Source: §IV · p. 3

V. MODEL-BASED CONTROL

The controller suite compares decoupled PID, full-state LQR, two-timescale MPC, and model-free RL. MPC predicts a reduced slow state, while an integral-action estimator improves loaded tracking without changing the controller gains or QP.

V. MODEL-BASED CONTROL

  • The comparison evaluates decoupled PID, linearized full-state LQR, two-timescale MPC, and SAC-based model-free RL on the stated tracking benchmarks.PID uses one loop per degree of freedom, with an inner pressure loop for each bellows; controllers compensating hysteresis add Bouc–Wen crease torque to pressure feedforward.Source: §V · p. 3
  • MPC predicts only the slow mechanical state and uses commanded pressure as input because full-state prediction is numerically unusable.The full model has a mode near 430 rad/s; the reduced prediction-matrix entry falls to 70–160.Source: §V · p. 4
  • MPC achieves zero colliding steps on both benchmarks with tracking comparable to LQR.The outer QP uses an exact adjacent-bellows constraint with a 0.10 rad margin.Source: §V · p. 4
  • At 0.25 kg, the estimator reduces pitch-joint error from 0.062 to 0.009 rad for LQR and from 0.077 to 0.001 rad for MPC.Worst-joint errors are 0.009 and 0.007 rad, respectively, versus PID's 0.049 rad.Source: §V · p. 4

VI. LEARNING A REAL-TIME POLICY

Model-free RL struggles on the strict settled-success metric, while MPC decisions provide targets for supervised policy distillation with DAgger. The learning setup canonicalizes goals to make the target actions consistent and uses bounded actuator commands.

VI. LEARNING A REAL-TIME POLICY

  • The goal-conditioned MDP observes plant state and end-effector goal positions, while bounded actions map to torque and commanded pressure; collisions are measured rather than enforced.The per-step reward combines goal distance, action effort, and a smooth collision potential; terminal events end the episode.Source: §VI · p. 4
  • At N=2, model-free RL reaches 46.8% settled success, rising to 85.0% when pressure dynamics are removed.At N=1 it reaches 74.2%; the pressure ablation leaves the remaining multisegment-coordination gap.Source: §VI · p. 4
  • Distillation fits a student to MPC decisions by supervised learning, and DAgger adds labels at states visited by student rollouts to address covariate shift.Training uses settled teacher episodes; labels from failed QP solves are dropped because the teacher then holds its previous command rather than producing its policy action.Source: §VI · p. 4
  • Deterministic inverse-kinematics canonicalization assigns one joint-space target to each task-space goal, avoiding conflicting action labels for identical student inputs.The procedure selects a posture holding below 30 kPa when available, otherwise the lowest-pressure converged candidate.Source: §VI · pp. 4–5
  • DAgger iterations roll out the student, label visited states with the canonicalized MPC action, refit from scratch, and stop when settled success does not improve.The student observation represents joint angles as sine and cosine for goals that swing the yaw through ±π; MPC commands are invertible into student action coordinates.Source: §VI · p. 5

VII. THE TEACHER’S CEILING AT THREE SEGMENTS

At three segments, pressure-aware posture selection broadens the admitted goal set and raises MPC teacher success. The remaining ceiling is associated with a limit cycle involving the bellows’ lightly damped mode.

VII. THE TEACHER’S CEILING AT THREE SEGMENTS

  • On the narrow task’s 300-goal diagnostic set, 268 settle, 27 time out, and 5 collide; the collisions involve non-adjacent segments 1 and 3.The outer QP does not constrain that segment pair; timeouts oscillate across the 10 mm boundary.Source: §VII · p. 5
  • On 40 diagnostic goals, integral action settles 39 with a 1,500-step budget but only 18 within the protocol’s 500 steps, versus 38 without it.Raising the pressure bound to 158.6 kPa lowers success to 40%.Source: §VII · p. 5
  • The MPC holds every tested goal at 30 kPa but loses most at 50 and 70 kPa, consistent with a limit cycle entered between 30 and 50 kPa.Without the outer QP, a 1 mm bellows kick rings out to 13 mm at 50 and 70 kPa but stays near 1 mm at 30 kPa.Source: §VII · p. 5, Table I · p. 5
  • 99.7% of 300 fresh goals settle without collisions when goals use pressure-aware postures, compared with 90.1% on the narrow task.The admitted set reaches 89% of raw draws, and median holding pressure falls from 0.55 to 0.29 of the limit.Source: §VII · p. 5

VIII. EXPERIMENTS

Experiments show that coupling varies across joints, and coupled model-based controllers outperform decoupled PID on tracking while PID remains competitive at the nearly decoupled pitch joint. DAgger distillation produces a fast, collision-free policy, while evaluations expose limits in model-free learning, real-time MPC, and robustness to plant mismatch.

RQ1: Pressure modeling

  • Pressure modeling kept bellows RMS error within 2.4% across modeled, naive-PI, and ideal pressure loops, although pressure error differed 2×.The modeled chamber rolls off near 8 Hz, attenuating excitation of the bellows’ lightly damped 14.35 Hz mode; ideal pressure resonates hardest.Source: §VIII · p. 5

RQ3: Coupled control

  • LQR tracked 2.5× and MPC 2.4× tighter than PID on the worst joint, with 3.7× less torque; PID won only on the pitch joint.On the five-leg benchmark, worst-joint RMS error was 0.076 rad for PID, 0.044 for LQR, and 0.060 for MPC, with no collisions; the coupled and decoupled LQRs tracked similarly.Source: Table II · p. 6, §VIII · p. 6

RQ2: Coupling

  • Bellows-bellows coupling had a median ratio of 0.708, compared with 0.111 for cross-class coupling; mean coupling was 0.523 for yaw joints and 0.062 for pitch.The pitch angle switches cross-class coupling on; Coriolis terms contributed 0.118% of control authority at benchmark speed.Source: §VIII · p. 6

RQ4: Hysteresis

  • As α fell from 0.985 to 0.80, naive-controller tracking error grew 61%, while compensated-controller error grew 7.5%.The comparison supports compensation reducing sensitivity to crease hysteresis over the tested range.Source: §VIII · p. 6

Distillation and RQ6: Model-free RL

  • At N = 2, DAgger reached 90.3% pooled success after one round and a second removed collisions; on identical goals, the student reached 92.4% versus 95.0% for its teacher.Model-free RL’s settled success plateaued, and its redesigned tracking remained an order of magnitude behind LQR and MPC.Source: §VIII · p. 6

Robustness

  • Under mass mismatch at N = 3, the student degraded further than the teacher, whose misses reflected uncorrected gravity sag; the student also had two collisions at +40%.With a sustained payload, the teacher held 100% at N = 3 and 0.10 kg, while the student reached 42.4%; the authors conclude N = 3 robustness needs a teacher with integral action.Source: §VIII · p. 6, Figure 4 · p. 7

RQ5: Computation

  • The student’s forward pass cost 0.18 ms at N = 2 and 0.32 ms at N = 3, while cached MPC exceeded the control step at both sizes.At N = 3, MPC also exceeded its own 20 ms cadence; rebuilding the QP cost 5–15× more than cached MPC.Source: §VIII · p. 6, Figure 3 · p. 7

Distillation

  • 94.2% pooled settled success was reached on the full task, with zero collisions across 1,000 full-task and 1,000 failing-region episodes.Three hard-region rounds gained 2.4, 2.6, and 1.4 points; a fourth fell to 93.6%.Source: §VIII · p. 7

IX. CONCLUSION

The coupled model measures substantial inertial coupling that predicts controller rankings joint by joint. Distilled MPC policies approach the teacher’s performance at lower per-decision cost, while the conclusion identifies scope limits and hardware validation as future work.

IX. CONCLUSION

  • The distilled policy comes within a few points of the teacher at N=2 and N=3, with zero collisions and a small fraction of its per-decision cost.At N=2 it also matches the teacher’s robustness.Source: §IX · p. 8
  • Inertial coupling is large and predicts controller rankings joint by joint; coupled LQR and MPC track tighter than decoupled PID at lower torque.A shared disturbance estimator also makes their disturbance rejection outperform PID’s.Source: §IX · p. 8
  • The tracking advantage over PID at benchmark speed comes from model-based feedforward and full-state feedback, not cross terms.The conclusion therefore grounds the case for coupling in its measurement, collision constraint, and higher speeds—not Table II.Source: §IX · p. 8
  • Low-pressure postures among redundant choices avoid the teacher’s ceiling, a limit cycle with the bellows’ lightly damped mode.Future work includes hardware validation, richer cycling measurements to pin down α, and a teacher with integral action for N=3 mismatch and payload robustness.Source: §IX · p. 8
Loading 2609.29424v1…