PaperlayerView exact revision on arXiv

Source-linked AI summary

Real-Time Shape Control of Multi-Segment Soft Robotic Arms Using Koopman Operators with Global and Local Observables

Jiahe Wang, Eron Ristich, Sultan Haidar Ali, Eric weissman, Lei Zhang, Wanxin Jin, Yi Ren, Jiefeng Sun

arXiv:2609.03175v1cs.ROeess.SY

TL;DR

Tip-based control leaves a soft arm’s body deformation undetermined, while global observables alone are insufficient for multi-segment shape control under nonlinear continuum dynamics. The paper combines global and local observables in a Koopman-based dense-MPC framework, demonstrating improved numerical scalability and real-time shape-control performance.

  • Problem

    Tip-based control leaves infinite-dimensional body deformation undetermined, while global-frame observables do not describe local deformation in multi-segment soft robots with strongly nonlinear dynamics.

  • Method

    The paper combines global and local observables within a unified Koopman-based dense-MPC architecture for full-body shape control.

  • Results

    The combined-observable controller achieves substantially greater accuracy and improved scalability in numerical experiments, including robots with up to 10 independently actuated segments.

  • Takeaways & Limitations

    Observable mappings and the control cost function are more critical to multi-segment shape-control performance than increasing model complexity, within the demonstrated scope.

  • Takeaways & Limitations

    The current hardware remains limited by external motion capture, predefined unplanned reference trajectories, carefully curated training data, and actuator and sensing constraints.

Abstract

from arXiv · show

Multi-segment soft robotic arms can continuously reconfigure their body shapes for safe interaction, but tip control alone is insufficient for constrained-space tasks. Therefore, shape control is a more important task for multi-segment soft arms than tip control, but remains challenging due to the high dimensionality and nonlinear dynamics of continuum deformation. In existing work, shape control accuracy is defined by the error in the global frame (global shape error). For multi-segment soft arms, using only global shape error as the control objective is insufficient, as segment coupling, gravity-induced loading, and inertial effects become more significant. This difficulty increases with the number of segments. In this paper, we present a Koopman-based model predictive control framework that combines global and local observables, enabling real-time shape control on multi-segment soft robotic arms. The framework is evaluated through numerical and physical experiments. Numerical experiments demonstrate the scalability of the proposed controller by achieving shape control on robots with up to 10 independently actuated segments. The physical experiments demonstrate that the controller is capable of (1) real-time shape control of 3- and 5-segment robotic arms with tip speeds up to 0.6 m/s, (2) robust tracking without retraining, including distal payloads up to 400~g and recovery from a 7~N lateral disturbance, and (3) the potential for future inspection applications through a confined-space demonstration. These results demonstrate that the proposed framework enables dynamic, scalable, and accurate real-time shape control on multi-segment soft robotic arms.

I. INTRODUCTION

Multi-segment soft-arm shape control requires regulating full-body deformation rather than only the tip, but real-time control remains difficult as nonlinear coupling and segment count increase. The paper addresses this with Koopman MPC using combined global and local observables.

  • Tip control leaves body deformation undetermined, making it inadequate for tasks such as navigating cluttered environments.
  • Increasing segment count strengthens gravity-induced coupling because proximal segments support and compensate for distal segments whose loading changes with robot shape.
  • Existing real-time shape-control studies are often restricted to planar or small-deformation motions and robots with few segments.
  • Global-frame Koopman observables can underfit local backbone deformation because they encode global pose without local-frame deformation information.
  • The proposed framework combines global and local observables in a Koopman-based dense MPC formulation to regulate full-body configuration and reduce both global and local shape errors.
  • Simulation scales to robots with up to 10 independently actuated segments, while hardware validation uses 3- and 5-segment robots with real-time full-body tracking and robustness tests.

A. Koopman Operator Theory

Koopman theory converts nonlinear controlled dynamics into a linear representation over observables. The paper approximates this representation from trajectory data using EDMDc and LASSO-regularized least squares.

  • The Koopman operator propagates scalar observables of a discrete-time controlled system and is linear but infinite-dimensional.
  • A finite-dimensional Koopman model is constructed from a finite set of functions spanning an approximated invariant subspace.
  • EDMDc extends EDMD to systems with control inputs by fitting the Koopman approximation from sampled state, next-state, and input trajectories.
  • The finite-dimensional operator is obtained through least squares, with the Moore–Penrose pseudoinverse used in the basic formulation.
  • LASSO adds L1 regularization to improve sparsity, reduce learned eigenvalue magnitudes, and improve stability under noise or outliers.

C. Koopman MPC and Dense Form

The Koopman MPC formulation uses a lifted linear model, reconstructs the original state through an output matrix, and optimizes controls efficiently in dense form. This supports high-frequency real-time control despite large lifting dimensions.

  • The control-affine Koopman formulation enables linear MPC as a quadratic program with positive semidefinite gain matrices.
  • The Koopman model uses a state-dependent lifting function and zero-order-hold inputs to produce a linear system in the lifted state.
  • An output matrix C reconstructs the original state from the lifted state, enabled by including the state in the dictionary.
  • MPC tracks a finite-horizon reference trajectory by defining error as e_k = Cz_k − r_k and optimizing control inputs under linear constraints.
  • Dense MPC directly optimizes the control sequence rather than propagating the high-dimensional lifted state across the prediction horizon.
  • Precomputed dense-MPC matrices make the quadratic program independent of the large lifting dimension and enable update rates exceeding 300 Hz on a standard CPU.

III. GLOBAL, LOCAL, AND COMBINED OBSERVABLES

The paper defines control-relevant observable mappings for Koopman lifting, using global task-space positions, reduced geometric features, and delay coordinates. The proposed combined representation integrates global consistency with local detail for shape control.

  • Observable mappings: The Koopman framework compares global, local, and combined observable mappings that determine the lifted state and control representation.The lifting function composes an observable mapping h with delay embedding Ψ.
  • Lifting function design: The reduced control state extracts geometric features from measured states instead of directly retaining all position and orientation coordinates.This avoids adding orientation constraints that increase model complexity without directly improving backbone shape control.
  • Lifting function design: Delay coordinates augment instantaneous geometric features to represent latent dynamics such as velocities and internal forces.The lifted state uses a delay horizon Nd through Ψ.
  • 1) Global observable: Global-coordinate representations work well for low-dimensional tip or planar-shape tasks with no more than 3 segments but can become unstable for multi-segment shape control.The paper motivates reduced global observables as a way to improve model consistency and control stability.
  • 1) Global observable: Global observables represent backbone positions in the fixed global frame, with projections onto its x–z normal plane for more stable modeling.The projection removes the poorly modeled extension direction associated with strong nonlinearity.

2) Local observable:

The local observable expresses each backbone point in its corresponding segment-base frame to capture segment-wise deformation. Because it omits explicit task-space position, local errors can accumulate into global drift.

  • 2) Local observable: The local observable introduces segment-frame information to better represent deformation for multi-segment soft robotic arms.Each measured point is assigned to a segment through λ(i), and its position is expressed in that segment’s local base frame.
  • 2) Local observable: Local coordinates capture segment-wise deformation by projecting each point onto the normal plane of its corresponding segment frame.The projection uses a segment-specific matrix ¯R{Bλ(i)} ∈ R2×3.
  • 2) Local observable: The local representation does not explicitly encode task-space position, so small local errors can propagate along the serial chain and produce global drift.This limitation motivates combining local and global observables.

3) Combined observable:

The combined observable concatenates global and local representations to preserve task-space consistency while capturing segment-level deformation. Its main control benefit is improved MPC objective conditioning rather than explicitly improved prediction accuracy.

  • 3) Combined observable: The combined observable unifies global and local representations to exploit their complementary strengths in shape control.It retains global position information while adding local deformation detail.
  • 3) Combined observable: The combined lifted state is formed by concatenating the independently obtained global and local lifted states.The corresponding Koopman model is constructed from independently trained global and local models.
  • 3) Combined observable: The block-diagonal combined model does not explicitly improve prediction accuracy; its benefit comes from richer state representation and better MPC objective conditioning.The combined MPC cost jointly penalizes deviations in global and local representations through QCombined = diag(QGlobal, QLocal).
  • C. Global and Local Shape Error: Global shape error measures average squared backbone-position discrepancies, but different segment deformations can produce similar global errors.The metric includes tip error as the special case at the final measured point.
  • C. Global and Local Shape Error: Local shape error captures deformation in segment frames, but local errors can accumulate into global task-space drift; therefore both metrics are reported.The two metrics provide complementary assessments of global position and local deformation.
  • C. Global and Local Shape Error: The MPC objective is optimized in the lifted state rather than directly minimizing exact geometric errors such as eglobal or elocal.Observable mappings are designed so that the lifted quadratic objective indirectly reflects geometric shape errors.

D. Special Considerations

The framework uses practical modeling and data-collection choices to keep Koopman-based MPC within a tractable dynamical regime. Simulation and training use Kirchhoff-rod dynamics, PD motor actuation, ramp-and-hold inputs, and reduced tip-point state representations.

  • Input increments Δu replace absolute inputs u in MPC, explicitly limiting actuation changes to avoid regimes dominated by unmodeled inertial effects.The constraint is ∥Δu_k∥∞≤Δu_max, while preserving sufficient actuation for meaningful motion.
  • Reference trajectories come from experimentally collected motions generated by predefined input sequences excluded from the Koopman training dataset.Reference generation is treated as outside the paper’s main focus.
  • The control state uses reduced tip-point representations because denser intermediate measurements reduce physical control stability, unlike noiseless numerical simulations.The same reduced representation is used in simulation for consistency.
  • The simulated arm separates backbone, cables, and plates, with Kirchhoff-rod dynamics solved using the implicit BDF-α method.Cable forces are incorporated into the backbone model.
  • Each segment has two independent control inputs, with motor angles driving cable forces through PD-controlled antagonistic cable pairs.The motor model computes cable force from motor torque using the pulley radius.
  • Training data comprise 200,000 snapshots for 3-, 5-, 8-, and 10-segment arms, generated using randomly sampled ramp-and-hold inputs.The Koopman interval is Δt=0.01s, with tr=40Δt, th=10Δt, and N=4000.

B. Numerical Experiment Results

Numerical experiments show that global observables become inadequate as the number of segments increases, whereas combined global and local observables maintain accurate tracking across tested configurations. The combined controller also achieves lower error variance, although performance degrades with increasing system complexity.

  • The global-observable controller fails beyond 3 segments, while combined-observable control tracks 5-, 8-, and 10-segment reference shapes with substantially lower residual error.Representative trajectories show the global controller converging to high-error configurations as complexity increases.
  • Combined observables reduce the magnitude of steady-state drift caused by modeling inaccuracies in the Koopman approximation.Both controllers can exhibit transient oscillations arising from the soft robot’s dynamics.
  • Orders of magnitude higher accuracy across global and local error metrics is achieved by combined observables than by global observables across all tested robots.The comparison covers 200 simulations with 200 time steps, and percentile ranges show smaller variance for the combined controller.
  • The converged global shape error of the combined-observable controller empirically grows exponentially as the number of segments increases.All controllers show decreasing performance in each metric with more segments.
  • In a three-cable force-driven setting with dense backbone measurements, global and combined controllers failed to converge, while the local-observable controller remained stable.These results motivate introducing a PD motor controller and support the observable-design conclusions.

V. PHYSICAL EXPERIMENTS

Physical experiments evaluate the proposed method on 3- and 5-segment prototypes using modular, locally actuated cable mechanisms and motion-capture state reconstruction. The setup supports comparisons across observables, varied target shapes, robustness tests, and confined-space evaluation.

  • The physical evaluation uses 3- and 5-segment prototypes to compare global, local, and combined observables under shared reference trajectories.The 5-segment evaluation includes multiple target shapes with varied deformation patterns.
  • Each segment carries two orthogonally mounted motors that actuate antagonistic cable pairs through dual-groove pulleys.The proposed architecture mounts motors at the base of each segment rather than centralizing all actuators at the robot base.
  • Motor position increments Δu∈R6 define the control input for the physical robot.Oppositely wound cable pairs allow one cable to be pulled while the other is released during motor rotation.
  • Motion capture measures robot shape at 100 Hz and reconstructs seven rigid-body poses along the 3-segment backbone, yielding x∈R49.Markers on intermediate and distal plates define global poses and a local base frame.
  • The integrated system uses internal motor PID control and ROS2 for state reconstruction, reference generation, synchronization, logging, and real-time K-MPC computation.

5) Data Collection:

The evaluation compares global, local, and combined observables for real-time shape tracking of a 3-segment soft robotic arm under slow and fast motion. The combined observable provides the strongest overall tracking performance, while faster motion increases errors for all controllers.

  • Observable comparison: The 3-segment evaluation uses identical training data and reference trajectories while comparing global, local, and combined observables with dense MPC at 100 Hz.The controller uses Nd = 20 time-delay coordinates and a prediction horizon of NH = 5.
  • Observable comparison: The combined observable provides the best tracking performance under both slow and fast motions.It achieves lower errors than either global or local observables alone, although faster motion increases errors for all controllers.
  • Slow shape control: During slow tracking, the global observable produces the largest global-frame and local shape errors, especially for distal points.The results indicate that global observables alone do not accurately capture segment-wise deformation.
  • Slow shape control: The combined observable has the smallest global and local shape errors for most of the slow-tracking trajectory.The local observable improves shape-error performance but leaves larger global-frame errors.
  • Fast shape control: 0.6 m/s is the approximate maximum tip speed during fast tracking, when motors operate near their speed limitation.At this speed, the combined observable still yields lower errors for Seg 1 and Seg 2, while Seg 3 shows a lower global-frame tip RMSE with the global observable but large shape errors.
  • Fast shape control: Tip error alone cannot determine the multi-segment robot shape because low tip error can coexist with incorrect backbone deformation.Full-body shape metrics are therefore needed to evaluate segment-wise deformation.

D. Control the 5-Segment Robot

The 5-segment experiments test diverse shapes, payloads, external disturbances, and confined-space motion. The combined observable maintains global and local tracking while remaining stable under unmodeled effects and supporting a 28 s inspection demonstration.

  • D. Control the 5-Segment Robot: Six target shapes span multi-curvature bending, asymmetric deformation, and helix patterns across the robot workspace.The shapes are executed in one sequential ramp-and-hold trial.
  • D. Control the 5-Segment Robot: 718.63 mm2 is exceeded by local shape errors under the global observable for large and asymmetric deformations.The local observable achieves significantly lower local shape errors for Shapes 1, 3, and 6.
  • D. Control the 5-Segment Robot: 0.4 m/s is the approximate maximum tip speed during 5-segment shape tracking.Oscillatory velocity reflects inertia and shape deformation, while speed decays toward zero during holds.
  • D. Control the 5-Segment Robot: 2739.92 mm2 (5.23%L) average global shape error and 52.19 mm2 (3.61%Lseg) average local shape error are achieved across all six shapes.The combined observable provides the best tracking performance across the diverse shape set.
  • Robustness evaluation: 400 g distal payloads and unknown disturbances increase tracking error, but the combined-observable controller remains stable without retraining.The evaluation includes 200 g and 400 g payloads, continuous airflow, and sudden lateral pulling.
  • F. Confined-Space Shape-Control Demonstration: 28 s covers approach, aperture traversal, target observation, and return motion in the confined-space demonstration.The 5-segment arm follows manually specified reference shapes through a circular aperture toward a target.

VI. DISCUSSION

The discussion identifies observable and error-metric design as central to scalable multi-segment shape control, while outlining practical limits and future sensing and robustness extensions.

  • Choice of local observable: Observable mappings can matter more than model complexity because global coordinates ambiguously represent multi-segment deformation.Global-frame projections may map multiple backbone configurations to the same coordinates, whereas local projections preserve segment-wise geometry.
  • Choice of local observable: Local segment-frame observables make the coordinate-to-deformation mapping locally unique and avoid overlap in global projections.The resulting representation better captures segment-wise geometry and improves modeling accuracy for multi-segment arms.
  • Shape Error Metric: Global and local shape errors are both required because optimizing either metric alone can miss local deformation or accumulate task-space drift.Global observables can achieve lower global shape error while missing local deformation, whereas local observables can accumulate global error along the arc-length.
  • Limitations and Future Work: External motion capture, predefined data-derived references, self-contact restrictions, sensing rate, actuator capability, and highly inertial regimes constrain current deployment.Future work proposes richer onboard sensing, self-contact and payload handling, force-based actuation, and adaptive robustness across loading conditions.
  • Conclusion: The proposed Koopman dense-MPC framework combines global and local observables and reduces both global and local shape errors, with simulations scaling to 10 independently actuated segments.Physical scaling remains constrained by motor capacity and space constraints.
Loading 2609.03175v1…