Real-Time Shape Control of Multi-Segment Soft Robotic Arms Using Koopman Operators with Global and Local Observables

• Tip control leaves body deformation undetermined, making it inadequate for tasks such as navigating cluttered environments.Source: §I · p. 1
• Increasing segment count strengthens gravity-induced coupling because proximal segments support and compensate for distal segments whose loading changes with robot shape.Source: §I · p. 1
• Existing real-time shape-control studies are often restricted to planar or small-deformation motions and robots with few segments.Source: §I · p. 1
• Global-frame Koopman observables can underfit local backbone deformation because they encode global pose without local-frame deformation information.Source: §I · p. 2
• The proposed framework combines global and local observables in a Koopman-based dense MPC formulation to regulate full-body configuration and reduce both global and local shape errors.Source: §I · p. 2
• Simulation scales to robots with up to 10 independently actuated segments, while hardware validation uses 3- and 5-segment robots with real-time full-body tracking and robustness tests.Source: §I · p. 2

• The Koopman operator propagates scalar observables of a discrete-time controlled system and is linear but infinite-dimensional.Source: A. Koopman Operator Theory · p. 3
• A finite-dimensional Koopman model is constructed from a finite set of functions spanning an approximated invariant subspace.Source: A. Koopman Operator Theory · p. 3
• EDMDc extends EDMD to systems with control inputs by fitting the Koopman approximation from sampled state, next-state, and input trajectories.Source: B. Discrete-time Koopman Operator… · p. 3
• The finite-dimensional operator is obtained through least squares, with the Moore–Penrose pseudoinverse used in the basic formulation.Source: B. Discrete-time Koopman Operator… · p. 3
• LASSO adds L1 regularization to improve sparsity, reduce learned eigenvalue magnitudes, and improve stability under noise or outliers.Source: B. Discrete-time Koopman Operator… · p. 3

• The control-affine Koopman formulation enables linear MPC as a quadratic program with positive semidefinite gain matrices.Source: C. Koopman MPC and Dense Form · pp. 3–4
• The Koopman model uses a state-dependent lifting function and zero-order-hold inputs to produce a linear system in the lifted state.Source: C. Koopman MPC and Dense Form · pp. 3–4
• An output matrix C reconstructs the original state from the lifted state, enabled by including the state in the dictionary.Source: C. Koopman MPC and Dense Form · p. 4
• MPC tracks a finite-horizon reference trajectory by defining error as e_k = Cz_k − r_k and optimizing control inputs under linear constraints.Source: C. Koopman MPC and Dense Form · p. 4
• Dense MPC directly optimizes the control sequence rather than propagating the high-dimensional lifted state across the prediction horizon.Source: C. Koopman MPC and Dense Form · p. 4
• Precomputed dense-MPC matrices make the quadratic program independent of the large lifting dimension and enable update rates exceeding 300 Hz on a standard CPU.Source: C. Koopman MPC and Dense Form · p. 4

Observable mappings

• The Koopman framework compares global, local, and combined observable mappings that determine the lifted state and control representation.The lifting function composes an observable mapping h with delay embedding Ψ.Source: §III · p. 4, A. Lifting Function Design · p. 4

Lifting function design

• The reduced control state extracts geometric features from measured states instead of directly retaining all position and orientation coordinates.This avoids adding orientation constraints that increase model complexity without directly improving backbone shape control.Source: A. Lifting Function Design · p. 4, B. Observable Mapping Design · p. 4
• Delay coordinates augment instantaneous geometric features to represent latent dynamics such as velocities and internal forces.The lifted state uses a delay horizon Nd through Ψ.Source: A. Lifting Function Design · p. 4

1) Global observable

• Global-coordinate representations work well for low-dimensional tip or planar-shape tasks with no more than 3 segments but can become unstable for multi-segment shape control.The paper motivates reduced global observables as a way to improve model consistency and control stability.Source: 1) Global observable · pp. 4–5
• Global observables represent backbone positions in the fixed global frame, with projections onto its x–z normal plane for more stable modeling.The projection removes the poorly modeled extension direction associated with strong nonlinearity.Source: 1) Global observable · p. 5

2) Local observable

• The local observable introduces segment-frame information to better represent deformation for multi-segment soft robotic arms.Each measured point is assigned to a segment through λ(i), and its position is expressed in that segment’s local base frame.Source: 2) Local observable · p. 5
• Local coordinates capture segment-wise deformation by projecting each point onto the normal plane of its corresponding segment frame.The projection uses a segment-specific matrix ¯R{Bλ(i)} ∈ R2×3.Source: 2) Local observable · p. 5
• The local representation does not explicitly encode task-space position, so small local errors can propagate along the serial chain and produce global drift.This limitation motivates combining local and global observables.Source: 2) Local observable · p. 5

3) Combined observable

• The combined observable unifies global and local representations to exploit their complementary strengths in shape control.It retains global position information while adding local deformation detail.Source: 3) Combined observable · p. 5
• The combined lifted state is formed by concatenating the independently obtained global and local lifted states.The corresponding Koopman model is constructed from independently trained global and local models.Source: 3) Combined observable · p. 5
• The block-diagonal combined model does not explicitly improve prediction accuracy; its benefit comes from richer state representation and better MPC objective conditioning.The combined MPC cost jointly penalizes deviations in global and local representations through QCombined = diag(QGlobal, QLocal).Source: 3) Combined observable · p. 6, C. Global and Local Shape Error · p. 6

C. Global and Local Shape Error

• Global shape error measures average squared backbone-position discrepancies, but different segment deformations can produce similar global errors.The metric includes tip error as the special case at the final measured point.Source: C. Global and Local Shape Error · p. 6, Figure 3 · p. 6
• Local shape error captures deformation in segment frames, but local errors can accumulate into global task-space drift; therefore both metrics are reported.The two metrics provide complementary assessments of global position and local deformation.Source: C. Global and Local Shape Error · p. 6, Figure 3 · p. 6
• The MPC objective is optimized in the lifted state rather than directly minimizing exact geometric errors such as eglobal or elocal.Observable mappings are designed so that the lifted quadratic objective indirectly reflects geometric shape errors.Source: C. Global and Local Shape Error · p. 6

• Input increments Δu replace absolute inputs u in MPC, explicitly limiting actuation changes to avoid regimes dominated by unmodeled inertial effects.The constraint is ∥Δu_k∥∞≤Δu_max, while preserving sufficient actuation for meaningful motion.Source: D. Special Considerations · p. 6
• Reference trajectories come from experimentally collected motions generated by predefined input sequences excluded from the Koopman training dataset.Reference generation is treated as outside the paper’s main focus.Source: D. Special Considerations · p. 6
• The control state uses reduced tip-point representations because denser intermediate measurements reduce physical control stability, unlike noiseless numerical simulations.The same reduced representation is used in simulation for consistency.Source: D. Special Considerations · p. 6
• The simulated arm separates backbone, cables, and plates, with Kirchhoff-rod dynamics solved using the implicit BDF-α method.Cable forces are incorporated into the backbone model.Source: A. Description of Simulation Setup · p. 7
• Each segment has two independent control inputs, with motor angles driving cable forces through PD-controlled antagonistic cable pairs.The motor model computes cable force from motor torque using the pulley radius.Source: A. Description of Simulation Setup · p. 7, Figure 4 · p. 7
• Training data comprise 200,000 snapshots for 3-, 5-, 8-, and 10-segment arms, generated using randomly sampled ramp-and-hold inputs.The Koopman interval is Δt=0.01s, with tr=40Δt, th=10Δt, and N=4000.Source: A. Description of Simulation Setup · pp. 7–8

• The global-observable controller fails beyond 3 segments, while combined-observable control tracks 5-, 8-, and 10-segment reference shapes with substantially lower residual error.Representative trajectories show the global controller converging to high-error configurations as complexity increases.Source: B. Numerical Experiment Results · p. 8
• Combined observables reduce the magnitude of steady-state drift caused by modeling inaccuracies in the Koopman approximation.Both controllers can exhibit transient oscillations arising from the soft robot’s dynamics.Source: B. Numerical Experiment Results · p. 8
• Orders of magnitude higher accuracy across global and local error metrics is achieved by combined observables than by global observables across all tested robots.The comparison covers 200 simulations with 200 time steps, and percentile ranges show smaller variance for the combined controller.Source: B. Numerical Experiment Results · pp. 8–9
• The converged global shape error of the combined-observable controller empirically grows exponentially as the number of segments increases.All controllers show decreasing performance in each metric with more segments.Source: B. Numerical Experiment Results · pp. 8–9
• In a three-cable force-driven setting with dense backbone measurements, global and combined controllers failed to converge, while the local-observable controller remained stable.These results motivate introducing a PD motor controller and support the observable-design conclusions.Source: B. Numerical Experiment Results · p. 9

• The physical evaluation uses 3- and 5-segment prototypes to compare global, local, and combined observables under shared reference trajectories.The 5-segment evaluation includes multiple target shapes with varied deformation patterns.Source: §V · p. 9
• Each segment carries two orthogonally mounted motors that actuate antagonistic cable pairs through dual-groove pulleys.The proposed architecture mounts motors at the base of each segment rather than centralizing all actuators at the robot base.Source: 2) Actuation · p. 9, A. Robot Design and Data Collection · p. 9
• Motor position increments Δu∈R6 define the control input for the physical robot.Oppositely wound cable pairs allow one cable to be pulled while the other is released during motor rotation.Source: 2) Actuation · p. 10
• Motion capture measures robot shape at 100 Hz and reconstructs seven rigid-body poses along the 3-segment backbone, yielding x∈R49.Markers on intermediate and distal plates define global poses and a local base frame.Source: 3) Shape Measurement · p. 10
• The integrated system uses internal motor PID control and ROS2 for state reconstruction, reference generation, synchronization, logging, and real-time K-MPC computation.Source: 4) System Integration · p. 10

Observable comparison

• The 3-segment evaluation uses identical training data and reference trajectories while comparing global, local, and combined observables with dense MPC at 100 Hz.The controller uses Nd = 20 time-delay coordinates and a prediction horizon of NH = 5.Source: C. Control the 3-Segment Robot · p. 11
• The combined observable provides the best tracking performance under both slow and fast motions.It achieves lower errors than either global or local observables alone, although faster motion increases errors for all controllers.Source: C. Control the 3-Segment Robot · p. 11, 2) Fast shape control · p. 12

Slow shape control

• During slow tracking, the global observable produces the largest global-frame and local shape errors, especially for distal points.The results indicate that global observables alone do not accurately capture segment-wise deformation.Source: 1) Slow shape control · p. 11
• The combined observable has the smallest global and local shape errors for most of the slow-tracking trajectory.The local observable improves shape-error performance but leaves larger global-frame errors.Source: 1) Slow shape control · p. 12

Fast shape control

• 0.6 m/s is the approximate maximum tip speed during fast tracking, when motors operate near their speed limitation.At this speed, the combined observable still yields lower errors for Seg 1 and Seg 2, while Seg 3 shows a lower global-frame tip RMSE with the global observable but large shape errors.Source: 2) Fast shape control · p. 12
• Tip error alone cannot determine the multi-segment robot shape because low tip error can coexist with incorrect backbone deformation.Full-body shape metrics are therefore needed to evaluate segment-wise deformation.Source: 2) Fast shape control · p. 12

D. Control the 5-Segment Robot

• Six target shapes span multi-curvature bending, asymmetric deformation, and helix patterns across the robot workspace.The shapes are executed in one sequential ramp-and-hold trial.Source: D. Control the 5-Segment Robot · pp. 12–13
• 718.63 mm2 is exceeded by local shape errors under the global observable for large and asymmetric deformations.The local observable achieves significantly lower local shape errors for Shapes 1, 3, and 6.Source: D. Control the 5-Segment Robot · p. 13
• 0.4 m/s is the approximate maximum tip speed during 5-segment shape tracking.Oscillatory velocity reflects inertia and shape deformation, while speed decays toward zero during holds.Source: D. Control the 5-Segment Robot · p. 13
• 2739.92 mm2 (5.23%L) average global shape error and 52.19 mm2 (3.61%Lseg) average local shape error are achieved across all six shapes.The combined observable provides the best tracking performance across the diverse shape set.Source: D. Control the 5-Segment Robot · p. 13

Robustness evaluation

• 400 g distal payloads and unknown disturbances increase tracking error, but the combined-observable controller remains stable without retraining.The evaluation includes 200 g and 400 g payloads, continuous airflow, and sudden lateral pulling.Source: E. Robustness Experiment · p. 13, 1) Distal Payload · pp. 13–14, 2) External Disturbances · p. 14

F. Confined-Space Shape-Control Demonstration

• 28 s covers approach, aperture traversal, target observation, and return motion in the confined-space demonstration.The 5-segment arm follows manually specified reference shapes through a circular aperture toward a target.Source: F. Confined-Space Shape-Control… · p. 14

Choice of local observable

• Observable mappings can matter more than model complexity because global coordinates ambiguously represent multi-segment deformation.Global-frame projections may map multiple backbone configurations to the same coordinates, whereas local projections preserve segment-wise geometry.Source: §VI · p. 14, A. Choice of local observable · p. 15
• Local segment-frame observables make the coordinate-to-deformation mapping locally unique and avoid overlap in global projections.The resulting representation better captures segment-wise geometry and improves modeling accuracy for multi-segment arms.Source: A. Choice of local observable · p. 15, Figure 14 · p. 15

Shape Error Metric

• Global and local shape errors are both required because optimizing either metric alone can miss local deformation or accumulate task-space drift.Global observables can achieve lower global shape error while missing local deformation, whereas local observables can accumulate global error along the arc-length.Source: B. Shape Error Metric · p. 15

Limitations and Future Work

• External motion capture, predefined data-derived references, self-contact restrictions, sensing rate, actuator capability, and highly inertial regimes constrain current deployment.Future work proposes richer onboard sensing, self-contact and payload handling, force-based actuation, and adaptive robustness across loading conditions.Source: C. Limitations · p. 15, §VII · pp. 15–16

Conclusion

• The proposed Koopman dense-MPC framework combines global and local observables and reduces both global and local shape errors, with simulations scaling to 10 independently actuated segments.Physical scaling remains constrained by motor capacity and space constraints.Source: §VII · p. 15