PaperlayerView exact revision on arXiv

Source-linked AI summary

Multimodal MPC via Bayesian Set Optimization for Dynamic Manipulation of Deformable Linear Objects

Youyuan Long, Gokhan Solak, Arash Ajoudani

arXiv:2610.31852v1cs.RO

TL;DR

Dynamic DLO manipulation is challenging because its nonlinear, underactuated dynamics respond with delay, and obstacles can leave multiple feasible motions. BM2PC combines Bayesian set optimization with execution-aware prediction; it generalizes to unseen tracking trajectories and reduces planning time in obstacle avoidance while maintaining comparable performance.

  • Problem

    Dynamic DLO manipulation must handle underactuated, high-dimensional dynamics, delayed deformation, and multiple feasible motions under obstacle constraints.

    Source: §I · p. 1

  • Method

    BM2PC uses Bayesian set optimization to maintain diverse high-quality control modes with an incremental update, and predicts actual end-effector motion to account for execution mismatch.

    Source: §I · p. 2

  • Results

    BM2PC generalizes better to unseen tracking tasks than the learning-based baseline and achieves comparable obstacle-avoidance solution quality with substantially less computation.

    Source: §VI · p. 15

  • Takeaways & Limitations

    The results support BM2PC as an efficient framework for real-time multimodal control of deformable linear objects.

    Source: §VI · p. 15

Abstract

from arXiv · show

Dynamic manipulation of deformable linear objects (DLOs) remains challenging due to underactuated, high-dimensional dynamics, delayed deformation responses, and the presence of multiple feasible motions under obstacle constraints. This paper presents BM2PC, a Bayesian Multimodal MPC framework for agile manipulation of DLOs. Building on the control-as-inference posterior, BM2PC formulates multimodal planning as a set optimization problem that balances control quality and behavioral diversity. An efficient incremental update maintains multiple promising control modes with low additional computational overhead. For real-world deployment, BM2PC incorporates an execution dynamics model that predicts actual end-effector motion and compensates for execution mismatch during fast manipulation. We evaluate BM2PC on rope tracking and obstacle avoidance in simulation and hardware, and on simulated disturbance rejection. In trajectory tracking, BM2PC generalizes better to unseen trajectories than a learning-based baseline without task-specific policy training or fine-tuning. Real-world experiments demonstrate accurate long-horizon tracking, reducing RMSE by 44.0% on the longest task. In obstacle avoidance, BM2PC achieves comparable performance to existing multimodal MPC methods while reducing planning time by 29% to 66%. These results demonstrate accurate dynamic control and efficient multimodal planning within a unified MPC framework. The core implementation of BM2PC and supplementary videos are available at https://github.com/Longyouyuan/BM2PC.

I. INTRODUCTION

Dynamic DLO control must predict delayed, nonlinear responses while preserving alternative motions as constraints change. BM2PC addresses the efficiency gap in multimodal MPC and is evaluated on tracking and obstacle avoidance.

I. INTRODUCTION

  • Fast DLO manipulation is difficult because underactuated, nonlinear dynamics produce delayed and complex deformation, while obstacles and changing conditions can make multiple motion strategies feasible.Existing learning-based methods may generalize poorly to unseen conditions, while multimodal optimization can add computation that limits agile manipulation.Source: §I · pp. 1–2
  • BM2PC uses a control-as-inference posterior to select a compact set of high-quality, behaviorally diverse control sequences through an incremental update, without task-specific policy training.The selected modes persist across MPC steps to support adaptation as constraints change.Source: §I · p. 2
  • In dynamic rope manipulation, BM2PC generalizes more strongly than a learning-based method to unseen trajectories without task-specific policy training or fine-tuning, and achieves comparable control performance to existing multimodal MPC methods with 29% to 66% less planning time.Source: §I · p. 2

A. Manipulation of Deformable Linear Objects

  • Prior DLO manipulation methods often assume quasi-static or slowly varying deformation, whereas fast endpoint motions require predicting delayed and oscillatory responses.Optimization-based methods can be costly as model complexity, horizon length, and constraints grow; learning-based methods often require task-specific data and can struggle to generalize.Source: A. Manipulation of Deformable Linear… · pp. 2–3

B. Sampling-Based and Multimodal Model Predictive Control

  • Sampling-based MPC supports parallel evaluation of complex predictive models and non-smooth costs, but standard single-mode sampling can miss separated low-cost solutions.Existing multimodal approaches use methods such as clustering, density estimation, or particle transport; BM2PC instead uses a lightweight incremental update and is compared under matched planning conditions.Source: B. Sampling-Based and Multimodal Model… · p. 3

III. PRELIMINARY

The preliminary formulation casts finite-horizon control as minimizing trajectory cost, then reframes control through probabilistic inference over optimal trajectories. A variational distribution approximates the resulting posterior over control sequences.

A. Finite-horizon optimal control

  • At each MPC step, finite-horizon optimal control starts from the observed state and selects a control sequence to minimize rollout cost over the horizon.The formulation defines state and control sequences, system dynamics, and stage and terminal costs.Source: A. Finite-horizon optimal control · p. 3

B. Control as probabilistic inference

  • Control-as-inference introduces an optimality variable and uses a variational distribution over admissible control sequences to approximate the posterior conditioned on optimality and the current state.The optimality variable is one when the trajectory is optimal; the posterior combines the optimality likelihood with a prior over controls.Source: B. Control as probabilistic inference · pp. 3–4

IV. METHOD

BM2PC combines multimodal planning, DLO simulation, execution-aware motion prediction, and collision detection in an MPC pipeline. Its set optimization selects high-posterior control sequences while encouraging behavioral diversity.

1) Problem formulation:

  • BM2PC optimizes a finite set of control sequences for high posterior probability and separation in trajectory-feature space, rather than approximating the full posterior distribution.The trajectory feature map used in the experiments retains predicted rope-tip positions over the planning horizon.Source: 1) Problem formulation · p. 4

IV. METHOD

  • The BM2PC pipeline evaluates candidate controls with the execution and DLO dynamics models and collision detection, then updates a diverse set of high-quality modes for the next MPC iteration.One mode is selected for execution while the updated set is propagated forward.Source: Figure 1 · p. 4, §IV · p. 4

U JBM2PC(U ) (4)

BM2PC balances control-sequence posterior quality against behavioral diversity, approximating multimodal planning with sampled candidates and incremental selection. The update avoids combinatorial subset enumeration, but its step-dependent normalization has no standard greedy approximation guarantee.

U JBM2PC(U ) (4)

  • BM2PC’s set objective combines each control sequence’s log-posterior density with a term encouraging diversity among predicted trajectories.The objective is formulated over a set of control sequences.Source: U JBM2PC(U ) (4) · p. 5

2) Sampled Set Optimization:

  • Sampling from an adaptive Gaussian-mixture prior concentrates computation around previous modes, while prior probability and rollout cost jointly determine candidate posterior quality.The mixture centers are updated after each optimization iteration, and the search is restricted to subsets of the finite sampled candidate set.Source: 2) Sampled Set Optimization · p. 5

3) Incremental Multimodal Update:

  • Because selecting n candidates from N creates combinatorially many subsets, the method adds candidates sequentially by maximizing marginal contribution to the objective.Each incremental score combines a candidate’s posterior score with its diversity contribution relative to the modes already selected.Source: 2) Sampled Set Optimization · p. 5, 3) Incremental Multimodal Update · p. 5
  • The objective is supermodular for nonnegative diversity weights but not necessarily monotone, and step-dependent normalization provides no standard greedy approximation guarantee.The normalized incremental update is evaluated against exhaustive set optimization in Sec. V-C.2.Source: 3) Incremental Multimodal Update · p. 5
  • The update replaces subset enumeration with n selections and adds modest overhead because rollout costs, prior densities, and trajectory-feature distances can be reused or precomputed.Each greedy step accumulates distance scores and evaluates an argmax over the candidate pool.Source: 3) Incremental Multimodal Update · p. 5

4) BM2PC algorithm:

Each BM2PC MPC update samples and rolls out control candidates around the current modes, then updates the multimodal set for the next iteration. One sequence is selected for execution, and shifted modes warm-start the next MPC step.

4) BM2PC algorithm:

  • Each optimization iteration samples candidates around the current control modes, rolls them out in parallel for trajectories and costs, and uses the resulting multimodal set as the next sampling center.The first iteration warm-starts from the shifted multimodal set produced by the previous MPC step.Source: 4) BM2PC algorithm · p. 5
  • Before multimodal updating, BM2PC retains the K lowest-cost candidates and excludes candidates above a cost threshold.Distances are computed on demand in parallel rather than by constructing a full pairwise distance matrix.Source: 4) BM2PC algorithm · p. 5
  • BM2PC executes the first control of one selected sequence, then shifts each control sequence forward by one step and copies the second-to-last input into the vacant final entry.The experiments select the minimum-cost sequence; probabilistic selection from a softmax over negative rollout costs is also described.Source: 4) BM2PC algorithm · pp. 5–6

B. DLO Model

The nominal DLO model represents a rope as linked mass points with linear and bending spring–damper forces, gravity, and air damping. It applies the robot’s commanded acceleration at the attached endpoint, while recognizing that real execution may deviate from the command.

B. DLO Model

  • The rope model discretizes the DLO into N+1 mass points connected by N links, with linear and bending stiffness and damping parameters.For a homogeneous rope, the material parameters are spatially uniform; gravity and air damping are also included.Source: B. DLO Model · p. 6, Figure 2 · p. 6
  • The full rope state concatenates the mass-point states, and the model defines link vectors, lengths, unit directions, rest lengths, and adjacent-link bending angles.The displayed interior-point force expressions omit terms with indices outside the valid range at boundary points.Source: B. DLO Model · p. 6
  • Each mass point’s motion follows Newton’s second law and is integrated with symplectic Euler, while link forces model linear spring–damper behavior and bending deformation.The model also includes bending damping and external forces from gravity and air damping.Source: B. DLO Model · p. 6
  • The nominal model assumes ideal execution and uses the commanded end-effector acceleration as the rope endpoint’s boundary acceleration to form the controlled discrete rope model.On hardware, low-level tracking errors and unmodeled execution dynamics can make executed motion differ from the command; the paper introduces a separate model to account for this mismatch.Source: B. DLO Model · pp. 6–7

C. Execution Dynamics Model

BM2PC uses a learned execution dynamics model to predict actual end-effector motion from desired trajectories and recent tracking history. Its recurrent rollout predicts acceleration and integrates it into velocity and position, using independently recorded trajectories for training and testing.

C. Execution Dynamics Model

  • A GRU-based history encoder initializes a latent representation of current execution dynamics from recent measured and desired end-effector states.The rollout is conditioned on that latent state and the desired future trajectory.Source: Figure 3 · p. 7, C. Execution Dynamics Model · p. 7
  • At each rollout step, a GRUCell and MLP predict executed acceleration, which a physics-informed integration head converts into the next velocity and position.The predicted state is recursively fed into the next rollout step, so the model predicts acceleration rather than directly predicting both next position and velocity.Source: C. Execution Dynamics Model · p. 7, Figure 3 · p. 7
  • Training uses approximately 1.2 million synchronized robot state samples recorded at 1 kHz during randomized motions spanning slow, medium, and fast segments.Autoregressive multi-step rollouts are supervised over the horizon, and training and test windows come from independently recorded trajectories.Source: C. Execution Dynamics Model · p. 7

D. Collision Detection

Collision checking assigns infinite cost to rollouts whose discretized rope nodes intersect conservatively inflated cylindrical obstacles. The experiments use an ℓ1 rope-tip tracking loss, compare BM2PC with three baselines, and assess both single-mode control and multimodal planning.

D. Collision Detection

  • Obstacle collision checks test discretized rope nodes against cylinders inflated by a safety margin; any intersecting rollout receives infinite cost and is excluded.The paper describes this dense-node approximation as reliable in its experiments with negligible computational overhead.Source: D. Collision Detection · pp. 7–8

V. EXPERIMENTS AND RESULTS

  • The evaluation uses one mode for trajectory tracking and disturbance rejection, and three modes for obstacle avoidance when multiple feasible strategies exist.The experiments also examine multimodal maintenance, diversity promotion, and the incremental update’s accuracy and efficiency.Source: §V · p. 8

1) Cost function:

  • The rope-tip trajectory loss is an ℓ1 distance from the target, while collisions receive infinite loss.Source: 1) Cost function · p. 8

2) Baseline Controllers:

  • SPiD is a physics-informed learning controller, whereas C-MPPI clusters rollout trajectories and SVMPC uses SVGD to optimize a Bayesian particle-based posterior.The study compares BM2PC against SPiD, C-MPPI, and SVMPC; the baselines were implemented from their respective paper descriptions.Source: 2) Baseline Controllers · p. 8

3) Validation scenarios:

Simulation evaluation covers trajectory tracking, obstacle avoidance, and disturbance rejection, with tracking tested on smooth and hand-drawn trajectories. BM2PC tracks nearly all tested trajectories accurately, while SPiD struggles to reproduce detailed geometry on complex paths.

3) Validation scenarios:

  • The simulation evaluates three capabilities: tracking trajectories of different shapes and durations, avoiding moving obstacles, and rejecting random external-force perturbations.Source: 3) Validation scenarios · p. 8

B. Simulation Experiments

  • The simulated rope is 1 m long and modeled as 40 rigid segments; control outputs endpoint accelerations at 40 Hz for execution by a 1 kHz PD controller.The high-gain controller yields negligible command-execution mismatch in simulation, so the execution dynamics model is not needed there.Source: B. Simulation Experiments · p. 8

1) Trajectory Tracking:

  • Tracking tests use three smooth parametric paths and three complex hand-drawn paths, each at three durations, with the same controller configuration across all 18 tasks.The hand-drawn paths include sharp corners, abrupt turns, and rapid local geometric variation.Source: 1) Trajectory Tracking · pp. 8–9
  • BM2PC tracks nearly all tested trajectories accurately, while SPiD follows complex hand-drawn paths only at the overall-trend level and misses fine geometric details.The reported tracking errors include SpongeBob finger details and the final zigzag in the fastest PatrickStar case (T = 12 s); SPiD was evaluated on a mass-spring-damper DLO model.Source: 1) Trajectory Tracking · pp. 9–10, Figure 4 · p. 9

2) Obstacle Avoidance:

BM2PC maintains comparable tracking accuracy to multimodal MPC baselines while avoiding obstacles, and its update has lower inference time. In simulations, it also recovers from disturbances and sudden route blockages, with ablations showing gains from maintaining and diversifying modes.

2) Obstacle Avoidance:

  • 1.55 ms average inference time for BM2PC’s multimodal update, versus approximately 4.99 ms for C-MPPI’s multimodal optimization stage.The three obstacle-avoidance controllers achieve comparable tracking accuracy, and none collides with the obstacle during execution.Source: 2) Obstacle Avoidance · p. 10
  • Distinct leftward, rightward, and upward avoidance modes emerge near the obstacle, while modes collapse to one solution in regions far from it.Source: 2) Obstacle Avoidance · p. 11

3) Disturbance Rejection:

  • Tracking error returns below 5 cm within 0.68–0.93 s after disturbance onset, averaging 0.78 s; post-recovery RMSE averages 4.69 cm.Four 1 N disturbances, each lasting 50 ms, were applied at different motion phases and caused substantial transient rope-tip deviations.Source: 3) Disturbance Rejection · p. 11

1) Ablation of the Multimodal Update:

  • 64.3% success for BM2PC, versus 45.2% for n = 1 and 47.0% for β = 0; collision rates are 25.2%, 39.3%, and 35.7%, respectively.The improvement is particularly pronounced at τ = 0.70–0.80 s, where BM2PC exceeds both ablated variants by about 20–30 percentage points in success rate.Source: 1) Ablation of the Multimodal Update · p. 11, Table III · p. 11
  • When a sudden obstacle blocks the preferred route, BM2PC switches to a mode on the unblocked side; the ablated variants fail to recover a feasible passage in the illustrated example.The study evaluates sudden blocking across lead times τ ∈ [0.65, 0.90] s.Source: 1) Ablation of the Multimodal Update · p. 11, Figure 7 · p. 12

2) Greedy vs. Exhaustive Set Optimization:

The incremental multimodal update selects near-optimal mode sets while substantially reducing planning time relative to exhaustive selection. The real-world evaluation also measures execution-model prediction against robot motion, where the model’s prediction error is much smaller than the commanded-versus-executed mismatch.

2) Greedy vs. Exhaustive Set Optimization:

  • 9.63 ms average planning time for BM2PC versus 90.14 ms for exhaustive selection, with rope-tip tracking RMSE of 6.22 cm versus 6.08 cm.These closed-loop results indicate comparable tracking accuracy alongside reduced planning time.Source: 2) Greedy vs. Exhaustive Set… · p. 12, Table IV · p. 12
  • The incremental update’s mean normalized optimality gap is 1.14%, its 95th-percentile gap is 6.80%, and it matches the exhaustive optimum in 56.07% of candidate pools.Source: 2) Greedy vs. Exhaustive Set… · p. 12

D. Real-world experiments

  • Hardware experiments use a Franka Panda arm with a gripper, updating desired end-effector acceleration at 66.67 Hz and executing position commands through a 1 kHz controller.Preliminary trials without the execution dynamics model showed tracking errors up to 5 cm during fast manipulation.Source: D. Real-world experiments · pp. 12–13

1) Execution Dynamics Model Evaluation:

  • The learned execution model’s average maximum position prediction error is 0.43 cm, compared with 4.57 cm average maximum deviation between commanded and executed trajectories.A test-time example shows the predicted end-effector trajectory closely matching measured motion despite an execution delay.Source: 1) Execution Dynamics Model Evaluation · p. 13

2) Trajectory Tracking:

BM2PC tracks real-world rope trajectories more accurately than SPiD and completes obstacle-avoidance trials without collision. The conclusions report stronger generalization to unseen tracking tasks and comparable obstacle-avoidance quality with less computation than multimodal MPC methods.

2) Trajectory Tracking:

  • BM2PC achieved higher tracking accuracy than SPiD across all tested trajectories and maintained performance through three lemniscate cycles.SPiD’s tracking error accumulated over the repeated lemniscate task.Source: 2) Trajectory Tracking · p. 14, Figure 10 · p. 14
  • Without the execution-dynamics model, tracking RMSE rose to 23.68 cm and the controller failed to reliably track the target.The ablation highlights execution-aware prediction under real-world execution mismatch.Source: 2) Trajectory Tracking · p. 14

3) Obstacle Avoidance:

  • Three obstacle-avoidance trials finished without collision, with 4.51 ± 1.11 cm mean tracking RMSE and 0.78 ± 0.61 s mean recovery time.Recovery time is measured from peak error until tracking error remains below 5 cm for at least 0.3 s.Source: 3) Obstacle Avoidance · p. 14, Figure 11 · p. 15
  • Perception failures accounted for approximately 4.96% of measurements during obstacle avoidance, where occlusions produced frozen or abruptly jumping rope-state estimates.Despite these degradations, the controller completed the task.Source: 3) Obstacle Avoidance · p. 14, Figure 11 · p. 15

VI. CONCLUSIONS

  • BM2PC outperforms the learning-based baseline on unseen tracking tasks and achieves comparable obstacle-avoidance quality with less computation than state-of-the-art multimodal MPC methods.Disturbance-rejection experiments also demonstrate robustness under large external perturbations.Source: §VI · p. 15
  • The execution-dynamics model compensates for command-execution mismatch by giving the planner a more accurate prediction of executed robot motion.The conclusions identify this model as enabling successful real-world deployment.Source: §VI · p. 15
Loading 2610.31852v1…