PaperlayerView exact revision on arXiv

Source-linked AI summary

Motion Planning Diffusion: Learning and Planning of Robot Motions with Diffusion Models

Joao Carvalho, An T. Le, Mark Baierl, Dorothea Koert, Jan Peters

arXiv:2308.01557v2cs.ROcs.AIcs.LG

TL;DR

Robot motion planners need useful trajectory priors because optimization depends on initialization and classical methods struggle with complex, multimodal motion distributions. The paper learns diffusion priors from expert trajectories and uses reverse diffusion to sample task-conditioned posteriors while incorporating motion-planning costs. Across simulated environments, diffusion priors produce more collision-free and multimodal trajectories than CVAE priors, while posterior guidance improves results in unseen-obstacle settings.

  • Problem

    Motion-planning optimization benefits from trajectory priors, but existing approaches must either initialize optimization from prior samples or incorporate priors through separate formulations.

  • Method

    The paper learns a diffusion trajectory model from expert plans and concurrently samples from its prior while biasing trajectories toward low-cost, high-likelihood regions.

  • Results

    DiffusionPrior achieves a 98% success rate versus 46% for CVAEPrior in PointMass2D Dense and produces more multimodal trajectories; MPD also improves success rate and multimodality with unseen obstacles.

  • Takeaways & Limitations

    Diffusion models are strong priors for encoding high-dimensional, multimodal robot-motion trajectories and can improve collision-free planning when combined with cost-guided sampling.

Abstract

from arXiv · show

Learning priors on trajectory distributions can help accelerate robot motion planning optimization. Given previously successful plans, learning trajectory generative models as priors for a new planning problem is highly desirable. Prior works propose several ways on utilizing this prior to bootstrapping the motion planning problem. Either sampling the prior for initializations or using the prior distribution in a maximum-a-posterior formulation for trajectory optimization. In this work, we propose learning diffusion models as priors. We then can sample directly from the posterior trajectory distribution conditioned on task goals, by leveraging the inverse denoising process of diffusion models. Furthermore, diffusion has been recently shown to effectively encode data multimodality in high-dimensional settings, which is particularly well-suited for large trajectory dataset. To demonstrate our method efficacy, we compare our proposed method - Motion Planning Diffusion - against several baselines in simulated planar robot and 7-dof robot arm manipulator environments. To assess the generalization capabilities of our method, we test it in environments with previously unseen obstacles. Our experiments show that diffusion models are strong priors to encode high-dimensional trajectory distributions of robot motions.

I. INTRODUCTION

Motion planning seeks feasible, smooth, collision-free paths, but classical sampling and optimization methods face practical inefficiencies and initialization or local-minimum challenges. This work uses diffusion models to learn trajectory priors that support multimodal motion planning.

  • Motion planning finds feasible, smooth, collision-free paths between start and goal configurations for execution by a lower-level controller.
  • Sampling-based planners can be sample-inefficient and produce non-smooth trajectories despite their completeness property.
  • Optimization-based planners can incorporate smoothness costs but depend on good initialization and may become trapped in local minima.
  • Learning-based methods use experience from successful plans, demonstrations, or contextual sampling distributions to improve classical motion planning.
  • The paper learns diffusion trajectory priors from expert data and merges prior sampling with motion optimization to model multimodal, high-dimensional trajectories.

B. Diffusion models for robotics

Diffusion models are positioned as trajectory priors that integrate generative modeling with classical motion planning. The paper emphasizes their suitability for high-dimensional, multimodal robot trajectories and direct posterior sampling.

  • B. Diffusion models for robotics: The method incorporates diffusion models as priors combined with differentiable motion-planning cost likelihoods.
  • B. Diffusion models for robotics: Unlike related diffusion applications, the approach learns higher trajectory derivatives and directly samples optimal trajectories from the posterior through reverse diffusion.
  • B. Diffusion models for robotics: Motion Planning Diffusion samples from trajectory posteriors while incorporating cost likelihoods representing motion-planning objectives.
  • B. Diffusion models for robotics: Robot motion planning objectives include collision-free paths, smoothness, start and goal configurations, and other constraints encoded through costs.

B. Motion Planning as Inference

Planning-as-inference represents motion planning as posterior inference over trajectories conditioned on task objectives. This formulation makes informative trajectory priors part of the planning problem.

  • The planning-as-inference framework samples trajectories from a posterior distribution conditioned on the task objective.
  • The trajectory prior represents likely trajectories, while the task likelihood represents how well a trajectory achieves the robot’s goals.
  • Task likelihoods can be factorized into independent components with temperatures associated with individual planning objectives.
  • Maximum-a-Posteriori inference recovers trajectory optimization with a prior-dependent log-probability term, enabling learned diffusion priors in planning.

C. Diffusion Models as Trajectory Generative Models

The trajectory diffusion model gradually corrupts expert trajectories with Gaussian noise and learns a reverse denoising process that reconstructs trajectory samples. A temporal U-Net parameterizes this trajectory model.

  • The forward diffusion process transforms expert trajectories into Gaussian noise through Markovian noise-injection steps.
  • At diffusion step t, trajectories can be sampled directly from a Gaussian distribution determined by the cumulative noise schedule.
  • The inverse process denoises Gaussian noise through successive conditional distributions parameterized by a learned mean and fixed covariance.
  • Rather than learning the reverse-process mean directly, the model can learn the injected noise using a simplified training loss.
  • A temporal U-Net encodes the diffusion model over trajectories.

D. Optimal sampling with guidance

The method samples task-conditioned posterior trajectories by combining diffusion denoising with gradients from differentiable motion-planning costs. During inference, it iteratively denoises trajectories, applies cost guidance, and hard-sets start and goal states.

  • MPD directly samples p(τ|O) by iteratively sampling task-conditioned reverse-diffusion posteriors instead of separately sampling and optimizing trajectories.The final denoised trajectory is τ0, while the initial state is Gaussian noise.
  • The denoising prior is modeled as a Gaussian with mean µt and covariance Σt, enabling posterior guidance through a likelihood gradient.The posterior cannot be sampled in closed form, so the method uses a classifier-guidance-style approximation.
  • The task likelihood is linearized around µt because the reverse-diffusion covariance concentrates near zero as denoising approaches completion.The guidance gradient is g = ∇τt−1 log p(O|τt−1) evaluated at µt.
  • Algorithm 1 alternates prior-mean computation, weighted cost-gradient evaluation, stochastic denoising, and hard enforcement of the start and goal configurations.Inference outputs an optimized batch of trajectories τ0.
  • Cost guidance biases smoothly sampled trajectories toward collision-free regions, goal sets, and valid joint-limit configurations while decreasing the overall cost objective.The implementation typically drops covariance scaling to preserve task-likelihood influence near the end of denoising.

E. Motion planning costs

The planner combines Gaussian-process, endpoint, collision, self-collision, joint-limit, and end-effector costs to encode smoothness, feasibility, safety, and task requirements.

  • Start and goal costs enforce endpoint configurations using Dirac-delta factors, implemented by hard-setting the initial and final trajectory states.A quadratic alternative is available when the generative model does not support Dirac delta distributions.
  • Collision costs sum penalties over robot-body collision spheres using differentiable forward kinematics and signed distances to obstacle surfaces.The signed-distance penalty is active within an ϵ margin.
  • Self-collision costs average pairwise distances between collision spheres belonging to the same robot links.
  • Joint-limit and end-effector costs penalize violations and pose discrepancies, while additional costs such as manipulability can be introduced for task requirements.Joint-limit penalties include an ϵ margin, and end-effector distance is defined on SE(3).
  • The GP prior models trajectories as a zero-mean Gaussian process with time-correlated covariance and binary factors between consecutive states.Under the planning-as-inference formulation, this cost promotes dynamic feasibility and smoothness.

IV. EXPERIMENTS

The experiments test whether diffusion priors represent multimodal trajectories, generalize to unseen obstacles, improve guided sampling, and support optimization-based planning.

  • The evaluation asks whether diffusion models learn collision-free, high-dimensional, highly multimodal trajectory distributions.
  • The experiments compare diffusion trajectory priors with Conditional Variational AutoEncoders as generative models.
  • The study tests collision-free generation when obstacles are not present during training.
  • The evaluation examines whether merging prior sampling with cost-gradient biasing improves results over sampling first and then optimizing likelihood.
  • The experiments assess whether diffusion models provide effective priors for optimization-based motion-planning algorithms.

A. Experimental Setup

Experiments span planar, three-dimensional, and 7-dof manipulation environments, using expert trajectory data, unseen extra obstacles, and comparisons against sampling, optimization, and generative baselines.

  • Environments: The benchmark includes PointMass2D Dense, PointMass3D Maze Boxes, Panda Spheres, and Panda Shelf environments of increasing difficulty.The Panda tasks use a 7-dof Franka Emika Panda, while extra-obstacle variants add spheres and boxes absent from training.
  • Tasks: All tasks seek smooth, collision-free trajectories between random start and goal configurations; Panda Shelf additionally penalizes changing end-effector orientation.
  • Algorithms and Baselines: The compared methods are MPD, DiffusionPrior, RRTConnect, GPMP without an informative prior, CVAEPrior, and CVAEPosterior.MPD samples with cost guidance, whereas CVAEPosterior samples from its prior before optimizing trajectory cost likelihood.
  • Metrics: Results report mean and standard deviation over 100 random contexts with 100 trajectories per context, using time, success, intensity, path length, and waypoint variance metrics.Waypoint variance measures how spread or multimodal generated trajectories are.
  • Environments: Figure 2 visualizes the tested point-mass and Panda environments, marking initial states in green, goals in red, and RRTConnect paths in blue.
  • Dataset generation and training: Training uses expert collision-free trajectories generated offline, with trajectory inputs represented as H × d; Panda experiments use H = 64 and d = 14.The data is split into training and validation sets, and diffusion models use 25 denoising steps with exponential scheduling.

B. Results in Simulation

In simulation, diffusion-based priors produced more diverse and successful trajectories than CVAE-based alternatives, including with unseen obstacles. Their sampling time remained comparable to CVAE methods, while diffusion initializations also improved GPMP performance over a straight-line prior.

  • 98% versus 46% success rate in PointMass2D Dense shows DiffusionPrior generated more collision-free trajectories than CVAEPrior.Across training environments, DiffusionPrior also produced more multimodal trajectories, reflected by higher variance measures.
  • In environments with randomly added unseen obstacles, diffusion-guided optimization achieved better success rates and multimodality than CVAEPosterior.Prior-only success rates decreased in these environments, while DiffusionPrior in Panda Shelf retained 100% success with collision intensity increasing from 3.6% to 5.9%.
  • Comparable computation times in Panda environments indicate that diffusion sampling was not the dominant cost relative to cost-gradient computation.Most computation was spent evaluating cost gradients rather than sampling from the diffusion model.
  • Diffusion samples initialized GPMP with higher success rates and more multimodal trajectories than a constant-velocity straight-line mean prior across all environments.The advantage was more noticeable in complex collision-free-space manifolds such as PointMass2D Dense.
  • RRTConnect sampling included sequential GPU trajectory generation and generally produced higher-jerk, longer paths requiring more GPMP optimization steps than DiffusionPrior.The sequential sampling was attributed to the difficulty of parallelizing RRT-like algorithms for batches.

C. Results in the Real World Panda Shelf

The real-world Panda Shelf task evaluated MPD on bottle transport around previously unseen obstacles while maintaining constant end-effector orientation. Three initial configurations and one goal were tested, with 25 collision-free trajectories obtained after sampling and filtering.

  • MPD was tested on real-world bottle transport around unseen obstacles while maintaining constant end-effector orientation.The obstacles were approximated as box models, and three initial configurations were paired with one desired goal configuration.
  • 25 collision-free trajectories were obtained after sampling 10 trajectories for each start-goal pair and filtering collisions.

V. CONCLUSION AND FUTURE WORK

The paper learns diffusion priors over waypoint-parameterized trajectories and uses diffusion guidance to sample while biasing trajectories toward low-cost regions. Results support diffusion models as effective priors because they better represent multimodal trajectories and improve collision-free planning; future work will extend trajectory parametrizations.

  • The method learns a diffusion generative model over waypoint-parameterized trajectories using motion plans generated by an optimal planner.
  • At inference, diffusion guidance concurrently samples from the prior and biases trajectories toward low-cost, high-likelihood regions.
  • Diffusion models better encode multimodal trajectories than commonly used CVAEs, and sampling while optimizing likelihood improves collision-free trajectory finding.
  • Future work will extend diffusion models to encode different trajectory parametrizations for robotic movements.
Loading 2308.01557v2…