PaperlayerView exact revision on arXiv

Source-linked AI summary

Kernelized Movement Primitives

Yanlong Huang, Leonel Rozo, João Silvério, Darwin G. Caldwell

arXiv:1708.08638v2cs.RO

TL;DR

Imitation learning still needs to handle unseen task constraints, extrapolation, and high-dimensional inputs. The paper proposes KMP, a kernelized non-parametric movement-primitive framework with local-coordinate extensions and time-scale modulation. Evaluations cover trajectory adaptation, superposition, extrapolation, and high-dimensional-input tasks.

  • Problem

    Imitation learning must transfer demonstrated motion patterns to unseen situations while coping with obstacles, external interventions, extrapolation, and high-dimensional inputs.

  • Method

    KMP uses kernelized non-parametric trajectory modeling, extends to local coordinate frames, and supports trajectory modulation, superposition, and time-scale modulation.

  • Results

    KMP is evaluated on trajectory adaptation, superposition, extrapolation, time-driven modulation, and high-dimensional-input robotic tasks.

  • Takeaways & Limitations

    KMP provides one framework for adapting and extrapolating probabilistic movement patterns across unseen temporal, spatial, and task-constraint conditions.

  • Takeaways & Limitations

    The time-driven KMP derivation does not easily generalize to high-dimensional inputs because estimating derivatives of the kernel features is non-trivial.

Abstract

from arXiv · show

Imitation learning has been studied widely as a convenient way to transfer human skills to robots. This learning approach is aimed at extracting relevant motion patterns from human demonstrations and subsequently applying these patterns to different situations. Despite many advancements have been achieved, the solutions for coping with unpredicted situations (e.g., obstacles and external perturbations) and high-dimensional inputs are still largely open. In this paper, we propose a novel kernelized movement primitive (KMP), which allows the robot to adapt the learned motor skills and fulfill a variety of additional constraints arising over the course of a task. Specifically, KMP is capable of learning trajectories associated with high-dimensional inputs due to the kernel treatment, which in turn renders a model with fewer open parameters in contrast to methods that rely on basis functions. Moreover, we extend our approach by exploiting local trajectory representations in different coordinate systems that describe the task at hand, endowing KMP with reliable extrapolation capabilities in broader domains. We apply KMP to the learning of time-driven trajectories as a special case, where a compact parametric representation describing a trajectory and its first-order derivative is utilized. In order to verify the effectiveness of our method, several examples of trajectory modulations and extrapolations associated with time inputs, as well as trajectory adaptations with high-dimensional inputs are provided.

1 Introduction

The paper identifies limitations in existing imitation-learning methods for adapting trajectories to unseen constraints, extrapolating beyond demonstrations, and handling high-dimensional inputs. It proposes KMP as a unified kernelized framework addressing these requirements while reducing reliance on manually specified basis functions.

  • Robots must adapt demonstrated trajectories when unseen obstacles, human intervention, or other task constraints arise.
  • DMP and ProMP rely on explicitly defined basis functions, while DMP also introduces additional open parameters through its trajectory dynamics.
  • Existing approaches also face challenges with trajectory adaptation, online refinement, or via-point constraints in changing tasks.
  • KMP is proposed as a single framework that preserves probabilistic properties, supports trajectory adaptation and superposition, enables extrapolation, and learns from high-dimensional inputs.
  • KMP uses a non-parametric kernel treatment to avoid explicit basis-function representations, yielding fewer open parameters and supporting high-dimensional demonstrations.
  • Local-KMP extends KMP with local coordinate frames for extrapolation across task situations, while time-driven KMP supports time-scale modulation and trajectory adaptation through via-points and end-points.

2 Kernelized Representation of Movement Trajectory

KMP represents demonstrations probabilistically, then matches a parametric trajectory distribution to that reference using KL divergence and kernelized prediction. Its covariance-aware formulation preserves demonstration variability and output correlations while avoiding explicit basis-function definitions, including for high-dimensional inputs.

  • Probabilistic demonstration encoding: Demonstrations are encoded as input-output data, where inputs and outputs may represent positions, velocities, time, or other variable types.The framework uses a probabilistic encoding of demonstrations and can model different input-output interpretations.
  • Probabilistic demonstration encoding: A probabilistic reference trajectory is obtained from conditional distributions that capture demonstration variability and correlations among outputs.Each input-associated trajectory point has a conditional mean and covariance.
  • KMP derivation: KMP minimizes KL divergence between parametric and reference trajectory distributions, then separately solves mean and covariance optimization subproblems.The mean and covariance solutions define the KMP formulation.
  • KMP derivation: The mean objective uses covariance-weighted fitting with regularization, allowing larger deviations at high-variance points and tighter fitting at low-variance points.The penalty term λµᵀ_wµ_w is introduced to reduce overfitting.
  • Kernelized prediction: Kernelization avoids explicitly defining basis functions, facilitating prediction for high-dimensional inputs through kernel inner products and kernel matrices.The kernel treatment replaces explicit basis-function definitions in the mean prediction.
  • Kernelized prediction: Unlike GPR and CrKR, KMP uses demonstration variability in its covariance prediction and produces a full covariance matrix that models correlations between output components.GPR and CrKR use identity or diagonal weighting structures, respectively.

3 Extensions of Kernelized Movement Primitive

KMP is extended to adapt trajectories to new constraints, combine prioritized feasible motions, and extrapolate learned skills across task situations using local coordinate systems.

  • 3.3 Local Movement Learning Using KMP: Local-KMP encodes demonstrations in task-dependent coordinate frames and transforms predictions back to the base frame for broader task-space extrapolation.Each local frame is defined by a rotation matrix and translation vector; demonstrations and queries are projected into these frames before prediction.
  • 3.1 Trajectory Modulation Using KMP: KMP adapts learned trajectories to new via-points and end-points by incorporating desired conditional distributions into the reference database.Desired-point covariances encode tracking preference: smaller values impose higher precision, while larger values allow greater error.
  • 3.1 Trajectory Modulation Using KMP: When desired points conflict with the reference trajectory, KMP may produce a trade-off between their means rather than exactly satisfying both.The paper addresses this by prioritizing desired points and updating nearby reference datapoints while retaining most original datapoints.
  • 3.2 Trajectory Superposition: KMP superposes multiple feasible reference trajectories by weighting their information according to assigned priorities.The mixed trajectory is obtained through weighted mean and covariance minimization, with priorities represented through scaled Gaussian distributions.
  • 3.3 Local Movement Learning Using KMP: Local-KMP updates local reference databases with transformed via-points or end-points before predicting trajectory points for new task requirements.The procedure projects constraints and queries into each local frame, updates the corresponding KMP quantities, and combines the local predictions in the base frame.

4 Time-driven Kernelized Movement Primitives

Time-driven KMP treats time as the input and models positions with first-order derivatives, providing a compact trajectory representation and allowing time-scale modulation.

  • 4.1 A Special Treatment of Time-Driven KMP: Time-driven KMP uses time t as the input to learn trajectories for robotic movements in which timing is critical.The formulation targets time-driven trajectories as a special case of KMP.
  • 4.1 A Special Treatment of Time-Driven KMP: The parametric trajectory includes positions and velocities, so the first-order derivative encodes observed motion dynamics.The representation uses basis functions together with their derivatives rather than treating position and velocity through separate duplicated basis blocks.
  • 4.1 A Special Treatment of Time-Driven KMP: GMM models the joint distribution P(t, ξ, ˙ξ), and GMR retrieves the conditional probabilistic reference trajectory P(ˆξn, ˆ˙ξn|tn).The resulting reference distribution is then used to derive time-driven KMP.
  • 4.1 A Special Treatment of Time-Driven KMP: Using basis functions and their derivatives yields a lower-dimensional parameter vector w ∈ R^BO than using an extended duplicated basis matrix.The alternative representation requires w ∈ R^2BO, whereas the derivative-inclusive representation uses w ∈ R^BO.
  • 4.1 A Special Treatment of Time-Driven KMP: The time-driven derivation does not generalize easily to high-dimensional inputs because estimating derivatives of the feature map with respect to such inputs is non-trivial.An additional model capturing dynamics between time and the input would be needed to estimate ˙ϕ(s).
  • 4.2 Time Modulation: For new movement durations, a monotonic time transformation maps the adapted interval [0, tD] to the demonstration interval [0, tN].This provides a direct mechanism for speeding up or slowing down the learned movement.

5 Evaluations of the Approach

The evaluations test KMP on trajectory modulation, superposition, extrapolation, force-based adaptation, and human–robot collaboration. Across these settings, KMP fulfills new trajectory requirements, extrapolates through local frames, and adapts motions using sensed forces and high-dimensional hand inputs.

  • 5.1 Trajectory Modulation/Superposition: KMP and ProMP both generate trajectories satisfying new via-point and endpoint requirements, while KMP also supports trajectory superposition.The mixed trajectory switches from the first to the second reference according to monotonically decreasing and increasing priorities.
  • 5.1 Trajectory Modulation/Superposition: KMP avoids explicit basis functions, making high-dimensional-input tasks feasible where ProMP requires many basis functions.The 3rd-hand task uses a 6-D input associated with user hand trajectories.
  • 5.2 Extrapolation with Local-KMPs: Local-KMPs extrapolate to new starting and ending frames while passing through additional desired points and maintaining demonstrated trajectory shape.The transportation task uses five demonstrations and local frames at the initial and final locations.
  • 5.3 Force-based Trajectory Adaptation: KMP adapts the robot end-effector trajectory according to measured corrective forces when obstacles intersect the learned motion path.The reaching evaluation introduces two obstacles during execution although obstacles are absent during training.
  • 5.3 Force-based Trajectory Adaptation: Collision forces can autonomously guide KMP away from obstacles through low-impact collisions, with tolerated force regulated by a threshold.The collision force substitutes for human correction in replanning.
  • 5.4 Human-Robot Collaboration: A single KMP performs the soldering task without trajectory segmentation and adapts robot motion to user hand trajectories with high-dimensional inputs.Because prediction does not depend on time, slower or faster hand movements are supported.

6 Related Work

Related work contrasts KMP with DMP, ProMP, GMM/GMR, and reinforcement-learning approaches. The comparison centers on trajectory constraints, basis-function requirements, high-dimensional inputs, and computational demands.

  • DMP: DMP generalizes toward arbitrary starts and goals but converges with zero velocity and lacks a straightforward way to incorporate desired via-points.These properties limit DMP for tasks requiring velocity conditions or via-point constraints.
  • ProMP: ProMP supports simultaneous via-point and endpoint adaptation but, like DMP, relies on explicitly defined basis functions for time-driven trajectories.This basis-function formulation becomes problematic as input dimensionality increases.
  • GMM/GMR: GMM/GMR can encode high-dimensional inputs, but re-optimizing its many variables for new constraints is expensive and difficult.The cited variables include mixture coefficients, means, and covariance matrices.
  • KMP: KMP adapts toward multiple via-points and endpoints without high-dimensional hyperparameter optimization and avoids explicit basis functions through kernels.These properties support implementation with high-dimensional inputs and outputs.
  • Comparison: Unlike approaches trained from one demonstration, ProMP, GMM, and KMP require trajectory sets, enabling probabilistic modeling of human skills.DMP training requires only a single demonstration.

7 Discussion

The discussion characterizes KMP as an information-theoretic, kernel-based alternative to basis-function methods and identifies computational and design limitations. It also outlines extensions involving sparse training, automated via-point selection, control, and alternative kernels.

  • KMP and ProMP: KMP and ProMP both model probabilistic demonstration properties, but KMP matches a lower-dimensional reference trajectory distribution rather than estimating ProMP’s movement-pattern distribution directly.This distinction is central to their respective imitation-learning formulations.
  • KMP Formulation: KMP derives its optimal trajectory distribution by minimizing KL-divergence between the parametric trajectory and reference trajectory distributions.The derivation produces solutions for both the mean and covariance before applying the kernel trick.
  • KMP and ProMP: ProMP can require many demonstrations because estimating a high-dimensional movement-pattern covariance may produce a singular covariance matrix.High-dimensional inputs can also make its basis-function count grow exponentially.
  • Limitations and Extensions: KMP’s computational complexity increases with training-data size, motivating sparse models based on partial training data.The reference database is the relevant training-data size in this setting.
  • Limitations and Extensions: The desired points for KMP adaptation are chosen empirically, so more complicated tasks may benefit from searching for optimal via-points with reinforcement learning.The proposed search can encode task requirements through cost functions.
  • Limitations and Extensions: KMP’s simultaneous mean and covariance prediction could connect probabilistic learning with optimal control, while alternative kernels remain a future direction.The mean and covariance could serve as an LQR desired trajectory and tracking-error weight, respectively.

8 Conclusions

KMP combines kernel-based information-loss minimization with probabilistic movement primitives to adapt demonstrated trajectories to unseen temporal and spatial requirements. Its local-frame extension supports extrapolation, and evaluations covered time-driven movements and human-robot collaboration.

  • KMP incorporates kernel-based treatment into information-loss minimization for robot movement primitives.
  • The method preserves probabilistic properties of demonstrations while adapting trajectories to unseen temporal or spatial requirements and mixing different trajectories.
  • KMP’s local-frame extension provides reliable extrapolation capabilities across different coordinate systems.
  • As a kernel-based non-parametric approach, KMP models complex and high-dimensional trajectories with fewer limitations than state-of-the-art methods.
  • Simulations and real robotic systems evaluated KMP on time-driven movements and human-robot collaboration scenarios.

A Gaussian Mixture Regression (GMR)

The GMR formulation models the joint input-output distribution with Gaussian components and computes an output conditional distribution for each query input. The mean and covariance are obtained from component-wise Gaussian conditioning and mixture weights.

  • The joint probability distribution over inputs and outputs is represented as a mixture of Gaussian components.
  • The superscripts distinguish input variables from output variables in the component means and covariances.
  • For a query input, GMR computes the mean of the corresponding output using Gaussian component weights and conditional component means.
  • The corresponding conditional covariance is then computed, yielding a Gaussian conditional distribution for the output given the input.
  • The mean minimization problem is interpreted as maximizing the posterior defined by the Gaussian formulation after logarithmic transformation.

C Proof of Weighted Mean Minimization Subproblem

The appendix shows that the weighted mean minimization problem is equivalent to the corresponding formulation obtained from the defined quantities and derivative calculations.

  • The derivative of the weighted mean objective with respect to the mean variable is computed.
  • The definitions introduced earlier are used to transform the derivative expression into the target formulation.
  • The weighted mean minimization problem is proved equivalent to the problem defined in the corresponding equation.

D Proof of Weighted Variance Minimization Subproblem

The appendix concludes that the weighted variance minimization subproblem has the same solution as its equivalent reformulation.

  • Equation (58) establishes the equivalence between the weighted variance minimization problem and the problem defined in equation (40).
  • The proof concludes that both variance minimization formulations have the same solution.
  • The result completes the appendix’s equivalence argument for the weighted variance minimization subproblem.
Loading 1708.08638v2…