Source-linked AI summary
PyRoki: A Modular Toolkit for Robot Kinematic Optimization
Chung Min Kim, Brent Yi, Hongsuk Choi, Yi Ma, Ken Goldberg, Angjoo Kanazawa
TL;DR
Kinematic optimization tools are fragmented by task-specific implementations and limited computation platforms. PyRoki provides modular problem specification and optimization across CPU, GPU, and TPU; in batched IK, it is reported 1.4–1.7x faster than previous GPU-accelerated methods and converges to lower errors than cuRobo.
Problem
Existing kinematic optimization tools rely on task-specific implementations that hinder adding objectives and often restrict computation to CPU or GPU.
Source: §I · p. 1Method
PyRoki combines modular variable and cost abstractions with a Levenberg–Marquardt optimizer and supports CPU, GPU, and TPU execution.
Source: §I · pp. 1–2Results
For batched IK, PyRoki is 1.4–1.7x faster than previous GPU-accelerated methods and converges to lower errors than cuRobo.
Source: §I · p. 2, C. Runtime Analysis · p. 6Takeaways & Limitations
PyRoki applies composed variables and costs across inverse kinematics, trajectory optimization, and motion retargeting.
Source: §I · pp. 1–2
Abstract
from arXiv · showhide
Robot motion can have many goals. Depending on the task, we might optimize for pose error, speed, collision, or similarity to a human demonstration. Motivated by this, we present PyRoki: a modular, extensible, and cross-platform toolkit for solving kinematic optimization problems. PyRoki couples an interface for specifying kinematic variables and costs with an efficient nonlinear least squares optimizer. Unlike existing tools, it is also cross-platform: optimization runs natively on CPU, GPU, and TPU. In this paper, we present (i) the design and implementation of PyRoki, (ii) motion retargeting and planning case studies that highlight the advantages of PyRoki's modularity, and (iii) optimization benchmarking, where PyRoki can be 1.4-1.7x faster and converges to lower errors than cuRobo, an existing GPU-accelerated inverse kinematics library.
I. INTRODUCTION
Kinematic optimization tools support diverse robotics tasks, but task-specific implementations fragment the field and restrict computation to CPU or GPU. PyRoki offers a modular, cross-platform toolkit for these problems, with applications and benchmarking across tasks.
I. INTRODUCTION
- Task-specific routines, kernels, and Jacobians fragment kinematic optimization tools, hinder adding objectives, and restrict optimizers to CPU or GPU.Table I compares documented features and supported functionalities of selected tools.Source: §I · p. 1, Table I · p. 2
- PyRoki combines modular variable and cost abstractions, a Levenberg–Marquardt optimizer, and a web visualizer for tuning cost weights.The toolkit supports inverse kinematics, trajectory optimization, and motion retargeting, and runs on CPU, GPU, and TPU.Source: §I · pp. 1–2
- 1.4–1.7x faster batched IK than previous GPU-accelerated methods is PyRoki’s reported benchmark result.PyRoki uses a Levenberg–Marquardt solver and evaluates both automatic and analytic Jacobians.Source: §I · p. 2
- The paper demonstrates PyRoki across inverse kinematics, trajectory optimization, and motion retargeting for robot hands and humanoids.Its toolkit supports URDF and includes a real-time web visualizer; the code is open-source.Source: §I · p. 2
II. RELATED WORK
The related work covers optimization approaches for inverse kinematics, trajectory optimization, and motion retargeting, alongside broader modular optimization frameworks. These areas differ in their objectives, structure, and computational needs.
A. Inverse Kinematics
- Iterative optimization makes inverse kinematics more flexible than closed-form methods that require assumptions on kinematic configuration.Optimization can accommodate secondary objectives and geometric or learned collision constraints, and generalizes across robot embodiments.Source: A. Inverse Kinematics · p. 2
B. Trajectory Optimization
- Trajectory optimization outputs a continuous sequence of joint configurations and can exploit temporal-neighbor structure for sparse optimization.PyRoki supports sparse linear algebra routines for such problems.Source: B. Trajectory Optimization · p. 2
C. Motion Retargeting
- Motion retargeting transfers motion between embodiments and must balance similarity with physical plausibility despite kinematic differences.Teleoperation may require real-time single-CPU operation, while offline processing of large datasets can benefit from GPU batches.Source: C. Motion Retargeting · p. 2
D. Modular Optimization Tools
- Modular optimization frameworks span machine learning, mathematical optimization, robotics, computer vision, and control, with tools such as TensorFlow, JuMP, and Drake.The paper positions PyRoki as aimed at kinematic optimization across diverse robots, tasks, and compute platforms.Source: D. Modular Optimization Tools · pp. 2–3
III. PYRO K I: MODULAR KINEMATIC OPTIMIZATION
PyRoki is designed around reusable optimization components, support for custom objectives, and execution across CPU, GPU, and TPU. Its design separates variables from costs and supports flexible Jacobian computation and interactive cost-weight tuning.
1) Modular:
- Separating variables from cost functions lets the same objectives, such as collision avoidance or pose matching, apply across tasks without reimplementation.The paper identifies reusable components as central to its optimization design.Source: 1) Modular · p. 3, §III · p. 3
2) Extensible:
- Automatic differentiation computes Jacobians for user-defined costs, while analytical Jacobians remain available when performance is critical.A real-time interface supports tuning cost weights.Source: 2) Extensible · p. 3
3) Cross-Platform:
- PyRoki runs natively on CPUs, GPUs, and TPUs, scaling from single-robot optimization to parallel batch processing.Its design also includes a quasi-Newton optimizer, variable abstractions, and composable cost functions.Source: 3) Cross-Platform · p. 3
A. Solver Backbone
PyRoki combines an LM solver with reusable kinematic variables and composable costs, covering common motion objectives while supporting CPU, GPU, and TPU execution. Its cost library includes motion-quality, feasibility, and joint-bound terms.
A. Solver Backbone
- JAX-based Levenberg–Marquardt optimization computes block-sparse Jacobians and represents joint limits and contact avoidance with differentiable penalties rather than direct hard constraints.The optimizer is designed for CPU, GPU, and TPU execution.Source: A. Solver Backbone · p. 3
B. Variable Abstractions
- Joint configurations and SE(3)/SO(3) variables provide reusable representations for articulation, poses, and orientations while handling geometric operations without requiring users to implement manifold math.Configurations support fixed, revolute, prismatic, and mimic joints.Source: B. Variable Abstractions · p. 3
C. Cost Functions
- Composable costs include smoothness, manipulability, self/world collision, and joint-pose objectives, with collision residuals based on signed distances and applicable across timesteps.Smoothness can also target acceleration and jerk; the manipulability cost penalizes near-singular configurations.Source: C. Cost Functions · pp. 3–4
- Pre-implemented terms also penalize joint-limit and velocity-limit violations and regularize solutions toward a user-defined default pose.The LM optimizer minimizes the sum of squared, weighted residuals.Source: C. Cost Functions · pp. 3–4
D. Interactive Robot Web Viewer
PyRoki’s web viewer supports real-time cost-weight tuning and interactive scene exploration. The toolkit also formulates collision-free trajectory optimization from straight-line initializations using swept collision geometry.
D. Interactive Robot Web Viewer
- The web viewer lets users tune cost weights in real time and immediately observe their effects on robot motion.This supports balancing end-effector accuracy, joint limits, and collision avoidance.Source: D. Interactive Robot Web Viewer · p. 4
- Users can modify scenes, including moving obstacles, and add visualizations such as manipulability ellipses.The viewer can also display robot configuration and set a goal.Source: D. Interactive Robot Web Viewer · p. 4
- Trajectory optimization can produce collision-free solutions from straight-line initializations, using spheres connected as capsules to check collisions between neighboring timesteps.The illustrated approach is described as similar to CHOMP [14].Source: Figure 3 · p. 4
IV. QUALITATIVE RESULTS
The qualitative case studies demonstrate PyRoki’s modularity and extensibility across inverse kinematics, trajectory optimization, and motion retargeting for humanoids and robot hands.
IV. QUALITATIVE RESULTS
- The case studies use predefined and custom costs to demonstrate the framework across four robot kinematic tasks.The section begins with inverse kinematics and trajectory optimization, then examines motion retargeting.Source: §IV · p. 4
- The inverse-kinematics and trajectory-optimization case study draws on existing optimization-based motion-planning approaches.Source: §IV · p. 4
- The retargeting case studies transfer human motion to humanoids and robot hands.Source: §IV · p. 4
A. Inverse Kinematics and Trajectory Optimization
PyRoki formulates inverse kinematics from a robot URDF and reusable cost terms, then reuses that logic for trajectory optimization with collision and motion-smoothness objectives.
A. Inverse Kinematics and Trajectory Optimization
- IK uses an imported robot URDF and assembled cost terms, and can be extended to seek solutions near the current state or avoid collisions.The described terms include joint pose, joint limits, and joint regularization.Source: A. Inverse Kinematics and Trajectory… · p. 4
- Trajectory optimization reuses IK costs and adds collision avoidance plus acceleration and jerk minimization for a UR5 moving between poses around an obstacle.The initial trajectory comes from collision-free start and goal IK configurations connected by linear interpolation, which may initially collide with the environment.Source: A. Inverse Kinematics and Trajectory… · p. 4
- Continuous collision costs connect each UR5 sphere at one timestep to its counterpart at the next, forming capsules checked against world obstacles.The passage describes this cost as quick to implement in a few lines of Python.Source: A. Inverse Kinematics and Trajectory… · pp. 4–5
B. Motion Retargeting
PyRoki’s modularity and extensibility support retargeting across full-body and hand-object scenarios, where motion fidelity must be balanced with physical constraints.
B. Motion Retargeting
- Retargeting may require scaling or motion warping, while balancing motion fidelity against physical constraints.The demonstrated scenarios range from full-body humanoid motion to hand-object interactions.Source: B. Motion Retargeting · p. 5
- Domain-specific retargeting costs can be prototyped quickly using auto-differentiated Jacobians.The passage presents this flexibility as useful for challenging retargeting scenarios.Source: B. Motion Retargeting · p. 5
- Sparse keypoint costs align positions while preserving relative joint distances and angles, using one approach for body and hand retargeting.This formulation adapts across skeletal structures with different scales and degrees of freedom.Source: B. Motion Retargeting · p. 5
1) Full-body Humanoid Retargeting:
PyRoki applies a shared retargeting approach to humanoids and robot hands, while adding domain-specific contact constraints for hand-object motion.
1) Full-body Humanoid Retargeting:
- Humanoid retargeting matches scaled relative joint positions and relative angles, while enforcing floor contact, foot orientation, self-collision avoidance, and joint limits.The same cost handles the different body proportions of the G1 (127 cm) and H1 (178 cm).Source: 1) Full-body Humanoid Retargeting · p. 5
2) Hand-Object Interaction Retargeting:
- Hand-object retargeting reuses the humanoid motion-transfer cost and adds contact costs to maintain contact between the robot hand and object surfaces.The example transfers DexYCB human hand motions to a Shadow Hand.Source: 2) Hand-Object Interaction Retargeting · p. 5
V. QUANTITATIVE RESULTS
- PyRoki’s global IK implementation of Dex-Retargeting’s keypoint vector loss achieves slightly lower final costs than the original differential IK approach.The figure identifies the method as Dex-Retargeting [6].Source: Figure 5 · p. 6
- The evaluation also covers extending IK to optimize a Fetch base pose and comparing autodiff and analytical Jacobian runtimes against cuRobo on CPU and GPU.The experiments ran on an AMD 5955WX CPU and NVIDIA RTX 4090 GPU.Source: §V · pp. 5–6
A. Reproducing Dex-Retargeting
PyRoki reproduces Dex-Retargeting’s hand-motion cost and extends mobile-robot IK by optimizing the base pose together with the arm.
A. Reproducing Dex-Retargeting
- PyRoki’s reimplementation of Dex-Retargeting reaches a keypoint cost of 0.068 ± 0.070 over a 621-timestep human hand trajectory, with visually similar motions and semantic poses.The reproduced cost can also be extended with contact loss and GPU batch processing.Source: A. Reproducing Dex-Retargeting · p. 6
B. Inverse Kinematics with a Mobile Base
- For mobile IK, PyRoki adds the SE(2) base pose as an optimization variable and evaluates targets created by translating reachable end-effector poses from a fixed base.Success requires position error under 5mm and rotation error below 0.05 radians; a fixed-base choice can leave the target unreachable.Source: B. Inverse Kinematics with a Mobile Base · p. 6
- Optimizing the base pose yields a 100% success rate with mean position errors under 0.005mm, versus 15% success and position errors >60cm with a fixed base.The table describes the Fetch IK comparison and reports improved position and rotation accuracy when base pose is a variable.Source: Table II · p. 6, B. Inverse Kinematics with a Mobile Base · p. 6
C. Runtime Analysis
IK-Beam combines parallel multi-seed optimization with analytical or autodiff Jacobians and CPU/GPU execution. It runs faster than cuRobo while converging to lower errors, and PyRoki reuses collision costs across tasks.
C. Runtime Analysis
- IK-Beam starts from 64 seeds, optimizes them in parallel for six LM steps, keeps four lowest-error seeds for ten more steps, and returns their best solution.Source: C. Runtime Analysis · p. 6
- Analytical Jacobians provide 3x–19x speedups over autodiff, while GPU parallelization scales runtime better as batch size increases.Source: C. Runtime Analysis · p. 6
- Depending on batch size, IK-Beam is 1.4x–1.7x faster than cuRobo and converges to lower errors.The authors believe the accuracy improvement is mostly explained by PyRoki's LM optimizer versus cuRobo's L-BFGS; Table IV compares the methods.Source: C. Runtime Analysis · p. 6, Table IV · p. 7
VI. CONCLUSION
- PyRoki reuses the same collision costs across inverse kinematics, trajectory optimization, and humanoid motion retargeting, and is open-source for extension.Source: §VI · p. 7