Source-linked AI summary
PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physics
Zhiao Huang, Yuanming Hu, Tao Du, Siyuan Zhou, Hao Su, Joshua B. Tenenbaum, Chuang Gan
TL;DR
Existing skill-learning environments largely model rigid bodies and usually lack useful simulation gradients, despite soft bodies’ broad applications and greater complexity. PlasticineLab introduces a differentiable elastoplastic soft-body benchmark and evaluates RL and gradient-based methods, finding that RL struggles on many tasks while gradient optimization is fast for simple tasks but limited on multistage planning.
Problem
Existing skill-learning environments typically simulate only rigid-body dynamics and provide few gradients for planning, while soft-body dynamics is more complex because of its high-dimensional behavior.
Method
PlasticineLab benchmarks 10 soft-body manipulation tasks across 50 configurations using differentiable physics for elastic and plastic deformation, enabling gradient-based optimization alongside RL evaluation.
Results
RL algorithms struggle on harder tasks, whereas gradient-based optimization can rapidly solve simple planning tasks but fails on local minima and multistage tasks requiring exploration or long-term planning.
Takeaways & Limitations
PlasticineLab provides a setting for systematically studying soft-body manipulation and motivates combining differentiable physics with RL or other sampling-based methods.
Takeaways & Limitations
Gradient-based methods are limited when tasks require detachment and reattachment or multistage manipulation, because local gradients may vanish or lead to local minima.
Abstract
from arXiv · showhide
Simulated virtual environments serve as one of the main driving forces behind developing and evaluating skill learning algorithms. However, existing environments typically only simulate rigid body physics. Additionally, the simulation process usually does not provide gradients that might be useful for planning and control optimizations. We introduce a new differentiable physics benchmark called PasticineLab, which includes a diverse collection of soft body manipulation tasks. In each task, the agent uses manipulators to deform the plasticine into the desired configuration. The underlying physics engine supports differentiable elastic and plastic deformation using the DiffTaichi system, posing many under-explored challenges to robotic agents. We evaluate several existing reinforcement learning (RL) methods and gradient-based methods on this benchmark. Experimental results suggest that 1) RL-based approaches struggle to solve most of the tasks efficiently; 2) gradient-based approaches, by optimizing open-loop control sequences with the built-in differentiable physics engine, can rapidly find a solution within tens of iterations, but still fall short on multi-stage tasks that require long-term planning. We expect that PlasticineLab will encourage the development of novel algorithms that combine differentiable physics and RL for more complex physics-based skill learning tasks.
1 INTRODUCTION
Existing skill-learning environments largely model rigid-body dynamics, while soft-body manipulation remains difficult because of high-dimensional dynamics. PlasticineLab addresses this gap with a differentiable benchmark covering diverse elastoplastic manipulation tasks.
- Existing virtual environments typically simulate rigid-body dynamics, leaving standard soft-body manipulation benchmarks sparse despite broad applications.Soft bodies are relevant to virtual surgery, computer graphics, robotics, and material science.
- Soft-body dynamics has effectively infinite degrees of freedom and high-dimensional governing equations, complicating simulation, control, and analysis.
- PlasticineLab introduces 10 soft-body manipulation tasks with 50 configurations involving pinching, rolling, chopping, molding, and carving.Plasticine deforms elastically under small deformation and plastically under large deformation.
- The benchmark provides analytical gradients through differentiable physics, enabling supervised learning and gradient-based optimization for soft-body manipulation.Its engine supports elastic and plastic deformation, soft-rigid interaction, and a differentiable contact model.
- PlasticineLab uses Taichi, MLS-MPM, the von Mises yield criterion, and reverse-mode differentiation to simulate and differentiate elastoplastic materials.The differentiation system handles numerically challenging SVD gradients from the plastic material model.
2 RELATED WORK
Prior learning environments and soft-body manipulation studies leave a gap: widely used simulators mainly provide rigid-body physics, while existing soft-body work often targets individual robot instances. PlasticineLab fills this gap with a benchmark combining soft-body tasks and differentiable simulation.
- Many virtual environments support reinforcement learning and robotics, but most rely on rigid-body engines such as MuJoCo and PyBullet.
- Existing environments generally lack assets and tasks for soft-body manipulation, and their engines do not provide differentiable information.Some platforms support soft-body dynamics in theory but do not provide the corresponding manipulation benchmark.
- Differentiable physics research includes neural approximations and direct automatic differentiation of physics-based simulators.
- Trajectory optimization and differentiable physics formulate manipulation planning as an optimization problem over trajectories.
- Prior learning-based soft-body manipulation studies address specific instances involving cloth, fluids, or rope, whereas PlasticineLab targets a comprehensive benchmark.
3 THE PLASTICINELAB LEARNING ENVIRONMENT
PlasticineLab is a differentiable-physics environment where rigid manipulators deform one or more plasticine bodies into target shapes. It defines particle-based states, landmark observations, bounded velocity actions, mass-tensor goals, and a suite of 10 tasks with 50 configurations.
- Environment: PlasticineLab contains challenging tasks in which rigid-body manipulators perform pinching, rolling, chopping, molding, and carving on 3D plasticine.
- Environment: Each task aims to deform one or more soft bodies into a target shape through planned manipulator motion within a reinforcement-learning framework.
- Task representation: The state represents soft-body particles with positions, velocities, deformation gradients, and affine velocity fields, together with manipulator states.Manipulator end effectors use position and quaternion orientation representations, with one-way rigid-soft coupling.
- Task representation: Observations downsample particle positions and velocities into fixed landmarks, while actions update bounded linear and optionally angular manipulator velocities.
- Goal and reward: The reward compares target and current mass tensors using L1 distance and an SDF-based term, while regularizing manipulator distance and grasping.A task-specific bias makes the initial reward nonnegative.
- Evaluation suite: The evaluation suite contains 10 tasks with five variants each, including Rope, Writer, Chopsticks, and RollingPin.Variants perturb initial and target shapes and manipulator locations; some tasks require multi-stage planning.
4 DIFFERENTIABLE ELASTOPLASTICITY SIMULATION
The simulator combines MLS-MPM continuum mechanics with elastoplastic material modeling and differentiable contact handling. Its plasticity and contact components are designed to support stable gradient computation through deformation and collisions.
- Simulation: The simulator runs in Taichi on CUDA and discretizes continuum mechanics with the Moving Least Squares Material Point Method.It uses both Lagrangian particles and Eulerian background grids.
- Elastoplasticity: Plasticity uses a von Mises yield criterion, with particles deforming plastically when deviatoric stress exceeds a threshold.Return mapping projects deformation gradients because plastic material forgets its rest state.
- Elastoplasticity: Return mapping is implemented as a projection on deformation-gradient singular values, requiring SVD and differentiated SVD gradients for backpropagation.
- Contact: The contact model uses grid-based Coulomb-friction treatment with rigid bodies represented as time-varying signed distance fields.A softened contact version is used to improve reward smoothness and gradient quality.
5 EXPERIMENTS
The experiments evaluate RL and gradient-based methods on soft-body manipulation tasks using normalized incremental IoU and reward-based trajectory optimization. RL struggles on many high-dimensional and multistage tasks, while differentiable optimization can exploit model gradients but remains sensitive to local minima and initialization.
- 5.1 EVALUATION METRICS: The normalized incremental IoU score measures how much task-state overlap with the goal increases from the initial to final state.The score uses normalized soft IoU between 3D mass tensors and averages results across five configurations per task.
- 5.2 EVALUATIONS ON REINFORCEMENT LEARNING: RL agents often achieve partial progress but fail on exact shape matching, long-horizon manipulation, or tasks requiring more manipulators.Observed failures include releasing plasticine during exploration, failing to move a rope around a pillar, and difficulty scaling to TripleMove.
- 5.2 EVALUATIONS ON REINFORCEMENT LEARNING: PPO generally performs better than SAC and TD3, although its advantage is reported as a suspected effect of on-policy samples and detailed shape variation.In RollingPin, PPO produces a more accurate shape and a higher normalized incremental IoU score than SAC.
- 5.2 EVALUATIONS ON REINFORCEMENT LEARNING: Harder tasks such as Chopsticks and Writer seldom yield reasonable RL solutions within 104 episodes, while Assembly agents commonly remain in local minima.The reported difficulties involve 3D rotation, complex carving trajectories, and lifting a spherical plasticine to the destination.
- 5.3 EVALUATIONS ON DIFFERENTIABLE PHYSICS FOR TRAJECTORY OPTIMIZATION: Gradient-based optimization backpropagates through full simulated trajectories to update open-loop action sequences for maximizing cumulative reward.The method assumes complete environment knowledge and compares Adam, momentum gradient descent, and Adam with a hard contact model.
- 5.3 EVALUATIONS ON DIFFERENTIABLE PHYSICS FOR TRAJECTORY OPTIMIZATION: Adam with differentiable physics can improve several manipulation solutions, but gradient-based methods struggle with local minima, multistage policies, and initialization sensitivity.The hard contact variant performs worse than the soft version in most tasks; gradient-based methods fail on tasks such as Pinch and Writer.
6 POTENTIAL RESEARCH PROBLEMS TO STUDY USING PLASTICINELAB
PlasticineLab opens research directions in planning, controller design, sim-to-real transfer, and generalization for soft-body manipulation. The passages emphasize combining differentiable physics with sampling-based methods and improving controllers and transfer across configurations and reality.
- Planning: Gradient-based trajectory optimization solves simple planning tasks extremely fast, but gradients vanish when manipulators detach and reattach to the plasticine.Sampling-based methods such as random search and reinforcement learning may be useful for these cases because they support multi-step exploration and cumulative rewards.
- Controller design: Controller design remains an open problem, with reward-function improvements and 3D neural architectures identified as possible directions.Experiments indicate substantial room for improved controller design and optimization.
- Sim-to-real transfer: Sim-to-real transfer could combine simulator-planned high-level trajectories with low-level controllers for complex real-world tasks.The authors also suggest using differentiable simulation gradients for systematic identification.
- Generalization: Generalization can be studied by generating varied procedures and configurations with different objects and goal conditions.PlasticineLab is positioned as a platform for evaluating algorithms across these variations.
7 CONCLUSION AND FUTURE WORK
The paper presents PlasticineLab as a differentiable elastoplastic soft-body manipulation benchmark with broad task coverage for comparing RL and gradient-based methods. It identifies future extensions involving articulation systems and task or controller optimization, while acknowledging inevitable modeling errors.
- Conclusion: PlasticineLab is presented as the first skill-learning environment that simulates elastoplastic materials while remaining differentiable.Its rich task coverage supports systematic study of state-of-the-art RL and gradient-based algorithms.
- Conclusion: The benchmark’s task coverage provides clues for future methods that combine RL and gradient-based algorithms.The paper frames this combination as a direction enabled by systematic behavioral analysis.
- Future work: Future work includes extending the benchmark with additional articulation systems such as virtual shadow hands.The paper also notes that modeling errors are inevitable in virtual environments.
- Impact: The authors argue that PlasticineLab can lower the barrier to future research on soft-body manipulation skill learning.
A SIMULATOR IMPLEMENTATION DETAILS
The simulator implementation includes a von Mises plasticity return-mapping procedure and performance measurements for forward and differentiated simulation steps. The implementation specifically addresses SVD-related gradient issues in backpropagation.
- Plasticity implementation: The forward return-mapping implementation uses an SVD whose gradients require special treatment during backpropagation.This is identified as a numerically challenging part of the plastic material model.
- Performance: Table 2 reports average running time for one forward or forward-plus-backpropagation step for each scene on an NVIDIA GTX 1080 Ti GPU.
B MORE DETAILS ON THE EVALUATION SUITE
The evaluation suite spans relocation, shaping, deformation, assembly, and articulated-object tasks with varying action-space dimensions. SAC, PPO, and TD3 are evaluated using fixed reward coefficients across environments.
- Task suite: Move relocates plasticine with two spherical manipulators and uses a 6D position-control action space.
- Task suite: TripleMove relocates three plasticine boxes using three gripper pairs and an 18-dimensional action space.The passage identifies this task as challenging for both RL and gradient-based methods.
- Task suite: Torus requires locating where to push a torus mold onto randomly relocated plasticine to produce a target shape.
- Task suite: Pinch requires discovering a sphere motion that creates dents matching a target generated from random collision angles.
- Task suite: Assembly combines deforming a spherical plasticine piece into a target shape with moving it onto another plasticine block.
- Task suite: Table requires pushing one leg of a plasticine table toward a target position with a spherical manipulator.
- RL setup: SAC, PPO, and TD3 are used with fixed reward coefficients c1 = 10, c2 = 10, and c3 = 1 across environments.Their parameter settings are listed in separate tables for each algorithm.
D ABLATION STUDY ON YIELD STRESS
Yield stress affects SAC learning speed in the Move configuration: agents in higher-yield-stress environments learn faster, especially early in training.
- Yield-stress effects: SAC agents achieved higher rewards as yield stress increased, with the strongest difference appearing early in training.The study varied yield stress from 10 to 1000 across six environments in the Move configuration.
- Experimental setup: The Move configuration was selected because SAC can solve it well, enabling comparison across yield-stress settings.
E DETAILED RESULTS ON ALL CONFIGURATIONS
The evaluation reports normalized incremental IoU across configurations using repeated random seeds, while differentiable physics is assessed through reward and variance trends during optimization.
- Evaluation protocol: Each algorithm used 15 random seeds per scene: 3 random seeds across each scene’s 5 configurations.Results are reported as normalized incremental IoU scores and standard deviations for each configuration.
- Optimization efficiency: Differentiable-physics optimization is evaluated through rewards and variances across configurations as training episodes increase.Figure 6 clamps rewards above zero for clearer visualization.
- Evaluation protocol: Table 6 reports normalized incremental IoU for each method and configuration as mean ± standard deviation.
- Evaluation metric: Soft IoU compares two positive normalized mass tensors after scaling their values to [0, 1], approximately measuring 3D-shape agreement.The formulation extends standard Intersection over Union from binary mass tensors to soft-valued tensors.