Source-linked AI summary
An End-to-End Differentiable Framework for Contact-Aware Robot Design
Jie Xu, Tao Chen, Lara Zlokapa, Michael Foshey, Wojciech Matusik, Shinjiro Sueda, Pulkit Agrawal
TL;DR
Separately developing robot morphology and control, alongside limits in design representations and contact handling, constrains co-optimization for manipulation. The paper combines cage-based deformation with a differentiable rigid-body simulator to optimize both for contact-rich tasks. Across manipulation tasks, the framework outperforms several gradient-free and model-free reinforcement-learning approaches, and its optimized designs can be manufactured.
Problem
Co-optimizing robot morphology and control is challenging because existing approaches have limited design representations and must contend with manufacturing constraints and contact-rich tasks.
Source: §I · pp. 1–2Method
The framework combines cage-based deformation for articulated morphology with differentiable rigid-body simulation and analytical gradients for contact-rich manipulation.
Source: §I · p. 2Results
The framework outperforms several state-of-the-art gradient-free and model-free reinforcement-learning methods at jointly optimizing morphology and control for manipulation tasks.
Source: §I · p. 2Takeaways & Limitations
The optimized designs can be easily manufactured, and the framework finds better morphology-control combinations with fewer simulation episodes than state-of-the-art approaches.
Source: §I · p. 2, §V · p. 10The morphology parameterization assumes a fixed robot topology.
Source: §I · p. 2
Abstract
from arXiv · showhide
The current dominant paradigm for robotic manipulation involves two separate stages: manipulator design and control. Because the robot's morphology and how it can be controlled are intimately linked, joint optimization of design and control can significantly improve performance. Existing methods for co-optimization are limited and fail to explore a rich space of designs. The primary reason is the trade-off between the complexity of designs that is necessary for contact-rich tasks against the practical constraints of manufacturing, optimization, contact handling, etc. We overcome several of these challenges by building an end-to-end differentiable framework for contact-aware robot design. The two key components of this framework are: a novel deformation-based parameterization that allows for the design of articulated rigid robots with arbitrary, complex geometry, and a differentiable rigid body simulator that can handle contact-rich scenarios and computes analytical gradients for a full spectrum of kinematic and dynamic parameters. On multiple manipulation tasks, our framework outperforms existing methods that either only optimize for control or for design using alternate representations or co-optimize using gradient-free methods.
I. INTRODUCTION
Manipulator design and control are often developed separately, despite their complex interplay with manufacturing constraints. The paper introduces a differentiable framework that jointly optimizes morphology and control for contact-rich manipulation.
I. INTRODUCTION
- Roboticists often explore rich designs with simple control or develop complex control algorithms for existing robots, a split driven by long iteration cycles.The paper notes that this can leave systems suboptimal for a given task.Source: §I · p. 1
- Design representations must support manufacturable articulated robots with complex geometry while remaining suitable for optimization.Simple primitives can be easy to optimize but may oversimplify shapes needed for complex grippers; other representations face connectivity or differentiability challenges.Source: §I · pp. 1–2
- Cage-based deformation represents articulated robot morphology with a small number of parameters, supports manufacturing and connectivity constraints, and is flexible and differentiable.It can be used with meshes or point clouds and allows the number of shape degrees of freedom to be controlled through the number of cage handles.Source: §I · p. 2
- The differentiable articulated rigid-body simulator extends frictional contact to multiple dynamic bodies, removes discontinuities in contact damping, and derives analytical gradients across simulation parameters.The gradients include contact-point positions.Source: §I · p. 2
- The framework outperforms state-of-the-art gradient-free approaches and model-free reinforcement learning methods when jointly optimizing robot morphology and control.The deformation-based parameterization provides an expressive design space, and optimized designs can be easily manufactured.Source: §I · p. 2
II. RELATED WORK
The paper situates its work at the intersection of differentiable contact simulation and articulated robot co-design, then introduces a cage-based, hierarchical morphology parameterization. Its constraints are designed to preserve manufacturability and connectivity while retaining a compact, expressive design space.
A. Differentiable Physics-Based Simulation
- The simulator extends differentiable frictional contact to multiple dynamic bodies and provides analytical gradients for kinematic and dynamic parameters, beyond control variables and limited material parameters.Earlier approaches include contact between a dynamic body and a stationary surface, while other differentiable rigid-body simulators provide gradients only for controls or a small set of material parameters.Source: A. Differentiable Physics-Based… · p. 2
B. Computational Robot Co-Design and Morphology Parameterization
- Existing co-design methods often use primitive link shapes, while richer representations can struggle with articulated connectivity, manufacturability, or differentiability.The paper positions its deformation-based representation as a constrained but expressive alternative for articulated designs.Source: B. Computational Robot Co-Design and… · pp. 2–3
III. METHOD
- Cage-based deformation uses a small set of handles and precomputed weights to deform complex meshes while preserving geometric features.The framework maps high-level morphology parameters through cage handles and deformed meshes to simulation inputs, with differentiable steps that support gradient-based optimization.Source: A. Hierarchical Morphology… · pp. 3–4, §III · p. 3
A. Hierarchical Morphology Parameterization
- Component-specific cage constraints accommodate fabrication requirements, while merged connection handles preserve connectivity between articulated components.The high-level parameters govern cage scales and auxiliary points; the allowed deformation can vary by component, such as scaling a joint cage only along its axis.Source: A. Hierarchical Morphology… · pp. 4–5
B. Differentiable Articulated Rigid Body Simulation
The differentiable simulator converts deformed morphology into kinematic and dynamic simulation parameters and computes gradients across that parameter space. It supports frictional contact between moving bodies and uses continuous contact damping within its articulated dynamics simulation.
B. Differentiable Articulated Rigid Body Simulation
- Analytical gradients cover the simulator’s full parameter spectrum, including kinematic and dynamic parameters, rather than only controls or a small set of material parameters.The simulation parameters include joint and body transformations, generalized inertia, contact point positions, and contact-point surface area.Source: B. Differentiable Articulated Rigid… · p. 5
- Reduced-coordinate dynamics use implicit time integration, with adjoint sensitivity analysis computing derivatives through forward and backward simulation passes.The method uses BDF2 with SDIRK2 for the initial step and solves the resulting nonlinear equations with Newton’s method and line search.Source: B. Differentiable Articulated Rigid… · p. 6
- The contact model supports friction between two dynamic bodies and differentiates contact forces with respect to robot coordinates and contact-point positions.It uses a signed distance field attached to the manipulated rigid object; force derivatives with respect to contact points and area contribute to backward-pass sensitivities.Source: B. Differentiable Articulated Rigid… · p. 6
- Contact damping is proportional to both penetration distance and penetration speed, keeping the force continuous at collision.The authors report that the earlier formulation’s discontinuity can cause convergence problems when two moving objects collide.Source: B. Differentiable Articulated Rigid… · p. 6
C. End-to-End Differentiable Co-Design Framework
The co-design framework composes differentiable morphology mappings with an articulated rigid-body simulator. Given high-level morphology parameters and controls, it computes task loss and analytical derivatives that support joint optimization.
C. End-to-End Differentiable Co-Design Framework
- The framework composes H, M, and P to map high-level morphology parameters ψc into simulation parameters ψp through analytically differentiable stages.H maps high-level parameters to cage handles, M deforms the mesh, and P derives simulation parameters from the mesh.Source: C. End-to-End Differentiable Co-Design… · p. 6
- Figure 5 illustrates randomly sampled morphology variants for single-finger and two-finger gripper configurations.The initial designs appear on the left and the deformed samples on the right.Source: Figure 5 · p. 7
- The simulator takes simulation parameters ψp and controls u as input, computes task loss L, and provides analytical derivatives with respect to morphology and controls.These derivatives are propagated through the morphology mappings using the chain rule.Source: C. End-to-End Differentiable Co-Design… · pp. 6–7
A. Implementation
The implementation connects a C++ simulator to a Python morphology parameterization and uses L-BFGS-B for co-optimization with analytical derivatives.
A. Implementation
- The simulator is implemented in C++ and the morphology parameterization in Python, with Python bindings; L-BFGS-B optimizes joint torques and morphology parameters using analytical derivatives.The control sequence supplies joint torques at each simulation time step.Source: A. Implementation · p. 7
B. Quality of Morphology Design Space
The parameterization produces varied, fabricable articulated morphologies with few design variables. In co-optimization experiments, the framework was more sample-efficient than ES and RL baselines and alone solved Assemble.
B. Quality of Morphology Design Space
- With only 9 and 17 parameters for single- and two-finger configurations, the method generated varied, fabricable designs while maintaining articulated connectivity.The authors describe the resulting design space as compact yet expressive.Source: B. Quality of Morphology Design Space · p. 7
C. Morphology and Control Co-Optimization
- The four tasks test sequential target reaching, energy-efficient box flipping, cube-layer rotation, and precise two-finger cube insertion.Baselines include evolutionary strategies, an RL-based co-optimization method, and control-only optimization with fixed morphology.Source: C. Morphology and Control… · p. 7
- Methods were run five times with different seeds; task metrics measured target distance, final rotation error, or box-to-hole distance, depending on the task.Figure 6 plots mean and standard deviation of objective loss against simulation episodes.Source: Table IV · p. 8, C. Morphology and Control… · p. 8, Figure 6 · p. 8
- The framework used 10–30 times fewer simulated episodes than ES and RL baselines, tracked Finger Reach targets most accurately, and alone solved Assemble.The authors identify Assemble as their most challenging, contact-rich task.Source: C. Morphology and Control… · p. 8
- Optimizing morphology alongside control outperformed the control-only baseline, and task-specific designs included a hooklike tip for flipping and differently sized fingers for assembly.For assembly, flat, larger fingers helped push the object more stably than thin fingers.Source: C. Morphology and Control… · p. 8
D. Flexibility of the Morphology Parameterization
The deformation-cage representation supports higher-dimensional gripper design and produces smoother shapes than direct mesh optimization. In a grasping comparison, it used far fewer variables while achieving a similar success rate and better average loss.
D. Flexibility of the Morphology Parameterization
- Adding cage handles raises morphology degrees of freedom; the free-form task deforms cube-like fingers to grasp a diamond-like object using a predefined control sequence.Both parameterizations optimize only the inner-side handles or mesh vertices of each finger.Source: D. Flexibility of the Morphology… · pp. 8–9
- The comparison used 30 independent runs for each method, with the deformation-cage and mesh parameterizations optimized in the same differentiable framework.The mesh baseline directly optimizes vertex positions, while the cage method optimizes deformation handles.Source: D. Flexibility of the Morphology… · p. 9
- With 396 variables versus mesh-based optimization’s 8,946, cage deformation achieved a 97% success rate versus mesh’s 100%, better average loss, and smoother fingers.The mesh-based result included reverted triangles; the cage-based method’s smoothness is attributed to feature-preserving deformation.Source: D. Flexibility of the Morphology… · p. 9, Figure 8 · p. 9
E. Manufacturing of Optimized Designs
The optimized finger designs were 3D printed with minimal preparation, and a manufactured Flip Box finger successfully flipped real cubes. The discussion notes that the method optimizes continuous morphology for a fixed topology and identifies simulation-to-reality transfer as future work.
E. Manufacturing of Optimized Designs
- Two optimized fingers were 3D printed in Onyx and assembled; minimal modification was needed to prepare the generated models for printing.The authors present this as evidence that the deformation-based parameterization maintains manufacturability.Source: E. Manufacturing of Optimized Designs · p. 9
- The manufactured Flip Box finger flipped real cubes, including cubes of varied sizes, using manually programmed controls that followed waypoints from the optimized trajectory.The tendon-driven finger was mounted on a UR5 arm and actuated by Dynamixel servos.Source: E. Manufacturing of Optimized Designs · p. 9
V. DISCUSSION
- The component database can generate different topologies, and cages and precomputed deformation weights can be reused across manipulator structures.Components with differently sized connection surfaces are also supported when proper cages are constructed.Source: §V · p. 10
- The optimization varies continuous morphology only for a fixed robot topology, so it does not search the discrete topology space.The discussion also identifies local minima and the simulation-to-reality gap as future challenges.Source: §V · p. 10
Appendix
The appendix describes a graph grammar that generates manipulator designs from components and computes their articulated cages during rule application.
I. MANIPULATOR GRAMMAR
- A context-sensitive stochastic graph grammar represents manipulator topology as a graph and generates designs while computing articulated cage parameterizations.The grammar starts from S and repeatedly replaces nonterminal nodes until only terminal component nodes remain.Source: §I · p. 13
- Each grammar rule replaces a nonterminal graph node with a subgraph; an empty-node rule can remove F from the graph.The grammar automatically computes articulated cages during rule application.Source: §I · p. 13
- A database of five basic components yields varied component combinations, and the paper’s two manipulator configurations are selected from this grammar space.Source: §I · p. 13
A. Tasks
The section defines five manipulation tests spanning reaching, contact-rich object manipulation, collaborative insertion, and shape-optimized grasping.
A. Tasks
- Finger Reach requires lengthening a finger to visit four scattered targets sequentially, two of which the initial design cannot reach.Source: A. Tasks · pp. 13–14
- Flip Box asks a finger to rotate a ground-hinged heavy box 90° while minimizing energy, using contact-force leverage.Source: A. Tasks · p. 14
- Rubik’s Cube rotation requires turning the fixed cube’s top layer 90° without an objective-function heuristic specifying where to touch.Source: A. Tasks · p. 14
- Assemble uses two fingers to insert a similarly sized box into a movable mount while keeping the mount near its original position.The close size match makes insertion challenging and demands high-accuracy manipulation.Source: A. Tasks · p. 14
- Free-form Gripper deforms initially cubic fingers to grasp a diamond-like object, then shakes it to test stability under fixed control.Source: A. Tasks · p. 14
B. More Experiment Results
On Assemble, the proposed method converges faster and reaches lower loss than the baselines, while contact continuation helps only the proposed method. On Free-form Gripper, cage-based parameterization has better average optimization performance than mesh-based parameterization despite one failed trial.
B. More Experiment Results
- The proposed method converges significantly faster and reaches lower loss than all baselines on Assemble, with or without contact continuation.Continuation further reduces task cost for the proposed method, but does not improve the baselines and slows CMA-ES.Source: Figure 10 · p. 15, B. More Experiment Results · p. 15
- Across 30 Free-form Gripper runs, cage-based parameterization has better average optimization performance, although it fails once while mesh-based parameterization succeeds in all 30 trials.Source: Figure 11 · p. 15, B. More Experiment Results · p. 15