Source-linked AI summary
NeuralSim: Augmenting Differentiable Simulators with Neural Networks
Eric Heiden, David Millard, Erwin Coumans, Yizhou Sheng, Gaurav S. Sukhatme
TL;DR
Analytical differentiable simulators remain limited by unmodeled dynamics that contribute to the sim-to-real gap. NeuralSim embeds neural networks within a differentiable rigid-body engine, learning such effects with physically meaningful inputs. The hybrid approach improves modeling efficiency and generalization, while neural replacement of a gait-controller QP solver reduces inference time ten-fold.
Problem
Analytical simulators cannot represent dynamical effects absent from their designed models, motivating methods that close the sim-to-real gap.
Method
NeuralSim embeds neural networks inside a differentiable rigid-body physics engine to learn unmodeled effects from physically meaningful quantities.
Results
The hybrid simulator requires less data and generalizes better than entirely data-driven baselines across frictional contacts and viscous drag, while supporting automatic augmentation discovery.
Takeaways & Limitations
Fine-grained neural augmentation supports sim-to-real modeling and can accelerate model-based control computations.
Takeaways & Limitations
The real-robot controller retains a sim-to-real gap: simulation-trained data produced a slightly lower torso height during trotting.
Abstract
from arXiv · showhide
Differentiable simulators provide an avenue for closing the sim-to-real gap by enabling the use of efficient, gradient-based optimization algorithms to find the simulation parameters that best fit the observed sensor readings. Nonetheless, these analytical models can only predict the dynamical behavior of systems for which they have been designed. In this work, we study the augmentation of a novel differentiable rigid-body physics engine via neural networks that is able to learn nonlinear relationships between dynamic quantities and can thus learn effects not accounted for in traditional simulators.Such augmentations require less data to train and generalize better compared to entirely data-driven models. Through extensive experiments, we demonstrate the ability of our hybrid simulator to learn complex dynamics involving frictional contacts from real data, as well as match known models of viscous friction, and present an approach for automatically discovering useful augmentations. We show that, besides benefiting dynamics modeling, inserting neural networks can accelerate model-based control architectures. We observe a ten-fold speed-up when replacing the QP solver inside a model-predictive gait controller for quadruped robots with a neural network, allowing us to significantly improve control delays as we demonstrate in real-hardware experiments. We publish code, additional results and videos from our experiments on our project webpage at https://sites.google.com/usc.edu/neuralsim.
I. INTRODUCTION
The paper addresses sim-to-real errors caused by effects omitted from analytical rigid-body models by embedding neural augmentations inside a differentiable simulator. Experiments show improved data efficiency and generalization, automatic augmentation discovery, and faster model-based control.
- Motivation: Differentiable simulators enable gradient-based estimation of simulation parameters, but rigid-body models remain limited by unmodeled real-world effects.Examples include link bending and air resistance.
- Method: The proposed augmentations are embedded within the physics engine, depend on physically meaningful quantities, and are trained through end-to-end gradient optimization.They modulate forces and other simulation variables rather than correcting only the predicted state externally.
- Results: Hybrid simulation outperforms deep learning baselines in training efficiency and generalizability across learned drag and contact-friction dynamics.The experiments include swimmer drag forces and planar pushing with neural contact-friction models.
- Results: Sparse regression identifies relevant inputs for data-driven models, enabling automatic discovery of locations for neural augmentations.The approach supports selecting physically meaningful inputs for inserted networks.
- Control: An order-of-magnitude inference-time reduction results when a neural network replaces the QP solver in a model-predictive gait controller on a Unitree A1 quadruped.The experiment is conducted on a real robot.
- Contribution: The authors release Tiny Differentiable Simulator, an open-source engine for articulated rigid-body dynamics, contact models, and automatic differentiation.Gradients can be computed with respect to quantities inside the engine.
II. RELATED WORK
Prior work spans traditional, differentiable, data-driven, physics-informed, and residual physics models. NeuralSim positions its contribution as fine-grained neural augmentation inside a differentiable physics engine with end-to-end optimization.
- Existing dynamics models: Dynamics-learning research includes regression, forward models, graph neural networks, and physics-based machine learning approaches.These approaches differ in how they use data and physical inductive bias.
- Differentiable simulation: Differentiable simulators compute parameter gradients through analytical Newton-Euler equations and rigid-body contact models using automatic, symbolic, or implicit differentiation.Differentiable simulation has also been applied to light propagation and continuum mechanics.
- Residual physics models: Residual physics models reduce the sim-to-real gap by adding learned models to physics engines, typically through residual corrections outside the predicted state.NeuralSim instead introduces models at selected internal simulation components.
- NeuralSim distinction: NeuralSim differentiates through the entire computation graph, allowing gradients from high-level states to optimize neural-network weights and other simulator components.The cited comparison contrasts this with supervised training of actuator-dynamics networks.
III. APPROACH
The approach combines articulated rigid-body dynamics and contact models with neural networks inserted into simulation variables. The differentiable engine computes dynamics and advances them with numerical integration while retaining physically interpretable analytical parameters.
- Rigid-body dynamics: The simulator uses the Articulated Body Algorithm to compute forward dynamics from joint positions, velocities, torques, and external forces.Semi-implicit Euler integration advances the resulting joint accelerations through time.
- Contact models: Contact is modeled with either an impulse-level projected Gauss-Seidel solver for nonlinear complementarity or a Hunt-Crossley penalty model.The two options address multiple simultaneous contacts and per-contact nonlinear spring-damper forces, respectively.
- Model parameters: Analytical parameters include link masses, inertias, centers of mass, joint properties, friction coefficients, stabilization parameters, and contact regularization terms.These parameters have physically interpretable effects and may be measured or optimized from data.
- Neural augmentation: Neural augmentation lets a variable combine an analytical function of inputs a and b with a neural network using inputs a, c, and d.The neural scalar mechanism permits learned dependencies to be inserted into the simulator computation.
- Neural augmentation: An exemplar hybrid architecture learns passive forces τ from joint velocities, with joint positions and network biases omitted from the illustration.This architecture represents fine-grained augmentation of the physics computation.
- Dynamics formulation: The simulator expresses dynamics through generalized inertia, Coriolis and centrifugal effects, and passive and gravitational forces.These terms are computed from the analytical model parameters.
A. Hybrid Simulation
The proposed hybrid simulation augments differentiable rigid-body physics with neural networks that can be inserted throughout the simulation and trained to learn unmodeled effects. The framework also supports defining neural augmentations separately from simulation code.
- A. Hybrid Simulation: Hybrid simulation augments analytical rigid-body dynamics with neural networks to learn effects that fitted physical models do not account for.The approach allows any part of the simulation to be replaced or augmented by neural networks with parameters θ_NN.
- A. Hybrid Simulation: Neural networks can augment any simulation variable and receive input connections from any other variable.Neural scalars are assigned unique names, while separate neural blueprints declare architectures and network weights.
- A. Hybrid Simulation: System identification estimates double-pendulum link lengths from joint-position trajectories, comparing local optimization with parallel basin hopping.The true link lengths are 3 and 4, and darker colors indicate lower ℓ2 parameter error.
B. Overcoming Local Minima
Contact-rich nonlinear systems produce loss landscapes with many local minima, so the method uses global optimization strategies that combine population evolution with local gradient-based optimization.
- B. Overcoming Local Minima: Nonlinear or discontinuous contact systems create highly nonlinear loss landscapes containing many local minima.This motivates global rather than purely local optimization strategies.
- B. Overcoming Local Minima: Parallel basin hopping and population-based methods run local L-BFGS optimizers in separate threads.After each population evolution, candidate solutions are combined and mutated for the next iteration.
C. Implementation Details
The physics engine is implemented in C++ with interchangeable mathematical backends and multiple automatic-differentiation frameworks. Benchmarking shows that precompiled gradient passes can provide orders-of-magnitude speed-ups.
- C. Implementation Details: Tiny Differentiable Simulator is implemented in C++ while keeping linear-algebra computations independent of specific library implementations.Template metaprogramming supports different mathematical libraries without changing experiment code.
- C. Implementation Details: Runtime comparisons use a logarithmic time scale to compare finite differences with the supported automatic-differentiation frameworks.The figure reports time in seconds.
- C. Implementation Details: The engine supports Jet, Stan Math, CppAD, and CppADCodeGen for forward- and reverse-mode automatic differentiation.It also supplies operators for differentiating through conditionals.
- C. Implementation Details: Orders-of-magnitude speed-up is observed when CppADCodeGen precompiles the gradient pass for the rolled-out cost function.The benchmark evaluates a 175-parameter neural network with pendulum simulations involving 5000 or 500 steps, depending on the task.
A. Friction Dynamics in Planar Pushing
The planar-pushing experiment augments an analytical contact model with neural networks to capture complex friction dynamics. The model uses predefined contact points and force-level augmentation based on contact and motion variables.
- A. Friction Dynamics in Planar Pushing: The Push Dataset contains planar shapes pushed by a cylindrical robot tip across different surfaces, with constant-velocity trajectories selected for modeling.The contact is modeled with an NCP-based solver and a nonlinear spring-damper ground model.
- A. Friction Dynamics in Planar Pushing: Predefined contact points are placed on each shape using a uniform 2 cm grid.The force-level model is easier to augment because normal and friction forces are computed independently at each contact point.
- A. Friction Dynamics in Planar Pushing: Even after tuning analytical contact parameters, simulated object poses retain significant error relative to real-world trajectories.The untuned setup also visibly diverges between real and simulated poses in the example push experiment.
- A. Friction Dynamics in Planar Pushing: Augmenting the lateral 2D friction force produces a significantly closer match to real-world ground truth.The neural network uses object pose, velocity, contact normal, and penetration depth as inputs, with gradients propagated through integrated dynamics.
B. Passive Frictional Forces for Articulated Systems
The hybrid simulator augments rigid-body dynamics to model passive forces from viscous media and improve long-horizon prediction in sparse-data settings.
- The augmentation learns passive viscous-friction and damping forces for a multi-link swimmer moving through unidentified media.The learned forces depend on each link’s angle of attack and velocity.
- Exact end-to-end gradients are computed through the simulator, integrator, and function approximator using trajectory MSE loss.
- The swimmer experiments cover water at 5 °C and 25 °C and air at 25 °C, which differ in viscosity and density.
- More accurate long-term rollouts were obtained over 900 timesteps after training on only the initial 200 timesteps of 10 trajectories.The experiment used a 2-link swimmer in water at 25 °C.
- The augmentation networks used 1637 trainable parameters, compared with 1900 for the LSTM baseline.
C. Discovering Neural Augmentations
The method automatically identifies compact neural augmentations by sparsifying their inputs and penalizing their overall influence on simulated dynamics.
- The sparse augmentation objective seeks minimally invasive neural models that retain predictive accuracy and generalizability.
- The predicted state fθ(st−1) is compared with the observed state s∗t using a weighted objective with regularization terms.
- The cost function sparsifies first-layer input weights and penalizes upper-layer weights to limit the augmentation’s contribution.This encourages dependence on a small subset of physical quantities while preserving analytical-model behavior where possible.
- In the double-pendulum experiment, the hybrid simulator required orders of magnitude fewer training samples than a fully learned dynamics model.
- The discovered augmentation identifies joint velocity as influencing the residual joint force associated with joint friction.
D. Imitation Learning from MPC
A neural policy imitates the MPC quadratic-program solver to accelerate quadruped control, while simulation-trained imitation retains a measurable sim-to-real limitation.
- The neural policy imitates the QP solver by mapping the same system state to foot contact forces used by inverse kinematics and PD control.The training set contains 10k solver-generated state transitions.
- The learned controller produces simulated Laikago walking gaits and in-place trotting on a real Unitree A1 quadruped.
- The learned QP solver runs at 0.2 ms inference time versus 2 ms for the original solver on an AMD 3090 Ryzen CPU.
- Control frequency increased from 160 Hz to 300 Hz after replacing the solver with the neural network.
- Simulation-only training leaves a sim-to-real gap that produces a slightly lower torso height during real-robot trotting.
V. CONCLUSION
The paper concludes that differentiable hybrid simulators can learn unmodeled dynamics, improve sim-to-real transfer, and accelerate model-based control, while several extensions remain open.
- Neural networks can be inserted throughout a differentiable rigid-body simulator to learn effects absent from its analytical model.
- Experiments show improved sim-to-real transfer for frictional pushing and learning of fluid-drag forces not present in the rigid-body simulator.
- Replacing the quadruped’s QP-based controller produced an order of magnitude computation speed-up.
- Future work will investigate incorporating physical constraints such as conservation of energy into neural-network training.