PaperlayerView exact revision on arXiv

Source-linked AI summary

RiCo: Neural Simulation of Rigid-Body Interactions via Local Contact Reasoning

Ruixiang Ouyang, Guanren Qiao, Fansen Meng, Yueci Deng, Ruixing Jin, Kui Jia, Guiliang Liu

arXiv:2610.12333v1cs.CVcs.AIcs.GRcs.LG

TL;DR

Rigid-body contact remains difficult to predict because collisions create abrupt, discontinuous motion changes, and point-cloud representations do not inherently identify which surface points may contact. RiCo uses sparse local contact neighborhoods and intra-object reasoning to predict rigid transformations, with reported gains in motion accuracy and contact fidelity and generalization to 270-object scenes.

  • Problem

    Collisions can abruptly change rigid bodies’ linear and angular velocities, making contact dynamics difficult to predict accurately.

    Source: §1 · p. 1, §2 · p. 3

  • Method

    RiCo combines each surface point’s state with nearby contact candidates and reasons across points within each object to decode rigid transformations.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    RiCo reduces position and orientation errors by up to 35% and 38%, respectively, while reporting 11.0% ∆PTR and 2.22 mm ∆MPD and generalizing to 270 objects.

    Source: §1 · p. 2, §4.3 · p. 8, §5 · p. 9

  • Takeaways & Limitations

    Sparse local contact reasoning supports accurate, contact-faithful simulation that scales to scenes containing 270 objects.

    Source: §5 · p. 9

    The framework covers passive rigid-body dynamics, not deformable objects or actuated systems.

    Source: §5 · p. 10

Abstract

from arXiv · show

Accurate simulation of rigid-body interactions is essential for predictive physical world models. Despite recent progress in modeling object dynamics, capturing how local contacts between surfaces shape object motion remains challenging. While end-to-end world models predict interactions across entire scenes or objects, in practice, rigid-body contact is inherently local, and only nearby surfaces can directly exchange contact forces. Motivated by this observation, we introduce Rigid-body Contact Reasoning (RiCo), which represents interactions between objects through sparse neighborhoods of contact surface points. RiCo combines each point's state with the relative geometry, motion, and physical properties of nearby surfaces, then reasons across the object's points to determine how these local contacts jointly affect its motion. By confining cross-object reasoning to nearby surfaces while propagating contact information within each rigid body, RiCo retains fine-grained interaction details without the cost of modeling every pair of scene points. Such properties enable RiCo a higher accuracy and contact fidelity. Experiments on MOVi-benchmark demonstrate that RiCo reduces 100-frame position and orientation errors by 31-35% and approximately 38%, respectively, compared with baselines. Moreover, RiCo achieves high contact fidelity, with ground-truth-relative penetration-time and mean-depth differences of 11.0% and 2.22 mm, respectively. RiCo further generalizes zero-shot from small-scale training scenarios to scenes containing 270 objects. Our real-world multi-ball collision experiments further provide preliminary evidence of sim-to-real transfer.

1 INTRODUCTION

Rigid-body collisions create abrupt motion changes that are difficult to predict, while existing geometric approaches can be costly or fail to identify which surface points may contact. RiCo models sparse local contacts and reasons across each body’s points; experiments report improved motion prediction, contact fidelity, and generalization.

1 INTRODUCTION

  • Collisions can abruptly change linear and angular velocities, making rigid-body interactions under varied geometry and motion difficult to predict accurately.Mesh connectivity and higher-order geometric relations can be computationally expensive, while point clouds do not inherently identify which surfaces may contact.Source: §1 · pp. 1–2
  • RiCo identifies sparse local contact relationships from point clouds and propagates their effects across each rigid body, linking fine-grained interactions to whole-object motion.It avoids mesh connectivity and dense interactions among all objects.Source: §1 · p. 2
  • For each surface point, RiCo combines its state with nearby candidates’ geometry, motion, and physical properties, then decodes anchor features into one rigid transformation per object.A point transformer performs intra-object reasoning over contact-conditioned features.Source: §1 · p. 2
  • Across diverse MOVi configurations, RiCo reduces position and orientation errors by up to 35% and 38%, reports 11.0% relative contact violations and 2.22 mm mean penetration depth, and generalizes to 270 objects.Real-world multi-ball experiments provide preliminary evidence of zero-shot sim-to-real transfer.Source: §1 · p. 2

2 RELATED WORK

Prior work ranges from learned dynamics and predictive world models to geometric simulators using meshes, signed-distance functions, or point clouds. Rigid-body contact remains difficult because collisions cause discontinuous changes in global motion; RiCo’s overview describes sparse local contacts followed by intra-object reasoning.

2 RELATED WORK

  • Learned world models span latent dynamics and feature-space prediction as well as explicit 3D representations for point-flow and multi-frame motion prediction.Source: §2 · p. 2
  • RiCo’s framework constructs sparse local contact neighborhoods, reasons across points within each object, and predicts rigid motion from anchors.The predicted state is recursively used for autoregressive rollout.Source: Figure 2 · p. 3
  • Rigid-body contact is challenging because interactions can abruptly change global object motion, producing discontinuous dynamics that smooth learned predictors struggle to capture.Source: §2 · p. 3
  • Geometric simulators progress from mesh message passing and face-level contact modeling to higher-order mesh relations, signed-distance functions, and mesh-free point-cloud inputs.RiCo retains point-level geometry for sparse local contact reasoning rather than relying primarily on coarse object-level representations.Source: §2 · p. 3

3 METHOD

RiCo predicts rigid-body dynamics by representing interactions as sparse neighborhoods of nearby surfaces and combining local contact information with within-object reasoning. It predicts motion from a small set of fixed surface anchors and trains both direct and rigid-projected anchor displacements.

3.1 SPARSE CONTACT NEIGHBORHOOD REPRESENTATION

  • RiCo retains object pairs whose surface-point AABBs are within contact radius ρ, then selects the K nearest candidates within ρ from those objects and the static environment.Dynamic-object candidates are sampled surface points, while static planar surfaces are queried by projection.Source: §3.1 · p. 4
  • Each point combines its own geometry, motion, and physical properties with nearby-surface descriptors, including signed distance as a cue to potential penetration.The point state includes inverse mass, friction, restitution, local normal, and displacement from its initial position.Source: §3.1 · p. 4

3.2 INTRA-OBJECT CONTACT REASONING

  • Contact-conditioned point tokens are aggregated with distance-dependent weights, then a shared point Transformer integrates contact information across each object without dense cross-object attention.The learnable temperature controls the concentration of candidate weights.Source: §3.2 · pp. 4–5

3.3 ANCHOR-BASED RIGID MOTION PREDICTION AND TRAINING

  • RiCo selects fixed anchors by farthest point sampling, predicts updates only at those anchors, and uses their predicted positions to recover a rigid transformation for the whole object.The predicted anchor displacements are used with the Kabsch algorithm to recover rotation and translation.Source: §3.3 · pp. 5–6
  • An anchor-to-point decoder uses each anchor query and the object’s point features to predict second-order anchor displacements, while training supervises both direct and rigid-projected predictions.Both loss terms use smooth-L1 in the normalized target space.Source: §3.3 · pp. 5–6

4 EXPERIMENT

The experiments evaluate RiCo on MOVi datasets using trajectory errors and ground-truth-relative penetration metrics, then examine geometry transfer, contact fidelity, large-scale prediction, and real-world transfer. Baselines include mesh-, point-, and transformer-based simulators.

4.1 MOTION PREDICTION PERFORMANCE ACROSS ROLLOUT HORIZONS

  • On MOVi-A, RiCo reduces position RMSE by approximately 35% and orientation RMSE by 38–43%; on MOVi-B, the reductions are 31–32% and 35–38%, respectively, versus the strongest baseline.On MOVi-Sphere, translational accuracy is comparable to RigidFormer, differing by at most 0.001 m at each horizon, while orientation error is lower.Source: §4.1 · p. 6, Table 1 · p. 7

4.2 GENERALIZATION ACROSS OBJECT GEOMETRIES

  • From MOVi-B, 100-step position RMSE on MOVi-Sphere is 0.085 m versus 0.160 m for HopNet and 0.207 m for RigidFormer; transfer toward more diverse geometries remains challenging.The reported generalization pattern is asymmetric: diverse training geometry transfers particularly well to simpler distributions, while sphere-only training does not consistently outperform RigidFormer on MOVi-A and MOVi-B.Source: Table 2 · p. 7, §4.2 · p. 7

4.3 IN-DEPTH STUDY ON THE CONTACT FIDELITY

  • RiCo’s ∆PTR is 11.0% and ∆MPD is 2.22 mm, versus HOPNet’s 42.7% and 286.62 mm and RigidFormer’s 44.8% and 267.70 mm.These ground-truth-relative penetration results accompany lower trajectory errors.Source: Figure 3 · p. 8, §4.3 · p. 8
  • Increasing resolution from 512 to 1024 points changes position RMSE from 0.112 to 0.111 m, while ∆PTR falls from 23.4% to 11.0% and ∆MPD from 6.21 to 2.22 mm.RigidFormer’s trajectory RMSE is nearly unchanged across resolutions, and its penetration statistics remain considerably higher despite increased surface sampling.Source: Figure 4 · p. 8, §4.3 · p. 9
  • Replacing signed distance with its absolute value raises ∆PTR from 11.0% to 23.6% and ∆MPD from 2.22 to 7.86 mm, while only moderately affecting trajectory RMSE.The authors suggest signed geometry helps capture contact configurations and reduce interpenetration.Source: §4.3 · p. 8, Figure 3 · p. 8
  • Across K ∈ {1, 4, 8}, prediction accuracy and penetration statistics remain relatively stable; K = 4 has the lowest ∆PTR and orientation RMSE, while K = 8 has slightly lower ∆MPD.The results suggest a compact neighborhood is sufficient for strong benchmark performance.Source: Figure 3 · p. 8, §4.3 · p. 9

4.4 ZERO-SHOT SCALABILITY TO LARGE-SCALE SCENES

  • Large-scale autoregressive accuracy is evaluated at 120, 240, and 360 steps; comparison with RigidFormer is qualified by differences in object geometry and training settings.The longer horizon examines prediction after extensive object interactions, which occur mainly in the latter parts of the trajectories.Source: Table 3 · p. 9, §4.4 · p. 9

4.5 TRANSFER FROM SIMULATED TO REAL-WORLD DYNAMICS

  • Trained only on simulated trajectories, RiCo is evaluated zero-shot on five real-world multi-ball sessions, with position RMSEs of 138.54, 214.69, and 272.42 mm at 50, 75, and 100 frames.The authors report that camera-based localization inaccuracy is a major source of error and describe these results as preliminary evidence of sim-to-real transfer.Source: §4.5 · p. 9

5 CONCLUSION

RiCo models rigid-body interaction through sparse local contacts and within-object reasoning, aiming to retain fine-grained contact geometry without dense scene-level interactions. The supplied sections describe its state representation, contact-neighborhood construction, and transition-model setup.

5 CONCLUSION

  • RiCo uses sparse local contact reasoning to model rigid-body interactions while avoiding dense scene-level interactions.The conclusion also characterizes its predictions as accurate and physically consistent, with high computational efficiency.Source: §5 · p. 9

A PROBLEM FORMULATION

  • Physical world models must predict interactions among rigid objects because contacts transfer momentum and can abruptly change translational and rotational motion.The formulation represents each rigid object with tracked surface samples and physical properties, and learns a transition from consecutive scene states for recursive rollouts.Source: A PROBLEM FORMULATION · p. 12

B ADDITIONAL METHOD DETAILS

  • The additional method details cover surface-state representation, sparse contact construction, network architecture, rigid-motion projection, and training and inference procedures.Source: B ADDITIONAL METHOD DETAILS · p. 12

B.1 SURFACE REPRESENTATION AND FEATURE PREPROCESSING

  • The default representation uses 1,024 ordered surface points per active body, preserves point identities across frames, and includes displacement from the initial surface configuration.The implementation also describes feature normalization and physical parameters broadcast to points.Source: §B.1 · pp. 12–13

B.2 SPARSE CONTACT NEIGHBORHOOD CONSTRUCTION

  • Contact neighborhoods retain nearby object pairs using an AABB separation threshold, then select up to K nearest external samples and include an analytic plane candidate.Training uses cached contact indices, whereas autoregressive inference queries neighborhoods from newly predicted geometry at each step.Source: §B.2 · pp. 13–14

B.3 NETWORK ARCHITECTURE

RiCo encodes source-point and contact features, reasons across points within each object, and predicts rigid motion through persistent anchors. The described setup uses anchor-based alignment and autoregressive inference.

B.3 NETWORK ARCHITECTURE

  • Source and contact descriptors are separately embedded to width 384, fused with masked distance-dependent weights, and processed by a shared six-block Transformer within each object.The anchor-query decoder attends to that object's point tokens and predicts second-order anchor displacement.Source: §B.3 · pp. 14–15, Table 5 · p. 15

B.4 ANCHOR-BASED RIGID MOTION PREDICTION

  • Eight fixed anchor indices provide queries; their predicted second-order displacements are rigidly aligned to recover a proper transform that advances the object's surface points and center of mass.The same anchor indices are retained through the rollout rather than dynamically reselected.Source: §B.4 · p. 15
  • At inference, RiCo starts from two consecutive observed states and applies the transition model autoregressively, recomputing local contacts from predicted geometry for the next step.Source: §B.4 · p. 16

C EXPERIMENTAL SETUP AND IMPLEMENTATION

  • The experimental setup section describes the offline data interface, optimization setup, evaluation protocol, and metric implementations for local-contact experiments.The standard model architecture is specified in Appendix B.Source: C EXPERIMENTAL SETUP AND IMPLEMENTATION · p. 16

C.1 BENCHMARKS AND DATA PROCESSING

MOVi benchmark processing uses released trajectory datasets and cached contact targets for training, while evaluation reconstructs contacts from predicted geometry. The fixed training representation does not limit inference object count.

C.1 BENCHMARKS AND DATA PROCESSING

  • Evaluation spans MOVi-Sphere, MOVi-A, and MOVi-B, which cover different geometric distributions and test rollout prediction and cross-dataset generalization.The standard offline implementation consumes precomputed trajectories and follows the released train/test partitions.Source: §C.1 · p. 16
  • Training data stores ten object slots, 1,024 surface points per active object, and eight fixed anchors; these storage dimensions do not constrain inference object count.Trajectory lengths and terminal flags prevent padded frames and windows crossing terminal transitions.Source: §C.1 · p. 16
  • RiCo's shared parameters across objects support zero-shot evaluation on scenes containing 270 objects, beyond the training representation's fixed object dimension.Source: §C.1 · p. 16
  • Teacher-forced training reconstructs contact descriptors from cached target indices, whereas autoregressive evaluation rebuilds neighborhoods from each predicted current geometry.Thus inference does not use future ground-truth contact assignments.Source: §C.1 · p. 16

C.2 OPTIMIZATION AND TRAINING CONFIGURATION

Training is offline and supervised, using two consecutive ground-truth states to predict the next state. The reference configuration and model-selection procedure are described separately from the reported training-loop details.

C.2 OPTIMIZATION AND TRAINING CONFIGURATION

  • The teacher-forced training loop predicts the next state from two consecutive ground-truth states and applies unit-weight smooth-L1 losses to direct and rigid-projected anchor displacements.AdamW is the optimizer, and only objects active at both current and target frames contribute.Source: §C.2 · p. 16
  • Table 6 summarizes the reference offline training configuration, while separate benchmark runs and ablations use configurations stored with their checkpoints.Source: Table 6 · p. 17, §C.2 · p. 17
  • Validation evaluates autoregressive predictions at stored trajectory frame indices 50, 75, and 100, and each final experiment specifies its checkpoint-selection criterion and computational resources.These are stored frame indices, not necessarily counts of neural updates after the two observation frames.Source: §C.2 · p. 17

C.3 EVALUATION METRICS AND MEASUREMENT PROCEDURES

The evaluation measures trajectory position and orientation error alongside ground-truth-relative penetration statistics. It estimates penetration using evaluation meshes and aggregates scene- and trajectory-level measurements before reporting them.

C.3 EVALUATION METRICS AND MEASUREMENT PROCEDURES

  • Position and orientation errors are computed per scene from active objects’ predicted and ground-truth states, with orientation measured by the geodesic angle of a rigid-alignment rotation.Reported frame values average per-scene RMSEs across trajectories; translation is in meters and orientation in degrees.Source: §C.3 · p. 17
  • Penetration depth is estimated by testing evaluation-mesh vertices for occupancy and measuring their distance to the other mesh’s surface, with ground-plane violations measured from minimum vertex height.The recorded depth is the maximum detected object–object or object–ground violation for an active object and frame.Source: §C.3 · p. 18
  • Penetration uses a geometric vertex-query approximation rather than an exact continuous-surface intersection solver, with no additional positive depth threshold.The reported differences are dataset-level summary differences, not per-frame positive excess penetration.Source: §C.3 · p. 18
  • Penetration-time ratio averages each trajectory–object pair’s fraction of penetrating active frames, excluding the first two frames from evaluation.Pairs are included if active in at least one evaluated frame.Source: §C.3 · p. 18
  • Ground-truth-relative penetration-time and mean-depth values are differences between aggregated predicted and ground-truth statistics, reported in percentage points and millimeters.Mean penetration depth averages detected penetrating active object–frame observations and is zero when its denominator vanishes.Source: §C.3 · p. 18

C.4 RUNTIME ANALYSIS

Runtime is measured during 100-step autoregressive rollouts on a single GPU using the same test trajectories. RiCo’s reported throughput is higher than the reported baseline throughputs, with implementation differences qualifying the comparison.

C.4 RUNTIME ANALYSIS

  • Timing used batch-one, 100-step autoregressive rollouts over 120 test trajectories on one RTX 5090, initialized with two ground-truth frames and then run without future ground truth.Five warm-up iterations and CUDA synchronization bracketed timing; loading, I/O, visualization, and rendering were excluded.Source: §C.4 · p. 18
  • On 120 test trajectories using a single NVIDIA RTX 5090 GPU, RiCo achieves 47.89 FPS at 1024 surface points per object, versus 41.73 FPS for our RigidFormer reimplementation and 2.61 FPS for HOPNet; at 512 and 256 points, it reaches 87.69 and 130.87 FPS, respectively.Source: §C.4 · p. 19
  • The RigidFormer throughput is from a reimplementation, while HOPNet throughput uses additional inference optimizations, so neither measurement represents an unmodified original implementation.All methods use the same GPU and test trajectories, but runtime comparisons remain subject to implementation differences.Source: §C.4 · p. 19

C.5 TEMPORAL ERROR ACCUMULATION IN LARGE-SCALE SCENES

In 270-object scenarios, prediction errors remain relatively small early in autoregressive rollouts and then rise faster as multi-object collisions become more frequent. The reported plots show this temporal accumulation across three scenarios.

C.5 TEMPORAL ERROR ACCUMULATION IN LARGE-SCALE SCENES

  • The rollout plots track position and orientation errors over time, with the approximate collision-transition point identified from qualitative observation of scene dynamics.The dashed marker does not specify a measured collision-count threshold.Source: §C.5 · p. 19
  • Prediction errors remain relatively small during earlier rollout stages, then increase more rapidly after more frequent multi-object interactions begin; this consistent trend across large-scale scenarios motivates reporting errors at 120, 240, and 360 frames.Source: §C.5 · p. 19
  • These trajectories complement endpoint metrics by showing error accumulation as long-horizon autoregressive simulation becomes increasingly challenging.The analysis concerns three 270-object scenarios.Source: §C.5 · p. 19

C.6 REAL-WORLD EXPERIMENTAL SETUP AND SIMULATED TRAINING

The real-world study uses three- and four-ball collisions on a billiard table, paired with a geometrically comparable simulated training environment. RiCo is trained only on simulated trajectories and evaluated on camera-observed real-world motion.

C.6 REAL-WORLD EXPERIMENTAL SETUP AND SIMULATED TRAINING

  • Real-world experiments capture three- and four-ball collisions on a billiard table with a 2.54 × 1.27 m playing area and balls of 33.5 mm radius.Source: §C.6 · p. 20
  • The simulated environment matches the table dimensions and ball geometry, while randomizing ball positions within ±0.25 m, moving-ball speed from 1.5–2.5 m/s, and direction within ±5°.The randomizations expose training to varied contact locations, collision angles, and subsequent interactions.Source: §C.6 · p. 20
  • RiCo is trained exclusively on 960 simulated trajectories of 2 seconds each, without real-world observations or real-world fine-tuning.Source: §C.6 · p. 20
  • Zero-shot evaluation covers five real-world sessions and 23 valid ball samples, comparing autoregressive predictions and physics-engine trajectories with observations using position RMSE at frames 50, 75, and 100.Initial real-world states are reconstructed from camera-estimated positions.Source: §C.6 · pp. 20–21

Related papers

Loading 2610.12333v1…