Virtual Elastic Objects

• Virtual Elastic Objects look like real-world objects and behave like them during novel interactions.Source: §1 · p. 1
• The full-loop pipeline spans capture, reconstruction, simulation, interaction, and re-rendering for deforming elastic objects.Source: §1 · pp. 1–2
• A differentiable particle-based simulator uses reconstructed deformation fields to estimate physical parameters and generate responses to novel interactions.Source: §1 · p. 2
• The PLUSH dataset records elastic objects deforming under known controlled force fields with an occlusion-free multi-camera capture rig.Source: §1 · p. 2
• The method reconstructs geometry and temporal deformation without object templates, extending Non-rigid Neural Radiance Fields with losses and point correspondences.Source: §1 · p. 2
• A mapping function re-renders Neural Radiance Fields from simulated point-cloud deformations rather than continuous deformation fields.Source: §1 · p. 2

Recovering Elastic Parameters for 3D Templates

• Prior elastic-parameter methods track pre-scanned templates or markers, whereas this method reconstructs geometry, appearance, deformation, and physics without templates, depth, or markers.Source: §2 · p. 2

Dynamic Reconstruction

• Dynamic reconstruction methods include template-based RGB or RGB-D approaches, model-free RGB-D reconstruction, non-rigid structure-from-motion, image-based methods, and neural representations.Source: §2 · p. 2

Data-Driven Physics Simulation

• Data-driven physics work learns natural laws, deformation subspaces, simulation details, or material parameters, but unrestricted learning remains computationally intractable for real-world systems.Source: §2 · p. 3

Meshless Simulation

• Meshless simulation complements mesh-based methods and has been extended with oriented particles, skinning, and Eulerian grids for fluid-like phenomena.Source: §2 · p. 3

3.1. Capture

• 19-camera capture records RGB and grayscale views while transparent fishing line and compressed air apply forces without occluding the object.The setup uses a dark, uniformly lit capture volume and reconfigures camera placement for object size.Source: §3.1 · p. 3

3.2. 4D Reconstruction

• NR-NeRF reconstructs a canonical radiance field and time-conditioned deformations, then inverse warping and surface sampling produce temporally corresponding point clouds.Foreground and density losses improve object interiors, and sequences are downsampled to 9–15k points when necessary.Source: §3.2 · pp. 3–4

3.4. Novel Interactions

• The inferred simulator supports novel force fields, altered gravity, contact forces, and manipulation by changing external-force terms or adding contact penalties.Contact with a secondary object is represented using a signed distance field and a penalty energy.Source: §3.4 · pp. 5–6

3.5. Rendering

• Simulation point clouds are converted into continuous backward-warping fields through inverse-distance interpolation, enabling neural-radiance-field rendering of interactively deformed objects.Opacity is suppressed away from the simulated surface, and point clouds can be upsampled when needed.Source: §3.5 · p. 6

4.1. Dataset

• The PLUSH dataset contains 12 soft everyday objects captured under controlled fishing-line and compressed-air forces using a tracked multi-camera rig.Recordings last 32–67 seconds at 40 FPS, with nozzle position and direction tracked to estimate applied forces.Source: §4.1 · p. 6

4.2. Virtual Elastic Objects

• The method achieves relatively small errors for all 12 objects, using 20–50 reconstruction frames to create each VEO and evaluating on the remaining 500–1500 frames.Quality is evaluated by ℓ2 distance between VEO surface points and reconstructed point clouds; Table 1 reports average, 95th-percentile, and maximum distances.Source: §4.2 · p. 6, Table 1 · p. 6
• Inhomogeneous material parameters better reproduce core-body, arm, and ear postures than a homogeneous-material baseline.The homogeneous model distributes deformation more evenly and has larger error, while the inhomogeneous model better matches the ground-truth point cloud.Source: Figure 3 · p. 7, §4.2 · p. 7
• The method identifies spatially varying softness, including softer ears and arms on Baby Alien and a softer mane and tail on Pony.These effective softness estimates remain reliable in thin regions and areas with unreliable tracking.Source: §4.2 · p. 7, Figure 2 · p. 7
• The learned simulator generalizes Baby Alien to asymmetric ear and arm poses absent from training, including pulling the ears in opposite directions.Training contains only symmetric forward and backward motions, whereas the tested poses move ears or arms independently.Source: Figure 4 · p. 7, §4.2 · p. 7
• Simulated VEOs render realistic novel interactions with secondary virtual objects, including dinosaur-neck bending and dents forming on a dog’s back.Simulator point clouds are reported to be very close to reconstructed point clouds in quantitative and qualitative comparisons.Source: §4.2 · p. 8, Figure 5 · p. 8

• Sparse 16-camera coverage causes viewpoint-dependent artifacts around specular regions, such as eyes.These artifacts affect extracted point clouds and final renderings.Source: §5 · p. 8
• Rapidly oscillating surfaces from the air compressor challenge reconstruction, material-parameter estimation, and calibration.The authors report examples including fish fins and note resulting artifacts in point clouds and renderings.Source: §5 · p. 8
• Despite noisy reconstructed point clouds, the physical simulator remains robust when temporal correspondences are available.Source: §5 · p. 8
• Known capture-time forces restrict the variety of applicable forces and therefore the kinds of objects compatible with the method.The authors identify handling unknown forces as an important direction for future work.Source: §5 · p. 8

• The paper estimates physical parameters from RGB observations and known forces, then renders physically plausible responses to novel interactions.Source: §6 · p. 8
• Virtual Elastic Objects synthesize deformed states that greatly differ from the observed deformations.Source: §6 · p. 8
• A physical simulator estimates plausible parameters from a captured object’s 4D reconstruction, enabling new simulated states.Source: §6 · p. 8
• The resulting deformed states can be re-rendered with high quality, and the accompanying dataset is intended to support future interactive-object research.Source: §6 · p. 8