VoMP: Predicting Volumetric Mechanical Property Fields

1 INTRODUCTION

• Accurate simulation requires spatially varying mechanical properties, but common 3D captures and repositories rarely contain them, leaving artists and engineers to guess or reuse coarse presets.This makes reliable simulation setup labor-intensive and subjective.Source: §1 · p. 1
• VoMP is a feed-forward model for predicting volumetric Young’s modulus, Poisson’s ratio, and density across 3D representations.It targets properties throughout object volumes rather than only surface values.Source: §1 · pp. 1–2
• VoMP accepts voxelizable, renderable geometries such as meshes, Gaussian Splats, NeRFs, and SDFs without per-object optimization or run-time aggregation of large-model supervision.It predicts real-world mechanical parameters throughout the object volume, rather than simulator-specific parameters.Source: §1 · p. 2
• A MatVAE trained on real-world material values supplies a latent space intended to keep decoded per-voxel properties physically valid, including under interpolation.The training-data annotation pipeline combines segmented 3D assets, material databases, visual textures, and a vision-language model.Source: §1 · p. 2
• VoMP is evaluated with high-fidelity simulations and quantitative metrics on existing and new benchmarks, with the authors reporting significant outperformance over prior art.The paper also contributes an automatic annotation pipeline and a new benchmark for volumetric physics materials.Source: §1 · p. 2

2.1 BACKGROUND

• For locally isotropic material models, Young’s modulus, Poisson’s ratio, and density are widely used parameters; accurate values at every point support consistent simulation predictions under load.Measured real-world parameters are portable across consistent simulation algorithms and material models that use these properties or derived quantities.Source: §2.1 · pp. 2–3
• Many simulation implementations favor speed over accuracy, so their material parameters may need modification and may not transfer reliably across frameworks.Figure 2 contrasts simulator behavior for a dropped solid sphere across XPBD, MPM, and FEM.Source: §2.1 · pp. 2–3

2.2 INFERRING MECHANICAL PROPERTIES OF STATIC OBJECTS

• Learning volumetric properties from shape and appearance is hampered by datasets that are small, noisy, coarse, biased, simulator-specific, or lack spatial material fields.Collecting data also requires difficult physical experiments and digitization and annotation work.Source: §2.2 · p. 3
• Prior methods include per-object feature-field optimization, coarse material categories, simulator-specific outputs, and models restricted to particular 3D representations or object surfaces.NeRF2Physics and PUGS predict coarse stiffness categories and density but have limited access to meaningful interior features; Pixie’s training segments are biased toward surfaces.Source: §2.2 · pp. 3–4
• SOPHY and PhysX-3D generate new shapes with physical attributes, whereas VoMP targets deterministic material prediction for existing assets and volumetric properties.The comparison also notes that VoMP uses multiple data sources rather than relying only on VLM knowledge.Source: §2.2 · pp. 3–4

3 MECHANICAL PROPERTIES LATENT SPACE

• MatVAE maps real-world material triplets into a 2D latent space that supports visualization, sampling, and interpolation while keeping decoded properties within the range of some materials.The mapping compresses triplets from three dimensions to two and acts as a continuous tokenizer for VoMP outputs.Source: §3 · p. 4
• The reconstruction loss is mean-squared error between input and reconstructed material triplets, with per-property normalization and log transforms for Young’s modulus and density.Poisson’s ratio is directly normalized to [0, 1]; the authors report that alternative schemes produce poorly conditioned heavy-tailed features.Source: §3 · p. 4
• MatVAE uses a normalizing flow for a more flexible variational distribution, penalizes total correlation to reduce dependence between latent coordinates, and adds a capacity constraint to encourage use of both dimensions.The authors motivate these changes by heavy-tailed property distributions and imbalanced reconstruction that can collapse the latent space toward one property.Source: §3 · p. 4

4 PREDICTING MECHANICAL PROPERTY FIELDS

• VoMP combines voxelized image features with a trained transformer and MatVAE latents to predict volumetric mechanical properties across 3D representations.The per-voxel materials can be transferred to the original representation using nearest-neighbour interpolation.Source: §4 · p. 5, §4.2 · p. 6

4.1 AGGREGATING FEATURES

• Each voxel receives the average of DINOv2 image features sampled at its projected center across rendered views, including features for interior voxels.Processing object interiors, not only surfaces, allows the model to predict internal material properties.Source: §4.1 · p. 5

4.2 GEOMETRY TRANSFORMER

• The Geometry Transformer serializes active voxel features, adds 3D positional encodings, and uses shifted-window attention to predict material latents.For assets exceeding the maximum sequence length, it samples a random subset of voxels each training epoch.Source: §4.2 · p. 5
• During training, the predicted per-voxel latents pass through a frozen MatVAE decoder, and the transformer is optimized against ground-truth material triplets.The loss is mean squared error averaged over voxels in the selected set.Source: §4.2 · pp. 5–6

5.1 MATERIAL TRIPLETS DATASET (MTD)

• The Material Triplet Dataset contains 100,562 real-world (E, ν, ρ) triplets sampled from measured material-property ranges, with duplicates filtered out.The number of samples per material is proportional to its range size.Source: §5.1 · p. 6

5.2 GEOMETRY WITH VOLUMETRIC MATERIALS (GVM) DATASET

• The GVM pipeline combines segmented 3D models, part material names and textures, and nearby MTD material ranges to guide VLM annotations.It covers 1,624 models and 8,089 parts, yielding 37M voxels annotated with (E, ν, ρ).Source: §5.2 · p. 6

6.1 IMPLEMENTATION DETAILS

• The Gaussian-splat voxelizer carves exterior space from multi-view depth maps, then samples the solid; its GPU and octree implementation voxelizes test objects in 31 ms.Training uses an 80-10-10 train/validation/test split and Qwen 2.5 VL-72B for material annotation; FEM is used for mesh simulations and sparse Simplicits for large-scale splat-and-mesh simulations.Source: §6.1 · pp. 6–7

6.2 END-TO-END QUALITATIVE EVALUATION

• Predicted spatially varying mechanical properties produce realistic simulations without hand-tweaks for meshes and 3D Gaussian Splats.The approach also works across meshes, 3D Gaussian Splats, SDFs, and NeRFs.Source: §6.2 · p. 7

6.3 QUANTITATIVE EVALUATION

• VoMP's material estimates are 5–100x faster than prior methods in the reported comparison.Timing averages 100 runs on objects with 53.9K Gaussians; rendering and preprocessing take the most time and could be further optimized.Source: Table 1 · p. 7, §6.3 · p. 8
• VoMP shows a dramatic quality boost across all properties and metrics versus prior art on the detailed benchmark.For released-data estimates, the unavailable vegetation subset is omitted; public-dataset averages are highly similar to full-dataset averages.Source: §6.3 · pp. 7–8
• For ABO-500 mass estimation, VoMP performs better or on par across most metrics, although mass is only an imperfect proxy for volumetric density accuracy.Predicted voxel densities are averaged and multiplied by known object volume to estimate mass.Source: §6.3 · p. 8
• On average, VoMP outputs more realistic materials than the compared methods when tested against real-material ranges.Validity is measured by each voxel’s relative error to the nearest possible material range in MTD; estimates within a range have zero error.Source: §6.3 · p. 8

6.4 RECONSTRUCTING AND GENERATING MATERIALS WITH MATVAE

• MatVAE has low reconstruction errors, maps latent-space samples to real-world material ranges, and supports smooth variation and valid interpolation between materials.These properties facilitate assigning predicted voxel materials back to the original geometry.Source: §6.4 · p. 8

• The method maps meshes, SDFs, Gaussian splats, or voxel grids to volumetric fields of physically valid E, ν, and ρ.Source: §7 · p. 8
• VoMP significantly outperforms prior art in accuracy and speed, with potential to lower barriers to physics in digital workflows.Source: §7 · p. 9
• Fixed-grid voxelization limits output resolution, causing oversmoothing in highly heterogeneous regions and possible approximation errors on more detailed input geometry.The annotation pipeline also assumes isotropic part-level materials, which does not hold for some materials such as wood.Source: §7 · p. 9
• MatVAE shows excellent reconstruction errors across all metrics on the MTD test set.Source: §7 · p. 10
• Figure 7 presents MatVAE’s material latent space as faithful, valid, smoothly varying, and interpolatable.Source: Figure 7 · p. 10

A.1 END-TO-END EXAMPLES WITH SIMULATION

• VoMP estimates properties for meshes and Gaussian splats and produces realistic simulations without fine-tuning; a multi-object Gaussian-splat scene is also simulated.The described workflow estimates properties in a couple of seconds, and the authors report qualitatively accurate voxelization of complex, noisy splat objects.Source: §A.1 · p. 22

A.2 MORE MECHANICAL PROPERTY PREDICTION RESULTS

• Examples show predictions for small details, unobserved interiors, noisy assets, thin boundaries, leaves, and complex volumetric materials.The appendix also reports the same performance boost over prior art with voxel-normalized metrics as with the main paper’s per-object normalization.Source: §A.2 · p. 22

Supplementary Material for VoMP: Predicting

• Voxel-averaged results are reported for the publicly released dataset, which excludes the vegetation subset.Source: §A.2 · p. 22

A.3 MASS ESTIMATION EXAMPLE RESULTS

• Three shown mass estimates are 1.58 kg vs 1.58 kg, 0.01 kg vs 0.01 kg, and 0.71 kg vs 0.70 kg predicted versus ground truth.The appendix also presents qualitative mass-estimation results on ABO-500.Source: §A.3 · pp. 22, 26

A.4.1 DISTRIBUTION LEARNED BY MATVAE

• MatVAE’s learned distribution has small errors against the MTD test-set distribution, suggesting it approximates the true material distribution well.Source: §A.4.1 · p. 27

A.4.2 MOVING ACROSS MATVAE LATENT SPACE

• Across latent-space traversals, material properties vary smoothly, and bunny displacement during FEM dropping simulations correlates with latent dimensions.The authors conclude that MatVAE provides smooth interpolation while generating physically valid material triplets.Source: §A.4.2 · p. 27

A.4.3 INTERPOLATION WITH MATVAE

• Each MatVAE traversal setting tests 25 decoded material triplets in bunny simulations, and the authors observe correlation between latent dimensions and simulation behavior.The appendix also shows additional interpolation examples in the latent space.Source: §A.4.3 · p. 27, Figure 13 · p. 28

• VoMP annotates from meshes, whereas Pixie obtains points from NeRF training and uses CLIP-feature segmentation, which the authors say can be noisy on difficult objects.Source: B COMPARISON WITH CONCURRENT WORK · p. 29
• Some mechanical-property ranges in Pixie’s in-context physics examples may fall outside real-material ranges, although they may yield pleasing simulations with a particular simulator.The authors compare these examples with real-material ranges from MTD.Source: B COMPARISON WITH CONCURRENT WORK · p. 29
• The appendix includes a comparison of Pixie and VoMP outputs for one object.Source: B COMPARISON WITH CONCURRENT WORK · p. 30, Figure 16 · p. 30

C ABLATIONS

• MatVAE outperforms a vanilla VAE on almost all reported metrics; the vanilla VAE collapses toward Young’s modulus and raises errors for other properties.Table 7 compares reconstruction and distributional metrics.Source: C ABLATIONS · p. 30, Table 7 · p. 31
• DINOv2 and CLIP perform slightly worse without TRELLIS initialization, while RGB features perform significantly worse.These feature ablations start from random weights; Table 8 reports voxel-level property-difference errors.Source: C ABLATIONS · p. 30, Table 8 · p. 32
• Directly predicting material triplets instead of using MatVAE produces significantly worse Young’s modulus and Poisson’s ratio estimates.The comparison treats MatVAE as a continuous tokenizer.Source: C ABLATIONS · p. 30
• Removing log scaling for Young’s modulus or using Z-score normalization severely harms prediction accuracy.The tested normalization changes all significantly degrade performance; Table 8 covers the normalization ablation.Source: C ABLATIONS · p. 30, Table 8 · p. 32
• Replacing ℓ2 with ℓ1 increases errors by 2-3x for most properties across metrics.The authors report that this change substantially reduces performance; Table 8 covers the loss ablation.Source: C ABLATIONS · p. 30, Table 8 · p. 32

D.1 METRICS FOR MASS AND FIELD ESTIMATION

• ADE is scale-sensitive and reports average absolute error in physical units, while ALDE measures average absolute error in logarithmic space for multiplicative differences.ALDE is particularly useful for quantities spanning several orders of magnitude.Source: §D.1 · p. 31

D.2 METRICS TO MEASURE DIFFERENCES IN MECHANICAL PROPERTIES

• Mechanical-property metrics cover individual properties, derived stiffness measures, and Ashby indices; Bray–Curtis summarizes overall divergence across property vectors.The listed measures include relative errors in E, ν, ρ, E/ρ, G, K, and two Ashby indices.Source: §D.2 · p. 32

D.3 METRICS TO MEASURE DIFFERENCES IN DISTRIBUTIONS

• Wasserstein–1 and Wasserstein–2 measure transport costs between distributions, while KL divergence measures extra expected log-likelihood when coding p-distributed data with q.The section also presents simulations to interpret relative error in relation to simulation error.Source: §D.3 · p. 33, Figure 17 · p. 33

D.4 INTERPRETING ERRORS FOR MATERIAL PROPERTY ESTIMATION

• Each baseline material triplet is perturbed linearly for density and Poisson’s ratio, but exponentially for Young’s modulus to accommodate its wide stiffness range.Each resulting triplet is applied to a unit cube under external forces.Source: §D.4 · p. 33
• The measured outcomes are final cube volume and potential energy after Newton iterations converge.Relative volume change is computed against the reference volume.Source: §D.4 · p. 33
• For robotic gripping, Figure 18 relates relative material errors to relative changes in potential energy and volume, showing confidence bounds in light-shaded regions under a 140 N compressive force.Source: Figure 18 · p. 34, §D.4 · p. 34
• Potential energy combines elastic strain energy with inertia, gravity, and external work, evaluated at the converged displacement field.The simulations use corotated linear elasticity for elastic strain energy.Source: §D.4 · pp. 33–34
• Each scenario uses 486 converged simulations: 140 N compression for robotic gripping, 120 N impact for package drops, 330 N tensile force, or 200 N tension.The impact force models a 1 kg package dropped from 0.6 m with a 5 cm deformation distance.Source: §D.4 · p. 34

E.1 ANNOTATION WITH VISION-LANGUAGE MODEL

• Qwen2.5-VL 72B was selected to annotate object segments using 3D assets, part segmentations, material databases, textures, and material names.The authors compared several VLMs and tuned system prompts with TextGrad.Source: §E.1 · p. 35
• On a tiny manually annotated dataset, Qwen2.5-VL 72B performed close to human annotators when given the additional information used by the pipeline.The comparison is reported in Table 9 using mechanical-property difference metrics.Source: §E.1 · p. 36, Table 9 · p. 36

E.2 DATASET STATISTICS

• The object dataset contains 1,692 objects and 8,128 segments drawn from four datasets; material names come from labels or, for some parts, PBR texture names.The authors also analyze property distributions and frequent material categories and object classes.Source: §E.2 · p. 36, Table 10 · p. 38, Table 11 · p. 38

F.1 DESIGN OF MATVAE

• MatVAE learns a two-dimensional latent space from real-world (E, ν, ρ) triplets; a radial normalizing flow expands the posterior family to avoid mode-averaging on non-Gaussian material data.The flow is motivated by heavy-tailed or multimodal log E and log ρ distributions and boundary-concentrated ν values.Source: §F.1 · pp. 38–39

F.2 NETWORK DESIGN

• The MatVAE encoder maps material triplets to a two-dimensional latent code, and its decoder reconstructs E, ν, and ρ; the Geometry Transformer predicts these latents from sparse voxel features.The transformer uses DINOv2 features and is trained with a frozen MatVAE decoder.Source: §F.2 · p. 41

F.3 TRAINING

• Training voxelizes normalized multi-segment meshes into solid 1/64-resolution volumes, assigns each voxel its segment material, renders 150 views, and extracts DINOv2-ViT-L/14 features.Training views use quasi-random Hammersley camera positions; images are rendered at 512×512 pixels, and feature extraction resizes inputs to 518×518 pixels.Source: §F.3 · pp. 42–44

F.4 SIMULATION AND RENDERING

• Mesh simulations use libuipc FEM, while comparisons also use MPM and XPBD; large-scale splat simulations use Simplicits, with splats rendered through gsplat.The additional simulation details also cover FEM and Warp implementations, and Simplicits for splat-plus-mesh simulations.Source: §F.4 · p. 44, G ADDITIONAL DETAILS ON THE SIMULATIONS · pp. 44–45

F.5 BASELINES

• NeRF2Physics Shore A/D hardness predictions are converted to Young’s modulus using ASTM mappings, and baseline methods are evaluated on the same object points as VoMP.The conversions are EMPa = e^(SA×0.0235)−0.6403 and EMPa = e^((SD+50)×0.0235)−0.6403; Phys4DGen components were reproduced with GPT-4o and author-provided prompts.Source: §F.5 · p. 44

G.1 INTERPOLATION SCHEME

• Nearest-neighbor interpolation assigns voxel-field materials at arbitrary simulation locations, preserving sharp interfaces rather than blending across piecewise-constant object parts.The authors avoid higher-order interpolation because convex blending could create intermediate materials that are not physically present or admissible.Source: §G.1 · p. 45

G.2 PREPARING SCENES AND ASSIGNING MATERIALS FOR THE FEM SOLVER

• For heterogeneous FEM assignments, tetrahedra take Lamé parameters from the nearest voxel to each centroid, while vertices take density from the nearest voxel to their positions.Uniform assignments instead derive Lamé parameters from E and ν and use constant density.Source: §G.2 · p. 45

G.3 DETAILS OF THE FEM SOLVER

• The FEM solver models corotational Hookean elasticity and advances each frame by minimizing an incremental potential, solving the resulting residual with Newton iterations and backtracking line search.A related simulation variant uses compressible Neo-Hookean elasticity with volumetric regularization and IPC collision handling.Source: §G.3 · pp. 46–47

G.4 PREPARING SCENES AND ASSIGNING MATERIALS FOR THE SIMPLICITS SOLVER

• Simplicits embeds objects in a regular grid with a low-dimensional displacement subspace and assigns each quadrature point material parameters from its nearest voxel.Uniform quadrature weights make elastic and inertial energies invariant to sampling density; contact uses an IPC-style barrier with Coulomb friction.Source: §G.4 · pp. 47–48

G.5 DEFORMING SPLATS AND RENDERING DEFORMED SPLATS

• During splat deformation, the local deformation gradient transforms each splat’s rest anisotropy into a world-space covariance, while its mean follows the embedded deformation.Color and opacity remain fixed; the renderer receives updated means and covariances, with a small positive scalar protecting covariance positive-definiteness under extreme compression.Source: §G.5 · p. 48

G.6 DETAILS OF THE SIMPLICITS SOLVER

• Simplicitis solves displacement in a low-dimensional grid-and-handle subspace using Newton steps and preconditioned conjugate gradients, with distinct splat–mesh and splat–splat contact formulations.Splat–mesh contact uses triangle meshes as kinematic colliders, while splat–splat contact uses particle pairs.Source: §G.6 · pp. 48–49

G.7 PREPARING SCENES AND ASSIGNING MATERIALS FOR THE XPBD SOLVER

• XPBD maps Young’s modulus and Poisson ratio to distance- and volume-constraint compliances, then projects distance, volume, and collision constraints after Verlet prediction.Objects use particles initialized at rest positions; after projection, the method updates positions and applies ground-plane contact with Coulomb friction.Source: §G.7 · p. 49

G.8 PREPARING SCENES AND ASSIGNING MATERIALS FOR THE MPM SOLVER

• MPM assigns per-object E and ν, derives Lamé parameters, and uses a fast corotated elastic model on particles embedded in a uniform Cartesian grid.Each step transfers particle mass, momentum, and internal forces to the grid, applies gravity and boundary conditions, then interpolates velocities back to update particles and deformation.Source: §G.8 · pp. 49–50

H OTHER RELATED WORKS

• Related work infers physical properties from video or manipulation, generates physically plausible shapes, predicts displacements instead of properties, or studies articulation; VoMP targets adding properties to existing 3D assets.Shape-generation methods cannot augment existing assets with mechanical properties.Source: H OTHER RELATED WORKS · p. 50