Source-linked AI summary
Adaptive Volumetric Mechanical Property Fields Invariant to Resolution
Rishit Dagli, Donglai Xiang, Vismay Modi, Xuning Yang, Gavriel State, David I. W. Levin, Maria Shugrina
TL;DR
Most 3D assets lack the spatially varying mechanical properties needed for physics simulation. AdaVoMP uses sparse adaptive voxels and a transformer to predict these fields, achieving higher-resolution, memory-efficient estimates that improve over prior methods across all three properties.
Problem
Most 3D assets lack Young’s modulus, Poisson’s ratio, and density, limiting the information available for reliable physics simulation.
Method
AdaVoMP uses sparse adaptive voxel structures and a sparse transformer encoder-decoder to represent shapes and generate spatially varying material fields.
Results
AdaVoMP significantly improves state-of-the-art performance across Young’s modulus, Poisson’s ratio, and density, including at lower effective resolution.
Takeaways & Limitations
The method converts high-resolution 3D assets into simulation-ready, physically interactable entities while maintaining memory efficiency.
Takeaways & Limitations
AdaVoMP cannot currently model truly directional materials represented by spatially varying tensor-valued Young’s moduli.
Abstract
from arXiv · showhide
Accurate mechanical properties (or materials) Young's modulus ($E$), Poisson's ratio ($ν$) and density ($ρ$) are essential for reliable physics simulation of digital worlds, but most 3D assets lack this information. We propose AdaVoMP, a method for predicting accurate dense spatially-varying ($E$, $ν$, $ρ$) for input 3D objects across representations, improving the resolution, accuracy, and memory efficiency over the state-of-the-art. The foundation of our technique is a sparse and adaptive voxel structure SAV that efficiently represents both the input 3D shape and the material field output. We replace the fixed-voxel model of the most accurate prior method, VoMP, with a novel sparse transformer encoder-decoder model that learns to generate a unique SAV autoregressively for every input shape to represent its materials, achieving a resolution $16^3\times$ higher than prior art. Experiments show that AdaVoMP estimates more accurate volumetric properties, even with lesser test-time compute than all prior art. This allows us to convert high-resolution complex 3D objects into simulation-ready assets, resulting in realistic deformable simulations.
1. Introduction
AdaVoMP addresses the lack of spatially varying mechanical properties in typical 3D assets by predicting E, ν, and ρ with an adaptive representation. Its sparse transformer architecture generates material fields at substantially higher resolution while improving accuracy and memory efficiency over prior voxel-based methods.
- Motivation: Typical 3D scenes lack volumetric Young’s modulus (E), Poisson’s ratio (ν), and density (ρ), which must be spatially defined for accurate physics simulation.Manual assignment is difficult, while real-world measurement does not scale with demand for digital simulation.
- Contribution: AdaVoMP predicts accurate spatially varying E, ν, and ρ for 3D shapes using an adaptive structure that improves resolution, accuracy, and memory efficiency over prior art.It replaces VoMP’s fixed-voxel model with a sparse transformer encoder-decoder that generates a unique adaptive structure for each input shape.
- Resolution and adaptivity: 1024^3 is AdaVoMP’s maximum operating resolution, compared with 64^3 for prior art.The adaptive structure uses few voxels in constant-material regions and concentrates predictive capacity near difficult regions and sharp material boundaries.
- Method: SAV represents both the input shape and material field, enabling efficient querying and autoregressive generation across multi-level voxels.The method aggregates multi-view visual features into an adaptive structure and uses sparse windowed attention with unified coordinate embeddings.
- Training and generation: The generator produces SAVs coarse-to-fine using multi-scale supervision, teacher forcing, and explicit empty-space negative supervision.The architecture includes an Adaptive Geometry Transformer and a jointly trained sparse transformer generator.
2. Related Works
Prior work obtains material parameters through inverse physics or feed-forward inference, but faces overfitting, simulator dependence, limited datasets, surface-only modeling, computational cost, or fixed-resolution representations. AdaVoMP builds on VoMP by using sparse adaptive voxels for coarse-to-fine, higher-resolution volumetric prediction.
- Inverse Physics vs. Static Inference: Inverse physics methods optimize material parameters from video or priors but suffer from overfitting, simulator dependence, and poor scalability.Feed-forward methods instead learn from ground-truth material datasets and infer volumetric parameters from static scenes for rapid runtime inference.
- Mechanical Property Datasets: Learning-based volumetric property prediction is hindered by limited, noisy, simulator-overfit, coarsely annotated, or rigid-object datasets.These dataset limitations make prediction from shape and appearance alone difficult.
- Inferring Materials for Static Scenes: NeRF- and Gaussian-splat approaches focus on surface feature fields and cannot model internal volume, while VLM methods are computationally heavy and depend on external segmentation.The cited methods target static-scene material inference from visual or 3D data.
- Comparison to Feed-Forward Methods: AdaVoMP replaces VoMP’s fixed-resolution grids with a sparse adaptive voxel tree, enabling coarse-to-fine predictions that scale to significantly higher effective resolutions in complex regions.Pixie, like VoMP, operates on a fixed-resolution grid; adaptive feature voxel structures themselves are not new.
3. SAV: Sparse Adaptive Voxels
SAV is a sparse adaptive voxel representation for encoding 3D shapes and spatially varying material fields. It learns multiresolution structures that refine heterogeneous regions while keeping homogeneous areas coarse, enabling substantially higher resolution than prior fixed-voxel methods.
- Adaptive representation: SAV encodes both the input 3D shape and output spatially varying materials using adaptive voxel structures that allocate computation to material-relevant regions.Fine heterogeneous regions and boundaries are recursively refined, while less important regions receive less compute.
- Adaptive representation: Unlike octrees, OpenVDB, VoMP, and TRELLIS, SAV is learned autoregressively for material prediction and stores voxels across multiple resolution levels simultaneously.SAV refines voxels where material heterogeneity demands finer detail and represents homogeneous regions with single coarse voxels.
- SAV definition: SAV represents a feature field over a bounded 3D domain with an adaptive voxel tree whose leaf voxels form an axis-aligned partition and may occupy different resolution levels.Each voxel is indexed by a level and integer grid coordinate, with levels ranging from 0 to Lmax.
- Material representation: The material tree stores (E, ν, ρ) vectors and subdivides voxels only when within-voxel material variation exceeds a tolerance.Coarse-to-fine construction enables partially specified regions to return coarser averaged materials for level-by-level supervision.
4. Learning Adaptive Material Fields
AdaVoMP learns adaptive material fields by encoding sparse input-shape tokens with a geometry transformer and autoregressively generating a sparse material structure. The generator refines only candidate voxels from coarse to fine, producing effective 1024^3 resolution without a dense grid while allowing test-time compute scaling.
- Adaptive Material Generator: 1024^3 effective resolution is achieved without instantiating a dense grid.The generated material field is represented as an adaptive sparse voxel structure.
- Adaptive Geometry Transformer: The encoder E processes mixed-level leaf voxels from the input SAV, enriched with aggregated DINOv3 features, as sparse tokens.Each voxel feature has dimension d_in = 1280.
- Adaptive Geometry Transformer: E combines learned level embeddings and RoPE positional information with sparse 3D shifted-window self-attention and feed-forward layers.RoPE is applied to unified voxel coordinates inside self-attention.
- Adaptive Material Generator: The material generator G autoregressively constructs the output SAV coarse-to-fine over resolution levels, restricting computation to sparse refinement frontiers.This produces lower-resolution outputs by stopping after fewer generator iterations.
- Adaptive Material Generator: For each candidate voxel, G predicts EMPTY, KEEP, or SUBDIVIDE structure actions and a 2D latent material vector for non-empty voxels.EMPTY explicitly represents empty space, while KEEP voxels provide material latents for the next refinement level.
- Adaptive Material Generator: G is shared across levels and conditions each refinement candidate on its level, octant identity, and parent hidden state.Parent states preserve context because each candidate set contains only the current refinement frontier.
3D Assets Adaptive Encoding
AdaVoMP encodes input shapes as sparse adaptive voxel structures and autoregressively generates material fields through coarse-to-fine candidate refinement. Cross-attention, sparse windowed self-attention, and a frozen MatVAE decoder support context-aware structure decisions and physically plausible material properties.
- Method overview: The method encodes each input shape as an SAV, processes it with an autoregressive Adaptive Material Generator, and outputs the material field as an SAV.The overview links shape encoding, generator processing, and material-field output within the SAV representation.
- Candidate encoding: Each candidate combines unified sparse coordinates with cross-attention to input latents and sparse windowed self-attention with RoPE before predicting structure actions and 2D material latents.Feed-forward heads predict both the structure action and material latent for every candidate.
- Coarse-to-fine decoding: SUBDIVIDE candidates generate eight children, EMPTY candidates are discarded, and KEEP voxels become leaves while hidden states preserve broader spatial context.At the finest level ℓ=0, refinement terminates and all non-empty voxels are leaves.
- Material decoding: The frozen MatVAE decoder maps predicted 2D voxel latents to E, ν, and ρ, supporting physically plausible generated properties.The predicted latent lies in MatVAE’s latent space and is mapped to the material-property triplet.
- Training: Training uses teacher forcing to jointly supervise structure decisions and node materials across the breadth-first refinement schedule.The ground-truth structure replaces predicted subdivision decisions during training, and losses are computed across all levels.
5. Experiments and Results
AdaVoMP significantly improves mechanical-property accuracy and resolution over prior methods, including at lower effective resolution, while producing compact adaptive material trees. Qualitative and end-to-end results further show faithful predictions and realistic simulations on complex objects.
- Quantitative evaluation: AdaVoMP significantly outperforms VoMP, Pixie, NeRF2Physics, PUGS, and Phys4DGen across metrics for Young’s modulus, Poisson’s ratio, and density.It remains better than prior art when evaluated at a lower effective resolution of 64^3.
- Qualitative evaluation: Qualitatively, AdaVoMP avoids the noisy, mislabeled, overly soft, and low-resolution predictions observed in competing methods, including VoMP’s missed object regions.The method also supports high-fidelity end-to-end simulations of complex objects.
- Additional benchmarks: AdaVoMP achieves on-par or better material validity than VoMP and improves mass estimation on the ABO benchmark.Material validity measures whether predicted materials fall within physically measured material ranges.
- Ablation and scaling: The B+ model outperforms VoMP despite having similar parameter count, showing that experimental gains are not solely attributable to model size.The model family spans SMALL, BASE, BASE+, LARGE, LARGE+, and HUGE, scaling to 0.6B parameters.
- Representation compactness: 7.24%: ground-truth material trees use this fraction of the occupied voxels required by dense 64^3 voxelization on GVT-TEST.Generated trees require 9.14% of dense 64^3 occupied voxels, combining 7.24% ground-truth compactness with their 26.14% leaf-node overhead.
6. Discussion
AdaVoMP’s accuracy and generalization are expected to improve with more training data, while its high-resolution predictions enable multiscale approximation of anisotropic materials. However, it cannot represent truly directional materials with spatially varying tensor-valued Young’s moduli, and future work may extend predictions beyond linear elasticity.
- Limitations and future work: More available training data should improve the model’s accuracy and generalization.The method is data-driven, so its performance depends on training-data availability.
- Limitations and future work: High-resolution mechanical-property prediction enables approximating anisotropic materials through multiscale modeling.This capability does not extend to truly directional materials with spatially varying tensor fields of Young’s moduli.
- Limitations and future work: Future extensions could predict yield strength, shear modulus, and thermal expansion beyond linear elasticity.
7. Conclusion
AdaVoMP predicts mechanical property fields for 3D assets at 163× higher resolution than prior works while maintaining memory efficiency. It transforms visual surface appearance into volumetric, physically interactable entities without manual parameter tuning.
- Conclusion: 163× higher resolution enables AdaVoMP to predict mechanical property fields for 3D assets while maintaining memory efficiency.This improves on prior works’ resolution without sacrificing memory efficiency.
- Conclusion: Surface-level visual appearance transforms 3D assets into volumetric, physically interactable entities.The approach obviates manual parameter tuning, identified as a bottleneck to realistic simulation at scale.
- Conclusion: The authors position AdaVoMP as a potential foundational block for physical AI and scalable simulation-generation pipelines.This stated goal follows the method’s conversion of 3D assets into simulation-ready volumetric entities.
Impact Statement
AdaVoMP could reduce the cost of building simulation-ready digital assets by learning physically plausible, spatially varying materials from geometric and visual cues. However, its estimates may be misused or fail under conditions such as distribution shift, so they should not replace measurement, testing, or certified engineering analysis.
- Potential positive impact: The method could reduce simulation-ready asset construction costs by providing a learned prior over physically plausible, spatially varying materials for simulation and interactive scene generation.It performs conditional generation of volumetric mechanical property fields from geometric and visual cues.
- Risks: Like many models, it could be misused to create realistic digital content or deepfakes.
- Risks: Predicted properties should not substitute for measurement, testing, or certified engineering analysis in safety-critical decisions.Outputs can be wrong under distribution shift, partial observability, or atypical materials, potentially causing unsafe designs or incorrect conclusions.
Supplementary Material for Adaptive Volumetric Mechanical … B.4. Core Operators
The supplementary material reports scaling, simulation, additional evaluation, and implementation details for AdaVoMP, centered on the sparse adaptive voxel representation and its core operators. It shows that larger models and higher test-time resolution improve accuracy while preserving efficient, resolution-flexible material-field queries.
- A.1. Scaling Experiments: Scaling experiments vary model size, training tokens, test-time resolution, memory, generated SAV dimensionality, and computational cost.The supplementary figures separately examine scaling of model, training, and test-time compute, memory, SAV dimensionality, and computation.
- A.2. End-to-end Examples with Simulation: AdaVoMP is evaluated on meshes and 3D Gaussian Splats, producing realistic physics simulations without hand-tuned material parameters.The simulations include elastodynamic scenes with stable objects under gravity and realistic robot–object interactions.
- A.4. Additional Mechanical Property Fields: Additional material-field visualizations and baseline comparisons are provided in Figures 12 to 19.Figures 12 to 14 show additional predicted mechanical property fields, while Figures 15 to 19 add comparisons with baseline methods.
- B. SAV: Our Sparse Adaptive Volumetric Voxels Backend; B.1. Representation: SAV is designed as a sparse, adaptive multi-resolution backend that supports high-resolution material prediction while remaining suitable for generation targets and conditioning inputs.The representation stores sparse voxel subsets across levels, and partial trees fall back to coarser region-average features when fine voxels are absent.
- A.1. Scaling Experiments; B.1. Representation: Larger models improve accuracy across (E, ν, ρ), while scaling test-time compute from 643 to 10243 is more effective for larger models.Figure 6 also reports lower displacement errors for ν and E with larger models at fixed training budgets and higher-resolution inference.
- A.3. Additional Mechanical Property Prediction Results; B.1. Representation: Voxel-averaged supplementary evaluations compare regular and hard datasets, with AdaVoMP outperforming baselines on all reported metrics and remaining close to object-averaged performance on GVT-HARD.The voxel metrics are averaged across all voxels; most baselines degrade substantially under voxel averaging, whereas AdaVoMP’s aggregation gap remains small.
- B.2. Baking DINO Features into SAV: Input conditioning features are reconstructed from multi-view DINOv3 tokens using inverse-depth weighting, then merged into feature-homogeneous SAV cells and recursively subdivided when non-uniform.Uniform cells become pooled leaves under a feature threshold, while non-uniform cells are refined over occupied children.
- B.3. Sparse Tensor Backend; B.4. Core Operators: The sparse tensor backend uses batched per-level coordinates and features, spatial hashing with O(log Nℓ) lookup, contiguous batching, cached coordinate metadata, and four lossless core operators.The operators construct material and conditioning trees, query finest available values, and flatten trees into encoder tokens; serialization preserves Equation (15).
C. Ablations · C.1. Material Gaussian Splats · D. Metrics
The ablations examine adaptive geometry, material-generation components, and a Gaussian-splat alternative, while the metrics section retains VoMP’s evaluation metrics. Material Gaussian Splats provide continuous sub-voxel fields but yield only modest gains and scale poorly to larger models.
- C. Ablations: The ablations evaluate the Adaptive Geometry Transformer and Adaptive Material Generator by removing or changing initialization, query embeddings, supervision, and material parameterization under matched compute budgets.All ablations use the B scale, with hyperparameters tuned within an identical compute budget; the Material Gaussian Splats comparison is not directly comparable to the baseline because training and architecture differ.
- C. Ablations: d ≈2.48 is the measured effective dimensionality of the sparse adaptive volumetric geometry, between surface scaling d = 2 and dense volumetric scaling d = 3.The generated geometry can sometimes represent a volume more efficiently than the same object’s surface in a dense voxel grid.
- C.1. Material Gaussian Splats: Material Gaussian Splats replace each occupied voxel’s single material latent with 32 anisotropic 3D Gaussians carrying MatVAE latent codes.The fixed-grid representation voxelizes objects at 64^3 and processes up to 32,768 occupied voxels per object, using stochastic subsampling when necessary.
- C.1. Material Gaussian Splats: Material Gaussian Splats produce a continuous, locally smooth material field within each voxel by density-weighting decoded MatVAE latents.The querying operation evaluates Gaussian densities at a 3D point, normalizes their contributions, decodes each latent, and blends the resulting material triplets.
- C.1. Material Gaussian Splats: Only modest gains over the voxel baseline are obtained by Gaussian refinement, while per-voxel parameterization increases.The ablation follows VoMP’s supervised fixed-grid recipe, changing only the single per-voxel latent into Gaussian-supported latents.
- C.1. Material Gaussian Splats: 1.4× more TFLOPs per GB is achieved by a model larger than H versus smaller models at mid-compute budgets in the 128^3 regime.At high-compute budgets in the 1024^3 regime, scaling from S to larger than H yields 43× more FLOPs for only 20× more memory.
- C.1. Material Gaussian Splats: The Material Gaussian Splats model outperforms at B size but cannot scale well enough to match the performance of higher model sizes.The comparison uses the same compute budget as other baselines as a fairness proxy, but is imperfect because this model uses only Data Parallelism.
- D. Metrics: The metrics section uses the same metrics as VoMP.The section introduces an explanation of the metrics used for evaluation.
D.1. Metrics for Mass and Field Estimation … E.3. Feature Adaptive Tree for Training and Inference
The appendix defines metrics for scalar, field, and mechanical-property estimation, then specifies GVT dataset construction, material supervision, and adaptive feature-tree preprocessing. These choices support voxel-level material targets and view-lifted features at G = 1024.
- D.1. Metrics for Mass and Field Estimation: Ablations evaluate initialization, query embeddings, supervision, material parameterization, and Material Gaussian Splats at 1024^3 and B scale.The compared variants include scratch or VoMP initialization, level and octant embeddings, empty-space or leaf-only supervision, and MatVAE removal.
- D.2. Metrics to Measure Differences in Mechanical Properties: Mechanical-property estimates are evaluated at queried interior points using relative and displacement errors for log(E), ν, and ρ.Young’s modulus is assessed in logarithmic space, while Poisson’s ratio and density use linear-space measures; units include Pa and kg/m3.
- E. Dataset Details: GVT is built from simulation-ready, textured USD assets pooled from multiple collections, with stable SHA-256 instance identifiers and source semantic labels.The dataset construction choices determine supervision targets and conditioning signals used in the experiments.
- E.1. Voxelizing for Training: Assets are solid-voxelized at G = 1024, representing occupied finest-grid indices throughout object interiors rather than only surfaces.Geometry is normalized to the paper’s coordinate convention, and the number of generated voxels is capped for robustness.
- E.1. Voxelizing for Training: Each occupied voxel receives a material triplet (E, ν, ρ), inherited from its object part, with E in Pa, ν unitless, and ρ in kg/m3.Each part is assigned a constant triplet for supervision.
- E.2. Material Adaptive Tree for Training: The material supervision tree refines the G = 1024 field by value ranges, averaging colliding samples and storing descendant means at coarser nodes.Refinement occurs when descendant ranges exceed per-channel merge tolerances.
E.4. Dataset Statistics · F. Additional Details on Training · F.1. Network Design
After preprocessing at G = 1024, GVT contains 1,725 high-quality objects with substantial occupancy and token-count variability. AdaVoMP’s architecture combines sparse-coordinate AGT encoding with a shared-weight, coarse-to-fine AMG generator and memory-conscious implementation choices.
- E.4. Dataset Statistics: GVT contains 1,725 high-quality objects spanning four source collections, dominated by SimReady (59.7%) and Residential (29.3%) assets.The dataset contains 55 semantic classes, although the supplied passage truncates the remaining statistics.
- E.4. Dataset Statistics: 22.5M occupied voxels is the median per object, with the 5th–95th percentile spanning 0.88M–123.0M at G = 1024.This heavy-tailed occupancy motivates adaptive processing for volumetric inputs.
- F. Additional Details on Training: At G = 1024, objects require ≈86.7k input tokens and up to 398,112 output tokens per object for training.The passage identifies these counts as the tokens used for training.
- E.4. Dataset Statistics: Adaptive feature trees average 86.7k nodes per object, motivating adaptive discretization and candidate-only computation.Material and feature-tree node distributions are summarized across levels, while material-property statistics are computed by averaging each property over nodes.
- F.1. Network Design: AGT is a sparse 3D Transformer on unified coordinates using pre-norm residual blocks, sparse shifted-window attention, and RoPE.The supplied architectural description identifies AGT’s core sparse-coordinate design and residual block structure.
- F.1. Network Design: AGT uses width dmodel = 768 with 12 heads, head dimension 64, per-head Q/K RMS normalization, and MLP ratio 4 with GELU activations.The attention uses shifted-window sparse 3D attention with RoPE on unified coordinates, and AGT is initialized from a pretrained TRELLIS encoder.
- F.1. Network Design: AMG applies candidate-to-latent cross-attention followed by candidate self-attention, shares block weights across refinement levels, and scales compute linearly with levels.Weight sharing makes parameter count independent of Lmax; AMG predicts a 2D latent decoded by the frozen MatVAE decoder from VoMP.
- F.1. Network Design: LayerNorm runs in float32, while token-wise operators are processed in exact chunks to bound peak memory without changing outputs.Shifted-window attention and RoPE depend only on discrete sparse coordinates, according to the supplied implementation passage.
F.2. Training Recipe … ADAVOMP VoMP
AdaVoMP uses normalized multi-property training, adaptive distributed optimization, and simulation-specific interpolation and evaluation pipelines. The supplementary material also documents baseline conversion and sampling procedures, related algorithms, and qualitative field comparisons with VoMP.
- F.2. Training Recipe: AdaVoMP normalizes material targets with log-minmax transforms for Young’s modulus and density, minmax normalization for the remaining property, and MatVAE-consistent bounds.A small ϵ avoids log(0).
- F.2. Training Recipe: Training uses AdamW with linear warmup and decay, global gradient clipping, exponential moving averages, and bfloat16 mixed precision.The learning rate ramps from ηpeak/Tw to ηpeak, then decays to ηend = 0.01 ηpeak by step T.
- F.3. Distributed Training: HSDP combines ZeRO-3/FSDP-2 sharding with DDP, while dummy candidates and masked dummy levels preserve identical collective schedules for adaptively terminating trees.Gradient norms are aggregated across ranks before local shard scaling, and candidate capping uses contiguous spatial windows.
- G.3. Baselines: For fair baseline comparisons, NeRF2Physics Shore hardness is converted to average Young’s modulus, while point- or voxel-based methods are evaluated on matched object points.Pixie and VoMP use fixed 64^3 voxel grids, with properties interpolated for evaluation where applicable.
- H. Additional Details on the Simulations: FEM and Simplicits simulations follow VoMP’s setup, converting predicted (E, ν, ρ) into Lamé parameters and using identical solver and preprocessing details.All simulations run on an RTX A6000 with 48 GB.
- H.1. Material Interpolation Scheme For Simulation: Material values returned on AdaVoMP’s sparse voxel grid are transferred to arbitrary simulator query locations using nearest-neighbor interpolation.Query locations include mesh vertices and Monte Carlo sampled cubature points.
- H.3. Evaluating on IsaacSim: Isaac Sim evaluations use NVIDIA Omniverse’s PhysX engine with a Temporal Gauss-Seidel solver and a 120 Hz simulation rate.The supplementary material lists the FEM and PhysX hyperparameters in Tables 18 and 19.
- ADAVOMP VoMP: Additional material-field figures compare AdaVoMP and VoMP across dartboard, foosball, Lombardy poplar, Phineas, and shield-controller objects.The related-work appendix also describes adaptive and coarse-to-fine strategies and provides algorithms for material-tree construction, feature-tree refinement, point queries, decoding, candidate capping, and teacher-forced loss evaluation.