Source-linked AI summary
MeshWeaver: Sparse-Voxel-Guided Surface Weaving for Autoregressive Mesh Generation
Jiale Xu, Wang Zhao, Ying Shan
TL;DR
Existing autoregressive mesh generators use inefficient coordinate-level tokenization and lack fine-grained geometric guidance. MeshWeaver predicts vertices with hierarchical sparse-voxel guidance, achieving state-of-the-art compression and geometric fidelity while scaling to meshes with up to 16K faces.
Problem
Coordinate-level mesh tokenization remains limited to about 22% compression, leaving more compact yet faithful representations an open challenge.
Method
MeshWeaver predicts next vertices as coarse-to-fine atomic tokens and uses hierarchical sparse-voxel features, cross-attention, and an inference scaffold for geometric guidance.
Results
MeshWeaver achieves state-of-the-art compression and geometric fidelity while generating meshes with up to 16K faces.
Takeaways & Limitations
MeshWeaver provides a surface-weaving formulation that directly produces structured polygonal meshes while coupling structural coherence with fine geometric detail.
Takeaways & Limitations
The method uses explicitly inserted BOS tokens rather than a potentially shorter separate-token design for patch-center and other vertices.
Abstract
from arXiv · showhide
Autoregressive mesh generation has gained attention by tokenizing meshes into sequences and training models in a language-modeling fashion. However, existing approaches suffer from two fundamental limitations: (i) low tokenization efficiency, which yields long token sequences and prevents scaling to high-poly meshes, and (ii) absence of geometry-aware guidance, as generation is conditioned only on global shape embeddings rather than local surface cues. We introduce MeshWeaver, an autoregressive framework that treats mesh generation as a surface weaving process by directly predicting the next vertex instead of independent coordinates. At its core is a multi-level sparse-voxel encoder that injects geometric context into the generative process in three complementary ways: providing voxel features as vertex representations, guiding token prediction via cross-attention to voxel features, and serving as a structural scaffold that constrains generation around the input surface. Our hierarchical design enables coarse-to-fine vertex prediction in a single decoding step, while tightly coupling the generative model with 3D geometry. Extensive experiments demonstrate that MeshWeaver achieves a state-of-the-art compression ratio of 18%, can generate meshes with up to 16K faces, and significantly improves geometric fidelity over prior approaches.
1. Introduction
MeshWeaver addresses the long-sequence and limited-scalability challenges of autoregressive mesh generation by reframing it as surface weaving under known geometry. It predicts vertices directly with hierarchical sparse-voxel guidance, enabling coarse-to-fine generation while focusing computation on structural reasoning.
- Motivation: Implicit representations simplify learning but often yield overly dense, topologically complex meshes that hinder downstream processing.The passage identifies mesh density and topological complexity as key drawbacks of Marching Cubes-based extraction.
- Related work: MeshGPT and MeshXL established autoregressive mesh generation through face tokenization, but long sequences limited scalability to high-poly meshes.These early methods modeled discrete coordinate sequences with transformers.
- Novelty: MeshWeaver formulates autoregressive mesh generation as surface weaving, analogous to re-topology under known geometry rather than conditional shape generation.This perspective distinguishes MeshWeaver from prior methods that use geometric conditions predominantly for conditional shape generation.
- Contribution: MeshWeaver directly predicts vertices as atomic tokens instead of independent coordinates, using multi-level coarse-to-fine prediction within a single decoding step.The design reduces sequence length and lets the transformer prioritize structural reasoning over redundant coordinate generation.
2. Related Work
Prior 3D generation methods trade efficiency, detail, local geometric fidelity, and training cost, while learning-based re-topology remains difficult. Autoregressive mesh generation addresses this by modeling ordered vertex sequences before face connectivity.
- 3D Generation: Early 3D generation methods adapted 2D models through optimization but were inefficient and produced impractical results.
- 3D Generation: Recent methods adopt a VAE + latent diffusion paradigm, with VecSet representations offering compact transferable shape sets but limited fine-grained detail.
- 3D Generation: Sparse-voxel methods capture local geometry more faithfully but require heavier training, while learning-based re-topology still struggles to balance fidelity, compactness, and workflow readiness.
- Autoregressive Mesh Generation: PolyGen pioneered autoregressive mesh generation by producing ordered vertex sequences and then connecting them into faces with two autoregressive transformers.
3. Method
MeshWeaver reformulates autoregressive mesh generation as vertex-level surface weaving, using hierarchical voxel indices for coarse-to-fine prediction. A sparse-voxel encoder further supplies geometry-aware vertex features, guides token prediction, and anchors generation near occupied surface regions.
- Vertex-Level Tokenization: Vertex-level tokenization predicts a complete vertex per decoding step, replacing coordinate-level prediction and yielding a 2D vertex sequence.The approach interprets mesh traversal as weaving the surface vertex by vertex.
- Multi-Level Vertex Representation: Hierarchical voxel indexing predicts each vertex coarse-to-fine across L levels, progressively narrowing from a coarse voxel to finer subvolumes.Each finer voxel is conditioned on its parent, and the finest resolution equals the coordinate quantization resolution.
- Tokenization Results: 18% compression ratio establishes a new state of the art for the patch-based, multi-level vertex-token representation.Patch centers receive BOS tokens, and EOS terminates the full sequence.
- Sparse-Voxel Encoder: The sparse-voxel encoder provides hierarchical surface features that replace static vertex embeddings with shape-dependent, local geometry-aware representations.PointNet aggregates points within non-empty voxels, followed by shifted-window sparse attention and multi-scale sparse-convolutional downsampling.
- Cross-Attention-Guided Token Prediction: Cross-attention guides each hierarchical voxel prediction using corresponding sparse-voxel features, restricting finer-level attention to the previously localized subvolume.This reduces computation while preserving spatial precision.
- Sparse Voxels as Generation Scaffold: Masking empty voxels makes sparse voxels an explicit generation scaffold that constrains predicted vertices toward occupied surface regions.This addresses the risk of drifting into empty space associated with implicit shape embeddings.
4. Experiments
Experiments evaluate MeshWeaver’s implementation, point-cloud-conditioned generation, tokenization efficiency, and sparse-voxel design. Results show efficient vertex-level tokenization, improved handling of complex geometry, and complementary benefits from voxel representations, cross-attention, and inference-time scaffolding.
- Implementation Details: The model is trained on 800K meshes with 1K–16K faces using a 24-layer, 600M-parameter LLaMA3-style transformer and 7-bit coordinate quantization.The corpus merges Objaverse++, ShapeNet, 3D-Future, HSSD, and ABO, with random scale and rotation augmentations.
- Point-Cloud-Conditioned Generation: MeshWeaver faithfully reproduces intricate details, whereas baselines exhibit error accumulation, surface drift, and difficulty capturing complex local structures.MeshAnythingV2 and EdgeRunner are further limited by training on meshes with fewer than 4K faces because of tokenization inefficiency.
- Mesh Tokenization Efficiency: 18% compression ratio is achieved by vertex-level tokenization, compared with about 22% for existing coordinate-level tokenization algorithms.The compression ratio is computed as L/(9N), where lower values indicate better efficiency.
- Sparse-Voxel Encoder: Removing either voxel features as vertex representations or cross-attention causes a substantial performance drop, while removing both causes the most severe degradation.These components provide complementary geometric priors and token-prediction guidance; generation scaffolding is used only during inference.
- Sparse-Voxel Encoder: Disabling generation scaffolding at inference causes a moderate but consistent decline, confirming its role in constraining generation around the input surface and mitigating error accumulation and surface drift.The result supports the surface-weaving paradigm.
- Level Partition: The (16, 8) and (8, 16) space partitions achieve comparable performance, while (8, 4, 4) performs a little worse; the model therefore adopts (16, 8).The deeper hierarchy reduces later levels’ spatial support, limiting the effective range of local geometry injected by sparse-voxel features.
5. Conclusion
MeshWeaver frames autoregressive mesh generation as sparse-voxel-guided surface weaving, predicting the next vertex while coupling decoding with a hierarchical sparse-voxel encoder. This design shortens sequences, strengthens structural reasoning, provides fine-grained geometric guidance, and supports state-of-the-art compression and geometric fidelity.
- 5. Conclusion: MeshWeaver casts mesh generation as a sparse-voxel-guided surface weaving process.The framework is autoregressive and uses sparse voxels to guide generation.
- 5. Conclusion: Predicting the next vertex rather than the next coordinate enables shorter sequences and stronger structural reasoning.The conclusion attributes these benefits to vertex-level prediction.
- 5. Conclusion: Coupling vertex decoding with a hierarchical sparse-voxel encoder provides more fine-grained geometric guidance.The encoder is integrated directly into the vertex-decoding process.
- 5. Conclusion: The design achieves state-of-the-art compression and geometric fidelity while scaling effectively to meshes with up to 16K faces.The passage summarizes compression, fidelity, and scalability as outcomes of the overall design.