PaperlayerView exact revision on arXiv

Source-linked AI summary

MeshAnything V2: Artist-Created Mesh Generation With Adjacent Mesh Tokenization

Yiwen Chen, Yikai Wang, Yihao Luo, Zhengyi Wang, Zilong Chen, Jun Zhu, Chi Zhang, Guosheng Lin

arXiv:2408.02555v3cs.CVcs.AIcs.GR

TL;DR

Autoregressive mesh-generation methods struggle with complex meshes, in part because existing tokenizations are inefficient. MeshAnything V2 introduces AMT, which uses one vertex per face when possible. The paper reports shorter sequences, a doubled maximum face count, and improved performance and efficiency; it also notes that accuracy remains insufficient for industrial applications.

  • Problem

    Existing mesh-generation methods struggle with large meshes because representing every face with three vertices creates redundant, resource-intensive sequences.

    Source: §1 · p. 1

  • Method

    MeshAnything V2 uses Adjacent Mesh Tokenization (AMT), which encodes adjacent faces with one additional vertex and inserts a restart token when adjacency is unavailable.

    Source: §1 · p. 3, §3.1 · p. 5

  • Results

    AMT halves token sequence length on average, and MeshAnything V2 doubles the maximum face count while improving generation performance and efficiency.

    Source: §1 · p. 3, §5 · p. 9

  • Takeaways & Limitations

    The compact, well-structured AMT sequence is reported to enhance sequence learning and improve the efficiency and performance of mesh generation.

    Source: §5 · p. 9

    MeshAnything V2’s accuracy remains insufficient for industrial applications, and the authors call for further work on stability and accuracy.

    Source: B. Limitations · p. 11

Abstract

from arXiv · show

Meshes are the de facto 3D representation in the industry but are labor-intensive to produce. Recently, a line of research has focused on autoregressively generating meshes. This approach processes meshes into a sequence composed of vertices and then generates them vertex by vertex, similar to how a language model generates text. These methods have achieved some success but still struggle to generate complex meshes. One primary reason for this limitation is their inefficient tokenization methods. To address this issue, we introduce MeshAnything V2, an advanced mesh generation model designed to create Artist-Created Meshes that align precisely with specified shapes. A key innovation behind MeshAnything V2 is our novel Adjacent Mesh Tokenization (AMT) method. Unlike traditional approaches that represent each face using three vertices, AMT optimizes this by employing a single vertex wherever feasible, effectively reducing the token sequence length by about half on average. This not only streamlines the tokenization process but also results in more compact and well-structured sequences, enhancing the efficiency of mesh generation. With these improvements, MeshAnything V2 effectively doubles the face limit compared to previous models, delivering superior performance without increasing computational costs. We will make our code and models publicly available. Project Page: https://buaacyw.github.io/meshanything-v2/

1. Introduction

Manual mesh creation is time-consuming, and existing autoregressive methods struggle with large meshes because their tokenizations are redundant and resource-intensive. MeshAnything V2 introduces Adjacent Mesh Tokenization (AMT) and supporting controls to improve mesh-generation efficiency and performance.

1. Introduction

  • Existing methods struggle to generate meshes with many faces because three-vertex face encoding creates nine tokens per face, increasing resource demands and sequence redundancy.The paper says this redundancy harms sequence learning and reduces performance.Source: §1 · p. 1
  • AMT represents each face with one vertex when possible, encodes adjacent faces with one additional vertex, and inserts '&' to restart when no adjacent face is available.The authors frame compactness and sequence regularity as important for effective mesh generation.Source: §1 · pp. 2–3
  • Face-count conditioning lets users specify an approximate generated face count, while masking invalid predictions prevents invalid tokens such as consecutive '&' tokens.The paper presents these as controls for application requirements and inference robustness.Source: §1 · p. 3
  • Up to 1,600 faces, MeshAnything V2 doubles prior models’ face limit; AMT halves sequence length and cuts attention-block compute and memory nearly fourfold.The paper also reports significantly better accuracy and efficiency.Source: Figure 1 · p. 2, §1 · p. 3

2. Related Works

Artist-created mesh generation treats meshes as ordered face sequences, while other 3D-generation approaches produce assets through feed-forward models or diffusion. The paper positions AMT as a more compact, structured tokenization for mesh generation.

2.1. Artist-Created Mesh Generation

  • AMT uses one vertex per face when possible, rather than the three used by prior methods, producing a more compact and well-structured representation.When adjacency is unavailable, it inserts '&' and restarts; the reported average sequence-length reduction is about half.Source: §2.1 · p. 4, Figure 2 · p. 4

2.2. 3D Generation

  • Artist-created mesh generation focuses on sequence learning to produce high-quality mesh topology, unlike other 3D-generation methods.The cited overview also describes GANs, feed-forward transformer methods, and diffusion approaches to 3D assets.Source: §2.2 · p. 4

3. Method

MeshAnything V2 introduces Adjacent Mesh Tokenization (AMT), which shortens mesh sequences, and integrates it into shape-conditioned generation with a doubled face limit and no increase in parameters.

3.1. Adjacent Mesh Tokenization

  • AMT reduces the token sequence length by approximately half for the same mesh, lowering mesh-generation computational overhead.Source: §3 · p. 5
  • AMT represents each face with one vertex when possible, encoding adjacent faces with one additional vertex and using & to restart when none is available.The method is described for triangle meshes and can be generalized to variable polygons.Source: §3.1 · p. 5
  • On the Objaverse test set, AMT reduces SeqV length by half on average; ideally it approaches one-third, but disconnected faces can make it perform worse than previous methods.The authors note that artist-created meshes generally have well-structured topologies.Source: §3.1 · p. 5
  • The $ swap token lets AMT form the next face from the previous face’s first and last vertices, reducing interruptions without noticeable impact in experiments.It broadens the adjacent-face search, although the added token could potentially make sequence learning harder.Source: §3.1 · p. 5
  • AMT follows sorted face order when choices arise, producing a unique sequence and prioritizing adjacent faces rather than potentially distant vertices.The discussion reports advantages over previous methods in speed, memory usage, and accuracy.Source: §3.1 · p. 6

3.2. MeshAnything V2

  • MeshAnything V2 raises the maximum generated face count from 800 to 1600 without increasing parameters and generates artist-created meshes conditioned on shape.It uses point clouds as conditions and a decoder-only transformer; AMT supplies compact mesh sequences, with discretized coordinates and a token for & forming transformer input.Source: §3.2 · p. 6
  • V2 adds distinct embeddings for three-vertex faces, one-vertex faces, and &, and explores a face-count condition for approximate count control.The face-count embedding is placed after the point-cloud prefix; training varies the count and drops the condition with 10% probability.Source: §3.2 · p. 6

4. Experiments

The experiments evaluate MeshAnything V2’s implementation, shape-conditioned outputs, and tokenization choices. They report better performance and efficiency with AMT, while highlighting the importance of sequence regularity alongside compression.

4.1. Implementation Details

  • The main setup uses OPT-350M [32], a pretrained point encoder [33], and a dataset of 230K point-cloud/mesh pairs with fewer than 1600 faces.The evaluation set contains 4K samples; point clouds use 8192 points, and training uses 32 A800 GPUs for four days.Source: §4.1 · p. 7

4.2. Qualitative Experiments

  • V2 generates high-quality Artist-Created Meshes aligned to given shapes and is reported to work in 3D asset production pipelines.The authors describe the generated meshes as highly controllable.Source: §4.2 · p. 7

4.3. Quantitative Experiments

  • Nearly twice the GPU hours were used by the no-AMT variant despite equal training iterations, while V2 significantly outperformed it.The comparison replaces AMT with naive mesh tokenization under otherwise identical settings.Source: §4.3 · p. 8
  • Unsort versus Baseline supports the importance of tokenization regularity for mesh generation.The Unsort method uses meshes in their stored order rather than sorting them.Source: §4.3 · p. 8
  • PolyGen(AMT) [15] achieves the best sequence-length compression ratio, but has a noticeable mesh-quality gap against Baseline and AMT.The authors suggest that difficulty interacting with previously generated vertices may hinder sequence learning.Source: §4.3 · p. 8
  • AMT shortens token sequences without compromising mesh quality, and AMT(Swap) further increases compression without affecting quality.The authors conclude that AMT and AMT(Swap) retain sequence regularity better suited to learning than PolyGen [15].Source: §4.3 · p. 8

5. Conclusion

MeshAnything V2 is a shape-conditioned Artist-Created Mesh generation model. Its AMT tokenization uses one vertex whenever possible, shortening sequences while improving sequence learning, efficiency, and generation performance.

5. Conclusion

  • MeshAnything V2 generates Artist-Created Meshes aligned to given shapes and significantly outperforms MeshAnything in performance and efficiency.Source: §5 · p. 8
  • AMT uses a single vertex whenever possible and reduces token sequence length by half on average, producing compact, well-structured sequences.The authors report that this improves sequence learning and the efficiency and performance of Artist-Created Mesh generation.Source: §5 · p. 9

A. Additional Experiments

The appendix points to additional quantitative experiments and a detailed discussion of PolyGen tokenization.

A. Additional Experiments

A.1. Experiments on Face Count Condition.

The face-count condition lets users request approximate mesh face counts, but mesh quality takes priority when the requested count becomes difficult to meet.

A.1. Experiments on Face Count Condition.

  • At a scale ratio of 0.8, the face-count condition significantly reduces face count while maintaining mesh quality.At 0.6, face count does not decrease further; the model prioritizes mesh quality when the condition is difficult to meet, with similar behavior above 1.Source: §A.1 · p. 11

A.2. User Study.

A user study compared MeshAnything V2 with MeshAnything on 30 meshes, with 43 users preferring V2 more often.

A.2. User Study.

  • Users compared MeshAnything and MeshAnything V2 on 30 randomly sampled evaluation meshes with fewer than 800 faces.Each model received the paired point clouds, and users selected their preferred result.Source: §A.2 · p. 11
  • 43 users participated in the study.Source: §A.2 · p. 11
  • 67.8% of votes favored MeshAnything V2, versus 32.2% for MeshAnything.The authors report that V2's results were more aligned with human preference.Source: §A.2 · p. 11

A.3. Discussion on PolyGen Tokenization Method.

PolyGen [15] first generates vertex coordinates, then uses vertex indices to assemble faces; the paper discusses its sequence-length trade-off and combines its face-generation stage with AMT.

A.3. Discussion on PolyGen Tokenization Method.

  • PolyGen [15] generates vertex coordinates autoregressively, then feeds them as a prefix to a second autoregressive model that connects them into faces.Source: §A.3 · p. 11
  • For a vertex referenced n times, PolyGen uses 3 + n tokens versus 3 × n for other methods, saving tokens when vertices are referenced multiple times.Source: §A.3 · p. 11
  • The face-count condition controls the number of generated faces using a scale ratio, with 1.0 denoting the ground-truth face count as the condition.Source: Table 3 · p. 11
  • PolyGen's requirement to predict vertex positions accurately first may be unsuitable for an autoregressive model.Source: §A.3 · p. 11
  • The experiments merge PolyGen's two generation stages into one model and combine its face-generation stage with AMT to shorten the token sequence further.Source: §A.3 · p. 11

B. Limitations.

Despite a large improvement over V1, MeshAnything V2's accuracy remains insufficient for industrial applications; improving stability and accuracy requires further effort.

B. Limitations.

Related papers

Loading 2408.02555v3…