TreeMeshGPT: Artistic Mesh Generation with Autoregressive Tree Sequencing

1. Introduction

• Artistic meshes offer compact geometry, sharp details, and regular wireframes, but creating them manually is time-consuming and labor-intensive.Dense meshes from other 3D representations can be unsuitable for real-time rendering, while down-sampling can degrade mesh quality.Source: §1 · pp. 1–2
• Earlier point-cloud-conditioned mesh generation faced sequence-length limits, restricting MeshAnything to 800 faces and later methods to 1,600 or 4,000 faces.Challenges also included gaps, missing components, and flipped normals.Source: §1 · p. 2
• TreeMeshGPT uses dynamically growing tree sequencing and represents each face with two tokens, reporting 22% compression relative to naive nine-token face encoding.With 7-bit discretization, it generates meshes of up to 5,500 triangles under conditioning with 2,048 point-cloud tokens.Source: §1 · p. 2
• TreeMeshGPT imposes strong normal-orientation constraints to minimize flipped normals encountered in MeshAnything [3] and MeshAnythingV2 [4].The authors also report higher-quality meshes and generalization to real-world 3D scans.Source: §1 · p. 2

2.1. Mesh Extraction

• Marching Cubes and Poisson reconstruction produce dense, over-tessellated meshes that lack the regularity and semantic alignment needed for efficient downstream workflows.Marching Cubes extracts triangles from cubes in a scalar field to approximate an isosurface.Source: §2.1 · p. 2

2.2. 3D Generation

• Feedforward 3D generation methods [10] [18] [36] improve generation speed over optimization-based methods, but their meshes often have lower quality and less diversity.Diffusion approaches such as CLAY [38] generate high-quality shapes, yet downstream use still requires non-trivial post-conversion.Source: §2.2 · pp. 2–3

2.3. Autoregressive Mesh Generation

• Autoregressive mesh methods progressed from MeshGPT [28] to adjacency-based tokenization in MeshAnythingV2 [4] and EdgeRunner [29], raising reported capacity from under 800 faces to 1,600 and 4,000.Other approaches include Meshtron [9]'s hourglass architecture and sliding-window inference, and BPT [34]'s block-wise indexing and localized patch aggregation; TreeMeshGPT instead uses a dynamically growing tree-based sequencing strategy.Source: §2.3 · p. 3

3.1. Autoregressive Tree Sequencing

• The decoder uses directed mesh edges as inputs and makes localized predictions to add an opposite vertex or stop expansion from that edge.Each generation step is associated with a pair of dequantized vertices.Source: §3.1 · p. 3
• A half-edge structure and depth-first search build the input-output sequence, while a dynamic stack supplies the next edge from its top.New triangles create two directed edges oriented counter-clockwise on potential adjacent faces; boundaries or previously visited triangles produce [STOP] without adding an edge.Source: §3.1 · p. 3
• The stack begins with a directed edge and its twin, and traversal continues until all triangles are visited.When the stack empties, the current component is complete; additional components start from the lowest-position remaining unvisited face.Source: §3.1 · pp. 3–4
• Generation initializes a component with [SOS] and [SOS2], traverses its edge stack, reinitializes tokens for later components, and ends with [EOS].The final mesh gathers faces formed from the initial vertex pairs and predicted opposite vertices, excluding auxiliary and stop labels.Source: §3.1 · p. 4
• Each edge embedding concatenates positional embeddings of its two vertices, and hierarchical MLP heads predict quantized x-, y-, and z-coordinates sequentially.The authors report easier coordinate sampling than with heads that predict all three coordinates simultaneously.Source: §3.1 · p. 4
• Tree sequencing adds two steps per triangular face and achieves approximately 22% compression for most meshes versus naive nine-token face encoding.The dynamic stack directs localized predictions; the decoder is trained with summed cross-entropy over the three coordinate axes, with [STOP] and [EOS] included as height-axis classes.Source: §3.1 · pp. 4–5

4.1. Dataset

• The training set contains 75,000 filtered Objaverse meshes, with 1,000 reserved for validation and meshes limited to fewer than 5,500 faces.Meshes must satisfy half-edge traversal requirements, be manifold, and have no flipped normals.Source: §4.1 · p. 5

4.2. Implementation Details

• Point-cloud conditioning encodes 8,192 sampled surface points into a latent code prepended to [SOS], while inference adjustments address duplicate faces and difficult [EOS] sampling.The decoder uses a 24-layer Transformer, 16 attention heads, and 7-bit coordinate quantization.Source: §4.2 · p. 5

4.3.1. Qualitative Results

• In qualitative Objaverse comparisons, TreeMeshGPT produces meshes with higher face counts and refined details than MeshAnything and MeshAnythingV2.The baselines use point clouds sampled from marching-cubes meshes with an 8-level octree.Source: §4.3.1 · p. 6, Figure 3 · p. 7

4.3.2. Quantitative Results

• On Objaverse, TreeMeshGPT generates meshes more faithful to ground truth, with consistent normal orientation where MeshAnything baselines show low NC from flipped normals.Evaluation uses 200 validation samples and one generation seed per model.Source: §4.3.2 · p. 6, Table 1 · p. 6
• In a controlled experiment, results on 200 test samples indicate that TreeMeshGPT’s tokenization provides an effective inductive bias.The experiment holds other factors constant across tokenizers and uses a filtered 24k-sample subset with at most 500 faces.Source: §4.3.2 · p. 6, Table 2 · p. 6

4.4. Results on GSO Dataset

• On GSO, TreeMeshGPT achieves the best results across evaluated metrics and generates meshes more consistent with inputs, with topology like artist-created meshes.The qualitative comparison with the decimated mesh highlights topology similarity to meshes created by 3D artists.Source: §4.4 · p. 6, Table 3 · p. 6

4.5. Ablation Study

• Simultaneous coordinate prediction produces noisy, incomplete meshes, whereas DFS traversal has better training perplexity than BFS in the traversal ablation.The proposed hierarchical MLP preserves sequential x-y-z prediction; BFS can introduce dependencies between spatially or structurally distant steps.Source: §4.5 · p. 8, Figure 6 · p. 8, Figure 7 · p. 8

• TreeMeshGPT uses Autoregressive Tree Sequencing and two tokens per face to generate point-cloud-aligned artistic meshes.Source: §5 · p. 8
• With 7-bit discretization and 2,048 point-cloud latent tokens, it generates up to 5,500 triangular faces and reports higher quality than previous methods.Source: §5 · p. 8
• Success rate decreases as sequence length increases, and enforcing optimal mesh topology remains challenging despite higher face-count capacity.Source: §5 · p. 8

• The supplement details the network's MLP heads and Normal Consistency metrics.Source: Supplementary Material for… · p. 11
• Section C demonstrates artistic mesh generation from text prompts through a multi-step process.Source: Supplementary Material for… · p. 11
• The supplement presents a 9-bit model that generates artistic meshes with up to 11,000 faces.Source: Supplementary Material for… · p. 11

• Hierarchical MLP heads predict a vertex's coordinates sequentially, using earlier predicted coordinates to condition later stages.Source: A. Vertex Prediction Heads · p. 11
• The first stage predicts an initial coordinate from decoder code c; the second predicts y using c and the discretized x-coordinate embedding, and the third predicts z using c and both prior coordinate embeddings.Source: A. Vertex Prediction Heads · p. 11
• For Objaverse, coordinates are predicted z, then y, then x; [STOP] and [EOS] are added to z classes, and teacher forcing conditions y and x on preceding ground-truth coordinates.Source: A. Vertex Prediction Heads · pp. 11–12

• For each source-face centroid, the metric finds the closest reference face using shortest point-to-face distance.Source: B. Normal Consistency Metrics · p. 12
• The matching is repeated in reverse, pairing each reference-face centroid with its closest source face.Source: B. Normal Consistency Metrics · p. 12
• Normal Consistency averages cosine similarity between paired face normals in both directions; |NC| omits flipping direction.Source: B. Normal Consistency Metrics · p. 12

• Luma AI Genie3 generates a dense mesh from each text prompt, typically with around 50,000 triangles.Source: C. Generating Artistic Meshes from… · p. 12, Figure 9 · p. 13
• The dense mesh is decimated before a point cloud is sampled from it for conditioning.Source: C. Generating Artistic Meshes from… · p. 12, Figure 9 · p. 13
• TreeMeshGPT uses the sampled point cloud as its input condition to generate the final artistic mesh.Source: C. Generating Artistic Meshes from… · p. 12, Figure 9 · p. 13

• For 7-bit training, meshes with more than 5,500 faces were excluded because many contained small triangles that collapse or merge after discretization.Source: D. 9-bit Model Supporting 10K+ Faces · p. 13
• Finer discretization reduces triangle collapses and merges that can violate the manifold condition required by TreeMeshGPT’s sequencing approach.Source: D. 9-bit Model Supporting 10K+ Faces · p. 13
• Up to 11,000 faces are supported by the 9-bit model, whose meshes have smoother surfaces and finer details than the 7-bit model.The 9-bit model took 25 days to train using 8× A100-80GB GPUs.Source: D. 9-bit Model Supporting 10K+ Faces · p. 13, Figure 10 · p. 14