Source-linked AI summary
Meshy T2: Fast Native Mesh Generation with Flow Matching
Jiale Xu, Rendong Liang, Yuhao Long, Siyuan Shen, Zangyueyang Xian, Xiaofei Wu, Zeyi Xu, Yuanming Hu
TL;DR
Generating compact, valid, artist-style meshes remains a central challenge for scalable 3D content creation. Meshy T2 uses a vertex-set mesh VAE and coarse-to-fine flow matching to generate meshes, achieving state-of-the-art geometric fidelity while running over an order of magnitude faster than autoregressive baselines.
Problem
Automatically generating compact, valid, ready-to-use meshes remains a central challenge in scalable 3D content creation.
Method
Meshy T2 uses a vertex-set mesh VAE and two-stage flow matching to jointly generate vertices, connectivity, and face orientation from an image.
Results
Meshy T2 achieves state-of-the-art geometric fidelity on retopology and image-to-mesh tasks while running over an order of magnitude faster than autoregressive baselines.
Takeaways & Limitations
The system provides interactive-speed generation, predictable face-count control, and native multicomponent asset generation without separate splitting or stitching.
Takeaways & Limitations
The vertex-set representation assumes manifold meshes, so non-manifold inputs must be repaired by splitting offending edges and vertices.
Abstract
from arXiv · showhide
Polygonal meshes are the standard surface representation of modern 3D pipelines, and generating high-quality meshes with artist-style topology is essential for film, gaming, and interactive 3D applications. Mainstream approaches serialize a mesh into a token sequence and decode it autoregressively, which is slow at inference and sensitive to error accumulation, making them impractical for interactive asset creation. We present Meshy T2, a fast native mesh generation framework built on flow matching. At its core is a vertex-set mesh VAE that encodes a mesh into one continuous latent token per vertex and decodes vertices, edge connectivity, and face winding order in a single pass, preserving high-precision geometry and artist-authored topology without vertex quantization or welding. Generation proceeds as a coarse-to-fine cascade of two flow-matching models: an image-conditioned voxel flow first sketches the overall shape as a coarse occupancy scaffold, and a mesh flow then populates the scaffold with per-vertex latent tokens, conditioned on the image, the scaffold, and a requested vertex budget. This design delivers three practical capabilities: interactive generation speed through parallel flow-based synthesis; effective face-count control through the requested vertex budget; and native support for multi-part assets, whose components emerge directly from the generated connectivity. In our experiments, Meshy T2 achieves state-of-the-art geometric fidelity and completes end-to-end image-to-mesh generation within a median of 6 seconds, over an order of magnitude faster than autoregressive baselines. Code and weights will be available at https://github.com/meshy-dev/meshy-t2.
1 Introduction
Meshy T2 addresses the challenge of producing compact, valid, ready-to-use meshes by directly generating geometry and topology with a parallel flow-based pipeline. Its vertex-set representation and two-stage generation design support faithful structure preservation, controllable face budgets, and end-to-end generation within ten seconds.
- Production-ready meshes must compactly capture shape while preserving sharp edges, thin structures, and valid geometry for scalable 3D content creation.
- Most high-quality 3D generative systems extract explicit surfaces from implicit or volumetric representations rather than generating meshes directly.
- Meshy T2 uses a nearly lossless vertex-set mesh VAE that jointly decodes vertices, edge connectivity, and face winding order without quantization or vertex welding.
- A two-stage flow-matching pipeline first predicts an image-conditioned coarse voxel scaffold, then jointly generates vertex tokens, connectivity, and face winding conditioned on the image and scaffold.
- Within ten seconds, Meshy T2 completes end-to-end image-to-mesh generation, while requested vertex slots directly determine the expected face budget without post-hoc simplification.
- Experiments report state-of-the-art geometric fidelity on retopology and image-to-mesh tasks while running over an order of magnitude faster than autoregressive baselines.
2 Method
Meshy T2 uses a vertex-set mesh VAE and a coarse-to-fine cascade of two flow-matching models for image-to-mesh generation. The VAE preserves continuous per-vertex geometry while predicting connectivity and oriented faces, and the cascade conditions mesh synthesis on image, voxel scaffold, and vertex count.
- Vertex-set mesh VAE: The vertex-set VAE assigns exactly one continuous latent token to each vertex and decodes positions, edges, and oriented faces in a single pass.Continuous regression avoids vertex quantization and welding, preserving fine geometry and distinct tokens for coincident vertices.
- Vertex-set mesh VAE: Topology is represented by an undirected edge set and per-vertex halfedge successor permutations, which losslessly determine oriented faces.At inference, edges are thresholded, soft permutations are rounded into single fans, and oriented triangles are assembled from successor mappings.
- Topology representation: Open boundaries are supported by adding a NULL element to each vertex fan, while non-manifold training meshes are repaired by splitting offending edges and vertices.The NULL element converts open strips into proper permutations, allowing non-watertight artist-created surfaces to be represented.
- Coarse-to-fine generation: A voxel flow first generates a coarse occupancy scaffold because directly flowing the large unordered vertex-token set from an image performed poorly.The scaffold is modeled in the continuous latent space of a pretrained Voxel VAE and recovered by thresholding predicted occupancy probability at 0.5.
- Coarse-to-fine generation: The mesh flow maps Gaussian noise to per-vertex latent sets conditioned on image features, voxel features, and vertex count, enabling practical face-count control.Image conditioning uses a frozen DINOv3 encoder at 768 × 768 resolution, while inference voxel conditioning comes from the generated scaffold.
3 Evaluation
Meshy T2 is evaluated on 115 diverse assets across high-poly retopology and image-to-mesh generation. It achieves the strongest retopology geometry, leads DINOv2 perceptual alignment, and provides 6-second, fully reliable image-to-mesh generation.
- Evaluation setup: The benchmark contains 115 assets spanning objects, architecture, creatures, and characters, evaluated in high-poly retopology and image-to-mesh tasks.Face-budget methods are evaluated at roughly 4,000 faces, or approximately 2,000 vertices.
- High-poly retopology: 0.020 CD, 0.044 HD, and 0.860 NC make Meshy T2 the best overall retopology method on geometric alignment.Meshy T2 leads MeshFlow by 0.020 vs. 0.319 CD.
- Ablation: Sobol OT converges to lower validation Chamfer, Hausdorff, and non-manifold edge ratio than index position encoding or Morton-order pairing.The strategy is adopted by default for both the VAE and second-stage latent flow.
- High-poly retopology: 0.14 mean non-manifold edges and 70.1% tri-to-quad mergeability indicate cleaner topology in Meshy T2’s retopologized artist meshes.These measurements reflect the final artist mesh rather than the raw image-to-mesh stage.
- Image-to-mesh generation: 2312.01 DINOv2 FD is Meshy T2’s best perceptual-alignment result, versus 2442.27 for Tripo P1 and 2577.00 for MeshFlow.MeshFlow achieves the lowest Inception FD at 254.06.
- Latency and reliability: 6 s median runtime with 100% success makes Meshy T2 faster than Tripo P1 at 12 s and substantially faster than every autoregressive baseline.MeshFlow also has 100% reliability but requires 94 s per asset.
4 Conclusion
Meshy T2 is a fast mesh-generation framework whose joint vertex-connectivity modeling enables predictable face-count control and naturally supports multicomponent assets. Future work targets greater topological robustness, richer controls, and scene-level generation.
- Practical capabilities: Meshy T2 generates compact meshes within ten seconds and supports predictable face budgets by sampling a requested number of vertex slots before decoding.The triangle-mesh Euler relation F≈2V makes the resulting face budget predictable.
- Multicomponent assets: Jointly modeled vertex connectivity naturally decomposes generated vertex-edge graphs into connected components, enabling multicomponent assets without splitting or stitching.No additional component-wise generation stage is required.
- Future work: Future work will improve topological robustness on highly irregular meshes, add richer material and part-level controls, and study scene-level generation.The scene-level setting involves generating multiple compact meshes.