PaperlayerView exact revision on arXiv

Source-linked AI summary

MeshFlow: Mesh Generation with Equivariant Flow Matching

Qi Sun, Kiyohiro Nakayama, Jing Nathan Yan, Qixing Huang, Alexander Rush, Leonidas Guibas, Gordon Wetzstein, Jing Liao, Guandao Yang

arXiv:2606.23489v1cs.GRcs.CV

TL;DR

Direct mesh generation must handle the permutation symmetries of faces and vertices without sacrificing quality or efficiency. MeshFlow models triangle soups with equivariant optimal-transport flow matching, achieving comparable mesh quality to autoregressive generators with about an 18× inference speedup.

  • Problem

    Direct mesh generation is difficult because faces and vertices have important permutation symmetries, while autoregressive mesh generation requires long serialized sequences.

  • Method

    MeshFlow generates triangle soups directly using an equivariant Diffusion Transformer and an optimal-transport training objective that respects their symmetries.

  • Results

    MeshFlow achieves mesh quality comparable to state-of-the-art autoregressive mesh generators while producing high-quality meshes in less than a second, 18 times faster.

  • Takeaways & Limitations

    The results indicate that symmetry-aware flow matching can provide high-quality direct mesh generation with sub-second inference for interactive graphics applications.

  • Takeaways & Limitations

    The coupling algorithm has O(n^3) complexity, generated meshes occasionally contain missing or overlapping faces, and scaling or artifact reduction remains future work.

Abstract

from arXiv · show

Meshes are among the most common 3D scene representations, but directly generating meshes is challenging because the representation contains important symmetries, including permutation invariance of faces and vertices. MeshFlow learns to generate triangle meshes directly as triangle soups, avoiding the need to serialize meshes into long autoregressive sequences. We adopt equivariant optimal-transport flow matching models that respect the key symmetries of triangle soups: arbitrary permutations of faces and permutations of the vertices within each face. Toward this goal, we propose a simple yet effective modification to the Diffusion Transformer architecture, resulting in a scalable network capable of modeling a velocity field while maintaining the desired equivariance. We further introduce an optimal-transport-based training objective that improves convergence by eliminating supervision signals that violate these symmetries. MeshFlow achieves mesh quality comparable to state-of-the-art autoregressive mesh generators while providing about an 18$\times$ speedup during inference. Project page is at https://qiisun.github.io/MeshFlow/.

1 Introduction

MeshFlow addresses the difficulty of generating high-quality meshes directly by learning over triangle soups with symmetry-aware flow matching. It combines an equivariant architecture and optimal-transport objective, achieving mesh quality comparable to autoregressive methods with substantially faster inference.

  • Traditional meshing tools are heuristic and require human intervention, motivating generative models that adapt to varied inputs while matching artist- and engineer-crafted mesh quality.
  • Two-stage pipelines generate intermediate representations before meshing, but conversion algorithms can introduce artifacts such as over-tessellated surfaces.
  • MeshFlow learns directly from human-created mesh data using equivariant optimal-transport flow matching instead of relying on intermediate representations or serialized mesh sequences.
  • MeshFlow achieves mesh quality on par with state-of-the-art mesh generators across ShapeNet categories and produces high-quality meshes in less than a second, 18 times faster than autoregressive methods.
  • Its modified Diffusion Transformer respects face and within-face vertex symmetries, while its optimal-transport loss removes symmetry-violating training signals.

2 Related Work

Prior mesh-generation methods use intermediate representations, autoregressive sequences, or diffusion models, each with distinct quality, efficiency, or representation limitations. MeshFlow instead applies equivariant generation directly to triangle-soup meshes.

  • Mesh Generation through Intermediate Representation: Intermediate-representation methods generate point clouds, voxels, or implicit functions before converting them into meshes with reconstruction algorithms.
  • Mesh Generation through Intermediate Representation: Voxel-based generation has a high memory footprint, making high-fidelity 3D generation expensive.
  • Mesh Generation through Intermediate Representation: Mesh conversion can create irregular faces, lose sharp features, or produce tessellations inferior to artist-created meshes, whereas MeshFlow outputs meshes directly.
  • Auto-regressive Mesh Generation: Autoregressive mesh generators achieve strong quality and human-like tessellations but suffer from slow inference, difficult control, and error accumulation in long sequences.
  • Diffusion-based Mesh Generation: Diffusion and flow-matching methods refine all data dimensions simultaneously, while MeshFlow applies equivariant generation directly to triangle soups to leverage their inherent invariances.

3 Preliminaries

Flow matching learns a time-dependent vector field that transports noise toward data along prescribed paths, with optimal-transport couplings producing straighter trajectories. MeshFlow adapts this framework to symmetry-invariant triangle soups through equivariant coupling and architecture choices.

  • Flow matching: Flow matching defines a conditional vector field and path that transform samples from a prior distribution, such as Gaussian noise, into target data.
  • Flow matching: A common flow-matching target uses linear interpolation between noise x_0 and data x_1, with constant velocity u_t = x_1 − x_0.
  • Flow matching: Optimal-transport coupling minimizes transport cost and can produce straighter trajectories, although exact optimal-transport maps are often intractable.
  • Data symmetry and Coupling: Data symmetries make distributions invariant under group actions, allowing equivariant optimal-transport methods to reduce transport costs for structured data.
  • MeshFlow combines a triangle-soup metric, symmetry-aware noise-data coupling, and an equivariant rectified-flow network for unconditional mesh generation.
  • Equivariant architecture: If the velocity field is equivariant and the prior distribution is invariant, the induced probability distribution remains invariant under the symmetry group.

4 Method

MeshFlow represents meshes as unordered triangle soups and combines a symmetry-aware equivariant DiT with optimal-transport coupling. The architecture preserves face and within-face vertex permutation symmetries while processing face-level global information efficiently.

  • 4.1 Triangle Soup Representation: MeshFlow models an unconditional mesh M = {V, F} as a triangle soup, avoiding sequential dependencies imposed by autoregressive representations.Each soup contains N triangular faces, with unordered vertices and no explicitly modeled orientation or topology.
  • 4.1 Triangle Soup Representation: Triangle soups are invariant to arbitrary face permutations and to permutations of the three vertices within each face, forming the group G = S3 ≀N SN.These two symmetry levels motivate the equivariant architecture and symmetry-aware training objective.
  • 4.2 Equivariant Architecture: The network encodes vertex coordinates with sinusoidal positional features, maps them to a hidden dimension, and uses equivariant DiT blocks with conditioning on time and face count.The architecture includes a vertex positional embedder, repeated equivariant DiT blocks, and an output layer.
  • 4.2 Equivariant Architecture: The equivariant DiT modifies standard transformer processing because full S3N equivariance is computationally expensive and less expressive than equivariance to G.Self-attention over all 3N vertices has quadratic token complexity; the proposed block instead aggregates vertex features within faces, applies self-attention to face features, and adds the result back to vertices.
  • 4.3 Symmetry-aware Training Objectives: Symmetry-aware training couples noise and data by minimizing squared distance across their G-orbits, using nested face and vertex matching rather than arbitrary correspondences.The face assignment is solved with a Hungarian algorithm after constructing pairwise face costs.
  • 4.3 Symmetry-aware Training Objectives: Nested coupling reduces transport-path crossings and achieves lower optimal-transport cost than independent or face coupling in the illustrated comparisons.The resulting interpolation follows a straighter path, while face coupling can still produce vertex crossings.

5 Experiments

Experiments evaluate MeshFlow on ShapeNet categories using distribution-similarity and topological-quality metrics, comparisons with autoregressive and diffusion baselines, and ablations of its architecture, coupling, and post-processing. MeshFlow achieves strong mesh quality and substantially faster inference, while nested optimal transport and equivariant design improve performance and efficiency.

  • Experimental Setup: MeshFlow is evaluated on four ShapeNet categories—Table, Chair, Lamp, and Bench—using 1-NNA for distribution similarity and metrics for topological quality.Meshes are normalized and augmented through quadric edge-collapse decimation.
  • Main Results: 18.55× speedup over autoregressive methods demonstrates MeshFlow’s inference-efficiency advantage.Post-processing adds minimal latency while preserving mesh quality.
  • Main Results: MeshFlow achieves the best 1-NNA score in 3 of 4 categories and shows superior generative performance with fewer face intersections than PolyDiff.Qualitative results also show fewer missing and intersecting faces than baselines.
  • Ablation Studies: The equivariant architecture produces superior 1-NNA performance and the most visually coherent results among the evaluated architecture variants.The ablation compares vanilla, face-equivariant, and the proposed architecture variants.
  • Ablation Studies: Nested optimal transport achieves the best 1-NNA among coupling variants and maintains high-fidelity results with 20 function evaluations, while independent coupling fails at that setting.Independent coupling converges more slowly, produces more curved trajectories, and can require more sampling steps.
  • Ablation Studies: Post-processing reduces average self-intersection rate by approximately 56% across categories while retaining comparable 1-NNA.The denoiser removes local geometric artifacts and severe self-intersections from raw outputs.

6 Discussion, Limitations, and Future Direction

MeshFlow directly applies equivariant flow matching to triangle soups and designs its architecture and training objectives around their symmetries. It matches state-of-the-art mesh quality while achieving sub-second inference, but scaling and artifact issues remain open limitations.

  • Discussion: MeshFlow applies equivariant flow matching directly to the triangle-soup representation and designs corresponding symmetry-aware training objectives and network architecture.
  • Discussion: MeshFlow matches state-of-the-art mesh generative models in mesh quality while achieving sub-second inference speed.
  • Limitations and Future Direction: The coupling algorithm may be difficult to scale to many faces because its complexity is O(n^3).Patch-based training and approximate optimal transport are proposed as possible directions.
  • Limitations and Future Direction: Generated meshes occasionally contain missing or overlapping faces, which the authors hypothesize may reflect limited computational resources.The authors suggest large-scale training as a possible remedy.

Appendix

The appendix reports that the authors released the full MeshFlow data and code.

  • Resources: The full data and code are released at the MeshFlow GitHub repository.

7 Proof: Equivariance of the Proposed DiT Block

The proof establishes that the proposed Equivariant DiT block respects face permutations and independent within-face vertex permutations. It verifies equivariance through mean pooling, self-attention, broadcasting addition, and point-wise feed-forward processing.

  • Equivariance Statement: The proposed Equivariant DiT block is equivariant to G = S_N × (S_3)^N, covering face permutations and independent vertex permutations within faces.
  • Setup: The proof represents the input as N faces with three vertices and C-dimensional features, then defines the group action through face and within-face vertex permutations.
  • Equivariant DiT Block: Mean pooling makes each face feature invariant to within-face vertex permutations while allowing face features to permute under face reordering.
  • Equivariant DiT Block: Self-attention remains permutation equivariant over the set of pooled face features because it uses no positional encoding.The proof analyzes the block as mean pooling, self-attention, broadcasting addition, and a point-wise feed-forward network.
  • Equivariant DiT Block: Broadcasting addition preserves the group action, and the point-wise feed-forward network commutes with permutation operators.Together these steps yield z′ = g · z and complete the equivariance proof.
  • Equivariance Motivation: Standard token-level positional encodings break permutation equivariance by assigning biases according to sequence order.The proposed design therefore avoids fixed ordering in contrast to standard autoregressive mesh-generation backbones.

8 Extended Results

The extended results describe implementation details for nested optimal-transport coupling, vertex welding, and denoiser training. These procedures support processing triangle soups and reconstructing valid meshes.

  • Optimal-transport coupling: The nested optimal-transport procedure uses face-level assignment and vertex-permutation retrieval to couple triangle soups.It is implemented with a Hungarian solver and enumeration of six vertex permutations per face pair.
  • Vertex welding: Vertex welding builds a spatial index, merges neighbors within a distance threshold, and reconstructs faces from canonical vertex indices.Degenerate faces created by collapsed vertices are filtered out to preserve geometric validity.
  • Optimal-transport coupling: The Hungarian algorithm solves the face assignment using dual variables, reduced costs, and augmenting paths.The procedure updates row and column duals using the minimum slack along candidate edges.
  • Denoiser training: The denoiser uses Gaussian noise augmentation with standard deviation η=0.02 to simulate positional inaccuracies during inference.It is trained with batch size 128 and an initial learning rate of 1×10^-4 using cosine decay.

8.5 Convergence of Topology

The extended experiments examine topology emergence, shape novelty, alternative triangulations, evaluation metrics, face-budget effects, broader-category performance, and scalability boundaries. Together, they report coherent topology, diverse outputs, strong fidelity, and limits for dense meshes.

  • Convergence of Topology: Topology quality emerges after 8,000 training steps, while Chamfer Distance improves more strongly during earlier training.Vertex ratio, face intersection ratio, and Chamfer Distance decrease as training progresses.
  • Shape novelty: Generated chairs span both close and distant regions of the training distribution, with substantial novelty from the 50th to 90th percentiles.Nearest-neighbor comparisons also show structural differences even for some low-Chamfer-Distance samples.
  • Topology diversity: MeshFlow can produce nearly identical geometry with distinct topological structures, enabling multiple triangulations for a fixed shape.Figure 14 illustrates this one-geometry, many-topologies behavior.
  • Quantitative evaluation: 14.85 on Chair and 25.20 on Lamp are the reported MMD scores, outperforming MeshXL despite MeshFlow using 124M rather than 350M parameters.The method also reports competitive COV and state-of-the-art FID on Chair and Lamp.
  • Face-budget effects: 800 faces yield smaller face areas and higher Gaussian curvature, whereas 100 faces produce a smoother low-poly abstraction.The reported values are face area 0.011 and Gaussian curvature 840.03 at 800 faces, versus 0.058 and 119.98 at 100 faces.
  • Generalization and efficiency: MeshFlow outperforms MeshXL across six additional ShapeNet classes, supporting generation quality beyond the original four-category evaluation.Inference timing experiments use 1k chair meshes with batch size 1 on an NVIDIA A6000 GPU.
  • Scalability: The Hungarian-based implementation supports meshes up to several thousand faces, while 10k+ face meshes motivate more efficient optimal-transport solvers.Dense production meshes remain a scalability target for future work.

9 Extended Ablation study

The extended ablations evaluate the denoising decoder, time shifting, and positional encoding. They show artifact removal, improved fidelity with time shifting, and unstable optimization without positional encoding.

  • Denoising decoder: The denoising decoder removes self-intersections, planar-region noise, and disconnected or jagged geometry from raw triangle-soup outputs.It acts as a geometric projection operator mapping noisy, non-manifold inputs to clean meshes.
  • Time shifting: MMD decreases from 16.50 to 14.85 with time shifting, while 1-NNA improves from 55.81% to 54.51%.Coverage increases slightly to 49.93%, while JSD remains comparable.
  • Positional encoding: Removing positional encoding causes loss divergence and gradient explosion to 10^10, whereas positional encoding stabilizes convergence.The ablation indicates that positional encoding helps the model distinguish and assemble geometric primitives.

10 More Limitations

The extended limitations concern mesh resolution and face-count flexibility. MeshFlow is currently intended for bounded face budgets and requires a predefined number of faces during inference.

  • Resolution limits: The proof-of-concept model supports a maximum of 800 faces, limiting representation of product-level meshes above 10k faces and intricate real-world geometry.The authors identify higher-resolution models and more efficient architectures as future directions.
  • Face-count flexibility: MeshFlow cannot currently vary the number of generated faces and requires a predefined face count during inference.Dynamic prediction of a task-aware face budget is proposed as future work.
Loading 2606.23489v1…