PaperlayerView exact revision on arXiv

Source-linked AI summary

PolyFlow: Continuous Topology Embedding Flow Matching for Artist-style Mesh Generation

Chunshi Wang, Haohan Weng, Junliang Ye, Biwen Lei, Yang Li, Zibo Zhao, Zeqiang Lai, Kaiyi Zhang, Yunhan Yang, Zhuo Chen, Chunchao Guo, Yawei Luo

arXiv:2606.30673v1cs.GRcs.CV

TL;DR

Mesh generation needs artist-like topology without the severe latency of sequential autoregressive decoding, but continuous flow methods cannot directly represent discrete connectivity. PolyFlow embeds topology continuously for parallel flow-matching generation and outperforms the strongest autoregressive baseline by 43% in Chamfer Distance and 40% in Hausdorff Distance.

  • Problem

    Autoregressive mesh generation produces artist-like topology but remains bottlenecked by sequential decoding, while continuous generative methods cannot directly handle discrete mesh connectivity.

  • Method

    PolyFlow learns continuous per-vertex topology embeddings recoverable through spacetime distance thresholding, then jointly denoises positions, normals, and topology in parallel with flow matching.

  • Results

    43% in CD and 40% in HD: PolyFlow outperforms the strongest autoregressive baseline on both Chamfer Distance and Hausdorff Distance.

  • Takeaways & Limitations

    PolyFlow generates clean-topology meshes up to tens of times faster than autoregressive baselines while providing exact vertex-count control.

Abstract

from arXiv · show

Autoregressive Transformers dominate high-quality mesh generation by producing artist-worthy topologies, yet their inherent sequential decoding induces substantial computational overhead, falling orders of magnitude slower than parallel generative models. On the other hand, while continuous diffusion and flow-matching methods support efficient parallel synthesis across a variety of domains, they cannot be directly applied to meshes: mesh connectivity is inherently discrete and incompatible with standard continuous noise injection and denoising operations. To resolve this fundamental incompatibility, we introduce a compact topology embedder that projects discrete mesh vertex positions and normals into continuous per-vertex embeddings, where the original discrete adjacency information can be faithfully recovered via spacetime distance thresholding. After pretraining and freezing this embedder, any raw mesh can be fully converted into a continuous per-vertex state space unifying position, normal, and implicit topological attributes. Built upon this novel continuous mesh representation, we present PolyFlow, a Transformer-based flow-matching framework that achieves fully parallel vertex state denoising conditioned on extracted point-cloud features. During inference, our model completes generation rapidly via an ODE solver, and supports explicit, precise control over output mesh resolution by directly specifying the target vertex count. Extensive evaluations on the Toys4K benchmark demonstrate that PolyFlow surpasses state-of-the-art autoregressive baselines in both Chamfer Distance and Hausdorff Distance.

1 Introduction

PolyFlow addresses the discrete-topology barrier that prevents continuous flow-matching methods from directly generating meshes. It uses a continuous topology embedding and parallel joint geometry–topology denoising to produce retopologized meshes in seconds with exact vertex-count control.

  • Motivation: Production-ready retopology remains necessary because dense triangle soups lack clean, purposeful, and resolution-controllable connectivity.Polygonal meshes are valued for explicit connectivity supporting rendering, simulation, and artist editing.
  • Limitations of Existing Methods: Autoregressive mesh generation remains a fundamental latency bottleneck, requiring tens of seconds to several minutes for one mesh.This persists despite BPT’s 75% compression ratio and XSpecMesh’s speculative decoding.
  • Problem: Continuous diffusion and flow-matching methods cannot directly generate meshes because mesh topology is inherently discrete, with edges defined by binary existence.These frameworks otherwise synthesize structured signals in parallel with high fidelity and reduced latency.
  • Method: A lightweight topology embedder learns low-dimensional continuous per-vertex coordinates supervised by spacetime distance to approximate discrete adjacency.The embedder takes ground-truth vertex positions and normals as input and enables topology recovery through spacetime distance thresholding.
  • PolyFlow: PolyFlow jointly denoises positions, normals, and topology coordinates with one Transformer flow model, replacing autoregressive decoding with parallel generation.Conditioned on point-cloud features, it uses an ODE solver and generates meshes in seconds with clean topology and exact vertex-count control.

2 Related Work

Prior work has advanced native polygon-mesh generation, 3D-native generative representations, and continuous flow-based modeling, while topology has been modeled separately through continuous embeddings, differentiable triangulation, or triangle prediction.

  • Autoregressive Mesh Generation: PolyGen treated meshes as token sequences by factoring their joint distribution into separate vertex and face models.MeshGPT later introduced residual vector quantization to learn geometric vocabularies, compress face representations, and improve topological coherence.
  • 3D-Native Representations: Recent 3D generation research has shifted from external 2D priors toward 3D-native generative representations.Examples include 3DShape2VecSet and TRELLIS, which construct 3D-native autoencoding spaces.
  • 3D-Native Representations: Structured 3D latents enable direct generative modeling with faster inference and improved geometric fidelity.These representations also support native 3D part segmentation, structure-coherent shape decomposition, and part-aware multimodal modeling.
  • Flow-Based Modeling: Flow matching models continuous 3D states through velocity fields along simple transport trajectories and can reduce sampling steps versus conventional diffusion.DiT-based architectures and Flux-style velocity prediction further support scalable flow-based modeling.
  • Topology Prediction: Topology prediction methods include continuous per-vertex embeddings, differentiable weighted Delaunay triangulation probabilities, and local classifiers for candidate triangles.SpaceMesh uses a spacetime distance metric and halfedge cycle construction to guarantee edge-manifoldness.

3 Methodology

PolyFlow represents mesh topology continuously through frozen per-vertex embeddings and jointly denoises positions, normals, and topology with a Transformer flow model. Its inference procedure uses ODE integration, distance-thresholded edge recovery, and clique-based face extraction while allowing direct control of vertex count.

  • Method overview: PolyFlow uses two stages: a topology embedder converts discrete adjacency into continuous per-vertex embeddings, followed by a Transformer flow model that jointly denoises positions, normals, and topology.The topology embedder is pretrained and frozen before Stage 2.
  • Topology embedder: The topology embedder predicts edges by thresholding pairwise spacetime distances between embeddings, with a learned threshold τ.The embedder is trained using sampled binary cross-entropy over positive and negative vertex pairs, then frozen as a fixed feature extractor.
  • Joint flow state: Each continuous vertex token concatenates 3D position, unit normal, and frozen topology embedding, while directly specifying V determines the output mesh resolution.The joint state has shape V×(3+3+d).
  • Flow model: The flow Transformer predicts a channel-weighted velocity field from noisy joint states, timestep, and frozen point-cloud features, balancing geometry, normals, and topology channels.Classifier-free guidance dropout is used during Stage 2 training.
  • Inference and mesh decoding: At inference, an Euler ODE solver denoises the requested number of Gaussian-initialized vertex tokens, after which edges are decoded by spacetime thresholding and triangular faces by enumerating 3-cliques.Edge decoding requires no additional neural network, and face extraction is exact.

4 Experiments · 4.1 Dataset · 4.2 Baselines and Evaluation Metrics

The experiments train PolyFlow on approximately 5 million meshes and evaluate it on unseen Toys4K objects conditioned on uniformly sampled surface point clouds. Comparisons use representative mesh-generation baselines and report Chamfer Distance and Hausdorff Distance computed from 1024 sampled points, including test-set standard deviations.

  • 4.1 Dataset: Approximately 5 million meshes from public repositories and licensed 3D assets are used to train PolyFlow.The training collection combines public repositories with licensed 3D assets.
  • 4.1 Dataset: Toys4K contains diverse 3D objects unseen during training and serves as the evaluation dataset.For each test shape, the conditioning input is a uniformly sampled surface point cloud.
  • 4.1 Dataset: Each test shape is conditioned on a point cloud uniformly sampled from its surface.This point cloud is extracted from the test shape before generation.
  • 4.2 Baselines and Evaluation Metrics: PolyFlow is compared with BPT, MeshAnythingV2, DeepMesh, and FastMesh using official released weights when publicly available.These methods are described as representative mesh-generation baselines.
  • 4.2 Baselines and Evaluation Metrics: Chamfer Distance (CD) and Hausdorff Distance (HD) measure geometric fidelity from 1024 uniformly sampled points on generated and ground-truth mesh surfaces.The evaluation also reports each metric’s standard deviation across test samples.
  • 4.2 Baselines and Evaluation Metrics: At d=8, topology embeddings cannot encode adjacency and produce collapsed geometry, while d=32 is visually faithful with no discernible improvement at d=64.The visual ablation reconstructs meshes at embedding dimensions d=8, 16, 32, and 64.

4.3 Implementation Details

PolyFlow uses a pretrained 32-dimensional topology embedder and a Flux-based DiT denoiser with explicit architectural and conditioning specifications. Inference uses the EMA model with Euler ODE integration, classifier-free guidance, and point-cloud conditioning.

  • Topology embedder: The topology embedder produces 32-dimensional per-vertex embeddings using a 512-dimensional encoder with 12 Transformer layers and 8 attention heads.The embedding splits equally into space and time components, with d_s=d_t=16.
  • Flow model: The Flux-based DiT denoiser has 12 double-stream blocks, 24 single-stream blocks, hidden size 768, 16 attention heads, and input/output dimensionality 38.Its 38 dimensions comprise 3 xyz coordinates, 3 normals, and 32 topology-embedding dimensions.
  • Inference: Inference uses the EMA model with an Euler ODE solver over 50 integration steps and classifier-free guidance at scale 3.0.Input surface point clouds are sampled at 40,960 points and passed to the frozen condition encoder.

4.4 Quantitative Results

PolyFlow achieves the strongest quantitative performance on Toys4K, outperforming the strongest autoregressive baseline in both Chamfer and Hausdorff Distance. Its parallel flow-matching formulation also supports stable generation across diverse inputs, while competing methods show structural and detail artifacts on complex shapes.

  • Quantitative comparison: PolyFlow outperforms the strongest AR baseline, BPT, by 43% in CD and 40% in HD.It achieves the best performance on both metrics.
  • Quantitative comparison: PolyFlow has the lowest CD standard deviation, indicating stable generation quality across diverse inputs.The reported stability accompanies its best mean CD and HD performance.
  • Qualitative comparison: BPT and FastMesh frequently produce broken geometry, missing parts, or distorted proportions on complex shapes such as the dragon and spaceship.DeepMesh improves structural integrity with reinforcement learning but retains artifacts in thin structures and fine details.

4.5 Qualitative Results

PolyFlow’s qualitative results show parallel mesh generation with controllable vertex counts and progressively coherent denoising trajectories. Across diverse Toys4K shapes, it produces cleaner topology, fewer broken regions, and more faithful geometry than comparison methods.

  • Vertex Distribution: FastMesh often produces uneven vertex distributions with clustering artifacts, especially in complex regions such as dragon extremities and robot limbs.The comparison covers four test shapes against ground-truth retopology vertices.
  • Resolution Control: Users can specify PolyFlow’s target vertex count directly, demonstrated on a tree model spanning 250 to 3,000 vertices.At 250 vertices, the model forms a coarse but recognizable approximation focused on salient features such as the canopy outline and trunk.
  • Denoising Trajectory: PolyFlow’s denoising trajectory evolves from dispersed vertices and a tangled decoded mesh into a gradually coalescing, coherent shape.At step 25, the decoded mesh has no recognizable structure; later ODE steps progressively organize the geometry.
  • Qualitative Comparison: On Toys4K, PolyFlow produces cleaner topology, fewer broken regions, and more faithful geometry across diverse object categories than BPT, FastMesh, and DeepMesh.The qualitative comparison shows front and back views for each method alongside the input dense mesh.

4.6 Ablation Study · Topology Embedding Dimension

The ablation varies the frozen topology embedder’s dimensionality and evaluates both edge reconstruction and PolyFlow generation. Results favor d=32, which provides near-perfect topology reconstruction and balances downstream mesh quality without the added learning difficulty of larger states.

  • Topology Embedding Dimension: d=8 is too compact to faithfully encode adjacency, yielding high recall but poor precision in edge reconstruction.The embedding also fails to distinguish nearby vertices, producing collapsed meshes with severely tangled faces.
  • Topology Embedding Dimension: Increasing the topology embedding from d=8 to d=16 substantially improves precision, although spurious long-range edges remain visible around the limbs.At d=16, the overall shape is recovered despite these connectivity errors.
  • Topology Embedding Dimension: d=32 achieves near-perfect edge reconstruction with an F1 of 0.9991.The reconstructed connectivity at d=32 closely matches the input and is visually indistinguishable from d=64.
  • Topology Embedding Dimension: Figure 6 provides a qualitative evaluation of PolyFlow’s vertex generation.The supplied passage identifies the figure but does not report additional qualitative findings.
  • Topology Embedding Dimension: HD (0.025 vs. 0.021) indicates worse downstream generation for the larger embedding state.The passage attributes this degradation to greater denoiser learning difficulty without meaningful additional topology information.
  • Topology Embedding Dimension: d=32 and d=64 produce closely matching connectivity, with d=32 already delivering visually indistinguishable reconstruction from d=64.Their near-identical F1 values support using the smaller of the two dimensions.
  • Topology Embedding Dimension: Table 2 evaluates topology embedding dimensions using edge precision, recall, F1, Chamfer Distance, and Hausdorff Distance.The evaluation combines frozen topology-embedder reconstruction with PolyFlow’s end-to-end mesh-generation performance.
  • Topology Embedding Dimension: Table 3 compares inference time in seconds, with all timings measured on a single NVIDIA A100 GPU.The supplied passage identifies the comparison but does not provide the individual timing values.

Inference Speed

PolyFlow substantially accelerates mesh generation through parallel denoising, completing 4,000-vertex generation in 5.88s compared with much slower autoregressive and two-stage baselines. Its inference advantage persists as vertex count increases.

  • Inference Speed: 5.88s: PolyFlow completes 4,000-vertex generation through parallel denoising.The reported total inference time is 5.88s at 4,000 vertices.
  • Inference Speed: Over 9 minutes: BPT reaches this inference time at 4,000 vertices, with runtime scaling linearly with vertex count.BPT is described as a purely autoregressive method.
  • Inference Speed: 36.72s: FastMesh requires this time for 4,000-vertex generation despite its two-stage pipeline reducing inference time substantially.FastMesh remains slower than PolyFlow at the same vertex count.

5 Conclusion

PolyFlow generates polygonal meshes with clean, artist-like topology by parallel flow-matching denoising of vertex positions, normals, and continuous topology coordinates. Its topology embedder makes discrete adjacency recoverable in continuous per-vertex embeddings, enabling a single Transformer flow model to handle the entire mesh state.

  • Conclusion: PolyFlow generates polygonal meshes with clean, artist-like topology by denoising vertex positions, normals, and continuous topology coordinates in parallel.The model is conditioned on point clouds.
  • Conclusion: The topology embedder converts discrete mesh adjacency into continuous per-vertex embeddings recoverable through spacetime distance.This provides the continuous topology representation used by PolyFlow.
  • Conclusion: A single Transformer flow model handles the entire mesh state by combining vertex positions, normals, and embedded topology coordinates.This unified representation is enabled by recovering adjacency from the continuous embeddings.
Loading 2606.30673v1…