PaperlayerView exact revision on arXiv

Source-linked AI summary

Neural Subdivision

Hsueh-Ti Derek Liu, Vladimir G. Kim, Siddhartha Chaudhuri, Noam Aigerman, Alec Jacobson

arXiv:2005.01819v1cs.GRcs.LG

TL;DR

Automatic mesh upsampling with classic subdivision can over-smooth shapes and miss local details. Neural Subdivision learns nonlinear vertex updates from high-resolution meshes using generated coarse variants and bijective correspondences. It reports closer high-resolution reconstructions than traditional subdivision and generalization to novel meshes, including after training on a single shape.

  • Problem

    Classic subdivision can over-smooth automatically upsampled shapes because fixed linear averaging ignores local geometric context and details to preserve.

    Source: §1 · p. 2

  • Method

    Neural Subdivision applies Loop’s fixed topological updates while a shared neural network predicts vertex positions from local patch geometry, trained using generated coarse variants and bijective correspondences.

    Source: §1 · p. 2

  • Results

    The method generates meshes closer to true high-resolution shapes than traditional interpolatory and non-interpolatory subdivision, with significant gains on an upsampling-fidelity benchmark.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Its deterministic mesh connectivity supports direct use in graphics pipelines, and it generalizes to novel meshes even when trained on a single shape.

    Source: §1 · p. 2

    The framework is limited to triangle meshes without boundaries; extensions to quadrilateral meshes and surfaces with boundaries are identified as future work.

    Source: §7 · p. 12

Abstract

from arXiv · show

This paper introduces Neural Subdivision, a novel framework for data-driven coarse-to-fine geometry modeling. During inference, our method takes a coarse triangle mesh as input and recursively subdivides it to a finer geometry by applying the fixed topological updates of Loop Subdivision, but predicting vertex positions using a neural network conditioned on the local geometry of a patch. This approach enables us to learn complex non-linear subdivision schemes, beyond simple linear averaging used in classical techniques. One of our key contributions is a novel self-supervised training setup that only requires a set of high-resolution meshes for learning network weights. For any training shape, we stochastically generate diverse low-resolution discretizations of coarse counterparts, while maintaining a bijective mapping that prescribes the exact target position of every new vertex during the subdivision process. This leads to a very efficient and accurate loss function for conditional mesh generation, and enables us to train a method that generalizes across discretizations and favors preserving the manifold structure of the output. During training we optimize for the same set of network weights across all local mesh patches, thus providing an architecture that is not constrained to a specific input mesh, fixed genus, or category. Our network encodes patch geometry in a local frame in a rotation- and translation-invariant manner. Jointly, these design choices enable our method to generalize well, and we demonstrate that even when trained on a single high-resolution mesh our method generates reasonable subdivisions for novel shapes.

1 INTRODUCTION

Classical subdivision is useful for interactive coarse-to-fine modeling but can over-smooth shapes during automatic upsampling. Neural Subdivision learns local, nonlinear vertex updates and self-supervises from high-resolution meshes, producing meshes closer to high-resolution shapes than traditional subdivision methods.

1 INTRODUCTION

  • Classic subdivision can over-smooth an automatically upsampled shape and misses details because fixed linear averaging ignores local geometric context.Its one-size-fits-all weights were designed for general convergence and smoothness, rather than exploiting information in existing 3D models.Source: §1 · p. 2
  • Neural Subdivision keeps Loop’s topological updates but uses a shared neural network conditioned on local patch geometry to learn nonlinear vertex positions.The training data tailors the learned subdivision style to the class, type, or diversity of geometries.Source: §1 · p. 2
  • Training uses only high-resolution meshes, generating stochastic coarse variants with bijective surface correspondences to supervise exact target positions and support generalization across discretizations.The paper presents this correspondence-based loss as more efficient and accurate than existing methods.Source: §1 · p. 2
  • The output retains deterministic connectivity based on the input, while shared local-patch weights avoid requiring a fixed genus, template, or large shape collection.The method is demonstrated to generalize to novel meshes even when trained on a single shape.Source: §1 · p. 2
  • Neural Subdivision produces meshes closer to true high-resolution shapes than traditional interpolatory and non-interpolatory subdivision, with significant gains on an upsampling-fidelity benchmark.The paper also demonstrates low-poly mesh upsampling and 3D modeling applications.Source: §1 · p. 2

2 RELATED WORK

The related work establishes Loop subdivision as the triangle-mesh foundation and contrasts learned local refinement with classical smoothing and neural geometry methods. Neural Subdivision combines data-dependent nonlinear rules with mesh connectivity and correspondence-based supervision.

2.1 Subdivision Surfaces

  • Loop’s linear subdivision for triangle meshes is the basis for Neural Subdivision, while classical schemes combine topological updates with local averaging and leave detail capture to users or procedural routines.Classical methods are designed to guarantee smoothness and are easier to analyze than nonlinear methods.Source: §2.1 · p. 2
  • Neural Subdivision treats its rule as a learned nonlinear function; its training data influences the resulting style, and the method is trained for a finite number of subdivision steps.Prior nonlinear schemes also studied geometric invariants, while an alternative probabilistic subdivision method is described as orthogonal to this work.Source: §2.1 · pp. 2–3

2.2 Neural Geometry Learning

  • Point-cloud upsampling lacks mesh connectivity and needs surface reconstruction, whereas template-based mesh generation is constrained by deformations of a shape or by template topology.Neural Subdivision locally refines the input mesh and respects its topology.Source: §2.2 · p. 3
  • For mesh-patch processing, Neural Subdivision learns generative features over half-flaps whose canonical orientations provide local frames for rigid-motion invariance.The approach is inspired by MeshCNN, which learns filters over local mesh structure for deterministic tasks.Source: §2.2 · p. 4
  • Closest-point reconstruction losses can leave target points unmatched and produce self-overlaps, while low-distortion bijective maps provide one-to-one correspondences for an ℓ2 loss.The generated maps are intended to ensure every part of the target surface has a pre-image on the coarse mesh.Source: §2.2 · p. 4

3 NEURAL SUBDIVISION

Neural Subdivision recursively refines a coarse triangle mesh with Loop’s topological updates while a neural network predicts vertex positions from local geometry. Bijective maps from coarse inputs to high-resolution training shapes provide corresponding targets for training and support generalization across discretizations.

Inference

  • At inference, Loop-style midpoint subdivision adds vertices recursively while the network predicts positions at each new level.The output is a sequence of meshes with increasing detail.Source: §3 · p. 4

Training and loss

  • A bijective coarse-to-fine map composed with subdivision-to-coarse maps assigns each predicted vertex a corresponding point on the training shape.The training loss measures the ℓ2 distance between corresponding positions at every subdivision level.Source: §3 · p. 4

Data generation

  • Bijective, continuous correspondences ensure the entire target surface is covered, unlike non-bijective closest-point correspondences.Random edge collapses produce multiple coarse versions of a training mesh, with maps maintained through successive collapses.Source: §3 · p. 4

Advantages of our training approach

  • Training on random decimations of the same shape helps the network generalize across discretizations.The low-distortion map also supports mapping additional signals such as texture.Source: §3 · p. 5

Network architecture

  • The recursively applied network uses local mesh neighborhoods to predict differential features that reconstruct vertex coordinates.Initialization, Vertex, and Edge modules process the initial vertices, existing vertices, and newly inserted edge-midpoint vertices, respectively.Source: §3 · p. 5

4 DATA GENERATION AND TRAINING

The training setup creates one-to-one correspondences between coarse and high-resolution meshes by extending self-parameterization to successive edge collapses. These correspondences support an ℓ2 training loss, while conformal local maps and validity checks help maintain usable, low-distortion mappings.

4 DATA GENERATION AND TRAINING

  • The training setup replaces unreliable closest-point correspondences with intrinsic bijective maps, making a simple ℓ2 loss sufficient for measuring vertex errors.Naive point-to-mesh or Chamfer-like matching can produce incorrect, self-overlapping matches and fail to capture the full target surface.Source: §4 · p. 6, Figure 10 · p. 6

4.1 Successive Self-Parameterization

  • Successive self-parameterization combines MAPS with successive mapping [Cohen et al. 1998, 1997, 2003] to produce bijective maps with any chosen edge-collapse algorithm.Its two-step module takes a triangle mesh and collapse algorithm as input and returns a decimated mesh with a corresponding map.Source: §4.1 · p. 6

4.2 Single Edge Collapse

  • For each collapse, a shared UV parameterization maps points between the pre-collapse and post-collapse neighborhoods while preserving injectivity.The boundary vertices are preserved through the collapse, allowing the same UV domain to support mappings in both directions.Source: §4.2 · p. 7, Figure 13 · p. 7, Figure 14 · p. 7

4.3 Implementation

  • Conformal flattening limits accumulated angle distortion, while the combined collapse and parameterization algorithm remains O(N log N).The mapping’s robustness depends on the underlying edge-collapse algorithm; collapses that create self-intersections can make maps unusable.Source: §4.2 · p. 7, §4.3 · p. 7

4.4 Training Data & Loss Computation

  • The loss compares each predicted vertex with its mapped ground-truth point using ℓ2 distance, and computes orders of magnitude faster than Chamfer loss.The ground-truth positions are obtained from barycentric coordinates through the bijective map.Source: §4.4 · pp. 7–8, Figure 17 · p. 7

5 NETWORK ARCHITECTURE

The architecture applies shared neural modules recursively to local mesh neighborhoods, using half-flaps and local differential features to predict vertex positions. Its local-frame representation is invariant to rigid motion, which the paper reports as important to output quality.

Network steps

  • Initialization, Vertex, and Edge modules successively update vertex features and predict displacement vectors for vertices and edge midpoints.The modules use edge vectors and differential coordinates or learned features as inputs.Source: §5 · pp. 8–9, Figure 19 · p. 9, Figure 20 · p. 9

Design choices

  • Local, shared modules operate on mesh patches and differential quantities in local frames, supporting generalization across meshes with different topologies.The design preserves input topology and provides many local training patches even from a single training pair.Source: §5 · p. 8

Half-flap operator

  • A half-flap operator processes four canonically ordered vertices with a shallow MLP whose weights are shared across subdivision levels.Initialization and Vertex pool outputs across outgoing half-flaps; Edge pools outputs from both directions of an edge.Source: §5 · p. 8, Figure 18 · p. 9

Design evaluation

  • The invariant representation improves result quality; slight rigid motion can make learned weights inapplicable, while differential-coordinate inputs speed convergence.The authors report that other tested design choices offer only small convergence improvements.Source: §5 · pp. 9–10, Figure 21 · p. 9

6 EVALUATIONS

Evaluations show that Neural Subdivision generalizes across deformations, shapes, and discretizations, while training data influences subdivision style. It also outperforms classic subdivision methods on the reported shape-upsampling metrics.

6 EVALUATIONS

  • Training on one pose generalizes to unseen approximately isometric poses and manually altered non-isometric coarse cages.The figure examples show the pose and non-isometric deformation tests.Source: §6 · p. 10, Figure 22 · p. 10, Figure 23 · p. 10
  • Training on a single shape generalizes to different geometries, and training on classic Loop sequences reproduces Loop subdivision on unseen shapes.The single-shape example includes subdivisions of other geometries.Source: §6 · p. 10, Figure 24 · p. 10
  • The method also generalizes to artist-created coarse discretizations rather than only meshes produced by edge collapse.Figure 25 illustrates the artist-created input meshes.Source: §6 · p. 10, Figure 25 · p. 11
  • Training shape affects subdivision style: smooth training shapes yield smoother results, while man-made training shapes can preserve sharp creases.The paper also reports stylized outputs biased toward the training shapes.Source: §6 · p. 10, Figure 27 · p. 11, Figure 26 · p. 11
  • Neural Subdivision produces smaller Hausdorff and mean surface distance errors than classic Loop and modified butterfly subdivision on the TOSCA evaluation.The evaluation trains on Centaur alone or on Centaur, David, and Horse, then tests on the remaining shapes.Source: §6 · p. 11, Table 2 · p. 11
  • Adding categories produced visually similar results in the reported experiment, while training on Centaur, David, and Horse slightly reduced error.The visual comparison uses a mixture of organic and non-organic shapes.Source: §6 · p. 11, Figure 28 · p. 11
  • Because the learned subdivision is nonlinear, a limit surface is not guaranteed; continuing with classic subdivision can ensure a smooth limit surface.The alternative applies neural subdivision at trained levels before switching to classic subdivision.Source: Figure 29 · p. 12
  • Using local geometry means the approach does not hallucinate semantic features.The paper illustrates this limitation with the Bratty Dragon example.Source: Figure 30 · p. 12

7 LIMITATIONS & FUTURE WORK

The paper identifies limits in mesh coverage, scale and limit-surface behavior, and the use of local geometry. It frames full super-resolution, detail hallucination, and surface stylization as future directions.

7 LIMITATIONS & FUTURE WORK

  • The framework is currently limited to triangle meshes; quadrilateral meshes and surfaces with boundaries are proposed extensions.These extensions are described as closer to real-world modeling scenarios.Source: §7 · p. 12
  • The method is not scale-invariant and does not guarantee a limit surface, both of which the authors identify as desirable properties.The paper suggests making the method scale-invariant and convergent to a limit surface.Source: §7 · p. 12
  • Local geometry limits semantic feature hallucination; incorporating global information is proposed as a possible way to address this.The paper also identifies wider neighborhoods and deeper subdivision as future directions.Source: §7 · p. 12
  • Robustness could be improved by training on data spanning a wider range of triangle aspect ratios and curvature information.The authors also suggest online learning with newly coarsened meshes.Source: §7 · p. 12
  • The paper presents fine-mesh reconstruction as a first step, with full super-resolution, detail hallucination, and surface stylization left for future work.Source: §7 · p. 12

A IMPLEMENTATION OF POINT CLOUD UPSAMPLING

The point-cloud upsampling comparison describes a sampling-and-reconstruction route and its practical trade-offs against mesh-based processing.

A IMPLEMENTATION OF POINT CLOUD UPSAMPLING

  • The point-cloud route samples a mesh, upsamples the points, then reconstructs a surface, but is expensive for interactive graphics and does not provide levels of detail or preserve textures.The number and placement of samples are additional design choices that influence results.Source: A IMPLEMENTATION OF POINT CLOUD… · p. 14
  • Sampling choices can change the result, and the lack of connectivity information can produce surface artifacts.The example compares uniform and farthest-point sampling before point-cloud upsampling and screened Poisson reconstruction.Source: A IMPLEMENTATION OF POINT CLOUD… · p. 14

B IMPLEMENTATION OF SUCCESSIVE SELF-PARAMETERIZATION

Successive self-parameterization builds a bijective map through local conformal flattenings during edge collapse. Collapse criteria screen geometric and UV validity, topology, and triangle quality; the comparison reports greater parameterization uniformity than MAPS on an uneven triangulation.

B IMPLEMENTATION OF SUCCESSIVE SELF-PARAMETERIZATION

  • Successive self-parameterization flattens the collapsing edge’s 1-ring, collapses it, then flattens the inserted vertex’s 1-ring with the prior boundary fixed, yielding a bijective map.Each flattening involves only a 1-ring when vertex valence is bounded.Source: B IMPLEMENTATION OF SUCCESSIVE… · p. 14

C CRITERIA FOR COLLAPSIBLE EDGES

  • Invalid collapses are skipped; checks prevent Euclidean normal flips and UV face flips, with δ = 0.2 sufficient to avoid Euclidean flips in the reported experiments.Source: C CRITERIA FOR COLLAPSIBLE EDGES · p. 14
  • UV overlaps are screened by checking whether each interior vertex’s total angle is 2π, alongside the UV-face signed-area check.Source: C CRITERIA FOR COLLAPSIBLE EDGES · p. 14
  • Collapse validity also requires the link condition and neighboring triangle quality Q > 0.2 in both UV and Euclidean domains.Q approaches 1 for equilateral triangles and 0 for skinny, degenerate ones.Source: C CRITERIA FOR COLLAPSIBLE EDGES · pp. 14–15

D COMPARISON TO [Lee et al. 1998]

  • On a very uneven triangulation, MAPS produces non-uniform parameterization, while successive self-parameterization with area-weighted qslim yields a more uniform one.MAPS is also described as sensitive to the input triangulation.Source: D COMPARISON TO [Lee et al. 1998] · p. 15

E DATA GENERATION FROM RANDOM COLLAPSES

The training data are generated from high-resolution meshes by producing varied coarse discretizations through semi-random edge collapses. This creates inputs with different vertex counts and triangulations.

E DATA GENERATION FROM RANDOM COLLAPSES

  • At equal vertex counts, Neural Subdivision creates a more uniform parameterization than MAPS, which is more sensitive to input triangulation.The comparison is for subdivision remeshing.Source: Figure 32 · p. 15
  • Semi-random edge collapses generate coarse meshes with varied vertex counts and triangulations to improve robustness across discretizations.The target vertex count is randomly selected between 150 and 300.Source: E DATA GENERATION FROM RANDOM COLLAPSES · p. 15
  • Each collapse selects the minimum-quadric-error edge from 100 randomly selected edges.The new vertex is inserted using the qslim procedure.Source: E DATA GENERATION FROM RANDOM COLLAPSES · p. 15

F EXPERIMENTAL SETUP

The experiments use a consistent training setup, generally training on two subdivision levels. A separate experiment finds that the network can reproduce classic Loop subdivision on novel meshes.

F EXPERIMENTAL SETUP

  • The standard setup uses 200 training discretizations and 700 epochs, with two subdivision levels for consistency across experiments.For multiple training shapes, discretizations are divided evenly while keeping the total at 200.Source: F EXPERIMENTAL SETUP · p. 16
  • The network can also be trained on more subdivision levels, although experiments generally use two.Figure 33 illustrates training on additional levels.Source: F EXPERIMENTAL SETUP · p. 16, Figure 33 · p. 16

G LEARNING CLASSIC LOOP SUBDIVISION

  • On novel meshes, the network reproduces Loop subdivision with visually indistinguishable results and 0.3% average per-vertex error relative to the bounding-box diagonal.Training uses Loop-subdivided sequences created from 200 randomly edge-collapsed meshes.Source: G LEARNING CLASSIC LOOP SUBDIVISION · p. 16, Figure 34 · p. 16

H ABLATION STUDIES (CONTINUED)

Ablations examine input features, loss design, displacement initialization, network depth, and feature choices. They report convergence or intermediate-level benefits for some choices, while other tested changes do not improve output quality.

H ABLATION STUDIES (CONTINUED)

  • Adding differential coordinates to the input can improve convergence.The study evaluates their influence on training convergence.Source: H ABLATION STUDIES (CONTINUED) · p. 16
  • Adding cross-level loss improves subdivision results at intermediate levels, which matters for meshes with different levels of detail.The comparison is against measuring loss only at the final level.Source: H ABLATION STUDIES (CONTINUED) · p. 16
  • Starting predicted displacements at edge midpoints or Loop-subdivided positions does not affect output quality, so the midpoint is chosen for simplicity.The comparison is reported in the Fig. 20 inset.Source: H ABLATION STUDIES (CONTINUED) · p. 16
  • Recursively repeating the vertex step does not improve results, so the method performs that step once.The authors suspect the resulting two-ring information may already suffice.Source: H ABLATION STUDIES (CONTINUED) · p. 16
  • Using edge vectors and differential-coordinate vectors converges to a better solution than the tested MeshCNN features.The tested MeshCNN features characterize undirected edges using flap geometry.Source: H ABLATION STUDIES (CONTINUED) · p. 16
Loading 2005.01819v1…