PaperlayerView exact revision on arXiv

Source-linked AI summary

MeshUDF: Fast and Differentiable Meshing of Unsigned Distance Field Networks

Benoit Guillard, Federico Stella, Pascal Fua

arXiv:2111.14549v4cs.CV

TL;DR

Non-watertight surfaces can be represented by UDFs, but existing meshing approaches are slow or reduce accuracy. MeshUDF adapts marching cubes using gradient-based reasoning and differentiable vertex positions. The paper reports better reconstruction accuracy than current deep-learning approaches in a fraction of the computation time.

  • Problem

    UDFs represent non-watertight surfaces, but fast direct meshing is unavailable, leaving slow point-cloud triangulation or accuracy-reducing thin-shell alternatives.

    Source: §1 · p. 2

  • Method

    MeshUDF uses gradient orientations to assign pseudo-signs for marching cubes and differentiates mesh-vertex positions with respect to latent vectors.

    Source: §1 · p. 2, Surface Detection within Cells · p. 4

  • Results

    MeshUDF achieves better reconstruction accuracy than current deep-learning approaches to non-watertight surfaces in a fraction of the computation time.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Differentiable triangulation gives non-watertight UDF surfaces an explicit mesh parameterization that can be integrated into end-to-end differentiable pipelines.

    Source: §5 · p. 14

    With small-capacity neural networks, poorly learned UDFs may fail to reach zero or approximate gradients poorly, producing artifacts such as single-cell holes at high resolution.

    Source: §4.6 · p. 14

Abstract

from arXiv · show

Unsigned Distance Fields (UDFs) can be used to represent non-watertight surfaces. However, current approaches to converting them into explicit meshes tend to either be expensive or to degrade the accuracy. Here, we extend the marching cube algorithm to handle UDFs, both fast and accurately. Moreover, our approach to surface extraction is differentiable, which is key to using pretrained UDF networks to fit sparse data.

1 Introduction

Existing implicit-surface methods handle watertight surfaces, while modeling non-watertight surfaces with UDFs lacks fast direct meshing. MeshUDF extends marching cubes using gradients and differentiates mesh vertices, achieving more accurate reconstructions than current deep-learning approaches in a fraction of their computation time.

1 Introduction

  • Watertight-only implicit methods can require time-consuming dataset preprocessing that discards useful inner components, such as car seats.An alternative—learning from raw data with network initialization or regularization—significantly slows training.Source: §1 · p. 2
  • UDF-based methods lack fast direct meshing, so they extract dense point clouds for slow triangulation or mesh a positive-level set that reduces accuracy.Thin watertight surfaces cannot be made arbitrarily thin, and some applications require single-faced surfaces.Source: §1 · p. 2
  • MeshUDF replaces marching cubes’ sign flips with gradient-based reasoning to create mesh vertices and facets from UDFs.Oppositely oriented neighboring gradients indicate that a surface element should be inserted between them.Source: §1 · p. 2
  • Differentiating mesh-vertex positions with respect to latent vectors lets pretrained UDF networks fit sparse 3D points or object silhouettes.The paper uses this capability to fit partial observations via gradient descent.Source: §1 · p. 2
  • MeshUDF reports better reconstruction accuracy than current deep-learning approaches for non-watertight surfaces, in a fraction of the computation time.The paper attributes this comparison to its approach to meshing UDFs and parameterizing differentiable meshes.Source: §1 · p. 2

2 Related Work

Related work covers classic marching cubes and alternatives for triangulating implicit fields, then reviews approaches to non-watertight surfaces. Existing options either require oriented normals, sacrifice accuracy, or produce point clouds that are slow to mesh.

2 Related Work

  • Classic marching cubes triangulates a scalar field’s 0-isosurface using sign changes at grid corners, lookup-table facets, and linear interpolation.Although newer dual methods can better capture sharp edges, they add complexity and require richer input.Source: §2 · p. 3
  • Gradient-based backpropagation through marching-cubes surface points requires unambiguous normal orientation, making it impractical for non-watertight surfaces.The original algorithm remains widely used for its simplicity, flexibility, and efficient implementations.Source: §2 · p. 3
  • Prior non-watertight methods either inflate a thin watertight surface, reducing accuracy, or sample UDF level sets into point clouds that are slowly meshed sequentially.NDF and AnchorUDF project and resample points, then use ball-pivoting to connect them.Source: §2 · p. 3

3 Method

MeshUDF extracts triangulated meshes from UDF zero-level sets and makes their vertex coordinates differentiable with respect to shape codes or network parameters. It adapts marching cubes with gradient-based pseudo-signs and derives gradients from nearby level-set values, including at surface borders.

3 Method

  • The method maps a parameter vector and 3D query point to an unsigned distance, then differentiates mesh vertices with respect to the parameters.The parameter vector may contain a latent code, network parameters, or both; the method creates a mesh from the UDF’s 0-level set.Source: §3 · p. 4

3.1 From UDF to Triangulated Mesh

  • Opposing gradient orientations at cell corners provide pseudo-signs that let marching cubes reconstruct a surface without UDF sign flips.The anchor corner is assigned a positive pseudo-signed distance; corners with gradients opposite to its gradient are assigned negative values. Breadth-first exploration stores corner values for consistency, and neighboring cells vote to estimate signs more robustly.Source: Surface Detection within Cells · pp. 4–5
  • The method prunes spurious faces by reevaluating UDF values and smooths jagged borders with Laplacian smoothing on single-triangle edges.A face is eliminated if any vertex’s distance-field value exceeds half the cubic-cell side length.Source: Figure 3 · p. 5, Surface Detection within Cells · p. 6

3.2 Differentiating through Iso-Surface Extraction

  • For points inside a facet, nearby points on opposite sides of the 0-level set provide UDF gradients for local surface-deformation derivatives.The method uses points offset along the facet normal; increasing the UDF on one side and decreasing it on the other moves the surface normally.Source: §3.2 · pp. 6–7
  • At border edges, the method samples the α-level set outward within the facet plane, allowing gradients to represent shrinking or extending the surface.Increasing the UDF at the outward sample moves the border inward; decreasing it extends the surface outward.Source: §3.2 · p. 7

4 Experiments

The experiments evaluate UDF meshing on learned garment and ShapeNet car fields, then test differentiable fitting, topology change, and use with other UDF networks. Results show accurate, fast meshing and useful fitting, alongside limitations from approximate fields and marching-cubes cases.

4.1 Network and Metrics

  • The evaluation trains a 128-dimensional latent-to-UDF network on 300 of 328 MGN meshes and tests on the remaining 28; comparison uses pretrained NDF, trained on raw ShapeNet meshes.Source: §4.1 · p. 8

4.2 Mesh Quality and Triangulation Speed

  • Ours is slightly more accurate than NDF on all three garment metrics and orders of magnitude faster; on cars the pattern persists except Inflation has slightly better NC.At very high resolution, Inflation approaches Ours’ accuracy at substantially higher computational cost.Source: §4.2 · p. 9, Table 1 · p. 9

4.3 Using Differentiability to Fit Sparse Data

  • Using the differentiable mesh loss yields the best results for fitting sparse 3D points and silhouettes among the compared losses.For point fitting, the comparison includes implicit-domain losses; for silhouettes, it compares mesh-rendering and UDF-domain losses.Source: Table 2 · p. 10, §4.3 · pp. 11–12, Table 3 · p. 11
  • Removing the border derivative term reduces performance, supporting its role in allowing garment shrinkage and expansion.The ablation compares the full normals-plus-border gradients with gradients computed everywhere using Eq. 5.Source: §4.3 · p. 12, Table 4 · p. 12

4.4 Differentiable Topology Change

  • Starting from a pair-of-pants latent code, gradient descent approximates a sweater from 10k target points despite topology changes during optimization.The optimization minimizes CHD, and the intermediate latent codes can correspond to invalid garments.Source: §4.4 · pp. 12–13, Figure 8 · p. 13

4.5 Generalization to other UDF Networks

  • The meshing procedure directly triangulates UDFs from AnchorUDF and NDF, replacing downstream ball pivoting while preserving differentiability.The authors report that this takes a fraction of the time and suits end-to-end differentiable pipelines.Source: §4.5 · p. 13, Figure 9 · p. 13

4.6 Limitations

  • Small-capacity networks can fail to learn a sharp zero-valued surface, causing artifacts such as single-cell holes at high resolution.The authors distinguish these approximate learned-field artifacts from meshing a real UDF.Source: §4.6 · p. 14
  • The method still needs high-resolution grids for high-frequency detail, with cubic scaling, and may produce holes on non-orientable surfaces.Marching-cubes cases are also missing for some surface borders or intersections; the method lacks geometric topology guarantees because UDFs have no inside/outside notion.Source: §4.6 · p. 14

5 Conclusion

The paper concludes that UDF-based non-watertight surfaces can be triangulated effectively and differentiably. This provides explicit meshes for end-to-end differentiable pipelines while retaining the ability to represent different topologies from one latent space.

5 Conclusion

  • Differentiable triangulation provides explicit meshes for end-to-end pipelines while retaining implicit representations’ ability to express different topologies from one latent space.The authors identify joint optimization of pose and clothing for people in loose attire as future work.Source: §5 · p. 14

Supplementary Material

The supplementary material expands on garment UDF training and evaluation, differentiable surface extraction and its limitations, meshing artifacts, pseudo-sign voting, and additional UDF meshing results. It also reports comparisons of BFE and PSD and optimization from random latent codes.

5.6 Fitting from Random Latent Codes

  • The supplementary material reports sparse-point fitting from random latent-code initialization and compares it with same-type initialization and alternative optimization routes.The table caption lists Chamfer distance, image consistency, and normal consistency for initial and optimized codes, but the supplied passages do not provide its values.Source: Supplementary Material · p. 18, Table 6 · p. 26

5.1 Network Training

  • Training uses 30,000 samples per garment, with UDF values clamped at 0.1; unsigned distances can be computed directly from raw triangle soups without mesh preprocessing.Samples comprise 6,000 surface points, 12,000 within distance 0.05, 8,000 within distance 0.3, and 4,000 in the bounding box.Source: §5.1 · p. 18

5.2 Metrics

  • Evaluation defines Chamfer distance between surface-point sets, image consistency from silhouette IoU and normal-map cosine similarity across eight views, and normal consistency from nearest-point normal similarities.Image consistency averages the product of silhouette IoU and normal-map cosine similarity over the eight cameras.Source: §5.2 · pp. 18–19

5.3 Differentiating through Iso-Surface Extraction

  • The latent-code gradient for a surface point is approximated from UDF derivatives at two points offset by α along opposite normal directions, paired around the surface point.This approximation assumes the offset points lie on the α-levelset and the surface point is their closest point on the zero-levelset; it is most appropriate when α is small relative to curvature. The paper reports α = 0.01 works across a wide range of shapes.Source: §5.3 · pp. 20–21
  • For border vertices, the method pairs each vertex with an outer α-levelset point and uses a tangent-plane direction perpendicular to the border, oriented by evaluating the UDF both ways.The network weights are frozen so optimization changes the latent code and moves the UDF zero-level set.Source: §5.3 · pp. 21–22

5.4 Meshing approximate or real UDFs

  • At resolution 512, meshing an approximate shallow-network UDF produced staircase artifacts, whereas meshing the exact UDF grid produced a smooth surface.The authors attribute the artifacts to an inflated zero-level set with many near-surface grid locations evaluating to zero, disrupting Marching Cubes interpolation.Source: §5.4 · pp. 22–23, Figure 13 · p. 22

5.5 Ablation study: pseudo-sign and breadth-first exploration

  • Breadth-first exploration reuses pseudo-signs across adjacent cells, while voting avoids arbitrary anchors and diagonal votes and handles opposing edge gradients that indicate a local field maximum.Ambiguous corners with vote sums below cos(π/4) are deferred to a lower-priority queue for additional votes; cells with multiple non-adjacent facets are explored last.Source: §5.5 · pp. 23–24
  • BFE yields consistently fewer excess holes and slightly lower Chamfer distance than PSD, except at resolution 64, while producing consistent facet orientations and closing most holes.Excess holes increase with resolution for both methods; the reported comparison uses the same garment network and dataset.Source: Table 5 · p. 24, §5.5 · pp. 25–26
  • BFE is slightly faster than PSD because breadth-first exploration reduces dot products: PSD computes 8 per cell, while BFE corners receive votes from at most 6 neighbors.Most corners receive fewer than six votes because exploration starts from one cell and proceeds breadth-first.Source: §5.5 · p. 26

5.6 Optimization from random initial latent codes

  • Starting from random latent codes, the approach succeeds better than direct supervision on UDF values, so initialization from a similar semantic class is not required.Same-category codes remain plausible when supplied by a regressor.Source: §5.6 · p. 26

5.7 Additional results

  • Additional results mesh NDF outputs from sparse point clouds, AnchorUDF outputs from images, and garment reconstructions compared with the ε-inflation baseline.For NDF, triangulation takes less than 10 seconds versus more than 2 hours for ball pivoting; Fig. 18 shows the baseline yields wrongly inflated meshes.Source: §5.7 · p. 27, Figure 16 · p. 27, Figure 17 · p. 28

Related papers

Loading 2111.14549v4…