SAL: Sign Agnostic Learning of Shapes from Raw Data

Matan Atzmon; Yaron Lipman

Source-linked AI summary

SAL: Sign Agnostic Learning of Shapes from Raw Data

Matan Atzmon, Yaron Lipman

arXiv:1911.10414v2Nov 23, 2019cs.CVcs.GRcs.LG

TL;DR

Raw point clouds and triangle soups lack readily available signed or occupancy supervision, making implicit surface learning difficult. SAL trains signed implicit representations directly from unsigned data, supports reconstruction and shape-space learning, and is reported to produce high-quality results on raw geometric data. The paper also provides geometric initialization and plane-reproduction justification, while noting limitations for thin structures.

Problem
Raw point clouds and triangle soups do not readily provide the signed distance or occupancy samples required by many implicit-surface methods, and computing them is notoriously difficult.
Method
SAL uses a family of sign-agnostic losses to learn signed implicit representations directly from raw unsigned geometric data, with geometric initialization and plane-reproduction analysis.
Results
SAL reconstructs surfaces from point clouds and learns human shape spaces from raw scans, with the paper reporting high-quality reconstructions and shape-space interpolations.
Takeaways & Limitations
SAL allows shape learning from raw geometric data without ground-truth normals, inside/outside labeling, or preprocessing surface reconstruction.
Takeaways & Limitations
SAL’s main stated limitation is capturing thin structures, with missing parts reported in chair-back and plane-wheel reconstructions.

Abstract

from arXiv · show

Recently, neural networks have been used as implicit representations for surface reconstruction, modelling, learning, and generation. So far, training neural networks to be implicit representations of surfaces required training data sampled from a ground-truth signed implicit functions such as signed distance or occupancy functions, which are notoriously hard to compute. In this paper we introduce Sign Agnostic Learning (SAL), a deep learning approach for learning implicit shape representations directly from raw, unsigned geometric data, such as point clouds and triangle soups. We have tested SAL on the challenging problem of surface reconstruction from an un-oriented point cloud, as well as end-to-end human shape space learning directly from raw scans dataset, and achieved state of the art reconstructions compared to current approaches. We believe SAL opens the door to many geometric deep learning applications with real-world data, alleviating the usual painstaking, often manual pre-process.

1. Introduction

Neural implicit surface methods previously required difficult-to-compute signed or occupancy supervision, whereas SAL learns signed implicit representations directly from unsigned geometric data. The paper applies SAL to raw point clouds and scans, reporting high-quality reconstructions and shape-space learning.

Implicit neural surfaces represent geometry as zero level-sets of neural networks, benefiting from flexibility, approximation power, optimization, and generalization.
Existing regression-based implicit methods require samples from signed distance or occupancy functions, which are not readily available for raw point clouds or triangle soups.
SAL uses raw unsigned geometric data to produce signed implicit surface representations and can support generative models that learn shape spaces.
SAL overcomes imperfections and artifacts in raw data while providing high-quality surface reconstructions and latent shape-space interpolations.
Experiments cover point-cloud reconstruction and human shape-space learning from raw D-Faust scans, positioning SAL as a method for learning shapes without preprocessing surface reconstruction.

2. Previous work

Prior work represents surfaces parametrically or implicitly and uses diverse methods for shape learning and unsigned reconstruction. SAL differs by training implicit functions directly from raw unsigned data without ground-truth inside/outside information.

Neural parameteric surfaces: Parametric neural methods represent surfaces through mappings from R2 to R3, using atlases, geometry images, or global parameterizations.
Neural implicit surfaces: Implicit neural methods define surfaces as zero level-sets of functions from R3 to R, with representations ranging from grids and octrees to neural networks.
Shape representation learning: Shape representation learning has used GANs, auto-encoders, variational auto-encoders, and auto-decoders across voxel, chart, and implicit representations.
Signed surface reconstruction: Signed reconstruction methods commonly require normals or inside/outside information, using techniques such as radial basis functions and Poisson reconstruction.
Unsigned surface reconstruction: Unsigned reconstruction methods operate on point clouds or triangle soups using level sets, variational RBFs, orientation estimation, or offset surfaces.
Unsigned surface reconstruction: In the 2D SAL experiment, networks optimized with unsigned L0 and L2 losses produce zero level-sets that connect point-cloud samples into complete shapes.

3. Sign agnostic learning

SAL optimizes a neural implicit function so its absolute output matches an unsigned distance to raw data, while initialization and loss design favor a signed solution. The method has theoretical plane-reproduction support and requires only local point randomization for one loss.

SAL optimizes network weights so the zero level-set of f approximates raw geometric data X, such as a point cloud or triangle soup.
SAL is defined by a family of losses using a data distribution, an unsigned distance measure, and a differentiable unsigned similarity function.
The unsigned similarity τ(a,b)=||a|−b| creates a local minimum where the network’s absolute output approximates the unsigned distance while allowing a signed function.
Proper initialization favors the signed solution; in one dimension, the learned function can realize f(x;θ*)=x−x0 for a point x0.
Plane reproduction is theoretically important because surfaces have approximate tangent planes almost everywhere.
For L2, SAL samples from Gaussian neighborhoods around data points, while L0 uses Gaussian noise plus a point-mass distribution.
The L0 loss requires only randomizing points near data samples, avoiding further computations involving X and supporting large or complex geometric data.
2D example: The 2D experiment uses an eight-layer, 100-neuron-wide MLP with a middle skip connection and optimizes L0 and L2 variants.

4. Geometric network initialization

SAL introduces a geometrically motivated MLP initialization that approximates a signed distance function to a sphere. Increasing hidden-layer width improves the approximation while preserving the sphere-like zero level-set.

The initialization sets network parameters so f(x;θ0) approximately equals ϕ(||x||−r), the signed distance function to an r-radius sphere passed through ϕ.
Theorem 1 specifies how to choose the MLP’s initial parameters to achieve this sphere-distance approximation.
With hidden-layer widths of 100, 200, and 2000 neurons, the approximation improves as width increases while the sphere-like zero level-set remains topologically correct.
For a single-hidden-layer ReLU MLP with suitable zero biases, output weights, and Gaussian input weights, the network approximates signed distance to a centered sphere.

5. Properties

SAL's plane-reproduction property is motivated by the local planarity of surfaces and established first for linear models, then extended locally to general MLPs. The SAL loss approaches signed versions of unsigned distance functions, with different limiting forms for L0 and L2.

Surfaces are locally planar because they have an approximating tangent plane almost everywhere.
For data lying on a plane, the linear SAL model has critical weights whose zero level-set reproduces that plane.The weights are of the form (w*, b*) = (αn, αc) for some α ∈ R+.
The plane-reproduction theorem extends locally from the linear model to general multilayer perceptrons optimized with SAL.
SAL drives its neural implicit function toward a signed version of the unsigned distance function hX.
In the L0 case the limiting function is an inside/outside indicator, whereas in the L2 case it is a signed Euclidean distance to the data.These discontinuous limits can cause contouring quantization errors, though stopping criteria can avoid the issue; regularization is proposed as future work.

6. Experiments

SAL is evaluated for unsigned point-cloud reconstruction and raw-scan shape-space learning, including unseen humans and poses. It produces high-fidelity reconstructions, while latent optimization improves test-time approximations and thin structures remain a limitation.

6.1. Surface reconstruction: SAL reconstructs surfaces from un-oriented point clouds without normal information, including sparse and irregular inputs.The comparison includes ball-pivoting and variational-implicit reconstruction methods.
6.2. Learning shape space from raw scans: The D-Faust experiment uses approximately 41k raw triangle-soup scans of 10 humans containing holes, ghost geometry, and noise.Evaluation uses a random 75%-25% train-test split and Chamfer distances to raw scans and ground-truth registrations.
6.2. Learning shape space from raw scans: SAL learns shape representations with a modified variational encoder-decoder whose PointNet encoder maps sampled scan point clouds to latent variables.The decoder is an implicit neural representation conditioned on a latent vector.
6.2. Learning shape space from raw scans: The SAL loss combines unsigned distances to triangle soups with variational regularization of the latent representation.The latent regularization uses a balancing weight λ = 10^-3.
6.2. Learning shape space from raw scans: Latent optimization further improves reconstruction quality over a single forward pass for unseen humans and poses.The unseen-human experiment trains on 8 humans and tests on 2 held-out humans; pose prediction also improves in shape and pose.
6.2. Learning shape space from raw scans: SAL’s main limitation is capturing thin structures, with parts of a chair back and plane wheel structure missing.The failure is illustrated on ShapeNet reconstructions.

7. Conclusions

The paper concludes that SAL processes raw geometric data without preprocessing, normal data, or inside/outside labels. It demonstrates high-fidelity reconstruction and shape-space learning, while identifying further generative, partial-data, and image-data applications.

7. Conclusions: SAL processes raw data without preprocessing or ground-truth normal data and inside/outside labeling.The method is presented as Sign Agnostic Learning for raw geometric data.
7. Conclusions: The paper demonstrates high-fidelity surface reconstruction from raw point clouds and integration with generative models for learning shape spaces.The conclusion also reports geometric initialization for sphere signed-distance approximation and theoretical planar reproduction.
7. Conclusions: Proposed future directions include generative adversarial models, global reconstruction from partial data, and combining SAL with image data.These directions are framed as applications for geometric deep learning with unorganized raw data.

8.1. Implementation details

The implementation samples Gaussian perturbations around scan points and uses multilayer perceptrons for reconstruction, with an encoder-decoder architecture for D-Faust shape-space learning. Training uses ADAM with task-specific schedules and hardware settings.

Data Preparation: Training data for point clouds uses two Gaussian variables centered at each input point, with scales based on local and global point-cloud distances.The first standard deviation uses the distance to the 50-th closest point; the second uses the furthest-point distance.
Network Architecture: The surface-reconstruction network is an 8-layer MLP with a fourth-layer skip connection concatenating the input point with hidden features.The hidden representation is 509-dimensional and concatenated with the 3D point to form 512 dimensions.
Training Details: Surface reconstruction training uses ADAM with learning rate 0.0001 for 5000 epochs on an Nvidia V-100 GPU.The implementation uses the PyTorch framework.
Data Preparation: For D-Faust, 500K points are sampled per scan, including uniform triangle-soup samples and Gaussian perturbations with prescribed scales.Unsigned distances are calculated to the closest triangle for each sample point.
Network Architecture: The D-Faust model uses PointNet and DeepSets layers in an encoder-decoder architecture, with a decoder conditioned on a 256-dimensional latent encoding and a 3D spatial point.The decoder input is the concatenation [z, x] in R^259.
Training Details: D-Faust training uses ADAM with initial learning rate 0.0005, batch size 64, halving every 500 epochs, and stopping after 2000 epochs on four Nvidia V-100 GPUs.The implementation uses PyTorch.

8.2. Additional Experiments

These experiments compare SAL single-scan reconstruction with VAE-based shape-space learning and examine reconstruction quality throughout training. VAE learning improves results on registered scans, while performance is comparable on original scans.

Single reconstruction versus VAE reconstruction: SAL supports both single-input surface reconstruction and VAE-based shape-space learning from an entire scan dataset.The experiment evaluates whether learning a shape space affects reconstruction quality.
Single reconstruction versus VAE reconstruction: VAE training performs better than SAL single reconstruction on registrations, while results on original scans are comparable.For the 5%, 50%, and 95% percentiles, single reconstruction scores are 0.10, 0.17, and 0.22 on registrations versus 0.07, 0.08, and 0.10 on scans.

8.3. Proofs

The proofs establish that SAL has plane-reproduction properties for linear models and general MLPs, and that suitable initialization approximates signed distance to a sphere. These results provide theoretical support for local surface reconstruction behavior.

Theorem 3: If the data lies on a plane, a linear model with a strong non-linearity has a critical loss point whose zero level-set reconstructs that plane.The critical parameters have the form (w*, b*) = (αn, αc) for α ∈ R+.
Theorem 3: The proof uses reflection symmetry across the plane, reducing the gradient analysis to an integral whose sign changes with a scaling parameter.The intermediate value theorem establishes α− and α+ with opposite signs, yielding a zero of the relevant scalar function.
Theorem 4: For an MLP, if the data locally lies on a plane and the network is locally linear before its final non-linearity, a critical point reconstructs that plane locally.The result requires the data distribution to be sufficiently concentrated around points in the plane.
Theorem 4: The MLP proof constructs parameters whose local affine behavior matches the plane, then scales the associated weights and bias to make the full loss gradient vanish.The resulting parameter vector θ* is critical for the loss.
Theorem 2: A single-hidden-layer ReLU MLP with zero biases, Gaussian weights, and specified output scaling approximates f(x) ≈ ∥x∥ − r, the signed distance to a centered sphere.The approximation follows from the law of large numbers and rotational invariance of the Gaussian distribution.