Source-linked AI summary
AtlasNet: A Papier-Mâché Approach to Learning 3D Surface Generation
Thibault Groueix, Matthew Fisher, Vladimir G. Kim, Bryan C. Russell, Mathieu Aubry
TL;DR
Generating high-resolution 3D surfaces remains difficult with voxel and point-cloud representations because they are limited in resolution or do not directly encode surface neighborhoods. AtlasNet learns a union of parametric surface elements from 2D squares and demonstrates benefits for reconstruction and related shape applications. Its main scope limitations are disconnected patches, synthetic shapes, and post-hoc mesh parameterization.
Problem
High-resolution 3D surface generation remains challenging because voxel grids are limited to voxel-scale sampling and point clouds do not directly represent neighborhood information.
Method
AtlasNet represents shapes as unions of learnable parametrizations that map 2D squares to 3D surface elements and optimizes them using sampled-point losses.
Results
AtlasNet outperformed existing baselines on 3D shape and single-view reconstruction and showed potential for interpolation, correspondence finding, and mesh parameterization.
Takeaways & Limitations
The learned atlas supports continuous surface generation, arbitrary-resolution sampling, tessellation and texture transfer, and several shape-analysis applications.
Takeaways & Limitations
Generated patches are not guaranteed to connect, experiments focus on synthetic object shapes, and mesh parameterizations are optimized post-hoc.
Abstract
from arXiv · showhide
We introduce a method for learning to generate the surface of 3D shapes. Our approach represents a 3D shape as a collection of parametric surface elements and, in contrast to methods generating voxel grids or point clouds, naturally infers a surface representation of the shape. Beyond its novelty, our new shape generation framework, AtlasNet, comes with significant advantages, such as improved precision and generalization capabilities, and the possibility to generate a shape of arbitrary resolution without memory issues. We demonstrate these benefits and compare to strong baselines on the ShapeNet benchmark for two applications: (i) auto-encoding shapes, and (ii) single-view reconstruction from a still image. We also provide results showing its potential for other applications, such as morphing, parametrization, super-resolution, matching, and co-segmentation.
1. Introduction
AtlasNet learns 3D surfaces directly as unions of learnable parametrizations, addressing limitations of voxel and point-cloud representations. The approach supports continuous, arbitrarily resolved surfaces and several downstream applications.
- Voxel representations provide only voxel-scale sampling, while point clouds do not directly encode neighborhood information needed to approximate smooth surfaces faithfully.
- Mapping 2D squares to 3D points favors continuous smooth 2-manifold structure and jointly provides a global UV surface parameterization.Regular meshes and texture atlases can be transferred or optimized through the learned parameterization.
- AtlasNet represents a shape as a union of learnable parametrizations that transform 2D squares into surface elements, analogous to papier-mâché strips.Transformation parameters come from both neural-network weights and a learned shape representation.
- The learned surface can adapt to geometric complexity, be sampled at any desired resolution, and transfer tessellations or texture maps.
- Experiments demonstrate high-resolution surface generation from point clouds and 2D images, alongside potential applications in interpolation, parameterization, and shape alignment.
2. Related work
Prior work represents 3D shapes with voxels, hierarchical volumetric structures, point-based methods, or parameterized meshes. AtlasNet instead learns parameterizations directly from point clouds without requiring predefined input correspondences.
- AtlasNet learns the surface parameterization directly from point clouds rather than requiring parameterized input data.
- Surface parameterization supports applications including texture mapping, remeshing, and shape correspondence.
- Voxel-based generation supports varied inputs but is memory-intensive and typically limited to coarse resolutions.Octrees and hierarchical part-level generation address this limitation by reducing volumetric costs.
- Parameterized mesh approaches use spherical or base-shape mappings, but require consistent correspondences between training shapes and those bases.Such correspondences become increasingly difficult for heterogeneous datasets.
3. Locally parameterized surface generation
AtlasNet learns local surface maps from 2D unit squares using ReLU MLPs, then combines multiple maps to approximate complex 3D surfaces. The formulation is trained using losses on sampled surface points.
- The learning objective fits a parametrization ϕθ from the unit square to a target local surface while regularizing its parameters.In practice, manifold losses are replaced by losses on sampled point sets, including Chamfer and Earth-Mover distance.
- ReLU MLPs are piecewise affine, and rank-2 local transformations yield neighborhoods that are 2-manifolds.
- A learnable atlas is the set of local functions used to approximate a 2-manifold, while MLPs can approximate any continuous surface in principle.
4. AtlasNet
AtlasNet decodes 3D surfaces from shape features using multiple learned parameterizations, supporting point-cloud and image inputs. Its surface-based representation enables direct, high-resolution mesh generation, while closed-mesh quality depends on how well the surface is represented by a sphere.
- Model: AtlasNet decodes a 3D surface from a shape encoding using multiple learnable parameterizations, with encodings supplied by point clouds or images.Each parameterization maps sampled points to surface points through an MLP.
- Model: The model concatenates each unit-square sample coordinate with the shape feature before passing it to the MLPs, whose outputs jointly cover the target shape.The parameterizations are not explicitly prevented from overlapping, but their union is intended to cover the full shape.
- Mesh generation: During inference, AtlasNet can generate meshes by transferring a regular mesh from the unit square to the learned 3D surface.The method typically uses 22500 points and can process points in batches for high-resolution meshes without memory issues.
- Mesh generation: A second mesh-generation route densely samples the learned surface and applies Poisson surface reconstruction to obtain a mesh.This route generates approximately 100000 points with oriented normals before reconstruction.
- Mesh generation: Direct closed-mesh generation samples inputs from a 3D sphere, but its quality depends on how well the underlying surface can be represented by a sphere.This avoids the Poisson surface reconstruction step.
5. Results
AtlasNet is evaluated for auto-encoding and single-view reconstruction on ShapeNet, with additional tests of generalization and surface-oriented applications. Across these experiments, its learnable surface representation improves reconstruction quality, supports high-resolution meshes, and produces meaningful interpolations and correspondences, while thin unseen structures remain challenging.
- Evaluation: AtlasNet is evaluated on ShapeNet for auto-encoding and single-view reconstruction, using Chamfer distance and Metro mesh distance alongside qualitative comparisons.The dataset contains 13 object categories, and Metro measures average Euclidean distance between output and ground-truth meshes.
- Auto-encoding 3D shapes: AtlasNet outperforms the Points baseline on both Chamfer distance and Metro criteria, even with a single learned parameterization.The comparison uses generated shapes from point clouds and averages results over ShapeNet categories.
- Auto-encoding 3D shapes: Increasing the number of learnable patches improves quantitative and visual quality, although many patches can introduce close-but-disconnected artifacts.The authors use 25 patches for single-view reconstruction experiments.
- Generalization: In leave-one-category-out tests, AtlasNet outperforms the point-generating baseline, and 125 patches outperform all baselines when generalizing to a new category.On cars, both AtlasNet variants clearly outperform the baseline when cars are excluded from training.
- Single-view reconstruction: For single-view reconstruction, AtlasNet directly generates triangular meshes whose surface details are more visible than in voxel, octree, or unstructured point-cloud outputs.PointSetGen can generate points inside the shape volume, whereas AtlasNet generates points on a surface by construction.
- Additional applications: AtlasNet also generates gradual shape interpolations, semantically meaningful correspondences, and texture-oriented parameterizations, though its inferred atlas has relatively high distortion.Reported average distortions are Ea = 1.9004 and Es = 6.1613, compared with 1 for an undistorted map.
6. Conclusion
The paper introduced AtlasNet, a learnable union of parametric surface elements for 3D shape generation. It demonstrated benefits for reconstruction and potential applications in interpolation, correspondences, and mesh parameterization.
- AtlasNet generates 3D surfaces as a union of learnable parametric surface elements.
- The method showed benefits for 3D shape reconstruction and single-view reconstruction compared with existing baselines.
- The approach showed potential for shape interpolation, finding shape correspondences, and mesh parameterization.
- The framework opens applications in generating and synthesizing meshes for 3D shapes.
7. Supplementary
The supplementary results examine AtlasNet’s reconstruction quality, resolution flexibility, correspondences, deformable-shape performance, and limitations. Across these experiments, AtlasNet produces fine or high-resolution surfaces and outperforms comparison methods in reported settings, while exposing trade-offs in parameterization count and patch connectivity.
- Limitations: Small parameterization sets cause excessive distortion, whereas many parameterizations increase reconstructed-mesh topology errors and can worsen Metro performance.With 25 versus 125 parameterizations, the 25-parameter model is better in Metro distance for 7 of 13 categories, but 125 is better on average because of cellphone results; this pattern does not hold for Chamfer distance.
- Single-view reconstruction: AtlasNet consistently produces better single-view reconstructions than PointSetGen and 3D-R2N2 across the shown categories.The comparison includes triangular meshes from AtlasNet, point clouds from PointSetGen, and voxel models from 3D-R2N2.
- Shape correspondences: AtlasNet transfers distance-based colors across inferred atlases to produce semantically meaningful correspondences without semantic supervision.Qualitative demonstrations cover planes and watercraft, with the reported behavior generalizing to all categories.
- Deformable shapes: On FAUST human shapes, AtlasNet is evaluated with Chamfer and Metro distances using 250 training shapes and 50 validation shapes without ground-truth correspondences.The supplementary experiment compares square-patch and sphere-based parameterizations.
- Super resolution: 122500-point meshes can be generated from inputs sampled at either the training resolution or one-tenth that resolution.AtlasNet obtains arbitrary output resolution by sampling more points, while PointNet accepts varying-resolution input point clouds.
- Quantitative comparison: AtlasNet with 25 learned parameterizations outperforms HSP in every category for both Chamfer and Metro distance after ICP alignment.The comparison uses 100 random test shapes per category and evaluates unnormalized meshes.