PaperlayerView exact revision on arXiv

Source-linked AI summary

Surface Representation for Point Clouds

Haoxi Ran, Jun Liu, Chengjie Wang

arXiv:2205.05740v2cs.CVcs.AIcs.GRcs.RO

TL;DR

Point-cloud coordinates alone do not directly describe local geometry, motivating a representation that captures local structure explicitly. RepSurf introduces triangular and umbrella surface features computed from geometric priors, and evaluations report superior performance and efficiency across classification, segmentation, and detection benchmarks. The authors identify noise-sensitive reconstruction and vulnerability to extremely messy points as limitations.

  • Problem

    Coordinates alone are insufficient to describe point-cloud local geometry directly, while existing approaches may represent local shape indirectly.

  • Method

    RepSurf explicitly represents local geometry with Triangular and Umbrella RepSurf, computed from reconstructed surfaces and designed as plug-and-play features for irregular point-cloud models.

  • Results

    RepSurf achieves superior performance and efficiency to previous state-of-the-art methods across classification, segmentation, and detection benchmarks.

  • Takeaways & Limitations

    RepSurf supports explicit local-geometry representation while retaining compatibility with irregular point-cloud models.

  • Takeaways & Limitations

    RepSurf may suffer from noise during kNN-based surface reconstruction and Umbrella RepSurf may be vulnerable to extremely messy points.

Abstract

from arXiv · show

Most prior work represents the shapes of point clouds by coordinates. However, it is insufficient to describe the local geometry directly. In this paper, we present \textbf{RepSurf} (representative surfaces), a novel representation of point clouds to \textbf{explicitly} depict the very local structure. We explore two variants of RepSurf, Triangular RepSurf and Umbrella RepSurf inspired by triangle meshes and umbrella curvature in computer graphics. We compute the representations of RepSurf by predefined geometric priors after surface reconstruction. RepSurf can be a plug-and-play module for most point cloud models thanks to its free collaboration with irregular points. Based on a simple baseline of PointNet++ (SSG version), Umbrella RepSurf surpasses the previous state-of-the-art by a large margin for classification, segmentation and detection on various benchmarks in terms of performance and efficiency. With an increase of around \textbf{0.008M} number of parameters, \textbf{0.04G} FLOPs, and \textbf{1.12ms} inference time, our method achieves \textbf{94.7\%} (+0.5\%) on ModelNet40, and \textbf{84.6\%} (+1.8\%) on ScanObjectNN for classification, while \textbf{74.3\%} (+0.8\%) mIoU on S3DIS 6-fold, and \textbf{70.0\%} (+1.6\%) mIoU on ScanNet for segmentation. For detection, previous state-of-the-art detector with our RepSurf obtains \textbf{71.2\%} (+2.1\%) mAP$\mathit{_{25}}$, \textbf{54.8\%} (+2.0\%) mAP$\mathit{_{50}}$ on ScanNetV2, and \textbf{64.9\%} (+1.9\%) mAP$\mathit{_{25}}$, \textbf{47.7\%} (+2.5\%) mAP$\mathit{_{50}}$ on SUN RGB-D. Our lightweight Triangular RepSurf performs its excellence on these benchmarks as well. The code is publicly available at \url{https://github.com/hancyran/RepSurf}.

1. Introduction

RepSurf addresses the difficulty of representing local geometry in irregular point clouds by explicitly augmenting coordinates with geometric surface features. Its triangular and umbrella variants are designed as efficient, plug-and-play representations for point-cloud models.

  • Motivation: Prior point-based methods learn from standalone points or indirectly capture local shapes through extra ingredients and transformations.These approaches may require complex preprocessing and significant computation while representing local shape implicitly.
  • RepSurf: RepSurf explicitly represents local point-cloud geometry using discrete, local, and curvature-sensitive features that collaborate with irregular points.The representation complements Cartesian coordinates with geometric information.
  • Triangular RepSurf: Triangular RepSurf reconstructs a triangle from each point and two neighbors, then uses its normal vector, surface position, and normalized coordinate as features.It is inspired by triangle meshes in computer graphics.
  • Umbrella RepSurf: Umbrella RepSurf expands the perceptive field by reconstructing an umbrella surface from K nearest neighbors ordered counterclockwise.It extends the triangular representation using K triangular features.
  • Contributions: RepSurf is presented as a high-efficiency plug-and-play module for point-cloud models.The paper identifies state-of-the-art performance across numerous point-cloud benchmarks as a contribution.

2. Related Work

Related work represents 3D data through transformed views, voxels, direct point processing, local aggregators, surface reconstruction, and curvature estimation. These approaches differ in how they preserve geometry, handle irregular points, and encode local structure.

  • Transformed representations: Multi-view and voxel-based methods transform 3D objects into images, lattice spaces, or voxels, but may incur computation and lose shape information.The stated information loss can result from occlusion or lower resolution.
  • Point-based methods: Point-based methods process raw point clouds directly, with PointNet using MLPs and max pooling and PointNet++ adding set abstraction and farthest point sampling.Later work explores local aggregation through convolutions, relations, and graphs.
  • 3D detection: 3D detection has progressed from convolution over 2D grids or 3D voxels toward detectors that process raw point clouds.VoteNet uses PointNet++ for feature extraction and groups points around voted centroids, while another method replaces grouping with Transformers.
  • Surface reconstruction: Surface-reconstruction methods include ball-pivoting triangles and spatial Poisson formulations for obtaining meshes from point clouds.These methods provide graphics-inspired ways to construct surfaces from point samples.
  • Curvature representation: Curvature-based methods estimate local geometry through least-square fitting or umbrella curvature computed from neighboring normals and directions.The cited umbrella-curvature approach constructs surfaces from homogeneous neighbors.

3. Surface Representation

RepSurf explicitly represents very local point-cloud geometry with geometric surfaces that retain explicit locality and curvature sensitivity while collaborating with irregular points. The paper develops lightweight Triangular RepSurf and broader-field Umbrella RepSurf, then integrates them into PointNet++.

  • Background and design goals: RepSurf is designed to represent local point-cloud geometry explicitly rather than learning it indirectly through added ingredients or transformations.Its stated properties are discreteness, explicit locality, and curvature sensitivity.
  • Triangular RepSurf: Triangular RepSurf uses two nearest neighbors to compute triangle centroids, cross-product normals, orientation handling, and surface positions.The implementation concatenates centroids, normals, and positions as the output representation.
  • Triangular RepSurf: Triangular RepSurf represents each point locally with a reconstructed triangle, its normal, and surface position, then replaces input points with triangle centroids.The representation is t_i = (a_i, b_i, c_i, p_i), with recomputed centroids X′ concatenated with triangular features T.
  • Umbrella RepSurf: Umbrella RepSurf expands the perceptive field of Triangular RepSurf to address unstable local representations caused by its limited field.It is inspired by umbrella curvature but avoids requiring given normals and can handle heterogeneous neighbors through position sensitivity.
  • Umbrella RepSurf: Umbrella surfaces are formed by sorting neighboring edges counterclockwise, constructing K adjacent triangles, and computing consistently oriented normals.The resulting features aggregate normalized coordinates, triangle information, and surface-related quantities through transformation and aggregation functions.

4. Experiments

RepSurf is evaluated across classification, segmentation, detection, and ablation studies, with Umbrella RepSurf improving accuracy over strong baselines while maintaining efficiency. Experiments also examine representation variants and design choices affecting performance and scalability.

  • Experimental Setup: RepSurf is evaluated on classification, segmentation, and detection benchmarks, with ablation studies assessing its designed modules.
  • Classification: 94.7% accuracy on ModelNet40 lets RepSurf-U surpass CurveNet by 0.5%, while training and inference are 5.4× and 4.0× faster.
  • Classification: 84.6% accuracy on ScanObjectNN surpasses MVTN by 1.8%, with around 1.8× fewer parameters and 1.2× fewer FLOPs.
  • Segmentation: 14.4% and 12.9% mIoU improvements over PointNet++ occur on S3DIS 6-fold and Area-5, while S3DIS 6-fold exceeds Point Transformer by 0.8% mIoU.
  • Segmentation: 1.6% higher mIoU than KPConv on ScanNet is achieved with 14.3× fewer parameters.

5. Discussion

RepSurf is evaluated across classification, segmentation, and detection, with results reported as efficient and superior to prior state of the art on different benchmarks. The authors also identify noise sensitivity during surface reconstruction and vulnerability to extremely messy points as limitations.

  • RepSurf may suffer from noise during surface reconstruction because the kNN algorithm is noise-sensitive.
  • Umbrella RepSurf may be vulnerable to extremely messy points, whose neighbor distributions can produce distorted surfaces when more neighbors are queried.
  • RepSurf variants are evaluated on shape classification, scene segmentation, and detection across various benchmarks.
  • The paper assumes Taylor Series can depict a local curve and extends this formulation to 3D space.

C. Details of Polar Auxiliary

Polar auxiliary augments Cartesian coordinates with spherical or cylindrical polar coordinates to distinguish neighboring points and express their relationship to a centroid. The coordinates are normalized for stable training and concatenated with Cartesian coordinates.

  • Two polar auxiliaries are provided: spherical and cylindrical coordinates based on their respective polar systems.
  • Spherical polar auxiliary maps a point to (ρs, θs, ϕs), with angular components normalized and radius commonly limited by the ball-query radius.
  • Cylindrical polar auxiliary maps a point to (ρc, θc, zc), with ϕs and zc normalized to [0, 1].
  • The implementation concatenates Cartesian coordinates with either spherical or cylindrical polar coordinates.
  • Polar angles can magnify differences between nearby neighbors, while ρs expresses a neighbor’s relationship to its centroid.

D. Details of Channel De-differentiation

Channel De-differentiation addresses distribution imbalance between mapped coordinates and high-level features in PointNet++ set-abstraction stages. It separately normalizes the two inputs before combining them, which the authors report can significantly boost point-based models.

  • Channel De-differentiation targets distribution imbalance between mapped coordinates and mapped last-stage features in each set-abstraction stage.
  • The imbalance may cause later MLP layers to ignore coordinate inputs because coordinates and high-level features have different distributions.
  • The first fused feature combines separate linear transformations of the point coordinate xi and high-level feature fi.
  • Channel De-differentiation applies separate batch normalization to the coordinate and feature branches before their combination.
  • The modification can significantly boost point-based models, including RepSurf inputs containing polar coordinates, RepSurf features, or RGB information.

E. Computation of FLOPs

The paper estimates FLOPs using common treatments for point-cloud operations and default THOP settings for basic operations, then complements theoretical complexity with practical speed measurements. Experiments use a fixed workstation and task-specific batch sizes.

  • FLOPs comparisons treat farthest point sampling, indexing, ball querying, and kNN querying identically across models.
  • Convolution, ReLU, and MLP operations use THOP’s default settings in the FLOPs calculation.
  • Tests run on one V100 GPU and four Intel Xeon CPU cores, with batch size 16 for classification and segmentation and 1 for detection.
  • The paper evaluates speed during both training and inference in addition to theoretical FLOPs.

G. Implementation details

RepSurf is implemented across classification, segmentation, and detection using PointNet++-based settings, with surface reconstruction preceding feature computation.

  • Classification: Classification uses Triangular and Umbrella RepSurf with PointNet++ SSG, Adam optimization, and dataset-specific augmentation settings.ModelNet uses random scale, shift, and dropout; ScanObjectNN uses no augmentation.
  • Segmentation: Segmentation uses RepSurf with PointNet++ SSG segmentation, SGD optimization, and augmentation on both S3DIS and ScanNet.Augmentations include scaling, color contrasting, color shifting, and color jittering.
  • Detection: Detection implements RepSurf on ScanNet V2 and SUN RGB-D following the practice of GroupFree.
  • Surface reconstruction: RepSurf reconstructs discrete surfaces before calculating Triangular and Umbrella RepSurf features.

H. Detailed Experimental Results

The detailed experiments report detection results by category and IoU threshold, while visualizations show RepSurf features computed after discrete surface reconstruction.

  • Detection: Detection results are detailed for ScanNet V2 and SUN RGB-D at mAP@0.25 and mAP@0.5.
  • Surface reconstruction: RepSurf visualizations follow discrete surface reconstruction before Triangular and Umbrella feature computation.

I.2. Geometry Sensitivity on Triangular RepSurf

Triangular RepSurf numerically reflects very local geometry: flat regions appear uniform, whereas edges produce clear changes.

  • Geometry sensitivity: Triangular RepSurf outputs numerically perceive local geometries in ScanObjectNN.
  • Visualization: Figure 10 visualizes three channels from the normal vectors of Triangular RepSurf.
  • Geometry sensitivity: Points on flat shapes have similar colors, while edge points show obvious color changes.

I.3. Geometry Sensitivity on Umbrella RepSurf

Umbrella RepSurf visualizations indicate sensitivity to local object geometry, including edges and planes, alongside category-level detection evaluations on two datasets.

  • Geometry sensitivity: Umbrella RepSurf recognizes local geometries, including object edges and planes.
  • Visualization: Figure 11 visualizes ten channels from Umbrella RepSurf.
  • Detection evaluation: ScanNet V2 detection is evaluated with per-category mAP@0.25 and mAP@0.5 results.
  • Detection evaluation: SUN RGB-D validation is evaluated with per-category mAP@0.25 and mAP@0.5 results.
Loading 2205.05740v2…