PaperlayerView exact revision on arXiv

Source-linked AI summary

A Closer Look at Local Aggregation Operators in Point Cloud Analysis

Ze Liu, Han Hu, Yue Cao, Zheng Zhang, Xin Tong

arXiv:2007.01294v1cs.CVcs.LG

TL;DR

The paper addresses the lack of fair evidence about how local aggregation operators affect point-cloud networks. It evaluates representative operators in a shared deep residual setting and proposes weight-free PosPool, finding similar performance across operators and strong benchmark results.

  • Problem

    Local aggregation operators are difficult to compare because prior studies use different network architectures and implementation details, while many operators are evaluated only in shallow networks.

  • Method

    The paper revisits representative operators using a common deep residual architecture, shared implementation details, and three benchmarks, then introduces weight-free PosPool.

  • Results

    Representative operators achieve similar performance under matched representation capacity and appropriate settings, while PosPool attains 53.8 part category mIoU on PartNet and exceeds the previous best by 7.4 mIoU.

  • Takeaways & Limitations

    The findings suggest that sophisticated or heavy local aggregation computation may not be necessary for strong point-cloud modeling performance.

  • Takeaways & Limitations

    The PosPool variant omits the encoding function ∆xij∆yij∆zij to ensure g = 18 and make fj’s channel number C divisible by g.

Abstract

from arXiv · show

Recent advances of network architecture for point cloud processing are mainly driven by new designs of local aggregation operators. However, the impact of these operators to network performance is not carefully investigated due to different overall network architecture and implementation details in each solution. Meanwhile, most of operators are only applied in shallow architectures. In this paper, we revisit the representative local aggregation operators and study their performance using the same deep residual architecture. Our investigation reveals that despite the different designs of these operators, all of these operators make surprisingly similar contributions to the network performance under the same network input and feature numbers and result in the state-of-the-art accuracy on standard benchmarks. This finding stimulate us to rethink the necessity of sophisticated design of local aggregation operator for point cloud processing. To this end, we propose a simple local aggregation operator without learnable weights, named Position Pooling (PosPool), which performs similarly or slightly better than existing sophisticated operators. In particular, a simple deep residual network with PosPool layers achieves outstanding performance on all benchmarks, which outperforms the previous state-of-the methods on the challenging PartNet datasets by a large margin (7.4 mIoU). The code is publicly available at https://github.com/zeliu98/CloserLook3D

1 Introduction

The paper argues that local aggregation operators have not been fairly compared because surrounding architectures and implementation details differ. Using a common deep residual testbed, it finds similar performance across operators and introduces PosPool as a simple alternative.

  • Background: Point-cloud networks combine point-wise transformation layers with local aggregation layers that process neighborhood features and relative positions.Local aggregation plays a role analogous to convolution for image pixels.
  • Motivation: Different local aggregation operators have not been carefully compared because network architectures and implementation details vary across studies.Differences include depth, width, residual connections, point sampling, and neighborhood computation.
  • Approach: The common evaluation uses a deep residual architecture with shared implementation details and the ModelNet40, S3DIS, and PartNet benchmarks.These datasets represent different tasks, scenarios, and data scales.
  • Findings: Different representative operators perform similarly under the same representation capacity when each receives appropriate settings.The paper also reports that settings effective in deep residual networks can differ from those used in shallower networks.
  • PosPool: PosPool combines neighbor features with 3-d relative coordinates by element-wise multiplication and then applies average pooling without learnable weights.The operator performs as well as or better than sophisticated operators across three datasets.
  • Results: 53.8 part category mIoU on PartNet exceeds the previous best method by 7.4 mIoU using a simple deep residual network with PosPool layers.The same system achieves state-of-the-art performance on three representative benchmarks.

3 Overview of Local Aggregation Operators

The paper formulates local aggregation as transforming neighborhood features and relative positions before reduction, then categorizes existing operators by their transformation function. The categories differ in how they combine features and geometry, with distinct computational costs.

  • General Formulation: A local aggregation layer transforms each neighbor feature and relative location, then reduces the transformed neighborhood features into the center point output.The neighborhood is denoted N(i), and reduction commonly uses MAX, AVG, or SUM.
  • Categories: Existing operators are categorized as point-wise MLP based, pseudo grid feature based, or adaptive weight based.The categorization follows the family of the transformation function G(·, ·).
  • Point-wise MLP based: Point-wise MLP methods concatenate relative positions and point features, process neighborhood inputs with point-wise layers, and usually apply MAX reduction.Their flexibility comes with substantial computation because fully connected layers process all neighboring points.
  • Pseudo Grid Feature based: Pseudo grid methods generate features at sampled regular grid points so that regular convolution can be applied.KPConv is a representative method using spherical grid points and depth-wise convolution weights indexed by relative position.
  • Complexity: For pseudo grid methods with depth-wise convolution, space complexity is O(space) = dM and time complexity is O(time) = ndKM.M is the number of grid points.
  • Adaptive Weight based: Adaptive weight methods compute aggregation weights from arbitrary relative positions and combine them with neighbor features using element-wise multiplication and typically SUM reduction.Variants may additionally use position variables, point density, edge features, Taylor approximation, or SoftMax normalization.
  • Complexity: For adaptive weight methods with h ≥2 fully connected layers, time complexity is O(time) = ((h −2)d/2 + d + 5) · d/2 · nK.When h = 1, space and computation complexity decrease to O(space) = 3d and O(time) = 5dnK.

4 Benchmarking Local Aggregation Operators in Common Deep Architecture

The paper benchmarks representative local aggregation operators under a shared deep residual architecture and implementation, finding that appropriately configured methods perform similarly across three benchmarks. Simpler settings and PosPool challenge the need for sophisticated aggregation designs.

  • Common Experimental Settings: A shared 5-stage deep residual architecture standardizes comparisons across local aggregation operators, point sampling, neighborhood selection, and model capacity.Capacity varies through network depth, width, and bottleneck ratio.
  • Common Experimental Settings: The evaluation covers ModelNet40, S3DIS, and PartNet, spanning classification and semantic segmentation across CAD models and real scenes.The datasets also differ in training-data scale and scenario complexity.
  • Performance of Two Baseline Operators: Baseline operators mostly perform marginally worse than previous best methods, while MAX pooling reaches 47.4 mIoU with 6.7G FLOPs versus 46.4 mIoU with 23.1G FLOPs.This baseline uses no relative-position encoding.
  • Point-wise MLP based Method: Using 1 FC layer without non-linearity outperforms 2 or 3 FC layers on S3DIS and remains competitive on ModelNet40 and PartNet.The authors associate the simpler design with easier optimization and lower computation.
  • Adaptive Weight based Method: For adaptive-weight methods, relative positions alone with 1 FC layer and no SoftMax normalization achieve the best performance and approach or surpass state-of-the-art results on all three datasets.MAX and AVG are slightly better than SUM, while SoftMax normalization significantly reduces accuracy.
  • Discussions: All three operator types achieve state-of-the-art performance under the same deep residual architecture, with smaller configurations remaining competitive at 16× fewer parameters and computations.These results favor relative position alone and one FC layer in most scenarios, motivating simpler aggregation such as PosPool.

5 PosPool: An Extremely Simple Local Aggregation Operator

PosPool is a simple local aggregation operator without learnable weights that combines relative positions with neighboring features before average pooling. Its variant retains this property and can slightly improve performance in some scenarios.

  • Operator design: PosPool has no learnable weights and combines each neighbor’s relative position with its point feature by element-wise multiplication.Because coordinates have three dimensions while features have d dimensions, multiplication is applied group-wise across feature channels.
  • Operator design: The operator applies average pooling after position-feature combination to abstract information from the neighborhood.The supplied passages describe the multiplication and pooling design, while the displayed equation fragments provide the operator notation.
  • Operator properties: PosPool preserves permutation and translation invariance, properties favorable for point cloud analysis.The paper names these invariance properties as retained by the operator.
  • Variant: A coordinate-embedding variant uses cosine/sine functions before element-wise multiplication with point features, while remaining weight-free.The embedding matches the feature dimension and is concatenated from six-dimensional coordinate-function groups.
  • Variant: The variant performs slightly better than direct multiplication in some scenarios, while both operators avoid learnable weights.The paper also reports zero space complexity for learnable weights and time complexity O(time) = ndK for the operator.

6 Experiments

The experiments evaluate operators across three benchmarks and varying model capacities using a shared evaluation setup. PosPool is consistently competitive, whereas some sophisticated operators are less stable as capacity changes.

  • 6.1 Benchmark Settings: The study uses ModelNet40, S3DIS, and PartNet, spanning classification, semantic segmentation, CAD models, and real scenes.The benchmarks also differ in training-data scale and scenario.
  • 6.1 Benchmark Settings: PartNet contains 26,671 pre-sampled 3D object models across 24 categories, averaging 18 parts per object.The experiments use official training, validation, and test splits, with comparison on 17 fine-grained categories.
  • 6.2 Comparing Operators with Varying Architecture Capacity: The comparison varies network width, depth, and bottleneck ratio across the three benchmarks.Figure 3 reports accuracy under these architecture-capacity changes.
  • 6.2 Comparing Operators with Varying Architecture Capacity: PosPool achieves top or near-top performance across datasets and varying network hyperparameters, showing strong stability and adaptability.Other sophisticated operators can match it in some settings but are less stable across scenarios and capacities.
  • 6.2 Comparing Operators with Varying Architecture Capacity: AdaptWeight accuracy drops significantly on S3DIS when width, depth, or bottleneck ratio reduces model capacity.This example illustrates the reported instability of some sophisticated operators.

7 Conclusion

The conclusion presents a common-testbed study showing similar operator performance under appropriate settings and introduces PosPool as a simpler alternative. Activation visualizations also suggest that different operators learn similar activations for the same input.

  • 7 Conclusion: A deep residual architecture and three representative benchmarks provide the common testbed for studying existing local aggregation operators.The study evaluates operators under shared experimental conditions.
  • 7 Conclusion: With appropriate settings, all evaluated operators achieve state-of-the-art performance on three tasks.The conclusion frames this as the central finding of the comparative investigation.
  • 7 Conclusion: PosPool performs as well as existing operators with sophisticated designs while using no learned weights.The operator is presented as an extremely simple alternative motivated by the comparative findings.
  • 7 Conclusion: Activation maps before prediction indicate that different operators tend to produce similar activations for the same input point cloud.Figure 4 visualizes these activation maps on PartNet validation shapes.

A1 Training/Inference Settings

Training and inference settings differ by benchmark but use specified optimizers, augmentation procedures, training schedules, and inference aggregation schemes. ModelNet40 and S3DIS use SGD, while PartNet uses AdamW.

  • Settings overview: The section documents training and inference settings separately for each dataset.These settings include optimization, augmentation, duration, and prediction aggregation choices.
  • ModelNet40: ModelNet40 training uses SGD for 600 epochs with anisotropic scaling and Gaussian-noise augmentation.Inference uses the final-epoch model and averages logits over 100 augmented versions.
  • S3DIS: S3DIS training uses SGD for 600 epochs with scaling, Gaussian noise, z-axis rotations, and random color dropping.Inference averages each point’s logits across overlapping spheres.
  • PartNet: PartNet training uses AdamW for 300 epochs with anisotropic scaling and Gaussian-noise augmentation.Inference uses a 10-augmentation voting scheme to compute point probabilities.

A2 Detailed Experimental Settings for Section 6.2

The experiments vary architecture width, depth, and bottleneck ratios under controlled settings, while specifying each operator’s reduction and implementation choices.

  • Architecture variations: Width experiments fix depth at Nr + 1 = 1 and bottleneck ratio at γ = 2.
  • Architecture variations: Depth experiments fix width at C = 36 and bottleneck ratio at γ = 2.
  • Architecture variations: Bottleneck-ratio experiments fix width at C = 144 and depth at Nr + 1 = 1.
  • Operator settings: PointMLP uses {∆pij, fi, ∆fij}, MAX pooling, and one fully connected layer.
  • Operator settings: PseudoGrid uses SUM reduction and 15 grid points, while AdaptWeight uses {∆pij}, AVG pooling, and one fully connected layer.
  • Operator settings: PosPool and PosPool* both use AVG pooling, following Eq. 8 and Eq. 9 respectively.

A3 More Variants of PosPool

The appendix evaluates weight-free PosPool variants built from alternative relative-position encodings and reduction functions, finding that full positional configurations outperform angle- or distance-only variants.

  • Variant formulation: Seven PosPool variants are constructed by changing the scalar encoding functions applied to relative positions.
  • Variant formulation: Second-order variants add six encoding scalars formed from squares and pairwise multiplications of relative coordinates.
  • Variant formulation: Third-order variants add third-order multiplications, omitting ∆xij∆yij∆zij so g = 18 and C remains divisible by g.
  • Variant formulation: Angle-and-distance variants decouple relative position into angular and distance components, while Gaussian-inversed-distance variants amplify closer points.
  • Results: All variants containing full relative-position configurations perform similarly well and significantly better than angle-only or distance-only variants.
  • Results: Changing whether distant points receive larger or smaller encoding amplitudes has insignificant effect on performance.

A4 The Robustness of Different Operators with Missing/Noisy Points

Robustness is evaluated on PartNet by training on clean data and testing with missing or noisy points; most learned operators handle missing points similarly, while PosPool is strongest under noise.

  • Experimental setup: The robustness curves compare AdaptWeight, PseudoGrid, MaxPool, PosPool, and PointMLP across noise and missing-point ratios.Models are trained on clean PartNet data, with corruption introduced only during inference.
  • Missing points: With varying missing-point ratios, AdaptWeight, PseudoGrid, PointMLP, and PosPool perform similarly and all significantly outperform MaxPool.
  • Noisy points: Under varying noise ratios, PosPool performs best, slightly exceeding AdaptWeight and PointMLP and significantly exceeding PseudoGrid and MaxPool.
  • Noisy points: Noisy features substantially contaminate some regular-point activations in MaxPool and PseudoGrid, whereas other operators are less affected.

A5 More Detailed Results on PartNet

Detailed PartNet evaluations show similar operator performance across categories, and qualitative results indicate that representative methods usually produce comparable shape predictions.

  • Quantitative results: Part-category mIoU is reported separately for each PartNet category.
  • Quantitative results: All operators show similar results on individual PartNet categories, further validating the paper’s operator-level findings.
  • Qualitative results: Qualitative results show that representative methods perform similarly well on most shapes.

A6 Detailed Space and Time complexity Analysis

The section analyzes the space and time complexity of several local aggregation methods, including depth-wise convolution and adaptive-weight approaches. It also documents supplementary benchmark tables and visualizations for comparing variants and segmentation outputs.

  • The section provides a detailed space-and-time complexity analysis for the aggregators introduced earlier.
  • For point-wise MLP operators with h = 1, the shared fully connected layer costs (d + 3)dnK time and (d + 3)d parameters.The layer is applied to each point across K neighborhoods.
  • Depth-wise convolution uses d · M parameters and has time cost ndKM.A d-dimensional learned weight vector is associated with each grid point.
  • Adaptive Weight methods combine shared-MLP weight computation with depth-wise aggregation, yielding total time ((h −2)d/2 + d + 5) · d/2 · nK and space ((h −2)d/2 + d + 3) · d/2.The aggregation step itself costs dnK time and zero additional parameters.
  • The supplementary material includes PosPool variant evaluations, PartNet validation and test mIoU tables, sample counts, and visualizations of activations, segmentation results, and point-wise MLP architectures.
Loading 2007.01294v1…