PaperlayerView exact revision on arXiv

Source-linked AI summary

FCAF3D: Fully Convolutional Anchor-Free 3D Object Detection

Danila Rukhovich, Anna Vorontsova, Anton Konushin

arXiv:2112.00322v2cs.CV

TL;DR

3D object detection must handle irregular point clouds while balancing accuracy and scalability, and existing methods may rely on geometric priors that limit generalization. FCAF3D uses a fully convolutional anchor-free architecture with sparse voxel convolutions and a novel OBB parametrization. It reports state-of-the-art mAP@0.5 results across ScanNet, SUN RGB-D, and S3DIS, including gains of 4.5, 3.5, and 20.5 points, respectively.

  • Problem

    3D object detection lacks a method that simultaneously provides precise estimates and scales well, while prior box assumptions may restrict generalization.

  • Method

    FCAF3D is a fully convolutional anchor-free detector using sparse convolutions and a novel OBB parametrization without object priors.

  • Results

    FCAF3D significantly outperforms previous state-of-the-art methods in mAP and inference speed on ScanNet, SUN RGB-D, and S3DIS.

  • Takeaways & Limitations

    The paper presents FCAF3D with its OBB parametrization as accurate, scalable, and generalizable for indoor 3D object detection.

  • Takeaways & Limitations

    VoteNet-like alternatives are limited by poor scalability and custom voting or grouping layers that complicate reproduction, debugging, and mobile deployment.

Abstract

from arXiv · show

Recently, promising applications in robotics and augmented reality have attracted considerable attention to 3D object detection from point clouds. In this paper, we present FCAF3D - a first-in-class fully convolutional anchor-free indoor 3D object detection method. It is a simple yet effective method that uses a voxel representation of a point cloud and processes voxels with sparse convolutions. FCAF3D can handle large-scale scenes with minimal runtime through a single fully convolutional feed-forward pass. Existing 3D object detection methods make prior assumptions on the geometry of objects, and we argue that it limits their generalization ability. To get rid of any prior assumptions, we propose a novel parametrization of oriented bounding boxes that allows obtaining better results in a purely data-driven way. The proposed method achieves state-of-the-art 3D object detection results in terms of mAP@0.5 on ScanNet V2 (+4.5), SUN RGB-D (+3.5), and S3DIS (+20.5) datasets. The code and models are available at https://github.com/samsunglabs/fcaf3d.

1 Introduction

FCAF3D addresses the accuracy–scalability gap in indoor 3D object detection with an anchor-free, data-driven method and novel OBB parametrization. It reports stronger benchmark performance while targeting faster inference and fewer geometry-related assumptions.

  • Motivation: 3D object detection methods struggle to process irregular point-cloud data at scale, with accuracy and computational efficiency often in tension.Large scenes can require impractical resources or excessive processing time, while sparse-convolution approaches may sacrifice detection accuracy.
  • Motivation: Prior bounding-box assumptions about aspect ratios and absolute sizes can restrict generalization while increasing hyperparameters and trainable parameters.The paper therefore targets objects with arbitrary shapes and sizes without additional hand-tuned assumptions.
  • Approach: FCAF3D is a fully convolutional anchor-free method that avoids object priors and uses a novel Mobius-strip-inspired OBB parametrization.The proposed approach is presented as purely data-driven and designed to reduce the number of hyperparameters.
  • Approach: SUN RGB-D experiments show that the novel OBB parametrization improves accuracy for several existing 3D object detection methods.The paper uses these experiments to evaluate the parametrization independently of the complete FCAF3D system.
  • Results: At least 3.5% mAP@0.5 separates FCAF3D from other methods on SUN RGB-D and ScanNet, while S3DIS shows a large margin over competitors.The paper reports evaluation on ScanNet, SUN RGB-D, and S3DIS, with superiority over previous state of the art across the benchmarks.
  • Results: FCAF3D is positioned as a first-in-class fully convolutional anchor-free detector for indoor scenes that improves accuracy while supporting scalable inference.Its contributions include the detector, the OBB parametrization, and benchmark results against prior methods.

2 Related Work

Prior 3D detection approaches use voting, transformers, dense or sparse convolutions, and anchor-based voxel representations, each with scalability, memory, accuracy, or implementation trade-offs. FCAF3D builds on sparse convolutions while using a fully convolutional design and anchor-free formulation.

  • Voting-based methods: Voting-based methods group points around voted centers but scale poorly with larger scenes and often rely on custom layers that hinder reproduction and deployment.These design choices can also complicate debugging and porting to mobile devices.
  • Transformer-based methods: Transformer-based methods provide end-to-end inference, but advanced variants still experience scalability issues; FCAF3D is described as faster and easier to implement.The comparison is made against both voting-based and transformer-based approaches.
  • 3D convolutional methods: Voxel-based 3D convolutional methods process sparse data efficiently, but dense volumetric features consume memory and 3D convolutions remain computationally expensive.Large scenes therefore cannot generally be processed within a single pass by these approaches.
  • 3D convolutional methods: GSDN improves memory efficiency and large-scene scalability with sparse 3D convolutions, but its accuracy is inferior to the current state of the art.GSDN uses an encoder–decoder architecture built from sparse 3D convolutional blocks.
  • FCAF3D: FCAF3D combines a backbone, neck, and head, with sparse three-dimensional convolutions and transposed convolutions enabling single-pass point-cloud processing.The architecture is presented as a fully convolutional alternative to the preceding voting, transformer, and voxel-based designs.
  • Anchor-free detection: GSDN uses 15 aspect-ratio anchors, whereas training it with a single aspect ratio reduces accuracy by 12%; FCAF3D is anchor-free while retaining sparse convolutions.This comparison motivates removing anchor-based geometric priors without abandoning sparse processing.

3 Proposed Method

FCAF3D is a fully convolutional, anchor-free indoor 3D detector that uses sparse convolutions and data-driven box representations to combine scalability with reduced reliance on geometric priors. Its method includes a sparse backbone and neck, an anchor-free head with multi-level assignment, and parametrizations for axis-aligned and oriented boxes.

  • Architecture: FCAF3D accepts RGB-colored 3D points and predicts object boxes through a backbone, neck, and head.The architecture is designed for indoor 3D object detection.
  • Architecture: The sparse backbone replaces 2D convolutions in ResNet with sparse 3D convolutions, while the neck uses sparse transposed and standard 3D convolutions.The neck simplifies GSDN-style decoding to limit memory growth from feature expansion.
  • Sparse neural network: The neck removes GSDN’s scoring layer and loss, reuses head classification probabilities for pruning, and retains at most Nvox voxels, where Nvox equals Npts.This avoids tuning a probability threshold while controlling sparsity with the input-point count.
  • Detection head: The anchor-free head predicts classification probabilities, box regression parameters, and centerness at each location using three parallel sparse convolutional layers shared across feature levels.At inference, classification scores are multiplied by centerness before NMS.
  • Location assignment: Multi-level assignment selects the last feature level covering at least Nloc locations for each box, defaults to the first level otherwise, and applies center sampling without dataset-specific threshold tuning.This replaces strategies requiring hand-tuned face-distance thresholds for each feature level.
  • Bounding box parametrization: For oriented boxes, FCAF3D addresses heading-angle ambiguity with a Mobius-inspired parametrization using ln(q) sin(2θ) and ln(q) cos(2θ), while AABBs use six distance and size parameters.The proposed representation maps equivalent oriented-box descriptions together and ignores the rare w = l case for simplicity and accuracy.

4 Experiments

FCAF3D is evaluated on three indoor 3D detection benchmarks using mAP at IoU thresholds 0.25 and 0.5, with standardized training and repeated randomized evaluation.

  • Datasets: FCAF3D is evaluated on ScanNet V2, SUN RGB-D, and S3DIS using mAP@0.25 and mAP@0.5.The datasets use different category counts and annotation formats, including AABBs for ScanNet and S3DIS and OBBs for SUN RGB-D.
  • Implementation Details: The experiments use dataset-specific output categories: 18 for ScanNet, 10 for SUN RGB-D, and 5 for S3DIS.SUN RGB-D additionally predicts two OBB targets, while the loss function remains unchanged.
  • Implementation Details: Training uses 12 epochs with Adam, an initial learning rate of 0.001, weight decay of 0.0001, and two NVIDIA V100 GPUs.Evaluation and performance tests run on a single GTX1080Ti.
  • Evaluation Protocol: Reported metrics include both best and average values across 25 randomized trials.Training and evaluation are each repeated five times to account for random point sampling and obtain statistically significant results.

5 Results

FCAF3D consistently outperforms prior indoor 3D detection methods while combining accuracy with efficient inference. Its Mobius OBB parametrization improves several detectors, and ablations identify accuracy–speed trade-offs in voxel size, point count, and backbone design.

  • Comparison with State-of-the-art Methods: FCAF3D surpasses previous state-of-the-art by 4.5% mAP@0.5 on ScanNet and 3.7% on SUN RGB-D, with a particularly large margin on S3DIS.It achieves the best results across all three indoor benchmarks.
  • Object Geometry Priors: Replacing standard OBB parametrization with the Mobius parametrization boosts VoteNet and ImVoteNet mAP@0.5 by approximately 4%.The parametrization reduces the number of trainable parameters and hyperparameters, including geometry priors and loss weights.
  • Comparison with Fully Convolutional Methods: With HDResNet34, FCAF3D reaches mAP@0.25 of 70.7 and mAP@0.5 of 56.0, versus 62.8 and 34.8 for GSDN.This comparison uses the accurate FCAF3D configuration with 0.01 m voxel size and default stride 2.
  • Inference Speed: The lightweight HDResNet34:2 variant processes 30 scenes per second versus 20 for GSDN without sacrificing superior mAP@0.25 and mAP@0.5.This speed comparison uses a voxel size of 0.02 m.
  • Ablation Study: Larger voxel sizes reduce accuracy, partly because nearly flat objects can be thinner than the localization tolerance induced by coarse voxels.At 2 cm voxel size, the head outputs locations with 16 cm tolerance, which can exceed one dimension of flat objects.
  • Ablation Study: Using more points generally improves detection accuracy, but values above 100k increase inference time while providing negligible accuracy gains.The experiments compare 20k, 40k, and 100k sampled points and use 100k as the default.
  • Ablation Study: Selecting 18 locations for center sampling performs best across all benchmarks, while reduced-feature-level backbones make FCAF3D faster than GSDN.The fastest variants use HDResNet34:3 and HDResNet34:2 with three and two feature levels.

6 Conclusion

The paper presents FCAF3D as an accurate, scalable, and generalizable anchor-free method for indoor 3D object detection. Its OBB parametrization avoids object-shape priors and improves accuracy across multiple detectors.

  • Conclusion: FCAF3D is a fully convolutional anchor-free method for indoor 3D object detection that outperforms prior state-of-the-art in mAP and inference speed.The evaluation covers SUN RGB-D, ScanNet, and S3DIS.
  • Conclusion: The novel OBB parametrization improves several 3D object detection methods while avoiding prior assumptions about object geometry.Avoiding these priors reduces the number of hyperparameters.

A Additional Comments on Mobius Parametrization

The appendix explains why conventional OBB parameters are ambiguous and verifies that the Mobius mapping gives equivalent representations of the same box the same encoded point.

  • OBB Ambiguity: A single OBB can have four equivalent representations because heading angles and frontal-versus-lateral face assignments are ambiguous.The four tuples arise from θ, θ + π/2, θ + π, θ + 3π/2 and aspect-ratio choices q or 1/q.
  • Mobius Verification: The Mobius parametrization maps all four representations of one OBB to the same point.The verification explicitly checks the transformations corresponding to these equivalent tuples.

B Metric values for Fig. 1

Inference speed is reported for point-cloud methods on ScanNet using the same single NVidia GTX1080Ti.

  • Inference speed is measured on the same single NVidia GTX1080Ti for all methods.
  • ScanNet inference speed is compared across 3D object detection methods that accept point clouds.

C Per-category results

FCAF3D achieves strong per-category results across ScanNet, SUN RGB-D, and S3DIS, with especially large gains on selected categories and on S3DIS overall.

  • ScanNet: 12 of 18 ScanNet categories achieve higher AP@0.25 with FCAF3D than with other methods.The largest gaps are window at 60.2 against 53.7, picture at 29.9 against 18.6, and other furniture at 65.4 against 56.4.
  • ScanNet: 13 of 18 ScanNet categories achieve the best AP@0.5 with FCAF3D.Reported large improvements include cabinet at 35.8 against 26.0, sofa at 85.2 against 70.7, picture at 17.9 against 7.8, shower at 64.2 against 44.1, and sink at 52.6 against 37.4.
  • SUN RGB-D: FCAF3D is more accurate on 7 of 10 SUN RGB-D categories at AP@0.25 and improves mAP@0.25 by 1.2% over GroupFree.The largest category gaps are 4.1% for desk and 5.2% for night stand.
  • SUN RGB-D: FCAF3D outperforms competitors on 9 of 10 SUN RGB-D categories at AP@0.5 and surpasses them by more than 3.5% in mAP@0.5.Examples include dresser at 30.1 against 21.9, night stand at 59.8 against 49.8, and table at 35.5 against 29.2.
  • S3DIS: On S3DIS, FCAF3D achieves AP@0.25 of 92.4 for sofas against GSDN's 20.8 and gains almost 19% in mAP.

D Visualization

The visualizations overlay estimated and ground-truth 3D bounding boxes on point clouds from three indoor benchmarks, using colors to distinguish object categories.

  • The visualizations draw ground-truth and estimated 3D bounding boxes over corresponding point clouds.
  • SUN RGB-D: SUN RGB-D visualizations place FCAF3D estimates on the left and ground truth on the right, with colors denoting object categories.
  • ScanNet: ScanNet visualizations show FCAF3D estimates beside ground-truth AABBs, with category-specific colors.
  • S3DIS: S3DIS visualizations compare FCAF3D estimates with ground-truth AABBs using different colors for object categories.
Loading 2112.00322v2…