PaperlayerView exact revision on arXiv

Source-linked AI summary

Learned Point Cloud Geometry Compression

Jianqiang Wang, Hao Zhu, Zhan Ma, Tong Chen, Haojie Liu, Qiu Shen

arXiv:1909.12037v1cs.CVeess.IV

TL;DR

Point cloud geometry compression must efficiently represent volumetric 3D data for applications such as 3D scenes and holoportation. Learned-PCGC addresses this with voxelization, scaling, cube partitioning, and a 3D convolutional VAE using hyperpriors and occupancy classification. It reports large BD-Rate gains over G-PCC, improved rendered quality, and a compact 2.5MB model, while remaining focused on static geometry.

  • Problem

    High-efficiency compression is needed for volumetric point-cloud data used in applications such as 3D scenes and holoportation.

  • Method

    Learned-PCGC voxelizes, scales, and partitions geometry into cubes, then uses stacked 3D convolutions in a VAE with WBCE training and adaptive occupancy thresholding.

  • Results

    Over 62% and 67% BD-Rate gains over G-PCC trisoup, and over 69% and 76% over G-PCC octree, are reported for D2 and D1 respectively.

  • Takeaways & Limitations

    The method combines improved objective and subjective compression performance with a 2.5MB model suitable for hardware or embedded implementation.

  • Takeaways & Limitations

    The paper focuses on static geometry compression and leaves other aspects, such as color attributes, for future study.

Abstract

from arXiv · show

This paper presents a novel end-to-end Learned Point Cloud Geometry Compression (a.k.a., Learned-PCGC) framework, to efficiently compress the point cloud geometry (PCG) using deep neural networks (DNN) based variational autoencoders (VAE). In our approach, PCG is first voxelized, scaled and partitioned into non-overlapped 3D cubes, which is then fed into stacked 3D convolutions for compact latent feature and hyperprior generation. Hyperpriors are used to improve the conditional probability modeling of latent features. A weighted binary cross-entropy (WBCE) loss is applied in training while an adaptive thresholding is used in inference to remove unnecessary voxels and reduce the distortion. Objectively, our method exceeds the geometry-based point cloud compression (G-PCC) algorithm standardized by well-known Moving Picture Experts Group (MPEG) with a significant performance margin, e.g., at least 60% BD-Rate (Bjontegaard Delta Rate) gains, using common test datasets. Subjectively, our method has presented better visual quality with smoother surface reconstruction and appealing details, in comparison to all existing MPEG standard compliant PCC methods. Our method requires about 2.5MB parameters in total, which is a fairly small size for practical implementation, even on embedded platform. Additional ablation studies analyze a variety of aspects (e.g., cube size, kernels, etc) to explore the application potentials of our learned-PCGC.

I. INTRODUCTION

Learned-PCGC applies an end-to-end learned framework to static point cloud geometry compression, combining voxel processing, cube partitioning, 3D convolutional VAEs, and classification-based distortion control. It reports substantial gains over G-PCC, improved rendered quality, and a compact model suited to practical implementation.

  • Motivation: Point cloud applications require high-efficiency compression for volumetric 3D scenes and objects.The paper targets geometry compression amid growing point-cloud-enabled applications such as 3D free-viewpoint video and holoportation.
  • Proposed framework: Learned-PCGC voxelizes, scales, and partitions point cloud geometry before VAE-based compression and post-processing reconstruction.The pipeline uses non-overlapped cubes and compact latent representations generated by stacked 3D convolutions.
  • Proposed framework: Weighted binary cross-entropy training and adaptive inference thresholding formulate voxel occupancy as a classification problem.This design determines whether decoded voxels are occupied and exploits voxel sparsity rather than directly optimizing D1 or D2 distortion.
  • Results: Approximately 67% and 76% D1 BD-Rate gains, and 62% and 69% D2 gains, are reported against G-PCC octree and trisoup models respectively.The evaluations use common datasets suggested by the MPEG PCC and JPEG Pleno groups.
  • Results: The method reports better visual quality, including smoother surfaces and appealing details, than existing MPEG-standard-compliant PCC methods.The paper attributes this mainly to an inherently 3D structural representation using learned 3D transforms.
  • Practicality: A 2.5MB model and parallel cube processing support low-complexity implementation on hardware or embedded platforms.Additional ablation studies examine partition size, kernel size, and thresholding for practical application.

II. RELATED WORK

Prior point cloud geometry compression methods use octrees, surface meshes, or projections into 2D images, while learned compression has mainly advanced in image and video domains. These approaches provide established baselines but retain representation-specific limitations.

  • Research landscape: Prior point cloud compression research includes geometry compression, learned image compression, and autoencoder-based point cloud processing.The related work is organized around these three research directions.
  • Conventional PCC: G-PCC and related methods use conventional octree, trisoup, hierarchical-transform, and 3D-to-2D projection models.G-PCC targets static point clouds, whereas V-PCC targets dynamic point clouds.
  • Conventional PCC: Octree coding represents occupied voxels or cubes with binary node labels and generally performs decently for sparse but less effectively for dense point clouds.G-PCC includes an octree geometry codec.
  • Conventional PCC: Surface models encode mesh vertices and reconstruct point clouds by sampling triangle meshes.MPEG G-PCC also includes a triangulated surface model.
  • Conventional PCC: Projection-based methods pack 3D point-cloud patches into 2D grids, convert geometry to depth images, and compress them with image or video codecs.The main technical issue is efficient 3D-to-2D projection.

B. Learned Image Compression

Learned-PCGC extends learned convolutional compression and autoencoder-based point-cloud processing into a cube-wise 3D geometry-compression pipeline. Its preprocessing uses voxelization, scaling, and partitioning to prepare data for learned compression while retaining parallel processing.

  • Learned compression background: Learned image compression commonly uses VAE structures, stacked 2D CNNs, and hyperpriors for compact latent features and conditional probability estimation.The paper identifies a lack of systematic effort to extend these learned compression principles to point cloud geometry.
  • Point-cloud autoencoders: Existing point-cloud autoencoders have supported tasks such as classification, shape completion, and voxel-based 3D object modeling.These models provide references for applying autoencoder structures to point cloud compression.
  • Pipeline: The proposed framework uses preprocessing, an end-to-end compression network, and post-processing for point cloud reconstruction and rendering.The detailed design is presented as a sequence of these three components.
  • Pre-processing: Voxelization maps occupied point-cloud positions to binary voxel values and exposes 3D inter-voxel correlations for subsequent 3D convolutions.A voxel is set to 1 for a positive attribute and 0 otherwise.
  • Pre-processing: Downscaling and rounding reduce the point cloud before compression, with inverse scaling applied after cube inference for rendering.The scaling factor satisfies s < 1, and duplicate coordinates created by rounding are removed.
  • Pre-processing: Partitioning divides the volumetric model into non-overlapped W × W × W cubes, enabling cube-wise processing and parallelism.The current study processes cubes independently without exploiting inter-cube correlations.

B. Cube-based Learned-PCGC

Learned-PCGC represents voxelized point-cloud cubes with stacked 3D convolutional transforms, using VRN units in the main codec and hyperpriors to improve latent-feature probability modeling.

  • The framework pursues compact representations of sparsely distributed voxel cubes through stacked 3D CNN transforms, rate estimation, and classification-based distortion optimization.
  • 3D Convolution-based Transforms: 3D CNN transforms exploit spatial correlation to encode each W × W × W cube as compact latent features and decode them into reconstructed voxel cubes.
  • 3D Convolution-based Transforms: Voxception-ResNet units form the main codec, with nine stacked VRNs used for both analysis and synthesis transforms.
  • Hyper-codec convolutions generate decoded hyperpriors that improve conditional probability modeling of latent features.
  • The method uses relatively small 1 × 1 × 1 and 3 × 3 × 3 kernels, integrated with VRN to capture essential information with lower complexity.

C. Quantization

Learned-PCGC uses differentiable noise during training to approximate inference-time rounding of latent features, preserving an end-to-end trainable compression pipeline.

  • Inference quantizes latent features with a rounding operation, mapping original representations y to quantized representations ˆy.
  • Direct rounding is nondifferentiable during backpropagation, so training approximates it by adding uniform noise to the latent representation.
  • The noisy latent representation ˆy follows a uniform distribution centered on y, with interval [y−1/2, y+1/2].

D. Entropy Rate Modeling

Entropy-rate modeling estimates latent-feature and hyperprior rates using probability models, with hyperpriors supplying side information for more accurate conditional modeling of latent features.

  • Arithmetic coding compresses each quantized latent-feature element, making accurate probability-based rate estimation central to source compression.
  • The latent-feature rate is approximated from negative log probabilities of quantized latent features under their conditional distributions.
  • Hyperpriors provide decoded prior knowledge that improves approximation of latent-feature probabilities when conditioned on the hyperprior distribution.
  • Decoded hyperpriors are modeled with a fully factorized probability model, while a Laplacian distribution approximates latent-feature densities conditioned on them.

E. Rate-distortion Optimization

Rate-distortion optimization combines estimated rates with a classification-based distortion objective, using WBCE during training and adaptive thresholding during inference for voxel decisions.

  • Rate-distortion Optimization: The framework uses a Lagrangian loss to trade off distortion and bit rate, with λ controlling the bit-rate operating point.
  • Rate Estimation: The total rate is the sum of latent-feature and hyperprior rates, R = Rˆy + Rˆz, with hyperprior bits treated as side-information overhead.
  • Distortion Measurement: WBCE measures distortion by treating decoded voxel values as probabilities of occupancy and balancing occupied and null voxel penalties.
  • Classification: Adaptive thresholding classifies decoded voxels into occupied or null states according to the number of occupied voxels, rather than using a fixed threshold.
  • Post-Processing: Post-processing applies inverse scaling with factor 1/s after cube inference, while extraction converts volumetric point clouds into storage or exchange formats when needed.

G. Bitstream Specification

Learned-PCGC encodes each cube’s metadata alongside entropy-coded latent features and hyperpriors in the bitstream. Metadata identifies cube location and occupancy count, while hyperpriors condition latent-feature probability modeling.

  • Each cube contributes its position, original occupied-voxel count, entropy-coded features, and hyperpriors to the binary bitstream.Cube position and occupancy count form metadata overhead; features and hyperpriors form the main payload.
  • Cube positions can be signaled with octree coding, while occupancy counts support voxel classification.
  • The worst-case occupancy-count metadata scales with the cube volume, although sparse cubes typically require fewer bits.
  • Arithmetic coding encodes hyperpriors and latent features consecutively, with latent-feature probabilities conditioned on decoded hyperpriors.

A. Training

Training uses ShapeNet-derived voxelized point clouds and a rate-distortion strategy spanning multiple compression levels. Higher-bitrate models initialize lower-bitrate models to improve convergence and stability.

  • 12,714 ShapeNet mesh models across 55 object categories provide the training data.Meshes are sampled into point clouds and voxelized in a 256 × 265 × 256 occupancy space.
  • Training samples are randomly collected as 64 × 64 × 64 cubes from voxelized point clouds.
  • The rate-distortion trade-off λ ranges from 0.75 to 16 to produce models with different compression performance.
  • Training proceeds from λ = 16 models toward lower bit rates through transfer learning, improving convergence speed and outcome stability.

B. Performance Evaluation

Across MPEG and JPEG Pleno test classes, Learned-PCGC is evaluated against conventional point-cloud codecs using rate-distortion and visual comparisons. It reports large BD-Rate gains over PCL and G-PCC, comparable performance with V-PCC, and smoother or repaired-looking reconstructions in selected cases.

  • Datasets: The evaluation covers full-body, incomplete upper-body, and sparse high-precision inanimate-object point clouds from MPEG PCC and JPEG Pleno datasets.Class A contains complete full bodies, Class B noisy incomplete upper bodies, and Class C sparse inanimate objects.
  • Objective comparison: BD-Rate gains average -88% and -82% against PCL under D1 and D2, respectively.
  • Objective comparison: BD-Rate gains average -77% and -69% against G-PCC (octree) under D1 and D2, respectively.
  • Objective comparison: BD-Rate gains average -67% and -62% against G-PCC (trisoup) under D1 and D2, respectively.
  • Generalization: Learned-PCGC remains effective across dense or sparse distributions and complete or incomplete surfaces, using simple scaling for sparse Class C data.
  • V-PCC comparison: Against V-PCC, Learned-PCGC records averaged +8.16% D1 BD-Rate loss but -4.31% D2 BD-Rate gains.V-PCC performs better on Loot and Longdress, while Learned-PCGC performs better on Redandblack and Soldier.
  • Subjective evaluation: Visual evaluation compares decoded clouds and point-to-point error maps against ground truth after normal-based Lambert rendering.
  • Subjective evaluation: ShapeNet-trained reconstructions tend toward complete, smooth, lower-noise shapes, whereas other methods tend toward random noise.

V. ABLATION STUDIES

The ablations examine partition size, hyperpriors, adaptive thresholding, and network design, showing trade-offs between rate-distortion efficiency and implementation complexity.

  • Partition Size: 20% BD-Rate gain occurs when cube size increases from W = 32 to W = 64, with almost no further gain at W = 128.The study uses Loot as an example, and other testing materials show similar outcomes.
  • Partition Size: W = 64 balances BD-Rate performance with processing time, memory consumption, parallelism, and blocky artifacts.Smaller cubes improve parallel processing and reduce memory and computation but sacrifice BD-Rate and increase blocky artifacts.
  • Hyperpriors: 14.65% BD-Rate gain results from adding hyperpriors to factorized entropy modeling through improved conditional probability estimation.The comparison is against a scenario using only a factorized entropy model for latent representations.
  • Adaptive Thresholding: 8.9% and 6.7% average BD-Rate gains arise from optimizing top-k voxel selection for D1 and D2 distortion, respectively.The optimal selection factors differ: ρ = 1.14 for D1 and ρ = 0.91 for D2 on the illustrated example.
  • Implementation Complexity: The model contains 658,092 embedded-convolution parameters, requiring approximately 2.52MB of storage.Parameters are buffered using a 4-byte floating-point format.
  • Convolution Kernels: Stacked VRN networks with small kernels and deep layers outperform shallow networks with larger kernels because down-scaling captures spatial correlations across scales.The paper attributes the alternative’s weaker performance to sparse voxel distributions limiting large-kernel spatial capture.

VI. CONCLUSION AND FUTURE WORK

Learned-PCGC combines learned 3D transforms, VAE-based entropy modeling, weighted BCE training, and adaptive voxel classification for point cloud geometry compression. It reports substantial gains over G-PCC, while future work highlights weaknesses in objective distortion metrics.

  • Conclusion: Learned-PCGC integrates stacked 3D convolutions, VAE entropy modeling, weighted BCE loss, and adaptive thresholding for voxel classification.These components target latent-feature extraction, probability modeling, training distortion optimization, and inference classification.
  • Conclusion: Over 62% and 67% BD-Rate gains over G-PCC (trisoup), and over 69% and 76% over G-PCC (octree), are reported for D2 and D1 respectively.The conclusion also reports comparable compression efficiency against MPEG V-PCC and perceptual improvements in subjective evaluation.
  • Future Work: D1 and D2 distortion measurements have low correlation with subjective assessment, motivating development of a better objective metric.The authors identify improved convolution efficiency as another future-study direction.
Loading 1909.12037v1…