PaperlayerView exact revision on arXiv

Source-linked AI summary

Point Cloud Quality Assessment: Dataset Construction and Learning-based No-Reference Metric

Yipeng Liu, Qi Yang, Yiling Xu, Le Yang

arXiv:2012.11895v4eess.IV

TL;DR

NR-PCQA lacks large-scale data because reference clouds are often unavailable and subjective annotation is expensive. The paper builds LS-PCQA with pseudo scores and proposes ResSCNN, a sparse-CNN metric operating directly on point clouds. ResSCNN achieves state-of-the-art performance among existing NR metrics and is competitive with FR metrics.

  • Problem

    NR-PCQA is hindered by unavailable reference point clouds and the lack of large-scale datasets needed to train generalizable learning-based metrics.

  • Method

    The paper constructs LS-PCQA with pseudo quality scores and develops ResSCNN, a sparse-CNN NR-PCQA metric using hierarchical features from 3D point clouds.

  • Results

    ResSCNN achieves state-of-the-art performance among existing NR-PCQA metrics and remains competitive with FR-PCQA metrics across three datasets.

  • Takeaways & Limitations

    LS-PCQA provides large-scale training data, and the proposed sparse-CNN metric supports effective no-reference quality assessment without requiring reference point clouds.

  • Takeaways & Limitations

    Dimensionality-reduction preprocessing should not be used in NR-PCQA because it introduces additional geometric distortions.

Abstract

from arXiv · show

Full-reference (FR) point cloud quality assessment (PCQA) has achieved impressive progress in recent years. However, in many cases, obtaining the reference point clouds is difficult, so no-reference (NR) metrics have become a research hotspot. Few researches about NR-PCQA are carried out due to the lack of a large-scale PCQA dataset. In this paper, we first build a large-scale PCQA dataset named LS-PCQA, which includes 104 reference point clouds and more than 22,000 distorted samples. In the dataset, each reference point cloud is augmented with 31 types of impairments (e.g., Gaussian noise, contrast distortion, local missing, and compression loss) at 7 distortion levels. Besides, each distorted point cloud is assigned with a pseudo quality score as its substitute of Mean Opinion Score (MOS). Inspired by the hierarchical perception system and considering the intrinsic attributes of point clouds, we propose a NR metric ResSCNN based on sparse convolutional neural network (CNN) to accurately estimate the subjective quality of point clouds. We conduct several experiments to evaluate the performance of the proposed NR metric. The results demonstrate that ResSCNN exhibits the state-of-the-art (SOTA) performance among all the existing NR-PCQA metrics and even outperforms some FR metrics. The dataset presented in this work will be made publicly accessible at http://smt.sjtu.edu.cn. The source code for the proposed ResSCNN can be found at https://github.com/lyp22/ResSCNN.

1 INTRODUCTION

The paper addresses limited NR-PCQA development by constructing a large-scale dataset and proposing a sparse-CNN metric that operates directly on point clouds. Experiments report robust performance across datasets, matching or surpassing some FR metrics.

  • Motivation: NR-PCQA is important because reference point clouds are often unavailable, while existing datasets are too small for highly generalizable learning-based metrics.Annotation is also costly: the dataset contains more than 22,000 samples, and each requires at least 16 subjects for MOS collection.
  • Dataset: LS-PCQA contains 22,568 distorted point clouds derived from 104 references, spanning 31 distortion types at 7 levels across geometry and color.Examples include Gaussian noise, contrast distortion, local missing, and compression loss.
  • Dataset: The dataset uses subjective scores from 1,240 samples to derive pseudo quality scores for annotating the full database.The subjective experiment recruited 224 candidates and collected at least 16 valid scores per sample after outlier removal.
  • Proposed method: ResSCNN uses sparse convolution and hierarchical feature extraction to assess quality directly from unevenly distributed 3D point clouds.Sparse tensors avoid the feature-map expansion, high memory use, and slow inference associated with dense convolution.
  • Evaluation: ResSCNN achieves robust and competitive performance on LS-PCQA, SJTU-PCQA, and WPC2.0, including performance above some FR metrics.The conclusion also reports improved generalization of learning-based NR-PCQA metrics when trained with the large-scale dataset.

2 RELATED WORK

Prior PCQA methods include FR, RR, and projection-based NR approaches, but they may require references, lose information through projection, or focus too narrowly on local features. The paper motivates direct sparse-CNN processing to capture hierarchical local and global information from point clouds.

  • FR and RR metrics: Point-wise FR metrics measure geometric distortion using distances, surface-normal projections, reconstructed meshes, or normal-angle differences.These approaches primarily target geometric attributes and include p2point, p2plane, p2mesh, and normal-based methods.
  • FR and RR metrics: Other PCQA methods incorporate structural and color information, use 2D projections, or exploit statistical geometry, color, and normal features.Projection direction can affect performance, and projection can cause information loss under multiple distortions.
  • PCQA metrics: FR and RR metrics require the reference point cloud or its features, whereas NR metrics use only distorted samples.This makes NR assessment relevant when references are difficult or impossible to obtain.
  • NR-PCQA: Existing NR-PCQA metrics are projection-based, whereas this work proposes a metric operating directly on 3D point clouds.The direct formulation is intended to avoid projection-induced information loss.
  • 3D feature representation: NR-PCQA needs both local details and global understanding because subjective perception begins with local stimuli and requires whole-point-cloud perception.This differs from 3D recognition methods that often process small patches or key points in lower-dimensional spaces.
  • 3D feature representation: Sub-manifold sparse CNNs provide sparsity invariance and reduced computing load, motivating their use for hierarchical feature extraction in NR-PCQA.The proposed approach uses the Minkowski Engine for sparse convolutional processing.

3 LS-PCQA: LARGE-SCALE POINT CLOUD QUALITY ASSESSMENT DATASET

LS-PCQA addresses the scarcity of large-scale point-cloud quality data by combining 104 references, 22,568 distorted samples, controlled subjective annotations, and pseudo MOS generation. The resulting labels show high consistency with subjective MOS across evaluation subsets.

  • Dataset construction: LS-PCQA contains 104 reference point clouds and 22,568 distorted samples spanning 31 impairment types at 7 distortion levels.The impairments cover point-cloud production, compression, transmission, and presentation.
  • Reference point clouds: The dataset combines mesh-derived and existing point clouds, with mesh surfaces randomly sampled and texture materials used to color sampled points.The references include human models, animal models, and inanimate objects, and are screened for high quality.
  • Subjective MOS: Subjective annotation uses a controlled double-stimulus experiment, with 1,240 samples selected across all distortion types and reference-first pairwise presentation.Participants rate quality on a five-level scale after examining each pair in a controlled viewing setup.
  • Pseudo MOS construction: FR metrics are selected separately by distortion type because individual metrics respond differently to geometric and photometric impairments.For example, PCQM reaches SROCC 0.950823 for Gaussian noise but only 0.524864 for down-sampling, while p2point is insensitive to photometric attributes.
  • Pseudo MOS construction: Logistic-5 is selected for nonlinear MOS normalization, and the resulting pseudo MOS labels the full large-scale dataset.The mapping preserves the metric scores’ common [1,5] range for dataset annotation.
  • Validation: Pseudo MOS agrees strongly with subjective MOS, with SROCCs of 0.902697 for Part I and 0.878517 for Part II.Most annotation errors have small magnitudes, and pseudo MOS accuracy improves under more severe distortions.

4 RESSCNN: A SPARSE CNN BASED METRIC FOR NR-PCQA

ResSCNN is an end-to-end no-reference point-cloud quality assessment metric that uses sparse convolutions to extract hierarchical features directly from complete point clouds. Global pooling and concatenation convert features from multiple depths into a fixed-size representation for quality prediction.

  • ResSCNN is an end-to-end learning-based NR-PCQA metric designed for point clouds.
  • Hierarchical feature extraction: The network accepts complete 3D point clouds and uses sparse CNN layers with residual blocks to extract hierarchical features without dimensionality reduction.Sparse convolution preserves point-cloud sparsity, whereas dense convolution can massively increase feature-map elements.
  • Hierarchical feature extraction: The input sparse tensor represents geometry coordinates and occupation indices together with RGB color features for each point.Coordinates encode geometry, while the occupation index distinguishes points sharing coordinates.
  • Pooling and concatenation: Four hierarchical blocks produce features that are globally pooled into four 64 × 1 vectors and concatenated into a 256 × 1 representative vector.
  • Quality prediction: Two fully connected layers map the 256-dimensional hierarchical feature vector to a one-channel predicted quality score.The output score uses the same scale as the training labels.
  • Loss and training: Smooth L1 loss is used to improve robustness, while SGD training uses a learning rate of 1e-3 and an exponential schedule with γ = 0.99.Training uses batch size 1 with accumulated losses and gradients, plus random scaling and rotation augmentation.

5 EXPERIMENTS

ResSCNN is evaluated on LS-PCQA, SJTU-PCQA, and WPC2.0 against existing FR and NR metrics using PLCC and SROCC. It achieves the strongest NR performance and remains competitive with FR metrics across the three datasets.

  • Evaluation setup: ResSCNN is evaluated on three datasets: LS-PCQA, SJTU-PCQA, and WPC2.0.The evaluation compares it with existing full-reference and no-reference metrics using PLCC and SROCC.
  • Prediction performance: ResSCNN achieves state-of-the-art performance among existing NR metrics and outperforms some FR metrics.The paper reports this result from Table 9.
  • Prediction performance: 0.60 and 0.62 are ResSCNN's reported scores on LS-PCQA, compared with 0.70 and 0.67 for MPED.The paper presents these paired values when comparing ResSCNN with the best-performing FR metrics.
  • Prediction performance: 0.86 and 0.81 are ResSCNN's scores on SJTU-PCQA, compared with 0.91 and 0.89 for GraphSIM.These values are reported as the comparison for the SJTU-PCQA dataset.

5.3 Efficiency of The Proposed Dataset

The experiments test whether pre-training on LS-PCQA improves cross-dataset generalization. Pre-training improves ResSCNN's transfer performance between SJTU-PCQA and WPC2.0.

  • Experimental design: Pre-training on LS-PCQA is tested as a way to improve learning-based NR-PCQA metrics.ResSCNN is pre-trained on LS-PCQA before training and testing across SJTU-PCQA and WPC2.0.
  • Results: Pre-training on LS-PCQA improves ResSCNN's generalization capability across datasets.The paper reports this conclusion from Tables 10 and 11.
  • Results: Up to about 40% and 33% increases in PLCC and SROCC occur when training on SJTU-PCQA and testing on WPC2.0 after LS-PCQA pre-training.These are the reported maximum improvements for this transfer direction.
  • Results: Up to about 12% and 15% increases in PLCC and SROCC occur when training on WPC2.0 and testing on SJTU-PCQA after LS-PCQA pre-training.These are the reported maximum improvements for the reverse transfer direction.

5.4 Effect of Sampling

The sampling experiments examine how dimensionality reduction affects ResSCNN's NR-PCQA performance. Results support using complete point clouds because removing points introduces additional distortion and reduces performance.

  • Experimental design: Dimensionality reduction is tested by down-sampling point clouds to 400,000, 100,000, 50,000, 10,000, and 2,500 points.The experiments report overall and down-sampling-distortion performance for ResSCNN.
  • Results: ResSCNN generally provides better performance with more points.This trend is reported across the sampling versions in Table 12.
  • Results: Removing points through dimensionality reduction affects NR-PCQA results.The paper states that this finding corroborates the need to consider complete samples.
  • Implication: ResSCNN should take entire point clouds as input to avoid additional distortions during pre-processing.This is the paper's stated implication for the proposed metric.

5.5 Effectiveness of Hierarchical Features

The experiments compare shallow, deep, and hierarchical features and examine network depth. Hierarchical features perform best overall, while a four-block network provides the best depth configuration.

  • Feature design: Hierarchical features are motivated by combining shallow detail and deep semantic information for varied distortions.Shallow features provide color and texture details, whereas deep features contain conceptual and semantic information.
  • Feature comparison: The hierarchical feature outperforms shallow and deep features used alone.Table 13 reports the hierarchical feature as providing the best overall performance.
  • Feature comparison: Shallow features better handle detailed distortions, while deep features are needed for conceptual and semantic distortions.The dataset includes detailed distortions as well as contrast and luminance distortions.
  • Network depth: Increasing network depth improves dataset handling, but more parameters may cause over-fitting.The paper identifies this trade-off when comparing network depths.
  • Network depth: Four blocks yield the best overall performance among the tested network depths.Each block contains three sparse convolutional layers.

5.7 Effectiveness of the Residual Module

ResSCNN uses residual connections to compensate for feature information lost during pooling. Experiments on LS-PCQA show that residual connections improve quality-prediction accuracy, supporting their inclusion in the final network.

  • Residual design: Pooling can partially damage feature information, so ResSCNN uses residual connections to compensate for this loss.The network is designed to handle point clouds with arbitrary numbers of points, with pooling used to normalize features.
  • Experimental setup: The experiment compares ResSCNN with several residual-connection alternatives using four identical blocks.For dimensionality matching, some alternatives use residual connections spanning two or three layers.
  • Findings: Residual connections improve the accuracy of quality prediction on LS-PCQA.The final network therefore adopts residual connections.

5.8 Performance for Different Distortions

Across distortion types, ResSCNN performs best among existing NR-PCQA metrics for almost all cases, while FR metrics remain more robust overall. NR metrics particularly struggle with some color distortions, and projection-based methods can miss geometric loss.

  • NR-PCQA comparison: ResSCNN achieves the best performance among existing NR-PCQA metrics for almost all distortion types.Its advantage includes some geometric distortions, such as Local Missing, where projection can make geometric loss difficult to detect.
  • Color distortions: NR-PCQA metrics perform poorly on some color distortions, including Contrast Distortion and Poisson Noise.The paper hypothesizes that NR designs need additional semantic features describing color distributions.
  • FR versus NR: FR-PCQA metrics are more robust than NR metrics when original point clouds are available as references.References make it easier to quantify distortion effects on human perception.
  • Overall assessment: NR-PCQA remains more challenging without references, leaving learning-based NR metrics room for further improvement.The paper characterizes ResSCNN as strongest among the compared NR metrics across almost all distortion types.

6 CONCLUSION

The paper constructs LS-PCQA and proposes ResSCNN, a sparse-CNN NR-PCQA metric. Experiments report SOTA performance among existing NR metrics, competitiveness with FR metrics, and improved generalization from the large-scale dataset.

  • Dataset: LS-PCQA contains more than 22,000 distorted samples from 104 reference point clouds, spanning 31 impairment types at 7 distortion levels.The dataset was built to meet the data-scale requirements of learning-based quality metrics.
  • Metric: ResSCNN is a no-reference point-cloud quality metric based on a sparse convolutional neural network.It is trained and evaluated using the newly built LS-PCQA dataset.
  • Results: ResSCNN achieves state-of-the-art performance among existing NR-PCQA metrics and is competitive with FR-PCQA metrics.The conclusion attributes these findings to the reported experiments.
  • Dataset impact: The experiments show that the large-scale dataset can improve the generalization ability of learning-based NR-PCQA metrics.This conclusion concerns the utility of LS-PCQA for training such metrics.

A DETAILS FOR DISTORTION TYPES IN LS-PCQA DATASET

LS-PCQA defines distortion families covering photometric attributes, geometry, local structure, reconstruction, and compression. Each impairment is implemented with controlled parameters across seven distortion levels.

  • Photometric distortions: Photometric distortions modify point-cloud colors through noise, quantization, intensity or contrast changes, saturation changes, and luminance changes.Examples include additive Gaussian noise, high-frequency noise, quantization noise, mean shift, contrast change, color-saturation change, correlated or multiplicative Gaussian noise, color quantization with dither, salt-and-pepper noise, Rayleigh noise, Gamma noise, uniform noise, and Poisson noise.
  • Local geometry distortions: Local Missing removes points from selected spatial anchors, and Local Offset or Local Rotation dislocates or rotates points within corresponding local regions.The anchor size is 0.3 times the bounding box, with the number of selected anchors increasing across distortion levels.
  • Photometric distortions: The dataset includes luminance distortion that offsets the Y component after conversion to YCbCr color space.The implementation modifies the luminance component with seven increasing offsets.
  • Reconstruction: Poisson Reconstruction downsamples a point cloud, reconstructs a mesh, and randomly samples mesh faces to restore the point count.Its down-sampling proportion matches that used for the separate down-sampling distortion.
  • Compression: Compression distortions include PCL octree compression and MPEG GPCC with lossless geometry and lossy attributes.Octree compression varies resolution, while GPCC supplies a compression-based impairment for point-cloud data.
Loading 2012.11895v4…