Source-linked AI summary
Perceptual Quality Assessment of Colored 3D Point Clouds
Honglei Su, Qi Liu, Zhengfang Duanmu, Wentao Liu, Zhou Wang
TL;DR
Colored 3D point-cloud quality assessment lacks effective, broadly useful benchmarks and models. The paper constructs the WPC database, evaluates existing methods, and proposes an information-content-weighted structural-similarity model. The proposed model correlates well with subjective opinions and significantly outperforms existing PCQA methods, while the database supports reproducible evaluation.
Problem
Existing point-cloud databases and objective PCQA models provide limited support for reliable subjective-quality assessment, despite practical applications requiring high-quality colored point clouds.
Method
The paper builds the WPC database from 20 diverse high-quality point clouds and 740 distorted versions, conducts a subjective study, and proposes the IW-SSIMp model.
Results
IW-SSIMp significantly outperforms existing objective PCQA methods and is statistically better than all existing models in the reported analysis.
Takeaways & Limitations
The WPC database provides a large subject-rated benchmark, while IW-SSIMp offers a strong objective predictor across the evaluated databases, except IRPC.
Takeaways & Limitations
Existing PCQA databases remain constrained by acquisition quality, viewpoints, and content diversity, boundaries the WPC database was designed to address.
Abstract
from arXiv · showhide
The real-world applications of 3D point clouds have been growing rapidly in recent years, but not much effective work has been dedicated to perceptual quality assessment of colored 3D point clouds. In this work, we first build a large 3D point cloud database for subjective and objective quality assessment of point clouds. We construct 20 high quality, realistic, and omni-directional point clouds of diverse contents. We then apply downsampling, Gaussian noise, and three types of compression algorithms to create 740 distorted point clouds. We carry out a subjective experiment to evaluate the quality of distorted point clouds. Our statistical analysis finds that existing objective point cloud quality assessment (PCQA) models only achieve limited success in predicting subjective quality ratings. We propose a novel objective PCQA model based on the principle of information content weighted structural similarity. Our experimental results show that the proposed model well correlates with subjective opinions and significantly outperforms the existing PCQA models. The database has been made publicly available to facilitate reproducible research at https://github.com/qdushl/Waterloo-Point-Cloud-Database.
1 INTRODUCTION
The paper addresses limitations in colored 3D point-cloud quality assessment by introducing the WPC database and a new objective model. The database supports subjective evaluation and benchmarking, while the proposed model improves correlation with subjective quality.
- 3D point clouds support applications requiring faithful geometry and perceptual attributes, but acquisition, compression, transmission, storage, and rendering can degrade quality.
- Subjective assessment is reliable because the human visual system is the ultimate receiver, while large subject-rated databases support behavioral analysis, benchmarking, and model development.
- Existing public databases often have inferior acquisition quality, constrained viewpoints, and insufficient content types, while subjective tests are costly and time-consuming.
- The WPC database contains 20 diverse high-quality source point clouds and 740 distorted point clouds evaluated through mean opinion scores.
- Existing objective PCQA models show limited correlation with subjective quality, whereas the proposed information-content-weighted SSIM model significantly outperforms them.
- The paper contributes detailed WPC analysis, comprehensive PCQA-model comparison, and a new model achieving state-of-the-art performance on the WPC and other popular databases.
2 RELATED WORK
Prior PCQA databases and subjective tests vary substantially in quality, viewing conditions, interaction, and content coverage. Existing objective models span geometry, color, point-based, and projection-based approaches, motivating more consistent evaluation and improved metrics.
- Existing PCQA Databases: Existing PCQA databases include limitations such as acquisition noise, irregular edges, front-only scanning, restricted viewpoints, and low content diversity.
- Existing PCQA Databases: Subjective PCQA tests differ in scoring methodology, display, interaction method, and rendering mode, making experimental settings inconsistent across studies.
- Existing PCQA Databases: Most tests use 2D monitors and point-based rendering; passive predetermined viewing offers repeatability and reproducibility advantages over interactive viewing.
- Objective PCQA Models: Objective PCQA models are categorized by distortion type into geometry-only and geometry-plus-color metrics, and by feature extraction into point-based and projection-based models.
- Objective PCQA Models: Geometry metrics use point-to-point, point-to-plane, angular, or curvature similarities to quantify geometric distortion.
- Objective PCQA Models: Colored-point-cloud methods estimate quality using color measures, geometric and color features, joint features, or reduced-reference representations.
- Objective PCQA Models: Projection-based models include PSNRp, SSIMp, MS-SSIMp, and VIFPp, extending image-quality measures to projected point clouds.
3 POINT CLOUD DATABASE CONSTRUCTION
The Waterloo Point Cloud database is constructed from diverse, omni-directionally acquired objects and systematically distorted to support perceptual quality assessment.
- Point Cloud Construction: The source collection includes objects with diverse geometric and textural complexity that are moderate in size and omni-directionally acquirable.Examples include snacks, fruits, vegetables, office supplies, and containers.
- Point Cloud Construction: Images are captured from multiple perspectives with a single-lens-reflex camera and turntable, then aligned, reconstructed, merged, refined, and resampled.The process uses Agisoft Photoscan, Screened Poisson Surface Reconstruction, and CloudCompare.
- Point Cloud Construction: 20 normalized voxelized source point clouds contain 400K to 3M points, averaging 1.35M points.Each cloud is normalized to a unit cube with step size 0.001, and duplicate points are removed.
- Distortion Generation: The database applies downsampling, Gaussian noise, and MPEG-PCC compression to generate diverse geometry and texture distortions.Downsampling uses octree levels 7, 8, and 9; Gaussian noise affects geometry and texture independently.
- Distortion Generation: 740 distorted point clouds are generated from 20 originals, yielding 760 original and distorted point clouds in total.The distortions include texture effects such as blockiness and blur, alongside point-cloud-specific geometric artifacts.
4 SUBJECTIVE EXPERIMENTS
The subjective study uses standardized passive viewing and DSIS scoring to collect ratings, then evaluates score reliability and existing PCQA models against MOS.
- Subjective User Study: Passive watching uses rendered video sequences with horizontal and vertical circular camera paths to standardize viewpoints and viewing time.The renderer uses a 960×960 window, point size 1, and point rendering.
- Subjective User Study: 60 subjects with normal or corrected-to-normal vision participate after training on separate videos.Subjects are aged 21–40 and view videos on a calibrated 23.6-inch LCD monitor.
- Subjective User Study: Each video is scored 30 times, producing 22,800 subjective ratings, including 600 scores for reference point clouds.Subjects are assigned overlapping sets of 10 objects in a circular fashion.
- Subjective Data Analysis: MOS is computed by averaging rescaled Z-scores after applying the specified outlier-removal scheme.The rescaled scores lie in the range [0, 100].
- Subjective Data Analysis: Individual subjects perform consistently with relatively low variation, while existing PCQA models only moderately correlate with human perception.PLCC and SRCC evaluate subject–MOS consistency and model predictions against MOS; the results leave substantial room for improvement.
5 OBJECTIVE QUALITY ASSESSMENT
The proposed IW-SSIMp assesses colored 3D point-cloud quality by projecting reference and distorted clouds into multiple views and averaging information-content-weighted image similarities. On the WPC database, it achieves the best predictive performance among evaluated models, with statistically significant advantages over existing methods.
- Projection pipeline: The method transforms each point cloud through translation, rotation, scaling, orthogonal projection, and rasterization to produce comparable images from multiple viewpoints.Translation uses the reference cloud’s geometric center, rotation follows icosphere viewpoints, and scaling aims to preserve detail while producing approximately watertight snapshots.
- Information-weighted similarity: IW-SSIMp removes background influence and averages IW-SSIM similarities between reference and distorted projections across viewpoints.The approach weights spatial regions by information content because background pixels contain no point-cloud information and perceptual importance varies across regions.
- Model configuration: Nv = 12 is used in reported results because performance is close for Nv = 12, 42, and 162 while computational complexity increases with Nv.The model does not require training and is independent of existing PCQA databases, including WPC.
- Validation results: The proposed model delivers the best performance on the whole database and almost every subset, with PLCC and SRCC at the level of an average human subject.Evaluation uses PLCC, SRCC, and RMSE comparisons against existing objective PCQA models.
- Statistical significance: Statistical testing finds IW-SSIMp significantly better than all existing models, while most geometry-plus-color metrics outperform geometry-only metrics.The analysis compares prediction-residual variances using F-statistics and marks pairwise outcomes as better, worse, or statistically indistinguishable.
- Generalization: Cross-database validation shows strong performance except on IRPC, while PCQM leads on SJTU-PCQA and GraphSIM leads on IRPC and M-PCCD.GraphSIM is competitive with IW-SSIMp but has high time complexity in the actual experiment.
6 CONCLUSION
The work establishes a large, diverse point-cloud database with subjective ratings and finds that existing PCQA models are unreliable, while the proposed IW-SSIMp model significantly outperforms them.
- Database construction: 20 high-quality, realistic, omni-directional dense point clouds cover diverse geometric and textural complexity, averaging 1.35M voxelized points.The standard deviation is 656K points.
- Database construction: 740 distorted point clouds have MOSs approximately evenly distributed from poor to excellent perceived quality levels.
- Model evaluation: Existing state-of-the-art PCQA models do not provide reliable predictions of perceived quality.
- Model evaluation: The projection-based IW-SSIMp model significantly outperforms existing objective PCQA methods.