Source-linked AI summary
No-Reference Quality Assessment for 3D Colored Point Cloud and Mesh Models
Zicheng Zhang, Wei Sun, Xiongkuo Min, Tao Wang, Wei Lu, Guangtao Zhai
TL;DR
Existing 3D quality assessment often requires a reference model and rarely incorporates color, motivating a no-reference approach for colored point clouds and meshes. The proposed method combines geometry and color projections with 3D natural scene statistics and entropy, achieving strong performance across point-cloud and mesh databases.
Problem
Many 3D quality metrics are full-reference and cannot operate without pristine reference models, while few incorporate color information.
Method
The method projects colored 3D models into geometry and color feature domains, extracts 3D-NSS and entropy features, and uses machine learning to predict visual quality.
Results
The method is validated on SJTU-PCQA, WPC, and CMDM, outperforming most compared NR 3D-QA metrics; it ranks first among compared NR-PCQA metrics on SJTU-PCQA and second on WPC.
Takeaways & Limitations
The framework provides a no-reference quality assessment approach for both colored point cloud and mesh models using geometry and color information.
Abstract
from arXiv · showhide
To improve the viewer's Quality of Experience (QoE) and optimize computer graphics applications, 3D model quality assessment (3D-QA) has become an important task in the multimedia area. Point cloud and mesh are the two most widely used digital representation formats of 3D models, the visual quality of which is quite sensitive to lossy operations like simplification and compression. Therefore, many related studies such as point cloud quality assessment (PCQA) and mesh quality assessment (MQA) have been carried out to measure the visual quality degradations of 3D models. However, a large part of previous studies utilize full-reference (FR) metrics, which indicates they can not predict the quality level with the absence of the reference 3D model. Furthermore, few 3D-QA metrics consider color information, which significantly restricts their effectiveness and scope of application. In this paper, we propose a no-reference (NR) quality assessment metric for colored 3D models represented by both point cloud and mesh. First, we project the 3D models from 3D space into quality-related geometry and color feature domains. Then, the 3D natural scene statistics (3D-NSS) and entropy are utilized to extract quality-aware features. Finally, machine learning is employed to regress the quality-aware features into visual quality scores. Our method is validated on the colored point cloud quality assessment database (SJTU-PCQA), the Waterloo point cloud assessment database (WPC), and the colored mesh quality assessment database (CMDM). The experimental results show that the proposed method outperforms most compared NR 3D-QA metrics with competitive computational resources and greatly reduces the performance gap with the state-of-the-art FR 3D-QA metrics. The code of the proposed model is publicly available now to facilitate further research.
I. INTRODUCTION
3D point clouds and meshes support many applications but are degraded by lossy processing, creating a need for objective quality assessment. Existing methods often rely on reference models, while color-aware and no-reference approaches remain limited.
- Point clouds and meshes represent 3D models in applications including virtual reality, medical reconstruction, and video post-production.
- Simplification, compression, blur, and noise can damage the visual quality of 3D models.
- Objective quality metrics are needed because subjective assessment requires substantial time and expense.
- Full-reference methods compare reference and distorted models, but cannot operate when pristine references are unavailable.
- Existing quality-assessment research includes geometry-based point-cloud metrics, color-aware point-cloud methods, and model-based or image-based mesh metrics.
- Only a few no-reference point-cloud and mesh methods had been developed, including learning-based and multi-view approaches.
B. Our Approach
The paper extends natural scene statistics to colored point-cloud and mesh quality assessment without reference models. Its framework projects geometry and color features, estimates statistical descriptors, and regresses them to quality scores.
- 3D-NSS is proposed as a framework for quantifying visual quality from distortion-induced changes in feature distributions without reference models.
- The method projects 3D models into geometry and color feature domains, extracts 3D-NSS and entropy characteristics, and uses SVR to produce quality values.
- The proposed metric handles both colored point clouds and colored meshes using geometry and color features within a common no-reference framework.
- The paper investigates feature and NSS-model combinations and reports computational efficiency relative to state-of-the-art methods.
- The code is released to promote further development of no-reference 3D quality assessment.
- The framework includes feature extraction, parameter estimation, and regression modules for uniformly processing point clouds and meshes.
A. Geometry Feature Projection
The geometry projection stage derives local structural descriptors from point-cloud neighborhoods and mesh geometry. For point clouds, neighborhoods and covariance eigenvalues support five geometry feature domains.
- Geometry projection maps 3D models into quality-aware geometric feature domains because geometry features correlate strongly with human perception.
- Point-cloud neighborhoods are obtained with k-nearest neighbors using k=10 and Euclidean distance before covariance estimation.
- The covariance matrix of each point neighborhood is formed from centered three-dimensional coordinate vectors.
- Eigenvalue analysis produces three ordered eigenvalues for each point, which are used to describe local geometric structure.
- The selected point-cloud geometry domains are curvature, anisotropy, linearity, planarity, and sphericity.
- These geometry features represent local distribution patterns and are calculated at the point level.
- For mesh curvature, the calculation uses edge contributions weighted by the portion of each edge inside a sphere around a vertex.
2) Mesh Geometry Feature Domains:
The method extracts mesh geometry features that capture local surface structure and degradation, including curvature, dihedral angles, face areas, and face angles.
- Mesh geometry feature extraction differs from point-cloud processing because meshes contain vertices, edges, and faces.
- Curvature: Weighted average curvature describes local mesh roughness or smoothness using the mesh’s edge and face information.
- Dihedral Angle: Oriented dihedral angles between adjacent faces indicate mesh simplification loss while distinguishing convex and concave configurations.
- Face Area and Angle: Face area and angle are computed from vertex coordinates and are related to mesh smoothing and compression operations.
- The mesh is projected into four geometry feature domains to represent quality-relevant structural characteristics.
B. Color Feature Projection
The method projects colored point clouds and meshes into LAB color feature domains because RGB has weak correlation with human perception.
- Color contributes significantly to visual quality assessment for both colored point clouds and colored meshes.
- The method uses LAB color transformation as the color feature projection instead of RGB.
- For colored point clouds, color comes directly from point attributes, whereas mesh surface color is generally rendered from contained vertex colors.
- The color projection transforms RGB inputs through XYZ-related quantities into LAB color channels.
III. ESTIMATING STATISTICAL PARAMETERS
The method estimates quality-aware statistical features from normalized geometry and color distributions using entropy and several natural scene statistic models.
- Entropy and generalized Gaussian, asymmetric generalized Gaussian, and shape-rate Gamma models quantify perceptual quality from feature distributions.
- Each feature domain undergoes normalization before statistical parameter estimation.
- Entropy is used because quantization distortion makes LAB-channel distributions sparser and reduces the number of distinct colors.
- The normalized feature distributions include both geometry and color domains, whose entropy is computed as a quality-aware feature.
B. GGD Parameters
The generalized Gaussian distribution models changes in 3D feature-distribution shapes, particularly their tails, using estimated shape and variance parameters.
- Reference geometry-feature distributions can appear Gamma-like or Gaussian-like, and distortions change their distribution shapes.
- The generalized Gaussian distribution captures a broad range of statistics and changes in distribution tails.
- The estimated GGD parameters α and β² represent distribution shape and variance.
- Because normalized distributions have fixed variance, GGD parameters are estimated before normalization.
C. AGGD Parameters
The method uses AGGD parameters to describe asymmetric feature distributions whose tails change under 3D distortions.
- Reference planarity is left-skewed, whereas a distorted planarity distribution can become right-skewed.
- Compression, downsampling, and Gaussian noise alter the tail weights of curvature and anisotropy distributions.
- AGGD models asymmetric distributions by estimating parameters for the different spread extents on their left and right sides.
- The AGGD extends GGD, and its four estimated parameters describe asymmetric distribution characteristics.
- Gamma distribution parameters are used for curvature and anisotropy because their distorted distributions retain Gamma-like appearances while changing shape and scale.
E. Parameters Summary
The method summarizes geometry and color feature domains with basic statistics, entropy, and fitted distribution parameters, then uses SVR to regress a quality score.
- The extracted feature vector includes average, standard deviation, entropy, and parameters from GGD, AGGD, and Gamma models.
- Colored point clouds yield 88 features from 8 domains, while colored meshes yield 77 features from 7 domains.
- An SVR regressor maps the normalized high-dimensional feature vector to an integrated quality score.
- Evaluation uses SJTU-PCQA and WPC point-cloud databases, with training and testing procedures designed to avoid content overlap.
- SJTU-PCQA contributes 378 publicly available samples, while WPC contains 740 distorted point clouds from 20 source point clouds.
- Comparisons include image-based and model-based metrics, spanning both full-reference and no-reference approaches.
2) Experiment Setup for MQA:
The MQA evaluation uses the CMDM database and compares the proposed colored-mesh method with image-based and model-based metrics using standard correlation and error criteria.
- Experiment Setup for MQA: CMDM contains 5 source models with geometry and color distortions across 4 distortion types and 4 strengths.
- Experiment Setup for MQA: The Aix, Chamleon, Fish, and Samurai models are selected for training, while Ari is reserved for testing.
- Experiment Setup for MQA: Competitors include FR image-based PSNR and SSIM, NR image-based NIQE and BRISQUE, FR-MQA CMDM, and geometry-only NR-MQA methods.
- Experiment Setup for MQA: Performance is measured with SRCC, KRCC, PLCC, and RMSE, where the first three should approach 1 and RMSE should approach 0.
- PCQA validation: The PCQA evaluation places the proposed method first among compared NR metrics on SJTU-PCQA and second on WPC.
- PCQA validation: The WPC comparison is interpreted alongside differences in sample complexity, colorfulness, and whether distortions are mixed or isolated.
2) The performance of MQA validated on the CMDM Database:
On CMDM, coarse-grained and visibly distinct distortions support strong performance, while color-aware model features outperform geometry-only no-reference mesh metrics.
- The CMDM database uses four distortion strengths spanning imperceptible to highly impaired visual-quality levels.
- Visible degradations at the four strengths allow simple full-reference image-based metrics to achieve competitive performance.
- The proposed method ranks second in CMDM PLCC and SRCC, slightly behind the full-reference CMDM metric.
- Geometry-only NR-MQA methods underperform the proposed method because they do not incorporate color information.
- Ablation Study: The ablation study finds geometry features contribute more than color features across all three databases.
- Ablation Study: Simple statistics can outperform the full feature set on SJTU-PCQA, whereas the full set is significantly better on WPC.
- Ablation Study: The authors recommend the full feature-group model for practical use because it better describes and distinguishes diverse distributions.
F. Statistical Test
The proposed method is statistically superior to most compared metrics across the evaluated databases, while remaining computationally practical. Its performance also improves with more diverse training data, and the paper reports competitive computational resources.
- Our method significantly outperforms 7 compared PCQA metrics on SJTU-PCQA and WPC, and 7 compared MQA metrics on CMDM.
- PCQM performs significantly better than our method on SJTU-PCQA and WPC, whereas CMDM is insignificantly distinguishable from our method on CMDM.
- Increasing training samples improves performance, with stronger gains on CMDM because its distortions are coarse-grained and its mesh sample count is relatively small.
- Greater training-sample diversity and quantity benefit robustness and performance.
- The method is evaluated for computational efficiency against GraphSIM, PointSSIM, PCQM, and PCMRR using a specified desktop CPU and 8 GB RAM.
- Overall, the method outperforms most compared NR 3D-QA metrics with competitive computational resources and narrows the gap with state-of-the-art FR metrics.