PaperlayerView exact revision on arXiv

Source-linked AI summary

Point Cloud Rendering after Coding: Impacts on Subjective and Objective Quality

Alireza Javaheri, Catarina Brites, Fernando Pereira, Joao Ascenso

arXiv:1912.09137v2eess.IVcs.MM

TL;DR

Point-cloud quality depends on both coding and rendering, yet prior work often used less efficient coding and did not consistently assess rendering effects. This paper evaluates subjective quality and objective geometry metrics across codecs and renderings; its results show that artifact visibility and metric performance vary with codec and rendering method.

  • Problem

    Earlier quality studies often relied on less efficient coding, while the impact of rendering on perceived quality and metric accuracy remained insufficiently assessed.

    Source: §I · pp. 1–2

  • Method

    The paper evaluates subjective quality and objective geometry metrics for three coding and three rendering solutions, using no color-attribute coding to isolate geometric artifacts.

    Source: §I · p. 2

  • Results

    Geometric distortions are visible for PCL across rendering methods, while color attributes and, to a lesser extent, surface reconstruction can mask them; selected metrics perform poorly for V-PCC data.

    Source: 1) Impact of Coding on the PC Metric… · p. 12, §VI · p. 14

  • Takeaways & Limitations

    Codec artifacts and rendering choices affect perceived quality differently, and objective metrics may not capture the higher perceived similarity associated with increased decoded-point density.

    Source: 1) Impact of Coding on the PC Metric… · p. 12, §VI · p. 14

    The evaluation excludes color-attribute coding to isolate the impact of geometric distortions.

    Source: §I · p. 2

Abstract

from arXiv · show

Recently, point clouds have shown to be a promising way to represent 3D visual data for a wide range of immersive applications, from augmented reality to autonomous cars. Emerging imaging sensors have made easier to perform richer and denser point cloud acquisition, notably with millions of points, thus raising the need for efficient point cloud coding solutions. In such a scenario, it is important to evaluate the impact and performance of several processing steps in a point cloud communication system, notably the quality degradations associated to point cloud coding solutions. Moreover, since point clouds are not directly visualized but rather processed with a rendering algorithm before shown on any display, the perceived quality of point cloud data highly depends on the rendering solution. In this context, the main objective of this paper is to study the impact of several coding and rendering solutions on the perceived user quality and in the performance of available objective quality assessment metrics. Another contribution regards the assessment of recent MPEG point cloud coding solutions for several popular rendering methods which were never presented before. The conclusions regard the visibility of three types of coding artifacts for the three considered rendering approaches as well as the strengths and weakness of objective quality metrics when point clouds are rendered after coding.

I. INTRODUCTION

Point-cloud quality depends on coding and rendering, but earlier studies often used less efficient coding, inconsistent conditions, or limited metric sets. This paper jointly examines subjective quality and objective geometry metrics for coded static point clouds across rendering methods.

I. INTRODUCTION

  • Point clouds can contain millions or billions of points, making efficient coding and quality assessment important for immersive applications.The paper focuses on geometry degradations, which can create holes, deformed or noisy surfaces, and reduce realism and immersion.Source: §I · pp. 1–2

II. RELATED WORK

  • Prior studies often used less efficient or simulated distortions and inconsistent test conditions, leaving rendering-method differences without solid quantification.Many also assessed only a single type of objective metric.Source: §I · pp. 1–2, §II · p. 3

I. INTRODUCTION

  • The study jointly evaluates coding and rendering impacts on subjective quality and objective geometry metrics for static point clouds, without coding color attributes.It examines geometry artifacts from state-of-the-art codecs across rendering scenarios.Source: §I · pp. 1–2
  • The paper contributes a subjective study of coding-rendering combinations, an evaluation of objective metrics across artifact types, and a public MOS dataset of coded point clouds.The dataset targets codec artifacts not considered when popular point-cloud metrics were designed.Source: §I · p. 2

III. POINT CLOUD CODING AND RENDERING SOLUTIONS

This section introduces the rendering and coding solutions used in the experiments. Rendering produces the displayable visual representation, and the choice between point-based and converted representations involves practical trade-offs.

III. POINT CLOUD CODING AND RENDERING SOLUTIONS

  • The section reviews three rendering solutions and three state-of-the-art codecs, characterizing their associated coding artifacts for later assessment.The reviewed approaches are used in the paper’s subjective and objective evaluations.Source: §III · p. 3

A. Selected Point Cloud Rendering Solutions

  • Rendering converts point-cloud data into a visual representation for a display, and can significantly affect perceived quality.The paper describes point-based rendering and conversion to another representation, commonly a surface such as a polygonal mesh.Source: A. Selected Point Cloud Rendering… · pp. 3–4
  • Point-based rendering can simplify processing of large point clouds, while conversion to another representation may lose information or be infeasible for complex or sparse scenes.Point-based models can also ease streaming and processing subsets when the full point cloud exceeds available memory.Source: A. Selected Point Cloud Rendering… · p. 4
  • A geometry shader constructs the final displayed image using primitives and produces lighting, darkness, and color.For geometry-only point clouds, shading commonly uses a single color; otherwise, point or vertex color attributes may be used.Source: A. Selected Point Cloud Rendering… · p. 4

1) Point-based Rendering without Color Attributes

The paper compares two point-based renderings and a mesh-based rendering approach, then selects three codecs that represent point clouds using tree, surface, and patch structures. Their rendering choices and coding structures shape how geometry is represented and assessed.

1) Point-based Rendering without Color Attributes

  • RPoint uses square point primitives sized to fill gaps, with no color attributes and normals estimated from local surface planes.The point size is set to avoid holes; normal orientation is determined with a minimum spanning tree algorithm.Source: 1) Point-based Rendering without Color… · p. 4

2) Point-based Rendering with Color Attributes

  • RColor applies the point-based rendering method with acquired color attributes, which may mask geometric distortions but leave them more visible at object boundaries.The decoded points are recolored with original attributes to isolate geometric distortions; no relighting is applied.Source: 2) Point-based Rendering with Color… · p. 4

3) Mesh-based Rendering

  • RMesh reconstructs a polygonal surface before rendering, producing a seamless surface but potentially removing high-frequency geometric detail.The paper uses Poisson Surface Reconstruction with default CloudCompare parameters and omits color attributes.Source: 3) Mesh-based Rendering · pp. 4–5

B. Selected Point Cloud Coding Solutions

  • The selected PCL, G-PCC, and V-PCC codecs represent point clouds using tree, surface, and patch structures, respectively, and produce distinct geometry artifacts.The artifacts include lost geometric detail, deformations, holes, and curved surfaces represented as planes.Source: B. Selected Point Cloud Coding Solutions · p. 5

1) PC Coding with Tree Structures

The section describes tree-, surface-model-, and patch-based point-cloud coding, then illustrates how low-rate PCL artifacts vary across rendering methods.

1) PC Coding with Tree Structures

  • PCL represents point-cloud geometry with an octree, where occupied voxels are subdivided and encoded as occupancy bytes traversed breadth-first.Decoding depth sets the minimum voxel size; points within each occupied voxel become one point at its center.Source: 1) PC Coding with Tree Structures · p. 5

2) PC Coding with Surface Models

  • G-PCC extends pruned-octree geometry coding with a parameterized surface model that represents points in leaf voxels using triangles.The paper uses G-PCC TMC1 v1.1; when octree level is below depth, TriSoup supplies additional geometry within voxel blocks.Source: 2) PC Coding with Surface Models · p. 5

3) PC Coding with Patch-based Projection

C. Coding Artifacts

  • The artifact examples use low-rate coding and were selected to show typical distortions; broader examples cover all codecs and rendering combinations in the supplementary material.The section frames artifact characterization as useful for interpreting subjective tests and objective-metric limitations.Source: C. Coding Artifacts · p. 6

1) PCL Codec

  • At low rate, PCL looks pixelated in RPoint and RColor but loses face and hand detail in RMesh as lower octree depth reduces decoded points.Fig. 2 shows the Loot example across the rendering methods.Source: Figure 2 · p. 6, 1) PCL Codec · p. 6

2) MPEG G-PCC Codec

G-PCC and V-PCC produce distinct rendering artifacts, and the study evaluates coded point clouds under three rendering approaches using varied test materials and codec settings.

2) MPEG G-PCC Codec

  • TriSoup in G-PCC can create false edges at block or triangle boundaries and artificial or enlarged polygonal holes in sparse regions.The resulting triangles may be visible at low rates.Source: 2) MPEG G-PCC Codec · p. 6

3) MPEG V-PCC Codec

  • V-PCC artifacts include blockiness, false edges, and missing points at patch boundaries; their visibility differs across RPoint, RColor, and RMesh.False edges are most visible with RPoint, distortions are not very visible with RColor, and RMesh produces smoother results but can lose details such as the bell tower.Source: 3) MPEG V-PCC Codec · p. 6, Figure 4 · p. 6

A. Test Conditions

  • The test set uses six MPEG point clouds spanning object, building, and people categories, with varied density, geometry, and color characteristics.The selection includes four class A and two class B point clouds.Source: A. Test Conditions · pp. 6–7, Table I · p. 7, Figure 5 · p. 7
  • Three codecs—PCL, G-PCC, and V-PCC—were tested at low, medium, and high perceptual quality, with three separate sessions using RPoint, RColor, and RMesh.The sessions rendered standardized videos on a 2D display; PCL and G-PCC settings are listed in Table II, and V-PCC settings in Table III.Source: A. Test Conditions · p. 7, Table II · p. 7, Table III · p. 7

C. Subjective Quality Assessment Methodology

The subjective study uses paired original and decoded renders to assess similarity while reducing the influence of source-point-cloud characteristics. Its analysis focuses on coding-artifact visibility across rendering approaches, with mean opinion scores (MOS) representing fidelity rather than intrinsic quality.

C. Subjective Quality Assessment Methodology

  • Paired original and decoded point-cloud views were used because source noise and density can affect scores, and the DSIS method helps mitigate those influences.Subjects scored the decoded render's similarity to the original.Source: C. Subjective Quality Assessment… · p. 8
  • Each session assessed 54 stimuli across six point clouds, three codecs, and three rate points, using a 1–5 impairment scale.Twenty subjects participated per session; one RMesh-session outlier was identified, and scores were averaged into MOS.Source: C. Subjective Quality Assessment… · p. 8

D. Experimental Results and Analysis

  • The analysis uses MOS as similarity between original and decoded point clouds, not as intrinsic point-cloud quality, and examines artifact visibility across renderings.The stated focus is the impact of rendering solutions on perceived quality for differently coded point clouds.Source: D. Experimental Results and Analysis · p. 8

1) Impact of Rendering on Perceived PC Quality

Across rendering sessions, RColor receives the highest subjective scores, followed by RMesh and RPoint. Coding-artifact visibility varies by codec and rendering approach: color and surface reconstruction can mask some distortions, but not all.

1) Impact of Rendering on Perceived PC Quality

2) Impact of Rendering on the Coding Artifacts Visibility

  • PCL artifacts remain visible across rendering methods because reduced point counts cause pixelation or detail loss, including reconstruction-related distortions in some RMesh surfaces.Lower octree depth reduces decoded point counts; larger point sizes are then needed for RPoint and RColor.Source: 2) Impact of Rendering on the Coding… · p. 9
  • For G-PCC, RColor masks surface distortions, while RMesh scores slightly better than RPoint by mitigating some holes and false edges.Severe cases can still show false edges, holes, and boundary distortions.Source: 2) Impact of Rendering on the Coding… · p. 9
  • V-PCC distortions are less visible with RColor and RMesh than with RPoint; color masks distortions, while surface reconstruction improves perceived quality over RPoint.V-PCC uses 2D texture and depth maps with HEVC coding, producing consistently represented surfaces with some error relative to the original.Source: 2) Impact of Rendering on the Coding… · p. 9

3) Statistical Significance Analysis of Subjective Assessment

Statistical tests establish an overall subjective-quality ranking of RColor above RMesh above RPoint, while codec-specific comparisons differ. The following objective-metrics section evaluates geometry metrics against MOS and considers their performance under coding artifacts and rendering.

3) Statistical Significance Analysis of Subjective Assessment

  • The analysis uses Welch ANOVA and Games–Howell tests at a 5% significance level, alongside Wilcoxon signed-rank tests for the non-normal ALL case.Welch ANOVA compares rendering-group means without requiring equal variances; the Wilcoxon test compares mean ranks without assuming a data distribution.Source: 3) Statistical Significance Analysis… · p. 10, Table IV · p. 10, Table V · p. 10
  • Codec-specific tests find no significant rendering differences for PCL; for G-PCC and V-PCC, RColor scores better than RPoint, with no significant difference between RPoint and RMesh.For G-PCC, RColor also does not differ significantly from RMesh; for V-PCC, RPoint and RMesh are statistically similar as the second-best methods.Source: 3) Statistical Significance Analysis… · p. 10
  • For all codecs together, Wilcoxon tests find significant differences across every rendering pair, ranking RColor above RMesh above RPoint.The comparison is reported as statistically significant for RPoint–RColor, RPoint–RMesh, and RColor–RMesh.Source: 3) Statistical Significance Analysis… · p. 10, Table VI · p. 10

V. PC RENDERING AFTER CODING: OBJECTIVE METRICS PERFORMANCE ASSESSMENT

  • The objective-metrics section evaluates geometry metrics by correlating them with MOS to assess metric performance across coding degradations and rendering solutions.The stated scope is geometry-error metrics, since coding artifacts may have a higher negative impact on user quality of experience through geometry.Source: §V · p. 10

A. Geometry Objective Assessment Metrics

The section introduces full-reference geometry metrics for comparing decoded point clouds with originals, then describes point-to-point correspondences and related error measures. Point-to-plane metrics instead project point errors onto local surface normals.

A. Geometry Objective Assessment Metrics

  • Geometry metrics support fidelity and rate-distortion assessment, coding optimization, and end-to-end quality measurement; they compare decoded and original point clouds using point correspondences and distances.The section groups metrics as point-to-point, point-to-plane, and plane-to-plane, according to the geometric quantities compared.Source: A. Geometry Objective Assessment Metrics · pp. 10–11

1) Point-to-Point (Po2Point) Objective Metrics

  • Point-to-point metrics find nearest-neighbor correspondences in both directions between original and degraded point clouds, then compute distances between matched points.The direct direction maps original points to degraded points; the inverse direction maps degraded points to original points.Source: 1) Point-to-Point (Po2Point) Objective… · p. 11
  • Point-to-plane metrics project point-to-point error vectors onto local surface normals, producing distances to the reference surface; MPEG uses Po2Plane PSNR as D2 for G-PCC and V-PCC evaluation.For the inverse direction, normals are estimated by averaging those of the nearest neighbors in the original point cloud.Source: 1) Point-to-Point (Po2Point) Objective… · p. 11

3) Plane-to-Plane (Pl2Plane) Objective Metrics

Plane-to-plane metrics compare local surface orientations using angular differences between matched planes. Point-to-plane metrics represent local surfaces with tangent planes, and the evaluated metrics normalize point-cloud coordinates to support comparisons across differing precisions.

2) Point-to-Plane (Po2Plane) Objective Metrics

  • Point-to-plane metrics fit a local plane to each point’s neighborhood and use its normal as the tangent plane’s normal; correspondences are computed in both directions.These planes serve as local linear approximations of the underlying object surface.Source: 2) Point-to-Plane (Po2Plane) Objective… · p. 11

3) Plane-to-Plane (Pl2Plane) Objective Metrics

  • Plane-to-plane metrics compare tangent-plane similarity for corresponding points using the angular difference between their normal vectors.The inverse cosine of cosine similarity gives the angular distance; the resulting distances are pooled using mean, mean-squared, or root-mean-squared angular difference, symmetrically across both directions.Source: 2) Point-to-Plane (Po2Plane) Objective… · pp. 11–12
  • All point clouds are normalized to coordinates in [0,1] before metric computation because differing coordinate precision can make error vectors incomparable; Po2Point and Po2Plane PSNR are exceptions because their peak value scales for bit depth.The peak constant depends on the point cloud’s coordinate precision.Source: 2) Point-to-Plane (Po2Plane) Objective… · p. 12

B. Experimental Results and Analysis

Objective-metric performance varies with both coding artifacts and codec: PCL and G-PCC are assessed more reliably than V-PCC, and no metric performs well across all codecs. Increased decoded-point density can raise perceived quality without being captured by geometry metrics.

1) Impact of Coding on the PC Metric Assessment

  • For PCL, Po2Plane and Po2Point correlate highly across renderings, while PSNR is best overall; octree pruning makes errors and distortions visible even after RMesh reconstruction.As compression increases, points are discarded and the remaining points lie farther from the original surface.Source: 1) Impact of Coding on the PC Metric… · p. 12, Table VII · p. 12
  • For G-PCC, metric performance is 4 to 5% lower than PCL, with Po2Point strongest only for RPoint and RColor; metrics perform poorly for V-PCC.G-PCC and V-PCC add points, increasing density and MOS, an effect the metrics miss; V-PCC errors are less visible, while G-PCC artifacts are more visible when triangulation fails.Source: 1) Impact of Coding on the PC Metric… · p. 12, Table VII · p. 12, Figure 9 · p. 13
  • At the same Po2Point PSNR of 60 dB, V-PCC has MOS 3.15 versus G-PCC’s 1.1 because its errors cluster nearer zero and are less visible.The mismatch also occurs for Po2Point and Po2Plane MSE.Source: 1) Impact of Coding on the PC Metric… · p. 13, Figure 10 · p. 13
  • Overall, metric correlation with MOS depends on coding distortions: it is satisfactory for PCL and G-PCC, poor for V-PCC, and no metric performs well across all codecs.This limits comparison of rate-distortion performance across different coding paradigms.Source: 1) Impact of Coding on the PC Metric… · pp. 12–13

2) Impact of Rendering on the PC Metrics Assessment

Rendering changes artifact visibility and metric correlation: RColor yields the best correlations overall, while RMesh generally yields low correlations. Across codecs and renderings, Po2Point and Po2Plane PSNR perform best, though performance varies by metric and coding distortion.

2) Impact of Rendering on the PC Metrics Assessment

  • Across all data, PLCC is below 78.4%; correlations are best for RColor and lower for RMesh, where most metrics correlate poorly, especially for G-PCC and V-PCC.The authors attribute the RColor advantage to subjects better perceiving medium- and high-quality degradations, while RMesh converts point clouds to polygonal meshes.Source: 2) Impact of Rendering on the PC… · p. 13

3) PC Objective Metrics Correlation Assessment

  • Po2Point metrics perform especially well for PCL and G-PCC with RPoint or RColor; Hausdorff variants can reach 90.07 for PCL in RPoint but are unreliable across mixed distortions because they use only maximum error.Average pooling, as in MSE and PSNR, can mitigate sensitivity to outliers.Source: 3) PC Objective Metrics Correlation… · p. 13
  • Po2Plane metrics perform similarly to Po2Point, and Po2Plane PSNR is more consistent than its MSE counterpart because its peak value normalizes the errors.The peak is computed from geometry coordinate precision.Source: 3) PC Objective Metrics Correlation… · p. 13
  • Pl2Plane generally has worse PLCC than Po2Point and Po2Plane because decoded-point normals can be unreliable, but it is best for V-PCC with RColor and RMesh.V-PCC geometry errors mainly come from depth-map coding artifacts, which are more consistent across the point cloud.Source: 3) PC Objective Metrics Correlation… · p. 13
  • Across all codecs and renderings, Po2Point and Po2Plane PSNR have the highest correlation with subjective scores; statistical tests find no significant performance difference between them.These are the metrics selected by MPEG for the PCC Call for Proposals and used in common test conditions.Source: 4) Statistical Significance Analysis… · p. 14
  • The authors report that the MPEG-selected metrics’ choice was validated against MOS, while noting substantial room to improve objective-metric performance.They describe this as the first validation using MOS from a well-defined procedure.Source: 4) Statistical Significance Analysis… · p. 14

VI. FINAL REMARKS

The study finds that coding artifacts and rendering jointly shape perceived point-cloud quality, while available geometry metrics are unreliable for some codec and rendering combinations. Its challenges discussion connects these findings to coding, subjective-test design, and render-aware metrics.

VI. FINAL REMARKS

  • Geometric coding artifacts vary in visibility by codec and rendering: color attributes mask geometry distortions, surface reconstruction masks them to a lesser extent, and PCL distortions remain visible across renderings.The paper finds current objective metrics unreliable for some settings, including V-PCC, and notes limited accuracy for particular rendering solutions.Source: §VI · p. 14
  • Evaluating color attributes coded at different rates or qualities remains future work, to identify geometry–color trade-offs for perceived quality.Source: §VI · p. 14

VII. PC QUALITY ASSESSMENT: CHALLENGES AND WAYS FORWARD

  • Increasing decoded-point density can improve perceived quality, but memory and computational-speed constraints may limit that approach in practical applications.The authors suggest coupling decoding and rendering, or sampling variable numbers of points from triangles or projection maps.Source: §VII · pp. 14–15
  • Subjective studies should account for rendering and assess both geometry-only and geometry-plus-color quality, since color can mask geometry deformations in applications where fidelity matters.The paper notes that mesh rendering can also mask some artifacts when the final rendering method is unknown.Source: §VII · p. 15
  • Objective metrics should account for rendering: point-based methods could incorporate projected point density, while mesh-based methods could use perceptually relevant surface and color features.The authors identify poor metric performance for mesh rendering as motivation for these directions.Source: §VII · p. 15
Loading 1912.09137v2…