PaperlayerView exact revision on arXiv

Source-linked AI summary

Deformable Radial Kernel Splatting

Yi-Hua Huang, Ming-Xian Lin, Yang-Tian Sun, Ziyi Yang, Xiaoyang Lyu, Yan-Pei Cao, Xiaojuan Qi

arXiv:2412.11752v2cs.CVcs.GR

TL;DR

Gaussian kernels are efficient for 3D scene representation but struggle with diverse shapes and sharp boundaries because of their symmetry and smoothness. DRK generalizes Gaussian splatting with learnable planar radial bases, shape controls, and efficient rendering strategies. Experiments report superior rendering quality and fewer primitives, while performance is less robust to camera noise on Tank&Temple.

  • Problem

    Gaussian kernels’ radial symmetry and smoothness limit representation of complex shapes and sharp boundaries, often requiring many primitives for detailed geometry.

    Source: §1 · p. 1, §3.4 · pp. 3–4

  • Method

    DRK extends Gaussian splatting with learnable radial bases, adjustable angles and scales, curvature and sharpness controls, and efficient planar rasterization.

    Source: §1 · p. 2, §4 · p. 4, §4.4 · p. 5

  • Results

    DRK achieves superior rendering quality with significantly fewer primitives while maintaining computational efficiency across synthetic and real-world scenes.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    DRK provides a flexible shape primitive that can represent diverse geometric features more efficiently than conventional Gaussian splatting.

    Source: §1 · p. 2, §6 · p. 8

    DRK is less robust to camera noise, which may explain its performance drop on the Tank&Temple dataset.

    Source: S3. More Experiments & Limitation… · p. 14

Abstract

from arXiv · show

Recently, Gaussian splatting has emerged as a robust technique for representing 3D scenes, enabling real-time rasterization and high-fidelity rendering. However, Gaussians' inherent radial symmetry and smoothness constraints limit their ability to represent complex shapes, often requiring thousands of primitives to approximate detailed geometry. We introduce Deformable Radial Kernel (DRK), which extends Gaussian splatting into a more general and flexible framework. Through learnable radial bases with adjustable angles and scales, DRK efficiently models diverse shape primitives while enabling precise control over edge sharpness and boundary curvature. iven DRK's planar nature, we further develop accurate ray-primitive intersection computation for depth sorting and introduce efficient kernel culling strategies for improved rasterization efficiency. Extensive experiments demonstrate that DRK outperforms existing methods in both representation efficiency and rendering quality, achieving state-of-the-art performance while dramatically reducing primitive count.

1 The University of Hong Kong 2 VAST

The paper motivates replacing Gaussian kernels with more flexible planar primitives because Gaussian symmetry and smoothness make diverse shapes and sharp boundaries inefficient to represent. DRK addresses this with learnable shape controls, efficient rendering, and evaluation across varied novel-view-synthesis settings.

1. The University of Hong Kong 2 VAST

  • A single DRK primitive can fit target patterns that require thousands of Gaussians, including detailed textures and shapes, with Figure 1 illustrating a 30-primitive fit.The paper presents this as improved representation efficiency alongside superior results.Source: Figure 1 · p. 1, §1 · p. 2

1. Introduction

  • Gaussian splatting’s rotational symmetry and smooth conic boundaries limit representation of diverse shapes, linear edges, and sharp transitions.These limitations can require numerous Gaussians while still producing artifacts and imperfect fidelity.Source: §1 · pp. 1–2
  • DRK uses multiple radial bases with learnable angles and scales, hybrid L1/L2 distances, and adaptive remapping to model curved surfaces and sharp geometric features.Its planar primitive can deform, rotate, stretch, and shrink while retaining a simple parametric form.Source: §1 · p. 2
  • DRK is evaluated on DiverseScene and MipNeRF 360 scenarios spanning rich textures, fine geometry, specular effects, large spatial scales, and unbounded scenes.The reported experiments emphasize more effective detail preservation and efficiency.Source: §1 · p. 2

2.1. Novel View Synthesis

  • The work situates DRK within novel-view synthesis methods ranging from light fields, layered planes, meshes, point clouds, voxels, and neural radiance fields to Gaussian splatting.Gaussian splatting is highlighted for differentiable rendering, computational efficiency, and high rendering quality, motivating extensions to its kernel.Source: §2.1 · p. 2, §2.2 · pp. 2–3

2.2. Gaussian Splatting

  • Prior Gaussian extensions constrain kernels to planar, temporal, cut, truncated, or convex forms, but DRK instead generalizes the kernel through controllable radial bases and sharpness.DRK also supports efficient differentiable rendering and kernel-based rasterization strategies.Source: §1 · p. 2, §2.2 · p. 3

3. Preliminaries

This section reviews 3D and 2D Gaussian splatting before generalizing planar splatting through kernels that map ray-plane UV coordinates to density. It then identifies Gaussian shape, boundary, and sharpness limitations that motivate deformable radial kernels.

3.1. 3D Gaussian Splatting (3D-GS)

  • 3D Gaussian Splatting represents scenes with colored 3D Gaussians and achieves high-quality representation with real-time rendering through optimized parameters and adaptive density.Each Gaussian is parameterized by center, covariance, opacity, and spherical-harmonic appearance coefficients, then projected and composited with α-blending.Source: §3.1 · p. 3

3.2. 2D Gaussian Splatting (2D-GS)

  • 2D Gaussian Splatting uses tangent-plane coordinates for surface-centric opacity, offering a more compact scene representation aligned with polygon-mesh modeling.Source: §3.2 · p. 3

3.3. General Kernel Splatting

  • General planar kernel splatting intersects rays with tangent planes, computes local UV coordinates, and maps them through a kernel function to density values that determine primitive shape.The framework extends conventional 2D Gaussians beyond a fixed Gaussian kernel.Source: §3.3 · p. 3, Figure 3 · p. 3

3.4. Discussions

  • Gaussian projections are rotationally symmetric elliptical distributions with conic boundaries, limiting representation of diverse shapes, linear edges, and intermediate edge forms.The covariance also couples decay rate with spatial extent, making sharp features difficult to represent across extended regions.Source: §3.4 · p. 3
  • Millions of fine-grained Gaussians may be needed to approximate arbitrary shapes, yet triangles and rectangles can retain interior inconsistencies and external floating points.A single DRK is presented as achieving superior geometric fidelity compared with multiple Gaussians by using radial bases with varying lengths and polar angles controlled for curvature and sharpness.Source: §3.4 · p. 4, Figure 2 · p. 4

4. Deformable Radial Kernel Splatting

DRK generalizes Gaussian splatting with learnable radial bases, hybrid distance metrics, edge sharpening, and rasterization methods tailored to planar kernels.

4. Deformable Radial Kernel Splatting

  • DRK represents planar primitives with radial lengths and angles, while η and τ control boundary curvature and sharpness.Its parameterization also retains position, orientation, opacity, and view-dependent appearance variables from 3D-GS.Source: §4 · p. 4, Figure 4 · p. 4

4.1. Radial Basis

  • Cosine-weighted interpolation between radial bases produces continuous kernel-shape transitions, and traditional 2D Gaussians arise as a special four-basis configuration.The formulation uses polar coordinates and ordered radial control points to define the kernel.Source: §4.1 · p. 4

4.2. L1&L2 Norm Blending 

  • Blending L1 and L2 norms maps adjacent radial endpoints to boundaries that interpolate between straight-line and smoothly curved shapes.The L1 transformation maps the diamond-shaped unit ball to the segment between neighboring endpoints, while η controls the blend.Source: §4.2 · p. 5

4.3. Edge Sharpening

  • The sharpening coefficient τ reshapes alpha values toward 0 or 1, producing sharper edge transitions without changing the kernel’s spatial extent.This decouples edge sharpness from the scale properties coupled by Gaussian variance.Source: §4.3 · p. 5, Figure 5 · p. 5

4.4. Rasterization

  • DRK rasterization combines view-dependent low-pass filtering, polygon-based tile culling, and intersection-depth cache sorting to improve efficiency and view consistency.Radial endpoints define the kernel boundary for culling; cache sorting uses ray-intersection distances and prevents popping artifacts while maintaining rendering efficiency.Source: §4.4 · pp. 5–6

5. Experiment

The experiments evaluate DRK across diverse synthetic and real-world scenes, comparing reconstruction quality, efficiency, qualitative detail, and mesh conversion. DRK achieves strong representation and rendering results while supporting direct conversion of traditional mesh assets.

5.1. Datasets and Evaluation Metrics

  • The evaluation uses DiverseScenes and Mip-NeRF360 with PSNR, SSIM, and LPIPS to test quality across scene complexity, scale, and perceptual fidelity.DiverseScenes contains 10 representative scenes spanning five categories, while density variants progress from DRK to DRK(S2) and DRK(S1).Source: §5.1 · p. 6

5.2. Quantitative Comparisons

  • DRK achieves the best average performance on DiverseScenes against 3D-GS, 2DGS, 3D-HGS, and GES under matched primitive-count comparisons.The benchmark covers texture complexity, geometric detail, view-dependent effects, and scene scale.Source: §5.2 · p. 6, §5.1 · p. 6
  • On Mip-NeRF360, DRK excels in LPIPS and SSIM but has less PSNR advantage on unbounded scenes, especially with sparser kernels.The authors hypothesize that distant regions with limited supervision may be overfit; foreground-only M-PSNR results indicate strength in well-supervised central regions.Source: §5.2 · p. 7, Table 3 · p. 7

5.3. Qualitative Comparison

  • Qualitative comparisons show better texture and geometry fitting, including fine flowers, leaves, tags, and bark, with comparable or fewer kernels than competing representations.Mip-NeRF360 comparisons also report clearer details and high-fidelity rendering.Source: §5.3 · pp. 7–8, Figure 8 · p. 7, Figure 9 · p. 8

5.4. Converting Mesh to DRK

  • Mesh2DRK converts triangle and polygon faces by placing DRK endpoints at polygon vertices and adjusting sharpness parameters, without training-data rendering or optimization.The conversion is presented as a low-cost way to incorporate traditional 3D assets into DRK scenes.Source: §5.4 · p. 8, Figure 10 · p. 8

6. Conclusion

DRK generalizes Gaussian splatting through learnable shape control and efficient rendering strategies. Experiments report high-quality scenes with fewer primitives, while mesh conversion extends the representation to traditional assets without training.

6. Conclusion

  • DRK achieves superior rendering quality with significantly fewer primitives while maintaining computational efficiency through rasterization optimizations.The conclusion also reports successful conversion of triangle and polygon meshes into DRK without training.Source: §6 · p. 8

Outline

The supplementary material expands the paper with applications, dataset information, implementation details, additional public-dataset results, and limitation analysis.

Outline

  • The supplement provides more mesh-to-DRK results, introduces DiverseScenes, adds public-dataset experiments and limitation analysis, and documents implementation details.It is organized into sections S1 through S4 covering these materials.Source: Outline · p. 12

S1. Converting Mesh to DRK

The supplementary mesh-conversion results show rapid DRK construction from traditional assets without training preparation or optimization. Cache-sorting supports conversion quality because Mesh2DRK uses larger kernels than image-trained representations.

S1. Converting Mesh to DRK

  • Mesh assets can be converted to DRK within seconds without training-data preparation or optimization, supporting composition with reconstructed multi-view scenes.The supplement also describes future retrieval of material properties through UV attributes and rendered maps.Source: S1. Converting Mesh to DRK · p. 12
  • Cache-sorting has little effect on reconstruction quality but helps Mesh2DRK because larger mesh-face kernels make sorting order important for satisfactory conversion.Image-trained kernels are smaller units that capture high-frequency appearance details.Source: S1. Converting Mesh to DRK · p. 12

S2. DiverseScenes Dataset

DiverseScenes is a 10-scene benchmark spanning varied geometry, textures, specular effects, and scene scales, with supplementary visualizations and metric tables supporting per-scene and category-level evaluation.

Dataset composition

  • DiverseScenes contains 10 scenes covering simple geometry, detailed textures, fine geometry, and large-scale environments.The dataset includes examples such as McCree, House, Newspaper, Book, PalmTree, Dress, Minecraft, and Street.Source: S2. DiverseScenes Dataset · p. 12

Supplementary materials

View sampling

  • The benchmark uses 200 training views and 30 test views, while Minecraft and Street use 230 training views with walking-camera paths.Source: S2. DiverseScenes Dataset · p. 12

Evaluation organization

S3. More Experiments & Limitation Analysis

Additional experiments show strong performance on NeRF-Synthetic but expose sensitivity to camera quality, especially on dynamic or changing-exposure scenes such as Tank&Temple.

NeRF-Synthetic evaluation

  • DRK achieves superior NeRF-Synthetic performance, while DRK (S3) uses very few kernels and a small model with average PSNR comparable to other methods.DRK (S1) remains compact and outperforms other methods in PSNR despite using slightly more kernels than 3D-GS.Source: S3. More Experiments & Limitation… · p. 13

Ablation study

Tank&Temple evaluation

Camera-noise robustness

Limitation analysis

S4. Method Details

The method details describe DRK’s parameterization, differentiable computation graph, and rendering pipeline, including culling and cache-based depth sorting for planar kernels.

Parametrization

  • DRK constrains opacity, scales, rotation, sharpness, blending, and basis angles through activation functions, using K = 8 to balance flexibility and memory efficiency.The angle parameterization enforces monotonic ordering, minimum angular separation, and a total span of 2π.Source: S4. Method Details · p. 14

Model training

  • DRK-specific parameters are optimized with a learning rate of 5e−3, decayed over 35K steps, under three density-control configurations.Source: S4. Method Details · p. 14

Rendering pipeline

  • Pre-sorting tile-kernel pairs by nearest distance, combined with cache-sorting, achieves the highest sorting accuracy and corrects most sorting disorders.Source: Figure S4 · p. 15

Tensor graph

  • The tensor graph traces dependencies from learnable parameters through intermediate variables to the final opacity output, with gradients back-propagated along those dependencies.Source: S4. Method Details · p. 15, Figure S5 · p. 15

Cache-sorting

  • Cache-sorting maintains a limited depth-ordered cache by inserting new DRKs according to depth and evicting the oldest or head element when necessary.The algorithm returns either a popped DRK index or a status code and requires backward processing from front to back for consistency.Source: S4. Method Details · p. 15

Sorting evaluation

  • Nearest-distance pre-sorting of DRK–tile pairs achieves the highest sorting accuracy and improves over cache-sorting alone and vanilla presorting.Source: S4. Method Details · pp. 15–16

Related papers

Loading 2412.11752v2…