Source-linked AI summary
3D Convex Splatting: Radiance Field Rendering with 3D Smooth Convexes
Jan Held, Renaud Vandeghen, Abdullah Hamdi, Adrien Deliege, Anthony Cioppa, Silvio Giancola, Andrea Vedaldi, Bernard Ghanem, Marc Van Droogenbroeck
TL;DR
3D Gaussian primitives make it difficult to represent physically bounded flat surfaces and hard edges efficiently. 3D Convex Splatting replaces them with differentiable 3D smooth convexes and fast splatting-based rendering. Across reported benchmarks, it delivers higher-quality reconstruction with fewer primitives while retaining real-time rendering capability.
Problem
3D Gaussian primitives lack defined boundaries and struggle with flat surfaces, hard edges, and geometrically meaningful scene representations.
Method
3DCS models radiance fields with 3D smooth convex primitives optimized through a differentiable GPU-based rendering pipeline.
Results
3DCS achieves higher visual quality than 3DGS while using fewer primitives, including 70% of 3DGS memory and retaining real-time rendering capability.
Takeaways & Limitations
3DCS provides dense, geometrically meaningful radiance-field representations with high-quality, real-time, and flexible reconstruction.
Abstract
from arXiv · showhide
Recent advances in radiance field reconstruction, such as 3D Gaussian Splatting (3DGS), have achieved high-quality novel view synthesis and fast rendering by representing scenes with compositions of Gaussian primitives. However, 3D Gaussians present several limitations for scene reconstruction. Accurately capturing hard edges is challenging without significantly increasing the number of Gaussians, creating a large memory footprint. Moreover, they struggle to represent flat surfaces, as they are diffused in space. Without hand-crafted regularizers, they tend to disperse irregularly around the actual surface. To circumvent these issues, we introduce a novel method, named 3D Convex Splatting (3DCS), which leverages 3D smooth convexes as primitives for modeling geometrically-meaningful radiance fields from multi-view images. Smooth convex shapes offer greater flexibility than Gaussians, allowing for a better representation of 3D scenes with hard edges and dense volumes using fewer primitives. Powered by our efficient CUDA-based rasterizer, 3DCS achieves superior performance over 3DGS on benchmarks such as Mip-NeRF360, Tanks and Temples, and Deep Blending. Specifically, our method attains an improvement of up to 0.81 in PSNR and 0.026 in LPIPS compared to 3DGS while maintaining high rendering speeds and reducing the number of required primitives. Our results highlight the potential of 3D Convex Splatting to become the new standard for high-quality scene reconstruction and novel view synthesis. Project page: convexsplatting.github.io.
1. Introduction
Radiance-field methods seek high-quality novel views efficiently, but Gaussian primitives struggle with physical boundaries, flat surfaces, and hard edges. 3D Convex Splatting addresses these limitations with differentiable 3D smooth convex primitives and fast splatting-based rendering.
- 3D Gaussian primitives lack defined physical boundaries, making flat surfaces and physically meaningful scene decompositions difficult to represent.
- Gaussian ellipsoid-like shapes struggle with angular boundaries and flat surfaces, requiring inefficient coverage that increases memory and computation.Their gaps near flat or sharp regions reflect the sphere-packing limitation described in the introduction.
- 3DCS uses flexible 3D smooth convexes to model dense volumes, hard edges, and detailed surfaces with fewer primitives than Gaussians.Smoothness and sharpness parameters control whether the shapes are soft or hard, and diffuse or dense.
- 3DCS is reported to outperform existing rendering primitives on MipNeRF360, Tanks and Temples, and Deep Blending while using fewer primitives per scene.
- 3DCS combines Gaussian-like rendering speed with smooth convex flexibility through a differentiable, splatting-based rasterization pipeline.The pipeline projects point-set convexes, forms a 2D convex hull, and optimizes primitive parameters from rendered images.
2. Methodology
3DCS represents radiance fields with differentiable 3D smooth convexes, projecting point-defined convex hulls into 2D for efficient splatting. Its parameters control geometric smoothness and boundary sharpness, while adaptive initialization, densification, and pruning optimize compact scene representations.
- 2.2. 3DCS: Splatting 3D Smooth Convexes: 3DCS combines smooth convexes with point-based representation, convex-hull projection, smoothing, splatting, and adaptive densification for real-time novel-view synthesis.The pipeline uses convex shapes from CvxNet and primitive-based rendering from 3DGS, with operations optimized from posed images.
- 2.1. Preliminaries on 3D Smooth Convexes: Larger δ produces harder convex edges, while higher σ makes boundary transitions steeper and more defined; lower values soften or diffuse the shape.Figure 4 summarizes δ as controlling vertex and edge smoothness and σ as controlling radiance-field transition sharpness.
- 2.2. 3DCS: Splatting 3D Smooth Convexes: A convex is represented as the hull of freely movable 3D points, enabling flexible positioning and shape morphing during optimization.The points need not be explicit polyhedron vertices; they define the convex hull of the shape.
- 2.2. 3DCS: Splatting 3D Smooth Convexes: Projected points form a 2D convex hull through differentiable camera projection and Graham Scan, preserving an efficient differentiable outline for rasterization.The rasterizer extends the smooth convex indicator to 2D, scales δ and σ by camera distance, and follows a tile-based 3DGS design.
- 2.3. Optimization: Initialization optimizes points, δ, σ, opacity, and spherical-harmonic colors, scaling convex sizes from local point-cloud density before adaptive refinement.Shapes are initialized around point-cloud points using Fibonacci-sphere samples; densification adds convexes, while transparent or oversized convexes are pruned.
- 2.3. Optimization: Smooth convexes approximate polyhedra and Gaussian-like shapes with fewer primitives while producing sharper geometric boundaries.The simple-shape comparison evaluates one or eight Gaussian primitives against a single convex defined by three or six points.
3. Experiments
Experiments show that 3DCS represents diverse shapes with fewer primitives and achieves strong quality, efficiency, and visual fidelity across real-world novel-view-synthesis benchmarks.
- 3.1. Experiments on Synthetic Data: A single smooth convex approximates polyhedra and Gaussians more efficiently than one or eight Gaussian primitives.This synthetic comparison demonstrates the broader shape-representation flexibility of convex primitives.
- 3.3. Real-world Novel View Synthesis: 3DCS consistently matches or surpasses existing methods across evaluated datasets, while balancing memory usage, training time, and rendering speed.It outperforms 3DGS, GES, and 2DGS on most metrics for Tanks and Temples and Deep Blending, and remains competitive on Mip-NeRF360.
- 3.3. Real-world Novel View Synthesis: 0.9 PSNR, 0.007 SSIM, and 0.023 LPIPS improvements over 3DGS occur on indoor Mip-NeRF360 scenes with structured surfaces and hard edges.The advantage is less pronounced outdoors dominated by natural, unstructured elements, where 3DGS can achieve better PSNR and SSIM.
- 3.4. Ablation Study and Discussion: Using at least four points per convex consistently outperforms 3DGS, whereas increasing beyond six points provides no significant performance gain.The ablation varies the point count per convex and reports the trade-off between representational flexibility and training time.
- 3.4. Ablation Study and Discussion: 3DCS decomposes objects into physically meaningful convex parts, preserving image clarity and producing compact, geometrically interpretable representations.The qualitative comparison reports sharper details than Gaussian splatting, while the lightweight variant retains high visual quality with reduced memory.
4. Conclusion
The conclusion presents 3DCS as a smooth-convex primitive for high-quality novel-view synthesis that improves density, compactness, and reconstruction flexibility.
- 4. Conclusion: 3DCS uses smooth convex primitives to deliver high-quality novel view synthesis with denser representations and fewer primitives and parameters.The method combines convex-shape adaptability with primitive-based rendering for high-quality, real-time, flexible radiance-field reconstruction.
- 4. Conclusion: 3DCS shows particularly substantial improvements on Tanks and Temples and indoor Mip-NeRF360 scenes.The conclusion identifies these settings as especially favorable for the proposed representation.
5. Initialization & Hyperparameters
3DCS initializes convex shapes from point-cloud locations using adaptive spherical neighborhoods, producing smaller shapes in dense regions and larger ones in sparse regions.
- 5. Initialization & Hyperparameters: Each convex is initialized from uniformly distributed points around a point-cloud center using the Fibonacci sphere algorithm.The initial radius is 1.2 times the mean distance to the three nearest neighbors.
- 5. Initialization & Hyperparameters: Adaptive initialization assigns many small convexes to dense regions and larger convexes to sparse regions.Indoor scenes require denser shapes because many objects are human-made.
6. Methodology Details
The 2D formulation transfers the smooth-convex representation from 3D to projected image-space shapes by replacing points and delimiting planes with their 2D counterparts.
- 6. Methodology Details: The 2D convex indicator function reuses the 3D equations while substituting a 2D point q for the 3D point p.The projected convex hull is delimited by lines rather than the planes used in 3D.
- 6. Methodology Details: T denotes the total number of lines delimiting the 2D convex shape.This quantity indexes the boundaries used by the 2D formulation.
- 6. Methodology Details: The smooth-convex indicator I(p) is the quantity defined by the method for determining the convex representation at a 3D point.The cited method text introduces I(p) immediately before its defining expression.
7. Ablation Study
The ablation examines perspective-aware scaling in the 2D projection, evaluating how scaling δ and σ by camera distance affects rendering quality. Table 4 reports PSNR under varying scaling magnitudes.
- Perspective-Aware Scaling in 2D Projection: Scaling δ and σ by camera distance keeps the projected convex appearance consistent across depths.The method incorporates perspective effects by scaling both quantities with distance d.
- Perspective-Aware Scaling in 2D Projection: The ablation tests both the necessity of scaling δ and σ and the effect of the chosen scaling magnitude.
- Perspective-Aware Scaling in 2D Projection: Table 4 evaluates PSNR under different scaling magnitudes for perspective-aware projection.
8. More Results
Additional results illustrate that smooth convexes can approximate diverse geometries and that 3D Convex Splatting performs across multiple datasets and evaluation metrics. Qualitative figures and metric tables cover Mip-NeRF360, Tanks & Temples, and Deep Blending.
- More Results: Results are evaluated with LPIPS, PSNR, and SSIM on Mip-NeRF360, Tanks & Temples, and Deep Blending.The supplied tables cover LPIPS, PSNR, and SSIM for these benchmark datasets.
- Experiments on Synthetic Data: Smooth convexes approximate a wide variety of shapes, including both hard or soft and dense or diffuse geometries.The convex representation can approximate polyhedra and Gaussians while using fewer primitives for accurate representation.
- More Results: Additional qualitative results highlight the capabilities of 3D Convex Splatting.