Source-linked AI summary
Triangle Splatting for Real-Time Radiance Field Rendering
Jan Held, Renaud Vandeghen, Adrien Deliege, Abdullah Hamdi, Silvio Giancola, Anthony Cioppa, Andrea Vedaldi, Bernard Ghanem, Andrea Tagliasacchi, Marc Van Droogenbroeck
TL;DR
Differentiable 3D representations must support flexible geometry while retaining efficient rendering, but triangles are difficult to optimize directly and volumetric primitives can blur surface structure. Triangle Splatting directly optimizes triangle soups through differentiable screen-space splatting, achieving high visual fidelity and fast rendering while remaining compatible with standard mesh renderers. The paper positions this as a step toward mesh-aware neural rendering, with connected meshing and some outdoor-scene behavior remaining limitations.
Problem
Triangles are difficult to optimize directly without a predefined topology, while volumetric primitives complicate surface definition and sharp-detail modeling.
Method
Triangle Splatting projects unstructured 3D triangles into screen space and applies differentiable window functions for end-to-end gradient-based optimization.
Results
Triangle Splatting outperforms concurrent methods in visual fidelity, training speed, and rendering throughput, while achieving superior perceptual quality to Zip-NeRF on indoor scenes.
Takeaways & Limitations
Optimized triangle soups bridge modern differentiable rendering with classical mesh-based pipelines and support real-time rendering in standard mesh renderers.
Takeaways & Limitations
Generating a connected mesh still requires additional steps, and the method occasionally suffers from floaters in large-scale outdoor scenes.
Abstract
from arXiv · showhide
The field of computer graphics was revolutionized by models such as Neural Radiance Fields and 3D Gaussian Splatting, displacing triangles as the dominant representation for photogrammetry. In this paper, we argue for a triangle comeback. We develop a differentiable renderer that directly optimizes triangles via end-to-end gradients. We achieve this by rendering each triangle as differentiable splats, combining the efficiency of triangles with the adaptive density of representations based on independent primitives. Compared to popular 2D and 3D Gaussian Splatting methods, our approach achieves higher visual fidelity, faster convergence, and increased rendering throughput. On the Mip-NeRF360 dataset, our method outperforms concurrent non-volumetric primitives in visual fidelity and achieves higher perceptual quality than the state-of-the-art Zip-NeRF on indoor scenes. Triangles are simple, compatible with standard graphics stacks and GPU hardware, and highly efficient: for the \textit{Garden} scene, we achieve over 2,400 FPS at 1280x720 resolution using an off-the-shelf mesh renderer. These results highlight the efficiency and effectiveness of triangle-based representations for high-quality novel view synthesis. Triangles bring us closer to mesh-based optimization by combining classical computer graphics with modern differentiable rendering frameworks. The project page is https://trianglesplatting.github.io/
1 Introduction
The paper asks whether unstructured triangles can be optimized directly for differentiable novel-view synthesis, avoiding the limitations of fixed mesh templates and smooth volumetric primitives. Triangle Splatting addresses this by combining triangle efficiency with differentiable splatting and standard renderer compatibility.
- Motivation: Fixed mesh templates require known topology, limiting differentiable triangle optimization when scene structure is unknown.These methods struggle to capture fine geometric details and adapt to novel structures.
- Motivation: Gaussian representations adapt without templates, but unbounded support complicates surfaces and smoothness hinders sharp details.The introduction contrasts these limitations with the geometric structure of triangles.
- Approach: Triangle Splatting directly optimizes unstructured triangle soups by splatting triangles into screen space with end-to-end gradients.The method is presented as a real-time differentiable renderer for novel-view synthesis and 3D reconstruction.
- Implications: The optimized triangle soup remains directly compatible with standard mesh-based renderers and traditional graphics pipelines.This compatibility connects differentiable rendering with established triangle-processing infrastructure.
- Approach: A differentiable window function creates soft triangle boundaries, enabling effective gradient flow during optimization.This design is listed as a core contribution of the renderer.
2 Related work
Related work develops differentiable representations for neural rendering and explicit scene primitives. Neural fields and accelerated variants address reconstruction and rendering challenges, while primitive-based methods enable gradient optimization of explicit geometry and real-time Gaussian rendering.
- Neural radiance fields: Neural radiance fields became a standard for image-based 3D reconstruction, with later work improving training, rendering, anti-aliasing, unbounded scenes, and few-shot generalization.Approaches include multiresolution grids, hybrid representations, hash tables, and baking procedures.
- Primitive-based differentiable rendering: Differentiable renderers back-propagate image loss to optimize explicit primitives including points, voxels, meshes, and Gaussians.This establishes primitive-based rendering as a route to end-to-end scene optimization.
- Primitive-based differentiable rendering: 3D Gaussian Splatting fits millions of anisotropic Gaussians in minutes and renders them in real time.Follow-up work extends the approach with anti-aliasing, exact volumetric integration, and dynamic modeling.
3 Method
Triangle Splatting represents scenes with directly optimized 3D triangle primitives and renders them through differentiable screen-space splatting. Its window function, pruning, densification, and optimization procedures support compact, depth-consistent, end-to-end rendering.
- Triangle representation and rendering: 3D triangles are parameterized by vertices, color, smoothness, and opacity, then projected into 2D and composited in depth order.The renderer uses a standard pinhole camera model and treats the window-function value as opacity.
- Differentiable rasterization: Prior sigmoid-based windows can exceed the triangle footprint as σ increases and become globally supported, making them unsuitable for rasterization workloads.In the limit, every triangle contributes to every pixel through a constant window value of 0.5.
- Differentiable rasterization: The proposed window function reaches one at the triangle incenter and is zero on the boundary and outside, tightly matching the triangle footprint.Its smoothness is controlled by σ, while the signed distance field defines the triangle interior and boundary.
- Depth-dependent scaling: Triangle Splatting avoids manual depth-dependent smoothness scaling because its window-function normalization produces consistent 2D windows across depths.This differs from Convex Splatting, whose pixel-unit smoothness parameter must be scaled manually.
- Pruning and densification: Pruning removes triangles with low blending weight or insufficient multi-view coverage, while densification adds shapes through an MCMC-guided process based on opacity and sharpness.Small triangles are split into four, whereas triangles below a size threshold are cloned with in-plane noise.
- Optimization: Training initializes one approximately equilateral triangle per SfM point and optimizes vertices, sharpness, opacity, and spherical-harmonic colors using a composite rendering loss.The loss combines photometric, opacity, distortion, normal, and size-regularization terms; thresholds and hyperparameters are specified separately.
4 Experiments
Triangle Splatting achieves strong visual quality and efficiency across standard novel-view-synthesis benchmarks, especially on perceptual metrics and indoor scenes. It also renders substantially faster than competing primitive-based methods, although PSNR can favor blurrier Gaussian reconstructions.
- Triangle Splatting improves LPIPS over 2DGS and BBSplat by 25% and 19%, respectively, on Mip-NeRF360.
- Triangle Splatting improves LPIPS over 3DGS and 3DCS by 10% and 7%, respectively, on Mip-NeRF360.
- Triangle Splatting matches Zip-NeRF within 0.002 LPIPS while rendering over 500× faster.
- Triangle Splatting achieves state-of-the-art performance on indoor scenes and consistently outperforms other non-volumetric primitives across the evaluated datasets.
- Speed & Memory: Triangle Splatting renders 4× faster than BBSplat, 40% faster than 2DGS, and 4× faster than 3DCS while using more primitives than BBSplat.
- PSNR favors smoother Gaussian reconstructions: 2DGS scores 21.27 versus Triangle Splatting’s 18.41 in the highlighted region despite appearing blurrier.
- Qualitative comparisons show sharper Triangle Splatting reconstructions with finer details and less blur than 2DGS, plus higher visual quality than 3DCS.
5 Ablations
Ablations identify opacity and shape-size regularization, bounded window functions, and direct triangle splatting as important for stable optimization, scene coverage, and quality. The optimized representation also remains compatible with conventional mesh renderers and reaches high frame rates.
- Loss terms: Opacity regularization is the most impactful loss component because it makes empty-region triangles transparent and enables their reallocation.
- Loss terms: Shape-size regularization increases PSNR and helps triangles expand into sparse or underrepresented scene regions, particularly indoors.
- Window functions: The proposed bounded window function recovers sparse background structure more effectively than sigmoid windows, which can grow arbitrarily large.
- Triangle vs. convex splatting: Triangle Splatting avoids the line artifacts of 3DCS with degenerate triangles and improves Mip-NeRF360 by 0.05 LPIPS, 0.61 PSNR, and 0.045 SSIM.
- Rendering speed with traditional mesh-based renderer: The optimized triangle soup integrates with mesh-based renderers and reaches 500 FPS at HD resolution on a consumer laptop and 2,400 FPS on an RTX 4090.
6 Conclusions
Triangle Splatting directly optimizes unstructured triangle primitives for novel-view synthesis, connecting neural rendering with traditional graphics pipelines. The method reports high visual fidelity and faster rendering, while identifying meshing and outdoor-scene floaters as remaining limitations.
- Triangle Splatting directly optimizes unstructured triangle primitives for novel-view synthesis.
- The method bridges neural rendering and traditional graphics pipelines by using the primitive of classical mesh representations.
- Triangle Splatting offers high visual fidelity with faster rendering performance than volumetric and implicit alternatives.
- Connected mesh generation still requires additional steps, motivating more direct and principled meshing strategies.
- Outdoor scenes can occasionally produce floaters because non-volumetric shapes receive fewer viewpoints and may overfit training views.
A.1 Methodology
The methodology makes triangle splatting depth-consistent and computationally efficient through normalized influence, precise tile assignment, and adaptive densification. It also notes that center-based depth sorting can cause view-rotation artifacts.
- Depth-dependant scaling: Normalized influence makes a single exponent σ sufficient across depths under uniform in-plane projection scaling.
- Tile assignment in Triangle Splatting: Tile assignment uses screen-space triangle intersections, with triangle bounds determined by the projection of their three vertices.
- Tile assignment in Triangle Splatting: The renderer tightens edge boundaries using a distance threshold, producing a more accurate bounding box and reducing unnecessary rasterization.
- Depth sorting: Triangles are currently sorted by center during rasterization, which can cause popping and blending artifacts during view rotation.
- Densification: Densification prioritizes low-σ triangles and adjusts softness according to local triangle density, while sparse regions receive more solid coverage.
A.2 Initialization & Hyperparameters.
Initialization assigns fixed opacity, sigma, and convex-hull scaling values selected empirically for stable rendering. Densification then increases the number of shapes on a fixed iteration schedule.
- Initialization: Initialization uses opacity 0.28, sigma 1.16, and convex-hull scaling constant k = 2.2.These values were selected empirically for stable initialization and consistent rendering across scenes.
- Densification: Densification occurs every 500 iterations from iteration 500 through 25,000, increasing the number of shapes by 30% each time.
- All other hyperparameters are specified in Table 5.
A.3 More novel-view synthesis results
Tables 7–10 provide detailed results on the Mip-NeRF360 and Tanks and Temples datasets, while Figure 10 adds qualitative novel-view synthesis results.
- Table 5 is identified as the hyperparameter table.
- Tables 7 to 10 present detailed analyses on the Mip-NeRF360 and Tanks and Temples datasets.
- Figure 10 presents additional qualitative results for novel-view synthesis.
A.4 Geometry analysis of Triangle Splatting
Triangle Splatting produces renderable triangle soups that support direct mesh-renderer use, while geometry can be analyzed through depth-based meshing and surface-normal visualization. The method also demonstrates high rendering throughput in game engines.
- Mesh extraction: Triangle soups can be converted directly into standard mesh-renderer formats, but connected-mesh construction still requires post-processing.The paper applies TSDF fusion to predicted depth maps for mesh extraction.
- Visual quality: Triangle Splatting captures finer details and less blurry renderings than 2DGS, while achieving higher visual quality than 3DCS.The comparison is reported qualitatively for real-world scenes.
- Surface geometry: Normal maps show smooth surfaces with triangle orientations consistently aligned to local geometry.This visualization provides a qualitative check of geometric coherence.
- Optimization and pruning: During the final 5,000 iterations, low-opacity triangles are pruned and an opacity loss encourages mostly solid, opaque triangles.The loss also encourages lower σ before conversion to mesh-renderer formats.