Source-linked AI summary
Recent Advances in 3D Gaussian Splatting
Tong Wu, Yu-Jie Yuan, Ling-Xiao Zhang, Jie Yang, Yan-Pei Cao, Ling-Qi Yan, Lin Gao
TL;DR
Realistic 3D content creation remains difficult because traditional methods can be noisy or time-consuming, while NeRF suffers from slow training and rendering. This survey explains 3DGS and reviews its growing methods across reconstruction, editing, and downstream applications, highlighting fast rendering and explicit geometry alongside remaining robustness and geometry-quality challenges.
Problem
Traditional reconstruction can be less faithful and modeling is time-consuming, while NeRF’s training and rendering remain slow for practical novel view synthesis.
Method
The paper surveys traditional point-based rendering and recent neural-based 3DGS methods, organizing research into reconstruction, editing, and downstream applications.
Results
3DGS achieves approximately 30 FPS on a common device with rendering quality comparable to NeRF, while its explicit geometry supports dynamic reconstruction and downstream applications.
Takeaways & Limitations
3DGS’s efficient rendering and explicit geometry support applications including large-scene reconstruction, SLAM, generation, dynamic reconstruction, and free-viewpoint video.
Takeaways & Limitations
Current 3DGS geometry quality is only comparable to prior SDF-based methods, and reconstruction quality degrades for sparse views, complex shading, and large-scale scenes.
Abstract
from arXiv · showhide
The emergence of 3D Gaussian Splatting (3DGS) has greatly accelerated the rendering speed of novel view synthesis. Unlike neural implicit representations like Neural Radiance Fields (NeRF) that represent a 3D scene with position and viewpoint-conditioned neural networks, 3D Gaussian Splatting utilizes a set of Gaussian ellipsoids to model the scene so that efficient rendering can be accomplished by rasterizing Gaussian ellipsoids into images. Apart from the fast rendering speed, the explicit representation of 3D Gaussian Splatting facilitates editing tasks like dynamic reconstruction, geometry editing, and physical simulation. Considering the rapid change and growing number of works in this field, we present a literature review of recent 3D Gaussian Splatting methods, which can be roughly classified into 3D reconstruction, 3D editing, and other downstream applications by functionality. Traditional point-based rendering methods and the rendering formulation of 3D Gaussian Splatting are also illustrated for a better understanding of this technique. This survey aims to help beginners get into this field quickly and provide experienced researchers with a comprehensive overview, which can stimulate the future development of the 3D Gaussian Splatting representation.
1 Introduction
Realistic 3D content is increasingly needed, but traditional reconstruction can be noisy and traditional modeling is time-consuming. Against NeRF’s quality–speed challenge, this survey organizes emerging 3DGS research across reconstruction, editing, and downstream applications.
- Traditional reconstruction may be less faithful because of imperfect capture and noisy camera estimation, while 3D modeling requires professional training and time-consuming interaction.
- NeRF improved novel view synthesis quality but still faced low training and rendering speed, motivating continued acceleration efforts.
- The survey covers traditional splatting methods and recent neural-based 3DGS methods to help readers become familiar with the field.
- The literature is divided by functionality into 3D reconstruction, scene editing, and downstream applications such as digital humans.
- The review concludes with a higher-level synthesis and future research directions, with representative works also organized chronologically in a timeline.
2 Gaussian Splatting for 3D Reconstruction
This section introduces point-based rendering and 3D Gaussian Splatting, then surveys reconstruction methods addressing quality, efficiency, compression, and challenging inputs. 3DGS directly optimizes Gaussian ellipsoids and rasterizes their projections, enabling approximately 30 FPS rendering with quality comparable to NeRF.
- 2.1 Point-based Rendering: Point-based rendering generates images by rendering discrete geometry primitives, while splats allow each ellipsoid to influence multiple pixels and reduce holes.Traditional methods further improve anti-aliasing, rendering efficiency, and shading continuity.
- 2.1 Point-based Rendering: NeRF combines sampled points’ view-dependent colors along camera rays, typically requiring 128 samples per ray for high-quality rendering.The dense sampling increases training and rendering time.
- 2.1 Point-based Rendering: 3DGS directly optimizes Gaussian ellipsoids with position, rotation, scale, opacity, and spherical-harmonic color attributes instead of predicting values with neural networks.Projected Gaussian ellipsoids determine pixel colors from a given viewpoint; 3DGS directly models opacity and uses rasterization-based rendering.
- 2.2 Quality Enhancement: Quality-enhancement methods address sampling artifacts, view-dependent effects, Gaussian popping, and unstable point distributions through filtering, alternative appearance models, sorting, and densification strategies.Mip-Splatting constrains representation frequency; StopThePop replaces per-ray sorting with tile-based sorting; Pixel-GS uses pixel coverage and camera distance during splitting.
- 2.2 Quality Enhancement: 3DGS reconstruction methods remain comparable to NeRF-based methods while offering faster rendering, and dynamic methods benefit from explicit geometry for modeling motion.The survey also reports applications to medical motion and endoscopic reconstruction.
- 2.3 Compression and Regularization: Compression methods reduce computational or storage requirements by quantizing Gaussian attributes, encoding grids, pruning or sampling Gaussians, and applying image or entropy compression.EAGLES quantizes color, position, opacity, rotation, and scaling; SASCGS uses DEFLATE, while SOGS uses JPEG XL and OpenEXR compression.
- 2.5 3D Reconstruction from Challenging Inputs: Beyond dense regular-view settings, reconstruction research targets sparse views, missing camera parameters, and large urban scenes using depth supervision, Gaussian refinement, semantic inputs, and dynamic scene models.FSGS initializes sparse Gaussians from structure-from-motion and adds supervision from a pretrained 2D depth network.
3 Gaussian Splatting for 3D Editing
3DGS editing methods span geometry, appearance, and physical simulation, using explicit Gaussian representations for operations such as inpainting, object removal, relighting, and dynamics. Despite these capabilities, current geometry-editing methods remain constrained in handling large-scale or detailed deformations.
- Editing taxonomy: 3DGS editing is organized into geometry editing, appearance editing, and physical simulation.The survey describes editing across these three categories.
- Geometry editing: GaussianEditor uses text prompts and semantic tracing to enable 3D inpainting, object removal, and object composition.
- Geometry editing: Geometry-editing methods support simple rigid transformations and non-rigid deformation but struggle with large-scale deformation, intense movements, detailed surface changes, and topology changes.SuGaR, SC-GS, and GaMeS each have limitations involving deformation scale, movement intensity, surface detail, or mesh topology.
- Appearance editing: Appearance editing modifies Gaussian attributes using language-guided image edits, segmentation masks, and hierarchical Gaussian splatting.Two independently named GaussianEditor methods follow related image-editing strategies, with one adding hierarchical Gaussian splatting for 3D editing.
- Appearance editing: GS-IR and RelightableGaussian disentangle texture and lighting by assigning material parameters to Gaussians and modeling illumination with a learnable environment map.
- Physical simulation: Physical-simulation methods combine 3DGS with continuum deformation, position-based dynamics, spring-mass models, or semantic priors for dynamic and object-level simulation.These approaches support cohesive rendering and dynamics, interactive VR manipulation, real-world physical-property learning, or object-level simulation.
4 Applications of Gaussian Splatting
The survey organizes Gaussian Splatting applications across scene understanding, reconstruction and SLAM, digital humans, and 3D/4D generation. These applications use explicit Gaussian representations for segmentation, geometry processing, simulation, human modeling, and content creation.
- 4.1 Segmentation and Understanding: 3DGS scene-understanding methods combine 2D segmentation or language features with Gaussians to support real-time, editable 3D representations.Methods use SAM-derived masks or CLIP/DINO features, with Gaussian attributes enabling semantic rendering and cross-view consistency.
- 4.2 Geometry Reconstruction and SLAM: Geometry reconstruction methods regularize Gaussians with depth, surfaces, signed-distance fields, normals, or 2D Gaussian projections.SuGaR aligns camera-to-Gaussian distances with rendered depth, while other methods transfer NeuS properties, use depth and normal priors, or construct meshes through TSDF fusion.
- 4.2 Geometry Reconstruction and SLAM: SLAM methods jointly localize cameras and reconstruct scenes, using adaptive Gaussian management, scale regularization, LiDAR initialization, and semantic extensions.3DGS enables flexible reprojection and real-time rendering, which the survey associates with better reconstruction and greater applicability than NeRF-based SLAM.
5 Conclusions and Discussions
The survey positions 3DGS as an efficient explicit representation supporting reconstruction, editing, and downstream applications, while identifying limitations in geometry quality, robustness, editing independence, 4D realism, and platform availability.
- 5 Conclusions and Discussions: 3DGS uses explicit geometry and rasterization-based rendering to support real-time visualization on consumer-level devices.Its anisotropic Gaussian scales compensate for the absence of edges and faces between primitives.
- 5 Conclusions and Discussions: 3DGS supports efficiency-sensitive applications such as large-scene reconstruction, SLAM, and generation, but its discrete geometry remains only comparable to prior SDF-based methods.The survey suggests combining 3DGS with other representations to improve geometry and surface quality.
- 5 Conclusions and Discussions: Novel-view reconstruction quality degrades for sparse views, complex shading effects, and large-scale scenes, leaving robustness and generalizability unresolved.The survey identifies generalizable pipelines as a way to reduce training cost.
- 5 Conclusions and Discussions: Current 3DGS editing methods do not accurately decompose geometry, texture, and lighting or may require Gaussian reoptimization, limiting independent and efficient editing.The survey points toward advanced rendering techniques and connections with mesh-based representations.
- 5 Conclusions and Discussions: Current 4D generation methods still miss realistic geometry, appearance, and physics-aware motion, while Python and CUDA-supported PyTorch implementations may restrict platform reach.The survey proposes incorporating video-generative-model priors, physical laws, and broader framework support.
Author biography
The authors are researchers and practitioners working across computer graphics, computer vision, neural rendering, geometric processing, and related academic and industry roles.
- Author biography: Tong Wu is a PhD candidate whose research interests include computer graphics and computer vision.He conducted the literature review and drafted the manuscript.
- Author biography: Yu-Jie Yuan is a PhD candidate whose research interests include computer graphics and neural rendering.
- Author biography: Ling-Xiao Zhang is an engineer whose research interests include computer graphics and geometric processing.
- Author biography: Jie Yang is an Assistant Professor whose research interests include computer graphics and geometric processing.
- Author biography: Yan-Pei Cao leads research and works on computer graphics and 3D computer vision.
- Author biography: Ling-Qi Yan is an Assistant Professor of Computer Science at UC Santa Barbara and codirector of the MIRAGE Lab.
- Author biography: Lin Gao is a Professor whose research interests include computer graphics and geometric processing.