Source-linked AI summary
DashGaussian: Optimizing 3D Gaussian Splatting in 200 Seconds
Youyu Chen, Junjun Jiang, Kui Jiang, Xiao Tang, Zhihao Li, Xianming Liu, Yinyu Nie
TL;DR
3DGS optimization is costly because rendering resolution and primitive count drive its computational complexity. DashGaussian schedules resolution and primitive growth by progressively fitting higher-frequency components, achieving 45.7% average acceleration across backbones while preserving rendering quality.
Problem
3DGS still requires further acceleration because its optimization cost depends heavily on rendering resolution and Gaussian primitive count.
Method
DashGaussian progressively fits higher-frequency components while coordinating dynamic rendering resolution with synchronized Gaussian primitive growth.
Results
45.7% average acceleration is achieved across 3DGS backbones while preserving rendering quality.
Takeaways & Limitations
DashGaussian is presented as a plug-and-play optimization strategy applicable across various 3DGS backbones.
Abstract
from arXiv · showhide
3D Gaussian Splatting (3DGS) renders pixels by rasterizing Gaussian primitives, where the rendering resolution and the primitive number, concluded as the optimization complexity, dominate the time cost in primitive optimization. In this paper, we propose DashGaussian, a scheduling scheme over the optimization complexity of 3DGS that strips redundant complexity to accelerate 3DGS optimization. Specifically, we formulate 3DGS optimization as progressively fitting 3DGS to higher levels of frequency components in the training views, and propose a dynamic rendering resolution scheme that largely reduces the optimization complexity based on this formulation. Besides, we argue that a specific rendering resolution should cooperate with a proper primitive number for a better balance between computing redundancy and fitting quality, where we schedule the growth of the primitives to synchronize with the rendering resolution. Extensive experiments show that our method accelerates the optimization of various 3DGS backbones by 45.7% on average while preserving the rendering quality.
1. Introduction
DashGaussian targets 3DGS optimization costs that grow with rendering resolution and Gaussian primitive count. It schedules both resources to reduce redundant computation while preserving rendering quality.
- 3DGS optimization remains time-intensive despite reducing scene optimization from days with NeRF to tens of minutes.
- Rendering resolution and Gaussian primitive count jointly determine the optimization complexity of forward rendering, back-propagation, and primitive updates.
- High-resolution rendering is wasteful early, while late optimization spends substantial time on many primitives for limited quality improvement.
- The resolution scheduler progressively fits higher-frequency components, avoiding the aliasing associated with simply reducing training resolution.
- The primitive scheduler grows Gaussian primitives alongside rendering resolution to balance computational redundancy and fitting quality.
- 45.7% average acceleration is achieved across 3DGS backbones without trading off rendering quality.
2. Related Work
Prior 3DGS acceleration methods refine computation or prune primitives, while scheduled-resolution approaches use fixed levels. DashGaussian instead adaptively coordinates rendering resolution and primitive growth.
- Novel View Synthesis: NeRF-based novel-view synthesis achieves high visual quality but can require tens of hours to optimize a scene.
- Acceleration for 3DGS Optimization: Engineering approaches accelerate 3DGS through more efficient forward, backward, and parameter-update computation.
- Acceleration for 3DGS Optimization: Pruning Gaussian primitives reduces parameters but can noticeably decrease rendering quality when many parameters are excluded.
- Scheduled 3DGS Optimization: Existing scheduled 3DGS methods use fixed resolution levels, whereas DashGaussian adaptively schedules rendering resolution and primitive growth together.
- Scheduled 3DGS Optimization: DashGaussian avoids heuristically predetermining the final primitive number through adaptive primitive growth.
3. Preliminary
3DGS represents scenes with differentiable Gaussian primitives projected onto image pixels. Rendering and densification determine optimization computation and model growth.
- 3DGS represents a scene with Gaussian primitives characterized by position, covariance, opacity, and color coefficients.
- Gaussian primitives are splatted onto camera-view pixels to render images for novel-view synthesis.
- Differentiability enables optimizing Gaussian primitives by back-propagating rendering-loss gradients against training images.
- Rendering resolution controls the number of processed pixels and therefore directly affects optimization time.
- 3DGS densification identifies under-reconstructed regions and clones or splits primitives during optimization.
4. DashGaussian
DashGaussian accelerates 3DGS by scheduling rendering resolution and Gaussian primitive growth throughout optimization. It progressively fits higher-frequency components while coordinating resolution and primitive budgets to reduce complexity without sacrificing rendering quality.
- Problem Formulation: 3DGS optimization with low-resolution supervision must address 3D aliasing to preserve fidelity when rendering at the original high resolution.Simple resolution reduction can introduce aliasing and compromise high-resolution rendering quality.
- Problem Formulation: DashGaussian formulates increasing-resolution 3DGS optimization as progressively fitting higher levels of frequency components.The formulation focuses on residual high-frequency components absent from downsampled training images.
- Frequency Guided Resolution Scheduler: Frequency-guided scheduling distributes optimization steps across resolutions according to the information contained in each resolution’s frequency domain.The schedule determines when to increase rendering resolution and can use multiple intermediate resolutions.
- Resolution Guided Primitive Scheduler: The primitive scheduler grows Gaussian primitives with rendering resolution, using a concave-up schedule that reduces early over-densification and optimization complexity.The schedule suppresses primitive growth early and encourages it later, producing a small area under the primitive-growth curve.
- Resolution Guided Primitive Scheduler: Momentum-based primitive budgeting determines the final primitive count during optimization rather than fixing it from a dataset prior.Experiments show the scheme works across datasets without requiring an upper bound for primitive growth.
5. Experiments
Experiments evaluate DashGaussian across datasets, fast-optimization baselines, multiple 3DGS backbones, scheduler visualizations, profiling, and ablations. The method consistently reduces optimization time while preserving or improving rendering quality and often using fewer Gaussian primitives.
- Experimental Setup: Experiments cover Mip-NeRF 360, Deep-Blending, and Tanks&Temples, with MatrixCity evaluated for large-scale reconstruction in supplementary material.Novel-view quality is measured with PSNR, SSIM, and LPIPS; optimization speed uses total time, and primitive efficiency uses average primitive count.
- Comparison with Fast Optimization Methods: Within 200 seconds, DashGaussian combined with Taming-3DGS improves optimization speed over existing fast methods while retaining comparable or better rendering quality.The compared methods include Reduced-3DGS, Mini-splatting, and Taming-3DGS; Taming-3DGS uses efficient back-propagation and Sparse Adam.
- Comparison with Fast Optimization Methods: 45.7% average acceleration is achieved when DashGaussian is plugged into various 3DGS backbones, with negligible quality compromise or improvement.The evaluation spans backbones differing in optimizers, primitive representations, and densification scores.
- Visualization of Schedulers: DashGaussian scene-adaptively schedules rendering resolution and represents the bicycle and stump scenes with fewer primitives than the unscheduled 3DGS version.Figure 4 visualizes resolution scheduling on the two Mip-NeRF 360 scenes and compares primitive growth between 3DGS and 3DGS+DashGaussian.
- Profiling of the Optimization Time: DashGaussian reduces the per-iteration time of forward rendering, gradient back-propagation, and primitive optimization across 3DGS and Taming-3DGS.The similar acceleration across these operations indicates reduced optimization complexity across different backbone implementations.
- Ablation Study: 41.3% optimization-time reduction is obtained for Taming-3DGS with the full DashGaussian, while rendering quality surpasses the backbone using fewer Gaussian primitives.Ablations separately evaluate the resolution scheduler and primitive scheduler, and the full method combines both.
6. Conclusion
DashGaussian accelerates 3DGS optimization by coordinating rendering resolution and primitive growth while preserving rendering quality. It is designed to integrate across 3DGS backbones and support larger reconstruction tasks.
- DashGaussian formulates 3DGS optimization as frequency-component fitting while jointly scheduling rendering resolution and primitive number.The method allocates computational resources throughout optimization and is presented as a potential optimization paradigm for 3DGS.
- DashGaussian accelerates optimization across various 3DGS backbones without compromising rendering quality.
- DashGaussian may improve the feasibility of training 3DGS on large-scale, time-intensive reconstruction tasks.
7. Implementation Details
The implementation equips DashGaussian with scheduled supervision, learning-rate modulation, resolution growth, and primitive densification across multiple backbones. These modifications are largely shared across backbones, with backbone-specific densification differences.
- DashGaussian downsamples ground-truth images with an anti-aliasing algorithm to supervise low-resolution renderings.This prevents aliased supervision from hindering optimization.
- The positional learning rate remains at its initial value during low-resolution optimization and decays only from a scheduled iteration.
- A logarithmic modulation adjusts the relative allocation between low- and high-resolution optimization stages.The stated goal is to suppress the low-resolution stage while expanding the high-resolution stage.
- DashGaussian modifies backbones through learning-rate, rendering-resolution, primitive-growth, and densification schedules.These modifications are common across backbones, with slight differences in densification operations.
- Densification follows each backbone's scoring mechanism, including positional gradients for 3DGS, rendering error for Revising-3DGS, and combined scores for Mip-Splatting.
8. Primitive Growth of DashGaussian
DashGaussian constrains primitive growth through threshold-based densification and top-k selection, coordinating primitive counts with the optimization stages. The double-condition scheme limits growth even when the primitive upper bound is large.
- Top-k selection limits early primitive growth, while late-stage thresholding and top-k selection jointly constrain densification.The early low-resolution stage can otherwise admit many Gaussians, whereas the late high-resolution stage has fewer primitives above threshold.
- The final optimized 3DGS contains fewer densified primitives than the specified upper bound P_fin.
- An extremely large P_fin can otherwise cause primitive-number explosion and out-of-memory issues, but the double-condition densification scheme prevents OOM when P_fin is arbitrarily large.
9. Scene-wise Qualitative Results
The paper reports complementary scene-wise quantitative results for all three evaluation datasets in separate supplementary tables.
- Scene-wise quantitative results are reported for Mip-NeRF 360, Deep Blending, and Tanks&Temples.
10. Ablation on Hyper-parameters
The ablations examine how primitive-budget and initial-resolution hyper-parameters affect DashGaussian’s quality, optimization time, and final primitive count. Larger γ increases the final primitive budget and generally trades more optimization time for higher rendering quality, while larger a reduces time without materially changing quality.
- Momentum-based Primitive Budgeting: γ and η determine Pfin, which controls the final Gaussian primitive count NGS.The analysis uses their relationship to select a primitive budget that approximates NGS and avoids under-reconstruction.
- Momentum-based Primitive Budgeting: Larger γ produces a larger Pfin, a bigger NGS, higher rendering quality, and more optimization time.Even without primitive scheduling, DashGaussian still provides significant acceleration.
- Momentum-based Primitive Budgeting: Changing γ and η is effectively equivalent to changing η/(1 −γ), so the ablation varies only γ.This follows because both parameters influence Pfin through the same combined quantity.
- Initial Resolution: Larger a substantially reduces optimization time while rendering quality remains basically insensitive to a.Here, larger a means a smaller initial rendering resolution.
11. Evaluation on Large-scale Reconstruction
On the large-scale MatrixCity reconstruction benchmark, DashGaussian reduces training time for each scene block while slightly improving reconstruction quality.
- Large-scale reconstruction: 41.6% training-time reduction is achieved on each MatrixCity block with DashGaussian and Taming-3DGS.The scene is divided into 9 blocks using VastGaussian’s divide-and-conquer strategy, and timing is rendered at 1080P.