PaperlayerView exact revision on arXiv

Source-linked AI summary

EffGS: Efficient and High-Fidelity Gaussian Splatting

Changbai Li, Shuo Yang, Yichen Yang, Shuwei Shao, Huobin Tan

arXiv:2609.39553v1cs.CV

TL;DR

Preserving rendering quality under aggressive primitive reduction remains challenging for 3DGS, particularly in large-scale scenes. EffGS combines frequency-aware guidance, localized density control, and learnable scale modulation; across bounded and city-scale scenes, it balances reconstruction quality, training time, and primitive count.

  • Problem

    Preserving rendering quality under aggressive primitive reduction remains challenging, particularly for large-scale scenes.

    Source: §1 · p. 2

  • Method

    EffGS combines reconstruction-error scoring with scheduled spatial masks, limits density operations to Gaussians with valid sampled-view footprints, and learns per-Gaussian scale modulation.

    Source: §1 · p. 2, §3.2 · p. 4, §3.3 · p. 6

  • Results

    EffGS achieves a favorable balance between reconstruction quality, training time, and primitive count across bounded and city-scale scenes.

    Source: §5 · p. 10

  • Takeaways & Limitations

    Component ablations and matched-primitive-budget comparisons support the framework’s effectiveness, with spectral magnitude analysis complementing standard rendering metrics.

    Source: §5 · p. 10

    The evaluation covers static-scene novel view synthesis, not time-varying reconstruction, and does not characterize trade-offs across all computational budgets or hardware settings.

    Source: D LIMITATIONS · p. 27

Abstract

from arXiv · show

3D Gaussian Splatting (3DGS) enables real-time novel view synthesis, but existing general-purpose acceleration methods suffer severe rendering quality degradation when extended to more complex, large-scale scenes. To address this issue, we propose EffGS, a more general acceleration framework that improves training and rendering efficiency while maintaining reconstruction quality comparable to or better than vanilla 3DGS across bounded and large-scale scenes. EffGS combines frequency-aware guidance, localized density control, and adaptive primitive scale modulation. First, an importance scoring mechanism combines pixel-wise reconstruction errors with a difference-of-Gaussians mask scheduled over training to provide stage-dependent spatial guidance. Second, localized densification and pruning restricts density modifications to Gaussians with valid projected footprints in the sampled views. Third, learnable per-Gaussian scale modulation adjusts effective primitive extent during optimization while retaining the Compact Box rasterization rule. Extensive experiments on bounded and large-scale scene datasets demonstrate a favorable balance between reconstruction quality, training time, and primitive count. Component ablations and matched-primitive-budget comparisons further support the effectiveness of the framework.

1 INTRODUCTION

Reducing primitive redundancy can lower 3DGS costs, but preserving rendering quality under aggressive reduction remains challenging, especially in large-scale scenes. EffGS combines spatial guidance, localized density control, and adaptive Gaussian extent to address this trade-off.

1 INTRODUCTION

  • Primitive redundancy increases optimization and rasterization costs, and preserving rendering quality under aggressive primitive reduction remains challenging in large-scale scenes.The introduction points to city-scale comparisons in Table 2 and Figure 6 as illustrating this trade-off.Source: §1 · pp. 1–2
  • Reconstruction errors and scale-dependent image responses provide complementary spatial signals for emphasizing selected image structures at different training stages.The paper motivates evaluating detail preservation alongside standard reconstruction metrics.Source: §1 · p. 2
  • Localized Densification and Pruning restricts densification and pruning to Gaussians with valid projected footprints in sampled views.This distinguishes primitives outside sampled-view coverage from those in the active set; the FastGS+LDP comparison supports the restriction’s benefit.Source: §1 · p. 2
  • EffGS combines frequency-aware scoring, localized density control, and adaptive primitive compactness to balance reconstruction quality, training time, and primitive count.Component ablations and matched-primitive-budget comparisons support the framework’s effectiveness.Source: §1 · p. 2
  • Learnable per-Gaussian scale modulation adapts effective primitive extent during optimization while retaining the Compact Box rasterization rule.It modulates each Gaussian’s three base-scale components.Source: §1 · p. 2

2 RELATED WORK

Neural Radiance Fields advanced novel view synthesis but have slow rendering, while 3D Gaussian Splatting uses explicit primitives and GPU-friendly rasterization. EffGS builds on error-based scoring and Compact Box rasterization while adding spatial guidance, sampled-view localization, and learnable scale modulation.

Neural Rendering

  • NeRF represents scenes implicitly with a multilayer perceptron and volume rendering, whereas 3DGS uses explicit Gaussian primitives and GPU-friendly rasterization with alpha compositing.3DGS was introduced in response to NeRF’s slow rendering efficiency.Source: §2 · p. 3
  • Gaussian redundancy reduces 3DGS efficiency, especially in city-scale scenes, where multi-GPU training is often used to improve training efficiency.The related work describes this as increasing training time.Source: §2 · p. 3

3DGS Acceleration

  • Acceleration methods use budgeted densification, importance-based pruning, or sampled-view reconstruction errors, and EffGS adds spatial guidance, active-set density control, and learnable per-Gaussian scale modulation.The methods trade off primitive count, computational cost, and rendering quality.Source: §2 · p. 3

3 METHOD

EffGS combines frequency-aware importance scoring, view-local density control, and learnable primitive compactness, then trains with image-based losses and scale regularization.

3.1 PRELIMINARY

  • The preliminary 3DGS review describes Gaussian primitives with view-dependent color, projected covariance, alpha compositing, and adaptive cloning, splitting, and pruning.Density control clones or splits Gaussians with large positional gradients and prunes those with low opacity or excessive scale.Source: §3.1 · p. 4

3.2 FREQUENCY-AWARE IMPORTANCE SCORING

  • A scheduled difference-of-Gaussians mask supplies stage-dependent spatial guidance, while reconstruction errors contribute to importance scores and mask-weighted supervision.The second-stage decreasing bandwidth emphasizes finer structures; densification uses valid-view average scores, whereas pruning uses summed, photometric-loss-weighted counts.Source: §3.2 · pp. 4–5

3.3 VIEW-VISIBLE LOCAL DENSIFICATION AND PRUNING

  • LDP limits pruning and densification to Gaussians with valid projected footprints in sampled views, leaving primitives outside that active set unchanged.Footprint eligibility is not explicit occlusion visibility; selected primitives are cloned or split according to their base scale and densification score.Source: §3.3 · p. 6

3.4 ADAPTIVE PRIMITIVE COMPACTNESS

  • Learnable per-Gaussian scale modulation adjusts overall spatial extent while preserving anisotropic axis ratios and FastGS’s Compact Box tile-culling rule.Both base scales and modulation remain learnable, making modulation an additional optimization and density-control variable.Source: §3.4 · p. 7

3.5 OPTIMIZATION

  • EffGS combines a frequency-guided reconstruction loss with photometric and structural terms, plus regularization on learnable scale factors.The mask weights reconstruction errors; the scale regularizer favors smaller modulation factors but does not directly penalize effective Gaussian volume.Source: §3.5 · pp. 7–8

4 EXPERIMENT

EffGS is evaluated on bounded and city-scale scenes, where it balances reconstruction quality and computational efficiency. Ablations examine the contributions of frequency-aware guidance, learnable compactness, and localized density control.

4.1 EXPERIMENTAL SETUP

  • Experiments cover Mip-NeRF 360, Deep-Blending, and Tanks and Temples, plus Mill-19, UrbanScene3D, and GauU-Scene.EffGS, Speedy-Splat, and FastGS use identical 24GB GPUs; baselines that exceed that capacity on large-scale scenes use NVIDIA A800 GPUs.Source: §4.1 · p. 8

4.2 RESULTS ANALYSIS

  • 27.77 and 24.22 PSNR on Mip-NeRF 360 and Tanks & Temples, respectively, are the highest among methods in the bounded-scene comparison.EffGS improves PSNR, SSIM, and LPIPS over vanilla 3DGS on all three benchmarks, and has higher SSIM and lower LPIPS than FastGS on all three, with additional training cost.Source: §4.2 · p. 9
  • 20–22 minutes of training and 124–148 FPS accompany EffGS’s higher PSNR, SSIM, and LPIPS performance than Taming-3DGS, Speedy-Splat, and FastGS across three city-scale datasets.It achieves the best PSNR and SSIM on Mill-19 and GauU-Scene while maintaining competitive quality on UrbanScene3D.Source: §4.2 · p. 9

4.3 ABLATION STUDY

  • Removing the frequency-aware mask, error-aware mask, or frequency-guided loss reduces PSNR and SSIM and increases LPIPS.The ablation supports combining reconstruction errors with scale-dependent spatial guidance for density control and reconstruction supervision.Source: §4.3 · p. 9
  • Without learnable scale modulation, Gaussian count rises to 4.98M, training time to 28 minutes, and peak memory to 16.5GB, while PSNR falls to 23.85.Extended evaluation reports that learnable compactness reduces frequency-specific reconstruction errors while improving rendering quality with fewer primitives across representative bounded and large-scale scenes.Source: §4.3 · p. 9
  • Removing LDP lowers Gaussian count, training time, and memory use, but gives the lowest reconstruction quality among evaluated variants.Adding LDP to FastGS on the Russian scene raises SSIM from 0.741 to 0.758 and PSNR from 23.48 to 23.89, while reducing LPIPS from 0.294 to 0.257.Source: §4.3 · p. 9

5 CONCLUSION

EffGS combines frequency-aware guidance, localized density control, and learnable scale modulation to balance reconstruction quality and efficiency across bounded and city-scale scenes. Ablations and matched-primitive-budget comparisons support the framework’s effectiveness.

5 CONCLUSION

  • EffGS balances reconstruction quality, training time, and primitive count across bounded and city-scale scenes through frequency-aware scoring, localized density control, and learnable per-Gaussian scale modulation.Component ablations and matched-primitive-budget comparisons support the framework, with spectral magnitude analysis complementing standard rendering metrics.Source: §5 · p. 10

AI USE STATEMENT

Generative AI tools were used for language polishing, grammar correction, and manuscript clarity, not for results, analysis, or scientific claims.

AI USE STATEMENT

  • Generative AI tools were limited to language polishing, grammar correction, and improving manuscript clarity; authors produced and verified the technical content, experiments, and conclusions.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

The work uses publicly available research datasets for efficient 3D scene reconstruction and reports no human subjects, personal data, or sensitive attributes.

ETHICS STATEMENT

  • Experiments use publicly available research datasets for their intended academic use, and the authors report no direct ethical concerns specific to the method.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

The supplementary material documents the experimental setup and implementation, adds bounded- and large-scale comparisons, and describes plans to release code and configurations.

REPRODUCIBILITY STATEMENT

  • The authors plan to release the implementation and configuration files to facilitate reproduction of the reported results.Experiments use consistent settings unless otherwise specified.Source: REPRODUCIBILITY STATEMENT · p. 10

A MORE EXPERIMENTS

  • Additional experiments extend comparisons to city-scale methods and report model size and GPU memory, alongside frequency-extraction memory and spectral-error analyses.The extended comparisons cover Mill-19, UrbanScene3D, and GauU-Scene; further bounded-scene comparisons cover Deep Blending, Mip-NeRF 360, and Tanks & Temples.Source: A MORE EXPERIMENTS · p. 13

A.1 EXPERIMENTAL SETUP AND IMPLEMENTATION

  • Bounded-scene evaluations use a single RTX 4090 with 24GB VRAM, while several large-scene baselines exceed the 24GB constraint and use A800 GPUs.EffGS-GPUs uses TITAN RTX GPUs and a tiling strategy; initial modulation parameters are set to βi = 0, corresponding to γi = 1.Source: §A.1 · p. 13, Figure 7 · p. 14

A.2 EXTENDED RESULTS ANALYSIS

Extended experiments report matched-budget comparisons, frequency-specific analyses, and memory behavior. EffGS shows quality and efficiency advantages in several tested settings, while the spectral metric has a stated phase-related limitation.

A.2 EXTENDED RESULTS ANALYSIS

  • EffGS-GPUs achieves competitive quality against specialized city-scale pipelines, with leading results across multiple scenes and metrics.The extended analysis also compares training time, primitive count, rendering speed, memory usage, and model size.Source: §A.2 · p. 14, Figure 9 · p. 17

A.4.3 SPATIAL VISUALIZATION AND QUANTITATIVE COMPARISON

  • On Rubble, EffGS has lower spectral magnitude errors across all three frequency bands, and its stacked bars compare methods band by band rather than decomposing total pixel-space error.The visualized high-pass response map emphasizes spatial distribution after 99th-percentile normalization.Source: §A.2 · p. 14, Figure 8 · p. 15, §A.4.3 · pp. 16–17

A.2 EXTENDED RESULTS ANALYSIS

  • On Russian, FastGS+LDP improves PSNR from 23.48 to 23.89 and reduces LPIPS from 0.294 to 0.257.The comparison supports restricting density operations to the sampled-view active set within FastGS.Source: §A.2 · p. 14
  • On the illustrated bounded-scene regions, EffGS shows clearer edges and finer texture than FastGS and Speedy-Splat, alongside quantitative and training-time comparisons.The per-scene comparisons cover indoor and outdoor environments.Source: §A.2 · p. 14

A.3 MEMORY FOOTPRINT OF FREQUENCY EXTRACTION

  • During sequential frequency extraction at fixed batch size 10 on Russian, peak VRAM fluctuates within a bounded range after an initial increase, without a sustained upward trend.The measurements use varying input resolutions.Source: §A.3 · p. 14, Figure 11 · p. 19

A.4.1 FREQUENCY-AWARE MASK VISUALIZATION

  • The frequency-aware masks show schedule-dependent spatial selection: finer filters emphasize progressively finer structures for f ≥50, while f < 50 complements the response before thresholding.Masks are thresholded at 0.5, and scores are normalized independently for each setting.Source: §A.4.1 · p. 15, Figure 10 · p. 18

A.4.2 SPECTRAL DECOMPOSITION AND ERROR FORMULATION

  • The spectral analysis compares mean absolute Fourier-magnitude errors in low, mid, and high bands, with thresholds defined in frequency-grid pixels relative to evaluation resolution.Because the metric does not capture phase differences, the authors interpret it alongside standard image-quality metrics and qualitative results.Source: §A.4.2 · pp. 15–16

B.1 REFERENCE AND MATCHED-BUDGET COMPARISONS

  • At matched Gaussian budgets, EffGS achieves the best reconstruction metrics and shortest training time on Deep Blending, Tanks & Temples, Mill-19, and UrbanScene3D.The large-scene budgets are 3.76M Gaussians on Mill-19 and 3.05M on UrbanScene3D; the authors note these comparisons do not isolate individual components.Source: §B.2 · pp. 18–19, §B.1 · p. 18

B.3 EXTENDED ANALYSIS OF ADAPTIVE PRIMITIVE COMPACTNESS

The compactness ablation finds that learnable per-Gaussian scale modulation improves average reconstruction quality while reducing the average primitive count across five representative scenes.

B.3 EXTENDED ANALYSIS OF ADAPTIVE PRIMITIVE COMPACTNESS

  • The learnable factor γi changes the optimization parameterization and participates in compactness-aware primitive control, without expanding a Gaussian’s static representational space.The evaluation covers Garden, Room, Bonsai, Rubble, and Modern Building.Source: §B.3 · pp. 20, 22
  • Across five representative scenes, the full model improves PSNR from 26.99 to 28.51 dB and SSIM from 0.810 to 0.855, reduces LPIPS from 0.264 to 0.194, and lowers average Gaussian count from 2.30M to 2.02M.Average errors also fall in all three frequency bands; the quality improvement accompanies a more compact representation rather than more primitives.Source: §B.3 · p. 22

C.1 LEARNABLE SCALE MODULATION FOR PRIMITIVE COMPACTNESS

EffGS adds learnable scale modulation to adapt Gaussian extent while retaining Compact Box culling; the appendix also details the scheduled frequency mask and density-control execution.

C.1 LEARNABLE SCALE MODULATION FOR PRIMITIVE COMPACTNESS

  • Learnable per-Gaussian scale modulation reparameterizes effective extent while retaining Compact Box tile culling, and its ablations report improved reconstruction quality and efficiency in the full framework.The modulation changes the optimization parameterization and its interaction with density control, not the static family of representable covariances.Source: §C.1 · pp. 22, 24
  • With base covariance fixed, a smaller modulation factor contracts ideal projected support and can reduce intersected tiles, although discrete tile assignment changes tile counts in steps.During training, effective support depends on the product of the optimized base scales and modulation factor; total rasterization cost also depends on primitive count and arrangement.Source: §C.1 · p. 24

C.2 FREQUENCY-AWARE MASK GENERATION

  • The frequency-aware mask uses a scheduled difference-of-Gaussians response: early training selects relatively weak responses, while later training selects strong responses with finer structures emphasized.The mask is used in both importance scoring and mask-weighted reconstruction loss; its bandwidth increases in the first stage and decreases in the second.Source: §C.2 · p. 25

C.3 DENSITY-CONTROL DEFINITIONS AND EXECUTION DETAILS

  • At scheduled density updates, the method samples K distinct training views uniformly without replacement and uses the same frequency schedule for density-control masks and reconstruction supervision.The complemented and direct masks participate in density operations before and after the midpoint, respectively; either scheduled intersection may be empty.Source: §C.3 · p. 27
  • Pruning is applied before densification when both are scheduled, and density modifications are restricted to the active set.New primitives inherit their parent’s modulation parameter, receive fresh visibility estimates and scores at the next scheduled update, and are not pruned during their creation step.Source: §C.3 · p. 27

D LIMITATIONS

The evaluation covers static-scene novel view synthesis on bounded and urban-scale benchmarks, not time-varying reconstruction. The paper also identifies privacy-sensitive misuse as a potential concern and calls for safeguards.

D LIMITATIONS

  • The evaluation covers static-scene novel view synthesis on bounded and urban-scale benchmarks; effectiveness for time-varying scene reconstruction remains unevaluated.Matched-primitive-budget comparisons do not characterize performance across all computational budgets or hardware settings, and extension beyond tested configurations needs further evaluation.Source: D LIMITATIONS · p. 27

E BROADER SOCIAL IMPACTS

  • The authors identify unauthorized spatial reconstruction in privacy-sensitive contexts as a potential misuse and call for dataset sanitization, ethical guidelines, and regulatory oversight.Source: E BROADER SOCIAL IMPACTS · p. 27
Loading 2609.39553v1…