PaperlayerView exact revision on arXiv

Source-linked AI summary

Efficient Confidence-Aware Densification for 3DGS

Manuel Hofer, Thomas Köhler, Felix Windisch, Bernhard Kerbl, Markus Steinberger, Lukas Radl

arXiv:2610.13541v1cs.CVcs.GR

TL;DR

Opacity-based sampling in fixed-budget 3DGS-MCMC can concentrate new primitives in already well-reconstructed foreground regions, leaving backgrounds under-reconstructed. The paper replaces those opacity decisions with confidence-weighted image-error sampling and contribution-based relocation. It improves on 3DGS-MCMC at every tested budget and leads compared fixed-budget methods from 500k primitives onward.

  • Problem

    In fixed-budget 3DGS-MCMC, opacity-based sampling and relocation favor well-reconstructed foreground regions, leaving backgrounds under-reconstructed.

    Source: §1 · p. 2

  • Method

    The method keeps fixed-budget gradual growth, places new primitives using image error weighted by learned confidence, and relocates those contributing least to rendered images.

    Source: §1 · p. 2, §7 · p. 10

  • Results

    The method outperforms 3DGS-MCMC at every tested budget and produces the most accurate reconstructions among compared fixed-budget methods from 500k primitives onward.

    Source: §1 · p. 2, §7 · p. 10

  • Takeaways & Limitations

    The same densification works from random points as robustly as 3DGS-MCMC and can use prior-based initializations to improve on those methods from 500k primitives onward.

    Source: §7 · p. 10

    At 100k, ConeGS stays ahead from SfM points and retains better LPIPS even when the method starts from ConeGS's initialization.

    Source: §6 · p. 10

Abstract

from arXiv · show

3DGS-MCMC has replaced the gradient-driven density control of 3D Gaussian Splatting with opacity-based sampling and relocation, and is now the de facto standard for prior-free densification under a fixed primitive budget. Because opacity stays high where the scene is already well reconstructed, this sampling sends most new primitives to the foreground and leaves the background under-reconstructed. At low budgets, prior-based methods such as ConeGS and EDGS outperform it, at the cost of a pre-trained radiance field or dense image matches and a separate initialization stage. We argue for a comeback of error-driven densification within the fixed-budget MCMC framework. Our confidence-aware densification keeps the fixed budget and gradual growth of 3DGS-MCMC, but places primitives by the image error instead of the opacity. New primitives are drawn mostly where screen-space gradients are large. We favor primitives with a low learned confidence, which marks regions the reconstruction still explains poorly, and relocate the primitives that contribute least to the rendered images to these places. Averaged over 24 scenes, our method outperforms 3DGS-MCMC in PSNR, SSIM and LPIPS at every budget from 100k to 2M primitives. At 100k, without a prior or an initialization stage, it recovers 0.84dB of the 1.05dB PSNR by which ConeGS leads 3DGS-MCMC, and from 500k on, it has the highest PSNR and SSIM of all fixed-budget methods we compare with, including ConeGS and EDGS. While inherently prior-free, our densification can also exploit priors: from the ConeGS or EDGS initialization, it matches ConeGS in PSNR and SSIM at 100k and is best in all three metrics at 500k.

1. Introduction

The introduction identifies opacity-based sampling as a source of foreground bias under fixed primitive budgets and proposes confidence-weighted, error-driven densification within the 3DGS-MCMC framework. The method improves on 3DGS-MCMC across tested budgets and can also use prior-based initializations.

1. Introduction

  • Opacity-based source sampling and relocation favor already well-reconstructed foreground regions, compounding background under-reconstruction as the fixed budget fills.Figure 2 shows 3DGS-MCMC sources clustered around the central object while the proposed method covers the background more evenly.Source: §1 · p. 2, Figure 2 · p. 2
  • It keeps fixed-budget gradual growth, samples new primitives by image error weighted toward low-confidence regions, and relocates the least-contributing primitives.Placement is sampled around source primitives and color comes from training images, replacing continuous position noise; the two main changes only work together.Source: §1 · p. 2
  • The method outperforms 3DGS-MCMC at every tested budget, with gains clearest when primitives are scarce, and achieves the highest PSNR and SSIM among compared fixed-budget methods at larger budgets.It also reconstructs backgrounds in more detail; most of its Mip-NeRF 360 gain over 3DGS-MCMC lies in the background.Source: §1 · p. 2
  • The method can start from a prior-based initialization, matching or exceeding that method in PSNR and SSIM, while training in under 3 min at 500k on Mip-NeRF 360.ConeGS takes 14 min, including 2.2 min for initialization.Source: §1 · p. 2

2. Related Work

Related work spans adaptive density control, budget-focused simplification, and prior-based initialization. The paper builds on 3DGS-MCMC’s fixed-budget, prior-free framework while changing its opacity-based sampling and relocation choices.

Densification

  • Adaptive density control clones or splits primitives with large image-space gradients and prunes low-opacity primitives; later work refines the error signal with absolute gradients and per-primitive SSIM errors.The paper reuses these error signals together with confidence learned for mesh extraction.Source: §2 · p. 2
  • Budget-focused alternatives include tailored-score splitting and cloning, followed by stepwise pruning from a larger optimized set to a compact, better-distributed set.The cited prior-based approaches instead initialize from a pre-trained radiance field or dense correspondences; ConeGS also inserts primitives at high-error pixels.Source: §2 · p. 3
  • 3DGS-MCMC is the prior-free baseline: it grows toward a fixed budget and samples, relocates, and perturbs primitives by opacity, including from random initialization.The proposed method retains its budget, growth, and relocation framework but changes which primitives are selected and where new ones go.Source: §2 · p. 3

3. Preliminaries and Analysis

3DGS-MCMC [KRS∗24] relocates primitives within a fixed budget using opacity-based sampling, while learned confidence can mark under-reconstructed regions. Its opacity rules concentrate primitives in the foreground and miss high-error regions, especially where primitives are scarce.

Learned Confidence

  • Learned confidence is trained per pixel from photometric error and marks under-reconstructed regions below 1 and well-reconstructed regions above 1.The confidence map is alpha-blended from per-primitive confidences; its minimum occurs at βconf/Lrgb(p).Source: §3 · p. 3

Fixed-Budget Densification

  • 3DGS-MCMC [KRS∗24] grows primitives by 5% every 100 iterations until its fixed budget, relocates primitives with opacity at or below 0.005, and adds gated position noise.The primitives are splatted onto the image plane and alpha-blended; relocation and growth use opacity-proportional sources.Source: §3 · p. 3

Foreground Bias of 3DGS-MCMC

  • Opacity-based sampling concentrates primitives in the well-reconstructed foreground and leaves few in the background, missing high-error regions.The imbalance is present at both 100k and 500k budgets; the distribution formed during growth is corrected least where primitives are scarcest.Source: §3 · p. 3, Figure 3 · p. 3

Learned Confidence

  • At iteration 500, confidence maps are nearly white because all confidences start at 1, so sampling begins with absolute gradients before confidence is blended in over 5,000 iterations.Figure 4 shows maps for Bonsai and Counter at iterations 500, 5k, and 30k.Source: Figure 4 · p. 4

4. Confidence-Aware Densification

The method retains 3DGS-MCMC’s fixed-budget, gradual-growth framework but places new primitives according to image error and relocates those with the least contribution to rendered views. It combines confidence-aware error sampling with maximum-blending-weight relocation and new position and color initialization.

4. Confidence-Aware Densification

  • The framework keeps a fixed budget and gradual growth, using the same image-error distribution for growth and relocation while changing where primitives are placed and which are moved.New primitives are positioned by sampling from their source Gaussian, and receive color initialization from a training image.Source: §4 · p. 4

4.1. Sampling-Based Densification

  • Absolute screen-space gradients guide sampling from the first iteration, while confidence weighting increasingly favors primitives in under-reconstructed regions over the first 5,000 iterations.Confidence alone is initially uniform because all confidences start at 1; the method combines it with gradients and uses ranks to prevent scale or very large gradients from dominating.Source: §4.1 · p. 4
  • The remaining 20% of source primitives are sampled by a per-primitive SSIM term, which can reach regions down-weighted by confidence.This structural-error draw addresses visible structural error that the gradient-based distribution may miss when L1 error is low.Source: §4.1 · p. 4

4.2. Relocation by Maximum Blending Weight

  • Relocation selects primitives by maximum blending weight across training views, accounting for occlusion; relocation occurs every 400 iterations and its share fades after iteration 10,000.Early relocation is limited to invisible primitives, then shifts to the 10% with the lowest blending weights before declining smoothly.Source: §4.2 · pp. 4–5

4.3. Position Sampling and Color Initialization

  • New positions are sampled from the source Gaussian rather than continuous position noise, and a visible training image supplies the new primitive’s color when available.If no suitable view appears within 400 iterations, the primitive keeps its inherited color.Source: §4.3 · p. 5

5. Evaluation

The evaluation compares confidence-aware densification with fixed- and variable-budget methods across 24 scenes, then analyzes its components, initialization choices, and growth beyond the budget. It improves over 3DGS-MCMC at every tested budget, while analysis reveals trade-offs involving perceptual quality, peak memory, and initialization.

5.1. Novel View Synthesis

  • Across 24 scenes, our method beats 3DGS-MCMC in all three metrics at every budget, by 0.84 dB PSNR at 100k and 0.34 dB at 2M.From 500k, it has the highest PSNR and SSIM among fixed-budget methods, with a PSNR lead growing from 0.18 dB at 500k to 0.34 dB at 2M.Source: Figure 7 · p. 6, §5.1 · p. 6

5. Evaluation

  • The evaluation averages PSNR, SSIM, and LPIPS over 24 scenes from four datasets at budgets of 100k, 500k, 1M, and 2M primitives.The fixed-budget comparisons include ConeGS, EDGS, Taming 3DGS, and 3DGS-MCMC; the study also compares variable-budget methods at matched final primitive counts.Source: §5 · p. 5, §5.1 · p. 6

5.1. Novel View Synthesis

  • Against 3DGS, DashGaussian, and 3DGS + GOF, our method leads in all three metrics across every scene group, except for a tie with DashGaussian in outdoor PSNR.Mini-Splatting2 and GaussianSpa lead only in outdoor scenes on LPIPS, while our method retains the PSNR lead; allowing 1.5–2× growth brings LPIPS on par with both simplification methods.Source: §5.1 · p. 6, Table 3 · p. 8, §5.2 · p. 10

5.2. Analysis

  • Ablations show that the full method’s components work together: replacing its relocation rule loses about 0.6 dB PSNR, while either proposed component alone adds less than 0.1 dB to 3DGS-MCMC.Error-based sampling generally beats opacity sampling; confidence weighting and the SSIM term each improve gradient-only sampling, and the SSIM term yields more detailed backgrounds.Source: §5.2 · pp. 6, 8, Table 4 · p. 9, Table 5 · p. 9
  • Removing positional randomness lowers our PSNR by up to 0.88 dB, while adding 3DGS-MCMC’s continuous noise to our position sampling costs up to 0.22 dB.Without position sampling, new primitives sit exactly on their sources, and the optimizer struggles to separate them.Source: §5.2 · p. 8, Table 4 · p. 9
  • With EDGS or ConeGS initialization, our method matches ConeGS in PSNR and SSIM at 100k and is best in all three metrics at 500k; from random points, it reaches 1.9 dB higher PSNR than 3DGS-MCMC at 100k.At original 3DGS model counts, both methods reach the same PSNR from random points, but ours has better LPIPS (0.188 against 0.210).Source: §5.2 · pp. 8–9, Table 7 · p. 10
  • At 500k on Mip-NeRF 360, our method takes 2.8 minutes, while ConeGS and EDGS take about 5× and 3× as long, respectively.ConeGS’s initialization alone takes 2.2 minutes; the comparison notes that implementation differences account for part of the runtime gap.Source: §5.2 · p. 9, Table 6 · p. 9
  • Allowing growth to 1.5–2× the budget brings LPIPS on par with Mini-Splatting2 and GaussianSpa while keeping PSNR 0.5–0.6 dB higher at every tested growth factor.At 3× growth, LPIPS worsens; the method defaults to no growth to bound peak memory, with 1.5–2× suggested when memory allows.Source: Figure 10 · p. 10, §5.2 · p. 10

6. Limitations and Future Work

The method’s low-budget quality remains limited by initialization and fine-detail reconstruction, while growth beyond the budget costs peak memory. Its growth and relocation schedules are also hand-set and shared across budgets.

6. Limitations and Future Work

  • At 100k, densification cannot fully compensate for a better initialization: ConeGS remains ahead from SfM points and retains better LPIPS even when our method starts from its initialization.The cost of obtaining that start remains an open question; ConeGS’s iNGP initialization alone takes nearly as long as the full 500k training run.Source: §6 · p. 10
  • Without dense initialization, fine-detail recovery under a strict budget remains challenging outdoors, where simplification methods achieve better LPIPS at equal budget.Growth beyond the budget closes this gap at increased peak memory, but only up to about twice the budget.Source: §6 · p. 10
  • Growth and relocation schedules are hand-set and shared across budgets, although the best relocation share varies with budget.The confidence signal could also indicate when further primitives stop paying off and help set a scene’s budget.Source: §6 · p. 10

7. Conclusion

The conclusion places error-driven densification within 3DGS-MCMC’s fixed-budget framework, using image error and learned confidence to place primitives and rendered-image contribution to choose which ones are relocated. The method improves on 3DGS-MCMC at every tested budget and leads the compared fixed-budget methods from 500k primitives onward without a prior.

7. Conclusion

  • Image error weighted by learned confidence places new primitives, while contribution to rendered images determines which primitives make room for them.The method retains 3DGS-MCMC’s fixed budget and gradual growth.Source: §7 · p. 10
  • The method improves on 3DGS-MCMC at every tested budget and leads the compared fixed-budget methods in reconstruction accuracy from 500k primitives onward, without a prior.It also outperforms 3DGS, DashGaussian, and 3DGS + GOF at their own primitive counts.Source: §7 · p. 10
  • One configuration adapts across scenes, budgets, and starting points; on the optimized Faster-GS pipeline, training a 500k-primitive scene takes under three minutes.It needs no pre-trained network, radiance field, or image matcher, only one confidence per primitive and one loss term.Source: §7 · p. 10

S.1.1. Per-Scene Results

Per-scene results report strong performance across fixed-budget, random-initialization, and matched-initialization settings. The reported outcomes vary by budget and initialization, with prior-based ConeGS ahead at 100k under SfM initialization and the proposed method strongest in several other settings.

Per-Scene Results

  • At 100k with SfM initialization, prior-based ConeGS attains better results, while the proposed method attains state-of-the-art results starting at 500k primitives.The authors attribute ConeGS’s 100k advantage to initialization.Source: §S.1.1 · p. 13
  • With random initialization, the proposed method has the strongest results across tested scenes at 100k and better perceptual quality than 3DGS-MCMC at high primitive counts.The per-scene results are reported at 100k and at the primitive counts used by 3DGS.Source: §S.1.1 · p. 13
  • With matched ConeGS or EDGS initialization, the proposed method benefits from both initialization types and achieves strong results across tested scenes.Per-scene comparisons are provided for 100k and 500k budgets.Source: §S.1.1 · p. 13

S.1.2. Additional Results and Details

The supplementary results examine relocation-share choices, indoor-scene reconstruction, comparisons with variable-budget and revising methods, and implementation settings. They also document the scopes of the accompanying per-scene tables.

Additional Results and Details

  • Relocation shares from 5% to 15% have a minor performance impact; higher shares help more at large budgets, while 100k metrics decrease slightly, and the chosen 10% attains state-of-the-art results against tested methods.The comparison averages over nine Mip-NeRF 360 scenes; the table reports shares before the share falls off from iteration 10k.Source: §S.1.2 · p. 13, Table S.10 · p. 19
  • Indoor-scene results improve across tested metrics and primitive counts, with stronger PSNR gains than SSIM or LPIPS gains.The authors note that indoor backgrounds lack intricate details such as fine foliage, limiting quantitative SSIM/LPIPS deltas; background pixels are defined using depth and foreground/background erosion rules.Source: §S.1.2 · p. 13, Table S.11 · p. 19
  • For fixed-budget comparisons, initializations exceeding the budget are subsampled to the required size, and EDGS is trained with pruning enabled and pruned primitives replaced by standard densification.The authors state that this procedure follows the ConeGS setting and is intended to ensure a fair comparison.Source: §S.1.2 · p. 13
  • SfM-initialized per-scene tables report results at 100k, 500k, 1M, and 2M primitive budgets.Source: Table S.1 · p. 14, Table S.2 · p. 14, Table S.3 · p. 15
  • The variable-budget qualitative comparison shows each baseline in the top row and the proposed method trained at that baseline’s primitive budget in the bottom row.The compared methods include 3DGS, GOF densification, DashGaussian, Mini-Splatting2, and GaussianSpa.Source: §S.1.2 · p. 13, Figure S.1 · p. 16
  • The random-initialization table covers 100k and original 3DGS primitive counts, while EDGS- and ConeGS-initialized tables cover 100k and 500k budgets.The ConeGS-initialized comparison specifies 30k training iterations for both methods.Source: Table S.5 · p. 17, Table S.6 · p. 17, Table S.7 · p. 17
  • The comparison with Revising Densification covers 100k, 500k, 1M, and 2M budgets across Mip-NeRF 360, Tanks&Temples, and Deep Blending scenes.Because source code was unavailable, the authors re-implemented the method from the original manuscript.Source: §S.1.2 · p. 13, Table S.12 · p. 20

Related papers

Loading 2610.13541v1…