PaperlayerView exact revision on arXiv

Source-linked AI summary

Gauss What You Need: Compact Gaussian Splatting Across Scene Scales

Afif Boudaoud, Jiayi Liu, Alexandru Calotoiu, Torsten Hoefler

arXiv:2609.31248v1cs.CV

TL;DR

Selecting Gaussian count automatically across capture scales remains unresolved, as standard-benchmark settings can leave larger captures too sparse for fine detail. TangoGS sets a learning allowance from capture coverage and adapts growth and pruning using training-view quality. It matches LeGS’s mean PSNR with roughly half as many Gaussians on 13 standard scenes and achieves the highest mean PSNR among evaluated methods on eight large captures.

  • Problem

    Density control must balance reconstruction quality and model size across capture scales, but standard-benchmark settings can leave larger captures with too few Gaussians for fine detail.

    Source: §1 · p. 1

  • Method

    TangoGS derives a learning allowance from total capture pixels adjusted for overlapping coverage, then uses training-view PSNR to guide densification and pruning.

    Source: §1 · p. 2, §2.1 · p. 3, §2.2 · p. 4

  • Results

    Roughly half as many Gaussians matches LeGS’s mean PSNR on 13 standard scenes; on eight large captures, TangoGS achieves the highest mean PSNR and lowest mean LPIPS among evaluated methods.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    A single configuration calibrated on standard benchmarks serves both standard and large captures, supporting a quality–size compromise across scene scales.

    Source: §1 · p. 2, §5 · p. 10

Abstract

from arXiv · show

3D Gaussian Splatting reconstructs a scene as a collection of Gaussian primitives from a set of posed photographs called the capture. The number of primitives used to represent the scene affects reconstruction quality, storage, and rendering cost. How to select this number automatically across capture scales remains unresolved: configurations effective on standard benchmarks can leave larger captures with too few Gaussians to reconstruct fine details. We observe that the surface to represent, given by the capture's extent and resolution, is known before training, whereas its content complexity becomes apparent during training, through the reconstruction quality on the training views. We introduce TangoGS, which combines capture-derived model sizing with training-based adaptation: the capture determines the scale of the model, and training feedback determines its final size within that scale. Before training, TangoGS derives a learning allowance for model growth from the capture's total pixels after discounting views that re-observe the same scene points. During training, reconstruction quality guides how many Gaussians to add and remove. On 13 standard benchmark scenes, TangoGS matches the mean PSNR of the best-performing evaluated baseline, LeGS, with $48\%$ fewer Gaussians. On eight large captures, the same configuration automatically scales to larger models when necessary, achieving the highest mean PSNR among evaluated methods: $0.54$ dB above the runner-up with $2.3\times$ as many Gaussians. Together, capture-derived learning allowances and training-quality guided density control enable a state-of-the-art quality--size compromise across scene scales without retuning.

1 INTRODUCTION

Choosing Gaussian count means balancing reconstruction quality and model size across captures whose scale and content complexity provide different sizing signals. TangoGS derives a learning allowance from capture coverage and uses training quality to adapt the model, matching or exceeding evaluated baselines across standard and large scenes.

1 INTRODUCTION

  • Configurations effective on standard benchmarks can leave larger captures with too few Gaussians for fine detail, making quality–size balance across capture scales a central challenge.The evaluated models on a single large capture range from 87K to 7.4M Gaussians and span 5 dB in PSNR.Source: §1 · p. 1
  • TangoGS discounts overlapping observations from total capture pixels to set a learning allowance, then uses mean training-view PSNR to guide densification and pruning.The allowance establishes a model-growth budget; training feedback sets the densification target and the fraction retained during pruning.Source: §1 · p. 2
  • A single configuration calibrated on standard benchmarks serves standard and large captures, with capture-derived sizing and training-quality density control forming the design principle.The paper also reruns 13 released implementations under a common protocol across 13 standard scenes and eight large captures.Source: §1 · p. 2
  • Roughly half as many Gaussians matches LeGS’s mean PSNR on 13 standard scenes; on eight large captures, TangoGS achieves the highest mean PSNR and lowest mean LPIPS among evaluated methods.Against the large-capture perceptual-quality runner-up, reference 3DGS, it uses 1.06× as many Gaussians and achieves 0.63 dB higher mean PSNR; LPIPS is lower on seven of eight captures.Source: §1 · pp. 2–3
  • Replacing the capture-derived allowance with a fixed one reduces large-capture PSNR by at least 0.43 dB, while disabling feedback adaptation increases Gaussian count by at least 1.9× without improving PSNR.These ablations support the contribution of both the capture-derived allowance and quality-feedback adaptation.Source: §1 · p. 3

2 METHOD

TangoGS changes density control at three stages: capture information sets a learning allowance, training quality guides growth and pruning, and a final selection sets the population retained for continued optimization. Its sizing uses track-adjusted pixels, while PSNR feedback adapts use of that allowance.

2 METHOD

  • TangoGS intervenes before training to set an allowance, during training to adjust growth and pruning, and after growth to select the final population for continued optimization.Standard 3DGS begins with Gaussians at sparse structure-from-motion points, then densifies and prunes before optimizing the final population.Source: §2 · p. 3

2.1 SETTING THE LEARNING ALLOWANCE

  • The learning allowance starts from total capture pixels divided by mean SfM track length, discounting repeated observations, then maps track-adjusted pixels to Gaussians with a fixed rate and square-root scaling.The allowance uses the geometric mean of the linearly converted value and a pivot; four times the track-adjusted pixels yields twice the allowance.Source: §2.1 · p. 3

2.2 QUALITY FEEDBACK DURING LEARNING

  • Mean training-view PSNR sets a keep ratio that scales the allowance into the densification target; lower quality permits more capacity, while higher quality allows more aggressive pruning.At or below the 20 dB floor, the ratio is at least 1/s and final selection removes nothing; the parameters are set on standard scenes and applied unchanged to large captures.Source: §2.2 · pp. 3–4

2.3 SELECTING RETAINED CAPACITY

  • Budgeted densification ramps toward the adapted target, and soft pruning keeps the top fraction by sensitivity; after growth, a single prune retains a selected population for the remaining training.The final selection ranks Gaussians by accumulated, opacity-weighted effect on training views; removed Gaussians at this stage cannot be regrown.Source: §2.2 · p. 4, §2.3 · p. 4

3 EXPERIMENTS

The experiments compare TangoGS with released baselines on standard and large captures, then examine its capacity-control components and capture statistics. Results show how quality and model size vary across these settings and ablations.

3.1 SETUP

  • Thirteen released baselines were rerun on 13 standard scenes and eight large captures, with PSNR, SSIM, LPIPS, and final Gaussian count reported.Comparisons include LiteGS, the closest comparison because TangoGS builds on it, and state-of-the-art baselines.Source: §3.1 · pp. 4–5, §3 · p. 4

3.2 EVALUATION ON STANDARD BENCHMARKS

  • Across 13 standard scenes, LiteGS trained to TangoGS’s per-scene counts trails by 0.21 dB mean PSNR and has worse LPIPS.TangoGS uses 18% more Gaussians on bicycle and 14% fewer on stump, improving PSNR by 0.85 dB and 0.33 dB, respectively, while lowering LPIPS on both.Source: §3.2 · p. 5

3.3 TRANSFER TO LARGER CAPTURES

  • Across eight large captures, TangoGS improves PSNR by 0.26–0.97 dB and lowers LPIPS by 0.029–0.067 at 2.32× LiteGS’s mean count.Against Taming 3DGS, it trails by 0.45–2.14 dB on seven captures and matches within 0.01 dB on sci-art; Taming has higher LPIPS on all eight.Source: §3.3 · p. 6

3.4 ABLATIONS: CAPTURE CONDITIONING AND ADAPTATION

  • Without adaptation, final counts are 2.6× TangoGS’s on standard scenes and 1.9× on large captures; standard-scene LPIPS improves by 0.021 without PSNR improvement.Only the complete method reaches the highest PSNR among combinations below the fully spent allowance’s count in both regimes, at about half that count.Source: §3.4 · p. 7

3.5 QUALITY FEEDBACK VARIATION AND ITS EFFECT ON CAPACITY

  • Captures ending below 27 dB—including five of eight large captures—have keep ratios of 0.54–0.58 and retain 82%–97% of the grown population.Holding the ratio at 0.5 selects smaller, lower-quality models on some captures; on Garden, Building, Rubble, and Sci-Art, adaptive ratios retain 1.29× to 1.66× as many Gaussians for reported quality gains.Source: §3.5 · p. 8

3.6 ABLATION: THE CAPTURE STATISTIC

  • On standard scenes, the three tested allowance statistics reach mean quality within 0.04 dB and 0.003 LPIPS; track-adjusted pixels use 4% fewer Gaussians than SfM points on standard scenes and 12% fewer on large captures.SfM-point allowances vary with matched-observation density, reaching 1.3×–1.7× TangoGS’s on aerial captures and 0.5×–0.8× on Zip-NeRF interiors; larger models gain at most 0.3 dB in each case.Source: §3.6 · p. 9
  • Duplicating every image doubles the total-pixel allowance and raises mean final count by 36%–38%, while track-adjusted counts remain within about 10% of the original.Both statistics nevertheless lose large-capture PSNR when duplication halves the number of training epochs.Source: §3.6 · p. 9

4 RELATED WORK

The section organizes density controllers by whether they choose the Gaussian budget, adapt model size during training, or decide where to grow and prune. TangoGS couples a capture-derived allowance with quality feedback to control how much the model grows and retains.

Choosing a Gaussian budget

  • Budgeted methods specify a target count or cap, whereas TangoGS computes a pre-training learning allowance from image count, resolution, and repeated scene-point observations using one global conversion.Examples include Taming 3DGS [Mallick et al., 2024], LiteGS [Liao et al., 2025], 3DGS-MCMC, and Mini-Splatting.Source: §4 · p. 9

Adapting model size during training

  • Training-time controllers vary from learned retention masks and growth-based budgets to reconstruction-loss pruning, reinforcement learning, and prescribed count trajectories.TangoGS differs by coupling its capture-derived allowance to quality feedback during growth and final retention.Source: §4 · p. 9

Choosing where to grow and prune

  • TangoGS sets how much to grow and retain by coupling a capture-derived learning allowance with quality feedback, while leaving where to grow and prune to other mechanisms.Its targets guide LiteGS budgeted growth and sensitivity pruning.Source: §4 · p. 9

5 CONCLUSION

The conclusion distinguishes the capacity required by capture scale from the capacity needed for content complexity. It proposes setting a capacity scale from the observed surface and adapting its use through learning feedback.

5 CONCLUSION

  • Capture extent and resolution set how much capacity to make available before training, while reconstruction quality provides feedback on how much to retain as content is learned.The authors frame these as complementary density-control decisions across scene scales.Source: §5 · pp. 9–10
  • Adapting within an unsuitable allowance can restrict reconstruction quality, while sizing capacity from capture scale alone can retain unnecessary Gaussians and reduce compactness.Source: §5 · p. 10
  • The authors report using generative AI tools across methodology, experiments, implementation, dataset preparation, result interpretation, and text editing, with code and conclusions checked by the authors.Source: AI USE STATEMENT · p. 11

SUPPLEMENTARY MATERIAL

The supplement collects qualitative comparisons, TangoGS method details, and extended evaluations of its results, controls, and quality–size compromise.

Comparisons and quality–size analysis

  • Sections A, I, and J provide qualitative comparisons, benchmark quality, storage and rendering-throughput measurements, and an assessment of TangoGS’s quality–size compromise.Source: SUPPLEMENTARY MATERIAL · p. 14

Method details

Results

  • Section C reports complete per-scene results on 13 standard scenes and a reproduction check, while Section D gives complete results on eight large captures.Source: SUPPLEMENTARY MATERIAL · p. 14

Controller and capture-statistic studies

  • Sections E and F cover controller ablations and capture-statistic studies, including natural captures and exact view duplication.Source: SUPPLEMENTARY MATERIAL · p. 14

Additional evaluations

A QUALITATIVE RESULTS

The supplement describes qualitative comparisons and the shared evaluation setup, then documents allowance computation, quality-guided pruning, training schedules, and benchmark reporting. Its comparisons use matched regions and views, while the benchmark tables report quality, final count, and training time.

A QUALITATIVE RESULTS

  • The qualitative comparisons use the same models as the quantitative evaluation, matched test views and regions, and ground-truth consistency checks; crops are selected using rendering error.Crops are 240 or 140 pixels square; results averaged over all test views are reported in Section D.Source: A QUALITATIVE RESULTS · p. 14
  • In handheld-capture windows, TangoGS recovers thin structures that comparators blur, including a keyhole, drawer fronts, shutter slats, and wood grain.Source: Figure A.1 · p. 15
  • On aerial captures, TangoGS recovers vehicles, a grass edge, soil texture, and parking-lot markings smoothed over by comparators, despite reference 3DGS and CDC-GS using up to about twice as many Gaussians.Source: Figure A.2 · p. 15

B.1 CONFIGURATION AND EVALUATION

  • Default TangoGS results average two seeded runs across one configuration for 21 captures, using all official training views and 30k standard-scene or 60k large-capture iterations.Source: §B.1 · p. 16

B.2 METADATA AND EXACT ALLOWANCE

  • The allowance metadata counts held-out images and loaded resolution; mean track length measures valid image–point observations per unique tracked point, with training-model tracks used for three captures.Building, Residence, and Sci-Art use training sparse models for tracks while counting images from both training and validation splits.Source: §B.2 · p. 16

B.3 QUALITY FEEDBACK AND SENSITIVITY PRUNING

  • The implementation uses ρ = 0.7382 − 0.0075P, where P is preceding-epoch mean per-image training PSNR, and aborts removals exceeding 90% of the population.The final no-removal threshold is approximately 19.995 dB.Source: §B.3 · p. 16

B.4 TRAINING SCHEDULE AND BASELINE COMPATIBILITY

  • The training schedule starts densification at 2% and ends at 80% of training, scales epoch intervals by run length, and recalibrates quality at growth and final-selection events.The implementation also uses an SH-rest learning-rate divisor of 20, a split/clone threshold of 0.001, and opacity decay instead of resets.Source: §B.4 · p. 16

C BENCHMARK RESULTS AND REPRODUCTION

C.3 REPRODUCTION AGAINST PUBLISHED RESULTS

The reported re-runs broadly reproduce published PSNR under a common protocol, while the large-capture results use shared evaluation settings across all eight captures. Per-capture tables document the comparisons, but the supplied passages do not state their outcomes.

C.3 REPRODUCTION AGAINST PUBLISHED RESULTS

  • Nine of twelve scored baselines differ from published PSNR by less than 0.35 dB on every reported dataset; three are exceptions.The exceptions are reference 3DGS (+0.71 dB), DashGaussian (+0.38 dB), and ControlGS (−0.39 dB). LeGS reproduces published means within 0.13 dB using its evaluation-script configuration.Source: §C.3 · p. 19, Table C.6 · p. 20

D LARGER-CAPTURE RESULTS

  • Large-capture comparisons cover all eight captures at 60k iterations with shared splits and resolutions, reporting the final Gaussian populations achieved.TangoGS uses the Section B configuration; LiteGS uses its released 1M allowance, while Taming uses its 15× SfM-point multiplier on aerial captures and all three released multipliers on Zip-NeRF.Source: D LARGER-CAPTURE RESULTS · p. 20

D.1 PER-CAPTURE COMPARISONS

  • Per-capture tables cover Mill19, UrbanScene3D, and Zip-NeRF scenes, including the released Taming multipliers for the Zip-NeRF captures.The supplied passages identify tables for Rubble and Building, Residence and Sci-Art, Alameda and London, and NYC and Berlin, without giving their result values.Source: §D.1 · pp. 20–22, Table D.1 · p. 20, Table D.2 · p. 21

D.2 BEHAVIOR OF RELEASED BASELINES

Released baselines exhibit varied count and quality trade-offs on large captures. The controls separately test capture conditioning, training adaptation, and whether a fixed keep ratio can replace the adaptive one.

D.2 BEHAVIOR OF RELEASED BASELINES

  • Released regularization and pruning methods retain 6% to 46% of reference 3DGS’s population, while DashGaussian and LeGS remain below TangoGS in PSNR across most reported comparisons.LP-3DGS, Compact 3DGS, and ControlGS reach PSNR 0.7 dB to 3.0 dB below TangoGS; DashGaussian trails by 0.4 dB to 2.3 dB, and LeGS by 0.8 dB to 1.6 dB.Source: §D.2 · p. 23

E.1 CAPTURE CONDITIONING AND ADAPTATION

  • The component controls vary the allowance and adaptation independently across all 21 captures, with shared schedules, camera order, learning rates, and iteration counts.The fixed-allowance control uses LiteGS’s 1M default; the no-adaptation control removes sensitivity ranking and quality-based growth and retention.Source: §E.1 · p. 23, Table E.1 · p. 24, Table E.2 · p. 25

E.2 LITEGS AT THE SELECTED FINAL COUNT

  • At equal final count, TangoGS improves mean PSNR by 0.210 dB on standard scenes and 0.281 dB on large captures, with lower mean LPIPS in both regimes.The comparison matches final counts and optimizer updates but not training profiles; mean training populations are 0.724M versus 0.586M on standard scenes and 2.474M versus 2.255M on large captures (TangoGS versus LiteGS).Source: §E.2 · p. 25, Table E.3 · p. 26

E.3 FIXED KEEP RATIO

  • A fixed 0.5 keep ratio matches the adaptive rule closely on Room and Dr Johnson, but on four other captures adaptive ratios of 0.54 to 0.58 retain more Gaussians with better quality.On Garden, Building, Rubble, and Sci-Art, the adaptive ratios retain 1.29× to 1.66× the Gaussians, with 0.13 dB to 0.16 dB higher PSNR and 0.006 to 0.019 lower LPIPS; Room and Garden share a dataset yet end at different ratios.Source: Table E.4 · p. 26, §E.3 · p. 27

E.4 TRAINING COST

The section reports training-cost measurement conventions, compares track-adjusted allowances with total-pixel and SfM-point controls, and tests sensitivity to duplicated views.

E.4 TRAINING COST

  • Training-cost reporting includes final size and mean training population, while a shared iteration count alone does not imply equal FLOPs.Reported subprocess times include loading, scoring backward passes, and diagnostics on one GH200.Source: §E.4 · p. 27, Table E.5 · p. 27

F.1 NATURAL CAPTURES

  • On large captures, track adjustment uses 4.6% fewer Gaussians than the total-pixel control, with 0.007 dB lower PSNR and 0.00064 higher LPIPS.Against the SfM-point control, it uses 30% fewer Gaussians on aerial captures with 0.16 dB lower PSNR and 0.0143 higher LPIPS, but 47% more on Zip-NeRF with 0.082 dB higher PSNR and 0.0093 lower LPIPS.Source: §F.1 · pp. 27–28, Table F.1 · p. 28, Table F.2 · p. 28

F.2 VIEW DUPLICATION

  • With duplicated views and a halved epoch count, track-adjusted counts stay within approximately 0.91–1.10× their originals, while total-pixel counts grow to 1.27–1.54×.Both variants lose PSNR on large captures: 0.34 dB for track adjustment and 0.27 dB for total pixels; the largest losses are on aerial captures.Source: §F.2 · p. 29, Table F.3 · p. 30

G RETUNING REQUIREMENTS FOR LARGE CAPTURES

The paper discusses why automatic count selection can still entail calibration, then evaluates the controller on reference 3DGS and reports model-size and rendering trade-offs. Its measurement protocol and per-capture tables define the scope of those comparisons.

G RETUNING REQUIREMENTS FOR LARGE CAPTURES

  • Explicit target counts still require selecting a useful capacity for a new capture, while indirect thresholds and regularization can have capture-dependent effects.Interacting growth and retention controls may require several trials, and the comparison does not measure retuning effort or establish that retuned baselines remain inferior.Source: G RETUNING REQUIREMENTS FOR LARGE… · pp. 31–32

H THE CONTROLLER ON REFERENCE 3DGS

  • On standard scenes, the controller is within 0.3 dB of released Taming 3DGS on every dataset with 2.3× to 3.6× fewer Gaussians; on large captures, it is 0.8 dB higher on average.On large captures it is ahead on seven of eight, using 1.0× to 2.2× Taming’s count on aerial captures and 5× to 9× on Zip-NeRF.Source: H THE CONTROLLER ON REFERENCE 3DGS · p. 33, Table H.1 · p. 33

I.1 MEASUREMENT PROTOCOL

  • Storage measures saved model files in decimal MB, while rendering throughput times each native implementation on the same held-out cameras and settings used for quality evaluation.Rendering does not isolate the speed effect of density control or compare models at matched quality.Source: §I.1 · p. 34

I.2 AGGREGATE COMPARISON

  • On standard scenes, TangoGS uses less saved storage than LiteGS and reference 3DGS while achieving higher mean native rendering throughput; on large captures, higher quality accompanies larger models and lower FPS than LiteGS.Aggregate results use equal-capture means; the table reports PSNR, saved model size, and throughput.Source: §I.2 · p. 34, Table I.1 · p. 34

I.3 STANDARD SCENES

  • Standard-scene per-capture tables pair test PSNR and VGG LPIPS with storage and throughput for the same saved checkpoints.The tables cover Mip-NeRF 360, Tanks & Temples, and Deep Blending scenes.Source: §I.3 · p. 35, Table I.2 · p. 35, Table I.7 · p. 37

I.4 AERIAL CAPTURES

  • Aerial-capture per-capture tables report quality, storage, and throughput for Mill19 and UrbanScene3D scenes.The supplied table captions identify Rubble and Building, and Residence and Sci-Art.Source: Table I.9 · p. 38, Table I.10 · p. 39

I.5 HANDHELD CAPTURES

J CAPACITY SELECTION AGAINST AN OFFLINE SWEEP

Across six captures, TangoGS meets the joint quality target automatically on four, with at most 1.34× the smallest evaluated qualifying count. Bicycle and Rubble need a larger allowance to meet both limits.

J CAPACITY SELECTION AGAINST AN OFFLINE SWEEP

  • The comparison sweeps six allowance multipliers on six captures with seed 4114, while keeping the controller and other training settings fixed.The thirty non-1× endpoints are offline references and provide no per-capture tuning to TangoGS.Source: J CAPACITY SELECTION AGAINST AN… · p. 41
  • A model qualifies when its losses versus the 2×-allowance reference are at most 0.30 dB PSNR and 0.020 VGG LPIPS jointly.The objective compares TangoGS’s automatic count with the smallest evaluated endpoint that meets both limits.Source: J CAPACITY SELECTION AGAINST AN… · p. 41, Table J.2 · p. 41, Figure J.1 · p. 42
  • 4/6 automatic models meet both quality limits; among qualifying models, count is at most 1.34× the smallest evaluated qualifying count.Residence and London match that smallest count; Bonsai and Berlin use 33.2% and 32.5% more Gaussians, respectively.Source: J CAPACITY SELECTION AGAINST AN… · p. 41, Table J.1 · p. 41
  • Bicycle misses the LPIPS limit despite a 0.160 dB PSNR loss; Rubble misses both limits, while 1.5×-allowance models meet both on each capture.Rubble’s losses are 0.408 dB PSNR and 0.0364 LPIPS; Bicycle’s LPIPS increase is 0.0356 against the 0.020 limit.Source: Table J.2 · p. 41, J CAPACITY SELECTION AGAINST AN… · p. 42

Related papers

Loading 2609.31248v1…