PaperlayerView exact revision on arXiv

Source-linked AI summary

BoundedGS: Memory-Efficient High-Resolution Training for 3D Gaussian Splatting

Xiaobin Deng, Changyu Diao, Min Li, Duanqing Xu

arXiv:2610.34233v1cs.CV

TL;DR

High-resolution 3DGS training increases rasterization work while rasterization, projection, and gradient state can coexist at peak memory. BoundedGS bounds sequential regional rasterization and uses candidate-indexed projection and compact gradients. It reports a nearly constant rendering working set in a fixed-count resolution sweep and reduced working set with similar quality and shorter training time on native 5K scenes.

  • Problem

    At higher observation resolution, pixel and Gaussian–tile intersection counts increase rendering resources, while rasterization, projection, and gradient state can coexist at peak memory.

    Source: §1 · p. 1

  • Method

    BoundedGS processes load-bounded image regions sequentially, separates rasterization and projection backward passes, and uses view candidates with compact gradient storage.

    Source: §1 · p. 2, §3.2 · p. 4, §3.3 · p. 5

  • Results

    The rendering working set stays within 1.073–1.077 GiB from 1K to 16K at about 6M Gaussians, and five native 5K scenes show an 82.3% mean working-set reduction with similar quality and shorter training time.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Regional contributions can be aggregated into one complete-view update while releasing local rasterization state, controlling the rendering caches as resolution grows.

    Source: §1 · pp. 1–2

    Leaf-wise SSIM uses independent windows, so boundary treatment differs from full-image SSIM and changes with leaf layout.

    Source: §3.2 · p. 4

Abstract

from arXiv · show

High-resolution 3D Gaussian Splatting training increases pixel and Gaussian-tile intersection counts, expanding rasterization caches that coexist with projection and gradient state at peak memory. A complete-view update aggregates gradients across the view; each region's forward and backward caches can be released after its contribution is collected. BoundedGS separates rasterization and projection backward passes and bounds the active local rasterization working set according to its measured load. Candidate-indexed projection and compact gradient storage address the view-level peaks exposed by this change, while CPU-assisted structure rebuilding and bounded image supply manage resources outside rendering. In a resolution sweep at a fixed count of about 6M Gaussians, the measured rendering working set stays within 1.073-1.077 GiB from 1K to 16K; at 11K, a warm pass takes 19.79 seconds versus 21.76 for full-frame execution. Across five native 5K outdoor scenes, full training reduces the mean rendering working set from 9.97 to 1.76 GiB (82.3%), retains similar reconstruction quality, and shortens training time from 36.7 to 33.9 minutes. Project page: https://xiaobin2001.github.io/BoundedGS-Web/

1 INTRODUCTION

High-resolution 3DGS training can exceed device memory because resolution-driven rasterization state coexists with projection and gradient state. BoundedGS bounds regional rasterization work and limits view-level projection and gradient storage while retaining complete-view updates.

1 INTRODUCTION

  • Higher observation resolution increases each view’s resource requirements even when scene extent and Gaussian count remain approximately fixed, and a peak beyond device capacity prevents training that view.The peak also depends on simultaneous rasterization, projection, and gradient state.Source: §1 · p. 1
  • Across five native 5K outdoor scenes, full training reduces mean rendering working set by 82.3%, with similar quality and shorter training time.At fixed count of about 6M Gaussians, the measured rendering working set stays within 1.073–1.077 GiB from 1K to 16K; diagnostics and native-scene training evaluate memory and time.Source: §1 · p. 2
  • BoundedGS completes each region’s forward and backward work, releases its local state after collecting gradient contributions, and aggregates them before one parameter update.Rasterization and projection backward computation are separated; load-bounded raster leaves, candidate-indexed projection, and compact gradients are integrated with structure rebuilding and image supply.Source: §1 · p. 2

2 RELATED WORK

Prior work improves 3DGS efficiency through representation changes, compact or culled rasterization state, and ways of partitioning high-resolution training. BoundedGS instead targets the co-residency and lifetime of temporary state within a complete view on one GPU.

Representation quality, model capacity, and training efficiency

  • Prior 3DGS work spans quality and convergence, model-size reduction, structured or dynamic representations, and scaling scene extent through spatial partitions and hierarchies.BoundedGS focuses on temporary memory peaks while retaining the global model and input resolution.Source: §2 · p. 2

Rasterization, projection, and intermediate state

  • Rasterization-efficiency methods use compact state, overlap culling, sorting, or backward-kernel changes, while BoundedGS controls which intermediate states coexist.The related work also notes that BoundedGS manages the time cost of local execution.Source: §2 · p. 2

Scaling resources for high-resolution training

  • High-resolution systems partition pixels or models, sample supervision, filter candidates, or distribute work; BoundedGS combines view-level candidates with load-bounded local rasterization.It retains full-pixel supervision and one update per view, and integrates projection, gradients, and other resource management in the same training path.Source: §2 · p. 3

3 METHOD

BoundedGS limits temporary memory by controlling the lifetime and range of rasterization state, then using view candidates to reduce detailed projection and gradient storage. It also manages structure rebuilding and image supply after rendering caches are bounded.

3.1 THE TRAINING STEP AND MEMORY LIFETIMES

  • A conventional view update retains regional backward states together, while rasterization, supervision, and loss state grow with image pixels and Gaussian–tile intersections.Projection caches and parameter gradients are temporary view-level state; persistent Gaussian parameters, Adam state, and density statistics form a separate storage class.Source: §3.1 · p. 3, Figure 1 · p. 4

3.2 BOUNDED EXECUTION OF RASTERIZATION

  • Rasterization and projection backward passes are separated so sequential regions can accumulate projected-attribute gradients and release local caches before one update.The figure traces co-resident projection, gradient, and rasterization/supervision/loss state for one view, with a 9.55 GiB peak.Source: §3.2 · p. 4, Figure 1 · p. 4, §3.1 · p. 4
  • Independent SSIM windows in each leaf change boundary treatment with leaf layout, although pixel-weighted additive L1 preserves the full-image pixel mean.Final quality is evaluated on complete images.Source: §3.2 · p. 4
  • Regions are split along tile boundaries when measured intersection, pixel, and buffer demand exceeds the rasterization budget, down to one tile.The CPU-cached leaf layout can be further split or siblings merged after current load is recounted; higher resolution distributes added load across more sequential leaves.Source: §3.2 · pp. 4–5

3.3 CANDIDATE-DRIVEN PROJECTION AND GRADIENT STORAGE

  • View candidates limit detailed projection and gradient storage to potentially contributing Gaussians while retaining the global model and exact intersection determination.Conservative screening uses position, opacity, maximum scale, and camera parameters; it retains uncertain cases and permits false positives.Source: §3.3 · p. 5
  • Candidate-indexed compact gradients accumulate rasterization contributions before projection backward, while global opacity regularization still requires a dense opacity gradient.Parameters and Adam state keep full layouts; outside-candidate image gradients are zero, and the optimizer reads compact gradients and IDs directly.Source: §3.3 · p. 5

3.4 RESOURCE PEAKS IN FULL TRAINING

  • After view caches are released, CPU-assisted structure rebuilding constructs the final parameter, optimizer-state, and statistics layout with reduced GPU overlap.The approach trades host memory and transfers for that reduced overlap.Source: §3.4 · p. 6
  • A fixed-capacity prefetch queue decodes compressed host images and supplies only the current and next leaf’s supervision to GPU memory.Leaf regions are transferred as uint8, converted to floating point when used, and released after loss backward.Source: §3.4 · p. 6

4 EXPERIMENTS

The experiments compare BoundedGS with full-frame Original across full-training benchmarks, component diagnostics, and a fixed-Gaussian-count resolution sweep. They report lower rendering working sets with similar reconstruction quality, alongside measured timing trade-offs and resource peaks outside rendering.

4.1 EXPERIMENTAL SETUP

  • The full-training comparison covers five outdoor and four indoor Mip-NeRF 360 scenes plus four Tanks and Temples scenes, with shared training schedules and model-count procedures.Training runs for 30K steps from SfM initialization; densification ends at 15K steps, followed by pruning toward 50% retention.Source: §4.1 · p. 6

4.2 FULL-TRAINING QUALITY AND COST

  • 82.3%: across five native 5K outdoor scenes, the mean rendering working set falls from 9.97 to 1.76 GiB, with similar quality and training time dropping from 36.7 to 33.9 minutes.Mean PSNR changes by −0.02, +0.12, and −0.01 dB across the 5K, 3K, and 4K groups, respectively; SSIM and LPIPS remain close at displayed precision.Source: §4.2 · pp. 6–7, Table 1 · p. 7
  • Figure 2 shows BoundedGS retaining similar local detail to Original in the displayed Bicycle and Train regions.The comparisons include Bicycle bench slats and occlusion edges, and Train lettering and panel texture.Source: §4.2 · p. 7, Figure 2 · p. 7

4.3 RENDERING COMPONENTS AND MEMORY LIFETIMES

  • 83.3%: the complete 384 MiB configuration cuts working set relative to Original, while time falls by 14.2% and terminal-rendering SSIM remains near one.Separating rasterization and projection backward reduces the mean working set from 9.98 to 7.64 GiB; candidate projection and compact gradients then reduce it to 3.27 and 1.66 GiB.Source: §4.3 · pp. 7–8, Table 2 · p. 8

4.4 RESOLUTION SCALING

  • 1.073–1.077 GiB: BoundedGS stays within this rendering working set from 1K to 16K at about 6M Gaussians, while Original reaches OOM at the second 12K step.At 11K, the 169-view warm pass takes 19.79 seconds with BoundedGS versus 21.76 seconds with Original.Source: §4.4 · p. 9, Figure 4 · p. 9

4.5 RESOURCE PEAKS IN FULL TRAINING

  • CPU-assisted rebuilding lowers the structure-update peak from 12.83 to 8.68 GiB (32.4%) but adds 11.7% training time; bounded JPEG supply lowers host RSS by 24.0% at a 5.5% time cost.The rendering path and its working sets are the same across these resource comparisons.Source: §4.5 · p. 9, Table 3 · p. 9

5 CONCLUSION

The conclusion presents BoundedGS as a way to control high-resolution 3DGS rendering working sets while retaining similar quality and reducing training time in native 5K outdoor scenes. The paper also reports its research context and supporting appendices.

5 CONCLUSION

  • On five native 5K outdoor scenes, full training reduces mean rendering working set by 82.3%, retains quality close to Original, and shortens time from 36.7 to 33.9 minutes.At about 6M Gaussians, the reported rendering working set stays within 1.073–1.077 GiB across 1K–16K.Source: §5 · p. 9
  • BoundedGS combines load-bounded raster leaves, separated backward stages, and compact candidate state to control high-resolution working sets.The paper identifies co-resident rasterization, projection, and gradient state as the source of the complete-view memory peak.Source: §5 · p. 9
  • The study evaluates 3D reconstruction on established benchmark datasets and reports no known ethics issues specific to its method or evaluation.Appendices document evaluation and memory accounting, execution rules, fixed-block resource comparisons, and per-scene training results.Source: REPRODUCIBILITY STATEMENT · p. 10

A EXPERIMENTAL PROTOCOL AND ACCOUNTING

The protocol compares full-frame Original and BoundedGS under shared training settings across Mip-NeRF 360 and Tanks and Temples scenes. It specifies the data, optimization schedule, and evaluation metrics used for the full-training comparison.

A.1 MAIN-TABLE RUNS AND COMPARISON CONFIGURATION

  • The comparison uses five outdoor and four indoor Mip-NeRF 360 scenes and Barn, Family, Train, and Truck from Tanks and Temples, with original-resolution Mip-NeRF 360 images and 4K-video-derived Tanks and Temples frames.Every eighth filename-sorted image is reserved for testing; the rest form the training pool.Source: §A.1 · p. 13
  • Both methods train for 30K steps from SfM initialization with shared densification, pruning, loss weights, optimizer, CPU-assisted rebuilding, and bounded JPEG supply.Densification ends at 15K steps, followed by gradual reduction toward 50% Gaussian retention; peak budgets are 12M, 6M, and 8M for the three scene groups.Source: §A.1 · p. 13
  • The comparison evaluates final models on all test views using PSNR, SSIM, and LPIPS, with scenes equally weighted within each group.Original uses full-frame projection and rasterization with dense parameter gradients; BoundedGS uses one projection group, a 384 MiB Auto Leaf, candidate projection, and compact gradients.Source: §A.1 · p. 13

A.2 MAIN-TABLE TRAINING TIME AND RENDERING WORKING SET

The experiments define rendering working set and training time measures, describe fixed-count component diagnostics, and compare resolution scaling with and without BoundedGS. In the sweep, Original runs out of memory at 12K while BoundedGS completes through 16K.

A.2 MAIN-TABLE TRAINING TIME AND RENDERING WORKING SET

  • Main-table time is synchronized training-loop wall-clock time, while rendering working set is measured at the iteration with the largest allocated peak after subtracting persistent state.The measure retains parameter gradients, projection, rasterization, supervision images, and loss caches; structure-update allocated peaks are reported separately.Source: §A.2 · p. 13

A.3 COMPONENT DIAGNOSTICS AT FIXED GAUSSIAN COUNTS

  • Component diagnostics hold each scene’s 15K SH3 model and Gaussian count fixed, then compare execution paths over two ordered passes, with quality measured against Original renders.The second pass supplies cumulative time and peak temporary working set; the final models are evaluated using view-averaged SSIM, then averaged across scenes.Source: §A.3 · p. 13

A.4 RESOLUTION SWEEP AT A FIXED GAUSSIAN COUNT

  • At 16K, Original’s extrapolated rendering working set is approximately 31.76 GiB; Original runs out of memory at the second 12K step, while BoundedGS completes 1K–16K.The extrapolation adds 0.78 GiB of projection state to the rasterization-cache estimate anchored at 11K.Source: §A.4 · p. 14

A.5 FULL-TRAINING RESOURCE ABLATION

The resource ablation compares structure rebuilding and image-supply configurations, alongside the execution details that bound per-view rendering state. The method screens candidate Gaussians, processes load-bounded leaves, accumulates gradients, and manages structure and image data separately.

A.5 FULL-TRAINING RESOURCE ABLATION

  • The 30K-step resource ablation compares the full CPU-assisted rebuilding and bounded JPEG-supply path against repeated GPU rebuilding and host-resident predecoded images.All configurations share a one-group, 384 MiB Auto Leaf, candidate projection, and compact gradients; time excludes evaluation and saving.Source: §A.5 · p. 14

B.1 STATE AND EXECUTION ORDER FOR ONE VIEW

  • For each view, groups are processed in row-major order, leaves accumulate local rasterization gradients, and projection backward runs once per group before the optimizer updates.Candidate IDs are screened first; each group releases its projection and gradient caches before the next group, and global regularization is computed once after all groups.Source: §B.1 · p. 14

B.2 CONSERVATIVE CANDIDATE COVERAGE

  • Candidate screening uses a rotation-independent scale bound to form conservative screen coverage; nonfinite projections remain candidates, and group membership follows rectangle intersection with tile-padded bounds.Detailed projection and ellipse–tile intersection then follow the original rasterizer.Source: §B.2 · p. 14

B.3 LEAF BUDGET AND LAYOUT REUSE

  • Leaf load is estimated from intersection, image, sampling, and output allocations; oversized tile-aligned regions are split until they fit the budget or reach one tile.The CPU-cached layout recomputes current loads on each visit, allowing further subdivision and rechecked mergers as conditions change.Source: §B.3 · p. 15

B.4 LEAF LOSS, COMPACT GRADIENTS, AND ADAM

  • Leaf losses combine 0.8 L1 and 0.2 (1 − SSIM), area-weight the L1 term, and apply the pruning-window global regularizer once per view.The global term follows GNS during steps 15K–19K; each leaf’s rasterization backward contributes to group gradients before projection backward.Source: §B.4 · p. 15
  • Compact gradients use deduplicated candidate rows for position, SH, scale, and quaternion, while opacity remains dense and Adam updates the full parameter layout.Rows outside current candidates have zero current image gradient, but their first- and second-order momentum continue to update.Source: §B.4 · p. 15

B.5 STRUCTURE UPDATES AND IMAGE SUPPLY

  • CPU-assisted rebuilding stages retained and new parameter, Adam, and density-statistic rows in host arrays before uploading the target layout; images are decoded through a bounded queue.Only the active leaf’s image region is uploaded asynchronously, converted to FP32 for loss computation, and released afterward.Source: §B.5 · p. 16

B.6 STORAGE COMPOSITION AND CAPACITY ESTIMATE

The storage estimate quantifies the persistent per-Gaussian state for the main-table SH3 setting and its implications for a 12M-Gaussian budget.

B.6 STORAGE COMPOSITION AND CAPACITY ESTIMATE

  • At SH3, storage is about 0.685 GiB per million Gaussians, making a 12M-Gaussian budget about 8.23 GiB.The estimate covers parameters, two Adam-momentum tensors, per-Gaussian steps, and persistent density statistics.Source: §B.6 · p. 16

C.1 FIXED-PIXEL BLOCKS ACROSS RESOLUTION

The comparison tests fixed-pixel blocks against load-bounded Auto Leaf across resolution. At 16K, Auto Leaf uses less rendering working set and less warm-pass time than accumulated fixed blocks.

C.1 FIXED-PIXEL BLOCKS ACROSS RESOLUTION

  • At 16K, Auto Leaf uses 1.073–1.076 GiB and takes 35.7 seconds, versus accumulated fixed blocks at 1.24 GiB and 49.3 seconds; it reduces both measures across tested resolutions.The figure’s first three panels show peak temporary rendering state for a representative Bicycle view; panel D reports synchronized second-pass time over 169 views.Source: §C.1 · pp. 16–17, Figure 5 · p. 17

C.2 INDEPENDENTLY UPDATED FIXED GRIDS

On Bicycle’s 11.98M-Gaussian model, independently updated fixed grids reduce temporary working-set size but take longer as grids become finer. BoundedGS Auto Leaf improves on fixed 2 × 2 in both measures.

C.2 INDEPENDENTLY UPDATED FIXED GRIDS

  • BoundedGS Auto Leaf used 1.57 GiB and 11.57 seconds, versus 1.94 GiB and 21.84 seconds for fixed 2 × 2.The second pass covered 169 views, with one Adam update per block.Source: §C.2 · pp. 17–18
  • Fixed 4 × 4 reduced the working set to 5.66 GiB but took 122.15 seconds, versus Original’s 9.58 GiB and 13.85 seconds.Fixed 2 × 2 used 6.55 GiB and took 35.46 seconds; each block repeated full projection.Source: §C.2 · p. 18
  • With stage separation, candidate projection, and compact gradients, fixed 2 × 2 and 4 × 4 used 1.94 and 0.85 GiB, taking 21.84 and 65.36 seconds.The figure reports all-view peak and time for the warm pass, plus one-view trace peaks.Source: §C.2 · p. 18, Figure 6 · p. 18
  • Panel F saved a further 0.72 GiB over BoundedGS atop about 8.21 GiB of persistent state, but required about 5.65 times the warm-pass time.Independent 2 × 2 and 4 × 4 block updates also perform four or sixteen Adam steps per view.Source: §C.2 · p. 18

D PER-SCENE FULL-TRAINING RESULTS

Across 13 scenes, BoundedGS reduces the temporary rendering working set while largely preserving PSNR and lowering mean training time across all three resolution groups.

D PER-SCENE FULL-TRAINING RESULTS

Related papers

Loading 2610.34233v1…