PaperlayerView exact revision on arXiv

Source-linked AI summary

ByteSplat: Efficient Distributed 3D Gaussian Splatting Training via Intra- and Inter-GPU communication reduction

Shuo Wu, He Zhu, Han Zhao, Xiaohui Zhang, Yaqian Zhao, Hui Wei, Ruyang Li, Hongzhi Shi, Lihua Lu, Jingwen Leng, Yu Feng, Minyi Guo

arXiv:2610.02851v1cs.DC

TL;DR

Large-scale 3DGS training is constrained by data movement within and between GPUs. ByteSplat combines fused rasterization, hardware-aware pruning, and zero-eliding gradient communication. It reduces intra-GPU and backward inter-GPU traffic and achieves up to 6.1× speedup while preserving reconstruction quality.

  • Problem

    Large-scale 3DGS training requires substantial memory and computation, while distributed training is constrained by intra-GPU off-chip traffic and inter-GPU partial-gradient communication.

    Source: §1 · pp. 1–2

  • Method

    ByteSplat combines fused rasterization, hardware-aware pruning, and zero-gradient elision to reduce data movement within and between GPUs.

    Source: §1 · p. 2, §10 · p. 19

  • Results

    63.4% and 65.8%: ByteSplat reduces intra-GPU and backward inter-GPU data traffic, respectively, and achieves up to 6.1× training speedup while preserving reconstruction quality.

    Source: §1 · p. 3

  • Takeaways & Limitations

    ByteSplat achieves up to 6.1× training efficiency while maintaining the same reconstruction quality.

    Source: §10 · p. 19

    ByteSplat is one step toward addressing distributed 3DGS data-movement challenges; more fundamental improvements require algorithm-architecture co-design.

    Source: §10 · pp. 19–20

Abstract

from arXiv · show

3D Gaussian Splatting (3DGS) enables photorealistic scene reconstruction, but training large-scale scenes requires substantial memory and computation. Distributing training across multiple GPUs increases available memory capacity, yet its performance is strictly constrained by data movement. We identify two dominant bottlenecks: repeated off-chip memory accesses during forward and backward rasterization, and inter-GPU communication of partial Gaussian gradients. To address these bottlenecks, we present ByteSplat, a distributed 3DGS training framework that jointly reduces intra- and inter-GPU data movement. First, ByteSplat fuses forward rasterization and backward rasterization into a single GPU kernel, retaining intermediate results on-chip to eliminate redundant off-chip transfers. Second, to alleviate the increased on-chip storage by fused rasterization, ByteSplat introduces hardware-aware pruning that considers both rendering quality and per-tile shared-memory constraints, increasing the overall performance of fused execution. Lastly, ByteSplat exploits gradient sparsity to eliminate zero partial gradients from inter-GPU communication. GPU-efficient encoder and decoder kernels compact the remaining records and directly aggregate the received gradients on their owner GPUs, reducing both communication volume and local aggregation overhead. We evaluate ByteSplat across six datasets. Compared with the baseline, ByteSplat reduces intra-GPU off-chip traffic and backward inter-GPU communication volume by 63.4% and 65.8%, respectively. On eight GPUs, ByteSplat achieves up to 6.1$\times$ training speedup while preserving reconstruction quality.

1 Introduction

Distributed 3DGS training is constrained by data movement within GPUs and between GPUs. ByteSplat combines fused rasterization, hardware-aware pruning, and zero-eliding gradient communication to address both bottlenecks.

1 Introduction

  • Large-scale scenes can require more than 80 GB of memory, motivating distribution across GPUs to provide sufficient capacity.In this setting, each GPU retrieves Gaussians needed for its assigned image regions and returns partial gradients to their owner GPUs.Source: §1 · pp. 1–2
  • Forward and backward rasterization account for 29.9% of end-to-end training time and 37.9% of off-chip traffic; gradient aggregation accounts for 23.9% of total training time.These findings identify intra-GPU off-chip traffic and inter-GPU gradient communication as bottlenecks.Source: §1 · p. 2
  • Fused rasterization combines forward rasterization, loss computation, and backward rasterization in one mega-kernel, retaining attributes and intermediate results on-chip.The fused dataflow reduces off-chip memory traffic by 63.4%.Source: §1 · p. 2
  • Hardware-aware pruning removes low-importance Gaussians from overloaded tiles to fit fused execution within register and shared-memory budgets while preserving reconstruction quality.Unlike pruning aimed primarily at model size, this strategy accounts for hardware resource constraints and execution efficiency.Source: §1 · pp. 2–3
  • Zero-eliding communication compacts and transfers only nonzero partial gradients, reducing inter-GPU communication volume by 46.2% and accelerating end-to-end training by 1.2×.Up to 72.0% of partial gradients can be zero; specialized encoder and decoder kernels support compacting and reconstructing the communicated records.Source: §1 · p. 3
  • 63.4% and 65.8%: ByteSplat reduces intra-GPU and backward inter-GPU data traffic, respectively, and achieves up to 6.1× speedup while retaining reconstruction quality.The evaluation covers six datasets.Source: §1 · p. 3

2 Background

3DGS iteratively refines scene Gaussians by rendering camera views and propagating image losses into parameter updates. Distributed training partitions Gaussians and image tiles across GPUs, exchanging Gaussian attributes and partial gradients between owner and rendering roles.

2.1 Single-GPU Training Pipeline

  • Each iteration projects Gaussians, sorts tile intersections by depth, renders pixels with front-to-back alpha compositing, computes loss, and backpropagates gradients to update Gaussian parameters.A Gaussian may contribute to multiple pixels, so its partial gradients are aggregated before parameter updates.Source: §2.1 · pp. 3–5
  • The single-GPU pipeline renders a selected camera view against its ground-truth image, then propagates the resulting pixel-wise loss through rasterization.The forward pass includes projection, sorting, rasterization, and loss computation; the backward pass includes rasterization, gradient aggregation, and parameter updates.Source: §2.1 · pp. 3–4

2.2 Distributed Training Pipeline

  • Distributed training assigns Gaussians to owner GPUs and image tiles to rendering GPUs, shifting between Gaussian-wise and tile-wise work.Owners send projected Gaussian attributes to rendering GPUs; those GPUs return partial gradients for local aggregation and updates.Source: §2.2 · pp. 5–6

3 Motivation

Distributed 3DGS training spends substantial time and moves substantial data in rasterization and gradient aggregation. Repeated rasterization traffic and zero-valued partial gradients identify opportunities for ByteSplat to reduce movement within and between GPUs.

3 Motivation

  • Forward rasterization, backward rasterization, distributed gradient aggregation, and parameter updates together consume over 71.9% of training time.Across evaluated scenes, the four stages take 17.0%, 13.0%, 23.8%, and 18.1% of training time, respectively.Source: §3 · p. 6
  • 63.4% of off-chip traffic in forward and backward rasterization comes from repeated Gaussian-attribute accesses and intermediate-data transfers.The two rasterization stages are primarily memory-bound, motivating retention of reusable data on-chip.Source: §3 · pp. 6–7
  • 45.0% of total communication volume is backward gradient communication, while total all-to-all volume nearly doubles as GPU count rises from two to eight.Distributing tiles across more rendering GPUs can require Gaussian attributes and partial gradients to move to multiple GPUs.Source: §3 · p. 7
  • Up to 72.0% of partial gradients are zero, and this sparsity remains consistently high across datasets.Accumulating zero gradients does not affect Gaussian updates, so these transfers consume bandwidth without affecting training.Source: §3 · pp. 7–8
  • ByteSplat targets both identified movement sources by fusing forward and backward rasterization and eliding zero-valued partial gradients.The first change reduces redundant off-chip accesses; the second reduces backward inter-GPU communication.Source: §3 · p. 8

4 Fused Rasterization

ByteSplat fuses forward rasterization, loss computation, and backward rasterization to reduce redundant off-chip data movement. Its tile-local loss design addresses cross-tile SSIM dependencies while preserving reconstruction quality.

4.1 Issues of the Existing Pipeline

  • The conventional pipeline loads Gaussian attributes in both rasterization passes and writes intermediate rendered pixels and gradients to off-chip memory for reloading.These redundant accesses make both rasterization stages memory-bound.Source: §4.1 · p. 8

4.2 Fused Rasterization Dataflow

  • ByteSplat fuses forward rasterization, loss computation, and backward rasterization in one kernel, retaining reusable data on-chip to eliminate off-chip round trips.The fused dataflow avoids repeated Gaussian-attribute loading and round trips for rendered pixels, intermediate states, and per-pixel gradients.Source: §4.2 · p. 9
  • ByteSplat applies stochastic structural similarity loss independently within each tile, allowing tile-local loss and pixel-gradient computation.This design removes dependencies on neighboring tiles, avoiding cross-block synchronization and boundary-data exchange between GPUs.Source: §4.2 · p. 9
  • Tile-local loss preserves the same reconstruction quality as the original implementation.Conventional SSIM windows can cross tile boundaries, so a block cannot compute that loss from its own tile's pixels alone.Source: §4.2 · p. 9

5 Hardware-Aware Training

ByteSplat uses hardware-aware pruning to increase the fraction of image tiles that can use fused rasterization while preserving reconstruction quality. Its training strategy balances rendering contribution against per-tile shared-memory limits.

5.1 Challenges

  • Fused rasterization avoids intermediate off-chip transfers but increases on-chip storage, and tiles exceeding shared-memory capacity must use conventional execution.Large shared-memory allocations can also reduce concurrently resident thread blocks and weaken compute–memory latency overlap.Source: §5.1 · pp. 9–10

5.2 Training Strategy

  • The three-stage strategy trains and densifies the initial representation, removes Gaussians that never dominate a training pixel, then prunes against GPU tile budgets.The sequence is summarized in Fig. 8.Source: Figure 8 · p. 10, §5.2 · pp. 10–11
  • Initial pruning retains the union of each training pixel’s largest-contribution Gaussians, then fine-tunes the survivors without densification.This hardware-independent stage removes redundant Gaussians before hardware-aware pruning.Source: §5.2 · pp. 10–11
  • Hardware-aware pruning computes each tile’s Gaussian budget from target-GPU shared memory and iteratively removes candidates until budgets are met or the round limit is reached.Its priority favors Gaussians affecting severely overloaded tiles while accounting for their relative importance to reconstruction quality.Source: §5.2 · pp. 11–12
  • 33.2% to 50.7% of tiles became eligible for fused rasterization while reconstruction quality was preserved, compared with prior pruning methods.The strategy reshapes the per-tile Gaussian distribution for the target GPU’s shared-memory capacity.Source: §5.2 · p. 12

6 Zero-Eliding Gradient Communication

ByteSplat reduces gradient communication by eliding zero partial gradients and transmitting the remaining records sparsely. Its decoder aggregates received gradients directly on their owner GPUs.

6 Zero-Eliding Gradient Communication

  • Up to 72.0% of partial gradients are zero, motivating ByteSplat to remove zero-value gradients before communication.The existing approach communicates all partial gradients of every Gaussian processed by a rendering GPU.Source: §6 · p. 12

6.1 Zero-Gradient Elision

  • Zero-gradient elision removes exactly zero partial-gradient records without deleting their Gaussians, preserving gradient aggregation results and training semantics.This reduces both inter-GPU communication volume and local aggregation overhead.Source: §6.1 · p. 12

6.2 Sparse Gradient Transmission

  • The encoder groups gradients by owner GPU, identifies nonzeros with a bitmap, and packs them contiguously for sparse transmission.Fig. 9 illustrates zero-gradient elision and sparse gradient transmission.Source: §6.2 · p. 13, Figure 9 · p. 13
  • On owner GPUs, the decoder uses the bitmap to recover retained-gradient positions and directly accumulates nonzero values into Gaussian gradient buffers.Atomic operations handle updates when a Gaussian receives gradients from multiple rendering GPUs.Source: §6.2 · p. 13

7 Experimental Setup

The evaluation uses six datasets, two eight-GPU server configurations, and native-resolution training. It measures end-to-end performance and reconstruction quality, with variants designed to isolate ByteSplat’s techniques.

7 Experimental Setup

  • The experiments use eight Nvidia A6000 GPUs or eight Nvidia A100 GPUs, with 48 GB GDDR6 or 80 GB HBM memory per GPU, respectively.Training runs use one camera per GPU.Source: §7 · p. 13
  • Evaluation covers six datasets spanning large-scale urban reconstruction and indoor and outdoor 3DGS scenes, using native image resolution.Every eighth view is reserved for testing following the standard 3DGS protocol [53].Source: §7 · p. 13
  • After initial convergence, pruning runs receive 30k fine-tuning iterations with densification disabled; performance is measured by end-to-end latency and speedup, and quality by PSNR and SSIM.The evaluation also reports pruning ratio and the percentage of tiles fitting the fused-kernel resource budget.Source: §7 · pp. 13–14
  • Four variants compare the baseline, fused rasterization, fused rasterization plus sparse communication, and the full ByteSplat design.The baseline follows Grendel-GS [53].Source: §7 · p. 14

8 Evaluation

The evaluation examines ByteSplat’s training speed and rendering quality across GPU platforms and model sizes, and studies how pruning settings affect efficiency and quality across scene sizes. It relates the speed gains to reduced off-chip traffic and backward communication, while showing that quality degrades more abruptly with pruning on large scenes.

8.1 Rendering Quality

  • ByteSplat’s largest PSNR degradation is 0.1 dB, while average PSNR increases by 0.04 dB; average SSIM is 0.81 versus Grendel-GS’s 0.82.The largest SSIM decrease is 0.01; hardware-aware training lets remaining Gaussians accommodate removed primitives, reshaping their distribution for fused execution without noticeable quality loss.Source: §8.1 · p. 14

8.2 Performance

  • On eight-GPU A6000s, ByteSplat averages 3.4× speedup (up to 6.1×) over Grendel-GS; on A100s, it averages 2.7× (up to 3.6×).Performance improves across large- and small-scale scenes, with variation reflecting pruning opportunities and the share of iteration time spent in rasterization and gradient aggregation.Source: §8.2 · pp. 14–15, Figure 11 · p. 15
  • Across six A100 model sizes from 20.3 million to 120.2 million Gaussians, ByteSplat maintains 1.9–2.0× speedup over Grendel-GS.At the largest scale, per-iteration latency falls from 114.4 to 57.8 ms.Source: §8.2 · p. 15
  • Fused rasterization reduces off-chip read traffic by 55.4% on average (up to 78.3%) and write traffic by 70.9% on average (up to 81.7%) across six datasets.It avoids repeated Gaussian-attribute and intermediate-state loads, and eliminates storing rendered colors and loss gradients between separate stages.Source: §8.2 · p. 16
  • Sparse communication and hardware-aware pruning together reduce backward inter-GPU communication by 65.8% on average and up to 86.5% versus Grendel-GS.Sparse communication alone reduces traffic by 46.2% on average (up to 72.0%); pruning contributes a further 19.6% on average (up to 35.5%). Exact-zero gradient elision does not change final rendering quality.Source: §8.2 · p. 16

8.3 Ablation Study

  • In the cumulative ablation, ByteSplat reaches 3.2× speedup over Base and is 2.1× faster than Base+Fused+SpComm.Hardware-aware pruning raises the fraction of tiles meeting the fused-kernel Gaussian budget by an average 17.5% (8.9%–24.7%) across six datasets.Source: §8.3 · pp. 16–17

8.4 Scalability Analysis

  • With eight GPUs, ByteSplat achieves an average 5.9× speedup and a maximum 10.4× over one GPU; large scenes reach 7.8–10.4×, versus 1.3–5.4× for small scenes.Large scenes provide enough work for added GPUs, while inter-communication accounts for a larger fraction of iteration time in smaller scenes.Source: §8.4 · p. 17

8.5 Sensitivity Study

  • On Mill-19, increasing the pruning ratio from 0% to 63.7% lowers per-iteration latency from 69.2 ms to 29.8 ms (2.3× speedup), while PSNR falls from 25.2 dB to 22.8 dB.The supplied passage begins to describe a light-pruning condition but provides no further result for it.Source: §8.5 · p. 17
  • At 34.7% pruning, Mill-19 achieves 1.8× speedup with unchanged reconstruction quality; at 45.5%, speedup reaches 1.9× but quality loss rises to 1.8 dB.This marks a quality-efficiency turning point for the large-scale scene.Source: §8.5 · pp. 17–18
  • On Mip-NeRF 360, increasing pruning from 0% to 66.6% reduces iteration latency from 64.0 ms to 36.1 ms for 1.8× speedup, while test PSNR drops only 0.3 dB.At 58.3% pruning, latency is 41.0 ms with 1.6× speedup and 0.1 dB quality loss; light and moderate settings retain essentially unchanged quality.Source: §8.5 · p. 18
  • Small-scale scenes degrade more gradually under pruning than Mill-19, because large scenes need more Gaussians to represent geometry and appearance.Removing that necessary representation capacity causes reconstruction quality to decrease rapidly.Source: §8.5 · p. 18
  • ByteSplat’s pruning ratios capture most available efficiency improvement without entering the range where further pruning causes significant quality loss.The Mill-19 turning point contrasts with the smoother degradation observed on small-scale scenes.Source: §8.5 · p. 18

9 Related Work

Related work positions ByteSplat around reducing data movement: fused execution limits repeated off-chip accesses, while pruning and zero-eliding address tile resources and distributed gradients.

9 Related Work

  • ByteSplat reduces repeated off-chip accesses across forward rasterization, loss computation, and backward rasterization through fused execution.This contrasts with representation compression and optimizations confined to individual rendering stages.Source: §9 · p. 18
  • Unlike methods primarily controlling representation size or optimization complexity, ByteSplat incorporates per-tile shared-memory constraints into pruning to increase tiles eligible for fused rasterization.Source: §9 · p. 19
  • ByteSplat eliminates exactly zero partial-gradient records from inter-GPU communication, and its sparse decoder aggregates retained gradients without dense communication buffers.Source: §9 · p. 19
  • ByteSplat implements its optimizations on existing GPUs and targets distributed scene training, unlike application systems focused on online tracking or interactive rendering.Source: §9 · p. 19

10 Conclusion

ByteSplat addresses data-movement constraints in distributed 3DGS training with fused rasterization, hardware-aware pruning, and zero-gradient elision. It reports up to 6.1× training efficiency while maintaining reconstruction quality, while identifying further algorithm-architecture co-design as future work.

10 Conclusion

  • ByteSplat reduces intra-GPU off-chip traffic and inter-GPU gradient communication through fused rasterization, hardware-aware pruning, and zero-gradient elision.Source: §10 · p. 19
  • Up to 6.1× training efficiency is reported while maintaining the same reconstruction quality.Source: §10 · p. 19
  • More fundamental improvements are framed as requiring joint optimization of on-chip storage, execution scheduling, and inter-GPU communication.Source: §10 · pp. 19–20
Loading 2610.02851v1…