PaperlayerView exact revision on arXiv

Source-linked AI summary

From Splats to Silicon: Rethinking Computational Efficiency of 3DGS

Minnan Pei, Qiwei Dong, Yihan Zhou, Gang Li, Yuchen Zhu, Wenju Zhao, Zhongtian Long, Siting Wang, Peisong Wang, Jian Cheng

arXiv:2609.06157v1cs.ARcs.AIcs.GR

TL;DR

3DGS efficiency is difficult to compare because reported gains affect different points in rendering and update paths. This paper uses workload tracing, reproduced measurements, and GPU profiling to connect those changes to end-to-end execution, finding that downstream work, stage granularity, and data movement determine system benefit.

  • Problem

    The central problem is that 3DGS efficiency varies across workloads and platforms, while existing gains target different rendering and update stages.

  • Method

    The paper applies a workload-centric framework that traces Gaussian selection, screen-space work, data movement, synchronization, cached results, gradients, and optimizer data across algorithms, runtimes, and architectures.

  • Results

    Across literature analysis, reproduced measurements, and controlled profiling, system gains depend on reductions reaching downstream execution and on granularity matching each stage.

  • Takeaways & Limitations

    End-to-end evaluation should use traceable workloads, aligned timed operations, rendering-quality constraints, and sustained measurements.

  • Takeaways & Limitations

    Faster iterations or matched settings do not guarantee matching rendering quality or final reconstruction quality.

Abstract

from arXiv · show

3D Gaussian splatting (3DGS) represents scenes with explicit primitives and supports real-time novel-view synthesis, yet its system efficiency varies substantially across scenes, viewpoints, rendering paths, and platform constraints. Existing studies pursue efficiency through representation and algorithm design, GPU runtime optimization, and architectural support, but their reported gains correspond to different points along the rendering and update paths. Connecting these indicators to end-to-end system benefit requires tracing how each optimization changes Gaussian selection, screen-space work, data movement, and stage or frame time. We therefore use a workload-centric framework to connect representation and algorithm research, GPU runtimes, and hardware architectures and to identify recurring workload patterns. We complement literature analysis with reproduced measurements and controlled GPU profiling of selected implementations, relating workload counts to stage time and memory traffic. Together, these comparisons show that system gains depend on workload reductions reaching downstream execution, granularity matching each stage, and the cost of data transfers, synchronization, and cached results, gradients, and optimizer data. Building on these findings, we discuss more consistent evaluation under rendering-quality constraints and identify key directions for future system design.

1 INTRODUCTION

3DGS enables real-time novel-view synthesis through explicit Gaussian primitives, but efficiency varies with view-dependent workloads and platform resources. The paper addresses this variation with a workload-centric comparison spanning algorithms, GPU runtimes, and hardware.

  • Motivation: 3DGS uses explicit anisotropic Gaussians to support direct manipulation and real-time rendering across reconstruction, dynamic editing, robotics, and mobile deployment.Its broader application scope makes efficient execution across heterogeneous platforms a central systems challenge.
  • Motivation: View-dependent costs vary with Gaussian count, screen-space coverage, and per-pixel blending depth.Visibility culling and projection create footprints, sorting expands them into Gaussian–tile associations, and traversal evaluates pixel contributions.
  • Research Scope: Efficiency research spans representation and algorithms, GPU runtimes, and specialized hardware.These approaches target model size, selected Gaussians, association generation, sorting, scheduling, dataflow, and storage.
  • Approach: The paper compares studies by tracing workloads from Gaussian selection through association construction, pixel processing, compositing, and updates.It records the first affected stage, computation or data-movement changes, and whether work is eliminated, reorganized, remapped, or reused under aligned task and quality conditions.
  • Findings: Three recurring patterns are that upstream savings depend on downstream costs, stage granularity redistributes overhead, and local acceleration shifts the remaining bottleneck.Literature analysis is combined with reproduced measurements and controlled GPU profiling relating workload counts to stage time and memory traffic.

2 UNDERSTANDING 3DGS EFFICIENCY

The paper frames 3DGS efficiency as the cost of converting explicit scene data into view-dependent rendering work. It introduces the representation, projection mathematics, and tile-based execution path that determine this cost.

  • Scene Representation: An explicit Gaussian stores position, shape, opacity, and appearance, with covariance parameterized by scale and rotation.The mean represents the center, while the covariance describes the 3D shape.
  • Scene Representation: Appearance combines opacity with view-dependent color represented in original 3DGS by spherical harmonics.The viewing direction selects the spherical-harmonic basis used to evaluate color from stored coefficients.
  • Projection: Projection transforms Gaussian centers and covariances into image space, where the resulting 2D covariance determines affected tiles and pixels.The camera transform W and local projection Jacobian Ji determine the screen-space covariance.
  • Execution Path: The tile-based renderer selects visible Gaussians, projects footprints, creates Gaussian–tile records, orders them by depth, and traverses tile lists for pixel evaluation.Pixel threads evaluate Gaussian response, opacity, transmittance, and color contribution.

PROJECTION SORTING BLENDING

The forward path consists of Projection, Sorting, and Blending, while training adds backward updates and optimizer state. A workload-centric analysis follows how interventions change stored Gaussians, associations, evaluations, data movement, and synchronization across these stages.

  • Stage Definitions: Projection, Sorting, and Blending respectively cover footprint construction, Gaussian–tile association and depth ordering, and ordered pixel-level alpha compositing.Forward denotes the complete path, while Backward propagates image-space losses to Gaussian parameters and accumulates updates.
  • Bottlenecks: Pixel-level blending can dominate inference time, but irregular association and blending work creates tile imbalance and weakens SIMT utilization and locality.After rasterization acceleration, sorting and its DRAM traffic can become the next major cost, so the bottleneck depends on scene, platform, and prior optimization.
  • Update Path: Training adds gradient conflicts, optimizer-state memory, and possible host–device transfer costs when model data exceed GPU capacity.Backward propagation can serialize on atomic updates while Gaussian parameters, gradients, and optimizer states grow with the model.
  • Analysis Framework: The taxonomy traces where optimizations occur, which physical workloads change, and whether implementation eliminates, reorganizes, remaps, or reuses work.Comparisons retain task, quality setting, implementation, platform, and timing scope; unmatched settings are used only for mechanism analysis.
  • Workload Counts: The five workload counts follow Nresident → Nactive → Npair → Neval → Nblend from stored Gaussians through selection, screen-space expansion, evaluation, and compositing.The counts measure different work items and therefore need not change monotonically.
  • Workload Counts: Nactive excludes rejected inputs that still incur Projection work, Npair counts Gaussian–tile records, and Nblend counts accepted contributions that update pixel color and transmittance.Neval includes rejected candidate tests, whereas Nblend is a subset of evaluations within a fixed frame and backend.
  • Analysis Framework: Successive workload-count ratios vary with scene, viewpoint, resolution, tile configuration, selection policy, and backend semantics.Reducing one count therefore need not reduce later counts proportionally, and savings matter only when fewer computations, accesses, synchronization events, or updates occur.

3 ALGORITHMIC WORKLOAD OPTIMIZATION

Algorithmic methods improve efficiency by reducing stored or selected Gaussians, screen-space work, or evaluation cost, but their benefits depend on downstream work and selection overhead.

  • Algorithmic changes improve runtime when they remove downstream computation or data movement.
  • Compression reduces storage, but decoding, feature recovery, lookup, or inference can add rendering cost and should be measured separately from file size.
  • View- and Scale-Adaptive Activation: View- and scale-adaptive methods expose fewer Gaussians for projection, with projected footprints determining association and pixel work.
  • View- and Scale-Adaptive Activation: Fewer selected Gaussians save rendering work but add hierarchy traversal, per-frame selection, prefetching, and quality-level transitions.
  • Screen-space methods reduce sorting and traversal inputs, while faster response evaluation lowers unit cost without necessarily reducing Neval.
  • Dynamic Scenes and Online Updates: Dynamic rendering and training methods add motion lookup, decoding, gradient reuse, pruning, or parameter-update work to the frame or update path.

4 GPU RUNTIME OPTIMIZATION

GPU runtime optimization reorganizes filtering, spatial partitioning, scheduling, backward reduction, and data placement to address irregular workloads and memory pressure.

  • GPU runtimes build association lists, schedule pixel and gradient tasks, and place parameters and intermediates, while uneven work limits SIMT utilization.
  • Spatial Partitioning and List Organization: Fine tiles raise Npair, whereas coarse tiles raise Neval; complete GPU time reaches a minimum between the tested extremes, with tile-list imbalance persisting.
  • Tighter Gaussian–tile intersection reduces associations, shrinking sorting input, later traversal, and intermediate list traffic.
  • Dynamic assignment, Gaussian-parallel rendering, parameter hoisting, warp culling, and coherent blending address uneven traversal and improve reuse.
  • Backward optimizations replace some global gradient updates with local reduction to reduce atomic updates and contention.
  • Data Placement at Scale: CPU–GPU tiering, microbatching, prefetching, caching, and multi-GPU partitioning distribute large parameter, optimizer, and rendering datasets.
  • GPU Runtime Design Space: Fewer operations and faster execution are distinct: Local-GS reduces time through organization and reuse despite nearly unchanged association volume.

5 ARCHITECTURAL SUPPORT FOR 3DGS

Architectural support assigns projection, sorting, blending, backward computation, filtering, and data movement to specialized units, buffers, and memory systems.

  • Architectures must accommodate view-dependent coverage, variable tile lists, and depth-ordered compositing across GPU, NPU, and dedicated dataflows.
  • Projection, sorting, and blending designs reduce candidates, reuse ordering, or accelerate ordered candidate processing through specialized pipelines and caches.
  • Hardware rows distinguish work removal from faster execution, data placement, and reuse, with reported values retaining each source paper’s scope.
  • Existing graphics and matrix hardware can support Gaussian evaluation, blending, early termination, tile batching, and balanced work distribution.
  • Training architectures use local merging, rearrangement, or dedicated accumulation to reduce irregular gradient writes and coordinate Gaussian–pixel computation.
  • Filtering, near-data computation, cross-frame caches, and locality-aware grouping reduce later work or repeated data movement, but larger groups require more buffering and may delay stages.
  • Programmable components remain useful because parameter updates and compositing rules can change as representations evolve.

6 WORKLOAD FINDINGS AND IMPLICATIONS

Efficiency gains depend on whether reduced work reaches downstream execution, how granularity redistributes work across stages, and whether transfers, updates, and cached data offset local speedups.

  • Workload Reduction Must Reach the Backend: Active Gaussian count is a weaker predictor of stage time than association and evaluation counts, whose explanatory power still depends on implementation and configuration.Associations and evaluations more directly track sorting and rasterization work than active-Gaussian count alone.
  • Workload Reduction Must Reach the Backend: 42% fewer median associations in gsplat reduced complete forward time by only 4%, while DRAM traffic increased 19% relative to Graphdeco.Median radix-sorting traffic fell from 0.58 to 0.34 GB, but generic indexing kernels transferred 0.47 GB per view.
  • Workload Reduction Must Reach the Backend: Workload reductions save time only when they eliminate downstream operations rather than adding comparable decoding, filtering, scheduling, or data-movement costs.Removal point determines whether association construction, sorting, gradient computation, or parameter updates are actually avoided.
  • Optimal Granularity Is Stage Dependent: Fine tiles increase associations and list-organization cost, whereas coarse tiles increase invalid evaluations and rasterization cost; total time is lowest between the tested extremes.Nblend changes little, so fewer evaluations do not necessarily produce faster execution.
  • Optimal Granularity Is Stage Dependent: Stage-specific grouping can improve efficiency, but added list conversion, records, scheduling, communication, or compositing costs must also be included.HiGS uses macro tiles for partitioning and sorting and finer tiles for rasterization; other work items can have different trade-offs.
  • Local Speedups Shift the Bottleneck: After rasterization is optimized, Adam updates can remain the main training bottleneck, while capacity-limited and cloud-assisted systems must account for transfers.Complete-iteration timing is needed because a rising time share alone does not establish absolute cost or bottleneck status.

7 EVALUATION PRINCIPLES AND OPEN ISSUES

Reliable comparisons must connect quality, workload generation, execution conditions, and sustained behavior. The paper identifies shared interfaces, adaptive execution, and programmable specialization as open directions under these constraints.

  • Evaluation under Quality Constraints: Efficiency comparisons must measure output quality because pruning, compression, transparency changes, and skipped updates can alter reconstruction results.Faster iterations do not establish final reconstruction quality, and matching settings do not guarantee matching quality.
  • Evaluation under Quality Constraints: Comparisons support implementation claims only when model, views, configurations, quality, and timed operations are held within the same evidence scope.The cited comparisons do not support ranking papers evaluated at different quality levels.
  • Sustained and Workload-Aware Evaluation: Sustained evaluation should use representative continuous operation and track tail latency, sustained power, and periods of concentrated reordering, recomputation, or transfers.Single-frame measurements and brief FPS averages capture only part of behavior affected by view history, training views, and cloud-client collaboration.
  • Open Issues: Shared workload interfaces must define Gaussian selection, coverage, and sort results so later stages can reuse results correctly without conversion and update costs outweighing the benefit.Open questions include which information can be shared and which culling or compositing rules remain implementation-specific.
  • Open Issues: Adaptive execution could observe association counts, traversal lengths, or imbalance to adjust grouping and resources while controlling observation, switching, and list-rebuilding costs.Fixed-policy benefits vary with work distribution, motivating stage-specific granularity and dynamic assignment.
  • Open Issues: Programmable specialization must balance specialized efficiency, functional coverage, and evolving representations across complete tasks rather than maximizing individual-kernel speedups.Training, dynamic scenes, and cross-frame reuse may require changed selection rules or recomputation.

8 CONCLUSION

The paper uses a workload-centric framework to compare 3DGS algorithms, GPU runtimes, and architectures across the path from representation to execution. It concludes that end-to-end gains depend on downstream work, granularity, transfers, updates, and quality-constrained evaluation.

  • Conclusion: A common workload-centric framework connects 3DGS algorithms, GPU runtimes, and architectures across representation, execution, and resource use.The framework distinguishes stored model size, executed workload counts, kernel timing, and end-to-end behavior.
  • Conclusion: End-to-end gains depend on where work is removed, how granularity shapes downstream work, and whether computation, transfers, and updates also decline.Cached results and optimizer data are part of the complete system cost.
  • Conclusion: Evaluation should use rendering-quality constraints, traceable inputs, identical timed operations, and sustained measurements to make efficiency comparisons more consistent.These conditions help distinguish supported findings from unresolved questions.
  • Conclusion: The literature review covers public research on efficient 3DGS with a search frozen on August 20, 2026.The retained corpus draws from major publication and project sources spanning graphics, systems, architecture, and machine learning venues.

APPENDIX C REPRODUCIBILITY PROTOCOL

The reproducibility protocol preserves provenance and measurement context while using controlled GPU experiments to validate workload counts before timing. It also limits interpretation to admitted, implementation-specific evidence rather than cross-paper rankings.

  • C.1 Cross-Paper Evidence Admission: Quantitative evidence retains provenance, baseline, scene, aggregation, platform, backend, resolution, timing scope, and quality context.Values are separated into reported, derived, and survey-reproduced categories; missing workload counters remain unavailable.
  • C.1 Cross-Paper Evidence Admission: Fig. 4 and Fig. 6(d) include only studies admitted under criteria requiring identified workloads or resources, comparison conditions, scope, quality settings, and measured operations.The protocol covers the listed representation, runtime, and backend configurations while preserving matched operations and available quality differences.
  • C.2 Controlled GPU Profiling: Controlled RTX 3090 experiments collect Gaussian–tile and per-pixel candidate-list counts, normalize each result to its reference tile size, and check output and workload-count consistency before timing.The experiment uses materialized lists and scene–view counterparts rather than cross-scene normalization.
  • C.2 Controlled GPU Profiling: Fig. 7 relationships are descriptive within one GPU and measurement population, while Fig. 8 holds models and views fixed for backend comparisons.Complete forward latency is measured without profiler attachment; Nsight Compute separately measures kernel composition and DRAM traffic.

C.3 Reproducibility Boundary

The reproducibility package separates evidence provenance from the requirements for recollecting GPU measurements. Offline analysis is lightweight, but GPU recollection depends on documented execution inputs and covers measured calls rather than complete training.

  • C.3 Reproducibility Boundary: The evidence package includes figure inputs, provenance, collection and plotting scripts, and measurement configurations.Records distinguish reported, derived, and survey-measured values.
  • C.3 Reproducibility Boundary: Offline plotting requires no GPU or external survey checkout, whereas GPU recollection requires the documented inputs, builds, and profiler settings.Instrumentation defines Nactive as post-projection pair-producing Gaussians and Npair as materialized list length.
  • C.3 Reproducibility Boundary: GPU recollection covers measured calls rather than complete training procedures.

APPENDIX D SUPPLEMENTARY BACKEND MEASUREMENTS

Supplementary backend measurements pair capacity checks with reconstruction-quality checks under matched-backend comparisons. Memory and quality differences are summarized using defined reference conventions.

  • Backend Memory and Reconstruction Quality: Table 4 checks backend capacity and reconstruction quality for the matched-backend comparison in Fig. 8.Peak allocated memory includes the representation, constructed buffers, and temporary storage.
  • Backend Memory and Reconstruction Quality: PSNR, SSIM, and LPIPS compare each backend with the same ground-truth images, while reported differences use Graphdeco as the reference.Each reported quality difference is the maximum absolute per-view score difference from Graphdeco over measured views.
  • Backend Memory and Reconstruction Quality: Peak memory is summarized over measured views, keeping capacity reporting aligned with the backend comparison scope.
Loading 2609.06157v1…