PaperlayerView exact revision on arXiv

Source-linked AI summary

A Hybrid and Hierarchical Ray Tracing Method for Efficient 3D Gaussian Rendering

Tiancheng Zhao, Sixu Li, Yuzhou Chen, Hengyu Ding, Zhiming Pan, Chaojian Li

arXiv:2610.13498v1cs.ARcs.GR

TL;DR

Rasterization-based 3DGS struggles with per-ray optical effects, while ray-traced 3DGS faces spatial imbalance, redundant neighboring-ray traversal, and GPU-mismatched mixed-primitive workloads. H2-Ray combines density-aware spatial partitioning, traversal reuse across neighboring rays, and a reconfigurable datapath for Gaussian and triangle intersections. H2-Ray reports a 3.77× geometric-mean speedup over RaySplats; arithmetic-mean runtime decreases from 514.59 to 131.65 ms.

  • Problem

    Rasterization-based 3DGS struggles with per-ray optical effects, while ray-traced 3DGS faces spatial imbalance, redundant neighboring-ray traversal, and GPU-mismatched mixed-primitive workloads.

    Source: §I · p. 1

  • Method

    H2-Ray combines density-aware spatial partitioning, traversal reuse across neighboring rays, and a reconfigurable datapath for Gaussian and triangle intersections.

    Source: §I · p. 2

  • Results

    H2-Ray reports a 3.77× geometric-mean speedup over RaySplats; arithmetic-mean runtime decreases from 514.59 to 131.65 ms.

    Source: B. End-to-End Results · p. 8

  • Takeaways & Limitations

    H2-Ray addresses the three workload challenges while maintaining small average PSNR differences across matched views.

    Source: §VII · p. 11

    GRTX had no public implementation at the time of writing, so the authors could not compare against it directly.

    Source: §VII · p. 10

Abstract

from arXiv · show

3D Gaussian Splatting (3DGS) is a powerful representation for real-time neural reconstruction and rendering. While existing 3DGS systems rely on rasterization for efficiency, recent work explores ray tracing to enable secondary lighting, highly distorted cameras, and depth-of-field effects. However, compared with rasterization, ray-traced 3DGS introduces three major challenges: (1) the highly non-uniform spatial distribution of massive numbers of Gaussian primitives creates severe load imbalance; (2) neighboring pixels often generate rays with highly similar traversal paths, leading to substantial redundant traversal overhead; and (3) supporting both Gaussian and triangle primitives yields heterogeneous intersection workloads that are not well matched to existing GPU architectures. We present H2-Ray, a hybrid and hierarchical ray tracing architecture for efficient 3D Gaussian rendering. H2-Ray introduces three techniques. Load-Balanced Spatial BVH (LB-SBVH) partitions the scene into uniform grids and uses density-aware representations to convert an imbalanced global workload into balanced local traversal units. Ray-Cluster Path Reuse (RCPR) lets a proxy ray share traversal state with neighboring rays to amortize redundant traversal. The Unified Intersection Engine (UIE) supports mixed Gaussian and triangle intersections through a mode-multiplexed datapath that shares 40% of its floating-point units across both primitive types. Across 8 scenes, H2-Ray achieves a geometric-mean speedup of 3.77x and a maximum speedup of 9.26x over RaySplats on the NVIDIA RTX A5000, combining a cycle-accurate model of the modified RT Core with measured CUDA timings.

I. INTRODUCTION

Rasterization-based 3DGS struggles with optical effects that need per-ray visibility, while ray-traced 3DGS faces spatial imbalance, redundant traversal, and heterogeneous intersection workloads. H2-Ray addresses these challenges with a hybrid, hierarchical architecture and reports a 3.77× geometric-mean speedup over RaySplats.

I. INTRODUCTION

  • Rasterization-based 3DGS struggles with secondary lighting, highly distorted cameras, and depth-of-field effects that benefit from per-ray visibility reasoning.Ray tracing broadens the rendering envelope, but making that flexibility practical requires support beyond triangle-centric ray tracing hardware and rasterization-based 3DGS pipelines.Source: §I · p. 1
  • Ray-traced 3DGS faces spatial imbalance, redundant traversal of neighboring-ray path prefixes, and heterogeneous Gaussian–triangle intersections mismatched to GPU support.Secondary rays amplify spatial non-uniformity; high-resolution sampling and deep BVHs strengthen shared traversal prefixes, while Gaussian intersections typically use general-purpose compute units.Source: §I · pp. 1–2
  • Earlier ray-traced 3DGS work explored rendering formulations or single-ray traversal, leaving these three workload challenges largely unexplored.The cited efforts include 3DGRT [13], 3DGUT [23], RaySplats [3], and GRTX [9].Source: §I · p. 2
  • H2-Ray combines rasterization and ray tracing for region-of-interest culling and addresses workloads at scene, ray, and intersection levels.It uses a co-design approach that adds targeted support at key points in the GPU rendering pipeline.Source: §I · p. 2
  • LB-SBVH uses density-aware spatial partitioning to convert globally skewed Gaussian distributions into balanced traversal regions.It achieves a microarchitectural traversal speedup of up to 1.31× over a monolithic BVH.Source: §I · p. 2
  • RCPR shares traversal across neighboring rays to reduce redundant BVH operations.In matched multi-view evaluation, H2-Ray decreases average PSNR by 0.034 dB relative to RaySplats.Source: §I · p. 2
  • UIE supports Gaussian and triangle primitives in a single reconfigurable datapath.The design shares floating-point units across the two primitive types.Source: §I · p. 2
  • 3.77× geometric-mean speedup, up to 9.26×, over RaySplats on the NVIDIA RTX A5000.Evaluation combines cycle-level modeling of the modified RT Core with measured CUDA timings.Source: §I · p. 2

A. 3D Gaussian Splatting

3DGS represents scenes with anisotropic Gaussians and efficiently renders them by projecting and compositing them in screen space. Ray-based formulations support more general visibility queries, with RaySplats directly intersecting Gaussian confidence ellipsoids.

A. 3D Gaussian Splatting

  • Each 3DGS primitive is an anisotropic Gaussian storing a mean position, covariance, opacity, and view-dependent appearance.The mean and covariance describe spatial support, while opacity controls contribution strength.Source: A. 3D Gaussian Splatting · p. 2
  • Rasterization projects Gaussians onto image-plane tiles and blends their contributions front-to-back using alpha compositing.The pixel equation uses projected opacity, color, and accumulated transmittance.Source: A. 3D Gaussian Splatting · p. 2
  • Rasterization-based 3DGS is less natural for arbitrary-ray queries, secondary visibility, and integration with conventional ray-traced geometry.Its efficient standard pipeline is described for pinhole-camera novel-view synthesis.Source: A. 3D Gaussian Splatting · pp. 2–3

B. Ray Tracing for 3DGS Rendering

  • 3DGRT uses bounding-mesh hits for Gaussian traversal, while 3DGUT retains splatted primary rendering and traces secondary rays.3DGRT integrates along proxy intersections; 3DGUT uses a sigma-point approximation for primary rays.Source: B. Ray Tracing for 3DGS Rendering · p. 3
  • RaySplats directly intersects rays with Gaussian confidence ellipsoids formed by thresholding the same Gaussian primitive, then accumulates contributions in ray order.Substituting the ray into the threshold surface yields a quadratic, whose smaller positive root gives the closest valid hit.Source: B. Ray Tracing for 3DGS Rendering · p. 3

C. Why Hybrid Rendering Matters

Ray-traced Gaussian rendering supports effects and mixed scene content that are awkward for rasterization-centered pipelines. Hybrid scenes therefore require handling rays that encounter Gaussians, triangles, or both.

C. Why Hybrid Rendering Matters

  • Ray-traced 3DGS supports reflections, refractions, shadows, stochastic sampling, distorted cameras, and rolling-shutter effects, and can combine Gaussian content with conventional geometry.Prior examples include mesh interactions, splatted primary rays with traced secondary rays, and mesh-based objects with lighting effects.Source: C. Why Hybrid Rendering Matters · p. 3

III. MOTIVATION

  • Hybrid rendering creates mixed-intersection workloads in which different rays may encounter Gaussians, triangles, or both.The paper treats unified ray–Gaussian and ray–triangle execution as an architectural requirement and characterizes these workloads alongside spatial imbalance and redundant traversal.Source: C. Why Hybrid Rendering Matters · p. 3, §III · p. 3

A. Scene-Level Challenge: Spatial Imbalance in Gaussian Distributions

Gaussian distributions vary sharply across scenes and within scenes, making a monolithic BVH poorly matched to traversal and scheduling. Neighboring rays also repeat shared BVH traversal, motivating spatial balancing and ray-cluster reuse.

A. Scene-Level Challenge: Spatial Imbalance in Gaussian Distributions

  • The coefficient of variation ranges from 2.7 in Playroom to 11.2 in Kitchen, illustrating scene-to-scene variation in Gaussian spatial skew.Kitchen and Stump are identified as high-CV scenes, while drJohnson and Playroom are lower-CV scenes.Source: A. Scene-Level Challenge: Spatial… · p. 3, Figure 3 · p. 4
  • A monolithic BVH must serve dense regions with long traversal and intersection queues alongside sparse regions that underutilize ray-tracing hardware.Dense regions dominate queues and cache traffic while sparse regions leave traversal capacity idle, reducing parallel efficiency and complicating load balance.Source: A. Scene-Level Challenge: Spatial… · p. 4
  • Across eight scenes, 69.8% of grids are empty, 20.9% are sparse, and 9.3% are dense; non-empty grids have an average CV of 6.7.The occupancy measurement uses a uniform 8^3 partition.Source: A. Scene-Level Challenge: Spatial… · p. 4
  • LB-SBVH restructures severe spatial imbalance into more uniform traversal units.The spatial-skew challenge persists alongside a second inefficiency: neighboring rays often share BVH prefixes before diverging near boundaries.Source: A. Scene-Level Challenge: Spatial… · p. 4, B. Ray-Level Challenge: Redundant… · p. 4

B. Ray-Level Challenge: Redundant Traversal Across Neighboring Rays

  • Repeated upper-level node tests, address generation, and state updates waste memory bandwidth and RT Core cycles in coherent workloads.RCPR amortizes traversal over a ray cluster by reusing a proxy ray’s shared traversal state.Source: B. Ray-Level Challenge: Redundant… · p. 4

C. Intersection-Level Challenge: Heterogeneous Mixed-Primitive Intersection Workloads

Mixed Gaussian–triangle workloads challenge fixed-function intersection hardware because the dominant primitive varies by scene, camera, and task. H2-Ray combines scene partitioning, ray-path reuse, and an adaptive intersection backend in a hybrid, hierarchical pipeline.

C. Intersection-Level Challenge: Heterogeneous Mixed-Primitive Intersection Workloads

  • A dominant primitive that varies by scene, camera, and task makes rigid single-primitive intersection pipelines inefficient for hybrid workloads.Triangle-centric hardware loses efficiency when Gaussian tests dominate, while Gaussian-only hardware cannot serve triangle work.Source: C. Intersection-Level Challenge:… · p. 4

IV. H2-RAY ACCELERATION FRAMEWORK

  • H2-Ray combines LB-SBVH scene partitioning, RCPR traversal reuse, and UIE mixed-primitive intersection in a hybrid, hierarchical framework.UIE shares floating-point units between Gaussian and triangle paths and selects the active mode per test.Source: §IV · p. 4, C. Intersection-Level Challenge:… · p. 4

A. The End-to-End Rendering Pipeline

  • A low-resolution raster pass builds a grid hit map for culling, then LB-SBVH dispatches rays to active grids using local BVHs for dense regions and flat lists for sparse ones.The hit map prunes empty regions and rays with negligible Gaussian energy.Source: A. The End-to-End Rendering Pipeline · p. 5
  • RCPR uses a proxy ray’s traversal state to help neighboring rays bypass redundant work, while UIE selects Gaussian or triangle intersection at leaf nodes before front-to-back accumulation.When a worker ray’s grid diverges, it falls back to root traversal.Source: A. The End-to-End Rendering Pipeline · p. 5, Figure 5 · p. 5

B. Load-Balanced Spatial BVH (LB-SBVH)

H2-Ray addresses imbalance, redundant traversal, and mixed-primitive intersection with local spatial representations, shared ray prefixes, and a reconfigurable intersection datapath.

B. Load-Balanced Spatial BVH (LB-SBVH)

  • LB-SBVH converts an imbalanced Gaussian distribution into locally balanced traversal regions using density-aware spatial partitioning.Dense grids receive local BVHs, sparse grids use flat lists, and conservative bounds preserve coverage without duplicating Gaussians.Source: B. Load-Balanced Spatial BVH (LB-SBVH) · p. 5

C. Ray-Cluster Path Reuse (RCPR)

  • RCPR shares intermediate BVH traversal state among a 3 × 3 ray tile, with the center proxy ray traversing fully and workers resuming at shared intermediate nodes.SPM stores proxy-ray prefixes, while pinned caching and direct color duplication provide additional hardware support.Source: C. Ray-Cluster Path Reuse (RCPR) · pp. 5–6

D. Unified Intersection Engine (UIE)

  • UIE uses a reconfigurable datapath for Gaussian and triangle intersections, sharing 31 of 78 floating-point units (40%) across both modes.The primitive-type field selects the active datapath per intersection without runtime workload prediction or software branching.Source: D. Unified Intersection Engine (UIE) · p. 6

E. Hardware Integration

H2-Ray integrates targeted support into the GPU pipeline while leaving shader execution, memory hierarchy, and rasterization largely intact. Its proxy and real ray phases pass traversal state to worker rays before intersection and blending.

E. Hardware Integration

  • The GPC-level design places an H2-Ray RT Core and Raster Engine beside an SM, with R2RFP forwarding BVH pointers and a pinned L2 cache supporting RCPR.The additions leave shader execution, memory hierarchy, and rasterization largely intact.Source: E. Hardware Integration · p. 7
  • The RT Core adds a Raster-to-RT interface, SPM, a reconfigurable intersection unit, and direct color duplication around the existing traversal and intersection flow.R2RFP supplies the starting BVH handle, SPM changes worker-ray traversal resumption, UIE changes the leaf test, and color duplication handles repeated output writes.Source: E. Hardware Integration · p. 7
  • In the proxy phase, culling forwards valid grid handles and the RT Core records prefix pointers; in the real-ray phase, workers load those pointers and skip redundant upper-level nodes.The two phases preserve front-to-back blending semantics.Source: E. Hardware Integration · p. 7

V. EVALUATION

The evaluation combines modeled modified-RT-Core behavior with measured CUDA work, and reports runtime and quality comparisons against RaySplats across eight scenes. Its end-to-end speedup is a combined-system result, not an isolated attribution to LB-SBVH or RCPR.

A. Evaluation Settings

  • The evaluation covers six Mip-NeRF 360 scenes and two Deep Blending scenes, using matched rendered cameras rather than a held-out test split.Matched views may overlap with views used to train the RaySplats scene representation.Source: A. Evaluation Settings · p. 7, Table III · p. 7
  • Modified RT Core traversal and intersection are cycle-accurately modeled, while CUDA work is measured on the RTX A5000.CUDA measurements include intersection, sorting, and auxiliary kernels; the core is synthesized and placed and routed at 28 nm, 0.9 V, and 1 GHz.Source: A. Evaluation Settings · p. 7

V. EVALUATION

  • 3.77× geometric-mean end-to-end speedup over RaySplats is a combined-system result, not an isolated attribution to LB-SBVH or RCPR.Figure 8 decomposes latency into modeled RT Core and measured CUDA components and overlays per-scene speedups.Source: Figure 8 · p. 8, A. Evaluation Settings · p. 8

A. Evaluation Settings

  • PSNR and SSIM compare each rendered image with ground truth, while runtime is per-frame latency for representative views and excludes training, preprocessing, loading, and image I/O.The per-view comparison table reports PSNR and SSIM across eight scenes, with deltas defined as H2-Ray minus RaySplats.Source: A. Evaluation Settings · pp. 7–8, Table IV · p. 8

B. End-to-End Results

H2-Ray is evaluated for runtime and image quality against RaySplats, with latency combining a modified RT Core model and measured CUDA execution. The results report a 3.77× geometric-mean speedup and a small average PSNR difference, while diagnosing view-level outliers.

B. End-to-End Results

  • 3.77× geometric-mean speedup over RaySplats; the arithmetic-mean runtime falls from 514.59 to 131.65 ms across the evaluation.Per-scene speedups range from 1.89× on Stump to 9.26× on Garden; the average runtime and speedup use different means.Source: B. End-to-End Results · p. 8
  • Latency combines modeled modified-RT-Core time with measured CUDA work; worker-ray intersection and sorting run as CUDA kernels, so these timings omit any benefit from the proposed Reconfigurable Intersection Unit.Intersection dominates CUDA latency in six scenes, while sorting dominates in Stump and Playroom.Source: B. End-to-End Results · p. 8
  • The arithmetic mean of per-scene average PSNR deltas is -0.034 dB for H2-Ray minus RaySplats on matched views.Figure 10 plots each scene’s ten lowest per-view differences; its outlined marker marks the worst view, and values closer to zero indicate better agreement with RaySplats.Source: B. End-to-End Results · p. 8, Figure 10 · p. 9
  • For the two largest drJohnson outliers, disabling proxy reuse or increasing hop distance did not consistently close the gaps, which may instead reflect implementation-level differences.RaySplats does not expose its exact OptiX BVH construction and traversal details.Source: B. End-to-End Results · p. 9, Table VI · p. 9

C. Sensitivity Analysis

The sensitivity analyses examine hop distance, grid granularity, and local-grid traversal. Two hops provide the reported accuracy–cost balance on representative views, while 27 local grid BVHs improve modeled traversal latency over one global BVH.

C. Sensitivity Analysis

  • At 2 hops, the maximum per-scene PSNR loss is 0.13 dB versus the C++ full per-ray traversal baseline on representative views; the across-scene mean losses are 2.99, 1.32, and 0.06 dB at 0, 1, and 2 hops.These representative-view statistics do not bound all camera views.Source: C. Sensitivity Analysis · p. 9, Table VII · p. 9
  • 0-hop reuse loses 5.55 dB in Garden and 10.19 dB in Playroom when tiles cross foreground/background boundaries; at 1 hop, Playroom still loses 6.94 dB.At 0 hops, neighboring rays reuse only the center ray’s leaf-level hits; 1 hop resumes traversal from a higher shared state.Source: C. Sensitivity Analysis · p. 9
  • Coarser-to-moderate grid partitions separate empty regions from dense clusters, while finer settings increase occupied and sparse grid fractions; the 33-grid default isolates hotspots while retaining enough Gaussians for local BVHs.Changing grid granularity does not change correctness because all candidate Gaussians remain reachable.Source: C. Sensitivity Analysis · p. 9
  • 27 local grid BVHs reduce modeled traversal latency by 1.08×–1.31× across eight scenes and DRAM reads by 5.9%–31.3%.The geometric mean of per-scene speedups is 1.18×; this table isolates modeled RT Core traversal rather than end-to-end runtime.Source: C. Sensitivity Analysis · p. 10

D. Ray Tracing with Heterogeneous Primitives

H2-Ray supports heterogeneous Gaussian–triangle workloads, and the paper positions its architecture against prior accelerators that target different parts of the rendering pipeline. Its conclusion reports a 3.77× geometric-mean speedup over RaySplats across eight scenes.

D. Ray Tracing with Heterogeneous Primitives

  • Gaussian tests dominate all evaluated mixed workloads: perspective-plus-mesh performs 1.90 billion ray–Gaussian tests versus 1.38 million ray–triangle tests, and fisheye-plus-mesh performs 1.75 billion versus 0.73 million.Fisheye changes ray directions and coverage; adding meshes leaves the perspective Gaussian workload nearly unchanged, from 1.87 to 1.90 billion tests.Source: D. Ray Tracing with Heterogeneous… · p. 10

VI. RELATED WORK

  • Prior accelerators target neural-field evaluation, splatting-style projection, or BVH traversal and triangle tests; RCPR instead reuses a nearby ray’s traversal path, orthogonal to GRTX’s single-ray checkpointing.H2-Ray also adds LB-SBVH partitioning and UIE for Gaussian–triangle datapaths.Source: §VI · p. 10

VII. CONCLUSION

  • Across eight scenes, H2-Ray achieves a 3.77× geometric-mean speedup over RaySplats while maintaining small average PSNR differences on matched views.The paper says GRTX had no public implementation at the time of writing, preventing direct comparison.Source: §VII · pp. 10–11
Loading 2610.13498v1…