PaperlayerView exact revision on arXiv

Source-linked AI summary

Ray-by-Ray: Fine-Grained Resource Management for GPU Ray Tracing Units

Junrui Pan, Cesar Avalos Baddouh, Kai Ze Ee, Timothy G. Rogers

arXiv:2610.31630v1cs.AR

TL;DR

Ray tracing divergence creates a mismatch between warp-level RT resource allocation and per-ray execution, wasting on-chip resources and limiting traversal parallelism. Ray-by-Ray decomposes buffers to per-ray granularity and dynamically manages active rays; it achieves a 1.91× average speedup with 0.19% area overhead.

  • Problem

    Warp-level resource allocation wastes on-chip capacity when rays diverge and inactive lanes remain in a warp, despite each ray progressing independently.

    Source: §I · p. 2

  • Method

    Ray-by-Ray uses fine-grained per-ray buffers and lightweight hardware to dynamically map active rays onto RT-unit resources.

    Source: §I · p. 2, §VII · p. 13

  • Results

    1.91× average speedup with 0.19% total area overhead is achieved by Ray-by-Ray.

    Source: §VII · p. 13

  • Takeaways & Limitations

    Ray-by-Ray tracks active lanes to enable more concurrent rays and better hide memory latency, while ray tracing receives limited benefit from contemporary GPU memory coalescing.

    Source: §VII · p. 13

Abstract

from arXiv · show

Ray tracing (RT) generates photorealistic images by simulating the physical behavior of light as it interacts with surfaces in a virtual scene. In film production and high-end rendering pipelines, this technique computes global illumination, reflections, refractions, and soft shadows by tracing millions of light rays per frame and accurately modeling their bounces and attenuation. Contemporary GPUs incorporate specialized ray tracing hardware to accelerate key operations in the traversal pipeline. By accelerating these stages in hardware, the unit enables real-time ray tracing for complex scenes that previously required offline computation. However, ray tracing introduces substantial control and data divergence. Each ray may traverse a different set of nodes in the acceleration structure, encounter distinct materials, or terminate at varying depths. This irregularity reduces thread coherence and exposes a fundamental disparity between the GPU SIMT and per-thread execution model of dedicated RT units, where each ray progresses independently. In this work, we seek to validate assumptions made in open-source infrastructure and identify potential inefficiencies in resource management within the ray tracing pipeline. To address inefficiency caused by mismatches between SIMT and RT granularities, we propose Ray-by-Ray, which decomposes on-chip buffers into the granularity of individual rays and introduces lightweight management structures that enable more warps to run in parallel without increasing buffer capacity. This approach improves ray tracing unit utilization and increases memory-level parallelism. Experimental results show that Ray-by-Ray achieves an average performance improvement of 1.91x over the baseline, with an area overhead of 0.19\%.

I. INTRODUCTION

Ray tracing divergence leaves RT units inefficient despite available occupancy, exposing a mismatch between warp-level resource allocation and per-ray execution. Ray-by-Ray uses finer-grained on-chip buffers and dynamic ray management to increase parallelism, achieving an average speedup of 1.91× with 0.19% area overhead.

I. INTRODUCTION

  • High RT occupancy can coexist with low thread efficiency because rays follow different traversal paths and terminate at different times.Recursive path tracing amplifies this divergence as new rays are launched at hit points.Source: Figure 1 · p. 1, §I · p. 2
  • Prior analyses assumed warp-granular RT resource allocation, while this study finds that inactive lanes waste on-chip resources and the RT pipeline is narrower than the SIMT pipeline.Ray traversal gains little from memory coalescing, making the narrower pipeline amenable to sharing among threads from different warps.Source: §I · p. 2
  • Ray-by-Ray decomposes ray buffer groups into per-ray buffers and dynamically maps active rays onto available RT resources while preserving each ray’s original lane and warp.The traversal instruction bounds the internal divergence phase, allowing hardware management without general-purpose dynamic warp formation.Source: §I · p. 2
  • 1.91× average speedup over the baseline with 0.19% area overhead is achieved by Ray-by-Ray across a wide range of workloads.The design is a lightweight, fine-grained per-ray resource-management mechanism within the RT unit.Source: §I · p. 2
  • Ray tracing’s irregular memory access patterns offer little benefit from traditional GPU memory coalescing, motivating dynamic regrouping of rays during hardware traversal.The paper also presents a microarchitectural characterization of NVIDIA RTX 5090 RT cores, including warp-granular resource allocation, shared-memory activity, and a narrower pipeline than the SIMT pipeline.Source: §I · p. 2

II. BACKGROUND

The background describes ray tracing and BVH traversal, then motivates characterizing RT-core behavior because proprietary implementations leave simulator assumptions unverified. The study profiles NVIDIA GPUs and isolates traversal latency using OptiX.

II. BACKGROUND

  • Ray tracing produces realistic images by tracing rays through a scene, while path tracing repeats bounces to model global illumination and creates divergent ray paths.BVHs organize scene geometry hierarchically so a missed parent bounding box lets traversal skip its children.Source: §II · pp. 2–3

III. RAY TRACING ON GPUS

  • RT-core implementation details are proprietary, so simulator assumptions about traversal, buffering, and scheduling may not match real GPUs.The study characterizes contemporary NVIDIA RT cores to investigate their architectural behavior.Source: §III · p. 3
  • A CUPTI- and GPUVision-based profiler samples hardware events at tens-of-cycles granularity, and OptiX traversal-only calls isolate traversal latency from shading.Experiments used optixTriangle on an RTX 5090 and reproduced the observations on an Ampere RTX 3070.Source: §III · p. 3

A. On-Chip Buffer Management

The buffer-management study finds that RT resources remain allocated per warp even when few lanes are active, limiting traversal concurrency. Shared-memory activity also tracks RT ray-casting, suggesting traversal state uses shared memory alongside RT-core resources.

A. On-Chip Buffer Management

  • The maximum concurrent RT occupancy remains 12 warps in both single-lane and full-warp execution, indicating inactive lanes still reserve warp-granular resources.The microbenchmark masks lanes and measures traversal latency to distinguish per-thread from per-warp allocation.Source: Figure 2 · p. 4, A. On-Chip Buffer Management · p. 4
  • A per-warp resource allocation wastes buffer capacity when only a few threads in a warp are active, restricting traversal concurrency.Each ray operates independently, so only active threads should need buffer space for runtime traversal information.Source: A. On-Chip Buffer Management · pp. 3–4

B. Shared Memory Management

  • Shared-memory accesses strongly correlate over time with RT-core ray-cast operations, and each thread block allocates 64B of shared memory per thread.The allocation could accommodate several traversal-stack entries if represented by 64-bit node addresses; the observations suggest traversal state consumes meaningful on-chip storage.Source: B. Shared Memory Management · pp. 4–5, Figure 3 · p. 4
  • Traversal-state use of shared memory creates trade-offs with other uses of that on-chip storage.The design implications include shared-memory capacity, access frequency, allocation granularity, and bank conflicts.Source: B. Shared Memory Management · p. 5

C. RT Cores Width

With identical, non-divergent rays, latency rises with active thread count, suggesting RT cores serialize warp threads on a narrower datapath. Warp-granular resource management then compounds traversal divergence by retaining resources for inactive or completed rays.

C. RT Cores Width

  • Latency increases with active threads for identical, non-divergent rays, suggesting a temporal SIMT pipeline serializes warp threads on a narrower datapath.The measured latency includes traceRay() setup and RT-core execution; the experiment isolates RT-core latency by removing memory divergence.Source: C. RT Cores Width · p. 5, Figure 4 · p. 5

IV. RAY-BY-RAY

  • Ray-by-Ray introduces thread-level resource allocation to improve traversal utilization under divergence.The design section frames this approach as addressing the baseline’s resource-allocation and warp-management inefficiencies.Source: §IV · p. 5

A. Baseline RT Unit

  • Warp-granular management wastes resources when path-tracing rays leave lanes inactive or when completed rays wait for the longest traversal before warp commit.These patterns reduce parallelism and prevent per-thread buffers and stacks from being released until the entire warp finishes.Source: A. Baseline RT Unit · p. 6

B. Fine-Grained Ray-based Allocation

Ray-by-Ray allocates and releases RT-unit resources per active ray rather than per warp, addressing fragmentation and delayed reclamation. A unified buffer pool and per-warp completion tracking support this finer-grained management while retaining warp-level admission.

B. Fine-Grained Ray-based Allocation

  • Per-ray allocation uses resources only for active threads and releases each ray’s resources when traversal completes, increasing the rays in flight under divergence.A pending warp can issue when sufficient resources become available.Source: B. Fine-Grained Ray-based Allocation · p. 6
  • With 128 ray buffers, the flattened pool avoids warp-group fragmentation and can admit a waiting warp as individual ray slots become free.The baseline groups buffers by warp and releases them only after collective warp completion.Source: B. Fine-Grained Ray-based Allocation · p. 7, Figure 7 · p. 7

1) Buffer Management:

  • The unified buffer uses a one-bit-per-entry Ray Buffer Mask, and a warp is admitted only when a warp-table entry and enough slots for all active threads are available.Allocation is serialized at one thread per cycle, or 32 cycles for a fully active warp; this is described as negligible against traversal lasting thousands to tens of thousands of cycles.Source: 1) Buffer Management · p. 7
  • A per-warp completion counter and each entry’s Warp ID enable immediate per-ray reclamation while tracking when the originating warp can commit.Stacks are attached to ray-buffer entries so their accesses follow a fixed pattern, avoiding potential bank conflicts; buffer-order groups of 32 rays preserve baseline coalescing width.Source: 1) Buffer Management · pp. 7–8

2) Memory Coalescing:

Ray-by-Ray can reorder rays because BVH traversal diverges quickly and concurrent misses to the same cache line can be merged. The evaluation reports a 1.91× geometric-mean speedup and uses a GPU-resolution pairing chosen to represent per-SM warp density within simulation limits.

2) Memory Coalescing:

  • Ray reordering may increase issued load instructions, but MSHRs merge concurrent requests to the same cache line, so this does not increase L2 traffic.BVH pointer-chasing permits only one outstanding miss per ray, and the authors state that GPU MSHR capacity accommodates the additional requests.Source: 2) Memory Coalescing · p. 8

V. METHODOLOGY

  • The evaluation uses 16 LumiBench scenes across three rendering modes and pairs a validated GPU configuration with 256 × 256 resolution to approximate representative per-SM warp density.A full data-collection cycle takes one week, preventing further increases in workload scale.Source: §V · pp. 8–9
  • 1.91× geometric-mean speedup across all workloads is reported for Ray-by-Ray.Fig. 8 distinguishes Shadow Rays, Ambient Occlusion, and Path Tracing workloads.Source: Figure 8 · p. 9

A. Performance

Ray-by-Ray improves traversal performance by freeing resources as individual rays finish, keeping more rays in flight. This raises L1 traffic, while L2 traffic remains nearly unchanged because MSHRs absorb redundant requests.

A. Performance

  • 1.42×, 1.70×, and 2.78× speedups over the baseline for Shadow Rays, Ambient Occlusion, and Path Tracing, respectively.Path Tracing achieves the highest speedup because rays spawn new rays and continue traversing rather than terminating immediately after a hit.Source: A. Performance · p. 9
  • Ray-by-Ray raises active rays in the ROBOT path-tracing scene to nearly the theoretical maximum of 128, versus only a few in the baseline.Allocating resources per ray lets more warps traverse in parallel instead of reserving resources for inactive lanes.Source: A. Performance · p. 9
  • WKND and SHIP show the least speedup; their small scenes fit largely in L1 cache, making the cited workloads compute-bound rather than memory-bound.WKND SH has a 5.6% L1 miss rate, and SHIP AO has a 5.7% L1 miss rate.Source: A. Performance · p. 9

B. Memory

  • 14% more L1 accesses on average but only 1.6% more L2 sector accesses on average; MSHRs coalesce redundant same-sector requests before they reach L2.The L1 and L2 sector-load comparison is normalized to baseline; Fig. 10 shows workloads with changes exceeding 10% in L1 accesses, with geomean across all workloads.Source: B. Memory · p. 10, Figure 10 · p. 10
  • 0.09 baseline versus 0.10 proposed-design L1 MSHR merge fails per kilo warp instruction on average, with no MSHR allocation failures in either design.The proposed design therefore does not exhaust MSHR capacity in the evaluated workloads.Source: B. Memory · p. 10, Figure 11 · p. 10
  • 1.94× the baseline L2 bandwidth is sustained by Ray-by-Ray as it keeps more rays traversing in parallel.The added bandwidth can cap speedup where L2 bandwidth is the binding constraint.Source: B. Memory · p. 10, Figure 12 · p. 10

C. Energy

Ray-by-Ray averages 0.98× the baseline energy consumption, with the largest reductions in path-tracing workloads and a few shadow workloads increasing energy. Its reported area overhead is 0.19%, and it achieves a higher geometric-mean speedup than CoopRT across evaluated workloads.

C. Energy

  • 0.98× the baseline average energy consumption is reported for Ray-by-Ray.The largest reductions occur in path-tracing workloads, while a few shadow workloads show increased energy consumption.Source: C. Energy · p. 10, Figure 13 · p. 11
  • A few shadow workloads increase energy; PARK SH has the largest increase in off-chip accesses and a 1.78× speedup.The passage identifies the added off-chip accesses as the primary cause of PARK SH’s higher energy usage.Source: C. Energy · p. 11

D. Area Overhead

  • 0.19% total area overhead is reported for the fine-grained resource-management design.The synthesized baseline RT core occupies 677,066.8 µm2, compared with 678,368.4 µm2 for the improved RT core; direct RT-unit power averages 9.17 mW versus 9.23 mW, a 0.7% overhead.Source: D. Area Overhead · p. 11

E. Comparison To Related Work

  • Ray-by-Ray outperforms CoopRT on most workloads and achieves a higher geometric-mean speedup across all workloads.The CoopRT results come from the authors’ artifact evaluation on an RTX 2060, a different experimental configuration.Source: E. Comparison To Related Work · pp. 11–12

VI. RELATED WORK

The related-work review distinguishes Ray-by-Ray from software reordering and prior traversal techniques by its focus on reclaiming RT-unit resources during traversal. The conclusion reports its resource-management approach and measured performance and area overhead.

VI. RELATED WORK

  • Software ray reordering, including SER, spills and reloads live state and cannot reclaim warp-managed resources during a monolithic traversal instruction.These approaches also cannot mitigate inter-ray divergence while rays are traversing the scene.Source: §VI · p. 12
  • Dynamic Warp Formation and Thread Block Compaction regroup threads at divergent branches, while persistent-thread ray replacement does not apply to monolithic RT traversal.Ray traversal has defined divergence boundaries, and its ray state resides in RT-unit structures rather than the vector register file.Source: §VI · p. 12
  • Prior ray-tracing architectures target traversal mechanisms or mobile devices, whereas CoopRT uses inactive lanes to assist BVH traversal and may perform unnecessary work.Source: §VI · p. 12
  • Other accelerator techniques include intersection caching and prediction, shared-memory stack-overflow storage, BVH-treelet prefetching, and ray sorting.The authors describe many of these microarchitectural techniques as orthogonal to Ray-by-Ray and potentially integrable alongside it.Source: §VI · p. 13

VII. CONCLUSION

  • Ray-by-Ray tracks active lanes for fine-grained per-ray allocation, allowing more rays to traverse concurrently and better hiding memory latency.The conclusion also reports that contemporary GPU memory coalescing provides limited benefit for ray tracing, allowing relaxed warp-level coalescing constraints.Source: §VII · p. 13
  • 1.91× average speedup over the baseline comes with a total area overhead of 0.19%.The RTL design was evaluated across diverse workloads and ray tracing techniques using Vulkan-Sim and LumiBench.Source: §VII · p. 13

APPENDIX ARTIFACT APPENDIX

The artifact packages the Ray-by-Ray speedup reproduction workflow in Docker images, with baseline results included and design results generated by the user. It documents the environment, workload set, and estimated resource and runtime requirements.

A. Abstract

  • The artifact reproduces Fig. 8’s per-workload normalized speedup over the baseline across 46 ray-tracing workloads.The set comprises 16 LumiBench scenes in SH, AO, and PT modes, plus CHSNT_PT.Source: A. Abstract · p. 15

B. Artifact check-list (meta-information) •

  • The Docker images include the simulator, Vulkan driver, benchmark app, scenes, and toolchain, with LumiBench’s 16 scenes included.Source: B. Artifact check-list… · p. 15
  • Reproduction can run on any x86-64 host with at least 128 GB RAM; no GPU is required, while additional RAM and CPU cores directly reduce wall time.Source: B. Artifact check-list… · p. 15
  • The workflow reports per-workload speedup as baseline/design and its geometric mean, using gpu_tot_sim_cycle.The output includes Fig. 8 and per-run Vulkan-Sim logs.Source: B. Artifact check-list… · p. 15
  • Preparation takes under 10 minutes; experiments are estimated at one week serially or about one day on a well-provisioned parallel host.The package requires approximately 15 GB of disk space.Source: B. Artifact check-list… · p. 15
  • The artifact is publicly available on Zenodo under DOI 10.5281/zenodo.21542536, and its code license is BSD.Source: B. Artifact check-list… · p. 15

C. Description

The artifact runs on a Docker-equipped x86-64 Linux host and bundles its software dependencies and LumiBench scenes. Installation begins by loading the supplied tarball, which registers design and baseline containers.

2) Hardware dependencies:

  • The host requirement is x86-64 Linux with Docker and at least 128 GB RAM; no GPU is needed.CHSNT_PT has a peak resident set of about 120 GB and is automatically skipped when the host lacks sufficient headroom.Source: 2) Hardware dependencies · p. 15

3) Software dependencies:

  • Docker is the only software dependency because the images contain the simulator, driver, toolchain, plotting software, and benchmark dependencies.Source: 3) Software dependencies · p. 16

4) Data sets:

  • LumiBench’s 16 scenes are shipped inside the images, with rendering at 256×256 resolution to match the paper.Source: 4) Data sets · p. 16

D. Installation

  • Loading vulkan-sim-artifact.tar registers rbr-design and rbr-baseline, with precomputed baseline results already in ./results/baseline/.Source: D. Installation · p. 16

E. Experiment workflow

The experiment workflow runs the design sweep and renders Fig. 8 against shipped baseline results. Its scheduler supports configurable workload concurrency and memory limits, alongside alternative benchmark entry points.

E. Experiment workflow

  • The design sweep uses all cores and available RAM by default, while JOBS and --max-mem can cap concurrency and memory.Workloads whose reservation exceeds the auto-clamped memory budget are skipped rather than launched.Source: E. Experiment workflow · p. 16
  • One design-sweep run generates Fig. 8 by running design workloads and plotting them against the shipped baseline results.The workflow launches all 46 workloads in parallel, subject to its scheduling settings.Source: E. Experiment workflow · p. 16
  • The artifact also provides a single-workload runner and a standalone in-container sweep driver.Source: E. Experiment workflow · p. 16
  • A host-side alternative launches one container per benchmark instead of one per variant.Source: E. Experiment workflow · p. 16

F. Evaluation and expected results

The evaluation script is expected to reproduce Fig. 8’s speedup bars and overall geometric mean, with small per-bar numerical variation. The notes distinguish baseline-image changes from a benign teardown abort after successful simulation completion.

F. Evaluation and expected results

  • Fig. 8’s per-application, per-shader-mode speedup bars should match the paper’s overall geometric mean and workload ranking, despite small numerical variation in individual bars.The figure covers SH, AO, and PT modes.Source: F. Evaluation and expected results · p. 16

G. Experiment customization

  • The WIDTH and HEIGHT settings can be lowered for a faster, lower-fidelity sweep; removing --only design reruns the baseline sweep from scratch.Source: G. Experiment customization · p. 16

H. Notes

  • The rbr-baseline image is essentially original Vulkan-Sim 2.0 (commit dc742fa), with added performance counters and software optimizations that reduce simulation wall time.Source: H. Notes · p. 16
  • The Mesa/lavapipe teardown assertion can occur after GPGPU-Sim has printed its exit marker and flushed statistics, so it does not mean the simulation failed.Check for the exit marker and a gpu_tot_sim_cycle value in the per-workload log.Source: H. Notes · p. 16
Loading 2610.31630v1…