PaperlayerView exact revision on arXiv

Source-linked AI summary

Characterizing Warp Divergence from Pascal to Blackwell

Alpin Dale

arXiv:2607.23402v1cs.ARcs.PF

TL;DR

Whether warp-divergence behavior has changed across NVIDIA GPU generations remains unclear, especially for Blackwell. This paper combines dynamic microbenchmarks, hardware counters, and static SASS analysis across Ampere, Hopper, Blackwell, and Pascal, finding stable linear divergence costs alongside evolving reconvergence machinery.

  • Problem

    Prior work provides limited cross-generational evidence on ITS divergence behavior and does not measure Blackwell control-flow divergence.

  • Method

    The paper combines divergence microbenchmarks, hardware counters, and compiler-emitted SASS analysis across post-ITS GPUs and a pre-ITS Pascal baseline.

  • Results

    Divergence serializes linearly as T(k) ≈ s·k across generations, while compiler and ISA reconvergence machinery changes substantially from Pascal through Blackwell.

  • Takeaways & Limitations

    Programmer-facing divergence cost models transfer across Ampere–Blackwell, while architects and tool builders must account for evolving static reconvergence mechanisms.

  • Takeaways & Limitations

    Bit-flip experiments show no runtime effect for the .RELIABLE field only in the tested executions.

Abstract

from arXiv · show

Since Volta introduced Independent Thread Scheduling (ITS), NVIDIA GPUs have been widely assumed to handle warp divergence in a fixed manner. We test this assumption across Ampere, Hopper, and datacenter and consumer Blackwell GPUs, using pre-ITS Pascal as a baseline. Combining cycle-accurate microbenchmarks, hardware counters, and static analysis of compiler-generated SASS, we separate stable behavior from architectural change. Across all tested generations, divergent paths serialize linearly with the number of paths $k$, following $T(k) \approx sk$ with no super-linear reconvergence penalty. Warp execution efficiency falls as $32/k$, the penalty is independent of occupancy, and predication removes the serialization cost. The same behavior appears on Pascal, showing that this programmer-visible cost model predates ITS. The compiler-emitted reconvergence machinery, however, has changed substantially. Pascal uses a per-warp SSY/SYNC instruction stack, whereas later generations use barrier-register instructions. Deferred reconvergence beyond the immediate post-dominator falls from 29 cases on Ampere to 2 on Blackwell. Blackwell also introduces a two-tier convergence-barrier classification, uniform-branch instructions, and explicit partial-mask warp synchronization, none of which appear on Ampere or Hopper. Controlled bit-flip experiments indicate that the new barrier class is a static compiler classification with no observable runtime effect in our tests. Thus, divergence retains a stable and predictable performance cost even as NVIDIA's control-flow ISA and reconvergence mechanisms continue to evolve.

I. INTRODUCTION

Across Pascal, Ampere, Hopper, and Blackwell, programmer-visible warp-divergence behavior remains invariant: paths serialize linearly, while compiler-emitted control-flow and reconvergence machinery changes substantially. The study combines microbenchmarks, hardware counters, and static SASS analysis to establish this separation across generations.

  • Introduction: The study addresses a gap in prior work by testing whether ITS divergence behavior changed across four post-ITS GPUs, two Blackwell variants, and pre-ITS Pascal.Prior microarchitecture studies largely omitted control-flow divergence, and detailed SASS reverse engineering covered only Turing.
  • Introduction: T(k) ≈ s·k with no super-linear reconvergence penalty, and the same linear serialization law governs pre-ITS Pascal.The result holds up to a full 32-way split across Ampere, Hopper, and two Blackwell variants.
  • Introduction: 32/k is the generation-invariant warp execution-efficiency result, while divergence cost is occupancy-invariant because resident warps cannot hide instruction-issue work.Hardware counters independently confirm the efficiency relationship.
  • Introduction: Pascal uses the literal SSY/SYNC instruction stack, whereas later generations use barrier-register control-flow machinery.The comparison concerns compiler-emitted reconvergence placement and the mechanism that replaced the pre-ITS stack.
  • Introduction: Blackwell introduces two-tier convergence barriers, BRA.U uniform branches, and explicit partial-mask WARPSYNC, none present on Ampere or Hopper.The barrier classes are .RECONVERGENT and .RELIABLE, represented by a 2-bit barrier class.

II. BACKGROUND … C. SASS control flow and the scheduling control word

NVIDIA warps execute divergent paths under active masks and must reconverge, while Volta’s Independent Thread Scheduling enabled independent per-thread scheduling without guaranteeing implicit reconvergence. Post-Volta SASS therefore uses explicit convergence barriers, alongside scheduling-control metadata in each instruction.

  • A. SIMT, reconvergence, and the IPDom stack: A warp issues one instruction across an active mask of up to 32 lanes, partitioning divergent branches into reduced-mask paths that later reconverge.The classical mechanism uses a per-warp stack targeting the branch’s immediate post-dominator.
  • A. SIMT, reconvergence, and the IPDom stack: The classical reconvergence mechanism is a per-warp stack that reunites threads at the earliest instruction reached by all branch paths.This location is the immediate post-dominator of the branch.
  • B. Independent Thread Scheduling: Volta replaced a single per-warp PC with per-thread state, allowing threads on different paths to be scheduled independently and guaranteeing forward progress.This change removed the classic spin-lock deadlock permitted by the stack model.
  • B. Independent Thread Scheduling: Volta’s Independent Thread Scheduling no longer guarantees implicit reconvergence, motivating explicit __syncwarp() convergence barriers.The passage contrasts this with the earlier stack model’s implicit behavior.
  • C. SASS control flow and the scheduling control word: Since Volta, each SASS instruction is 128 bits and includes a compiler-assigned scheduling control word.The word encodes a stall count, yield hint, scoreboard dependencies, and reuse-cache information.
  • C. SASS control flow and the scheduling control word: Post-Volta control flow uses explicit BSSY and BSYNC instructions rather than relying on an implicit hardware stack.BSSY establishes a reconvergence barrier at a target PC, while BSYNC waits on that barrier.

III. EXPERIMENTAL SETUP · IV. THE COST OF DIVERGENCE · A. Divergence is linear path serialization

Dynamic microbenchmarks and static SASS analysis span post-ITS Ampere through Blackwell and pre-ITS Pascal. Across generations, divergent paths serialize linearly with k, including a 31.7–31.9× cost for a full 32-way split without measurable super-linear reconvergence.

  • III. EXPERIMENTAL SETUP: Dynamic experiments microbenchmark tested GPUs, while static experiments compile kernels across architectures and analyze compiler-generated SASS.The static study covers sm_80 through sm_120; Pascal SASS required compilation on a Pascal host.
  • III. EXPERIMENTAL SETUP: CUDA 13 runs on post-ITS GPUs, whereas the pre-ITS Pascal baseline runs CUDA 12.4.
  • III. EXPERIMENTAL SETUP: Cycle counts use clock64() around the measured region, with each datum reported as the median of 201 launches.Cycle counts are frequency-independent, and the coefficient of variation is below 10^-4.
  • IV. THE COST OF DIVERGENCE: The central experiment varies the number of distinct paths k through a balanced divergent region whose paths perform identical work.The measured latency is compared across four post-ITS GPUs and pre-ITS Pascal.
  • A. Divergence is linear path serialization: T(k) ≈ s·k + c fits the data with high precision, with c roughly 2 k cycles on post-ITS parts and negligible on Pascal.
  • A. Divergence is linear path serialization: 54.1 k cycles on Ampere, 58.1 k on Hopper, 46.1 k on both Blackwell parts, and 70.1 k on pre-ITS Pascal are the measured per-path slopes.The two Blackwell parts are indistinguishable.
  • A. Divergence is linear path serialization: 31.7–31.9× is the cost of a full 32-way split relative to a single path on every tested part.Pascal remains flat to within 0.6% across k, and no measurable super-linear reconvergence penalty appears.

B. Efficiency falls as 32/k

Warp execution efficiency falls as 32/k for k-way divergence across all tested post-ITS GPUs, with measured active threads per issued instruction closely matching the ideal and overlapping across generations.

  • B. Efficiency falls as 32/k: 31.9, 16.2, 8.3, 4.25, 2.23, 1.21 active threads per warp-instruction match the 32/k ideal for k = 1, 2, 4, 8, 16, 32.Hardware counters show the measured values track the ideal almost exactly on all four post-ITS GPUs.
  • B. Efficiency falls as 32/k: 32/k: Warp execution efficiency falls with k-way divergence across every tested post-ITS generation, and the four generation curves overlap.Figure 2 measures active threads per issued instruction with Nsight Compute.

C. Predication removes the serialization · D. The penalty is occupancy-invariant · E. Forward progress holds, but paths run in order

Predication removes divergent-path serialization, while occupancy does not hide the resulting issue-rate cost. Forward progress holds across tested generations, but divergent paths still execute in order without fine-grained interleaving.

  • C. Predication removes the serialization: 2.00× to 1×: predicating a two-way divergent branch removes its serialization cost on all four post-ITS GPUs.Measured ratios were within 0.5% of 2.00.
  • C. Predication removes the serialization: 1.94×: predication reduces the corresponding cost on pre-ITS Pascal as well.The result supports predicating small branches across Pascal through Blackwell.
  • D. The penalty is occupancy-invariant: 28–31×: the 32-way divergence penalty remains stable from light to oversubscribed occupancy.Pascal measures 29.9–30.9×, and resident warps cannot hide the increased instruction issue count.
  • D. The penalty is occupancy-invariant: A divergent warp issues k× more instructions, so occupancy hides latency rather than the divergence-driven issue-rate cost.This explains why interleaving other resident warps does not reduce the measured penalty.
  • E. Forward progress holds, but paths run in order: All 32 lanes contending for one lock complete successfully on every tested post-ITS generation and on pre-ITS Pascal.The lock releases inside the acquiring branch and therefore does not exercise the classical unbounded intra-warp peer-wait deadlock.
  • E. Forward progress holds, but paths run in order: At most two spin iterations: consumer lanes observe the producer’s result even when the producer performs 107 iterations of work.ITS provides forward progress without fine-grained interleaving of divergent paths absent contended synchronization.

V. RECONVERGENCE ACROSS GENERATIONS · A. From an instruction stack to barrier registers

Across generations, NVIDIA’s compiler-emitted reconvergence machinery changes from Pascal’s instruction stack to post-Volta barrier registers, while measured divergence cost remains fixed. In the analyzed corpus, deferred reconvergence decreases substantially from Ampere to Blackwell.

  • V. RECONVERGENCE ACROSS GENERATIONS: 38 kernels spanning branches, switches, loops, short-circuit evaluation, and irreducible control flow were compiled from sm_80 through sm_120 for SASS analysis.For each divergent branch, the study reconstructs the control-flow graph and computes the immediate post-dominator.
  • A. From an instruction stack to barrier registers: Pascal brackets divergent regions with SSY and SYNC, which respectively push and pop reconvergence program counters.Nested divergence produces nested SSY/SYNC pairs.
  • A. From an instruction stack to barrier registers: No BSSY, BSYNC, or BREAK instructions appear anywhere in the Pascal corpus.This distinguishes Pascal’s reconvergence-stack mechanism from later barrier-register generations.
  • A. From an instruction stack to barrier registers: Every post-Volta generation uses BSSY and BSYNC barrier-register instructions to manage reconvergence.Blackwell additionally layers reliability-tagged barrier registers onto this mechanism.
  • A. From an instruction stack to barrier registers: 29 deferred reconvergence cases on Ampere collapse to 2 on Blackwell relative to the immediate post-dominator.Deferred reconvergence means placement later than the immediate post-dominator.
  • A. From an instruction stack to barrier registers: Despite these changes in reconvergence machinery, the measured cost of divergence stays fixed across the tested barrier-register generations.The passage explicitly relates this stable cost to the divergence measurements reported in Section IV.

B. Deferred reconvergence disappears · C. A two-tier convergence barrier on Blackwell

Deferred reconvergence shrinks sharply from Ampere to Blackwell, while Blackwell introduces a two-tier barrier scheme that distinguishes early partial reconvergence from the eventual post-dominator merge. Controlled field mutations show no observable runtime effect, supporting a static compiler classification.

  • B. Deferred reconvergence disappears: 29 Ampere branches reconverge later than the IPDom, versus 7 on Hopper and 2 on Blackwell.Deferred reconvergence is the regime where the classical stack postpones merging through unrolled or nested loops.
  • B. Deferred reconvergence disappears: 72.7% of Ampere branches reconverge exactly at the IPDom, rising to 90.8% on Hopper and reaching 83.2% on Blackwell.Blackwell trades the vanished deferred tail for a new class of earlier-than-IPDom reconvergences.
  • C. A two-tier convergence barrier on Blackwell: On sm_100 and later, BSSY/BSYNC instructions carry one of two modifiers that do not exist on Hopper or earlier generations.Earlier-generation barriers are bare, while Blackwell’s modifiers occupy a 2-bit reliability field with two non-zero values.
  • C. A two-tier convergence barrier on Blackwell: .RECONVERGENT marks the true post-dominator merge and the genuine reconvergence point, which is never broken out of in the observations.It is the merge that all lanes ultimately reach.
  • C. A two-tier convergence barrier on Blackwell: .RELIABLE marks an early, partial reconvergence at unrolled-iteration and short-circuit-clause boundaries inside the enclosing .RECONVERGENT region.BREAK is encoding-restricted to .RELIABLE, allowing a subset of lanes to exit early.
  • C. A two-tier convergence barrier on Blackwell: Blackwell’s two-tier scheme layers an early partial-reconvergence barrier inside an IPDom-exact barrier, unlike the single bare barrier of earlier generations.This accounts for Blackwell’s controlled population of before-IPDom reconvergences.
  • C. A two-tier convergence barrier on Blackwell: A positive-control cubin patch changed output, confirming that the driver executed the edited bytes rather than re-deriving the barrier field.The field was rewritten directly in compiled cubins, and the single edit was verified in disassembly.
  • C. A two-tier convergence barrier on Blackwell: Flipping the field across all values left output, completion, warp-level forward progress, partner exchange, and cycle counts unchanged on every tested kernel.The result included mismatched BSSY/BSYNC pairs and a mutated BREAK.RECONVERGENT encoding, supporting interpretation of the field as a static compiler and assembler classification.

D. Uniform branches and partial-mask synchronization

Blackwell uniquely introduces compiler-emitted uniform-branch instructions that mark provably non-divergent control flow and distinguish it statically from divergent control flow.

  • Uniform branches and partial-mask synchronization: 23 BRA.U occurrences on Blackwell versus 0 on Ampere/Hopper mark provably non-divergent branches.BRA.U is predicated on a uniform datapath register and preceded by a uniform ISETP.
  • Uniform branches and partial-mask synchronization: BRA.U replaces divergent BRAs when the compiler can prove uniformity, creating a static distinction between uniform and divergent control flow.

VI. DISCUSSION

The discussion finds a stable programmer-visible divergence cost model from Ampere through Blackwell: k-way divergence costs k×, efficiency is 32/k, predication recovers the cost, and occupancy does not hide it. In contrast, compiler and ISA reconvergence machinery continues to evolve, including disappearing deferred reconvergence and a new two-tier barrier distinction on Blackwell.

  • Stable performance model: A divergence cost model calibrated on one post-Volta generation transfers across Ampere–Blackwell: k-way divergence costs k×, efficiency is 32/k, predication recovers it, and occupancy does not hide it.The cost model can therefore remain fixed across these generations for analytical models and simulators.
  • Stable performance model: Occupancy does not absorb divergence because divergence inflates issue count rather than exposing latency.This corrects the intuition that a busy GPU automatically hides divergence overhead.
  • Evolving reconvergence machinery: Blackwell’s evolving static machinery is marked by the disappearance of deferred reconvergence and the appearance of a two-tier barrier distinction.These changes indicate that compiler and ISA representations of reconvergence distinctions are moving even while the performance cost remains stable.

VII. RELATED WORK · VIII. CONCLUSION

Prior work spans NVIDIA microarchitecture, divergence, scheduling, and compiler analyses, while this paper shows that divergence cost remains linear from Pascal through Blackwell despite substantial changes in reconvergence machinery.

  • VII. RELATED WORK: Prior studies characterize successive NVIDIA generations, core microarchitecture, issue logic, and numerous approaches to warp divergence and reconvergence.The cited approaches include dynamic warp formation, thread-block compaction, warp subdivision, large warps, intra-warp compaction, thread frontiers, stack-less reconvergence, and divergence-aware scheduling.
  • VII. RELATED WORK: Earlier work also examined compiler divergence analysis, scalarization, melding, and Turing’s control-flow instructions and reconvergence mechanism.This body of work largely predates or assumes the IPDom-stack model.
  • VII. RELATED WORK: This study measures how production hardware has moved its reconvergence points and characterizes the forward-progress behavior introduced by Independent Thread Scheduling.The related-work discussion frames the contribution as extending beyond prior IPDom-stack assumptions.
  • VIII. CONCLUSION: The measured divergence cost is stable across Ampere, Hopper, and both Blackwell variants, while its linear form already appears on pre-ITS Pascal.The conclusion qualifies the answer to whether Independent Thread Scheduling has stood still since Volta as no.
  • VIII. CONCLUSION: Divergent regions serialize linearly, warp execution efficiency falls as 32/k, predication recovers lost utilization, and additional occupancy does not hide the issue cost.These findings summarize the measured programmer-visible divergence cost model.
  • VIII. CONCLUSION: The conclusion contrasts stable divergence cost with reconvergence machinery that has changed substantially in the compiler and ISA.The supplied conclusion passage establishes this contrast without detailing the individual mechanisms.
Loading 2607.23402v1…