PaperlayerView exact revision on arXiv

Source-linked AI summary

Ozaki 2.5: Engineering the Deconstruction Path of fp64-Emulated Dense Matrix Multiplication on FP8 Tensor Cores

Satoshi Matsuoka

arXiv:2609.09095v1cs.MScs.ARcs.DCcs.PF

TL;DR

Deconstruction imposes a per-element load that makes large multi-cluster Rubin outputs plateau below the arithmetic roof. The paper models that floor and engineers the conversion path; its projections put large-output performance at ≈235 TFLOPS and move the crossover to ≈480–730, subject to stated conditions and validation.

  • Problem

    On Rubin, per-element deconstruction load becomes constant beyond cluster reach, producing a modeled large-output floor of 235 TFLOPS, or 0.50 of the 473-TFLOPS roof.

    Source: §5.2 · p. 26

  • Method

    The method leaves Ozaki II reconstruction unchanged and engineers conversion with convert-once workspaces and exact two-limb integer-tensor reduction.

    Source: §8 · p. 49

  • Results

    ≈235 TFLOPS is the modeled large-DGEMM floor on Rubin for codesigned set E at NB≳1024, while the published set S is modeled at ≈182 TFLOPS.

    Source: §8 · p. 49

  • Takeaways & Limitations

    The closed-form floor makes conversion load a hardware co-design coordinate, with cluster reach, TMEM capacity, and L2 service among the specified axes.

    Source: §5.2 · p. 26

    The 235-TFLOPS set-E floor applies only at NB≳1024, and its accuracy guarantees are per-set obligations rather than the round-to-nearest theorem covering set S.

    Source: §5.2 · p. 31

Abstract

from arXiv · show

FP8 Ozaki II emulates FP64 matrix multiplication by tensor-core products over a CRT residue system; converting the operands into residue planes (the deconstruction term in the Tensor-Memory Equilibrium model of the companion paper "FP8 is All You Need, Part 1") costs integer-pipe and memory resources before tensor instructions issue. This paper engineers that path; every result is a model projection pending measurement. First, a deconstruction-aware model: on the NVIDIA Rubin GPU the emulated rate reaches the arithmetic roof $P_{\rm FP8}/(3r+1)$ ($\approx 473$ TFLOPS at $r=12$) only within one thread-block cluster; larger outputs are re-split on the fly and held at a floor of $\approx 235$ TFLOPS (half the roof, a ratio of three design integers, not a fit), while real solvers' tall/skinny shapes stay near the crossover, $1.6$-$1.9\times$ over simple deconstruction today. Second, the method: convert-once residue workspaces, an exact two-limb constant-reduction GEMM on integer tensor pipes (or pure-SIMT dp4a), and conversion pipelined behind the MMAs, moving the crossover from $\approx 1211$ to $\approx 480$-$730$. Third, modulus co-design: all-byte and hybrid sets, two supply bounds and a carry-corrected E4M3 split of tail moduli. Fourth and central, the closed-form floor names its hardware escape, and the prize is Rubin's: a stream-side residue-conversion mode on the asynchronous copy path (Option C), a narrow fixed-function block sized as a bill of materials, takes plane formation off the arithmetic pipes and lifts the floor from 235 TFLOPS to the full 473-TFLOPS roof at unchanged cluster reach, about doubling HPL-class FP64 per Rubin GPU, and unbinds conversion-bound sparse kernels. The NVIDIA GB300 GPU, whose 135-TFLOPS roof sits at its own floor, gains little; floor and remedy are Rubin-scale. Application traces ground the analysis; constants are script-checked.

1 Introduction

The paper asks whether deconstruction explains part of the gap between Rubin’s announced emulated-DGEMM rate and the FP8 arithmetic quotient, then engineers that conversion path. It introduces a deconstruction-aware model, Ozaki 2.5, modulus/encoding co-design, and hardware co-design proposals, while treating its performance figures as projections.

1 Introduction

  • The introduction frames the gap as a potentially removable component of the ≈42% Rubin specification-to-roof ratio, distinct from size-independent efficiency loss.The announced ≈200-TFLOPS specification lacks a published algorithm, mode, or benchmark dimensions; the comparison uses the 473-TFLOPS arithmetic quotient.Source: §1 · pp. 2–3
  • Rubin’s ≈235-TFLOPS large-output floor is half the 473-TFLOPS roof, while tall/skinny solver shapes remain near crossover and reach ≈1.6–1.9× over simple deconstruction.The introduction labels these performance figures as reduced-model projections, not measured kernels.Source: §1 · p. 3

1.1 Prior work

  • The cited prior work establishes Ozaki schemes, accuracy results, and throughput, but leaves engineering the FP8 deconstruction path and modulus set as performance variables to this paper.The authors also distinguish this paper from Part 1, which prices deconstruction in the TME model rather than engineering that path.Source: §1.1 · p. 3, §1.2 · p. 4

1.2 Contributions and roadmap

  • The deconstruction-aware model places the Rubin crossover near n*=1,211 and the large multi-cluster rate at 0.50 of the roof, a derived ratio rather than a fitted efficiency.The reduced model passes through the published 200-TFLOPS figure at inferred n≈500–900 as a falsifiable hypothesis, not a finding.Source: §1.2 · p. 4
  • Ozaki 2.5 engineers conversion with convert-once workspaces, exact two-limb integer-tensor reduction, a SIMT residual, and pipelining, moving the Rubin crossover to ≈480–730.The method leaves Ozaki II mathematics unchanged.Source: §1.2 · p. 4
  • Modulus/encoding co-design treats the set as a design variable, including an all-byte one-pass system, a hybrid set, and exhaustive supply bounds that motivate regime-switched dispatch.The selected designs are not an exhaustive search of all mixed-modulus possibilities.Source: §1.2 · p. 4
  • The hardware proposal names cluster reach, TMEM capacity, L2 bandwidth, and a conversion-capable copy-engine datapath as measurable co-design asks; application traces and a size sweep ground the analysis.The proposed datapaths are not present in silicon, and the decisive DGEMM size sweep is described as a future experiment.Source: §1.2 · pp. 4–5

2 Background: the Ozaki Schemes and the Substrate Transition

Ozaki Scheme II replaces slice-pair products with exact residue products over a CRT system, but FP8’s narrow significand makes residue-plane formation a substantial part of the path. The background explains the scheme, its accuracy and reconstruction conditions, and why the shift from int8 to FP8 makes deconstruction important.

From error-free splitting to modular arithmetic

  • Ozaki I uses error-free low-precision slices with cost quadratic in slice count, whereas Scheme II computes one exact integer product per CRT modulus with cost linear in modulus count.The exact integer core gives componentwise fp64-grade accuracy under the stated scheme description.Source: §2 · p. 5

The scheme in equations

  • The round-to-nearest variant has a cited error theorem within its hypotheses, but the shipping truncation-toward-zero rule does not inherit that bound and remains an accuracy-validation obligation.The integer product is exact under either quantizer; the projections assume truncation toward zero.Source: §2 · p. 6
  • Scheme II forms operand residues, computes a product per modulus, and reconstructs outputs in Garner form; reconstruction touches outputs, while deconstruction touches every input.The residue products are exact through low-precision decomposition, and rounding-free FP32 accumulation holds for k≤216 before k-blocking is needed.Source: §2 · p. 6

The fp8 variant

  • FP8 residues for 9–11-bit moduli use two limbs and three plane-product MMAs per modulus, with stored-plane load distinct from the issued-MMA count.Square moduli use two stored planes; nonsquare moduli store a third sum plane for the Karatsuba identity.Source: §2 · pp. 6–7
  • At r=12, accurate mode uses 37 FP8 MMA passes per FP64 product; dividing the official rates gives ≈135 TFLOPS on GB300 and ≈473 TFLOPS on Rubin.The GB300 point is NVL72-derived per-GPU data, not the distinct air-cooled HGX B300 rating.Source: §2 · p. 7

The substrate transition, and why it matters here

  • The substrate shift makes deconstruction costly: int8 supports byte extraction or direct byte-modulus operands, while FP8 requires per-modulus reduction and three FP8 pieces before an MMA.The paper notes that the int8 route’s all-byte modulus set needs 15 moduli to meet its stated 117.8-bit supply, while FP8 scales to much higher tensor rates.Source: §2 · p. 7

3 The Updated TME Model and the Fourth Term

The updated TME model charges deconstruction per input on its host pipe, alongside tensor and memory service, and includes a reconstruction tail. The section refines that cost, describes moving reduction work to tensor arithmetic, and treats reconstruction implementations and their costs as modeled or validation-dependent.

3 The Updated TME Model and the Fourth Term

  • The model bounds emulated-kernel service with tensor, memory, deconstruction, and reconstruction terms, distinguishing ideal host-pipe overlap from the service-tail endpoint.The displayed tail is not fully serialized and does not by itself upper-bound measured time; imperfect overlap may move measured time beyond it.Source: §3 · p. 8
  • On the shipping conversion path, deconstruction costs 16 integer-pipe instructions per modulus per converted input, including 13 counted conversion instructions and three FP8 Karatsuba converts.The model charges deconstruction by input count, unlike reconstruction, which is charged per output.Source: §3 · p. 8
  • The proposed conversion ledger moves byte-plane modular reduction to tensor-side matrix arithmetic, leaving a ≈6.3-instruction SIMT residual for scale/truncate, final centering, and the structural split.The redundant construction was exhaustively checked over all ordered centered pairs for six moduli, with zero mismatches.Source: §3 · p. 8
  • The lazy-reduction variant lowers the modeled SIMT residual but doubles tensor cost to α′≈6r+1, so the paper excludes it from dense-kernel analysis.The authors characterize it as benign for memory-bound kernels but say it would halve the dense ceiling.Source: §3 · p. 9

The reconstruction term, quantified

  • Garner reconstruction requires about 90 narrow integer operations per output at r=12; at k=1000 its modeled service time is 8.1% on GB300 and 28.4% on Rubin.It falls below 1% only at k≳8.1×10^3 on GB300 and k≳2.84×10^4 on Rubin.Source: §3 · p. 9
  • SIMT Garner is used as the conservative reconstruction-cost floor, while ≈r-op integer or fractional-CRT hosting remains a target or sensitivity rather than a delivered result.Fractional-CRT reconstruction still requires fp64-exactness validation; trace composites use the conservative SIMT-Garner cost.Source: §3 · p. 9
  • The trace comparison changes when using conservative SIMT-Garner rather than the ≈r-op target, with some small-k shipping-relative ratios and native-reference multipliers falling.Examples include UMFPACK GB300 1.00→1.19 and dense-QR Rubin 2.06→1.60; the target would restore these pending validation.Source: §3 · p. 9

4 The Dense Puzzle: a Size Crossover, and Where 200 TFLOPS Falls

The modeled dense-GEMM rate reaches the arithmetic roof only within one cluster; beyond that reach, repeated deconstruction imposes a large-output floor. The section also examines where the published ≈200-TFLOPS result falls on the modeled curve and how the floor could be removed.

The size crossover

  • For dense GEMM, conversion stops binding when operand deconstruction time no longer exceeds the tensor-multiplication time; the crossover depends on the problem dimensions and the deconstruction cost.The convert-once model assumes each operand is deconstructed once and its residue planes reused thereafter, an assumption exact only within one cluster’s reach.Source: §4 · pp. 9, 11
  • The convert-once envelope rises with output size only up to reach R=256; beyond it, operands are re-split across clusters and deconstruction load stops falling.On-chip planes cannot be shared across clusters, and materializing them through L2 or HBM is feed-starved; Option C moves plane formation to the copy path and returns Rubin to the roof at unchanged reach.Source: §4 · pp. 11–12

The large-output ceiling

  • ≈235 TFLOPS on Rubin is the modeled cluster-aligned large-square ceiling, 0.50 of the 473-TFLOPS roof, with the ratio arising from design integers rather than a fitted efficiency.The floor follows from reach and the per-element deconstruction load; changing the int8-tensor provisioning moves it, while GB300's floor does not bind its roof.Source: §4 · pp. 11–12

Removing the floor

  • Option C forms residue planes on the copy engine, removing the deconstruction charge from arithmetic pipes and returning Rubin’s rate to the roof at unchanged reach.Materializing planes through L2 or HBM is feed-starved; the added deposit traffic is a fraction of an operand read the kernel already performs.Source: §4 · p. 12, Figure 2 · p. 13

Where 200 TFLOPS falls

  • ≈200 TFLOPS, NVIDIA’s published Emulated-DGEMM figure, is consistent with the deconstruction-limited branch at modeled sizes ¯n=512 to ¯n≈900, but is not decisive evidence for that explanation.A flat ≈40% delivered efficiency could explain the same result; a size sweep with a slicing-based int8 control would distinguish the predicted size-dependent knee from flat efficiency artifacts.Source: §4 · pp. 12–13

Route comparison at HPL depth

  • At HPL-relevant depth k=4096, route E has a 235-TFLOPS floor versus route A’s 231, although A keeps the best roof-fraction.This finite-k ordering reverses beyond k≈6777, so the table’s crossover and large-square floor are distinct quantities.Source: Table 2 · p. 15

5 The Ozaki 2.5 Method

This section co-designs Ozaki 2.5 residue-plane formation across SIMT, integer-tensor, and asynchronous-copy paths, including reduction routes, moduli, storage, and pipeline choices. It models how deconstruction costs and a multi-cluster floor shape dispatch and throughput, and evaluates Option C and fallback hardware changes as conditional routes toward each path’s roof.

5 The Ozaki 2.5 Method

  • Ozaki 2.5 retains Ozaki II’s MMA schedule and reconstruction mathematics while reusing residue planes, routing reduction to integer tensor pipes or SIMT dp4a, and overlapping conversion behind the tensor stream.The method also keeps the SIMT residual at narrow-integer grain and does not take the lazy-reduction trade.Source: §5 · p. 15
  • Materialized or persistent planes achieve λ=1 but incur plane traffic, whereas fused modes avoid plane traffic but may require λ>1; reused planes serve every MMA touching a panel.For fixed-operand Krylov workloads, persistent planes amortize conversion across calls; the method keeps conversion and HBM-byte accounting separate because their relationship depends on schedule.Source: §5 · pp. 16, 18
  • Each modulus reduction uses two unsigned-byte-limb int8→int32 GEMMs with exact bounded accumulators, recombined as U(0)+256U(1), at a cost of 2·8r≈192 MACs per element.A one-pass single-limb reduction is exact only with a sub-256 modulus system; the published set’s two-limb route has a modeled demand of ∼380 TOPS at the SIMT-residual knee, so capacity and narrow-shape efficiency remain constraints.Source: §5 · p. 18
  • The carry-corrected three-plane layout makes every stored plane e4m3-representable for nonsquare moduli m≤513, while m=514 fails this rule.Exhaustive checks across the six tail moduli found zero mismatches; the correction costs ≈1 SIMT instruction per modulus and is included in the ∼6.3 instruction ledger.Source: §5 · p. 20
  • Double-buffered asynchronous-copy pipelining converts panel t+1 while panel t undergoes MMA; at n̄≈500, the required conversion rate is P_fp8/(37n̄), about one third of the full HBM stream rate.This overlap moves a real kernel from the serialized toward the overlapped cost-model bracket.Source: §5 · p. 20
  • Lazy reduction lowers c_q to ≈4.3 on the published set but halves Rubin’s dense ceiling to ≈240 TFLOPS, so Ozaki 2.5 excludes it.The passage characterizes lazy reduction as appropriate for memory-bound streamed kernels, not dense GEMM.Source: §5 · p. 20

5.1 Modulus/Encoding Codesign: Sub-256 Systems

  • Sub-256 modulus codesign seeks pairwise-coprime moduli preserving at least 111.8 CRT bits, e4m3-representable residues and Karatsuba sums, and 28^k mod m_i≤255 for one-pass reduction.Candidate A’s constants fit one byte, eliminating the second limb and recombination and yielding rigorously counted c_q≈7.3 for the pure-SIMT route.Source: §5 · p. 19, §5.1 · p. 20
  • The exhaustive supply bounds show square moduli with s≤33 provide at most 94.1 bits and moduli ≤64 at most 89.9 bits, below fp64’s 111.8-bit range.Thus squares alone cannot span the range and a tail is mandatory; the selected S/A/E/D designs are not exhaustive over mixed moduli, schedules, and encodings.Source: §5.1 · p. 22
  • S keeps Nacc=3 on six 9-bit tail moduli despite an exact E4M3 merge, because the merged term’s 4095 maximum limits exact accumulation to K≤4097.The merge is declined to preserve margin at Kslab=4096; consequently S and A both require 30 accumulator tiles, while E requires 26.Source: §5.1 · pp. 23–24, Table 4 · p. 24
  • At n̄=256, A beats S by 27% at k=4096 (231 vs 182 TFLOPS) but trails by 22% in the cubic regime (131 vs 168 TFLOPS).A’s 15-modulus Garner reconstruction is not amortized when k=n̄=256; its roof is 380 TFLOPS versus S’s 473, and its tmem footprint uses the full 256-KiB budget.Source: §5.1 · pp. 24–25

5.2 Hardware Co-Design: Lifting the Deconstruction Floor to the Roof

  • Beyond Rubin’s 256-element cluster reach, conversion load becomes constant and the modeled large-DGEMM rate plateaus at 235 TFLOPS, half the 473-TFLOPS roof.The closed-form floor is R·Pint/(cqr); the passage says traced tall/skinny, small-k, or sub-cluster applications are largely spared, while GB300’s 135-TFLOPS roof sits at or below its floor.Source: §5.2 · p. 26

5.1 Modulus/Encoding Codesign: Sub-256 Systems

  • Figure 5’s modeled dispatch selects A below n̄≈410, E to ≈620, and S above on Rubin; on GB300, L1 direct reaches 135 TFLOPS at n̄≈220.The switched envelope is at least as fast as any single route; on GB300, dp4a A is within ≈7% of tensor A without int8-tensor use.Source: §5.1 · pp. 25–26, Figure 5 · p. 27

5.2 Hardware Co-Design: Lifting the Deconstruction Floor to the Roof

  • Raising cluster count or tile size can extend reach, but cluster layouts must divide launch-grid dimensions and larger tiles face a 256-KiB-per-CTA TMEM bound.At nblk=2, 128² tiles require 3.5–4.0× TMEM and 192² tiles 7.9–9.0×; increasing blocking also re-reads fp64 operands.Source: §5.2 · pp. 28–29
  • Materialized conversion reaches each route’s roof only with about 10× HBM bandwidth: S needs 221.7 TB/s, E 225.6 TB/s, and A 261.5 TB/s.These requirements correspond to 10.08×, 10.25×, and 11.89× L2 service bandwidth, respectively, versus the assumed ≈4×.Source: §5.2 · p. 29
  • Route performance depends on available cluster reach: all-byte A and hybrid E reach their lower roofs at 64 CTAs, while the 473-TFLOPS two-limb and pure-SIMT routes need 128 CTAs or more.Thus, the best route is reach-dependent, and sparse or streaming kernels instead need deferred or persistent deconstruction to amortize conversion [11].Source: §5.2 · p. 30
  • At HPL scale, panel width amortizes reconstruction rather than changing the deconstruction floor, and NB multiples of reach 256 keep local edges cluster-aligned.For ragged local edges of 8000 versus 8192, the alignment penalty is ≈2%; the larger NB effect is the reconstruction amortization, which falls as 1/NB.Source: §5.2 · p. 31
  • At NB=1024, modeled per-GPU HPL rates are 182 TFLOPS for theorem-backed S and 235 TFLOPS for hybrid E, both against Rubin’s 473-TFLOPS roof.E reaches 235 TFLOPS only for NB ≳1024, while at NB=256 it delivers 159 TFLOPS; reported HPL practice is NB ≈892–1024. These are modeled projections, not measurements.Source: §5.2 · pp. 31–32
  • 4.73 EFLOPS projected DGEMM peak and 3.6–4.0 EFLOPS HPL Rmax on 10,000 GPUs follow if Option C lets theorem-backed set S reach its 473-TFLOPS roof.This is a modeled projection, not a submission or measurement; it is roughly 2.6× today’s S row and doubles today’s E row while retaining the round-to-nearest contract.Source: §5.2 · p. 32
  • Option C moves residue conversion into the asynchronous copy engine, removing the compute-pipe deconstruction term and enabling each route to reach its own roof at today’s cluster reach, conditionally.The transform must sustain 166–325 TMAC/s and plane deposits 55–65 TB/s of SMEM writes; conversion time remains and the requirements are modeled vendor asks, not existing hardware.Source: §5.2 · pp. 32–33, Figure 7 · p. 33, Table 9 · p. 34
  • A faster L2 with software-blocked materialization reaches 0.60, 0.79, and the roof at 6×, 8×, and ≈10–12× HBM L2 bandwidth, respectively.It is dense-only and requires a real bandwidth investment; unlike Option C, it does not help SpMV/GEMV. A 3.5–4.0× TMEM tile reaches 0.92, while increasing NB alone reaches only the 0.50 floor.Source: §5.2 · p. 35

6 Projected Performance

This section projects software-route performance and application scenarios, comparing current-GPU results with convert-once envelopes and a hardware-enabled ceiling, while modeling deconstruction and plane-schedule costs across large dense GEMMs and traced workloads. The projections depend on storage schedules, memory traffic, feed bandwidth, residency, shape, and concurrency assumptions; they are not measured kernel or application performance.

6 Projected Performance

  • At ¯n=256, switched dispatch reaches 243 TFLOPS (≈2.4× shipping); at ¯n=512, projected rates are 364–438 TFLOPS, conditional on conversion and shape.The larger-¯n figures are convert-once envelopes: tall/skinny and small-batch shapes can realize them, while large square outputs are otherwise clipped near 235 TFLOPS on Rubin.Source: §6 · pp. 35–36, Table 10 · p. 36
  • At ¯n=512 on Rubin, tensor-service contention brackets switched dispatch at [352, 438] TFLOPS and the published set at [310, 364] TFLOPS.The lower endpoint assumes complete serialization; pure-dp4a routes use no int8 tensor capacity, while interpolation depends on measured concurrency.Source: §6 · pp. 35–36
  • Complete SIMT–fp8 serialization projects Rubin ¯n=512 rates of ≈183 TFLOPS for A-dp4a and ≈187 for L1, but this is a shared-overlap sensitivity, not an unconditional floor.With the shipping baseline degraded consistently, the like-for-like uplift is ≈1.30× at 512 and ≈1.46× at 256; both concurrency parameters await measurement.Source: §6 · pp. 36–37

6.1 Application-Geometry Scenarios: Measured Call-Shape Traces

  • Measured traces place all LOBPCG b=64 GEMM work at ¯n≤256, half of blocked QR BLAS-3 work near ¯n≈64, and UMFPACK’s median GEMM near ¯n≈670.The traces show block-width-bounded work in LOBPCG and QR panels, while multifrontal fronts occupy the crossover band; sequential SuperLU’s panel updates are dgemv-based, not BLAS-3.Source: §6.1 · pp. 37–38, Figure 8 · p. 40
  • After schedule, memory-roof, and reconstruction corrections, Rubin multifrontal projections fall from 2.07× to 1.19× and QR from 2.27× to 1.60×.These are GEMM-portion scenario projections over measured call shapes, not application benchmarks; Table 11 reports the composites and counterfactual roof-lifted cases.Source: §6.1 · pp. 38–39, Table 11 · p. 39
  • Lifting the deconstruction floor to the compute roof yields modeled headroom of ≈2.5–3.3× on Rubin but only ≈1.1–1.5× on GB300.Even with deconstruction free, these traced workloads remain bandwidth-limited; the counterfactual is a hardware ceiling, not a delivered rate.Source: §6.1 · p. 38, Table 11 · p. 39

6.2 Storage Modes: Where the Planes Live, and What Each Mode Charges

  • Storage modes trade conversion multiplicity against plane traffic: fused planes avoid HBM traffic but may be reconverted, while transient materialization converts once and incurs plane writes and reads.The per-call selector chooses the best eligible schedule under compute, additive-memory-traffic, and feed-bandwidth caps.Source: §6.2 · pp. 40, 42
  • Auditing the schedule engine exposed an omitted HBM plane write-plus-read charge; corrected roof-attainment crossovers for full materialization on Rubin are 1548/1764/1551 for S/A/E on square shapes.The earlier branch charged reduction traffic and a plane-feed cap but omitted the c=2p traffic term, an error that could be hidden on square shapes and surfaced on tall-and-skinny shapes.Source: §6.2 · p. 41
  • 276 HBM-materialized records now fall back to on-the-fly scheduling, revising CCSD on Rubin from ≈1.73× to ≈1.69× over shipping (116 →114 TF delivered).The correction also moves co-design headroom from ×2.49 to ×2.54; every other application, platform, projection, and co-design table remains unchanged to printed precision.Source: §6.2 · p. 43
  • L2-resident materialization avoids HBM plane traffic only when the full workspace fits; the 126-MB capacity assumption gives square reach k ≤1449/1196/1381 for S/A/E.This is a testable but unmeasured hypothesis requiring usable capacity, plane residency between write and read, and no write-back spill; halving effective capacity lowers S reach from 1449 to 1024.Source: §6.2 · p. 43
  • The fused-hybrid schedule is genuinely tall/skinny: its square-shape surcharge δ is 2.50/3.75/2.75 for S/A/E, and δ ≤0.2 requires aspect ratios m/n ≥36.5/55.25/40.25.The earlier blanket 10–20% surcharge claim is withdrawn for general shapes; it applied to tall-skinny cases.Source: §6.2 · p. 44
  • Materialized-plane feed caps are 188/171/128 TFLOPS for L2 and 47/43/32 for HBM (S/E/A), so the design generates planes on the fly.Those are feed-only caps; charging the one-off L2 write lowers the small-NB caps to 186/169 TFLOPS. On-the-fly generation avoids plane storage but incurs cluster-level re-splitting.Source: §6.2 · pp. 44–45
  • ≈235 TFLOPS, or 0.50 of Rubin’s 473-TFLOPS roof, is the large-square on-the-fly floor; GB300’s 135-TFLOPS roof binds before its floor.The floor arises from re-splitting operands across clusters; outputs within one cluster’s 256² reach keep λ = 1. The 0.50 floor assumes large k and a 16-CTA cluster; it is ≈0.35 at k=256, while an 8-CTA cluster lowers the reach to 170.67 and floor to ≈157 TFLOPS.Source: §6.2 · pp. 45, 47, Figure 9 · p. 46
  • Table 12 reports 0.89–0.94× for LOBPCG, 1.00× for multifrontal and blocked dense LU, 0.95× for dense QR, and 0.82× for CCSD.The floor mainly affects large-k dense square products; sparse and streaming kernels are exempt because each value is converted once and consumed on-chip.Source: §6.2 · p. 45, Table 12 · p. 46
  • Persistent sparse conversion raises set S’s stream from ≈12 to ≈34 B/nnz (×2.83), capping throughput near 0.35 of the raw-value memory roof.It is strictly worse than fused conversion on GB300, comparable on Rubin (0.35 versus 0.36–0.39), and mainly preferable when planes are L2-resident or SIMT issue is contended.Source: §6.2 · p. 47

7 Validation Plan and Falsifiable Predictions

The section specifies falsifiable predictions and staged measurements to test the reduced model, distinguish storage modes and deconstruction effects, and establish which claims are proved, projected, or still unmeasured.

Linearity

  • Below the knee, modeled CRT/fp8 throughput rises linearly with n̄ at approximately 0.39 TFLOPS per unit n̄.The prediction is for the reduced model at the reference constants.Source: §7 · p. 47

Knee position

  • Convert-once sweeps are predicted to knee at n̄≈n*(cq), whereas deployable on-the-fly large squares knee near cluster reach n̄≈256 and plateau at about half the roof.For on-the-fly execution, cq is inferred from the rising-branch slope rather than the knee.Source: §7 · p. 47

Substrate structure

  • Convert-once sweeps predict knees near n̄≈730 for B200 CRT/int8 and n̄≈1,211 for Rubin CRT/fp8; the slicing-int8 control predicts neither practical-size signature.Flat sub-ceiling margins across all three paths would refute the branch hypothesis and point to generic-efficiency artifacts instead.Source: §7 · p. 47

Ragged-edge boundary

  • The rigid integer schedule’s enumerated worst case is 0.36 at (513, 513), while a ragged-edge-aware schedule is modeled at 0.54 but remains unconstructed.The latter is a Part-3 validation target, not an established minimum.Source: §7 · p. 47

Ladder shift

  • Validation proceeds from exact residue equivalence through isolated and concurrent conversion tests to end-to-end knee movement and application-level replay.A dummy SIMT/int8-work negative control separates deconstruction service demand from generic small-GEMM inefficiency.Source: §7 · p. 48

Decisive experiment

  • A single DGEMM size sweep across Rubin CRT/fp8, B200 CRT/int8, and a slicing-int8 control can distinguish predicted storage-mode signatures and fit each path’s deconstruction cost.The sweep fixes storage mode and uses square and tall/skinny shapes; the predicted signatures are a size knee for convert-once and a reach-256 floor for on-the-fly execution.Source: §7 · p. 48

Pass/fail criteria

  • Pass/fail checks cover bit-exactness, instruction and occupancy accounting, reduction shape, concurrency, knee fitting, numerical accuracy, and application replay.Knee fitting uses confidence intervals and held-out shapes; application reporting separates GEMM-portion and whole-run Amdahl numbers.Source: §7 · p. 48

Claim status

  • The paper classifies algebraic proofs and script checks separately from memo-counted or modeled quantities; only call-shape traces are measured, and all route knees and speedups remain conditional projections.The supplied artifact regenerates tables and figures from stated constants and traces, but generation is not independent verification of those constants.Source: §7 · pp. 47–49

8 Conclusion

The conclusion projects a deconstruction-limited floor for large Rubin DGEMM while finding better modeled outcomes for solver-like shapes. It presents an engineered conversion path and modulus co-design, with hardware co-design as the route toward the arithmetic roof.

Conclusion

  • ≈235 TFLOPS is the projected large-square Rubin DGEMM floor on set E, versus ≈182 TFLOPS on published set S; tall/skinny solver shapes are projected at ≈1.6–1.9× over simple deconstruction.Large outputs plateau beyond one cluster’s reach; solver-like shapes stay near the crossover, and the cited rates remain model projections pending measurement.Source: §8 · p. 49
  • An in-flight-convert copy-engine datapath is identified as the minimal co-design route to lifting Rubin’s large-square floor toward the roof; higher-bandwidth L2 is an alternative for software-blocked materialisation.The conclusion also identifies the deconstruction hypothesis as testable by a size sweep with a slicing-int8 control.Source: §8 · p. 49
  • The method leaves Ozaki II mathematics and reconstruction unchanged while using convert-once workspaces, exact two-limb integer-tensor reduction, SIMT residual work, and conversion overlapped with MMA.The exact two-limb reduction is used because the actual moduli exceed a byte.Source: §8 · pp. 49–50
  • The reduced model projects the Rubin crossover moving from ≈1,211 to ≈480–730 and 364–438 TFLOPS at n̄=512, a conditional 1.8–2.2× convert-once envelope.Tall/skinny shapes are mildly clipped, while large squares remain clipped to the ≈235-TFLOPS floor.Source: §8 · p. 50
  • All-byte moduli are projected faster below n̄≈540 despite a 20% lower roof, while the hybrid set concedes 7.5% asymptotically and leads in the ≈410–620 band.The conclusion frames modulus choice as a runtime-switchable performance parameter.Source: §8 · p. 50
  • On GB300, the all-byte set leads the small-size band, while the two-limb route takes the roof from n̄≈290; every modeled route exceeds native fp64 (≈1.4 TFLOPS) throughout evaluated n̄≥32.Source: §8 · p. 50

A Proof Obligations per Candidate Modulus Set

Each candidate modulus set has an explicit numerical contract: changing moduli preserves the reconstruction framework but does not automatically inherit the source paper’s accuracy argument.

Proof obligations per candidate modulus set

  • The appendix treats every accuracy statement as scoped to a named numerical-contract variant rather than inherited automatically when the modulus set changes.Table 13 names the variants and their status.Source: A Proof Obligations per Candidate… · p. 50, Table 13 · p. 51
  • Candidate sets A, E, and D are script-checked for coprimality, CRT product range, centred digit maps, E4M3 representability, and int32-safe reduction accumulation.Their reported log2 CRT products are 117.8, 113.3, and 113.5 bits, respectively; the published set is 111.8 bits.Source: A Proof Obligations per Candidate… · p. 51
  • Signed limb equivalence is checked against trusted integer modulo, while scaling, truncation, escalation, and reconstruction conditions remain validation obligations until re-verified per set.SASS-level and accuracy-suite confirmation are still assigned to later validation.Source: A Proof Obligations per Candidate… · p. 51

B Provenance of the Memo-Counted Constants

The memo-derived constants are presented as conditional anchors, and an alternative SIMT normalization substantially lowers the projected floor.

Provenance of the memo-counted constants

  • The shipping-path estimate assigns about 16 deconstruction instructions per element per modulus, amortizing scale/truncation, byte-plane reduction, final reduction, and Karatsuba splitting.These are SASS-counted estimates, not throughput measurements, and the source note’s emulation realization remains conditional.Source: B Provenance of the Memo-Counted… · p. 51
  • Using the spectral companion’s 41.7 T inst/s normalization instead of the memo’s 75 makes the Rubin large-square floor fall from 235 to ≈148 TFLOPS.The tensor-side quantities do not depend on the normalization, but the floor does; the paper reports both cases as engine-checked sensitivity results.Source: B Provenance of the Memo-Counted… · p. 52
  • The 41.7 T inst/s value is derived by halving published B300 fp32 TFLOP/s under an assumption that fp32 and integer issue share Blackwell lanes.The paper retains 75 as its reference because that is the normalization used for the memo’s counts.Source: B Provenance of the Memo-Counted… · p. 52
Loading 2609.09095v1…