ForgeStencil: Automating Per-Case Stencil Specialization from Kernels to 100+ Real Applications

1 Introduction

• General stencil methods reuse one human-authored recipe because specialized solutions were too expensive to build per case, a performance cost termed the generality tax.The paper argues code-synthesis agents now make correct per-case solutions affordable.Source: §1 · p. 1
• 37/37 operator cases beat per-case public-SOTA upper bounds; across 100+ applications, end-to-end median speedup is 1.41× against same-architecture GPU baselines.The kernel and application results are measured at separate levels.Source: Figure 1 · p. 2, §1 · p. 2
• The Kernel Agent writes a CUDA kernel for each stencil, grid shape, and precision, retaining it only after it passes the gate and beats the per-case baseline.Accepted kernels populate a configuration-indexed map of specialized operators.Source: §1 · pp. 1–2
• The App Agent locates performance-critical regions, forges per-application optimizations, and validates and integrates them against each application's tests and timing.The workflow spans 100+ real industrial and scientific codes.Source: §1 · p. 2
• Separate operator and application gates keep kernel microbenchmark evidence distinct from end-to-end application evidence.The harness retracts reported numbers when executable checks fail.Source: §1 · pp. 2–3

2.1 Stencils, their variety, and the bandwidth limit

• For most tested instances, operators reach 74–85% of peak HBM bandwidth, with a 76.6% median, confirming the memory-bound regime.Stencil type and run shape determine whether on-chip reuse is sufficient to move a case toward compute-bound performance.Source: §2.1 · p. 3

2.2 Generic stencil methods and the generality tax

• General methods reuse transformation rules, search templates, or implementations, so cases outside their built-in forms pay a performance cost; comparable head-to-head measurement has been rare.The paper calls that cost the generality tax.Source: §2.2 · p. 4

2.3 Why per-case specialization becomes practical now

• Code synthesis makes per-case implementation affordable, but performance engineering still requires an agent loop that measures each change and revises from feedback.Prior work leaves gaps in stencil coverage, repeatable specialization practice, and trustworthy measurement.Source: §2.3 · pp. 4–5

System overview

• The Kernel Agent forges one kernel per stencil type, grid shape, and precision case, using an operator map that binds bottleneck diagnoses to gate-cleared code.It commits a change only when it is correct and faster than the per-case public-SOTA upper bound.Source: §3 · p. 5
• The App Agent measures an application baseline, rewrites framework structure, obtains the hot operator from the Kernel Agent, then validates and integrates the result.Framework changes address scheduling, fusion, and host–device traffic before operator-level work.Source: §3 · p. 5
• Independent gates measure proposed changes on real hardware, with the operator and application gates judging only their own levels.The App Agent invokes the Kernel Agent, but applications cannot access or author the operator map.Source: §3 · p. 5

4 Kernel Agent: Forging Specialized Operators

• For each stencil type, grid shape, and precision, the Kernel Agent synthesizes a specialized CUDA kernel and keeps it only when it is correct and faster.The operator gate also checks for regressions against accepted cases.Source: §4 · p. 5, §4.1 · p. 5

4.1 The code-synthesis autoloop

• Each autonomous round reads the rulebook, logs, baseline, and recent commits, makes one change, measures it, then commits or reverts and records the outcome.The agent tests the affected case first, then the full matrix.Source: §4.1 · p. 5
• Fresh rounds retain no in-memory history; factual performance and idea logs preserve outcomes, including rejected directions and their measured causes.Logs cover the blind spot left when reverted code no longer records a failed attempt.Source: §4.1 · pp. 5–6
• Disruptive rewrites and single-shape kernels can branch; after five rounds without progress, a branch is tested against the main line and merged only if faster.Otherwise it is dropped without losing accepted main-line rounds.Source: §4.1 · p. 6

4.2 The map as the loop’s memory

• The operator map transfers a neighboring cell's bottleneck diagnosis, not its code, and the new case re-derives and profiles its own structure.Rejected directions and measured causes are recorded in logs; after three non-improving rounds, the loop must search outside its history.Source: §4.2 · p. 6

4.3 How forging removes the generality tax

• The loop profiles each case and selects among standard bandwidth-bound levers, because the useful choice depends on the binding bottleneck.For star_1, it raises occupancy; for register-pressured box_1, it lowers occupancy and uses a j-blocked micro-tile.Source: §4.3 · pp. 6–7
• Per-case public-SOTA winners are split among Halide, Devito, and EBISU, while equal-compute ablations find neither search nor one-shot generation reaches the loop's result.Table 1 summarizes the 37 cases with a public-SOTA baseline.Source: §4.3 · p. 7, Table 1 · p. 7
• The optimization axis shifts by case: box_2 uses warp-level shfl reduction, while diamond temporal blocking fuses steps and keeps intermediate planes on-chip.The fused approach also combines stencils with application physics in 13 kernels; measured fusion merged the 4- and 8-step variants.Source: §4.3 · p. 7

5 App Agent: Extending Specialization to Whole Applications

• Most validated application wins are per-application structural solutions; drop-in stencil replacement is a small minority, and generic-library operators account for none of the 100 wins.Examples include fusing launches, removing host-device round-trips, fixing layouts, or simplifying arithmetic.Source: §5 · p. 7

5.1 States, milestones, and the Amdahl ceiling

• The workflow classifies integrations through five terminal states and milestones that locate the hotspot, establish the application’s GPU baseline, optimize, validate correctness and timing, and integrate.Correctness and speed are independent gates, both measured against the application’s own GPU baseline.Source: §5.1 · p. 7
• End-to-end speedup is bounded by 1/(1 − f + f/S); when the stencil fraction is small, structural changes that raise the accelerated fraction can matter more than speeding up one kernel.The passage identifies removing redundant launches, host-device traffic, and recomputation as structural means.Source: §5.1 · p. 8

5.2 Two-layer forging, framework before operators

• Framework-level optimization precedes operator optimization, addressing scheduling, fusion, and redundant work before tuning the hot kernel.This order fixes the operator boundary where fusion is involved and stabilizes its runtime fraction for measuring end-to-end impact.Source: §5.2 · p. 8
• For the operator stage, the App Agent invokes the Kernel Agent with the hot-operator case; the Kernel Agent owns map matching and returns a validated kernel, or forges one when needed.Forging may be needed for a fused operator, an uncovered primitive, a shape or precision mismatch, or a transplanted kernel that fails on application data.Source: §5.2 · p. 8
• The Kernel Agent’s loop can also forge non-stencil operators, which may become hotspots after stencil acceleration leaves the complementary runtime as a bottleneck.The loop rules contain no stencil-specific knowledge; that knowledge resides in the operator map.Source: §5.2 · p. 8

6.1 Positioning

• The harness precludes fabricated speedups by enforcing executable requirements on measurement surfaces outside the agent’s write scope before a run.Operator and application claims use separate gates and their evidence is not mixed.Source: §6.1 · p. 8

6.2 The operator gate

• Every Kernel Agent round must pass correctness and zero-regression checks using an independent oracle, persisted prior-best baselines, and idle-GPU canonical remeasurement.The driver, timer, and oracle remain outside the agent’s reach; the prior-best baseline is distinct from the external SOTA used for final evaluation.Source: §6.2 · p. 8

6.3 The application gate

• Application comparisons pair two variants of the same real program on its own inputs and shipped baseline, differing by one switch and timed fairly.Table 2 presents the requirements a trustworthy speedup must satisfy and the fraud each precludes.Source: §6.3 · p. 8, Table 2 · p. 9
• Application correctness comes from the program itself, through a built-in verdict, same-program output comparison, or matched cross-variant stdout.The application supplies the oracle through one of three manifest modes.Source: §6.3 · p. 9
• Multi-dataset applications report an all-dataset geometric mean, and only same-architecture GPU measurements qualify as REAL; all 100 validated applications are REAL.Proxies and cross-architecture candidates are excluded from the headline.Source: §6.3 · p. 9
• The application gate enforces requirements in code before execution, while the App Agent can edit only the candidate and its switch, not the driver, timer, oracle, original branch, or manifest.Both variants must build and run before timing begins.Source: §6.3 · p. 9

7.1 Setup

• Operator claims rely on kernel microbenchmarks, while application claims use a separate application gate; neither evidence set endorses the other.Headline numbers are measured on an A100, and application comparisons use each application's own GPU implementation.Source: §7 · p. 9, §7.1 · p. 9

7.2.3 A refuted tensor-core mapping: forging a neg-

• On A100, tensor-core mappings for box_2 remained slower than the winning CUDA-core branch, and the authors scope this ceiling to that architecture rather than ruling out every tensor-core formulation.The analysis attributes the ceiling to operand-feed limits from shared-memory bandwidth and MMA shapes; Table 3 compares the attempts with the CUDA-core branch.Source: Table 3 · p. 10, §7.2.3 · p. 10

7.2.4 Synthesis beats search: an equal-compute ab-

• The forge loop reaches 2.29×–2.73×, compared with 1.06×–1.28× for autotuning and 0.85×–0.87× for one-shot generation in the equal-compute ablation.The authors attribute the gain to measurement feedback followed by re-synthesis, rather than search or one-shot generation alone.Source: §7.2.4 · p. 10

7.2.5 The generality tax has an architectural axis:

• A100-forged production kernels transfer unchanged to H100 at a 1.81× geometric mean, while the library transfers at 1.51× across 41 B200 cells.Re-forging recovers headroom when the hardware moves a kernel off its binding constraint; the needed change can be a configuration adjustment or a structural rewrite.Source: §7.2.5 · p. 10

7.2.2 Library vs. per-case SOTA upper bounds.

• Across 37 cases, the library records 37 wins and no losses, with geometric-mean speedups of 2.35× over same-precision f32-vs-f32 baselines (n=13) and 1.95× over mixed-precision fp16-vs-f32 baselines (n=24).Source: §7.2.2 · p. 10

7.3 Application-Level Forging: The Specialization–Generality Trade-off

• Across 100 validated applications, end-to-end speedup has a median of 1.41× and a geometric mean of 2.05×.The distribution includes 11 applications below 1.05×, 46 at 1.05–1.5×, 22 at 1.5–3×, 14 at 3–10×, and 7 at ≥10×.Source: §7.3.1 · p. 11, Figure 6 · p. 11
• Fifty wins came from bespoke stencil kernels, 33 from forged non-stencil kernels, and 17 from host or structural rewrites; none reused an unchanged generic-library operator.The forged solutions also beat all 46 already hand-tuned application baselines, with a median speedup of 1.33×; realized gains anti-correlate with baseline quality.Source: §7.3.2 · p. 11, §7.3.3 · pp. 11–12

7.4.1 The protocol retracts single-dataset cherry-

• Every ablated measurement requirement distorted a real result; a wrong-answer box_2 measurement reported 13.2× instead of an honest 6.23×.Requiring the all-dataset geometric mean exposes single-dataset distortions; this rule binds for the five applications that ship multiple datasets.Source: §7.4.2 · p. 12, §7.4.1 · p. 12, Table 5 · p. 12

8 Conclusion

• 37 of 37 measured cases: the specialized kernel map beats per-case public-SOTA upper bounds.The conclusion attributes these wins to solutions forged for concrete cases, not a single generic method.Source: §8 · p. 12
• Across 100+ real applications, every reported win is a bespoke kernel or structural rewrite rather than a generic operator swap.The App Agent extends specialization from kernels to complete applications.Source: §8 · p. 12
• An agent-independent harness enforces correctness, fair measurement, and honest reporting; testing beyond stencil and broadening multi-dataset coverage remain next steps.The harness retracts favorable numbers when they fail its checks.Source: §8 · p. 12

A.1 Pure-kernel arbitration

• For star_1 at 128^3, the 0.87× wall-clock ratio is a timing artifact; symmetric pure-kernel arbitration yields a 1.63× win.The arbitration uses Nsight Compute pure-kernel time for cases distorted by fixed launch and timing overheads.Source: §A.1 · p. 14

A.2 Dispatch breadth

• Each benchmarked type × shape × precision configuration dispatches to a distinct kernel branch, with no cell reusing a shelf-general kernel.Figure 8 presents this as a dispatch view on A100.Source: Figure 8 · p. 14, §A.2 · p. 14

A.3 Per-case public-SOTA baseline identity

• Halide and Devito split the per-case baseline matrix without either dominating; EBISU wins only targeted temporal-blocking cells, and some cells lack a public baseline.The per-case method selects the strongest public system for each case rather than one fixed competitor.Source: Figure 9 · p. 14, §A.3 · p. 14

A.4 Tensor-core scope and relation to concurrent work

• The tensor-core refutation is bounded: dense packing hits an operand-feed wall, but the analysis does not rule out every possible tensor-core formulation.The operand-feed floor is ≈1.4–2.8 ms, above warp16’s measured 1.251 ms; the theoretical minimum density penalty is 2–3, near the stated 2×-win threshold.Source: §A.4 · p. 15

A.5 Synthesis-vs-search ablation

• The three equal-compute arms are launch-configuration autotuning over a fixed naive kernel, one-shot LLM generation without feedback, and iterative forging against the harness.All arms use the same CUDA-event timer and correctness gate on A100, f32, at 512^3.Source: §A.5 · p. 15, Table 6 · p. 15
• Autotuning reaches only 1.06×–1.28×, while one-shot LLM generation is slower than naive at 0.85×–0.87×.The passage attributes the forge loop’s gain to measurement feedback and re-synthesis rather than fixed-structure search or generation without feedback.Source: §A.5 · p. 15
• 2.29×–2.73×: the forge loop beats the naive kernel despite full dispatch and H2D overhead while other arms are timed on kernel launch alone.The caption characterizes the forge arm’s lead as a conservative lower bound.Source: §A.5 · p. 15, Table 6 · p. 15

A.6 Cross-architecture transfer (A100→H100→B200)

• On H100, A100-forged kernels transfer across 26 cubic cells at a 1.81× geometric-mean speedup, while re-forging recovers headroom when the binding constraint changes.The passage reports that the forge-arm speedups shrink for box_1 and diamond_1 on H100; re-forging can require a configuration change or structural rewrite.Source: §A.6 · p. 15
• On H100, forged kernels hold a 2.16× geometric mean across 17 cells with per-case public baselines and no losses.The reported edge over Halide widens from 2.16× to 2.38× overall; eight-step fused diamond_ts8 also wins against EBISU in stated cases.Source: §A.6 · p. 16
• On B200, carried kernels reach a 1.51× geometric mean across 41 cells against an HBM ratio of 2.39×; public baselines were not rebuilt on B200.The binding constraint shifts from bandwidth to latency and memory-level parallelism, and the reported B200 measurements are transfer and re-forge results, not an SOTA comparison.Source: §A.6 · p. 16

A.7 How the specialization emerges

• The autonomy trace is a lightweight characterization, not causal attribution, and includes disruptive refactors that run slower for several rounds before paying off.The described trajectory spans rounds in which later shape-specific branches emerge.Source: §A.7 · p. 16

A.8 Generalization within the library: unseen shapes

• 4.11× on cubic shapes and 4.04× on untuned edge shapes (ratio 0.98) show that the measured operator speedups do not collapse on those unseen shapes.The weakest cited case, box_1 on thin slabs, still beats SOTA at 2.70×, versus 3.42× on cubes.Source: §A.8 · p. 16

B Applications the Framework Rejected or Downgraded

• Alongside 100 validated integrations, 16 further candidates are recorded as non-success: 6 research, 5 out-of-scope, 3 dropped, and 2 blocked.The ledger does not force those candidates into an integrated result.Source: B Applications the Framework Rejected… · p. 16

• 58.39× for bspline_vgh/QMCPACK and 36.2× for haccmk/HACC come from launch-fusion and occupancy wins, not operator substitutions.Source: C Representative Benchmark Cases · p. 16
• 2.474× for fdtd_em/gprMax sits at the modest end and comes from pure stencil replacement.Source: C Representative Benchmark Cases · p. 16
• For multi-dataset applications, values are all-dataset geometric means; only minisweep uses one dataset.Source: C Representative Benchmark Cases · p. 16

D Experiment Manifest

• Table 8 maps each numbered experiment to its supporting claim and measured artifact; its E-numbers are cited throughout the paper.Source: D Experiment Manifest · p. 16, Table 8 · p. 17
• Table 7 indicates that large application speedups come from naive baselines and structural or launch-level rewrites, not faster generic stencils.Source: Table 7 · p. 17

E Evidence Map

• Table 9 assigns a validation gate to each result; kernel microbenchmarks and the application gate do not endorse each other's numbers.Source: E Evidence Map · p. 17, Table 9 · p. 17