PaperlayerView exact revision on arXiv

Source-linked AI summary

AMDKernelVault: Large-Scale Datasets and Agentic Training for AMD GPU Kernel Optimization

Ji Liu, Saptarshi Majumder, Yiqing Huang, Wenwen Ouyang, Umang Pandey, Zeping Li, Chushi Chen, Zihao An, Puyuan Yang, Zekai Li, Sina Rafati, Ziqiong Liu, Pratik Prabhanjan Brahma, Dong Li, Zicheng Liu, Sharon Zhou, Emad Barsoum

arXiv:2609.12471v1cs.CL

TL;DR

Kernel-agent data and workflows remain largely CUDA/NVIDIA-centric, while AMD kernel behavior requires ROCm-native compilation, correctness checks, and latency measurement. AMDKernelVault provides an execution-verified HIP/Triton corpus and trains a compact model on it; under fixed budgets, the model achieves the highest correctness on selected benchmarks.

  • Problem

    Existing kernel-agent data and tooling are largely CUDA/NVIDIA-centric, while AMD lacks large-scale execution-verified data across native HIP and Triton.

    Source: §1 · p. 1

  • Method

    AMDKernelVault builds HIP and Triton kernels from PyTorch references through AMD-native generation, compilation, correctness validation, and latency profiling, and trains Qwen3-8B with SFT and execution-aware RL.

    Source: §1 · pp. 1–2

  • Results

    Under the same evaluation budget, the trained 8B model achieves the highest correctness among compared models on selected benchmarks, including 34.0% Pass@1 on PyTorch-to-HIP.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The corpus and AMD-native generation and validation infrastructure provide a resource for AMD kernel research, while the training results demonstrate its utility for locally deployable kernel LLMs.

    Source: §1 · p. 2, §6 · p. 9

    Evaluation is limited to available AMD GPU environments and kernel-generation benchmarks, and reported performance depends on the ROCm, Triton, and hardware configurations used.

    Source: Limitations · p. 9

Abstract

from arXiv · show

We introduce AMDKernelVault, an open HIP and Triton kernel corpus and training framework for recent AMD CDNA GPUs. Existing LLM-based kernel agents are largely CUDA/NVIDIA-centric and often depend on repeated frontier-LLM calls for generation, reflection, and optimization. To address this gap, we develop HIPKernelGen and TritonKernelGen, agent-driven pipelines that transform PyTorch references into HIP or Triton kernels, compile and validate candidates under ROCm, and latency-profile them on AMD hardware. The corpus contains 62,153 execution-verified HIP kernel samples, 2,377 production-grounded ROCm Libraries QA entries, and 39,893 Triton kernels. We further train Qwen3-8B with supervised fine-tuning and execution-aware reinforcement learning as a demonstration of the corpus's utility. Under fixed evaluation budgets, it achieves the highest correctness among the compared models on PyTorch-to-HIP (34.0% Pass@1), TritonBench-G (33.2% Corr@3), and ROCmBench (41.94% Corr@3), but does not uniformly lead compilation or speed metrics. The corpus and documentation are available at https://huggingface.co/datasets/amd/AIG-Datasets, and the associated training and kernel-generation code is available at https://github.com/AMD-AGI/hip_kernel_llm_lab.

1 Introduction

AMD kernel research lacks large-scale, execution-verified HIP and Triton data, while existing agent workflows often depend on repeated frontier-model calls. AMDKernelVault pairs AMD-native data-generation pipelines with a compact-model training demonstration that achieves leading correctness on selected benchmarks.

1 Introduction

  • HIP and Triton kernel performance depends on AMD-specific compiler behavior, wavefront execution, LDS use, memory access, and backend constraints.Syntax conversion can retain NVIDIA-shaped assumptions, so reliable AMD data requires native compilation, execution-based correctness checks, and ROCm latency measurement.Source: §1 · p. 1
  • HIPKernelGen and TritonKernelGen generate kernels from PyTorch references, then compile, check correctness, and profile latency under ROCm.The pipelines use a generate–evaluate–reflect loop.Source: §1 · pp. 1–2
  • The paper trains Qwen3-8B with supervised fine-tuning and execution-aware reinforcement learning to serve as generator, reflector, and optimizer in a GEAK-style loop.The training recipe is an effectiveness test for the corpus and pipeline, not a new RL algorithm.Source: §1 · p. 2
  • 34.0% Pass@1 on PyTorch-to-HIP and 33.2% Corr@3 on TritonBench-G were the highest correctness scores among compared models under the same evaluation budget.The trained 8B model runs locally on AMD hardware; comparisons are specific to the reported metrics, benchmarks, and agent budgets.Source: §1 · p. 2
  • AMDKernelVault releases 64K+ HIP/ROCm data and approximately 40K Triton kernels, validated or supervised for recent AMD CDNA GPUs.The corpus includes benchmark-derived tasks, synthesized workloads, and production-grounded supervision.Source: §1 · pp. 1–2

2 Related Work

Prior kernel optimization and agent research has largely focused on Nvidia GPUs and CUDA. AMD’s distinct execution and compiler environment, along with limited AMD-validated kernel resources, motivates native HIP/ROCm validation and a unified AMD corpus.

GPU Kernel Optimization

  • Most GPU optimization methods have been developed and evaluated primarily on Nvidia GPUs and CUDA, leaving AMD-focused optimization comparatively underexplored.Earlier work includes fusion, tiling, memory-hierarchy optimization, custom kernels, and automated tuning.Source: §2 · p. 2

HIP, Triton, and ROCm

  • HIP and Triton portability abstractions do not guarantee AMD correctness or performance because ROCm execution differs in compiler behavior, wavefronts, LDS, coalescing, and scheduling.The paper therefore motivates native HIP/ROCm validation rather than post-hoc translation.Source: §2 · p. 2

GPU Kernel Datasets and Benchmarks

  • Existing kernel datasets remain CUDA-centric, and CASS does not provide large-scale PyTorch-to-HIP, HIP-to-HIP, or AMD Triton data.AMDKernelVault addresses this gap with AMD-validated HIP/Triton data and compact local policies; related agent systems include Kevin, AutoTriton, TritonRL, and GEAK.Source: §2 · pp. 2–3

3 Agent-Based Dataset Generation

The paper builds HIP and Triton kernel data through a shared agent-driven workflow that standardizes PyTorch references, synthesizes candidates, and validates them on AMD GPUs. It combines varied source datasets with production-grounded ROCm Libraries supervision and classifies the resulting data by implementation difficulty.

3.1 Unified Pipeline Architecture

  • The unified framework instantiates HIPKernelGen and TritonKernelGen as backend-specific pipelines within a shared generate–evaluate–reflect workflow.Both begin from standardized, function-style PyTorch references for consistent invocation and execution-based validation.Source: §3.1 · p. 3
  • Candidates are compiled, checked against PyTorch references, and latency-profiled; failures trigger reflector-guided regeneration until correctness or the attempt limit.Test inputs are augmented to cover shapes, strides, dtypes, and boundary conditions, and shape-specialized-only Triton candidates are discarded.Source: §3.1 · pp. 3–4

3.2 Source Data

  • The source data spans synthesized CUDA-agent tasks, GitHub-derived PyTorch workloads, curated Triton pairs, TritonBench-8k, and production ROCm libraries.GPUMODE-KernelBook supplies broad operator coverage; TritonBench-8k data is curated to remove DSL-specific artifacts and check correctness across shapes, strides, and precisions.Source: §3.2 · p. 4, §3.2.2 · p. 4, §3.2.5 · p. 4

3.3 Generated Kernel Corpus

  • HIP-CudaAgent adapts PyTorch tasks into function-style references, while HIP-GPUMode generates and profiles variants and HIP2HIP produces kernel-optimization pairs.HIP-GPUMode contains 5,910 unique entries with 22,397 variants; HIP2HIP yields 34,368 validated optimization pairs.Source: §3.3 · p. 5
  • The released corpus contains 62,153 validated HIP samples, 39,893 validated Triton–PyTorch samples, and 2,377 production-grounded ROCm Libraries QA entries.The HIP/ROCm total of 64,530 combines the execution-verified HIP samples and QA entries; separately released HIP-AICUDA data is excluded.Source: §3.3 · pp. 5–6

3.4 Production-Grounded ROCm QA Supervision

  • ROCm Libraries QA is interface-level supervision, not execution-verified kernel-pair data, grounded in rocBLAS and rocSOLVER code.The workflow traverses interface call graphs, controls scope with project-specific allowlists and disallowlists, and generates questions about kernel semantics, mappings, memory access, synchronization, and edge cases.Source: §3.4 · p. 6
  • The pipeline yields 1,858 rocBLAS entries and 519 rocSOLVER entries, complementing benchmark-derived data with production HIP patterns and host–device orchestration.The entries include both kernel-implementation samples and QA-style explanations.Source: §3.4 · p. 6

3.5 Difficulty Distribution

  • The corpus assigns kernels three difficulty levels: standalone single-function kernels, fused or single-file implementations, and multi-file kernels with cross-module dependencies.The full Triton corpus comprises 11,824 L1 (29.6%), 24,765 L2 (62.1%), and 3,304 L3 (8.3%) samples.Source: §3.5 · p. 6

4 Training Compact AMD Kernel LLMs

The training framework specializes Qwen3-8B for AMD kernel tasks using supervised fine-tuning followed by execution-aware reinforcement learning. One policy handles generation, reflection, and optimization, with HIP and Triton training using different RL interaction patterns.

4 Training Compact AMD Kernel LLMs

  • AMDKernelVault trains Qwen3-8B with supervised fine-tuning for cold-start generation, followed by execution-aware reinforcement learning using compilation, correctness, and latency feedback.Source: §4 · p. 6

4.1 Kernel Tasks and Metrics

  • The training tasks cover PyTorch-to-HIP, HIP-to-HIP, Text-to-Triton, and Triton-to-Triton kernel generation or optimization.Evaluation metrics distinguish compilation, numerical correctness, and speed; Comp@k and Corr@k report outcomes after k iterations, while Pass@k measures the chance of at least one correct sample.Source: §4.1 · pp. 6–7

4.2 Supervised Fine-Tuning

  • Supervised fine-tuning teaches kernel syntax, binding conventions, memory-access idioms, and common implementation patterns before RL.HIP examples pair a PyTorch task or source HIP kernel with a validated implementation; Triton examples use reasoning traces when available.Source: §4.2 · p. 7

4.3 Execution-Aware Reinforcement Learning

  • HIP RL uses single-turn GRPO, whereas Triton RL uses multi-turn generation, feedback, reflection, and revision.The reward combines compilation, correctness, and bounded speedup signals to provide feedback in a low-success regime.Source: §4.3 · p. 7

4.4 Agentic Integration

  • A single Qwen3-8B policy serves as generator, reflector, and optimizer, producing Triton candidates and interpreting feedback to emit revisions.The recipe evaluates the corpus's utility for compact AMD kernel policies rather than proposing a new RL algorithm.Source: §4.4 · p. 7

4.5 Training Configuration

  • Training runs on AMD MI-series GPUs, with HIP using single-turn GRPO and Triton using multi-turn on-policy reflection RL with curriculum scheduling.Further objectives, reward implementation, schedules, hyperparameters, and sensitivity analyses are described in Appendix E.Source: §4.5 · p. 7

5 Experiments

The experiments compare corpus-trained Qwen3-8B with frontier models on held-out HIP and Triton tasks under controlled agent budgets. It achieves the strongest reported correctness on HIP translation and both Triton benchmarks, while expert Triton kernels and semantic-overlap controls bound the findings.

5 Experiments

  • Evaluation uses held-out task sets and a fixed three-iteration Triton agent scaffold, covering diverse kernel patterns but not reporting per-family pass rates.HIP tasks include 200 KernelBench translations and 100 HIP-to-HIP optimizations; Triton sets contain 184 TritonBench-G and 31 ROCmBench kernels.Source: §5 · pp. 7–8
  • Exact task-ID filtering does not rule out every functional near-duplicate, so results support benchmark-specific—not overlap-free generalization—claims.The authors did not conduct an exhaustive corpus-wide semantic-overlap search.Source: §5 · p. 8

5.1 HIP Kernel Results

  • 34.0% correctness for trained Qwen3-8B on PyTorch-to-HIP outperforms all frontier baselines; SFT reaches 31.5%.Table 2 reports Pass@1 on 200 KernelBench Level-1/2 translation tasks.Source: Table 2 · p. 8, §5.1 · p. 8
  • 84% correctness on HIP-to-HIP optimization approaches GPT-5’s 87% and exceeds the base model’s 18%, while maintaining positive speedup.Table 3 evaluates 100 held-out modules.Source: Table 3 · p. 8, §5.1 · p. 8

5.2 Triton Kernel Results

  • 33.2% Corr@3 on TritonBench-G exceeds Claude Sonnet 4 (29.8%) and GPT-5 (15.2%) under the fixed evaluator; D5 remains unsolved by all models.The model leads frontier baselines on D1–D3, while Claude Sonnet 4 solves more D4 kernels (18/84 versus 13/84).Source: Table 4 · p. 9, §5.2 · p. 9
  • Expert-kernel failures include unusual-shape tile choices, multi-stage reduction synchronization, and memory coalescing under AMD wavefront execution.The authors suggest richer profiler-guided feedback and broader expert-kernel supervision as future work.Source: §5.2 · p. 9

5.3 Ablation Studies

  • More training data improves performance, SFT provides a critical initialization before RL, and multi-turn RL contributes +13.1% on Triton.Detailed ablation tables and reward-sensitivity results are in Appendix J.Source: §5.3 · p. 9

5.2 Triton Kernel Results

  • 41.94% Corr@3 on ROCmBench exceeds Claude Sonnet 4 (35.48%) and GPT-5 (29.03%), but Claude leads compilation and speed.Claude Sonnet 4 reaches 67.74% Comp@3 and 1.82× speed, versus Qwen3-8B at 58.06% and 1.61×.Source: Table 5 · p. 9, §5.2 · p. 9

6 Conclusion

AMDKernelVault provides an open HIP and Triton resource for recent AMD CDNA GPUs, backed by AMD-native generation and validation infrastructure.

6 Conclusion

  • AMDKernelVault releases an open HIP and Triton resource for recent AMD CDNA GPUs, supported by AMD-native generation and validation infrastructure.The Qwen3-8B demonstration achieves the highest correctness on selected benchmarks under fixed budgets, but does not uniformly lead compilation or speed.Source: §6 · p. 9

Limitations

The evaluation is limited to AMD GPU environments and available kernel-generation benchmarks, and reported performance depends on the validation configuration.

Limitations

  • Evaluation covers AMD GPU environments and the kernel-generation benchmarks available to the authors.Expert-level kernels remain difficult across all tested models.Source: Limitations · p. 9
  • Reported performance depends on the specific ROCm, Triton, and hardware configurations used during validation.Some generated kernels may inherit suboptimal structures from upstream CUDA/Triton sources or LLM-generated candidates.Source: Limitations · p. 9
  • Future work calls for broader profiling, more held-out HIP evaluation, and evaluation beyond deep-learning workloads.The proposed extensions include latency, memory footprint, register pressure, occupancy, per-operator pass rates, HPC, sparse/graph, and database kernels.Source: Limitations · p. 9
  • The paper reports neither a corpus-wide semantic near-duplicate audit nor uniformly aggregated candidate-rejection rates because source pipelines use different logging formats.Software environments also vary across data-production snapshots, although the reported Triton workflow fixes Triton at version 3.3.0.Source: Limitations · p. 9

Ethical Considerations and Impact Statement

The paper describes the work’s expected impacts and validation safeguards, documents released artifacts and provenance, and details its AMD kernel data sources and construction.

Ethical Considerations and Impact Statement

  • The expected benefits include improved AI-workload efficiency and potentially lower computational cost and energy use for AMD GPU deployments.Generated code may be incorrect, inefficient, or unsafe without validation; the authors emphasize compilation, correctness testing, and latency profiling and recommend target-environment validation before production use.Source: Ethical Considerations and Impact… · p. 10

A Artifact, License, and AI-Assistance Statement

  • Released artifacts include code, execution metadata, task data, and public-ROCm-code-grounded QA, with filtering for executability, determinism, and task validity.The release documents provenance, usage conditions, and per-source licenses; AI assistants generated candidates and reflection feedback, while authors checked and approved the experimental design, analysis, and claims.Source: A Artifact, License, and AI-Assistance… · p. 12

B More Related Work

  • Production rocBLAS and rocSOLVER code contributes interface-level supervision spanning files and reflecting production engineering practices.The paper distinguishes its AMD HIP/Triton setting and locally trained agent policy from prior multi-turn feedback work, without claiming that multi-turn feedback alone is new.Source: B More Related Work · p. 12

C.1.1 HIP-CudaAgent Dataset

  • HIP-CudaAgent normalizes PyTorch tasks, generates HIP candidates, and iteratively retries compilation, runtime, or numerical failures; it yields 5,388 validated samples.Candidates are compiled with ROCm and tested against the PyTorch reference.Source: §C.1.1 · p. 13

C.1.2 HIP-GPUMode Dataset

  • HIP-GPUMode converts PyTorch modules to function-style references, then generates, validates, profiles, and iteratively refines HIP kernels, yielding 5,910 modules and 22,397 variants.The reported average is 3.8 variants per entry.Source: §C.1.2 · p. 13

C.1.3 HIP-to-HIP Optimization Pairs

  • HIP-to-HIP optimization operates on existing HIP functions and produces semantically equivalent optimized kernels, yielding 34,368 validated pairs.The input is a base-line HIP kernel; the output is an optimized kernel with equivalent semantics.Source: §C.1.3 · p. 13

C.2.1 Stack-v2-dedup (curated); C.2.2 TritonBench-train (modified)

  • TritonKernelGen curates Stack-v2 data into 2,269 medium-difficulty correct Triton–PyTorch pairs from about 13,000 candidates and expands TritonBench tests across shapes, strides, and precision types.The TritonBench curation includes 7,713 samples and removes DSL-specific artifacts and input-specific patterns.Source: §C.2.1 · p. 13, §C.2.2 · pp. 13–14

C.2.4 AI-CUDA-Engineer-Archive; C.2 Triton Kernel Dataset

  • The AI-CUDA-Engineer-Archive provides 11,911 usable PyTorch–CUDA pairs, with Triton synthesized against shared references when absent; the combined Triton corpus has 39,893 validated samples.Available metadata varies by source subset and representation; where present, it records construction-time validation outcomes, latency, and precision/shape coverage.Source: §C.2.4 · p. 14

C.3 ROCm Libraries QA Dataset

The ROCm Libraries QA dataset provides production-grounded supervision from rocSOLVER and rocBLAS; its difficulty profile includes substantial interface-level material.

C.3 ROCm Libraries QA Dataset

  • Interface-level extraction from AMD ROCm Libraries yields 519 rocSOLVER entries and 1,858 rocBLAS entries, totaling 2,377 production-grounded supervision entries.The entries include kernel implementations and QA explanations grounded in production HIP patterns, interface-driven logic, and host–device orchestration.Source: §C.3 · p. 14

C.4 Difficulty Distribution

  • The ROCm Libraries QA subset contains a larger fraction of Level 3 examples than the Triton corpus, which is dominated by Level 2 examples.Level 3 includes multi-file or interface-level code with cross-module dependencies; production interfaces often involve host-side orchestration and multi-file kernel dispatch.Source: §C.4 · p. 14

D Dataset Analysis

  • The analysis examines dataset scale, operator coverage, diversity, and architecture-relevant properties.Source: D Dataset Analysis · p. 14

D.1 Dataset Summary

AMDKernelVault combines AMD-validated HIP and Triton corpora with training and evaluation analyses spanning kernel generation, optimization, and agentic learning. The supplied results describe benchmark performance, corpus construction and validation, and execution-aware training choices.

D.1 Dataset Summary

  • The corpus combines 64K+ HIP/ROCm entries with approximately 40K Triton kernels across PyTorch-to-kernel generation, kernel optimization, and production-grounded code understanding.HIP/ROCm sources include HIP-CudaAgent, HIP-GPUMode, HIP2HIP, and ROCm Libraries QA; Triton-Curated combines four complementary sources.Source: §D.1 · p. 14

D.2 Operator and Workload Coverage / D.3 Kernel Diversity and Optimization Variants

  • Real-program workloads cover common deep-learning operators and production linear-algebra primitives, while multiple implementations expose different optimization choices and performance tradeoffs.ROCm Libraries QA includes rocBLAS and rocSOLVER examples; HIP-GPUMode and Triton-Curated provide diverse implementations for overlapping operators.Source: §D.2 · p. 14, §D.3 · p. 15

D.4 Data Quality and Bias Mitigation

  • Curation filters CUDA-derived seeds through ROCm compilation and testing, re-optimizes variants on AMD hardware, and includes production HIP examples to reduce NVIDIA-shaped assumptions.Triton samples are refactored and tested for shape generality, stride awareness, and AMD-compatible scheduling.Source: §D.4 · p. 15

D.5 Portability and Correctness Outcomes / D.6 Validation Metadata / D.7 Architecture-Relevant Properties (Triton)

  • Released HIP and Triton kernels pass AMD execution-based correctness checks, but validation metadata varies by subset and may not reconstruct every record’s validation trajectory.The Triton corpus also has greater scheduling-parameter diversity and fewer hardware-tied constants than raw crawled or DSL-generated code.Source: §D.5 · p. 15, §D.6 · p. 15, §D.7 · p. 16

D.8 Comparison with Existing GPU Kernel Datasets / D.9 Implications for Learning-Based Kernel Research

  • Compared with CUDA-focused resources, the AMD-validated HIP and Triton datasets support research on AMD kernel optimization, cross-platform transpilation, and heterogeneous GPU performance modeling.Their real workloads, executable kernels, and diverse variants support supervised fine-tuning, reinforcement learning, and agent-based synthesis under realistic hardware constraints.Source: §D.8 · p. 16, §D.9 · p. 16

E.2 Reward Design / E.6 Reward Sensitivity and Design Rationale

  • Execution-aware RL combines compilation, correctness, and bounded speedup rewards; ablations report lower performance when multi-turn reflection, SFT cold start, or hierarchical rewards are removed.The reward design supplies partial compilation credit in a low-success regime, while disabled variance normalization addresses sparse or clustered rewards.Source: §E.6 · p. 17

F Evaluation Details

  • 33.2% Corr@3 on TritonBench-G and 41.94% to 35.48% on ROCmBench: Q3-8B+SFT+RL improves from 22.58% to 41.94%, versus Claude Sonnet 4’s 19.35% to 35.48%.On TritonBench-G, the comparison is 12.0% to 33.2% for Q3-8B+SFT+RL, 7.6% to 29.8% for Claude Sonnet 4, and 3.8% to 15.2% for GPT-5.Source: F Evaluation Details · pp. 17–18

E.4 Multi-Turn On-Policy Reflection / G.3 Curriculum Learning Phases (Triton) / H Training Dynamics / F Evaluation Details

  • Triton training uses multi-turn generate–evaluate–reflect updates and difficulty fading; training dynamics show reward rising from approximately 0.19 to peaks above 1.0 with benchmark gains.The evaluation uses execution-intensive task sets but does not establish performance over every AMD operator or workload family.Source: §E.4 · p. 17, F Evaluation Details · p. 17, H Training Dynamics · p. 18

I Additional Experimental Analysis

The additional experiments examine local deployment and transfer across AMD hardware, alongside recurring kernel failure modes. The supplied excerpt introduces ablation studies but does not report their outcomes.

I Additional Experimental Analysis

  • The trained Qwen3-8B model runs locally on AMD GPUs and serves as generator, reflector, and optimizer, unlike frontier-LLM pipelines that require repeated external calls.Those external calls introduce API cost, latency variance, and deployment constraints.Source: I Additional Experimental Analysis · p. 18
  • 1.68× Speed@10 on MI250: kernels optimized on MI325 retain substantial gains when evaluated on MI250.Triton evaluations on MI325 and MI350 show similar qualitative trends, and the passage attributes transfer to learned locality, tiling, and parallel-reduction patterns.Source: I Additional Experimental Analysis · pp. 18–19
  • Observed failures include shape-specialized indexing, numerically unstable reductions, and inefficient layouts inherited from CUDA/NVIDIA assumptions.The passage identifies profiler-guided feedback and additional production AMD kernels as future extensions.Source: I Additional Experimental Analysis · p. 19

J.1 Data Scale (HIP)

The ablations and analyses report gains from training scale and agentic training, while examining sparse reward behavior and multi-turn recovery on difficult kernels. They connect execution feedback and reward design to compact-model training outcomes.

J.1 Data Scale (HIP)

  • With 10% of training data, the model reaches 62.5% compilation and 17.5% correctness, exceeding the base model’s 50.0% and 14.5%.The analysis covers 200 PyTorch-to-HIP tasks from KernelBench Levels 1 and 2.Source: §J.1 · p. 19, Table 10 · p. 19

J.2 Triton RL Components

  • Multi-turn reflection contributes +13.1%, while SFT cold start contributes +24.7% in the Triton component ablations.The passage says three-turn reflection enables refinement after initial failures, while SFT raises initial success from approximately 15% to approximately 51%.Source: §J.2 · p. 19, Table 11 · p. 19

J.3 Fixed Agent Scaffold Ablation

  • Under a fixed GEAK OptimAgent-v2 scaffold, the large gain from base Q3-8B to Q3-8B+SFT+Agentic RL indicates the scaffold alone is insufficient.Only the model checkpoint changes across the stated comparison; the trained policy uses compiler, correctness, and performance feedback.Source: §J.3 · pp. 19–20, Table 12 · p. 20

K.1 Pass@K Analysis (Parallel Scaling)

  • 41.94% pass@5 on ROCmBench: the model leads Claude Sonnet 4’s 38.71% on the 31-kernel comparison.The table reports pass@k, the probability of generating at least one correct kernel within k samples.Source: §K.1 · p. 20, Table 14 · p. 21

K.2 Full Difficulty Breakdown / K.3 Training Progression

  • 33.2% after RL, versus 10.3% after SFT and 4.9% for the base model, shows a large training progression on the reported breakdown.The complete difficulty-wise table includes all model variants.Source: §K.3 · p. 20, §K.2 · p. 20, Table 15 · p. 22

L.1 Zero-Advantage Collapse in GRPO / L.1.1 Binary Reward Collapse (Motivation) / L.1.2 Our Solution: Hierarchical Rewards + Disabled Normalization

  • Over 60% of training batches produce zero gradients under the stated binary-reward collapse analysis for typical kernel-generation success rates.The analysis motivates hierarchical rewards and disabled variance normalization to provide partial-success signal and avoid unstable gradients.Source: §L.1.1 · pp. 20–21, §L.1.2 · p. 21

L.2 On-Policy Reflection / L.3 Multi-Turn Credit Assignment

  • With discount factor γ = 0.6 over T = 3 turns, normalized credit assigns approximately 51.0% of the weight to Turn 1, encouraging early success while still rewarding improvement through reflection.Source: §L.3 · p. 21

M Multi-Turn Self-Correction Examples / M.1 Example 1: Cross-Attention (Medium-Hard) / M.2 Example 2: PreNorm Transformer Decoder (Hard) / M.4 Summary: Recovery Statistics

  • The examples report recovery from unstable softmax in one turn at 2.17× speedup and from oversized layer normalization in two turns at 9.13×.The summary passage describes learned handling of multi-pass reductions, numerical stability, arbitrary tensor dimensions, and fused operations.Source: §M.1 · p. 22, §M.2 · p. 23, §M.4 · p. 23

M.5 Kernel Quality Analysis

Across five medium-to-hard case studies, the agent-trained model produced correct kernels with speedups over PyTorch; code inspection found workload-specific strategies and feedback-targeted revisions. The examples cover multiple shape, stride, and dtype combinations, while full-corpus latency and memory studies remain future work.

M.5 Kernel Quality Analysis

  • 2.17×, 9.13×, 5.18×, 6.79×, and 4.22× speedups over the PyTorch reference yielded a geometric mean of ≈4.94× and median of 5.18×.These results cover five medium-to-hard case studies.Source: §M.5 · p. 23
  • Generated strategies varied by workload, including multi-pass safe softmax, fused layer-norm with running statistics, and masked tile loads with tl.exp stabilization.After failures, revisions typically targeted the reported error class, such as adding numerical stabilization after NaNs or changing reduction strategy after out-of-bounds errors.Source: §M.5 · p. 24
  • Each case was validated across multiple shape, stride, and dtype combinations, but a full-corpus latency distribution and memory-consumption study is left for future work.Source: §M.5 · p. 24
Loading 2609.12471v1…