Less is more: learning what to profile for GPU kernel optimization

1 INTRODUCTION

• Existing systems typically collect predefined profiling signals, despite profiling being costly and its utility varying across kernels.This leaves the choice of what to profile outside the optimization process.Source: §1 · p. 1
• Increasing the number of profiling metrics does not monotonically improve kernel optimization, while useful profiling information differs across kernels.Theoretical analysis shows kernel-conditioned selection can outperform any single fixed configuration.Source: §1 · pp. 1–2
• AutoProfiler represents profiling as a deterministic tree that selects later measurements based on earlier profiling results.An LLM proposes REWRITE and SPLIT modifications, and empirical paired evaluation determines whether to accept them.Source: §1 · p. 2, Figure 1 · p. 2
• AutoProfiler improves kernel optimization over fixed profiling configurations under constrained budgets and remains effective across iterative optimization rounds.Its policy recovers a substantial portion of the gain available from kernel-conditioned selection, without repeated LLM-based profiling decisions.Source: §1 · p. 2

2 PRELIMINARY

• A profiler P produces information I from kernel k and metric subset m, which the rewriter g uses to generate an optimized kernel.The formulation denotes this output as the optimized kernel ˆk.Source: §2 · p. 2
• The profiler, rewriter, and target hardware are fixed throughout the work.The policy chooses profiling information; it does not change these components.Source: §2 · p. 2
• Existing approaches use the same predefined metric set for every kernel, whereas this formulation allows profiling metrics to vary by kernel.The optimized kernel is generated using both the original kernel and acquired profiling information.Source: §2 · p. 2
• The policy ϕ(k) selects a subset of profiling metrics for kernel k, and determines what additional information to acquire from observations so far.This makes profiling a sequential reasoning process over hardware signals.Source: §2 · p. 3
• The objective is to maximize expected optimization reward, which is positive only when profiling-guided rewriting improves on the original kernel.The reward uses resulting speedup, defined from original-kernel and optimized-kernel latency; profiling is subject to a budget.Source: §2 · p. 3

Theoretical Analysis

• Dynamic selection never performs worse than the best fixed configuration, and strictly outperforms it when no single configuration covers the candidate success regions almost surely.The advantage depends on heterogeneity in profiling utility across kernels.Source: §3 · p. 3

Empirical Validation

• The oracle evaluation applies the same frozen optimizer to each of 128 kernels under four roofline-based profiling configurations, selecting the highest-speedup configuration per kernel.Detailed experimental settings are provided in Appendix A.2.Source: §3 · pp. 3–4
• 1.429× geomean speedup for per-kernel oracle selection versus 1.332× for the best fixed configuration, a 7.3% relative gain on 128 KernelBench kernels.The oracle selects different configurations across kernels, with the best configuration varying substantially among four candidates.Source: Figure 2 · p. 3, §3 · p. 4
• The oracle is an analysis reference, not an inference-time policy, because it requires evaluating every candidate configuration for each kernel.This motivates learning which configuration to use without exhaustive evaluation.Source: §3 · p. 4

4.1 TREE-BASED PROFILING POLICY

• The tree starts with SM utilization and DRAM throughput, routing kernels into latency-, memory-, compute-, or algorithm-bound regimes before finer-grained selection.Internal nodes route on profiling information; leaves determine the final metric configuration for kernels reaching them.Source: §4.1 · p. 5
• A deterministic tree routes each kernel through profiling conditions, collecting metrics until a leaf selects the kernel-specific profiling information for one LLM rewriter call.The policy treats profiling as sequential: later measurements depend on observations already collected, and profiling stops when no additional information is needed.Source: §4.1 · pp. 4–5

4.2.1 LEAF-LEVEL ATTRIBUTION AND STRUCTURAL UPDATES

• Leaf-level profiling observations and downstream optimization rewards guide REWRITE, which adds metrics, or SPLIT, which separates kernels with different profiling needs.An LLM proposes added metric groups or feature-threshold pairs; empirical evaluation determines whether to incorporate each candidate.Source: §4.2.1 · pp. 5–6

4.2.2 CONSERVATIVE EMPIRICAL ACCEPTANCE

• Candidate trees are compared with the incumbent on the same training kernels and accepted only when their paired downstream rewards exceed a conservative margin.REWRITE is tried before SPLIT; rejected changes leave the incumbent unchanged, and the profiling budget limits tree complexity and information collected per kernel.Source: §4.2.2 · p. 6

5.1 EXPERIMENTAL SETUP

• The profiling tree is constructed on CUDA-Agent-Ops-6K without KernelBench exposure, making KernelBench an out-of-distribution evaluation of transfer to unseen kernels.CUDA-Agent-Ops-6K contains 6,000 synthesized operator-level tasks; KernelBench spans individual operators, fused operators, and complete models.Source: §5.1 · pp. 6–7
• Evaluation compares AutoProfiler with four training-free kernel-optimization agents using execution accuracy and geometric-mean speedup relative to reference kernels.The baselines use their original static NCU configurations, while AutoProfiler dynamically selects profiling metrics.Source: §5.1 · p. 7

5.2 MAIN RESULTS

• 1.704× average speedup on KernelBench exceeds CudaForge’s 1.570× by 8.5%, with Level 3 speedup rising from 1.099× to 1.352×.AutoProfiler also reaches 87.2% execution accuracy, 2.4–8.8 percentage points above other baselines, while requiring one LLM call per task versus two for CudaForge and KernelMem and four for KernelAgent.Source: §5.2 · p. 7

5.3.1 COMPARISON OF PROFILING FEEDBACK STRATEGIES

• AutoProfiler achieves 1.379× speedup versus 1.197× for Blind and also outperforms Random and Static feedback.The remaining gap to Oracle leaves room to improve profiling selection.Source: Table 2 · p. 8, §5.3.1 · p. 8

5.3.2 MORE PROFILING IS NOT ALWAYS BETTER

• Geomean speedup is non-monotonic in metric count: four metrics yield 1.370×, versus 1.299× with two and 1.261× or 1.231× with six or eight.Both metric scarcity and redundant feedback degrade performance relative to the four-metric setup.Source: Table 3 · p. 8, §5.3.2 · p. 8

5.3.4 ITERATIVE KERNEL REFINEMENT

• AutoProfiler reaches 2.112× after seven rounds, while CudaForge reaches 1.846× after four rounds under a similar cumulative time budget.The comparison measures cumulative wall-clock time per sample; AutoProfiler also requires less time per iteration.Source: Figure 3 · p. 9, §5.3.4 · p. 9

5.3.3 GENERALIZATION ACROSS LLM BACKBONES

• With the profiling tree frozen, Claude-Sonnet-5 reaches 1.47× geomean speedup and 91.7% execution accuracy, and the policy also guides optimization across other model families.The tree is trained using GPT-5.6-Sol feedback; only the leaf rewriting LLM is swapped at test time.Source: §5.3.3 · pp. 8–9, Table 4 · p. 9

5.4 ABLATION STUDY

• Removing REWRITE lowers geomean speedup from 1.61× to 1.47×, while removing SPLIT lowers it to 1.53×.The full policy combines both operators and reaches 1.61×.Source: §5.4 · p. 9
• Without SPLIT, kernels with distinct bottlenecks share a profiling setup because the tree cannot separate them into specialized branches.The authors describe SPLIT as grouping kernels into specialized hardware regimes.Source: §5.4 · p. 9
• Without REWRITE, each leaf provides only basic metric feedback, leaving the downstream LLM without sufficient hardware context for fine-grained optimization.The authors describe REWRITE as enriching profiling details within each regime.Source: §5.4 · p. 9

6 CONCLUSION

• AutoProfiler learns a deterministic, kernel-conditioned profiling policy by iteratively constructing a tree with REWRITE and SPLIT operations.The framework is motivated by the observation that less profiling can yield better optimization.Source: §6 · p. 10
• On KernelBench, AutoProfiler outperforms traditional static profiling configurations and strong agentic baselines under constrained profiling budgets.The conclusion also reports minimal LLM invocation overhead.Source: §6 · p. 10
• The authors frame hardware profiling as a learnable decision process and a direction for future AI-driven systems optimization.They describe this shift as moving from static metric collection toward profiling reasoning.Source: §6 · p. 10

A.1 RELATED WORK

• Compiler frameworks expanded program search and tensor abstractions, but generally use rigid, template-driven spaces and passive or fixed hardware metrics.The passage cites TVM, AutoTVM, Ansor, MetaSchedule, FlexTensor, and TensorIR.Source: §A.1 · p. 11
• Recent LLM-driven kernel systems use generative models for GPU-kernel synthesis and iterative optimization, while agentic frameworks integrate execution and runtime feedback into iterative search loops.CudaForge (Zhang et al., 2025) and AutoKernel (Jaber & Jaber, 2026) apply iterative compilation and error-repair heuristics; KernelAgent and Astra (Wei et al., 2025) employ multi-agent orchestration for kernel refactoring.Source: §A.1 · p. 11
• Prior agentic frameworks treat profiling as invariant preprocessing or static feedback, despite profiling cost and kernel-varying metric utility.The paper identifies these properties as motivation for adaptive profiling.Source: §A.1 · p. 12
• Existing profiling methods either collect a universal metric subset or use hand-crafted rules; AutoProfiler instead learns a deterministic, kernel-conditioned policy from optimization rewards.The paper calls this approach profiling reasoning.Source: §A.1 · p. 12

• Four fixed candidate configurations capture algorithmic behavior, memory utilization, latency and stalls, or compute utilization.The candidates are Algorithm, Memory, Latency, and Compute.Source: §A.2 · p. 12
• Each candidate was evaluated on 128 sampled kernels with a frozen rewriter and identical optimization, hardware, and random-seed settings.The recorded outcome was speedup relative to the PyTorch reference implementation.Source: §A.2 · p. 12
• The static baseline maximizes benchmark-wide geometric mean speedup, whereas the empirical oracle selects the best configuration separately for each kernel.Source: §A.2 · p. 12
• Across 128 kernels, the oracle’s relative gain had a 95% confidence interval of [1.045×, 1.107×]; Memory, Latency, Algorithm, and Compute led on 31, 27, 26, and 23 kernels, respectively.Source: §A.2 · p. 13
• No fixed configuration dominates across heterogeneous workloads, and tailoring profiling signals to kernel bottlenecks offers substantial optimization headroom.Source: §A.2 · pp. 12–13
• 1.332× geometric mean speedup for the strongest fixed baseline versus 1.429× for the per-kernel oracle, a 7.3% relative performance improvement.Table 6 reports the comparison on 128 KernelBench kernels using the same frozen optimizer for each candidate configuration.Source: §A.2 · p. 12, Table 6 · p. 13