PaperlayerView exact revision on arXiv

Source-linked AI summary

KernelEvolve: Scaling Agentic Kernel Coding for Heterogeneous AI Accelerators at Meta

Gang Liao, Hongsen Qin, Ying Wang, Alicia Golden, Michael Kuchnik, Yavuz Yetim, Jia Jiunn Ang, Chunli Fu, Yihan He, Samuel Hsia, Zewei Jiang, Dianshi Li, Uladzimir Pashkevich, Varna Puvvada, Feng Shi, Matt Steiner, Ruichao Xiao, Liyuan Li, Nathan Yan, Xiayu Yu, Zhou Fang, Roman Levenstein, Kunming Ho, Haishan Zhu, Alec Hammond, Richard Li, Ajit Mathews, Kaustubh Gondkar, Abdul Zainul-Abedin, Ketan Singh, Hongtao Yu, Wenyuan Chi, Barney Huang, Sean Zhang, Noah Weller, Zach Marine, Wyatt Cook, Carole-Jean Wu, Gaoxiang Liu

arXiv:2512.23236v4cs.LGcs.AIcs.ARcs.MAcs.PF

TL;DR

Recommendation workloads span diverse model architectures, kernel primitives, and accelerators, while existing kernel-generation prototypes have limited workload and platform coverage. KernelEvolve uses graph-based search and retrieval-augmented prompting to generate and optimize kernels; the paper reports 100% correctness across 480 operator-platform configurations and production speedups of 1.25–17×.

  • Problem

    Recommendation workloads span diverse models, kernel primitives, and heterogeneous accelerators, while existing prototypes are limited in workload realism and cross-platform synthesis.

    Source: §1 · p. 3, §1.1 · p. 7

  • Method

    KernelEvolve treats kernel optimization as graph-based search, using selection, transformation, and fitness evaluation to iteratively generate candidates.

    Source: §3.1 · p. 12

  • Results

    100% correctness across 480 operator-platform configurations and 1.25–17× speedups on production workloads are reported for KernelEvolve.

    Source: §8 · p. 40

  • Takeaways & Limitations

    KernelEvolve’s MTIA kernels provide on-device execution where native operator coverage is limited and deliver 2–3× speedups on MTIA v3.

    Source: §5.4.3 · p. 36

    The paper characterizes existing prototypes as relying on synthetic static-shape workloads and mostly homogeneous NVIDIA targets, limiting their production and cross-platform scope.

    Source: §1.1 · p. 7

Abstract

from arXiv · show

Making deep learning recommendation model (DLRM) training and inference fast and efficient is important. However, this presents three key system challenges - model architecture diversity, kernel primitive diversity, and hardware generation and architecture heterogeneity. This paper presents KernelEvolve-an agentic kernel coding framework-to tackle heterogeneity at-scale for DLRM. KernelEvolve is designed to take kernel specifications as input and automate the process of kernel generation and optimization for recommendation model across heterogeneous hardware architectures. KernelEvolve does so by operating at multiple programming abstractions, from Triton and CuTe DSL to low-level hardware agnostic languages, spanning the full hardware-software optimization stack. The kernel optimization process is described as graph-based search with selection policy, universal operator, fitness function, and termination rule, dynamically adapts to runtime execution context through retrieval-augmented prompt synthesis. We designed, implemented, and deployed KernelEvolve to optimize a wide variety of production recommendation models across generations of NVIDIA and AMD GPUs, as well as Meta's AI accelerators. We validate KernelEvolve on the publicly-available KernelBench suite, achieving 100% pass rate on all 250 problems across three difficulty levels, and 160 PyTorch ATen operators across three heterogeneous hardware platforms, demonstrating 100% correctness. KernelEvolve reduces development time from weeks to hours and achieves substantial performance improvements over PyTorch baselines across diverse production use cases and for heterogeneous AI systems at-scale. Beyond performance efficiency improvements, KernelEvolve significantly mitigates the programmability barrier for new AI hardware by enabling automated kernel generation for in-house developed AI hardware.

1 Introduction

KernelEvolve addresses the diversity of recommendation models, kernel primitives, and accelerator architectures that makes manual kernel development difficult to scale. It automates kernel generation and optimization across heterogeneous hardware, reporting production speedups and shorter development times.

1.1 The Curse of Dimensionality

  • Model, kernel, and hardware diversity creates a combinatorial scaling challenge that manual kernel development cannot address.Hardware implementations require 2–8 weeks of expert optimization, and hardware-generation updates every 12–18 months can invalidate existing optimizations.Source: §1.1 · pp. 3–4
  • Missing accelerator preprocessing kernels can force disaggregated serving, adding network latency and complicating production deployment.A production MTIA comparison reports P99 latency of 61 ms for client-side preprocessing versus 97 ms with a dedicated CPU tier.Source: §1 · p. 3, §1.1 · p. 6
  • KernelEvolve automates synthesis, compilation, profiling, correctness verification, and benchmarking, targeting Triton and Triton-TLX across NVIDIA, AMD, and MTIA.Triton-TLX supports low-level hardware-specific tuning while retaining Triton’s portability advantages.Source: §1.1 · pp. 7–8
  • 1.2–17× speedups over PyTorch baselines, with development time reduced from weeks to hours, summarize KernelEvolve’s production results.The reported workloads include LLM inference, convolutional transformers, data preprocessing, ranking, MTIA-specific, and retrieval kernels.Source: §1.1 · pp. 7–8, Figure 4 · p. 8
  • KernelEvolve is presented as a production-deployed system for recommendation training and inference, with kernel-generation workflows and production-safety insights.The paper describes deployment across hundreds of models and discusses debugging, performance validation, and organizational integration.Source: §1.1 · p. 8

2 Background

Recommendation systems span distinct ranking stages and model architectures, creating diverse kernel needs; preprocessing adds irregular, latency-sensitive operations whose coverage is a first-order requirement.

2.1 Meta’s Recommendation Model

  • Meta products rely on DLRMs and multi-stage recommendation pipelines that process trillions of daily inferences.Source: §2.1 · p. 8
  • Retrieval, early-stage ranking, and late-stage ranking differ in candidate counts, model complexity, batch sizes, and latency demands.Late-stage models reach up to 2 GFLOPS per sample, with over 60× complexity variation.Source: §2.1 · pp. 8–9
  • Recommendation architectures combine sparse embeddings, dense transformations, and feature interactions, with sequence and generative models adding new computational patterns.HSTU and InterFormer introduce sequence modeling; generative systems add autoregressive decoding, token-based retrieval, and quantization operations.Source: §2.1 · p. 9
  • These architectural differences create specialized kernel requirements, from high-throughput retrieval to jagged sequence operations and generative primitives.The resulting operator landscape requires hundreds of optimized kernel implementations.Source: §2.1 · p. 9

2.2 Data Preprocessing Operations

  • Inference preprocessing runs synchronously with model invocation, so its latency affects end-to-end response time and comprehensive kernel coverage is a first-order requirement.Its hundreds of operators include irregular memory access, data-dependent control flow, and sparse computation.Source: §2.2 · p. 9
  • Preprocessing handles dense, sparse, weighted sparse, and event-based features through normalization, sparse processing, and feature-derivation transformations.Feature derivation includes bucketization, set operations, and n-gram hashing.Source: §2.2 · p. 9

3 System Architecture

KernelEvolve’s system architecture combines graph-based kernel search, hardware-aware runtime retrieval, standardized artifacts and interpreter environments, and automated profiling and evaluation to generate and refine kernels across heterogeneous accelerators. Persistent storage, fast MTIA debugging, and scalable remote evaluation support reuse and optimization across hardware.

3 System Architecture

  • KernelEvolve evaluates generated kernels for correctness and speed, then uses profiling and MPP’s federated tooling to expose metrics across system and compiler layers.TritonBench compares kernels with PyTorch references, while system profiling and hardware-specific tools provide execution and platform insights.Source: §3 · p. 11

3.1 Tree Search and State Machine

  • KernelEvolve models optimization as graph search, selecting nodes, transforming implementations, and scoring candidates by speedup while assigning zero fitness to incorrect or failed kernels.Selection can use greedy search, MCTS, or evolutionary algorithms; search stops when budgets expire, progress stalls, or a fitness threshold is reached.Source: §3.1 · p. 12
  • A single universal operator replaces fixed Draft, Debug, and Improve prompts with retrieval-augmented prompts that adapt kernel generation to runtime context.This lets each generation step address correctness, performance, memory access, or hardware-specific concerns without predefined operator boundaries.Source: §3.1 · pp. 12–13

3.2 Agentic Retrieval & Self-Managed Context

  • Runtime retrieval loads task-relevant knowledge into prompts: context memory diagnoses artifacts and bottlenecks, then deep search retrieves targeted guidance.Conditioning retrieval on runtime analysis avoids irrelevant generic guidance and conserves context-window capacity.Source: §3.2 · pp. 13–14

3.2.1 Deep Search Sub-Agent

  • The deep-search knowledge base organizes correctness constraints, cross-platform guidance, and hardware-specific expertise, using indexed retrieval to specialize advice progressively.Its platform knowledge spans NVIDIA, AMD, and MTIA; retrieval can progress from general guidance to architecture-specific techniques and code samples.Source: §3.2.1 · pp. 14–15

3.2.2 Context Memory Sub-Agent

  • Persistent context memory supports concurrent exploration, runtime-artifact analysis, profiling-prioritized prompt synthesis, iterative learning from prior attempts, and cross-session reuse.Metadata queries execute in milliseconds even with millions of nodes, and production histories span months across hundreds of operator types and multiple platforms.Source: §3.2.2 · pp. 15–16

3.2.3 MTIA Knowledge Injection

  • KernelEvolve injects MTIA-specific expertise absent from public training data, covering dedicated hardware operations, helper functions, tuning options, and cross-PE communication.Documented tuning includes circular-buffer multipliers and dual-core execution; extensions include cross-PE broadcasting and reductions.Source: §3.2.3 · pp. 16–17
  • MTIA-specific documentation retrieval teaches the model hardware idioms and enables production-grade kernels using SFU operations, inter-PE communication, and dual-core synchronization without model retraining for new platforms.Runtime feedback can guide further retrieval, such as fetching language-extension documentation after compilation errors.Source: §3.2.3 · p. 18

3.3 File and Code Search

  • An MCP code-search interface offers exact-match, regex, and filename queries, automatically dereferencing repository links to combine optimization guidance with production code examples.Search spans documentation and production code indices, returning paths and contextual snippets for prompt synthesis.Source: §3.3 · p. 18

3.4.1 Kernel Output Format

  • Standardized artifacts pair PyTorch and Triton nn.Module implementations with shared inputs, enabling automated correctness comparisons and performance profiling.The same interface supports correctness checks with precision-dependent tolerances and speedup measurement.Source: §3.4.1 · pp. 18–19

3.4.2 AI Hardware Interpreters

  • Hardware-specific interpreters provide unified evaluation interfaces while retaining platform-specific optimization capabilities across NVIDIA, AMD, and MTIA.Conveyor rebuilds and deploys interpreter packages as dependencies update, supporting reproducible profiling across search iterations.Source: §3.4.2 · pp. 19–20

3.4.3 Evaluation Code Generation

  • A deterministic evaluation-code generator creates profiler-specific harnesses, while pre-deployed interpreters reduce compilation latency from ≥10 minutes to seconds and keep profiling consistent across variants.Harnesses invoke correctness and speedup checks, execution-trace capture, and hardware-specific profiling tools.Source: §3.4.3 · pp. 20–21

3.4.4 Unified Profiling: Triton MPP

  • Triton MPP provides structured instruction-level performance data without vendor-specific parsers, using compiler-centric instrumentation and minimally invasive probes to capture asynchronous overlap.Targeted probes isolate instructions while guarding instrumentation against interference with execution.Source: §3.4.4 · pp. 21–22

3.4.5 Agentic Debugging in JIT Flow

  • MTIA compiler introspection exposes generated C++, letting agents diagnose hardware-specific issues, modify kernels, and replay fixes within seconds rather than minutes.The emit-modify-replay cycle avoids full Triton recompilation, including constexpr resolution, MLIR lowering, and backend code generation.Source: §3.4.5 · p. 22

3.4.6 Kernel Evaluation on FaaS

  • FaaS-based evaluation decouples CPU-bound kernel generation from accelerator runs and distributes evaluations across hundreds of GPUs and MTIA devices, avoiding queues on limited local hardware.Candidates are dispatched asynchronously to remote workers, which return correctness status, speedup ratios, and profiling metrics.Source: §3.4.6 · pp. 22–23

4 OSS Operator Evaluation

KernelEvolve is evaluated for correctness across standard PyTorch operators and the KernelBench suite, then examined for optimization behavior. The basic ATen operators primarily validate correctness rather than optimization potential.

4 OSS Operator Evaluation

  • The evaluation tests 160 ATen operators on NVIDIA H100, AMD MI350, and MTIA v3 against PyTorch references using precision-appropriate tolerances.The suite includes element-wise, transcendental, reduction, and activation operators.Source: §4 · p. 23
  • 100% correctness across 480 operator-platform configurations and a 100% pass rate across all three KernelBench levels validate kernel generation across diverse operator complexities.The 480 configurations comprise 160 operators on three platforms; KernelBench spans single operators, fused patterns, and full model blocks.Source: §4 · p. 23
  • Four of six operators exceed 1.0× fitness, while optimization headroom varies: torch.cos rises from 2.8× to 3.05× and torch.amax and torch.div remain near 1.0×.Fitness is generated-kernel speedup over the PyTorch reference; tree expansion uses execution feedback, unlike the initial independent-sampling draft phase.Source: §4 · p. 23, Figure 10 · p. 24
  • Basic ATen operators primarily validate end-to-end correctness, while their limited optimization headroom makes them a weak measure of optimization potential.The following evaluation instead considers higher-level ads operators with fusion opportunities and distinct memory-access patterns.Source: §4 · p. 24

5 Monetization Case Study

KernelEvolve improves performance across production workloads and uses automated search to optimize representative kernels. In convolutional transformers, its conv1d implementation gains speed over PyTorch baselines through fusion and search-guided tuning.

5 Monetization Case Study

  • Production workloads show 1.2–17× speedups over PyTorch baselines across convolution, recommendation-model fusion, and data-preprocessing case studies.The analyzed kernels include convolutional transformers, WuKong’s Optimized FM, InterFormer’s PFFN, MapId, MBDT, and Batch Event Truncate.Source: §5 · p. 24

5.1 Convolutional Transformer

  • The convolutional-transformer case targets production shapes on H100, where 1D convolution layers compress event sequences into shorter segment representations.These conv1d operations dominate the computational workload; the comparison evaluates FP16 serving and FP32 training shapes.Source: §5.1 · pp. 24–25
  • 2.30× speedup over conv1d and 1.62× over conv2d in FP16, and 1.73× and 1.30× respectively in FP32, are reported for the production conv1d shape.The generated kernel is specialized for production distributions and underperforms the baselines on the cited out-of-distribution shapes, at 0.49–0.63×.Source: Table 3 · p. 25, §5.1 · p. 25
  • KernelEvolve fuses the conv1d operation into two kernels, compared with five for PyTorch conv1d and four for the conv2d baseline.The fused approach avoids redundant layout transformations and their associated memory traffic; profiling-trace durations include overhead and are not actual kernel latency.Source: §5.1 · p. 25, Figure 11 · p. 26, Table 4 · p. 26

5 Monetization Case Study

  • Beyond fusion, search explores more than 20 autotuning configurations across block sizes, warp counts, and pipeline stages.Other production kernels benefit from expanded configuration searches, platform-specific compilation flags, cache modifiers, and alignment hints.Source: §5 · p. 24, §5.1 · p. 25

5.1 Convolutional Transformer

  • Search-based optimization raises the conv1d fitness score from about 2,000 to 6,889 over 300 steps, using accumulated execution feedback to refine candidates.The search identifies fusion strategies and tiling configurations that manual development would require weeks to explore.Source: §5.1 · p. 26

5.2 Convolution on Heterogeneous Hardware

KernelEvolve’s conv1d kernels are evaluated across five hardware platforms using a production shape and FP16 precision. Performance varies by platform, and the system generates hardware-specific kernels and retains library fallbacks.

5.2 Convolution on Heterogeneous Hardware

  • KernelEvolve beats the conv1d baseline on all five platforms, with speedups ranging from 1.75× on AMD MI300 to 6.54× on MTIA v3.Against conv2d, speedups range from 1.06× on AMD MI350 to 4.71× on MTIA v3.Source: §5.2 · p. 28, Figure 13 · p. 28
  • KernelEvolve targets NVIDIA, AMD, and MTIA from unified operator specifications, with platform-specific strategies including Tensor Core tiling, Infinity Cache-aware tiling, and MTIA-specific units and inter-PE communication.The paper attributes these strategies to systematic search guided by platform-specific documentation retrieved from its knowledge base.Source: §5.2 · p. 28
  • Because optimizations may not transfer across platforms, KernelEvolve generates and validates platform-specific kernels offline and uses the best-performing variant with vendor-library fallbacks when generated kernels underperform.The evaluation uses the production shape (2048, 96, 96, 200) at FP16 precision.Source: §5.2 · p. 28

5.3 WuKong and InterFormer: Kernel Fusion

KernelEvolve uses fused, shape-aware kernels to optimize WuKong Optimized FM and InterFormer PFFN across production shapes. Performance varies with dimensions and batch size, with gains maintained or selectively bypassed when tiling overhead dominates.

5.3.1 Optimized FM in WuKong

  • For Optimized FM, KernelEvolve fuses two matrix multiplications, keeps their intermediate in SRAM, and cuts memory traffic by approximately 2×.Shape-specific tiles fit the complete computation chain within SRAM for the stated production configuration, B=1024 and D=224.Source: §5.3.1 · p. 29
  • Optimized FM delivers 3.6–3.9× speedup for 24 features and 2.0–2.5× for 40–64 features, while larger feature counts approach 1×.The cited shape analysis also reports 2.0–3.5× across output dimensions for smaller feature counts; larger N requires more SRAM tiles, whose management overhead can outweigh on-chip computation.Source: §5.3.1 · p. 29, Figure 14 · p. 30
  • KernelEvolve deploys Optimized FM with 2–4× speedups on production shapes where N≤64 and falls back to PyTorch for larger feature counts.This shape-specific dispatch avoids regressions when tiling overhead dominates.Source: §5.3.1 · p. 30

5.3.2 PFFN in InterFormer

  • The PFFN module applies a feed-forward layer, GELU, RMSNorm, a second feed-forward layer, and final RMSNorm to its input tensors.KernelEvolve generates variants fusing the feed-forward network with RMSNorm, or with GELU and RMSNorm.Source: §5.3.2 · p. 30
  • For PFFN, shape-specific tiling and cross-operation tile reuse keep the operator chain in SRAM, reducing execution to one tile load and one output write.The PyTorch baseline uses separate BMM and fused elementwise kernels with three total memory passes.Source: §5.3.2 · p. 31
  • PFFN speedup reaches 2.0–2.6× at B ≤256 and stabilizes at 1.2–1.4× beyond B=512; all tested configurations remain at or above 1.0×.At fixed B=1024 and K=256, speedup varies non-monotonically with D, from 1.6–1.9× at D≤100 to 1.1–1.2× around D=200, then 1.2–1.4× for D>200.Source: §5.3.2 · p. 31

5.4 Data Preprocessing Kernels on MTIA

KernelEvolve addresses MTIA's incomplete ATen coverage by generating on-device preprocessing kernels, while also optimizing kernels on better-supported hardware. Its MapIdTransform and MBDT evaluations show speedups across MTIA generations, with gains varying by workload and baseline.

5.4 Data Preprocessing Kernels on MTIA

  • MTIA lacks native support for some ATen operators, so KernelEvolve supplies missing on-device implementations on v2i and tunes kernels on v3.Missing kernels can force model rewrites or CPU fallback; generated kernels address both model enablement and optimization.Source: §5.4 · pp. 31–32

5.4.1 MapId Transform

  • MapIdTransform remaps sparse categorical IDs to dense indices, reserving index 0 for unknown values; KernelEvolve fuses four PyTorch operators into one accelerator invocation.The fused kernel eliminates three intermediate tensor materializations and reduces global memory traffic by 4× relative to operator-by-operator execution.Source: §5.4.1 · pp. 32–33
  • MapIdTransform reaches 4.07× on MTIA v2i and 1.36× on v3 for large-batch workloads, with runtime dispatch falling back to PyTorch for regression cases.The v2i speedup scales from 0.78× at batch size 2000 to 3.23× at 10000; the v3 range is 1.05–1.36×.Source: §5.4.1 · pp. 33–34, Table 7 · p. 34

5.4.2 MergeBucketizedDense Transform

  • MBDT maps continuous features to discrete bin indices, using per-feature offsets so output indices are globally unique.Its generated kernel fuses border lookup, binary search, and offset computation; vectorized counting and adaptive block sizes target MTIA execution.Source: §5.4.2 · pp. 34–36
  • MBDT speedups are 2.94–9.25× on MTIA v2i and 2.31–3.09× on MTIA v3.On v2i, the reported speedup rises from 3.19× for 64 × 2 × 2 to 9.25× for 2048 × 2 × 4; Figure 17 presents configurations as Batch × Features × Borders.Source: §5.4.2 · pp. 35–36, Figure 17 · p. 36

5.4.3 Summary

  • On MTIA v2i, generated preprocessing kernels provide the on-device path where native operator coverage is limited; on v3 they deliver 2–3× speedups.Without supported operators, PyTorch can fall back to CPU and incur order-of-magnitude latency penalties.Source: §5.4.3 · p. 36

5.5 Sequence Learning: Batch Event Truncate

KernelEvolve generates a batched Triton implementation of Batch Event Truncate, an operation over nested jagged event-feature data. The batched kernel achieves 2× speedup in production end-to-end benchmarks over the sequential PyTorch implementation.

5.5 Sequence Learning: Batch Event Truncate

  • KernelEvolve batches multiple features into one Triton kernel, replacing PyTorch's sequential per-feature processing.The operation coordinates index arithmetic across nested jagged tensors; production sequences can reach 200 events and 5–32 features.Source: §5.5 · p. 37
  • When truncation is required, constant-launch parallel processing yields 1.4–2.0× speedups; when none is needed, vectorized comparison yields 9.8× and 14.5× at higher feature counts.The PyTorch baseline iterates sequentially, while the batched kernel processes all features in one launch.Source: §5.5 · p. 37
  • The batched kernel achieves 2× speedup over PyTorch in production end-to-end benchmarks.The reported speedup scales with batch size and supports additional event-based features without proportional latency increase.Source: §5.5 · p. 38, Table 8 · p. 38

6 Future Directions

The future directions extend KernelEvolve across more hardware, abstraction levels, and stages of the AI stack. They also propose scaling search, integrating kernel generation into deployment, and measuring its resource costs.

Heterogeneous Hardware at Scale

  • Future work proposes hardware abstractions for memory hierarchies, compute capabilities, and ISA constraints, alongside execution-feedback adaptation for proprietary accelerators.The proposed adaptation loop collects execution traces and fine-tunes the agent for new hardware.Source: §6 · pp. 38–39

From Operators to Models

  • KernelEvolve could expand from operators to model-level optimization, including cross-layer fusion and joint optimization of model structure and kernel implementation.The paper also proposes deeper code generation through MLIR dialects, PTX/SASS, and hardware diagnostics where Triton abstractions limit control.Source: §6 · p. 38

Massively Parallel Search

  • The current sequential tree search could be extended to evaluate thousands of candidates in parallel across distributed infrastructure.The proposed search relaxes consistency guarantees and considers whether kernel quality improves predictably with compute investment.Source: §6 · p. 38

End-to-End Production Integration

  • A future production integration would connect KernelEvolve with AOT Inductor, serving infrastructure, and continuous integration to generate, validate, and deploy kernels as models and hardware evolve.The stated aim is to reduce human effort maintaining performance across the AI stack.Source: §6 · p. 39

Sustainable AI Infrastructure

  • Future work would quantify token consumption and carbon footprint, enabling trade-offs between optimization depth and environmental impact.The paper also identifies prompt design as a target for reducing inference cost.Source: §6 · p. 39

7 Related Work

KernelEvolve builds on prior kernel optimization and LLM-based code-generation work, but targets heterogeneous hardware and production recommendation workloads. Its stated distinctions include broad operator coverage, deployment-integrated validation, and at-scale deployment.

7 Related Work

  • Earlier optimization abstractions reduce development effort but still require expertise for novel transformations and manual adaptation across heterogeneous hardware.The discussion spans vendor libraries, auto-tuning, Halide, TVM, Triton, and newer higher- and lower-level DSLs.Source: §7 · p. 39
  • Prior LLM kernel systems use approaches including RL, supervised generation, execution-guided rewards, agentic workflows, and evolutionary search, while remaining limited in hardware and production scope.The paper also relates its tree-based search with execution feedback to test-time scaling and multi-path reasoning.Source: §7 · pp. 39–40
  • KernelEvolve targets NVIDIA GPUs, AMD GPUs, and Meta’s MTIA accelerators from unified operator specifications, including proprietary MTIA through injected hardware knowledge.The paper describes this as a distinction from prior systems targeting single platforms and synthetic workloads.Source: §7 · pp. 39–40
  • It addresses production operator diversity, including 200+ preprocessing operators whose irregular patterns and data-dependent control flow can determine deployment architecture.The paper contrasts these with canonical benchmark operators.Source: §7 · p. 40
  • Deployment-integrated optimization combines continuous validation, multi-level profiling, and serving compatibility for production rollout.Source: §7 · p. 40
  • KernelEvolve is presented as the first LLM-based kernel coding system deployed at-scale for business-critical recommendation inference.Source: §7 · p. 40

8 Conclusion

KernelEvolve applies graph-based search with retrieval-augmented prompting and persistent knowledge bases to kernel generation across heterogeneous AI infrastructure. The paper reports full correctness across operator-platform configurations, production speedups, and reduced development time, and presents automated synthesis as a response to deployment barriers from incomplete kernel coverage.

8 Conclusion

  • 100% correctness across 480 operator-platform configurations, 1.25-17× production speedups, and development time reduced from weeks to hours are reported for KernelEvolve.Source: §8 · p. 40
  • Incomplete kernel coverage forces disaggregated architectures with 10-20ms network overhead, while automated synthesis eliminates these architectural penalties.Source: §8 · p. 40
  • KernelEvolve is presented as a first step toward LLM agents acting as a universal compilation layer, adapting to new hardware through knowledge injection rather than manual porting.The deployment insights include multi-granularity profiling, automated validation, and architecture-aware optimization as design principles for AI-assisted systems software.Source: §8 · p. 40

9 Contributors

The section names the project leads, three contributor groups, and supervisors.

Project Leads

Core Contributors

  • Core Contributors span Monetization Infra and Ranking, FAIR SysML, and MTIA Software.Monetization Infra and Ranking: Gang Liao, Hongsen Qin, Ying Wang, Yavuz Yetim, Jia Jiunn Ang, Xiayu Yu, Yihan He, Feng Shi, Zewei Jiang, Chunli Fu, Ruichao Xiao, Dianshi Li, and Gaoxiang Liu; FAIR SysML: Alicia Golden, Michael Kuchnik, Samuel Hsia, and Carole-Jean Wu; MTIA Software: Roman Levenstein, Kunming Ho, Haishan Zhu, Alec Hammond, Richard Li, Ajit Mathews, and Kaustubh Gondkar.Source: Core Contributors · p. 41

Supervision

  • Supervision is credited to Liyuan Li, Nathan Yan, Varna Puvvada, Uladzimir Pashkevich, and Matt Steiner.Source: Supervision · p. 41

A 1D Convolution

This section presents a Triton 1D convolution using GEMM-style tiling and packed weights, evaluated with production-like benchmark inputs. It describes the implementation and a PyTorch comparison harness that checks accuracy, reports latency and speedup statistics, and assigns pass or failure outcomes.

A 1D Convolution

  • The candidate is a randomly selected high-performing variant, not the single best; evaluation uses synthetic data validated for equivalent speedup characteristics to production data.The synthetic data preserves production Min, Max, Mean, and STD; the production Triton trunk also differs slightly from the open-source release.Source: A 1D Convolution · p. 47
  • Autotuning configurations are expanded for M-heavy shapes and small K, with larger K tiles and very large M tiles for selected workloads.The kernel also double-buffers input and weight tiles across channel chunks and caches weights in L2 for reuse.Source: A 1D Convolution · pp. 49, 52, 55
  • The Conv1D kernel packs weights from [Cout, Cin_g, Ksz] to [Cin_g*Ksz, Cout] and computes convolution with a GEMM-style kernel.The implementation uses 3D tiling and includes a fused bias epilogue; weight packing can be cached by pointer, shape, and device.Source: A 1D Convolution · pp. 53, 54, 56
  • The generated operator supports fp32, fp16, and bf16 and can disable weight caching to measure cold-start performance including packing.The file is generated by the KernelEvolve TritonBench Operator generator.Source: A 1D Convolution · pp. 47, 48, 56
  • The Triton 1D convolution uses GEMM tiling with packed weights and supports fp32, fp16, and bf16 inputs.The implementation packs and caches weights, and checks that weight and optional bias dtypes match the input.Source: A 1D Convolution · pp. 57–58
  • Benchmark inputs match the original production-data shape (2048, 96, 200) and production dtype float32; multiple batch sizes are created by slicing the full input.The independent get_inputs() function is accessed by both models, ensuring consistent test data generation across all implementations.Source: A 1D Convolution · pp. 59–60
  • The TritonBench harness evaluates latency, speedup, and accuracy against PyTorch reference implementations.The passages describe the evaluation metrics but provide no measured benchmark results.Source: A 1D Convolution · pp. 61–62
  • It prints average latency and average speedup as summary statistics.The corresponding labels are “Average Latency” and “Average Speedup.”Source: A 1D Convolution · p. 64
  • The harness checks accuracy across all input sizes against the PyTorch reference and reports a numerical mismatch when outputs differ.It prints a success message and KERNEL_TEST_PASSED when the outputs match.Source: A 1D Convolution · p. 64
  • Failures print KERNEL_TEST_FAILED and FITNESS_SCORE: 0.0, while a positive average speedup allows the computed fitness score to be printed.The harness also reports exceptions and exits with status 0 on success or 1 on failure.Source: A 1D Convolution · pp. 64–65

Related papers

Loading 2512.23236v4…