Source-linked AI summary
Agentic Code Optimization via Compiler-LLM Cooperation
Benjamin Mikek, Danylo Vashchilenko, Bryan Lu, Panpan Xu
TL;DR
Compilers can miss optimizations requiring high-level reasoning, while LLM-generated code is often incorrect. Compiler-LLM cooperation coordinates LLM optimization agents and compiler components across abstraction levels; ACCLAIM achieves mean speedups of 1.25× against clang -O3 and outperforms the specified baselines. Its computational cost is high, with each run requiring many model inference calls.
Problem
Compilers may miss optimization opportunities requiring high-level reasoning, while LLMs can produce incorrect code.
Source: §1 · p. 2Method
Compiler-LLM cooperation interleaves level-specific LLM optimization with compiler components under a guiding agent that selects tools and uses feedback.
Source: §1 · pp. 2–3Results
1.25× mean speedups against clang -O3, outperforming level-specific and naive multi-level baselines with equal computation budgets on standard C programs.
Source: §1 · p. 3Takeaways & Limitations
The approach combines multi-level LLM optimization with compiler orchestration and is reported to outperform single-level and naive multi-level baselines.
Source: §1 · p. 3A run on one program requires b × n × k model inference calls, each using tokens proportional to input program size.
Source: §5.3 · p. 18
Abstract
from arXiv · showhide
Generating performant executables from high level languages is critical to software performance across a wide range of domains. Modern compilers perform this task by passing code through a series of well-studied optimizations at progressively lower levels of abstraction, but may miss optimization opportunities that require high-level reasoning about a program's purpose. Recent work has proposed using LLMs to fill this gap. While LLMs can achieve large speedups on some programs, they frequently generate code that is incorrect. In this work, we propose a method to balance the correctness of conventional compiler optimizations with the ``creativity'' of LLM-based code generation: compiler-LLM cooperation. Our approach integrates existing compiler optimization passes with LLM-based code generation at multiple levels of abstraction, retaining the best features of both types of code optimization. We realize our approach with a multi-agent system that includes (1) LLM-based optimization agents for each level of abstraction, (2) individual compiler constituents as tools, (3) an LLM-based test generation agent that probes the correctness and performance of generated code, and (4) a guiding LLM that orchestrates the other components. The strategy enables LLM-based optimization of input programs at multiple levels of abstraction and introduces a method for distributing computational budget between levels. Our extensive evaluation shows that compiler-LLM cooperation outperforms both existing compiler optimizations and level-specific LLM-based baselines, producing speedups up to 1.25x.
1 Introduction
Compiler optimizations are robust but may miss opportunities requiring high-level reasoning, while LLM optimizations can be creative but often produce incorrect code. Compiler-LLM cooperation interleaves both across abstraction levels, using agents and compiler tools to choose optimizations; ACCLAIM outperforms compiler-only and LLM-based baselines in its evaluation.
1 Introduction
- LLMs can reason about some complex optimization opportunities that compilers miss, but frequently generate incorrect code, whereas compilers are conservative and robust.The paper frames cooperation as balancing these contrasting strengths.Source: §1 · p. 1, Figure 1 · p. 2
- Compiler-LLM cooperation interleaves LLM optimization with compiler components across abstraction levels, balancing LLM optimization opportunities with compiler correctness.The approach does not require training data beyond the target program, though it remains compatible with training- and test-time tuning methods.Source: §1 · pp. 2–3
- ACCLAIM combines level-specific rewriting agents, a test-generation agent, and a guiding agent with access to compiler components as tools.The guiding agent chooses among the components and uses feedback to inform optimization decisions.Source: §1 · p. 3
- 1.25× mean speedups against clang -O3, with ACCLAIM outperforming level-specific and naive multi-level baselines under equal computation budgets.The evaluation uses a standard set of C programs.Source: §1 · p. 3
- The paper's contributions include a framework for multi-level LLM optimization and an orchestration workflow that chooses when and where to apply it.The combined strategy is reported to outperform single-level and naive multi-level baselines.Source: §1 · p. 3
2 Motivating Example
The example shows an LLM and compiler contributing different optimizations to a function that sums the population counts of values from 1 to k. The LLM replaces inner-loop bit counting with a population-count intrinsic, and LLVM then vectorizes it, with a further speedup on suitable hardware.
2 Motivating Example
- The original function sums the number of set bits for each value from 1 to k, with nested loops and O(k log k) runtime.The source-level LLM rewrite reduces this to O(log k) by replacing inner-loop bit counting with a mathematical function.Source: §2 · pp. 3–4
- At IR level, the LLM recognizes bit counting and replaces the inner loop with LLVM’s @llvm.ctpop.i32 intrinsic, yielding O(k) runtime.This improvement depends on the underlying hardware supporting constant-time bit counting.Source: §2 · p. 4, Figure 2
- 4× further reduction in running time follows LLVM vectorization of LLM-introduced population-count calls, for a processor with suitable hardware.The compiler vectorizes the intrinsic calls; the example notes that constant-time bit counting requires hardware support.Source: §2 · pp. 4–5
- The example demonstrates complementary strengths: the LLM introduces ctpop, which the compiler does not, and LLVM vectorizes the calls, which the LLM does not.The example attributes the additional 4× reduction to compiler vectorization on suitable hardware.Source: §2 · p. 5
3 Approach
Section 3 frames compiler-LLM cooperation as coordinating rewrites, lowerings, and testing to produce correct, performant programs. ACCLAIM uses a guiding agent to orchestrate compiler components, level-specific optimization agents, and program-dependent testing.
3 Approach
- Section 3 presents compiler-LLM cooperation as coordination of compiler and LLM rewrites and compiler lowerings, with testing to seek correct, performant output.The framework models a compiler as rewrites and lowerings across abstraction levels, including C source, LLVM IR, and x86 assembly.Source: §3.1 · p. 5, §3 · p. 5
3.1 Preliminaries
- The formal objective is to compose compiler and agent functions into a compilation path while maximizing performance subject to full correctness.Correctness is measured as the proportion of tests passed, while performance is the ratio of original runtime to final runtime.Source: §3.1 · p. 6
3.2 Architecture
- The architecture has a guiding agent that invokes compiler components and level-specific agents, which return rewritten programs for further orchestration.The organizational structure is described as similar to the planner-executor model of Erdogan et al. [13].Source: §3.2 · p. 6, Figure 3 · p. 7
3.3 Guiding Agent
- The guiding agent selects among compiler and level-specific optimization tools, using feedback and a compute budget to allocate LLM-agent calls.Compiler-component calls have zero cost; level-specific agent calls have unit cost, and repeated calls and backtracking are allowed.Source: §3.3 · pp. 6–7
3.4 Level-Specific Agents
- Each level-specific agent rewrites code at its own abstraction level, sampling variants and testing them before retaining the best-performing correct program.If a sample has no correct variants, testing feedback such as compiler errors or failing cases guides later iterations.Source: §3.4 · p. 8
3.5 Testing Agent
- ACCLAIM’s testing agent generates correctness and performance inputs, using deterministic scripts with edge-case exploration and inputs spanning multiple orders of magnitude.The scripts are program-dependent; large-scale inputs grow with a size parameter so runtime behavior can be assessed across input scales.Source: §3.5 · pp. 8–9
4 Evaluation
The evaluation tests ACCLAIM on filtered CodeNet C programs, compares multi-level optimization with level-specific baselines, and examines how the guiding agent works and how design choices affect results.
4.1 Dataset
- The benchmark contains 5,640 programs sampled from 564 problems selected for at least 10% runtime variation among solutions.The dataset is filtered to exclude programs that fail to compile and problems for which the testing agent cannot produce scripts.Source: §4.1 · p. 10
4.2 Experimental Setup
- The experiments address effectiveness, process, and design questions, using LLVM 18.1.3 and measuring speedups against clang -O3.Experiments use Claude 3.7 Sonnet, and each agent run is limited to one hour.Source: §4.2 · pp. 10–11
4.3 RQ1: Effectiveness
- ACCLAIM outperforms all four level-specific and naive multi-level baselines under a fixed computation budget.The baselines include source-only, IR-only, assembly-only, and a per-input portfolio of the three levels.Source: §4.3 · p. 11, Table 2 · p. 12
- Most programs show negligible speedup, while a smaller number gain substantially; ACCLAIM speeds up one grid-pathfinding benchmark by 1384×.Across that benchmark’s 10 test inputs, grid sizes range from 5 × 5 to 1000 × 1000.Source: §4.3 · pp. 11–12, Figure 4 · p. 12
- For the grid-pathfinding benchmark, ACCLAIM reduces grid allocation calls from linear in H to constant by allocating the grid in one block.Other program changes, including BFS simplifications, also contribute to the speedup.Source: §4.3 · pp. 12–13, Figure 5 · p. 13
4.4 RQ2: Process
- Source-level optimization contributes most, but level contributions interact; one illustrated run gains at both source and IR levels, primarily from an IR transformation.The table’s speedup contributions are multiplicative rather than additive.Source: §4.4 · pp. 13, 15, Table 3 · p. 13
- The guiding agent calls all compiler components and optimization agents, repeats some calls, and can restart optimization from the source level after reaching the backend.Transitions from the backend to source-level optimization occur 30% of the time.Source: Figure 6 · p. 14, §4.4 · p. 15
4.5 RQ3: Design
- Claude 3.7 Sonnet outperforms the evaluated open-source models; with that model, more feedback iterations outperform all-parallel sampling, and tripling compute raises mean speedup from 1.19× to 1.25×.With n = 1, k = 4, results are best; n = 2, k = 2 is close, while all-parallel sampling yields 1.11× speedup.Source: §4.5 · pp. 16–17, Table 4 · p. 17
5 Discussion and Future Work
The discussion considers extending compiler-LLM cooperation beyond its LLVM and CPU-focused realization, while identifying correctness verification and computational cost as practical boundaries.
5.1 Generalization to Other Compilers
- Compiler-LLM cooperation could extend to other compilers, Rust, and GPU stacks, while compiler components and LLM rewrites may also help explore verified decompilation.The paper describes adapting ACCLAIM to GCC with minimal engineering effort and identifies PyTorch-to-Triton-to-LLVM GPU workflows as a potential target.Source: §5.1 · pp. 17–18
5.2 Correctness and Verification
- Testing can miss bugs in generated code at any addressed abstraction level; fully verifying output-program correctness remains future work.The modular design could replace the testing agent with a verification agent; the testing agent also withholds full test cases to reduce reward-hacking risk.Source: §5.2 · p. 18
5.3 Computational Cost
- Each run requires b×n×k model calls, with token use proportional to program size; proposed savings include smaller agents and targeting amenable code regions.The paper notes that inference costs may be worthwhile for widely used infrastructure, but not for a program run once by one user.Source: §5.3 · pp. 18–19
6 Related Work
Related work covers LLM-based optimization, compiler-directed optimization, and compiler phase ordering. ACCLAIM combines compiler calls with direct LLM code generation across abstraction levels under a guiding agent.
LLM-based Code Optimization
- Prior LLM optimization spans source code, x86 assembly, PyTorch, and Triton, using techniques such as repeated sampling and iterative refinement with feedback.ACCLAIM provides a proof-of-concept CPU system using repeated sampling and iterative refinement, while leaving reinforcement learning and supervised fine-tuning for future work.Source: §6 · pp. 19–20
Integration of Compilers and LLMs
- Other compiler-and-LLM approaches direct existing compiler passes or settings; ACCLAIM adds direct LLM code generation and lets its guiding agent choose among compiler front-, middle-, and back-ends.The cited approaches include compiler-tool feedback, pass selection, and compiler-optimization search.Source: §6 · p. 20
Compiler Phase Ordering
- ACCLAIM addresses compiler phase ordering by letting an LLM-based guiding agent choose between compiler-provided rewrites and level-specific LLM optimization agents.The paper situates this challenge among prior search-based and machine-learning approaches to ordering optimizations.Source: §6 · p. 20
7 Conclusion
Compiler-LLM cooperation integrates LLM code generation with compiler components across three abstraction levels, coordinated by a guiding agent. The evaluation reports an average speedup of 1.25× and orders-of-magnitude gains on some individual programs.
7 Conclusion
- 1.25× average speedup: compiler-LLM cooperation outperforms existing compiler optimizations and single-level LLM optimization in the reported evaluation.The conclusion also reports orders-of-magnitude improvement on individual input programs.Source: §7 · p. 20
- The approach combines LLM code generation with existing compiler components at source, intermediate-representation, and low-level assembly levels.A guiding agent coordinates the use of LLM-based and compiler-based optimization techniques.Source: §7 · p. 20
- Compiler-LLM cooperation combines LLM-based optimization with the correctness of existing compiler optimizations.The conclusion characterizes the approach as tempering LLM optimization’s creativity with compiler correctness.Source: §7 · p. 20