PaperlayerView exact revision on arXiv

Source-linked AI summary

MLGOPerf: An ML Guided Inliner to Optimize Performance

Amir H. Ashouri, Mostafa Elhoushi, Yuzhe Hua, Xiang Wang, Muhammad Asif Manzoor, Bryan Chan, Yaoqing Gao

arXiv:2207.08389v2cs.PLcs.AIcs.LGcs.NEcs.PF

TL;DR

ML-guided compiler optimization has had limited integration into production compiler pipelines, and LLVM's MLGO inliner focused on code-size reduction. MLGOPerf uses predicted post-inlining function speedups as rewards to train a performance-oriented inlining agent. The paper reports speedups over LLVM O3 and more downstream code-region opportunities on SPEC CPU2006 and Cbench.

  • Problem

    Few ML-guided compiler optimizations have been integrated as standalone components in compilers such as LLVM, and MLGO's inliner targets code-size reduction rather than performance.

    Source: §1 · pp. 1–2

  • Method

    IR2Perf predicts post-inlining function speedup and supplies rewards for training a performance-oriented reinforcement-learning inliner.

    Source: §1 · p. 2

  • Results

    Up to 1.8% and 2.5% performance speedup over LLVM O3 was reported on SPEC CPU2006 and Cbench, respectively.

    Source: §6 · p. 10

  • Takeaways & Limitations

    The approach increases downstream code-region opportunities by up to 26%, which can add a further 3.7% speedup to the binary after O3.

    Source: §6 · p. 10

    In 15% of training cases, function speedup and global speedup moved in opposite directions, limiting the function-level proxy's alignment with program performance.

    Source: §5.1 · p. 10

Abstract

from arXiv · show

For the past 25 years, we have witnessed an extensive application of Machine Learning to the Compiler space; the selection and the phase-ordering problem. However, limited works have been upstreamed into the state-of-the-art compilers, i.e., LLVM, to seamlessly integrate the former into the optimization pipeline of a compiler to be readily deployed by the user. MLGO was among the first of such projects and it only strives to reduce the code size of a binary with an ML-based Inliner using Reinforcement Learning. This paper presents MLGOPerf; the first end-to-end framework capable of optimizing performance using LLVM's ML-Inliner. It employs a secondary ML model to generate rewards used for training a retargeted Reinforcement learning agent, previously used as the primary model by MLGO. It does so by predicting the post-inlining speedup of a function under analysis and it enables a fast training framework for the primary model which otherwise wouldn't be practical. The experimental results show MLGOPerf is able to gain up to 1.8% and 2.2% with respect to LLVM's optimization at O3 when trained for performance on SPEC CPU2006 and Cbench benchmarks, respectively. Furthermore, the proposed approach provides up to 26% increased opportunities to autotune code regions for our benchmarks which can be translated into an additional 3.7% speedup value.

1 INTRODUCTION

Standard compiler pipelines offer average benefits, but optimization selection remains difficult to automate within the compiler. MLGOPerf adapts LLVM's inliner for performance by using predicted function speedups to train its reinforcement-learning agent and expose downstream optimization opportunities.

1 INTRODUCTION

  • Standard optimization levels apply fixed-length pass sequences that have shown average benefits, with LLVM O3 containing about 160 passes and roughly 60 unique passes.Several subsequences repeat in the O3 sequence.Source: §1 · p. 1
  • Earlier ML autotuning approaches often depended on external components, while MLGO brought a standalone ML-guided inlining pass into LLVM but targeted code-size reduction.The paper describes MLGO as upstreamed into LLVM trunk.Source: §1 · p. 1
  • Inlining can remove call overhead and expand later analyses, but excessive inlining can increase code size and hurt performance through instruction-cache misses.Choosing well is difficult because the number of inlining choices grows at least exponentially.Source: §1 · p. 1
  • MLGOPerf adapts LLVM's inliner from code-size reduction toward performance optimization and increases code regions available to subsequent O3 passes.The paper reports that these added opportunities can provide further speedup.Source: §1 · pp. 1–2
  • IR2Perf predicts post-inlining caller speedup from LLVM IR features, generating rewards for training the existing LLVM ML-Inliner agent without thousands of program executions.The paper also adds features intended to better represent callee behavior and improve agent accuracy.Source: §1 · p. 2

2 RELATED WORK

Compiler autotuning research addresses optimization selection and phase ordering, commonly using machine learning to speed iterative search. Prior integrated approaches still relied on high-level optimization engines, while MLGOPerf builds on LLVM's inliner to target performance.

2 RELATED WORK

  • Compiler autotuning research focuses on the still-open optimization-selection and phase-ordering problems, often pairing ML with autotuning to accelerate iterative search.The paper characterizes many approaches as black-box optimization.Source: §2 · p. 2
  • Prior end-to-end compiler approaches wrapped a high-level engine around pass selection, rather than making optimization decisions through an ML model built into a standalone pass.The authors describe this as an unmet capability in the approaches they survey.Source: §2 · p. 2
  • Rotem and Cummins [45] embed decision-tree branch-probability guidance in LLVM, while Phothilimthana et al. tune graph- and subgraph-level optimizations across compilation stages.The paper distinguishes the latter work as orthogonal to its approach.Source: §2 · p. 2
  • MLGO [54] provides LLVM inlining guidance for code-size minimization; MLGOPerf reuses that inlining infrastructure to make decisions aimed at performance.MLGOPerf uses IR2Perf-predicted function speedups as rewards for training an RL agent to make call-site decisions.Source: §2 · p. 2

3 PROPOSED METHODOLOGY

MLGOPerf retargets LLVM’s ML-Inliner from code-size reduction to performance by using IR2Perf-predicted function speedups as RL rewards. Its methodology collects inlining data, trains the predictor and policy, then deploys the trained policy without IR2Perf.

3 PROPOSED METHODOLOGY

  • MLGOPerf adapts LLVM’s ML-Inliner from code-size reduction to performance, using a second model to address reward needs and impractical runtime measurements during RL training.The paper identifies thousands of evaluations and long benchmark execution times as barriers to using actual execution time for rewards.Source: §3 · pp. 2–3

3.1 IR2Perf to the Rescue

  • At function granularity, IR2Perf predicts caller speedup as a proxy for generating rewards that guide call-site inlining decisions.Source: §3.1 · p. 3

3.2 MLGOPerf Phases

  • The three-phase workflow uses autotuning to collect call-site inlining configurations, function and module execution times, and function features for training.Source: §3.2 · p. 3, §3.2.1 · p. 3

3.2.1 IR2Perf Training.

  • IR2Perf is a deep regression model trained on post-inlining function features, measured execution time, and inlining configuration to predict relative speedup against O3.Source: §3.2.1 · p. 3

3.2.2 RL Model Training.

  • The RL agent treats each visited call site as its state, chooses whether to inline, updates the call graph deterministically, and receives function execution speedup versus baseline as reward.Source: §3.2.2 · p. 3
  • PPO trains the policy to maximize summed IR2Perf rewards; after sufficient iterations, IR2Perf is unplugged and the pretrained model is built into LLVM.Training can stop based on performance evaluation, loss trajectory, or an iteration or time budget.Source: §3.2.2 · p. 5, §3.2.3 · p. 5

4 EXPERIMENTAL RESULTS

The experiments evaluate IR2Perf’s function-speedup predictions, use them to train MLGOPerf’s reinforcement-learning inliner, and test its deployment performance and downstream autotuning opportunities. MLGOPerf improves performance over LLVM O3 on SPEC CPU2006 and Cbench, with increased binary size and more tunable code regions also reported.

4.1 IR2Perf Model

  • IR2Perf estimates function speedup from handcrafted LLVM IR features, with a four-layer fully connected regression model and PCA reducing the feature space to seven components.The study collected 370K inlining configurations from 10 SPEC CPU2006 benchmarks; training data excluded speedups above 3× and function overheads below 1%.Source: §4.1 · pp. 5–6
  • Function runtime is estimated from total program runtime and the function’s share of execution time; function speedup compares the baseline runtime with runtime under an inlining configuration.Runtime share and total runtime are collected with Linux perf, while call counts are collected with instrumentation and llvm-profdata.Source: §4.1 · p. 5

4.2 IR2Perf Accuracy

  • IR2Perf’s average error across SPEC CPU2006 benchmarks was 12.8%, with a best error of 9.1% on 401.bzip2 and a reported 19.1% prediction accuracy for 464.h264ref.The model was evaluated using leave-one-benchmark-out cross-validation; the authors selected the pretrained model with the lowest cross-validation error.Source: §4.2 · p. 6, Table 4 · p. 6

4.3 MLGOPerf Training

  • IR2Perf’s predicted caller speedup guides the RL policy, while two added callee features—Block Frequency and Loop Level—were associated with about 1% benefit in early results.IR2Perf generates rewards from IR features, avoiding the need to execute each program thousands of times; RL training took around four days on SPEC CPU2006 module samples.Source: §4.3 · pp. 6–7

4.4 MLGOPerf Deployment

  • At deployment, the AOT-compiled RL model uses 13 callee features to decide whether to inline; IR2Perf is no longer needed in this phase.The deployment evaluation used single-threaded runs on an ARMv8.2-A Kunpeng 920 system and repeated measurements with variance above 2%.Source: §4.4 · p. 7

4.5 Performance Improvement

  • 1.8% on SPEC CPU2006 and 2.2% on Cbench are MLGOPerf’s average speedups over LLVM O3; it also averages 3.1% and 4.1% over MLGO, respectively.Binary size increases versus both baselines: 17.8% and 23.4% on SPEC CPU2006, and 12% and 16% on Cbench, respectively.Source: §4.5 · p. 7, Table 5 · p. 8

4.6 Performance Enablement with Autotuning

  • Up to 21.8% more tunable code regions than LLVM O3 and 26% more than MLGO were found; the reported higher-speedup rates were 2.5%, 2.7%, and 3.7% for O3, MLGO, and MLGOPerf, respectively.The autotuner explored configurations for 120 iterations per benchmark on Cbench.Source: Table 6 · p. 9, §4.6 · p. 9
  • The autotuner searches loop-unroll and loop-vectorize settings, compiling and measuring successive configurations until its stopping criterion is met.The explored settings include unroll-count values 0, 2, 4, and 8, and interleave-count values 1, 2, and 4.Source: §4.6 · p. 9

5 DISCUSSION

MLGOPerf remains an early step toward standalone ML-guided compiler optimization, with challenges in balancing inlining, aligning function-level rewards with global speedup, and characterizing performance and software.

5 DISCUSSION

  • The authors describe MLGOPerf as an early step, still distant from an ideal standalone compiler pass that makes optimal decisions using ML.Source: §5 · p. 9
  • Inlining can hurt performance at either extreme—when omitted or applied aggressively—so choosing between them requires navigating a fine line.Source: §5 · p. 10

5.1 Challenges

  • Function-level speedup and global speedup move in opposite directions in 15% of training cases, while cache, memory, and hardware factors remain outside the work’s scope.An increase in function speedup can coincide with a decrease in global speedup.Source: §5.1 · p. 10
  • In some cases, performance-oriented inlining increases emitted code size; the paper leaves balancing these objectives through Pareto optimality to future work.Source: §5.1 · p. 10
  • Compiler optimization selection is NP-hard, the sequence space is unbounded, and no general algorithm can solve selection and phase ordering for all program-input pairs.Source: §5.1 · p. 10
  • Performance prediction depends on architecture and compiler, while unknown factors and noisy measurements make speedup difficult to predict.Source: §5.1 · p. 10
  • Handcrafted software features introduce discretization noise, and the paper identifies the lack of ideal ML-ready software characterization as a broader compiler challenge.Finer-grained embeddings could reduce this noise, but are outside the work’s scope.Source: §5.1 · p. 10

6 CONCLUSION

The conclusion presents MLGOPerf as an end-to-end framework for performance optimization using ML-Inliner. Its reported results include speedups over LLVM O3 and expanded opportunities for subsequent code-region tuning.

6 CONCLUSION

  • Up to 1.8% and 2.5% performance speedup over LLVM’s O3 on SPEC CPU 2006 and Cbench, respectively.Source: §6 · p. 10
  • Up to 26% more code-region opportunities for subsequent passes can add 3.7% speedup to the emitted binary at the end of O3.Source: §6 · p. 10
  • The authors identify generalizing the approach to other compiler optimizations as future work.Source: §6 · p. 10
Loading 2207.08389v2…