PaperlayerView exact revision on arXiv

Source-linked AI summary

ACPO: AI-Enabled Compiler Framework

Amir H. Ashouri, Muhammad Asif Manzoor, Duc Minh Vu, Raymond Zhang, Colin Toft, Ziwen Wang, Angel Zhang, Bryan Chan, Tomasz S. Czajkowski, Yaoqing Gao

arXiv:2312.09982v4cs.PLcs.AIcs.LGcs.PF

TL;DR

The paper asks how to make ML-guided optimization decisions integrate with LLVM passes in a scalable, user-friendly, and comprehensive framework. ACPO separates compiler-facing model abstractions from ML APIs and uses autotuner-generated benchmark data to train models for optimization passes. Combined results show performance benefit rose from 4% to 4.5% on Polybench and 2% to 2.4% on Cbench; ACPO-FI was not deployed on Polybench, so its gain was indirect.

  • Problem

    The paper asks how to make ML-guided optimization decisions integrate with LLVM passes in a scalable, user-friendly, and comprehensive framework.

    Source: §I · p. 1

  • Method

    ACPO separates compiler-facing model abstractions from ML APIs and uses autotuner-generated benchmark data to train models for optimization passes.

    Source: §I · p. 1, A. ACPO Training · p. 3

  • Results

    Combined results show performance benefit rose from 4% to 4.5% on Polybench and 2% to 2.4% on Cbench; ACPO-FI was not deployed on Polybench, so its gain was indirect.

    Source: C. Combined Results · p. 7

  • Takeaways & Limitations

    ACPO provides a flexible framework for LLVM users to apply ML to optimization passes with minimal effort.

    Source: §VI · p. 7

    The work optimizes performance but does not jointly optimize code size, which can increase with inlining and loop unrolling.

    Source: §VI · p. 8

Abstract

from arXiv · show

The key to performance optimization of a program is to decide correctly when a certain transformation should be applied by a compiler. This is an ideal opportunity to apply machine-learning models to speed up the tuning process; while this realization has been around since the late 90s, only recent advancements in ML enabled a practical application of ML to compilers as an end-to-end framework. This paper presents ACPO: An AI-Enabled Compiler Framework, a novel framework that provides LLVM with simple and comprehensive tools to benefit from employing ML models for different optimization passes. We first showcase the high-level view, class hierarchy, and functionalities of ACPO and subsequently, demonstrate \taco{a couple of use cases of ACPO by ML-enabling the Loop Unroll and Function Inlining passes used in LLVM's O3. and finally, describe how ACPO can be leveraged to optimize other passes. Experimental results reveal that the ACPO model for Loop Unroll can gain on average 4%, 3%, 5.4%, and 0.2% compared to LLVM's vanilla O3 optimization when deployed on Polybench, Coral-2, CoreMark, and Graph-500, respectively. Furthermore, by including both Function Inlining and Loop Unroll models, ACPO can provide a combined speedup of 4.5% on Polybench and 2.4% on Cbench when compared with LLVM's O3, respectively.

I. INTRODUCTION & RELATED WORKS

ACPO addresses the challenge of making ML-guided compiler optimization scalable and easy to integrate by separating model interfaces from LLVM and providing reusable features. Its workflow uses autotuning to collect training data, then routes compiler decisions through an ACPOModel and ML APIs.

I. INTRODUCTION & RELATED WORKS

  • The paper frames ACPO as a response to the need for scalable, user-friendly, comprehensive integration of ML-guided decisions into LLVM optimization passes.It contrasts existing end-to-end work with MLGO approaches built into passes [16], [28], [15], [8].Source: §I · p. 1
  • ACPO provides predefined features, libraries, and algorithms through a user-friendly interface, while keeping compiler and ML layers separate when inputs and outputs match.Users can revise the ML side without rebuilding LLVM after every change.Source: §I · pp. 1–2

II. RELATED WORK

  • CompilerGym uses RL to further reduce Cbench code size, whereas ACPO targets performance optimization in passes used standalone or within LLVM’s O3 pipeline [16].The paper also distinguishes ACPO’s performance objective from MLGO’s code-size objective [15].Source: §II · p. 2

III. PROPOSED METHODOLOGY

  • During inference, an ML-enabled pass creates an ACPOModel to collect code-region features and request advice through remote procedure call APIs to the ML framework.The compiler applies the transformation prescribed by the returned inference result.Source: §III · pp. 2–3
  • ACPO’s fine-grained autotuner searches parameters for individual code regions and records their effects to generate training data and target labels.The autotuner is built on OpenTuner.Source: §III · p. 3

A. ACPO Training

ACPO trains models from autotuner-collected benchmark data and uses them to advise compiler passes at inference time. Its abstractions separate model inputs and outputs from the compiler and underlying ML framework.

A. ACPO Training

  • Training uses benchmark programs and iterative autotuning to record optimization decisions and their speedups relative to LLVM O3, then processes the trial data for model training.The search overrides compiler heuristics to explore decisions for each application.Source: A. ACPO Training · p. 3

B. ACPO Inference

  • At inference, LLVM checks transformation legality first, then asks the ML framework for profitability advice and applies the transformation only when the result recommends it.An inference result can be a numeric prediction, such as an optimal loop unroll factor.Source: B. ACPO Inference · p. 3

C. ACPOModel

  • ACPOModel stores model-specific information, accepts selected features, and returns a configured advice type through getAdvice().Its virtual input-preparation and model-running methods allow custom model subclasses.Source: C. ACPOModel · p. 3

D. ML APIs & Framework

  • ACPOMLInterface abstracts the underlying ML framework from ACPOModel and the compiler, while ACPOCollectFeature offers roughly 130 handcrafted features across LLVM scopes.Users can select features individually or by scope; the Python-side separation supports adding tools without compiler changes.Source: D. ML APIs & Framework · pp. 3–4

IV. ACPO MODELS

ACPO is presented as applicable to passes with internal decisions and defined inputs and outputs, with loop unrolling serving as a detailed use case. Its model advises profitability after LLVM has checked legality and respected higher-priority user directives.

IV. ACPO MODELS

  • ACPO is proposed for passes with an internal decision mechanism and defined inputs and outputs; the paper demonstrates this with Loop Unroll and Function Inlining.The authors select these as important compiler optimizations used in state-of-the-art compilers, including LLVM.Source: §IV · p. 4

A. ACPO Loop Unrolling Model

  • Finding an optimal loop unroll count is a hard problem with nonpolynomial complexity,.Prior work has attempted to construct automatic heuristics for choosing the count,,,.Source: A. ACPO Loop Unrolling Model · p. 4
  • For loop unrolling, ACPOLUModel supplies unroll type and factor after legality checks, while explicit user counts and pragma directives retain priority.Only profitability checks are replaced by ML; LLVM continues to determine transformation legality.Source: A. ACPO Loop Unrolling Model · pp. 4–5

B. ACPO Function Inlining

ACPO instantiates a model for LLVM function inlining and invokes it after legality checks while retaining high-priority overrides. Evaluation compares vanilla O3 with O3 using vanilla or ML-enabled passes, individually and in combination.

B. ACPO Function Inlining

V. EXPERIMENTAL EVALUATION

  • The evaluation compares vanilla O3 with O3 using vanilla or ML-enabled passes, first individually and then in combination.Source: §V · p. 5

A. ACPO Loop Unrolling Model

ACPOLUModel predicts loop unrolling factors from handcrafted loop features, with experiments measuring training, overhead, accuracy, and generalization. It reports speedups on several loop-oriented benchmarks, including 5.4% on CoreMark and an average 3% on Coral-2.

A. ACPO Loop Unrolling Model

  • ACPOLUModel uses a five-layer ReLU neural network with a final Softmax layer to classify unrolling factors from 30 handcrafted loop features.The features are derived from LLVM IR; the framework also supports other feature types.Source: A. ACPO Loop Unrolling Model · p. 5

a) Data Collection & Setup

  • 100,000 unrolling configurations were collected from Polybench by tuning loop unroll counts in the set {0, 2, …, 64}.Measurements were single-threaded on ARMv8.2-A at 2.6 GHz; runs with more than 1% variance were repeated.Source: A. ACPO Loop Unrolling Model · p. 6

b) Analysis of Compilation Time Overhead

  • Compile-time overhead depends chiefly on module count because the model loads once per module, with loop count contributing to inference frequency.For Polybench’s bicg, two modules and nine loops resulted in 0.857 seconds of overhead.Source: A. ACPO Loop Unrolling Model · p. 6

c) ML Training and Cross Validation

  • 80% geometric-mean Top-1 accuracy with 7% standard deviation was measured on Polybench using leave-one-application-out cross-validation.The authors attribute lower accuracy in some applications to imbalanced training data and identify preprocessing and augmentation as possible mitigations.Source: Table III · p. 6, A. ACPO Loop Unrolling Model · p. 6

d) Generalization

  • 5.4% speedup on CoreMark, average 3% on Coral-2 including 18% on AMG, and 1.3% on Graph500’s SSSP were reported for generalization tests.Most performance benefit came from selecting the correct unrolling factor for hot loops; a hot AMG loop improved runtime by almost 15%.Source: Table IV · p. 6, A. ACPO Loop Unrolling Model · p. 6

B. ACPO Function Inlining Model

For function inlining, ACPO integrates the existing MLGOPerf model rather than creating a new model. The Cbench results table specifies comparisons against O3 and MLGO.

B. ACPO Function Inlining Model

  • ACPO uses the existing MLGOPerf model to demonstrate integration instead of designing a new model.Source: B. ACPO Function Inlining Model · p. 6, Table V · p. 7
  • MLGOPerf is a reinforcement-learning model that uses 13 function features to predict whether a callsite should be inlined.The listed features include block frequency, callsite height, and block counts.Source: B. ACPO Function Inlining Model · p. 6
  • The Cbench results table compares columns 1, 3, and 4 against O3, while column 2 compares ACPO-FI against MLGO.Source: Table V · p. 7

C. Combined Results

The combined deployment modestly improves performance on Polybench and Cbench, though interactions across LLVM’s pipeline make individual model contributions difficult to quantify.

C. Combined Results

  • 4.5% on Polybench and 2.4% on Cbench, up from 4% and 2%, respectively, with both ACPO models deployed versus LLVM O3.The combined results show a slight increase in performance benefit on both benchmarks.Source: C. Combined Results · p. 7
  • LLVM’s complex, repeated pass activation and intervening transformations make the separate benefits of jointly deployed models difficult to measure accurately.Passes between the models, including DCE, change IR code size, so combined code-size bloat does not equal the sum of the deployments.Source: C. Combined Results · p. 7
  • ACPO-FI is not deployed on Polybench’s small nested-loop kernels, yet the combined result shows an indirect benefit there.Function inlining precedes the first loop-unrolling pass in O3, and enhanced inlining may benefit subsequent passes.Source: C. Combined Results · p. 7
  • On security_sha, ACPO’s combined models achieve a 50% performance gain, while loop unrolling alone had already improved it by 7% over O3.The authors report that LLVM O3’s unroll-count and inliner thresholds constrain decisions in this benchmark.Source: C. Combined Results · p. 7

VI. DISCUSSION

The discussion presents ACPO as a flexible framework for bringing ML to LLVM optimization passes, with model deployment choices and practical trade-offs. It also notes that performance-oriented unrolling and inlining can increase emitted code size.

VI. DISCUSSION

  • ACPO aims to let LLVM developers apply ML to optimization passes with minimal effort and reproduce models across architectures.The framework lays out steps to regenerate data, retrain models, and deploy them with LLVM.Source: §VI · p. 7
  • Separating the ML framework from the compiler supports licensed tools, while ahead-of-time compilation provides an alternative to inference through Python named pipes.The named-pipe approach offers flexibility but adds slight compilation-time overhead; developers can choose between the modes.Source: §VI · p. 8
  • Once models are optimized, deployment can have efficient compilation overhead, while design and training can proceed without rebuilding the compiler after each change.The passage describes these as benefits at different stages of the model workflow.Source: §VI · p. 8
  • Performance-focused unrolling and inlining can increase emitted code size, but jointly optimizing performance and size is outside this work’s scope.The discussion suggests a future exploration strategy could account for both objectives using Pareto optimality.Source: §VI · p. 8

VII. CONCLUSION

The conclusion presents ACPO as a scalable, modular ML-guided framework for LLVM and reports average benchmark improvements from its Loop Unroll and Function Inline models. It also highlights reusable feature collection and utilities for other feature types.

VII. CONCLUSION

  • The two ACPO models achieved average improvements of 4% and %4.5 on Polybench, and 2.1% and 2.5% on Cbench.The conclusion attributes these results to ACPOLUModel and ACPOFIModel, respectively, without specifying a separate model-to-value mapping for each benchmark in the passage.Source: §VII · p. 9
  • ACPO is a scalable, modular ML-guided framework capable of optimizing LLVM performance.The conclusion says it enables Loop Unroll and Function Inline to use ML models through seamless integration.Source: §VII · p. 9
  • ACPO provides a generalized feature-collection class and utility classes for other feature types.Compiler developers can add features to an instantiated ACPOModel as needed.Source: §VII · p. 9

APPENDIX

The appendix describes ACPOMLInterface as a virtual abstraction between the compiler and an ML framework. ACPOModel uses it to configure models, pass input data, run inference, and retrieve results.

A. ACPO ML Interface Class

  • ACPOMLInterface abstracts data exchange between LLVM and the ML framework, whether the framework is linked as a library or runs as a separate process.Its virtual implementation is adapted to how a particular ML framework is accessed.Source: A. ACPO ML Interface Class · p. 11
  • ACPOModel uses the interface to load and free models and to register model inputs and outputs.The listed API includes model-loading and registration methods.Source: A. ACPO ML Interface Class · p. 11
  • After input data is ready, ACPOModel invokes inference and retrieves the model result through the interface.The interface exposes runModel and result-retrieval methods.Source: A. ACPO ML Interface Class · p. 11

B. ACPO Loop Unrolling Model

The Loop Unroll model obtains compiler advice through ACPO’s ML interface, which loads a model, supplies inputs, and runs inference. The compiler can choose how to handle inference failures; this implementation terminates with an error.

B. ACPO Loop Unrolling Model

  • getAdviceML() retrieves the ML interface, loads the configured model, sets its inputs, and invokes inference for the compiler.The model is specified in model.acpo, and results are retrieved through getModelResultI methods.Source: B. ACPO Loop Unrolling Model · p. 11
  • Model loading is skipped in effect when the framework has already loaded that model, avoiding repeated loading.The runModel call blocks until inference completes.Source: B. ACPO Loop Unrolling Model · p. 11
  • This implementation terminates with an error if inference fails, while ACPO developers may instead provide a fallback mechanism.Source: B. ACPO Loop Unrolling Model · p. 11

C. ACPO Function Inlining Model

The Function Inlining model uses ACPO to pass custom features to an ML model and return an inlining decision to LLVM. In this work, ML replaces only the pass’s profitability checks.

C. ACPO Function Inlining Model

Loading 2312.09982v4…