PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey of Machine Learning for Computer Architecture and Systems

Nan Wu, Yuan Xie

arXiv:2102.07952v2cs.LGcs.AR

TL;DR

Computer architecture and systems must support rapidly increasing ML compute demands while conventional heuristic design offers limited scalability and optimality. This survey organizes ML applications by predictive modeling versus design methodology and reviews techniques across architecture, systems, and design automation. It identifies broad opportunities, while noting that scalability and fully automated, holistic, portable design remain unresolved.

  • Problem

    Rapidly increasing ML compute demands and the limits of heuristic design motivate reconsidering how ML can transform computer architecture and systems design.

  • Method

    The paper provides a comprehensive survey that classifies ML by predictive modeling or design methodology and synthesizes applications across architecture, systems, data centers, compilers, and design automation.

  • Results

    The survey finds applications spanning fast modeling, resource management, placement, and analog and digital design automation, including reported speedups and generalization to unseen designs.

  • Takeaways & Limitations

    ML-based techniques can support a virtuous cycle in which powerful computers run ML efficiently while ML informs the design of next-generation computers.

  • Takeaways & Limitations

    Scalability remains challenging, and fully automated design still requires holistic system-wide optimization and migration across different systems.

Abstract

from arXiv · show

It has been a long time that computer architecture and systems are optimized for efficient execution of machine learning (ML) models. Now, it is time to reconsider the relationship between ML and systems, and let ML transform the way that computer architecture and systems are designed. This embraces a twofold meaning: improvement of designers' productivity, and completion of the virtuous cycle. In this paper, we present a comprehensive review of the work that applies ML for computer architecture and system design. First, we perform a high-level taxonomy by considering the typical role that ML techniques take in architecture/system design, i.e., either for fast predictive modeling or as the design methodology. Then, we summarize the common problems in computer architecture/system design that can be solved by ML techniques, and the typical ML techniques employed to resolve each of them. In addition to emphasis on computer architecture in a narrow sense, we adopt the concept that data centers can be recognized as warehouse-scale computers; sketchy discussions are provided in adjacent computer systems, such as code generation and compiler; we also give attention to how ML techniques can aid and transform design automation. We further provide a future vision of opportunities and potential directions, and envision that applying ML for computer architecture and systems would thrive in the community.

1 INTRODUCTION

The survey frames ML as both a tool for faster system modeling and a design methodology, motivated by rising AI compute demands and limits of heuristic design. It reviews these applications across architecture, data centers, adjacent systems, and design automation.

  • Motivation: AI training compute has increased exponentially since 2012, while Moore’s law is approaching its end, intensifying pressure on architecture and systems innovation.The reported doubling period for training compute is 3.4 months, compared with 18 months for transistor density under Moore’s law.
  • Motivation: Human expert designs based on intuition and heuristics cannot guarantee scalability and optimality for increasingly complicated systems.These designs require expertise in both ML and architecture or systems.
  • Motivation: Applying ML to architecture and systems aims to improve designers’ productivity and complete the feedback loop between systems for ML and ML-informed system design.The paper describes this as reducing manual design burdens while reconsidering the relationship between ML and system design.
  • Taxonomy: Existing work divides into ML for fast, accurate system modeling and ML as the design methodology.System modeling targets metrics such as power consumption, latency, and throughput.
  • Scope: The survey organizes ML applications by their role, summarizes design problems and techniques, and covers data centers, adjacent systems, and analog and digital design automation.It also discusses challenges and future prospects for architecture and system design.

2 DIFFERENT ML TECHNIQUES

The paper distinguishes supervised, unsupervised, and reinforcement learning by their data and model-building processes, then describes their typical roles in architecture and systems design. Supervised learning predicts from labels, unsupervised learning finds patterns without labels, and reinforcement learning learns sequential decisions through environment interaction.

  • Frameworks: The three general ML frameworks are supervised learning, unsupervised learning, and reinforcement learning, distinguished by sampled data and how those data build models.Technique selection can depend on hardware resources, data, implementation overhead, and performance targets.
  • Supervised Learning: Supervised learning maps inputs to outputs using labeled datasets and generalizes the learned rules to unseen inputs.In system design, it is commonly used for performance modeling, configuration prediction, and higher-level behavior prediction.
  • Supervised Learning: Common supervised techniques include regression, SVM or SVR, decision trees, artificial neural networks, and ensemble learning.Different models prefer different feature types; SVMs and ANNs generally perform better with multidimensional continuous features, while logic-based systems favor discrete or categorical features.
  • Unsupervised Learning: Unsupervised learning finds previously unknown patterns in unlabeled datasets through methods such as clustering and PCA.Clustering groups similar objects, while PCA represents high-dimensional variables with a few orthogonal variables capturing most variability.
  • Unsupervised Learning: Semi-supervised learning combines a small amount of labeled data with a large amount of unlabeled data to reduce human effort and improve accuracy.Because unsupervised learning lacks labels, it is difficult to measure model performance and decide when to stop training.
  • Reinforcement Learning: Reinforcement learning models sequential decision-making as agent interaction with an environment, using states, actions, policies, and rewards.Its suitability for control and scheduling lets agents explore design spaces and encode optimization goals through reward functions.

3 ML FOR FAST SYSTEM MODELING

ML-based fast system modeling replaces costly simulation for performance estimation and design-space exploration, using learned predictions across processors, GPUs, memory systems, and networks. The surveyed work primarily uses supervised regression and related models to estimate metrics and support optimization decisions.

  • System Modeling and Performance Prediction: Cycle-accurate simulators provide accurate estimates but run multiple orders of magnitude slower than native execution, whereas ML models balance prediction accuracy and simulation cost.This enables exploration of large configuration spaces and nonlinear configuration effects.
  • Optimization and Design-Space Exploration: ML performance models support resource allocation, task scheduling, auto-tuning, power-constrained adaptation, and rapid exploration across architecture and memory-network configurations.One regression-based uncore exploration method reduces exploration time by up to four orders of magnitude.
  • System Modeling and Performance Prediction: Accurate and fast performance estimation supports system optimization and design-space exploration as systems and workloads become more complex.ML-based techniques can surpass commonly used cycle-accurate simulators while retaining reasonable simulation costs.
  • GPU Modeling: GPU modeling distinguishes cross-platform predictions for CPU-to-GPU offloading from GPU-specific predictions for metrics estimation and design-space exploration.Cross-platform prediction addresses whether an application benefits enough from GPU execution to justify porting effort.
  • Processor Performance Modeling: Regression techniques are mainstream for predicting performance metrics from micro-architectural parameters or other features because they provide accurate estimates with reasonable training costs.ANNs and nonlinear regression commonly predict throughput, latency, power, and energy, with explainability and generalization varying across model types.
  • Data-Driven Modeling: Recent data-driven approaches include recurrent models for basic-block throughput and generative models that address data scarcity while improving prediction of optimal configurations.The surveyed work also emphasizes that domain knowledge can aid system optimization even without improving overall prediction accuracy.

3.3 Performance Modeling in Chip Design and Design Automation

ML accelerates performance estimation in chip design and design automation where synthesis, simulation, and layout analysis are expensive. The surveyed methods learn from pre-layout, HLS, synthesis, placement, routing, and implementation information to predict design metrics and guide exploration.

  • Analog Circuit Analysis: Post-layout simulations are accurate but extremely time-consuming, while schematic-based estimates become less accurate with device scaling, creating a bottleneck for analog design iteration.ML methods are discussed as a way to bridge pre-layout and post-layout performance differences.
  • High-Level Synthesis (HLS): HLS transforms behavioral languages such as C/C++/SystemC into RTL designs, but synthesis time limits sufficient design-space exploration and motivates ML-based performance estimation.Input features come from HLS directives, intermediate representations, and HLS reports.
  • High-Level Synthesis (HLS): Random forests and other ML models predict HLS metrics including area, latency, throughput, logic utilization, post-implementation resource utilization, timing, power, and throughput-to-area ratio.Transfer learning supports reuse of knowledge across prior design experiences.
  • High-Level Synthesis (HLS): Cross-platform HLS prediction uses CPU execution measurements or heterogeneous-platform data to estimate FPGA cycle counts, power, or application speedup.HLSPredict targets the same FPGA platform during training and testing, whereas XPPE considers different FPGA platforms.
  • Digital Design and Physical Synthesis: Logic- and physical-synthesis studies use CNNs, LSTMs, regression, decision trees, MARS, and conditional GANs to predict delay, area, design-rule violations, timing, power, and routing congestion.The inputs include synthesis flows, global-routing results, placement data, congestion maps, and post-placement images.

4 ML AS DESIGN METHODOLOGY

ML can serve not only as a predictive modeling tool but also as the design methodology for increasingly complex computer architecture and systems. This direction targets systems capable of designing, configuring, adapting, diagnosing, and repairing themselves.

  • ML as Design Methodology: Human design and optimization become expensive and inefficient as computer architecture and systems grow more complicated.The surveyed design-methodology work responds by applying ML directly to system design and configuration.
  • ML as Design Methodology: The envisioned systems can design and configure themselves, adapt behavior to workloads or user constraints, diagnose failures, and repair detected failures.These capabilities define the broad scope of ML used as a design methodology.

4.1 Memory System Design

ML-based memory-system design addresses the memory wall and changing workload demands through learned cache, prefetching, controller, translation, and interconnect optimizations. The surveyed methods combine supervised learning, reinforcement learning, representation learning, and online adaptation.

  • Memory System Design: The memory wall arises because computation is orders of magnitude faster than memory access, motivating hierarchical memory systems and optimization across memory levels.Growing workload variety and size make heuristic designs less able to meet performance demands.
  • Cache Management: Cache optimization studies improve replacement policies and prefetching policies using perceptrons, contextual-bandit reinforcement learning, and LSTM-based regression or classification.Context-based prefetching models semantic locality through correlations in program semantics and data structures.
  • Memory Controllers: Reinforcement-learning memory controllers select legal DRAM commands according to expected long-term performance benefits and can optimize multiple objectives through reward design.The controllers are intended to adapt to dynamically changing workloads.
  • Other Memory-System Components: Learned index structures can achieve almost 100% accuracy for tested virtual addresses, but their unacceptably long inference latency leaves practical hardware implementation as future work.Other memory-system methods target data-movement energy in interconnects and related system optimizations.

4.2 Branch Prediction

ML-based branch predictors extend beyond limited-history traditional predictors by using richer histories and learned models to improve prediction accuracy and address difficult branch patterns.

  • Perceptron/MLP-based predictors handle long branch histories with reasonable hardware budgets and outperform prior non-ML predictors.
  • An MLP can predict branch direction at compile time from static program and control-flow features, while dynamic predictors use perceptron methods.
  • Ahead pipelining and path-history-based perceptron selection can further improve predictor latency and accuracy.
  • Piecewise linear prediction generalizes perceptron-based methods to improve accuracy on non-linearly separable branches.
  • CNN helper predictors target hard-to-predict static branch instructions that current perceptron/MLP predictors systematically mispredict.

4.3 NoC Design

ML techniques address NoC power, routing, reliability, and design optimization as growing core counts and dynamic workloads make conventional approaches less adaptable.

  • Increasing core counts make NoCs crucial for inter-core and memory communication, while communication energy scales slower than computation energy.
  • ANN-based link management dynamically computes regional utilization thresholds, adapting link activation to changing workloads.
  • Q-routing uses local delivery-time estimates to minimize total packet delivery time while handling irregular topologies and higher network loads than shortest-path routing.
  • Per-router Q-learning agents select among four fault-tolerant modes to minimize end-to-end packet latency and power consumption.
  • MLNoC uses supervised learning trained on thousands of real-world and synthetic SoC designs to quickly find near-optimal NoC designs under multiple goals.
  • STAGE alternates local base search with meta-search that uses SVR-learned evaluation functions to optimize link placement in 3D NoCs.

4.4 Resource Allocation or Management

ML-based resource management and task allocation adapt policies across power, configuration, workflow, placement, and scheduling problems shaped by complex workloads and hardware interactions.

  • Resource allocation coordinates architectures and workloads, whose growing complexity and interaction make optimization difficult.
  • Power management applies ML because energy is a runtime cost and may be subject to hard or soft power budgets.
  • PACSL improves energy-delay product by 22% on average, up to 46%, over independently applying DVS to each component.
  • Pack & Cap uses multinomial logistic regression to identify optimal thread-packing and DVFS operating points under arbitrary power caps.
  • Semi-supervised RL achieves linear complexity with core count while maximizing throughput under power constraints and coordinating cores with uncore components.
  • JouleGuard uses a multi-arm bandit to select energy-efficient configurations and maximize compute accuracy within application energy budgets.
  • ML methods explore large design spaces for automated resource management and task allocation across diverse workloads and performance targets.
  • Statistical machine learning quickly finds multi-core configurations optimizing runtime and energy efficiency without application or microarchitecture domain knowledge.

4.5 Data Center Management

Data-center management uses ML and RL to address scaling challenges in resource allocation, traffic optimization, caching, and workload-dependent quality of experience.

  • Rapid data-center expansion makes problems that are minor on a single machine increasingly challenging.
  • Early resource-allocation work models dynamic server assignment as RL with service-level utility rewards and local value-function estimates.
  • Two-level RL systems support traffic optimization by combining locally trained end-host decisions with broader data-center coordination.
  • Future-content popularity prediction can be combined with cache policies to make smarter content-eviction decisions.
  • Video workloads naturally fit RL because fluctuating network conditions, multiple QoE goals, coarse actions, and long-term decision effects must be handled together.

4.6 Code Generation and Compiler

ML-based methods are applied to code completion, code generation, program translation, and compiler optimization. These approaches use language-modeling, neural translation, and reinforcement-learning techniques to automate or improve programming-system tasks.

  • Code Generation and Compiler: Code completion uses statistical language models to select constraint-satisfying sentences for program holes.Explored models include N-gram, RNN, and combined approaches.
  • Code Generation and Compiler: CLgen trains LSTM models on handwritten code to learn OpenCL semantics and structures, then generates programs by iterative sampling.
  • Code Generation and Compiler: NMT-based methods translate programs between languages, including Java to C#, CUDA to OpenCL, and binaries to high-level code.Tree-to-tree and sequence-to-sequence architectures support these translation tasks.
  • Code Generation and Compiler: ML optimizes compilers through instruction scheduling, heuristic selection, optimization ordering, hot-path identification, auto-vectorization, and application-specific compilation.Instruction-scheduling preferences can be computed with temporal-difference reinforcement learning.

4.7 Chip Design and Design Automation

ML is being used to automate analog and digital chip-design workflows that are increasingly complex, manual, and expensive. The reviewed methods span circuit generation, transistor sizing, routing, high-level synthesis, logic synthesis, placement, and clock-tree optimization.

  • 4.7.1 Analog Design: Analog design remains difficult because it has large topology and sizing spaces, lacks a general optimization framework, and requires substantial manual expertise.ML techniques are introduced to expedite analog design automation.
  • 4.7.1 Analog Design: RL and GNNs enable automatic transistor sizing that can generalize across circuit topologies or technology nodes.
  • 4.7.1 Analog Design: 40× speedup over a traditional genetic algorithm is reported for an analog design automation approach.
  • 4.7.1 Analog Design: GeniusRoute uses a generative neural network and VAE-learned routing likelihoods to automate analog routing, achieving competitive manual-layout performance and cross-functionality generalization.
  • 4.7.2 Digital Design: Digital-design automation is organized across HLS, logic synthesis, and physical synthesis, with ML selecting directives, optimizers, and placement or CTS configurations.These tasks target constrained optimization of design quality, resources, timing, power, wirelength, or skew.
  • 4.7.2 Digital Design: DREAMPlace achieves over 30× speedup without quality degradation compared to RePlAce, while deep RL placement generalizes to unseen netlists but is several times slower.Other methods use RL for macro placement and design-rule-violation fixing.

5 DISCUSSION AND POTENTIAL DIRECTIONS

The discussion identifies data, scalability, interpretability, deployment, and automation as central challenges and opportunities for ML-based computer architecture and systems. It envisions holistic, portable, and agile design methods that improve practical deployment and designers’ productivity.

  • ML applications span the development and deployment stack, including data, algorithms, implementation, and design targets.
  • 5.1 Bridging Data Gaps: Expensive simulation creates data scarcity, motivating small-data algorithms, active learning, Bayesian optimization, and synthetic-data generation.
  • 5.2 Developing Algorithms: Hybrid methods, including combinations of ML techniques with heuristics, are presented as opportunities for more flexible system optimization.
  • 5.2 Developing Algorithms: Scalability remains challenging, with multi-level techniques and transfer learning proposed to reduce computation or amortize pre-training costs.
  • 5.3 Improving Implementations and Deployments: Practical deployment requires scenario-aware complexity limits, model maintenance, and attention to workload variation and hardware aging.
  • 5.5 Facilitating General Tool Design and Hardware Agile Development: Future design automation should support system-wise holistic optimization, portability across platforms, and rapid hardware development.

6 CONCLUSION

The conclusion frames ML for computer architecture and systems as a reciprocal relationship: systems enable ML at scale, while ML can transform system design. Existing work is organized around fast modeling and ML-based design methodology, with the goal of completing this virtuous cycle.

  • Existing work falls into ML-based fast modeling and ML-based design methodology that directly uses ML as a design tool.
  • The paper envisions a virtuous cycle in which ML runs efficiently on powerful computers while helping design next-generation computers.
Loading 2102.07952v2…