PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning to Design Circuits

Hanrui Wang, Jiacheng Yang, Hae-Seung Lee, Song Han

arXiv:1812.02734v5cs.LG

TL;DR

Analog IC parameter search is difficult because simulations are slow and training data are scarce, while circuit IP is often proprietary. L2DC trains an RL agent to optimize circuit parameters through simulation feedback, and reports 250× higher sample efficiency than grid-search-aided human design alongside comparable or better performance than several baselines.

  • Problem

    Analog circuit optimization has limited training data because simulations are slow and much circuit IP is proprietary.

    Source: §1 · pp. 1–2

  • Method

    L2DC trains an RL agent to generate circuit parameters from simulation observations and rewards, without prior circuit-optimization rules.

    Source: §1 · p. 2

  • Results

    250× higher sample efficiency than grid-search-aided human design, with comparable or better performance than human experts, Bayesian Optimization, and random search.

    Source: §1 · p. 2, §5 · p. 5

  • Takeaways & Limitations

    Across two transimpedance amplifiers, the agent first meets hard constraints and then optimizes good-to-have targets such as area and power.

    Source: §4 · p. 5, §5 · p. 5

    Bayesian Optimization was run for only 1,160 samples because its runtime grows cubically with sample count.

    Source: §3.2 · p. 4

Abstract

from arXiv · show

Analog IC design relies on human experts to search for parameters that satisfy circuit specifications with their experience and intuitions, which is highly labor intensive, time consuming and suboptimal. Machine learning is a promising tool to automate this process. However, supervised learning is difficult for this task due to the low availability of training data: 1) Circuit simulation is slow, thus generating large-scale dataset is time-consuming; 2) Most circuit designs are propitiatory IPs within individual IC companies, making it expensive to collect large-scale datasets. We propose Learning to Design Circuits (L2DC) to leverage reinforcement learning that learns to efficiently generate new circuits data and to optimize circuits. We fix the schematic, and optimize the parameters of the transistors automatically by training an RL agent with no prior knowledge about optimizing circuits. After iteratively getting observations, generating a new set of transistor parameters, getting a reward, and adjusting the model, L2DC is able to optimize circuits. We evaluate L2DC on two transimpedance amplifiers. Trained for a day, our RL agent can achieve comparable or better performance than human experts trained for a quarter. It first learns to meet hard-constraints (eg. gain, bandwidth), and then learns to optimize good-to-have targets (eg. area, power). Compared with grid search-aided human design, L2DC can achieve $\mathbf{250}\boldsymbol{\times}$ higher sample efficiency with comparable performance. Under the same runtime constraint, the performance of L2DC is also better than Bayesian Optimization.

1 Introduction

Analog circuit design requires difficult trade-offs among coupled metrics, while slow simulation makes search costly. L2DC uses reinforcement learning to generate circuit parameters and reports comparable or better performance than expert and automated baselines, with higher sample efficiency than grid-search-aided design.

1 Introduction

  • Circuit metrics are strongly coupled, so improving one aspect can deteriorate another and requires trade-offs informed by design experience.The introduction cites Analog Circuits Octagon for these coupled relations.Source: §1 · p. 1
  • Slow circuit simulation makes random or exhaustive search impractical, particularly for complex circuits such as ADCs, DACs, and PLLs.The passage identifies simulation time as a constraint on search.Source: §1 · p. 2
  • Existing search approaches face limitations including local optima, slow convergence, stochastic outcomes, or prohibitive computational complexity.The introduction discusses particle swarm intelligence, simulated annealing, evolutionary algorithms [4], and Bayesian Optimization.Source: §1 · p. 2
  • L2DC trains an RL agent from scratch to generate circuit parameters from simulation observations and optimize them using rewards based on circuit metrics.Observations include DC operating points, AC responses, and transistor states.Source: §1 · p. 2
  • 250× higher sample efficiency than grid-search-aided human design, with comparable or better performance than human experts, Bayesian Optimization, and random search.The experiments cover two circuit environments.Source: §1 · p. 2

2 Methodology

L2DC formulates analog circuit design as constrained optimization, then trains a DDPG agent through repeated interaction with a multi-step simulation environment. The method supplies global and transistor-local observations, stages reward priorities around hard constraints, and uses an encoder-decoder actor to generate parameters.

2.1 Problem Definition

  • The design objective maximizes the sum of spec-satisfaction ratios for optimization targets while satisfying all hard constraints.For each spec, q_c is metric/spec when the metric should be larger, and spec/metric otherwise.Source: §2.1 · p. 2

2.2 Multi-Step Simulation Environment

  • The multi-step environment provides global simulation features and per-transistor observations, then scales normalized joint actions to each parameter’s allowed range.Global features include DC operating points, AC responses, and the environment step; local features include transistor states and capacitances.Source: §2.2 · p. 3, Figure 2 · p. 3
  • The score prioritizes hard constraints before optimization targets; once constraints are met, the agent focuses on optimization targets.Step reward is the change in scalar score between consecutive steps.Source: §2.2 · p. 3

2.3 DDPG Agent

  • The DDPG actor uses an encoder-decoder to translate observations into actions, generating transistor widths and lengths in signal-propagation order.Truncated uniform output noise and parameter noise are used for exploration.Source: §2.3 · p. 3
  • Under the same runtime constraint, DDPG satisfies all spec hard constraints with the smallest gate area and achieves the highest score.The table caption reports 250× higher sample efficiency than human design.Source: Table 1 · p. 4

3 Experimental Results

L2DC is evaluated against random search, Bayesian Optimization (BO), and human designs on three-stage and two-stage transimpedance amplifiers. It meets the hard constraints in both settings and uses fewer simulations than the human designs.

3.1 Three-Stage transimpedance Amplifier

  • The three-stage evaluation uses a Stanford EE214A project amplifier and compares L2DC with random search, grid search-aided human design, and MACE, a BO algorithm.MACE uses a batch size of 10 and 50 initialization samples.Source: §3.1 · p. 4
  • 250× fewer simulations than grid search-aided human design; DDPG meets all hard constraints and achieves the highest score on the three-stage amplifier.Its power is slightly higher, but remains within the course specification; its gate area is smaller than the human design's.Source: §3.1 · p. 4

3.2 Two-stage transimpedance Amplifier

  • The two-stage evaluation uses a Stanford EE214B design-contest circuit, with noise, gain, peaking, and power as hard constraints and bandwidth as an optimization target.The comparison runs DDPG, BO, and random search for about 30 hours; BO is limited to 1,160 samples under the matched runtime constraint because of its sample-count time and space complexity.Source: §3.2 · p. 4
  • DDPG meets all hard constraints and reaches 5.78 GHz bandwidth, 97.143% of the human result, on the two-stage amplifier.Random search and BO fail the noise hard constraint; DDPG's sample efficiency is 25× higher than the human design's.Source: §3.2 · p. 5, Table 2 · p. 5

4 Discussion

In the three-stage amplifier, the agent first raises gain and bandwidth, accepting higher power; after meeting those hard constraints, it reduces power while holding gain and bandwidth steady.

4 Discussion

  • After hard constraints are met, the agent keeps gain and bandwidth constant while optimizing power downward.Before that point, power rises as the agent increases gain and bandwidth.Source: §4 · p. 5
  • The learning curves indicate a two-phase strategy: meet bandwidth and gain constraints first, then optimize the soft power target.The transition is marked by the step at which the hard constraints are satisfied.Source: §4 · p. 5
  • The observed sequence suggests the RL agent learned strategies for analog circuit optimization.Source: §4 · p. 5

5 Conclusion

L2DC uses reinforcement learning to optimize transistor parameters and is evaluated on two transimpedance amplifiers. The reported results include stronger performance than human experts, random search, and BO, with higher sample efficiency than grid search-aided human design.

5 Conclusion

  • L2DC is reported to outperform random search, BO, and human experts on both two-stage and three-stage transimpedance amplifiers.Under the same runtime constraint, it also achieves better circuit performance than BO.Source: §5 · p. 5
  • L2DC uses reinforcement learning to optimize circuit parameters through repeated observations, parameter generation, rewards, and model updates.Source: §5 · p. 5
  • About 250× higher sample efficiency than grid search-aided human design is reported with comparable performance.Source: §5 · p. 5

Related papers

Loading 1812.02734v5…