Source-linked AI summary
AutoCkt: Deep Reinforcement Learning of Analog Circuit Designs
Keertana Settaluri, Ameer Haj-Ali, Qijing Huang, Kourosh Hakhamaneshi, Borivoje Nikolic
TL;DR
Analog circuit sizing is hindered by lengthy expert-driven iteration and limitations in existing automated methods, especially for post-layout design. AutoCkt uses deep reinforcement learning with sparse design-space sampling and transfer learning to optimize across environments. It achieves broad target coverage and substantially higher sample efficiency, including post-layout designs that pass LVS.
Problem
Analog sizing remains dependent on lengthy expert iteration, while existing methods can be schematic-only, inaccurate, sample-inefficient, or not generalizable.
Method
AutoCkt trains a reinforcement-learning agent on a sparse design-space subsample and transfers knowledge from schematic simulation to post-layout environments.
Results
AutoCkt reaches at least 96.3% of tested schematic design goals and is approximately 40X faster than a traditional genetic algorithm, while post-layout design is 9.6X faster than the state-of-the-art.
Takeaways & Limitations
AutoCkt designs analog circuits across multiple topologies and simulation environments, including layout-parasitic-aware designs with 40 LVS-passed operational amplifiers.
Abstract
from arXiv · showhide
Domain specialization under energy constraints in deeply-scaled CMOS has been driving the need for agile development of Systems on a Chip (SoCs). While digital subsystems have design flows that are conducive to rapid iterations from specification to layout, analog and mixed-signal modules face the challenge of a long human-in-the-middle iteration loop that requires expert intuition to verify that post-layout circuit parameters meet the original design specification. Existing automated solutions that optimize circuit parameters for a given target design specification have limitations of being schematic-only, inaccurate, sample-inefficient or not generalizable. This work presents AutoCkt, a machine learning optimization framework trained using deep reinforcement learning that not only finds post-layout circuit parameters for a given target specification, but also gains knowledge about the entire design space through a sparse subsampling technique. Our results show that for multiple circuit topologies, AutoCkt is able to converge and meet all target specifications on at least 96.3% of tested design goals in schematic simulation, on average 40X faster than a traditional genetic algorithm. Using the Berkeley Analog Generator, AutoCkt is able to design 40 LVS passed operational amplifiers in 68 hours, 9.6X faster than the state-of-the-art when considering layout parasitics.
I. INTRODUCTION
Analog circuit sizing remains time-consuming and dependent on expert iteration, while existing automation methods trade off efficiency, accuracy, generalizability, or layout awareness. AutoCkt addresses the need for a sample-efficient, accurate, generalizable sizing method without constraint-generation overhead.
- Advanced technology nodes increase circuit-design time because complex design rules and layout parasitics require substantial tapeout effort.
- Knowledge-based methods encode designer expertise as equations but require substantial manual effort when introducing a new design.
- Equation-based optimization methods are efficient after formulation but apply to only a few circuits and require constraint equations.
- Simulation-based genetic algorithms are sample inefficient, lack convergence guarantees, and must restart when the target changes.
- Existing learning- and layout-aware approaches can require retraining, approximate parasitics, or exhaustive lookup tables, motivating a sample-efficient and accurate alternative.
A. Our Contributions
AutoCkt learns analog design-space tradeoffs through sparse subsampling and reinforcement learning, enabling faster convergence, broad target coverage, and post-layout optimization through transfer learning.
- AutoCkt trains over a sparse subsample of the design space to reduce deployment convergence time for new design specifications.
- AutoCkt models design-space tradeoffs, converges approximately 40X faster than a traditional evolutionary algorithm, and reliably reaches many target specifications.
- Transfer learning lets AutoCkt account for layout parasitics and operate 9.6X faster than the state-of-the-art.
II. THE PROPOSED FRAMEWORK
AutoCkt consists of a reinforcement learning agent and a simulation environment that together support analog circuit design against target specifications.
- The framework’s system-level diagram contains two main blocks: a reinforcement learning agent and a simulation environment.
A. The Reinforcement Learning Agent
AutoCkt uses an RL agent that observes simulated circuit performance and target specifications, takes constrained parameter actions, and learns through accumulated rewards during training and deployment.
- Reinforcement learning uses trial-and-error interaction between an agent and environment, with simulation in the loop to verify outputs.
- At each step, the neural-network agent observes the environment state, acts, receives a new state, and accumulates rewards across a trajectory.
- AutoCkt discretizes N tunable circuit parameters into K grids and represents M normalized target design specifications.
- The agent uses simulated performance, target specifications, and current parameters to increment, decrement, or retain each parameter, subject to optional circuit constraints.
- Training samples 50 target specifications, generates trajectories from the target set, and uses reward accumulation to train the agent.
- The reward increases as observed performance approaches the target, and training ends when mean reward reaches 0 so all targets are consistently satisfied.
- During deployment, trained trajectories use unique targets sampled from O∗, and final simulated outcomes are compared with targets for counting.
B. Simulation Environment
AutoCkt interfaces with schematic-level and layout-aware simulation environments, including predictive technology models, Spectre, and BAG.
- AutoCkt supports predictive-model and Spectre schematic simulations, plus BAG simulations that automatically include layout parasitics.BAG runs simulations in Cadence with layout parasitics automatically.
III. EXPERIMENTS
The experiments evaluate AutoCkt across multiple simulation environments and circuit topologies, with repeated training sessions to assess robustness.
- AutoCkt is evaluated across three simulation environments and three circuit topologies.Training sessions are repeated several times to assess robustness to random-seed variation.
A. Transimpedance amplifier
For a 45nm transimpedance amplifier, AutoCkt learns from sparse design-space sampling and efficiently generalizes to randomly selected target specifications.
- A. Transimpedance amplifier: Figure 5 plots mean episode reward for the transimpedance amplifier, increasing above zero after training.A positive reward indicates that the agent learned to reach the positive goal state.
- A. Transimpedance amplifier: The transimpedance amplifier varies transistor width and multiplier, resistor-series and parallel counts, and three target specifications.The target specifications are settling time, cutoff frequency, and input-referred noise.
- A. Transimpedance amplifier: 25.1× speedup over a vanilla genetic algorithm is achieved in sample efficiency across 500 randomly chosen target specifications.Sample efficiency is measured by the number of simulations required to converge to the target specification.
- A. Transimpedance amplifier: 97.4% of the transimpedance amplifier design space is generalized by the trained agent.The genetic-algorithm comparison uses its best result across swept initial population sizes and several target specifications.
B. Two stage operational amplifier
For a two-stage operational amplifier, AutoCkt handles a large discrete action space and reaches most unseen targets while converging substantially faster than a traditional genetic algorithm.
- B. Two stage operational amplifier: The two-stage amplifier uses transistor widths and a compensation capacitor to target gain, unity-gain bandwidth, phase margin, and bias current.The action space contains 10^14 possible values, making random parameter generation infeasible.
- B. Two stage operational amplifier: 1.3 hours of wall-clock training time is achieved on an 8-core CPU by running environments in parallel.Schematic simulation takes 25 ms, while the agent requires approximately 10^4 environment steps to reach mean reward zero.
- B. Two stage operational amplifier: 963 of 1000 unseen target specifications are reached, corresponding to 20× generalization beyond the specifications used during training.The experiment uses randomly generated targets within the training range and excludes phase margin from the 3D plot because it has only a lower-bound requirement.
- B. Two stage operational amplifier: The 37 unreached targets cluster where bias current is very low, suggesting those requirements may be unreachable under the power constraint.For these points, the agent attempts to meet gain and bandwidth while minimizing power.
C. Two stage OTA with negative gm load
AutoCkt is evaluated on a challenging two-stage operational amplifier with a negative gm load, spanning a large parameter space and multiple target specifications. It converges substantially faster than a traditional genetic algorithm.
- Circuit and setup: Figure 9 presents the amplifier schematic and corresponding action space used for the evaluation.The action-space ranges are tied to circuit parameters and target design specifications.
- Circuit and setup: The evaluated amplifier uses negative gm and diode-connected loads in its first stage, introducing positive feedback and greater layout-parasitic sensitivity.The topology is described as more challenging than a traditional amplifier.
- Circuit and setup: 10^11 parameter combinations define the circuit’s action space across gain, unity gain bandwidth, and phase margin targets.The specified ranges are gain [1, 40] V/V, unity gain bandwidth [1.0e6, 2.5e7] Hz, and phase margin [60, 75]°.
- Results: 40.6× faster convergence than a traditional genetic algorithm is achieved for reaching a target specification.AutoCkt takes approximately 10 simulations on average to converge to a solution.
D. Two stage operational amplifier with negative gm load and layout parasitics
AutoCkt transfers knowledge from inexpensive schematic simulations to post-layout PEX simulation without additional training. In the negative-gm amplifier experiment, it reaches LVS-passed designs while retaining strong sample efficiency despite layout-parasitic differences.
- Results: 38× separates schematic simulation time from BAG simulation with layout parasitics, at 2.4 seconds versus an average 91 seconds.The authors identify this simulation-time gap as a scalability challenge for prior approaches.
- Transfer learning: Transfer learning moves an agent trained with inexpensive schematic simulations into a PEX environment without training after the environment changes.The transferred agent is deployed using the Berkeley Analog Generator and layout parasitics.
- Transfer learning: Training across a range of phase margins transfers better than training with only a 60° lower bound.The authors attribute this to greater exploration of the design space.
- Transfer learning: 11 time steps suffice for the transferred agent to converge to one target design objective that meets specification.The result is illustrated by a sample trajectory in Figure 14.
- Transfer learning: The transferred agent generally converges more slowly than its schematic counterpart because layout parasitics are included.The authors posit that learned parameter-specification tradeoffs remain useful despite potentially large schematic-to-PEX differences.
- Results: 9.56× greater sample efficiency than the prior state-of-the-art is reported, while AutoCkt obtains 40 LVS-passed designs from 40 random targets.The experiment uses a single-core CPU and completes in under three days without parallelization.
IV. CONCLUSION
The conclusion presents AutoCkt as a reinforcement-learning framework for analog circuit design that improves sample efficiency across topologies and simulation environments, including layout parasitics.
- Conclusion: 40× greater average sample efficiency than a genetic algorithm is reported across three circuit topologies and different simulation environments.The conclusion also reports 9.6× greater sample efficiency than the state-of-the-art when considering layout parasitics.
- Conclusion: 40 LVS-passing two-stage OTA designs are produced on a one-core CPU in under three days.The designs use a negative gm load and account for layout parasitics through transfer learning.