Source-linked AI summary
DRiLLS: Deep Reinforcement Learning for Logic Synthesis
Abdelrahman Hosny, Soheil Hashemi, Mohamed Shalan, Sherief Reda
TL;DR
Logic synthesis flow design is difficult because optimization sequences create an exponentially large search space and different flows can yield different area and delay. DRiLLS uses an A2C agent to explore optimization flows and minimize area under a delay constraint. On ten representative benchmarks, it meets the delay constraint in all designs and improves area by an average of 13.19%.
Problem
Optimization permutations create an exponentially large search space, and different flows can produce divergent area and delay results.
Source: A. Problem Statement · pp. 1–2Method
DRiLLS frames logic synthesis as a game-like environment in which an A2C agent uses AIG features and a multi-objective reward to select transformations for area reduction under a delay constraint.
Source: §I · p. 1, §IV · p. 3Results
13.19% average area improvement; DRiLLS meets the delay constraint in all designs and outperforms EPFL best results except on Barrel shifter.
Source: B. Comparison to Other Techniques · p. 5Takeaways & Limitations
The paper concludes that reinforcement learning can be used for combinatorial optimization of hardware circuit designs, with potential application to related physical synthesis tasks.
Source: §VI · p. 6
Abstract
from arXiv · showhide
Logic synthesis requires extensive tuning of the synthesis optimization flow where the quality of results (QoR) depends on the sequence of optimizations used. Efficient design space exploration is challenging due to the exponential number of possible optimization permutations. Therefore, automating the optimization process is necessary. In this work, we propose a novel reinforcement learning-based methodology that navigates the optimization space without human intervention. We demonstrate the training of an Advantage Actor Critic (A2C) agent that seeks to minimize area subject to a timing constraint. Using the proposed methodology, designs can be optimized autonomously with no-humans in-loop. Evaluation on the comprehensive EPFL benchmark suite shows that the agent outperforms existing exploration methodologies and improves QoRs by an average of 13%.
I. INTRODUCTION
Logic synthesis optimization flows require circuit-specific tuning because transformation sequences affect quality of results and no fixed sequence works best for every circuit. DRiLLS uses reinforcement learning to generate optimization flows, targeting area reduction under a delay constraint and evaluating against existing techniques on EPFL benchmarks.
I. INTRODUCTION
- Logic synthesis converts a high-level design into a gate-level representation through pre-mapping, technology mapping, and post-mapping optimization.Pre-mapping transforms an And-Inverter Graph (AIG) to reduce graph size and area while adhering to a delay constraint.Source: §I · p. 1
- Transformation choices and their order affect QoR, and no predefined sequence produces the best QoR for every circuit, making flow tuning complex.The authors attribute this challenge to an exponentially large search space and increasingly complex circuit designs.Source: §I · p. 1
- The introduction cites RL agents' ability to navigate complex environments, including reported human-level or better performance and AlphaGo's win over a professional human Go player.The passage also cites other demonstrations of RL agent capabilities [4].Source: §I · p. 1
- The paper proposes reinforcement learning to produce synthesis optimization flows and maps the exploration task to a game-like environment using AIG-derived features and a multi-objective reward.The reward guides the agent to minimize area while meeting a delay constraint.Source: §I · p. 1
- DRiLLS is presented as an RL framework for generating optimization flows without human-expert tuning, and is evaluated against EPFL best results, greedy algorithms, and expert-developed flows.The authors report that DRiLLS outperforms previous techniques, on the EPFL arithmetic benchmark suite.Source: §I · p. 1
A. Problem Statement
Different optimization sequences and repetitions produce different quality of results, making logic-synthesis flow exploration exponentially large. The complexity of designs and optimizations motivates autonomous exploration beyond human-guided search.
A. Problem Statement
- Optimization permutations and repetitions change circuit quality of results, creating an exponentially growing search space.Synthesis flows for large circuits often contain tens or hundreds of optimization commands.Source: A. Problem Statement · pp. 1–2
- Assuming optimizations can run independently, a flow of length k has n^k possible sequences from n available optimizations.Different flows can produce divergent area and delay results.Source: A. Problem Statement · p. 2
- Increasing design and synthesis complexity highlights the need for autonomous exploration beyond the traditional guidance of human experts.Source: A. Problem Statement · p. 2
B. Related Work
Prior design-space exploration work uses prediction, pruning, and flow classification, while this work proposes reinforcement learning for synthesis-flow search. The approach targets particular synthesis metrics and supports variable-length flows without pre-defined training flows.
B. Related Work
- Earlier design-space exploration studies use predictive models, pruning, or random forests across computing architecture and high-level synthesis domains,,.Source: B. Related Work · p. 2
- The proposed reinforcement-learning agent explores synthesis flows to optimize metrics such as area and delay, allowing variable-length flows without pre-defined sample flows for training.This contrasts with Yu et al.’s fixed-length flow classification approach, which requires large samples of pre-defined flows.Source: B. Related Work · p. 2
- SynTunSys iteratively evaluates low-cost candidate scenarios and updates its cost estimator from learned costs,.Source: B. Related Work · p. 2
- Deep Q-networks extended reinforcement learning to complex environments, while Lillicrap et al. extended its action domain to continuous control, [12].Earlier RL work focused on fully observable state spaces or handcrafted features,.Source: B. Related Work · p. 2
III. BACKGROUND ON REINFORCEMENT LEARNING
Reinforcement learning trains an agent to choose actions that change a system’s state and earn rewards, aiming to maximize future reward. Actor-critic methods pair an action-selecting actor with a critic that evaluates actions; advantage functions can reduce variability in action probabilities.
III. BACKGROUND ON REINFORCEMENT LEARNING
- An agent iteratively selects actions from its current state, receives a reward as the system moves to a new state, and learns from experience to maximize future reward.A policy maps each state to a probability distribution over actions.Source: §III · p. 2
- Value-based methods learn values for state-action pairs and select the highest-valued action, while policy-based methods optimize the policy directly.Actor-critic algorithms combine elements of both approaches.Source: §III · p. 2
- Actor-critic methods pair an actor that chooses actions from the current state with a critic that evaluates how good the chosen action is.The policy update also uses information from the critic, including its learning history.Source: §III · p. 2
- Advantage functions are proposed to reduce variability in actor-critic action probabilities; the paper estimates advantage without learning Q, using current reward and discounted state values [16].The critic and policy gradients are optimized separately, with β setting different learning rates for policy and value.Source: §III · pp. 2–3
IV. METHODOLOGY
DRiLLS frames logic synthesis design-space exploration as a game in which an A2C agent selects transformations based on circuit-state features. A multi-objective reward guides area reduction under a delay constraint.
IV. METHODOLOGY
- DRiLLS casts combinatorial optimization on a circuit as a game and uses an A2C agent to search for transformations at each state.The framework pairs a logic-synthesis environment with a reinforcement-learning environment.Source: §IV · p. 3
- The agent’s state is based on synthesis-tool metrics for a circuit, normalized against the initial design to support neural-network input and generalization to unseen designs.The described state-vector characteristics include node count and number of levels, which guide choices such as restructuring and balancing.Source: §IV · p. 3
- The agent explores seven ABC primitive transformations: six target AIG size reduction, while balance reduces the number of levels.The six size-oriented transformations are resub, resub -z, rewrite, rewrite -z, refactor, and refactor -z.Source: §IV · p. 3
- The agent’s multi-objective reward prioritizes meeting a delay constraint while rewarding area reduction, including when delay improves but remains above the constraint.The reward formulation gives its highest reward to area reduction that meets the constraint and its lowest negative reward to increases in both area and delay when the constraint is unmet.Source: §IV · p. 3, Table I · p. 3
- Across episodes, the agent applies a sequence of up to k transformations, records states, actions, and rewards, then trains the A2C networks on collected experience.The environment restarts from the original design for each episode, and the actor and critic are updated to minimize loss.Source: §IV · p. 4
- The hybrid actor-critic architecture combines diverse optimization-flow exploration with a path toward optimal designs.The actor outputs probabilities over transformations, while the critic receives the reward during training.Source: §IV · p. 4
V. EXPERIMENTAL RESULTS
The experiments use ABC and an A2C implementation to evaluate DRiLLS on EPFL arithmetic benchmarks. The reported setup specifies the synthesis library, training schedule, network sizes, and discount rate.
V. EXPERIMENTAL RESULTS
- Experiments use ABC v1.01, Python v3.5.2, TensorFlow r1.12, and the ASAP7 7 nm standard-cell library at the typical processing corner.The evaluation uses EPFL arithmetic benchmarks with a range of circuit characteristics.Source: §V · p. 4
- Training uses 50 episodes and caps each episode at 50 iterations.Source: §V · p. 4
- The actor has two fully connected layers with 20 hidden units each, while the critic has one hidden layer with 10 units.Source: §V · p. 4
- The discount rate is 0.99, and the authors report that deeper networks exhibit random behavior and do not train well in this framework.They attribute this to the small number of features and transformations used.Source: §V · p. 4
A. Design Space Exploration
The agent’s episode traces illustrate its search for low-area designs that meet a delay constraint. The traces show area-delay trade-offs and occasional exploration saturation.
A. Design Space Exploration
- One episode finds the global minimum for a number of representative benchmarks while searching for designs that minimize area and meet the delay constraint.Source: A. Design Space Exploration · p. 4
- In Log2 at iteration 30 and Max at iteration 26, transformations reduce delay toward the constraint while increasing area.Source: A. Design Space Exploration · p. 4
- Near-straight trace segments indicate occasional exploration saturation, while the actor-critic networks guide exploration and sometimes try other transformations.Source: A. Design Space Exploration · p. 4
B. Comparison to Other Techniques
DRiLLS improves area while meeting the delay constraint across the evaluated designs and is compared with greedy, expert-crafted, and EPFL approaches. The reported results also give baseline-specific trade-offs and exploration times.
B. Comparison to Other Techniques
- 13.19% average area improvement, with the delay constraint met in all designs; in two cases, area increased while bringing initially infeasible designs within the delay limit.Table II presents the area-delay comparison.Source: Table II · p. 5, B. Comparison to Other Techniques · p. 5
- EPFL size results improved three designs and met the delay constraint in four.The comparison used EPFL size results rather than depth results.Source: B. Comparison to Other Techniques · p. 5
- DRiLLS outperformed EPFL best results in every design except Barrel shifter.EPFL results were selected for size rather than depth, and their methods were not designed for standard-cell library mapping.Source: B. Comparison to Other Techniques · p. 5
- 3.25 minutes for Adder, 25.46 minutes for Hypotenuse, and 12.76 minutes average per episode; trained models can be reused on new circuits without retraining.Exploration runtime includes agent interaction with the synthesis environment, AIG feature extraction, and network-parameter optimization.Source: B. Comparison to Other Techniques · p. 5
VI. CONCLUSIONS
The conclusions present DRiLLS as an autonomous reinforcement-learning approach to exploring logic-synthesis optimization. It uses an A2C agent to seek minimum area under a delay constraint and reports outperforming existing methods on ten representative benchmarks.
VI. CONCLUSIONS
- The framework aims to produce high QoR without a human in the loop, using trial and error analogous to how human experts gain optimization experience.Source: §VI · pp. 5–6
- An A2C agent explores the logic-synthesis design space by iteratively choosing primitive transformations to reduce area subject to a delay constraint.The approach uses an AIG state representation and a multi-objective reward function to guide exploration.Source: §VI · p. 6
- Across ten representative benchmarks, the methodology outperformed existing methods.Source: §VI · p. 6
- DRiLLS is presented as evidence that reinforcement learning can address combinatorial hardware-design optimization, with potential application to related physical-synthesis tasks.The framework is described as open-source under a BSD-3 license and publicly available on GitHub.Source: §VI · p. 6