Source-linked AI summary
DeepCircuitX: A Comprehensive Repository-Level Dataset for RTL Code Understanding, Generation, and PPA Analysis
Zeju Li, Changran Xu, Zhengyuan Shi, Zedong Peng, Yi Liu, Yunhao Zhou, Lingfeng Zhou, Chengyu Ma, Jianyuan Zhong, Xi Wang, Jieru Zhao, Zhufei Chu, Xiaoyan Yang, Qiang Xu
TL;DR
Existing RTL datasets have limited scope, while RTL-focused machine learning needs data spanning code and circuit implementation. DeepCircuitX combines multilevel repositories, annotations, and synthesized circuit data; its experiments report improved LLM performance on RTL tasks, alongside challenges in PPA prediction.
Problem
Existing RTL datasets have a narrow scope, limiting the breadth of designs available for RTL modeling and evaluation.
Source: §I · p. 1Method
DeepCircuitX organizes repository-level RTL projects across repository, file, module, and block levels and combines annotations with synthesized circuit modalities.
Source: §I · pp. 1–2, §V · p. 6Results
Fine-tuned LLMs significantly outperform their original counterparts on RTL tasks, while accurate early-stage PPA prediction remains challenging.
Source: 2) Analysis · pp. 5–6, §V · p. 6Takeaways & Limitations
Synthesized netlists and PPA metrics provide a resource for early-stage design exploration and a foundation for future hardware design automation research.
Source: §V · p. 7
Abstract
from arXiv · showhide
This paper introduces DeepCircuitX, a comprehensive repository-level dataset designed to advance RTL (Register Transfer Level) code understanding, generation, and power-performance-area (PPA) analysis. Unlike existing datasets that are limited to either file-level RTL code or physical layout data, DeepCircuitX provides a holistic, multilevel resource that spans repository, file, module, and block-level RTL code. This structure enables more nuanced training and evaluation of large language models (LLMs) for RTL-specific tasks. DeepCircuitX is enriched with Chain of Thought (CoT) annotations, offering detailed descriptions of functionality and structure at multiple levels. These annotations enhance its utility for a wide range of tasks, including RTL code understanding, generation, and completion. Additionally, the dataset includes synthesized netlists and PPA metrics, facilitating early-stage design exploration and enabling accurate PPA prediction directly from RTL code. We demonstrate the dataset's effectiveness on various LLMs finetuned with our dataset and confirm the quality with human evaluations. Our results highlight DeepCircuitX as a critical resource for advancing RTL-focused machine learning applications in hardware design automation.Our data is available at https://zeju.gitbook.io/lcm-team.
I. INTRODUCTION
DeepCircuitX addresses limitations in RTL datasets with a repository-level collection spanning multiple code scales and circuit modalities. It pairs this resource with annotations and benchmarks for RTL tasks and PPA prediction.
I. INTRODUCTION
- Existing RTL datasets are limited in design diversity, and many omit circuit-level information needed to relate RTL code to implementations.This limits coverage of the RTL-to-circuit stages relevant to PPA.Source: §I · p. 1
- Repository-level projects can be transformed into circuit modalities, including netlists and layouts, connecting RTL designs with circuit implementations.The annotation and transformation pipeline is illustrated in Figure 1.Source: §I · p. 1, Figure 1 · p. 2
- Fine-tuning on DeepCircuitX significantly improves LLM performance across metrics, while PPA prediction remains an ongoing challenge for accurate early-stage forecasting.The experiments include LLMs across model scales and learning-based prediction of area, power, and delay.Source: §I · p. 2
- The dataset contains over 4,000 repository-level RTL projects and supports training across repository, file, module, and block scales.Its scope includes chip, IP, and module designs with diverse functional and algorithmic keywords.Source: §I · p. 2
- GPT-4 and Claude generate comments, descriptions, and question-answer pairs to support RTL code understanding and generation.Annotations cover the dataset’s four levels: repository, file, module, and block.Source: §I · pp. 1–2
II. RELATED WORK
Related EDA datasets offer either circuit netlists or multimodal physical-design data, while RTL-focused datasets emphasize code tasks. The described resources have limits in cross-stage coverage, design diversity, or repository-level context.
A. Previous Dataset in EDA
- Open-source netlist benchmarks and physical-design datasets provide EDA data, but the described resources lack comprehensive cross-stage insights linking front-end RTL optimization to back-end PPA.The examples include physical designs generated from IWLS’05 using Skywater’s 130nm PDK and OpenROAD.Source: A. Previous Dataset in EDA · p. 2
- CircuitNet combines synthesized gate-level netlists and layouts, including million-gate designs, but its datasets cover a limited range of circuit designs.The expanded data includes CPUs, GPUs, and AI chips at the 14nm FinFET node.Source: A. Previous Dataset in EDA · p. 2
B. RTL-stage Understanding, Completion, and Generation
The section reviews limitations in file-level RTL datasets and describes DeepCircuitX’s scope and preparation: designs are organized by abstraction level, collected across many categories, annotated, and synthesized for circuit data.
B. RTL-stage Understanding, Completion, and Generation
- Existing RTL datasets focus on file-level code and often omit repository, module, and block context, synthesis-derived netlists and PPA metrics, and annotations for code understanding.The review describes prior work as concentrating mainly on RTL generation and completion.Source: B. RTL-stage Understanding,… · pp. 2–3
A. Data Preparation
- Chip-, IP-, and module-level designs represent different scopes, from whole-system integration to reusable components and their subcomponents.These abstraction levels distinguish system-wide constraints from standalone functionality and local optimization.Source: A. Data Preparation · p. 3
- DeepCircuitX collects over 4,000 repository-level projects, covering 140,000 RTL files across 77 functional categories.The collection uses 222 keywords representing chip-, IP-, and module-level designs.Source: A. Data Preparation · p. 3
- The dataset organizes RTL at repository, file, module, and block levels, adds GPT-4 and Claude CoT annotations, and synthesizes projects for netlists, PPA metrics, and layouts.The annotations support RTL understanding and generation, while synthesis provides multimodal data.Source: A. Data Preparation · p. 3
B. Language-RTL Code Dataset Construction
DeepCircuitX applies CoT annotation at multiple RTL levels, using structured descriptions of modules and code blocks and a repository-level summary assembled from files and module annotations.
1) Chain of Thought (CoT) for RTL Code Annotation:
- The annotation process covers module-, block-, and repository-level RTL, aiming to describe designs comprehensively at different levels.The paper frames CoT as structured reasoning used to create detailed Verilog annotations.Source: 1) Chain of Thought (CoT) for RTL Code… · p. 3
- Module annotations use rounds of questions about purpose, ports, and signals to build a specification and summarize functionality and implementation.The specification also breaks down functional blocks within the module.Source: 1) Chain of Thought (CoT) for RTL Code… · p. 3
- Block annotations identify constructs such as always, initial, task, function, generate, assign, and final blocks; GPT-4 handles complex segmentation alongside regular expressions for simpler patterns.Each identified block is annotated for its purpose and function.Source: 1) Chain of Thought (CoT) for RTL Code… · p. 3
- Repository annotations combine file-structure information with module annotations to summarize how files and modules work together.This produces a top-down view of project structure, purpose, interconnections, and design details.Source: 1) Chain of Thought (CoT) for RTL Code… · p. 3
2) Dataset for RTL Code Understanding, Completion and Generation:
DeepCircuitX supports RTL understanding, completion, and generation tasks, with data organized across RTL categories. Each task evaluates a different ability, from explaining existing code to implementing a full module from specifications.
2) Dataset for RTL Code Understanding, Completion and Generation:
- The dataset covers understanding, completion, and generation tasks, with data counts reported across IP, module, and chip categories.Table III summarizes counts for code generation, code completion, and comment generation across categories.Source: 2) Dataset for RTL Code Understanding,… · p. 3, Table III · p. 4
- RTL code understanding asks models to describe a module’s purpose, signals, internal logic, and behavior from its code.The task assesses production of concise, human-readable code explanations.Source: 2) Dataset for RTL Code Understanding,… · p. 4
- RTL code completion provides partial code, typically a module header, and asks the model to generate missing logic, control structures, and signal definitions.The task tests code generation from the supplied context.Source: 2) Dataset for RTL Code Understanding,… · p. 4
- RTL code generation asks the model to produce a full Verilog implementation from a high-level description and specified input and output parameters.The implementation is intended to follow the provided specifications.Source: 2) Dataset for RTL Code Understanding,… · p. 4
1) Graph-based Code Representation:
The section describes graph-based representations of RTL and the synthesis process used to produce circuit netlists and related data. The dataset is evaluated through human review, RTL tasks, and PPA prediction experiments.
1) Graph-based Code Representation:
- RTL designs are represented with Abstract Syntax Trees and Control/Data Flow Graphs generated using Yosys and customized Pyverilog.AST nodes denote variables or operators, while CDFGs visualize data movement between registers and the role of control signals.Source: 1) Graph-based Code Representation · p. 4
2) Circuit Netlist Synthesis:
- Synopsys Design Compiler 2019.12 synthesizes RTL into netlists mapped to five technology libraries, with default and maximum-area-constrained versions.The libraries span GlobalFoundries 180nm, skywater 130nm, ihp-sg 130nm, nangate 45nm, and asap 7nm.Source: 2) Circuit Netlist Synthesis · p. 4
- The synthesis flow records power and delay measurements, netlists in Verilog and SDF formats, reports, and AIG representations.PrimeTime assesses internal, transition, and leakage power; Design Compiler reports critical-path delay; abc converts mapped netlists to AIG format.Source: 2) Circuit Netlist Synthesis · p. 4
A. overview
- Dataset evaluation includes human evaluation, RTL code tasks, and PPA prediction experiments.The paper assigns these experiments to Sections IV-B, IV-C and IV-D, and IV-E, respectively.Source: A. overview · p. 4
B. Human evaluation
Independent engineers reviewed DeepCircuitX annotations for accuracy, completeness, and understandable clarity. The reported scores exceed 3.5 out of 4 across all metrics.
B. Human evaluation
- The evaluation criteria are accuracy, completeness, and understandable clarity.Table IV presents the detailed grading criteria.Source: B. Human evaluation · p. 5, Table IV · p. 5
- Engineers evaluate RTL code and accompanying annotations against specific criteria, grading each metric from 1 to 4 (4 highest, 1 lowest); five individuals review each generated text.Source: B. Human evaluation · p. 5
- All annotation-quality metrics score above 3.5 out of 4 in the human evaluation.Table V reports grading for repository- and module-level annotations using accuracy, completeness, and understandable clarity.Source: B. Human evaluation · p. 5, Table V · p. 5
C. RTL Code Understanding
The RTL code understanding benchmark evaluates generated descriptions using text-overlap metrics. Fine-tuning on DeepCircuitX improves results across the reported metrics and model sizes.
C. RTL Code Understanding
- The benchmark reports BLEU-4, METEOR, ROUGE-1, ROUGE-2, and ROUGE-L to evaluate generated descriptions against references.BLEU and ROUGE measure forms of n-gram or sequence overlap, while METEOR accounts for synonymy and stemming.Source: 2) Analysis · p. 5, 1) Evaluation Metrics · p. 5
2) Analysis:
- DeepSeek-Coder-V2-lite (original) leads the described baseline examples, with BLEU-4 2.2387 and ROUGE-1 30.3208.CodeLLama (original) has BLEU-4 0.0828, while CodeGen2.5 (original) has BLEU-4 0.1060.Source: 2) Analysis · p. 5
- Fine-tuned LLMs significantly outperform their original counterparts across BLEU-4, METEOR, ROUGE-1, ROUGE-2, and ROUGE-L.The reported gains span models from 220M to 16B parameters.Source: 2) Analysis · p. 5
D. RTL Code Completion and Generation
RTL completion and generation are evaluated with Pass@k, which checks whether the correct code appears among a model’s top-k predictions. The section identifies Table VII as comparing original and fine-tuned models across two benchmarks.
D. RTL Code Completion and Generation
- Pass@k measures whether a correct code snippet appears among a model’s top-k outputs.The metric is used to evaluate RTL completion and generation.Source: 1) Evaluation Metrics · p. 5
- For RTL code completion and generation, Pass@k measures code-generation accuracy by assessing valid solutions and whether a correct code snippet appears among a model’s top k outputs.Source: 1) Evaluation Metrics · p. 5
- Pass@k provides insight into the effectiveness and reliability of model predictions.The passage describes this as an additional interpretation of the metric.Source: 1) Evaluation Metrics · pp. 5–6
2) Analysis:
The analysis reports gains from fine-tuning for RTL generation and describes a repository-based setup for predicting PPA metrics from RTL designs.
2) Analysis:
- Every dataset-fine-tuned model significantly outperforms its original counterpart, and CodeGen2.5 (7B) exceeds CodeV on the reported benchmarks.CodeV scores 14.80% Pass@1 and Pass@5 on RTLLM and 4.5% on VerilogEval; the comparisons use Table VII.Source: Table VII · p. 6, 2) Analysis · p. 6
1) Overview:
- Repository-level RTL data enables post-mapping netlists and synthesis reports to be used for early-stage PPA prediction.The task evaluates learning-based prediction models using metrics parsed from the reports.Source: 1) Overview · p. 6
2) Evaluation Metrics:
- PPA ground truth comes from default-setting Design Compiler post-mapping netlists; prediction accuracy uses MAPE and RRSE with the SkyWater 130nm library.The passage states that predictors trained with one technology library do not generalize to another because PPA metrics are strongly linked to the library.Source: 2) Evaluation Metrics · p. 6
3) Analysis:
PPA prediction accuracy improves with more training data, but delay prediction remains difficult on the practical-sized designs evaluated. The conclusion presents DeepCircuitX as a multilevel RTL resource that includes synthesized netlists and PPA metrics.
3) Analysis:
- Delay prediction remains weaker: model has 4.7392 MAPE and model has 3.4769 MAPE.The authors note that synthesis tools heavily optimize logic to minimize maximum path delay, making early timing estimation from RTL-based graph path features difficult.Source: 3) Analysis · p. 6
- PPA predictors perform worse on designs exceeding 10k cells than on the simple benchmarks originally reported, leaving practical-design prediction open.The authors identify further exploration as necessary.Source: 3) Analysis · p. 6
V. CONCLUSION
- DeepCircuitX spans repository, file, module, and block RTL levels and includes synthesized netlists and PPA metrics for early-stage design exploration.The conclusion presents it as a resource for EDA researchers and practitioners.Source: §V · pp. 6–7
3) Analysis:
- 4.3201 area-prediction MAPE with 10% data falls to 0.3303 with 100% data.The evaluation uses 146 training designs and 10 test designs, each containing more than 10k synthesized cells.Source: 3) Analysis · p. 6, Table VIII · p. 7