Source-linked AI summary
LLM-based Hardware Development with Hierarchical IRs and End-to-End Multi-Agent Workflow
Chenyang Yin, Agasthi Haputhanthri, Aditya Anirudh Jonnalagadda, Zhenyu Bai, Yuanming Song, Saranyu Chattopadhyay, Mohammad Fadiheh, Tom Zelazny, Subhasish Mitra, Tulika Mitra
TL;DR
Complex hardware design requires LLMs to reason about module hierarchy, interconnections, functionality, and verification beyond direct RTL generation. The paper addresses this with hierarchical IRs and an end-to-end multi-agent workflow, achieving strong Verilog-Eval performance and functional generation of complex systems. The results identify structured representations and targeted debugging as central to the proposed workflow.
Problem
LLM hardware-design research largely targets small single-module designs, leaving end-to-end generation and verification of complex hierarchical systems insufficiently addressed.
Method
The framework uses an Architectural Sketch for topology and interconnection, an Operational Specification for module behavior and interfaces, and a multi-agent workflow for generation, verification, and debugging.
Results
95.5% pass@5 is achieved on Verilog-Eval, and the framework produces functionally correct end-to-end designs in a seven-system complex case study.
Takeaways & Limitations
The framework provides an end-to-end approach for decomposing, implementing, verifying, and debugging complex hierarchical hardware designs.
Takeaways & Limitations
MAGE does not support hierarchical design, and its CNN success aligns with that case being the simplest by execution time, token count, and lines of code.
Abstract
from arXiv · showhide
Large language models (LLMs) are increasingly used in software development, but their use in complex hardware design remains limited. This gap stems from both the scarcity of public hardware training data and the fundamentally different methodologies used in hardware design. In particular, applying LLMs to hardware requires more than direct RTL generation: the model must understand module boundaries, inter-module connections, and verification requirements. In this paper, we present an LLM-based hardware development framework with hierarchical intermediate representations (IRs) and an end-to-end multi-agent workflow. The core idea is to provide an abstraction of hardware design to LLMs through two structured IRs: Architectural Sketch, which captures module topology and interconnection, and Operational Specification, which defines per-module functionality and interfaces. Our framework uses these IRs to decompose a complex design into sub-modules, specify the per-block functionality, and derive how each module should be tested and verified. We incorporate a multi-agent debug loop in the framework, allowing agents to get the error feedback and control the debug details such as the signals to be probed for simulation. We evaluate our framework on Verilog-Eval benchmark, achieving a pass@5 rate of 95.5%, which surpasses current state-of-the-art LLM generation frameworks. To better assess performance on complex, realistic designs, we introduce a new case study spanning applications from general-purpose processors to digital signal processing systems. Experimental results indicate that such complex designs exceed the capabilities of existing approaches, whereas our framework is the only one capable of producing functional end-to-end design. Our generated RTL follows all industry-standard design rules, is lint-clean, functionally correct and fully synthesizable.
1 Introduction
Existing LLM hardware-design work mainly targets small, single-module RTL, leaving complex hierarchical systems and verification insufficiently addressed. The paper introduces hierarchical IRs and an end-to-end multi-agent workflow, reporting strong benchmark performance and functional complex-system generation.
- Research gap: Existing LLM hardware studies mainly focus on relatively small, single-module designs, unlike real-world systems with interacting modules and verification requirements.Complex systems require architectural organization, inter-module connections, and system-level verification.
- Research gap: Existing decomposition frameworks generate modules block-by-block but rely on manually written testbenches, while testbench-focused methods do not cover hierarchical design.This leaves no end-to-end workflow from high-level requirements through verification and complete system delivery.
- Proposed approach: The framework introduces an Architectural Sketch for structural organization and an Operational Specification for module functionality and interfaces.The IRs structure requirements for RTL and testbench generation and support divide-and-conquer design reasoning.
- Proposed approach: The multi-agent debug loop probes internal signals, analyzes execution feedback, and maintains debug history to improve complex-design debugging.The framework uses specialized agents and records error patterns, advice, and RTL modifications across iterations.
2 The Intermediate Representations
The framework uses two hierarchical IRs to translate high-level hardware intent into structured architectural and behavioral descriptions. These representations support modular RTL generation, interface specification, and verification-oriented implementation.
- Motivation: Hierarchical hardware designs challenge LLMs because reliable generation requires connectivity descriptions, precise port mapping, and operational partitioning across modules.Inter-module timing and state transitions make complex systems harder than single-module tasks.
- Architectural Sketch: The Architectural Sketch captures high-level module hierarchy and interconnection, providing the structural basis for task decomposition.For the CNN accelerator, the sketch identifies five sequential sub-modules: Conv2D, ReLU, Maxpool2D, Flatten, and Linear.
- Architectural Sketch: The sketch is generated from high-level inputs such as block diagrams and natural-language prompts, eliminating manual RTL wiring and port specification.It establishes a modular design and verification flow across individual sub-modules.
- Operational Specification: The Operational Specification lists each module’s operations, input and output patterns, port widths and types, control signals, and functional descriptions.Its JSON format captures module behavior, interfaces, data movement, and control protocols in machine-readable form.
- Operational Specification: The Operational Specification converts architectural intent and natural-language prompts into structured behavioral contracts that guide RTL implementation.By making control, data patterns, and functional descriptions explicit, it reduces ambiguity between system architecture and module-level implementation.
3 The Multi-Agent Framework
The framework separates planning from generation: agents first construct hierarchical IRs, then generate and verify modules bottom-up. Its workflow addresses testbench and debugging challenges through explicit behavioral information and multi-agent feedback.
- Planning stage: The planning stage converts block diagrams and textual specifications into an Architectural Sketch and Operational Specifications for the system hierarchy.Architecture and function-decomposition agents identify modules, connections, and module behaviors.
- Generation stage: The generation stage uses the IRs to schedule block-level RTL tasks, verify sub-modules independently, and construct the top module after successful lower-level verification.This bottom-up sequence verifies each hierarchy level before proceeding upward.
- Testbench generation: Purely LLM-generated testbenches can be unreliable, while prior hierarchical workflows often depend on manually written testbenches.Generated testbenches may diverge from design semantics or impose inconsistent constraints.
- Testbench generation: The Operational Specification supports automated testbench generation by encoding input/output flow patterns, trigger conditions, sampling conditions, and reference functions.The reference function represents output = F(input).
- Multi-agent debug loop: Vanilla self-reflection is limited by output-only feedback and independent iterations that can repeat the same errors in complex hierarchical systems.These limitations make internal diagnosis and top-module debugging difficult.
- Multi-agent debug loop: The enhanced loop classifies errors, probes relevant internal signals, supplies diagnostic advice, revises RTL, and tracks prior debug history.Syntax errors are regenerated directly, whereas functional errors use the multi-agent diagnostic path.
4 Evaluation
The evaluation combines Verilog-Eval with a seven-design hierarchical case study to test both single-module RTL generation and realistic end-to-end hardware development. The framework outperforms existing approaches on both settings, including all seven complex cases.
- Evaluation Setup: The evaluation uses Verilog-Eval for small single-module designs and a new case study covering seven complex hardware systems with 29 sub-modules.The case study spans signal processing, communications, computer vision, machine-learning acceleration, processors, and programmable architectures.
- Verilog-Eval: 149 out of 156 Verilog-Eval cases pass with the framework, compared with 146 for native GPT-5 and MAGE; excluding seven problematic cases yields a 100% pass rate.The problematic cases involve issues such as reset handling, synthesizability constraints, prompt ambiguity, and incorrect reference behavior.
- Evaluation Setup: The case study stresses programmability, intricate finite-state-machine transitions, large-scale structural interconnection, arithmetic, and spatial search logic.Its modules include processors and vector units, OFDM, a systolic array, CNN and beamformer engines, and an image-match core.
- Evaluation Setup: 80 times more execution time, approximately 55 times more tokens, 12 times more lines of code, and nearly 260 times more cells distinguish the case study from the largest Verilog-Eval case.These statistics indicate substantially larger and more intricate generated designs.
- Case Study Results: All seven complex cases pass with the framework using GPT-5 and GPT-5.4, while ROME passes none and MAGE passes one.Across GPT-5, GPT-5.4, Claude Sonnet, and Claude Opus, the framework passes at least five cases for every model.
- Ablations: Direct generation and repair pass only 1/7 and 2/7 cases, while removing Operational Specification passes 2/7 and removing Architectural Sketch passes 5/7.The complete framework passes all seven cases, indicating complementary functional and structural information from the two IRs.
5 Related Work
LLM hardware research has expanded from single-module RTL generation toward broader design-flow support, including hierarchical decomposition and verification. However, existing approaches still leave gaps in end-to-end hardware development.
- RTL Generation: LLM hardware research initially emphasized improving single-module RTL generation through fine-tuning and larger, higher-quality Verilog datasets.These efforts include Verigen, RTL++, and VerilogDB, but primarily target single-module code generation.
- Hardware Design Flow: Chip-chat and ChatCPU applied LLMs to CPU design, while ROME and HiveGen decomposed complex systems into sub-modules.HiveGen additionally reused existing modules through retrieval-augmented generation.
- Hardware Design Flow: MAGE introduced a multi-agent framework that converts waveform information into design-related guidance.
6 Conclusion and Future Work
The paper presents an end-to-end multi-agent hardware-development framework using hierarchical IRs and reports strong results on both Verilog-Eval and complex case-study designs. Future work aims to add physical-design feedback for PPA optimization.
- Conclusion: The framework combines Architectural Sketch and Operational Specification IRs with multi-agent debugging to decompose, implement, verify, and refine hierarchical hardware designs.Architectural Sketch captures topology and interconnection, while Operational Specification defines per-module functionality.
- Conclusion: 95.5% pass@5 on Verilog-Eval surpasses state-of-the-art LLM-based generation baselines.
- Conclusion: The framework passes all complex case-study cases and is the only evaluated framework that consistently generates structurally and functionally correct RTL for them.
- Future Work: Future work will close a PPA optimization loop by incorporating physical-design feedback on delay, area, and power into iterative IR and implementation refinement.