Source-linked AI summary
SeqFeed: Improving Agentic RTL Code Generation with Sequential Behavior Feedback
Yuxin Du, Juxin Niu, Tao Hu, Xi Wang, Zhe Jiang, Nan Guan
TL;DR
Agentic RTL generation lacks effective feedback about how signals evolve across clock cycles, limiting agents’ ability to diagnose and revise sequential behavior. SeqFeed combines event-based waveform querying with cycle-aware dependency tracing; across six LLM configurations, it improves pass rates, with complementary benefits from both components.
Problem
Agents need effective feedback to understand and revise sequential behavior, but RTL source code and simulation waveforms each have limitations for conveying it.
Method
SeqFeed combines SeQuery for event-based waveform querying with SeGraph for cycle-aware dependency tracing.
Results
SeqFeed improves pass rates across all six model settings, with gains ranging from 8.7 percentage points to 18.3 percentage points over Baseline.
Takeaways & Limitations
SeQuery and SeGraph are independently effective and complementary, while agent reasoning shifts toward evidence-driven diagnosis.
Takeaways & Limitations
SeqFeed leaves some scenarios uncovered, including multi-clock-domain designs, prioritizing LLM accessibility over completeness.
Abstract
from arXiv · showhide
RTL code generation is a critical stage in hardware design, and the emergence of agentic systems offers new opportunities to automate this process. To generate correct RTL code, agents must understand sequential behavior, including how signals evolve and propagate over multiple clock cycles. However, effectively conveying such temporal information to agents remains a significant challenge. RTL code does not expose cycle-level signal behavior for a specific execution, whereas full simulation waveforms are too voluminous and noisy for effective LLM analysis. To address these limitations, we study how human engineers reason about sequential behavior and identify three requirements for effective feedback: it should be event-addressable, dependency-traceable, and iteratively-queryable. Guided by these requirements, we propose \textit{SeqFeed}, which comprises two complementary mechanisms: (1) \textit{SeQuery}, an SQL-like waveform query language that enables agents to anchor queries to semantic events and sample signal values at relative time points; and (2) \textit{SeGraph}, a dependency graph that tracks signal propagation across clock cycles. Experimental results across multiple LLMs demonstrate the effectiveness of SeqFeed in improving pass rates. SeQuery and SeGraph are each effective independently and provide complementary benefits when used together.
Introduction
The introduction frames sequential behavior feedback as essential for agentic RTL generation but difficult to provide effectively. It motivates SeqFeed, which combines event-based waveform querying with cycle-aware dependency tracing to support evidence-based diagnosis and refinement.
- Motivation: Correct RTL generation requires agents to understand how signals evolve and propagate across clock cycles.This sequential understanding is needed to identify appropriate code revisions during iterative improvement.
- Limitations of Existing Feedback: RTL source code and simulation mismatches do not directly reveal the cycle-level behavior or path conditions responsible for an error.Source code describes transition rules, while a mismatch indicates incorrect behavior without identifying its sequential cause.
- Limitations of Existing Feedback: Full simulation waveforms provide concrete execution evidence but can overwhelm context and fail to connect observed behavior to responsible RTL logic.They contain many signals and cycles unrelated to the agent’s current reasoning and show what occurred without directly explaining the underlying logic.
- Limitations of Existing Feedback: Diagnostic instrumentation requires advance specification and repeated simulation, potentially slowing iterative refinement when complex designs take minutes or hours to simulate.Changing observed signals or trigger conditions requires modifying the instrumentation and rerunning the simulation.
- Contributions: SeqFeed derives three design requirements and provides SeQuery for event-based waveform querying plus SeGraph for cycle-aware dependency tracing.The framework is evaluated across RTL code generation benchmarks and multiple LLMs, improving success rates and enabling more evidence-based diagnosis and refinement.
Motivation
The paper motivates sequential-behavior feedback for agentic RTL generation by distilling hardware engineers’ inspection practices into three requirements: event-addressable, dependency-traceable, and iteratively-queryable feedback.
- Feedback requirements: These practices motivate three requirements for LLM sequential feedback: event-addressable, dependency-traceable, and iteratively-queryable.The requirements determine where to look, what information is relevant, and how much information to examine.
- Engineer reasoning: Hardware engineers inspect sequential executions by anchoring analysis on specific events and examining surrounding cycles rather than scanning waveforms cycle by cycle.An event provides a reference point, while its surrounding window supplies context.
- Engineer reasoning: Engineers explain a signal by tracing upstream assignments, enabling conditions, and earlier signal values backward across clock cycles.This focuses inspection on relevant signals at progressively earlier cycles.
- Feedback requirements: An event-addressable mechanism locates cycles containing a specified signal event and inspects selected signals within a surrounding window without scanning the entire waveform.A dependency-traceable mechanism identifies assignments, enabling conditions, and earlier values determining a signal at a specific cycle; an iteratively-queryable mechanism supports successive queries.
SeqFeed
SeqFeed provides sequential feedback through SeQuery and SeGraph, enabling agents to query event-anchored waveform values and trace signal dependencies across clock cycles. The tools are designed for iterative debugging, prioritizing LLM usability while accepting limited completeness.
- SeQuery: SeqQuery anchors waveform queries to semantic events, retrieves selected signals at relative cycle offsets, and supports iterative refinement as debugging hypotheses evolve.Its core primitives are select, where, and s@k, with positive offsets looking forward and negative offsets looking backward.
- Agentic debugging: Agents autonomously choose when to query waveforms, trace dependencies, and interleave both mechanisms as their hypotheses evolve.The accumulator example combines dependency tracing with an event-specific waveform query to identify an incorrect in_valid gating condition instead of accept.
- SeGraph: SeGraph traces a target signal backward through producing statements and input signals, organizing dependencies by clock cycle rather than RTL source order.Cycle annotations make cross-cycle value propagation explicit, while traversal terminates at primary inputs, undefined or visited nodes, or a depth limit.
- Design trade-offs: SeqFeed favors LLM accessibility by extending familiar SQL syntax with s@k and unifying signals, assignments, and conditional branches in SeGraph, while leaving some scenarios uncovered.The stated tradeoff prioritizes stable, actionable feedback during iterative debugging at the cost of completeness, including limited support for multi-clock-domain designs.
Evaluation
Evaluation on 256 practical RTL cases across six model settings shows that SeqFeed improves pass rates over the baseline in every setting. Token-cost and tool-use analyses indicate that structured feedback extends productive iteration while changing how agents allocate actions and reasoning.
- Benchmark: The benchmark contains 256 practical hardware-design cases spanning six categories, excluding purely combinational and toy circuits.Cases combine existing RTL-generation benchmarks with recent open-source hardware-design cases.
- Experimental setup: The evaluation compares Baseline, SeQuery-only, SeGraph-only, and SeqFeed configurations across six DeepSeek Flash, DeepSeek Pro, and MiniMax M3 settings.Each model configuration runs all 256 benchmark cases under a fixed token budget per case.
- Pass-rate results: 18.3 percentage points is SeqFeed’s largest pass-rate gain over Baseline, rising from 53.0% to 71.3% on DeepSeek Flash think; the smallest gain is 8.7 points on DeepSeek Pro think, from 60.4% to 69.1%.SeqFeed improves pass rate over Baseline across all six model settings and reaches the highest absolute pass rate on DeepSeek Flash think at 71.3%.
- Token-cost analysis: SeqFeed continues producing passes after Baseline nearly stops gaining: on DeepSeek Flash Think, it gains 57.8 points in the first 100K tokens and 13.5 points thereafter, versus Baseline’s 46.5 and 6.9 points.On MiniMax M3 No-Think, Baseline leads at 10K tokens but finishes at 41.7%, while SeqFeed overtakes and reaches 57.0%.
- Tool-use analysis: SeqFeed raises tool-use density to 2.4–3.9× Baseline on no-think models, whereas DeepSeek Pro Think reaches 69.1% pass rate while tool calls per token fall to 0.62× Baseline.Tool-use density counts iterative-loop calls for editing, simulation, and SeQuery or SeGraph queries, excluding unrelated calls.
Related Work
Related work applies LLMs and agentic systems across hardware-design workflows, including RTL generation, while emphasizing feedback mechanisms for iterative refinement. Existing RTL agents use diverse diagnostics and traces, but conveying sequential behavior for evolving hypotheses remains challenging.
- LLM-Aided Hardware Design: LLMs have been applied across the EDA pipeline, including RTL generation, verification, and optimization.The agentic paradigm extends RTL tasks by enabling LLM agents to invoke tools autonomously and iteratively refine their outputs.
- Sequential Feedback: Effective feedback enables LLM agents to iteratively refine their outputs, paralleling software agents’ use of compiler diagnostics and unit-test outcomes.This establishes feedback as a central mechanism for identifying program-level errors during agentic refinement.
- Sequential Feedback: RTL agents use compiler errors, simulation results, EDA reports, and waveform traces as feedback sources.These modalities support RTL refinement but do not by themselves resolve the challenge of tracking evolving hypotheses.
Conclusion
The conclusion identifies ineffective sequential feedback as a central bottleneck in agentic RTL generation and presents SeqFeed as a solution guided by three feedback requirements.
- Feedback requirements: Effective sequential feedback should be event-addressable, dependency-traceable, and incrementally expandable.These requirements were derived from examining how human engineers interpret sequential behavior.
- SeqFeed: SeqFeed combines SeQuery, a waveform query language, with SeGraph, a cycle-organized dependency graph.Together, the two mechanisms are designed to meet the identified feedback requirements.
- Evaluation: Experiments evaluated SeqFeed across six LLM configurations.