PaperlayerView exact revision on arXiv

Source-linked AI summary

ACE-RTL: When Agentic Context Evolution Meets RTL-Specialized LLMs

Chenhui Deng, Zhongzhi Yu, Guan-Ting Liu, Nathaniel Pinckney, Brucek Khailany, Haoxing Ren

arXiv:2602.10218v2cs.ARcs.LG

TL;DR

RTL generation requires both hardware-specific knowledge and strong iterative reasoning, but prior approaches typically provide only one. ACE-RTL combines an RTL-specialized model with a frontier reasoning LLM through Generator, Reflector, and Coordinator components, achieving up to a 41.02% pass-rate improvement over all baselines on CVDP.

  • Problem

    Prior RTL-specialized and agentic approaches have complementary limitations, motivating a framework that combines hardware knowledge with general reasoning and simulation-guided refinement.

  • Method

    ACE-RTL integrates a Generator trained on 1.7 million RTL samples with a Reflector and Coordinator that iteratively use feedback and evolving context to refine RTL.

  • Results

    41.02% pass-rate improvement is achieved by ACE-RTL over all baselines on the CVDP benchmark.

  • Takeaways & Limitations

    ACE-RTL combines RTL specialization and agentic context evolution to achieve strong performance across CVDP and VerilogEval-Human-v2.

Abstract

from arXiv · show

Recent advances in LLMs have sparked growing interest in applying them to hardware design automation, particularly for accurate RTL code generation. Prior efforts follow two largely independent paths: (i) training domain-adapted RTL models to internalize hardware semantics, (ii) developing agentic systems that leverage frontier generic LLMs guided by simulation feedback. However, these two paths exhibit complementary strengths and weaknesses. In this work, we present ACE-RTL that unifies both directions through Agentic Context Evolution (ACE). ACE-RTL integrates an RTL-specialized LLM, trained on a large-scale dataset of 1.7 million RTL samples, with a frontier reasoning LLM through three synergistic components: the generator, reflector, and coordinator. These components iteratively refine RTL code toward functional correctness. We further analyze a parallel scaling strategy that reduces wall-clock iterations to first success by exploring diverse debugging trajectories concurrently. On the CVDP benchmark, ACE-RTL achieves up to a 41.02% pass rate improvement over 14 competitive baselines.

I. INTRODUCTION

RTL code generation is important but difficult, and existing approaches separately trade off hardware specialization against general reasoning. ACE-RTL unifies these strengths through iterative collaboration between an RTL-specialized model and a frontier LLM.

  • Motivation: RTL design remains labor-intensive and error-prone, while functionally correct RTL generation can accelerate design cycles and downstream exploration.Accurate RTL generation supports performance optimization and architectural design space exploration.
  • Prior directions: Prior work follows two directions: RTL-specialized models trained on hardware data and agentic systems guided by simulation feedback.The specialized-model direction is illustrated by RTLCoder, CraftRTL, and ScaleRTL.
  • Research gap: Training-based approaches often lack long-context reasoning, planning, and instruction following, whereas agent-based methods lack hardware knowledge from large RTL datasets.These complementary limitations motivate a unified framework combining both capabilities.
  • ACE-RTL: ACE-RTL combines a 1.7-million-sample RTL-specialized Generator with a frontier LLM Reflector and Coordinator that iteratively refine RTL code.The Generator also supports editing and debugging, while the Reflector analyzes simulation feedback and the Coordinator evolves context.

II. BACKGROUND

RTL-LLM research has advanced through specialized training and agentic collaboration, but existing benchmarks and systems leave gaps in realistic, sustained RTL reasoning and broad workflow coverage.

  • Specialized RTL models: RTL-specialized models use methods including automated data synthesis, correct-by-construction samples, compiler-guided reflection, and scaled training with iterative reasoning.Examples include RTLCoder, CraftRTL, OriGen, and ScaleRTL.
  • Agentic systems: Agentic systems such as VerilogCoder and MAGE use multi-agent decomposition, waveform tracing, simulation feedback, and specialized roles for RTL workflows.These systems demonstrate the potential of multi-agent collaboration for RTL generation.
  • Benchmark scope: Mainstream benchmarks mostly test small self-contained specification-to-RTL problems, while RTL-Repo focuses on code completion and omits broader design tasks.The omitted tasks include specification-to-RTL generation, code modification, and debugging.

III. THE ACE-RTL FRAMEWORK

ACE-RTL is an iterative framework that combines an RTL-specialized Generator with a frontier LLM and evolving context. Its training data covers generation, editing, and debugging workflows for refinement-oriented RTL coding.

  • Framework overview: ACE-RTL integrates the Generator, Reflector, and Coordinator in an iterative loop that progressively refines RTL toward functional correctness.The framework also executes multiple instances concurrently through a parallel scaling strategy.
  • Generator: The ACE-RTL-Generator is trained on synthetic data designed to capture diverse hardware patterns, semantics, and refinement workflows.Unlike specification-only training, the data includes code editing and debugging tasks.
  • Generator: X → C⋆ unifies three training categories by mapping an input condition to a correct RTL implementation.The input may be a specification alone or a specification paired with an existing design requiring refinement or correction.
  • Data construction: The training pipeline collects, filters, and validates RTL seeds before constructing specification-to-RTL, editing, and debugging samples.Quality control includes syntax validation, benchmark decontamination, and semantic alignment scoring.
  • Generator: Editing and debugging supervision enables the Generator to analyze, refine, and correct RTL implementations within the agentic framework.This training aligns the model with later iterative refinement rather than only initial synthesis.

B. Reflector: A Frontier Reasoning LLM

The Reflector is a frontier LLM that interprets simulation feedback and error reports to guide subsequent RTL refinement. Its broader reasoning and context capacity support diagnosis of complex failures.

  • Reflector: The Reflector uses Claude4-Sonnet to analyze simulation feedback and long-context error reports, producing guidance for subsequent refinement.It combines the design specification and current RTL implementation with structured feedback.
  • Reflector: At each iteration, iverilog executes the candidate RTL against the test harness, and failures are converted into structured reports of errors and signal mismatches.Claude4-Sonnet uses these reports to produce a concise diagnosis and high-level fix guidance.

C. Coordinator: A Context Evolution Engine

The Coordinator maintains a self-evolving context that records debugging history, monitors progress, and triggers restarts when refinement stalls. This combines preserved progress with controlled exploration of fresh RTL implementations.

  • The Coordinator aggregates identified errors, suggested fixes, and outcomes across iterations to preserve progress and prevent regressions.
  • It monitors debugging trajectories and triggers a restart when the same primary failure persists without meaningful improvement.
  • Cross-iteration history and adaptive restarts keep the evolving context informative while enabling efficient convergence to correct RTL code.
  • The Generator interprets the self-evolving context as an augmented specification because it was trained on editing and debugging tasks.

D. Parallel Scaling in ACE-RTL

ACE-RTL uses parallel execution to search multiple debugging trajectories concurrently, stopping when any process passes all functional tests. The evaluation uses CVDP across four RTL task categories and compares against multiple baselines and variants.

  • Parallel scaling launches multiple ACE-RTL processes with distinct RTL implementations and terminates the others when one passes all functional tests.
  • The strategy reduces time-to-first-success by exploiting generation diversity, while trading off increased total inference-time compute.
  • ACE-RTL is compared with 14 state-of-the-art baselines, including frontier generic models, RTL-specialized models, and MAGE.
  • CVDP evaluates code completion, specification-to-RTL generation, code modification, and code debugging across 94, 78, 55, and 35 problems, respectively.
  • Inference runs five parallel processes, each limited to 30 iterations, with the Generator served via vLLM and other components using Claude4-Sonnet.

B. Main Results

ACE-RTL performs strongly on the challenging CVDP benchmark, where its specialized Generator achieves the highest standalone pass rate and the full system surpasses all baselines. Prior RTL LLMs and agentic baselines show limitations on these tasks.

  • Prior RTL LLMs struggle on CVDP despite performing well on earlier RTL benchmarks.
  • ACE-RTL-Generator achieves the highest standalone average CVDP Pass@1 and lies beyond prior baselines’ Pareto frontier of model scale and functional correctness.
  • MAGE achieves only marginal gains over its corresponding standalone model, partly because it lacks self-evolving context that preserves debugging history.
  • 41.02% APR improvement on cid003 is achieved by ACE-RTL over all baselines on CVDP.

C. Ablation Study

The ablation study shows that iterative refinement, evolving debugging history, and restart-based regeneration each contribute to ACE-RTL’s performance. Restart is especially useful when incremental fixes cannot recover difficult trajectories.

  • ACE-RTL without the Coordinator improves over the standalone Generator, confirming the value of simulation-guided iterative refinement.
  • The largest ablation gain comes from enabling the Coordinator without restart, which preserves debugging history across iterations.
  • Adding restart further improves performance because some failed trajectories benefit from fresh implementations under accumulated debugging context.
  • The full ACE-RTL system outperforms ACE-Claude4, indicating complementary contributions from agentic context evolution and RTL specialization.

D. Case Studies

The case studies show ACE-RTL overcoming stalled RTL debugging by identifying underlying behavioral mistakes and restarting unproductive trajectories. These mechanisms guide the Generator toward simpler, correct implementations.

  • Case studies: ACE-RTL analyzes two cases where stagnation prevents repeated local fixes from reaching functional correctness.The cases cover a 64b/66b Decoder and a Clock Jitter Detection module.
  • Case Study I: In the 64b/66b Decoder, the Reflector infers an implicit alignment transformation from output discrepancies and the specification.The resulting high-level guidance directs the Generator to revise extraction logic, after which all remaining tests pass.
  • Case Study II: In the Clock Jitter Detection case, repeated attempts fail because detection occurs before a valid baseline interval is established.The Generator produces syntactic variants, but they retain the same early-trigger behavior and violate timing semantics.
  • Case Study II: The Coordinator detects repeated assertion patterns without improvement, then restarts generation using distilled insights from failed attempts.This mechanism replaces an unproductive trajectory with a fresh implementation guided by clarified constraints.
  • Case Study II: After two restarts, the Generator adds a validity flag that delays jitter detection until the first full interval is observed, restoring correctness.The restart produces a simpler implementation that properly validates the measurement interval.

E. Efficacy of Parallel Scaling

Parallel scaling accelerates convergence by exploring multiple stochastic ACE-RTL trajectories concurrently while preserving averaged solution quality. It also maintains strong performance on the VerilogEval-Human-v2 benchmark.

  • Convergence speed: Parallel scaling consistently reduces the average iterations needed to reach a correct solution across all four CVDP categories.Figure 7 and Table 2 report this convergence trend across categories.
  • Inference cost: Using five parallel processes increases relative token usage to only 1.12× because single-process contexts become increasingly expensive across later iterations.Accumulated debugging history makes later LLM calls substantially more costly in the single-process setting.
  • VerilogEval-Human-v2: On VerilogEval-Human-v2, ACE-RTL-G outperforms prior RTL LLMs such as OriGen and CraftRTL and achieves the highest pass rate among agentic baselines.The result supports transfer beyond CVDP to an earlier RTL benchmark.

V. FAILURE ANALYSIS AND FUTURE WORK

ACE-RTL does not yet solve all CVDP tasks, particularly those requiring fine-grained temporal diagnosis or reasoning through specification ambiguity and specification-test mismatch. The authors identify waveform information as a future auxiliary signal while retaining specification-grounded reasoning.

  • Failure analysis: ACE-RTL does not yet reach a 100% pass rate on CVDP.Remaining failures motivate the paper’s failure analysis and future-work discussion.
  • Failure analysis: Some failures require fine-grained temporal diagnosis that textual simulation feedback alone cannot localize reliably.These cases involve timing-related bugs whose source is difficult to identify from textual feedback.
  • Failure analysis: Other failures arise from specification ambiguity or subtle mismatches between intended behavior and behavior emphasized by test cases.These cases require reasoning about design intent in addition to debugging observed failures.
  • Future work: The authors propose integrating waveform information as an auxiliary signal because it can reveal temporal mismatches more directly than textual logs.They caution that waveform-driven debugging should avoid overfitting to observed test behavior and should not replace specification-grounded reasoning.
  • Conclusion: The paper concludes that ACE-RTL achieves state-of-the-art results on multiple RTL benchmarks and identifies automated testbench generation and waveform-enhanced reasoning as future directions.These directions extend the framework beyond its current evaluation and debugging setup.
Loading 2602.10218v2…