PaperlayerView exact revision on arXiv

Source-linked AI summary

VFlow: Discovering Optimal Agentic Workflows for Verilog Generation

Yangbo Wei, Zhen Huang, Huang Li, Wei W. Xing, Ting-Jung Lin, Lei He

arXiv:2504.03723v2cs.ARcs.MA

TL;DR

Verilog generation needs workflows that handle hardware-specific requirements while improving quality and efficiency beyond fixed prompts and manually designed flows. VFlow searches graph-structured LLM workflows using cooperative multi-objective evolution, shared experience, and multi-level verification, achieving stronger benchmark performance and cost efficiency, including up to 10.9× ROI.

Abstract

from arXiv · show

Hardware design automation faces challenges in generating high-quality Verilog code efficiently. This paper introduces VFlow, an automated framework that optimizes agentic workflows for Verilog code generation. Unlike traditional approaches relying on fixed prompts or manually designed flows, VFlow treats workflow discovery as a search over graph-structured LLM invocation sequences. It introduces a multi-population cooperative evolution (CEPE-MCTS) algorithm that balances multiple hardware objectives -- functional correctness, area, power, timing and token cost -- while sharing successful patterns and avoiding repeated failures. Integrated multi-level verification ensures syntactic correctness, functional behavior, and synthesizability. Experiments on VerilogEval and RTLLM2.0 show VFlow improves pass@1 by 20--30\% over prompting baselines and closely matches designer-level area/power. Remarkably, VFlow enables small LLMs to outperform larger models with up to 10.9$\times$ ROI, offering a cost-effective solution for RTL design. This work paves the way for intelligent, automated hardware development, advancing LLM applications in EDA.

I. INTRODUCTION

Verilog generation must handle hardware-specific semantics, timing, synthesizability, and verification, while existing workflows remain difficult to optimize systematically. VFlow addresses this gap by searching graph-structured LLM workflows with cooperative multi-objective optimization and multi-level verification.

I. INTRODUCTION

  • VFlow searches graph-structured LLM invocation workflows instead of relying on fixed prompts or manually designed multi-stage processes.Its CEPE-MCTS strategy uses multiple populations, fragment migration, and shared failure experience to explore alternative workflow patterns.Source: §I · p. 1, Figure 1 · p. 1
  • VFlow combines hardware-specific verification operators with syntax checking, functional simulation, and boundary-condition testing.The process is designed to ensure generated code meets synthesizability and timing requirements.Source: Figure 1 · p. 1, §I · p. 2
  • 20%-30% average pass@1 improvement over direct invocation and mainstream multi-prompt methods was reported on VerilogEval and RTLLM2.0.VFlow also approached or exceeded human design reference values on area and power metrics.Source: §I · p. 1
  • Smaller specialized LLMs outperformed larger general-purpose models through optimized workflows while reducing inference overhead and cost.Source: §I · p. 2

A. Hardware Description Languages and Verilog

B. LLMs for Verilog Code Generation

LLM-based Verilog generation has progressed through stronger benchmarks, specialized models, graph-aware architectures, and verifiable reinforcement learning. Agentic workflows extend this progress by representing LLM actions as searchable graphs, enabling automated workflow discovery beyond manual design.

B. LLMs for Verilog Code Generation

  • Specialized and graph-enhanced models improve Verilog generation through hardware-focused training or textual representations of control-flow and data-flow graphs.RTL-Coder reached about 34% pass rate, while RTL++ improved correctness by encoding graph structure textually.Source: B. LLMs for Verilog Code Generation · p. 2
  • CodeV-R1 achieved 68.6% pass@1 on VerilogEval v2 and 72.9% on RTLLM v1.1, outperforming previous best models by 12-20 percentage points.Source: B. LLMs for Verilog Code Generation · p. 2
  • Hardware-focused LLM research now spans code generation, hallucination control, simulation and synthesis debugging, coverage-driven testing, and formal verification.Source: B. LLMs for Verilog Code Generation · p. 2

C. Agentic Workflows

  • Agentic workflows structure LLM invocations into sequences that support complex tasks across code generation, data analysis, question answering, and mathematical reasoning.Source: C. Agentic Workflows · p. 2
  • Automated workflow methods reformulate workflow design as a search over code-represented graphs whose nodes are connected by edges defining action logic, dependencies, and flow.Approaches such as AFLOW and MaAS use systematic search to discover workflows without extensive human intervention.Source: C. Agentic Workflows · pp. 2–3

III. VFLOW FRAMEWORK

VFlow formulates Verilog generation as a search over workflows that transform hardware specifications into code. It represents workflows as connected LLM nodes and seeks the configuration maximizing an evaluation function over models, prompts, parameters, formats, and connections.

A. Problem Formulation

  • VFlow represents a workflow as connected LLM-invoking nodes that transform a hardware task into Verilog code.The node set is N = {N1, N2, ..., Nk}, while E defines information flow and execution order.Source: A. Problem Formulation · p. 3
  • Each workflow node is characterized by a language model, prompt, temperature setting, and output format.Source: A. Problem Formulation · p. 3
  • VFlow seeks the workflow W* that maximizes an evaluation function G(W,T) for hardware task T.Source: A. Problem Formulation · p. 3
  • The search space covers combinations of models, prompts, parameters, and connections.Source: A. Problem Formulation · p. 3

B. Architecture of VFlow

VFlow uses CEPE-MCTS to search Verilog-generation workflows through multiple specialized populations that optimize distinct hardware objectives while sharing failure experience. Its architecture separates objective-focused exploration from cooperative knowledge exchange.

B. Architecture of VFlow

  • CEPE-MCTS maintains multiple specialized search populations that explore Verilog-generation workflows in parallel while sharing a global failure repository.The framework is designed to avoid repetitive errors during workflow discovery.Source: B. Architecture of VFlow · p. 3

1) Multi-population Cooperative Architecture:

  • The architecture represents each population with an MCTS tree, a specialized objective function, and a failure-experience repository shared across populations.Source: 1) Multi-population Cooperative… · p. 3
  • Four populations target functional correctness, area, timing, and balanced optimization across objectives.This specialization supports deep objective-specific searches and avoids conflicts inherent in traditional multi-objective optimization.Source: 1) Multi-population Cooperative… · p. 3

2) Fragment Migration Mechanism for Cooperative Evolution:

VFlow evaluates workflow fragments by integrating them into a target workflow and migrates them across populations only when their fitness exceeds a threshold.

2) Fragment Migration Mechanism for Cooperative Evolution:

  • Fragment migration evaluates a candidate workflow fragment after integrating it into the target population’s base workflow.Source: 2) Fragment Migration Mechanism for… · p. 3
  • Migration occurs when fragment fitness exceeds θ_migrate, allowing valuable patterns to propagate while filtering ineffective migrations.Fixed migration intervals enable recurring exchange of successful design experiences among populations.Source: 2) Fragment Migration Mechanism for… · p. 4

3) Global Failure Experience Sharing Mechanism:

VFlow shares failure experiences globally by recording workflow failures and weighting them according to structural similarity and recency. This mechanism reduces the influence of outdated patterns during failure-risk assessment.

3) Global Failure Experience Sharing Mechanism:

4) Cooperative Enhanced Selection Strategy:

The cooperative enhanced selection strategy extends MCTS by jointly considering specialized objective performance, exploration, failure risk, and diversity across populations.

4) Cooperative Enhanced Selection Strategy:

5) Algorithm Execution Flow:

CEPE-MCTS searches workflows in parallel, sharing valuable fragments while rejecting likely failures, and combines this search with progressive verification and hierarchy-aware design support.

5) Algorithm Execution Flow:

  • CEPE-MCTS runs parallel MCTS searches, performs cooperative evolution through thresholded fragment migration, and updates shared failure experience and Pareto-optimal workflows.Similarity checks reject candidates resembling historical failures, while migration propagates only fragments whose fitness gain exceeds θ_migrate.Source: 5) Algorithm Execution Flow · p. 4

1) Multi-Level Simulation Verification:

  • Progressive verification filters workflows through syntax analysis, functional simulation, and boundary-case testing, reducing computation spent on invalid candidates.The levels use binary syntax validity, test pass rate, and an all-pass requirement.Source: 1) Multi-Level Simulation Verification · p. 4

2) Hierarchical Design Pattern Support:

  • Hierarchical design support rewards appropriate modular depth while penalizing excessive hierarchy that may harm synthesis results.The metric compares hierarchical depth D_hier with a recommended maximum D_max.Source: 2) Hierarchical Design Pattern Support · p. 4

3) Verilog-Specific Operator Design:

VFlow combines Verilog-specific operators with benchmarked evaluation across diverse RTL tasks, models, prompting baselines, and synthesis-aware implementation settings.

3) Verilog-Specific Operator Design:

  • Verilog-specific operators validate syntax, execute simulations, analyze waveforms, optimize circuit metrics, and compose hierarchical modules.These operators target language rules, functional behavior, area, power, timing, and modular assembly.Source: 3) Verilog-Specific Operator Design · pp. 4–5

A. Experimental Setup

  • The evaluation covers 156 VerilogEval problems and nine RTLLM2.0 modules spanning combinational logic, finite-state machines, arithmetic, memory, and control designs.The datasets assess both broad benchmark coverage and more realistic medium-scale RTL designs.Source: A. Experimental Setup · p. 5
  • VFlow is compared with seven prompting baselines using Claude-3.7-Sonnet for baseline fairness, while VFlow selects among four LLMs.The baselines include IO, Chain-of-Thought, Self-Consistency CoT, MultiPersona Debate, Self-Refine, PromptV, and AFlow.Source: A. Experimental Setup · p. 5
  • Experiments use Icarus Verilog for functional verification and Yosys for synthesis metrics, with up to 20 MCTS iterations and early stopping after five stagnant rounds.Simple tasks optimize the trade-off between pass rate and API cost.Source: A. Experimental Setup · p. 5

B. Experimental Results and Analysis

VFlow improves Verilog generation success rates and hardware implementation quality while exploring workflow evolution, component contributions, and cost-performance trade-offs.

1) Overall Performance Comparison:

  • +19.6% machine and +30.9% human improvement over IO accompanies VFlow’s highest pass@1, pass@5, and pass@10 on VerilogEval.Table II reports the overall benchmark comparison against direct invocation and prior prompting strategies.Source: 1) Overall Performance Comparison · p. 5
  • VFlow matches or surpasses baseline area, power, and timing results across nine modules, achieving minimum area or power in several cases while remaining close to designer timing references.The reported balance spans resource efficiency, power consumption, and timing closure.Source: 1) Overall Performance Comparison · pp. 5–6
  • Figure 4 reports ablation studies of VFlow components using pass@1 performance.Source: Figure 4 · p. 6

2) Cost-Performance Analysis:

VFlow combines cost-performance gains with component-level validation and an evolving verification-integrated workflow, supporting its overall effectiveness for Verilog generation.

2) Cost-Performance Analysis:

  • +41.2% pass@1 over GPT-4o, with token consumption at 0.4×, cost at 0.13×, and 10.9× ROI for DeepSeek-V3.VFlow with GPT-4o-mini also achieves 34.7% higher pass@1 at less than half the cost.Source: 2) Cost-Performance Analysis · p. 6

3) Ablation Studies:

  • Removing MCTS causes over 20% performance loss, while removing domain-specific operators causes around 10% loss.Removing cooperative evolution or global failure sharing causes approximately 5–7% degradation, and omitting multi-level verification causes about 3–4%.Source: 3) Ablation Studies · p. 6

4) Discovered Workflow Analysis:

  • The discovered workflow progresses from a score of 0.56 to 0.84 through multi-implementation generation, Claude-based generation, comprehensive testing, and prompt optimization.Its final five-step process covers problem analysis, implementation generation, ensemble integration, testing, and targeted refinement.Source: 4) Discovered Workflow Analysis · p. 6

V. CONCLUSION

  • VFlow integrates MCTS-based search, domain-specific operators, and multi-level verification to improve Verilog code quality and efficiency.The conclusion presents the framework as having strong potential for advancing hardware design automation.Source: §V · p. 6

Related papers

Loading 2504.03723v2…