Source-linked AI summary
FPGAgent: An LLM-Assisted Framework for Autonomous HLS Code Generation and Verification in FPGA Environments
Tianyu Wang, Wenjie Wang, Jianguo Yao, Haibing Guan, Xijun Li
TL;DR
Existing LLM-based HLS systems often stop at simulation or synthesis, even though timing, place-and-route, and hardware-runtime constraints can prevent deployable FPGA designs. FPGAgent addresses this gap with multi-agent evolutionary generation, functional repair, and board-level executable validation. On HLS-Eval, it improves synthesizable rate, executability, and functional correctness over zero-shot baselines.
Problem
Existing HLS-generation approaches often lack real-hardware executable validation and well-established benchmark evaluation, limiting evidence of practical generalization.
Method
FPGAgent uses multi-agent evolutionary HLS generation, C/C++ functional verification and repair, and host-code compilation and execution on FPGA hardware.
Results
FPGAgent improves synthesizable rate by 16.9% on average, executability by 26.7%, and functional correctness by 30.6% over zero-shot baselines.
Takeaways & Limitations
FPGAgent demonstrates improved practical usability of LLM-based HLS generation through end-to-end validation from task specification to FPGA execution.
Takeaways & Limitations
The evaluation relies primarily on HLS-Eval, and executable generation is hardware-dependent, especially for host-side code and runtime-stack configurations.
Abstract
from arXiv · showhide
Large language models (LLMs) have shown substantial promise for high-level synthesis (HLS) code generation, but most existing approaches validate only simulation or synthesis results. Because of timing and place-and-route constraints, \emph{HLS code that passes simulation and synthesis may still fail to produce deployable, runnable designs on real FPGA platforms}. Moreover, the lack of public benchmarks has limited many evaluations to small, self-curated test suites. We propose FPGAgent, a multi-agent framework tailored to real FPGA environments for autonomous HLS coding with end-to-end executability validation. To the best of our knowledge, FPGAgent is \emph{the first task-specification-to-executable HLS generation framework experimentally validated on a well-established benchmark}. Given a natural-language task specification, FPGAgent injects HLS-specific knowledge and employs evolutionary search to iteratively derive reliable HLS kernel implementations. It then generates a C++ validation program to verify functional correctness, diagnoses potential defects, and guides targeted repairs. Finally, it synthesizes host code for compilation and board-level execution on FPGA hardware. We comprehensively evaluate FPGAgent with five established LLMs on HLS-Eval, a benchmark containing 78 tasks across multiple domains, and verify board-level executability on a real FPGA platform. Compared with existing baselines, FPGAgent improves the synthesizable rate by 16.9% on average, executability by 26.7%, and functional correctness by 30.6%. These results show that FPGAgent substantially improves the practical usability of LLM-based HLS generation and demonstrates the value of end-to-end validation.
1 Introduction
Existing LLM-based HLS approaches benefit from HLS’s software-like abstractions and compiler feedback, but often stop at simulation or synthesis and lack benchmarked, task-level, real-hardware validation. FPGAgent addresses these gaps with an end-to-end multi-agent framework combining evolutionary generation, functional repair, and FPGA executable validation.
- Motivation: HLS offers software-like abstractions and structured toolchain feedback, making it an attractive intermediate layer for LLM-based hardware generation.It narrows the gap between natural-language specifications and hardware implementations while supporting iterative optimization and repair.
- Research Gap: Existing approaches often validate HLS designs through compilation, co-simulation, or synthesis without confirming deployment and execution on real FPGA platforms.Timing closure, place-and-route, and resource constraints can prevent synthesized designs from becoming runnable hardware.
- Research Gap: Existing evaluations frequently rely on self-curated test suites, limiting empirical validation on medium- to large-scale benchmarks.This makes generalization difficult to assess, particularly for approaches that do not generate HLS directly from task specifications.
- FPGAgent: FPGAgent maps natural-language task descriptions to FPGA executables through multi-agent HLS generation and verification targeting real hardware.The framework invokes an industry-standard toolchain for simulation, synthesis, and executable construction.
- FPGAgent: FPGAgent combines evolutionary HLS code generation, multi-level feedback, functional verification and repair, and comparative evaluation on HLS-Eval.Its functional module validates C/C++ behavior, diagnoses defects, and guides targeted repairs.
2 Motivating Example
Simulation and synthesis do not guarantee that an HLS design can produce a deployable FPGA executable or correct board-level behavior. Aggressive pipelining and loop unrolling can instead create timing or routing failures during hardware generation.
- Failure Mechanism: An HLS kernel using pipeline initiation interval II=1 and full loop unrolling can create excessive transactional parallelism within one cycle.This may lengthen the critical path or create many long-distance cross-region interconnects.
- Failure Mechanism: Excessive critical-path length causes timing-closure failures, while extensive interconnects can cause routing congestion or unroutable designs.These failures can occur after the kernel has already passed simulation-based validation.
- Motivating Example: Simulation and synthesis are necessary but insufficient for executable FPGA designs with correct board-level behavior.The motivating case calls for end-to-end executable generation and board-level verification.
3 Methodology
FPGAgent uses three coordinated stages: evolutionary kernel generation, functional verification and repair, and host-code compilation and execution on physical FPGA hardware. The workflow combines domain knowledge with error-driven feedback across the HLS-to-hardware path.
- 3.1 Kernel Code Implementation: FPGAgent generates kernel code through evolutionary program search guided by HLS expertise and multi-level error feedback.The framework initializes candidates from task descriptions and applies Exploration and Modification operators during iterative optimization.
- 3.1 Kernel Code Implementation: Vitis HLS simulation and synthesizability checks provide feedback for revising generated kernels during implementation.Candidates that pass simulation enter the kernel pool, while failures trigger log-based revision.
- 3.2 Functional Reflection: Functional Reflection verifies generated kernels at the C/C++ level and repairs defects through verification, debugging, and coding agents.The verifier constructs test inputs and expected outputs, while the debugger analyzes discrepancies and the coder applies targeted fixes.
- 3.3 Compilation & Execution: Compilation & Execution generates host code, compiles kernel and host code into an FPGA executable, and verifies runtime behavior on physical hardware.Compilation failures are analyzed from toolchain logs, and the resulting program is deployed for board-level testing.
4 Experiment Setup
The evaluation studies FPGAgent through comparative and ablation experiments, using multiple metrics and comparisons across base models. Table 1 contrasts zero-shot pass@10 results with FPGAgent outcomes.
- 4.1 Research Questions: The experiments use comparative and ablation studies with multiple evaluation metrics to assess FPGAgent and its components.The study is designed to evaluate framework effectiveness across the reported research questions.
- 4.1 Research Questions: FPGAgent attains its highest functional correctness of 92.3% with Gemini 3.1 Pro.The table caption states that improvements are consistent across backbone models.
Research Questions
The study asks whether FPGAgent improves end-to-end HLS generation through FPGA execution, how its design choices contribute, and which failures remain challenging. It evaluates these questions using HLS-Eval, multiple code-generation models, and board-level experiments.
- RQ1 evaluates whether FPGAgent improves success from task specification through HLS generation, FPGA executable creation, and board-level execution.The question targets implementations that run correctly on real FPGA hardware.
- RQ2 examines how the kernel implementation and functional reflection modules affect overall end-to-end HLS generation performance.This is addressed through an ablation study.
- RQ3 analyzes the primary sources of end-to-end failures, identifying which FPGAgent mitigates and which remain challenging.The analysis motivates directions for future research.
- Evaluation setup: The evaluation uses HLS-Eval because public HLS code-generation benchmarks remain limited, while the dataset provides relatively large and well-specified benchmarks across application domains.The dataset includes task descriptions, kernel headers, and interface specifications.
- Evaluation setup: The experiments evaluate five representative general-purpose code-generation models and validate deployment using Vitis HLS 2022.2 on a Xilinx U200 FPGA at 200 MHz.The generated FPGA executable is treated as the final deliverable.
5 Evaluation
FPGAgent consistently improves HLS generation across models and evaluation stages, from synthesizability to hardware executability and functional correctness. Ablations and failure analysis attribute these gains to evolutionary kernel optimization, functional reflection, and host-side repair.
- Main Results: FPGAgent improves synthesizability by 16.9% on average, executability by 26.7%, and functional correctness by 30.6% over zero-shot generation.
- Main Results: FPGAgent improves generation stability through iterative refinement, with designs progressively improving and converging toward a reasonable performance regime.
- Ablation Study: Eoh solves 9 more instances than FPGAgent_base, a relative improvement of 31.0%, while improving synthesizability, executability, and functional correctness.
- Ablation Study: Reflect solves 8 more instances than FPGAgent_base, a relative improvement of 27.6%, without degrading synthesizability or executability.
- Ablation Study: The complete workflow solves 52 benchmark instances, outperforming variants that use only evolutionary optimization or functional reflection.
- Failure Analysis and Discussion: Error_Kernel is strongly model-dependent: failures decrease from 18 with Qwen3-Coder-30B-A3B to 2 with Gemini 3.1 Pro.
6 Threats to Validity
FPGAgent’s evaluation is bounded by benchmark, platform, and efficiency considerations. The study relies on HLS-Eval, hardware-dependent host-code generation, and expensive FPGA executable builds.
- Internal Validity: Reliance on HLS-Eval may bias the reported results because the primary experiments use a single benchmark.The authors focus on HLS-Eval because established benchmarks remain limited and it offers a specified interface, non-trivial scale, and diverse application domains.
- External Validity: Platform dependence is concentrated in host-code generation and compilation, which may require adapting prompts or interface templates when changing environments.The kernel-generation and verification mechanism is described as broadly applicable across platforms, while host-side implementation depends on the target device and runtime stack.
- Efficiency and Cost Validity: FPGA executable generation is the dominant time bottleneck, typically requiring approximately 3–5 hours per xclbin build.The workflow uses low-cost kernel-level synthesis and screening before one-shot xclbin generation to avoid repeatedly rebuilding hardware.
7 Related Work
Related work spans benchmark infrastructure, indirect HLS generation and repair, performance optimization, and agentic workflows. However, these directions commonly emphasize toolchain-level evaluation rather than complete FPGA deployment.
- Benchmark Building: HLS-Eval and Bench4HLS extend evaluation beyond text similarity to parsing, compilation, simulation, and synthesis.Their end-to-end evaluation frameworks establish infrastructure for executable correctness and synthesizability assessment.
- Indirect Generation: Prior methods convert C/C++ into HLS-compatible code, refactor implementations, or repair non-synthesizable programs through LLM-guided transformations.These approaches target reducing manual engineering effort required for HLS adoption.
- Performance Optimization: LLM-based optimization methods generate or refine directives such as pipelining, unrolling, and array partitioning to improve latency/resource tradeoffs.They may combine graph models, retrieval augmentation, or iterative search, but are often evaluated mainly at synthesis-level QoR.
- Agentic Workflow: Agentic HLS workflows iteratively refine kernels using compiler, simulator, and synthesis feedback for multiround repair and optimization.These systems align with established HLS toolchains such as Vitis HLS, while the supplied passage indicates notable limitations remain.
- Agentic Workflow: The paper positions FPGAgent within work that increasingly automates HLS generation, optimization, repair, and feedback-driven refinement.The supplied title passage identifies FPGAgent as an LLM-assisted framework for autonomous HLS code generation and verification in FPGA environments.
8 Conclusion and Future Work
FPGAgent coordinates multiple agents to generate FPGA executables from high-level task specifications, combining evolutionary kernel search with functional verification and repair. The framework improves HLS generation accuracy over zero-shot baselines, while future work targets performance and task-specific adaptation.
- Conclusion: FPGAgent generates FPGA executables through coordinated multi-agent HLS generation, toolchain execution, functional feedback, and error analysis.Its workflow covers simulation, synthesis, executable generation, and functional-correctness improvement from high-level task specifications.
- Conclusion: 30.6% average accuracy improvement over zero-shot baselines demonstrates substantially higher HLS code-generation accuracy.Using Gemini 3.1 Pro, the framework reaches up to 92.3% accuracy.
- Future Work: Future work will reduce latency and resource utilization while preserving executability and functional correctness.The authors also plan task-specific HLS fine-tuning for more accurate, efficient, and high-performance generation.
9 Data Availability Statement
The paper states that reproducibility materials will be available upon acceptance, while complete hardware execution may require specialized resources because of platform and toolchain dependencies.
- Data Availability: Source code, scripts, benchmark instance lists, and documentation for reproducing the main results will be available upon acceptance.Full end-to-end hardware execution may require an Xilinx FPGA board and a Vitis HLS environment.