Source-linked AI summary
LLM Agent-Based Design of RISC-V CPUs with FPGA-in-the-Loop Evaluation
Leo Itoh, Junichiro Kadomoto
TL;DR
Long RTL simulations can take hours, and short diagnostics may miss degradation that develops during execution. The paper presents an FPGA-in-the-loop environment that uses board measurements to guide agent-based CPU development. The final core reaches 5.61 CoreMark/MHz, and FPGA evaluation is reported as 9.6× faster than RTL simulation for CoreMark.
Problem
Full-length RTL simulation can take hours, while short diagnostics may miss performance degradation caused by predictor and scheduler state that emerges later.
Source: §I · p. 1Method
The environment feeds long-run FPGA board measurements into human-supervised LLM-agent CPU candidate selection and RTL analysis.
Source: §I · p. 1Results
The final design reaches 5.61 CoreMark/MHz at 130.137 MHz, and FPGA evaluation including initial implementation is 9.6× faster than RTL simulation for CoreMark.
Source: A. CPU Performance · p. 3, §V · p. 4Takeaways & Limitations
In the development case, board measurements guided candidate selection and subsequent RTL analysis and fixes.
Source: §V · p. 4, C. Case Study · p. 4
Abstract
from arXiv · showhide
Long-run RTL simulation of CPU candidates can take hours, while short diagnostic runs may miss performance degradation that emerges later in execution. This evaluation cost can delay design decisions in LLM-agent-based CPU development. We present a development environment in which LLM agents implement CPU candidates on an FPGA and use on-board performance measurements to guide design decisions. We applied it to the human-supervised development of a single-cycle RV32I core into a three-wide out-of-order RV32IM core. The final design reaches 5.61 CoreMark/MHz at 130 MHz on a Artix-7 FPGA. For the same CPU candidate running 8,000 CoreMark iterations, FPGA evaluation, including implementation, was 9.6x faster than single-threaded RTL simulation. In one case, board measurements revealed long-run performance degradation in a candidate that improved in a short diagnostic run. The result guided candidate selection and led to RTL-based root-cause analysis and a fix.
Index Terms—LLM, RISC-V, FPGA
CPU design-space studies and LLM-based RTL generation provide context for this work. It addresses the challenge of evaluating candidates over long runs by feeding physical-board measurements back into agent decisions.
I. INTRODUCTION
- Prior RISC-V studies compare pipeline and issue choices, while BOOM-Explorer and ChatDSE search design spaces using optimization or LLM-based estimates –.Kadomoto et al. compare performance, power, and area; Fu et al. compare single-issue, multiissue, and out-of-order cores,.Source: §I · p. 1
- LLM-related CPU work includes RTL generation and verification, prompt-structure comparisons, and instruction-mapped fault localization –.El-Helwani and Khalid evaluated an FPGA implementation of a generated single-cycle RV32I core.Source: §I · p. 1
- Full-length RTL simulation can take hours, while short diagnostics may miss degradation caused by predictor and scheduler state that develops later.Prior auto-arch-tournament agents run CoreMark on Verilator and combine iterations per cycle with place-and-route Fmax, without programming or running a physical board.Source: §I · p. 1
- FPGA execution is used to feed long-run board measurements back to LLM agents for CPU candidate selection and RTL analysis.The work focuses on this feedback loop rather than FPGA execution alone.Source: §I · p. 1
- The work connects agent-driven RTL changes to human-supervised on-board evaluation, reports development from single-cycle RV32I to three-wide out-of-order RV32IM, and compares evaluation time.The reported development includes long-run board results guiding candidate selection and analysis.Source: §I · p. 1
II. DEVELOPMENT FRAMEWORK
Agents extend a single-cycle RV32I core using verification and FPGA evaluation.
II. DEVELOPMENT FRAMEWORK
- Agents extend a single-cycle RV32I core using the verification and FPGA evaluation flow.Source: §II · p. 1
A. Human and Agent Roles
Human supervision sets the goals and boundaries of development, while agents implement and assess CPU changes. The framework records development activity and routes checks and board measurements into agent decisions.
A. Human and Agent Roles
- Humans set stage goals, suggest directions, approve work, and decide when development stops; agents handle RTL changes, design, implementation, optimization, analysis, and verification.Development used both single-agent and orchestrator–worker configurations.Source: A. Human and Agent Roles · p. 1
- The orchestrator–worker setup specifies change hypotheses, target files, verification conditions, acceptance criteria, and compute allocation for workers.Orchestrator and worker models and task exchange are described in the passage.Source: A. Human and Agent Roles · p. 1
- Functional checks and on-board performance measurements feed agent decisions through a shared candidate record.The FPGA testbed figure identifies the board and host connection/on-chip design.Source: Figure 1 · p. 2, Figure 2 · p. 2
- Recorded activity covers three changes adopted in the final CPU, not a complete census; FPGA implementation records include checkpoint resumptions.The changes were a load-wait predictor fix, register-move aliasing, and a two-way BTB.Source: A. Human and Agent Roles · p. 2
B. FPGA Evaluation
The FPGA evaluation setup combines host-side implementation tools with board measurements and uses functional verification alongside performance evaluation. Final acceptance requires agreement between full-length RTL simulation and independent board runs.
B. FPGA Evaluation
- Agents in WSL2 use Vivado, hw_server, and the serial API on Windows; Vivado reports provide resource and timing data, while board UART provides cycle counts and CRCs.The setup is shown in Fig. 2.Source: B. FPGA Evaluation · p. 2
- Verification combines lint, directed arithmetic, memory-dependence, and branch-recovery tests with per-retirement PC and register-writeback comparison against a Python RV32IM reference model.Final acceptance requires matching UART output from full-length simulation and two independent board runs using frozen RTL and firmware.Source: B. FPGA Evaluation · p. 2
C. Experiment Management
Experiment management records candidate lineage, artifacts, decisions, measurements, and handovers. Figures identify the final CPU pipeline and evaluation time including FPGA implementation.
C. Experiment Management
- Each candidate has an identifier and parent link, with frozen RTL, firmware, implementation settings, logs, hashes, and an accept-or-reject decision recorded.Tasks and reviews include measurement results and failure reasons; handover notes capture measurements, running jobs, and next steps.Source: C. Experiment Management · pp. 2–3
- Figure 3 marks register-move aliasing and the two-way BTB in orange in the final CPU pipeline.Source: Figure 3 · p. 3
- Figure 4 reports evaluation time including FPGA implementation, with broken axes scaled separately for each workload.Source: Figure 4 · p. 3
III. EXPERIMENTAL SETUP
The evaluation uses CoreMark and Embench-IoT on the final bare-metal RV32IM design, and reports performance and timing measures. Published soft-core comparisons differ in implementation and configuration conditions.
III. EXPERIMENTAL SETUP
- CoreMark ran for 8,000 iterations, while 18 of 19 Embench-IoT benchmarks were evaluated because cubic did not fit in the 32 KiB instruction memory.The final design targets bare-metal RV32IM with 32 KiB instruction memory and 64 KiB data memory.Source: §III · p. 3
- CoreMark performance is reported in iterations per second, elapsed time uses board-reported ticks divided by MMCM frequency, and IPC divides retired instructions by measured-interval ticks.Source: §III · p. 3
- Published soft-core results use speed grade -3 versus this work's -1, and differ in memory configuration, compiler settings, and design scope.Source: §III · p. 3
- RTL simulation counts were obtained using the same RTL and firmware.Source: §III · p. 3
IV. RESULTS
The final three-wide out-of-order RV32IM core reaches 5.61 CoreMark/MHz and a 1.759 frequency-normalized Embench score relative to Cortex-M4. Its development included staged pipeline and execution changes, with a predictor case study spanning evaluation, diagnosis, and fix selection.
A. CPU Performance
- Development progressed from a single-cycle RV32I core through a five-stage RV32IM pipeline, dual issue, out-of-order execution with register renaming, and frequency-oriented restructuring.Source: A. CPU Performance · p. 3
- 5.61 CoreMark/MHz at 130.137 MHz was the highest in Table II, 12% above the published NaxRiscv result.The final design is a three-wide out-of-order RV32IM core with register renaming and reached 730.6 iter/s over 8,000 CoreMark iterations.Source: A. CPU Performance · p. 3
- Figure 5 presents the predictor case study as long-run evaluation, RTL diagnosis, and fix selection.Source: Figure 5 · p. 4
- 1.759 was the geometric-mean frequency-normalized Embench-IoT score relative to the reference Cortex-M4 over 18 benchmarks.Source: A. CPU Performance · p. 4
B. Evaluation Time
With identical RTL and firmware, FPGA evaluation including implementation was faster than RTL simulation for both workloads. Existing-bitstream programming and result capture took a median of 27.5 seconds for CoreMark.
B. Evaluation Time
- 9.6× faster for CoreMark (3.91 h RTL versus 24.5 min FPGA) and 13.7× for Embench (15.05 h versus 65.7 min), using identical RTL and firmware.Source: B. Evaluation Time · p. 4
- With an existing bitstream, CoreMark programming and result capture took a median of 27.5 s.Source: B. Evaluation Time · p. 4
- The reported evaluation-time comparison uses identical RTL and firmware for the RTL and FPGA runs.Source: B. Evaluation Time · p. 4
C. Case Study
On-board measurements exposed a predictor-modified core that improved in a short diagnostic but degraded over a long run, guiding its rejection and subsequent RTL analysis. Analysis identified a load-wait predictor state behind the slowdown, while timing results also guided a separate RTL revision.
C. Case Study
- 9.98% more cycles over 8,000 iterations led the implementation agent to reject the predictor-modified core and retain the baseline after two independent board runs.Full-length RTL simulation reproduced the board tick count more than three hours later.Source: C. Case Study · p. 4
- 10.29% more cycles over iterations 239–2,000 followed a 2.62% reduction over iterations 2–237, with the slower state beginning at iteration 238.The transition was traced during per-iteration RTL analysis.Source: C. Case Study · p. 4
- An absorbing load-wait predictor state was linked to the slowdown; clearing its entry instead of storing distance 7 eliminated the abrupt slowdown in a 400-iteration RTL run.The baseline running the modified-core firmware remained flat through 300 iterations, ruling out the firmware difference as the cause of the transition; the selected fix added no state.Source: C. Case Study · p. 4
- A −0.291 ns setup slack at 130.137 MHz prompted separating substitution data from the instruction queue into a separate LUTRAM with a ready bit.The final design combined this revision with the two-way BTB and passed on-board acceptance.Source: C. Case Study · p. 4
V. CONCLUSION
The paper presents an FPGA-in-the-loop environment for human-supervised, agent-based CPU development. In the reported evaluation, FPGA evaluation was faster than RTL simulation, and the resulting core reached 5.61 CoreMark/MHz at 130 MHz on board.
V. CONCLUSION
- On-board measurements guided candidate selection and subsequent RTL analysis and fixes; in this development case, the resulting core reached 5.61 CoreMark/MHz at 130 MHz, above published VexRiscv and NaxRiscv results.Source: §V · p. 4
- 9.6× faster for CoreMark and 13.7× faster for the 18 Embench benchmarks was FPGA evaluation than RTL simulation, including initial FPGA implementation cost.Source: §V · p. 4
- On-board measurements guided candidate selection and subsequent RTL analysis and fixes in the development case.Source: §V · p. 4