Source-linked AI summary
UVMarvel: an Automated LLM-aided UVM Machine for Subsystem-level RTL Verification
Junhao Ye, Dingrong Pan, Hanyuan Liu, Yuchen Hu, Jie Zhou, Ke Xu, Xinwei Fang, Xi Wang, Nan Guan, Zhe Jiang
TL;DR
Subsystem-level UVM testbench construction and high-quality stimulus generation remain manual and expertise-intensive, despite verification consuming nearly 70% of front-end effort. UVMarvel combines an IR and Bus Protocol Library for testbench construction with Signal Tracker- and Verilog Patching-guided stimulus refinement. It reports 95.65% average code coverage and a 4.5-hour automated execution, compared with several working days for manual verification.
Problem
Subsystem-level UVM testbench construction and high-quality stimulus generation remain manual and require protocol and micro-architectural expertise, while verification consumes nearly 70% of front-end effort.
Source: §1 · p. 1Method
UVMarvel uses an IR and Bus Protocol Library to construct subsystem-level UVM testbenches, then Signal Tracker and Verilog Patching-guided refinement to generate targeted stimuli.
Source: §1 · p. 2, §2 · p. 2, §3.2 · p. 3Results
95.65% average code coverage; verification runs in 4.5 hours, 20.17× faster than the manual verification flow.
Source: §4.3 · p. 5Takeaways & Limitations
Subsystem-level verification tasks that required several working days of human-driven development and regression are completed within an hour-scale automated run.
Source: §4.3 · p. 5Comparative and ablation studies use code coverage as the primary metric because it is comparable across designs; functional coverage depends on engineer-defined covergroups and remains subjective.
Source: §4.2 · p. 5
Abstract
from arXiv · showhide
Verification presents a major bottleneck in Integrated Circuit (IC) development, consuming nearly 70% of total effort. While the Universal Verification Methodology (UVM) improves reuse through structured verification environments, constructing subsystem-level UVM testbenches and generating high-quality stimuli still require extensive manual coding, repeated EDA tool runs, and deep protocol and micro-architectural expertise. We present UVMarvel, an automated verification framework that leverages Large Language Models (LLMs) to build UVM testbenches for subsystem-level RTL. UVMarvel introduces an Intermediate Representation (IR) and a Bus Protocol Library to translate heterogeneous specifications into protocol-correct subsystem-level UVM testbenches, and employs a Signal Tracker and a Verilog Patching Library to guide LLM-based stimuli refinement. UVMarvel is the first framework capable of automatically constructing subsystem-level UVM testbenches across mainstream bus protocols, and it achieves an average code coverage of 95.65%, while reducing verification time from several human working days to a 4.5-hour automated execution.
1 Introduction
Subsystem-level UVM verification remains a manual, expertise-intensive bottleneck: template-based flows do not capture design semantics, and LLM-based methods struggle with subsystem complexity and deep stimulus sequences. UVMarvel combines specification-to-IR testbench construction with coverage-guided stimuli refinement, reaching 95.65% coverage.
Motivation
- Nearly 70% of the front-end cycle is consumed by verification, while assembling testbenches for real designs still requires deep engineering expertise.Template-driven UVM flows automate routine code but leave protocol timing, transaction semantics, and functional constraints to engineers.Source: §1 · p. 1
Challenges
- Subsystem-level UVM automation is difficult because multimodal specifications and inter-IP dependencies exceed general-purpose LLMs’ capacity to reliably assemble correct testbenches.Prior LLM-aided UVM advances described here are confined to IP-level designs.Source: §1 · p. 1
- Generated stimuli average below 40% coverage and often miss corner cases, long dependency chains, rare event sequences, and intricate state transitions.Power request/acknowledgement handshakes are one example of multi-component sequences that require stepwise orchestration.Source: §1 · pp. 1–2
Contributions
- 95.65% code coverage is achieved by UVMarvel, which combines an IR, Bus Protocol Library, Signal Tracker, and Verilog Patching Template Library for subsystem-level verification.The framework targets nuanced multi-step sequences needed to cover corner cases and long inter-IP dependencies.Source: §1 · p. 2
2 UVMarvel: An Overview
UVMarvel organizes subsystem-level verification into testbench construction followed by stimuli refinement. Specifications and RTL inform testbench generation, while uncovered coverage points guide targeted stimulus updates and waiving candidates.
2 UVMarvel: An Overview
- The framework has two stages: construct a UVM testbench from specifications and RTL, then refine stimuli using coverage feedback.The workflow produces verification reports with error logs, waiving candidates, and coverage results.Source: §2 · p. 2
- An IR summarizes subsystem modules, connections, and interface roles, while the Bus Protocol Library supplies transaction guidance for testbench generation.The generated testbench runs with the DUT to produce an initial verification report.Source: §2 · p. 2
- For refinement, the Coverage Analyser decomposes uncovered points, the Signal Tracker locates relevant RTL paths, and patching creates a compact filtered DUT for LLM guidance.Generated stimuli or justified waiving candidates are then tested on the original DUT, producing the final report.Source: §2 · p. 2
3 UVMarvel: The Framework Pipeline
The framework pipeline turns specifications and RTL into structured UVM testbench inputs, then uses coverage feedback to identify RTL dependencies and guide targeted sequence generation. Its components address protocol semantics, coverage reporting, cross-file tracing, and the syntactic reconstruction needed to present relevant RTL to LLMs.
3.1 Intermediate Representation
- The five-part IR captures module name, interface description, register configuration, timing characteristics, and functional description for UVM construction.It distills information needed for environment, agent, register-model, and scenario construction.Source: §3.1 · pp. 2–3
3.2 Bus Protocol Library
- Protocol-specific UVM skeletons provide fixed control flow and handshake structure, while LLMs specialize designated regions with DUT details such as signal names, widths, and address ranges.The Bus Protocol Library supplies skeletons for interfaces, drivers, monitors, and agents.Source: §3.2 · p. 3
3.3 Coverage Analyser
- The Coverage Analyser converts HTML reports into structured, task-oriented summaries of uncovered or partially covered items, grouped by coverage type and module.It includes hierarchical names and source files while removing report UI markup.Source: §3.3 · p. 3
3.4 Signal Tracker
- The Signal Tracker starts from uncovered coverage signals, traces related statements across files, and links the resulting dependency slice to top-level I/O stimulus entry points.It addresses deep cases whose signal chains span modules and whose assignments are scattered across files and processes.Source: §3.4 · pp. 3–4
3.5 Verilog Patcher
- The Verilog Patcher groups tracked statements by RTL construct and restores minimal syntax, producing a filtered DUT for LLMs to use during stimuli refinement.Templates rebuild structures such as case blocks, always blocks, module shells, assignments, and instance connections.Source: §3.5 · p. 4
3.6 LLM-Guided Sequence Execution
- Three LLMs propose candidate sequences from the filtered DUT and uncovered items; sequences run on the original DUT, while unreachable-point waivers and error feedback enter the refinement loop.Compilation or simulation errors can prompt sequence repair or discard and another attempt.Source: §3.6 · p. 4
4 Evaluation
Evaluation spans heterogeneous subsystem designs and bus interfaces, measuring generation success, code and functional coverage, runtime, and effects of UVMarvel’s staged inputs and refinement.
4 Evaluation
- The evaluation uses GPT-4.1 by default, with Claude 4.5 and Gemini 2.5 Pro as comparative baselines, and runs generated testbenches on Synopsys VCS.Success requires completing VCS compilation and simulation without errors, passing all checkers, and matching outputs from an experienced-engineer reference under the same regression stimuli.Source: §4 · p. 5, §4.1 · p. 5
4.1 Evaluation Metrics
- Code coverage is the primary comparison metric because it is defined on RTL structure; functional coverage uses engineer-defined covergroups and is reported for completeness.The benchmarks cover heterogeneous IPs connected through APB, AHB, AXI, P-Channel, and Q-Channel interfaces.Source: §4.2 · p. 5, §4.1 · p. 5
4.3 Overall Framework Effectiveness
- 4.5 hours and 20.17× faster than manual verification to reach 90% code coverage from a human-provided IR.The Figure 6 time evaluation excludes test planning and IR authoring; its caption reports nearly twenty times faster to 90% coverage and comparable final coverage.Source: §4.3 · p. 5, Figure 6 · p. 5
- 95.65% average code coverage, with UVMarvel higher on all five coverage components than MEIC and UVM2 across the benchmarks.The comparison isolates stimulus generation by supplying both baselines with a subsystem-level UVM testbench and Bus Protocol Library.Source: §4.3 · pp. 5–6, Figure 7 · p. 6
4.4 Impact of IR on Stage a
- Nearly 30% average coverage improvement with IR inputs over original specifications, which often led to incomplete environments missing bus monitors or register-model connections.The IR presents design intent in a predictable, composable format to help infer handshake directions, transaction ordering, and interface semantics.Source: §4.4 · p. 6, Figure 8 · p. 6
4.5 Impact of Bus Protocol Library on Stage a
- 65.78% higher SRG with the Bus Protocol Library; for AXI, success rises from below 7% without guidance to above 90% with it.The comparison fixes non-bus UVM components and asks the LLMs to generate only bus-side components; AXI stresses channel ordering, responses, and backpressure.Source: §4.5 · p. 6, Figure 9 · p. 6
4.6 Impact of Signal Tracker & Verilog Patcher on Stage b
- 11.53% average coverage gain with the Filtered DUT, versus 6.66% with Vizier and 3.35% with the raw DUT.The Filtered DUT supplies compact, dependency-preserving context; the paper reports that Vizier requires many optimization iterations and that unguided LLM refinement makes limited progress on the full subsystem RTL.Source: §4.6 · p. 6, Figure 10 · p. 6
4.7 Code coverage achieved by different LLMs
- Coverage improvement remains stable across three LLMs using the same IR, protocol scaffolding, and Filtered DUT.Table 3 reports code coverage achieved by different LLMs.Source: §4.7 · pp. 6–7, Table 3 · p. 6
5 Conclusion
UVMarvel combines an IR and Bus Protocol Library to construct subsystem-level UVM testbenches, then uses signal tracking and Verilog patching to refine stimuli.
5 Conclusion
- The framework reports 95.65% score coverage with a 4.5-hour automated execution.The conclusion presents UVMarvel as the first UVM-based verification framework for subsystem-level RTL.Source: §5 · p. 7