Source-linked AI summary
Back to the Future: Rethinking EDA Infrastructure for Agentic Systems in Chip Design Verification
Je Yang, Ivan Lobov, Thomas Karpati
TL;DR
Post-simulation waveform debugging remains underexplored, and raw dumps lack an agent-ready query interface. BTTF makes waveforms relational and uses collaborative agents for verification queries, achieving 95.33% overall accuracy across a 150-query taxonomy.
Problem
Post-simulation waveform debugging and dynamic verification remain underexplored, while engineers manually inspect large temporal dumps that are difficult for agents to query.
Source: §1 · p. 1Method
BTTF stores waveforms in a normalized relational SQLite schema and uses specialized agents to translate natural-language verification queries into SQL and carry out checks.
Source: §3 · p. 2, §3.2 · p. 3Results
95.33% overall accuracy across the full 150-query taxonomy.
Source: §4 · p. 4Takeaways & Limitations
The results support the paper’s thesis that data infrastructure is a binding constraint on agentic waveform debugging and suggest improving agent–data interfaces as a route toward autonomous EDA verification.
Source: §5 · p. 4
Abstract
from arXiv · showhide
The unprecedented computational scale of modern artificial intelligence depends on complex, multi-billion-transistor Systems-on-Chip, yet the workflows that verify these chips remain stubbornly manual. Although Large Language Models (LLMs) have made rapid inroads into Electronic Design Automation (EDA), approximately 74.6% of existing studies target static Register-Transfer Level (RTL) code generation, leaving post-simulation verification and interactive waveform debugging largely untouched. We introduce Back-to-the-Future (BTTF), an end-to-end agentic framework that closes this infrastructural gap. BTTF distills massive, unstructured simulation dumps into a normalized relational SQLite database and couples it with a collaborative multi-agent orchestration engine that translates natural-language verification queries into schema-aware SQL while correlating signal anomalies with versioned RTL repositories. Across a 150-query benchmark, BTTF attains 95.33% execution accuracy, charting a practical path toward autonomous EDA verification.
1 Introduction
Modern chip verification remains labor-intensive, while most LLM-for-hardware research targets RTL generation rather than dynamic, post-simulation debugging. BTTF turns waveform dumps into a relational database for agentic querying and reports 95.33% overall accuracy on 150 expert-annotated queries.
1 Introduction
- Verification consumes over 70% of the development cycle, and bugs escaping to full-chip regression or post-silicon testing carry substantial financial and schedule penalties.The introduction identifies automated RTL verification and analysis as an urgent EDA goal.Source: §1 · p. 1
- Approximately 74.6% of LLM-for-hardware studies focus on front-end RTL generation and synthesis, while post-simulation waveform debugging and dynamic verification remain underexplored.The paper notes that hardware development requires rigorous testing alongside synthesis.Source: §1 · p. 1
- Raw ASCII waveform dumps exceed model context windows by orders of magnitude and lack a structured semantic index or native low-latency query interface.During failures, engineers manually inspect terabytes of temporal dumps to trace illegal transitions through the HDL hierarchy.Source: §1 · pp. 1–2
- BTTF converts unstructured waveforms into a normalized relational SQLite database and pairs it with collaborative agents for natural-language verification queries.On 150 expert-annotated queries, it reports 95.33% overall accuracy.Source: §1 · p. 2
2 Related Work
Prior LLM-assisted verification systems largely address static source code, assertions, or toy-scale logs, leaving production-scale interactive waveform debugging comparatively unaddressed. BTTF’s related-work rationale is to make waveforms relational so agents can use SQL and database capabilities.
2 Related Work
- Existing LLM-assisted hardware verification and multi-agent systems predominantly operate on static source code, compile-time assertions, or toy-scale simulation logs.These approaches often assume functional errors can be localized from static HDL traces or terse compiler messages.Source: §2 · p. 2
- The closest cited waveform-tracing approach uses an AST-based tool, but remains tied to small, code-centric examples rather than production dumps.The work is cited as [11].Source: §2 · p. 2
- BTTF’s alternative is to relationalize waveforms, leveraging LLMs’ natural-language-to-SQL capability and database indexing, aggregation, and low-latency filtering.The SQL-generation observation is cited as [12].Source: §2 · p. 2
3 BTTF Framework
BTTF connects an offline waveform-preprocessing pipeline to an online agentic orchestration engine through a relational SQLite schema. The pipeline reduces and organizes simulation data, while specialized agents query it and perform verification.
3 BTTF Framework
- BTTF couples offline waveform preprocessing, which creates an optimized relational SQLite schema, with online orchestration that converts natural-language verification queries into schema-aware SQL.The two subsystems share a single relational contract.Source: §3 · p. 2
3.1 Waveform Preprocessing Pipeline
- For SoC-scale blocks, raw VCD files can exceed 10 GB, so preprocessing extracts target hierarchies into SQLite tables for signal metadata and timestamped changes.Separating signal identity from its temporal behavior allows agents to join structure and time when a query requires it.Source: §3.1 · pp. 2–3
3.2 LLM-Driven Agent Orchestration
- A collaborative orchestrator routes query sub-goals to analysis, verification, and evaluation agents, which query waveforms, check SVA rules, and audit answers against five criteria.Agents share session context, and failed queries can trigger self-correction; assertion violations can be localized to a timestamp and problematic sub-signals.Source: §3.2 · p. 3
4 Results
BTTF was evaluated on diverse waveform windows and expert-annotated verification queries, with results covering preprocessing trade-offs, database-generation speed, benchmark accuracy, and agent architecture.
Experimental setup
- The benchmark used Gemini-2.5-Pro by default and covered multiple SoC hierarchies, 4.2 GB of VCD, 18,450 nets, 50–800 ns windows, and 150 expert-annotated queries.The windows ranged from narrow high-resolution glitch-debug captures to full-transaction simulation dumps.Source: §4 · p. 3
Preprocessing Performance
- Clock pruning reduced footprint by 66.20% and query latency by 3.28× while preserving 98.51% of functional toggle coverage; further pruning untoggled signals reduced footprint by 72.18% with full active-net coverage.Aggressive below-average pruning compressed furthest but sacrificed 35.77% of coverage; Table 1 presents the pruning-strategy trade-offs.Source: §4 · p. 3, Table 1 · p. 4
- 2.89× maximum database-generation speedup over single-row insertion was achieved with SQL EXECUTEMANY batching.Figure 2 reports the insertion speedup; the text attributes it to removal of the I/O bottleneck for multi-terabyte waveforms.Source: Figure 2 · p. 4, §4 · p. 4
BTTF Framework Performance
- 95.33% overall accuracy across 150 queries spans routine debugging, toggle-based performance analysis, and formal verification.Table 2 summarizes the verification-task coverage.Source: Table 2 · p. 4, §4 · p. 4
- 17.3% higher reasoning quality came with an average 1.96× latency overhead for collaborative agents versus a monolithic single agent.The monolithic design scored 3.35/5.00 on the Evaluation audit and was faster, but suffered context clutter, schema hallucination, and degraded causal tracing on multi-hop tasks.Source: §4 · p. 4
5 Discussion and Conclusion
The discussion argues that data infrastructure, rather than model capability, constrains agentic waveform debugging. BTTF's results support relational data interfaces and specialized agents as a route toward autonomous EDA verification.
5 Discussion and Conclusion
- BTTF argues and empirically supports that data infrastructure, not model capability, is the binding constraint on agentic waveform debugging.It reports 95.33% benchmark accuracy and a 17.3% quality gain from agent specialization.Source: §5 · p. 4
- Scaling to larger full-chip designs calls for streaming ingestion, standardized scope-path normalization, and a waveform-to-RTL symbol index.Evaluating larger frontier reasoning models for multi-hop causal tracing is identified as future work.Source: §5 · p. 4
- The normalize-then-specialize recipe is proposed as a template for regression logs, coverage databases, and power traces, and for treating data interfaces as a first-class research object.The paper frames BTTF as more than a waveform-specific point tool.Source: §5 · p. 5
A.1 Handling Assertion Failures and Root-Cause Diagnostics
BTTF's assertion-failure handling turns a binary failure into a timestamped, signal-level root-cause diagnostic by tracing antecedent values backward.
A.1 Handling Assertion Failures and Root-Cause Diagnostics
- On assertion failure, the agent extracts the exact violation timestamp when the condition becomes true or an invalid X/Z state is detected.The procedure applies to rules such as ASSERT_NEVER and ASSERT_KNOWN.Source: §A.1 · p. 6
- It queries prior values of all assertion antecedent operands at times t ≤ t_violation to trace the failure backward.Source: §A.1 · p. 6
- The diagnostic identifies the specific sub-signal responsible, such as req_sel becoming unknown while clk is high.This connects high-level assertion monitoring with low-level root-cause debugging.Source: §A.1 · p. 6
A.2 End-to-End Query-to-Response Pipeline
The query-to-response pipeline routes natural-language verification questions to schema-aware tools, retrieves deterministic waveform results, and correlates signal hierarchy with versioned RTL source.
A.2 End-to-End Query-to-Response Pipeline
- The Orchestration Agent routes a query such as a module-scope request to the Analysis Agent, which maps it to a function call or SQL join.The example uses find_signal_in_scope or joins signal_metadata with changes.Source: §A.2 · p. 7
- Queries over indexed waves.db return exact timestamps, values, and transition counts without context-window truncation or string hallucination.Values may be returned in hexadecimal or decimal format.Source: §A.2 · p. 7
- For questions spanning waveform behavior and design intent, BTTF correlates the resolved hierarchy with the versioned repository to return RTL line numbers and declaration semantics.Source: §A.2 · p. 7