PaperlayerView exact revision on arXiv

Source-linked AI summary

Evidence-Guided Repository-Level RTL Repair

Yuxin Du, Juxin Niu, Zhe Jiang, Nan Guan

arXiv:2610.03219v1cs.AR

TL;DR

Repository-level RTL repair must localize failures across files, modules, and clock cycles, then propagate fixes consistently, but source code alone does not show the failing execution. The paper grounds diagnosis and repair checking in a reproduced waveform and coordinates edits before replaying the same scenario. On the evaluated HWE-Bench tasks, the framework resolves more tasks than the baseline for every tested model.

Abstract

from arXiv · show

Repository-level RTL repair must localize a failure that spans files, modules, and clock cycles, then propagate the fix consistently. Existing methods reason over source code, which reveals possible behaviors but not the failed execution, and cannot tell whether a local fix was propagated consistently. We therefore present an evidence-guided framework with three modules. Failure grounding converts a problem statement into a reproduced failing run and a failure anchor. Waveform-guided localization uses a localization toolbox to narrow the observed violation into a candidate mechanism and an evidence trail. Consistency-aware repair and validation then expand that seed into a coordinated patch and replay the same scenario to check that the violation disappears. We conducted experiments on HWE-Bench and achieved better performance than the baseline.

I. INTRODUCTION

Repository-level RTL repair must trace failures across files, modules, and clock cycles, then propagate fixes consistently. The paper introduces an evidence-guided framework and reports that it resolves more HWE-Bench tasks than the baseline for every tested model.

I. INTRODUCTION

  • Repository-level repair takes a repository and problem statement as input, and must produce a patch that passes fail-to-pass tests despite cross-file, cross-module, and cross-cycle failures.Unlike module-level repair, no specification or reference design is available to confirm a candidate cause, and every affected location must be changed consistently.Source: §I · p. 1
  • Source code shows possible paths rather than the failing run, leaving competing causes plausible; Figure 1 illustrates this cycle-level evidence gap.Even after finding a local cause, an agent may not preserve the same scenario or propagate the fix to stale interfaces, widths, or state updates elsewhere.Source: §I · p. 1, Figure 1 · p. 1
  • The framework grounds repair in a reproduced failing run, localizes a candidate mechanism with an evidence trail, and expands it into a coordinated patch checked by replay.The same scenario is replayed to check that the violation disappears without breaking surrounding behavior.Source: §I · pp. 1–2
  • Across the same 388 runnable HWE-Bench tasks, the framework resolves more tasks than the baseline for every tested model.The setup excludes 29 OpenTitan cases that could not be adapted to a runnable VCS environment.Source: §I · p. 2

II. RELATED WORK

Prior RTL repair methods primarily work at module scale or stop at localization, while repository-level coding agents address software repositories. This work targets end-to-end repair of existing RTL repositories.

A. Automated RTL Repair

  • Module-scale RTL repair assumes a supplied failing testbench, while repository-level work takes a whole repository and problem statement and may require changes across files and modules.Prior methods include repair tools, multi-agent and solver-based approaches, waveform tracing, and formal-counterexample debugging; BluesFL stops at localization.Source: A. Automated RTL Repair · p. 2

B. Repository-Level Coding Agents

  • Repository-level coding agents target software repositories, whereas this work establishes reproduction, localization, and validation in project-specific flows for existing RTL repositories.Prior hardware-repository work cited here uses knowledge graphs over Verilog sources and descriptions.Source: B. Repository-Level Coding Agents · p. 2

A. Overview

The evidence-guided loop centers on a waveform from a reproduced failing run. It uses that execution to move from the problem statement through localization to coordinated edits and replay-based checking.

A. Overview

  • Failure grounding turns the problem statement into a concrete failing execution and its waveform.The waveform is the shared artifact used by the subsequent stages.Source: A. Overview · p. 2
  • Waveform-guided localization uses a reusable toolbox to extract, trace, and semantically interpret signal behavior from that execution.The localization stage narrows the observed violation into a diagnosis used by repair.Source: A. Overview · p. 2
  • Consistency-aware repair turns the local diagnosis into coordinated repository edits and replays the same scenario to check that the violation disappears.The replay also checks whether the revised design preserves the reproduced scenario.Source: A. Overview · p. 2

B. Failure Grounding

Failure grounding constructs a run aligned with the reported scenario, then records the situation, observed violation, and reproduction recipe in a stable failure anchor. Later localization and validation use that anchor to stay tied to the same failure.

B. Failure Grounding

  • The framework parses a problem statement into preconditions P, a triggering event or transaction E, and required behavior Q that is violated.The held-out fail-to-pass tests are not visible to the agent, so it must construct its own failing run.Source: B. Failure Grounding · p. 2
  • A reproduced run is accepted only when its waveform confirms that P and E occur and Q is observably violated.The stimulus and configuration must follow intended usage assumptions; unclear assumptions are recorded rather than treated as facts.Source: B. Failure Grounding · p. 2
  • The failure anchor F = (S, V, R) records the reproduced situation S, observed violation V, and stimulus and configuration R.S contains the preconditions and trigger, while V records the required behavior, its location, and the cycle or interval.Source: B. Failure Grounding · p. 2
  • Validation replays R and checks that S still occurs while V disappears, keeping diagnosis and repair checking aligned to the reproduced failure.Localization consumes (V, R) to explain the observed symptom.Source: B. Failure Grounding · p. 2

C. Waveform-Guided Localization

Waveform-guided localization queries the reproduced failure alongside the elaborated design to narrow an observed violation into a candidate mechanism and evidence trail. It progresses from timing and active-driver evidence to RTL-level semantics, widening or comparing evidence when needed.

C. Waveform-Guided Localization

  • Localization takes the observed violation in the reproduced run and returns a candidate mechanism with an evidence trail.The toolbox supports queries across module boundaries, instantiations, and generated hierarchy.Source: C. Waveform-Guided Localization · pp. 2–3
  • The framework’s loop reconstructs a failing run, uses its waveform to localize a candidate mechanism, and checks a coordinated patch against the same scenario.Validation checks whether the observed violation is removed while the scenario is preserved.Source: Figure 2 · p. 3
  • Five structured query classes progressively narrow analysis from a symptom signal to a bounded interval, exercised logic, and design-level semantics.The query classes are intended to make large, heterogeneous waveforms systematically queryable.Source: C. Waveform-Guided Localization · p. 3, Table I · p. 3
  • Semantic lifting decodes packed fields, maps state encodings to symbolic states, and aligns related waveforms on a shared timeline.These artifacts support reasoning about mechanisms at the RTL semantic level rather than as raw integers.Source: C. Waveform-Guided Localization · p. 3
  • The evidence-narrowing loop time-bounds the violation, traces active drivers, and lifts values when needed, widening the window or comparing alternatives if evidence remains insufficient.It stops when a candidate mechanism and an evidence trail recording support and remaining ambiguities can be summarized.Source: C. Waveform-Guided Localization · p. 3

D. Consistency-Aware Repair and Validation

Consistency-aware repair expands a local seed across structurally coupled repair sites and validates the coordinated patch by replaying the reproduced scenario. The relation graph combines source and elaboration evidence to identify propagation scope.

D. Consistency-Aware Repair and Validation

  • The repository relation graph links repair sites at declaration, implementation, structural-binding, and generated-RTL levels, with edges marking relations that may require coordinated updates.Enclosing modules or files may also be included when needed for propagation.Source: D. Consistency-Aware Repair and… · pp. 3–4
  • Graph construction combines source parsing with elaboration to recover concrete bindings, resolved hierarchy, parameters, widths, and generated replicas.Generated RTL is linked to its generator when metadata is available; otherwise conservative structural links are marked uncertain.Source: D. Consistency-Aware Repair and… · p. 4
  • Table II contrasts source-declared instantiation sites with instances in the configured design for five elaborated modules.The relation graph follows elaborated structure because textual counts can overstate scope for unbuilt sites and understate it for replicated instances.Source: D. Consistency-Aware Repair and… · p. 4, Table II · p. 4
  • Consistency closure over-approximates structurally coupled sites, then selects those requiring synchronized edits before replaying the reproduction recipe.A patch is accepted only when replay preserves the reproduced preconditions and trigger while eliminating the required-behavior violation.Source: D. Consistency-Aware Repair and… · p. 4
  • If replay preserves the preconditions and trigger but the violation remains, the consistency scope expands before another repair attempt.The loop propagates the revised patch across selected sites and repeats validation.Source: D. Consistency-Aware Repair and… · p. 4

A. Experiment Setup

The evaluation uses HWE-Bench fail-to-pass tests to measure resolved rate across a common pool of runnable tasks. It compares the framework with model-specific baselines and describes the framework’s added modules.

A. Experiment Setup

  • 417 HWE-Bench tasks span six RTL repositories, but all conditions use the same remaining 388 tasks after 29 OpenTitan cases could not be adapted to runnable VCS.Resolved rate is the share of tasks whose held-out fail-to-pass tests all pass; agents do not see those tests.Source: A. Experiment Setup · p. 4
  • Baseline models run under their respective unchanged scaffolds: DeepSeek V4 Flash (0731) with OpenHands SDK, DeepSeek V4 Pro with Claude Code, and Kimi K2.6 with Kimi CLI.Source: A. Experiment Setup · p. 4
  • The framework adds failure grounding, waveform-guided localization, and consistency-aware repair and validation to the baseline.The additions reproduce a failing run, expose waveform queries, propagate local edits to structurally coupled sites, and replay the scenario.Source: A. Experiment Setup · p. 4
  • Figure 3 presents resolved rate per repository for DeepSeek V4 Flash (0731) under the baseline, the framework, and one-arm ablations.Its radial axis is percent and begins at 50%.Source: Figure 3 · p. 5

B. Results and Analysis

Every tested model resolves more HWE-Bench tasks with the framework than with its baseline, with gains concentrated in the two largest repositories. The results also distinguish the contributions of evidence-guided localization and consistency-aware repair.

RQ1: Does the framework resolve more tasks?

RQ2: How much does each arm contribute?

  • Either ablation reduces repairs: waveform evidence supplies a mechanism, while consistency checks catch coupled edits a single waveform cannot reveal.The consistency arm matters most for repairs spanning coupled modules.Source: RQ4: Why do the remaining failures… · p. 5

RQ3: What changes in how the agent works?

  • Reasoning-token share shifts by repository within baseline variation, while tool-call share is higher with the localization toolbox mounted.The agent combines reasoning with execution; verified toolbox scripts return evidence it can act on.Source: RQ3: What changes in how the agent… · p. 5, Figure 4 · p. 5

RQ4: Why do the remaining failures persist?

  • Unresolved baseline failures are partitioned into cases accepted by the alignment check, rejected for inadequate waveforms, and lacking a waveform.The no-waveform cases arise when the repository's simulation flow cannot be driven in the evaluation setup.Source: RQ3: What changes in how the agent… · p. 5, Table IV · p. 6
  • Even when waveforms expose failure values, they may not reveal an absent interface or the protocol ordering and handshake needed for a correct edit.The examples include an unbuilt port, a generator-level control-bus change, and key-manager field mappings.Source: RQ3: What changes in how the agent… · p. 6

Cost

  • The framework's cumulative spend stays close to the baseline across repositories, with the largest increase on OpenTitan and little visible difference on Ibex.Queried waveform values cost tokens of a short command, and reduced guessing offsets some query costs.Source: Cost · p. 6, Figure 5 · p. 6

V. CONCLUSION

The framework grounds repository-level repair in a reproduced failing run, uses waveform evidence to localize a candidate mechanism, and validates a coordinated patch by replay. On 388 HWE-Bench tasks, it resolves more tasks than the baseline for every tested model.

V. CONCLUSION

  • On 388 HWE-Bench tasks, the framework resolves more tasks than the baseline for every model tested.Remaining failures show that obtaining relevant evidence does not always produce complete repository-level fixes.Source: §V · p. 6
  • The framework grounds repairs in a reproduced failing run, narrows the violation to a candidate mechanism with an evidence trail, and replay-checks a coordinated patch.Source: §V · p. 6
  • Trajectories show agents extracting signal values, interpreting them in design semantics, and using reruns to expose incomplete repairs.The conclusion identifies a continuing gap between obtaining relevant evidence and producing complete repository-level fixes.Source: §V · p. 6

Related papers

Loading 2610.03219v1…