PaperlayerView exact revision on arXiv

Source-linked AI summary

SparseCraft: Agentic Hardware-Software Co-Optimization for Sparse Computing

Rajatabha Chakraborty, M P Samartha, Vedant Pahariya, Priyesh Shukla

arXiv:2610.05037v1cs.ARcs.AI

TL;DR

Sparse-accelerator design is often evaluated against analytical models, while sparse-workload efficiency requires changes across RTL, memory, and schedule. SparseCraft puts a language model in a measured-feedback CHIA loop, and its resident-X schedule cut off-chip traffic 9.8× and cycles 1.34× at zero area.

  • Problem

    Prior language-model hardware work has compiled code, set accelerator parameters, or optimized kernels, but generally generated once, used analytical-model scoring, or tuned one layer rather than iterating across measured RTL, memory, and software changes.

    Source: §I · p. 1

  • Method

    SparseCraft runs an LLM inside a closed CHIA loop that uses measured feedback to change RTL, memory, and schedule, with gate-based admission and a second LLM repairing failed changes.

    Source: §I · p. 1

  • Results

    9.8× less off-chip traffic and 1.34× fewer cycles followed the resident-X schedule at zero area in iteration 2.

    Source: A. Trajectory · p. 4

  • Takeaways & Limitations

    The trajectory shows software scheduling, RTL mechanisms, and memory sizing as distinct optimization levers with different measured effects in this workload.

    Source: A. Trajectory · p. 4

Abstract

from arXiv · show

Sparse-accelerator design spaces are usually searched against analytical models, so a design point is admitted on what a model predicts rather than on what the hardware does. SparseCraft closes that gap with a language model inside a closed CHIA loop. In each of 15 iterations the model reads the measured outcome of the previous one and edits the Chisel RTL, the memory configuration and the sparse-kernel schedule of a Gemmini accelerator through MCP tool servers, and no candidate counts until it has been checked for legality, elaborated, simulated cycle-accurately, checked bit-for-bit on every output against a golden reference, and synthesised. The harness turns each measurement into the next work order, a diagnosed bottleneck with matching strategy guidance, the history of tried designs and a score of the model's own prediction, and a second model repairs changes that fail a gate. On a $512 \times 512$ GraphChallenge sparse-DNN layer the loop reaches 2.1x fewer cycles, 9.8x less off-chip traffic and 22.8% less area than the block-sparse Gemmini baseline, with 5.61x higher modelled perf/W and 11.8x lower EDP. The levers span three layers: a schedule that keeps the dense operand resident removes 9.8x of the traffic, a zero-gated MAC and a zero-row skip unit that the model wrote in Chisel cut energy, and resizing the memories cuts area.

I. INTRODUCTION

SparseCraft puts an LLM inside a closed CHIA loop that changes sparse-accelerator RTL, memory, and schedule using measured feedback. Its 15-iteration run records both rejected proposals and regressions.

I. INTRODUCTION

  • Recovering sparse-workload waste requires RTL, memory, and schedule changes together: occupied blocks are only 6.25% dense, yielding 8,388,608 issued MACs for 524,288 useful ones.The block-sparse kernel skips empty blocks, but substantial work remains within occupied blocks.Source: §I · p. 1
  • SparseCraft is a CHIA-based loop for sparse matrix multiply on Gemmini, with the LLM reading measured outcomes and changing designs over iterations.No human proposes, edits, or selects designs; the harness, energy model, and benchmark are fixed and hashed.Source: §I · p. 1
  • The closed loop drives RTL, memory, and schedule changes from measured feedback, repairs failed changes with a second LLM, and admits designs only after the gate ladder.Admission follows legality checks, building, simulation, checking every output, and synthesis.Source: §I · p. 1
  • The 15-iteration run traces every number to a published record and includes its rejections and regressions.Source: §I · p. 1

II. THE CLOSED LOOP

Each run starts from the baseline and uses a fixed 15-iteration budget. The harness checks that inputs remain immutable, admits passing designs as parents, and rolls rejected ones back exactly.

II. THE CLOSED LOOP

  • The run starts by evaluating the unmodified baseline, then asserts that legality rules, Pareto admission, energy model, metrics parser, and kernel remain unchanged.Source: A. Structure · p. 1
  • A passing, admitted design becomes the next parent; a rejected design is rolled back exactly, including its RTL, and recorded with its failure reason.Source: A. Structure · p. 1
  • The model proposes and the harness evaluates each later iteration, but the fixed 15-iteration counter—not the model—ends the run.Source: A. Structure · p. 1

B. The proposer

The proposer operates through a tightly constrained tool interface, while composed prompts structure its design changes and prediction reporting. The figure traces proposal, evaluation, repair, and feedback through the loop.

B. The proposer

  • The proposer has only three MCP tools and can write only the design-point parameters, PE, and zero-bitmap unit.Its other tools return the last measured design and the history of tried designs and results.Source: B. The proposer · p. 1

C. Prompt structure

  • After measurement, the loop turns diagnosis, strategy modules, counters, tried designs, and prediction score into the next work order; failed gates send changes to a repairer for re-entry.Source: Figure 1 · p. 2
  • Composed prompts give the proposer a fixed reasoning order: inspect the parent state and measured diagnosis, use selected strategy modules, then make and verify one change before compiling.The model returns candidates, its implemented mutation, and predictions for time, energy, area, and clock, which the harness scores against measurement.Source: C. Prompt structure · pp. 1–2

D. Design state and legality

SparseCraft represents designs as typed hardware-and-schedule records and applies legality checks before building. Its run then proposes, evaluates, repairs when possible, admits or rolls back designs, and records results.

D. Design state and legality

  • A design records 28 hardware fields and 6 schedule fields; legality checks 27 rules, including generator invariants, hardware-kernel couplings, metadata and area budgets.Two further rules withhold the workload and dense-schedule flag; edits outside the writable set and previously evaluated designs are also rejected.Source: D. Design state and legality · p. 2
  • Each run seeds the baseline, proposes changes, evaluates them through the ladder, attempts bounded repairs for repairable failures, then admits a passing design or rolls back the tree.The run records each trial and scores the model's prediction against the measured outcome before diagnosing the next work order.Source: D. Design state and legality · p. 2
  • The model can add a zero-gated MAC that holds PE operands on zero and a zero-bitmap unit that suppresses scratchpad reads of all-zero sparse-operand rows.Source: D. Design state and legality · p. 2

E. Measurement

Measurement combines cycle-accurate simulation, output checking, synthesis, and counter-based energy modelling. It reports performance metrics using a fixed 2.0 ns clock.

E. Measurement

  • The elaborated Gemmini design is simulated cycle-accurately to measure cycles, off-chip bytes, and hardware counters; all 32,768 outputs are compared with a host-computed golden result.The host-side reference keeps a scheduling bug from appearing on both sides of the comparison.Source: E. Measurement · pp. 2–3
  • Synthesis uses yosys and OpenSTA on NanGate45, while area combines synthesised logic area with estimated SRAM area.Synthesis runs in parallel with simulation; every design uses a fixed 2.0 ns clock.Source: E. Measurement · p. 3
  • Modelled energy is calculated from measured counters, alongside performance, power, perf/W, perf/area, energy/op, and EDP metrics.The metrics are normalised to iteration 1.Source: E. Measurement · p. 3

F. Admission, diagnosis and repair

The harness admits only Pareto-nondominated designs and uses measured counters to diagnose bottlenecks. Failed gates can trigger bounded, evidence-based repair attempts.

F. Admission, diagnosis and repair

  • A design is admitted only if no previously admitted design matches or beats it in time, energy, and area.The rule implements Pareto non-dominance.Source: F. Admission, diagnosis and repair · p. 3
  • DMA-wait and array-activity counters label the bottleneck as memory, compute, idle, or balanced, guiding the next work order’s strategy modules.Infrastructure faults do not reach a model.Source: F. Admission, diagnosis and repair · p. 3
  • For a failed gate, the repairer receives shrunk evidence, the proposer’s mechanism, design states, and earlier attempts, then returns a fixed tree for full re-evaluation or a root cause.Repairs are capped at three attempts per iteration, and reverting repairs are detected.Source: F. Admission, diagnosis and repair · p. 3

III. EXPERIMENTAL SETUP

The experiment evaluates a block-sparse GraphChallenge DNN layer on stock Gemmini and tracks performance, energy, and area metrics. Figures describe metric evolution and the loop’s wall-clock profile.

III. EXPERIMENTAL SETUP

  • The workload is Y=AX, using a 512×512 matrix block with 8,192 nonzeros and 512 occupied 16×16 blocks, multiplied by dense INT8 X of shape 512×64.Nonzero values are seeded and never zero, keeping the equivalence check sensitive to indexing errors.Source: §III · p. 3
  • Iteration 1 uses stock Gemmini with a 16×16 weightstationary array, 256 KB scratchpad, 64 KB accumulator, and the block-sparse kernel.Reported gains are relative to this baseline.Source: §III · p. 3
  • Figure 3 shows metric evolution normalised to iteration 1, with absolute values for iterations 1 and 15; Figure 4 profiles the loop’s wall-clock time.Figure 3’s step line identifies the design held after each iteration.Source: Figure 3 · p. 3, Figure 4 · p. 4
  • The metrics use 1,048,576 useful operations, cycle count at 2.0 ns, modelled energy, and area to calculate performance, power, perf/W, perf/area, energy/op, and EDP.Metrics are normalised to iteration 1.Source: §III · p. 4

A. Trajectory

The trajectory combines schedule, RTL, and memory changes, with the resident-dense-operand schedule cutting traffic and later scratchpad resizing reducing cycles. The run also records rejected proposals and its active-time breakdown.

A. Trajectory

  • Keeping X resident cut off-chip traffic 9.8× and cycles 1.34× at zero area; later scratchpad halving cut cycles from 79,433 to 50,862 (36%).RTL mechanisms T-A and T-B cost 1.7% and 4.2% area for 6.6% and 17.5% less energy, respectively, without changing cycles.Source: A. Trajectory · p. 4
  • Five proposals were rejected; a 32×32 array produced wrong outputs in 32,674 of 32,768 cases because the kernel’s 16×16 blocking admits no other array size.Two further proposals duplicated that change, while iterations 13 and 14 failed legality; directional predictions held in 30 of 36 cases.Source: A. Trajectory · p. 4

B. Time breakdown

  • The run took 328 minutes of active time, averaging 21.9 minutes per iteration; RTL simulation accounted for 75% and 17 LLM sessions for 20%.Synthesis ran alongside simulation and was off the critical path.Source: B. Time breakdown · p. 4

A. Trajectory

  • Table I compares baseline iteration 1 with best design iteration 15, and states that neither produces a wrong output.The supplied table caption does not specify cell values.Source: Table I · p. 4

V. CONCLUSION

SparseCraft uses measured feedback in a CHIA loop to co-design a sparse Gemmini accelerator across RTL, memory, and software schedule. It reports gains over the block-sparse baseline, with every reported design built, simulated, checked on every output, and synthesised.

  • 2.1× fewer cycles, 9.8× less off-chip traffic, 22.8% less area, 5.61× higher modelled perf/W, and 11.8× lower EDP than the block-sparse baseline.Source: §V · p. 4
  • Across 15 iterations, measured feedback drove RTL, memory, and schedule co-design; legality, compilation, elaboration, cycle-accurate simulation, bit-exact output equivalence, and synthesis gated admission.A second model repaired changes that failed a gate.Source: §V · p. 4
  • The model kept the dense operand resident in software, wrote a zero-gated MAC and zero-row skip unit in Chisel, and resized the memories.Source: §V · p. 4
  • The model's directional predictions were right in 30 of 36 cases.Source: §V · p. 4
Loading 2610.05037v1…