PaperlayerView exact revision on arXiv

Source-linked AI summary

The World Model Hardware Accelerator

Shashank Chaurasia

arXiv:2609.16244v1cs.AR

TL;DR

Diffusion inference has static-shape denoising steps but no token-by-token recurrence, motivating a latency-first accelerator with a compile-time schedule. WMHA implements and verifies that design, meeting its semantic acceptance criterion by a factor of 23.

  • Problem

    Diffusion inference has static-shape full-sequence steps, but element-wise agreement with a reference can still accept a device that is semantically useless.

    Source: §I · p. 1

  • Method

    WMHA uses a static VLIW control path and workload-specific operators, removing training-oriented hardware while retaining diffusion sampler and activation operations.

    Source: §I · p. 1, B. What the workload removes and adds · p. 2

  • Results

    WMHA meets its frozen semantic acceptance criterion by a factor of 23.

    Source: §XI · p. 11

  • Takeaways & Limitations

    Restructuring the instruction schedule alone raised multi-engine concurrency from 1.83% to 44.66% of cycles and produced a measured 1.484× speedup.

    Source: §XI · p. 11

    The full chip is synthesized but not routed because of a host limitation rather than a design limitation.

    Source: A. Full-chip layout as a scoped project · p. 11

Abstract

from arXiv · show

Diffusion transformers invert the arithmetic that autoregressive decoding made familiar. There is no token-by-token recurrence: every denoising step is a full-sequence forward pass over static shapes, so the entire schedule is known at compile time and the only serial dimension is the step count itself. We exploit that structure in WMHA, a latency-first diffusion-transformer inference accelerator: a very-long-instruction-word sequencer issues four engines from one instruction word, a weight-stationary 16x16 dual-dot array streams FP8 and BF16 contractions, and a single-pass online-softmax attention pipeline keeps keys and values resident through a skewed software pipeline. The design is specified in a frozen micro-architecture document, implemented in synthesizable SystemVerilog, and verified against a double-precision reference model by a UVM environment whose acceptance criterion is semantic: the device must run a real denoising trajectory and reduce mean squared error against a clean latent by at least a factor of ten. It does so by a factor of 23, at both synthesized configurations, with zero element failures across 237 million checked values. Eleven application benchmarks built from published model shapes, including the original diffusion-transformer configuration, run on the device and report measured occupancy beside separately labelled projections. Five engines are taken to routed layout in sky130 with parasitic-annotated timing and measured-activity power; the full chip is synthesized, and the host limit that stopped its place-and-route is quantified together with the machine that would remove it.

I. INTRODUCTION

Diffusion inference repeats full forward passes over static shapes, making denoising steps serial but leaving each step’s work schedulable at compile time. WMHA uses that structure for latency-first execution and evaluates its accelerator with a semantic denoising criterion.

I. INTRODUCTION

  • Diffusion steps are serial full-sequence passes over fixed shapes, so the schedule is known before execution and no within-step autoregressive recurrence exists.This workload structure supports compile-time placement of memory movements and contractions rather than runtime dependence discovery.Source: §I · p. 1
  • One instruction word issues up to four engines with explicit wait-mask dependences, and a representative program keeps engines busy for 99.4% of wall time.Measured control cost is nine cycles per instruction word.Source: §I · p. 1
  • The frozen semantic gate requires a real sampling trajectory to reduce mean squared error against a clean latent by at least tenfold.It was set before RTL existed because element-wise tolerance checking can accept a device that is semantically useless.Source: §I · p. 1, Contributions · p. 1
  • WMHA is specified in a frozen micro-architecture document, implemented in synthesizable SystemVerilog, and verified against a double-precision reference using UVM.The accelerator is also taken through an open-source physical flow to routed layout with parasitic-annotated timing and measured-activity power.Source: Contributions · p. 1
  • 1.484× speedup followed schedule restructuring alone, as multi-engine concurrency rose from 1.83% to 44.66% of cycles with arithmetic unchanged.The arithmetic was proven unchanged.Source: Contributions · p. 1
  • Eleven application benchmarks built from published model shapes run real programs on the device with hardware counters read back.The original diffusion-transformer head dimension does not fit the fused attention pipeline at this on-chip capacity.Source: Contributions · pp. 1–2

A. Why diffusion inference is not decoding

  • Weights recur across T steps and batch B while the latent stays resident without a growing key/value cache; per-step latency sets time-to-image.The paper therefore treats per-step latency, rather than aggregate throughput, as the product metric.Source: A. Why diffusion inference is not… · p. 2

B. What the workload removes and adds

The workload removes training-oriented hardware needs while adding diffusion-specific operators and sampler updates. WMHA combines a single-pass online-softmax attention design with an instruction model that makes dependences explicit at compile time.

B. What the workload removes and adds

  • Diffusion inference removes the optimizer, backward dataflows, gradient storage, and on-chip random generation, while adding activation, modulation, guidance, sampler-update, and mixed-precision operations.The sampler updates use host-supplied scalars and cover DDPM and DDIM families and learned covariance.Source: B. What the workload removes and adds · p. 2

C. The attention decision

  • Single-pass online-softmax attention avoids the quadratic score-matrix intermediate, keeps keys and values resident across a query stripe, and shares a streaming pattern with feed-forward contractions.Its running-maximum increases require rescaling the accumulated output and denominator, a numerically delicate chain for saturated low-precision scores.Source: C. The attention decision · p. 2

A. Instruction model

  • A WMHA instruction word issues up to four engine slots with explicit wait masks, while issue-time descriptor capture and zero-overhead loop control support software pipelining.The instruction model includes array contractions and drains, vector operations spanning normalization, modulation, activations, guidance, sampler updates, and casting.Source: A. Instruction model · p. 2

B. Contraction array

WMHA’s contraction array uses a weight-stationary systolic organization, with two independently verified configurations. Its banked memory, crossbar port budgets, and descriptor walkers define how programs move and reuse data.

B. Contraction array

  • The 16×16 weight-stationary grid uses dual-dot cells that compute four FP8 E4M3 or two BF16 products with FP32 accumulation.A physical-design configuration halves the array to 8×8, and both configurations are independently verified.Source: B. Contraction array · pp. 2–3
  • The top level routes four engines from one VLIW sequencer, with softmax and epilogue on the array output path and traffic crossing to four banked regions.The per-region port budget is an architectural contract.Source: Figure 1 · p. 3
  • Array operations can span multiple passes when the stationary operand exceeds one array image, and a deposit interlock prevents drains from seeing partially written accumulators.Retirement runs from issue through the final output write.Source: B. Contraction array · p. 3

C. Memory system

  • Four on-chip memory regions cover activations, stationary weights, accumulator/output, and descriptors; each has a fixed port budget, typically two reads and one write per cycle.A program requesting more than the budget commits a program error that the hardware raises.Source: C. Memory system · p. 3
  • Hardware descriptor walkers step a base address by stride and wrap within a window, providing ring buffers and enabling double-buffering of stationary operands under a running contraction.Walker commitment follows the same capture-at-issue rule as the rest of the machine.Source: C. Memory system · p. 3

D. Attention pipeline

Attention uses a software-pipelined, single-pass online-softmax datapath, while adversarial review and pre-committed fallbacks shaped the specification before implementation.

D. Attention pipeline

  • Attention streams scores through online softmax into value accumulation while keys and values remain resident across a query stripe, avoiding a quadratic score matrix.A beat-valid interlock couples probability production to consumption; generation tags prevent stale probabilities from being read.Source: D. Attention pipeline · p. 4

A. Freezing before building

  • Five adversarial review rounds combined binding briefs, targeted rewrites, regression of prior findings, and checks for newly introduced mechanisms.Hands-on encoding, port-grant, latency, and descriptor-reachability checks produced more blocking findings than prose review.Source: A. Freezing before building · p. 4

B. Pre-committed fallbacks

  • The attention schedule’s two-stage skew failed derivation, so the pre-approved three-stage fallback was used without renegotiation.Pre-committing a fallback removes the incentive to rescue an attractive mechanism with optimistic arithmetic.Source: B. Pre-committed fallbacks · p. 4

C. Errata as the only post-freeze edit

After the specification freeze, changes required dated, justified errata; pinned latencies made timing contracts enforceable, while numerical tolerances were composed conservatively.

C. Errata as the only post-freeze edit

  • Twenty-two post-freeze errata were issued, mostly correcting underspecification; one narrowed an alignment requirement that differed between configurations.RTL–specification mismatches after the freeze required either an RTL correction or a dated, justified erratum.Source: C. Errata as the only post-freeze edit · p. 4

D. Pipeline structure and pinned latencies

  • At 16×16, an array operation takes 223 cycles from injection through drain, the epilogue adds 9, attention’s reciprocal takes 19, and widest-head drain normalization takes 150.Pinned latency constants are bound to RTL by assertions, which report their attempt counts.Source: D. Pipeline structure and pinned… · p. 4

V. NUMERICS AND THE TOLERANCE MODEL

  • WMHA uses FP8 E4M3 and BF16 with FP32 accumulation and checks results against a double-precision reference, so nearly all comparisons require tolerances.The paper treats the tolerance model as part of the design because almost no result is bit-exact.Source: §V · p. 4

A. Composing a band

  • The tolerance band conservatively composes reduction depth, accumulation length, piecewise-linear-unit envelopes, and storage quantization using an ℓ1 walk.Exact-class values are checked bit-for-bit, while values flushed to zero receive an explicit floor.Source: A. Composing a band · p. 5

B. Storage saturation and its ceiling

The tolerance model accounts for storage saturation and propagated error, and the verification environment checks its limits with a negative control and replayed programs.

B. Storage saturation and its ceiling

  • FP8 E4M3 saturates at ±448, and its tolerance band prices clamping only when the ideal value exceeds that representable range.Within range, the saturation-ceiling term is zero; outside it, a correctly clamped output is not reported as an error.Source: B. Storage saturation and its ceiling · p. 6

C. Propagation without double counting

  • Across programs, bands separate self-injected from propagated error to avoid double-counting; normalization propagation uses a Jacobian bound that can reach thousands for small-norm rows.Element-wise comparison uses the full band, while chain composition uses only the self-injected portion.Source: C. Propagation without double counting · p. 6

D. The negative control

  • The deliberately perturbed negative control misses its required rejection-margin floor of 100 by a factor of 2131.The paper treats a live, measured negative control as minimum evidence that the tolerance model is still doing work.Source: D. The negative control · p. 6

A. Environment

  • A UVM replay model executes the device’s instruction stream in double precision, records visible values with tolerance metadata, and compares memory at program boundaries.The 34-row suite contains 24 base rows and ten application rows; every base row runs at both synthesized configurations.Source: A. Environment · p. 6

B. The semantic acceptance gate

The frozen semantic gate required element-wise and trajectory checks plus actual denoising; WMHA passed at both configurations, with coverage reported under authored exclusions.

B. The semantic acceptance gate

  • The frozen gate required every swept element to stay within its composed band, trajectory error to meet its per-step budget, and final-latent MSE to be at most one tenth of starting-noise MSE.The denoising condition distinguishes a working generative accelerator from one that is merely arithmetically defensible.Source: B. The semantic acceptance gate · pp. 6–7

C. Results

  • The acceptance gate passed at both configurations, and the device denoised roughly 23× better than required.All 24 base rows passed at both configurations with zero errors and zero fatals; all ten application rows passed at 16×16.Source: Table III · p. 7, C. Results · p. 7
  • The acceptance row was rerun after every compile-path change and produced bit-identical numbers across seven independent runs.Functional and FSM coverage reached 100% at both configurations; code metrics were reported as measured with authored exclusions.Source: C. Results · p. 7, Figure 11 · p. 8, Table IV · p. 8

D. Coverage

The functional and FSM checks reach full coverage, while code coverage remains below 100%; control-path measurements and software pipelining show that instruction issue exposes little idle time and that scheduling changes can create substantial overlap.

D. Coverage

  • Functional and FSM coverage reach 100% at both configurations, while code coverage is measured with authored exclusions checked to ensure excluded objects are not covered.Exclusions require a reason tied to a parameter, specification clause, error code, or structural bound.Source: D. Coverage · p. 8
  • Code-coverage residue is concentrated in wide datapath buses and reciprocal seeds outside the reachable input domain, though directed tests could reach these objects.The authors decline to manufacture exclusions to raise the score.Source: D. Coverage · p. 8

A. What the control path costs

  • The nine-cycle instruction-word walk accounts for 5.25% of wall time but exposes only 0.006% as idle; full sequencer pipelining would recover 185 cycles from 3.21 million.The mean engine window is 152 cycles per word.Source: A. What the control path costs · p. 8

B. Software pipelining

  • At least 95% of the 1,047,239-cycle saving is genuine overlap, with array and vector busy cycles bit-identical before and after software pipelining.The work removed by restructuring is bounded at approximately 52.5k cycles.Source: B. Software pipelining · p. 8, Table V · p. 8
  • Attention was not pipelined because resident key and value groups occupy 12 of the 16 KB stationary region, leaving no second buffer.The serialization is capacity-forced.Source: B. Software pipelining · p. 8

C. Application benchmarks

The application benchmarks run real programs at scaled shapes and report measured occupancy separately from analytical projections. Across these workloads, weight streaming limits the full block, while isolated attention reaches 47% array occupancy without softmax-interlock stalls; the original DiT head dimension also exposes a residency boundary.

C. Application benchmarks

  • Load DMA holds about 70% occupancy while the array holds about 16% at testbench shapes, and a 128-to-1024-token sweep leaves both figures unchanged.The stripe-outer GEMM loop streams each output tile’s weight image into region W for every 16-row stripe because the region cannot hold one weight matrix.Source: C. Application benchmarks · p. 9, Table VI · p. 9
  • 47% array occupancy is reached by the isolated attention term under the skewed software pipeline, with the softmax interlock never stalling.The interlock’s consumer-wait counter reads zero.Source: C. Application benchmarks · p. 9, Table VI · p. 9
  • Pair-bias and cross-attention run without stalls; the protein workload repeats its single attention pass across 24 layers, 200 steps, and 5 samples.That workload entails 24,000 attention passes per query stripe per structure.Source: C. Application benchmarks · p. 9, Table VI · p. 9
  • The guidance-combine row is vector work with 30.2% vector-unit occupancy and no array involvement.On a host, the combine is a device round trip on every sampler step; WMHA performs it as one native operation.Source: C. Application benchmarks · p. 9, Table VI · p. 9
  • The original DiT head dimension of 72 is legal through ragged closure but costs 1.58× the cycles of 64 for 1.125× the useful work.The two K and two V groups fill region W at head dimension 64; dimension 72 would require 26.6 KB, so DiT-XL/2 and PixArt are excluded. The learned-covariance DDPM step is one vector operation using a host-formed σz tile, checked against the reference.Source: C. Application benchmarks · p. 9

VIII. PHYSICAL IMPLEMENTATION

WMHA is evaluated with routed sky130 measurements for engines that fit the host’s place-and-route ceiling, while the full chip is synthesized hierarchically. The physical results show that pre-layout timing can be substantially optimistic and that activity-based power estimates depend on the design’s clock fraction.

VIII. PHYSICAL IMPLEMENTATION

  • The physical flow reports sky130 measurements without advanced-node extrapolations.Standalone engines proceeded to routed layout with parasitic extraction; the full chip was synthesized hierarchically.Source: §VIII · p. 9, Figure 13 · p. 10

A. Host limits and what was routed

  • Five engines were routed; the four large engines and sequencer exceeded the host’s approximately 536k-instance place-and-route ceiling.On the 22-core, 30 GB workstation, 318 memories were lowered to flip-flops, and the crossbar alone became 530k flops.Source: A. Host limits and what was routed · p. 9, Figure 13 · p. 10

B. Pre-layout timing is not timing

  • The vector unit’s pre-layout path was 89.43 ns of a 90.95 ns path in one nor2b_1 driving 6.98 pF, so pre-layout frequency is not reported for the four affected blocks.The issue is that Yosys performs no high-fanout buffering.Source: B. Pre-layout timing is not timing · p. 9, Figure 13 · p. 10
  • 24.2 MHz pre-layout versus 76.7 MHz post-route gives a 3.2× timing difference on the epilogue.The figure compares post-route frequency per engine with the epilogue’s pre-layout figure; missing fanout buffering makes pre-layout timing optimistic on that block.Source: Figure 14 · p. 10, B. Pre-layout timing is not timing · p. 10

C. Measured versus assumed activity

  • With measured switching activity, the assumed 0.05 activity estimate was within −8% to +4% on three of four engines.The blocks are 70–80% internal power, with the clock group alone contributing 29–38%; the shortcut may not generalize to designs with a smaller clock fraction.Source: C. Measured versus assumed activity · p. 10

D. Where the area goes

The full chip’s synthesized area is dominated by storage and interconnect, while the reported physical results are limited to sky130 and five routed engines. The paper quantifies the host and design work needed to pursue full-chip layout.

D. Where the area goes

  • The full chip synthesizes to 5,771,328 cells and 68.355 mm2, with roughly 25 mm2 in the crossbar and banked tile memory and 20 mm2 in the contraction array.The vector and softmax units each take 8–9 mm2; the authors caution that standard-cell storage makes this an unrepresentative area estimate for an architecture using compiled memories.Source: D. Where the area goes · p. 10

E. Sequencer frequency is a flow artifact

  • The sequencer’s 18.1 MHz pre-layout figure is a flow artifact from a 255-level ripple-carry path, and it has no routable layout on the tested host.Routing failed to converge at 40% density and exhausted host memory at 18% density; the paper does not claim this frequency as a micro-architecture property.Source: E. Sequencer frequency is a flow… · p. 10

F. What we do not claim

  • The authors report sky130 measurements only: five engines are routed, while the full chip is synthesized but not routed because 5.77 M instances exceed the host’s place-and-route capacity.They report neither an advanced-node extrapolation nor a full-chip frequency or TOPS figure.Source: F. What we do not claim · p. 10

IX. RELATED WORK

  • The paper identifies its contribution as a quantified sky130-flow limitation: absent fanout buffering creates a 3.2× pre-layout-to-post-route frequency gap on one block.It recommends labeling pre-layout figures from this flow as a lower bound.Source: §IX · p. 11

A. Full-chip layout as a scoped project

  • The proposed macrohierarchical closure replaces 318 flip-flop-lowered memories with macros, hardens the array, and restructures the sequencer’s ripple chains; the estimated machine has 256 GB and 32–64 cores.The flat full chip as written is estimated to need roughly 768 GB and eight days, and is not recommended because it is expected not to converge. The estimates extrapolate from measured runs.Source: A. Full-chip layout as a scoped project · p. 11, Table IX · p. 11

B. Specification revisions the benchmarks motivate

The benchmarks motivate two concrete specification revisions, while verification reports measurable code-coverage residue. The conclusion highlights semantic acceptance, schedule-level speedup, and the bounded scope of physical results.

B. Specification revisions the benchmarks motivate

  • The benchmarks motivate enlarging the attention pipeline’s resident ring for head dimension 72 and changing weight residency or stripe height to let batching amortise weight streams.At head dimension 64 the resident ring exactly fits region W; at 72 it needs 26.6 KB. The block program is weight-bound at every token count and batch.Source: B. Specification revisions the… · p. 11

C. Verification residue

  • Code coverage is 95.6% and 94.1% with authored exclusions; the remaining reachable residue needs directed stimulus, and the application rows still need a full merged coverage rerun.Source: C. Verification residue · p. 11

XI. CONCLUSION

  • WMHA meets its frozen semantic acceptance criterion by a factor of 23 and reports 100% functional and FSM coverage at both configurations.Source: §XI · p. 11
  • Restructuring the instruction schedule alone raises multi-engine concurrency from 1.83% to 44.66% of cycles and yields a measured 1.484× speedup, with arithmetic proven unchanged.Source: §XI · p. 11
  • Five engines have sky130 post-route timing and measured-activity power, while the full chip is synthesized but not routed; closing that gap is identified as the next project.Source: §XI · p. 11

Related papers

Loading 2609.16244v1…