PaperlayerView exact revision on arXiv

Source-linked AI summary

Realizing Low-Precision Storage and Sparse Execution in a Folded FPGA Speech Enhancer

Clément Laroche

arXiv:2610.35144v1eess.AScs.ARcs.LG

TL;DR

Folded accelerators decouple precision, weight storage, and scheduled work, making a single bit-operation count an incomplete cost measure. This paper trains compression objectives for a folded streaming speech-enhancement engine and deploys the bit-exact implementation on a Kria KV260. Its best sparse model skips 47.3% of convolution MACs and achieves lower latency and energy than the most efficient dense build, with higher measured quality.

  • Problem

    In a folded accelerator, shared arithmetic means a single bit-operation count does not determine how compression changes hardware cost.

    Source: §I · p. 1, B. The shared data lane · p. 2

  • Method

    The authors adapt training objectives to the measured costs of a folded engine, pricing packed weight storage and scheduled cycles separately.

    Source: §I · p. 1, B. The shared data lane · p. 2, §IV · p. 3

  • Results

    47.3% of SKIP-PLAIN's convolution MACs belong to skipped exact-zero channels, yielding 1.03 ms and 33.4 µJ per frame at PESQ 2.78, with 42% lower latency and 48% less consumption than UNIFORM-6.

    Source: B. Main comparison · p. 3

  • Takeaways & Limitations

    Variable-width storage and exact-zero channels removed from the schedule are the two compression forms the folded engine directly exploits.

    Source: B. The shared data lane · p. 2

Abstract

from arXiv · show

Hardware-aware training often optimizes FPGA resources, latency, or bit operations, but folding changes what compression saves: shared arithmetic decouples precision, weight storage, and scheduled work. We derive training objectives from the measured costs of a folded streaming speech-enhancement accelerator. Mixed-precision weights reduce on-chip storage; structured channel sparsity removes entire operations from the engine schedule. We deploy a bit-exact implementation on the Kria KV260. The best sparse model eliminates 47.3% of convolution MACs, cutting frame time by 45% (1.86 to 1.03 ms) and dynamic energy by 60% (83 to 33 $\mu$J/frame) against a storage-priced model measured in the same session, with higher speech quality. Relative to the most efficient dense build measured separately, frame time falls by 42% and energy by 48%. Reducing weight width from 8 to 6 bits lowers dynamic power by 27% but changes frame time by less than 5%. Plain channel group-lasso removes more work than cycle-cost-weighted lasso. Thus, folded-accelerator training should price storage and removable work separately, rather than rely on a single bit-operation metric.

I. INTRODUCTION

On-device speech enhancement motivates compact inference, but a folded accelerator changes how compression maps to hardware cost. This work studies weight storage and scheduled work as distinct costs for training and deploying such an engine.

I. INTRODUCTION

  • A fixed-datapath int8 NPU uses the same 8 × 8 multiplier array for every operand, so sub-byte gains require hardware built for trained widths.Trainable quantization schemes assign precision at layer, channel, or weight granularity –.Source: §I · p. 1
  • Folding reuses one physical engine across network blocks, so reducing precision does not by itself determine the accelerator's cost.Bit-operation objectives capture designs where precision reduction lowers arithmetic work or instantiated logic, but the correspondence depends on the datapath,, [22]–.Source: §I · p. 1
  • The study adapts training objectives to weight storage and scheduled cycles, carrying learned choices through FPGA tools and checking bit-exact deployment.Every build exceeds the same-board 16-bit reference's quality while using 20–21 rather than 899 DSPs.Source: §I · p. 1

II. THE SPEECH-ENHANCEMENT NETWORK

Conv-FSENet is a causal, frame-streaming dilated temporal convolutional network that estimates a mask from STFT magnitude frames. Its accelerator returns logits, with the host applying the sigmoid.

II. THE SPEECH-ENHANCEMENT NETWORK

  • Conv-FSENet estimates a mask from STFT magnitude frames using a Conv-TasNet-style dilated temporal convolutional network.Its pointwise frontend feeds nine TCN blocks and a pointwise mask layer.Source: §II · p. 1
  • The nine blocks are arranged as three stacks of three, with pointwise expansion, dilated depthwise convolution, projection, and a residual addition.The depthwise kernel has size 3, with dilations 1, 2, and 4 within each stack.Source: §II · p. 1
  • The accelerator returns 257 logits, and the host applies σ.The network streams frame by frame and has approximately 0.66 M weights and 668 k MACs per frame.Source: §II · pp. 1–2, Figure 1 · p. 2

III. THE FOLDED ENGINE

The 16 ms frame period leaves time for hardware reuse, while a spatial implementation exceeds the FPGA's resource budget. The design therefore folds the network onto one engine that executes its blocks sequentially.

A. Trading latency margin for hardware reuse

  • Even at one-eighth production channel width, the spatial design required 356 k LUTs, about three times the device's 117 k.The 16 ms frame period provides substantial time for hardware reuse.Source: A. Trading latency margin for hardware… · p. 2
  • One physical TCN engine executes the nine same-structure blocks sequentially instead of instantiating nine copies.The blocks differ in weights and depthwise-convolution dilation.Source: A. Trading latency margin for hardware… · p. 2
  • In the folded design, each iteration fetches the block's weights, depthwise history, and operation output formats as data.This replaces the area-heavy spatial mapping with a shared datapath, stored weights, and cycles for the layer schedule.Source: A. Trading latency margin for hardware… · p. 2

B. The shared data lane

The shared data lane preserves operation-specific learned precision while reusing a common datapath. It separates variable-width weight storage from exact-zero channels that can be removed from the execution schedule.

B. The shared data lane

  • Activations use a common fixed-point carrier and a shared accumulator, with one ROM-controlled requantizer applying each operator's learned output format.This avoids separate casts for every operator and activation element.Source: B. The shared data lane · p. 2
  • The weight lane stores each output channel's consecutive weights using a shared code width bf and shift sf.The decoder reconstructs weights from their signed codes.Source: B. The shared data lane · p. 2
  • A channel with bf = 0 contains only zero weights, so removing it from the schedule also removes its MAC cycles.The MAC receives the same fixed-point weight as with a uniform-width representation; only storage format changes.Source: B. The shared data lane · p. 2
  • The shared multiplier must support the widest operand, so narrower weights shrink packed storage but not multiplier width.The folded engine directly exploits variable-width weight storage and exact-zero channels removed from the schedule.Source: B. The shared data lane · p. 2

IV. TRAINING PRICED FOR THE ENGINE

The training objectives price the folded engine’s two directly exploitable costs: packed weight storage and scheduled cycles. Channel sparsity removes scheduled work, while storage constraints target the emitted weight tape.

IV. TRAINING PRICED FOR THE ENGINE

  • The engine-aware objectives add terms for weight storage and scheduled cycles to the baseline compression objective.All models minimize compressed spectral MSE plus two HGQ regularizers.Source: §IV · p. 2
  • Cycle-weighted group-lasso encourages whole output channels to become zero so their MAC loops can be skipped.Each channel’s pruning pressure is weighted by its MAC cost; setting c_l=1 recovers plain group-lasso.Source: §IV · pp. 2–3
  • At epoch 50, training freezes the zero-channel mask and removes fold-cost terms, while task loss, monotone-L1, and β·EBOPs continue adapting surviving weights and precisions.A projection keeps masked channels at zero, fixing the hardware schedule.Source: §IV · p. 3
  • The storage recipe learns one weight width per tensor and can constrain the packed weight tape to a target capacity.The forward pass uses exact packed size including per-channel headers, while a differentiable estimate from learned widths supplies the gradient.Source: §IV · p. 3

A. Experimental Setup

Experiments use VoiceBank-DEMAND and compare builds that share Conv-FSENet but differ in training objective and weight granularity. They evaluate board execution, output parity, speech quality, frame time, and resource use.

A. Experimental Setup

  • The study uses 16 kHz VoiceBank-DEMAND, with four of 28 training speakers held out for validation and an 824-utterance test set reserved for final scoring.The held-out validation speakers are p279, p282, p286, and p287; test speakers are p232 and p257.Source: A. Experimental Setup · p. 3
  • All builds share Conv-FSENet and the AMD Kria KV260, using the same Vitis HLS/Vivado 2024.1 flow at 100 MHz.The compared builds vary in training objective and weight granularity.Source: A. Experimental Setup · p. 3
  • Board replay produced identical raw output words across deployments, and repeated runs over all 824 test utterances were also identical.An integer NumPy reference matched all 75 layer outputs in C simulation, including cold start; host-side feature quantization was checked separately.Source: A. Experimental Setup · p. 3
  • Dynamic power is measured as active-minus-idle board power, and energy per frame divides that difference by achieved frame rate.Quality is wideband PESQ over all 824 test utterances; frame time is board wall-clock per frame at flat-out rate, and resource counts are post-route.Source: A. Experimental Setup · p. 3

B. Main comparison

On the shared KV260 engine, SKIP-PLAIN is the best measured build, while reducing weight width mainly saves power rather than frame time. More MAC lanes reduce energy per frame but increase dynamic power, and DEN16 remains faster on its different network.

B. Main comparison

  • Sparse builds use 79 BRAM tiles, compared with STORE’s 113 tiles and 20% fewer tiles than UNIFORM-8; STORE meets the 3.0 Mbit tape target but skips no MACs.Every build fits within the device’s 144 BRAM tiles.Source: B. Main comparison · p. 3
  • From eight to six bits in the pinned-width sweep, dynamic power falls by 27% and energy per frame by 30%, while frame time changes by under 5%.The comparison is the pinned-width sweep; the passage does not identify a specific pair of builds.Source: B. Main comparison · p. 3
  • Our builds score 0.23–0.37 PESQ higher than DEN16, with 4.2–8.3× fewer LUTs and 43× fewer DSPs.The DSP difference is largely because nine TCN blocks share four MAC lanes.Source: B. Main comparison · p. 3
  • 47.3% of SKIP-PLAIN’s convolution MACs are skipped, yielding 1.03 ms and 33.4 µJ per frame at PESQ 2.78—42% lower latency and 48% lower consumption than dense UNIFORM-6.The eight builds all route and meet timing on the same engine; SKIP-PLAIN also beats cycle-cost-weighted SKIP on latency, energy, and PESQ.Source: B. Main comparison · p. 3, Table I · p. 4
  • As SKIP-PLAIN’s MAC lanes increase from K=1 to K=4 to K=8, dynamic power rises from 26.1 to 32.4 to 43.8 mW, while energy per frame falls by 2.8× and 1.12× respectively.Fixed per-frame work and shared-window decoding limit lane efficiency; at K=16, decoder logic prevents a new convolution iteration every cycle.Source: Table II · p. 4, B. Main comparison · p. 4
  • DEN16 is faster, with RTF 0.010 against our 0.064 and a 9.7 ms first sample against our 32 ms window.DEN16 is an independently reported implementation of a different speech-enhancement network on the same KV260 FPGA.Source: B. Main comparison · p. 4

C. Effects of compression

Channel skipping removes substantial scheduled work, but frame-time gains are smaller because other per-frame operations remain. The training objectives differ in how much whole-channel computation they make skippable.

C. Effects of compression

  • With MAC skipping disabled, the four builds take nearly the same cycles at K=4, so narrower weights do not meaningfully shorten the engine’s full schedule.The comparison isolates row skipping from the effect of weight width.Source: C. Effects of compression · p. 4
  • SKIP-PLAIN skips 47.3% of MACs and reduces frame time by 44.6%, while SKIP skips 44.1% and reduces frame time by 41.3%.The gains are smaller than the skipped-work shares because frontend, mask, normalization, and residual operations still run, totaling roughly 24k cycles per frame.Source: C. Effects of compression · p. 4
  • Channel group-lasso zeros entire output channels, removing TCN blocks 4 and 5 and reducing work in later blocks; SKIP-PLAIN leaves only 18.8% of block 8’s scheduled MACs.By contrast, STORE produces no skippable channels, while stock HGQ produces whole zero channels only in depthwise layers, accounting for 6.1% of MACs.Source: C. Effects of compression · p. 4

VI. CONCLUSION

SKIP-PLAIN reduces frame time and energy versus UNIFORM-6 at the same four-lane configuration, with higher measured PESQ. Training-induced channel sparsity zeros output-channel rows so the engine can skip their MAC loops; the authors note Lasso-related training instability as a future-work concern.

VI. CONCLUSION

  • At the same four-lane configuration, SKIP-PLAIN reaches 1.03 ms and 33.4 µJ versus UNIFORM-6’s 1.77 ms and 64.5 µJ, with PESQ 2.78 versus 2.64.Source: §VI · p. 4
  • Channel sparsity pressure zeros entire output-channel weight rows, allowing the engine to skip their MAC loops and reduce scheduled work per frame.Source: §VI · p. 4
  • Future work could explore alternative structured sparsity priors to reduce training instability observed with Lasso regularization.Source: §VI · p. 4

Related papers

Loading 2610.35144v1…