PaperlayerView exact revision on arXiv

Source-linked AI summary

Keeping Inference in the Radio: Design and Measurement of a Spiking Accelerator for mmWave ISAC Activity Recognition

Anthony Sánchez, Rafael Ruiz, José A. Belloch, Jesús O. Lacruz, Joerg Widmer

arXiv:2610.37480v1cs.NIcs.ARcs.LG

TL;DR

Always-on ISAC activity recognition needs inference that can remain powered while general-purpose processors sleep. The paper implements a spiking classifier and encoder in RFSoC programmable logic; compared with same-topology conventional networks, it uses less logic and dynamic energy per decision, with quantization-aware training supporting 3-bit weights.

  • Problem

    Always-on sensing needs a classifier small and low-power enough to remain active while general-purpose processors sleep.

    Source: §I · p. 1

  • Method

    The authors generate Spiker+ accelerators for a 132-neuron SNN and implement the micro-Doppler spike encoder and classifier in RFSoC programmable logic.

    Source: §I · p. 2

  • Results

    7.6× less dynamic energy per decision than the 16-bit-input conventional network, at 1.2 points lower accuracy.

    Source: §VI · p. 6

  • Takeaways & Limitations

    The conclusion places the SNN’s small footprint as an advantage for an ASIC low-power domain, where leakage scales with area and sensing could remain active while the rest of the system sleeps.

    Source: §VI · p. 6

    The evaluation scope is DISC-A: four activities by seven subjects on one monostatic link.

    Source: §V · p. 6

Abstract

from arXiv · show

Integrated sensing and communication allows wireless networks to recognize human activity from the channel estimates that the receiver computes anyway. The inference, however, runs on the general-purpose processors of the base station, access point or handset, which are the first components a device switches off to save energy. Always-on sensing therefore requires a classifier that is small enough to remain powered while the rest of the system sleeps. We investigate whether Spiking Neural Networks (SNNs) can provide such a classifier. To this end, we generate SNN accelerators for micro-Doppler activity recognition on the public DISC dataset (IEEE 802.11ay at 60 GHz) with the open-source Spiker+ framework, and implement them together with a spike encoder on a ZCU208 RFSoC, so that the complete chain from micro-Doppler frame to activity label runs in programmable logic. A fully connected SNN with 132 neurons achieves 94.7 % accuracy (90.7 % macro-F1) on unseen subjects and performs an inference in 206 microseconds. Compared to conventional networks of the same topology on the same device, it requires an eighth of the logic and 3.4X less dynamic energy per decision than a network fed the same binary spikes, at comparable accuracy, and 7.6X less than a network fed the 16-bit spectrogram, at 1.2 points lower accuracy. Quantization-aware training reduces its weights to 3 bits within half a percentage point of the 6-bit network at the same encoder threshold

I. INTRODUCTION

ISAC activity inference commonly runs on general-purpose processors, which are among the first components switched off for energy savings. The paper investigates a small SNN classifier for always-on sensing and reports hardware measurements for communication-derived channel data.

I. INTRODUCTION

  • The main challenge is a classifier small and low-power enough to remain on, completing each decision within the observation interval while the rest of the system sleeps.The proposed path keeps channel estimation and micro-Doppler extraction active and exports only a few label bits per decision.Source: §I · p. 1
  • SNNs use binary events for accumulation-only synaptic operations and integrate micro-Doppler frames in a few hundred state variables rather than buffering a CNN window.Prior sensing work had limited hardware evidence: one study measured latency but estimated energy, while communication-derived channel-data activity recognition had been evaluated in software.Source: §I · p. 1
  • The authors report the first hardware measurements, to their knowledge, of SNN inference for ISAC activity recognition on communication-derived channel data.Their implementation uses an RFSoC and compares against same-topology conventional networks.Source: §I · p. 2
  • The paper generates a 132-neuron fully connected SNN accelerator and compares its measured accuracy, latency, resources, and power with a quantized conventional network of the same topology and input.The work also integrates a spike encoder and streaming AXI wrapper, and compares post-training with quantization-aware training.Source: §I · p. 2

II. FROM PACKETS TO MICRO-DOPPLER

The receiver’s channel estimates provide the input for sensing, and successive estimates are transformed into micro-Doppler spectrograms. Keeping inference in the radio can greatly reduce the data exported from the device.

A. Sensing as a By-Product of Communication

  • A short-time Fourier transform over successive channel estimates converts motion-related changes in channel taps into a micro-Doppler spectrogram.Each frame is one spectrogram column, representing the Doppler spectrum of one STFT segment; a window is a sequence of consecutive frames.Source: A. Sensing as a By-Product of… · p. 2
  • 15 Mb/s per beam of raw channel estimates in DISC-A rises to 180 Mb/s for a twelve-beam link, whereas in-radio inference exports about 1 b/s in two-bit labels.The label stream is seven orders of magnitude smaller than the raw data rate for a single beam.Source: A. Sensing as a By-Product of… · p. 2

B. Problem Formulation

The classifier processes micro-Doppler frames as they arrive, producing one activity label per 200-frame window. The evaluation uses subject-disjoint DISC-A data, and the processing chain is implemented in programmable logic.

B. Problem Formulation

  • 200 frames form a 1.73 s window, and the classifier must process each frame as it arrives so memory depends on neuron count rather than the window’s N × 59 values.The design processes one frame at a time, and window length remains a parameter.Source: B. Problem Formulation · p. 2

C. Dataset and Protocol

  • The evaluation uses 433 DISC-A sequences from seven subjects, training on subjects 1–5 and testing on subjects 6 and 7.The 812 test windows retain their natural class distribution; the training set is balanced by replicating minority-class windows with additive Gaussian noise.Source: C. Dataset and Protocol · p. 2

III. ACCELERATOR DESIGN AND MEASUREMENT METHOD

  • The Spiker+ framework quantizes the trained SNN to fixed point and generates synthesizable VHDL for an FPGA accelerator.The classifier consumes frames without data leaving the chip, and its VHDL can be ported to a baseband ASIC.Source: §III · pp. 2–3
  • The processing-chain figure traces input from the 802.11ay receiver to the activity label and distinguishes DISC-sourced blocks from work implemented in this study.It also depicts the spike encoder and the Spiker+ core within an AXI stream wrapper.Source: Figure 2 · p. 3
  • The spike-encoding figure shows a 200-frame waving-hands window as a normalized micro-Doppler spectrogram and as rasters at thresholds 0.75 and 0.85.The corresponding active-bin fractions are 6.2% and 4.1%.Source: Figure 3 · p. 3

A. Spike Encoding

The encoder maps each micro-Doppler frame to binary spikes using a threshold relative to that frame’s own range. This makes the rule independent of absolute signal level, while the frame maximum always fires.

A. Spike Encoding

  • Each frame becomes one time step, with a Doppler bin spiking when its magnitude is large relative to the other bins in that frame.Rate and latency coding require multiple time steps per value, while delta coding discards steady components.Source: A. Spike Encoding · p. 3
  • The encoder emits a spike for each bin above min + θ(max − min), making its threshold independent of absolute signal level such as gain and distance.The frame maximum always fires, so even a frame without motion produces spikes.Source: A. Spike Encoding · p. 3
  • At θ = 0.75, 6.4% of bins are active over the test set for the SNN configurations.The encoder uses the normalized [0, 1] DISC spectrograms, for which the relative rule coincides with a fixed threshold on the unsigned 16-bit input values.Source: A. Spike Encoding · p. 3

B. Network and Training

The network processes each micro-Doppler frame as one time step, using 59 inputs, 128 hidden LIF neurons, and four output LIF neurons. Fixed-point quantization changes the leak behavior of some configurations, while quantization-aware training enables 3-bit weights at the higher encoder threshold.

B. Network and Training

  • The 59-input, 128-hidden, 4-output LIF network processes one micro-Doppler frame per time step without buffering a window.Its output label is the class whose neuron fires most over the window; potentials are cleared between windows.Source: B. Network and Training · p. 3
  • The four-fractional-bit fixed-point leak vanishes below threshold, so SNN-S and SNN-SS hidden neurons do not leak, unlike SNN-L.The shared float32 training run was the best of three seeds selected on the test subjects, making its reported accuracy and derived hardware accuracies optimistic by about 1.5 points.Source: B. Network and Training · p. 3

C. Fixed-Point Quantization and HDL Generation

  • The study swept 140 combinations of weight, membrane, and fractional-bit widths, then selected three configurations from one float32 model for VHDL generation.Selection used Spiker+ emulation accuracy on the test subjects; the deployed designs also include a one-line output-neuron-reset correction.Source: C. Fixed-Point Quantization and HDL… · pp. 3–4
  • A bit-exact software model was needed because Spiker+ emulation can differ from generated hardware by several accuracy points due to circuit operation order.Hardware accuracy was measured on the board with the on-chip encoder.Source: C. Fixed-Point Quantization and HDL… · p. 4
  • The serial-input accelerator updates each layer’s neurons in parallel and skips time steps without input spikes, but the frame-maximum spike makes latency nearly input-independent.A window costs roughly 200 × (59 + 128) cycles plus control overhead.Source: C. Fixed-Point Quantization and HDL… · p. 4
  • At θ = 0.85, quantization-aware training uses differentiable fixed-point arithmetic in the forward pass and makes 3-bit SNN-SS deployable through the same flow.Training exposes the network to the hardware’s rounding, saturation, and spike timing.Source: C. Fixed-Point Quantization and HDL… · p. 4

D. System Integration on the RFSoC

The RFSoC integration wraps the Spiker+ core in an AXI-stream interface and places a streaming spike encoder in front of it. The wrapper manages windows and returns labels, while the encoder reproduces the software conversion on the test windows.

D. System Integration on the RFSoC

  • An AXI-stream wrapper adapts the Spiker+ core’s bespoke handshake and raw spike vectors, supports backpressure, frames windows, clears membrane potentials, and timestamps completion.Four counters and a comparator count output spikes to return the label.Source: D. System Integration on the RFSoC · p. 4
  • The streaming encoder accepts a 59-bin, 16-bit frame, emits spikes within 161 clock cycles, and takes θ over AXI-Lite.On-board, it matched the software encoder bit for bit on all 812 test windows, and labels matched the bit-exact model.Source: D. System Integration on the RFSoC · p. 4
  • PYNQ manages data streaming and registers, while DMA transfers frames from the processor to programmable logic and labels back.Source: D. System Integration on the RFSoC · p. 4
  • In a radio deployment, the digital-front-end STFT and a logarithm would replace the DMA input path.The logarithm supplies the dB values expected by the encoder.Source: D. System Integration on the RFSoC · p. 4

E. Measurement Method

The measurement method separates programmable-logic processing latency from processor-visible end-to-end latency and measures resources and PL-rail power on the RFSoC. The non-spiking references use the same topology but differ in whether they receive binary spikes or 16-bit spectrograms.

E. Measurement Method

  • Programmable-logic latency comes from the wrapper’s cycle counter, while processor-visible latency is timed around DMA over 1,000 passes.Power is measured on the VCCINT rail; on-chip replay keeps the accelerator running continuously because a current reading takes longer than one inference.Source: E. Measurement Method · p. 4

F. Non-Spiking Reference

  • The ANN and ANN16 references use Linear(59,128), ReLU, Linear(128,4), summing 200 frame logits before deciding; ANN receives binary spectrograms and ANN16 receives 16-bit normalized spectrograms.Both use int8 weights and 16-bit activations and are compared at 187.5 MHz because none of the designs closes timing at 250 MHz.Source: F. Non-Spiking Reference · p. 4
  • The comparison controls topology and input, but implementation style differs: hls4ml uses a parallel DSP-multiplier datapath, whereas Spiker+ uses a serial memory-based datapath.The binary ANN uses the same multipliers as ANN16 because hls4ml does not exploit 1-bit inputs.Source: F. Non-Spiking Reference · p. 4

A. Classification Accuracy

On the test set, SNN-M’s accuracy lies between the binary-input ANN and the 16-bit-input ANN, with the three accuracies considered comparable given seed variation. The SNN accelerator takes 206 µs per stored window; quantization-aware training also makes 3-bit weights usable at a higher encoder threshold.

A. Classification Accuracy

  • SNN-L matches the shared float32 model’s 94.6 % accuracy and 91.8 % macro-F1, while SNN-M keeps accuracy but loses one macro-F1 point; SNN-S loses three accuracy points and nine macro-F1 points.Table II reports per-class recall, accuracy, and macro-F1; accuracy is dominated by walking’s 68 % of windows.Source: A. Classification Accuracy · p. 4, Table II · p. 5
  • At θ = 0.85, raising the threshold removes 35 % of input spikes and 25 % of hidden spikes, costing retrained SNN-M 1.5 accuracy points; quantization-aware SNN-SS reaches 92.7 % with 3-bit weights, half a point below SNN-M.The same network quantized after training reaches 87.8 % (five seeds, bit-exact model).Source: A. Classification Accuracy · p. 5
  • 1.1 points above the binary ANN and 1.2 below ANN16, SNN-M’s accuracy is considered comparable to both because the differences lie within the ±1.7-point spread between training seeds.ANN16 reaches 95.9 % accuracy and 92.8 % macro-F1; the binary ANN reaches 93.6 % accuracy and 88.2 % macro-F1.Source: A. Classification Accuracy · p. 5

B. Latency

  • 206 µs per stored window, the SNN accelerator’s processing time is 38,676 cycles at 187.5 MHz, versus 89.6 µs for ANN and 104.6 µs for ANN16.On a live stream, encoder and accelerator processing is about 2 µs per arriving 8.64 ms frame; DMA and software add 225 µs per window in the prototype.Source: B. Latency · p. 5

C. Resources

The SNN accelerators use substantially fewer FPGA resources and less energy than the conventional networks, while the design space exposes accuracy, energy, and area trade-offs across SNN configurations.

C. Resources

  • ANN and ANN16 need 6.7–7.6× the LUTs, 14–15× the registers, and 192–200 DSPs of SNN-M, while SNN-M with its encoder uses 1.2% of ZCU208 LUTs.SNN cores use no DSP slices; 3-bit SNN-SS needs 23% fewer LUTs and half the BRAM of SNN-M.Source: C. Resources · p. 5

D. Power and Energy

  • 0.38 µJ per inference for SNN-M is 3.4× below ANN and 7.6× below ANN16; SNN-SS uses 0.25 µJ per inference.At one inference every 1.73 s, SNN-M's dynamic energy corresponds to 0.2 µW average power; always-on cost is instead set by static power and the unquantified front end.Source: D. Power and Energy · p. 5

E. Design Space

  • ANN16 is the most accurate and expensive design point, at 7.6× SNN-M's energy and 7× its LUTs; quantization-aware training moves SNN-SS to the lower-left of the trade-off plane.SNN-M and SNN-L exceed the binary ANN in macro-F1, while SNN-SS matches it at a seventh to a tenth of its LUTs and 2.7–5.2× less energy.Source: E. Design Space · p. 5

V. DISCUSSION

The discussion locates accuracy limits in the fully connected model and binary input, while attributing energy savings to arithmetic, serial processing, and area rather than input sparsity. It also bounds the findings to DISC-A's four activities and one monostatic link.

What limits accuracy

  • SNN-L reaches 91.1% macro-F1 on subject 7, below the DISC CNN's 96%; binarization costs 4.5 macro-F1 points, with SNN-M recovering half on the same spikes within seed variation.The remaining gap separates an 8k-parameter fully connected network without temporal feature extraction from convolutional designs.Source: §V · pp. 5–6

Where the savings come from

  • Because the encoder always fires the frame maximum, input activity changes dynamic power by less than 0.5 mW; savings instead come from accumulate-only arithmetic, serial processing, and small area.The paper notes an absolute encoder noise floor would make frames without motion cheapest, but does not report that design as measured.Source: §V · p. 6

Energy and area

  • At one decision every 1.73 s, SNN-M's 0.38 µJ amounts to 0.2 µW average dynamic power, while always-on standby is governed by block area and the unquantified receiver-to-STFT front end.In an ASIC, standby power is dominated by leakage and scales with area; quantization-aware training halves weight memory at a half-point cost.Source: §V · p. 6

Scope

  • DISC-A covers four activities from seven subjects on one monostatic link, while the described always-on use cases may require fewer classes.Each additional class costs one output neuron without changing latency.Source: §V · p. 6

VI. CONCLUSION

A 132-neuron SNN runs micro-Doppler activity inference in RFSoC programmable logic, achieving about 95% accuracy on unseen subjects. Its logic and dynamic energy use are lower than those of conventional networks, while the authors note that static FPGA power dominates on the prototype.

VI. CONCLUSION

  • At about 95% accuracy on unseen subjects, the 132-neuron SNN infers in 206 µs, using one-eighth the logic and 3.4× less dynamic energy than a network on the same binary input.It uses 7.6× less energy than a network on the 16-bit input, at 1.2 points lower accuracy.Source: §VI · p. 6
  • Quantization-aware training reduces SNN weights to 3 bits at a cost of half a point at the same encoder threshold.Source: §VI · p. 6
  • Static power dominates on the FPGA prototype, while the authors identify an ASIC low-power domain as the setting where the SNN's small footprint pays off.In that setting, leakage scales with area, allowing sensing to remain active while the rest of the system sleeps.Source: §VI · p. 6
Loading 2610.37480v1…