PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Low-Pass Dynamics: Frequency-Selective Spiking Reservoirs with Resonant Neurons

Aurora Micheli, Claudio Gallicchio, Nergis Tomen

arXiv:2610.37906v1cs.AI

TL;DR

Spiking reservoirs offer activity-driven inference, but their usual LIF neurons provide low-pass dynamics that limit frequency discrimination. HRF-Res uses heterogeneous resonant spiking neurons; across temporal benchmarks it reports accuracy gains over LIF-Res and lower inference energy than a non-spiking oscillatory baseline.

  • Problem

    Existing spiking reservoirs mostly use LIF neurons, whose low-pass dynamics limit their ability to distinguish information encoded at different temporal frequencies.

    Source: §1 · p. 2

  • Method

    HRF-Res replaces LIF neurons with a heterogeneous population of Harmonic Resonate-and-Fire neurons that selectively respond to input frequencies.

    Source: §1 · p. 2

  • Results

    Across diverse temporal benchmarks, HRF-Res gains accuracy over LIF-Res by up to +26.6% and uses up to 19× lower inference energy than a non-spiking oscillatory baseline.

    Source: §5 · p. 9

  • Takeaways & Limitations

    The results identify frequency-selective dynamics as a promising computational principle for spiking reservoirs, supporting richer temporal representations without sacrificing efficiency.

    Source: §5 · p. 9

    Energy estimates are model-based and have not yet been validated on neuromorphic hardware.

    Source: §5 · p. 9

Abstract

from arXiv · show

Spiking reservoir computing combines the training efficiency of fixed recurrent reservoirs with the energy efficiency of event-driven inference, where computation is triggered by sparse spikes rather than executed densely at every timestep. However, existing spiking reservoirs almost exclusively rely on Leaky Integrate-and-Fire (LIF) neurons, whose low-pass dynamics limit their ability to distinguish information encoded at different temporal frequencies. We introduce HRF-Res, a liquid state machine in which LIF neurons are replaced by a heterogeneous population of Harmonic Resonate-and-Fire (HRF) neurons. These neurons exhibit intrinsic oscillatory dynamics and selectively respond to input frequencies, enabling the reservoir to capture frequency-specific temporal patterns. The result is more discriminative representations and higher classification accuracy than LIF-based reservoirs, achieved without sacrificing the energy efficiency of spiking computation, with up to 19x lower energy consumption than an equivalent non-spiking oscillatory reservoir. Evaluated on ten benchmarks spanning time series, sequential images, audio spike trains, and neuromorphic vision, HRF-Res achieves state-of-the-art accuracy among spiking reservoir methods. These results demonstrate that introducing frequency-selective dynamics into spiking reservoirs improves representational quality and classification performance while preserving their inherent energy efficiency.

1 Introduction

Fixed reservoirs avoid recurrent training, while spiking networks make inference activity-driven; existing spiking reservoirs, however, mostly use LIF neurons that attenuate fast inputs and do not distinguish temporal frequencies. HRF-Res replaces LIF units with heterogeneous resonant neurons to add frequency-selective representations, and the paper reports improved results across benchmarks alongside lower energy than a non-spiking oscillatory baseline.

1 Introduction

  • Reservoir computing avoids backpropagation through time, while spiking networks make inference computation activity-driven; conventional reservoirs still compute densely at every timestep.The introduction frames their combination in liquid state machines as a way to retain training efficiency and activity-driven inference.Source: §1 · pp. 1–2
  • LIF neurons attenuate rapidly varying inputs, so a reservoir of LIF units encodes input amount but not its temporal frequencies.The paper identifies this as a representational limitation for signals with multiple temporal scales.Source: §1 · p. 2
  • HRF-Res replaces LIF units with heterogeneous Harmonic Resonate-and-Fire neurons, combining spiking computation with frequency-selective representations.The neurons exhibit damped oscillations and preferentially respond to inputs matching their intrinsic resonance frequencies.Source: §1 · p. 2
  • The study tests whether band-pass behavior persists in a recurrent spiking reservoir and improves class discrimination and accuracy at low energy cost.Its controlled comparison varies neuron dynamics while holding the other model aspects fixed.Source: §1 · p. 2
  • HRF neurons act as distributed band-pass filters, yielding more class-discriminative representations than an equivalent LIF reservoir before readout training.A controlled ablation reports that this representational advantage translates into improved classification accuracy across multiple datasets and modalities.Source: §1 · p. 2
  • Across ten benchmarks, HRF-Res achieves state-of-the-art performance among spiking reservoir methods and uses up to 19× less energy than non-spiking RON, with modest accuracy loss.The evaluation spans univariate time series, sequential images, audio spike trains, and neuromorphic vision.Source: §1 · p. 2

2 Related Work

Prior reservoir work has explored oscillatory dynamics, and heterogeneous damped oscillators have outperformed standard reservoir models, but those systems compute densely. Spiking reservoirs offer event-driven inference yet predominantly use LIF neurons, while prior uses of resonate-and-fire neurons have largely not integrated them into fixed-weight reservoir computing.

2 Related Work

  • Earlier liquid state machine work suggested resonant or oscillatory subthreshold dynamics could improve representation separability relative to standard integrator neurons.Later reservoir research examined oscillatory systems from theoretical and practical perspectives.Source: §2 · pp. 2–3
  • Heterogeneous damped-oscillator reservoirs significantly outperform standard reservoir models, but retain dense computation at every timestep.Oscillatory dynamics have also been revisited in gradient-trained networks, including coupled-oscillator RNNs and GNNs [27] [28] [29] and Kuramoto-oscillator units [30].Source: §2 · p. 3
  • Existing spiking reservoirs explore heterogeneous time constants, efficient scaling, and ensemble or receptive-field approaches, yet predominantly rely on LIF neurons without intrinsic frequency selectivity [14].Some hybrid models combine non-spiking feature extractors with trainable spiking components.Source: §2 · p. 3
  • Resonate-and-fire neurons have been studied for frequency-selective processing, including Fourier computation and classification, but their integration into fixed-weight reservoirs remains largely unexplored [24] [37] [38].Related applications include analog-to-spike conversion for downstream SNNs [36].Source: §2 · p. 3

3 Methods

HRF-Res combines LIF encoding with a heterogeneous, recurrent HRF reservoir whose damped oscillations select input frequencies; only a linear readout is trained. The methods compare this reservoir with matched LIF dynamics and characterize its frequency response, representations, and computational cost.

3.1 Neuron Models

  • HRF neurons use damped oscillations to respond selectively to input frequencies, while LIF neurons integrate inputs and attenuate rapid fluctuations.The HRF resonance band is determined by stiffness γ and damping ε.Source: §3.1 · p. 3

3.2 The HRF-Res Model

  • HRF-Res connects a LIF encoder to a heterogeneous HRF reservoir with fixed weights, training only a linear readout.Membrane-based and spike-based readouts support different deployment settings, with the paper reporting no significant accuracy loss from the choice.Source: §3.2 · pp. 4–5

3.3 Energy Cost Model

  • Spiking cost is measured in spike-triggered synaptic operations, while non-spiking cost is measured in dense multiply-accumulate operations.The energy model converts these operation counts using standard CMOS estimates.Source: §3.3 · p. 5

3.4 Reservoir Characterisation

  • The controlled LIF-Res baseline differs from HRF-Res only in reservoir-neuron dynamics, isolating oscillatory versus integrative behavior.Each model receives an independent dataset-specific hyperparameter search; characterization tests frequency tuning and untrained class alignment.Source: §3.4 · p. 5
  • 4–20× more HRF-Res neurons are resonant than LIF-Res neurons, and HRF neurons have sharper mean Q-factors across datasets.The largest reported mean Q-factor gaps are FordA (4.6 vs. 2.0), DVS Gesture (3.4 vs. 1.4), and SHD (2.5 vs. 0.6).Source: Figure 2 · p. 6

4 Results

HRF-Res shows competitive or state-of-the-art accuracy across temporal modalities, with frequency-tuned representations and improved classification over LIF-Res on most tested datasets. Against non-spiking RON, it reduces energy use substantially with modest accuracy losses.

4.1 HRF Reservoirs Function as Distributed Band-Pass Filters

  • HRF-Res neurons retain distributed frequency tuning in recurrent reservoirs, and mean CKA exceeds LIF-Res on all four datasets, with the largest CKA gap on FordA (+322%).LIF-Res neurons collapse to the lowest sweep frequency; the FordA HRF responses concentrate around 0.42 Hz.Source: §4.1 · p. 6, Figure 3 · p. 7

4.2 HRF Neurons Improve Classification Across Modalities

  • On sMNIST, HRF-Res reaches 93.64% versus 67.00% for LIF-Res; it outperforms LIF-Res on three of four datasets, while DVS Gesture is comparable.The comparison uses the same datasets and reservoir sizes with final-state readout; Table 1 reports HRF-Res matching or outperforming LIF-Res across all four datasets.Source: §4.2 · p. 7, Table 1 · p. 8

4.3 Readout Strategy Ablation

  • The best membrane-based readout exceeds the spike-rate readout on every dataset, while the combined RMS, standard-deviation, and final-state readout is best or near-best across datasets.The combined readout triples the parameter count; temporal averaging beats the final state by about 4% on FordA.Source: §4.3 · pp. 7–8, Figure 4 · p. 8

4.4 Energy Efficiency of the Spiking Formulation

  • HRF-Res reduces energy use by 9–19× versus RON across four benchmarks, with accuracy losses of only 1.8–4.9 percentage points.Including activity-independent state updates leaves a 9–17× reduction; the recurrent cost is gated by spiking activity.Source: §4.4 · p. 8, Table 2 · p. 9

4.5 Comparison to Spiking Reservoir Baselines

  • HRF-Res achieves competitive or state-of-the-art accuracy across modalities, including 89.78% on SHD with 6,000 neurons, surpassing a 16,000-neuron ELSM.With 800 neurons it also outperforms Dey et al. baselines on FordB, Wafer, and Earthquakes; results are averaged over five seeds.Source: §4.5 · p. 9

5 Conclusion

HRF-Res uses heterogeneous Harmonic Resonate-and-Fire neurons to produce frequency-selective, more class-discriminative representations, with accuracy gains over LIF-Res and lower inference energy than a non-spiking oscillatory baseline. Its gains vary with task and sampling conditions, and its energy estimates and evaluation scope have limitations.

5 Conclusion

  • Up to +26.6% accuracy over LIF-Res and up to 19× lower inference energy than a non-spiking oscillatory baseline accompany HRF-Res's frequency-selective reservoir representations.HRF neurons act as distributed band-pass filters and produce more class-discriminative representations before readout training.Source: §5 · p. 9
  • Energy estimates are model-based and not yet validated on neuromorphic hardware; the readout is trained offline, and theoretical characterization remains open.The cited open questions include memory capacity, separation properties, and stability; evaluation is based on classification benchmarks, with generalization to other settings unexplored.Source: §5 · pp. 9–10
  • HRF-Res achieves competitive or state-of-the-art accuracy across modalities, often with fewer reservoir neurons than prior spiking reservoir methods.The comparison reports HRF-Res results as mean ± standard deviation over five seeds.Source: Table 3 · p. 10
  • HRF-Res does not improve over LIF-Res on DVS Gesture, where class identity plausibly depends more on spatial trajectories than stable temporal frequency content.Under distribution shift, HRF-Res degrades more gracefully under additive noise but is more sensitive to changes in input sampling rate.Source: §5 · p. 10

A Experimental Details and Hyperparameter Search

The appendix states that it provides the implementation and evaluation information needed to reproduce the paper's results.

A Experimental Details and Hyperparameter Search

  • The self-contained appendix provides implementation details, data preprocessing, training and evaluation procedures, and hyperparameter configurations for reproducing the results.Source: A Experimental Details and… · p. 14

A.1 Architecture Details

The architecture uses fixed randomly initialized weights, heterogeneous HRF parameters, and sparse connectivity and input projections in specified settings.

A.1 Architecture Details

  • Input weights and biases are sampled from uniform distributions and scaled by ν; recurrent HRF-to-LIF weights are rescaled to spectral radius ρ, while LIF-to-HRF weights are not spectrally normalized.All matrices are fixed after initialization; the model uses a logistic-regression readout and starts each sequence from a zero-initialized reservoir state.Source: §A.1 · p. 14
  • Per-neuron stiffness γ_i and damping ε_i are sampled from centered intervals with widths Δγ and Δε, then clamped to strictly positive values.Source: §A.1 · p. 14
  • A neuron's natural resonance frequency is ω_i = √γ_i; the centers and widths of the stiffness and damping distributions are hyperparameters, as is timestep Δt.The timestep is treated as a hyperparameter for the discrete-time system rather than a fixed integration step.Source: §A.1 · p. 14
  • HRF position y is not reset after a spike, and a refractory threshold increment decays with τ_ref = 0.25 to prevent firing at every consecutive timestep above threshold.Source: §A.1 · p. 14
  • When LIF-to-HRF connectivity p_V is below 1, a Bernoulli mask sparsifies V; HRF-to-LIF connectivity remains fully dense in all experiments.Source: §A.1 · p. 14
  • High-dimensional inputs use a sparse random projection with density ρ_inp and weights rescaled by 1/√ρ_inp to preserve expected input-drive norm.Rejection sampling ensures at least one active connection per neuron and per input channel.Source: §A.1 · p. 14

A.2 Energy Estimation Details

Energy estimation counts spike-driven synaptic operations and neuron state updates, then compares their estimated costs with non-spiking operation costs.

A.2 Energy Estimation Details

  • Energy comparisons use accumulate operations for spiking networks and multiply-accumulate operations for non-spiking networks, with hardware energy constants specified for each operation.The cited constants are 0.9 pJ per accumulate operation and 4.6 pJ per multiply-accumulate operation.Source: §A.2 · p. 14
  • Synaptic operations per sequence are estimated from the mean HRF and LIF firing rates, reservoir size, sequence length, and active connections in each direction.The SOP formulation sums the spike-driven synaptic events for both population-to-population directions.Source: §A.2 · p. 14
  • Spiking synaptic cost scales with spike activity, while non-spiking models perform dense matrix multiplications at every timestep regardless of input sparsity.The dominant recurrent cost in spiking reservoirs therefore follows effective firing activity rather than full N^2 connectivity.Source: §A.2 · pp. 14–15
  • Neuron state updates are paid every timestep: HRF neurons require 3 MACs and LIF neurons 1 MAC, while state-update costs scale as O(N) and synaptic costs as O(N^2).The energy comparisons in the main paper are derived from SOP counts and the stated hardware constants.Source: §A.2 · p. 15
  • Including neuron state updates, the reduction in HRF-Res energy over RON remains 9–17× across four datasets; state updates add less than 10% to HRF-Res energy.For the Table 1 comparison, state updates increase HRF-Res energy by at most 0.12 × 10^-4 J on sMNIST and do not change its ranking against LIF-Res on any dataset.Source: §A.2 · p. 15

A.3 Data Preprocessing

The benchmarks use sequential presentations of images and event streams, alongside raw univariate time series, with dataset-specific lengths and downsampling.

A.3 Data Preprocessing

  • MNIST images become 784-step pixel sequences; psMNIST applies a fixed random pixel-order permutation with seed 0.The permutation is independent of the model seed.Source: §A.3 · p. 15
  • npCIFAR-10 images produce 32 steps of 96-dimensional input, followed by 968 shared fixed-noise steps for 1,000 total.The Gaussian noise is drawn once and shared across trials.Source: §A.3 · p. 15
  • FordA, FordB, Wafer, and Earthquakes use unmodified UCR sequences of 500, 500, 152, and 512 steps, respectively.The UCR archive is cited as [47].Source: §A.3 · p. 15
  • SHD events are binned into 250 sparse binary frames over a 1.4 s window; DVS Gesture uses 200 frames, with factor-4 spatial downsampling yielding 2,048 input channels per step.N-MNIST uses 30 frames; factor-2 spatial downsampling yields 578 input channels.Source: §A.3 · p. 15

A.4 Training and Evaluation Protocol

The protocol trains only a logistic-regression readout on fixed reservoirs and evaluates HRF-Res against a similarly configured LIF-Res baseline. Random search selects dataset-specific configurations, with selected HRF-Res settings summarized for ablation and baseline-comparison datasets.

A.4 Training and Evaluation Protocol

  • Only the linear logistic-regression readout is trained; SHD uses a multinomial objective and other datasets use one-vs-rest.The L-BFGS solver uses ℓ2 regularization, and the test set is not accessed during hyperparameter search.Source: §A.4 · p. 16
  • Results are means ± standard deviations over five trials, using matched seeds for HRF-Res and LIF-Res and refitting the readout each trial.UCR datasets use their provided training partitions; other datasets use standard splits, with a stratified 10% training validation split.Source: §A.4 · p. 16
  • Random search uses one validation-split seed per configuration, then re-evaluates the best configuration with five seeds on the full train/test split.Parameters spanning orders of magnitude are sampled log-uniformly; other parameters are sampled uniformly, with regularization C selected from a discrete set.Source: §A.4 · p. 16

A.5 LIF-Res Baseline

  • LIF-Res replaces the HRF reservoir with a second LIF population while retaining the encoder, connectivity structure, initialization, and soft-reset spiking.Its heterogeneous membrane time constants and thresholds mirror HRF-Res parameter heterogeneity; the encoder time constant is fixed at 20, and LIF-Res receives an independent dataset-specific search.Source: §A.5 · p. 16

A.6 HRF-Res: Hyperparameter Configurations

  • HRF-Res searches LIF-to-HRF connectivity and, for high-dimensional inputs, input density; HRF-to-LIF connectivity stays dense and encoder time constant stays fixed at 20.The searched HRF-Res ranges and selected settings are tabulated for ablation datasets in Table 6 and baseline-comparison datasets in Table 7.Source: §A.6 · p. 16, Table 6 · p. 19, Table 7 · p. 20
  • Larger-reservoir settings transfer oscillator parameters from smaller reservoirs, while re-searching size-sensitive parameters; FordB, Wafer, and Earthquakes share FordA's search space.The Combined readout concatenates RMS, standard deviation, and final state into a 3N-dimensional representation; it is used for larger-reservoir configurations and N-MNIST.Source: §A.6 · p. 16, Table 7 · p. 20

B.1 Frequency Sweep Parameters

The frequency sweep estimates each neuron's preferred frequency and resonance sharpness from its steady-state response. Sweep ranges account for slower dataset timescales, and a Hann-window check preserves the reported qualitative comparisons.

B.1 Frequency Sweep Parameters

  • Each sweep tests 30 log-spaced frequencies, discards the first half as transient, and assigns each neuron the frequency with maximum steady-state DFT response power.The same frequency range is used for both models per dataset, though each model has its own tuned timestep.Source: §B.1 · p. 21, Table 8 · p. 21
  • Larger sweep durations and lower minimum frequencies for SHD and DVS Gesture resolve slower resonances; shorter sweeps or higher minimum frequencies would underestimate Q-factors.The slower intrinsic timescales reflect larger selected timesteps for longer temporal dependencies in audio and event-camera streams.Source: §B.1 · p. 21
  • Q is preferred frequency divided by the interpolated half-power bandwidth, and is set to zero when either crossing is absent.This zero value denotes purely integrative low-pass behavior without a well-defined resonance peak; independently tuned timesteps preclude direct comparison of absolute frequency scales.Source: §B.1 · p. 21
  • With a Hann window, HRF-Res retains more resonant neurons, higher Q-factors, and broader preferred-frequency distributions than LIF-Res on all four datasets.Windowing shifts estimates slightly but leaves every qualitative conclusion unchanged.Source: Table 9 · p. 21, §B.1 · p. 21

B.2 CKA: Implementation Details

CKA compares normalized temporal-mean reservoir representations with label structure, using sampled test sets for practical computation. Sensitivity checks find that the HRF-Res advantage remains across sample sizes and transient washout choices.

B.2 CKA: Implementation Details

  • CKA uses full-sequence temporal means from zero-initialized states, row-normalizes them, then compares the centered feature and label kernels.The feature kernel is formed from normalized mean states, and the label kernel marks whether each pair shares a label.Source: §B.2 · p. 22
  • The temporal mean is a conservative trajectory summary because averaging partially cancels HRF oscillations; RMS or combined statistics may accentuate model differences.The comparison uses 500 random test examples except for DVS Gesture, which uses all 264; both models use their best searched configurations.Source: §B.2 · p. 22
  • The HRF-Res–LIF-Res CKA gap remains positive and comparable across sample sizes, including the smallest, while estimates approach full-test-set values as n increases.The full-test-set counts are 1,320 for FordA, 10,000 for sMNIST, 2,264 for SHD, and 264 for DVS Gesture.Source: §B.2 · p. 22, Table 10 · p. 22
  • Discarding the first 10%, 25%, or 50% of trajectories changes CKA only marginally, and HRF-Res scores remain higher in every setting.Classification with the final-state readout is unaffected by washout by construction.Source: §B.2 · p. 22

C Robustness to Temporal Perturbations

This section examines HRF-Res and LIF-Res under additive noise and sampling-rate shifts on sMNIST, and notes that LLMs were used only for writing and editing, not as part of the method or to affect scientific rigor or originality.

C Robustness to Temporal Perturbations

  • Tests kept trained reservoirs and readouts fixed and perturbed inputs only at test time; time warping was excluded because it lacks a single standard operationalisation.The study tested additive zero-mean Gaussian noise and sampling-rate shifts on FordA and sMNIST, using three independently initialised reservoirs per model; rate shifts were implemented by linear interpolation.Source: C Robustness to Temporal Perturbations · pp. 22–23
  • At σ = 0.5, HRF-Res loses 5.6 percentage points on FordA versus 14.4 for LIF-Res, and retains above-chance sMNIST accuracy where LIF-Res drops to chance.Noise results are test accuracies under additive zero-mean Gaussian input noise, averaged over three reservoirs.Source: C Robustness to Temporal Perturbations · p. 23, Table 12 · p. 23
  • On sMNIST, HRF-Res drops to near-chance accuracy for every tested sampling-rate shift while LIF-Res degrades less; on FordA, the models degrade similarly.Neither model is robust to sampling-rate mismatch; both would require retraining or adaptation in that setting.Source: C Robustness to Temporal Perturbations · p. 23, Table 13 · p. 23
  • The band-pass mechanism rejects energy outside preferred bands, but sampling-rate changes move informative signal content away from those bands.This mechanism accounts for HRF-Res’s contrasting responses to additive noise and sampling-rate shifts.Source: C Robustness to Temporal Perturbations · p. 23

NeurIPS Paper Checklist

  • LLMs were used only for writing and editing and did not affect the core methodology, scientific rigor, or originality.The checklist marks the question of LLM use as an important, original, or non-standard component of the core methods as not applicable.Source: NeurIPS Paper Checklist · p. 30
Loading 2610.37906v1…