Source-linked AI summary
QKAN-LSTM: Quantum-inspired Kolmogorov-Arnold Long Short-term Memory
Yu-Chao Hsu, Jiun-Cheng Jiang, Chun-Hua Lin, Kuo-Chung Peng, Nan-Yow Chen, Samuel Yen-Chi Chen, En-Jui Kuo, Hsi-Sheng Goan
TL;DR
Conventional LSTMs have parameter redundancy and limited nonlinear expressivity. This paper integrates quantum-inspired DARUAN activations into LSTM gates and extends the approach to HQKAN-LSTM. The proposed models reduce trainable parameters while maintaining or improving predictive performance across multiple datasets.
Problem
Conventional LSTMs have high parameter redundancy and limited nonlinear expressivity in sequential modeling.
Source: §I · p. 1Method
Results
79% fewer trainable parameters than classical LSTMs, while maintaining or improving predictive performance across multiple datasets.
Source: §I · p. 2Takeaways & Limitations
The models provide a quantum-inspired approach to sequential modeling with enriched spectral representation that does not require multi-qubit entanglement.
Source: §VI · p. 8
Abstract
from arXiv · showhide
Long short-term memory (LSTM) models are a particular type of recurrent neural networks (RNNs) that are central to sequential modeling tasks in domains such as urban telecommunication forecasting, where temporal correlations and nonlinear dependencies dominate. However, conventional LSTMs suffer from high parameter redundancy and limited nonlinear expressivity. In this work, we propose the Quantum-inspired Kolmogorov-Arnold Long Short-Term Memory (QKAN-LSTM), which integrates Data Re-Uploading Activation (DARUAN) modules into the gating structure of LSTMs. Each DARUAN acts as a quantum variational activation function (QVAF), enhancing frequency adaptability and enabling an exponentially enriched spectral representation without multi-qubit entanglement. The resulting architecture preserves quantum-level expressivity while remaining fully executable on classical hardware. Empirical evaluations on three datasets, Damped Simple Harmonic Motion, Bessel Function, and Urban Telecommunication, demonstrate that QKAN-LSTM achieves superior predictive accuracy and generalization with a 79% reduction in trainable parameters compared to classical LSTMs. We extend the framework to the Jiang-Huang-Chen-Goan Network (JHCG Net), which generalizes KAN to encoder-decoder structures, and then further use QKAN to realize the latent KAN, thereby creating a Hybrid QKAN (HQKAN) for hierarchical representation learning. The proposed HQKAN-LSTM thus provides a scalable and interpretable pathway toward quantum-inspired sequential modeling in real-world data environments.
I. INTRODUCTION
Conventional LSTMs face parameter and expressivity challenges in complex temporal settings, while quantum implementations are constrained by hardware limitations. The paper introduces QKAN-LSTM and HQKAN-LSTM as quantum-inspired approaches to temporal and hierarchical representation learning.
I. INTRODUCTION
- Urban telecommunication forecasting supports real-time network resource allocation, traffic optimization, and anomaly detection.The data may exhibit irregular periodicity, bursty behavior, and strong spatiotemporal correlations.Source: §I · p. 1
- Conventional LSTMs face overparameterization and limited nonlinear expressivity, especially for complex oscillatory patterns and nonlinear feedback in communication signals.The introduction also identifies vanishing gradients and computational overhead as challenges for high-frequency, high-dimensional telecommunication data.Source: §I · p. 1
- NISQ hardware constraints, including limited qubit counts and two-qubit gate fidelities, restrict quantum machine learning's scalability for real-world telecommunication prediction.The paper presents quantum-inspired architectures as retaining quantum-model expressivity while running on classical hardware.Source: §I · p. 1
- The proposed framework combines QKAN-LSTMs for temporal modeling with HQKANs for hierarchical feature representation.HQKAN generalizes QKANs to hierarchical encoder–decoder architectures.Source: §I · p. 2
- QKAN-LSTM achieves a 79% reduction in trainable parameters versus classical LSTMs while maintaining or improving predictive performance across multiple datasets.The evaluations cover damped harmonic motion, Bessel function regression, and urban telecommunication forecasting.Source: §I · p. 2
- QKAN-LSTM replaces conventional LSTM-cell affine transformations with quantum-inspired DARUAN modules, while HQKAN-LSTM adds the JHCG Net mechanism.DARUAN uses data re-uploading to provide an exponentially rich Fourier representation without multi-qubit entanglement.Source: §I · p. 2
II. RELATED WORK
Related work covers quantum-enhanced LSTM approaches and KAN applications to time series forecasting. The paper then proposes QKAN-LSTM, replacing conventional LSTM gate transformations with quantum-inspired KART-based modules.
II. RELATED WORK
- Quantum-enhanced LSTM research includes variational-circuit, quantum-kernel, quantum-convolutional, and Quantum-Train variants, with applications across several domains.The cited application areas include telecommunication, weather prediction, cosmology, fraud detection, traffic, solar power, stress monitoring, and indoor localization.Source: §II · p. 2
- KAN studies generalized the Kolmogorov–Arnold representation theorem to arbitrary widths and depths and reported accuracy and interpretability advantages over MLPs.Subsequent studies have applied KANs to time-series forecasting.Source: §II · p. 2
A. Quantum-inspired Kolmogorov-Arnold Long Short-term Memory
- QKAN-LSTM replaces conventional affine transformations in LSTM gates with KART-based quantum-inspired functional modules.The gate modules aggregate one-dimensional quantum transformations to approximate high-dimensional nonlinear mappings.Source: A. Quantum-inspired Kolmogorov-Arnold… · pp. 2–3
- The proposed design is described as enriching recurrent dynamics and improving the modeling of long-range temporal dependencies.The passage attributes this to stronger nonlinear expressivity from structured aggregation of one-dimensional quantum transformations.Source: A. Quantum-inspired Kolmogorov-Arnold… · p. 3
1) Classical LSTM Equations:
The classical LSTM equations describe how forget, input, and output gates interact with the memory cell and hidden state over time.
1) Classical LSTM Equations:
- The conventional LSTM has forget, input, and output gates, together with a memory cell state.Its component evolution is expressed through the gate and state equations.Source: 1) Classical LSTM Equations · p. 3
- At time t, the input is an n-dimensional vector and the previous hidden state is an m-dimensional vector.The input and previous hidden state are the gate inputs represented in the equations.Source: 1) Classical LSTM Equations · p. 3
- Each gate has associated learnable weight matrices and bias vectors, and the equations use sigmoid and tanh activations.The cell-state update uses element-wise multiplication.Source: 1) Classical LSTM Equations · p. 3
2) Integration of Quantum-inspired Kolmogorov–Arnold Networks into LSTM:
QKAN-LSTM replaces the LSTM gates' affine transformations with QKAN mappings that aggregate trainable quantum variational activation functions. Its gate dynamics retain the classical LSTM structure while gaining an enriched functional representation.
2) Integration of Quantum-inspired Kolmogorov–Arnold Networks into LSTM:
- QKAN-LSTM replaces each gate's affine transformation with an aggregation of edge-wise quantum variational activation units.These units approximate high-dimensional nonlinear functions through learnable one-dimensional mappings without changing the classical neuron structure.Source: 2) Integration of Quantum-inspired… · p. 3
- Each edge unit is a trainable nonlinear activation realized by DARUAN, and its parameters are trained end-to-end.The gate input is formed using concatenation, and the activation is defined for each edge unit.Source: 2) Integration of Quantum-inspired… · p. 3
- Stacking r data re-uploading layers yields an exponentially enriched Fourier spectrum for compact nonlinear representations.The layers use scalar encoding parameters and trainable single-qubit variational blocks.Source: 2) Integration of Quantum-inspired… · p. 3
- The quantum state is initialized with a Hadamard gate to prepare a uniform superposition.The described DARUAN operation uses the specified parameter choices for the data re-uploading circuit.Source: 2) Integration of Quantum-inspired… · p. 3
- The QKAN mappings preserve the classical LSTM gating dynamics, while sigmoid and tanh remain for gating stability.Quantum activations enrich the inner functional space of each gate.Source: 2) Integration of Quantum-inspired… · p. 3
3) Training and Optimization:
QKAN-LSTM training optimizes classical and quantum gate parameters together, using an MSE objective and hybrid gradient computation. The implementation uses analytic functions with autograd, while quantum backends use parameter-shift gradients.
Loss Function
- The regression objective is mean squared error between predicted and target outputs at each time step.The model optimizes classical and quantum parameters that define each LSTM gate’s nonlinear mapping.Source: 3) Training and Optimization · pp. 3–4
Gradient Computation
- Classical gate parameters use backpropagation through time, while quantum parameters use the parameter-shift rule; the exact-solver implementation instead computes gradients with PyTorch autograd.On real quantum hardware or simulated quantum backends, the parameter-shift rule is used to estimate derivatives of quantum observables.Source: 3) Training and Optimization · p. 4
- Gradients are computed for the quantum parameters by shifting only θ_k while holding the other parameters fixed.The parameter-shift rule is used for quantum parameters; classical parameters use backpropagation through time.Source: 3) Training and Optimization · p. 4
Optimization Algorithm
- Adam or RMSprop jointly updates classical and quantum parameters until convergence.At each iteration, the optimizer updates parameters using the gradient of the loss and the learning rate.Source: 3) Training and Optimization · p. 4
A. Training Data
The evaluation uses Damped Simple Harmonic Motion, Bessel Function, and urban telecommunication time series. The benchmark datasets pair temporal or spatial inputs with corresponding displacement, amplitude, or SMS-in activity targets.
A. Training Data
- The evaluation covers Damped Simple Harmonic Motion, Bessel Function, and Urban Telecommunication time-series datasets.The proposed QKAN-LSTM and HQKAN-LSTM models are evaluated on all three.Source: A. Training Data · p. 5
Damped Simple Harmonic Motion
- Damped SHM samples pair time t with displacement x(t) from an oscillation whose amplitude decays exponentially.The dataset serves as a benchmark for learning and predicting harmonic oscillatory dynamics.Source: A. Training Data · p. 5
Bessel Function
- The Bessel dataset uses second-order J_2(x), pairing each x value with its corresponding amplitude.The Bessel Function dataset represents nonlinear oscillatory dynamics and is governed by Bessel’s differential equation.Source: A. Training Data · pp. 5–6
Urban Telecommunication
- Urban telecommunication data comes from Milan’s activity dataset and uses the more complete univariate SMS-in channel, normalized to [0, 1].Instances use fixed-length input sequences to predict the next value; tested lengths are 4, 8, 12, 16, 32, and 64, with a 70/15/15 train/validation/test split.Source: A. Training Data · p. 6
B. Experiment setup
The experiments compare LSTM, QLSTM, QKAN-LSTM, and HQKAN-LSTM across three datasets using dataset-specific training and model configurations.
B. Experiment setup
- All four models were evaluated on Damped SHM, Bessel Function, and Urban Telecommunication data using PennyLane, PyTorch, and QKAN implementations.The experiments used LSTM and QLSTM settings from ref. [8].Source: B. Experiment setup · p. 6
- Training rates are 10−2 for Damped SHM and Bessel Function and 10−3 for Urban Telecommunication, except HQKAN-LSTM uses 2 × 10−3.Urban Telecommunication training lasts 50 epochs and tests sequence lengths {4, 8, 12, 16, 32, 64}; experiments ran on Tesla V100 GPUs and Intel Xeon CPUs.Source: B. Experiment setup · pp. 6–7
- QKAN-LSTM and HQKAN-LSTM use hidden sizes of 2 for Damped SHM and 1 for Bessel Function and Urban Telecommunication, versus 5 or 4 for LSTM and QLSTM.All models use input and output dimensions of 1.Source: B. Experiment setup · p. 6
- QLSTM uses 6 qubits for Damped SHM and Bessel Function and 5 for Urban Telecommunication, while QKAN-LSTM and HQKAN-LSTM use a single-qubit DARUAN layer across datasets.QLSTM applies RY encoding, repeated CNOT plus trainable RY blocks, and Pauli-Z measurements.Source: B. Experiment setup · p. 6
C. Result Analysis
QKAN-LSTM and HQKAN-LSTM perform strongly across synthetic and urban time-series tasks, with parameter savings and task-specific differences in convergence and accuracy.
C. Result Analysis
- On Urban Telecommunication, QKAN-LSTM and HQKAN-LSTM use approximately 50–70% fewer quantum parameters than QLSTM and substantially fewer total parameters than LSTM.For Damped SHM, QKAN-LSTM uses two hidden units to provide sufficient expressive power, increasing its parameter count relative to its use on other datasets.Source: C. Result Analysis · p. 7
- Across Damped SHM and Bessel Function, all quantum-enhanced models show steady convergence and high predictive accuracy as training progresses.Source: C. Result Analysis · p. 7
- On Damped SHM, QKAN-LSTM reaches a final testing loss of 1.02 × 10−3 and R2 of 0.9771 after 30 epochs, surpassing LSTM baselines.HQKAN-LSTM achieves R2 of 0.9903 with reduced quantum parameters; QKAN-LSTM converges more slowly early in training.Source: C. Result Analysis · p. 7
- Bessel testing losses are 3.27×10−4 for QKAN-LSTM and 3.21×10−4 for HQKAN-LSTM, with corresponding R2 scores exceeding 0.986.Both models are described as more robust and stable than LSTM and QLSTM on this dataset.Source: C. Result Analysis · p. 7
- HQKAN-LSTM achieves the lowest MAE across Urban Telecommunication sequence lengths, while both QKAN-based models have lower MAE and MSE than LSTM and QLSTM.Their advantages persist as sequence length increases; they also use fewer quantum parameters than QLSTM and fewer classical parameters than LSTM on this dataset.Source: C. Result Analysis · p. 7
V. DISCUSSION
The discussion attributes QKAN-LSTM’s expressivity, interpretability, and efficiency to its DARUAN-based gate transformations and additive structure.
V. DISCUSSION
- Replacing LSTM gatewise affine maps with DARUAN-based QVAFs yields compact nonlinearities whose controllable Fourier spectrum can expand exponentially with re-uploading repetitions.The discussion links this spectral adaptability to mixed periodicity, burstiness, and multi-scale temporal patterns without increasing hidden width.Source: §V · p. 7
- Additive compositions of learnable univariate functions make gate-level effects of inputs and hidden states more transparent and support targeted diagnostics of temporal control.Edge-wise QVAF parameterization also encourages parameter reuse across time steps and alleviates redundancy in gate modeling.Source: §V · pp. 7–8
- Bounded single-qubit observable expectations constrain gate pre-activations, acting as an implicit regularizer that the discussion says can improve recurrent training stability.Avoiding multi-qubit entanglement reduces dependence on deep entangling circuits; increasing re-uploading depth provides a way to scale frequency content while preserving learned features.Source: §V · p. 8
- QKAN-LSTM and HQKAN-LSTM achieve competitive or superior forecasting performance while reducing trainable parameters relative to LSTM and quantum resources relative to QLSTM.DARUAN computations use two-level quantum states and GPU-friendly tensor operations; HQKAN-LSTM also compresses latent gate dimensionality and memory footprint relative to standard LSTM.Source: §V · p. 8
VI. CONCLUSION
The conclusion presents QKAN-based LSTMs as compact sequence models with enriched spectral representations and potential applications in resource-limited settings.
VI. CONCLUSION
- QKAN-LSTM and HQKAN-LSTM enhance temporal sequence modeling across synthetic and real-world datasets through frequency adaptability and exponentially enriched spectral representations without multi-qubit entanglement.Source: §VI · p. 8
- The models deliver higher predictive accuracy and faster convergence while requiring fewer trainable parameters and quantum resources.Source: §VI · p. 8
- The authors identify edge computing environments and resource-limited quantum devices as potential deployment settings for QKAN-based LSTM models.Source: §VI · p. 8