PaperlayerView exact revision on arXiv

Source-linked AI summary

Complementary Matrix-Gated QKAN Fast-Weight Programmers for Quantum Dynamics Forecasting

Kuo-Chung Peng, Samuel Yen-Chi Chen, Jiun-Cheng Jiang, Chen-Yu Liu, En-Jui Kuo, Yun-Yuan Wang, Tzung-Chi Huang, Prayag Tiwari, Chi-Sheng Chen, Chun-Hua Lin, Yu-Chao Hsu, Tai-Yue Li, Saif Al-Kuwari, Simon See, Kuan-Cheng Chen, Nan-Yow Chen, Hsi-Sheng Goan

arXiv:2607.27945v1quant-phcs.AIcs.LG

TL;DR

Scalar gates force every fast-state coordinate to share one memory timescale, limiting coordinate-wise control in QKAN-based fast-weight programmers. The paper introduces complementary matrix gating, which most consistently improves QKAN-based models and achieves at least 91.2% improvement over scalar gating in direct multi-step forecasting.

  • Problem

    Scalar gating gives every fast-state coordinate the same retention-write balance, motivating coordinate-wise memory control without losing boundedness or affine prefix-scan structure.

  • Method

    The paper adds self-modulating QKAN-FWPs and CMG, using one sigmoid matrix gate for old-state retention and its complement for writing the new proposal.

  • Results

    CMG is the most consistent update rule across seven single-step benchmarks and five sequence lengths, improving scalar gating by at least 91.2% in direct multi-step forecasting.

  • Takeaways & Limitations

    Coordinate-wise complementary modulation is a stable and effective update for QKAN-based FWPs within the evaluated forecasting settings.

Abstract

from arXiv · show

Sequence models must decide what to write into memory and what to retain. In quantum and quantum-inspired sequence learning, nonlinear recurrent updates often require repeated circuit evaluations and sequential backpropagation through time, making long contexts costly. Gated fast-weight programmers (FWPs) based on quantum-inspired Kolmogorov-Arnold networks (QKANs) alleviate this bottleneck by storing context in time-varying fast parameters. However, their scalar gate applies one retention-write balance to every fast-state coordinate, forcing all parameters to share a memory timescale. We introduce Self-Modulating QKAN-based FWPs, which replace this broadcast gate with low-rank-generated element-wise modulation of the new-proposal branch, a bounded old-state branch, or both. We further propose Complementary Matrix Gating (CMG), which uses one sigmoid matrix gate to retain the old state and its complement to write the new proposal. CMG provides coordinate-wise memory control while preserving the bounded convex update and affine prefix-scan structure of scalar gating, at the modulation-head cost of a single-branch rule. We compare four self-modulating rules with scalar gating across four FWP architectures combining classical and QKAN-based slow and fast programmers. Across seven single-step forecasting benchmarks and five sequence lengths, CMG gives the most consistent improvements for architectures whose fast programmer incorporates a QKAN-based module. In direct multi-step forecasting of Jaynes-Cummings and transmon-resonator dynamics simulated with CUDA-Q Dynamics, CMG models maintain mean-squared errors on the order of 0.001 or lower across forecasting horizons of 4, 8, and 16 steps, while improving on their scalar-gated counterparts by at least 91.2%. These results establish coordinate-wise complementary modulation as a stable and effective update for QKAN-based FWPs.

I. INTRODUCTION · II. RELATED WORK · III. METHODS

The paper introduces coordinate-wise self-modulation for QKAN fast-weight programmers, including Complementary Matrix Gating, to improve memory control while preserving boundedness and affine prefix-scan compatibility. It situates this approach among QML sequence models, fast-weight memory, QKAN architectures, and established element-wise recurrent gating.

  • I. INTRODUCTION: QML recurrent sequence models face costly repeated circuit evaluations and sequential backpropagation through time over long contexts.Fast-weight programming instead accumulates context in adaptive fast parameters.
  • I. INTRODUCTION: The paper asks whether QKAN-FWPs can gain coordinate-wise memory control without losing boundedness or affine parallel-prefix efficiency.This frames the central design constraint for the proposed updates.
  • III. METHODS: Self-modulating QKAN-FWP updates use low-rank-generated element-wise modulators on the new branch, old branch, or both.These variants are named Only-new, Only-old, and Full; old-state modulation is bounded with tanh.
  • III. METHODS: Complementary Matrix Gating retains the old state with one sigmoid matrix gate and writes the proposal with its complement.CMG provides coordinate-wise control while preserving bounded convex dynamics and prefix-scan compatibility.
  • I. INTRODUCTION: Coordinate-wise memory control yields more reliable gains than scalar gating, with CMG showing the most consistent improvements across forecasting benchmarks.The comparison covers write-side, memory-side, and full self-modulation in single-step and direct multi-step forecasting.
  • II. RELATED WORK: QML sequence models combine quantum reservoirs, recurrent quantum circuits, or variational quantum circuits with classical temporal architectures for forecasting and reinforcement learning.These approaches can be expressive, but long sequences still require repeated temporal circuit evaluation.
  • II. RELATED WORK: KANs learn univariate edge functions, motivating KAN forecasters, temporal KAN cells, mixture-of-expert variants, and decomposition or frequency modules for time series.QKAN replaces spline-style edge functions with DARUAN functions for quantum-inspired nonlinear modeling.
  • II. RELATED WORK: Recent QKAN sequence models place QKAN blocks in LSTM-style gates, fast-weight programmers, and transformers, while LSTMs and GRUs establish element-wise gating for memory and candidate states.This prior work motivates coordinate-wise gating in QKAN-FWPs.

A. QKAN and HQKAN programmer backbones · B. Gated fast-weight update · C. Low-rank self-modulating updates

The paper uses classical or QKAN-based slow and fast programmers, with temporal information stored in time-varying fast states. It replaces scalar fast-state gating with low-rank, element-wise modulation, including complementary matrix gating and bounded recurrent updates.

  • A. QKAN and HQKAN programmer backbones: Fast-weight programming separates slow and fast programmers, storing temporal information in the trajectory of the fast states rather than a recurrent hidden state.The slow programmer reads the scalar input and updates the time-dependent fast state before prediction.
  • A. QKAN and HQKAN programmer backbones: Programmers are either classical linear/MLP modules or QKAN-based modules using DARUAN functions and parameterized single-qubit data-reuploading circuits.HQKAN combines a classical encoder, a QKAN nonlinear-transformation block, and a classical component.
  • B. Gated fast-weight update: The gated fast-weight update begins with a slow-programmer proposal for the new fast-state change.For a generic fast state Θt, the slow programmer first proposes ∆t.
  • B. Gated fast-weight update: CMG uses one low-rank-generated matrix gate, assigning complementary weights to the retained old state and proposed new state.This preserves the complementary retain/write structure while allowing coordinate-wise control.
  • C. Low-rank self-modulating updates: Self-modulation replaces scalar gating with element-wise modulation of a reshaped fast state Θt ∈ R^P × Q using rank-one matrices from affine-head outer products.Separate modulation branches are defined for the new and old state components.
  • C. Low-rank self-modulating updates: The new branch controls write amplitude, whereas the old branch controls coordinate-wise retention, attenuation, amplification, or sign reversal.These branches can be used separately or together in the raw update family.
  • C. Low-rank self-modulating updates: Bounding old-state modulation prevents recurrent products of old-state multipliers from amplifying stored updates.The bounded Full and Only-old rules apply this constraint to the recurrent update kernel.

D. Complementary Matrix Gating

Complementary Matrix Gating (CMG) replaces scalar gating with a single element-wise matrix gate whose old-state and new-proposal branches remain complementary. It preserves coordinate-wise bounded convex updates and affine prefix-scan evaluation.

  • Gate construction: CMG replaces the scalar gate with an element-wise matrix gate generated by the slow programmer.It uses the same low-rank factorization while generating one gate matrix.
  • Gate construction: The old-state and new-proposal branches remain complementary, using one modulation matrix with the same modulation-head size as Only-new and Only-old.CMG differs from Full self-modulation by using a single modulation matrix.
  • Bounded update: CMG inherits scalar gating’s bounded convex update coordinate-wise under the same bounded-proposal condition, without requiring a global scalar gate.Each coordinate remains within the same bound because the update weights, including the initial-state coefficient, form a convex combination.
  • Efficient evaluation: CMG retains the affine prefix-scan structure, with At = Gt and Bt = (1−Gt)⊙∆t, so the fast-state trajectory is obtained through associatively composable pairs.All considered update rules remain evaluable by parallel prefix scan after the slow-programmer heads are computed.

IV. EXPERIMENTAL PROTOCOL · A. Tasks and preprocessing

The experiments evaluate seven univariate forecasting benchmarks using normalized chronological sliding-window samples with an 80%/20% train–test split. Tasks span smooth and nonlinear classical dynamics, memory-dependent signals, delayed quantum control, and CUDA-Q Dynamics simulations of open and closed quantum systems.

  • A. Tasks and preprocessing: Seven univariate time-series benchmarks are min–max normalized to [−1, 1], converted into chronological sliding-window samples, and split 80% for training and 20% for testing.The setup follows prior works [14], [20].
  • A. Tasks and preprocessing: The damped simple harmonic motion task predicts angular velocity from a nonlinear damped pendulum and tests smooth damped oscillatory prediction.Parameters include g = 9.81, b = 0.15, L = m = 1, θ(0) = 0, and ˙θ(0) = 3.
  • A. Tasks and preprocessing: The Bessel benchmark uses the second-order first-kind function J2(x) to test nonlinear approximation under changing amplitude and phase.The benchmark is designed around amplitude and phase variation.
  • A. Tasks and preprocessing: NARMA-5 and NARMA-10 test memory orders n = 5 and n = 10 through a standard nonlinear autoregressive moving-average recurrence.The recurrence includes delayed outputs and input interactions.
  • A. Tasks and preprocessing: Delayed quantum control provides a non-Markovian feedback-like signal formed by decaying localized pulses over t ∈[−2, 20].Its pulse construction is x(t) = P10 n=0 exp[−10(t − 2n)2] exp(−t/16).
  • A. Tasks and preprocessing: Open Jaynes–Cummings dynamics uses a two-level qubit coupled to a cavity truncated to 5 Fock levels and simulated with CUDA-Q Dynamics.The target is qubit excitation expectation probability ⟨σ+σ−⟩(t), with photon loss rate γ = 0.05 over t ∈[0, 50].
  • A. Tasks and preprocessing: Closed dispersive transmon–resonator dynamics uses a two-level transmon and a resonator truncated to 20 Fock levels, also simulated with CUDA-Q Dynamics.The target is the resonator position quadrature expectation ⟨ˆx⟩(t) over t ∈[0, 25] ns; trajectories contain 3000 equally spaced time steps.

B. Single-step prediction

Single-step prediction uses sequences of length N to predict the next observation after sequentially processing inputs and updating the fast state. The evaluation spans N=4, 8, 16, 32, and 64 to test information retention as context grows.

  • Each input sequence contains N observations, xt,N = [xt−N, . . . , xt−1], and targets xt.
  • The model processes observations sequentially, updating its fast state at every internal step before predicting after the final observation.
  • N=4, 8, 16, 32, and 64 are evaluated to assess how update rules retain information as sequence length increases.

C. Direct multi-step forecasting

Direct multi-step forecasting uses a fixed 64-step input and predicts horizons of 4, 8, or 16 steps in one forward pass without recursive feedback. This protocol tests whether the fast state retains enough history for longer-horizon prediction.

  • Protocol: The input length is fixed at N = 64 for direct forecasting.The input is xt,64 = [xt−64, . . . , xt−1].
  • Protocol: The model outputs the complete prediction horizon in one forward pass for H ∈ {4, 8, 16}.The target is yt,H = [xt, xt+1, . . . , xt+H−1].
  • Evaluation objective: The protocol avoids recursive prediction feedback to isolate whether the fast state contains sufficient history for longer-horizon forecasting.Predictions are not fed back into the model during the forecast.

D. Training and evaluation

Training uses a GPU-efficient FlashQKAN implementation with CUDA acceleration, five random seeds, and standardized Adam optimization across experimental configurations. Evaluation primarily uses MSE, including horizon-averaged MSE for multi-step forecasting.

  • Implementation: FlashQKAN uses PyTorch with CuTe DSL fused operators and block tiling on CUDA devices, building on the open-source QKAN repository.The implementation is designed for GPU-efficient training.
  • Training protocol: Each dataset, architecture, update rule, sequence length, and horizon is trained for 100 epochs with five independent seeds, Adam at learning rate 10^-3, and batch size 4.The primary evaluation metric is MSE.
  • Evaluation metrics: Multi-step forecasting reports MSE averaged over the prediction horizon, samples, and random seeds.The horizon metric is computed from squared differences between predicted and target values at each forecast offset.
  • Evaluation metrics: Relative improvement is defined against the corresponding scalar-gated baseline, with positive values indicating lower MSE.This interpretation applies to the reported relative-improvement measure.
  • Benchmark reporting: Final test MSE on NARMA and quantum-dynamics benchmarks is reported as mean (standard deviation) over five seeds for N = {4, 8, 32, 64}.The benchmark tables identify best and second-best results using bold and underlined formatting.

V. EXPERIMENTAL RESULTS AND ANALYSIS · A. Single-step prediction

Across single-step forecasting, CMG provides the most consistent improvement over scalar gating, especially for QKAN-based backbones. Its advantage persists across sequence lengths and training dynamics, becoming increasingly important as context grows.

  • A. Single-step prediction: CMG provides the most consistent improvement over scalar-gated baselines across four model families and sequence lengths N ∈{4, 8, 16, 32, 64}.Paired scatter plots show the cleanest shift below the diagonal, especially for QKANFWP and QKAN-QKANFWP.
  • A. Single-step prediction: Across 28 configurations, CMG variants achieve the best MSE in 24 cases.The selected top-performing arms at N = 16 were re-evaluated across the remaining sequence lengths.
  • A. Single-step prediction: CMG QKAN-QKANFWP and CMG QKANFWP decisively outperform classical FWP across most tasks.The cross-sequence results strongly favor QKAN-based backbones over classical FWP.
  • A. Single-step prediction: At N = 64, CMG QKANFWP achieves the best MSE, confirming that CMG becomes increasingly critical as the effective context window grows.NARMA datasets show slight heterogeneity at shorter sequences.
  • A. Single-step prediction: On Jaynes–Cummings, CMG remains among the lowest-error rules after early training at N = 64.This indicates that its advantage is not merely a final-epoch artifact.
  • A. Single-step prediction: On transmon–resonator, CMG reaches the low-error regime quickly and remains close to Only-old and Full through epoch 100.Scalar gating and Only-new remain in a higher-error regime, especially on transmon–resonator.

B. Direct multi-step prediction

Direct multi-step forecasting on CUDA-Q Dynamics datasets used N = 64 and horizons H ∈ {4, 8, 16}, focusing on QKANFWP and QKAN-QKANFWP. CMG maintained 10−4-scale expectation-value forecasts and strong long-horizon behavior, while Only-new consistently underperformed scalar gating.

  • Quantitative results: Every CMG QKANFWP and QKAN-QKANFWP cell was no worse than 9.3 × 10−4, placing forecasts at the 10−4 rather than 10−2 scale.The comparison is against scalar gating.
  • Quantitative results: Only-old, Full, and CMG reduced MSE by orders of magnitude over scalar gating on both quantum systems, whereas Only-new was worse in every multi-step setting.These findings concern the Jaynes–Cummings and transmon–resonator systems.
  • Quantitative results: Full was best in four dataset-horizon cells and Only-old in three, while CMG reached the same competitive regime with Only-old’s modulation-head size and fewer heads than Full.The listed cells included both QKANFWP and QKAN-QKANFWP across the two systems.
  • Qualitative forecasts: Full and CMG QKAN-QKANFWP tracked held-out oscillatory trajectories with accurate phase and amplitude, while scalar gating showed larger mismatches and wider seed variation.CMG remained visually comparable to Full despite using one matrix-gating head.

VI. CONCLUSION

The conclusion presents Self-Modulating QKAN-FWP and CMG as coordinate-wise memory-control mechanisms that preserve scalar gating’s bounded-convex stability and affine prefix-scan compatibility. CMG delivers consistent forecasting gains while motivating extensions to structured spatio-temporal fast states.

  • Core contribution: CMG uses one low-rank-generated matrix gate and its complement to write the new proposal, enabling coordinate-wise bounded-convex updates with affine parallel-prefix scans.This generalizes scalar gating while preserving its stability and parallelization properties.
  • Core contribution: CMG has the modulation-head cost of Only-old self-modulation and less than Full self-modulation.It relaxes the constraint that one coefficient govern every fast-state coordinate.
  • Empirical findings: Across seven single-step benchmarks and five sequence lengths, CMG is the most consistent update rule for QKANFWP and QKAN-QKANFWP.The comparison covers both architectures across all reported benchmark lengths.
  • Empirical findings: 10^-4-scale expectation-value error or below is maintained across H ∈{4, 8, 16}, with CMG improving on scalar gating by at least 91.2%.This result concerns direct multi-step Jaynes–Cummings and transmon–resonator forecasting.
  • Limitations and outlook: The formulation targets univariate temporal memory and does not explicitly represent spatial structure among variables.Extending CMG to structured spatio-temporal fast states is proposed for multivariate physical dynamics and quantum-control forecasting.
Loading 2607.27945v1…