PaperlayerView exact revision on arXiv

Source-linked AI summary

Hamiltonian Spectral-Temporal Dissipative Dynamics for Sequential Recommendation

Shuiying Liao, P. Y. Mok

arXiv:2608.25755v1cs.IR

TL;DR

Sequential recommendation needs to capture inertia, periodicity, and abrupt shifts that first-order models do not explicitly represent. HSR models preference evolution as a dissipative Hamiltonian system with spectral propagation, local impulse refinement, and phase-space extrapolation, and experiments show consistent gains over Transformer- and SSM-based baselines.

  • Problem

    User behavior includes inertia, periodicity, and abrupt shifts, but existing first-order approaches do not explicitly model these richer preference dynamics.

  • Method

    HSR uses a dissipative Hamiltonian latent phase space with position and momentum, combining spectral propagation, local impulse refinement, and one-step extrapolation.

  • Results

    Experiments on three benchmarks show consistent gains over state-of-the-art Transformer- and SSM-based recommenders.

  • Takeaways & Limitations

    HSR provides a sequential recommendation framework that jointly models long-range inertial evolution, periodic structure, and localized behavioral fluctuations.

Abstract

from arXiv · show

Sequential recommendation requires understanding how user preferences evolve over time, yet most existing models treat such evolution as a first order process where the next state depends solely on the current latent representation. Nevertheless, real user behavior often exhibits richer dynamics, including inertia, periodicity, and sudden shifts that cannot be fully captured by these first order assumptions. Motivated by these behavioral characteristics, we reconceptualize sequential recommendation through the lens of second order dynamical systems and introduce the Hamiltonian Spectral Recommender (HSR), which recasts preference evolution as a dissipative Hamiltonian system in a latent phase space of position (stable preference) and momentum (short-term tendency). The linear time-invariant structure of the governing equation admits a closed-form solution in the frequency domain. A learnable dissipation mechanism further captures natural interest decay, while a short local impulse refinement module models abrupt behavioral fluctuations commonly observed in sparse interaction logs. This design jointly accounts for global periodic patterns, inertial evolution, and localized shocks, where three phenomena that are underrepresented in existing sequential models. Extensive experiments on three benchmark datasets demonstrate that HSR consistently outperforms state-of-the-art Transformer-based and state space model (SSM)-based recommenders.

P. Y. Mok∗

Figure 1 presents the main motivation of HSR.

  • Figure 1 is identified as the main motivation of HSR.
  • The figure is associated with HSR.
  • The supplied figure caption provides no further details about the motivation.

1 Introduction

Sequential recommendation must model evolving interests whose inertia, temporary deviations, and abrupt shifts exceed first-order state updates. HSR addresses this gap with dissipative Hamiltonian dynamics, spectral propagation, local impulse refinement, and phase-space extrapolation.

  • A first-order model can drift toward a temporary topic, whereas second-order dynamics represent inertia, deviation, and damped recovery.
  • User behavior exhibits inertia, periodicity, and abrupt shifts, so preference evolution depends on both the current state and its rate of change.
  • HSR models preferences as a dissipative Hamiltonian system with position for stable preference and momentum for short-term tendency.
  • The model combines a physics-derived spectral propagator, a local impulse branch, and one-step phase-space extrapolation for next-step prediction.

2 Related Work

Related sequential recommenders include neural sequence models, frequency-domain methods, and structured state-space models. These approaches capture temporal structure efficiently but generally do not explicitly model higher-order preference dynamics such as momentum and inertia.

  • Neural Models for Sequential Recommendation: Neural recommenders use convolutional, recurrent, attention-based, and linear-complexity architectures to model temporal dependencies and long contexts.
  • Existing neural and state-space approaches generally do not explicitly represent inertia, oscillation, momentum, or distinct preference velocity.
  • Frequency-Domain and Spectral Approaches: Frequency-domain methods use spectral representations to capture periodicity, denoise sequences, filter fluctuations, or amplify stable patterns.
  • Structured State-Space Models (SSMs): Structured state-space models parameterize linear time-invariant systems for scalable long-sequence processing with linear-time complexity.

3 Method

HSR maps item sequences into latent representations, processes them through Hamiltonian spectral and local impulse components, extrapolates the terminal phase-space state, and ranks candidate items.

  • 3.1 Problem Formulation: Sequential recommendation predicts the next item from a user’s observed interaction prefix, with item IDs mapped to learnable embeddings.
  • 3.2 Overview: HSR’s architecture includes embedding, Hamiltonian spectral encoding, one-step extrapolation, and candidate-item prediction.
  • 3.2 Overview: The architecture combines global frequency-domain evolution with local time-domain impulse modeling, gated fusion, and residual normalization.
  • 3.2 Overview: The prediction layer scores candidate items by compatibility with the extrapolated latent preference and ranks them for top-K recommendation.

3.3 Primer

HSR models user preference evolution as a dissipative second-order Hamiltonian system in latent position–momentum phase space. Its Fourier-domain formulation converts the driven dynamics into a frequency-domain propagator and derives momentum from position.

  • HSR represents stable preference as position q(t) and short-term behavioral tendency as momentum p(t) in a conjugated phase space.
  • Learnable mass, damping, and stiffness encode inertia, forgetting rate, and preference stability, while item interactions provide the external force.
  • Fourier transformation turns the computationally expensive time-domain oscillator solution into an algebraic frequency-domain relation and spectral propagator.
  • Momentum is reconstructed spectrally from position using p̂_i(ω) = m_i(iω)q̂_i(ω), preserving the Hamiltonian structure.

3.4 HSR Layer Implementation

The HSR layer combines projected force, impulse, and gate branches with global Hamiltonian spectral propagation and local time-domain refinement. Fourier-domain filtering models long-range temporal patterns, while gated fusion incorporates abrupt local deviations.

  • HSR Layer Implementation: Each HSR block maps an input latent sequence to an updated sequence through projection, global Hamiltonian propagation, and local impulse refinement with gated fusion.
  • Stage I: Projection Mapping: Projection produces driving-force, local-impulse, and gate branches; force enters the dynamics, impulse models bursty behavior, and the gate controls fused-signal expression.
  • Stage II: Global Hamiltonian Spectral Propagation: Real FFT transforms each channel’s temporal signal into non-redundant non-negative frequency modes for modeling long-range and periodic preference dynamics.
  • Stage II: Global Hamiltonian Spectral Propagation: The propagator selectively amplifies, attenuates, and phase-shifts temporal patterns, functioning as a frequency-dependent filter for global evolution.
  • Stage II: Global Hamiltonian Spectral Propagation: Momentum is recovered from the evolved position spectrum rather than learned independently, coupling both phase-space variables through second-order dynamics.
  • Stage III: Local Impulse Refinement and Gated Fusion: Local impulse refinement captures abrupt deviations that a globally smooth trajectory misses, and gated fusion combines it with the global position trajectory.

3.5 Stacking Blocks and Final Phase-Space Recovery

HSR stacks Hamiltonian spectral blocks and then recomputes terminal momentum from the final fused trajectory. This preserves phase-space consistency while retaining local impulse refinement for final prediction.

  • 3.5 Stacking Blocks and Final Phase-Space Recovery: The model stacks L Hamiltonian spectral blocks with local refinement, gating, residual fusion, and normalization to produce the final hidden sequence.
  • 3.5 Stacking Blocks and Final Phase-Space Recovery: Using momentum from an earlier branch would mismatch the final position, so a jointly trained terminal mass recovers momentum from the final hidden sequence.
  • 3.5 Stacking Blocks and Final Phase-Space Recovery: The terminal phase-space state derives position and momentum from the same final latent trajectory, preserving consistency while retaining local impulse refinement.

3.6 Phase-Space Extrapolation and Prediction

HSR uses the terminal phase-space state to represent current preference and its short-term evolution, then extrapolates one step forward for next-item ranking. Prediction is trained end-to-end with softmax item scores and regularized loss.

  • 3.6 Phase-Space Extrapolation and Prediction: The terminal position captures current interest, while terminal momentum captures the direction and strength of short-term preference change.
  • 3.6 Phase-Space Extrapolation and Prediction: A one-step Euler extrapolation advances the phase-space state using learned mass parameters to estimate the next-step latent preference.
  • 3.6 Phase-Space Extrapolation and Prediction: The extrapolated sequence is scored against item embeddings with a dot-product softmax for the ground-truth next item.
  • 3.6 Phase-Space Extrapolation and Prediction: All model components are optimized end-to-end with the prediction objective and weight decay.

4 Experiments

HSR is evaluated across three sequential-recommendation datasets against diverse architectures, including Transformer- and Mamba-based models. It achieves strong overall results, with gains concentrated in sparse and irregular settings, while ablations and sensitivity studies show that its dynamical components and moderate capacity are important.

  • Overall Performance: HSR achieves the best performance across most metrics on all three datasets, with particularly notable improvements on the sparse Amazon datasets despite significantly fewer parameters.The authors attribute this result to Hamiltonian spectral modeling of inertia and periodic frequency behavior.
  • Comparison with Attention-Based Methods: HSR attains the best performance on all metrics and datasets among attention-based models, with relatively larger improvements in NDCG and MRR.The authors connect this pattern to next-step trajectory modeling and improved top-of-list re-ranking.
  • Comparison with SSM-Based Methods: On dense MovieLens-1M, SSM-based baselines appear stronger than attention-based models, yet HSR remains competitive and surpasses all baselines, especially on NDCG and MRR.The comparison frames second-order propagation as a general-purpose approach beyond sparse-data settings.
  • Component-wise Ablation Analysis: Ablations show that removing any single HSR module degrades at least one metric, with the largest drops caused by removing Hamiltonian structure or short-impulse refinement.The short-impulse branch is especially important for noisy, bursty interaction patterns, while phase-space extrapolation produces a smaller systematic gain.
  • Hyperparameter Sensitivity Analysis: HSR performance generally improves with more Hamiltonian blocks before saturating; the optimal depth is three blocks for Beauty and four for Game.Increasing hidden dimension from 64 to 256 improves Hit@10 by 11.5% on Amazon-Beauty and 13.0% on Amazon-Video-Games, while expansion to 512 shows diminishing returns.

5 Discussions

HSR’s diagnostic studies show consistent advantages across sequence lengths, interaction noise, cold-start settings, and online reward, while its learned dynamics and spectral implementation support behavioral interpretability and computational efficiency.

  • Performance across Sequence Lengths: HSR consistently outperforms state-of-the-art methods across interaction-history lengths, retaining a margin on ultra-short sequences and widening it on histories longer than 100.The local impulse branch recovers short-term signals, while the Hamiltonian propagator encodes long-range temporal correlations through frequency-dependent impedance.
  • Robustness to Interaction Noise: HSR shows the flattest performance degradation as synthetic interaction noise increases from 0% to 30%.The damping coefficient attenuates high-frequency components of the driving force, where the injected noise concentrates.
  • Cold-Start Robustness: HSR performs best across nearly the entire cold-start range, with its largest margin when users have only a few observed interactions.Local impulse refinement recovers short-term signals, and momentum-based extrapolation supplies a directional prior when context is scarce.
  • Interest Evolution in Phase Space: HSR’s representative latent trajectory makes a large high-momentum excursion before spiraling inward toward a compact attractor as damping dissipates energy.This orbit-and-settle behavior contrasts with the monotonic state transitions typical of conventional first-order models.
  • Comparison with Multi-Armed Bandit Recommender: HSR consistently outperforms BMAB across all values of K in average online next-item reward among the K most popular arms.
  • Computational Efficiency: HSR combines lower resource consumption with stronger performance through structured frequency-domain propagation, FFT-based computation, shallow depth, and reduced sequential dependence.Each Hamiltonian spectral block uses one FFT and one inverse FFT with O(TlogT) scaling, while inference adds one Euler extrapolation step and supports cached intermediate states.

6 Conclusion

The paper introduces HSR as a dissipative Hamiltonian recommender that models preference evolution through latent position and momentum. Its spectral propagator, local impulse branch, and phase-space extrapolation achieve O(TlogT) propagation and consistent gains over state-of-the-art baselines, while learned damping and physical parameters provide behaviorally meaningful interpretations.

  • HSR recasts sequential preference evolution as a dissipative Hamiltonian system with latent position and momentum, replacing first-order recurrences used by existing Transformer- and SSM-based models.
  • HSR combines a physics-derived O(TlogT) spectral propagator, a local impulse branch for short-term fluctuations, and momentum-based phase-space extrapolation for next-item prediction.
  • Experiments on three benchmarks show consistent gains over state-of-the-art baselines, with damping filtering noise and learned physical parameters adapting to dataset characteristics.
Loading 2608.25755v1…