PaperlayerView exact revision on arXiv

Source-linked AI summary

Enriching Sequential Recommendation with Graph Laplacian Positional Embeddings

Ekaterina Trushkova, Artur Gimranov, Anton Lysenko

arXiv:2609.31253v1cs.IR

TL;DR

Sequential recommenders encode interaction order with learnable positional embeddings, but item co-occurrence graphs provide another source of structure. This paper replaces SASRec’s positional embeddings with frozen Laplacian embeddings computed from training interactions, improving most ranking metrics across four benchmarks while remaining competitive with positional and temporal baselines.

  • Problem

    Sequential recommenders use positional embeddings for interaction order, while it remains an open question whether item–item graph structure can be injected through that encoding mechanism.

    Source: §1 · p. 1

  • Method

    The method computes eigenvectors of the symmetric normalized Laplacian of the training-interaction item co-occurrence graph and uses them as frozen SASRec positional embeddings without changing its backbone or objective.

    Source: §1 · pp. 1–2

  • Results

    Across four public benchmarks, Laplacian positional embeddings improve SASRec on most ranking metrics and remain competitive with strong positional and temporal baselines.

    Source: §1 · pp. 1–2

  • Takeaways & Limitations

    The findings suggest item–item graph structure provides recommendation signal not fully captured by ordinal positional embeddings alone.

    Source: §1 · p. 1, §5 · p. 5

Abstract

from arXiv · show

Sequential recommenders typically rely on learnable positional embeddings to encode the order of user interactions. In this work, we ask whether this ordinal signal can be replaced by a structural one derived from the item space. We propose to use Laplacian positional embeddings in SASRec: we build an item co-occurrence graph from training interactions, compute eigenvectors of its symmetric normalized Laplacian, and use them as frozen graph-derived positional embeddings. The backbone architecture and training objective remain unchanged. Experiments on four public sequential-recommendation benchmarks show that this simple replacement improves SASRec performance on most ranking metrics and remains competitive with strong positional and temporal encoding baselines. These findings indicate that item-item graph structure can be an effective substitute for standard ordinal positional embeddings in sequential recommendation.

1 Introduction

Sequential recommenders use positional embeddings to encode interaction order, while item co-occurrence also supplies global structure beyond local sequence context. This work replaces SASRec’s learnable positional embeddings with frozen Laplacian embeddings from the item graph, improving most ranking metrics across four benchmarks and remaining competitive with positional and temporal baselines.

1 Introduction

  • The motivating question is whether global item–item structure can enter a sequential recommender through positional encoding, rather than only indirectly through training and item embeddings.Items frequently consumed by the same users may be related even when they do not share a local sequential context.Source: §1 · p. 1
  • The method replaces SASRec’s learnable positional embeddings with frozen eigenvectors of the symmetric normalized Laplacian of a training-interaction item co-occurrence graph.The SASRec architecture and training objective remain unchanged.Source: §1 · pp. 1–2
  • The method adapts graph-Transformer spectral coordinates to recommendation, adding each item’s graph-derived embedding to its item representation at the input layer.The graph eigendecomposition is performed once, and the embeddings remain fixed during training.Source: §1 · p. 1
  • Across four public benchmarks, Laplacian positional embeddings improve SASRec on most ranking metrics and remain competitive with rotary and time-aware baselines.The results suggest item-graph structure supplies recommendation signal not fully captured by ordinal positional embeddings alone.Source: §1 · pp. 1–2

2 Related Work

Related work spans sequential models with alternative order and time encodings, graph Transformers that encode structural position, and hybrid recommenders that combine graph and sequence representations. This work instead computes graph spectral coordinates once and uses them in SASRec’s positional slot.

2 Related Work

  • The paper adopts SASRec with full-catalog cross-entropy as its backbone, following SASRec+ [7], which closes much of the gap to BERT4Rec [16] while preserving lower training cost.BERT4Rec [16] uses a bidirectional masked-item objective.Source: §2 · p. 2
  • Transformer sequential recommenders use positional or temporal encodings because self-attention alone has no intrinsic sequence-order information.TiSASRec adds both absolute position and time-interval information; relative and rotary alternatives include [15] [17].Source: §2 · p. 2
  • Graph Transformers use Laplacian eigenvectors as structural node coordinates, an idea introduced by Dwivedi and Bresson [3] and developed in SAN [9] and GraphGPS [14].The paper adapts this structural encoding to item co-occurrence graphs in sequential recommendation.Source: §2 · p. 2
  • Hybrid graph-aware recommenders add branches, objectives, or fusion modules, whereas this approach uses a frozen spectral item lookup in SASRec’s positional slot and leaves its backbone, loss, and inference path unchanged.Examples include MCLSR, MRGSRec [1], GSAU [2], and LOOM.Source: §2 · p. 2

3 Method

The method predicts a user’s next item from an ordered history, while deriving frozen positional inputs from the training-only item co-occurrence graph. It selects low-frequency Laplacian coordinates and integrates them into SASRec’s input representation.

3.1 Problem statement

  • Given a user’s chronological interaction prefix, the task is to rank items by their predicted probability of being the next interaction.The item graph is built from training interactions only, with edge weights counting users whose histories contain each item pair.Source: §3.1 · p. 2
  • The undirected weighted item graph uses A = R^T R with its diagonal set to zero, so each edge weight counts users who interacted with both items.R is the binary user-item interaction matrix for the training split.Source: §3.1 · p. 2

3.2 Laplacian positional encoding

  • Degree normalization reduces highly popular items’ dominance, and selected non-trivial eigenvectors of the normalized Laplacian form structural item coordinates.Each row of the encoding is a k-dimensional graph-spectral representation of an item.Source: §3.2 · p. 2
  • The graph, normalized Laplacian, and eigenvectors are computed once before training, and the resulting Laplacian positional encoding table remains frozen through training and inference.When k differs from the Transformer hidden size, a learnable projection maps the coordinates to that size.Source: §3.2 · p. 2

3.3 Eigenvector selection and model integration

  • SASRec-LapPE uses the k = d eigenvectors associated with the smallest non-zero eigenvalues, capturing low-frequency structure; an alternative passes the spectral input through a two-layer MLP.LPE modifies only the input embedding layer.Source: §3.3 · pp. 2–3
  • The input combines the learnable item embedding with a scaled graph-spectral embedding, replacing SASRec’s learnable sequence-position embedding.The scalar α controls the graph signal’s contribution; LPE depends on item identity and the training graph, not sequence position or time.Source: §3.3 · p. 3

4 Experiments and Results

Experiments compare SASRec-LapPE with sequential-recommendation baselines on four benchmarks, using a shared evaluation setup. LapPE improves most ranking metrics, remains competitive with stronger encoding methods, and adds only one-time preprocessing overhead.

4.1 Datasets

  • The four benchmarks are Amazon-Beauty, Amazon-Clothing, Amazon-Sports, and Yelp2018; interactions are treated as positive feedback and chronologically ordered.Users with fewer than three interactions are removed.Source: §4.1 · p. 3

4.2 Experimental setup

  • Testing uses each user's most recent 20% of interactions, while the graph and LapPE vectors are computed from training interactions only.Testing averages ten random holdout splits; tuning targets validation NDCG@10 and reuses the best dataset-specific SASRec backbone across variants, except for TiSASRec's reduced batch size.Source: §4.2 · p. 3

4.3 Baselines and variants

  • The comparison includes full-softmax SASRec, RoPE, and validation-selected TiSASRec configurations; all SASRec variants share a backbone, while TiSASRec adds time intervals to attention.RoPE uses rotary positional embeddings, and TiSASRec's max_time_interval is selected from a validation-searched grid.Source: §4.3 · p. 3

4.4 Results

  • NDCG@10 rises from 0.89 to 1.11 and NDCG@100 from 1.80 to 2.29 on Amazon-Clothing; across four benchmarks, LapPE improves SASRec on most ranking metrics.Table 2 reports results over ten random holdout splits.Source: §4.4 · p. 3, Table 2 · p. 4
  • At @100, SASRec-LapPE outperforms TiSASRec on all four datasets and improves over RoPE on three, without online positional or temporal computation.Its spectral signal is computed once from the training item graph and used as a frozen item-level lookup.Source: §4.4 · p. 4
  • LapPE is consistently stronger than TiSASRec in Coverage@10 and Coverage@100, but Amazon-Clothing has lower Coverage despite large NDCG and Recall gains.The authors suggest low-frequency components may concentrate recommendations within coherent item communities.Source: §4.4 · p. 4

4.5 Efficiency Comparison

  • Graph construction and eigendecomposition are one-time preprocessing; afterward, training is comparable to SASRec and inference marginally faster.Preprocessing took 2.52 seconds on Beauty and 95.86 seconds on Yelp2018; TiSASRec is substantially slower in training and inference.Source: §4.5 · p. 5, Figure 1 · p. 5

5 Conclusion and Future Work

The conclusion frames positional embeddings as a way to represent static item-space structure, not only interaction order. It presents frozen Laplacian embeddings as a lightweight SASRec substitute and identifies spectral frequency selection as future work.

5 Conclusion and Future Work

  • Positional embeddings can provide a channel for static item-space structure as well as mark interaction order.The authors describe frozen Laplacian embeddings from the item co-occurrence graph as a viable substitute for standard learnable positions in SASRec, with the sequential backbone unchanged.Source: §5 · p. 5
  • The authors identify systematic spectral frequency selection, such as validation-based band search or learnable filtering, as future work.The proposed approach uses the graph once before training rather than adding a graph–sequence hybrid architecture.Source: §5 · p. 5

GenAI Usage Disclosure

The authors disclose generative-AI assistance during manuscript preparation, code work, and result description, while attributing the research question, method, experimental work, and conclusions to themselves.

GenAI Usage Disclosure

  • The AI assistant helped scaffold paper structure, suggest LaTeX boilerplate, and check prose consistency against result tables; authors reviewed and verified generated text.The authors state they wrote the narrative, framing, and initial idea-level drafts.Source: GenAI Usage Disclosure · p. 5
  • AI tools assisted with scripts and implementation optimization, and the authors reviewed, modified, and tested the generated code.Source: GenAI Usage Disclosure · p. 5
  • No generative-AI tools generated or augmented the data; the authors report producing the experiments, searches, aggregations, and analytical conclusions.They identify the research question, method, experimental design, and conclusions as their intellectual contributions.Source: GenAI Usage Disclosure · p. 5

Related papers

Loading 2609.31253v1…