PaperlayerView exact revision on arXiv

Source-linked AI summary

MIDA: Multi-Interest Decoding with Adaptive Aggregation for Session-based Recommendation

Linjiang Guo, Nitin Bisht, Shiqing Wu, Guandong Xu

arXiv:2610.34192v1cs.IR

TL;DR

Session-based recommendation models can underuse distant, semantically relevant interactions, and longer sessions show lower accuracy across existing model families. MIDA reformulates decoding with differentiable multi-interest clustering and adaptive aggregation; experiments report consistent gains across benchmark datasets and encoder architectures.

  • Problem

    Existing decoders give limited attention to decoding and may blur item distinctions or overweight recent interactions, leaving long-session context underused.

    Source: §I · p. 1

  • Method

    MIDA clusters semantically related items dynamically and fuses persistent and evolving intent through stability and plasticity branches with adaptive gating.

    Source: §I · p. 2

  • Results

    Experiments on four benchmark datasets show that MIDA consistently outperforms strong baselines across encoder architectures, especially on longer and semantically diverse sessions.

    Source: §VI · p. 12

  • Takeaways & Limitations

    MIDA is a lightweight decoder framework intended to model stable and evolving intent while using long-range, multi-interest dependencies in session-based recommendation.

    Source: §VI · p. 12

    Pairwise similarity may incur non-trivial overhead on extremely long sessions, and the two-component intent model may not capture all complex multi-interest structures.

    Source: §VI · p. 13

Abstract

from arXiv · show

Session-based recommendation (SBR) infers user interest from limited interaction sequences, yet current models show a consistent performance drop when longer histories are available. This degradation is associated with decoder designs that overemphasize recent items while underutilizing distant, yet semantically relevant, interactions, leading to poor modeling of long-range dependencies and multi-interest behaviors. To address this challenge, we introduce MIDA (Multi-Interest Decoding with Adaptive Aggregation for Session-based Recommendation), a lightweight and encoder-agnostic framework that provides a reformulation of the decoding stage. The central design principle is a two-space decoupling: clustering operates on raw item embeddings so that semantic groupings are not distorted by encoder-learned relational biases; encoder outputs are reserved for aggregation, where sequential context remains intact. Concretely, MIDA (i) employs differentiable multi-interest clustering on raw embeddings, preserving semantic structure that encoder representations would otherwise obscure, and (ii) fuses stability and plasticity branches through a gate conditioned on average cluster size, so that the inferred clustering structure directly governs aggregation behavior through a structural coupling across the two stages. Extensive experiments on multiple benchmark datasets demonstrate that MIDA consistently outperforms state-of-the-art methods, with meaningful accuracy improvements while remaining computationally practical.

I. INTRODUCTION

MIDA reframes session-based recommendation around the decoder, addressing the limited attention to decoding and the underuse of distant, semantically relevant interactions. It combines adaptive multi-interest clustering and stability–plasticity aggregation, and experiments across benchmark datasets report gains over strong baselines.

I. INTRODUCTION

  • Existing decoders underuse long-range context: pooling blurs item distinctions, while last-item-anchored methods overweight recent interactions.Longer sessions show lower accuracy despite richer historical information, across GRU-, graph-, and Transformer-based model families.Source: §I · pp. 1–2

III. THE PROPOSED FRAMEWORK

  • MIDA uses differentiable clustering to group semantically related items across time, then gates stability and plasticity branches using semantic cohesion signals.The gate is conditioned on branch representations and average cluster size.Source: §I · p. 2

I. INTRODUCTION

  • MIDA outperforms strong baselines across diverse encoder architectures in experiments on multiple benchmark datasets.The framework replaces only the final aggregation step while preserving encoder parameters and training schedules.Source: §I · p. 2

II. RELATED WORK

  • Unlike encoder-level multi-interest methods, MIDA clusters dynamically at the decoder stage and separates persistent from evolving intents.Its clustering uses within-session semantic similarity without a preset number of interests or heavy overhead.Source: §II · p. 2
  • MIDA clusters raw item embeddings for semantic similarity and reserves encoder outputs for aggregation, preserving sequential dependencies without changing encoder parameters.This two-space decoupling distinguishes which items are related from how they are aggregated in session context.Source: §II · p. 3

A. Notation

The paper defines session-based recommendation as predicting the next item from an ordered interaction history. MIDA clusters raw item embeddings using a learnable threshold, then uses the resulting clusters to preserve semantically related signals across temporal gaps.

A. Notation

  • Table I is identified as a summary of the paper’s key symbols.The supplied passage does not specify the table’s individual entries.Source: A. Notation · p. 3, Table I · p. 3

B. Problem Formulation

  • A session S is an ordered sequence of interacted items, and the task is to predict the next item from that history.Each item is mapped to a raw embedding that captures semantic information; the encoder produces contextualized embeddings that capture sequential dependencies.Source: B. Problem Formulation · p. 3

C. Dynamic Multi-Interest Clustering

  • MIDA clusters raw item embeddings by cosine similarity, allowing semantically coherent items to be grouped even when they are far apart in the session.The clustering module uses soft weights derived from a global learnable threshold; the supplied notation passage also identifies a regularizer and a small constant that prevents weights from becoming numerically zero.Source: C. Dynamic Multi-Interest Clustering · pp. 3–4

D. Adaptive Aggregation

Adaptive Aggregation balances persistent and evolving intents through separate branches. Their queries attend over encoder outputs and are fused by a gate that uses average cluster size as a signal of session structure.

D. Adaptive Aggregation

  • The stability branch emphasizes items in broad clusters, while the plasticity branch focuses on the most recent cluster output.The stability query excludes the last item, which anchors the plasticity branch, to avoid conflating the two signals.Source: D. Adaptive Aggregation · pp. 4–5
  • Each branch query attends over encoder outputs through dedicated attention heads, retaining the backbone’s sequential dependencies during aggregation.The design learns query and attention representations separately to reduce gradient interference and support training stability.Source: D. Adaptive Aggregation · p. 5
  • The gate fuses stability and plasticity using average cluster size: larger clusters emphasize persistent intents, while smaller clusters favor evolving preferences.The gate also conditions on the branch representations.Source: D. Adaptive Aggregation · p. 5

E. Training Objective

The training objective combines next-item prediction loss with a regularizer on similarity assignments. The regularizer applies upward pressure on the learnable threshold to discourage overly dense clusters.

E. Training Objective

  • The prediction loss is the negative log-likelihood of the ground-truth next item.Minimizing prediction loss alone can drive the threshold downward and produce overly dense clusters that blur semantic boundaries.Source: E. Training Objective · p. 5
  • A regularization term penalizes excessive similarity assignments, applying upward pressure on the threshold because the soft weights decrease monotonically with it.The final objective combines prediction loss and regularization, with η as a trade-off hyperparameter.Source: E. Training Objective · p. 5

IV. THEORETICAL ANALYSIS

The analysis characterizes how the learnable threshold shapes clustering and the resulting stability and plasticity queries. Sparse and dense clustering regimes yield distinct ways of summarizing session history.

  • The regularizer is continuous and non-increasing in τ, with limits L^2 as τ approaches −∞ and L^2ϵ as τ approaches +∞.Source: §IV · p. 5
  • As τ decreases, more pairwise assignments activate and regularization cost increases, penalizing fully dense clustering and favoring compact, semantically selective clusters.Source: §IV · p. 5
  • In sparse singleton-like regimes, stability aggregates historical items while plasticity remains anchored to the last item; dense clustering makes both queries use a shared global summary.The threshold controls how the queries summarize the session.Source: §IV · p. 6
  • When raw embeddings are normalized and τ exceeds every off-diagonal similarity but remains below 1, each item forms a singleton cluster.The diagonal similarity exceeds τ while every off-diagonal similarity falls below it.Source: §IV · p. 6

A. Learning Procedure

Each forward pass forms clusters from item similarities, uses the resulting embeddings to construct stability and plasticity queries, and fuses them into a session representation. Training updates the decoder and clustering threshold jointly.

  • Each forward pass normalizes and subsamples item embeddings to compute similarities, forms soft clusters under the current threshold, and applies position-aware attention to produce cluster-enhanced embeddings.Source: A. Learning Procedure · p. 6
  • Cluster-enhanced embeddings form stability and plasticity queries that attend over the session relational context and are fused into the session representation.Source: A. Learning Procedure · p. 6
  • Prediction-loss gradients update decoder weights, while gradients through soft cluster weights adjust the threshold and regularization prevents it from collapsing to overly dense assignments.At inference, clustering, aggregation, and prediction run in a single forward pass with moderate overhead relative to the naive decoder.Source: A. Learning Procedure · p. 6

B. Complexity Analysis

The complexity analysis accounts for MIDA's clustering, attention, feed-forward, gating, and item-scoring operations. Its per-session complexity includes both sequence processing and full-vocabulary scoring.

  • Clustering costs O(L^2·d/k) for similarity computation with stride k = 4, while selected-neighbor attention costs O(L·e·d) and the feed-forward network adds O(L·d^2).Source: B. Complexity Analysis · p. 6
  • Preference aggregation adds O(L·d^2) for position-aware attention, O(d) for gating, and O(d·|V|) for scoring items.Source: B. Complexity Analysis · pp. 6–7
  • MIDA's overall per-session complexity is O(L^2d/k + Ld^2 + d|V|), with full-vocabulary score computation dominating because L is typically small and d ≪ |V|.Source: B. Complexity Analysis · p. 7

V. EXPERIMENTS

The experiments are designed to assess recommendation accuracy, efficiency, session-length behavior, component contributions, parameter sensitivity, representation alignment, and cluster coherence. The evaluation varies decoders while holding encoders fixed, across four datasets and several backbone architectures.

V. EXPERIMENTS

  • The experiments ask whether MIDA improves recommendation accuracy and efficiency, performance across session lengths, component contributions, parameter sensitivity, representation alignment, cluster coherence, and benefits by semantic diversity.Source: §V · p. 7

A. Datasets and Baseline Models

  • Evaluation covers four real-world datasets, filters out sessions shorter than three interactions and items with fewer than five occurrences, and uses chronological 8:1:1 splits.For Yoochoose, the experiments use the most recent 1/64 fraction of sequences.Source: A. Datasets and Baseline Models · p. 7
  • MIDA replaces decoders across RNN, graph, and Transformer backbones and is compared with Naive, RL, Ms-PoE, AttenMixer, and IDEA alternatives.The controlled evaluation holds each encoder fixed and varies only the decoder; metrics include Recall@K, MRR@K, training time, and full-test-set latency.Source: A. Datasets and Baseline Models · pp. 7–8

B. Implementation Details

MIDA replaces decoder components while holding backbone encoders and training settings fixed, enabling decoder-side comparisons across heterogeneous architectures. Across these comparisons, it improves performance over decoder baselines while adding moderate inference cost and remaining faster than AttenMixer.

B. Implementation Details

  • MIDA replaces only each backbone’s decoder while preserving its encoder parameters, configuration, and training schedule.The experiments use fixed embedding dimension, batch size, and initialization settings; results average five runs with significance testing against the best baseline.Source: B. Implementation Details · p. 8

C. RQ1: Accuracy and Efficiency Comparison

D. RQ2: Performance across Varying Session Lengths

MIDA’s gains generally grow with session length, and ablations show contributions from clustering, both intent queries, adaptive selection, and gradient decoupling. The long-session groups are partitions of the original test sets, not additional datasets.

D. RQ2: Performance across Varying Session Lengths

  • On Yoochoose, MIDA’s R@5 improvement rises from 0.85% for 1–5-item sessions to 3.43% for 16+ sessions, with gains generally increasing as sessions grow longer.Across datasets, improvement magnitude varies; RetailRocket has the strongest reported MRR@5 gains, while Diginetica shows stable R@5 improvements.Source: D. RQ2: Performance across Varying… · p. 10, Table VIII · p. 10

E. RQ3: Ablation Study

  • Removing Dynamic Multi-Interest Clustering lowers Diginetica R@5 from 31.47 to 30.13 and M@5 from 18.77 to 18.09.Source: E. RQ3: Ablation Study · p. 10
  • Removing the stable query lowers Yoochoose R@5 from 46.21 to 45.14, while removing the plastic query lowers it to 44.80.The corresponding M@5 scores fall from 26.31 to 25.37 without the stable query and to 25.14 without the plastic query.Source: E. RQ3: Ablation Study · p. 10
  • A learned per-session threshold performs the same as or slightly below MIDA, with its session statistics offering little additional signal and extra parameters carrying mild overfitting risk.Source: E. RQ3: Ablation Study · pp. 10–11
  • Using a fixed single-item selection instead of threshold-based aggregation lowers Yoochoose R@5 from 46.21 to 41.83, a relative decrease of over 9.4%.Source: E. RQ3: Ablation Study · p. 11
  • Directly stacking the components without gradient detachment lowers Yoochoose R@5 from 46.21 to 45.21, consistent with a benefit from decoupling gradient flows.Source: E. RQ3: Ablation Study · p. 11

F. RQ4: Sensitivity to Hyperparameter τ

MIDA’s learned similarity threshold performs robustly across a broad range, while analyses find target-alignment gains, coherent clusters, and gate behavior associated with session structure. Its global threshold also yields different effective cluster coverage across dispersion groups.

F. RQ4: Sensitivity to Hyperparameter τ

  • MIDA’s performance is relatively stable over a broad range of τ, although optimal thresholds vary by dataset; learned values converge to 0.85, 0.82, and 0.88 for Diginetica, Yoochoose, and RetailRocket.Low τ includes dissimilar items, while high τ excludes relevant items; the learned values closely approximate grid-search optima.Source: F. RQ4: Sensitivity to Hyperparameter τ · p. 11, Figure 3 · p. 12
  • On Diginetica, MRR@5 is 18.61, 18.71, 18.77, and 18.52 for strides k=1, 2, 4, and 8, respectively; k=4 reduces similarity-matrix construction cost fourfold versus k=1.Source: F. RQ4: Sensitivity to Hyperparameter τ · p. 11

G. RQ5: Representation Alignment and Cluster Analysis

  • MIDA moves session representations closer to the target item across all four datasets, including long sessions, with stronger gains when the target-aligned historical item occurs early.The comparison measures the difference in cosine similarity between the final session representation and the ground-truth next-item embedding.Source: G. RQ5: Representation Alignment and… · p. 11, Table X · p. 11
  • On Diginetica, sessions average 1.83 non-singleton clusters and cluster category purity of 0.67, above the 0.26 uniform-random expectation.Source: G. RQ5: Representation Alignment and… · p. 11
  • Across four datasets, average intra-cluster raw-embedding similarity is 0.83 ± 0.04, exceeding the 0.31 ± 0.07 across-cluster baseline.Source: G. RQ5: Representation Alignment and… · p. 11
  • The learned gating coefficient is −0.47 on Diginetica and has the expected negative sign across all four tested backbones, shifting larger-cluster sessions toward stability.Source: G. RQ5: Representation Alignment and… · p. 11
  • On Diginetica, mean gate values rise from 0.41 to 0.52 to 0.63 across low-, medium-, and high-dispersion groups, indicating a shift toward plasticity as dispersion increases.At the global τ⋆=0.85, effective cluster coverage falls from 0.72 to 0.41 to 0.21 across those dispersion groups.Source: G. RQ5: Representation Alignment and… · pp. 11–12

H. RQ6: Behavioral Analysis: When Does MIDA Help Most?

MIDA’s gains increase with semantic dispersion across four datasets, and the conclusion reports consistent advantages across encoder architectures, especially on longer and more diverse sessions. The paper identifies overhead on extremely long sessions and the two-component intent model as limitations.

H. RQ6: Behavioral Analysis: When Does MIDA Help Most?

  • Across all four datasets, MIDA’s improvement over the Naive decoder increases monotonically with semantic dispersion, reaching 4.6%–19.7% MRR@5 gains for highly dispersed sessions versus 0.7%–5.2% for cohesive sessions.Dispersion is defined from pairwise cosine similarity of raw item embeddings; higher values indicate more diverse item semantics.Source: H. RQ6: Behavioral Analysis: When Does… · p. 12, Table XI · p. 12
  • In low-dispersion sessions, clusters tend to merge similar items and MIDA degrades gracefully toward average pooling; in high-dispersion sessions, clustering and stability–plasticity separation capture distinct interests.Source: H. RQ6: Behavioral Analysis: When Does… · p. 12
  • The behavioral analysis links long-session gains to higher semantic dispersion, as longer sessions tend to contain more diverse items.Source: H. RQ6: Behavioral Analysis: When Does… · p. 12

VI. CONCLUSION

  • Across four benchmarks, MIDA consistently outperforms strong baselines across encoder architectures, with particularly significant gains on longer and semantically diverse sessions.The conclusion also cites ablations, cluster-coherence analysis, hyperparameter sensitivity, and behavioral analysis as validation of the design.Source: §VI · pp. 12–13
  • Pairwise within-session similarities may add non-trivial overhead on extremely long sessions, and the stability–plasticity decomposition assumes a two-component intent model.Category-purity validation is limited to Diginetica because it has the relevant annotations.Source: §VI · p. 13

Related papers

Loading 2610.34192v1…