PaperlayerView exact revision on arXiv

Source-linked AI summary

BehaveRec: When Does Long-Term Behavioral History Help Sequential Recommendation? A Study with Session, History, and Community Signals

Aysenur Inan, Reza Yousefi Maragheh, Jayesh Uddhav Kudase, Selene Xu, Kaushiki Nag, Jianpeng Xu, Sushant Kumar

arXiv:2610.36239v1cs.AIcs.IR

TL;DR

Session-only models cannot distinguish users who share the same short-term session. BehaveRec combines session, long-term history, and community signals; its history-stratified results show a growing advantage with richer histories, supporting a roughly 20-interaction deployment point for category-level intent prediction.

  • Problem

    Session-only models assign identical predictions to users with identical short-term sessions, despite differences in who they are.

    Source: §I · p. 1

  • Method

    BehaveRec combines a causal session transformer, a 90-day bidirectional history encoder, and GraphSAGE over a behavioral-similarity graph, fused by a learned gate.

    Source: C. Behavioral Neighborhood Graph · p. 3, §VII · p. 9

  • Results

    History-stratified results show a monotonic relationship between history length and BehaveRec's advantage, corroborated across industrial and public datasets.

    Source: §VII · pp. 9–10

  • Takeaways & Limitations

    The authors identify roughly 20 interactions per user as a practical deployment point for category-level intent prediction.

    Source: §VII · p. 10

    The industrial headline R@5 gain comes from one training run, with variance estimated indirectly from a five-seed MovieLens sweep.

    Source: M. Limitations · p. 8

Abstract

from arXiv · show

Sequential recommendation models typically predict a user's next action from recent session interactions, which limits personalization when short-term signals are weak or ambiguous. Rather than assuming that incorporating longer-term history and community signals is universally beneficial, we study when it actually helps. We propose BehaveRec, a three-signal framework combining a causal session transformer, a bidirectional User History Encoder over a 90-day window, and a GraphSAGE module over a user-user graph built from behavioral embedding similarity rather than item co-occurrence; a learned gate blends the three signals per user. On an industrial dataset of 137M sequences from 12.6M users, BehaveRec improves R@5 by +3.2% over SASRec, with a larger-capacity SASRec baseline (more parameters than BehaveRec) still trailing by +2.5%, ruling out capacity as the explanation. More importantly, a user-level stratification by history length reveals a clear dose-response relationship: BehaveRec's advantage is negligible below roughly 10 interactions and grows monotonically with history richness, reaching +0.017 R@5 for users with 101+ events. This pattern is corroborated across one industrial and three public datasets spanning history-rich to history-sparse regimes, where session-only models remain preferable. These headline results, and the resulting deployment threshold, are established for category-level intent prediction; a preliminary, single-run check on the full item-level vocabulary yields a directionally mixed result and is scoped as an open question rather than a claim. We use these results to derive a practical deployment rule -- roughly 20 interactions per user -- for when long-term history and behavioral-graph signals are worth their added complexity. Despite this added complexity, BehaveRec remains lightweight: it uses 5.67M parameters and trains on a single GPU in under two days.

I. INTRODUCTION

Session-only recommendation cannot distinguish users with identical recent activity, motivating long-term personal history and community signals. BehaveRec combines three signals, and the paper studies when their added complexity is worthwhile.

A. Limitations of Existing Approaches

  • Session-only models cannot personalize users with identical short-term sessions, while LLM-based approaches are impractical at 137 million training sequences.Long-term behavior and behavioral communities provide complementary signals about users’ preferences and transition patterns.Source: §I · p. 1, A. Limitations of Existing Approaches · p. 2

B. Our Approach

  • BehaveRec combines a causal session transformer, a 90-day User History Encoder, and a behavioral GraphSAGE signal through a learned gate.The history-similarity graph connects users by learned history embeddings rather than item co-occurrence; InfoNCE aligns session and history representations.Source: B. Our Approach · p. 2

C. Contributions •

  • BehaveRec’s advantage over a session-only baseline is negligible below ∼10 history interactions and grows monotonically, supporting a practical deployment threshold of ≈20 interactions per user.The study also reports cases where the added signals do not help, including sparse-history Amazon benchmarks and a directionally mixed full item-level MovieLens evaluation.Source: C. Contributions • · p. 2
  • A controlled ablation found that behavioral-similarity edges outperform item-co-occurrence edges, while a larger SASRec baseline indicates most gains are architectural rather than capacity-driven.BehaveRec uses 5.67M parameters and trains on a single A100 GPU in under two days.Source: C. Contributions • · p. 2

II. RELATED WORK

Prior work spans session-based sequential models, graph-based recommendation, LLM-based approaches, and industrial long-term user modeling. BehaveRec distinguishes itself by using one history embedding for both user-level fusion and behavioral graph construction.

A. Sequential Recommendation

  • SASRec uses causal self-attention, while BERT4Rec adds bidirectional attention; both rely on the current session, so identical recent sequences yield identical predictions.The paper also notes GRU4Rec [7] and Caser as earlier recurrent and convolutional approaches.Source: A. Sequential Recommendation · p. 2

B. Graph-Augmented Recommendation

  • Unlike item-co-occurrence or user-item-overlap graphs, BehaveRec connects users by similarity between learned 90-day history embeddings.GraphSAGE provides inductive aggregation suited to production systems with continuously arriving users.Source: B. Graph-Augmented Recommendation · p. 2

C. LLM-Based Recommendation

  • TALLRec and CoLLM are described as impractical at the scale of 137M training sequences and tens of millions of users.BehaveRec has 5.67M parameters and runs on a single GPU.Source: C. LLM-Based Recommendation · p. 2

D. Industrial Long-Term User Modeling and User-User Graphs

  • Industrial long-/short-term systems use auxiliary retrieval over raw item vocabularies, whereas BehaveRec encodes a 90-day intent history into one embedding reused for fusion and graph construction.The paper does not claim established superiority because these systems are not directly portable to its intent-cluster task and are not benchmarked head-to-head.Source: D. Industrial Long-Term User Modeling… · pp. 2–3

E. User History Modeling

  • The User History Encoder uses a bidirectional transformer with a [USER] aggregation token and InfoNCE [6] to align session and history embeddings for the same user.Source: E. User History Modeling · p. 3

A. Intent Clusters

The setup represents interactions as intent-cluster and activity pairs, then uses session, history, and behavioral-neighbor inputs for prediction. BehaveRec combines these signals through a learned gate.

A. Intent Clusters

  • The production setting uses 2,371 intent clusters created through an LLM-assisted pipeline that generates product intents, clusters them, and names the clusters.The authors state that the clustering pipeline itself is not a contribution.Source: A. Intent Clusters · p. 3

B. Notation

  • Each interaction is an intent-cluster and activity-type pair, with a session of up to 20 events and a 90-day history of up to 100 events.Source: B. Notation · p. 3

C. Behavioral Neighborhood Graph

D. Task Definition

  • The prediction task takes the user’s session, 90-day history, and behavioral neighborhood as inputs.Source: D. Task Definition · p. 3

IV. BEHAVEREC ARCHITECTURE

  • BehaveRec encodes three complementary signals and fuses them with a learned gate; all encoders share a cluster and activity embedding table.Source: §IV · p. 3

A. Shared Embeddings

The architecture shares cluster and activity embeddings across its encoders. A causal session transformer produces a short-term representation, while a bidirectional history transformer aggregates longer-term behavior.

A. Shared Embeddings

  • The cluster embedding table has dimension 256 and is shared with the output classifier.The activity embedding table has dimension 16.Source: A. Shared Embeddings · p. 3

B. Session Encoder (hshort)

  • A 4-layer causal Transformer encodes the session, and its last non-padding position produces the layer-normalized 256-dimensional hshort representation.The encoder uses 4 heads and an FFN dimension of 512.Source: B. Session Encoder (hshort) · p. 3

C. User History Encoder (hlong)

  • A 4-layer bidirectional Transformer prepends a learnable [USER] token that attends to every history event to aggregate recurring patterns.The same history encoder weights encode neighbor histories in a shared 256-dimensional space.Source: C. User History Encoder (hlong) · p. 3

D. Behavioral Graph Construction

BehaveRec constructs a behavioral graph from long-term user embeddings, aggregates neighbor signals, and learns to fuse them with session and history signals.

D. Behavioral Graph Construction

  • Top-(K+1) approximate nearest neighbors are retrieved from L2-normalized hlong embeddings, with self-matches discarded and K = 20.Graph construction takes under 40 minutes on one GPU for the full user population and is periodically recomputed as embeddings drift.Source: D. Behavioral Graph Construction · p. 3

E. GraphSAGE Aggregation (hgraph)

F. ThreeWayGate

  • The gate upweights session for long sessions, history for cold sessions with rich history, and neighbors for tight behavioral communities.It fuses the session, history, and neighbor signals into hfused.Source: F. ThreeWayGate · p. 3, Figure 1 · p. 4

G. Weight-Tied Classifier

BehaveRec ties its classifier weights to cluster representations, uses label-smoothed cross-entropy and InfoNCE alignment, and trains with a cached, low-latency serving path.

G. Weight-Tied Classifier

  • Weight tying halves parameters attributable to the cluster vocabulary and enforces consistency between input and output cluster representations.Source: G. Weight-Tied Classifier · p. 4

H. Training Objective

  • Training uses label-smoothed cross-entropy with ϵ = 0.1 and InfoNCE alignment with temperature τ = 0.5 and batch size B.Source: H. Training Objective · p. 4

I. Implementation Details

  • The model has 5.67M parameters and uses d = 256, four Transformer layers, four heads, and a 512-unit FFN.V5 trained for 50 epochs, with its best checkpoint at epoch 46; reported test R@5 was 0.7201.Source: §I · p. 4
  • Cached histories and neighbor embeddings keep total serving latency below 3 ms at P99, against a <100 ms SLA.The offline graph is recomputed every 24 hours in under 40 minutes on one GPU.Source: §I · pp. 4–5

V. EXPERIMENTS

On the industrial test set, BehaveRec V5 beats SASRec in R@5, with contributions from history and graph signals; results also distinguish continuation from shift sequences.

A. Datasets

  • The industrial dataset contains 137M training sequences from 12.6M users, with average session length 6.6 events and average raw 90-day history length 119.History inputs are capped at 100 recent events, while stratification uses raw pre-truncation history counts.Source: A. Datasets · p. 5
  • Public benchmarks span rich-history MovieLens ML-1M (165.6 events on average) and sparse-history CDs & Vinyl and Electronics; all use leave-one-out evaluation.The industrial test set is held out by time.Source: A. Datasets · p. 5

B. Baselines

  • Comparisons include SASRec and BERT4Rec as literature baselines and V4, which adds UHE but disables the graph; significance tests use user-level bootstrap CIs.The authors note that sequence-level independence understates uncertainty because about eight sequences per user are correlated.Source: B. Baselines · p. 5, C. Evaluation Metrics and Statistical… · p. 5

C. Evaluation Metrics and Statistical Significance

  • The industrial result is a single training run, while its +0.0226 R@5 gap is about 13× the MovieLens seed-to-seed delta standard deviation of ±0.0017 R@5.The authors flag single-run uncertainty as a limitation.Source: C. Evaluation Metrics and Statistical… · p. 5
  • +0.0226 R@5 (+3.2% relative) for V5 vs. SASRec, with a user-level 95% CI of [0.0219, 0.0233].This is the canonical headline effect size; UHE accounts for 83% of the gain.Source: C. Evaluation Metrics and Statistical… · p. 5
  • V4 vs. SASRec gains +0.0188 R@5 (+2.7% relative), while V5 vs. V4 gains +0.0038 R@5 (+0.5%) from the graph.The graph comparison has a 95% CI of [0.0020, 0.0060] and swaps edge definitions on a shared backbone rather than using independent retraining.Source: C. Evaluation Metrics and Statistical… · p. 5

D. Repetition Bias Analysis

  • On 2,982,331 test sequences, Repeat-Last R@5 = 0.4548, while V5 reaches R@5 = 0.6932 on shift sequences where Repeat-Last scores 0%.Overall, V5 outperforms Repeat-Last by +58.3% R@5.Source: D. Repetition Bias Analysis · p. 5

E. Main Results

BehaveRec’s advantage over session-only models is not explained by model capacity and grows with users’ history richness. Cross-domain results show that history-rich settings favor BehaveRec, while sparse-history Amazon settings favor SASRec.

E. Main Results

  • +3.2% over SASRec: BehaveRec V5 reaches R@5=0.7201, while larger SASRec-Large reaches 0.7024 and trails V5 by 2.5%.SASRec-Large has 14% more parameters than BehaveRec; the authors attribute 78% of BehaveRec’s total gain to the UHE and graph mechanisms.Source: E. Main Results · p. 5
  • +0.5%: adding behavioral-similarity graph edges improves on UHE-only V4; BERT4Rec’s R@5=0.7032 is 2.3% below BehaveRec V5.The behavioral-similarity graph outperforms item co-occurrence edges by +0.0013 R@5, though this comparison swaps edge definitions at inference rather than retraining each variant.Source: E. Main Results · pp. 5–6

F. User-Level History Stratification: A Practical Deployment Threshold

  • On the fixed industrial domain and vocabulary, BehaveRec’s advantage rises with 90-day history length, crossing parity around 6–10 events and reaching +0.017 R@5 at 101+ events.The stratification controls for dataset-level confounds; the figure reports the same dose-response data as Table III.Source: F. User-Level History Stratification:… · p. 6, Figure 2 · p. 6
  • The authors choose approximately 20 interactions as a practical operating point, not a discovered constant; full return-on-latency analysis depends on each platform’s per-bin traffic share.The behavioral graph’s sub-0.5ms latency cost buys gains concentrated in history-rich bins.Source: F. User-Level History Stratification:… · p. 6

G. Cross-Domain Corroboration

  • Across domains, history-rich industrial and MovieLens datasets favor BehaveRec (+3.2% and +0.67% R@5), whereas sparse Amazon CDs & Vinyl and Electronics favor SASRec (4.6% and 15.9% relative R@5).Amazon results use category-level prediction; the caption notes Amazon average histories of 14–17 events and that MovieLens is a single run.Source: G. Cross-Domain Corroboration · p. 6, Table II · p. 7
  • Amazon datasets are dominated by users with 1–10 events, where added model complexity does not offset SASRec’s simpler inductive bias.The authors conclude BehaveRec is not universally superior and that practitioners can estimate expected gains from their platform’s history-length distribution.Source: G. Cross-Domain Corroboration · p. 7

H. Additional Practitioner-Relevant Knobs

Additional analyses examine how gains vary with session and history length, whether they transfer to full-item prediction, and which graph edges work better. They find that gains concentrate on ambiguous transitions, item-level results are mixed, and behavioral-similarity edges outperform item co-occurrence edges in the reported comparison.

H. Additional Practitioner-Relevant Knobs

  • +0.0371 R@5 on shift sequences versus +0.0052 R@5 on continuation sequences; BehaveRec’s advantage is roughly 7× larger when the target differs from the last cluster.The weighted deltas recover the +0.0226 R@5 aggregate result; on continuation sequences the last event already predicts the target, while history does most of the work on shifts.Source: H. Additional Practitioner-Relevant… · pp. 7–8
  • +0.0251 R@5 is the largest joint-grid gain, for cold sessions with 51+ history events; the only negative cell is −0.0009 R@5 for warm sessions with 1–10 events.For cold sessions, the gap rises from +0.0051 to +0.0182 to +0.0251 as history grows; the table jointly stratifies by session and 90-day history length and reports one run without confidence intervals.Source: H. Additional Practitioner-Relevant… · p. 8, Table V · p. 8

I. Item-Level Evaluation on the Full MovieLens Vocabulary

  • On full-vocabulary MovieLens, BehaveRec V4 improves HR@10 by +1.3% and NDCG@10 by +0.3% relative, but underperforms SASRec on R@5 by −2.5% relative.The comparison uses 3,706 movies; the authors call it preliminary, note the lack of a multi-seed estimate, and scope headline claims to category-level intent prediction.Source: §I · p. 8

J. Graph Construction Ablation

  • Behavioral-embedding similarity edges outperform item co-occurrence edges by +0.0013 R@5 (95% CI [0.0006, 0.0020], p<0.05) in the reported graph comparison.The edge definitions are swapped at inference on a shared trained backbone rather than independently retrained from scratch.Source: J. Graph Construction Ablation · p. 8

K. Ablation Study

The ablations identify the User History Encoder as BehaveRec’s largest contributor, while five-seed MovieLens results show a consistent gain over SASRec. The authors also qualify the evidence with limits on independent retraining, item-level replication, threshold choice, and uncertainty for additional stratifications; the system is reported live on two recommendation surfaces.

K. Ablation Study

  • Removing the User History Encoder causes the largest ablation drop (−0.0226 R@5); secondary mechanisms each reduce R@5 by −0.0074 to −0.0077, while removing the graph gives the smallest drop (−0.0038).The authors treat the near-equality among InfoNCE, ThreeWayGate, and weight tying as evidence that none is individually dominant, not as a mechanistic claim about redundancy or ensembling.Source: K. Ablation Study · p. 8, Table VIII · p. 9

M. Limitations

  • The industrial +0.0226 R@5 headline gain comes from one training run; its variance estimate is a proxy based on five MovieLens seeds, not a direct industrial measurement.The reported industrial gap is approximately 13–25× the MovieLens standard-deviation spread.Source: M. Limitations · p. 8
  • Graph-edge variants reuse a trained backbone with edges swapped at inference, so the +0.0013 R@5 difference is not a fully controlled comparison with independent retraining.The authors describe it as evidence that the backbone transfers well to behavioral edges.Source: M. Limitations · pp. 8–9

L. MovieLens ML-1M: Multi-Seed Variance

  • +0.0061 mean R@5 (std 0.0017): BehaveRec V5 outperforms SASRec on all five MovieLens ML-1M seeds.The reported absolute values differ slightly from the single-run table, but the direction and approximate magnitude are consistent.Source: Table IX · p. 9

M. Limitations

  • The full-item MovieLens check is preliminary and single-run, with no multi-seed estimate to distinguish its mixed result from noise..Source: M. Limitations · p. 9
  • The ≈20-interaction threshold is a practical operating choice, set where per-bin gain first reaches ≥0.004 R@5, rather than a discovered constant.Platforms with different complexity tolerances could choose a different cutoff.Source: M. Limitations · p. 9
  • The additional session-by-history stratifications re-slice one run, so individual grid cells lack confidence intervals from independent retraining..Source: M. Limitations · p. 9

VI. DEPLOYMENT, BUSINESS IMPACT, AND QUALITATIVE ANALYSIS

  • BehaveRec has been live since 2025 on Item Detail Page intent carousels and post-add-to-cart basket-completion recommendations.Carousel titles use the predicted cluster’s human-readable label.Source: §VI · p. 9

A. A/B Test Results

BehaveRec’s gains vary with user history: history-richness evidence supports an operating point near 20 interactions per user, while illustrative cases show how signals contribute in different contexts. Production tests also report adoption and efficiency outcomes beyond in-session conversion.

A. A/B Test Results

  • A 2% new-product-type adoption rate within 10 days and 3.1× iOS GMV-per-impression efficiency versus benchmark carousels suggest gains beyond in-session conversion.Table X presents production A/B test lift versus existing carousel baselines.Source: A. A/B Test Results · p. 9, Table X · p. 10

B. Qualitative Signal Attribution

  • In 10 of 12 illustrative cases the User History Encoder was dominant, while the graph added lift in 8 of 12 cases.The gate upweighted session signal for long unambiguous sessions, history for cold sessions with rich history, and graph signal for tight behavioral communities.Source: B. Qualitative Signal Attribution · p. 9, §VII · p. 10

VII. CONCLUSION

  • A monotonic advantage with history length, corroborated across four datasets, yields a practical deployment point of approximately 20 interactions per user.The threshold and headline findings are established for category-level intent prediction; a preliminary single-run full item-level evaluation is directionally mixed.Source: §VII · pp. 9–10
  • The full item-level-vocabulary check was preliminary, single-run, and directionally mixed, leaving it an open question beyond the category-level claims.The authors identify multi-seed replication as future work.Source: §VII · p. 10

Related papers

Loading 2610.36239v1…