PaperlayerView exact revision on arXiv

Source-linked AI summary

Disentangling Long and Short-Term Interests for Recommendation

Yu Zheng, Chen Gao, Jianxin Chang, Yanan Niu, Yang Song, Depeng Jin, Yong Li

arXiv:2202.13090v1cs.IR

TL;DR

Recommendation must account for stable long-term and dynamic short-term interests, yet interest labels are unavailable and existing approaches can leave the two entangled. CLSR uses self-supervised contrastive proxies, separate encoders, and adaptive fusion; on two real-world datasets, it outperforms SOTA methods and achieves stronger counterfactual disentanglement.

  • Problem

    Long- and short-term interests differ, but interaction logs lack labels for separating them, and existing methods may leave the interests entangled.

    Source: §1 · p. 2

  • Method

    CLSR uses separate encoders, interaction-derived proxies, and contrastive self-supervision to disentangle long- and short-term interests, then adaptively fuses them for prediction.

    Source: §1 · p. 2

  • Results

    AUC and GAUC improved by over 0.02, and NDCG by over 10.7%, as CLSR outperformed SOTA methods on two real-world datasets, with stronger counterfactual disentanglement.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Counterfactual evaluations indicate that CLSR achieves stronger disentanglement of long- and short-term interests than SOTA methods.

    Source: §1 · p. 2

    Long- and short-term interests may overlap, so CLSR does not impose a constraint forcing the learned factors to be dissimilar.

    Source: §3.2.4 · p. 5

Abstract

from arXiv · show

Modeling user's long-term and short-term interests is crucial for accurate recommendation. However, since there is no manually annotated label for user interests, existing approaches always follow the paradigm of entangling these two aspects, which may lead to inferior recommendation accuracy and interpretability. In this paper, to address it, we propose a Contrastive learning framework to disentangle Long and Short-term interests for Recommendation (CLSR) with self-supervision. Specifically, we first propose two separate encoders to independently capture user interests of different time scales. We then extract long-term and short-term interests proxies from the interaction sequences, which serve as pseudo labels for user interests. Then pairwise contrastive tasks are designed to supervise the similarity between interest representations and their corresponding interest proxies. Finally, since the importance of long-term and short-term interests is dynamically changing, we propose to adaptively aggregate them through an attention-based network for prediction. We conduct experiments on two large-scale real-world datasets for e-commerce and short-video recommendation. Empirical results show that our CLSR consistently outperforms all state-of-the-art models with significant improvements: GAUC is improved by over 0.01, and NDCG is improved by over 4%. Further counterfactual evaluations demonstrate that stronger disentanglement of long and short-term interests is successfully achieved by CLSR. The code and data are available at https://github.com/tsinghua-fib-lab/CLSR.

1 INTRODUCTION

Users have stable long-term and dynamic short-term interests, but existing approaches may leave them entangled without explicit supervision. CLSR uses self-supervision to disentangle the two and adaptively fuse them, outperforming state-of-the-art methods in experiments and counterfactual evaluations.

1 INTRODUCTION

  • Earlier hybrid methods combined collaborative-filtering and sequential models but lacked explicit supervision to ensure their learned interests remained disentangled.The learned long- and short-term interests could therefore be entangled.Source: §1 · p. 1
  • Long-term interests reflect stable overall preferences, whereas short-term interests evolve with recent interactions, and behavior logs lack labels to distinguish them.The paper argues that a unified representation is insufficient to capture these different dynamics.Source: §1 · pp. 1–2
  • Because the relative importance of the two interests varies across user-item interactions, CLSR adaptively fuses them for prediction.The paper gives continuous browsing of similar items and switching to different items as cases where short- and long-term interests, respectively, matter more.Source: §1 · p. 2
  • CLSR uses separate encoders and contrastive self-supervision, comparing their interest representations with proxies extracted from users’ entire histories and recent interactions.The proxies provide supervision without manually labeled interest data.Source: §1 · p. 2
  • AUC and GAUC improved by over 0.02, and NDCG by over 10.7%, as CLSR outperformed SOTA methods on two real-world datasets.Counterfactual evaluations also found steadily stronger disentanglement of long- and short-term interests than in SOTA methods.Source: §1 · p. 2

2 PROBLEM FORMULATION

The problem takes users’ historical interaction sequences as input and seeks a model that predicts clicks while considering both long- and short-term interests.

2 PROBLEM FORMULATION

  • Each user's interaction sequence is a timestamp-ordered list of items indexed within the item catalog.The catalog contains N items.Source: §2 · p. 2
  • The task takes historical interaction sequences for all users as input.Source: §2 · p. 2
  • The output is a predictive model estimating whether a user will click an item while considering both long- and short-term interests.Source: §2 · p. 2

3 METHODOLOGY

CLSR models stable long-term and dynamic short-term interests separately, then combines them to predict interactions. Self-supervised contrastive tasks use interaction-derived proxies to guide the two interest representations.

3.1 User Interests Modeling

  • CLSR frames user-interest modeling as long-term representation, short-term evolution, and interaction prediction mechanisms.Long-term interests draw on the full history, while short-term interests evolve with recent interactions.Source: §3.1 · p. 3

3.2 Our Self-supervised Implementation

  • Separate attentive encoders capture long-term and short-term interests, with the short-term encoder using recurrent sequence patterns.The long-term encoder aggregates the interaction history; the short-term encoder models evolving sequential patterns.Source: §3.2.1 · p. 3, §3.2.2 · p. 4, §3.2.3 · p. 4
  • CLSR contrasts encoder outputs with proxies from the full history and recent interactions to supervise the corresponding interests.Four symmetric tasks encourage each representation to match its corresponding proxy; the design allows long- and short-term interests to overlap.Source: §3.2.4 · pp. 4–5
  • An attention-based fusion model uses the historical sequence and target item to adapt the importance of each interest for prediction.The history is compressed with a GRU, and the fusion weights depend on the history, target item, and interest representations.Source: §3.2.5 · p. 5

4 EXPERIMENTS

Across Taobao and Kuaishou, the experiments compare CLSR with state-of-the-art recommenders and test its disentanglement and model components. Results cover overall recommendation, counterfactual interest interventions, and ablations.

4.1 Overall Performance Comparison (RQ1)

  • GAUC improves by about 0.005 on Taobao and 0.01 on Kuaishou against SOTA methods; Taobao NDCG improves by about 5%.The reported overall comparison covers both datasets.Source: Table 2 · p. 6, §4.1 · p. 6

4.2 Study on Disentanglement of Long and Short-Term Interests (RQ2)

  • CLSR outperforms SLi-Rec in every one-side and full-interest case; combining both interests improves performance over either side alone.Reported AUC gains include about 0.03 for short-term and full interests on Taobao, and 0.1, 0.2, and 0.4 for long-term, short-term, and full interests on Kuaishou.Source: §4.2.1 · p. 6, Figure 3 · p. 6
  • Under shuffled histories, AUC falls by over 0.04 for click prediction and rises by about 0.02 for purchase/like prediction; CLSR still beats SLi-Rec.The shuffle protocol removes short-term interests; CLSR also improves click AUC by over 0.04 and purchase MRR by over 30% against SLi-Rec.Source: Table 4 · p. 7, §4.2.2 · p. 7
  • For click and purchase/like predictions, CLSR beats SLi-Rec across behaviors, and its long-term attention weight changes more consistently with behavior type.On Taobao, CLSR’s α for purchase is about 4% higher than for click; on Kuaishou, its like-versus-click increment is +9.06%, versus +3.91% for SLi-Rec.Source: §4.2.2 · p. 7, Table 3 · p. 7
  • As available history grows, purchase prediction improves markedly while click prediction rises more slowly; long-term-only click accuracy barely changes.Under truncation, the different click and purchase trends support the authors’ claim that long-term representations capture desired interests while distilling short-term interests.Source: Figure 5 · p. 7, §4.2.2 · p. 7

4.3 Ablation and Hyper-parameter Study (RQ3)

  • On Kuaishou, removing contrastive tasks drops CLSR GAUC by over 0.01; β=0.1 is optimal, and self-supervision also improves DIEN.The authors report that too-large β can conflict with the main interaction-prediction task and lower accuracy.Source: §4.3.1 · p. 8, Figure 6 · p. 8
  • Adaptive fusion outperforms every tested fixed α value across the two datasets.The adaptive attention network aggregates interests according to the target item and historical sequence.Source: §4.3.2 · p. 8, Figure 7 · p. 8

5 RELATED WORK

Prior approaches distinguish long- and short-term interests but lack supervision guaranteeing that learned representations are disentangled. CLSR uses self-supervision to align learned representations with interest proxies.

5 RELATED WORK

  • Earlier methods explicitly modeled long- and short-term interests, but imposed no supervision to ensure their representations were disentangled.Examples include combining matrix factorization with an RNN, or asymmetric SVD with a modified LSTM.Source: §5 · p. 8
  • Self-supervised recommendation methods based on mutual-information maximization or latent intention prototypes did not address differences between long- and short-term interests.The passage identifies Zhou et al. and Ma et al. as examples.Source: §5 · p. 8
  • CLSR contrasts learned interest representations with proxies extracted from interaction sequences to provide self-supervision for disentanglement.The related work also notes that other disentanglement approaches often lack specific semantics for their learned representations.Source: §5 · p. 8

6 CONCLUSION AND FUTURE WORK

The conclusion presents CLSR as a self-supervised contrastive framework for disentangling long- and short-term interests. Experiments and counterfactual evaluations on two large-scale datasets show gains over SOTA baselines, and future work includes alternative encoders or proxies and industrial deployment.

6 CONCLUSION AND FUTURE WORK

  • CLSR consistently outperforms SOTA baselines on two large-scale datasets, and counterfactual evaluations support its stronger disentanglement of long- and short-term interests.The authors report that unsupervised modeling can entangle the two aspects and lead to poorer performance.Source: §6 · p. 8
  • The authors identify exploring other encoder or proxy designs as possible extensions of CLSR.They describe the framework as highly general.Source: §6 · p. 8
  • Deploying CLSR to industrial systems is identified as future work.Source: §6 · p. 8

A.1 Datasets

The experiments use million-scale real-world e-commerce and short-video interaction datasets. Both are filtered with 10-core settings and use click data, with dataset-specific time-based train, validation, and test splits.

A.1 Datasets

  • The study uses a million-scale public e-commerce dataset and an industrial short-video dataset collected from real-world applications.The datasets are Taobao and Kuaishou, respectively.Source: §A.1 · p. 11
  • Taobao interactions span November 25–December 3, 2017; click data and 10-core filtering are used, with December 1 and earlier for training, December 2 for validation, and December 3 for testing.The source includes click, cart, and purchase behaviors, but the experiments use clicks.Source: §A.1 · p. 11
  • Kuaishou interactions span October 22–28, 2020; click data and 10-core filtering are used, with the first six days for training and the final day split at noon for validation and testing.The source logs include clicks, likes, follows, and forwards.Source: §A.1 · p. 11

A.2 Baselines

The baselines cover general, long-term, sequential, and joint long-/short-term recommendation approaches. They include matrix-factorization, attention, graph, convolutional, recurrent, and self-attention methods.

A.2 Baselines

  • General and long-term baselines include NCF, which combines matrix factorization and multilayer perceptrons, DIN, which attends over history using the target item, and LightGCN, which aggregates neighborhood information.Source: §A.2 · p. 11
  • Sequential baselines include Caser’s convolutional patterns, GRU4REC’s recurrent session modeling, DIEN’s attention and GRU, SASRec’s self-attention, and SURGE’s graph convolution and pooling.Caser treats item sequences as images; DIEN extends DIN with interest evolution.Source: §A.2 · p. 11
  • SLi-Rec is the joint long-/short-term baseline, using asymmetric SVD for long-term interests and a modified LSTM for short-term interests.Source: §A.2 · p. 11

A.3 Implementation Details

The implementation uses Microsoft Recommenders with TensorFlow and Adam, plus a two-layer MLP for interaction estimation. Hyperparameters and sequence lengths are selected for the Taobao and Kuaishou datasets.

A.3 Implementation Details

  • Models use Microsoft Recommenders with TensorFlow, the Adam optimizer, and a two-layer MLP with hidden sizes [100, 64] for interaction estimation.Batch normalization and ReLU are enabled for the MLP.Source: §A.3 · p. 11
  • The embedding size is 40, with batch size 500, learning rate 0.001, and L2 regularization weight 1e-6.The selected beta is 0.1.Source: §A.3 · p. 11
  • Maximum sequence lengths are 50 for Taobao and 250 for Kuaishou; l_t is 5 and 10, and k is 3 and 5, respectively.The selected contrastive loss is Ltri for Taobao and Lbpr for Kuaishou; hyperparameters are chosen by grid search.Source: §A.3 · p. 11

A.4 More Studies on the Proposed Method

The studies test proxy choices, interest evolution, encoder and loss alternatives, fusion, attention, discrepancy supervision, counterfactual history length, and complexity. They report accuracy drops when interest evolution or the fusion predictor GRU is removed, alongside several findings about component choices and interest-specific history sensitivity.

A.4 More Studies on the Proposed Method

  • Removing interests evolution significantly decreases accuracy on both datasets.The study identifies a GRU-generated query vector as the mechanism for modeling short-term interest evolution.Source: §A.4 · p. 11, Table 7 · p. 12
  • Using only the last interacted item as the short-term proxy performs worse except for AUC, which the authors attribute to the possibility of a noisy single interaction.The proxy is mean pooling over the recent k interacted items.Source: §A.4 · p. 11
  • Removing the fusion predictor GRU significantly drops recommendation performance, indicating that the importance of long- versus short-term interests depends largely on historical sequence.The GRU takes the historical sequence as input, and its final state feeds the MLP.Source: §A.4 · p. 12, Table 9 · p. 12
  • Time4LSTM outperforms LSTM and GRU on both datasets, while neither BPR nor triplet contrastive loss consistently wins across datasets.The study suggests time-interval features help LS-term interest modeling; CLSR can use different sequential encoders and losses.Source: §A.4 · p. 12, Table 8 · p. 12
  • MLP-based attention significantly outperforms inner-product attention on Taobao, and adding an independent-interest loss reduces AUC by 0.01.The authors attribute the attention result to a nonlinear relation between user interests and historical sequences.Source: §A.4 · p. 12, Table 10 · p. 12
  • On Kuaishou, like AUC improves markedly as available history grows from 50 to 250, whereas click AUC improves little.The paper relates likes to long-term interest and clicks to short-term interest; Figure 8 reports the counterfactual truncate evaluation.Source: §A.4 · p. 12, Figure 8 · p. 12
  • CLSR and SLi-Rec each take 4.1Gb GPU memory on Taobao, while training-time comparisons are reported in Table 11.The comparison uses a single GPU.Source: §A.4 · p. 12, Table 11 · p. 12
Loading 2202.13090v1…