PaperlayerView exact revision on arXiv

Source-linked AI summary

Hierarchical Gating Networks for Sequential Recommendation

Chen Ma, Peng Kang, Xue Liu

arXiv:1906.09217v1cs.IR

TL;DR

Sequential recommendation must infer long-term interests from sparse implicit feedback while capturing short-term interests from recent item histories. The paper proposes HGN with feature and instance gating plus an item-item product module, integrated with BPR, and reports improved performance across five real-world datasets. The results also support the effectiveness of the gating and item-item product components.

  • Problem

    Sequential recommendation must model long-term interests from sparse implicit feedback and short-term interests from several recently accessed items.

  • Method

    HGN integrates feature gating, instance gating, and an item-item product module with matrix factorization and the BPR objective.

  • Results

    HGN achieves the best performance on five datasets across all reported evaluation metrics and outperforms state-of-the-art methods.

  • Takeaways & Limitations

    The reported experiments validate HGN's performance and demonstrate the effectiveness of its gating and item-item product modules for Top-N sequential recommendation.

  • Takeaways & Limitations

    Reported training times exclude negative sampling time.

Abstract

from arXiv · show

The chronological order of user-item interactions is a key feature in many recommender systems, where the items that users will interact may largely depend on those items that users just accessed recently. However, with the tremendous increase of users and items, sequential recommender systems still face several challenging problems: (1) the hardness of modeling the long-term user interests from sparse implicit feedback; (2) the difficulty of capturing the short-term user interests given several items the user just accessed. To cope with these challenges, we propose a hierarchical gating network (HGN), integrated with the Bayesian Personalized Ranking (BPR) to capture both the long-term and short-term user interests. Our HGN consists of a feature gating module, an instance gating module, and an item-item product module. In particular, our feature gating and instance gating modules select what item features can be passed to the downstream layers from the feature and instance levels, respectively. Our item-item product module explicitly captures the item relations between the items that users accessed in the past and those items users will access in the future. We extensively evaluate our model with several state-of-the-art methods and different validation metrics on five real-world datasets. The experimental results demonstrate the effectiveness of our model on Top-N sequential recommendation.

1 INTRODUCTION

Sequential recommendation uses chronological interaction histories, but must capture both long-term preferences and short-term interests. HGN addresses this with hierarchical feature and instance gating plus explicit item-item relations, and is reported to outperform state-of-the-art baselines on five datasets.

  • Motivation: Sequential recommendation predicts future items from chronologically ordered interaction histories whose recent items may influence subsequent behavior.The task must account for both long-term and short-term user interests.
  • Motivation: Existing CNN- and RNN-based methods model short-term sequence dynamics but may overlook representative item features and differences in item importance.The paper identifies these omissions as limitations of treating sequence elements or features insufficiently selectively.
  • Proposed approach: HGN combines feature gating, instance gating, and an item-item product module with matrix factorization and the BPR objective.Feature gating selects user-relevant latent features, while instance gating selects important recent items for prediction.
  • Proposed approach: The item-item product module explicitly models relations between previously accessed items and items users may access in the future.This module complements the hierarchical gating network by representing item relations directly.
  • Results: HGN is reported to improve over state-of-the-art baselines on five real-world datasets and to show the effectiveness of its gating and item-item product modules.The listed contributions also claim fewer parameters and faster training than complex recurrent or convolutional networks.

2 RELATED WORK

Related work covers implicit-feedback recommendation and sequential recommendation, including transition-matrix, Markov-chain, CNN, and RNN approaches. HGN differs by combining feature- and instance-level gating with explicit item-item products.

  • Personalized recommendation: Implicit feedback such as browsing or clicking history is more ubiquitous than explicit feedback such as ratings and is studied as one-class collaborative filtering.Prior work includes conventional and neural network-based methods for learning user-item interactions.
  • Sequential recommendation: Early sequential recommenders use item-item transition matrices or Markov chains to model sequential patterns and personalized short-term transitions.Examples include factorized personalized Markov chains and similarity-based high-order Markov chains.
  • Sequential recommendation: Later methods apply CNNs to item-embedding sequences or RNNs and GRUs to learn short-term contexts and sequential dynamics.These approaches draw on sequence-learning techniques developed in natural language processing.
  • Difference from prior work: HGN adaptively controls item latent features and relevant items through feature-level and instance-level gating.The paper contrasts this with prior methods that omit representative items or neglect feature-level importance.
  • Difference from prior work: HGN also explicitly models relations between past and near-future items through an item-item product, which the paper describes as rarely considered previously.This provides a distinct relation-modeling component beyond gating.

3 PROBLEM FORMULATION

The task uses chronological user-item sequences as sequential implicit-feedback training data. Given an earlier subsequence, the model recommends items from the catalog that appear in the later sequence and evaluates the recommendation list.

  • Sequential recommendation task: Each user's preference is represented by a chronological item sequence Si = (Si1, Si2, ..., Si|Si|).The sequence records items in operating-timestamp order.
  • Sequential recommendation task: Given an earlier subsequence Si1:t, the task recommends a list from N items for each user.The user-item sequence is the training representation for sequential implicit feedback.
  • Sequential recommendation task: The recommendation is evaluated by whether items in the later sequence Sit:|Si| appear in the recommended list.The formulation contrasts the earlier history with future interactions.
  • Notation: Uppercase bold letters denote matrices, lowercase bold letters denote column vectors, and non-bold letters denote scalars.The paper states that major symbols are listed in Table 1.

4 METHODOLOGIES

HGN models sequential recommendation by combining long-term preferences with short-term group-level influence through feature and instance gating, aggregation, and item-item relations. Its gating modules select informative item features and relevant prior items before prediction.

  • Problem formulation: Sequential recommendation predicts subsequent items from successive interaction histories while modeling both long-term and short-term user interests.The input consists of |L| successive items, with future items serving as prediction targets.
  • Hierarchical gating: The hierarchical gating network selects effective latent features and relevant items without complex recurrent or convolutional architectures.The two gating modules model group-level user-item interactions and are described as effective and efficient.
  • Feature gating: HGN uses feature-level gating to select item latent features tailored to a user’s preferences.The feature gate modifies item embeddings using user-specific information before downstream processing.
  • Instance gating: Instance-level gating assigns greater influence to representative items in the input subsequence and largely neglects irrelevant items.This produces a group-level representation after feature and instance selection.
  • Item-item product: The item-item product module captures relations between previously interacted items and future candidate items through inner products of their embeddings.The accumulated scores sum item-item relation contributions from each item in L to other items.
  • Prediction layer: The prediction layer combines matrix-factorization modeling of long-term interests, short-term interest modeling, and item-pair relations.The output item embedding q_j participates in the prediction score, while the three terms correspond to the three modeled effects.

5 EXPERIMENTS

The experiments evaluate HGN on five real-world datasets spanning multiple domains and sparsity levels, using chronological train-validation-test splits and Recall@k and NDCG@k metrics.

  • Datasets: HGN is evaluated on MovieLens-20M, Amazon-Books, Amazon-CDs, Goodreads-Children, and Goodreads-Comics.The datasets cover different domains and sparsity levels.
  • Data Splits: Each user sequence is split chronologically into 70% training, 10% validation, and 20% test interactions.Testing uses both training and validation interactions as input.
  • Evaluation Protocol: All models are run independently five times, and the reported results are averages.
  • Metrics: Model performance is measured with Recall@k and NDCG@k for Top-N recommendation.NDCG@k accounts for the positions of correctly recommended items.

5.3 Methods Studied

The study compares HGN with classical implicit-feedback, session-based, and sequential recommendation methods, including matrix factorization, recurrent, convolutional, and attention-based models.

  • Classical Methods: BPRMF provides a classical Bayesian Personalized Ranking matrix-factorization baseline for implicit feedback.
  • Session-Based Methods: GRU4Rec, GRU4Rec+, and NextItNet represent session-based recommendation methods based on recurrent or convolutional sequence modeling.GRU4Rec+ uses a different loss and sampling strategy from GRU4Rec.
  • Sequential Methods: Caser and SASRec represent sequential methods using convolutional sequence embedding and self-attention, respectively.
  • Proposed Method: HGN learns group-level sequence representations with hierarchical gating and explicitly models item-item relations through an item-item product module.
  • Compared Methods: FMC, FPMC, and Fossil are omitted because Caser and SASRec had already outperformed them.

5.4 Experiment Settings

The experiments use fixed embedding dimensions and tuned model-specific settings, with HGN configured using short sequence and target lengths and trained on GPUs.

  • Common Settings: All models use latent dimension 50, while neural baselines largely follow their original architectural settings.
  • HGN Settings: HGN uses |L| = 5, |T| = 3, embedding size d = 50, learning rate 0.001, λ = 0.001, and batch size 4096.Its hyperparameters are tuned by grid search on the validation set.
  • Hardware: The experiments run in PyTorch on Nvidia GeForce GTX 1080 Ti GPU machines.

5.5 Performance Comparison

HGN achieves the best reported performance across five datasets and all evaluation metrics, with the paper attributing gains to hierarchical selection, long-term modeling, and explicit item-item relations.

  • Overall Results: HGN achieves the best performance on all evaluation metrics across the five datasets.The comparisons are reported in Figures 3–7 and Table 3.
  • Comparison with SASRec: HGN outperforms SASRec by adding feature-level selection and explicit item-item relation modeling alongside instance-level selection.
  • Comparison with Caser: HGN outperforms Caser because Caser models successive-item group representations without considering user-specific item importance.
  • Comparison with Session-Based Methods: HGN outperforms GRU4Rec, GRU4Rec+, and NextItNet, which are described as lacking explicit long-term-interest modeling and equal item treatment in short contexts.
  • Comparison with BPRMF: HGN outperforms BPRMF by combining sequential-dynamics modeling with item-item relation modeling on top of matrix factorization.
  • Additional Observations: Performance is generally lower on sparser datasets, while SASRec and Caser usually outperform the session-based baselines.

5.6 Ablation Analysis

The ablation analysis shows that gating components improve sequential modeling beyond BPR matrix factorization, while HGN also trains faster than compared recurrent and convolutional methods. The reported training-time comparison excludes negative sampling.

  • Ablation findings: BPR matrix factorization captures long-term interests but does not effectively model short-term user interests.
  • Ablation findings: Feature gating performs slightly better than instance gating, while average pooling slightly outperforms max pooling.The authors suggest average pooling accumulates representative features across successive items.
  • Efficiency comparison: HGN outperforms GRU and CNN while using fewer learnable parameters: 5,350 versus 15,300 for GRU and 26,154 for CNN.The comparison uses item embedding size d = 50.
  • Efficiency comparison: HGN achieves the fastest training speed on all datasets in the reported per-epoch comparison.Training-time measurements exclude negative sampling and were conducted on a single Nvidia GeForce GTX 1080 Ti.

5.8 The Sensitivity of Hyper-parameters

The sensitivity analysis examines embedding dimension and successive-item lengths. Performance improves with embedding capacity before stabilizing, while shorter histories and predicting three future items perform better in the reported settings.

  • Embedding dimension: Increasing the item-embedding dimension improves performance until the results become steady.Small dimensions are insufficient to express item latent features, whereas larger dimensions provide more modeling capacity.
  • Successive-item lengths: With the history length |L| fixed, setting the future-item length |T| to 3 achieves better performance.
  • Successive-item lengths: Smaller history lengths |L| outperform larger ones, possibly because longer histories include irrelevant items.

6 CONCLUSION

The paper concludes that HGN combines hierarchical feature and instance gating with an item-item product module for sequential recommendation. Experiments on five real-world datasets validate its performance and the effectiveness of these components.

  • HGN controls which item features and items reach downstream layers through feature and instance gating.
  • The item-item product module captures relations between closely relevant items in users’ sequential behavior.
  • Experiments on five real-world datasets validate HGN against many state-of-the-art methods.The experiments also demonstrate the effectiveness of the gating and item-item product modules.
Loading 1906.09217v1…