PaperlayerView exact revision on arXiv

Source-linked AI summary

Context-aware Sequential Recommendation

Qiang Liu, Shu Wu, Diyi Wang, Zhaokang Li, Liang Wang

arXiv:1609.05787v1cs.IRcs.AI

TL;DR

Sequential recommendation methods have difficulty modeling contextual information that matters for user behavior. CA-RNN addresses this by replacing conventional RNN matrices with context-specific input and transition matrices, and it significantly improves over sequential and context-aware methods on two public datasets.

  • Problem

    Existing sequential recommendation methods, including RNN-based methods, have difficulty modeling contextual information despite its importance for behavior modeling.

  • Method

    CA-RNN uses adaptive context-specific input matrices for external situations and adaptive context-specific transition matrices for time intervals between adjacent behaviors.

  • Results

    CA-RNN significantly outperforms state-of-the-art sequential and context-aware recommendation methods on the Taobao and Movielens-1M datasets.

  • Takeaways & Limitations

    Incorporating both input contexts and transition contexts enables CA-RNN to achieve the best performance among the proposed context variants across the reported datasets.

Abstract

from arXiv · show

Since sequential information plays an important role in modeling user behaviors, various sequential recommendation methods have been proposed. Methods based on Markov assumption are widely-used, but independently combine several most recent components. Recently, Recurrent Neural Networks (RNN) based methods have been successfully applied in several sequential modeling tasks. However, for real-world applications, these methods have difficulty in modeling the contextual information, which has been proved to be very important for behavior modeling. In this paper, we propose a novel model, named Context-Aware Recurrent Neural Networks (CA-RNN). Instead of using the constant input matrix and transition matrix in conventional RNN models, CA-RNN employs adaptive context-specific input matrices and adaptive context-specific transition matrices. The adaptive context-specific input matrices capture external situations where user behaviors happen, such as time, location, weather and so on. And the adaptive context-specific transition matrices capture how lengths of time intervals between adjacent behaviors in historical sequences affect the transition of global sequential features. Experimental results show that the proposed CA-RNN model yields significant improvements over state-of-the-art sequential recommendation methods and context-aware recommendation methods on two public datasets, i.e., the Taobao dataset and the Movielens-1M dataset.

I. INTRODUCTION

Context-aware sequential recommendation must model both behavioral order and the external situations surrounding behaviors. CA-RNN addresses this gap with adaptive matrices for input and transition contexts.

  • Motivation: Sequential methods model behavioral order, but RNN-based recommendation methods have difficulty incorporating rich contextual information in real-world applications.Collected contexts include situations such as location, time, and weather.
  • Context Types: Input contexts represent external situations under which users conduct behaviors, including location, time, and weather.Examples include home or workplace, weekdays or weekends, and sunny or rainy conditions.
  • Context Types: Transition contexts represent time intervals between adjacent behaviors and capture how interval length changes transitions from past behaviors to future behaviors.Shorter intervals usually have more significant effects than longer intervals.
  • Proposed Approach: CA-RNN models sequential and contextual information in one framework using adaptive context-specific input and transition matrices.These replace the constant matrices used in conventional RNNs.
  • Contributions: Experiments on two real-world datasets show that CA-RNN significantly outperforms state-of-the-art sequential recommendation and context-aware recommendation methods.The reported evaluations use the Taobao and Movielens-1M datasets.

II. PROPOSED MODEL

The model predicts a user’s next item from a behavioral history enriched with input contexts and transition contexts. These contexts are attached to behaviors and time gaps in the sequence.

  • Problem Definition: For each user, the model uses a behavioral history of selected items and associated timestamps to predict the next selected item.The target prediction is conditioned on the next item’s input and transition contexts.
  • Problem Definition: Each selected item in a behavioral history is associated with a timestamp identifying its position in the user’s behavioral sequence.The sequence notation indexes both selected items and timestamps by behavior order.
  • Context Representation: Input contexts describe external conditions associated with a behavior, such as weather or location.The input context at a timestamp is denoted as c^u_I,k in the paper’s notation.
  • Context Representation: Transition contexts are determined by the time interval between the current behavior’s timestamp and the previous behavior’s timestamp.They are represented separately from the input context of the behavior.

B. Recurrent Neural Networks

A conventional RNN propagates sequential information through recurrent hidden states, but it has difficulty modeling varied contextual information. CA-RNN is presented as an adapted recurrent architecture for this setting.

  • Recurrent Neural Networks: An RNN uses recurrent hidden layers, predicts an output from the current hidden layer, and feeds the new output into the next hidden state.This recurrent structure supports sequential modeling across time steps.
  • Recurrent Neural Networks: The hidden state represents a user’s sequential status, while the item latent vector represents the corresponding behavior.Both are d-dimensional representations in the described formulation.
  • Recurrent Neural Networks: The transition matrix propagates sequential signals from the previous hidden status, and the input matrix captures the user’s current behavior.The conventional formulation uses constant matrices for these two roles.
  • Motivation: Conventional RNNs have difficulty modeling varied contextual information, motivating an adapted RNN architecture for context-aware sequential recommendation.The motivation follows the increasing availability of contextual information in practical applications.
  • CA-RNN Overview: The proposed model overview separates CA-RNN’s forward propagation from the computational procedure of an individual hidden layer.Figure 2 presents both views of the architecture.

C. Modeling Input Contexts

CA-RNN models input contexts as external situations surrounding user behaviors. It replaces the conventional RNN’s constant input matrix with adaptive matrices specific to those contexts.

  • C. Modeling Input Contexts: Input contexts are external situations under which users conduct behaviors.They provide contextual information for modeling behavior sequences.
  • C. Modeling Input Contexts: CA-RNN replaces the constant input matrix with adaptive context-specific matrices.The matrices vary according to different input contexts and incorporate those contexts into item representations.

D. Modeling Transition Contexts

CA-RNN treats the time intervals between adjacent behaviors as transition contexts. It uses adaptive transition matrices, generated through discretized time bins, to model their differing effects on sequential-state transitions.

  • D. Modeling Transition Contexts: Transition contexts are the lengths of time intervals between adjacent behaviors in historical sequences.Longer intervals generally have weaker effects on predicting the next behavior than shorter intervals.
  • D. Modeling Transition Contexts: CA-RNN replaces the constant transition matrix with adaptive matrices conditioned on transition contexts.These matrices capture how intervals between adjacent behaviors affect transitions of global sequential features.
  • D. Modeling Transition Contexts: Continuous time intervals are discretized by assigning them to the floor of corresponding time bins.This avoids learning an independent context-specific transition matrix for every possible interval value.
  • D. Modeling Transition Contexts: The resulting context-specific transition matrices are used in each hidden layer of CA-RNN.The formulation is presented as the hidden-layer computation shown in Figure 2.

E. Context-aware Prediction

Context-aware prediction incorporates current contexts in addition to historical behavioral contexts. The prediction function determines whether a user selects an item at the next timestamp.

  • E. Context-aware Prediction: Historical behavioral contexts alone are insufficient for context-aware prediction at the next timestamp.Current contexts also have significant effects and should be incorporated when predicting user behavior.
  • E. Context-aware Prediction: The prediction function estimates whether user u will select item v at timestamp t_u,k+1.It is defined as the context-aware prediction function for the next behavior.
  • E. Context-aware Prediction: Current input and transition contexts are represented by d × d dimensional matrices in prediction.The current input-context and transition-context representations are generated within the CA-RNN framework.

F. Parameter Learning

CA-RNN is learned with Bayesian Personalized Ranking and Back Propagation Through Time. BPR supplies a pairwise ranking objective, while BPTT propagates gradients through the sequence to estimate model parameters.

  • F. Parameter Learning: CA-RNN uses BPR and BPTT for parameter learning.These methods are applied to estimate parameters in the recurrent recommendation model.
  • F. Parameter Learning: BPR assumes that users prefer selected items over negative items in implicit-feedback data.At each sequential position, CA-RNN maximizes the probability that a selected item ranks above a negative sample.
  • F. Parameter Learning: The equivalent negative-log-likelihood objective includes regularization over the parameters Θ = {R, M, W}.The regularization strength is controlled by λ.
  • F. Parameter Learning: BPTT calculates gradients through hidden layers iteratively across the whole sequence.Stochastic gradient descent can then estimate the model parameters.

III. EXPERIMENTS

The experiments evaluate CA-RNN and related methods on two timestamped real-world recommendation datasets using ranking metrics and controlled preprocessing.

  • Experimental Settings: Experiments use Taobao and Movielens-1M, both containing user behavior records associated with timestamps.Taobao has 13,611,038 shopping records from 1,103,702 users and 462,008 items; Movielens-1M has 1,000,209 ratings from 6,040 users for 3,900 movies.
  • Experimental Settings: Each sequence is split chronologically, using its first 80% for training and remaining 20% for testing.Users with fewer than 10 records and items with fewer than 3 records are removed to reduce sparsity.
  • Experimental Settings: The evaluation reports Recall@k, F1-score@k, MAP, and NDCG, with Recall and F1-score measured at k = 1, 5, and 10.Higher values indicate better performance for all metrics.
  • Compared Methods: Comparisons include popularity and BPR baselines, FM and CARS2 context-aware methods, FPMC, HRM and RNN sequential methods, and CA-RNN variants.The variants include CA-RNN-input and CA-RNN-transition, which use only input or transition contexts.
  • Compared Methods: Table I evaluates input-context and transition-context variants on Taobao and Movielens-1M at dimensionality d = 10.The comparison uses Recall, F1-score, MAP and NDCG.

B. Input Contexts VS. Transition Contexts

Input and transition contexts have dataset-dependent effects, but combining both types produces the strongest performance among the proposed CA-RNN variants.

  • Input Contexts VS. Transition Contexts: On Taobao, CA-RNN-input outperforms CA-RNN-transition, whereas the reverse holds on Movielens-1M.The results indicate that the relative importance of the two context types differs across situations.
  • Input Contexts VS. Transition Contexts: CA-RNN achieves the best performance among the proposed methods by incorporating both input contexts and transition contexts.It significantly outperforms both CA-RNN-input and CA-RNN-transition.
  • Performance Comparison: CA-RNN greatly outperforms POP, BPR, FM, CARS2, FPMC, HRM and RNN on both datasets across all reported metrics.The comparison covers baseline, context-aware and sequential recommendation methods.
  • Performance Comparison: Compared with RNN, CA-RNN relatively improves Recall@1, Recall@5, Recall@10, MAP and NDCG by 91.8%, 59.7%, 50.8%, 60.4% and 31.5% on Taobao.On Movielens-1M, the corresponding relative improvements are 19.1%, 23.9%, 13.9%, 13.8% and 9.5%.

D. Impact of Dimensionality

The study examines how dimensionality affects RNN-based methods and finds that CA-RNN variants remain superior to conventional RNN across tested settings.

  • D. Impact of Dimensionality: On Taobao, the best performances for CA-RNN, CA-RNN-transition and CA-RNN-input occur at d = 20, d = 20 and d = 30, respectively.Figure 3 compares these variants with conventional RNN while varying dimensionality d.
  • D. Impact of Dimensionality: Across both datasets, the proposed methods consistently outperform conventional RNN with a significant advantage in all tested dimensionality settings.CA-RNN can still outperform conventional RNN even when its dimensionality is not optimal.
  • D. Impact of Dimensionality: CA-RNN replaces conventional RNN’s constant input and transition matrices with context-specific matrices for external situations and time-interval transitions.The modeled contexts include time, location and weather, while transition contexts represent intervals between adjacent behaviors.
Loading 1609.05787v1…