PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Item-based Collaborative Filtering for Top-N Recommendation

Feng Xue, Xiangnan He, Xiang Wang, Jiandong Xu, Kai Liu, Richang Hong

arXiv:1811.04392v1cs.IR

TL;DR

Existing ICF methods largely model only linear, shallow, or pairwise item relationships, limiting their treatment of complex higher-order effects in user decision-making. DeepICF uses nonlinear neural networks to integrate higher-order interactions into ICF, with attention-based weighting as a further refinement. Experiments on MovieLens and Pinterest support its effectiveness, while the paper notes limitations in relation types and explanation granularity.

  • Problem

    Existing ICF methods mainly model linear, shallow, or pairwise item relationships, leaving higher-order item relations insufficiently captured for complex user decisions.

  • Method

    DeepICF integrates higher-order item-relation learning into an ICF predictive model using nonlinear neural networks, with DeepICF+a adding attention to weight pairwise interactions.

  • Results

    DeepICF outperforms other state-of-the-art item-based approaches on top-N recommendation, with relative NDCG improvements of 4.8% on MovieLens and 6.6% on Pinterest over FISM.

  • Takeaways & Limitations

    Higher-order item interactions can be modeled effectively with deep neural networks, while attention can further differentiate the importance of interacted items.

  • Takeaways & Limitations

    The paper focuses on implicit similarity and notes that similarity-based explanations may be too coarse-grained for user trust.

Abstract

from arXiv · show

Item-based Collaborative Filtering(short for ICF) has been widely adopted in recommender systems in industry, owing to its strength in user interest modeling and ease in online personalization. By constructing a user's profile with the items that the user has consumed, ICF recommends items that are similar to the user's profile. With the prevalence of machine learning in recent years, significant processes have been made for ICF by learning item similarity (or representation) from data. Nevertheless, we argue that most existing works have only considered linear and shallow relationship between items, which are insufficient to capture the complicated decision-making process of users. In this work, we propose a more expressive ICF solution by accounting for the nonlinear and higher-order relationship among items. Going beyond modeling only the second-order interaction (e.g. similarity) between two items, we additionally consider the interaction among all interacted item pairs by using nonlinear neural networks. Through this way, we can effectively model the higher-order relationship among items, capturing more complicated effects in user decision-making. For example, it can differentiate which historical itemsets in a user's profile are more important in affecting the user to make a purchase decision on an item. We treat this solution as a deep variant of ICF, thus term it as DeepICF. To justify our proposal, we perform empirical studies on two public datasets from MovieLens and Pinterest. Extensive experiments verify the highly positive effect of higher-order item interaction modeling with nonlinear neural networks. Moreover, we demonstrate that by more fine-grained second-order interaction modeling with attention network, the performance of our DeepICF method can be further improved.

1 INTRODUCTION

ICF is widely used because it models users through consumed items and supports interpretable, easily personalized recommendations. The paper identifies a gap in existing methods' treatment of higher-order item relations and proposes DeepICF, a neural framework that integrates such modeling into prediction.

  • 1 INTRODUCTION: ICF builds user profiles from consumed items and recommends items similar to those profiles.Its industry adoption is associated with user-interest modeling and online personalization.
  • 1.2 Why Higher-Order Item Relations?: Existing ICF methods primarily model linear, shallow, or pairwise item relationships, overlooking complex higher-order relations among multiple items.Examples include shared attributes, co-occurrence, and overlapping properties across itemsets.
  • 1.2 Why Higher-Order Item Relations?: HOSLIM identifies frequent itemsets separately before prediction, making its support threshold difficult to tune and its higher-order effects static and linear.Useful itemsets may be missed, while unhelpful identified itemsets may affect the predictive model uncontrollably.
  • 1.3 Our Proposal and Contributions: DeepICF integrates higher-order relation learning into a neural predictive model that also captures second-order item relations.The framework uses nonlinear layers above pairwise interaction modeling; its variants differ in whether pairwise interactions receive equal or attention-based weights.
  • 1.3 Our Proposal and Contributions: The paper evaluates the proposed framework on two real-world datasets and releases code to support further development of deep item-based CF methods.The stated contributions include a generic neural framework and two specific methods, DeepICF and DeepICF+a.

2 PRELIMINARIES

The preliminaries define ICF as predicting user-item interactions from similarities between a target item and previously interacted items. They then review SLIM, HOSLIM, FISM, and NAIS, highlighting the progression from learned pairwise similarities toward higher-order and attention-weighted modeling.

  • 2 PRELIMINARIES: ICF predicts a user-item interaction from the target item's similarity to items previously interacted with by the user.The formulation uses the user's historical item set, item similarities, and observed preferences.
  • 2 PRELIMINARIES: ICF offers accuracy, interpretability, and online-recommendation advantages by representing users through their interacted items rather than static user parameters.New purchases can refresh recommendations through item-similarity retrieval without retraining user-specific parameters.
  • SLIM: SLIM learns an item-item similarity matrix by reconstructing historical interactions while enforcing non-negativity, sparsity, and regularization constraints.The matrix is learned directly from data, with L1 regularization encouraging sparsity and L2 regularization reducing overfitting.
  • SLIM and Higher-Order SLIM: HOSLIM extends SLIM with itemset similarities, but its two-step frequent-itemset mining procedure depends on a difficult support-threshold choice.Its predictive model combines item-item and item-itemset similarities in a linear, static manner.
  • FISM and NAIS Methods: FISM factorizes item similarity through low-rank embeddings, while NAIS adds dynamically learned attention weights to distinguish historical items' contributions.FISM uses latent vectors for target and historical items; NAIS parameterizes attentive weights with a neural network.

3 METHODS

DeepICF models higher-order item relations with nonlinear neural layers above pairwise interactions, while DeepICF+a additionally uses attention to weight historical-item contributions.

  • Input and Embedding Layer: The framework uses multi-hot interacted-item inputs for users and one-hot target-item inputs, producing embeddings for historical and target items.
  • Pairwise Interaction Layer: Element-wise products explicitly model pairwise interactions between each historical item and the target item.The resulting vectors encode second-order item relations before pooling.
  • Pooling Layer: Pooling converts variable-size pairwise interaction sets into fixed-size vectors through weighted average pooling in DeepICF or attention-based pooling in DeepICF+a.The two pooling choices define the two proposed methods.
  • Pooling Layer: DeepICF’s weighted average gives historical items equal influence, although user decisions may depend mainly on a few relevant items.The normalization parameter α controls smoothing across users with different history lengths.
  • Pooling Layer: Assigning separate contribution weights to historical items did not significantly improve performance, so that extension was not explored further.
  • Attention-Based Pooling: DeepICF+a addresses non-uniform historical-item contributions with attention-based pooling, using an MLP attention function and a smoothed softmax weighting scheme.The β parameter smooths the denominator, and suitable β values allow omission of DeepICF+a’s α normalization term.
  • Deep Interaction Layers: An MLP stacked above the pooled vector captures higher-order interactions among historical and target-item embeddings.The final representation is projected to a prediction score using a linear regression layer with a global weight vector and user and item biases.

4 EXPERIMENTS

The experiments use two public datasets to evaluate DeepICF against state-of-the-art recommenders, examine hyperparameters, and test deeper layers for higher-order interaction modeling.

  • Experimental Goals: Experiments on two publicly accessible datasets evaluate the effectiveness of the proposed methods.
  • Research Questions: RQ1 compares DeepICF and DeepICF+a with other state-of-the-art recommender models.
  • Research Questions: RQ2 examines how key hyperparameter settings influence DeepICF performance.
  • Research Questions: RQ3 tests whether deeper hidden layers capture higher-order and nonlinear item interactions and enhance FISM’s expressiveness.
  • Experimental Organization: The section presents experimental settings before answering the research questions sequentially.

4.1 Experimental Settings

The experiments use filtered MovieLens and Pinterest datasets, leave-one-out evaluation, and comparisons against heuristic, latent-factor, neural, and item-based baselines.

  • Datasets: MovieLens uses one million ratings with at least 20 ratings per user, while Pinterest is filtered to retain users with at least 20 interactions.Both datasets are publicly accessible and are used to evaluate collaborative-filtering algorithms.
  • Evaluation Protocols: The evaluation sorts each user’s interactions by timestamp, holds out the latest interaction for testing, and trains on the remaining interactions.For each test case, 99 non-interacted items are randomly sampled as negative instances.
  • Baselines: The baseline set spans popularity ranking, ItemKNN, HOSLIM, YouTube Rec, BPR, eALS, MLP, and FISM.These methods cover non-personalized, conventional item-based, higher-order, neural, user-based, and representation-learning approaches.
  • Baselines and Parameters: FISM is treated as the state-of-the-art item-based baseline, while DeepICF models are compared with embedding sizes of 8, 16, 32, and 64.The experiments tune α for FISM and regularization λ for learning-based approaches.

4.2 Performance Comparison (RQ1)

DeepICF and DeepICF+a achieve the best recommendation performance on both datasets, with gains over FISM and attention-based differentiation of historical-item contributions. Additional analyses examine convergence, explainability, and initialization effects.

  • Performance Comparison: DeepICF and DeepICF+a achieve the highest HR and NDCG scores on both datasets, significantly outperforming FISM with p-Value < 0.05.The authors attribute the gains to higher-order interaction learning and attention-based weighting of historical items.
  • Convergence: At convergence, DeepICF and DeepICF+a improve NDCG over FISM by 4.8% on MovieLens and 6.6% on Pinterest, respectively.Initialized models already exceed FISM in the first epoch, and performance improves during training.
  • Explainability: DeepICF uses equal weighting of pairwise interactions, whereas DeepICF+a uses attention to assign different weights to historical-item interactions.The attention mechanism is intended to differentiate the importance of historical items in a user representation.
  • Explainability: For MovieLens user #1188 and target item #1549, DeepICF+a predicts 0.52 versus 0.37 for DeepICF after sigmoid.DeepICF+a assigns higher attention to items #495 and #43, which share romance-drama content with the target.
  • Pre-training: The study also evaluates DeepICF models with and without FISM item-embedding pre-training because random initialization can impair convergence and final performance.The pre-training analysis is motivated by the non-convexity of the DeepICF objectives.

4.3 Sensitivity to Hyper-parameter (RQ2)

DeepICF performance depends on normalization, negative sampling, and embedding size. DeepICF+a generally improves performance, while optimal settings vary by dataset and model.

  • Normalization hyper-parameter α: DeepICF performs best on MovieLens with α between 0.4 and 0.5, exceeding FISM across α settings.
  • Normalization hyper-parameter α: DeepICF performs best on Pinterest with α between 0.5 and 1, exceeding FISM when α equals 0.2.
  • Normalization hyper-parameter α: DeepICF+a achieves its best performance on both datasets at α = 0 and outperforms DeepICF.The authors attribute this improvement to the attention mechanism.
  • Embedding size: DeepICF outperforms competing methods across most embedding sizes, while DeepICF+a addresses its small-size weakness on dense MovieLens.MLP performs better than DeepICF on MovieLens at embedding size 8.
  • Negative sampling: Sampling more negative instances generally improves DeepICF and DeepICF+a, with an optimum of around four negatives per positive.Both models outperform FISM with one negative instance per positive instance.

4.4 Depth of Hidden Layer in Network (RQ3)

Increasing the number of nonlinear hidden layers can improve DeepICF by modeling higher-order item relations. ReLU-based nonlinear layers are substantially more effective than replacing them with linear identity activations.

  • Depth of hidden layer: Stacking more nonlinear hidden layers is beneficial for DeepICF to capture higher-order item relations and improve performance.The reported comparison uses DeepICF variants with different hidden-layer counts.
  • Activation function: ReLU hidden layers perform much better than identity-function layers, supporting nonlinear higher-order interaction learning.The identity-function results are omitted because of space limitations.
  • Evaluation: Table 4 reports HR@10 and NDCG@10 for DeepICF models with different hidden-layer counts at embedding size 16.The best result in the table is highlighted in bold.

5 RELATED WORK

Related work progresses from heuristic and linear item-based collaborative filtering toward neural models. The paper positions DeepICF as a neural approach for learning nonlinear, higher-order item interactions.

  • Collaborative filtering: Collaborative filtering predicts preferences from user-item interactions and is commonly used for candidate selection in recommender systems.CF includes user-based and item-based approaches; implicit-feedback CF is typically formulated as top-N recommendation.
  • Item-based collaborative filtering: Early ICF methods estimate item similarity with cosine or Pearson measures, but require manual tuning and generalize poorly across datasets.SLIM and FISM learn item similarities from data using linear or factorized formulations.
  • Item-based collaborative filtering: SLIM learns a sparse similarity matrix, whereas FISM models item similarity through inner products of latent item vectors.SLIM also has high training cost and does not capture transitive item relationships.
  • Deep collaborative filtering: CADE applies neural modeling to ICF but retains a linear inner product for user-item interactions, limiting nonlinear relation modeling.CADE can recover SVD++ when its hidden-layer activation is replaced by an identity function.
  • Deep collaborative filtering: DeepICF uses an MLP over item embedding vectors to learn complex item interactions, while DeepICF+a adds attention to distinguish pairwise importance.The paper presents these models as approaches to higher-order and nonlinear item-relation modeling.

6 CONCLUSIONS AND FUTURE WORK

The paper introduces DeepICF, a deep neural item-based CF method for top-N recommendation, and reports that it outperforms state-of-the-art item-based approaches on two real-world datasets. Future work targets heterogeneous item relations, finer explanations, and sequential preferences.

  • Conclusions: DeepICF uses deep neural networks to learn higher-order item relations nonlinearly for top-N item recommendation.The model is proposed to overcome limitations of linear approaches such as FISM.
  • Conclusions: DeepICF outperforms other state-of-the-art item-based approaches in experiments on two real-world datasets.
  • Future work: Future work will incorporate heterogeneous item attributes and content, provide feature-based explanations, and model sequential preference evolution.The planned directions include relational knowledge, side information, reinforcement learning, and memory networks.
Loading 1811.04392v1…