PaperlayerView exact revision on arXiv

Source-linked AI summary

Research Commentary on Recommendations with Side Information: A Survey and Research Directions

Zhu Sun, Qing Guo, Jie Yang, Hui Fang, Guibing Guo, Jie Zhang, Robin Burke

arXiv:1909.12807v2cs.IR

TL;DR

Traditional recommender systems face data sparsity and cold-start problems when relying on user-item interactions. This Research Commentary systematically surveys methods that use side information by organizing them across recommendation methodologies and side-information representations. It synthesizes current approaches, analyzes their development, and identifies challenges and future directions.

  • Problem

    Traditional collaborative filtering relies on sparse user-item interaction matrices, creating data sparsity and cold-start problems for new users and items.

  • Method

    The paper conducts a systematic survey organized by fundamental recommendation methodologies and structural or non-structural side-information representations.

  • Results

    The survey provides a comprehensive review and taxonomy of state-of-the-art recommendation approaches with side information, alongside comparative analysis and future research directions.

  • Takeaways & Limitations

    Recommendation research should consider side-information representation together with the methodology used to exploit it, because complex information structures require suitable advanced methods.

  • Takeaways & Limitations

    Video-based recommendation research remains limited because video features are difficult to obtain and manage and require substantial computational resources.

Abstract

from arXiv · show

Recommender systems have become an essential tool to help resolve the information overload problem in recent decades. Traditional recommender systems, however, suffer from data sparsity and cold start problems. To address these issues, a great number of recommendation algorithms have been proposed to leverage side information of users or items (e.g., social network and item category), demonstrating a high degree of effectiveness in improving recommendation performance. This Research Commentary aims to provide a comprehensive and systematic survey of the recent research on recommender systems with side information. Specifically, we provide an overview of state-of-the-art recommendation algorithms with side information from two orthogonal perspectives. One involves the different methodologies of recommendation: the memory-based methods, latent factor, representation learning, and deep learning models. The others cover different representations of side information, including structural data (flat, network, and hierarchical features, and knowledge graphs); and non-structural data (text, image and video features). Finally, we discuss challenges and provide new potential directions in recommendation, along with the conclusion of this survey.

1 INTRODUCTION

Recommender systems address information overload, but traditional collaborative filtering is limited by sparse interactions and cold-start users or items. This survey organizes side-information research by recommendation methodology and side-information representation to provide a systematic overview.

  • Motivation: Recommender systems help users navigate large volumes of products, news, and movies using historical interaction data.Their use has expanded across domains as information on the web has grown rapidly.
  • Traditional recommendation: Traditional collaborative filtering infers preferences from user-item interaction matrices by aggregating the tastes of similar users.Its effectiveness depends on available interaction data.
  • Challenges: Data sparsity makes preferences difficult to learn because users rate few items and many items receive limited feedback.The problem is especially acute when the item catalog is very large.
  • Challenges: Cold start affects new users and items because they lack historical data, making generic popular-item recommendations insufficiently personalized.Side information is introduced as a way to address both sparsity and cold start.
  • Survey rationale: Recommendation performance depends jointly on the representation of side information and the methodology used to exploit it.More complex side-information structures often require more advanced recommendation methods.
  • Survey scope: The survey reviews recommendation approaches across fundamental methodologies and side-information representations, including conventional methods, deep learning, structural data, and non-structural data.It combines literature collection from major conferences and journals with a taxonomy, comparative analysis, and future directions.

2 EVOLUTION OF RECOMMENDERS WITH SIDE INFORMATION

The survey classifies recommender systems by both fundamental methodology and side-information representation, because existing taxonomies do not capture their interaction. It traces methods from memory-based and latent-factor approaches toward representation learning and deep learning, alongside structural and non-structural side information.

  • Fundamental recommendation methodologies: Collaborative filtering methods use user-item interactions, while model-based methods learn low-dimensional user and item vectors for prediction.Memory-based methods search for similar users or items; model-based methods commonly predict with the dot product of latent vectors.
  • Overview of recommender systems: Recommender systems can be classified by strategies, tasks, and outputs, but these taxonomies do not fully explain recommendation with side information.The survey therefore focuses on hybrid methods across general, temporal, and sequential tasks and rating- and ranking-based outputs.
  • Side information representations: Side information is organized into structural data—flat, network, hierarchical, and knowledge-graph features—and non-structural data such as text, images, and videos.Structural information becomes more complex from homogeneous flat features toward heterogeneous knowledge graphs with multiple entity and relation types.
  • Side information representations: Knowledge graphs connect heterogeneous users, items, and feature types through multiple relations, enabling subtler relationships than homogeneous information alone.The survey reports that incorporating knowledge graphs can further boost recommendation accuracy.
  • Fundamental recommendation methodologies: Memory-based, latent factor, representation learning, and deep learning models form the survey’s taxonomy of fundamental recommendation methodologies.The survey emphasizes CF-based approaches and reviews their evolution before examining how side information is incorporated.
  • Survey scope: The survey analyzes 136 representative hybrid algorithms from two perspectives, with around 95% of the covered papers published in the recent 10 years.Its coverage spans conventional models with varied side information and newer deep-learning approaches.

3 CONVENTIONAL MODELS WITH SIDE INFORMATION

Conventional recommendation models incorporate structural and non-structural side information through memory-based, latent-factor, and representation-learning approaches. The survey finds that richer side information can improve recommendation quality, while scalability, feature quality, and manual design remain important constraints.

  • Memory-based methods: Memory-based methods fuse side information such as feature hierarchies with user- or item-neighborhood recommendation, but their search costs limit large-scale use.Taxonomy-driven methods represent products and users with distributions over hierarchical topics.
  • Latent factor models: Latent-factor models integrate side information through feature regularization, collective factorization, tensor factorization, and factorization machines.These approaches extend matrix-factorization-style representations to model user, item, feature, and pairwise interactions.
  • Latent factor models: Flat-feature methods improve over plain latent-factor models, while factorization machines often outperform comparable approaches by explicitly modeling pairwise interactions.Tensor-factorization methods can achieve comparable performance but have substantially higher time complexity than CMF and SVDFeature.
  • Structural side information: Hierarchies, knowledge graphs, and combined trust information encode richer relations that can support more accurate recommendations than simpler structural features.The survey reports that hierarchy-based methods outperform flat-feature methods and that knowledge-graph methods outperform other structural latent-factor approaches.
  • Non-structural side information: Text and image features extend latent-factor models toward finer content understanding, but extraction quality and the complexity of visual representations constrain further progress.Text methods progress from word- to aspect-, sentiment-, and topic-level modeling, while image methods require separate feature-extraction and recommendation components.
  • Representation learning models: Representation-learning methods learn local item relations rather than only global user and item factors, but basic Item2Vec provides limited personalization.The survey identifies adaptation of additional feature types and NLP techniques as directions for extending representation-learning recommenders.

4 DEEP LEARNING MODELS WITH SIDE INFORMATION

Deep learning models with side information are presented as recommendation approaches capable of learning nonlinear latent representations. The survey treats shallow autoencoders as deep learning models for presentation purposes.

  • Deep learning models: Deep learning recommendation approaches can learn nonlinear latent representations, unlike latent-factor and representation-learning models described in the survey.Examples include AutoRec and neural collaborative filtering.
  • Deep learning models: For presentation purposes, the survey classifies all artificial neural networks as deep learning models, including networks with one hidden layer.This convention includes shallow autoencoders.

4.1 Basic deep learning models

Basic deep learning recommendation models include auto-encoders, MLPs, CNNs, RNNs, and attention-based architectures. These models learn representations, capture sequential or spatial patterns, and identify relevant interaction history for recommendation.

  • Overview: The surveyed basic deep learning models form essential bases for more sophisticated models incorporating side information.The survey classifies these models into five categories and summarizes representative architectures.
  • Auto-Encoder based methods: Auto-encoders compress partially observed user or item vectors and reconstruct them to predict missing ratings.They minimize reconstruction error to learn compact representations.
  • MLP based methods: MLPs extract high-level user and item representations for approximating user-item interactions.The NCF framework combines generalized matrix factorization with MLP.
  • CNN based methods: CNNs capture local spatial patterns and support sequential recommendation from embedded action sequences.Caser uses previous 4 actions (L = 4) to predict the next 2 interaction steps (T = 2).
  • RNN based methods: RNNs model temporal and sequential recommendation by memorizing historical information and finding patterns across time.DREAM represents basket histories dynamically, while ST-RNN uses time- and distance-specific transition matrices.
  • Attention based methods: Attention mechanisms identify relevant parts of input data, while self-attention can replace RNNs and CNNs in sequence learning.AttRec considers both short- and long-term user interests.

4.2 Deep learning models with flat features (DLMs+FFs)

Deep learning models incorporate flat features such as user demographics and item categories through pre-filtering, concatenation, projection, and feature-specific fusion. These designs provide contextual user and item representations alongside interaction information.

  • Overview: Flat features include attributes such as user gender and item categories, which can be incorporated into deep learning recommenders.They provide contextual representations of users and items.
  • Auto-Encoder based methods: HDS jointly uses the user-item rating matrix and flat features through stacked denoising auto-encoders.The auto-encoders receive features such as user gender and item categories as input.
  • MLP based methods: Wide&Deep and related MLP methods embed categorical features and feed them into neural network hidden layers.Wide&Deep jointly trains wide linear regression models and deep neural networks.
  • Feature-specific fusion: Feature encoders can transform item embeddings for each category and combine the resulting category impacts.This approach represents the influence of K categories as the sum of their respective impacts.
  • Fusion strategies: Flat features are integrated through pre-filtering, direct concatenation, or projection within deep models.These strategies are summarized in the survey’s classification of DLMs+FFs.
  • Summary: The survey characterizes more elaborate feature fusion as a source of additional recommendation performance increments.The claim concerns the combination of side-information features with the representational capacity of deep learning models.

4.3 Deep learning models with network features (DLMs+NFs)

Deep learning models with network features progress from simple filtering and summation toward deep projection of network information. The survey reports higher recommendation accuracy for DLMs+NFs than for LFM-based network-feature models, while noting limited study of this area.

  • Fusion strategies: Network features have been incorporated into deep models through pre-filtering, SVDFeature-style summation, and deep projection.The progression ranges from simple use of network information to learned deep representations.
  • Reported effectiveness: DLMs+NFs consistently outperform LFMs+NFs in recommendation accuracy across the surveyed experimental results.This is reported as a comparative pattern across the reviewed studies, not as a single benchmark value.
  • Research coverage: Research on fusing network features into deep learning models is less extensive than research on LFMs+NFs.The survey identifies this imbalance as an area requiring further investigation.
  • Future directions: Graph convolutional and graph neural networks are identified as additional deep-learning approaches for network-feature recommendation.The survey connects network features with graph-based model families.

4.4 Deep learning models with feature hierarchies (DLMs+FHs)

Deep learning models with feature hierarchies combine hierarchical category information with interaction or content features. The survey reports verified effectiveness but describes this area as inadequately studied relative to other side-information settings.

  • Hierarchical feature models: A 3D-CNN combines session clicks, item descriptions, and category hierarchies using character-level encoding.Category paths are represented by concatenating each category with its ancestors to the root.
  • Hierarchical feature models: The 3D-CNN approach cannot distinguish different impacts of categories at different hierarchy layers.This limitation follows from representing the category path as a resulting character sequence.
  • Hierarchical projection: IARN models hierarchical categories through recursive parent-child projections from the root toward the leaf layer.Recursive projection gradually maps the item into a more general feature space.
  • Summary: Deep learning with feature hierarchies is inadequately studied compared with latent-factor models and other deep-learning side-information settings.The survey nevertheless reports empirically verified effectiveness for DLMs+FHs.
  • Practical scope: Feature hierarchies are described as more easily and cost-effectively obtained in applications such as Amazon and Tmall than some other complex side information.The survey presents this as one advantage of investigating hierarchical features.

4.5 Deep learning models with knowledge graphs (DLMs+KGs)

DLMs+KGs are organized into graph, path, and propagation-based approaches that incorporate knowledge-graph structure into recommendation. The survey reports higher accuracy for DLMs+KGs than LFMs+KGs but identifies scalability and KG-heterogeneity as continuing challenges.

  • Approach types: DLMs+KGs comprise graph embedding, path embedding, and propagation-based approaches.Graph methods learn KG embeddings directly, path methods encode semantic user-item paths, and propagation methods discover high-order interactions without directly extracting paths.
  • Graph embedding methods: Graph embedding methods incorporate conventional KG embeddings, such as TransE, TransR, TransH, and TransD, into recommendation models.These methods learn entity and relation representations from the knowledge graph and then integrate them into recommendation.
  • Path embedding methods: Path embedding methods extract semantically different connected paths between users and items and encode them with deep learning models.RKGE, for example, represents user-item relations through KG paths and encodes the extracted paths with recurrent neural networks.
  • Propagation methods: Propagation methods discover high-order interactions between items and KG entities, effectively performing automatic path mining.RippleNet propagates preferences from interacted items, while KGAT recursively and attentively propagates node embeddings along high-order connectivity.
  • Summary: DLMs+KGs show greater recommendation accuracy than LFMs+KGs, but their high computational cost limits scalability on large-scale datasets.The survey also reports that propagation-based methods outperform approaches using only graph or path embeddings, while better exploiting KG heterogeneity remains an open direction.

4.6 Deep learning models with text features (DLMs+TFs)

DLMs+TFs integrate textual features into deep recommendation architectures using auto-encoders, MLPs, CNNs, RNNs, and attention. Compared with latent-factor approaches, they fuse text more seamlessly and model textual information through deeper, finer-grained nonlinear transformations.

  • Text-feature integration: Deep recommendation models incorporate reviews, tips, item descriptions, and other textual features into recommendation.The surveyed methods use text to learn user and item representations, support explainability, or improve predicted preferences.
  • Auto-encoder methods: Auto-encoder methods jointly learn recommendation representations and reconstructive or content-based representations from textual inputs.CDL and SH-CDL connect matrix factorization with SDAE or DBN representations learned from abstracts, plots, categories, descriptions, or comments.
  • Neural architectures: MLP, CNN, and RNN methods combine interaction histories or contextual features with textual and other side information to estimate recommendations.Examples include wide-and-deep ranking, review-based user-item models, attention-enhanced CNNs, and recurrent models for spatial-temporal and semantic features.
  • Summary: DLMs+TFs seamlessly fuse textual features through deep learning advances such as CNN and RNN, whereas LFMs+TFs depend more heavily on external feature-extraction toolkits.Deep models use word-level embeddings, multiple nonlinear transformations, and attention to capture salient words and reviews.
  • Summary: DLMs+TFs model text more deeply and finely than approaches relying on averaged words, aspect or sentiment extraction, or conventional topic models.Neural attention can distinguish the saliency of individual words and reviews for users and items.

4.7 Deep learning models with image features (DLMs+IFs)

DLMs+IFs use deep architectures to incorporate visual features into recommendation, especially through CNN-based representation learning. Unlike latent-factor models that feed extracted features into linear models, deep models use nonlinear architectures to accommodate visual information.

  • Visual representation learning: CNN-based methods extract visual representations from images such as movie posters, book covers, facial images, and other item visuals.Examples include jointly trained convolutional auto-encoders, CNN-based image representation learning, and facial-feature models for makeup recommendation.
  • Role of image features: Image features support recommendation in domains including fashion, restaurants, hotels, Flickr, and Instagram.They can improve both recommendation accuracy and the attractiveness of recommended items.
  • Model comparison: DLMs+IFs generally use CNNs to capture local visual features, while LFMs+IFs feed extracted visual features into linear latent-factor models.Deep architectures add multiple nonlinear hidden layers to accommodate visual features and obtain more effective recommendation results.

4.8 Deep learning models with video features (DLMs+VFs)

Research on fusing video features into deep recommendation models is comparatively limited. The survey attributes this scarcity to the difficulty and time required to obtain and manage video features and their large computational demands.

  • Research landscape: Studies combining video features with deep recommendation models are fewer than studies using other side-information types.The survey identifies video processing and computation as the main reasons for this limited research base.
  • Challenges: Video features are difficult and time-consuming to obtain and manage, while their large volume requires substantial computational cost.ACF simplifies processing by extracting frame-level visual features with ResNet-152 and using the pool5 output as each frame’s feature vector.

4.9 Discussion of DLMs with side information

Deep learning models capture complex user-item interactions and flexibly incorporate diverse side information, but their gains involve substantially higher computational cost and training time. Side information can be integrated through preprocessing, early fusion, or jointly trained feature and rating modules.

  • Comparative performance: Deep learning models capture highly complex user-item interactions more effectively than memory-based, latent-factor, and representation-learning models.Their deep architectures and nonlinear transformations support this empirical advantage.
  • Side-information flexibility: Deep learning models flexibly incorporate structural and non-structural side information, including knowledge graphs, text features, and image features.Conventional models generally require feature engineering before using such information.
  • Trade-offs: Deep learning improvements are accompanied by heavy computational cost and longer training time, often requiring powerful hardware.Conventional models are generally more time-efficient than deep learning models.
  • Integration strategies: Side information can enter deep recommenders through pre-filtering, early fusion, or late fusion of feature and rating modules.Early fusion combines side information with user or item inputs, whereas late fusion jointly trains contextual-feature and interaction-based rating modules.

5 FUTURE DIRECTIONS

The commentary identifies future directions for using side information in structured deep recommenders, crowdsourced data collection, reinforcement and adversarial recommendation, and cross-domain and package recommendation. It emphasizes richer structural modeling, better-quality side information, and underdeveloped recommendation settings.

  • Deep recommenders with structured side information: Future work should improve deep recommenders’ ability to exploit complex structured side information.The authors identify both the intrinsic complexity of structured data and the difficulty of adapting deep models as central challenges.
  • Deep recommenders with structured side information: Knowledge-graph recommendation should move beyond paths and meta-paths toward higher-level structures such as meta-graphs and hyper-graphs.Existing methods are described as using only basic knowledge-graph information, leaving richer structures underexplored.
  • Deep recommenders with structured side information: An alternative to preprocessing structured information is to adapt neural networks so they directly model that structure, including through graph convolutional networks.Most existing methods convert structured information into formats consumable by common neural architectures.
  • Crowdsourcing side information for recommendation: Crowdsourcing could collect more varied side information than ratings, clicks, views, and check-ins, but suitable data types and collection methods remain open questions.The commentary points to active learning from crowds and gamification as potentially relevant approaches.
  • Reinforcement and adversarial recommendation: Side information could reduce reinforcement-learning action spaces and improve adversarial-example selection or item-embedding perturbations.The commentary states that existing reinforcement-learning work has not fully used side information’s rich structure, while adversarial recommendation remains at an early stage.
  • Cross-domain and package recommendation: Cross-domain and package recommendation are highlighted as settings where side information can support transfer or model relationships among jointly recommended items.Structured side information may identify geographical or complementary relationships in packages, but this application remains underdeveloped.

6 CONCLUSION

The commentary surveys recommendation with side information across recommendation methodologies and side-information representations, then discusses challenges and future directions. Its stated goal is to provide a comprehensive and systematic resource for researchers and practitioners.

  • Survey scope: The survey organizes recommendation algorithms by memory-based, latent-factor, representation-learning, and deep-learning methodologies.These constitute one of the commentary’s two orthogonal organizing perspectives.
  • Survey scope: It also organizes side information as structural data—flat, network, hierarchical, and knowledge-graph features—and non-structural data such as text, image, and video features.This is the second orthogonal perspective used to structure the survey.
  • Future directions: The commentary additionally identifies challenges and potential research directions in recommendation with side information.The authors present the resulting survey as a resource for researchers and practitioners.
Loading 1909.12807v2…