PaperlayerView exact revision on arXiv

Source-linked AI summary

Tensor Methods and Recommender Systems

Evgeny Frolov, Ivan Oseledets

arXiv:1603.06038v2cs.LGcs.IRstat.ML

TL;DR

Tensor-based recommender methods address the limited coverage of multifaceted recommendation settings in existing surveys. This paper synthesizes tensor techniques across application domains, finding that they flexibly incorporate additional information and support non-trivial recommendation models, while noting scope and interpretability limitations.

  • Problem

    Existing recommender-system surveys provide limited coverage of tensor-based methods, despite their use for multifaceted recommendation settings.

  • Method

    The paper surveys tensor-based recommender techniques across domains, covering their concepts, applications, and future directions.

  • Results

    Tensorization provides tools for merging additional information, extending recommendations beyond the standard user–item paradigm across application domains.

  • Takeaways & Limitations

    Tensor-based models can support flexible recommendation setups and provide compressed latent representations for other recommendation techniques.

  • Takeaways & Limitations

    Content-based approaches depend on obtaining thorough user and item properties, while tensor reconstructions can produce negative values without meaningful interpretations.

Abstract

from arXiv · show

A substantial progress in development of new and efficient tensor factorization techniques has led to an extensive research of their applicability in recommender systems field. Tensor-based recommender models push the boundaries of traditional collaborative filtering techniques by taking into account a multifaceted nature of real environments, which allows to produce more accurate, situational (e.g. context-aware, criteria-driven) recommendations. Despite the promising results, tensor-based methods are poorly covered in existing recommender systems surveys. This survey aims to complement previous works and provide a comprehensive overview on the subject. To the best of our knowledge, this is the first attempt to consolidate studies from various application domains in an easily readable, digestible format, which helps to get a notion of the current state of the field. We also provide a high level discussion of the future perspectives and directions for further improvement of tensor-based recommendation systems.

1 Introduction

Recommender systems filter information by learning relationships between users, items, and their interactions. Modern systems also incorporate additional entities such as tags and time, motivating tensor factorization methods and this survey’s structured overview.

  • Recommender systems: Recommender systems process large amounts of information and predict new interactions from user–item observation histories.Interactions include views, purchases, ratings, reviews, likes, and dislikes.
  • Multifaceted data: Entities in recommender systems can extend beyond users and items to include same-type entities, tags, time, and other contextual relations.These relations can represent user motivations, item relevance, or changing preferences.
  • Multifaceted data: Modeling multiple relations can produce more relevant, dynamic, and situational recommendations while increasing model complexity.The added complexity creates challenges and motivates tensor factorization algorithms.
  • Scope boundary: Production-ready recommender systems also require business logic, dataflow, infrastructure integration, service delivery, user experience, domain knowledge, and comprehensive evaluation.The survey therefore focuses on mathematical and algorithmic aspects, illustrated with examples from multiple application domains.
  • Survey scope: The survey covers recommender-system concepts and challenges, tensor fundamentals, tensor-based techniques across domains, and future directions.Its organization separates general recommender-system material from tensor concepts, applications, and concluding perspectives.

2 Recommender systems at a glance

Recommender systems use different prior information and modeling strategies to generate relevant items. Collaborative filtering relies on collective behavior, content-based filtering uses item and user attributes, and hybrid approaches combine their strengths.

  • Recommendation task: Product recommendation estimates a relevance or utility function from explicit or implicit user feedback to rank items for users.Explicit feedback includes ratings and likes; implicit feedback includes clicks and purchases.
  • Core approaches: Collaborative filtering uses observed collective user behavior, whereas content-based filtering matches users and items through item properties and user attributes.Collaborative filtering is divided into memory-based and model-based methods.
  • Content-based filtering: Content-based filtering can alleviate cold start and produce recommendations for unseen items when descriptive content information is available.Its practical drawbacks include limited content analysis, over-specialization, and difficulty collecting thorough user and item properties.
  • Collaborative filtering: Memory-based collaborative filtering commonly uses k-nearest neighbours to compute relevance from similar users or items.The simplest implementation stores all prior user–item interactions in memory.
  • Model-based methods: Factorization models embed users and items in a lower-dimensional latent space and are better suited to large-scale problems than pure similarity models.Their advantages include high performance and high-quality recommendations in many practical settings.
  • Hybrid methods: Hybrid recommender systems combine collaborative and content-based methods to use advantages from both approaches.

3 Challenges for recommender systems

Recommender systems must address cold start, missing interactions, evaluation validity, reproducibility, online updates, and changing context. Tensor and context-aware approaches are presented as ways to represent these multifaceted settings, with important interpretability and evaluation boundaries.

  • 3.1 Cold-start: Cold start makes recommendations difficult for new users or items because little preference, interaction, or content information is available.The problem affects both users and items.
  • 3.2 Missing values: Unobserved interactions create uncertainty because users engage with only a small subset of items, potentially biasing models and reducing recommendation quality.Regularization, non-zero weights for missing data, content-based smoothing, and clustering are described as mitigation techniques.
  • 3.4 Model evaluation: Reliable recommender evaluation requires more than accuracy, including relevance, ranking, coverage, diversity, novelty, serendipity, and business indicators.Online testing and user studies are described as the most reliable evaluation approaches, while offline metrics include precision, recall, F1-score, NDCG, MAP, MRR, and AUC.
  • 3.4 Model evaluation: RMSE or MAE performance does not guarantee high-quality top-n rankings because predicted relevance scores may not align with recommendation quality.Most recommender systems aim to generate ranked item lists rather than only predict relevance scores.
  • 3.5 Reproducible results: Reproducibility is constrained by inconsistent results across platforms and by differences in datasets, preprocessing, and evaluation metrics.The survey therefore emphasizes descriptions of method features over side-by-side quantitative comparisons.
  • 3.3 Online updates: High-quality systems must respond quickly to new users, items, and feedback, requiring low-complexity algorithms that accommodate dynamic environments.
  • 3.6 Context: Context-aware modeling treats user, item, and contextual variables jointly, extending model dimensionality to N + 2 when there are N context domains.Context can include time, location, day, mood, or social setting; contextual modeling preserves interrelations within context more directly than prefiltering or postfiltering.
  • 3.6 Context: Tensor methods naturally represent multifaceted and multirelational recommendation settings, but broad context definitions can create interpretability problems.Reconstruction may assign non-zero values to combinations that were not observed, making their interpretation dependent on the initial problem formulation.

4 Introduction to tensors

Tensor factorization extends matrix-based dimensionality reduction to multifaceted data with more than two modes. The section introduces tensor structure, matricization, latent-factor modeling, CP decomposition, and Tucker decomposition, while emphasizing that tensor low-rank approximation can be ill-posed.

  • Tensor fundamentals: Tensor factorization reveals patterns in data while modeling multifaceted user-item interactions across multiple modes.Unlike matrix factorization, tensor methods can represent relationships involving additional entities such as tags or time.
  • Tensor fundamentals: A tensor is a higher-order numerical array; an m-way tensor has m dimensions or modes, with fibers and slices generalizing matrix structures.Fibers fix all but one index, while slices fix all but two indices to form matrices.
  • Matricization: Matricization reshapes a tensor into a matrix by arranging mode-n fibers as columns, producing mode-specific matrix dimensions.For a tensor of size M × N × K, the three matricizations have sizes M × (NK), N × (MK), and K × (MN).
  • Dimensionality reduction: Matrix factorization uses low-rank latent representations to denoise observations, recover hidden user-item structure, and uncover previously unobserved interactions.Truncated SVD provides the best rank-r matrix approximation under the Eckart-Young theorem.
  • Tensor factorization: Tensor factorization generalizes matrix factorization through mode products and rank-1 tensor components, yielding multilinear latent models.The resulting model approximates real observations using factor matrices associated with the tensor’s modes.
  • Tensor decompositions: CP decomposition approximates a tensor with rank-1 components, but tensor low-rank approximation lacks an Eckart-Young analogue and may be ill-posed.Additional constraints such as orthogonality or non-negativity may be needed to improve numerical stability and convergence.
  • Tensor decompositions: Tucker decomposition extends SVD by replacing the diagonal matrix with a core tensor and adding a factor matrix for the additional mode.The factor matrices are orthogonal and represent latent features for the tensor modes.

5 Tensor-based models in recommender systems

The survey organizes tensor-based recommender models by application domain and uses a unified notation to describe their tensorization techniques. It reviews methods intended to add flexibility or quality to recommender systems while recognizing domain-specific nuances.

  • Survey scope: The section surveys tensorization techniques for building advanced recommender systems across varied application domains.The examples are presented using unified notation where possible to make concepts reusable across studies.
  • Survey scope: The unified notation may differ from original papers but is used to support consistent presentation across the reviewed methods.The survey explicitly notes that tensor-based recommender models involve nuances requiring careful treatment.

5.1 Personalized search and resource recommendations

Tensor models can treat personalized search as context-aware recommendation by modeling users, resources, and queries jointly. The reviewed approaches recover relevance scores or rankings, but higher-order representations increase sparsity, computational demands, and interpretive challenges.

  • Personalized search: Personalized search and recommender systems are closely related because recommendations can be viewed as zero-query search and personalized search as query-based recommendation.Search logs contain both clicked resources and the queries that contextualize users’ choices.
  • CubeSVD: CubeSVD represents user-resource-query interactions in a third-order tensor whose values encode relevance associations observed in search history.Observed entries correspond to user, resource, and query triplets, with unobserved entries set to zero.
  • Data representation: Search association scores can use click co-occurrence frequency, binary interaction values, or logarithmically rescaled frequencies with document-frequency adjustment.These transformations are intended to reduce bias toward pages with high click rates.
  • CubeSVD: HOSVD embeds users, web pages, and queries into separate latent spaces, while the core tensor represents the strength and form of their multilinear relations.The resulting factors and core tensor recover relevance scores for arbitrary triplets.
  • Sparsity mitigation: Smoothing techniques based on constant imputation or web-page content similarity improved overall model quality in the authors’ report.These techniques address numerical instability associated with the increased sparsity of higher-order data.
  • CubeSVD: Decomposition reconstructs non-zero values for potential user-resource-query associations, and higher tensor values rank resources as more relevant.The recovered tensor values can directly generate ranked lists of resources.
  • TOPHITS: TOPHITS aggregates resource-resource-query relations into an adjacency tensor and applies CP decomposition to model hubs, authorities, and keywords.Its component weights represent extracted topics, while factor vectors characterize hubs, authorities, and keyword terms within each topic.
  • Interpretation and limitations: Tensor values may be negative and therefore lack meaningful interpretation in many applications; non-negative tensor factorization is proposed as a remedy.This issue is especially relevant when reconstructed values are interpreted as relevance or association scores.

5.2 Social tagging

Social-tagging recommender systems model users, items, and tags as mutually related entities, using tensor factorization to uncover latent structure and generate recommendations. The surveyed methods address sparsity, ranking, and real-time updates through smoothing, pairwise objectives, and folding-in procedures.

  • Social tagging: Tags expose motives behind user preferences, so social-tagging systems model user-item-tag relations jointly as a multirelational tensor.The tensor encodes observed triplets as binary values, with 1 for observed assignments and 0 otherwise.
  • Unified framework: Tensor dimensionality reduction uncovers latent semantic structure, and reconstructed values estimate the likelihood of new user-item-tag links.These links can support tag assignment, item discovery, or like-minded-user identification.
  • Unified framework: HOSVD-based models address sparse tensor matricizations with kernel-based smoothing to improve learning from highly sparse data.The matricized tensors can otherwise prevent accurate model learning.
  • Unified framework: Higher-order folding-in embeds unseen entities without recomputing the whole model by appending updated latent-factor rows.Prestored terms make the reconstructed-tensor update more efficient, although repeated updates can reduce factor orthogonality and accuracy over time.
  • RTF and PITF: PITF applies Bayesian Personalized Ranking to pairwise tag comparisons, treating unobserved entries as unknown rather than uniformly negative.Experiments report close or better recommendation quality than BPR-tuned Tucker and CP models while learning features faster.
  • RTF and PITF: Original PITF lacks real-time support because rebuilding the full model for each new entity can be prohibitive, motivating compatible folding-in variants.The surveyed work reports that a PITF-compatible folding-in procedure retains high recommendation quality.
  • Improving the prediction quality: Hybrid tensor-based models can incorporate content information, but their suitability depends strongly on the type of content available.The survey does not identify one universally suitable hybrid approach.

5.3 Temporal models

Temporal tensor models represent changing or periodic user preferences and item relevance by incorporating time into latent-factor recommendations. The surveyed approaches model seasonality, evolving trends, and within-account user patterns to support forecasts and more personalized recommendations.

  • Temporal models: Temporal models account for user-consumption patterns that evolve with time, while evolving and periodic events may require different treatments.Examples include time-of-day effects in television use and seasonal variation in retail preferences.
  • Periodic models: BPTF uses seasonality and previous purchasing history to forecast ongoing-season orders, including sales of products absent from previous seasons.The model captures changes in product designs and customer preferences.
  • Probabilistic temporal factors: Probabilistic latent factors let user, item, and time features vary while modeling relevance-score variance with Gaussian assumptions.The time factor rescales user-item relevance according to evolving trends, while priors help regularize the model.
  • Probabilistic temporal factors: Although the dynamic model has many parameters and makes optimization nearly infeasible, an MCMC-based integration approach makes it nearly parameter-free and scalable.The formulation is equivalent to optimizing a weighted square-loss objective with dynamic regularization terms.
  • Subspace clustering and preference consolidation: TCC exploits periodic IPTV usage to identify users sharing an account and improve recommendations for those users.It builds an account-item-time tensor from play counts and represents real users as compositions of time-specific virtual users.
  • Subspace clustering and preference consolidation: TCC learns time sub-periods by clustering rows of the time-factor matrix, then consolidates similar virtual users with graph-based techniques before applying user-based kNN recommendations.The model avoids relying on manually specified time slots.

5.4 General context-aware models

General context-aware tensor models extend recommendation beyond user-item interactions by representing arbitrary contextual variables and optimizing recommendation objectives. The surveyed models address ranking, scalability, sparsity, and flexible multirelational structure.

  • Multiverse: The Multiverse model represents any set of preference-influencing variables as N−2 contextual dimensions in an N-th order tensor decomposition.Each contextual variable is embedded into a reduced latent space, and the factors need not be orthogonal.
  • Multiverse: Its weighted objective uses a pointwise loss based on l2, l1, or other distance measures, with regularization chosen for straightforward optimization.The formulation is illustrated for third-order tensors but generalizes to higher orders.
  • Multiverse: The modified SGD procedure updates latent-factor rows independently, allowing the updates to be performed in parallel as model order increases.The learning-step parameter η controls the update size.
  • Multiverse: Experiments on semi-synthetic data show that stronger context influence improves context-aware methods, with the proposed tensor-factorization approach performing best while context-unaware quality degrades.The study also includes results on a real dataset.
  • Ranking-oriented models: TFMAP optimizes mean average precision for implicit-feedback systems by using a negative-MAP loss and smooth approximations to ranking terms.A representative-item buffer controls the otherwise prohibitive computational complexity.
  • CARTD: CARTD generalizes context-aware ranking to an arbitrary number of contexts by extending the BPR-based approach used in PITF.Its framework targets optimal ranking rather than rating prediction.
  • General formulation: The unified entity formulation maps an entity and contextual variables to a relevance score, allowing users, items, tags, time, and other information to occupy entity or context roles.The associated learning construction relaxes exact positive-negative context conditions to reduce sparsity.
  • Scalability and interaction selection: Efficient tensor computation and selective interaction schemes improve scalability: iTALS-CD can learn faster than matrix-factorization methods, while GFF avoids complexity growth by omitting useless interactions.The surveyed evidence reports iTALS variants at approximately the accuracy of state-of-the-art Factorization Machines and more accurate GFF models when useless interactions are excluded.

5.5 Other models

Other tensor-based recommender models cover social, temporal, geographic, cross-domain, multi-criteria, and specialized factorization settings. These applications use tensor structure to combine additional relations, transfer information, or reduce computation for sparse data.

  • Social interactions: Social-interaction models recommend new connections within specific communities while separating heterogeneous groups to avoid inappropriate same-group recommendations.The cited example targets settings such as online dating and professional networks.
  • Social interactions: TOPHITS is applicable to authority ranking in Twitter and may support followee recommendations.
  • Social tagging: Social-tagging approaches stack adjacency matrices describing different relations among users and images into tensors for image-tagging tasks.
  • Temporal models: Temporal models incorporate social relationships as a regularizer or combine user-item-time tensors with auxiliary user attributes through coupled tensor-matrix factorization.A separate model combines social tagging and temporal information in a fourth-order user-item-tag-time tensor.
  • Multi-criteria ratings: Multi-criteria rating models extract sentiment aspects from product reviews and factorize user-item-aspect tensors to reconstruct missing aspect ratings.The tensor values include ratings for individual aspects and explicit ratings.
  • Cross-domain recommendations: Cross-domain models use consumption patterns from books, movies, music, or other domains for knowledge transfer and can model cross-domain relations jointly.
  • Special factorization methods: Pseudo-skeleton-style higher-order factorization is suited to sparse data because sampling a small subset of tensor elements reduces factorization complexity.

6 Conclusion

The survey presents tensor methods as flexible tools for modeling complex recommender environments and combining heterogeneous information, including as latent representations for larger systems. It also identifies computational growth, storage demands, and CP ill-posedness as important boundaries for higher-order models.

  • Conclusion: Tensor-based methods merge additional information, extend beyond the standard user-item paradigm, and support recommendation models across diverse domains.The survey also describes their use as compressed latent representations for other techniques.
  • Future directions: Increasing the number of tensor dimensions inevitably raises computational complexity and can delay recommendations for new users in dynamic production environments.Incremental updates and higher-order folding-in are identified as ways to address these issues.
  • Future directions: When dimensions substantially exceed three, tensor-decomposition methods may become infeasible because of storage growth, while CP decomposition remains generally ill-posed and potentially numerically unstable.The survey identifies TT/HT decomposition as a promising direction for further investigation.
Loading 1603.06038v2…