Source-linked AI summary
Fast ALS-based tensor factorization for context-aware recommendation from implicit feedback
Balázs Hidasi, Domonkos Tikk
TL;DR
Implicit-feedback recommendation is common but less studied, and explicit-feedback methods do not transfer straightforwardly while preserving scalability. The paper proposes iTALS, an ALS-based tensor factorization method for context-aware implicit recommendation. Experiments report improved recommendation quality from incorporating context, with seasonality and sequentiality helping in different datasets and settings.
Problem
Implicit feedback is common in practice but less studied, while explicit-feedback methods and context-aware tensor methods do not straightforwardly provide scalable solutions for implicit data.
Method
iTALS is an ALS-based tensor factorization method for implicit recommendation that scales with tensor non-zeroes and supports contextual information such as seasonality and sequentiality.
Results
Context improved recommendation performance overall: seasonality improved all datasets, while sequentiality produced large gains on Grocery and LastFM 1K but not the movie databases.
Takeaways & Limitations
iTALS provides a factorization framework for integrating arbitrary contextual information into implicit-feedback recommendations.
Takeaways & Limitations
Adding context dimensions can make tensors sparser, potentially producing poorer results than using only a few context dimensions.
Abstract
from arXiv · showhide
Albeit, the implicit feedback based recommendation problem - when only the user history is available but there are no ratings - is the most typical setting in real-world applications, it is much less researched than the explicit feedback case. State-of-the-art algorithms that are efficient on the explicit case cannot be straightforwardly transformed to the implicit case if scalability should be maintained. There are few if any implicit feedback benchmark datasets, therefore new ideas are usually experimented on explicit benchmarks. In this paper, we propose a generic context-aware implicit feedback recommender algorithm, coined iTALS. iTALS apply a fast, ALS-based tensor factorization learning method that scales linearly with the number of non-zero elements in the tensor. The method also allows us to incorporate diverse context information into the model while maintaining its computational efficiency. In particular, we present two such context-aware implementation variants of iTALS. The first incorporates seasonality and enables to distinguish user behavior in different time intervals. The other views the user history as sequential information and has the ability to recognize usage pattern typical to certain group of items, e.g. to automatically tell apart product types or categories that are typically purchased repetitively (collectibles, grocery goods) or once (household appliances). Experiments performed on three implicit datasets (two proprietary ones and an implicit variant of the Netflix dataset) show that by integrating context-aware information with our factorization framework into the state-of-the-art implicit recommender algorithm the recommendation quality improves significantly.
1 Introduction
Implicit feedback is practically important but less studied than explicit feedback, while existing context-aware tensor methods do not handle the implicit task efficiently. The paper introduces iTALS, a fast tensor factorization approach with context-aware variants for seasonality and sequential usage patterns.
- Motivation: Implicit feedback infers preferences from user actions rather than ratings, but this important setting has received less research attention.Users often do not provide ratings, so systems must interpret events such as product-page navigation.
- Research gap: Existing solutions often adapt explicit-feedback algorithms to implicit data, while context-aware tensor methods previously addressed only the explicit problem.Classical matrix factorization uses user-item interactions, whereas contextual information can include time, location, or metadata.
- Contribution: iTALS is a fast tensor factorization method adapted to implicit recommendation and designed to incorporate contextual information.The paper presents iTALS as a general method for factorizing large tensors and applies it to implicit feedback.
- Context-aware variants: The proposed context-aware variants use seasonality to distinguish behavior across time intervals and sequentiality to learn repetitive usage patterns.Sequential patterns help distinguish items or categories purchased repetitively from those purchased once.
- Paper organization: The paper is organized around related work, the iTALS method, seasonal and sequential applications, experiments, and conclusions.The notation defines tensor dimensions, feature counts, preference and weight tensors, and factor matrices.
2 Related work
Context-aware recommendation incorporates information beyond user-item interactions, and tensor factorization represents such data in higher-dimensional tensors. Existing tensor methods scale poorly when implicit data produce large, dense tensors.
- Context-aware recommendation: Context-aware recommenders use information such as time, location, social networks, and user or item metadata in addition to interactions.The field includes applications such as movie, music, point-of-interest, and citation recommendation.
- Tensor factorization: Tensor factorization generalizes matrix factorization by representing users, items, and multiple contexts in a higher-dimensional tensor.The first dimensions correspond to users and items, while additional dimensions encode contextual variables.
- Scalability limitation: Existing methods scale linearly with explicit ratings and features but do not scale well when the tensor is large and dense in implicit recommendation.This scalability problem motivates a tensor factorization method designed for implicit data.
3 ALS based fast tensor factorization
iTALS is an ALS-based tensor factorization method for implicit recommendations that models sparse binary event tensors and scales linearly with non-zero examples. It alternates matrix updates using weighted observations while exploiting precomputation to keep computation practical.
- iTALS factorizes a sparse tensor of binary events into one low-rank matrix per dimension, predicting each cell from the elementwise product of corresponding factor columns.The method uses weights for tensor elements and applies ALS updates to the factor matrices.
- The loss is optimized by fixing all but one factor matrix, making the objective convex in the remaining variables and allowing its columns to be computed separately.The update sets the derivative with respect to the non-fixed matrix to zero.
- Sparse non-zero entries reduce the first update component to O(K^2N+) rather than requiring computation over every zero tensor element.The method exploits the fact that only N+ cells contain ones and uses weights for zero elements.
- The second update component can be computed through reusable matrix products, avoiding the naive cost associated with enumerating all combinations of tensor dimensions.The relevant expressions are computed from the factor matrices and reused when unchanged.
- Algorithm 3.1 initializes factor matrices and their products, then repeatedly updates each dimension's matrix using unfolded tensors, weights, and non-zero entries.The implementation returns the learned K × Si factor matrices after the prescribed number of epochs.
- Each matrix update costs O(K^3Si + K^2N+), so an epoch is linear in the number of non-zero examples and cubic in the number of features.The cost is also linear in tensor dimensions and the summed dimension lengths; the O(K^2) term is dominant in practical problems.
4 Context-aware iTALS algorithm
The paper derives seasonal and sequentiality-aware variants of iTALS to incorporate temporal and purchase-context information into implicit recommendations. These variants model time-dependent preferences and repetitiveness patterns within a common tensor-factorization framework.
- Context-aware iTALS algorithm: iTALS derives two context-aware algorithms: one models seasonality, while the other treats user history as sequential data to learn repetitiveness patterns.The variants are derived from the generic iTALS method and use different contextual information.
- 4.1 Seasonality: Seasonality is modeled by selecting a data-dependent season length and assigning events to time bands before factorizing a user–item–time-band tensor.The method assumes similar aggregate behavior at corresponding offsets across seasons and does not expect repetition within a season.
- 4.1 Seasonality: At recommendation time, the seasonal model determines the current time band and combines its feature vector with the user and item representations to calculate preferences.The factorized tensor yields feature vectors for users, items, and time bands.
- 4.2 Sequentiality: Sequentiality addresses category-driven repetition by using association-rule ideas to represent patterns across subsequent user events within a time window.The approach can express category–category or category–item relationships, but conventional rules require difficult parameter selection and may miss negated or overly specific patterns.
- 4.2 Sequentiality: The sequential model uses the last purchased item’s category as context, factorizing a user–item–category tensor whose feature vectors reweight user–item relations.The learned patterns are global and integrated into the factorization model without post-processing promotional or demotional weights or filters.
- 4.2 Sequentiality: Several recent purchases can be incorporated by activating multiple category contexts in a three-dimensional tensor, avoiding the sparsity of adding one dimension per purchase.The additional category cells may receive reduced weights.
5 Experiments
Experiments evaluate iTALS variants against iALS and an implicit context-aware baseline across five datasets, measuring recall, precision, and running time. Context generally improves recommendation quality, while iTALS retains scalable computation.
- Seasonality improved performance on all datasets, while sequentiality produced large gains on Grocery and LastFM 1K but not on the movie datasets.The sequentiality variant significantly surpassed seasonality on Grocery and LastFM 1K; seasonality increased VoD performance by an average of 30%.
- At 20 features the last category was best, whereas at 40 features the last two categories became better.The authors conjecture that this reflects the greater expressive power of models with more features.
- At 40 features, the precision–recall curves show a larger distance between context-aware iTALS variants and iALS.The paper attributes this to finer context reweighting and more specific item properties with more features.
- 5.1 Running times: iTALS scales quadratically with K like iALS, but its per-epoch running time is larger because it recomputes more feature matrices.Even with many context states, the O(K^2) term remains dominant when tensor non-zeroes greatly outnumber distinct users, items, and context states.
6 Conclusion
The paper presents iTALS as an efficient ALS-based tensor factorization method for context-aware implicit recommendation. Its seasonality and sequentiality variants improve performance, but high-dimensional tensors can become sparse and yield poorer results.
- iTALS scales linearly with the number of non-zero tensor elements and incorporates contextual information into implicit-feedback recommendation.The paper presents seasonality and sequentiality as two specific context-aware implementations.
- Seasonality separates periodic behavior into time bands, while sequentiality distinguishes items by repetitiveness patterns in user behavior.Both variants integrate arbitrary contextual information within the factorization framework.
- Adding many context dimensions can make tensors sparser because the number of non-zero elements remains unchanged, potentially worsening results.The authors recommend avoiding high-dimensional tensors and identify further characterization of reweighting, context features, and K as future work.