PaperlayerView exact revision on arXiv

Source-linked AI summary

Translation-based Recommendation

Ruining He, Wang-Cheng Kang, Julian McAuley

arXiv:1707.02410v1cs.IR

TL;DR

Sequential recommendation needs to model third-order interactions among users, previous items, and next items, while existing approaches often split them into pairwise components. TransRec embeds items in a transition space and users as translation vectors, achieving superior results across large real-world sequential and item-to-item recommendation tasks.

  • Problem

    Next-item recommendation must model complex third-order interactions among users, previously consumed items, and next items at scale and under sparsity.

  • Method

    TransRec embeds items in a transition space and represents each user with a translation vector that maps previous items toward personalized next-item predictions.

  • Results

    TransRec achieves superior sequential-prediction results across large real-world datasets and significantly outperforms multiple state-of-the-art models on eight large Amazon co-purchase datasets.

  • Takeaways & Limitations

    Success on sequential prediction and item-to-item recommendation suggests translation-based architectures are promising for general-purpose recommendation problems.

  • Takeaways & Limitations

    The aggregation used by some unified predictors can remain hard to interpret and may not provide metric-embedding benefits; TransRec also relies on metric-space assumptions for generalization.

Abstract

from arXiv · show

Modeling the complex interactions between users and items as well as amongst items themselves is at the core of designing successful recommender systems. One classical setting is predicting users' personalized sequential behavior (or `next-item' recommendation), where the challenges mainly lie in modeling `third-order' interactions between a user, her previously visited item(s), and the next item to consume. Existing methods typically decompose these higher-order interactions into a combination of pairwise relationships, by way of which user preferences (user-item interactions) and sequential patterns (item-item interactions) are captured by separate components. In this paper, we propose a unified method, TransRec, to model such third-order relationships for large-scale sequential prediction. Methodologically, we embed items into a `transition space' where users are modeled as translation vectors operating on item sequences. Empirically, this approach outperforms the state-of-the-art on a wide spectrum of real-world datasets. Data and code are available at https://sites.google.com/a/eng.ucsd.edu/ruining-he/.

1 INTRODUCTION

Sequential recommendation must model interactions among users, previously consumed items, and next items at large scale and under sparsity. TransRec unifies these interactions by translating previous-item embeddings toward personalized next-item predictions, with strong results across real-world tasks.

  • Next-item recommendation requires modeling third-order interactions among a user, recently consumed items, and the next item under scale and sparsity.
  • Traditional methods separate user-item compatibility from previous-item/next-item continuity, effectively combining matrix factorization and Markov-chain components.
  • TransRec embeds items in a transition space and represents each user as a translation vector mapping a previous item approximately to the next item.
  • Recommendations are generated by nearest-neighbor search around the translated point γ_i + t_u.
  • TransRec uses one component for third-order interactions, retains metricity-based generalization, and scales to sequences containing millions of instances.
  • TransRec significantly outperforms multiple state-of-the-art models on eight large Amazon co-purchase datasets and introduces a large Google Local sequential dataset.

2 RELATED WORK

Related work models recommendation through matrix factorization, Markov chains, metric embeddings, and aggregation schemes. TransRec instead uses a translation-based structure for personalized Markov transitions and complex sequential data.

  • General recommendation methods commonly use matrix factorization to model user preferences while ignoring sequential signals.
  • Metric embeddings improve generalization through distances, while pooling-based methods model dependence between users and previous items but are harder to interpret.
  • TransRec introduces a translation-based structure for third-order interactions in personalized Markov transitions and targets large-scale, complicated sequential data.

3 THE TRANSLATION-BASED MODEL

TransRec represents items in a transition space and users as translation vectors, allowing personalized item transitions to be modeled in one metric-based component. It trains these representations for sequential ranking and supports nearest-neighbor recommendation.

  • Problem Formulation: TransRec formulates sequential recommendation as predicting the next item j from user u and preceding item i in each interaction sequence.The model ranks the ground-truth successor above alternative items using pairwise ranking optimization.
  • The Proposed Model: Items are embedded as latent vectors in a transition space, while each user is represented by a translation vector capturing personalized transition tendencies.For a transition from i to j, the translated point γ_i + t_u should be near γ_j under a distance metric.
  • The Proposed Model: Metric-space neighborhoods encode item similarity and translation encodes transition relationships, with triangle inequality supporting generalization under sparsity.If users transition from item A to both B and C, the metric structure also places B and C close together.
  • The Proposed Model: The model uses a global translation vector plus a user-specific offset, regularizing cold-start users toward the average transition behavior.The combined translation is T_u = t + t_u; the user offset adds no expressive power but supports sparse-data settings.
  • Inferring the Parameters: TransRec learns parameters with stochastic gradient ascent on pairwise sequential rankings and constrains item embeddings to a subspace such as a unit ball.Its prediction includes an item-popularity bias, and embeddings are re-normalized during optimization.
  • Nearest Neighbor Search: At test time, recommendation becomes nearest-neighbor search using a query formed from the preceding item embedding and the user’s total translation.Bias terms are shifted and absorbed into the search representation without changing item rankings.

4 EXPERIMENTS

The experiments evaluate TransRec and comparison methods across diverse, large public datasets, using sequential recommendation metrics and model-property comparisons. The datasets vary in domain, size, sparsity, and variability, while evaluation uses held-out sequential actions.

  • 4.1 Datasets and Statistics: Experiments cover publicly available datasets that differ substantially in domain, size, data sparsity, and variability or complexity.
  • 4.1 Datasets and Statistics: Amazon datasets contain review and timestamp corpora spanning multiple product categories and are characterized by high sparsity and variability.
  • 4.1 Datasets and Statistics: Users and items with fewer than five associated actions are discarded, and available star ratings are treated as positive feedback for implicit-feedback evaluation.
  • 4.2 Comparison Methods: The comparison includes popularity, matrix-factorization, Markov-chain, hybrid, metric-embedding, hierarchical, and translation-based methods.
  • 4.2 Comparison Methods: Models are compared using their personalization, sequential-awareness, metric-based formulation, and ability to unify third-order relations.
  • 4.3 Evaluation Methodology: Historical user sequences are split into recent test, second-most-recent validation, and earlier training portions, with hyperparameters tuned on validation data.

1 |I \ Su |

TransRec outperforms the comparison methods in nearly all reported cases, with especially large gains on variable and sparse datasets. Baseline behavior differs with dataset density and model complexity.

  • Results: BPR-MF and FMC generally outperform popularity ranking, showing value in modeling user-item and item-item relationships even separately.
  • Results: FPMC performs best among the relevant baselines mainly on denser datasets, while its larger parameter count may hurt on sparse data; HRM is strong across most cases.
  • Results: TransRec outperforms other methods in nearly all cases.
  • Variability: 32.5% and 24.7% Hit@50 improvements occur on Google and Clothing, the datasets with the largest item vocabularies.
  • Sparsity: TransRec performs especially well on comparatively sparse Epinions, Automotive, and Google datasets, except for Flixter Hit@50.
  • Results: Squared L2 distance typically outperforms L1 distance, although L1 also beats baselines in most cases.

4.5 Convergence

Convergence behavior varies with dataset sparsity: simpler methods converge faster on sparse data, whereas methods converge at more comparable speeds on denser datasets.

  • 4.5 Convergence: BPR-MF and FMC converge faster than other methods on sparse datasets, reflecting the simpler dynamics they capture.
  • 4.5 Convergence: FPMC also converges quickly on sparse datasets, reportedly because its tendency to overfit triggers early termination.
  • 4.5 Convergence: On denser Electronics, Foursquare, and Flixter datasets, methods converge at comparable speeds as they model denser relationships.

4.6 Sensitivity

Increasing representation dimensionality from 10 to 100 preserves TransRec’s dominance on Electronics and Foursquare and narrows its Flixter Hit@50 gap with PRME.

  • 4.6 Sensitivity: TransRec continues to dominate other methods on Electronics and Foursquare as dimensionality K increases from 10 to 100.
  • 4.6 Sensitivity: On Flixter, increasing dimensionality closes the Hit@50 gap between TransRec with L2 distance and PRME.

4.7 Implementation Details

The experiments examine convergence and dimensionality sensitivity under specified training and embedding settings. TransRec uses a unit L2-ball, while the unit L2-sphere performed slightly worse.

  • Training comparisons track test AUC across four datasets as iterations increase with K = 10.
  • TransRec uses the unit L2-ball as its subspace Ψ, while the unit L2-sphere produced slightly worse results.

4.8 Recommendations

Recommendations from TransRec are illustrated for a random sample of seven Electronics users. The examples use users’ historical sequences and display each user’s top-1 recommendation.

  • The examples indicate that TransRec can capture long-term dynamics in users’ histories.

4.9 Item-to-item recommendation

The paper adapts TransRec to item-to-item recommendation by removing personalization and evaluates it on eight Amazon co-purchase datasets, including a content-based variant. TransRec considerably outperforms the compared baselines, with larger improvements on sparse datasets.

  • Removing personalization adapts TransRec to item-to-item recommendation by modeling relationships among items.
  • The evaluation uses eight Amazon datasets containing directed cross-subcategory co-purchase edges, emphasizing complementary rather than substitute products.
  • The content-based variant extracts Bag-of-Words features from product reviews and uses an embedding layer E(·) to project items into a relational space.
  • The comparison includes Weighted Nearest Neighbor, Low-rank Mahalanobis Transform, and Mixtures of Non-metric Embeddings.
  • TransRec considerably outperforms all baselines in every case, with larger improvements on sparse datasets such as Office than on denser datasets such as Toys and Games.

5 CONCLUSION

The paper concludes that TransRec is a scalable translation-based method for third-order and item-to-item recommendation. Results across both tasks support translation-based architectures as promising for general-purpose recommendation, alongside a large Google Local dataset contribution.

  • TransRec is introduced as a scalable method for modeling semantically complex relationships among entities in recommender systems.
  • The method models third-order interactions among users, previously consumed items, and next items, while also achieving superior sequential-prediction results across large real-world datasets.
  • Success on sequential and item-to-item recommendation suggests that translation-based architectures are promising for general-purpose recommendation problems.
  • The paper introduces a large-scale Google Local dataset containing information about millions of businesses, ratings, and reviews worldwide.
Loading 1707.02410v1…