Source-linked AI summary
A Context-Aware User-Item Representation Learning for Item Recommendation
Libing Wu, Cong Quan, Chenliang Li, Qian Wang, Bolong Zheng
TL;DR
Static user and item representations may miss the candidate-specific relevance of review information for rating prediction. CARL learns joint user-item representations from reviews and interactions, models their higher-order interactions, and reports significantly better accuracy than state-of-the-art alternatives across five datasets while highlighting review information for interpretation.
Problem
Existing methods mainly learn independent, static user and item representations without considering the characteristics of each candidate item.
Method
CARL learns context-aware user-item representations from reviews and interactions, applies convolution and attention to reviews, and uses Factorization Machines for higher-order interactions.
Results
CARL consistently outperforms existing state-of-the-art alternatives across five real-world datasets in rating prediction accuracy.
Takeaways & Limitations
Attention in CARL highlights relevant review information and enables semantic interpretations of recommendation decisions.
Takeaways & Limitations
The review-based feature learning component may be incomplete, so CARL adds complementary features from interaction data.
Abstract
from arXiv · showhide
Both reviews and user-item interactions (i.e., rating scores) have been widely adopted for user rating prediction. However, these existing techniques mainly extract the latent representations for users and items in an independent and static manner. That is, a single static feature vector is derived to encode her preference without considering the particular characteristics of each candidate item. We argue that this static encoding scheme is difficult to fully capture the users' preference. In this paper, we propose a novel context-aware user-item representation learning model for rating prediction, named CARL. Namely, CARL derives a joint representation for a given user-item pair based on their individual latent features and latent feature interactions. Then, CARL adopts Factorization Machines to further model higher-order feature interactions on the basis of the user-item pair for rating prediction. Specifically, two separate learning components are devised in CARL to exploit review data and interaction data respectively: review-based feature learning and interaction-based feature learning. In review-based learning component, with convolution operations and attention mechanism, the relevant features for a user-item pair are extracted by jointly considering their corresponding reviews. However, these features are only review-driven and may not be comprehensive. Hence, interaction-based learning component further extracts complementary features from interaction data alone, also on the basis of user-item pairs. The final rating score is then derived with a dynamic linear fusion mechanism. Experiments on five real-world datasets show that CARL achieves significantly better rating prediction accuracy than existing state-of-the-art alternatives. Also, with attention mechanism, we show that the relevant information in reviews can be highlighted to interpret the rating prediction.
1 INTRODUCTION
CARL addresses the limitation of static, independently learned user and item representations by learning context-aware representations for user-item pairs from reviews and interactions. It combines review-based and interaction-based features for rating prediction, and experiments report improved performance across five real-world datasets.
- Motivation: Existing review-based methods learn user and item features independently and statically, overlooking which review information is relevant to a particular candidate item.The paper motivates jointly considering user and item reviews to identify relevant semantic information for each user-item pair.
- Proposed approach: CARL uses convolution operations and attention to extract relevant semantic information by jointly considering a user's reviews and an item's reviews.The attention mechanism is intended to identify pair-relevant review information and support interpretation of recommendation decisions.
- Proposed approach: CARL contains separate review-based and interaction-based feature learning components that exploit textual reviews and user-item rating scores.The two components provide complementary information for the final rating prediction.
- Results: Experiments on five real-world datasets report that CARL significantly outperforms existing state-of-the-art alternatives in rating prediction accuracy.The experiments also validate that fusing the two feature-learning components improves prediction performance.
- Proposed approach: CARL learns latent representations for user-item pairs from individual characteristics and their feature interactions instead of using static user or item vectors.The model treats the pair as the basis for representation learning and further models feature interactions for rating prediction.
2 RELATED WORK
Related work spans interaction-based collaborative filtering, text-based rating prediction, neural recommendation, and context-based feature learning. CARL differs by learning context-aware representations for user-item pairs from reviews and rating scores.
- 2.1 Interaction based Collaborative Filtering: Interaction-based collaborative filtering commonly represents users and items with static latent vectors learned from user-item interaction matrices.Matrix factorization methods typically use dot products to match user and item latent features for rating prediction.
- 2.2 Rating Prediction from Text: Collaborative filtering methods suffer when interaction data is sparse and cannot handle new users or items, motivating the use of textual auxiliary information.Reviews, item descriptions, and labels are identified as common auxiliary sources for addressing these limitations.
- 2.2 Rating Prediction from Text: Text-based rating prediction methods use topic models, matrix factorization, and neural networks to derive latent features from review text.Examples include LDA-like topic models, jointly modeled topics and interactions, and CNN-based semantic representations.
- 2.3 Attention-based Recommendation: Attention mechanisms identify important words or features in auxiliary information and can provide semantic interpretations for recommendations.Prior attention-based systems combine weighted review text with neural representation learning to distinguish relevant information.
- 2.4 Context-based Features Learning: CARL extends context-aware representation learning to user-item pairs by jointly using review data and user-item rating scores.The paper presents this pair-based context-aware learning as its distinction from static embedding approaches.
3 THE PROPOSED MODEL
CARL predicts ratings from context-aware user-item representations learned from review and interaction data. Its review component uses convolution and attention to identify pair-relevant semantics before combining them into the pair representation.
- Overall framework: CARL uses independent review-based and interaction-based feature-learning components to estimate personalized ratings from heterogeneous review and interaction data.The model is designed for new user-item pairs and combines information from item reviews and the user-item interaction matrix.
- Rating prediction framework: Factorization Machines model first- and second-order interactions in the context-aware pair vector for rating prediction.The authors report that Factorization Machines outperform linear regression and multilayer perceptrons in their experiments.
- Rating prediction framework: User and item biases are added to accommodate rating tendencies such as users or items consistently receiving higher scores.The predictor is optimized with square loss and regularization to reduce overfitting.
- Review-based feature learning: Review-based learning merges each user’s reviews and each item’s reviews into documents, then applies convolution to extract contextual features.Word order is preserved through document matrices, while sliding-window filters capture local semantic context rather than bag-of-words statistics.
- Review-based feature learning: Attention computes pairwise relatedness between user and item contextual features, assigning larger weights to aspects more relevant to the specific pair.The weighted review features are combined with user- and item-specific representations to form the context-aware latent feature vector.
- Review-based feature learning: A further convolution over relevance-weighted features incorporates a broader 2·s−1-word context while reducing noise in higher-level semantic extraction.This design uses the attention weights to make the subsequent semantic representation more precise for the user-item pair.
4 EXPERIMENT
Experiments across five real-world datasets show that CARL consistently improves rating prediction by combining context-aware review and interaction representations. Additional analyses indicate benefits from dynamic fusion, attention, latent feature interactions, and review-based modeling, while attention also supports interpretation.
- Overall Performance: Review information improves rating prediction over interaction-only PMF, which performs worst across all five datasets.The experiments associate review-based methods with richer semantic information for understanding rating behavior.
- Overall Performance: CARL achieves the best MSE scores across all five datasets, with a 4.55% average improvement over the best baseline and 4.17% on Tools Improvement.The improvements are described as consistent and stable across datasets with different characteristics.
- Parameter Analysis: CARL remains effective across latent dimensions l from 15 to 300, with little performance variation and near-optimal accuracy at l = 15 on most datasets.Larger dimensions can provide a small additional gain for Office Products, but the passage does not report the complete condition for that gain.
- Component Analysis: The review-based and interaction-based components complement each other, and their combination produces better prediction accuracy.The interaction-based component performs especially well on denser datasets, while review-based learning is stronger on sparse datasets.
- Fusion Analysis: Dynamic linear fusion outperforms static combinations across different α settings and removes the need to tune the static fusion parameter.The comparison is reported for Video Games and Musical Instruments.
- Review-based Feature Analysis: The attentive layer improves review-based prediction by 4.76% on average and highlights pair-specific review information for interpretation.Different items associated with the same user receive different attention patterns over review words and aspects.
5 CONCLUSION
CARL learns context-aware user-item representations by combining reviews and rating scores, and it consistently outperforms existing state-of-the-art alternatives across five real-world datasets. Attention also supports semantic interpretations of recommendation decisions.
- CARL learns user-item representations from individual characteristics and their interactions, jointly exploiting reviews and rating scores.
- CARL consistently outperforms existing state-of-the-art alternatives over five real-world datasets.
- Attention-based review processing highlights semantic information that can interpret recommendation decisions.
- Future work will incorporate sentiment factors and unify reviews with rating scores in a jointly learned user-item representation.