PaperlayerView exact revision on arXiv

Source-linked AI summary

MMALFM: Explainable Recommendation by Leveraging Reviews and Images

Zhiyong Cheng, Xiaojun Chang, Lei Zhu, Rose C. Kanjirathinkal, Mohan Kankanhalli

arXiv:1811.05318v2cs.IR

TL;DR

Matrix factorization provides accurate rating prediction but remains vulnerable to cold-start, limited transparency, and poor individual user-item fit. MMALFM combines reviews and images through MATM and ALFM to model aspect importance and aspect ratings, with experiments reporting advantages over prior approaches and acknowledged ranking limitations.

  • Problem

    The paper addresses cold-start, non-transparency, and suboptimal predictions for individual user-item pairs in latent factor recommendation.

  • Method

    MATM jointly uses textual reviews and item images to learn aspect-level user preferences, item properties, and user-item aspect importance, which ALFM integrates with rating-based latent factors.

  • Results

    The model learns aspect ratings and combines them using aspect importance to predict overall ratings, with experiments conducted on Yelp 2017 and Amazon product datasets.

  • Takeaways & Limitations

    The approach is designed to improve rating prediction, sparse-data recommendation, and interpretability by linking review-image aspects with rating factors.

  • Takeaways & Limitations

    The authors do not claim optimal use of image and review information and report that MMALFM is not as good as JRL for top-n recommendation.

Abstract

from arXiv · show

Although the latent factor model achieves good accuracy in rating prediction, it suffers from many problems including cold-start, non-transparency, and suboptimal results for individual user-item pairs. In this paper, we exploit textual reviews and item images together with ratings to tackle these limitations. Specifically, we first apply a proposed multi-modal aspect-aware topic model (MATM) on text reviews and item images to model users' preferences and items' features from different aspects, and also estimate the aspect importance of a user towards an item. Then the aspect importance is integrated into a novel aspect-aware latent factor model (ALFM), which learns user's and item's latent factors based on ratings. In particular, ALFM introduces a weight matrix to associate those latent factors with the same set of aspects in MATM, such that the latent factors could be used to estimate aspect ratings. Finally, the overall rating is computed via a linear combination of the aspect ratings, which are weighted by the corresponding aspect importance. To this end, our model could alleviate the data sparsity problem and gain good interpretability for recommendation. Besides, every aspect rating is weighted by its aspect importance, which is dependent on the targeted user's preferences and the targeted item's features. Therefore, it is expected that the proposed method can model a user's preferences on an item more accurately for each user-item pair. Comprehensive experimental studies have been conducted on the Yelp 2017 Challenge dataset and Amazon product datasets to demonstrate the effectiveness of our method.

1 INTRODUCTION

The paper targets cold-start, limited transparency, and suboptimal individual user-item predictions in matrix factorization by linking reviews and images to aspect-aware rating factors.

  • Motivation: Matrix factorization suffers from cold-start when users or items have few ratings and can produce suboptimal individual recommendations because latent factors are optimized globally.Reviews provide complementary information about user preferences, item features, and reasons underlying ratings.
  • Motivation: Users discuss aspects they care about more often, allowing reviews to estimate aspect attention that varies across users and items.Examples include food, service, ambience, and price in restaurant reviews.
  • Approach: The proposed system uses reviews and item images in a multi-modal aspect-aware topic model to learn preferences, item properties, and user-item aspect importance.The learned results are integrated into an aspect-aware latent factor model for rating prediction.
  • Approach: ALFM introduces a weight matrix associating latent factors with aspects, enabling aspect ratings whose importance differs across user-item pairs.The overall rating is computed as a linear combination of aspect ratings weighted by corresponding aspect importance.
  • Contributions: Unlike prior models requiring one-to-one topic-factor mappings, the model separately tunes topic and factor parameters and automatically learns aspect weights for each user-item pair.This design is intended to improve local recommendation quality and flexibility.
  • Evaluation: Experiments report significantly better performance than previous approaches on rating prediction, sparse-data recommendation, and recommendation interpretability.The study uses benchmark datasets and also evaluates top-n recommendation with precision and NDCG.

2 RELATED WORK

Related work spans review-aware, visually aware, and multi-modal recommendation, while the paper distinguishes its aspect-level integration of review-image topics with rating factors.

  • Review-aware recommendation: Review-aware recommendation methods use ratings with review text through sentiment analysis, topic or aspect extraction, and related factorization approaches.Some methods assume reviews are available at prediction time, whereas real-world prediction targets uncommented and unrated items.
  • Review-aware recommendation: Topic-based methods commonly assume a one-to-one mapping between latent review topics and rating factors, whereas this paper removes that constraint.The paper represents aspects as distributions of latent topics rather than treating each topic or factor as an aspect directly.
  • Deep learning-based recommendation: Deep learning approaches use autoencoders, neural collaborative filtering, and deep textual or multi-modal models for recommendation.The cited multi-modal deep learning work combines heterogeneous information sources for top-n recommendation.
  • Visually aware recommendation: Visually aware recommendation incorporates item visual features, especially where appearance matters, but some methods require handcrafted techniques and carefully annotated data.Examples include clothing recommendation systems designed for specific domains.
  • Multi-modal topic models: Multi-modal topic models assume shared latent topics explain correlations between modalities, with prior work modeling text and image topics in different ways.The paper instead generates text and visual words from topic distributions associated with aspects.

3 THE PROPOSED MODEL

MMALFM models user–item satisfaction through explainable aspects, combining review and image evidence with aspect-aware latent factors. Its MATM estimates aspect preferences, item characteristics, and aspect importance, while ALFM estimates aspect ratings and combines them into overall predictions.

  • Problem setting: The model treats overall satisfaction as depending on aspect ratings and user-specific aspect importance for each item.Aspects are latent item attributes learned from reviews, with all items assumed to share the same aspect set.
  • Aspect-aware latent factor model: ALFM extends matrix factorization with a binary weight matrix that associates latent factors with explainable aspects.The aspect vector w_a selects factors related to aspect a, enabling aspect-specific rating estimation.
  • Aspect rating estimation: Each aspect rating combines review-derived user–item aspect matching with the corresponding aspect-specific latent-factor interaction.The matching score s_u,i,a and masked factor vectors jointly determine r_u,i,a.
  • Multi-modal aspect-aware topic model: MATM jointly models user interests, item characteristics, aspect distributions, and aspect importance from reviews and item images.User interests are inferred from text reviews, while item characteristics use reviews and visual content; latent topics generate textual and visual observations.
  • Multi-modal aspect-aware topic model: MATM represents review generation through either user preferences or item characteristics, with aspects and topics sampled accordingly.A user-level indicator parameterized by π_u selects the source, while λ_u and λ_i govern aspect selection.
  • Inference: The model estimates its topic-model parameters using collapsed Gibbs sampling and learns rating parameters by minimizing training prediction error.The rating model includes user, item, and global bias terms.

4 EXPERIMENTAL SETUP

The experiments use public Amazon and Yelp datasets, evaluate rating prediction and top-n recommendation, and compare MMALFM with tuned state-of-the-art baselines under controlled data splits and hyperparameter settings.

  • Datasets and tasks: The study evaluates rating prediction as its main task and additionally measures top-n recommendation performance.
  • Datasets and tasks: Experiments use publicly accessible Amazon review and Yelp Challenge datasets, incorporating reviews, ratings, and item images where available.The Amazon data include item metadata and images; the evaluation focuses on rating prediction and also includes top-n recommendation.
  • Baselines: MMALFM is compared with state-of-the-art rating-prediction methods whose hyperparameters are tuned on the validation set for fair comparison.
  • Hyperparameters: The number of latent factors and topics is tuned in [25], separately for TALFM and MMALFM but jointly for several competing methods.HFT, CTR, RMR, and RBLT require equal numbers of factors and topics.
  • Evaluation protocol: Each dataset is randomly split per user into training, validation, and testing sets with an 80:10:10 ratio.Reviews from validation and testing interactions are unavailable during prediction.
  • Metrics: RMSE evaluates rating prediction, with smaller values indicating better performance.The metric is computed from prediction errors over testing ratings.

5 EXPERIMENTAL RESULTS

The results examine latent-topic and factor settings, image integration, rating prediction, cold-start behavior, interpretability, and top-n recommendation. These analyses address the study’s six experimental questions across the reported tasks.

  • Experimental scope: The evaluation first studies how latent-topic and latent-factor numbers affect performance, then examines item-image effects and comparisons with state-of-the-art methods.
  • Experimental scope: The study evaluates whether the model alleviates cold-start problems when users have few ratings.
  • Experimental scope: The experiments assess whether the model provides explicit interpretations for high or low predicted ratings.
  • Experimental scope: The study also reports performance on the top-n recommendation task.

5.1 Model Analysis

Model analysis examines factor-topic settings and image contributions, including performance comparisons at f = K = 5. Images consistently improve performance, with larger gains for visually driven domains and limited gains where images poorly reveal product quality.

  • Factors and topics: Earlier models tie the number of latent factors and topics, whereas TALFM and MMALFM can tune these parameters separately.The analysis motivates testing whether equal factor and topic counts are optimal.
  • Item images: With image features, MMALFM consistently outperforms TALFM across datasets, although the gains are generally modest because reviews already provide semantic product details.
  • Item images: Image-based improvement is more obvious on Clothing, where users’ preferences are strongly affected by visual features.
  • Performance comparisons: Table 3 compares model performance using RMSE with f = K = 5, and statistically significant differences from TALFM are marked with an asterisk.
  • Item images: Improvements are limited in domains such as movies and CDs because product quality is difficult to judge from images.

5.2 Performance Comparison (RQ3)

MMALFM and TALFM outperform rating-only and review-based baselines by modeling preferences across aspects, with MMALFM additionally using item images. The results attribute gains to fine-grained preference modeling and multimodal information.

  • Model comparison: MMALFM outperforms TALFM by combining text reviews and item images in preference modeling.The paper reports that MMALFM performs better than TALFM across datasets, while image gains vary with visual importance.
  • Model comparison: 17.52% average RMSE improvement distinguishes TALFM from BMF when f = K = 5.Against review-based methods, average improvements range from 10.07% over EFM to 2.04% over RBLT.
  • Model comparison: Fine-grained aspect modeling helps TALFM outperform review-based competitors that do not carefully model users’ aspect preferences.The paper contrasts this with external sentiment tools in EFM and topic-based methods lacking explicit aspect preference modeling.
  • Model comparison: Neural review representations in TransNet are less competitive here, partly because noisy review information can deteriorate performance.The passage also notes that TransNet uses reviews as user and item representation inputs.

5.3 Cold-start Setting (RQ4)

The cold-start evaluation measures RMSE gains for users with only 1–10 training ratings. TALFM substantially improves over rating-only BMF and outperforms review-based baselines in this sparse setting.

  • Results: Table 4 reports the percentages of Clothing and Yelp users having 1–10 training samples.The table organizes percentages by the specific number of training samples.
  • Evaluation setup: Fig. 6 reports RMSE gain over baselines as average baseline RMSE minus MMALFM RMSE, grouped by historical-rating count.Positive values indicate better prediction by the proposed model.
  • Results: TALFM substantially improves over rating-only BMF and greatly outperforms review-based baselines for users with limited ratings.The passage presents these comparisons as evidence of the model’s cold-start capability.

5.4 Model Interpretability (RQ5)

MMALFM represents preferences through aspect-specific topics, aspect importance, matching scores, and sentiment polarity. Examples show how these quantities explain ratings for particular user–item pairs.

  • Interpretability mechanism: The model decomposes a user’s preference for an item into preferences on aspects and the importance of those aspects.Aspects are represented through latent-topic distributions, while aspect attitudes are determined by rating-trained latent factors.
  • Interpretability mechanism: Top aspect terms are extracted from topic distributions for one user and two items, with background words removed.The reported terms highly match their corresponding aspects.
  • User–item examples: For user_2397, Food, Service, and Ambience receive greater importance than Price.Table 6 compares aspect importance, matching scores, and sentiment polarity for items 137 and 673.
  • User–item examples: Item_137 matches the user’s Food and Service preferences with positive sentiment, whereas item_673 has poor Food matching and negative Ambience sentiment.The passage notes that aspect matching derives from reviews and can coexist with negative comments.
  • Interpretability conclusion: Aspect importance, matching, and sentiment polarity jointly explain recommendation reasons in depth.These quantities connect user priorities, item properties, and attitudes for the examined items.

5.5 Top-n Recommendation (RQ6)

MMALFM achieves the best Top-n recommendation performance across datasets and evaluation metrics by combining ratings, reviews, and images. The results also show benefits from fine-grained aspect modeling, while image gains depend on item appearance.

  • Evaluation setup: The Top-n experiment uses 70% of each user’s interactions for training and 30% for testing.Evaluation uses Precision, Hit Ratio, and NDCG, which assess correctness, users receiving at least one correct item, and ranking position.
  • Experimental results: MMALFM achieves the best performance across datasets and evaluation metrics among the considered competitors.The model integrates images, reviews, and ratings for preference modeling.
  • Experimental results: TALFM outperforms BPR-HFT and EFM by a large margin, supporting fine-grained modeling of users’ preferences across aspects.The comparison is reported as evidence for aspect-level preference modeling in Top-n recommendation.
  • Multimodal effects: Adding images to TALFM improves performance, especially for items whose visual appearance is important, such as clothing.Image-only VBPR is weaker than rating-only BPR, and image benefits are limited for Movies and CDs.
  • Conclusion: The study concludes that aspect modeling, item images, and integrated reviews can improve preference inference and recommendation accuracy.The conclusion attributes the supported benefits to capturing user preferences on different item aspects.

6 CONCLUSIONS

MMALFM combines reviews and item images in aspect-aware topic and latent-factor models to estimate aspect importance and ratings. It improves rating prediction, particularly for users with few ratings, while acknowledging that its rating-error objective is not designed for ranking.

  • 6 CONCLUSIONS: MATM learns users’ interests and items’ properties from reviews and images, while ALFM integrates these results into aspect-aware latent factors.The model jointly estimates aspect ratings and aspect importance for overall rating prediction.
  • 6 CONCLUSIONS: MMALFM outperforms existing review-based personalized rating prediction methods, especially for users with few ratings.The paper reports this result on a public dataset.
  • 6 CONCLUSIONS: Item images with visual features related to important item properties improve performance to some extent.
  • 6 CONCLUSIONS: The model provides detailed interpretations of recommendation results by modeling interactions between latent topics and latent factors at the aspect level.
  • 6 CONCLUSIONS: MMALFM minimizes rating prediction error rather than using an objective specifically designed for ranking.Consequently, the paper states that it is not expected to compete with JRL on top-n recommendation.
Loading 1811.05318v2…