Source-linked AI summary
Multi-View Graph Convolutional Network for Multimedia Recommendation
Penghang Yu, Zhiyi Tan, Guanming Lu, Bing-Kun Bao
TL;DR
Existing multimedia recommendation methods can propagate modality noise and treat modalities equally despite users’ distinct modality preferences. MGCN purifies modality features with behavior information, enriches behavior and modality representations in separate graph views, and adaptively fuses modalities; experiments on three public datasets show effectiveness, including a 23.3% improvement over the best baseline on Clothing.
Problem
Existing GCN-based multimedia recommenders suffer modality-noise contamination and incomplete user-preference modeling from equal treatment of modality features.
Method
MGCN uses a behavior-guided purifier, separate user-item and item-item information encoding, and a behavior-aware fuser with a self-supervised auxiliary task.
Results
23.3% improvement over the best baseline methods on the Clothing dataset; MGCN outperforms existing methods on all three datasets.
Takeaways & Limitations
MGCN combines purified multimodal information with behavior information to model user preferences across three public multimedia-recommendation datasets.
Takeaways & Limitations
Future work proposes integrating external knowledge with item information through large-scale language models to potentially address cold-start recommendation.
Abstract
from arXiv · showhide
Multimedia recommendation has received much attention in recent years. It models user preferences based on both behavior information and item multimodal information. Though current GCN-based methods achieve notable success, they suffer from two limitations: (1) Modality noise contamination to the item representations. Existing methods often mix modality features and behavior features in a single view (e.g., user-item view) for propagation, the noise in the modality features may be amplified and coupled with behavior features. In the end, it leads to poor feature discriminability; (2) Incomplete user preference modeling caused by equal treatment of modality features. Users often exhibit distinct modality preferences when purchasing different items. Equally fusing each modality feature ignores the relative importance among different modalities, leading to the suboptimal user preference modeling. To tackle the above issues, we propose a novel Multi-View Graph Convolutional Network for the multimedia recommendation. Specifically, to avoid modality noise contamination, the modality features are first purified with the aid of item behavior information. Then, the purified modality features of items and behavior features are enriched in separate views, including the user-item view and the item-item view. In this way, the distinguishability of features is enhanced. Meanwhile, a behavior-aware fuser is designed to comprehensively model user preferences by adaptively learning the relative importance of different modality features. Furthermore, we equip the fuser with a self-supervised auxiliary task. This task is expected to maximize the mutual information between the fused multimodal features and behavior features, so as to capture complementary and supplementary preference information simultaneously. Extensive experiments on three public datasets demonstrate the effectiveness of our methods.
1 INTRODUCTION
Multimedia recommendation combines behavioral and multimodal item information, but existing methods face modality-noise contamination and incomplete preference modeling. MGCN addresses these issues with behavior-guided purification, separate multi-view encoding, and adaptive behavior-aware fusion.
- Existing methods mix modality and behavior features during propagation, allowing preference-irrelevant modality noise to spread and contaminate item representations.Examples of modality noise include redundant text, image backgrounds, and image brightness.
- MGCN introduces a behavior-guided purifier that filters preference-irrelevant modality features using behavior information.
- Its multi-view information encoder separately captures collaborative signals from user-item relationships and semantic signals from item-item relationships.
- A behavior-aware fuser adaptively combines modality features according to users’ modality preferences distilled from behavior features.
- The model adds a self-supervised auxiliary task intended to maximize mutual information between fused multimodal and behavior features.
2.1 Problem Definition
The problem represents users and items as nodes connected by observed interactions, with multimodal item features and the goal of ranking items by predicted user preference.
- Users and items form sets U and I, while each item has modality features indexed by modality m.The paper mainly considers visual and textual modalities but allows multiple modalities.
- Historical interactions define a sparse bipartite behavior graph whose edges connect users to clicked items.An interaction matrix entry is one when a user clicked an item and zero otherwise.
- Multimedia recommendation ranks items for each user according to predicted preference scores.
2.2 Behavior-Guided Purifier
The behavior-guided purifier transforms raw item modality features and separates preference-relevant information from modality representations using behavior features.
- The purifier is introduced because modality information contains useful item content alongside modality noise.
- Raw item modality features are transformed into high-level features before purification.
- The framework presents purification as a behavior-guided module that filters modality noise, alongside adaptive modality fusion in the overall architecture.
- Behavior features guide the separation of preference-relevant modality features from each item’s transformed modality representation.The purification uses a learnable transformation, element-wise product, and sigmoid nonlinearity.
2.3 Multi-View Information Encoder
The multi-view information encoder enriches behavior and modality representations through separate graph views: user-item propagation captures collaboration, while item-item propagation captures semantic similarity.
- Multi-View Information Encoder: The encoder separates collaborative signals from user-item relationships and semantically correlative signals from item-item relationships.
- User-Item View: In the user-item view, a GCN propagates user and item ID embeddings over the interaction graph to encode high-order neighbors.The l-th layer represents information from l-order neighbors, which are aggregated into final representations.
- User-Item View: The user-item graph uses the interaction matrix to define adjacency and degree-based normalization for propagation.The normalized factor depends on the neighboring users’ and items’ degrees.
- Item-Item View: In the item-item view, cosine similarities between modality features define affinity graphs, which are sparsified by retaining each item’s K greatest-similarity edges.
- Item-Item View: A GCN propagates modality features over each sparsified affinity graph to capture common features among similar items.The item-item module is kept shallow because deeper propagation can cause over-smoothing and noisy feature capture.
- Feature Construction: User modality features are obtained by aggregating the modality features of interacted items and concatenating them with item modality features.
2.4 Behavior-Aware Fuser
The behavior-aware fuser allocates modality weights from user behavior features, separates shared and modality-specific information, and uses self-supervision to align fused multimodal features with behavior features.
- Behavior-aware modality weighting: The fuser learns flexible modality weights from user behavior features to represent user-specific modality preferences.Modality preferences are first distilled from behavior features through a nonlinear sigmoid gate.
- Shared and specific features: Modality-shared features are extracted with attention because users’ attention is consistent for features shared across modalities.The design distinguishes shared information from modality-specific information before fusion.
- Shared and specific features: Modality-specific features are obtained by subtracting shared features, then adaptively fused with shared features into final multimodal representations.This fusion preserves shared information while allowing modality-specific contributions to vary.
- Self-supervised alignment: A self-supervised auxiliary task maximizes mutual information between behavior features and fused multimodal features during fusion.The task is intended to promote exploration of both behavior and multimodal information.
- Self-supervised alignment: The fusion contrastive objective uses a softmax temperature hyperparameter τ.The temperature controls the softmax used in the auxiliary task.
2.5 Predictor
The predictor forms final user and item representations from enhanced behavior and multimodal features, then estimates interaction likelihood using their inner product.
- Representation construction: Final user and item representations are formed from enhanced behavior features and multimodal features.These representations provide the inputs to interaction prediction.
- Interaction prediction: The predictor uses the inner product between user u and item i to determine interaction likelihood.This follows the paper’s adopted interaction-scoring approach.
2.6 Optimization
Model training uses Bayesian Personalized Ranking as the basic optimization task together with auxiliary self-supervised objectives and L2 regularization.
- Primary objective: Bayesian Personalized Ranking loss assumes users prefer historically interacted items over unclicked items.BPR is used as the basic optimization task during model training.
- Joint training: The BPR objective is jointly optimized with auxiliary self-supervised tasks to update user and item representations.The auxiliary tasks are combined with the main ranking objective during training.
- Objective weighting: The hyperparameters λ_C and λ_E control the contrastive auxiliary task and L2 regularization, respectively.These parameters regulate the contributions of the auxiliary contrastive objective and regularization.
3 EXPERIMENTS
Experiments on three Amazon datasets evaluate MGCN against collaborative-filtering and multimedia baselines, then examine its modules, modality inputs, hyperparameters, and representation distributions. MGCN achieves the strongest reported recommendation performance, with improvements attributed to purification, multi-view encoding, adaptive fusion, and self-supervision.
- Experimental Setup: MGCN is compared with general collaborative-filtering models and multimedia recommenders, including MF, LightGCN, VBPR, MMGCN, GRCN, SLMRec, and BM3.The baselines cover methods using interaction data alone and methods combining interaction and multimodal information.
- Overall Performance: MGCN significantly outperforms both general and multimedia recommendation models across the three datasets.On Clothing, it achieves a 23.3% improvement over the best baseline methods.
- Modality Analysis: Both textual and visual features improve performance, while visual features have a greater impact; text descriptions are described as more likely to contain irrelevant information.The modality comparison attributes the stronger visual contribution to users purchasing items based on appearance.
- Sensitivity Analysis: The item-neighbor setting typically performs best at k = 15, while the self-supervised loss weight is optimal near λ_C = 0.01 and declines when too large.Smaller neighborhoods can reduce noise from unrelated items, whereas excessive auxiliary-task weight can mislead the model.
4 RELATED WORK
Multimedia recommendation combines behavioral interactions with multimodal item information, increasingly using GCNs to model high-order user preferences. However, direct modality use can contaminate item representations with irrelevant features.
- Multimedia recommendation incorporates multimodal item information into collaborative filtering to better model user preferences.
- Directly using modality information can contaminate item representations with preference-irrelevant features.
- GCN-based methods use the user-item interaction graph to extract and enhance user behavior features.
5 CONCLUSION
The paper proposes MGCN to address modality noise and improve user-preference modeling through purification, multi-view encoding, behavior-aware fusion, and self-supervision. Future work will integrate external knowledge with item information through large-scale language models.
- MGCN uses a behavior-guided purifier to avoid modality noise contamination.
- It separately enriches purified modality features and behavior features through a multi-view information encoder.
- A behavior-aware fuser and self-supervised auxiliary task are introduced to comprehensively model user preferences.
- Future work aims to integrate external knowledge with item information through large-scale language models, potentially addressing cold-start recommendation.