Source-linked AI summary
A Tale of Two Graphs: Freezing and Denoising Graph Structures for Multimodal Recommendation
Xin Zhou, Zhiqi Shen
TL;DR
Multimodal recommendation benefits from item content, but prior feature fusion does not capture latent item-item structures, motivating scrutiny of LATTICE’s learned graph. FREEDOM freezes an item-item graph, denoises the user-item graph with degree-sensitive pruning, and achieves stronger accuracy with lower memory cost than LATTICE.
Problem
Prior multimodal recommendation methods fuse content into item representations without capturing latent semantic item-item structures, while LATTICE’s learned graph is argued to be inefficient and unnecessary.
Method
FREEDOM freezes an item-item graph built from raw multimodal features and denoises the user-item graph using degree-sensitive edge pruning.
Results
FREEDOM improves LATTICE by an average of 19.07% in recommendation accuracy across three datasets and reduces its memory cost by up to 6× on large graphs.
Takeaways & Limitations
Freezing the item-item graph can achieve better performance while avoiding learned graph structure, and degree-sensitive pruning outperforms random edge dropout.
Abstract
from arXiv · showhide
Multimodal recommender systems utilizing multimodal features (e.g., images and textual descriptions) typically show better recommendation accuracy than general recommendation models based solely on user-item interactions. Generally, prior work fuses multimodal features into item ID embeddings to enrich item representations, thus failing to capture the latent semantic item-item structures. In this context, LATTICE proposes to learn the latent structure between items explicitly and achieves state-of-the-art performance for multimodal recommendations. However, we argue the latent graph structure learning of LATTICE is both inefficient and unnecessary. Experimentally, we demonstrate that freezing its item-item structure before training can also achieve competitive performance. Based on this finding, we propose a simple yet effective model, dubbed as FREEDOM, that FREEzes the item-item graph and DenOises the user-item interaction graph simultaneously for Multimodal recommendation. Theoretically, we examine the design of FREEDOM through a graph spectral perspective and demonstrate that it possesses a tighter upper bound on the graph spectrum. In denoising the user-item interaction graph, we devise a degree-sensitive edge pruning method, which rejects possibly noisy edges with a high probability when sampling the graph. We evaluate the proposed model on three real-world datasets and show that FREEDOM can significantly outperform current strongest baselines. Compared with LATTICE, FREEDOM achieves an average improvement of 19.07% in recommendation accuracy while reducing its memory cost up to 6$\times$ on large graphs. The source code is available at: https://github.com/enoche/FREEDOM.
1 INTRODUCTION
Multimodal recommendation improves item representations but conventional fusion does not capture latent item-item semantics. FREEDOM shows that LATTICE’s item-item structure learning can be frozen, then combines graph freezing with user-item graph denoising.
- Multimodal recommender systems typically outperform models using only historical user-item interactions.
- Conventional methods fuse projected multimodal features with item ID embeddings, while graph-based methods capture higher-order semantics.
- LATTICE-Frozen slightly outperforms original LATTICE on Baby and Sports for Recall and NDCG.
- FREEDOM freezes the item-item graph and denoises the user-item graph for multimodal recommendation.It constructs the item-item graph from raw multimodal contents and applies degree-sensitive edge pruning to the user-item graph.
2 RELATED WORK
Related work integrates multimodal content through feature fusion, attention, and graph neural networks. Graph denoising methods differ in whether edge pruning is permanent or temporary and in how pruning decisions are determined.
- Multimodal recommendation extends collaborative filtering by fusing visual or stylistic content with item ID embeddings.
- Attention mechanisms model users’ preferences for different image regions or textual and visual item features.
- Graph denoising methods use permanent or temporary edge pruning based on scores, predefined functions, learned distributions, or uniform sampling.DropEdge is described as a widely used method that prunes edges following a uniform distribution.
3 FREEZING AND DENOISING GRAPH STRUCTURES
FREEDOM constructs and freezes a multimodal item-item graph before training while repeatedly denoising the user-item graph during training. It then propagates information over both graphs to produce user and item representations for recommendation.
- 3.1 Constructing Frozen Item-Item Graph: FREEDOM constructs modality-aware item-item graphs from raw multimodal features using cosine similarity, top-k sparsification, normalization, and modality-weighted aggregation.Visual and textual modalities are combined using importance scores, with textual importance set to 1 − α_v.
- 3.1 Constructing Frozen Item-Item Graph: The aggregated item-item graph is precomputed and frozen, removing its O(N^2d_m) graph-construction burden during training.FREEDOM calculates the similarity matrix before training rather than reconstructing it at each training step.
- 3.2 Denoising User-Item Bipartite Graph: FREEDOM samples a fixed number of user-item edges with degree-sensitive probabilities, making edges between high-degree nodes more likely to be pruned.The sampled adjacency matrix is renormalized each training epoch, while inference uses the original normalized adjacency matrix.
- 3.3 Integration of Two Graphs for Learning: The model applies lightweight GCN propagation separately to the frozen item-item graph and the sampled, normalized user-item graph.The item-item graph uses L_ii convolutional layers, while the user-item graph uses L_ui layers and a readout over hidden representations.
- 3.3 Integration of Two Graphs for Learning: User representations come from the user-item graph, while item representations sum outputs from the user-item and item-item graphs.Projected multimodal features are also generated through modality-specific MLPs.
- 3.4 Top-K Recommendation: FREEDOM optimizes pairwise BPR loss and ranks candidate items by predicted interaction scores to select the top K recommendations.Prediction uses user and item ID embeddings; projected multimodal features support user-representation learning through the reconstruction losses.
4 SPECTRAL ANALYSIS
FREEDOM’s frozen item-item matrix has a tighter upper bound on its eigenvalues than LATTICE’s corresponding matrix. Empirical eigenvalue calculations and validation-accuracy plots support the spectral analysis and show higher, more stable accuracy for FREEDOM.
- The analysis compares normalized item-item adjacency matrices for FREEDOM and LATTICE within a specific modality.
- FREEDOM’s frozen item-item matrix possesses a tighter upper bound on its eigenvalues than LATTICE’s corresponding matrix.
- The largest eigenvalues of the item-item matrices are computed on different datasets to validate the theoretical analysis.
- A tighter eigenvalue bound lets FREEDOM’s frozen item-item graph act as a low-pass filter that eliminates negative coefficients at large frequencies.
- FREEDOM achieves higher accuracy and more stable validation performance than LATTICE in the frozen-versus-learnable graph comparison.
5 EXPERIMENTS
Experiments on three Amazon review datasets compare FREEDOM with general collaborative-filtering and multimodal recommendation baselines under standardized top-K evaluation. Results show that freezing and denoising graph structures improves recommendation accuracy while reducing efficiency costs, with ablations and modality studies examining component contributions and sensitivity.
- Experimental Setup: Experiments use Baby, Sports, and Clothing Amazon review datasets with visual and textual item features.The data are 5-core filtered, using 4,096-dimensional visual features and 384-dimensional textual embeddings.
- Experimental Setup: Evaluation reports Recall@K and NDCG@K at K=10 and K=20 using an 80%/10%/10% train-validation-test split.Negative sampling pairs each observed training interaction with one unobserved item, and all-ranking computes recommendation metrics.
- Performance Comparison: FREEDOM improves over LATTICE by an average of 19.07% across datasets and outperforms the other evaluated baselines.The reported gains are attributed to denoising false-positive user-item interactions and freezing an item-item graph built from raw multimodal features.
- Efficiency: FREEDOM reduces LATTICE’s memory cost by 6× and training time by 4× on Clothing.Pre-building and freezing the item-item graph removes its per-epoch construction burden, making FREEDOM preferable for large graphs.
- Ablation Study: Freezing contributes substantially to accuracy, while degree-sensitive denoising provides additional gains over freezing alone.Random edge dropout performs slightly better than freezing alone but remains worse than FREEDOM, supporting degree-sensitive pruning.
- Hyperparameter Sensitivity Study: Textual features appear more informative than visual features for constructing an effective item-item graph.FREEDOM varies the visual-feature ratio from 0.0 to 1.0 in increments of 0.1 when building the graph.
- Hyperparameter Sensitivity Study: Large graphs favor a high edge-pruning ratio, whereas performance is less sensitive to the multimodal loss weight.A high multimodal-loss weight may limit FREEDOM’s expressive power and degrade performance.
6 CONCLUSION
The paper finds that LATTICE’s graph structure learning plays a trivial role and proposes FREEDOM, which freezes the item-item graph while denoising the user-item graph. Experiments and analysis support FREEDOM’s effectiveness and efficiency.
- LATTICE’s graph structure learning plays a trivial role in recommendation performance.
- FREEDOM’s performance is evaluated across different loss weights 𝜆 and edge pruning ratios 𝜌 on the Baby and Clothing datasets.
- FREEDOM freezes the item-item graph and denoises the user-item graph simultaneously for multimodal recommendation.Its denoising component uses degree-sensitive edge pruning and performs better than random edge dropout.
A PROOF OF LEMMA 4.1
The proof compares normalized adjacency matrices for LATTICE and FREEDOM. It uses monotonicity of a matrix-entry function to identify the diagonal as the location of the maximum value.
- The proof denotes LATTICE’s normalized adjacency matrix as e𝑺′ and FREEDOM’s as e𝑺.
- The function g(˜𝑠′𝑖𝑗) is monotonic increasing, so maximizing the relevant expression requires maximizing ˜𝑠′𝑖𝑗.
- The maximum normalized adjacency value for either matrix lies on the diagonal and is expressed as 1/d𝑖𝑖.