Mining Latent Structures for Multimedia Recommendation

• Multimedia recommendation predicts user interactions with items containing modalities such as images, text, and videos.Source: §1 · p. 1
• Traditional methods incorporate multimodal contents as item side information but primarily model direct user-item interactions, limiting their expressiveness.Source: §1 · p. 1
• Prior graph-based methods model high-order user-item relationships, but item relationships are only implicitly discovered through item-user-item co-occurrences.Source: §1 · p. 1
• Semantic item relationships from multimodal features can supplement collaborative signals and help recommend visually or semantically related items that interaction patterns miss.Source: §1 · p. 2, Figure 1 · p. 2
• LATTICE learns modality-aware item structures, aggregates modality-specific graphs, and applies graph convolutions to inject high-order item relationships into item representations.Source: §1 · p. 2
• LATTICE mines item graphs from multimodal features while applying graph convolutions to ID embeddings, then adds the resulting representations to downstream CF item embeddings.Source: §1 · p. 2
• LATTICE addresses limited-interaction settings through neighbor feedback and serves as a plug-and-play module for existing recommender models.Source: §1 · p. 2

Modality-aware latent structure learning

• LATTICE learns item graph structures separately from multimodal features, fuses them, and uses the resulting latent graph to model item relationships.Its framework combines modality-aware structure learning, graph convolutional embedding, and downstream CF methods.Source: §2 · p. 2, §2.2 · p. 2, Figure 2 · p. 3
• For each modality, LATTICE constructs a cosine-similarity kNN graph, suppressing negative similarities and retaining only top-k edges.The resulting adjacency matrices are sparsified and normalized for computational efficiency and gradient stability.Source: §2.2 · pp. 2–3, §2.2.2 · p. 3

Learning latent structures

• The model dynamically learns graph structures from transformed high-level features and combines them with initial raw-feature graphs through a skip connection.The coefficient λ controls how much information is retained from the initial structure.Source: §2.2.2 · p. 3

Aggregating multimodal latent graphs

• Learned modality-specific graphs are adaptively fused with learnable modality weights into a normalized graph representing multimodal item relationships.The weights assign different importance scores to modality-specific graphs.Source: §2.2.3 · pp. 3–4

Graph convolutions

• Graph convolutions propagate item representations across neighbors, and stacking layers captures high-order item-item relationships constructed from multimodal information.LATTICE uses simple message passing without feature transformations or nonlinear activations, then enhances downstream CF item embeddings with the learned representations.Source: §2.3 · p. 4, §2.4 · p. 4

Combining with collaborative filtering

• LATTICE combines its item representations with downstream CF methods and is designed as a plug-and-play module for different collaborative filtering methods.The framework enhances item embeddings before computing user-item preference scores.Source: §2.4 · p. 4

• The experiments compare LATTICE with state-of-the-art multimedia recommendation and collaborative filtering methods.Source: §3 · p. 4
• Evaluation covers both warm-start and cold-start recommendation settings.Source: §3 · p. 4
• The study examines the effectiveness of item graphs learned from multimodal features.Source: §3 · p. 4
• The experiments assess sensitivity to several key hyperparameters.Source: §3 · p. 4

Datasets

• Experiments use Amazon Clothing, Sports, and Baby datasets containing visual and textual modalities.Visual features have 4,096 dimensions, while textual embeddings have 1,024 dimensions.Source: §3.1.1 · p. 4

Baselines

• Baselines comprise CF methods MF, NGCF, and LightGCN, plus content-aware models VBPR, MMGCN, and GRCN.Source: §3.1.2 · p. 5

Evaluation Settings

• Warm-start data splits each user’s historical interactions into 80% training, 10% validation, and 10% testing.Observed positives are paired with one negatively sampled item.Source: §3.1.3 · p. 5
• Cold-start evaluation removes interactions for a randomly selected 20% item set, dividing those unseen items equally between validation and testing.Source: §3.1.3 · p. 5

Metrics

• Performance is measured with Recall@20, NDCG@20, and Precision@20, averaged across testing users.Source: §3.1.3 · p. 5

Overall Performance

• 12.5%, 9.8%, and 9.9% are LATTICE’s Recall@20 improvements over the strongest baselines on Clothing, Sports, and Baby, respectively.The reported improvements are significant with p-value ≤0.05.Source: §3.2.1 · p. 6
• LATTICE significantly outperforms both collaborative filtering and content-aware methods across the evaluated datasets.The comparison includes CF and content-aware baselines, with Table 2 reporting Recall@20, Precision@20, and NDCG@20.Source: §3.2.1 · p. 6

Cold-start Performance

• LATTICE outperforms all baselines across Clothing, Sports, and Baby in cold-start settings.Its learned item graphs transfer feedback from relevant neighbors to cold-start items through neighborhood aggregation.Source: §3.2.2 · p. 6
• MF and LightGCN generally perform poorly in cold-start settings because unseen items lack user-item interactions for representation updates.Source: §3.2.2 · p. 6
• VBPR generally outperforms CF methods for cold-start items, whereas GRCN performs poorly because cold-start items become isolated in its user-item graphs.Source: §3.2.2 · p. 7

Ablation Results

• LATTICE–CF consistently outperforms original CF methods and two comparison variants when combined with MF, NGCF, and LightGCN.Source: §3.3 · p. 7
• 17.6% average improvements are obtained over CF+feats variants that directly use multimodal features as item side information.Source: §3.3 · p. 7
• LATTICE–CF applies graph convolutions to item ID embeddings using the same learned structures as LATTICE/feats–CF.This design aims to directly model item affinities.Source: §3.3 · p. 7

Impact of varied k values

• LATTICE improves significantly from k=0 to k=10, supporting the usefulness of item relationships mined from multimodal features.Here, k=0 excludes item relationships and reduces the model to original LightGCN.Source: §3.4 · p. 7
• Performance initially increases with k because aggregating more item neighbors provides additional meaningful information.Source: §3.4 · p. 7
• Performance declines when k becomes too large because unimportant neighbors introduce noise during information propagation.The result motivates kNN sparsification of the learned dense graph.Source: §3.4 · p. 7

Impact of varied coefficients

• With λ=0, performance is poor because the model relies only on transformed high-level features and continually updates the adjacency matrix.Source: §3.4.2 · p. 8
• Performance first rises as λ increases, then deteriorates when raw multimodal features receive excessive weight because those features can be noisy.Source: §3.4.2 · p. 8
• The absence of sharp performance changes across λ values indicates that LATTICE is not highly sensitive to selecting this coefficient.All tested models surpass the baselines according to the cited analysis.Source: §3.4.2 · p. 8

Multimedia Recommendation

• CF-based multimedia recommenders extend collaborative filtering by adding multimodal item features as side information.Examples include VBPR, DVBPR, Sherlock, DeepStyle, and ACF.Source: §4.1 · p. 8
• Existing multimedia methods generally use multimodal features directly for each item rather than discovering relationships among items.LATTICE instead mines fine-grained item-item relationships from those features.Source: §4.1 · p. 8

Graph Structure Learning

• Graph neural networks aggregate neighborhood information, but graph noise can propagate through repeated updates and affect many node embeddings.Source: §4.2 · p. 8
• Graph structure learning jointly optimizes graph structures and representations through metric learning, probabilistic modeling, or direct optimization.Source: §4.2 · p. 8
• NeuralSparse learns k-neighbor subgraphs by selecting at most k neighbors for each node, while other methods learn adaptive or probabilistic graphs.Source: §4.2 · p. 8
• Because item-item relations are underexplored in personalized recommendation, LATTICE uses metric learning to represent edge weights as distances between item nodes.This formulation accommodates multimodal content when measuring semantic relationships.Source: §4.2 · p. 8

• LATTICE learns and fuses modality-aware item graphs, then uses graph convolutions to inject high-order neighbor affinities into item representations.The enriched representations are combined with downstream collaborative filtering methods for recommendation.Source: §5 · p. 8
• Empirical results on three public datasets demonstrate the effectiveness of LATTICE.Source: §5 · p. 8