Source-linked AI summary
Beyond Modality Harmony: Orthogonal Purification and Topology-Guided MoE for Conflict-Aware Multimodal Recommendation
Jialin Liu, Zhaorui Zhang, Ray C. C. Cheung
TL;DR
Multimodal recommenders often assume that modalities harmonize with collaborative behavior, despite conflicts that can distort latent representations. OrthoRec combines collaborative-guided purification, topology-aware decoupled routing, and safe contrastive learning to address these conflicts. It consistently improves accuracy over competitive baselines and improves robustness to modality noise and item sparsity in the evaluated settings.
Problem
Multimodal recommendation commonly assumes that multimodal features are beneficial and aligned with collaborative patterns, despite conflicts from deceptive or mismatched semantics.
Method
OrthoRec combines CGOP geometric purification, decoupled TAR-MoE routing, and safe-SSL to filter conflicting directions and regulate multimodal injection and alignment.
Results
OrthoRec consistently improves accuracy over competitive baselines and robustness under modality noise and item sparsity within the evaluated settings.
Takeaways & Limitations
The evaluated framework supports conflict-aware multimodal recommendation rather than assuming that all modality signals should be aligned or fused equally.
Abstract
from arXiv · showhide
Multimodal Recommender Systems (MRSs) typically rely on a flawed "modality harmony" assumption, presuming that multimodal features are inherently beneficial and strictly aligned with users' collaborative interaction patterns. However, modality-topology conflicts are ubiquitous in real-world scenarios due to deceptive visual clickbaits and mismatched semantics. Blindly integrating these noisy modalities inevitably pollutes the pristine collaborative space, causing severe representation distortion. To address this, we propose Orthogonal purification and topology-guided MoE for conflict-aware multimodal Recommendation (OrthoRec). At its core, OrthoRec introduces Collaborative-Guided Orthogonal Purification (CGOP), which geometrically decouples multimodal features into directions parallel and orthogonal to a pure collaborative anchor. By adaptively truncating the orthogonal noise with an energy-preserving normalization, CGOP rectifies deceptive semantic directions while preserving the modality's intrinsic representation capacity. Furthermore, we design a Topology-Aware Routing Mixture-of-Experts (TAR-MoE). Guided by the collaborative topology, TAR-MoE employs decoupled sigmoid gating to break the zero-sum bottleneck of traditional softmax attention, autonomously determining the injection scale for each purified modality. Finally, a safe-SSL objective is introduced to dynamically penalize the forced contrastive alignment of contradictory pairs. Experiments on three real-world Amazon datasets show that OrthoRec consistently outperforms competitive recent baselines and exhibits improved robustness under modality noise and item sparsity.
1 Introduction
OrthoRec challenges the modality harmony assumption by identifying modality-topology conflicts that distort collaborative representations. It addresses these conflicts through orthogonal purification, topology-aware routing, safe contrastive learning, and evaluation on three Amazon datasets.
- Problem: Multimodal features can conflict with collaborative patterns because of visual clickbaits, exaggerated claims, and mismatched semantics.These deceptive signals introduce false-positive user-item edges and distort the latent representation space.
- Problem: Blind alignment or fusion can pollute the collaborative space, while softmax attention forces zero-sum competition among modalities.The resulting distortion can collapse user preferences toward spurious visual appeal.
- Method: OrthoRec uses collaborative-guided orthogonal purification to disentangle deceptive modality directions without arbitrarily discarding multimodal features.The method treats topological collaborative embeddings as a trustworthy anchor and applies geometric decoupling.
- Method: TAR-MoE uses the collaborative anchor for routing and decoupled sigmoid gates to set each purified modality’s injection scale independently.A safe-SSL objective also down-weights contrastive penalties for inherently contradictory modalities.
- Evaluation: OrthoRec consistently outperforms competitive recent baselines on three real-world Amazon datasets and is more robust to modality noise and item sparsity.The contribution summary reports both accuracy improvements and robustness under the stated stressors.
2 Related Work
Related work addresses multimodal recommendation through graph modeling, contrastive learning, and denoising, but these approaches often retain harmony, alignment, or fusion assumptions. OrthoRec targets the resulting conflicts with purification and routing mechanisms designed to preserve modality utility.
- Multimodal graph methods: Multimodal GNNs model user-item connectivity through modality-specific graphs or latent semantic structures, but often use early concatenation or heuristic summation.These fusion choices remain vulnerable to destructive interference from modality-topology conflicts.
- Contrastive learning: Contrastive learning reduces sparsity by aligning multimodal views with ID embeddings or distilling modality-invariant representations.Existing approaches can distort latent representations when inherently contradictory visual and textual modalities are forced to align.
- Denoising and fusion: Disentanglement and denoising methods face trade-offs because heuristic masking discards features, while softmax gating restricts independent multimodal utilization.OrthoRec instead combines purification with routing that determines modality injection scales without zero-sum constraints.
3 Methodology
OrthoRec first builds a pure collaborative anchor, then purifies multimodal features against it before topology-guided fusion and safe contrastive optimization. Its modules address deceptive modality noise, representation-capacity loss, zero-sum fusion, and forced alignment of contradictory modalities.
- Collaborative Anchor Extraction: Pure ID embeddings are propagated over the user-item graph to establish a multimodal-independent collaborative anchor for subsequent processing.The anchor can be enhanced with a normalized global hypergraph embedding to capture global structural dependencies.
- Collaborative-Guided Orthogonal Purification: CGOP projects each modality onto the collaborative anchor, separating a topology-consistent parallel component from an orthogonal component containing unique information and possible noise.The orthogonal component may include beneficial exploratory semantics as well as deceptive clickbait noise.
- Collaborative-Guided Orthogonal Purification: A cosine-based conflict score drives soft gating of the orthogonal component, with lower or negative scores indicating more severe modality-topology conflict.The score is batch-centered so gating responds to relative item-level conflict severity, and it also guides safe global alignment.
- Collaborative-Guided Orthogonal Purification: Energy-preserving normalization rescales each truncated feature to retain its original L2 magnitude while rectifying deceptive semantic directions.Direct truncation would cause magnitude decay and reduce intrinsic representation capacity.
- Topology-Aware Routing MoE: TAR-MoE uses the collaborative topology as its routing condition and decoupled sigmoid gates to determine each purified modality’s injection scale independently.This avoids the zero-sum competition imposed by softmax fusion, while identity-initialized experts and L2 normalization stabilize injected magnitudes.
- Conflict-Aware Safe Contrastive Learning: Safe-SSL dynamically down-weights contrastive penalties for contradictory modality pairs, protecting the purified semantic space from optimization distortion.Forced alignment between inherently contradictory visual and textual modalities can otherwise distort latent representations.
4 Experiments
Experiments evaluate OrthoRec on three Amazon datasets using ranking metrics, baseline comparisons, ablations, robustness tests, popularity groups, hyperparameter sensitivity, and latent-space analyses. OrthoRec ranks first across reported metrics, while its design choices support performance under modality noise and item sparsity.
- Overall Performance: OrthoRec ranks first on every reported metric across all three datasets, with relative gains over baselines reaching 11.28%.At K=20, NDCG gains exceed Recall gains throughout.
- Overall Performance: Several multimodal baselines underperform the strongest pure-CF model, indicating that additional modality information can harm recommendation quality.The result directly challenges the modality harmony assumption targeted by CGOP.
- Ablation Study: Replacing TAR-MoE with softmax is the most damaging ablation on every dataset, especially Clothing, confirming the value of decoupled routing.Magnitude-matched retuning attributes the effect to mutually exclusive routing rather than injection scale.
- Ablation Study: Removing SSL reduces R@20 by 14.1–19.7% across datasets, while standard InfoNCE can be as damaging as no alignment on Baby.These results support alignment that is explicitly conflict-aware.
- Item Popularity and Sparsity Analysis: OrthoRec’s largest popularity-group advantage is about +41% R@20 over LGMRec for Cold items, while Popular-item performance differs by less than 4%.The gain concentrates where collaborative evidence is scarce.
- Hyperparameter Sensitivity: Routing-temperature sensitivity remains below 4% across τ_r∈[0.01, 1.0], whereas Clothing requires α≥0.3 and α=0.5 lies within the stable region for all datasets.The global hypergraph enhancement is especially important for the sparsest dataset.
- Hyperparameter Sensitivity: Performance peaks within γ∈[1.25, 1.5], while λ_cl∈[10^-5, 10^-4] gives the best alignment and λ_cl≥5×10^-4 markedly degrades Clothing.Excessive modality amplification or contrastive pressure can overwhelm collaborative signals.
5 Conclusion
OrthoRec addresses modality-topology conflicts in multimodal recommendation by combining geometric noise filtration, decoupled routing, and safe-SSL. Within the evaluated settings, it improves accuracy over competitive baselines and robustness under modality noise and item sparsity.
- OrthoRec combines Collaborative-Guided Orthogonal Purification, decoupled TAR-MoE routing, and safe-SSL to address modality-topology conflicts.
- OrthoRec delivers consistent accuracy improvements over competitive baselines within the evaluated settings.
- OrthoRec exhibits improved robustness under modality noise and item sparsity within the evaluated settings.