Source-linked AI summary
CoNet: Collaborative Cross Networks for Cross-Domain Recommendation
Guangneng Hu, Yu Zhang, Qiang Yang
TL;DR
Cross-domain recommendation seeks to alleviate sparse interaction data by leveraging relevant domains, but shallow transfer and fixed sharing are limited. The paper proposes CoNet, neural networks with bidirectional cross connections and joint losses, and reports stronger benchmark performance, adaptive transfer, and substantial training-example reduction.
Problem
Cross-domain recommendation needs a neural transfer approach that learns complex user-item interactions while addressing how representations should be shared between domain networks.
Method
CoNet uses domain-specific multilayer feedforward networks connected by dual cross mappings and jointly optimized losses for bidirectional knowledge transfer.
Results
CoNet outperforms baselines by a reported 7.84% relative improvement in NDCG and can reduce tens of thousands of training examples without performance degradation.
Takeaways & Limitations
Adaptive selection of transferred representations is necessary because transferring all representations with equal importance may harm performance.
Takeaways & Limitations
The formulation assumes two domains with shared users and sparse implicit-feedback interaction matrices.
Abstract
from arXiv · showhide
The cross-domain recommendation technique is an effective way of alleviating the data sparse issue in recommender systems by leveraging the knowledge from relevant domains. Transfer learning is a class of algorithms underlying these techniques. In this paper, we propose a novel transfer learning approach for cross-domain recommendation by using neural networks as the base model. In contrast to the matrix factorization based cross-domain techniques, our method is deep transfer learning, which can learn complex user-item interaction relationships. We assume that hidden layers in two base networks are connected by cross mappings, leading to the collaborative cross networks (CoNet). CoNet enables dual knowledge transfer across domains by introducing cross connections from one base network to another and vice versa. CoNet is achieved in multi-layer feedforward networks by adding dual connections and joint loss functions, which can be trained efficiently by back-propagation. The proposed model is thoroughly evaluated on two large real-world datasets. It outperforms baselines by relative improvements of 7.84\% in NDCG. We demonstrate the necessity of adaptively selecting representations to transfer. Our model can reduce tens of thousands training examples comparing with non-transfer methods and still has the competitive performance with them.
1 INTRODUCTION
Cross-domain recommendation addresses sparse interactions by transferring knowledge across relevant domains, while CoNet applies deep neural transfer to learn complex user-item relationships. It uses bidirectional cross-network mappings and is evaluated for performance, adaptive transfer, and data reduction.
- Motivation: Cross-domain recommendation transfers knowledge from relevant domains to address sparse user-item interactions.The paper motivates jointly learning sparse interaction matrices from domains such as app installation and news reading.
- Contributions: CoNet uses neural networks as base models to combine deep interaction learning with cross-domain transfer.The approach connects hidden layers across domain-specific networks rather than relying on matrix factorization alone.
- Contributions: 7.84% relative NDCG improvement over baselines is reported for the proposed approach.
- Contributions: CoNet transfers knowledge in both directions through cross connections between the two base networks.The networks use dual mappings and joint loss functions and are trained by back-propagation.
- Contributions: Adaptive representation selection is necessary, and transfer can reduce tens of thousands of training examples without performance degradation.The paper also evaluates performance against single-domain neural and shallow cross-domain methods.
2 PRELIMINARY
The paper formulates cross-domain recommendation over shared users and sparse implicit-feedback matrices, using feedforward neural networks to model nonlinear user-item interactions. The base network maps one-hot user and item indices through embeddings and hidden layers to a positive-interaction score.
- 2.1 Notation: The setting contains source and target domains with shared users and separate item sets.App installations and news readings are represented as binary interaction matrices.
- 2.1 Notation: User-item interaction matrices are sparse because each user consumes only a small subset of available items.
- 2.1 Notation: Users receive ranked top-N items according to predicted interaction scores.
- 2.2 Base Network: Neural collaborative filtering parameterizes the interaction function with a neural network instead of a fixed matrix-factorization dot product.The neural model learns the interaction function from observed interactions.
- 2.2 Base Network: In transfer learning, domain-specific neural networks are jointly learned so that they can improve through mutual knowledge transfer.
- 2.2 Base Network: The base FFNN embeds one-hot user and item encodings, transforms them through multiple hidden layers, and predicts a score.The hidden layers learn nonlinear user-item interactions, while the output represents the probability of a positive interaction.
3 CROSS-STITCH NETWORKS
Cross-stitch networks combine activations from two tasks, but their scalar, dense sharing imposes assumptions that are poorly suited to typical recommendation networks. The limitations concern layer dimensions, uniform transfer weights, and transferring every representation.
- Cross-stitch mechanism: Cross-stitch networks linearly combine activation maps from two tasks before feeding them to successive layers.
- Cross-stitch mechanism: The scalar αD controls transferred information, while αS controls task-specific information.
- Limitations: Cross-stitch networks require contiguous layers to share the same vector space, which conflicts with tower-shaped FFNNs whose layer widths change.
- Limitations: A single scalar transfer weight treats representations from the other network as equally important, although their predictive usefulness may differ.
- Limitations: Dense transfer assumes every representation is useful, whereas sparse structure is important for general learning.
4 COLLABORATIVE CROSS NETWORKS
CoNet is a soft-sharing deep transfer approach that couples neural recommendation networks through bidirectional cross connections and joint training. Its matrix-based transfer units accommodate differing hidden-layer spaces and can selectively transfer representations through sparsity.
- 4.1 Cross Connections Unit: CoNet transfers knowledge between source and target recommendation networks by adding cross connection units to multilayer feedforward networks.The model is motivated by soft sharing and is designed to transfer knowledge in both directions.
- 4.1 Cross Connections Unit: Unlike cross-stitch networks, CoNet uses a matrix rather than a scalar for cross-domain transfer.The matrix H controls information transfer between the app and news networks.
- 4.1 Cross Connections Unit: Bidirectional cross connections let each domain receive information from the other, allowing a sparse target domain to learn residual representations relative to the source.The authors state that this can help the target network when its data is sparse.
- 4.1 Cross Connections Unit: Matrix H can match hidden layers with different dimensions, such as mapping a 128-dimensional layer to a 64-dimensional successive layer.This addresses a limitation of cross-stitch units, which assume contiguous layers share a vector space.
- 4.3 Adaptive Model: The adaptive SCoNet variant applies lasso regularization to task relationship matrices so transferred representations are sparse and selectively chosen.The regularization uses an ℓ1-norm penalty, with λ controlling sparsity.
- 4.4 Model Learning: CoNet jointly minimizes app and news recommendation losses, sharing user embeddings and transfer matrices while retaining domain-specific parameters.The joint objective can be optimized with stochastic gradient methods and trained efficiently by back-propagation.
5 EXPERIMENT
Experiments on Mobile and Amazon evaluate recommendation quality, transfer benefits, representation selection, data reduction, optimization, and training cost. CoNet models outperform baselines, while sparse transfer improves selection and transfer remains useful under reduced training data.
- 5.1 Experimental Setup: Experiments use two real-world cross-domain datasets—Mobile and Amazon—with top-10 HR, NDCG, and MRR evaluation.Mobile contains app-installation and news-reading interactions; Amazon uses Books and Movies & TV, with data sparsity over 99.9%.
- 5.2 Comparing Different Approaches: CoNet models outperform MLP, shallow cross-domain models, and deep cross-domain baselines on both datasets.The comparison covers MLP, CMF, CDCF, MLP++, and CSN under three ranking metrics.
- 5.2 Comparing Different Approaches: 7.84% relative NDCG improvement is achieved on Amazon versus the best baseline, MLP++; Mobile shows 4.28% MRR improvement versus non-transfer MLP.The results attribute these gains to knowledge transfer and report more than 20% MRR improvement over CDCF and CMF on Mobile.
- 5.3 Impact of Sparsity: Selecting Representations to Transfer: 2.29% relative NDCG improvement on Mobile and 4.21% on Amazon result from introducing sparsity in transfer matrices.The findings support selecting useful representations rather than transferring all representations equally.
- 5.4 Benefit of Transferring: Reducing Labelled Data: About 30,000 Mobile training examples and 20,000 Amazon examples can be saved while retaining comparable performance with the non-transfer MLP.The Amazon result is notable in a dataset with only about 16 examples per user and 0.017% density.
- 5.6 Optimization Performance: Training loss decreases and recommendation performance improves with optimization iterations, while proposed-model training costs about 100 seconds per epoch.The strongest updates occur during the first 15 iterations; MLP and CSN require about 70 and 90 seconds per epoch, respectively.
6 RELATED WORKS
Cross-domain recommendation extends collaborative filtering and transfer learning to address sparse data while modeling complex user-item interactions. Related approaches range from shallow matrix-factorization methods to neural networks that share information across domains or tasks.
- Recommender systems: Collaborative filtering predicts user preferences from past user-item interactions, while content-based methods match user profiles with item descriptions.
- Cross-domain recommendation: Cross-domain recommendation alleviates sparse data by transferring knowledge across relevant domains, including through collective matrix factorization, codebook transfer, and entity correspondence.
- Cross-domain recommendation: Existing cross-domain methods are generally shallow and have difficulty learning highly nonlinear user-item interaction relationships.
- Transfer and multitask learning: Deep transfer approaches let two base networks benefit from each other during training, extending information sharing beyond one-directional transfer.
7 CONCLUSIONS
The paper proposes deep transfer learning for cross-domain recommendation to reconstruct sparse target interactions using knowledge from an auxiliary domain. It finds that adaptive representation selection matters, while transfer can preserve performance with substantially fewer training examples.
- CoNet uses deep transfer learning to reconstruct sparse target user-item interactions with guidance from a source domain.
- Adaptive selection of auxiliary-domain representations is necessary because transferring all representations with equal importance may harm performance.
- Naive deep transfer models can be inferior to shallow or neural non-transfer methods in some cases.
- The transfer model can reduce tens of thousands of training examples relative to non-transfer methods without performance degradation.
- Experiments on two large real-world datasets compare shallow and deep methods across single-domain and cross-domain settings.