PaperlayerView exact revision on arXiv

Source-linked AI summary

Cross-Domain Recommendation to Cold-Start Users via Variational Information Bottleneck

Jiangxia Cao, Jiawei Sheng, Xin Cong, Tingwen Liu, Bin Wang

arXiv:2203.16863v1cs.IRcs.SI

TL;DR

Cross-domain recommendation aims to address user cold-start by transferring preferences from a source domain, but independently pretrained representations can retain domain-specific information. CDRIB jointly models cross-domain and in-domain interactions with information-bottleneck regularizers; the paper reports a 34.38% average MRR increase over state-of-the-art approaches across four real-world datasets.

  • Problem

    Independently pretrained domain representations can encode domain-specific information that limits the effectiveness of transferring preferences for cold-start recommendation.

    Source: §I · p. 2

  • Method

    CDRIB uses variational information-bottleneck regularizers to model cross-domain and in-domain user-item interactions jointly, alongside a contrastive regularizer for overlapping users.

    Source: §I · p. 2

  • Results

    34.38% average MRR increase over state-of-the-art approaches on four real-world cross-domain datasets is reported for CDRIB.

    Source: §I · p. 2

  • Takeaways & Limitations

    CDRIB learns representations that emphasize domain-shared information and limit domain-specific information for cross-domain recommendations to cold-start users.

    Source: §I · p. 2

Abstract

from arXiv · show

Recommender systems have been widely deployed in many real-world applications, but usually suffer from the long-standing user cold-start problem. As a promising way, Cross-Domain Recommendation (CDR) has attracted a surge of interest, which aims to transfer the user preferences observed in the source domain to make recommendations in the target domain. Previous CDR approaches mostly achieve the goal by following the Embedding and Mapping (EMCDR) idea which attempts to learn a mapping function to transfer the pre-trained user representations (embeddings) from the source domain into the target domain. However, they pre-train the user/item representations independently for each domain, ignoring to consider both domain interactions simultaneously. Therefore, the biased pre-trained representations inevitably involve the domain-specific information which may lead to negative impact to transfer information across domains. In this work, we consider a key point of the CDR task: what information needs to be shared across domains? To achieve the above idea, this paper utilizes the information bottleneck (IB) principle, and proposes a novel approach termed as CDRIB to enforce the representations encoding the domain-shared information. To derive the unbiased representations, we devise two IB regularizers to model the cross-domain/in-domain user-item interactions simultaneously and thereby CDRIB could consider both domain interactions jointly for de-biasing.

I. INTRODUCTION

CDR addresses cold-start recommendations by transferring preference information across domains, but independently pretrained embeddings can retain domain-specific information that limits transfer. CDRIB jointly learns representations with information-bottleneck regularizers to emphasize domain-shared information, and the paper reports higher average MRR than state-of-the-art approaches on four real-world datasets.

I. INTRODUCTION

  • EMCDR methods pre-train representations independently, which can encode domain-specific information and limit the effectiveness of cross-domain transfer.Shared preferences can help transfer, while domain-specific cues may be unhelpful or cause negative transfer.Source: §I · pp. 1–2
  • CDRIB uses a variational bipartite graph encoder and two information-bottleneck regularizers for cross-domain and in-domain interactions, plus a contrastive regularizer to align overlapping users.The regularizers encourage domain-shared information while limiting domain-specific information in the representations.Source: §I · p. 2
  • 34.38% average MRR increase over state-of-the-art approaches on four real-world cross-domain datasets is reported for CDRIB.The paper also reports ablation studies and analyses of the method.Source: §I · p. 2

A. Problem Formulation

  • The task is to recommend target-domain items to users observed only in the source domain, using interaction data from both domains.Source: A. Problem Formulation · p. 3

B. Information Bottleneck Principle

The information bottleneck (IB) balances concise representations against predictive power. CDRIB uses a variational bipartite graph encoder and cross-domain regularizers to learn user/item representations that capture domain-shared information.

B. Information Bottleneck Principle

  • IB learns representations that compress input information while remaining predictive of the target, balancing these aims with a trade-off parameter.The paper describes the resulting representation as a minimal sufficient statistic for prediction.Source: B. Information Bottleneck Principle · p. 3

III. METHODOLOGY

  • CDRIB combines a variational bipartite graph encoder with cross-domain and in-domain user-item regularizers and a contrastive regularizer for overlapping users.The regularizers constrain representations to encode domain-shared information.Source: §III · p. 3

A. Embedding Layer

  • The embedding layer initializes separate user and item representation matrices for domains X and Y.Each matrix uses representation dimension F.Source: A. Embedding Layer · p. 3

B. Variational Bipartite Graph Encoder

  • VBGE first aggregates homogeneous even-hop neighbors into interim representations, then uses variational inference to generate latent user and item representations.The process is applied separately in each domain.Source: B. Variational Bipartite Graph Encoder · p. 4

C. Cross-Domain Information Regularizer

The cross-domain information regularizer models correlations in user-item interactions across domains, while the contrastive regularizer models correlations between overlapping users. The section distinguishes observed interaction, user, and item information.

C. Cross-Domain Information Regularizer

  • The two regularizers target cross-domain user-item correlations and cross-domain correlations between overlapping users, respectively.Both aim to learn representations containing domain-shared information.Source: C. Cross-Domain Information Regularizer · p. 4
  • Observed interaction information comprises all observed user-item behaviors; user information includes user features and homogeneous structure, with item information defined similarly.The section divides users into overlapping and non-overlapping groups.Source: C. Cross-Domain Information Regularizer · p. 4
  • The user representations are divided according to the overlapping and non-overlapping user groups in each domain.Source: C. Cross-Domain Information Regularizer · p. 4

1) Information Bottleneck Regularizer:

The information bottleneck regularizers model cross-domain and in-domain user-item interactions. They constrain representations to retain predictive interaction information while limiting redundant source-domain information.

1) Information Bottleneck Regularizer:

  • Together, the cross-domain and in-domain regularizers learn domain-shared rather than independently pre-trained domain-specific representations, supporting recommendations in both domains.Source: 1) Information Bottleneck Regularizer · p. 5
  • The cross-domain information bottleneck trains source-domain user representations to predict interactions with target-domain items.The regularizer is applied to overlapping users and is formulated bidirectionally across domains.Source: 1) Information Bottleneck Regularizer · p. 5
  • Cross-domain regularization balances compression of source user/item information against reconstructing interactions in the target domain.The formulation also applies to users from Y and items from X.Source: 1) Information Bottleneck Regularizer · p. 5
  • The in-domain regularizer compresses redundant user/item information while preserving the ability to reconstruct interactions in the same domain.It is applied to non-overlapping users and yields regularizers for both domains.Source: 1) Information Bottleneck Regularizer · p. 5

2) Contrastive Information Regularizer:

The contrastive information regularizer refines overlapping-user representations across domains by encouraging them to encode domain-shared information.

2) Contrastive Information Regularizer:

  • The contrastive information regularizer measures mutual information between representations of overlapping users in domains X and Y.Its objective encourages those representations to predict within-domain information while penalizing their joint information with the interaction data.Source: 2) Contrastive Information Regularizer · p. 5
  • The regularizer is intended to make user representations from each domain encode domain-shared information.Source: 2) Contrastive Information Regularizer · p. 5

D. Tractable Objective Function

Because direct mutual-information optimization is intractable, CDRIB derives variational objectives for its information bottleneck regularizers. These objectives use lower bounds for minimality and reconstruction terms.

D. Tractable Objective Function

1) Optimizing the Information Bottleneck Regularizer:

  • Minimality terms are optimized using KL divergence with variational posterior approximations and a normal Gaussian prior.For domain-X user representations, the proposed VBGE generates the variational posterior.Source: 1) Optimizing the Information… · p. 6
  • Reconstruction terms use variational approximation to reconstruct observed interactions from learned latent variables.The example defines a score function for interaction plausibility and maximizes a lower bound; interactions are represented as positive or negative user-item pairs.Source: 1) Optimizing the Information… · p. 6
  • The interaction score uses an inner product followed by a sigmoid function.The passage describes this choice as simple and effective.Source: 1) Optimizing the Information… · p. 6

2) Optimizing the Contrastive Information Regularizer:

The contrastive information regularizer measures similarity between overlapping users’ latent variables from two domains. A discriminator provides a lower bound for its contrastive term.

2) Optimizing the Contrastive Information Regularizer:

  • The regularizer compares latent variables for the same overlapping user across domains using a similarity score.The discriminator distinguishes domain-X and domain-Y representations; negative variables are sampled from domain-Y user representations.Source: 2) Optimizing the Contrastive… · p. 6
  • A three-layer MLP followed by a sigmoid function implements the discriminator.The discriminator’s similarity scores are used to form a lower bound for the contrastive term.Source: 2) Optimizing the Contrastive… · p. 6
  • Concatenated latent variables receive higher similarity scores when they are more similar.The passage identifies vector concatenation as the operation denoted by ⊕.Source: 2) Optimizing the Contrastive… · p. 6

3) Optimizing the Overall Model:

CDRIB combines tractable objectives for cross-domain and in-domain user-item interactions with a contrastive objective for cross-domain user-user correlations. The resulting model is optimized end to end.

3) Optimizing the Overall Model:

  • The overall model combines user-item interaction objectives across domains with a user-user correlation objective.The paper states that these tractable objectives support end-to-end optimization.Source: 3) Optimizing the Overall Model · p. 6
  • The β parameters penalize representations biased by their training information.The overall objective uses β = {β1, β2}.Source: 3) Optimizing the Overall Model · p. 6
  • Mini-batch optimization has time complexity O((|EX| + |EY|)F^2), described as mostly similar to other graph neural network methods.The cited comparison is to methods.Source: 3) Optimizing the Overall Model · p. 6

IV. EXPERIMENTS

Experiments use bidirectional cross-domain recommendation scenarios built from selected Amazon item domains. The evaluation reserves overlapping users for cold-start validation and testing.

A. Datasets

  • Experiments use four bidirectional domain pairs from Amazon’s 24 disjoint item domains.The pairs are Music-Movie, Phone-Elec, Cloth-Sport, and Game-Video.Source: A. Datasets · pp. 6–7
  • Users and items with fewer than 5 and 10 domain interactions, respectively, are filtered out before evaluation.The filtering follows prior-work preprocessing settings.Source: A. Datasets · p. 7
  • About 20% of overlapping users are randomly assigned to cold-start validation and test sets, with the remainder used for training.The passage gives a 10% split in each transfer direction as an example.Source: A. Datasets · p. 7, Table II · p. 7

B. Experiments setting

The experiments use leave-one-out evaluation across four bidirectional CDR scenarios, comparing CDRIB with single-domain and cross-domain baselines under shared tuning and validation procedures.

1) Evaluation Protocol:

  • Leave-one-out evaluation ranks one positive item against 999 randomly selected negatives using MRR, NDCG@{5,10}, and HR@{1,5,10}.The learned source-domain and target-domain representations are used for ranking.Source: 1) Evaluation Protocol · p. 7

2) Compared Methods:

  • The comparison includes merged-domain CF models, cross-domain models, and EMCDR-based cold-start methods with several pretraining variants.Cross-domain baselines are run in both directions for bidirectional CDR.Source: 2) Compared Methods · p. 7

3) Parameter Settings:

  • Experiments use five random seeds, selecting the best validation-MRR result after grid search, with common embedding dimension 128 and batch size 1024.CDRIB’s multiplier, dropout, L2 weight, learning rate, and graph-layer count are tuned by grid search.Source: 3) Parameter Settings · p. 9

C. Performance Comparisons

  • Table III–VI report mean and standard deviation for MRR, NDCG, and HR across four bidirectional CDR scenarios.The scenarios are Music-Movie, Phone-Elec, Cloth-Sport, and Game-Video.Source: Table III · p. 8, Table IV · p. 8, Table V · p. 8

1) Comparison with Single-Domain Models:

CDRIB improves over EMCDR-based baselines across all four CDR scenarios and metrics. Ablations and robustness analyses examine its regularizers, overlap availability, source-history size, and graph depth.

1) Comparison with Single-Domain Models:

2) Comparison with Cross-Domain Models:

  • CDRIB achieves large improvements over all EMCDR-based baselines across every CDR scenario and metric.The authors attribute this to representations that emphasize domain-shared information and compress domain-specific information.Source: 2) Comparison with Cross-Domain Models · p. 10

D. Ablation Study

  • Removing both in-domain IB and contrastive regularization still beats cross-domain baselines; adding in-domain IB and then contrastive regularization yields further gains.The authors associate in-domain IB with shared-information encoding for non-overlapping users, and contrastive regularization with alignment of overlapping users.Source: D. Ablation Study · p. 10

E. Discussions of the Overlapping User Proportion

  • Performance improves steadily as the training proportion of overlapping users rises, yet CDRIB remains robust against SA-VAE and beats other baselines even at 20%.The authors report that it captures domain-shared information even with few overlapping users.Source: E. Discussions of the Overlapping User… · p. 10

F. Discussions of the Cold-Start User Interaction Number

  • CDRIB outperforms SA-VAE in every tested source-interaction group, including the worst group for each scenario.Performance generally improves with more observed source interactions, but fluctuations occur in some cases.Source: F. Discussions of the Cold-Start User… · pp. 10–11

G. Hyperparameter Analysis

  • More VBGE layers help aggregate neighborhood information, but four layers underperform three in many cases, which the authors attribute to over-smoothing.The preferred Lagrangian multiplier varies with scenario interaction scale and tends to be smaller in scenarios with more interactions.Source: G. Hyperparameter Analysis · p. 11

V. RELATED WORK

The section surveys CDR approaches for overlapping and cold-start users, reviews prior information-bottleneck work, and concludes with CDRIB’s design and reported performance.

A. Cross-Domain Recommendation

1) Cross-Domain Recommendation to Overlapping Users:

2) Cross-Domain Recommendation to Cold-Start Users:

  • Cold-start CDR methods bridge domains through user/item correlations; EMCDR maps source-pretrained user representations, while later approaches add multi-hop, meta-path, meta-learning, or VAE techniques.CDRIB differs by using a variational graph encoder and information bottleneck to learn domain-shared representations.Source: 2) Cross-Domain Recommendation to… · pp. 11–12

B. Information Bottleneck

  • Prior information-bottleneck work spans neural networks, disentanglement, graph tasks, and recommendation, but those methods were not designed for CDR.Source: B. Information Bottleneck · p. 12

VI. CONCLUSION

  • CDRIB combines a variational bipartite graph encoder, two user-item IB regularizers, and a user-user contrastive regularizer; it outperforms state-of-the-art methods in four CDR scenarios.Source: §VI · p. 12
Loading 2203.16863v1…