Source-linked AI summary
ESAM: Discriminative Domain Adaptation with Non-Displayed Items to Improve Long-Tail Performance
Zhihong Chen, Rong Xiao, Chenliang Li, Gangfeng Ye, Haochuan Sun, Hongbo Deng
TL;DR
Ranking models train on displayed items but retrieve from the full item space, where long-tail items have sparse feedback and different feature distributions. ESAM adapts from displayed to non-displayed items using attribute-correlation alignment, center-wise clustering, and self-training. Experiments report improved performance, especially in the long-tail space, and improved Taobao search performance online.
Problem
Ranking models learn from displayed-item feedback but retrieve across the full item space, where non-displayed long-tail items have sparse records and inconsistent feature distributions.
Source: §1 · p. 1Method
ESAM treats displayed and non-displayed items as source and target domains, aligning high-level attribute correlations and adding center-wise clustering and self-training.
Source: §1 · p. 2Results
ESAM improves retrieval performance over existing baselines, especially in the long-tail space, and online experiments show improved performance at Taobao search.
Source: §1 · p. 2, §6 · p. 10Takeaways & Limitations
ESAM is presented as a framework that can be integrated into existing ranking models without auxiliary information or domains.
Source: §1 · p. 2
Abstract
from arXiv · showhide
Most of ranking models are trained only with displayed items (most are hot items), but they are utilized to retrieve items in the entire space which consists of both displayed and non-displayed items (most are long-tail items). Due to the sample selection bias, the long-tail items lack sufficient records to learn good feature representations, i.e. data sparsity and cold start problems. The resultant distribution discrepancy between displayed and non-displayed items would cause poor long-tail performance. To this end, we propose an entire space adaptation model (ESAM) to address this problem from the perspective of domain adaptation (DA). ESAM regards displayed and non-displayed items as source and target domains respectively. Specifically, we design the attribute correlation alignment that considers the correlation between high-level attributes of the item to achieve distribution alignment. Furthermore, we introduce two effective regularization strategies, i.e. \textit{center-wise clustering} and \textit{self-training} to improve DA process. Without requiring any auxiliary information and auxiliary domains, ESAM transfers the knowledge from displayed items to non-displayed items for alleviating the distribution inconsistency. Experiments on two public datasets and a large-scale industrial dataset collected from Taobao demonstrate that ESAM achieves state-of-the-art performance, especially in the long-tail space. Besides, we deploy ESAM to the Taobao search engine, leading to significant improvement on online performance. The code is available at \url{https://github.com/A-bone1/ESAM.git}
1 INTRODUCTION
Ranking models train on displayed items but retrieve across the full item space, where non-displayed long-tail items have sparse feedback and different feature distributions. ESAM applies domain adaptation between displayed and non-displayed items, using attribute-correlation alignment and regularization to improve long-tail ranking.
1 INTRODUCTION
- 82% of displayed items are hot, while 85% of non-displayed items are long-tail in analyses of MovieLens and CIKM Cup 2016.The paper defines hot and long-tail items by display frequency.Source: §1 · p. 1
- Sparse feedback leaves long-tail items with weaker feature representations and a distribution shift from hot items, contributing to poor long-tail retrieval.Ranking models are trained on displayed-item feedback but retrieve from the entire item space.Source: §1 · pp. 1–2
- Existing approaches use auxiliary information or domains, while ESAM treats displayed and non-displayed items as source and target domains for unsupervised adaptation.The paper notes that auxiliary resources are not easily accessible and that architectural solutions may not generalize across applications.Source: §1 · p. 2
- Attribute correlation alignment uses pair-wise correlations between high-level item attributes as a distribution signal shared across displayed and non-displayed items.The paper names this approach attribute correlation congruence (A2C).Source: §1 · p. 2
- Center-wise clustering organizes items by feedback similarity, while self-training assigns high-confidence pseudo-labels to non-displayed items to address missing target labels.The paper reports offline improvements on two public datasets and a Taobao dataset, plus improved Taobao search performance in online A/B testing.Source: §1 · p. 2
2 PROPOSED METHOD
ESAM adds an unlabeled non-displayed-item stream to a ranking model and applies domain adaptation and two regularization strategies to improve item representations across the entire item space. Its method aligns high-level attribute correlations, clusters source items by feedback, and self-trains on reliable target pseudo-labels.
2.1 Preliminaries
- The BaseModel maps queries and items to features, scores their relevance, trains on available feedback, and retrieves the top K items.The source and target item domains share a query set; source items have feedback, while target items are non-displayed.Source: §2.1 · pp. 2–3, §2.2 · p. 3
2.3 Entire Space Adaptation Model (ESAM)
- ESAM adds an unlabeled non-displayed-item stream to the BaseModel and applies three constraints to improve item features across the entire item space.The BaseModel omits the unlabeled non-displayed-item stream.Source: §2 · p. 2, §2.3 · p. 3
- Attribute correlation congruence aligns displayed and non-displayed items by reducing differences between their high-level attribute-correlation distributions.The method treats correlations among high-level item attributes as the distribution to align.Source: §2.3 · p. 4
- Center-wise clustering pulls items with the same query feedback together and pushes items with different feedbacks apart in the source feature space.Its intra-class compactness and inter-class separability provide a more discriminative source feature space for adaptation.Source: §2.3 · p. 4
- Self-training uses high-confidence target pseudo-labels to add target-discriminative information and mitigate negative transfer during alignment.The method selects target items below a confidence threshold as negatives and above another threshold as positives; uncertain scores can receive false labels early in training.Source: §2.3 · pp. 4–5
2.4 Optimization
- ESAM’s total loss is formulated with λ1, λ2, and λ3 as hyperparameters that control the impact of their corresponding terms.Source: §2.4 · p. 5
3 APPLICATIONS
This section applies ESAM to item recommendation and personalized search, describing their feature inputs, scoring, and training setup.
3.1 Item Recommendation
- Recommendation combines user-profile, recent-behavior, and item-profile features with neural feature extractors and a point-wise cross-entropy objective for click prediction.Click labels are binary; the objective is defined over displayed items for each query.Source: §3.1 · p. 5
- Each training epoch assigns non-displayed items to queries and adds a nondisplay input stream with three item-side constraints.The model described above is trained using only the displayed-item loss before this stream is added.Source: §3.1 · p. 5
3.2 Personalized Search System
- Personalized search adds a keyword feature, represented by averaging its word vectors, and uses cosine similarity for scoring.Its ESAM integration, feature extractor, and loss function match the recommendation setup.Source: §3.2 · p. 6
4 EXPERIMENTS
Experiments across two public datasets and Taobao evaluate ESAM against ranking and adaptation baselines. ESAM improves long-tail performance, and visual analyses show changes in feature and score distributions.
4.1 Datasets; 4.2 Baselines and Evaluation Metrics
- Experiments cover MovieLens-1M, CIKM Cup 2016, and a week of Taobao logs, comparing neural-ranking, domain-adaptation, and unbiased-imputation baselines using NDCG, Recall, and MAP.The public datasets use random 8:1:1 splits; Taobao records are split chronologically across training, validation, and test.Source: §4.1 · p. 6, §4.2 · p. 6
4.4 Performance Comparison
- With ESAM, long-tail personalized search gains averaged 3.3% / 3.4% / 2.5%, and item recommendation gains averaged 2.7% / 2.0% / 2.2% in NDCG@20 / Recall@20 / MAP.These are average absolute gains over baselines; Table 3 covers MovieLens-1M and Table 4 covers CIKM Cup 2016.Source: §4.4 · p. 7, Table 3 · p. 7, Table 4 · p. 7
- In the cold-start test, ESAM significantly improves BST performance by including non-displayed items that contain the held-out cold-start items.The test removes training records containing displayed items selected from 20% of the test records.Source: §4.4 · p. 7, Table 5 · p. 7
- On Taobao, attribute-correlation alignment improved industrial long-tail performance by average gains of +2.73% or +3.05%; the three-constraint ESAM combination performed best and outperformed UIM.The industrial ablation uses Recall@1k and Recall@3k; the passage does not map each gain to a specific comparison.Source: §4.4 · pp. 7–8, Table 6 · p. 7
4.5 Feature Distribution Visualization
- The Taobao t-SNE visualizations show a source-target feature gap reduced by alignment, while ESAM features have greater within-class compactness and between-class separability.ESAM is described as extracting domain-invariant and discriminative features.Source: §4.5 · p. 8, Figure 6 · p. 8
4.6 Score Distribution
- Score visualizations show that alignment alone concentrates scores, clustering improves their discriminability, and self-training partially addresses a target-score spike associated with negative transfer.The authors associate the spike around score 0.15 with positive samples being aligned to negative samples when target labels are ignored.Source: §4.6 · p. 8, Figure 7 · p. 8
4.7 Similarity Matrix
- Compared with BaseModel, ESAM yields greater within-category and smaller between-category similarity, and its similarity matrix better reflects the stated relationship between dress, shoe, and phone.The paper reports dress–shoe similarity as higher than dress–phone similarity, which BaseModel does not display.Source: §4.7 · p. 8
(b) ESAM
ESAM produces a more discriminative category-similarity pattern, and its sensitivity analyses favor suitable distance constraints and reliable target pseudo-labels. In Taobao online testing, it improved four business metrics over BaseModel.
(b) ESAM
- ESAM has greater intra-class and smaller inter-class similarity than BaseModel, with the dress–shoe similarity exceeding dress–phone similarity in the reported matrix.The authors say BaseModel does not display this relationship.Source: (b) ESAM · p. 9, Figure 8 · p. 9
4.8 Parameter Sensitivity
- Bell-shaped performance curves indicate suitable intra-class and inter-class distance constraints help retrieval; low-confidence pseudo-labels can collapse the model, while stricter confidence thresholds improve performance.The sensitivity analysis varies m1 and m2 for clustering and p1 and p2 for self-training.Source: §4.8 · p. 9
4.9 Online Experiments
- 0.85% CTR, 0.63% collection rate, 0.24% CVR, and 0.4% GMV gains were reported for ESAM over BaseModel in a seven-day Taobao search A/B test.The passage attributes the gains to improved long-tail search capability from reducing displayed/non-displayed distribution discrepancy.Source: §4.9 · p. 9
5 RELATED WORK
Prior ranking work uses deep neural networks, while domain adaptation addresses source–target distribution differences and discriminative-feature methods seek more useful representations. ESAM combines domain adaptation with non-displayed items and transfers knowledge through correlations between high-level item attributes, without auxiliary information or domains.
5.1 Neural Network-Based Ranking Model
- ESAM combines domain adaptation with non-displayed items, using high-level item-attribute correlations to transfer knowledge without auxiliary information or domains.The paper presents this as a contrast with approaches relying on auxiliary data or domains.Source: §5.1 · p. 9
5.2 Discriminative Domain Adaption
- Domain adaptation methods learn domain-invariant features through moment matching, including MMD, CORAL, and CMD, or through adversarial training, including DANN and ADDA.Other methods seek more discriminative features through contrastive loss and center loss.Source: §5.2 · p. 9
- Inspired by discriminative-feature methods, the paper applies domain adaptation to learn more discriminative item features across the entire item space.The stated aim is better item-feature discriminative power across that space.Source: §5.2 · p. 10
6 CONCLUSION
ESAM applies discriminative domain adaptation with non-displayed items to improve long-tail ranking performance. Offline evaluations and Taobao online experiments report improved retrieval performance, and ablations examine the proposed constraints.
6 CONCLUSION
- Offline experiments on two public datasets and a Taobao industrial dataset show improved retrieval performance over existing state-of-the-art baselines, especially in the long-tail space.The conclusion also reports that online experiments demonstrate ESAM's superiority at the Taobao search engine.Source: §6 · p. 10
- ESAM improves long-tail ranking performance through discriminative domain adaptation that introduces non-displayed items.The paper describes ESAM as a general framework that can be integrated into existing ranking models.Source: §6 · p. 10
- Online experiments demonstrate ESAM's superiority at the Taobao search engine.The conclusion reports online results separately from the offline evaluations.Source: §6 · p. 10
- Ablation studies verify the necessity of each proposed constraint.The conclusion does not specify the individual ablation outcomes in this passage.Source: §6 · p. 10