PaperlayerView exact revision on arXiv

Source-linked AI summary

A Model of Two Tales: Dual Transfer Learning Framework for Improved Long-tail Item Recommendation

Yin Zhang, Derek Zhiyuan Cheng, Tiansheng Yao, Xinyang Yi, Lichan Hong, Ed H. Chi

arXiv:2010.15982v2cs.IR

TL;DR

Long-tail item distributions concentrate feedback on a small set of popular items, making tail-item recommendation a central challenge. MIRec transfers knowledge from head to tail items through model-level meta-mapping and item-level curriculum learning. On two benchmark datasets, it outperforms state-of-the-art algorithms and reports improvements across tail, head, and overall recommendation performance.

  • Problem

    Long-tail interaction distributions concentrate feedback on a small fraction of popular items, and diverse head and tail items make it challenging to improve tail recommendations without compromising overall performance.

    Source: §1 · pp. 1–2

  • Method

    MIRec combines model-level meta-learning, which maps few-shot model parameters toward many-shot parameters, with curriculum learning that transfers knowledge among items.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    On two benchmark datasets, MIRec consistently outperforms state-of-the-art algorithms and improves tail-item recommendations alongside head-item and overall performance on HR@K and NDCG@K.

    Source: §1 · p. 2, §6 · p. 11

  • Takeaways & Limitations

    MIRec's learned tail-item representations preserve better semantics, with similar items close together in embedding space.

    Source: §1 · p. 2

    The evaluation targets one model serving both head and tail recommendations; separate slice-specific models would most likely achieve better results on their respective slices.

    Source: §5.2 · p. 7

Abstract

from arXiv · show

Highly skewed long-tail item distribution is very common in recommendation systems. It significantly hurts model performance on tail items. To improve tail-item recommendation, we conduct research to transfer knowledge from head items to tail items, leveraging the rich user feedback in head items and the semantic connections between head and tail items. Specifically, we propose a novel dual transfer learning framework that jointly learns the knowledge transfer from both model-level and item-level: 1. The model-level knowledge transfer builds a generic meta-mapping of model parameters from few-shot to many-shot model. It captures the implicit data augmentation on the model-level to improve the representation learning of tail items. 2. The item-level transfer connects head and tail items through item-level features, to ensure a smooth transfer of meta-mapping from head items to tail items. The two types of transfers are incorporated to ensure the learned knowledge from head items can be well applied for tail item representation learning in the long-tail distribution settings. Through extensive experiments on two benchmark datasets, results show that our proposed dual transfer learning framework significantly outperforms other state-of-the-art methods for tail item recommendation in hit ratio and NDCG. It is also very encouraging that our framework further improves head items and overall performance on top of the gains on tail items.

1 INTRODUCTION

MIRec addresses the long-tail recommendation problem by transferring knowledge from head items to tail items through model-level and item-level learning. Experiments on two benchmark datasets report gains on tail items alongside improvements in head-item and overall performance.

1 INTRODUCTION

  • Long-tail interactions concentrate feedback on a small fraction of popular items, while existing transfer approaches face a head–tail distribution gap and diverse item preferences and features.The paper identifies the challenge of improving tail slices without compromising overall performance.Source: §1 · pp. 1–2
  • Model-level transfer learns how model parameters change as user feedback grows, using the resulting meta-mapping to enhance tail-item representations.The meta-mapping can capture implicit data augmentation without changing the data distribution.Source: §1 · p. 2
  • Item-level curriculum transfer connects head and tail items so that a meta-mapping learned from head items can be applied to tail items.The framework accounts for divergence between head and tail items when learning these connections.Source: §1 · p. 2
  • MIRec jointly transfers knowledge at the model and item levels to apply head-item knowledge to tail items in long-tail settings.The framework pairs model-level transfer with a curriculum that smoothly transfers the learned meta-mapping from head items to tail items.Source: §1 · p. 2
  • On two benchmark datasets, MIRec improves tail-item recommendations and also improves head-item and overall performance on HR@K and NDCG@K.The introduction reports consistent, significant tail-item gains and solid improvements for head items and overall performance.Source: §1 · p. 2

2 RELATED WORK

Prior work addresses long-tail recommendation through data balancing, loss adjustments, clustering, and side information, while cold-start approaches often emphasize tail items alone. This paper instead transfers knowledge from head items and combines model-level meta-learning with curriculum learning across items.

2 RELATED WORK

  • Resampling can overfit minor classes through redundant examples or lose majority-class information through undersampling, while loss refinement adjusts weights or regularization.The cited approaches include frequency-aware logQ corrections.Source: §2 · p. 2
  • Long-tail approaches include resampling, item or interaction reweighting, clustering, and cold-start methods using user or item side information.The paper distinguishes its focus on transferring from head items, which contain rich feedback, rather than relying purely on cold-start items.Source: §2 · pp. 2–3
  • Most cited meta-learning recommenders use optimization to learn generalized models for cold-start recommendation, with less attention to long-tail distribution patterns.MIRec instead maps few-shot model parameters toward many-shot parameters and enhances transfer to tail items through learned item connections.Source: §2 · p. 3
  • Curriculum learning organizes examples during training, but the paper notes few studies apply it to recommendation for transferring knowledge between head and tail items.The cited curriculum approaches include teacher-student mechanics and dynamic curriculum learning.Source: §2 · p. 3

3 DUAL TRANSFER LEARNING FRAMEWORK: MIREC

MIRec transfers knowledge from head to tail items through a model-level parameter mapping and an item-level curriculum that connects their features. Together, these mechanisms aim to make the learned mapping applicable to sparse tail items.

3.1 Transfer Learning Across Models: Meta-level Knowledge Transfer

  • The meta-learner maps few-shot model parameters to many-shot parameters, capturing how added feedback changes a model and implicitly augmenting sparse-item representations.The paper illustrates this as implicitly adding similar users’ feedback through parameter changes rather than explicitly adding data points.Source: §3 · p. 3, §3.1 · p. 4
  • Few-shot and many-shot models use datasets built from user-item-reward triplets, with downsampled feedback for Ω(k) and richer feedback for Ω*.The paper describes Ω(k) as using fewer samples per item and Ω* as using all user feedback.Source: §3.1 · p. 4
  • The training objective jointly fits the few-shot model on Ω(k) and minimizes the gap between predicted and learned many-shot parameters on Ω*.The parameter-matching term supports the meta-mapping, while the base-learner loss supports recommendation performance on the few-shot data.Source: §3.1 · p. 4
  • To avoid costly, unstable mappings across large networks, MIRec maps parameters from the final user and item tower layers using a fully connected network.The authors describe this choice as efficient and generalizable across items.Source: §3.1 · p. 5

3.2 Transfer Learning Across Items: Curriculum Transfer

  • LTCur constructs training sets containing head and tail items under different distributions, transferring learned features while relatively preserving the original distribution to alleviate distribution bias.The curriculum is intended to smooth transfer from head items to tail items.Source: §3.2 · pp. 5–6

3.3 Prediction by MIRec

  • MIRec combines preference predictions from the many-shot base model and the meta-mapped model, with λp balancing their contributions.The paper motivates the combination by noting that rich feedback can give head-item representations from the base learner an advantage over meta-mapped representations.Source: §3.3 · p. 6

4 DISCUSSION

The discussion describes extensions of MIRec’s transfer process and its applicability beyond item-side long-tail distributions.

4 DISCUSSION

  • Varying k across an increasing sequence creates multiple training subsets and recursively learned mappings between successive model stages.The final preference prediction can combine these mappings with the many-shot model.Source: §4 · p. 6
  • For user-side long-tail distributions, MIRec can be adapted by reversing users and items; user- and item-based versions can also be combined for distributions on both sides.The paper presents combining the two versions as one simple approach.Source: §4 · p. 6

5 EXPERIMENT

The experiment evaluates MIRec against state-of-the-art methods and tests its components, curriculum strategy, and learned tail-item representations.

5 EXPERIMENT

  • The section evaluates MIRec for recommendation tasks on two datasets.Table 2 summarizes the datasets and their content information.Source: §5 · p. 6, Table 2 · p. 7
  • The experiments ask whether MIRec outperforms state-of-the-art methods, whether meta-learning and curriculum learning complement each other, and how its curriculum compares with alternatives.They also ask whether MIRec learns better tail-item representations that can be distinguished visually.Source: §5 · pp. 6–7

5.1 Datasets

The evaluation uses MovieLens 1M and Bookcrossing, datasets with rich user and item information and highly skewed long-tail item distributions.

5.1 Datasets

  • MovieLens 1M and Bookcrossing are benchmark datasets with rich user and item information.Explicit ratings are positive examples, other user-item pairs are negative examples, and pairs with invalid or missing features are filtered.Source: §5.1 · p. 7
  • Both datasets have highly skewed long-tail item distributions, and a small fraction of popular Bookcrossing items contributes more than half of user feedback.The authors consider the datasets suited to evaluating tail recommendation under long-tail distributions.Source: §5.1 · p. 7
  • For MovieLens 1M, each user's most recent item is held out for testing and the second most recent for validation; remaining items are used for training.Bookcrossing lacks timestamps, so one item per user is randomly selected for testing and one for validation.Source: §5.1 · p. 7

5.2 Setup

The setup evaluates top-K recommendation on overall, head, and tail slices, comparing MIRec with a shared two-tower backbone and several long-tail strategies.

5.2 Setup

  • HR@K measures whether a test item appears among the top K recommendations, while NDCG@K rewards correctly recommended items ranked higher.Both metrics are used to evaluate recommendation performance.Source: §5.2 · p. 7
  • The evaluation reports head, tail, and overall performance for a single model intended to improve tail recommendations without reducing overall performance.The head/tail split uses the most frequent 20% of MovieLens1M items and 0.1% of Bookcrossing items as head items.Source: §5.2 · p. 7
  • The stated best outcome improves head, tail, and overall performance; the comparison uses a shared content-aware two-tower backbone with matching layers and dimensions.The authors also describe a case that improves tail performance while keeping head performance neutral and overall performance improved.Source: §5.2 · p. 7
  • Baselines include over-sampling and under-sampling, which respectively repeat tail examples or down-sample head examples.The setup also includes ClassBalance, which weights item classes using their effective sample numbers, and LogQ, which corrects predicted logits using estimated item frequency.Source: §5.2 · p. 8
  • Curriculum baselines train first on head items then tail items, or in the reverse order; MeLu adapts optimization-based meta-learning to implicit-feedback prediction.MeLu uses the two-tower backbone for a fair comparison.Source: §5.2 · p. 8
  • Embedding dimensions are 64 across methods, and two-tower approaches use the same layer counts and hidden units.Curriculum methods use 100 epochs per stage, non-curriculum methods use 200 epochs, and MIRec is optimized with Adam.Source: §5.2 · p. 8

5.3 Recommendation Performance (RQ1)

MIRec generally improves top-K recommendation across tail, head, and overall slices, and outperforms representative alternatives across embedding dimensions. Higher embedding dimensions can coincide with weaker tail-item performance.

5.3.1 TopK Performance.

  • MIRec generally improves both head and tail items over the two-tower model and performs best among the compared long-tail strategies.Resampling performs poorly; LogQ can improve tail performance while remaining neutral or performing poorly on head items, and some tail-strong baselines lose head and overall performance.Source: §5.3.1 · pp. 8–9
  • MIRec generally improves HR@K and NDCG@K for tail, head, and overall items across MovieLens1M and Bookcrossing.The reported comparisons are in Tables 4 and 5, with the same embedding dimensions used across methods for fairness.Source: §5.3.1 · p. 8, Table 4 · p. 9, Table 5 · p. 9

5.3.2 Influence of Embedding Dimension.

  • MIRec consistently outperforms representative methods across embedding dimensions, while tail-item performance tends to degrade as embedding dimension increases.The authors hypothesize that tail items are more prone to overfitting because they have less training data.Source: §5.3.2 · p. 9

5.4 Dual Transfer Learning (RQ2)

The ablation study reports that MIRec outperforms its tested variants. Its findings highlight the curriculum transfer, logQ correction, and combined transfer design as important to performance.

5.4 Dual Transfer Learning (RQ2)

  • MIRec outperforms the ablation variants in the MovieLens1M study.The reported comparison is in Table 6; the supplied text does not give cell values.Source: §5.4 · p. 10
  • MIRec-M, which uses vanilla meta-learning, performs poorly relative to MIRec-C, supporting curriculum transfer to adapt the meta-mapping for tail items.MIRec-M uses head items for its meta-learning component, while MIRec-C uses curriculum learning alone.Source: §5.4 · pp. 9–10
  • MIRec-C performs well on tail items but not head items; the authors hypothesize that second-phase head-item downsampling may cause forgetting.The forgetting explanation is presented as a hypothesis.Source: §5.4 · p. 10
  • MIRec performs better than MIRec-LogQ, which omits logQ correction, indicating that item-frequency-based correction can further help knowledge transfer.MIRec-LogQ substitutes the uncorrected preference score for the logQ-corrected score.Source: §5.4 · pp. 9–10
  • Two-tower2 performs poorly on tail items despite having a similar number of learned parameters; the authors suggest overfitting as one possible reason.Two-tower2 combines predictions from two two-tower models.Source: §5.4 · pp. 9–10

5.5 Curriculum Learning Process (RQ3)

The curriculum experiments compare loss trajectories across strategies and item groups. LTCur shows a two-stage decrease in tail-item training and validation loss, while single-slice training can worsen validation performance on the other slice.

5.5 Curriculum Learning Process (RQ3)

  • LTCur produces a two-stage decrease in both training and validation loss for tail items.Figure 5 plots epochs against loss, with curriculum strategies in rows, item groups in columns, and training and validation shown separately.Source: §5.5 · p. 10
  • Training on only head or tail items decreases validation performance on the other group, reflecting variation between head and tail items.The authors say this supports transferring knowledge between the groups.Source: §5.5 · p. 10
  • With training limited to head or tail items, validation loss can rise even as training loss falls, supporting use of both groups across curriculum stages.The reported pattern appears in the first two curriculum rows of Figure 5.Source: §5.5 · p. 10

5.6 Embedding Visualization and Case Study (RQ4)

The visualizations qualitatively examine MIRec's tail-item embeddings and item clusters. They report more genre-coherent clusters than the baseline and show similar movies grouped together.

5.6 Embedding Visualization and Case Study (RQ4)

  • MIRec's tail-item clusters are more coherent with movie genres than the baseline's, which the authors attribute to better capture of semantic information.Figure 6 uses t-SNE for the embeddings and colors items by genre.Source: §5.6 · p. 10, Figure 6 · p. 11
  • The case study shows similar movies clustered together, including titles with similar themes highlighted in Figure 7.The authors interpret the clusters as evidence that MIRec can group tail items using content information despite their having less collaborative information.Source: §5.6 · p. 10, Figure 7 · p. 11
  • Figure 6 visualizes tail-item embeddings in two dimensions, with movie genre encoded by color.The visualization uses t-SNE.Source: Figure 6 · p. 11

6 CONCLUSION

The conclusion presents MIRec as a dual-transfer framework for long-tail recommendation, combining model-level meta-learning with item-level curriculum learning. Experiments on two benchmark datasets show consistent outperformance of state-of-the-art algorithms.

6 CONCLUSION

  • Across two benchmark datasets, MIRec consistently outperforms state-of-the-art algorithms.The conclusion reports this result for the proposed framework.Source: §6 · p. 11
  • MIRec integrates meta-learning to transfer knowledge among models with curriculum learning to transfer knowledge among items.The curriculum within meta-learning is intended to ensure smooth knowledge transfer from head items to the larger number of tail items.Source: §6 · p. 11
  • Future work will improve the curriculum within meta-learning and explore other content information, including item knowledge graphs.Source: §6 · p. 11
Loading 2010.15982v2…