PaperlayerView exact revision on arXiv

Source-linked AI summary

Popularity Bias Is Not Always Evil: Disentangling Benign and Harmful Bias for Recommendation

Zihao Zhao, Jiawei Chen, Sheng Zhou, Xiangnan He, Xuezhi Cao, Fuzheng Zhang, Wei Wu

arXiv:2109.07946v1cs.IR

TL;DR

Popularity bias in recommendation can reflect either item quality or conformity, but observed popularity alone does not identify which factor is at work. TIDE uses temporal information to model static quality, dynamic conformity, and user–item matching, and the authors report that it outperforms state-of-the-art methods on three benchmark datasets.

  • Problem

    Popularity bias can reflect both benign item quality and harmful conformity, but observed interaction patterns do not explicitly distinguish the two factors.

    Source: §1 · p. 1, §2.1 · p. 2

  • Method

    TIDE uses temporal information to model clicks through static item quality, dynamic conformity, and user–item matching, then intervenes on conformity at inference.

    Source: §1 · p. 2, §6 · p. 9

  • Results

    TIDE outperforms a range of state-of-the-art methods on three benchmark datasets.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The approach aims to retain quality-related popularity information while avoiding harmful conformity effects in recommendations.

    Source: §6 · p. 9

    The conformity model uses a simple sum-exponential structure; exploring more sophisticated models is left for future work.

    Source: §6 · p. 9

Abstract

from arXiv · show

Recommender system usually suffers from severe popularity bias -- the collected interaction data usually exhibits quite imbalanced or even long-tailed distribution over items. Such skewed distribution may result from the users' conformity to the group, which deviates from reflecting users' true preference. Existing efforts for tackling this issue mainly focus on completely eliminating popularity bias. However, we argue that not all popularity bias is evil. Popularity bias not only results from conformity but also item quality, which is usually ignored by existing methods. Some items exhibit higher popularity as they have intrinsic better property. Blindly removing the popularity bias would lose such important signal, and further deteriorate model performance. To sufficiently exploit such important information for recommendation, it is essential to disentangle the benign popularity bias caused by item quality from the harmful popularity bias caused by conformity. Although important, it is quite challenging as we lack an explicit signal to differentiate the two factors of popularity bias. In this paper, we propose to leverage temporal information as the two factors exhibit quite different patterns along the time: item quality revealing item inherent property is stable and static while conformity that depends on items' recent clicks is highly time-sensitive. Correspondingly, we further propose a novel Time-aware DisEntangled framework (TIDE), where a click is generated from three components namely the static item quality, the dynamic conformity effect, as well as the user-item matching score returned by any recommendation model. Lastly, we conduct interventional inference such that the recommendation can benefit from the benign popularity bias while circumvent the harmful one. Extensive experiments on three real-world datasets demonstrated the effectiveness of TIDE.

1 INTRODUCTION

Popularity bias can reflect both valuable item quality and harmful conformity, so removing it wholesale risks discarding useful recommendation signal. TIDE uses temporal patterns to distinguish these factors and applies inference-time intervention; experiments report superiority over state-of-the-art methods on three benchmark datasets.

1 INTRODUCTION

  • Popularity bias reflects both intrinsic item quality and conformity, making its wholesale removal a trade-off between losing quality signal and retaining conformity effects.The paper argues that higher-quality items can be genuinely more popular, while conformity can skew observed behavior away from users’ own preferences.Source: §1 · p. 1
  • Temporal patterns can help distinguish the factors: item quality is stable, whereas conformity tied to recent clicks is time-sensitive.The reported analyses observe that more popular items tend to receive higher average ratings, while many items’ ratings are negatively correlated with popularity at a given time.Source: §1 · pp. 1–2
  • The paper frames disentangling quality-driven and conformity-driven popularity as an underexplored problem without an explicit signal identifying each factor.Its stated contribution is to study this distinction in recommendation.Source: §1 · pp. 1–2
  • TIDE models clicks using static item quality, dynamic conformity, and a user–item matching score, then intervenes on conformity during inference.The intervention aims to retain the quality and interest-matching components while avoiding harmful conformity effects.Source: §1 · p. 2
  • Experiments on three benchmark datasets show TIDE outperforms a range of state-of-the-art methods.The introduction does not specify individual metrics or comparison values.Source: §1 · p. 2

2 PRELIMINARIES

This section defines popularity bias as an uneven interaction distribution with both quality- and conformity-related sources, then examines rating and temporal patterns in real-world recommendation data. The analyses show that popularity can carry quality information while time-varying popularity is associated with conformity-related preference deviation.

2.1 Problem Definition

  • Recommendation is framed as learning from historical user behavior to capture user preference and make high-quality recommendations.The historical data consists of timestamped user–item click triples collected before time T.Source: §2.1 · p. 2
  • Popularity bias is the uneven, often long-tailed interaction distribution, attributed to benign item quality and harmful conformity; the goal is to disentangle these factors.The stated aim is to preserve the quality-related signal while circumventing the conformity-related factor.Source: §2.1 · p. 2

2.2 Empirical Analyses of Popularity Bias

  • Analyses use ratings from Amazon, Ciao, Douban, and Movielens, grouping items by popularity and examining rating correlations, including against clicks in the preceding half-year.Items are divided into 30 popularity groups for average-rating comparisons; instant popularity counts clicks during the past half-year.Source: §2.2 · p. 2, Figure 1 · p. 3
  • Across datasets, more-popular items tend to have higher average ratings, but popularity–rating patterns also fluctuate and are weakly positive or negative in many datasets.Figure 1 compares popularity-group ratings on Douban-Movie and rating–popularity correlations across datasets.Source: §2.2 · p. 3, Figure 1 · p. 3
  • For a large proportion of items, ratings are negatively correlated with time-aware instant popularity, whose dynamics reflect the conformity effect.The analyses relate instant popularity to recent clicks and observe that increased conformity impact accompanies behavior deviating from users’ own preferences.Source: §2.2 · p. 3

3 TIME-AWARE DISENTANGLED FRAMEWORK

TIDE models clicks as a combination of item quality, time-varying conformity, and user–item matching. At inference, it intervenes on conformity to retain the quality signal while removing conformity’s effect.

3.1 Disentangled Learning

  • TIDE models clicks through static item quality, time-aware conformity estimated from past interactions, and a user–item matching score.Item quality is represented by a time-irrelevant item-specific variable; conformity depends on the interaction time and prior interactions with the item.Source: §3.1 · p. 3
  • The conformity module discounts past-click stimulation and uses an item-specific β_i to scale its effect; training uses BPR with uniformly sampled negatives.The item-specific parameter reflects that conformity can be more severe for some items than others.Source: §3.1 · pp. 4–5

3.2 Intervention-based Inference

  • At inference, TIDE fixes conformity to c* to cut its path to prediction while retaining the item-quality and matching paths; experiments set c* to 0.This intervention is intended to remove conformity’s improper effect while preserving the useful factors.Source: §3.2 · p. 5

3.3 Links to Recent Work

  • TIDE splits popularity’s path into item quality and conformity, unlike prior approaches that retain complete popularity or remove it wholesale.The paper describes PDA as using complete popularity and DICE and other methods as removing popularity bias.Source: §3.3 · p. 5

4 EXPERIMENTS

Experiments on three chronological datasets evaluate click and preference prediction, TIDE variants, and whether learned item-quality parameters reflect ratings. TIDE performs well across both tasks, while ablations and exploratory analyses examine disentanglement and the learned quality signal.

4.1 Experimental Setup

  • Chronological splits of Ciao, Amazon-CDs, and Douban-Movie support click prediction and preference prediction evaluations against five method types.Training uses the first nine of ten time-ordered parts; the last part is divided between validation and testing.Source: §4.1 · pp. 5–6

4.2 Performance Comparison (RQ1)

  • TIDE generally outperforms comparison methods on click prediction, with Amazon improvements of 23.43% in Precision, 16.45% in Recall, and 19.69% in NDCG.The reported overall result has a few exceptions.Source: §4.2 · p. 6, Table 2 · p. 7
  • On preference prediction, TIDE-int generally outperforms compared methods, while PDA trails PD and TIDE-full trails TIDE-int.The authors attribute these comparisons to the costs of injecting conformity without filtering it and of failing to disentangle popularity’s factors.Source: §4.2 · p. 6, Table 3 · p. 7

4.3 Ablation Study (RQ2)

  • TIDE-int consistently outperforms variants modeling only item quality or conformity, and also TIDE-e and TIDE-full, supporting both-factor modeling and interventional inference.TIDE-e uses matching scores; TIDE-full retains conformity’s effect at inference.Source: §4.3 · pp. 6–7, Table 4 · p. 8

4.4 Exploratory Analysis (RQ3)

  • On Douban-Movie, learned qi has a positive relationship with average ratings, and its curve is more stable than the popularity comparison in Figure 1(a).Figure 5 groups items into 30 qi-based groups and plots each group’s average rating.Source: §4.4 · p. 7, Figure 5 · p. 7
  • Kendall ranking correlation between qi and average rating exceeds that between popularity and average rating, though both correlations are relatively small.Popularity’s ranking correlation with average ratings is negative on Amazon-CDs and Ciao; the authors note that value correlation need not imply ranking correlation.Source: §4.4 · p. 7, Table 5 · p. 8
  • TIDE-int generally outperforms fixed-quality TIDE-fixq, especially on Douban-Movie and Amazon-CDs, but performs similarly on Ciao.The authors suggest conformity may overwhelm item quality on Ciao, where qi and average rating have weakly positive correlation.Source: §4.4 · pp. 7–8, Figure 6 · p. 7

5 RELATED WORK

Related work discusses popularity bias and other recommendation biases, along with methods for debiasing and disentangled representations. The paper distinguishes its approach by separating popularity’s beneficial item-quality signal from harmful conformity.

Popularity Bias in recommendation

  • Prior work links popularity bias to conformity and skewed recommendations, with potential costs to serendipity, fairness, user experience, and retention,,,,,,,, [24],,,,,,.Source: §5 · p. 8
  • Debiasing approaches include inverse propensity scoring,, ranking adjustment,, regularization,,,, and causal inference,,.Ranking adjustment may sacrifice accuracy, while regularization and causal approaches use different strategies to address bias.Source: §5 · p. 8
  • Unlike methods that eliminate popularity bias or directly inject predicted popularity, this paper seeks to retain item-quality signal while filtering conformity.Source: §5 · p. 8

Biases in recommendation

  • Recommendation research also studies selection [36],,,, exposure [24], [36],,, and position bias [36],, alongside unfairness,.Source: §5 · p. 8

Disentanglement in recommendation

  • Disentanglement work includes debiasing user preference from data bias, and representation learning, such as separating user intentions from item attributes, or disentangling user representations from interaction graphs.Source: §5 · pp. 8–9

6 CONCLUSION

The conclusion frames the work as disentangling beneficial item-quality popularity from harmful conformity using their different temporal patterns. It also identifies richer conformity modeling and applying the approach to other biases as future directions.

6 CONCLUSION

  • The paper studies disentangling benign popularity bias from item quality and harmful popularity bias from conformity.Source: §6 · p. 9
  • TIDE models clicks using static item quality, dynamic conformity, and user-item matching, then uses interventional inference to retain the benign component while circumventing the harmful one.The paper reports experiments on three real-world datasets demonstrating the effectiveness of its models and inference strategy.Source: §6 · p. 9
  • Future work could explore a more sophisticated conformity model than the simple sum-exponential structure and extend the benign-versus-harmful analysis to other biases.Source: §6 · p. 9

Related papers

Loading 2109.07946v1…