Cross Pairwise Ranking for Unbiased Item Recommendation

1 INTRODUCTION

• Standard losses fit observed interactions despite their popularity bias, allowing popular items to gain exposure while niche-item recommendation quality suffers.The interaction data has a long-tail popularity distribution, and these losses leave its bias effect untouched.Source: §1 · p. 1
• IPS reweights samples by inverse propensity, but exposure mechanisms are seldom known and reweighted losses can have high variance, especially for implicit feedback.These practical limits motivate debiasing approaches that do not require accurately estimated propensity scores.Source: §1 · pp. 1–2
• CPR combines purposefully chosen user-item pairs to offset user and item propensities under a factorization assumption, without quantifying the exposure mechanism.The paper also describes CPR as ensuring unbiased learning for individual samples.Source: §1 · p. 2
• CPR adds dynamic sampling inspired by hard negative sampling to train selected samples more frequently for faster convergence.The strategy aims to assign hard samples higher sampling probability.Source: §1 · p. 2
• The paper analyzes pointwise and pairwise losses as biased for ranking user preference, and theoretically analyzes CPR’s sample-level unbiasedness.The stated contribution also includes empirical studies on two benchmark datasets and multiple backbone models.Source: §1 · p. 2

2.1 Notations

• The setup treats observed interaction data as labels for supervised recommendation, distinguishing observed pairs from missing-data pairs.Interactions occur when a user both likes and observes an item; relevance and exposure are not assumed independent.Source: §2.1 · p. 2

2.2 Definition of Unbiasedness

• Unbiasedness is defined by whether a loss optimizes predicted relevance rankings toward rankings of true relevance.This differs from the expectation-based definition associated with IPS.Source: §2.2 · p. 2

2.3 Biasedness of Pointwise and Pairwise Loss

• Pointwise and pairwise losses are introduced as standard approaches: the former fits single-item predictions, while the latter ranks positive items above negative ones.The section analyzes both from the perspective of recovering users’ true preference rankings.Source: §2.3 · p. 2

2.3.1 Biasedness of Pointwise Loss.

• Pointwise loss is biased because its predicted relevance score models true relevance together with exposure probability, which tends to favor popular items.The analysis frames the pointwise objective as binary classification between positive and negative pairs.Source: §2.3.1 · p. 3

2.3.2 Biasedness of Pairwise Loss.

• Pairwise loss is biased because its ranking also incorporates exposure probability, which tends to be higher for popular items.The analysis derives a pairwise ranking condition from the expected ranking and concludes that the resulting scores favor popular items.Source: §2.3.2 · p. 3

3.1 CPR Loss

• CPR samples two observed positive user-item pairs from different users, requiring their cross-combinations to be negative pairs.The resulting D2 tuples form the training data.Source: §3.1 · p. 3

3.2 Unbiasedness of CPR Loss

• Pointwise ranking is biased by user and item propensities, while pairwise ranking is biased by item propensity.These biases are assessed against the correct ranking under the exposure assumption.Source: §3.2 · p. 3
• Under the factorized exposure assumption, CPR loss is unbiased because user and item propensities cancel between positive and negative pairs.The assumption factorizes exposure probability into user propensity, item propensity, and user-item relevance.Source: §3.2 · pp. 3–4

3.3 Extending to More Interactions

• CPR extends from two interactions to k ≥ 2, with samples requiring observed positive pairs and cyclic cross-combinations that are negative.The extended loss scales samples with different interaction counts by 1/k; its unbiasedness proof is omitted as similar to the earlier proof.Source: §3.3 · p. 4
• CPR performs best when k = 2 or 3; larger k does not improve performance and may make training inflexible.The experiments therefore draw samples only from D2 and D3.Source: §3.3 · p. 4

3.4 Dynamic Sampling

• Dynamic sampling selects hard CPR samples more often by ranking valid samples by their score margin and choosing those with the smallest values.It first draws an enlarged candidate set, discards samples without the required negative cross-combinations, then selects a batch from the remaining samples.Source: §3.4 · p. 4

4.1 Comparison with IPS

• IPS defines unbiasedness by matching expected loss to ideal loss, then reweights labels using propensity scores to pursue that criterion.The discussion identifies difficulty estimating propensities and high variance as practical problems with this approach.Source: §4.1 · pp. 4–5
• CPR changes loss composition rather than reweighting samples, achieving sample-level unbiasedness without explicit propensity scores under a mild exposure assumption.IPS can require inaccurate propensity estimates and have high loss variance; variance reduction by clipping can remove unbiasedness.Source: §4.1 · p. 5
• This unbiasedness comes with a relaxed goal: the sum score of k positive pairs should exceed that of k negative pairs, which may not ensure one-to-one comparisons.The authors note that CPR’s empirical performance justifies this relaxation.Source: §4.1 · p. 5

4.2 Comparison with Setwise Ranking

• Setwise Ranking aims to improve BPR by ranking item sets, not to debias recommendations; CPR instead constructs cross-pair samples that provide debiasing ability.Setwise Ranking uses user-level positive and negative item sets, whereas CPR combines multiple user-item pairs.Source: §4.2 · p. 5

5.1.1 Datasets.

• The study uses MovieLens-10M, Netflix Prize, and Alibaba iFashion, with a 3-core setting to retain more unpopular items.MovieLens and Netflix ratings are binarized; iFashion clicks are treated as interactions.Source: §5.1.1 · p. 5

5.1.2 Evaluation Protocol.

• Validation and test records are sampled with equal item probability, and performance is measured with Recall@20, NDCG@20, and ARP@20.The training/validation/testing split is 70/10/20; higher Recall and NDCG and lower ARP indicate better performance and debiasing under this protocol.Source: §5.1.2 · p. 5

5.1.3 Baselines.

• The comparison includes BPR and Mult-VAE as traditional approaches and CausE, Rel-MF, UBPR, and DICE as debiasing approaches.MF is the common backbone except for Mult-VAE, with additional backbone comparisons in Section 5.4.1.Source: §5.1.3 · pp. 5–6

5.1.4 hyperparameter Settings.

• For CPR, training samples combine D2 and D3, with their sampling ratio and dynamic sampling rate tuned on validation data.The dynamic sampling choosing rate is fixed to 2.Source: §5.1.4 · p. 6

5.1.5 Significance Test.

• CPR and the best baseline on each dataset are each run five times, and their results are compared with a two-tailed pairwise t-test.Source: §5.1.5 · p. 6

5.2.1 Effectiveness Comparison.

• CPR significantly outperforms all baselines on Recall and NDCG across the datasets, with p-values below 0.05.On iFashion, CPR-rand has Recall similar to the best baseline but lower ARP; CPR further improves CPR-rand on all datasets.Source: Figure 2 · p. 6, §5.2.1 · p. 6

5.2.2 Efficiency Comparison.

• 2.6% and 1.3% of DICE’s training time: CPR’s training time on MovieLens and Netflix, respectively.CPR converges to its best Recall performance in the fewest epochs; Figure 3 compares total training time on a log-scaled y-axis.Source: Figure 3 · p. 6, §5.2.2 · p. 6

5.2.1 Effectiveness Comparison.

• Higher Recall and NDCG on the unbiased test set indicate stronger prediction accuracy, especially on unpopular items; lower ARP indicates less popularity bias.Source: §5.2.1 · p. 6

5.3 Distribution of Recommendation

• Figure 4 compares the percentage of recommended items across item-degree groups for each approach.The groups are formed by sorting items by degree and making each group’s total degree approximately equal.Source: Figure 4 · p. 7, §5.3 · p. 7
• The test-set share of recommended items decreases as group ID rises, reflecting the greater share of low-degree items in that set.The training-set curve is flat because of the grouping method.Source: §5.3 · p. 7
• CPR and DICE make recommendation distributions closer to the test set than BPR and Mult-VAE, which amplify training-set bias toward popular items.The test set contains a larger proportion of low-degree items than the training set.Source: §5.3 · p. 7

5.4.1 Different Backbones.

• On MovieLens, CPR exceeds DICE by 7.9%, 26.4%, and 10.4% in NDCG@20 on LightGCN, NeuMF, and NGCF, respectively.The evaluation also covers Netflix; CPR consistently outperforms BPR and DICE across the tested backbones.Source: §5.4.1 · p. 7, Table 3 · p. 7

5.4.2 Different Degrees of Bias.

• With both amplified and reduced bias in sampled training data, CPR outperforms DICE by around 10% on MovieLens and 14% on Netflix in NDCG@20.The samples use 70% of the training set, with item-degree sampling exponents θ=0.5 and θ=−0.5, respectively.Source: §5.4.2 · p. 7
• The degree-bias comparison amplifies or reduces bias by sampling according to item degree, then compares CPR with DICE.θ=0.5 amplifies bias relative to the original training set; θ=−0.5 reduces it.Source: §5.4.2 · p. 7

5.5 Composition of Samples

• The experiments compare fixed single values k=1, 2, and 3 with mixtures of k=2 and 3 at different sampling ratios.Other hyperparameters are tuned for each setting.Source: §5.5 · pp. 7–8
• For k=2 or 3, combining sample types can outperform using a single type when the sampling ratio is appropriately chosen.The best ratios are typically around 3.Source: §5.5 · p. 8, Figure 5 · p. 8
• Recall decreases as k increases, but the performance at k=4 remains better than all baselines in Table 2.The passage attributes the drop as k grows probably to less flexible training.Source: §5.5 · pp. 7–8

6 RELATED WORK

• The review organizes unbiased recommendation methods for implicit feedback into three research lines.These are domain adaptation, counterfactual learning, and IPS.Source: §6 · p. 8

6.0.1 Domain Adaptation.

• Domain adaptation uses a small unbiased target dataset to guide training on biased source data, but such data is often rare in real scenarios.CausE [5] aligns biased and unbiased embeddings, while KDCRec uses knowledge distillation; ESAM [10] instead treats displayed and non-displayed items as domains.Source: §6.0.1 · p. 8

6.0.2 Counterfactual Learning.

• Counterfactual methods model or remove popularity-related effects through causal operations, unlike CPR’s focus on constructing an unbiased loss function.DICE [44] separates relevance and exposure embeddings; MACR [37] and CR [34] remove direct item-property effects on scores.Source: §6.0.2 · p. 8

6.0.3 Inverse Propensity Score (IPS).

• IPS methods reweight losses using estimated exposure probabilities, but estimating propensity scores is difficult and reweighting can have high variance.Rel-MF [30] applies IPS to implicit feedback; UBPR extends its pointwise model to pairwise ranking.Source: §6.0.3 · p. 8

• CPR achieves better debiasing performance on multiple backbone models.Source: §7 · p. 8
• Under a factorization assumption, CPR combines carefully selected user-item pairs to offset user/item propensity and approach unbiased user-preference ranking.Source: §7 · p. 8
• Using more interactions in one sample makes CPR training inflexible, limiting the ability to combine more sample types to further improve recommendation accuracy.Source: §7 · p. 8
• The assumed factorization of exposure probability may not accurately reflect the real exposure mechanism.The authors also identify group fairness, list ranking, and graph learning as future extension areas.Source: §7 · p. 8