PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarial Counterfactual Learning and Evaluation for Recommender System

Da Xu, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, Kannan Achan

arXiv:2012.02295v1cs.IRcs.LGstat.ML

TL;DR

Recommender feedback reflects which products users were exposed to, while exposure is only partly observed and may not be identifiable from feedback alone. The paper proposes minimax counterfactual learning as an adversarial game between recommendation models, and reports theoretical analyses and experiments illustrating its benefits.

  • Problem

    Feedback depends on product exposure, but feedback data only partly capture the exposure mechanism, complicating preference learning and unbiased evaluation.

    Source: §1 · pp. 1–2

  • Method

    The paper formulates counterfactual recommendation as minimax empirical risk and relaxes its dual into an adversarial game between two recommendation models.

    Source: §1 · p. 2

  • Results

    The paper analyzes supervised-learning inconsistency, proves generalization bounds, and reports simulation, real-data, and online experiments illustrating the approach's benefits.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The framework offers a principled route to counterfactual learning and evaluation when exposure cannot be identified from feedback data alone.

    Source: §1 · p. 2

    Improvement depends on the uncertainty in the feedback data, and suboptimal tuning can cause undesired propensity-model behavior during adversarial training.

    Source: §6 · p. 9

Abstract

from arXiv · show

The feedback data of recommender systems are often subject to what was exposed to the users; however, most learning and evaluation methods do not account for the underlying exposure mechanism. We first show in theory that applying supervised learning to detect user preferences may end up with inconsistent results in the absence of exposure information. The counterfactual propensity-weighting approach from causal inference can account for the exposure mechanism; nevertheless, the partial-observation nature of the feedback data can cause identifiability issues. We propose a principled solution by introducing a minimax empirical risk formulation. We show that the relaxation of the dual problem can be converted to an adversarial game between two recommendation models, where the opponent of the candidate model characterizes the underlying exposure mechanism. We provide learning bounds and conduct extensive simulation studies to illustrate and justify the proposed approach over a broad range of recommendation settings, which shed insights on the various benefits of the proposed approach.

1 Introduction

Recommender feedback depends on which products users were exposed to, creating challenges for both preference learning and unbiased offline evaluation. The paper addresses these challenges with a minimax counterfactual framework that treats uncertainty about exposure as an adversarial component.

1 Introduction

  • Feedback depends on prior product exposure, so supervised learning can inconsistently detect preferences and offline evaluation can favor the historical exposure mechanism.The candidate model's exposure distribution is almost never reflected in feedback-only evaluation.Source: §1 · p. 1
  • Because exposure is only partially captured in feedback, its mechanism is not identifiable from feedback data alone, limiting existing counterfactual methods that rely on additional assumptions.The paper notes that full exposure logs are often restricted or absent from public datasets.Source: §1 · p. 2
  • The proposed minimax approach treats uncertainty about exposure as adversarial and converts the relaxed dual problem into a game between two recommendation models.The opponent model characterizes the underlying exposure mechanism.Source: §1 · p. 2
  • The paper analyzes supervised-learning inconsistency, proves generalization bounds for its adversarial learning approach, and reports simulation, real-data, and online experiments.The experiments are presented as illustrating the approach's benefits.Source: §1 · p. 2

2 Preliminaries

The preliminaries distinguish preference learning with a known exposure mechanism from the inconsistent results possible when that mechanism is unknown. They introduce propensity weighting and discuss the hidden independence assumption behind the click model.

2 Preliminaries

  • For implicit feedback, supervised learning minimizes a surrogate loss over user-item pairs, with exposure incorporated into the joint distributions of positive and negative feedback.Source: §2 · p. 3
  • With exposure fixed, the optimal loss depends on both user preference and exposure, while the optimal model depends on p(Y|o,x,z), not the exposure mechanism.Source: §2 · p. 3
  • When exposure is unknown, the optimal supervised model can depend on the exposure mechanism, so identical feedback collected under different mechanisms may yield different inferred preferences.Source: §2 · p. 3
  • Propensity weighting uses exposure probabilities to form a synthetic sample whose observed-feature distribution is independent of exposure, recovering the ordinary loss in expectation.Source: §2 · pp. 3–4

The hidden assumption of the missing-data (click) model

  • The click model assumes relevance is independent of exposure given features, a condition rarely true or verifiable unless features include every factor affecting exposure and preference.Source: The hidden assumption of the… · p. 4

3 Method

The method handles exposure-related uncertainty by optimizing recommendation risk against worst-case distributions and recasting the dual problem as an adversarial exposure-model game. It supplies generalization guarantees, practical surrogate regularizers, and a robust offline evaluation procedure.

3 Method

  • The candidate model minimizes risk over a Wasserstein neighborhood of the ideal exposure-eliminated distribution, and duality recasts this as an adversarial exposure-model game.A regularizer keeps the estimated exposure mechanism close to the underlying mechanism.Source: §3 · pp. 4–5

3.1 Theoretical property

  • The minimax approach retains the standard ERM 1/√n rate while adding a δρ bias term from adversarial uncertainty.Data-dependent bounds connect the result to propensity-weighted empirical loss and standard ERM.Source: §3.1 · pp. 5–6

3.2 Practical implementations

  • Because the true exposure model is unknown and Wasserstein distance can be difficult to compute, practical implementations use tractable regularizers for the adversary.For implicit feedback, the paper focuses on using loss on feedback data as a regularizer because it applies broadly without excessive assumptions.Source: §3.2 · p. 6
  • For unobserved factors, the approach uses Tukey’s factorization to model the exposure–outcome relationship and a simple logistic regression for the exposure model.The paper notes that unobserved factors are practically inevitable.Source: §3.2 · pp. 6–7

3.3 Minimax optimization and robust evaluation

  • Sequential training updates the candidate model first and the exposure adversary second; with sufficiently different timescales, stationary points are locally minimax up to degenerate cases.The cited simultaneous-training convergence guarantees do not apply to this sequential setup.Source: §3.3 · p. 7
  • Robust offline evaluation uses the worst-case propensity score model specialized for the candidate, rather than pursuing unbiased evaluation.The authors state that this frees offline evaluation from potential exposure-mechanism effects and gives a robust view of performance.Source: §3.3 · p. 7

4 Relation to other work

The paper distinguishes its exposure-focused counterfactual approach from prior work on GANs, distributional robustness, and adversarial robustness. It also contrasts its approach with recommender counterfactual methods that rely on assumptions about the data or model.

4 Relation to other work

  • Unlike GANs, this work studies the sampling and propensity-score distributions rather than generating and feature distributions, and addresses identifiability rather than deliberate adversarial behavior.The paper also distinguishes its setting from adversarial robustness and model defense.Source: §4 · p. 7
  • Prior counterfactual recommender methods often rely on data or model assumptions, while this work does not assume recommenders have a generative nature.The paper notes that assumption violations can make earlier approaches vulnerable in practice [35] [18] [1] [23] [33] [2].Source: §4 · p. 7
  • Table 1 reports true-exposure unbiased evaluations on semi-synthetic data, with PS/ACL configurations above original baselines and results averaged over ten repetitions.The caption says bold and underlining mark the best and second-best outcomes; full numerical results are deferred to Appendix A.6.Source: Table 1 · p. 8

5 Experiment and Result

The approach is evaluated in simulation, real-data analysis, and online experiments. The results report gains under unbiased and standard offline evaluation, and closer agreement between robust offline evaluation and online outcomes.

5 Experiment and Result

  • The study covers simulation, real-data analysis, and online experiments, using Pop, CF, MLP, NCF, GMF, and Attn as representative baselines.The next-item task uses all but the last two interactions for training, the second-to-last for validation, and the last for testing.Source: §5 · p. 8
  • Across simulation, real-data, and online studies, the approach performs favorably under the stated offline evaluations, while robust evaluation better approximates online results.The reported settings include synthetic data with oracle exposure, benchmark real data, and online A/B tests.Source: §5 · pp. 8–9
  • In synthetic data generated with oracle exposure, proposed training gives baseline models their best unbiased-evaluation performance apart from oracle-enhanced counterparts.The simulation uses MovieLens-1M and Goodreads explicit-feedback data to construct an oracle exposure mechanism.Source: §5 · pp. 8–9
  • On MovieLens-1M, Goodreads, and LastFM real data, the proposed approach yields the best outcome under standard evaluation, which does not account for exposure.Table 2 presents standard evaluations and identifies the fθ and gψ configurations for the PS and ACL approaches.Source: §5 · p. 9, Table 2 · p. 9
  • In eight Walmart.com online experiments, robust evaluation is reported as a more robust approximation to online results than standard, popularity-debiased, or propensity-score offline evaluation.The reported comparison uses MSE to online evaluation results.Source: §5 · p. 9, Table 3 · p. 9

6 Conclusion

The conclusion presents adversarial counterfactual learning and evaluation as a theoretically grounded framework, while noting that its improvement depends on feedback-data properties and that optimization dynamics remain to be studied.

6 Conclusion

  • The paper proposes a theoretically grounded adversarial counterfactual learning and evaluation framework after analyzing drawbacks of supervised learning for recommenders.The conclusion also reports theoretical and empirical results illustrating benefits of the approach.Source: §6 · p. 9
  • Improvement depends on feedback-data properties, including the extent to which identifiability issues create uncertainty.Source: §6 · p. 9
  • Suboptimal tuning parameters can cause undesirable propensity-model behavior during adversarial training, and the effects of optimization dynamics on the two-model interaction remain open.Source: §6 · p. 9

Broader Impact

The paper reports no major ethical concerns or societal consequences to its knowledge and identifies robust offline learning and evaluation as a potential benefit for recommender-system researchers and practitioners.

Broader Impact

  • The authors report that, to their knowledge, the approaches raise no major ethical concerns or societal consequences.Source: Broader Impact · p. 10
  • Researchers and practitioners may benefit because robust offline learning and evaluation is described as a significant real-world challenge.Source: Broader Impact · p. 10
  • If the approach fails, its worst possible outcome is reducing to standard offline learning when the propensity model stops having the desired impact.The extent of the data identifiability issue depends on data properties.Source: Broader Impact · p. 10

Appendix

The appendix supplies proofs and further discussion of the relaxation underlying the two-model objective, along with experiment details and complete numerical results.

Appendix

  • The appendix provides proofs for the paper’s major theoretical results.Source: Appendix · p. 13
  • It discusses the relaxation for Claim 2 and the origins and implications of Tukey’s factorization on unobserved factors leading to the final two-model adversarial objective in (9).Source: Appendix · p. 13
  • It provides experiment details and complete numerical results, including demonstrations of the two models’ adversarial training process.Source: Appendix · p. 13

A.1 Proof for Claim 1

The proof considers a fixed exposure mechanism and derives properties of a function defined from exposure-probability ratios. It also characterizes the optimal candidate model through an optimization problem.

A.1 Proof for Claim 1

  • For a fixed exposure mechanism, the proof defines μ(o) as the ratio of exposure probabilities and uses it to form Δ(μ).The passage states μ(o) = p(−1)(o)/p(1)(o).Source: §A.1 · p. 13
  • Δ(μ) is convex because it is the supremum of affine functions, and the proof also states that it is continuous.Source: §A.1 · p. 13
  • At the infimum in (A.1), the optimal fθ is obtained by solving for a*.Source: §A.1 · p. 13

A.2 Proof for Claim 2 and the relaxation

This appendix section proves the dual formulation for minimax ERM and discusses its relaxation. The derivation uses a Wasserstein representation and identifies conditions and empirical surrogates for the result.

A.2 Proof for Claim 2 and the relaxation

  • The section first establishes the dual formulation for minimax ERM, then derives a relaxation of the dual problem.Source: §A.2 · p. 13
  • The duality argument assumes propensity scores are bounded away from zero, which allows the weighted distributions to exist and supports use of Slater’s condition.Source: §A.2 · p. 14
  • Kantorovich-Rubinstein duality yields a Wasserstein relaxation, with data-dependent surrogates used in practice because the distance is intractable.The passage identifies the Wasserstein distance representation and says the specific cost functions do not matter for the surrogates.Source: §A.2 · p. 15
  • The proof relates the distance between propensity-weighted empirical distributions to a Wasserstein distance between exposure distributions.Source: §A.2 · p. 15

A.3 Proof for Theorem 1

The proof of Theorem 1 combines bounded-difference concentration with symmetrization and complexity bounds. It concludes by combining the intermediate bounds.

A.3 Proof for Theorem 1

  • The proof bounds deviations using the uniform bound on |δfθ(h)| and McDiarmid’s inequality.Source: §A.3 · p. 16
  • It applies symmetrization with independent Rademacher variables and copies of the sample observations.Source: §A.3 · p. 16
  • The argument bounds expected complexity using Rademacher complexity and Dudley’s entropy integral.Source: §A.3 · p. 16
  • The proof combines the intermediate bounds in (A.11), (A.12), and (A.15) to obtain the theorem’s result.Source: §A.3 · p. 16

A.4 Proof for Corollary 1

This appendix characterizes the exposure mechanism despite unobserved factors and describes evaluations of the proposed adversarial approach, including training, ablation, sensitivity, real-world, semi-synthetic, and online experiments. The online experiments compare offline-trained models against online click-through rate while keeping the recommendation architecture unchanged across deployment iterations.

A.5 Implications from Tukey’s Factorization on Unobserved Factors for Exposure

  • Tukey’s factorization characterizes the missing-data distribution despite unobserved factors, with the unknown exposure mechanism remaining the component needed to obtain counterfactual outcome distributions.The passage notes that unobserved factors confound the two potential outcomes, violating an assumption of many causal-inference methods; the exposure mechanism is described as learnable in this setting.Source: §A.5 · p. 18

A.6.1 Real-world datasets

  • Real-world evaluations cover movie, music, and book recommendation datasets, using each user’s final interaction for testing and the preceding one for validation.The appendix describes filtering infrequent users and items and excluding users with more than 1,000 interactions; online experiments use personalized lists of ten items on item pages under implicit feedback.Source: §A.6.1 · p. 18, §A.6.2 · p. 19, §A.6.9 · p. 25

A.6.3 Simulation settings

  • The semi-synthetic simulation uses a two-stage process to preserve signal from real explicit-feedback data while adding a controllable exposure component.The first stage estimates relevance and partial exposure from MovieLens-1M and Goodreads data; the second adds a self-defined exposure function to control the mechanism.Source: §A.6.3 · pp. 19–20

A.6.4 Model configuration and implementation

  • The proposed approach uses separate configuration for fθ and gψ plus two-timescale updates, and its per-step space and time costs equal the sum of those models’ costs.The method may require more epochs to converge depending on the relative learning rates; its specific tuning parameters include rθ, rψ, and α.Source: §A.6.4 · p. 20, §A.6.5 · p. 21

A.6.6 Visualization of the adversarial training process

  • Training traces show that fθ eventually overtakes gψ on evaluation metrics while retaining a larger objective, consistent with gψ acting adversarially as the models approach local minimax behavior.The same training pattern holds on real-world sequential recommendation data; excluding Y from Gβ preserves the pattern but yields less ideal loss and evaluation metrics.Source: §A.6.6 · p. 21

A.6.7 Complete ablation study

  • The complete propensity-score ablation results again favor adversarial counterfactual training, highlighting the adversarial process between fθ and gψ.The reported comparison covers propensity-score models on three real-world datasets.Source: §A.6.7 · p. 21, Table A.2 · p. 22
  • Adversarially trained models uniformly outperform their real-world counterparts without relying on greater model complexity, since baseline dimensions were doubled for parameter parity.The appendix also reports that adversarial counterfactual training outperforms propensity-score counterparts on three real-world datasets.Source: §A.6.7 · pp. 21–22, Table A.2 · pp. 22–23

A.6.8 Sensitivity analysis

  • Larger hidden dimensions most often improve outcomes, while α performs best at an intermediate value: too little weakens gψ and too much shifts the minimax game away from fθ.The authors caution that the α analysis covers a limited range.Source: §A.6.8 · p. 23, Figure A.3 · p. 24, Figure A.4 · p. 25

A.6.9 Online experiment settings

  • Across deployment iterations, new item and user features were added while the recommendation model’s major architecture remained unchanged.This stability made the approach favorable for examination.Source: §A.6.9 · p. 25
  • The study included four online A/B experiments covering eight offline-trained adversarial counterfactual models, evaluated with historical implicit feedback and compared using click-through rate.Real-time user features, page layout, and same-page advertisements were excluded because they continually changed.Source: §A.6.9 · p. 25

Related papers

Loading 2012.02295v1…