PaperlayerView exact revision on arXiv

Source-linked AI summary

IRGAN: A Minimax Game for Unifying Generative and Discriminative Information Retrieval Models

Jun Wang, Lantao Yu, Weinan Zhang, Yu Gong, Yinghui Xu, Benyou Wang, Peng Zhang, Dell Zhang

arXiv:1705.10513v2cs.IRcs.LG

TL;DR

Information retrieval has generative and discriminative modelling traditions with complementary limitations in using relevance signals and unlabelled data. IRGAN unifies them through an adversarial minimax game in which each model trains against the other. Across web search, item recommendation, and question answering, the framework produced significant improvements over strong baselines, while convergence remains unresolved when the true relevance distribution is unknown.

  • Problem

    Generative models have difficulty leveraging links and clicks, while discriminative models lack a principled way to exploit text statistics and relevant-document distributions in massive unlabelled data.

  • Method

    IRGAN iteratively trains a generative retrieval model and a discriminative retrieval model in a minimax game, with the discriminator guiding generation and the generator supplying difficult examples.

  • Results

    IRGAN achieved statistically significant improvements across all evaluation metrics and datasets in the reported experiments, including web search, item recommendation, and question answering.

  • Takeaways & Limitations

    The framework uses generative signals to guide document selection and adversarially selected examples to enhance discriminative ranking.

  • Takeaways & Limitations

    When the true relevance distribution is unknown, how the generative and discriminative models converge to the theoretical equilibrium remains an open problem.

Abstract

from arXiv · show

This paper provides a unified account of two schools of thinking in information retrieval modelling: the generative retrieval focusing on predicting relevant documents given a query, and the discriminative retrieval focusing on predicting relevancy given a query-document pair. We propose a game theoretical minimax game to iteratively optimise both models. On one hand, the discriminative model, aiming to mine signals from labelled and unlabelled data, provides guidance to train the generative model towards fitting the underlying relevance distribution over documents given the query. On the other hand, the generative model, acting as an attacker to the current discriminative model, generates difficult examples for the discriminative model in an adversarial way by minimising its discrimination objective. With the competition between these two models, we show that the unified framework takes advantage of both schools of thinking: (i) the generative model learns to fit the relevance distribution over documents via the signals from the discriminative model, and (ii) the discriminative model is able to exploit the unlabelled data selected by the generative model to achieve a better estimation for document ranking. Our experimental results have demonstrated significant performance gains as much as 23.96% on Precision@5 and 15.50% on MAP over strong baselines in a variety of applications including web search, item recommendation, and question answering.

1 INTRODUCTION

Information retrieval spans generative models that model document generation and discriminative models that predict relevance from query-document features. IRGAN unifies these approaches in a minimax game, improving both model training and retrieval performance across applications.

  • Generative retrieval: Generative IR models represent document generation from an information need, while related models may reverse the direction by generating queries from documents.Examples include relevance models, query-likelihood models, word embeddings, and recommender systems.
  • Discriminative retrieval: Discriminative IR models combine queries and documents as features and predict relevance or ranking labels from labelled or proxy feedback.Learning to rank is a major web-search example of this approach.
  • Motivation: Generative models struggle to use signals such as links and clicks, whereas discriminative models lack a principled way to exploit text statistics and relevant-document distributions in unlabelled data.These complementary limitations motivate a unified framework.
  • IRGAN: IRGAN treats generative and discriminative retrieval as two players sharing an objective: the discriminator guides generation, while the generator supplies difficult examples by adversarially minimising that objective.The models iteratively compete to improve against one another.
  • Empirical results: 23.96% on Precision@5 in web search was achieved by the pairwise adversarial model, while pointwise training improved strong baselines by 22.56% in web search and 14.38% in item recommendation.The experiments covered web search, item recommendation, and question answering.

2 IRGAN FORMULATION

IRGAN is a unified framework that fuses generative and discriminative information retrieval through adversarial training in a minimax game.

  • IRGAN formulation: IRGAN combines generative and discriminative IR models in an adversarial minimax framework inspired by GANs.The framework is applied to concrete information retrieval problems in subsequent sections.

2.1 A Minimax Retrieval Framework

The minimax retrieval framework models generation and discrimination as opposing objectives over query-document relevance. The generator approximates the relevance distribution and challenges the discriminator with sampled documents, using policy gradients for discrete sampling.

  • Model roles: The generative model pθ(d|q,r) approximates the true relevance distribution over candidate documents, while the discriminative model fϕ(q,d) classifies relevant and nonrelevant query-document pairs.The discriminator is a binary classifier over matching and nonmatching tuples.
  • Minimax game: IRGAN lets the generator produce relevant-looking documents that can fool the discriminator, while the discriminator separates true relevant documents from generated ones.Both models optimise the same minimax objective in opposite directions.
  • Discriminator optimisation: The discriminator maximises the log-likelihood of distinguishing observed relevant documents from documents sampled from the current generator.Its optimisation can use stochastic gradient descent when fϕ is differentiable.
  • Generator optimisation: The generator minimises the objective by fitting the underlying relevance distribution and sampling documents from the full document set to fool the discriminator.Unlike the original GAN setting, IRGAN directly generates known document identifiers rather than document features.
  • Training procedure: REINFORCE optimises the discrete generator: log(1 + exp(fϕ(d,q))) serves as the reward, and an advantage function reduces learning variance.The algorithm alternates generator updates with discriminator training on generated negatives and given positives.

2.2 Extension to Pairwise Case

IRGAN extends the minimax framework to pairwise learning-to-rank data, where the generator creates ranked document pairs and the discriminator evaluates whether their ordering resembles labelled preferences.

  • Pairwise setting: Pairwise IR uses ordered document pairs because relative preference judgements are often easier to obtain than absolute relevance labels.Click-through logs are one source of such pairwise feedback.
  • Pairwise minimax game: In the pairwise game, the generator produces document pairs with correct rankings, while the discriminator distinguishes generated pairs from labelled real pairs.The discriminator estimates pairwise correctness through a sigmoid of score differences.
  • Pairwise objective: The pairwise discriminator probability uses a logistic ranking loss equivalent to RankNet’s loss, with hinge and exponential alternatives corresponding to Ranking SVM and RankBoost.These alternatives can define the pairwise discrimination probability.
  • Pair generation: IRGAN generates new pairs by retaining a lower-ranked labelled document and pairing it with an unlabelled document selected to imitate the higher-ranked document.The strategy targets unlabelled documents likely to resemble more relevant examples.
  • Optimisation: The generator can be trained with REINFORCE in the pairwise setting, using the same procedure described for pointwise retrieval.The stated formulation assumes a softmax generator in the relevant derivation.

2.3 Discussion

IRGAN’s minimax game has a theoretically characterized equilibrium when the true relevance distribution is known, but practical convergence remains task-dependent because that distribution is unknown. The generator and discriminator iteratively challenge one another, with training complexity reducible through hierarchical softmax.

  • Theoretical equilibrium: At Nash equilibrium, the generator matches the true relevant-document distribution and the discriminator cannot distinguish generated from true documents.This result is established for both pointwise and pairwise formulations when the true relevance distribution is known exactly.
  • Practical behavior: In practice, the true relevance distribution is unknown, so convergence to the theoretical equilibrium remains an open problem.Empirically, the two models reach different performance levels depending on the task, with at least one model significantly improving over its non-adversarial counterpart.
  • Adversarial interaction: The generator produces difficult samples near the discriminator’s decision boundary, while the discriminator scores those generated samples down during subsequent training.The two models are described as having positively correlated scores on observed and unobserved positive documents.
  • Adversarial interaction: IRGAN training uses upward REINFORCE pressure from the generator and downward pressure from the discriminator to iteratively move samples around the decision boundary.The accompanying illustration labels observed, unobserved, and generated samples alongside the two opposing training forces.
  • Efficiency: O(NKM) per GAN iteration can be reduced to O(NK log M) by applying hierarchical softmax during generator sampling.The complexity is linear in the number of candidate documents M before this reduction.

2.4 Links to Existing Work

IRGAN extends adversarial learning to retrieval by jointly training conditional generative and discriminative models over discrete documents. It differs from NCE, SCE, pseudo relevance feedback, and ordinary GANs in its adversarial interaction and retrieval-specific sampling process.

  • Relation to GANs: Unlike ordinary GANs, IRGAN performs stochastic sampling over discrete candidate documents and trains its generator with REINFORCE.The generator can be viewed as an actor selecting a candidate document for a query.
  • Relation to GANs: IRGAN’s generative and discriminative models are both conditional on the query, linking the framework to conditional GANs.The query conditions both players rather than serving only as an external evaluation context.
  • Relation to retrieval methods: IRGAN differs from pseudo relevance feedback because its generator-selected documents serve as negative ranker samples and training proceeds iteratively.Pseudo relevance feedback instead treats top retrieved documents as positive samples and usually does not perform further iterations.
  • Adversarial retrieval: IRGAN trains generative and discriminative retrieval models together in an adversarial fashion, with each model learning against the other.The discriminator distinguishes true from generated samples, while the generator produces samples intended to fool it.
  • Relation to estimation methods: Compared with NCE and SCE, IRGAN enables the two retrieval models to learn together through adversarial competition rather than contrastive estimation alone.NCE and SCE are presented as alternatives or related estimation procedures for learning probability distributions and retrieval features.

3 APPLICATIONS

The paper applies IRGAN across web search, item recommendation, and question answering, using task-specific relevance scoring functions within a shared adversarial framework. These applications span neural ranking, matrix factorization, and convolutional sentence matching implementations.

  • Application scenarios: IRGAN is evaluated in web search with learning to rank, item recommendation, and question answering.These three scenarios instantiate the framework across distinct retrieval settings.
  • Shared framework: Lower generator temperature concentrates sampling on top-ranked documents, while temperature 0 makes the generator select the highest-ranked documents deterministically.The generator distribution depends on its scoring function, and temperature controls the concentration of sampling.
  • Shared framework: The generator and discriminator use separate parameterized scoring functions, although the implementation section chooses the same function form for both players.The shared form is written as gθ(q,d) = sθ(q,d) and fϕ(q,d) = sϕ(q,d).
  • Web search: In web search, query-document pairs are represented by feature vectors containing statistics such as BM25, PageRank, and TFIDF.The relevance score is implemented with a two-layer neural network following RankNet.
  • Item recommendation: In item recommendation, the query is a user profile and the basic relevance model is matrix factorization with item biases and user/item latent vectors.The paper omits global and user biases for top-N recommendation and notes that more sophisticated models could replace the basic implementation.
  • Question answering: In question answering, convolution and max-pooling produce question and answer representations whose matching score transforms QA into query-document ranking.The implementation uses a cosine-distance matching probability similar to a basic QA-CNN model.

4 EXPERIMENTS

The experiments evaluate IRGAN across web search, item recommendation, and question answering, with web-search tests covering pointwise and pairwise formulations. Results show improvements over strong baselines, while sampling temperature and model capacity materially affect performance.

  • Experimental Scope: IRGAN is evaluated on web search, item recommendation, and question answering, with both pointwise and pairwise formulations tested for web search.The experiments use three real-world information-retrieval applications and compare pointwise and pairwise IRGAN within web search.
  • Web Search Setup: 784 queries average about 5 positive and 1,000 unlabelled documents, supporting evaluation under implicit-feedback and semi-supervised conditions.Pairs with relevance above 0 are treated as positive, while relevance levels −1 and 0 are treated as unlabelled; data use a 4:1 random split.
  • Web Search Results: IRGAN improves performance across all reported metrics on MQ2008-semi, where its generative model ranks documents by sampling with temperature near 0.Comparisons include RankNet, LambdaRank, and LambdaMART, using Precision@N, NDCG@N, MAP, and MRR.
  • Web Search Results: IRGAN-pairwise performs better on Precision@3 and NDCG@3, while IRGAN-pointwise performs better on Precision@10, NDCG@10, and MAP.The pairwise formulation focuses on the top few webpages, whereas the pointwise formulation performs better when more high-ranked webpages are considered.
  • Experimental Caveat: LambdaMART is not particularly effective in the tested implicit-feedback setting, possibly because it relies on ΔNDCG scoring for document pairs.The dataset contains implicit feedback, with relatively little labelled data and substantial unlabelled data.
  • Sensitivity Analyses: 0.2 is the empirically optimal sampling temperature for pointwise IRGAN; performance rises up to this value and declines afterward.The result indicates that appropriately increasing the generator’s aggressiveness toward top-ranked documents matters for ranking performance.
  • Sensitivity Analyses: Model capacity should be at least as high for the prediction-making model as for its opponent: the generator in pointwise IRGAN and discriminator in pairwise IRGAN.For pointwise IRGAN, the neural-network generator outperforms the linear generator; for pairwise IRGAN, the neural-network discriminator outperforms its linear version.
  • Item Recommendation: IRGAN achieves statistically significant improvements across all evaluation metrics and datasets in item recommendation, outperforming LambdaFM despite not directly optimising final ranking measures.The authors attribute this comparison to greater learning flexibility from adversarial training than from single-model training.

5 CONCLUSIONS

IRGAN unifies generative and discriminative retrieval through adversarial training in a minimax game. Experiments across three IR tasks report significant gains, while different task settings can reach different equilibria.

  • IRGAN unifies generative and discriminative retrieval models through adversarial training in a minimax game.The generator receives signals from the discriminator, while the discriminator uses strategic negative sampling from the generator.
  • The framework guides generative retrieval with discriminator signals and enhances document ranking through strategic negative sampling.
  • Experiments on four real-world datasets across web search, item recommendation, and question answering observed significant performance gains in each set of experiments.
  • Different task settings can reach different equilibria; pointwise training improves the generator more, whereas pairwise training shows the opposite pattern.The paper identifies this difference as warranting further investigation.
  • Further work will evaluate more real-world datasets and extend the framework to word-token generation and learned ranking features.

APPENDIX

The appendix reviews original GANs for continuous data and IRGAN for discrete data, emphasizing that their gradient analyses lead to different practical objective implementations.

  • The appendix compares continuous-data GANs with IRGAN's discrete-data formulation through gradient analysis and objective derivations.It clarifies why the two GAN settings require different implementations in practice.

A GAN FOR CONTINUOUS DATA

For continuous data, GAN training defines a minimax objective between a generator and discriminator. The discriminator maximizes the objective while the generator minimizes it, with an ideal equilibrium matching generated and true data distributions.

  • The original GAN objective combines real-data discrimination and generated-data discrimination in a minimax game.
  • The generator maps noise z to data x, while the discriminator is a sigmoid-based binary classifier.
  • The discriminator maximizes the objective with the generator fixed, whereas the generator minimizes it with the discriminator fixed.
  • The ideal equilibrium has the generated data distribution equal to the true data distribution and the discriminator returning 0.5 for each generated instance.
  • The implementation replaces log(1 − Dϕ(Gθ(z))) with −log Dϕ(Gθ(z)) to provide more sufficient generator gradients early in training.The paper notes that this setting is widely adopted despite later theoretical qualifications.
  • The continuous-data gradient mechanism requires differentiating the discriminator's prediction with respect to generated data, which works for image pixels and speech audio.

B GAN FOR DISCRETE DATA AND IMPLEMENTATION DETAILS

For discrete data, IRGAN models document selection as a reinforcement-learning policy and trains it with policy gradients. The implementation modifies rewards to improve training stability.

  • IRGAN models its generator as a reinforcement-learning policy that selects a document given a query and relevance metric.The policy is trained with policy gradients because document sampling is discrete.
  • The discrete-data generator uses REINFORCE, with the discriminator-derived value functioning as the reward for selecting a document.
  • Using logσ(fϕ(d,qn)) directly as the reward can destabilize REINFORCE and cause parameter explosion when discriminator logits are strongly negative.
  • The continuous-data gradient issue does not arise in the original GAN because its gradient remains well behaved when the discriminator sigmoid is near zero.
  • IRGAN drops the discriminator logarithm in its experimental implementation to address discrete-data training issues.
  • A baseline or advantage function is used to reduce policy-gradient variance, and the implementation reports promising results across three application cases.
Loading 1705.10513v2…