PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarial Sampling and Training for Semi-Supervised Information Retrieval

Dae Hoon Park, Yi Chang

arXiv:1811.04155v2cs.IRcs.LG

TL;DR

Implicit-feedback retrieval has few clicks and many non-clicks, while linearity makes retrieval models vulnerable to adversarial examples. The paper combines adversarial augmentation and sampling; its approaches outperform baselines on Web search and item recommendation, match IRGAN on question answering, and outperform IRGAN in Web-search NDCG@5 with 5% labeled data.

  • Problem

    Implicit feedback produces relatively few clicked documents and many non-clicked ones, creating risks to generalization, effectiveness, and training efficiency; retrieval models are also vulnerable to adversarial examples due to linearity.

    Source: §1 · p. 1

  • Method

    The framework adversarially augments clicked examples and samples then perturbs unlabeled examples, alongside virtual adversarial training and its selective variation.

    Source: §1 · p. 1, §7 · p. 10

  • Results

    The approaches significantly outperform baselines on Web search and item recommendation, are on par with IRGAN on question answering, and outperform IRGAN in Web-search NDCG@5 with 5% labeled data.

    Source: §1 · p. 2, §7 · p. 10

  • Takeaways & Limitations

    The approaches perform especially well for high-ranked documents, and selective virtual adversarial training is more effective and efficient than virtual adversarial training.

    Source: §7 · p. 10

    The setup assumes far fewer clicked than unlabeled documents, and non-clicked documents may still be relevant.

    Source: §3 · p. 3

Abstract

from arXiv · show

Ad-hoc retrieval models with implicit feedback often have problems, e.g., the imbalanced classes in the data set. Too few clicked documents may hurt generalization ability of the models, whereas too many non-clicked documents may harm effectiveness of the models and efficiency of training. In addition, recent neural network-based models are vulnerable to adversarial examples due to the linear nature in them. To solve the problems at the same time, we propose an adversarial sampling and training framework to learn ad-hoc retrieval models with implicit feedback. Our key idea is (i) to augment clicked examples by adversarial training for better generalization and (ii) to obtain very informational non-clicked examples by adversarial sampling and training. Experiments are performed on benchmark data sets for common ad-hoc retrieval tasks such as Web search, item recommendation, and question answering. Experimental results indicate that the proposed approaches significantly outperform strong baselines especially for high-ranked documents, and they outperform IRGAN in NDCG@5 using only 5% of labeled data for the Web search task.

1 INTRODUCTION

Implicit feedback leaves retrieval models with few clicks and many non-clicks, while model linearity creates vulnerability to adversarial examples. The paper proposes adversarial sampling and training, reporting gains across retrieval tasks and against IRGAN for Web-search NDCG@5 with 5% labeled data.

1 INTRODUCTION

  • Clicked documents are relatively rare, while many non-clicked documents are uninformative and can slow training or harm effectiveness.Too few clicked examples can also impair model generalization.Source: §1 · p. 1
  • Retrieval models are vulnerable to adversarial examples because small input changes can greatly alter outputs in models with linearity.The passage notes that this vulnerability also applies to recent deep neural networks.Source: §1 · p. 1
  • The framework augments clicked examples adversarially and samples difficult non-clicked examples before generating still more informational negatives.It also proposes virtual adversarial training that generates examples without labels, including a more efficient and effective variation.Source: §1 · pp. 1–2
  • The proposed approaches are effective on ad-hoc retrieval tasks and outperform IRGAN in Web-search NDCG@5 with only 5% of the labeled data.Source: §1 · pp. 1–2
  • The experiments cover benchmark tasks in Web search, item recommendation, and question answering, and the approaches are reported as data-efficient.The paper compares mainly against IRGAN, which uses a different form of adversarial training for negative-document sampling.Source: §1 · pp. 1–2

2 RELATED WORK

The related work positions the paper at the intersection of adversarial training and negative sampling: it adapts label-free training to implicit-feedback retrieval and combines sampling with perturbation to make unlabeled examples more informative.

2.1 Adversarial Training

  • Earlier virtual adversarial training avoids requiring labels, while this work focuses on informativeness among abundant unlabeled examples and selectively trains on labeled and adversarially sampled examples.The retrieval setting can include both queries and documents as inputs, and the approach also uses pairwise learning to rank.Source: §2.1 · p. 2

2.2 Adversarial Training for Ad-hoc Retrieval

  • Unlike IRGAN, which samples negatives from unlabeled data, this approach perturbs existing examples and can further perturb adversarially sampled negatives.The paper also distinguishes its input perturbations and semi-supervised retrieval focus from a contemporaneous item-recommendation method that perturbs model parameters.Source: §2.2 · p. 2

2.3 Negative Sampling for Ad-hoc Retrieval

  • Prior negative-sampling methods include similarity-based and dynamic sampling, but the authors identify adversarially generating negatives on sampled examples as their distinct contribution.The related-work discussion also covers frequency-based sampling, uniform sampling, view information, and weighting informative negatives.Source: §2.3 · p. 3

3 PROBLEM DEFINITION

The paper formulates ad-hoc retrieval as ranking candidate documents by relevance to a query, with users/items and questions/answers as retrieval instances. Its implicit-feedback setting has relatively few clicked examples and many unlabeled documents, some of which may still be relevant.

3 PROBLEM DEFINITION

  • Ad-hoc retrieval ranks documents by estimated relevance to a query, and the query and candidates may represent users and items or questions and answers.The formulation assumes the Probability Ranking Principle.Source: §3 · p. 3
  • For each query, the setting assumes relatively few labeled clicked documents and many unlabeled non-clicked documents, which are not necessarily irrelevant.The paper describes this configuration as common in industry.Source: §3 · p. 3
  • Training examples are query-document-label triples, with clicks assigned relevance 1 and nonrelevant examples drawn from unlabeled documents.The learned function predicts a relevance score for a query-document pair.Source: §3 · p. 3
  • Few clicks risk overfitting, while many redundant or obviously irrelevant unlabeled documents provide little information for learning.The paper therefore motivates careful use of the imbalanced data.Source: §3 · p. 3

4 ADVERSARIAL SAMPLING AND TRAINING FRAMEWORK FOR AD-HOC RETRIEVAL

The framework treats labeled and unlabeled documents differently: adversarial training augments the positive class, while adversarial sampling followed by further adversarial training targets more informative negatives.

4 ADVERSARIAL SAMPLING AND TRAINING FRAMEWORK FOR AD-HOC RETRIEVAL

  • Adversarial training augments the relatively few labeled examples, while adversarial sampling selects informational unlabeled examples that are further perturbed to produce more informative negatives.The generated examples are also intended to help models cope with their linearity-related weakness.Source: §4 · p. 3
  • The framework presents adversarial examples, multiple adversarial-training methods, adversarial sampling, and pairwise learning as components for learning from implicit feedback.Source: §4 · p. 3

4.1 Adversarial Examples for Ad-hoc Retrieval

Adversarial examples are small input perturbations that can greatly alter a model’s output, a vulnerability linked to linear behavior in retrieval models. Small per-dimension changes can accumulate, potentially changing predictions for semantically similar documents.

4.1 Adversarial Examples for Ad-hoc Retrieval

  • Small perturbations can change activation by w⊺η, with maximum change growing linearly as ϵmn across n input dimensions.That change can propagate through piece-wise linear or nearly linear activation functions to the model output.Source: §4.1 · p. 3
  • Semantically identical documents can have different raw text or TF-IDF features, and subtle latent-vector differences may yield large output changes.The passage illustrates this with topic-probability vectors estimated for semantically identical documents.Source: §4.1 · pp. 3–4
  • Recent neural and traditional ad-hoc retrieval models can be vulnerable to adversarial examples because they behave in a quite linear way.Examples include ReLU-based neural networks, RankNet, and matrix factorization.Source: §4.1 · p. 4

4.2 Adversarial Training for Robust Ad-hoc Retrieval Models

The section describes adversarial training for robust retrieval models: it augments examples with bounded, loss-increasing perturbations and extends the approach to unlabeled data through virtual adversarial examples. For discrete inputs, perturbations are applied at the input so the method can handle both discrete and continuous features.

4.2.1 Adversarial Training.

  • Adversarial training adds perturbed versions of query-document examples while retaining their original relevance labels, encouraging robustness to difficult examples.The objective seeks perturbations that increase loss within a magnitude bound; the perturbation strength is controlled by ϵ, and α is set to 1.Source: §4.2.1 · p. 4, §4.2.2 · p. 4

4.2.2 Generating Adversarial Perturbation.

  • The perturbation is chosen to maximize loss subject to a norm bound and approximated with a first-order Taylor expansion; the work uses the L2 norm.The gradient can be computed by backpropagation, requiring only a few additional computations; the same ϵ is used for query and document perturbations in this work.Source: §4.2.2 · p. 4

4.2.3 Virtual Adversarial Examples.

  • Virtual adversarial training adds a KL-divergence cost that reduces changes in conditional relevance distributions under perturbation, encouraging local smoothness without requiring labels.A second-order approximation and a single power-method iteration are used; unlabeled examples can be used, though processing all non-clicked documents may be inefficient.Source: §4.2.3 · pp. 4–5

4.2.4 Adversarial Examples for Discrete Input.

  • For discrete inputs, the method perturbs the input before multiplying it by the embedding matrix, yielding a perturbed embedding that can also accommodate mixed discrete and continuous inputs.The proposed representation can act like a mixture of words, but its potential interpretability is left for future work.Source: §4.2.4 · p. 5

4.3 Adversarial Training with Adversarial Sampling

Adversarial sampling-based adversarial training augments labeled positives and creates difficult negatives by sampling unlabeled examples adversarially, then perturbing them further. The section also proposes selectively applying virtual adversarial training to difficult unlabeled examples rather than processing them all.

4.3 Adversarial Training with Adversarial Sampling

  • Adversarial sampling selects difficult unlabeled negatives, which are further perturbed, while adversarial training also augments labeled positive examples.The framework targets informational examples because unlabeled data may contain many redundant or obviously irrelevant documents.Source: §4.3 · p. 5
  • The training process uses all labeled positives and the same number of dynamically sampled negatives, drawing each negative from pθ(d|q,y = 1).The sampling distribution is estimated with a softmax whose smoothness is controlled by temperature τ.Source: §4.3 · p. 5
  • Unlike prior adversarial sampling approaches, this framework further perturbs sampled negatives and also generates adversarial examples for positives.The comparison names dynamic negative sampling and IRGAN as related approaches.Source: §4.3 · p. 5
  • Selective virtual adversarial training generates perturbations only for difficult unlabeled examples instead of all unlabeled documents.It samples one negative document per positive and is described as more efficient than original virtual adversarial training; the authors report it is empirically more effective as well.Source: §4.3 · pp. 5–6

4.4 Pairwise Training

Pairwise training uses relative document preferences to extend the adversarial approach, combining pairwise and pointwise virtual adversarial learning. Preferred documents come from labeled data, while non-preferred documents are sampled from unlabeled data.

4.4 Pairwise Training

  • Relative preference is useful when query-document relevance levels are harder to collect and user ratings may differ; pairwise and listwise learning have been studied for this setting [19, 29].Source: §4.4 · p. 6
  • Clicked and non-clicked documents can form query-wise pairs, but the authors leave potentially better pair-formation strategies outside this work’s scope.The described pairing uses clicked documents as positives and non-clicked documents as negatives.Source: §4.4 · p. 6
  • The pairwise objective compares a preferred labeled document with a non-preferred document sampled from unlabeled data according to pθ, using adversarial perturbations for the query and both documents.The perturbations are computed as in equation (4), and y+ ⊲ y− means the preferred document outranks the non-preferred one.Source: §4.4 · p. 6
  • Hybrid learning combines pairwise adversarial training with pointwise virtual adversarial loss for each positive and negative example, sampling one negative per labeled positive.The pairwise perturbation assumes relative relevance, while virtual adversarial perturbation assumes no labels.Source: §4.4 · p. 6

5 APPLICATION TO AD-HOC RETRIEVAL TASKS

The paper applies its retrieval approaches to Web search, item recommendation, and question answering, comparing primarily with IRGAN under shared models and datasets. It describes task-specific model choices, including adversarial training for models vulnerable to perturbations.

5 APPLICATION TO AD-HOC RETRIEVAL TASKS

  • The evaluation covers Web search, item recommendation, and question answering, with IRGAN as the main baseline and the same models used for a fair comparison except for a small Web search change.The tasks were chosen as popular, actively studied settings also used by IRGAN.Source: §5 · p. 6

5.1 Web Search

  • Web search represents query-document inputs as feature vectors and uses a two-layer neural network, replacing the tanh activation used by RankNet and IRGAN with ReLU.The authors also compare against a ReLU version of IRGAN; they state adversarial training makes ReLU’s vulnerability to adversarial examples not a concern.Source: §5.1 · p. 6

5.2 Item Recommendation

  • Item recommendation is treated as retrieval of items for users, using matrix factorization whose linear structure is vulnerable to adversarial perturbation.The authors therefore consider adversarial training desirable for this model.Source: §5.2 · p. 7

5.3 Question Answering

  • For retrieval-based question answering, answer relevance is modeled by cosine similarity between question and answer latent vectors, and adversarial training is applied to the model.The cited CNN- and LSTM-based approaches for obtaining those vectors can behave linearly and are vulnerable to adversarial perturbation.Source: §5.3 · p. 7

6 EXPERIMENTS ON THREE AD-HOC RETRIEVAL TASKS

The experiments evaluate pairwise AdvIR and its virtual-adversarial variants on Web search and item recommendation. The proposed approaches outperform baselines, with particular gains for high-ranked results and under reduced labeling.

6 EXPERIMENTS ON THREE AD-HOC RETRIEVAL TASKS

  • The experiments compare pairwise AdvIR with AdvIR-SVAT and AdvIR-VAT, using previously published baseline results for some comparisons.The authors state that AdvIR uses pairwise adversarial training and that AdvIR-VAT trains over all unlabeled examples.Source: §6 · p. 7

6.1.1 Experimental Design.

  • MQ2008-semi provides 784 queries averaging five labeled and 1,000 unlabeled examples each; evaluation uses precision and NDCG at N.The study treats relevance levels 1 and 2 as relevant labeled examples and all other pairs as unlabeled.Source: §6.1.1 · p. 7

6.1.2 Result Analysis.

  • On Web search and item recommendation, the proposed approaches significantly outperform baselines, especially for high-ranked documents.Table 1 reports Web search results; Table 2 reports item recommendation results.Source: Table 1 · p. 7, §6.1.2 · p. 8, Table 2 · p. 9
  • AdvIR’s learning curves surpass baseline best measures on test data from epoch 50, while only 5% of labeled data lets AdvIR and AdvIR-SVAT outperform IRGAN in NDCG@5.Figure 1 presents learning curves in panels a-b and data efficiency in panels c-d; with 20% of labeled data, the two methods outperform IRGAN in Precision@5.Source: Figure 1 · p. 8, §6.1.2 · p. 8
  • Adversarial sampling improves AdvIR over uniform negative sampling; even with uniform sampling, AdvIR outperforms IRGAN on Web search.AdvIR generates difficult examples through adversarial training, including for positive examples, whereas IRGAN samples negatives; switching to adversarial sampling improves AdvIR further.Source: Figure 2 · p. 8, §6.1.2 · p. 8

6.2.1 Experimental Design.

  • For item recommendation, the study uses Movielens 100k, labels 4- and 5-star ratings, and compares against BPR, LambdaFM, and IRGAN.The setup uses a 4:1 random training/test split and standard precision-at-N and NDCG-at-N metrics.Source: §6.2.1 · pp. 8–9

6.2.2 Result Analysis.

  • AdvIR-SVAT begins outperforming baseline best measures at about 100 epochs, while the item-recommendation learning curves oscillate less than Web search curves.With 50% of labeled data, the proposed models outperform IRGAN in Precision@5 and NDCG@5; their data efficiency is lower than for Web search.Source: §6.2.2 · p. 9
  • For item recommendation, AdvIR with adversarial sampling consistently outperforms AdvIR with uniform sampling.The paper attributes this to adversarial sampling providing more difficult examples for adversarial training to build on.Source: §6.2.2 · p. 9

6.3 Question Answering

On Insurance QA, dynamic-negative-sampling approaches outperform uniform-sampling QA-CNN, but the proposed approaches’ gains over IRGAN are not statistically significant. AdvIR reaches high precision quickly despite not using a pretrained model.

6.3.1 Experimental Design.

  • Insurance QA has 12,887 training and 1,000 development question-answer pairs, with two 1,800-pair test sets retrieving one correct answer from 500 candidates.The model uses CNNs for raw-text questions and answers, and the experiments compare against QA-CNN, LambdaCNN, and IRGAN.Source: §6.3.1 · p. 9

6.3.2 Result Analysis.

  • Dynamic-negative-sampling methods outperform QA-CNN, while the proposed approaches’ improvements over IRGAN are not statistically significant.The reported performance differences among the proposed approaches are also small; the authors describe the task as relatively easy because test negatives come from answers to other questions.Source: §6.3.2 · pp. 9–10, Table 3 · p. 10
  • Without a pre-trained model, AdvIR reaches high precision quickly, in 20 epochs.Figure 5 shows learning curves for question answering on test-1 and test-2.Source: §6.3.2 · p. 10, Figure 5 · p. 10

7 CONCLUSIONS

The paper proposes adversarial sampling and training for semi-supervised ad-hoc retrieval with implicit feedback. Across Web search, item recommendation, and question answering, the approaches outperform baselines on the first two tasks and are on par with IRGAN on question answering.

7 CONCLUSIONS

  • The framework generates informational positive examples adversarially and obtains negative examples through adversarial sampling followed by further adversarial generation.Virtual adversarial training and selective virtual adversarial training generate examples without labels; selective virtual adversarial training is reported as more effective and efficient than virtual adversarial training.Source: §7 · p. 10
  • Across Web search, item recommendation, and question answering, the approaches outperform baselines on the first two tasks and are on par with IRGAN on question answering.They perform especially well for high-ranked documents and are data-efficient; the reported results also indicate that adversarial sampling amplifies adversarial training.Source: §7 · p. 10
Loading 1811.04155v2…