PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning to select data for transfer learning with Bayesian Optimization

Sebastian Ruder, Barbara Plank

arXiv:1707.05246v1cs.CLcs.LG

TL;DR

Existing transfer-learning approaches often rely on ad hoc similarity measures, while prior domain-adaptation work commonly focuses on one-to-one settings and limited metric choices. The paper uses Bayesian Optimization to learn data-selection measures from similarity and diversity features across models, domains, and tasks. The learned measures outperform existing domain-similarity metrics on sentiment analysis, POS tagging, and parsing, with promising transferability but some overfitting-related negative transfer.

  • Problem

    Existing approaches use ad hoc task-specific data-selection measures, while prior work studies similarity mostly in isolation and often focuses on one task or one-to-one adaptation.

  • Method

    The paper uses Bayesian Optimization to learn model-independent data-selection measures from similarity and diversity features for selecting training data from multiple sources.

  • Results

    The learned measures outperform existing domain-similarity metrics on sentiment analysis, POS tagging, and parsing and are competitive with a state-of-the-art domain-adaptation approach.

  • Takeaways & Limitations

    Diversity complements similarity, and learned data-selection measures show promising transfer across models, domains, and related tasks.

  • Takeaways & Limitations

    Similarity-plus-diversity features can overfit to the domain and produced two instances of negative transfer when transferred across domains.

Abstract

from arXiv · show

Domain similarity measures can be used to gauge adaptability and select suitable data for transfer learning, but existing approaches define ad hoc measures that are deemed suitable for respective tasks. Inspired by work on curriculum learning, we propose to \emph{learn} data selection measures using Bayesian Optimization and evaluate them across models, domains and tasks. Our learned measures outperform existing domain similarity measures significantly on three tasks: sentiment analysis, part-of-speech tagging, and parsing. We show the importance of complementing similarity with diversity, and that learned measures are -- to some degree -- transferable across models, domains, and even tasks.

1 Introduction

The paper learns model-agnostic data-selection measures for transfer learning from multiple sources, addressing limits of ad hoc similarity metrics and one-to-one adaptation. Across tasks and domains, it highlights the complementary value of diversity and examines transferability across models, domains, and tasks.

  • Motivation: Prior work primarily studied one-to-one adaptation and task-specific similarity measures, with limited use of multiple sources.Earlier metrics were generally examined in isolation and focused on similarity for a single task.
  • Contribution: The paper proposes the first model-independent approach to learn data-selection measures for transfer learning.The approach is inspired by curriculum learning but targets selecting relevant data from multiple sources rather than speeding up learning.
  • Results: The learned measure outperforms baselines across three tasks and multiple domains while remaining competitive with state-of-the-art domain adaptation.The evaluated tasks are sentiment analysis, part-of-speech tagging, and parsing.
  • Findings: Diversity is empirically as important as domain similarity and complements it for transfer learning.The study evaluates a range of similarity metrics together with diversity rather than relying on similarity alone.
  • Transferability: The paper studies how learned data-selection measures transfer across models, domains, and tasks.This extends evaluation beyond a single model, domain, or task.

2 Background: Transfer learning

The paper frames transfer learning through source and target domains and tasks, focusing on domain adaptation when domains differ but tasks remain the same. It motivates many-to-one adaptation because dissimilar source-target pairs can produce negative transfer.

  • Definitions: A domain consists of a feature space X and a marginal distribution P(X), while a task consists of labels Y and a conditional distribution P(Y|X).Tasks are learned from labeled training pairs {x_i, y_i}.
  • Transfer learning: Transfer learning uses information from source domain-task pairs to learn the target conditional distribution when the source and target domains or tasks differ.The paper focuses on domain adaptation, where domains differ but the task is shared.
  • Adaptation settings: Traditional domain adaptation evaluates one-to-one adaptation across every source-target domain pair.The source and target sets are evaluated over their Cartesian product.
  • Adaptation settings: Many-to-one adaptation addresses multiple source domains with performance on the target domain as the primary objective.The paper presents this as a more realistic multiple-source setting.
  • Motivation: Adapting between dissimilar domains can lead to negative transfer, making source selection important.This motivates choosing useful sources rather than treating every source-target pairing equally.

3 Data selection model

The paper learns a task- and target-aware linear data-selection measure with Bayesian Optimization, combining similarity and diversity features to choose source training instances for transfer learning.

  • Motivation: Existing selection methods rank source examples by pre-existing domain similarity, which cannot adapt to the task and target-domain characteristics.They also select only by similarity, overlooking intrinsic qualities such as data diversity.
  • Optimization: Bayesian Optimization selects weights by evaluating task performance after scoring, sorting, and training on the highest-ranked source examples.The objective is measured on a small validation set from the corresponding target domain.
  • Features: Similarity features compare source examples with target-domain representations using Jensen-Shannon, Rényi, Bhattacharyya, cosine, Euclidean, and variational measures.Representations include term distributions, topic distributions, and word embeddings; KL divergence is excluded because zero-probability events can make it undefined.
  • Features: Diversity features capture intrinsic variation through word-type count, type-token ratio, entropy, Simpson’s index, and quadratic entropy.The paper treats diversity as complementary information to similarity for selecting useful transfer-learning data.

4 Experiments

The experiments evaluate learned data-selection measures across sentiment analysis, POS tagging, and parsing, using multiple domains, models, representations, and baselines.

  • Tasks, datasets, and models: The evaluation covers sentiment analysis, POS tagging, and dependency parsing, selecting the n highest-scoring examples for model training.Each domain is treated as a target while the remaining domains provide source data.
  • Tasks, datasets, and models: Sentiment analysis uses Amazon reviews with tf-idf unigram and bigram features, a linear SVM, a 10,000-word vocabulary, and n = 1600 training examples.The study conducts 10 runs for each feature set and reports mean and variance.
  • Tasks, datasets, and models: POS tagging and parsing use SANCL 2012 domains, with approximately 2,000–5,000 labeled sentences and more than 100,000 unlabeled sentences per domain except WSJ.Both tasks use n = 2000 selected examples.
  • Tasks, datasets, and models: Model choice is tested with a Structured Perceptron and a state-of-the-art Bi-LSTM tagger, while parsing uses a state-of-the-art Bi-LSTM parser.The POS and parsing experiments use one run because observed variance was low.
  • Training details: Table 1 reports labeled and unlabeled sentence counts for Amazon Reviews and SANCL 2012, while Table 2 reports sentiment-analysis accuracy scores.The experiments optimize accuracy for sentiment analysis and POS tagging, and LAS for parsing, using 100 target-domain validation examples and 300 optimization iterations.
  • Training details: The method is compared with random selection, JS-based example selection, JS-based most-similar-domain selection, and training on all available data.Bayesian Optimization is applied to embedding-, term-, topic-, diversity-, and combined feature sets.

5 Results

Across sentiment analysis, POS tagging, and parsing, Bayesian Optimization learns data-selection measures that generally outperform baselines, while results show complementary roles for similarity and diversity and some transfer across models, domains, and tasks.

  • Sentiment analysis: Bayesian Optimization outperforms baselines with almost all feature sets for sentiment analysis, with considerable gains across domains except smaller improvements for Books.Term-distribution and topic-distribution representations perform best for similarity, while combining them adds only marginal gains.
  • Sentiment analysis: Diversity performs comparably to the best similarity features, outperforms them on two domains, and combined similarity-diversity features add around 1 percentage point on almost all domains.The combination indicates that similarity and diversity capture complementary information.
  • Sentiment analysis: Training on all 6,000 available source instances is detrimental when domains are dissimilar, complementing the most-similar-domain baseline.The result supports selective transfer rather than indiscriminate use of all source data in dissimilar-domain settings.
  • POS tagging: For POS tagging, Bayesian Optimization beats baselines with almost all feature sets, although a few diversity, word-embedding, topic-distribution, and term-distribution cases do not.Term-distribution similarity is the strongest individual feature; adding diversity is less beneficial than in sentiment analysis but often gives the second-best results.
  • Cross-task comparison: The most-similar-domain baseline is ineffective or harmful for POS tagging and parsing, whereas selecting individual instances by JS similarity performs better.The paper relates this contrast to the greater benefit of varied training instances for POS tagging and parsing.
  • Parsing: For parsing, diversity features are stronger than for POS tagging, while term-distribution similarity and similarity-diversity combinations perform best across most domains.These features outperform baselines for every domain except Reviews.
  • Transfer across models and domains: Learned measures transfer across models and domains, with transferred weights often robust and sometimes outperforming metrics learned directly for the destination domain.Similarity or diversity weights trained on Books beat the strong JS-D baseline in all six cases, while Electronics and Kitchen do so in four of six cases.

6 Related work

Prior data-selection research largely developed task-specific similarity measures, especially for machine translation and selected NLP tasks. This work is distinguished by learning transfer-selection measures with Bayesian Optimization rather than merely ordering data or relying on similarity alone.

  • Most prior data-selection work targets phrase-based machine translation, using language-model perplexity or cross-entropy to select target data.
  • Domain-similarity research proposed different metrics for different tasks, including proxy A distance, Rényi divergence, and Jensen-Shannon divergence.
  • Curriculum learning selects or orders training examples to alter learning progression, whereas this work learns metrics for selecting transfer-learning data.
  • Adaptive data selection also relates to active learning, but active learning seeks difficult instances for labeling, while this model-agnostic approach selects transfer data.
  • Bayesian Optimization is the closest precedent, but it previously learned curricula for word embeddings rather than selecting relevant instances for transfer learning.

7 Conclusion

The paper proposes Bayesian Optimization for learning data-selection measures in transfer learning and evaluates their performance and transferability. The learned measures outperform existing similarity metrics across three tasks and show promising transfer across models, domains, and related tasks.

  • Bayesian Optimization learns data-selection measures that outperform existing domain-similarity metrics on sentiment analysis, POS tagging, and parsing.
  • The learned measures are competitive with a state-of-the-art domain-adaptation approach and can be transferred across models, domains, and related tasks.
Loading 1707.05246v1…