Source-linked AI summary
Semi-Supervised QA with Generative Domain-Adaptive Nets
Zhilin Yang, Junjie Hu, Ruslan Salakhutdinov, William W. Cohen
TL;DR
Semi-supervised question answering seeks to improve QA models with abundant unlabeled text when labeled data are limited. The paper proposes GDANs, which generate questions from unlabeled answer chunks and adapt generated data to human-generated data using reinforcement learning. On SQuAD, GDANs substantially improve over supervised learning and several baselines, including a 9.87-point F1 gain with 8K labeled pairs.
Problem
Semi-supervised question answering asks whether unlabeled text can improve QA models when only a small amount of labeled data is available.
Method
GDANs generate questions from answer chunks and contexts in unlabeled text, combine them with human-generated pairs, and use domain adaptation with reinforcement learning to reduce distribution discrepancy.
Results
9.87 points in F1 improve over supervised learning when 8K labeled question-answer pairs are used, while GDANs also consistently improve over supervised learning and baseline methods.
Takeaways & Limitations
With 0.1 training instances, the semi-supervised approach achieves better performance than supervised learning with 0.2 training instances, saving more than half of labeling costs.
Takeaways & Limitations
The generative objective can produce degenerate questions, such as copying answers, because questions may overrepresent the answers.
Abstract
from arXiv · showhide
We study the problem of semi-supervised question answering----utilizing unlabeled text to boost the performance of question answering models. We propose a novel training framework, the Generative Domain-Adaptive Nets. In this framework, we train a generative model to generate questions based on the unlabeled text, and combine model-generated questions with human-generated questions for training question answering models. We develop novel domain adaptation algorithms, based on reinforcement learning, to alleviate the discrepancy between the model-generated data distribution and the human-generated data distribution. Experiments show that our proposed framework obtains substantial improvement from unlabeled text.
1 Introduction
The paper addresses semi-supervised question answering by using abundant unlabeled text alongside limited labeled data. It proposes GDANs, combines generated and human-generated question-answer pairs, adapts between their distributions, and reports substantial improvements.
- Large labeled datasets are expensive and time-consuming to collect, limiting domain-specific question answering.
- Semi-supervised question answering asks whether unlabeled text can improve QA models when only limited labeled data are available.
- GDANs extract answer chunks from unlabeled text, generate questions from answers and contexts, and combine generated pairs with human-generated pairs.
- Domain tags help the discriminative model separate domain-specific from domain-invariant representations, while reinforcement learning fine-tunes the generative model against discriminative loss.
- A simpler baseline is easier to implement than GDANs and still provides substantial improvement when labeled data are limited.
- 9.87 points in F1 separate GDANs from supervised learning when 8K labeled question-answer pairs are used.
2 Semi-Supervised Question Answering
The semi-supervised QA setting combines labeled question-answer data with unlabeled paragraphs containing extracted answer chunks. It trains a discriminative QA model using both sources, including a context-based baseline that treats text surrounding an answer as a question-like hint.
- Extractive QA predicts an answer span from a paragraph given a question, with the answer represented as consecutive paragraph tokens.
- The semi-supervised setting adds unlabeled instances consisting of paragraphs and available answer chunks to the labeled dataset.
- The learning goal is a discriminative model D that captures P(a|p, q) using both labeled and unlabeled data.
- A Simple Baseline: The baseline extracts a window of five tokens on each side of an answer and combines these context-based pairs with human-generated pairs.
- A Simple Baseline: This context-based baseline serves as a hint for the QA model and yields substantial improvements when labeled data are limited.
3 Generative Domain-Adaptive Nets
Generative Domain-Adaptive Nets combine a discriminative QA model with a question-generating model trained on unlabeled text. Domain tags and reinforcement learning adapt generated questions toward human-generated data while optimizing answer prediction.
- Model framework: GDANs consist of a discriminative model D for P(a|p,q) and a generative model G for P(q|p,a).D predicts answer chunks, while G generates questions from paragraphs and answers.
- Domain adaptation: Domain tags distinguish human-generated data from model-generated data so D can account for domain differences while learning answer prediction.The tags “d true” and “d gen” are appended to questions and paragraphs.
- Generative model: The generative model uses a GRU encoder-decoder with attention, answer-indicator features, and a copy mechanism.The copy mechanism mixes vocabulary generation with copying tokens from the paragraph.
- Training algorithm: GDAN training alternates updates to D on labeled and generated data with updates to G using Reinforce because generated questions are discrete and non-differentiable.D maximizes objectives on labeled data tagged d true and generated data tagged d gen; G is updated using a reward based on D’s answer-prediction objective.
- Training objective: Reinforcement learning steers G toward useful questions that D can answer correctly, avoiding trivial questions such as copying the answer.Unlike MLE, RL does not require labels and optimizes log P_D(a|q,p) rather than only log P(q|p,a).
4 Experiments
Experiments evaluate answer extraction, generated-question quality, semi-supervised performance, baselines, ablations, and unlabeled-data scale. GDAN consistently benefits from unlabeled data, especially when labeled data is limited.
- 4.1 Answer Extraction: The framework extracts answer chunks from unlabeled Wikipedia paragraphs using linguistic tags, POS tagging, constituency parsing, and answer-type sampling.The extracted answers are used to generate additional question-answer pairs.
- 4 Experiments: The evaluation uses SQuAD with article-level train, development, and test splits, reporting F1 and exact matching scores.Ten percent of the training set is held out for testing, while development-set F1 guides tuning and early stopping.
- 4.3 Results and Analysis: 9.87 F1 points and 7.26 EM points are gained over supervised learning at labeling rate 0.1.Using 0.1 of the training instances achieves better performance than supervised learning with 0.2, saving more than half of labeling costs.
- 4.3 Results and Analysis: At labeling rate 0.1, GDANs outperform dual learning and GAN-based adaptation by 2.47 and 4.29 F1 points, respectively.The comparison uses the proposed adversarial domain-adaptation approach against Gen + dual and Gen + GAN.
- 4.3 Results and Analysis: Domain tags and adversarial training both contribute when labeling rates are at most 0.5, whereas adversarial training adds little at rate 0.9.Domain tags still improve performance at the highest labeling rate.
- 4.3 Results and Analysis: Increasing unlabeled data from 50K to 5M raises GDAN performance by 0.38 F1 points and 0.52 EM points.The gain is smaller than that from changing model architectures.
5 Related Work
Related work covers semi-supervised learning, domain adaptation, question answering, and multi-model learning. The paper distinguishes GDANs by applying domain adaptation to generative-model outputs.
- Semi-Supervised Learning: Earlier semi-supervised methods combine supervised and unsupervised objectives, while newer approaches use generative models, ladder networks, or graph embeddings.The cited literature frames semi-supervised learning as a broader representation-learning problem.
- Domain Adaptation: Domain-adaptation research learns invariant representations, matches domain distributions, or models transitions between source and target domains.GDANs draw on domain tags but apply adaptation to generative-model outputs rather than a natural data domain.
- Question Answering: Neural question-answering models achieve strong performance but largely depend on substantial labeled training data.This dependence motivates the paper’s semi-supervised QA setting.
- Learning with Multiple Models: Prior multi-model approaches include GANs for realistic generation and domain adaptation, review networks as generative regularizers, and related machine-translation methods.These methods provide precedents for combining generative and discriminative models.
6 Conclusions
The paper studies semi-supervised question answering and proposes Generative Domain-Adaptive Nets that combine generative modeling with domain adaptation. Experiments report substantial improvements over supervised learning and several baselines.
- 6 Conclusions: The paper identifies semi-supervised question answering as a critical and challenging problem.The setting uses unlabeled text alongside limited labeled data.
- 6 Conclusions: Generative Domain-Adaptive Nets combine domain-adaptation techniques with generative models for semi-supervised learning.The framework is presented as the paper’s central neural approach.
- 6 Conclusions: The approach substantially improves over supervised learning models and outperforms GAN and dual-learning baselines.The conclusion reports this empirically without specifying a single aggregate metric.
- 6 Conclusions: The authors plan to evaluate GDANs on more question-answering datasets and different domains, and to generalize them to other applications.These directions define the stated future scope of the work.