PaperlayerView exact revision on arXiv

Source-linked AI summary

Generating Training Data with Language Models: Towards Zero-Shot Language Understanding

Yu Meng, Jiaxin Huang, Yu Zhang, Jiawei Han

arXiv:2202.04538v2cs.CLcs.LG

TL;DR

Zero-shot NLU remains difficult because prompt-based methods perform poorly without task-specific data, while transfer methods rely on cross-task annotations. SuperGen generates class-conditioned training data with a unidirectional PLM and fine-tunes a bidirectional PLM, achieving strong seven-task GLUE results that outperform zero-shot prompting and compare favorably with 32-shot methods. Its data selection and regularization strategies improve training quality and stability, although hyperparameter tuning and generated-data quality remain limitations.

  • Problem

    Zero-shot NLU is difficult because prompt-based PLMs perform poorly without task-specific samples, while existing transfer approaches rely on cross-task annotations.

  • Method

    SuperGen uses label-descriptive prompts with a unidirectional PLM to generate class-conditioned texts, then fine-tunes a bidirectional PLM on them.

  • Results

    Across seven GLUE classification tasks, SuperGen significantly outperforms zero-shot prompting and achieves better overall performance and stability than few-shot methods using 32 samples per class.

  • Takeaways & Limitations

    Quality-data selection, label smoothing, and temporal ensembling support effective and stable fine-tuning on generated data.

  • Takeaways & Limitations

    Zero-shot learning limits hyperparameter tuning, and generated data may be inadequate for challenging tasks or distributions far from pretraining data.

Abstract

from arXiv · show

Pretrained language models (PLMs) have demonstrated remarkable performance in various natural language processing tasks: Unidirectional PLMs (e.g., GPT) are well known for their superior text generation capabilities; bidirectional PLMs (e.g., BERT) have been the prominent choice for natural language understanding (NLU) tasks. While both types of models have achieved promising few-shot learning performance, their potential for zero-shot learning has been underexplored. In this paper, we present a simple approach that uses both types of PLMs for fully zero-shot learning of NLU tasks without requiring any task-specific data: A unidirectional PLM generates class-conditioned texts guided by prompts, which are used as the training data for fine-tuning a bidirectional PLM. With quality training data selected based on the generation probability and regularization techniques (label smoothing and temporal ensembling) applied to the fine-tuning stage for better generalization and stability, our approach demonstrates strong performance across seven classification tasks of the GLUE benchmark (e.g., 72.3/73.8 on MNLI-m/mm and 92.8 on SST-2), significantly outperforming zero-shot prompting methods and achieving even comparable results to strong few-shot approaches using 32 training samples per class.

1 Introduction

SuperGen addresses weak zero-shot NLU performance by generating class-conditioned training data with a unidirectional PLM and fine-tuning a bidirectional PLM. Across seven GLUE classification tasks, it outperforms zero-shot prompting and compares favorably with 32-shot methods.

  • Motivation: Zero-shot prompting performs much worse than few-shot prompting, especially on challenging natural language inference tasks.PLMs struggle to interpret task prompts in formats unseen during pretraining without task-specific examples.
  • Approach: SuperGen generates class-conditioned texts from label-descriptive prompts and uses them to fine-tune a bidirectional classifier PLM.The approach requires neither target-task nor cross-task human annotations.
  • Results: SuperGen significantly outperforms prompt-based zero-shot learning across seven GLUE classification tasks.It also achieves better overall average performance and stability than strong few-shot methods using 32 annotated samples per class.
  • Results: Quality-data selection, label smoothing, and temporal ensembling are identified as key factors behind SuperGen’s performance.These strategies regularize fine-tuning on automatically generated data.

2 Related Work

Prior zero-shot approaches commonly rely on prompt-based prediction or transfer learning from cross-task annotations. SuperGen instead generates target-task training data with prompted language models, extending controlled generation to diverse NLU tasks despite potential noise and domain mismatch.

  • Few-Shot and Zero-Shot Learning: Few-shot prompting leverages small amounts of task-specific data, whereas strict zero-shot learning provides no task-specific data.Prompt-based methods are widely used to inject task descriptions into pretrained language models.
  • Few-Shot and Zero-Shot Learning: Mainstream zero-shot transfer methods fine-tune models on annotated tasks reformulated as instructions, entailment pairs, or question-answer formats.These methods seek cross-task transfer to unseen tasks with similar formulations.
  • Controlled Text Generation with PLMs: Controlled text generation steers pretrained language models toward desired contents, styles, domains, or attributes through training or inference-time methods.Prompts can guide generation by providing natural-language contexts or learnable parameters.
  • Controlled Text Generation with PLMs: Earlier category-conditioned data generation methods may lack sufficient capacity for challenging NLU tasks.SuperGen applies prompted generation without labeled samples to a broad range of NLU tasks.
  • Controlled Text Generation with PLMs: Generated texts can contain label noise and differ from downstream domains, motivating strategies for effective fine-tuning.These distributional differences may reduce generalization when generated data are used directly.

3 Method

SuperGen creates class-conditioned training data with a unidirectional generator and fine-tunes a bidirectional classifier, using selection and regularization to improve robustness.

  • Training-data generation: SuperGen maps each label to a descriptive prompt, generates class-conditioned texts with a unidirectional PLM, and fine-tunes a bidirectional PLM classifier.The classifier is assumed bidirectional because such PLMs generally perform better on NLU tasks.
  • Training-data generation: Single-sequence tasks begin generation with the label prompt, while sequence-pair tasks sample a first sequence and generate a related second sequence.Generated samples are paired with labels for classifier training; sampling the first sequence from the pretraining corpus improves diversity.
  • Training-data generation: For syntactic acceptability, lower temperature favors likely acceptable sequences and higher temperature produces less constrained sequences for contrasting examples.CoLA generation varies sampling temperatures rather than using prompts.
  • Training-data generation: Repetition controls promote novel tokens from the first sequence and discourage repeated tokens in the generated sequence to mitigate degenerate repetition.The method uses α < 1 and β > 1 for these effects.
  • Effective fine-tuning: Generated texts are ranked by average log probability, with selected examples forming the training set; label smoothing and temporal ensembling address noise and domain differences.For most tasks, the top-N examples per class are selected, while CoLA uses top-N and bottom-N examples for acceptable and unacceptable sequences.
  • Effective fine-tuning: Temporal ensembling filters samples when the ensemble prediction strongly agrees with the label, while label smoothing reduces overconfidence and improves robustness to label noise.Only samples with ensembled label probability above δ are retained for training.

4 Experimental Setup

The experiments evaluate SuperGen across seven GLUE classification tasks using fixed evaluation protocols, specified generator/classifier models, and comparisons with zero-shot and few-shot methods.

  • Downstream Tasks and Metrics: Seven GLUE classification tasks are evaluated, excluding STS-B because it is a regression task.The original development sets are used for testing, with averages and standard deviations reported over five random seeds.
  • Downstream Tasks and Metrics: F1 measures QQP and MRPC, Matthews correlation measures CoLA, and accuracy measures the remaining tasks.
  • Models: CTRL with 1.63B parameters serves as the generator, while COCO-LMLarge with 367M parameters serves as the classifier.The study also reports experiments using GPT-2 and RoBERTa as similarly sized generator/classifier models.
  • Fine-Tuning Settings and Hyperparameters: SuperGen uses prompt-based fine-tuning with manual templates and keeps fine-tuning hyperparameters identical across tasks because zero-shot learning provides no validation sets.The shared settings include learning rate, batch size, training epochs, generated-sample count, label smoothing, and temporal-ensembling hyperparameters.
  • Compared Methods and Ablations: The comparisons include zero-shot prompting, standard few-shot fine-tuning, four few-shot prompt-based methods, and ablations removing data selection, label smoothing, or temporal ensembling.

5 Evaluation

SuperGen evaluation shows that generated-data selection, regularization, prompt choices, data quantity, model choices, and task difficulty affect classifier performance and stability. Generated data can improve few-shot learning, while label noise and generator quality remain important constraints.

  • 5.1 Main Results: SuperGen significantly outperforms zero-shot prompting and achieves better overall results than few-shot methods, with smaller variance across most tasks.The ablations identify quality-data selection, label smoothing, and temporal ensembling as important for improving and stabilizing performance.
  • 5.2 Using Different Prompts: Different synonymous prompt groups produce generally similar performance, except RTE shows higher variance because its test set is very small.The authors manually select simple prompts and leave automatic optimal-prompt search for future work.
  • 5.3 Results with Different Amount of Generated Data: More generated data generally improves average performance and stability, but using 10,000 samples slightly reduces performance as lower-quality, noisier texts enter the training set.With only 100 samples, fine-tuning variance is high; increasing data improves stability, while fixed-ratio selection would cost more computation.
  • 5.4 Using SuperGen in Few-Shot Settings: Adding SuperGen data after few-shot fine-tuning notably improves MNLI performance when training and validation sets contain 32–64 samples per label in total.The few-shot classifier helps filter noisy generated samples through temporal ensembling.
  • 5.6 Using Different PLMs: CTRL consistently generates better training data than GPT-2, while smaller GPT-2 models produce significantly less coherent text.The comparison uses GPT-2XLarge as generator and RoBERTaLarge as classifier with other settings unchanged.
  • 5.7 Case Studies: Generated texts are usually coherent and label-consistent for SST-2, but MNLI examples sometimes contain incorrect labels, motivating fine-tuning regularization.The paper presents examples for SST-2, MNLI, and QQP and identifies improved label control as future work.

6 Discussions and Conclusions

The paper concludes that SuperGen generates supervision for zero-shot NLU by combining prompted generation with classifier fine-tuning, while acknowledging data-quality and tuning limitations. It also identifies regularization and data selection as important for stable performance and discusses risks from generated content.

  • Ethical Considerations: Generated PLM data can pose misinformation and harmful-bias risks, so SuperGen can be combined with bias-reduction and correction techniques.
  • Limitations: Zero-shot learning limits hyperparameter tuning because task-specific samples are unavailable, and generated data may be inadequate for challenging tasks with distribution shifts.
  • Conclusions: SuperGen automatically creates training data with a prompted unidirectional PLM and uses it to fine-tune a bidirectional PLM for zero-shot NLU.
  • Conclusions: SuperGen does not rely on cross-task annotations and reduces the mismatch between training and inference tasks.
  • Conclusions: Quality-data selection, label smoothing, and temporal ensembling are identified as important for effective and stable fine-tuning on generated data.

Checklist

The checklist records that the paper addressed its main claims, limitations, societal impacts, reproducibility materials, training details, error bars, compute, and asset licensing. Items concerning theoretical results, human subjects, and new assets were marked not applicable where indicated.

  • Author checklist: The paper reports that its main claims reflect its contributions and scope, and that limitations and potential negative societal impacts were discussed.
  • Theoretical results: Theoretical-result assumptions and proofs were marked not applicable because the paper does not present theoretical results.
  • Reproducibility: The authors report providing code, data, reproduction instructions, training details, and hyperparameter-selection information for the experiments.
  • Experimental reporting: The experiments reportedly include error bars, compute totals, and resource information.
  • Assets and human subjects: Existing assets were cited and licensed, while new assets and human-subject procedures were marked not applicable where specified.

A Details of Prompts Used for Different Tasks

The appendix details prompt construction for GLUE tasks, including task-specific sequence constraints, alternative prompt groups, and generation settings. Prompting differs between single-sequence and sequence-pair classification tasks.

  • Prompt design: Table 7 organizes prompts for GLUE tasks, distinguishing single-sequence SST-2 and CoLA from sequence-pair classification tasks.
  • Generation settings: Table 9 specifies sampling temperature, repetition parameters, and the number of generated texts per label, with top-k sampling using k = 10 when temperature is positive.
  • Prompt design: The appendix provides expanded prompt details and separate prompt groups for the experiments.
  • Task-specific prompts: SST-2 generation begins with movie-review prefixes, while CoLA begins with a random stop word.
  • Task-specific prompts: QNLI and QQP generation constrains the first sequence to questions and requires sampled sequences to use question words and end with question marks.
  • Task-specific prompts: For QNLI, the entailment-generated sequence immediately follows the sampled sequence, while prompt groups vary one label prompt at a time.

B Hyperparameters and Reproducibility

The appendix documents generation and fine-tuning hyperparameters, data-selection rules, regularization schedules, baselines, and hardware. These details target reproducibility across tasks while allowing task-specific handling for CoLA and selected labels.

  • Generation hyperparameters: Sequence-pair generation uses greedy sampling for reproducibility, while α controls overlap or repetition according to the target relation.
  • Data selection: Training data are generated beyond the target set and selected using score r, with task-specific top-, bottom-, or random-selection rules.
  • Fine-tuning hyperparameters: Fine-tuning hyperparameters are shared across tasks except CoLA, where δ = 0 because intentionally low-quality examples are retained.
  • Fine-tuning hyperparameters: The fine-tuning schedule ramps λ during the first 10 prediction ensembles using λ(t) = λmax exp(−5(1 −t/10)2).
  • Reported settings: The appendix includes fine-tuning settings and MNLI prompts for CTRL zero-shot prompting and knowledge-distillation baselines.
  • Computation environment: Experiments use NVIDIA GeForce RTX 3090 GPUs, and SuperGen is reported to run on typical research hardware with more than 10GB of GPU memory.

C GLUE Tasks

The GLUE tasks cover sentence acceptability, sentiment, entailment, question answering, paraphrase detection, and question equivalence across seven classification benchmarks. The SST-2 appendix also documents prompts used for a zero-shot and knowledge-distillation baseline.

  • Natural language inference: GLUE includes MNLI, which classifies whether a premise entails, contradicts, or is neutral toward a hypothesis on matched and mismatched test sets.MNLI-m uses sources resembling training data, whereas MNLI-mm uses samples that do not resemble the training data.
  • Semantic equivalence: QQP and MRPC classify whether pairs of questions or sentences are semantically equivalent.QQP concerns Quora question pairs, while MRPC concerns sentence pairs.
  • Question answering: QNLI predicts whether a sentence contains the answer to a question sentence.
  • Sentiment: SST-2 determines whether a movie review expresses positive or negative sentiment.
  • Linguistic acceptability: CoLA determines whether a sentence is linguistically acceptable.
  • Natural language inference: RTE predicts whether a premise entails a hypothesis.
  • Baseline prompts: The SST-2 baseline table lists prompts for CTRL zero-shot prompting and knowledge distillation, with x denoting the input sequence.

D Knowledge Distillation Baseline Details

The knowledge-distillation baseline selects the best of three prompts separately for MNLI and SST-2, then trains a classifier to reproduce CTRL’s soft labels.

  • Prompt selection: The baseline uses prompt #1 for both MNLI and SST-2 because it had the highest zero-shot test-set prediction accuracy among three prompts.
  • Knowledge distillation: CTRL generates soft labels over the entire training set, and the classifier is trained with Kullback–Leibler divergence to approximate them.

E Negative Generation Results

Negative-generation examples show that generated texts can fail to match the label specified by their prompts. These errors motivate regularization and suggest that improved generators or filtering could improve label quality.

  • Observed label noise: Generated MNLI texts sometimes contradict their prompted labels, including entailment texts that are neutral and contradiction texts that are neutral.
  • Methodological response: Label noise in generated training texts motivates regularization during fine-tuning.
  • Future improvements: Larger generator PLMs may improve generation quality, while better filtering strategies may select texts with correct labels.
Loading 2202.04538v2…