PaperlayerView exact revision on arXiv

Source-linked AI summary

Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners

Ningyu Zhang, Luoqiu Li, Xiang Chen, Shumin Deng, Zhen Bi, Chuanqi Tan, Fei Huang, Huajun Chen

arXiv:2108.13161v7cs.CLcs.AIcs.CVcs.IRcs.LG

TL;DR

Few-shot learning with pre-trained language models is constrained by reliance on large model scale and prompt design, complicating practical deployment. DART optimizes prompt templates and labels through backpropagation, improving few-shot performance across 15 NLP datasets, including up to 23.28% average absolute gains over conventional fine-tuning at K = 8 for relation extraction.

  • Problem

    Few-shot learning seeks effective NLP adaptation with limited labeled data, while strong performance often depends on large models and prompt design that challenge real-world deployment.

  • Method

    DART uses unused language-model tokens as continuously optimized prompt templates and labels, jointly training them with backpropagation in a model-agnostic framework.

  • Results

    Across 15 NLP datasets, DART improves few-shot performance across all tasks, with up to 23.28% absolute improvement over conventional fine-tuning on average at K = 8 for relation extraction.

  • Takeaways & Limitations

    DART supports few-shot real-world classification without the high cost of collecting and annotating large amounts of data.

  • Takeaways & Limitations

    DART may fail when the task-corpus distribution differs from the pre-training corpus.

Abstract

from arXiv · show

Large-scale pre-trained language models have contributed significantly to natural language processing by demonstrating remarkable abilities as few-shot learners. However, their effectiveness depends mainly on scaling the model parameters and prompt design, hindering their implementation in most real-world applications. This study proposes a novel pluggable, extensible, and efficient approach named DifferentiAble pRompT (DART), which can convert small language models into better few-shot learners without any prompt engineering. The main principle behind this approach involves reformulating potential natural language processing tasks into the task of a pre-trained language model and differentially optimizing the prompt template as well as the target label with backpropagation. Furthermore, the proposed approach can be: (i) Plugged to any pre-trained language models; (ii) Extended to widespread classification tasks. A comprehensive evaluation of standard NLP tasks demonstrates that the proposed approach achieves a better few-shot performance. Code is available in https://github.com/zjunlp/DART.

1 INTRODUCTION

The introduction identifies the deployment and prompt-engineering limitations of large pre-trained language models and proposes DART, a model-agnostic, parameter-efficient method that optimizes prompt templates and label tokens through backpropagation. Across 15 NLP datasets, DART improves few-shot performance, reaching up to 23.28% absolute improvement over conventional fine-tuning and 90% of SOTA performance with eight labeled samples per class.

  • Motivation: Large few-shot models remain difficult to deploy: GPT-3 reaches 80% of SOTA results but uses 175B parameters.GPT-3 uses natural-language prompts with 16 labeled demonstrations.
  • Motivation: Manual prompt design requires domain expertise and large validation sets, while prompt formats can be sub-optimal and cause accuracy variation.Prompting can compensate for hundreds of data points on average across classification tasks, but constructing effective prompts remains costly.
  • DART: DART uses unused language-model tokens as template and label tokens, then optimizes them in continuous space with backpropagation.The approach is designed to be model-agnostic, parameter-efficient, pluggable, extensible, and efficient without introducing additional parameters apart from the model.
  • Results: 23.28% absolute improvement over conventional fine-tuning is achieved on average at K = 8 for relation extraction datasets with complex label semantics.The reported improvement is 1.55% for fully supervised settings.
  • Results: 90% performance of the SOTA results is achieved with only 8 labeled samples per class across a systematic evaluation of 15 NLP tasks.The evaluation reports improvements across all 15 tasks.

2 RELATED WORK

Related work has explored prompting and few-shot learning with pre-trained language models, but often relies on handcrafted prompts or external parameter optimization. DART instead targets reduced prompt engineering and noninvasive, broadly applicable adaptation.

  • Language Model Prompting: GPT-3 established language-model prompting as a route to strong few-shot performance, but relies mainly on handcrafted prompts and is not designed for fine-tuning.This motivated subsequent prompting studies.
  • Language Model Prompting: Continuous-prompt methods learn trainable prompt representations while keeping language-model parameters fixed or optimizing task-specific vectors.Examples include P-tuning, OPTIPROMPT, and prefix-tuning.
  • Language Model Prompting: DART aims to reduce prompt engineering for templates and labels as well as external parameter optimization through noninvasive model modification.The approach is designed to plug into any pre-trained language model and extend to widespread classification tasks.
  • Few-shot Learning: Few-shot learning improves machine intelligence and practical adaptive applications by using only a small number of labeled examples.The paper positions DART among few-shot NLP approaches including meta-learning, which optimizes quantities of auxiliary tasks.

3 BACKGROUND

The background contrasts conventional fine-tuning, which predicts classes from [CLS] embeddings, with prompt-based methods that use task-specific templates and label tokens to elicit textual class outputs from masked-language models.

  • Input and prompt formulation: An input sentence Xin = {x1,x2,...,xL} is converted into a fixed token sequence and mapped to hidden vectors.L denotes the number of tokens, and xi is the ith input token.
  • Input and prompt formulation: Conventional fine-tuning applies a generic head layer, such as an MLP, over the [CLS] embedding to predict an output class.The input sequence is represented as ˜Xin = [CLS]Xin[SEP].
  • Input and prompt formulation: Prompt-based methods design a task-specific pattern string T and a label token M(Y) to coax the model into producing a textual output for a class.Together, the template and label token constitute a prompt.
  • Input and prompt formulation: The prompt contains one [MASK] token and is directly provided as input to the masked-language model.The model then obtains a probability distribution over candidate classes and their label tokens.

4 OUR APPROACH

DART replaces discrete prompt and label searches with continuously optimized pseudo-token templates and labels, reducing reliance on prompt engineering. It jointly trains class discrimination and fluency objectives while reusing the pre-trained model’s transformer architecture.

  • Differentiable Templates: DART addresses the limitations of discrete prompts by representing template positions as trainable pseudo-token embeddings optimized with backpropagation.This enables expressive templates beyond the original vocabulary V.
  • Differentiable Templates: DART jointly optimizes prompt embeddings because their values must be co-dependent, using an auxiliary fluency constraint instead of an additional bidirectional LSTM.The fluency constraint associates prompt embeddings and encourages contextual representation learning.
  • Differentiable Labels: DART maps class labels into a continuous vocabulary space, avoiding computationally intensive brute-force searches that scale exponentially with the number of classes.Earlier approaches search over candidate vocabulary words, while DART directly learns label representations.
  • Training Objectives: The method optimizes two objectives: a class discrimination loss for sentence classification and a fluency constraint loss based on masked language modeling.The fluency objective masks an input token and maximizes its masked-token prediction probability.
  • Optimization Procedure: DART jointly optimizes templates and labels to mitigate few-shot fine-tuning instability while reusing the same transformer architecture without an additional LSTM.The approach uses the golden label in the fluency constraint rather than the [MASK] token.

5 EXPERIMENTS

Across 15 NLP tasks, DART improves few-shot classification over conventional fine-tuning, matches or exceeds LM-BFF, and remains effective across model architectures. Ablations and representation analyses attribute these gains to jointly optimized differentiable templates and labels.

  • Evaluation setup: The evaluation spans 15 NLP tasks, including sentiment analysis, natural language inference, paraphrase detection, sentence similarity, relation extraction, and event extraction.Ten sentence-classification datasets are included, alongside complex-label-space extraction tasks.
  • Standard classification: DART outperforms conventional fine-tuning and achieves performance comparable to LM-BFF on standard classification tasks.On MR, DART improves performance by 11.3% with only 16 training samples per class, comparable to LM-BFF without external T5-generated templates.
  • Complex-label-space classification: DART outperforms fine-tuning and LM-BFF by a large margin on relation and event extraction in both few-shot and fully supervised settings.It improves TACRED-Revisit by 2.8% with full supervision and by 18.4% with only 8 training samples per class.
  • Ablation study: Removing the fluency constraint, differentiable template, or differentiable label causes performance decay, confirming that all modules contribute to DART.Differentiable label optimization is especially beneficial and sensitive in low-resource settings.
  • Model transferability: With GPT-2-medium, DART outperforms conventional fine-tuning and achieves performance on par with BERT-large.The GPT-2-medium experiment omits the fluency constraint because GPT-2 was not pretrained with a masked-language-modeling objective.
  • Representation analysis: Differentiable prompts produce more compact masked-token representations than fixed prompts and generate better semantic representations through optimized pseudo-labels.The analysis visualizes representations on CR and optimized label neighborhoods on Wiki80.

6 CONCLUSION AND FUTURE WORK

DART is presented as a simple, effective fine-tuning approach that improves few-shot learning in pretrained language models. The method provides satisfactory gains over conventional fine-tuning, supports other language models such as BART, and extends to intent detection and sentiment analysis.

  • Conclusion: DART is a simple yet effective fine-tuning approach for improving few-shot learning in pretrained language models.
  • Conclusion: DART produces satisfactory improvements in few-shot scenarios compared with conventional fine-tuning approaches.
  • Future Work: DART is pluggable for other language models, including BART, and can extend to intent detection and sentiment analysis.

A APPENDIX

The appendix documents reproducibility, training procedures, evaluation splits, and framework-specific implementation choices. DART jointly optimizes prompts and global parameters, while comparisons use matched RoBERTa-large settings and LM-BFF prompt initialization.

  • Training setup: Experiments use PyTorch on one Nvidia 3090 GPU, with AdamW, 10% linear warmup, subsequent linear decay, and gradient clipping.These settings support reproducible training across datasets.
  • Evaluation protocol: Following LM-BFF, performance averages results across five randomly sampled Dtrain and Ddev splits, with grid search per split for learning rate, weight decay, and batch size.Hyperparameter combinations are optimized separately on each split.
  • DART implementation: DART jointly optimizes prompts and global parameters, using base prompts as pseudo-token templates to accelerate convergence.The base prompts serve as templates during joint optimization.
  • Framework comparison: For fair comparison, DART and P-tuning both use RoBERTa-large and LM-BFF’s best discrete prompts with label words as base settings.P-tuning omits anchor tokens because of its limited search space.

A.1 HYPER-PARAMETER SEARCH SPACE OF OUR METHOD IN GRID SEARCH · SST-2, MR, CR, Subj, TREC, QNLI, MRPC, QQP

The method uses a grid-search hyper-parameter space whose optimal settings may vary across tasks and data splits. The search covers optimization, training duration, batching, sequence length, and gradient accumulation.

  • A.1 HYPER-PARAMETER SEARCH SPACE OF OUR METHOD IN GRID SEARCH: The optimal hyper-parameter set may vary across different tasks and data splits.
  • SST-2, MR, CR, Subj, TREC, QNLI, MRPC, QQP: Learning rates are searched over 1e-5, 5e-5, 1e-4, and 2e-4.
  • SST-2, MR, CR, Subj, TREC, QNLI, MRPC, QQP: The number of epochs is searched over 20 and 30.
  • SST-2, MR, CR, Subj, TREC, QNLI, MRPC, QQP: The maximum sequence length is fixed at 128.
  • SST-2, MR, CR, Subj, TREC, QNLI, MRPC, QQP: Gradient accumulation steps are searched over 1 and 2.

MNLI, SNLI

The MNLI and SNLI experiments use a task- and split-dependent hyper-parameter search covering optimization, training duration, batch configuration, sequence length, and gradient accumulation.

  • Search space: The optimal hyper-parameter set may vary across different tasks and data splits.The search is therefore not assumed to use one universally optimal configuration.
  • Search space: Learning rates searched were 1e-5, 5e-5, 1e-4, and 2e-4, with weight decay values from 0.0 to 0.10.The candidate weight decay values were 0.0, 0.01, 0.05, and 0.10.
  • Search space: Training searched 30 or 40 epochs with batch sizes of 4, 8, or 16.These candidates define the training-duration and minibatch portions of the search.
  • Search space: Maximum sequence length was fixed at 256, while gradient accumulation steps were 1 or 2.Sequence length was not searched, whereas accumulation had two candidate settings.

TACRED-Revisit, WiKi80, SemEval … A.8 BROADER IMPACT

DART uses task-specific prompt templates and label words, with experiments showing that template length has limited overall impact while differentiable labels improve representation separability and performance. The paper also reports full-data competitiveness, limitations under domain shift and few-shot instability, and broader practical relevance for low-data settings.

  • A.3 TEMPLATE LENGTH ANALYSIS: For SST-2, template length three yields the best performance, although template length has rather limited overall impact.Template length is defined as the number of tokens excluding the input sentence and <mask> token.
  • A.4 PERFORMANCE ON FULL TRAINING SET: DART obtains better or comparable results than standard fine-tuning and LM-BFF on GLUE tasks with full-sized training data.The authors interpret this as evidence that prompt-based tuning methods benefit less from full-sized data.
  • A.5 PERFORMANCE WITH CONSTRAINED LABEL TOKENS: DART learns more semantic label representations, while fixed-prompt performance relates to the cosine similarity of selected label tokens.Nearest-neighbor vocabulary embedding search projects optimized differential label tokens to readable natural tokens.
  • A.6 MORE EXPERIMENTS: The RD ratio measures average intra-class distance divided by average inter-class distance, with values near 0 indicating discriminative masked-token representations.Distances are Euclidean, and Hc[i] denotes the masked-token hidden state for sample i in class c.
  • A.6 MORE EXPERIMENTS: DART produces lower RD ratios than fixed labels, indicating better linear separability that leads to better performance under the masked-language-model decoder.The decoder is a linear transformation sharing weights with the model’s word embeddings.
  • A.7 LIMITATIONS; A.8 BROADER IMPACT: DART may fail under task-corpus distribution shift, while few-shot learning remains unstable; intermediate training is proposed as a possible remedy.The authors expect practical value in low-data settings such as low-resource languages or expert annotation.
Loading 2108.13161v7…