PaperlayerView exact revision on arXiv

Source-linked AI summary

Selective Annotation Makes Language Models Better Few-Shot Learners

Hongjin Su, Jungo Kasai, Chen Henry Wu, Weijia Shi, Tianlu Wang, Jiayi Xin, Rui Zhang, Mari Ostendorf, Luke Zettlemoyer, Noah A. Smith, Tao Yu

arXiv:2209.01975v1cs.CL

TL;DR

Large language models can learn tasks from a few demonstrations, but creating effective annotated datasets remains costly. This paper introduces selective annotation with prompt retrieval and vote-k, which selects diverse, representative examples and improves performance across 10 tasks, matching supervised finetuning with 10-100× less annotation cost.

  • Problem

    The paper studies how to reduce manual annotation costs while retaining high in-context learning performance for new natural language tasks.

  • Method

    The framework first selectively annotates a small pool from unlabeled data, then retrieves task examples at test time; vote-k uses a graph to select diverse and representative instances.

  • Results

    Across 10 diverse tasks, vote-k substantially improves in-context learning over random selection and achieves similar performance to supervised finetuning with 10-100× less annotation cost.

  • Takeaways & Limitations

    Selective annotation provides an annotation-efficient practice for applying in-context learning to new natural language tasks across model sizes and domain-shift settings.

  • Takeaways & Limitations

    Finetuning comparisons use different pretrained models rather than identical models, because finetuning large left-to-right language models is often computationally infeasible.

Abstract

from arXiv · show

Many recent approaches to natural language tasks are built on the remarkable abilities of large language models. Large language models can perform in-context learning, where they learn a new task from a few task demonstrations, without any parameter updates. This work examines the implications of in-context learning for the creation of datasets for new natural language tasks. Departing from recent in-context learning methods, we formulate an annotation-efficient, two-step framework: selective annotation that chooses a pool of examples to annotate from unlabeled data in advance, followed by prompt retrieval that retrieves task examples from the annotated pool at test time. Based on this framework, we propose an unsupervised, graph-based selective annotation method, voke-k, to select diverse, representative examples to annotate. Extensive experiments on 10 datasets (covering classification, commonsense reasoning, dialogue, and text/code generation) demonstrate that our selective annotation method improves the task performance by a large margin. On average, vote-k achieves a 12.9%/11.4% relative gain under an annotation budget of 18/100, as compared to randomly selecting examples to annotate. Compared to state-of-the-art supervised finetuning approaches, it yields similar performance with 10-100x less annotation cost across 10 tasks. We further analyze the effectiveness of our framework in various scenarios: language models with varying sizes, alternative selective annotation methods, and cases where there is a test data domain shift. We hope that our studies will serve as a basis for data annotations as large language models are increasingly applied to new tasks. Our code is available at https://github.com/HKUNLP/icl-selective-annotation.

1 INTRODUCTION

This work studies how in-context learning changes dataset annotation, proposing selective annotation to reduce manual cost while retaining performance. It reports broad gains over random selection, lower annotation costs, and greater stability across settings.

  • In-context learning adapts large language models to downstream tasks from a few demonstrations without parameter updates.
  • Selective annotation chooses examples before testing, then retrieves task demonstrations from the annotated pool for each test instance.
  • Vote-k selects diverse, representative examples with an unsupervised graph-based method.
  • 12.9% relative gain at 18 annotations and 11.4% at 100 annotations were achieved versus random selection on average.
  • 10-100× fewer annotations produced similar performance to state-of-the-art supervised finetuning across 10 tasks.
  • Vote-k largely reduces in-context learning variance and shows larger improvements under domain shift.

2 SELECTIVE ANNOTATION FOR IN-CONTEXT LEARNING

The framework treats annotation selection as a distinct first step in few-shot learning, selecting a small labeled pool before retrieving prompts at test time. Vote-k uses graph-based selection to balance representativeness and diversity.

  • Selective annotation addresses the often-unexamined total annotation cost of in-context learning when all labeled data are assumed available for retrieval.
  • The framework selects a labeled subset L from unlabeled samples X, with |L| = M defining the annotation budget.
  • Vote-k represents unlabeled instances with Sentence-BERT embeddings and builds a directed graph linking each instance to its k nearest neighbors.
  • Vote-k promotes representativeness for finding similar demonstrations and diversity for increasing total coverage.
  • The method iteratively selects high-scoring vertices, using a discount for vertices close to already selected instances to encourage diversity.
  • Random and other selective-annotation baselines underperform vote-k, supporting the importance of selective annotation for reducing annotation cost.
  • After annotation, prompt retrieval uses Sentence-BERT cosine similarity to find examples matching each test instance.

3 EXPERIMENTS

Experiments evaluate selective annotation across 10 diverse datasets and 9 tasks using multiple language-model settings and evaluation protocols. The results compare random selection with vote-k and examine stability under subsampled unlabeled data.

  • 3.1 DATASETS AND TASKS: 10 diverse datasets span 9 tasks, enabling evaluation across varying task formulations.The datasets include tasks represented in the GLUE benchmark.
  • 3.2 EXPERIMENTAL SETTINGS: Evaluation follows standard train/dev./test splits, with test or development data used according to dataset availability.GeoQuery combines development and test data for more reliable evaluation on a larger set.
  • 3.1 DATASETS AND TASKS: GPT-J is the default in-context learning model, while Codex-davinci-002 is used for MWoZ, GeoQuery, and NQ.Codex is used for structured prediction tasks in these experiments.
  • 3.2 EXPERIMENTAL SETTINGS: Prompt retrieval feeds as many retrieved samples as possible until the language model reaches its maximum token length.Retrieved examples are ordered by ascending similarity so more similar examples benefit from recency bias.
  • 3.3 MAIN RESULTS: Across 10 datasets, vote-k substantially outperforms random annotation at budgets of 100 and 18 examples.At 100 examples, the average gain is 5.2% absolute and 11.4% relative; at 18 examples, it is 5.8% absolute and 12.9% relative.
  • 3.3 MAIN RESULTS: Vote-k reduces variance across trials, and 18 selected examples outperform 100 random examples on 6 of 10 tasks.The experiments simulate unlabeled-data variability through repeated random subsampling.

4 ANALYSIS

The analysis evaluates selective annotation across finetuning comparisons, model sizes, domain shifts, prompt retrieval strategies, and alternative selection methods. Vote-k consistently benefits in-context learning, especially with limited annotations and domain shifts, while prompt retrieval remains essential.

  • In-context learning vs. finetuning: Vote-k in-context learning outperforms strong finetuning baselines across three representative tasks, requiring 10–100× fewer annotations for similar performance.Finetuning requires 1000 annotations for HellaSwag and 800 for MWoZ to match vote-k using 18 or 100 examples.
  • In-context learning vs. finetuning: Selective annotation helps in-context learning but not finetuning, possibly because similarity-based retrieval is more robust to outliers and small annotation sizes.Random retrieval does not benefit from vote-k, while explicitly removing outliers helps finetuning benefit from vote-k.
  • Language models with various sizes: 7.5% improvement with GPT-Neo versus 2.6% with GPT-3 on HellaSwag shows that vote-k gains remain consistent across model sizes but are larger for smaller models.The comparison covers GPT-Neo 2B, GPT-J 6B, and GPT-3 across HellaSwag, SST-5, and MWoZ.
  • Effects of domain shift: 9.9 versus 5.5 accuracy-point improvements on CivilComments show that vote-k gains are more pronounced under domain splits than random splits.The experiments use CivilComments and Amazon with random and domain-based splits, simulating deployment in unseen domains.
  • Random prompt retrieval: Random prompt retrieval can erase vote-k’s advantage, showing that selective annotation and similarity-based prompt retrieval must be combined.Random retrieval yields 62.5 versus 63.2 on HellaSwag and 35.7 versus 43.8 on MWoZ for vote-k versus random annotation.
  • Alternative selective annotation methods: Vote-k outperforms alternative selective annotation methods, while fast vote-k achieves similar performance with greater computational efficiency.The alternatives include facility location, diversity, least-confidence, and fast vote-k; performance is averaged over three random trials with 100 examples.

5 RELATED WORK

Prior work improves in-context learning through task adaptation, instructions, and formulation, while active learning targets performance with fewer labeled instances. The paper’s selective annotation shares active learning’s cost-reduction goal but avoids iterative parameter updates for large language models.

  • In-context learning research has explored meta-training, task instructions, and task formulation to improve performance.
  • Active learning aims to achieve similar or better model performance with fewer labeled training instances.
  • The paper’s selective annotation step shares active learning’s goal of reducing annotation cost.
  • Unlike many active learning methods, the approach does not involve iterative parameter updates, which are computationally expensive for large language models.

6 CONCLUSION

The paper studies dataset annotation through selective annotation for in-context learning and introduces vote-k, which selects diverse, representative examples. Across 10 tasks, vote-k improves performance and reaches similar performance to supervised finetuning with substantially fewer annotations.

  • The paper introduces vote-k, which selects diverse, representative examples for annotation.
  • Vote-k improves task performance across 10 diverse tasks by a large margin.
  • Vote-k achieves similar performance to state-of-the-art supervised finetuning with 10-100× less annotation cost.
  • The reported effectiveness of vote-k is consistent across different language model sizes and domain shifts between training and test data.

A DATASETS AND TASKS

The supplied dataset examples span commonsense reasoning, paraphrase detection, dialogue or SQL-style tasks, factual question answering, classification, and summarization. They illustrate the varied task formats used in the paper’s benchmark materials.

  • The examples include commonsense reasoning prompts involving descriptions of bathing or grooming dogs.
  • Paraphrase detection is represented by sentence-pair questions asking whether two statements are equivalent.
  • The materials include sentiment classification prompts requiring positive or negative judgments about movie reviews.
  • Dialogue and text-to-SQL examples provide conversational contexts, database schemas, and SQL answers.
  • Additional examples cover factual question answering, truth-value judgments, and short summarization.

C DETAILED MAIN RESULTS

The detailed-results section expands the main results by reporting mean, maximum, and minimum performance across three trials. The supplied figure passage identifies an OPT-175B comparison between random and vote-k annotation on HellaSwag.

  • Tables 7 and 8 report mean, maximum, and minimum performances across three trials.
  • Figure 4 compares OPT-175B performance for ICL-random and ICL-vote-k on HellaSwag.

E REMOVING OUTLIERS FOR FINETUNING

Explicitly removing outliers helps finetuning benefit from vote-k.

  • Removing outliers improves finetuning when using vote-k selective annotation.

F DIVERSITY AND REPRESENTATIVENESS OF SELECTED SAMPLES

The analysis evaluates whether selected samples are diverse and representative, finding that vote-k improves both properties compared with random selection.

  • The finetuning comparison uses a 100-example annotation budget and removes 10% of examples farthest from the training data before selection.
  • Vote-k improves both diversity and representativeness compared with random selection.
  • The evaluation measures diversity in input space, diversity in feature space, and representativeness, with subscripts denoting standard deviations.

G DETAILS OF SELECTIVE ANNOTATION METHODS

This section details selective annotation methods, including vote-k and embedding-based diversity selection, together with their algorithmic procedures.

  • Selective annotation methods used in Section 4.5 are described here.
  • Algorithm 1 specifies the vote-k selective annotation method introduced in Section 2.1.
  • Embedding diversity: The selected diverse embeddings support prompt retrieval by helping find similar in-context examples for many test instances.
  • Vote-k algorithm: Algorithm 1 initializes a k-nearest-neighbor graph over unlabeled samples, selects an initial subset, scores remaining candidates, and returns the selected samples.
  • Facility location: Algorithm 2 adapts a greedy facility-location procedure for selecting samples.
  • Embedding diversity: The diversity-selection procedure begins with an empty labeled set, tracks maximum similarities, and continues until the annotation budget M is reached.
  • Embedding diversity: Embedding-based diversity selection iteratively chooses an instance farthest from already selected samples using cosine distance.
Loading 2209.01975v1…