Source-linked AI summary
Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning
Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, Colin Raffel
TL;DR
Few-shot ICL is flexible but repeatedly processes labeled examples, creating computational and memory costs, while evidence comparing PEFT in very low-data settings has been limited. The paper compares the paradigms and introduces T-Few, a T0-based recipe using (IA)3 and additional losses. T-Few achieves higher accuracy than ICL at lower computational cost and reaches super-human performance on RAFT.
Problem
Evidence has been limited on whether PEFT works well with very little labeled data, despite ICL’s computational costs and performance and prompt-sensitivity drawbacks.
Method
The paper develops T-Few, a fixed T0-based recipe using (IA)3 activation rescaling, unlikelihood and length-normalized losses, and pre-training of PEFT parameters.
Results
T-Few attains higher accuracy than few-shot ICL at lower computational cost, and achieves 75.8% accuracy on RAFT versus 73.5% for the human baseline.
Takeaways & Limitations
T-Few can be applied as-is to novel real-world tasks without task-specific hyperparameter tuning or other changes.
Takeaways & Limitations
ICL inference costs grow with the number of in-context examples, while PEFT adds a one-time fine-tuning cost and can incur method-dependent inference overhead.
Abstract
from arXiv · showhide
Few-shot in-context learning (ICL) enables pre-trained language models to perform a previously-unseen task without any gradient-based training by feeding a small number of training examples as part of the input. ICL incurs substantial computational, memory, and storage costs because it involves processing all of the training examples every time a prediction is made. Parameter-efficient fine-tuning (PEFT) (e.g. adapter modules, prompt tuning, sparse update methods, etc.) offers an alternative paradigm where a small set of parameters are trained to enable a model to perform the new task. In this paper, we rigorously compare few-shot ICL and PEFT and demonstrate that the latter offers better accuracy as well as dramatically lower computational costs. Along the way, we introduce a new PEFT method called (IA)$^3$ that scales activations by learned vectors, attaining stronger performance while only introducing a relatively tiny amount of new parameters. We also propose a simple recipe based on the T0 model called T-Few that can be applied to new tasks without task-specific tuning or modifications. We validate the effectiveness of T-Few on completely unseen tasks by applying it to the RAFT benchmark, attaining super-human performance for the first time and outperforming the state-of-the-art by 6% absolute. All of the code used in our experiments is publicly available.
1 Introduction
The paper contrasts few-shot ICL with PEFT and proposes T-Few, a fixed recipe using T0, (IA)3, and additional loss terms for unseen tasks.
- ICL performs new tasks from prompted examples without gradient-based training, but requires processing those examples for every prediction.
- (IA)3 rescales attention keys, values, and feed-forward activations with learned vectors while introducing relatively few parameters.
- PEFT updates or adds only a small fraction of model parameters, supporting efficient adaptation and mixed-task batches.
- The paper proposes T-Few, a T0-based recipe combining (IA)3 with unlikelihood and length-normalized losses for novel tasks.
- T-Few is evaluated against strong ICL baselines and designed for application without task-specific tuning or modifications.
2 Background
The background compares ICL and PEFT in terms of task adaptation, computation, memory, and storage, highlighting ICL’s inference costs and PEFT’s efficiency trade-offs.
- Few-shot in-context learning (ICL): ICL feeds prompted input-target examples and an unlabeled query to an autoregressive model, enabling immediate few-shot task performance.
- Few-shot in-context learning (ICL): k-shot ICL increases computational cost by approximately k + 1 times relative to processing the unlabeled example alone.
- Few-shot in-context learning (ICL): ICL memory costs scale approximately linearly with k, while storing 32 prompted 512-token examples requires about 66 kilobytes on disk.
- Few-shot in-context learning (ICL): Example ordering strongly influences ICL predictions, and performance can remain strong when in-context labels are swapped incorrectly.
- Parameter-efficient fine-tuning: PEFT updates or adds relatively few parameters, reducing memory and storage requirements while supporting some mixed-task batch configurations.
- Parameter-efficient fine-tuning: PEFT methods differ in inference overhead and mixed-task support, and their one-time fine-tuning cost is amortized during inference.
3 Designing the T-Few Recipe
T-Few combines T0 with few-shot PEFT, loss terms tailored to rank classification, and the new (IA)3 activation-rescaling method. Across held-out tasks, these choices improve accuracy while targeting low parameter, memory, storage, and mixed-task inference costs.
- 3.1 Model and Datasets: T-Few uses T0 as its pretrained model and evaluates generalization on held-out sentence-completion, natural-language-inference, coreference, and word-sense-disambiguation tasks.The evaluation uses few-shot subsets, prompt templates, rank classification, and median accuracy across datasets, prompts, and subsets.
- 3.2 Unlikelihood Training and Length Normalization: 63.3% accuracy results when both length normalization and unlikelihood loss are added to full-model fine-tuning, improving over 60.7% without length normalization.Length normalization alone raises accuracy to 62.71%; the combined losses add no hyperparameters and are retained in the recipe.
- 3 Designing the T-Few Recipe: The recipe is designed to minimize updated parameters, retain strong few-shot accuracy, and support mixed-task batches without task-specific model modifications.Activation-modifying methods are preferred because task-specific changes can be applied independently to examples in a batch.
- 3.3 Parameter-efficient fine-tuning with (IA)3: (IA)3 rescales selected attention and feed-forward activations with learned vectors initialized to one, adding L(2d_k + 2d_v + d_ff) decoder parameters.The vectors are applied to keys and values in self-attention and encoder-decoder attention, plus intermediate feed-forward activations.
- 3.3 Parameter-efficient fine-tuning with (IA)3: Results for prompt tuning differ from prior reports, potentially because this study uses a different model and datasets, with validation performance fluctuating during training.The authors describe this as a hypothesis and note possible optimization issues for prompt tuning.
4 Outperforming ICL with T-Few
T-Few uses one fixed recipe across tasks and outperforms few-shot ICL on held-out tasks while substantially reducing inference costs. It also performs strongly on RAFT, where it exceeds the human baseline, while ablations show benefits from its training components.
- Held-out T0 tasks: T-Few outperforms GPT-3 175B few-shot ICL by 6% accuracy while using a model about 16× smaller.It also exceeds zero-shot T0, few-shot ICL with T5+LM, and smaller GPT-3 variants.
- Computational costs: 1,000× lower computational cost is reported for T-Few than for GPT-3 175B ICL in the held-out-task comparison.The table caption summarizes the accuracy and computational-cost comparison.
- Inference costs: 1.1e12 FLOPs are required for a T-Few prediction, versus 1.4e15 FLOPs for GPT-3 175B ICL.The GPT-3 ICL estimate is more than three orders of magnitude larger.
- Training costs: 20 GPT-3 175B ICL examples cost about as much FLOPs as training T-Few.Training T-Few requires approximately 2.7e16 FLOPs, while fine-tuning on one dataset takes about half an hour on one NVIDIA A100 GPU.
- Storage costs: 4.2 MB per task for (IA)3 is larger than ICL’s 16 kB example storage, but 10,000 adaptations occupy about one T0 checkpoint.The T0 checkpoint is 41.5 GB.
- RAFT benchmark: 75.8% accuracy on RAFT exceeds the 73.5% human baseline and is 6% above the next-best method.GPT-3 175B achieves 62.7%, while T-Few is applied using the standard prompts without task-specific changes.
- Ablation experiments: Removing unlikelihood training and length normalization decreases average accuracy by 4.1%.Removing pre-training decreases accuracy by 1.6%, while removing both pre-training and the additional losses decreases it by 2.5%.
5 Conclusion
The paper concludes that T-Few combines higher accuracy with lower computational cost than few-shot ICL. It also reports strong RAFT performance and identifies generative tasks as future work.
- Conclusion: T-Few attains higher accuracy than few-shot ICL at lower computational cost using the (IA)3 PEFT method.(IA)3 rescales inner activations with learned vectors while adding only a tiny number of parameters.
- Conclusion: T-Few exceeds the human baseline on RAFT and outperforms prior submissions by a large margin without task-specific tuning.The recipe is applied as-is to the benchmark.
- Future work: The experiments cover classification tasks, and applying T-Few to summarization and question answering remains future work.
A Compute resources used
The reported experiments used several GPU configurations and training durations for T0 models and PEFT methods. Pre-training (IA)3 required about one day on four GPUs.
- Compute resources: T0-3B models were trained on 48GB A6000 GPUs.
- Compute resources: Training T0-3B with most PEFT methods took about one hour, while Intrinsic SAID and FishMask each took about two hours.
- Compute resources: Pre-training (IA)3 took one day on four A6000 GPUs.
- Compute resources: T0 models were trained on 80GB A100 GPUs, with each model taking about half an hour.
C Full Unlikelihood Training and Length Normalization Results
Table 3 reports results using both unlikelihood training and length normalization.
- Table 3 includes unlikelihood training and length normalization.
D Full PEFT Results
The full PEFT evaluation compares multiple parameter-efficient methods under specified training configurations, alongside full-model fine-tuning and loss ablations.
- The evaluation compares PEFT methods including adapters, Compacter variants, prompt tuning, prefix tuning, FishMask, Intrinsic SAID, LoRA, BitFit, and LayerNorm.
- The ablation tables vary inclusion of the language-modeling, length-normalized, and unlikelihood losses.
- Full-model fine-tuning is trained for 300 steps with a learning rate of 3e−4.
E Full Pre-training Results
Table 8 reports per-dataset results for pre-training with (IA)3.
- Table 8 shows per-dataset results for pre-training with (IA)3.
F Full Main Results
The main results compare T-Few with zero-shot T0, T5+LM in-context learning, and GPT-3 in-context learning baselines. T0 itself was unable to benefit from few-shot in-context examples in preliminary experiments.
- The evaluation compares T-Few against zero-shot T0, T5+LM few-shot ICL, and GPT-3 models with 6.7, 13, and 175 billion parameters.
- T0 was unable to perform few-shot ICL, with performance decreasing as the number of in-context examples increased.
- GPT-3 results are reported from Brown et al. because the models were not publicly released.
G Full Ablation Results
Table 10 presents the T-Few ablation results.
- Table 10 reports the T-Few ablation results.
H RAFT Experiment Details
The RAFT experiment applies T-Few across 11 tasks with task-specific handling for Banking 77. Results are reported per dataset alongside other top-performing methods.
- RAFT comprises 11 tasks spanning classification, safety, review, and language-understanding datasets.
- T-Few is applied across RAFT without putting labels into the input string, except for Banking 77.Unlikelihood training is disabled for Banking 77 because its 77 classes cause memory issues.
- Per-dataset results compare T-Few with the other top-5 methods on RAFT.