PaperlayerView exact revision on arXiv

Source-linked AI summary

LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition

Chengsong Huang, Qian Liu, Bill Yuchen Lin, Tianyu Pang, Chao Du, Min Lin

arXiv:2307.13269v3cs.CLcs.AI

TL;DR

The paper asks whether LoRA modules trained on different tasks can be composed for efficient adaptation to unseen tasks from only a few examples. LoraHub combines such modules and uses gradient-free coefficient optimization, achieving BBH performance close to few-shot in-context learning with fewer inference tokens. Its main scope boundary is a trade-off between reduced inference cost and extra ADAPT-stage inference steps.

  • Problem

    The paper addresses limited investigation of LoRA modularity and composability for few-shot cross-task generalization to unseen tasks.

  • Method

    LoraHub combines LoRA modules trained on diverse upstream tasks and refines their scalar composition coefficients with a gradient-free optimizer using a few examples from the unseen task.

  • Results

    On BBH, LoraHub closely matches few-shot in-context learning on average while using the equivalent token count as zero-shot inference and fewer tokens than ICL.

  • Takeaways & Limitations

    LoraHub demonstrates the promise of reusing and composing LoRA modules to adapt LLMs to diverse tasks without additional model parameters or human expertise.

  • Takeaways & Limitations

    LoraHub reduces inference-token costs but adds extra inference steps during ADAPT, creating a trade-off with in-context learning.

Abstract

from arXiv · show

Low-rank adaptations (LoRA) are often employed to fine-tune large language models (LLMs) for new tasks. This paper investigates LoRA composability for cross-task generalization and introduces LoraHub, a simple framework devised for the purposive assembly of LoRA modules trained on diverse given tasks, with the objective of achieving adaptable performance on unseen tasks. With just a few examples from a new task, LoraHub can fluidly combine multiple LoRA modules, eliminating the need for human expertise and assumptions. Notably, the composition requires neither additional model parameters nor gradients. Empirical results on the Big-Bench Hard benchmark suggest that LoraHub, while not surpassing the performance of in-context learning, offers a notable performance-efficiency trade-off in few-shot scenarios by employing a significantly reduced number of tokens per example during inference. Notably, LoraHub establishes a better upper bound compared to in-context learning when paired with different demonstration examples, demonstrating its potential for future development. Our vision is to establish a platform for LoRA modules, empowering users to share their trained LoRA modules. This collaborative approach facilitates the seamless application of LoRA modules to novel tasks, contributing to an adaptive ecosystem. Our code is available at https://github.com/sail-sg/lorahub, and all the pre-trained LoRA modules are released at https://huggingface.co/lorahub.

1 Introduction

LoraHub investigates whether modular LoRA adapters can be composed automatically for efficient generalization to unfamiliar tasks. On BBH, it approaches few-shot in-context learning while using fewer inference tokens and enabling reusable modules.

  • Motivation: LoRA reduces LLM fine-tuning costs by freezing the base model and training a lightweight module.This parameter-efficient strategy reduces memory demands and computational costs during training.
  • Research gap: Prior work largely specialized LoRA modules for individual tasks, leaving their composability for cross-task generalization underexplored.LoraHub addresses this gap by automatically assembling compatible modules without task-specific human design.
  • Proposed direction: With only a handful of examples from a new task, LoraHub autonomously combines LoRA modules without assuming which task-specific modules are compatible.The framework allows modules trained on diverse tasks to be combined when they conform to the same specification, such as using the same LLM.
  • Empirical findings: On BBH, LoraHub’s average performance closely matches few-shot in-context learning while substantially reducing inference cost by removing examples from the LLM input.Its inference uses fewer tokens per example than in-context learning.
  • Broader vision: The proposed LoRA platform envisions users sharing or accessing trained modules for automated distribution and composition.The authors present this as a collaborative ecosystem for reusing capabilities while preserving providers’ data privacy.

2 Problem Statement

The paper frames cross-task generalization as adapting an LLM to an unseen task using limited labeled examples. It motivates LoRA-based adaptation because full-model fine-tuning is inefficient, time-consuming, and unstable when examples are scarce.

  • Cross-Task Generalization: Cross-task generalization is the ability of an LLM to perform novel tasks it has not encountered before.The paper distinguishes zero-shot learning from few-shot learning, which uses a handful of labeled examples.
  • Cross-Task Generalization: For an unseen target task, users provide only a limited set of labeled examples to guide adaptation.The problem setting contains N upstream tasks and a target task outside that set.
  • Cross-Task Generalization: Fine-tuning the full LLM on the small example set is inefficient, time-consuming, and unstable.This motivates seeking a more efficient adaptation method for few-shot cross-task generalization.
  • LoRA Tuning: LoRA adapts LLMs with lightweight low-rank modules while leaving the original model weights frozen.Traditional LoRA primarily trains and tests within the same tasks rather than addressing few-shot cross-task generalization.

3 Methodology

LoraHub first composes LoRA modules trained on upstream tasks and then adapts their scalar coefficients using few-shot examples from an unseen task. The framework uses low-rank parameterization and gradient-free optimization, while pruning may be needed when many modules expand the search space.

  • Method Overview: LoraHub trains separate LoRA modules on multiple upstream tasks before adapting their composition to a new task.The new task’s examples guide the subsequent composition and adaptation process.
  • COMPOSE: The COMPOSE stage integrates all available modules into one module using scalar coefficients w_i.The coefficients may be positive or negative, and the resulting module is later combined with the base LLM.
  • LoRA tuning: LoRA represents a weight update as W0 + δW = W0 + AB, where A and B are trainable low-rank matrices.Their product defines the LoRA module and reduces the number of trainable parameters relative to full-model adaptation.
  • ADAPT: During ADAPT, a gradient-free optimizer evaluates the composed module on few-shot examples and refines w for K iterations.The objective uses cross-entropy loss with L1 regularization on the coefficients.
  • Limitations: Combining too many LoRA modules can expand the search space exponentially and destabilize optimization.The method mitigates this by randomly pruning the candidate space, while more advanced pre-filtering remains future work.

4 Experimental Results

Experiments on BBH with FLAN-T5-large evaluate LoraHub against zero-shot, ICL, and gradient-based methods. LoraHub approaches ICL performance with fewer inference tokens, but incurs adaptation costs and trails LoRA or full fine-tuning on substantially different tasks.

  • Experimental setup: The evaluation uses FLAN-T5-large on BBH’s 27 multiple-choice tasks, with exact match as the metric.Twenty candidate LoRA modules are randomly selected during each experimental sequence, and gradient-based baselines train for 40 epochs on the same three runs of five examples.
  • Main results: LoraHub outperforms zero-shot learning on most tasks while closely matching few-shot ICL across three runs with different demonstrations.The reported averages use three runs, each with different few-shot examples.
  • Main results: LoraHub uses the same number of input tokens as zero-shot learning and fewer than ICL, reducing inference cost while approaching peak performance.The paper motivates this trade-off because inference cost increases with input length.
  • Main results: 18 tasks show an upper-bound LoraHub performance that surpasses ICL across runs with different demonstration examples.The authors present this upper bound as evidence of potential for future development.
  • Comparisons and limitations: LoraHub improves average performance over IA3 by 3.1% but remains behind LoRA tuning and full fine-tuning, especially when tasks diverge from the upstream task.On Dyck Languages, LoraHub and ICL average nearly 1.0%, whereas LoRA and full fine-tuning perform strongly with five examples.
  • Comparisons and limitations: LoraHub shifts cost from token processing during inference to extra ADAPT-stage inference steps, creating a use-case-dependent trade-off with ICL.The experiments use 40 adaptation steps; the paper frames LoraHub as complementary to rather than a replacement for ICL.

5 Experimental Analysis

The experiments examine LoraHub’s module count, gradient-free selection, rank sensitivity, applicability beyond instruction-tuned models, and memory efficiency. Results indicate stronger peak performance with more modules, effective selection and generalization, limited rank sensitivity on FLAN-T5, and substantially lower memory use.

  • Gradient-free optimization: LoraHub’s gradient-free optimization identified the task-specific WTQ LoRA module as the most relevant, while their combination was marginally better.The study ranked candidate modules using few-shot loss and evaluated the best candidate on the unseen task.
  • Non-instruction-tuned models: LoraHub learning enabled non-instruction-tuned T5 models to generalize effectively to unseen tasks, despite lower performance than FLAN-T5.The analysis focused on models without zero-shot capabilities, for which training has a larger effect on parameters.
  • Rank sensitivity: Rank 16 consistently outperformed ranks 4 and 64 across runs, although rank had minimal impact for FLAN-T5 and some influence for T5.The comparison considered both average and optimal performance values.
  • Module count: More LoRA modules increased performance variance but also improved the maximum achievable performance.The main experiments randomly selected 20 modules for LoraHub learning.
  • Computational resources: 5GB of memory was required by LoraHub, compared with 34GB for LoRA fine-tuning and 40GB for full fine-tuning.LoraHub’s inference-only mode avoids storing gradients and optimization states.

6 Related work

Related work places LoraHub within model merging, modular composition, mixture-of-experts, and cross-task generalization research. Its distinguishing focus is composing LoRA modules for cross-task generalization without additional training or specialized LoRA manipulation.

  • Model Merging: Model-merging research combines entire models to approximate ensembling or multi-task benefits, whereas LoraHub targets better cross-task generalization.Earlier approaches commonly assume shared model architectures, while the cited comparison distinguishes LoraHub’s objective.
  • Module Composition: Module-composition studies decompose, recompose, modulate, or weight parameter-efficient modules according to functionality, task vectors, or task similarity.These works form the research line most closely aligned with LoraHub.
  • Mixture of Experts: LoraHub resembles mixture-of-experts by treating upstream-trained LoRA modules as task-specialized experts assembled dynamically.Unlike MoE, LoraHub requires no specialized manipulation during training and supports assembly at any scale.
  • Cross-Task Generalization: Cross-task generalization frameworks such as CrossFit, ExT5, FLAN, T0, Instruct-GPT, and ReCross pursue generalization across diverse tasks, aligning with LoraHub’s objective.The passage identifies CrossFit and ReCross as particularly noteworthy connections.

7 Conclusion

LoraHub composes LoRA modules trained on diverse tasks to adapt models to new tasks using a few examples, without extra parameters or human expertise. On BBH, it matched few-shot in-context learning while supporting adaptation without inference-time in-context examples.

  • Conclusion: LoraHub dynamically combines LoRA modules from diverse tasks to adapt to novel tasks using only a few examples.The framework is designed for adaptable cross-task performance.
  • Conclusion: The composition requires neither additional model parameters nor human expertise.The conclusion presents this as a central property of the framework.
  • Conclusion: On BBH, LoraHub matched few-shot in-context learning and removed the need for in-context examples during inference.The conclusion frames this result as evidence for strategic LoRA composability.
  • Conclusion: Reusing and combining LoRA modules may support more adaptable LLMs while minimizing training costs.This is the paper’s stated broader direction.

Reproducibility Statement

The reproducibility statement describes detailed experimental documentation and planned public release of code, scripts, and pretrained LoRA modules.

  • Reproducibility Statement: The paper reports that experiment settings, evaluation metrics, and datasets were described in detail in Section 4.1.This documentation is presented as part of the reproducibility effort.
  • Reproducibility Statement: The authors state that source code, reproduction scripts, and pretrained LoRA modules will be made publicly available.The passage repeats the planned code release in connection with paper acceptance.

A More Analysis

The analysis examines which LoRA modules transfer effectively across BBH tasks and whether gradient-free optimization can identify suitable modules for unseen tasks.

  • The top five influential modules were associated mainly with reading comprehension and reasoning tasks, suggesting stronger cross-task transfer for cognitively demanding skills.Their influence was measured using maximum average absolute weights.
  • No individual module consistently improved all BBH tasks; only the second-ranked module significantly exceeded the original FLAN-T5-large average.
  • The WTQ study evaluated whether gradient-free optimization could identify a suitable downstream-task module by adding a WTQ-trained candidate to the module pool.

B Result of Best Results

LoraHub’s best-run performance is compared with other few-shot adaptation methods on BBH using FLAN-T5-large and five demonstrations per task.

  • Table 4 compares ICL, IA3, LoRA tuning, full fine-tuning, and LoraHub using the maximum performance across three runs.
  • LoraHub’s best results are reported alongside parameter-efficient and full fine-tuning baselines on the BBH benchmark.
  • Five examples per task serve as demonstrations for every method in the comparison.

C Result of non-instrcution-tuned models

These analyses evaluate LoraHub across backbone models, candidate-selection strategies, module counts, and coefficient thresholds, while identifying sources of instability and performance limits.

  • Result of non-instrcution-tuned models: LoraHub is evaluated with T5-large across different ranks, while its zero-shot baseline scores 0.0% on every BBH task except Dyck Languages at 0.67%.
  • Result of larger model: FLAN-T5-xl experiments compare zero-shot learning with few-shot LoraHub, whose average is computed over five random-seed runs; the reported trend resembles FLAN-T5-large.
  • Candidate pre-filtering: Pre-filtering the 20 candidates with the lowest few-shot loss produces a slight overall performance improvement, addressing instability from candidate selection.
  • Influence of Number of LoRA modules: Increasing the number of LoRA candidates increases performance variance, with candidate choice—not random seeds or optimization algorithms—identified as the primary source.
  • The Impact of Threshold: Removing the coefficient threshold decreases average performance by 1.2%, with notable effects on Date Understanding, Disambiguation, and Hyperbaton.
  • The Impact of Threshold: A reasonable threshold is therefore used to mitigate extreme scenarios in coefficient optimization.
Loading 2307.13269v3…