PaperlayerView exact revision on arXiv

Source-linked AI summary

Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey

Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, Sai Qian Zhang

arXiv:2403.14608v7cs.LG

TL;DR

Large-model adaptation is costly because billions of parameters demand substantial computational resources, especially on constrained hardware. The survey reviews PEFT algorithms, applications, cost-reduction techniques, and system designs, concluding with a comprehensive guide to algorithmic and implementation considerations.

  • Problem

    Adapting large models to downstream tasks is computationally and resource intensive because their scale creates substantial fine-tuning and system-design demands.

  • Method

    The survey systematically categorizes PEFT algorithms, examines their implementation costs, reviews applications, and discusses strategies and systems for reducing computation.

  • Results

    The survey provides a comprehensive examination and taxonomy of PEFT algorithmic design, computational efficiency, application scenarios, and system implementation.

  • Takeaways & Limitations

    The survey serves as a guidance and knowledge base for readers seeking to understand PEFT concepts and system implementation.

Abstract

from arXiv · show

Large models represent a groundbreaking advancement in multiple application fields, enabling remarkable achievements across various tasks. However, their unprecedented scale comes with significant computational costs. These models, often consisting of billions of parameters, require vast amounts of computational resources for execution. Especially, the expansive scale and computational demands pose considerable challenges when customizing them for particular downstream tasks, particularly over the hardware platforms constrained by computational capabilities. Parameter Efficient Fine-Tuning (PEFT) provides a practical solution by efficiently adjusting the large models over the various downstream tasks. In particular, PEFT refers to the process of adjusting the parameters of a pre-trained large model to adapt it to a specific task or domain while minimizing the number of additional parameters introduced or computational resources required. This approach is particularly important when dealing with large-scale language models with high parameter counts, as fine-tuning these models from scratch can be computationally expensive and resource-intensive, posing considerable challenges in the supporting system platform design. In this survey, we present comprehensive studies of various PEFT algorithms, examining their performance and computational overhead. Moreover, we provide an overview of applications developed using different PEFT algorithms and discuss common techniques employed to mitigate computation costs for PEFT. In addition to providing an extensive survey from an algorithmic standpoint, we also examine various real-world system designs to investigate the implementation costs associated with different PEFT approaches. This survey serves as a valuable resource for researchers aiming to understand both the PEFT algorithm and its system implementation, offering detailed ......

I. INTRODUCTION

The survey motivates PEFT as a way to adapt increasingly large language models while reducing fine-tuning costs, then organizes its review around model computation and PEFT operations.

  • LLMs generalize to unseen tasks, but fine-tuning remains essential for stronger performance on new datasets and tasks.
  • PEFT selectively updates a small proportion of parameters while leaving the remainder of the large model unchanged.
  • The survey categorizes PEFT algorithms as additive, selective, reparameterized, or hybrid according to their computational operations.
  • The introduction examines LLM computation through LLaMA’s embedding, decoder, attention, feedforward, and output-generation components.
  • Attention computation scales quadratically with input length, while KV-cache storage depends on layers, batch size, context length, head dimension, and number of heads.The stated KV-cache size is L ˆ 2 ˆ b ˆ l ˆ dhead ˆ nhead.
  • LLaMA generates tokens autoregressively, repeatedly using previous tokens and the predicted token until a complete sequence is produced.

B. Overview on Parameter Efficient Fine Tuning

PEFT addresses the cost of full fine-tuning by updating minimal parameters, and the survey introduces its operation-based taxonomy alongside representative evaluation tasks.

  • Full fine-tuning of growing language models can require thousands of GPUs in parallel, motivating PEFT for downstream adaptation.
  • PEFT is increasingly used beyond language models because vision and multimodal models can also be adapted from large datasets to smaller or different modalities.
  • Additive PEFT injects trainable modules, selective PEFT trains a parameter subset, reparameterized PEFT uses equivalent low-dimensional forms, and hybrid PEFT combines methods.
  • GLUE evaluates nine sentence or sentence-pair language-understanding tasks spanning varied dataset sizes, text genres, and difficulty levels.
  • Common-sense reasoning evaluation includes OpenBookQA, PIQA, and Social IQA, while vision evaluation covers recognition, action recognition, and dense prediction.

D. Evaluation Benchmarks for PEFT

The survey presents benchmarks for evaluating PEFT methods from both algorithmic and system perspectives, covering standardized tasks, realistic interactions, and controlled workload patterns.

  • Algorithmic benchmarks compare PEFT methods across performance, convergence, efficiency, combinability, scalability, and transferability.
  • ShareGPT evaluates system handling of diverse, realistic conversational queries and responses, including response accuracy and request-processing efficiency.
  • Azure Function traces from 2019 and 2021 simulate irregular and peak event-driven workloads that approximate computational demands faced by LLM systems.
  • Gamma-process workloads generate synthetic request-arrival patterns with varied rates for controlled evaluation of system performance under realistic user activity.

1) Adapters:

Adapter methods add small trainable modules within Transformer blocks, while related prompt-based approaches introduce trainable representations or vectors to adapt frozen models.

  • Adapters: Adapters typically use down-projection, nonlinear activation, and up-projection layers, with bottleneck dimension r controlling their configuration.The adapter maps hidden dimension d through a smaller bottleneck of dimension r.
  • Adapters: Serial Adapter inserts modules after self-attention and FFN layers, whereas parallel designs place adapter side-networks alongside Transformer sublayers.
  • Adapters: CoDA combines parallel adapters with soft top-k token selection to improve inference efficiency.
  • Adapters: Multi-task adapter methods fuse, mix, merge, or otherwise combine pretrained adapters to incorporate information from multiple tasks.
  • Soft Prompt: Soft prompts prepend adjustable vectors to input sequences, exploiting trainable continuous embeddings rather than discrete token representations.
  • Soft Prompt: Prefix-tuning prepends learnable vectors to keys and values across Transformer layers and uses an MLP reparameterization for optimization stability.
  • Soft Prompt: Soft-prompt training can be unstable and slow to converge, motivating source-task initialization and transferable prompt approaches.

3) Other Additive Methods:

Other PEFT methods rescale activations, selectively update existing parameters, or reparameterize weights into efficient forms while preserving the frozen model structure where possible.

  • Activation-based methods: (IA)3 introduces learnable rescaling vectors for key, value, and feed-forward activations.
  • Activation-based methods: SSF injects layers after MSA, FFN, and normalization operations to scale and shift generated features.
  • Selective PEFT: Selective PEFT applies a binary mask to existing parameters, updating only parameters whose mask value is 1 during backpropagation.The mask avoids introducing additional trainable parameters.
  • Selective PEFT: Selective methods determine trainable parameters using magnitude, Fisher information, initial parameter changes, or second-order approximations.
  • Selective PEFT: Unstructured masking can reduce hardware efficiency through uneven non-zero distributions, motivating structured masks with regular patterns.
  • Reparameterized PEFT: Reparameterized PEFT trains low-rank parameters and converts them back to the original weight form for inference without changing inference speed.
  • Reparameterized PEFT: DyLoRA trains across a predefined rank range and dynamically selects a rank at each iteration, reducing the search time for a fixed LoRA rank.

››QQT ´ I

LoRA derivatives refine low-rank adaptation through adaptive gating, shared parameters, weight decomposition, and related mechanisms that target efficiency or performance.

  • AdaLoRA dynamically adjusts rank within each LoRA module according to the significance of weight matrices, managing parameter counts during training.
  • SoRA replaces AdaLoRA’s orthogonality premise with an optimized gate and avoids the heuristic importance scores and extra calculations associated with AdaLoRA.
  • Other extensions address calibration, overfitting, learning-rate allocation, subspace mixtures, multi-task composition, and adapter weight reparameterization.
  • VeRA shares frozen low-rank matrices across layers and learns small scaling vectors, reducing trainable parameters compared with LoRA.
  • DoRA decomposes weights into magnitude and direction, applying LoRA reparameterization only to the directional component.
  • DoRA consistently outperforms LoRA across various tasks and models according to the survey.

D. Hybrid PEFT

Hybrid PEFT methods combine multiple adaptation strategies to address task-dependent effectiveness and explore better configurations. These approaches include manually combining methods, grouping layers, and using neural architecture search.

  • PEFT methods can differ substantially in effectiveness across tasks, motivating combinations that exploit their complementary advantages.
  • UniPELT integrates LoRA, prefix-tuning, and adapters within each Transformer block, with controls over which submodules are activated.
  • Layer grouping assigns similar PEFT strategies to layers with similar behaviors and searches over parameter allocation, group tuning, and strategy assignment.
  • MAM Adapter develops Parallel, Multi-head Parallel, and Scaled Parallel Adapter variants by combining adapter, prefix-tuning, and LoRA ideas.
  • NOAH uses neural architecture search to identify task-specific combinations of Adapter, LoRA, and Visual Prompt Tuning for each dataset.

IV. EFFICIENT PEFT DESIGN

Efficient PEFT design targets latency, memory, and resource consumption through pruning, quantization, and memory-efficient training techniques. The survey also describes system-level challenges arising from autoregressive inference and gradient storage.

  • Efficient PEFT design addresses processing latency and peak memory overhead through pruning, quantization, and memory-efficient techniques.
  • Autoregressive inference repeatedly transfers model weights to the graphics processor for each generated token, complicating single-user scheduling and multi-user workload balancing.
  • AdapterDrop and related pruning methods improve training and inference efficiency with minimal performance decrease, while some configurations benefit from high sparsity.
  • Quantization reduces computational and memory costs; BI-Adapter reports that 1-bit adapter quantization minimizes storage while achieving superior performance among tested precisions.
  • QLoRA enables fine-tuning a 65B language model on a single 48GB GPU with performance similar to full 16-bit fine-tuning, while extreme low-bit quantization can harm initialization.
  • Memory-efficient methods reduce gradient or activation storage through parallel branches, disentangled tuners, reversible models, hypernetworks, or pretraining PEFT modules on smaller language models.

V. PEFT FOR DNNS OF OTHER APPLICATIONS

The survey extends PEFT beyond conventional language benchmarks to diverse architectures and downstream applications. It highlights visual instruction following, continual learning, context extension, and broader model families.

  • The survey organizes applications by LLM, Vision Transformer, vision-language alignment, and diffusion architectures, then classifies them by downstream task.
  • PEFT applications include commonsense question answering, discourse relation recognition, out-of-distribution detection, privacy protection, federated learning, and social-bias mitigation.
  • Visual Instruction Following: Visual instruction-following LLMs process images and text to produce textual responses, with evaluation including image captioning and visual question answering.
  • Visual Instruction Following: LLaMA-Adapter prepends learnable prompts in higher LLaMA layers and uses zero-initialized attention to control adaptation contributions during training.
  • Visual Instruction Following: LayerNorm Tuning achieves comparable or better performance than fine-tuning while providing about 10× more parameter efficiency than LoRA.
  • Continual Learning: Continual learning addresses sequences of new tasks within one model, where catastrophic forgetting motivates architecture-based methods with task-specific parameters.
  • Context Window Extension: LLaMA and LLaMA2 have predefined context sizes of 2048 and 4096 tokens, respectively, and performance drops when inputs exceed those lengths.
  • Context Window Extension: Context extension methods include fine-tuning longer contexts and combining context compression with LoRA for information extraction from compressed representations.

1) Image Classification:

PEFT supports adaptation across image, vision-language, and diffusion applications, including image classification, multimodal prompting, grounded generation, and personalized concept generation.

  • Image Classification: Image classification methods such as AdaptFormer and VPT tune small task-specific modules or prompts rather than the full ViT backbone.
  • Image Classification: VPT can underperform with self-supervised ViT, motivating adaptable gates that modulate contributions from transformer blocks according to pretraining and downstream-task dependencies.
  • Video Recognition: ST-Adapter and AIM insert adapters into ViT blocks to model spatial-temporal information for video tasks, surpassing traditional full-model fine-tuning.
  • Vision-Language Alignment: Vision-language methods address prompt adaptation, gradient costs, and multimodal representation by using context optimization, gradient regularization, and encoder adapters.
  • Diffusion Models: Diffusion applications use trainable gated layers or copied encoding layers to incorporate grounding inputs while retaining pretrained model knowledge.
  • Diffusion Models: IP-Adapter separates text and image features with decoupled cross-attention to improve controllable generation of user-specific visual concepts.

VI. SYSTEM DESIGN CHALLENGE FOR PEFT

The survey identifies centralized serving, distributed training, and concurrent multi-PEFT tuning as major system-design challenges, evaluated through accuracy, cost, throughput, memory, and service metrics.

  • System challenges: Centralized PEFT systems must handle many task-specific queries efficiently, while distributed training addresses privacy and data-transfer concerns.Concurrent multi-PEFT training introduces additional challenges involving memory, gradients, model-weight storage, and batching kernels.
  • System challenges: Personalized tasks often require additional fine-tuning because pre-trained models do not fully support them.This motivates PEFT system designs for downstream task customization.
  • Serving metrics: PEFT serving evaluation measures throughput in tokens per second, runtime memory footprint, accuracy across context lengths, and quality of service.Serving memory includes model parameters and KV-cache, while quality of service includes query latency and deadline-related measures.
  • Training metrics: PEFT training evaluation covers downstream-task accuracy, compute cost, and communication cost across cloud servers and edge devices.Compute cost includes forward and backward propagation, while communication cost concerns intermediate-data transfer.

B. Centralized PEFT Serving Frameworks

Centralized PEFT serving frameworks combine different task-specific adapters with shared backbones and schedule requests to improve inference efficiency and parallelism.

  • PetS: PetS unifies diverse PEFT tasks into integrated computation kernels and combines coordinated batching with scheduling to improve throughput and task parallelism.Its framework exploits the distinction between modifiable and immutable model weights.
  • PetS: PetS registers each task with a model tag, compressed PEFT parameters, and an algorithm identifier before assigning it a unique identifier for inference.Supported algorithm examples include LoRA, Adapter, and Bitfit.
  • PetS: Adapter and Bitfit modify the same MLP component differently: Adapter adds weight segments, whereas Bitfit adjusts bias elements.The two operations therefore apply distinct PEFT parameterizations to the shared backbone computation.
  • PetS: Diff-Pruning computes the shared backbone contribution and pruned-weight difference separately, then adds the resulting outputs.The difference term can be represented as sparse matrix-vector multiplication.
  • PetS: PetS uses two-level scheduling: Coordinated Batching groups queries by input length and shared operator, while Macro-batch Streaming supports parallel execution.Grouping by input length avoids wasted padding.
  • DLoRA: DLoRA dynamically merges and unmerges LoRA adapters and migrates requests across worker replicas to address memory footprint, GPU utilization, and load imbalance.These issues arise from variable input and output lengths in traditional serving systems.

C. Distributed PEFT Training Frameworks

Distributed PEFT training separates backbone-model execution from task-specific adaptation to address privacy, ownership, hardware, and communication constraints.

  • Motivation: Fine-tuning large models raises dual privacy concerns and substantial computational-resource requirements for downstream users.Sharing data can expose privacy, copyright, or proprietary information, while sharing model weights can compromise ownership.
  • DLoRA: DLoRA executes the backbone LLM on cloud servers while training PEFT modules entirely on user devices.This distributed arrangement is presented as a response to privacy and hardware constraints.
  • Offsite-Tuning: Offsite-Tuning sends a compressed emulator and adapter to the data proprietor, who fine-tunes the adapter and returns it for integration into the complete model.The emulator is lossy, and complete model weights are not shared.
  • Multi-PEFT training: Multi-PEFT training must support efficient concurrent execution across shared backbones and multi-tenant serving across different backbones.These requirements create distinct kernel-design and system-architecture challenges.
  • Punica: Punica uses backbone matrix multiplication with an SGMV CUDA kernel to batch PEFT add-ons for requests sharing an LLM backbone.The kernel groups requests by PEFT model while parallelizing feature-weight multiplication.
  • Punica: Punica consolidates workloads on active GPUs, migrates older requests, and loads LoRA weights on demand with millisecond-level latency.Its design prioritizes decode-stage optimization because decode dominates serving cost.

VII. CONCLUSION AND FUTURE DIRECTIONS

The survey synthesizes PEFT algorithms, applications, computational efficiency, and system implementation, while identifying unresolved challenges in tuning, benchmarking, scaling, efficiency, and privacy.

  • Conclusion: The survey provides a taxonomy and comprehensive examination of PEFT algorithmic design, computational efficiency, applications, and system implementation.It is intended to help readers from different disciplines grasp PEFT concepts and implementations.
  • Future directions: Future research should develop simpler methods for automatically selecting PEFT hyperparameters instead of relying heavily on manual tuning.Relevant parameters include adapter bottleneck dimension, LoRA rank, and additive-layer arrangement.
  • Future directions: A comprehensive, accepted benchmark is still lacking, limiting fair comparison of PEFT performance and efficiency.The survey calls for a standard set of tasks and metrics.
  • Future directions: Parameter efficiency does not always yield computational or memory savings because full-model activations and gradients may still be required during training.Compression and memory-optimization techniques are identified as possible directions.
  • Future directions: PEFT methods developed for smaller Transformer models do not necessarily retain their design effectiveness as foundation models become larger.Scaling PEFT strategies to larger architectures remains an open research direction.
  • Future directions: Future PEFT systems must address privacy risks in centralized serving and fine-tuning, including inversion attacks that can reconstruct user data.Encryption protocols for personal data and intermediate results are proposed as one direction.
  • Future directions: Compression techniques such as quantization and pruning require dedicated hardware, and their impact on PEFT performance remains a systemic challenge.Building hardware platforms for compressed models is identified as another research direction.
Loading 2403.14608v7…