PaperlayerView exact revision on arXiv

Source-linked AI summary

Automated Federated Pipeline for Parameter-Efficient Fine-Tuning of Large Language Models

Zihan Fang, Zheng Lin, Zhe Chen, Xianhao Chen, Yue Gao, Yuguang Fang

arXiv:2404.06448v2cs.LGcs.AI

TL;DR

Private and heterogeneous edge-server resources make federated fine-tuning of large language models computationally and communicatively difficult. FedPipe automates weight selection, heterogeneous low-rank adapter configuration, quantization, and partial adapter aggregation. Experiments and the reported conclusion indicate that it reduces resource demands and achieves better performance than state-of-the-art baselines.

  • Problem

    Heterogeneous edge-server resources and the computational, memory, and communication demands of LLMs make privacy-preserving federated fine-tuning difficult.

    Source: §I · p. 2

  • Method

    FedPipe formulates automated LLM federated fine-tuning as an MILP and combines important-weight identification, heterogeneous adapter configuration, quantization, and partial adapter aggregation.

    Source: §I · p. 2, F. LoRA Adapters Aggregation and… · p. 7

  • Results

    FedPipe achieves significantly better performance than state-of-the-art LLM federated-learning baselines in extensive evaluations.

    Source: §VII · p. 11

  • Takeaways & Limitations

    FedPipe provides an automated framework for fine-tuning LLMs across edge servers with heterogeneous computing resources.

    Source: §I · p. 2, §VII · p. 11

Abstract

from arXiv · show

Recently, there has been a surge in the development of advanced intelligent generative content (AIGC), especially large language models (LLMs). However, for many downstream tasks, it is necessary to fine-tune LLMs using private data. While federated learning offers a promising privacy-preserving solution to LLM fine-tuning, the substantial size of an LLM, combined with high computational and communication demands, makes it hard to apply to downstream tasks. More importantly, private edge servers often possess varying computing and network resources in real-world scenarios, introducing additional complexities to LLM fine-tuning. To tackle these problems, we design and implement an automated federated pipeline, named FedPipe, to fine-tune LLMs with minimal training cost but without adding any inference latency. FedPipe firstly identifies the weights to be fine-tuned based on their contributions to the LLM training. It then configures a low-rank adapter for each selected weight to train local low-rank adapters on an edge server, and aggregate local adapters of all edge servers to fine-tune the whole LLM. Finally, it appropriately quantizes the parameters of LLM to reduce memory space according to the requirements of edge servers. Extensive experiments demonstrate that FedPipe expedites the model training and achieves higher accuracy than state-of-the-art benchmarks.

I. INTRODUCTION

FedPipe addresses the difficulty of fine-tuning LLMs through federated learning when private edge servers have limited and heterogeneous resources. It automates weight selection, adapter configuration, quantization, and aggregation to reduce training and communication costs.

Motivation

  • Private data prevents parties from directly sharing training data, while edge servers struggle to support full-parameter LLM fine-tuning under standard federated learning.Hardware differences in memory, bandwidth, and computing capability create substantial training and synchronization challenges.Source: §I · p. 1
  • Existing PEFT-based LLM federated-learning frameworks generally assume homogeneous edge-server resources and therefore adapt poorly to heterogeneous environments.FedAdapter focuses mainly on standard language models, while other LLM-FL studies assume homogeneous computing power and memory.Source: §I · pp. 1–2

FedPipe

  • FedPipe automates LLM federated fine-tuning by decoupling and configuring the operations and parameters of existing PEFT methods.The framework is intended to let application and algorithm developers avoid implementing the detailed mechanics of LLM federated fine-tuning.Source: §I · p. 2
  • FedPipe formulates pipeline design as an MILP optimization problem and solves it with trainable-weight identification and fast adapter-configuration algorithms.The optimization selects important weights and configures rank and batch size for local training under heterogeneous computing budgets.Source: §I · p. 2
  • FedPipe selects important weights using singular-value information, configures heterogeneous low-rank adapters, and quantizes local models according to edge-server memory budgets.The design prioritizes updates associated with larger singular values, adapts rank and batch size, and uses different quantization bits to improve training efficiency.Source: §I · p. 2
  • FedPipe aggregates only local adapter weights rather than whole models, reducing communication overhead without adding inference latency.Partial weight aggregation is designed to merge adapter updates while avoiding full-model transfers.Source: §I · p. 2

II. BACKGROUND AND MOTIVATION

LLM federated fine-tuning faces severe compute and communication costs, while PEFT—especially LoRA—reduces trainable parameters without adding inference latency.

II. BACKGROUND AND MOTIVATION

  • The section frames PEFT-FL integration as the central motivation for addressing LLM fine-tuning under constrained edge resources.The background places fine-tuning between LLM pre-training and deployment, while the supplied visuals cover LoRA, model scale, and overhead.Source: §II · p. 2, Table I · p. 3, Figure 2 · p. 3

A. Injecting PEFT into Federated Learning on LLMs

  • Full-parameter LLM fine-tuning is impractical on constrained edge servers because model size creates unacceptable training and aggregation latency.GPT-2 has 774M parameters, over seven times the 110M parameters of BERT.Source: A. Injecting PEFT into Federated… · pp. 2–3
  • PEFT trains only part of a model to reduce computing cost, with LoRA avoiding the inference latency introduced by ADAPTER layers.LoRA injects trainable low-rank matrices into transformer layers while freezing pretrained weights.Source: A. Injecting PEFT into Federated… · p. 3
  • For GPT-2, full-parameter fine-tuning requires more than 1.4 times the training time and 1000 times the exchanged parameters of PEFT.These measurements motivate replacing entire-model updates with parameter-efficient updates.Source: A. Injecting PEFT into Federated… · p. 3, Figure 3 · p. 3

B. Severe Straggler Problem in LLM FL

Heterogeneous edge resources create severe LLM-FL straggler effects, while LoRA configuration requires balancing performance against compute and identifying changing weight priorities.

B. Severe Straggler Problem in LLM FL

  • Resource heterogeneity causes significantly different local training times and severe stragglers during LLM-FL aggregation.Existing frameworks generally assume edge servers have sufficient and homogeneous resources.Source: B. Severe Straggler Problem in LLM FL · p. 3
  • A 10 TFLOPS computing difference makes heterogeneous LLM training substantially worse than standard-model training in under-training rate and aggregation time.The experiments define under-training rate and average aggregation time to quantify the straggler problem.Source: B. Severe Straggler Problem in LLM FL · pp. 3–4

C. Weight-Level Configuration

  • Adding more LoRA trainable weights improves GPT-2 performance but increases computing overhead, making weight-count selection a budget-dependent trade-off.The section uses perplexity as its performance indicator, with lower perplexity representing better performance.Source: C. Weight-Level Configuration · p. 4, Figure 5 · p. 4
  • Batch size and decomposition rank are the two most important evaluated LoRA parameters, with larger values improving performance while consuming more computation.Their performance and computing effects are summarized in Fig. 7.Source: Figure 7 · p. 4
  • Different weight vectors produce different LoRA performance, and their importance changes across training rounds, so one-shot measurements cannot determine priorities.The evaluated vectors are Wq, Wk, Wv, and Wo; the resulting variation motivates automatic per-server identification.Source: C. Weight-Level Configuration · p. 4, Figure 6 · p. 4

D. Budget-Aware Model Parameters Alignment

FedPipe aligns LoRA adapter structures with heterogeneous edge-server budgets by modeling training as an MILP and selecting budget-compatible parameters. It uses budget-aware batch sizes and ranks while coordinating local adapter training and aggregation.

D. Budget-Aware Model Parameters Alignment

  • Larger batch sizes or decomposition ranks improve LoRA performance but consume more resources, motivating optimal combinations under each edge server’s computing budget.Source: D. Budget-Aware Model Parameters… · p. 4

A. Overview of FedPipe

  • FedPipe’s pipeline identifies important weights, configures heterogeneous adapters, quantizes models for server memory budgets, and aggregates local models with different adapter sizes.Source: A. Overview of FedPipe · p. 4

B. Modelling Heterogeneous LoRA Adapters

  • LoRA freezes the pre-trained model and represents each update as a low-rank decomposition ΔW = BA, reducing trainable dimensions for edge-server training.Source: B. Modelling Heterogeneous LoRA Adapters · p. 5
  • The global objective is a data-weighted average of local losses across edge servers and seeks the optimal model parameters.Source: B. Modelling Heterogeneous LoRA Adapters · p. 5
  • FedPipe models heterogeneous adapters as an MILP that selects trainable weights, ranks, and batch sizes while constraining each server’s FLOPS budget.The formulation uses LoRA’s low-rank update structure and includes constraints for batch-size ranges, important transformer weights, allowable ranks, and per-server computing capacity.Source: B. Modelling Heterogeneous LoRA Adapters · p. 5

C. Identification of important weights

FedPipe identifies trainable weights by combining singular-value information with sensitivity-based scoring. Higher-scoring weights are associated with better fine-tuning performance, while smoothed sensitivity addresses noisy single-minibatch estimates.

C. Identification of important weights

  • FedPipe applies SVD to each LoRA update before every round and uses average singular values to quantify each trainable weight’s information content.The method is motivated by PCA, where larger singular values contain richer data information.Source: C. Identification of important weights · pp. 5–6
  • The importance score also incorporates sensitivity based on the gradient–weight product, which approximates average loss change when a parameter is zeroed.The sensitivity function uses model parameters and their corresponding gradients.Source: C. Identification of important weights · p. 6
  • Single-minibatch sensitivity estimates are unreliable because random sampling and training dynamics create substantial variability and uncertainty.FedPipe addresses this issue with sensitivity smoothing and uncertainty quantification.Source: C. Identification of important weights · p. 6
  • Higher importance metrics correspond to lower PPL values, supporting the effectiveness of FedPipe’s proposed weight-importance metrics.Source: C. Identification of important weights · p. 6
  • The selected weights are then used to configure each edge server’s LoRA adapter.Source: C. Identification of important weights · p. 6

D. Heterogeneous LoRA Adapters Configuration

FedPipe configures batch sizes and LoRA ranks for heterogeneous edge servers through a two-stage search. It scales batch sizes with computing capability and iteratively selects important weights and feasible ranks under each server’s budget.

D. Heterogeneous LoRA Adapters Configuration

  • FedPipe’s two-stage search first determines batch size from data diversity, then selects a suitable rank under the edge server’s computing budget.Source: D. Heterogeneous LoRA Adapters… · p. 6
  • Batch sizes are proportional to edge-server computing-capability ratios, with values below the minimum batch size replaced by that minimum.The rule uses B = U_r × b_max and enforces b_i ≥ b_min.Source: D. Heterogeneous LoRA Adapters… · p. 6
  • After batch-size selection, FedPipe derives a rank-related computing constraint for each server and searches ranks from high to low across weights ordered by importance.Source: D. Heterogeneous LoRA Adapters… · p. 6
  • The search repeats across weights and ranks until the computing constraint is met, then dynamically adjusts ranks for every weight during each training round.Source: D. Heterogeneous LoRA Adapters… · pp. 6–7

E. Quantization for Adapters’ Training

FedPipe addresses heterogeneous GPU-memory constraints by quantizing pre-trained models according to each server’s budget while keeping LoRA adapters at higher precision. It uses NormalFloat quantization to compress model weights.

E. Quantization for Adapters’ Training

F. LoRA Adapters Aggregation and Deployment

FedPipe aggregates heterogeneous LoRA adapters by preprocessing, combining, and redispatching incremental matrices rather than whole models. It then updates and quantizes local models according to each edge server’s memory constraint.

F. LoRA Adapters Aggregation and Deployment

  • FedPipe transfers rank-decomposition matrices B and A instead of whole models, reducing communication overhead while addressing heterogeneous adapter aggregation.Different edge servers may use different ranks, so their matrices cannot be directly added; naive recovery can also degrade training.Source: F. LoRA Adapters Aggregation and… · p. 7
  • The aggregation pipeline preprocesses edge-server incremental matrices, aggregates them centrally, and dispatches the resulting updates for local model updating.These stages avoid directly updating each server’s original B matrices.Source: F. LoRA Adapters Aggregation and… · p. 8
  • Local updates are quantized before merging into each model so they satisfy the edge server’s memory constraint.The quantization function is applied to the update during local model updating.Source: F. LoRA Adapters Aggregation and… · p. 8

IV. IMPLEMENTATION AND EXPERIMENTAL SETUP

Experiments evaluate FedPipe on LLaMA2 and GPT-2 across instruction-following and natural-language-generation tasks under controlled heterogeneous hardware. Comparisons use full fine-tuning, LoRA, and FedAdapter with common aggregation and client-sampling settings.

IV. IMPLEMENTATION AND EXPERIMENTAL SETUP

  • Experiments deploy LLaMA2-7B and GPT-2 medium and large models for language tasks, including text generation, summarization, question answering, completion, and translation, with 3.52 billion, 355 million, and 774 million parameters, respectively.Source: §IV · p. 8
  • FedPipe is evaluated on Alpaca instruction-following and E2E restaurant-domain natural-language-generation tasks.Alpaca contains approximately 52,000 samples, while E2E contains around 42,000 training, 4,600 validation, and 4,600 test examples.Source: §IV · p. 8
  • Benchmarks include full fine-tuning, LoRA, and FedAdapter, using FedAvg aggregation and random client sampling for fair comparison.LoRA trains low-rank modules while freezing the remaining model parameters and adds no inference latency.Source: §IV · p. 8
  • Resource heterogeneity is modeled by randomly limiting each GPU’s available training memory between 12 GB and 24 GB.Experiments use 16 RTX 3090-Ti GPUs, with task-specific batch sizes and 15 participating edge servers by default.Source: §IV · p. 8

V. EVALUATION

The evaluation measures FedPipe’s training performance and the effectiveness of its designed components.

V. EVALUATION

  • The evaluation reports numerical results for FedPipe’s training performance and component effectiveness.The section covers both overall framework performance and component-level evaluation.Source: §V · p. 8

A. Overall Performance Evaluation

FedPipe achieves stronger accuracy, faster convergence, and fewer trainable parameters than the evaluated benchmarks, especially under heterogeneous resources. Ablations attribute these gains to important-weight selection, adaptive rank configuration, and quantization.

A. Overall Performance Evaluation

  • FedPipe outperforms FT, FedAdapter, and LoRA in converged accuracy under heterogeneous and ideal settings, with a nearly 0.01 PPL increase from ideal to heterogeneous conditions.On LLaMA2, its PPL advantage over FT, FedAdapter, and LoRA is 0.22, 0.31, and 0.06, respectively, under the reported homogeneous comparison.Source: A. Overall Performance Evaluation · pp. 8–9
  • FedPipe has the fewest trainable parameters under heterogeneous settings, approximately 25% fewer than under ideal settings.It uses approximately 7.69M parameters on LLaMA2 and 0.37M on GPT-2; FT and FedAdapter require substantially more.Source: A. Overall Performance Evaluation · p. 9, Figure 14 · p. 9
  • FedPipe converges fastest, reaching GPT-2 PPL values of 3.6 and 4.0 up to 6, 2.7, and 2 times faster, and 24, 18, and 2 times faster than FT, FedAdapter, and LoRA, respectively.Its adaptive rank adjustment preserves convergence and accuracy under heterogeneous client resources.Source: A. Overall Performance Evaluation · p. 9, Figure 15 · p. 9

B. Ablation Evaluation

  • Important-weight identification yields lower PPL than three random combinations of three trainable weights on GPT-2 large.The method prioritizes weights associated with large singular values, which represent principal directions of variation.Source: B. Ablation Evaluation · p. 9, Table III · p. 10
  • Adaptive heterogeneous configuration is more stable than fixed-rank LoRA: fixed-rank PPL rises from 3.23 to 3.51 as rank increases from 4 to 16, while the proposed method rises only slightly.The widening performance gap reflects adaptive rank adjustment to heterogeneous computing capabilities.Source: B. Ablation Evaluation · p. 10, Table IV · p. 10
  • Quantization reduces memory substantially with a small accuracy change: FP32 uses 20.5 GB, while FP32 PPL is 3.224 versus approximately 3.228 for FP16, NF8, and NF4.The reported memory use for FP32 is approximately 1.9, 3.2, and 4.9 times that of FP16, NF8, and NF4.Source: B. Ablation Evaluation · p. 10, Table V · p. 10

VI. RELATED WORK

This section reviews parameter-efficient fine-tuning and federated learning for LLMs, then identifies quantization and resource heterogeneity as key deployment concerns. FedPipe addresses heterogeneous-resource LLM federated fine-tuning through automated weight selection, LoRA configuration, and adapter quantization.

VI. RELATED WORK

  • PEFT reduces trainable parameters while preserving fine-tuning performance; ADAPTER adds modules with overhead, whereas LoRA uses parallel low-rank matrices to avoid additional training latency.Compacter further reduces adapter complexity through shared information and low-rank subspaces.Source: §VI · p. 11
  • Existing LLM federated-learning studies address privacy-preserving fine-tuning, but prior work either targets traditional language models or assumes homogeneous edge resources.FedAdapter accelerates FedNLP convergence by progressively upgrading and profiling adapter configurations, while FedNLP evaluates four common NLP task formulations.Source: §VI · p. 11
  • Quantization reduces model storage and can accelerate inference by lowering bit precision, with methods including GPTQ, AWQ, SwitchBack, and QLoRA.QLoRA quantizes weights into 4-bit NormalFloat and backpropagates gradients into a low-rank structure.Source: §VI · p. 11
  • Resource heterogeneity can delay or exclude clients with limited computing or memory, yet prior work had not addressed this challenge for LLM federated fine-tuning.FedPipe is designed for edge servers with heterogeneous computing resources.Source: §VI · p. 11, §VII · p. 11

VII. CONCLUSION

  • FedPipe selects important weights, configures LoRA adapters under edge-server computing budgets, and quantizes adapters to reduce memory space.Its evaluations report significantly better performance than state-of-the-art LLM federated-learning baselines.Source: §VII · p. 11
Loading 2404.06448v2…