PaperlayerView exact revision on arXiv

Source-linked AI summary

Ferret: Federated Full-Parameter Tuning at Scale for Large Language Models

Yao Shu, Wenyang Hu, See-Kiong Ng, Bryan Kian Hsiang Low, Fei Richard Yu

arXiv:2409.06277v3cs.LGcs.AI

TL;DR

Federated full-parameter tuning of LLMs faces prohibitive communication overhead while preserving computational efficiency, convergence, privacy, and accuracy. Ferret uses first-order local updates, low-dimensional projection, and shared-randomness reconstruction to scale full-parameter tuning; experiments and theory report reduced communication, fast convergence, and competitive accuracy.

  • Problem

    Federated full-parameter tuning of LLMs must reduce communication overhead while preserving computational efficiency, convergence speed, and competitive accuracy.

    Source: §1 · p. 1, §2 · p. 2

  • Method

    Ferret combines first-order local updates with low-dimensional update projection and shared-randomness reconstruction for full-parameter global aggregation.

    Source: §1 · p. 2

  • Results

    Ferret achieves high computational efficiency, reduced communication overhead, fast convergence, and competitive model accuracy while improving the scalability of federated full-parameter tuning.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    Ferret offers a scalable approach to federated full-parameter LLM tuning that balances computational efficiency, communication cost, convergence speed, and model accuracy.

    Source: §6 · p. 9

    The convergence analysis primarily considers a homogeneous setting and assumes smooth nonconvex loss with bounded gradient-estimation variance.

    Source: §4.2 · p. 6

Abstract

from arXiv · show

Large Language Models (LLMs) have become indispensable in numerous real-world applications. However, fine-tuning these models at scale, especially in federated settings where data privacy and communication efficiency are critical, presents significant challenges. Existing approaches often resort to parameter-efficient fine-tuning (PEFT) to mitigate communication overhead, but this typically comes at the cost of model accuracy. To this end, we propose federated full-parameter tuning at scale for LLMs (Ferret), the first first-order method with shared randomness to enable scalable full-parameter tuning of LLMs across decentralized data sources while maintaining competitive model accuracy. Ferret accomplishes this through three aspects: (i) it employs widely used first-order methods for efficient local updates; (ii) it projects these updates into a low-dimensional space to considerably reduce communication overhead; and (iii) it reconstructs local updates from this low-dimensional space with shared randomness to facilitate effective full-parameter global aggregation, ensuring fast convergence and competitive final performance. Our rigorous theoretical analyses and insights along with extensive experiments, show that Ferret significantly enhances the scalability of existing federated full-parameter tuning approaches by achieving high computational efficiency, reduced communication overhead, and fast convergence, all while maintaining competitive model accuracy. Our implementation is available at https://github.com/allen4747/Ferret.

1. Introduction

Ferret targets the communication and accuracy trade-off in federated LLM tuning by enabling scalable full-parameter updates with first-order optimization and shared randomness. The paper combines theoretical analysis with experiments showing improved scalability and competitive model accuracy.

1. Introduction

  • Federated LLM tuning preserves data privacy but incurs prohibitive communication overhead as model parameters reach billions.Source: §1 · p. 1
  • PEFT reduces bandwidth by updating subsets of parameters, but often compromises accuracy, while zeroth-order full-parameter methods can require greater computation and more communication rounds.Source: §1 · p. 2
  • Ferret combines first-order local updates, low-dimensional update projection, and shared-randomness reconstruction for scalable full-parameter federated tuning.Source: §1 · p. 2
  • Theoretical analyses establish advantages over baselines, while experiments report superior scalability and competitive model accuracy.Source: §1 · p. 2

2. Problem Setup

The paper formulates privacy-preserving federated full-parameter LLM tuning as decentralized optimization over local data, using repeated local training and global aggregation. Its central challenge is reducing billion-scale communication without sacrificing efficient computation or convergence speed.

2. Problem Setup

  • Federated full-parameter tuning minimizes average loss across decentralized client datasets without sharing raw data.Source: §2 · p. 2
  • Each communication round alternates independent local model updates with server aggregation and redistribution of the updated global model.Source: §2 · p. 2
  • First-order FL offers efficient updates and fast convergence but requires O(d) communication, whereas zeroth-order FL reduces communication while increasing computation and convergence rounds.Source: §2 · pp. 2–3
  • Ferret is designed to combine high computational efficiency, reduced communication overhead, and fast convergence in federated full-parameter tuning.Source: §2 · p. 3

3. The Ferret Algorithm

Ferret repeatedly aggregates reconstructed updates, performs first-order local optimization, and projects each local update into a low-dimensional shared-randomness representation. Its reconstruction design further improves scalability through bounded random bases, inversion-free computation, and block-wise processing.

3.1. Overview of Ferret

  • Ferret combines first-order local optimization with low-dimensional random-basis projection and shared-randomness reconstruction to reduce communication while retaining efficient updates.Source: §3.1 · p. 3
  • Each round performs global aggregation, T-step local first-order updates, and transmission of a seed with K projected coordinates that regenerate the reconstruction bases.Source: §3.1 · p. 3
  • Using the same seed across rounds can further reduce communication overhead.Source: §3.1 · p. 3

3.2. Update Projection and Reconstruction

  • Ferret projects updates ∆∈R^d into K-dimensional coordinates with K ≪ d by solving a convex minimization problem over random bases.Source: §3.2 · p. 3
  • Bounded-norm random bases are chosen to avoid updating only selected dimensions and support full-parameter tuning with competitive performance.Source: §3.2 · p. 4
  • Approximating V^⊤V with the identity removes matrix inversion, reducing reconstruction complexity to O(Kd) and storage to O(max{K,d}).Source: §3.2 · p. 4
  • Block-wise reconstruction partitions d parameters into blocks, reducing computational and storage costs and further improving global-aggregation scalability.Source: §3.2 · p. 4

4. Theoretical Analyses and Insights

Ferret’s theoretical analysis establishes accurate shared-randomness reconstruction, convergence comparable to SGD, and scalability advantages for federated full-parameter LLM tuning.

4.1. Reconstruction Analysis

  • Ferret’s reconstruction is unbiased, avoids zeroth-order bias and error accumulation across local iterations, and has error rate O~(d/K) when d ≥ K.Theorem 1 identifies 1/(ρK) as crucial for unbiasedness; Theorem 2 states error decreases linearly with K and is independent of T.Source: §4.1 · pp. 4–5
  • Block-wise reconstruction reduces computational complexity by a factor of 1/L under equal block sizes and bases, while adaptive basis allocation minimizes error according to block gradient norms.Increasing L further improves computational efficiency, whereas larger-gradient blocks receive more random bases for improved reconstruction accuracy.Source: §4.1 · pp. 5–6

4.2. Convergence Analysis

  • Ferret requires R = O(1/ϵ2) communication rounds for ϵ convergence error, matching standard SGD asymptotically and avoiding zeroth-order methods’ dimension-dependent factors.The convergence result assumes homogeneous nonconvex smooth objectives and an appropriately chosen learning rate.Source: §4.2 · p. 6

4.3. Scalability and Beyond

  • Ferret combines first-order computation with O(K) communication, fewer local iterations than zeroth-order methods, and faster convergence while supporting adaptability, generalization, and privacy.Its per-round computation is O(τ1T), communication is O(K) with K ≪ d, and it transmits seeds and projected coordinates instead of full models or gradients.Source: §4.3 · pp. 6–7

5. Experiments

Ferret is evaluated across accuracy, scalability, memory consumption, and ablations, showing competitive model quality with substantial efficiency and communication gains. Its ablations identify effective choices for random-basis count, block allocation, and block size.

5. Experiments

  • Experiments compare Ferret with zeroth-order FedZO and FedKSeed, first-order FedAvg, and PEFT-based methods across multiple datasets and model scales.The evaluation covers DataJuicer-1.3B, LLaMA-3B, LLaMA2-7B, and LLaMA2-13B across Natural Instructions, Dolly-15K, CodeAlpaca, and GSM8K.Source: §5 · p. 7, §5.1 · p. 7

5.1. Comparison on Accuracy

  • Ferret outperforms all other methods on Natural Instructions across model sizes, improves over FedKSeed by up to 2.66%, and remains competitive on Dolly-15K, CodeAlpaca, and GSM8K.It improves over FedIT and zeroth-order baselines on CodeAlpaca and GSM8K but slightly underperforms FedAvg on complex tasks, likely because of reconstruction errors.Source: §5.1 · p. 8

5.2. Comparison on Scalability

  • Ferret reduces overall computational cost per round by 6.5× on LLaMA-3B and 6.8× on LLaMA2-7B versus FedKSeed, while reducing communication costs versus FedAvg by 106× and 107×, respectively.The gains combine first-order local updates, block-wise reconstruction, and precise reconstruction.Source: §5.2 · p. 8, Table 4 · p. 8, Table 5 · p. 8
  • Ferret converges in two communication rounds, matching FedAvg and reducing communication-round complexity 20× versus FedKSeed on both DataJuicer-1.3B and LLaMA-3B.Figure 2 compares convergence rounds on Natural Instructions for both model sizes.Source: §5.2 · p. 8, Figure 2 · p. 8

5.3. Comparison on GPU Memory Consumption

  • Ferret matches FedAvg’s peak GPU memory footprint, requiring 9.9 GB for DataJuicer-1.3B and 19.1 GB for LLaMA-3B on Natural Instructions.FedKSeed uses less memory, but Ferret provides stronger converged performance and scalability without extra memory over FedAvg.Source: Table 6 · p. 8, §5.3 · p. 9

5.4. Ablation Studies

  • Increasing K generally improves convergence while leaving generalization comparable; 2K0 already performs compellingly, with only marginal convergence gains from further increases.Generalization may slightly decrease as K increases because noisy-gradient regularization is reduced.Source: §5.4 · p. 9, Figure 3 · p. 9
  • Allocating K_l proportional to ||∆_l|| produces faster convergence and better generalization, supporting the best practice guided by Proposition 2.This allocation is the scheme used for the reported results in the preceding sections.Source: §5.4 · p. 9, Figure 4 · p. 9
  • Increasing the number of blocks improves convergence and reduces projection and reconstruction time while keeping generalization comparable.The paper attributes the convergence improvement to lower reconstruction error from smaller blocks and the speedup to block-wise reconstruction.Source: §5.4 · p. 9, Figure 5 · p. 9

6. Conclusion

Ferret targets scalable federated full-parameter LLM tuning by combining shared randomness with first-order optimization, addressing communication and computational constraints that limit existing approaches. The paper supports this design through theory and experiments while identifying privacy analysis as future work.

6. Conclusion

  • Ferret is presented as delivering computational efficiency, reduced communication overhead, fast convergence, and competitive model accuracy in large-scale federated settings.The conclusion characterizes the approach as a robust solution supported by theoretical analyses and extensive experiments.Source: §6 · p. 9
  • A remaining scope boundary is the need for in-depth privacy analysis and privacy-preserving mechanisms for data security.The paper identifies these topics as future research while focusing its current evaluation on performance and efficiency.Source: §6 · p. 9

A. Related Work

  • Existing federated PEFT and gradient-compression methods reduce communication but can sacrifice accuracy or provide insufficient compression for high-dimensional LLM parameters.PEFT may compromise accuracy, while conventional compression methods are not tailored to the scale of full-parameter LLM tuning.Source: A. Related Work · p. 13

6. Conclusion

  • Ferret combines shared randomness with first-order federated optimization to improve the scalability of full-parameter LLM tuning.Shared randomness reduces communication, while first-order local updates address the computational inefficiency of zeroth-order methods.Source: §6 · p. 9, A. Related Work · p. 13

A. Related Work

  • Shared-randomness methods transmit compact information but typically use computationally expensive zeroth-order local updates, limiting their scalability.Ferret addresses this gap by integrating shared randomness with first-order federated learning.Source: A. Related Work · p. 13

B.1. Proof of Thm. 1

The proof establishes the stated random-basis properties by sampling each element independently from a truncated normal distribution and applying the resulting distributional definitions.

B.1. Proof of Thm. 1

  • The truncated-normal sampling provides the distributional quantities used to conclude the proof.The supplied proof passages identify the PDF and CDF of the standard normal distribution evaluated at the truncation boundary.Source: §B.1 · p. 13
  • Each element of the random basis is independently sampled from the specified truncated normal distribution.The proof then uses the corresponding probability density and cumulative distribution functions for its analysis.Source: §B.1 · pp. 13–14

B.2. Proof of Thm. 2

These proof sections establish reconstruction-related bounds using matrix concentration, norm properties, and smoothness assumptions. The analysis also explains why bounded random bases are needed for controlling reconstruction error.

B.2. Proof of Thm. 2

  • The proof of Theorem 2 applies Matrix Bernstein’s inequality to independent, zero-mean symmetric matrices with bounded operator norms.Triangle inequalities and shared operator-norm properties of the outer products supply intermediate bounds.Source: §B.2 · pp. 14–15
  • Truncated-normal sampling ensures bounded random-basis norms, which is crucial for obtaining a bounded reconstruction error.This is the stated reason for using truncated rather than standard normal sampling.Source: §B.2 · p. 15

B.3. Proof of Thm. 3

  • Theorem 3’s proof uses β-smoothness and the reconstruction-related bounds to control the analyzed update quantities.The proof also relies on norm bounds for the sampled vectors and standard inequality steps.Source: §B.3 · p. 15
  • The proof relates FedKSeed’s scalar gradient to an approximation of a directional derivative corresponding to Ferret’s projected update.The comparison replaces the local update with the gradient in the projected-update expression.Source: §B.3 · p. 15

B.4. Proof of Prop. 1

  • The remaining proposition proof concludes after applying the preceding reconstruction result.The supplied passage only states that the argument thereby reaches its conclusion.Source: §B.4 · p. 15

B.5. Proof of Prop. 2

The appendix derives a block-wise reconstruction bound and convergence analysis, then describes the experimental setup used to compare federated tuning methods on two datasets and model scales.

B.5. Proof of Prop. 2

  • The block-wise bound is looser than the vanilla reconstruction bound, but experiments report comparable performance between the two approaches.The looser bound arises from an inequality used in the derivation.Source: §B.5 · p. 16

C.1. Experimental Setup

  • The experiments compare Ferret with eight federated tuning baselines, including PEFT, full-parameter, zeroth-order, and FedAvg-based methods.The comparison includes FedPTuning, FedPrompt, FedIT, FedIT-SGD, FedAvg, FedZO, FedMeZO, and FedKSeed.Source: §C.1 · p. 20

C.1.1. SETUP ON THE NATURAL INSTRUCTION AND DOLLY-15K DATASETS

  • Experiments use Natural Instructions and Dolly-15K with non-IID client distributions, Rouge-L evaluation, and DataJuicer-1.3B and LLaMA-3B base models.Natural Instructions uses 738 training tasks and 119 test tasks, while Dolly-15K distributes tasks among 200 clients with label-distribution skew.Source: §C.1.1 · p. 20
  • The experimental inputs are formatted with Alpaca prompt templates for both datasets.The corresponding templates are reported in the appendix tables.Source: §C.1.1 · p. 20, Table 7 · p. 20

C.1.2. SETUP ON THE CODEALPACA AND GSM8K DATASETS

Experiments extend Ferret to code generation and mathematical reasoning with larger LLaMA models, while evaluating computational and communication efficiency. Across these settings, Ferret maintains near-FedAvg accuracy and substantially reduces computational and communication costs.

C.1.2. SETUP ON THE CODEALPACA AND GSM8K DATASETS

  • Ferret is evaluated on CodeAlpaca and GSM8K with LLaMA2-7B and LLaMA2-13B to assess scalability for code generation and mathematical reasoning.CodeAlpaca uses language-based client partitions, while GSM8K uses three IID client subsets.Source: §C.1.2 · pp. 20–21

C.2. Calculation of Computational Cost and Communication Overhead

  • Computational cost is calculated as local update plus global aggregation, with projected gradients aggregated and reconstructed for Ferret and FedKSeed.FedZO and FedAvg instead average raw gradients during global aggregation.Source: §C.2 · p. 21

C.3. More Comparison on LLaMA3-8B and Qwen2.5-7B Models

  • Ferret achieves near-FedAvg performance across additional CodeAlpaca and GSM8K experiments on LLaMA3-8B and Qwen2.5-7B while reducing communication overhead.Table 9 reports final-round Rouge-L means and standard deviations across four random seeds.Source: §C.3 · pp. 21–22, Table 9 · p. 22

C.4. More Comparison of Computational Cost and Communication Overhead

  • On LLaMA2-13B with GSM8K, Ferret reduces FedKSeed costs by 9.8× for local updates, 3.9× for global aggregation, and 4.5× overall.These results concern computational cost and show scalability even for the larger model.Source: §C.4 · p. 22, Table 10 · p. 22
  • Ferret reduces communication costs by 107× versus FedAvg on LLaMA2-13B while using 12 rounds for Ferret and FedAvg versus 40 for FedZO and FedKSeed.Communication counts include exchanged 16-bit raw or projected gradients and aggregated gradients.Source: §C.2 · p. 21, §C.4 · p. 22

C.5. Ablation Studies on Reconstruction

Reconstruction ablations support Ferret’s theoretical design: reconstruction accuracy scales with dimension and improves over zeroth-order estimation, while block-wise reconstruction reduces complexity without sacrificing accuracy.

C.5. Ablation Studies on Reconstruction

  • 1/ρ follows O(d) as dimension d increases, confirming the dimension-rate claim associated with Theorem 2.Figure 6 directly evaluates the rate of 1/ρ with respect to d.Source: §C.5 · p. 22, Figure 6 · p. 23
  • Ferret’s reconstruction accuracy exceeds the zeroth-order method, especially at optimal ϵ = 0.1, while both improve approximately linearly as K increases.The comparison uses cosine similarity and supports the stated implications of Theorems 2 and 3.Source: §C.5 · p. 22
  • Ferret maintains reconstruction accuracy as gradient-descent iterations T increase, whereas the zeroth-order method declines because its error accumulates over iterations.This comparison uses cosine similarity for a 5 × 10^4-dimensional gradient.Source: §C.5 · p. 23
  • Both Ferret and the zeroth-order method show the same approximately linear accuracy rate as dimension d becomes large.The result aligns with the dimension dependence predicted by Theorem 2.Source: §C.5 · p. 23
  • Increasing the number of blocks L reduces projection and reconstruction time while preserving consistent reconstruction accuracy.This supports the computational-complexity benefit predicted for block-wise reconstruction.Source: §C.5 · pp. 23–24, Figure 8 · p. 23

C.6. More Ablation Studies on Convergence and Generalization

Ferret converges faster with improved local optimizers while retaining comparable generalization performance.

C.6. More Ablation Studies on Convergence and Generalization

  • Using Adam instead of SGD for local updates accelerates Ferret’s convergence while maintaining comparable generalization performance.The result supports Ferret’s adaptability to different local optimizers.Source: §C.6 · p. 24

Related papers

Loading 2409.06277v3…