PaperlayerView exact revision on arXiv

Source-linked AI summary

Federated In-Context LLM Agent Learning

Panlong Wu, Kangshuo Li, Junbao Nan, Fangxin Wang

arXiv:2412.08054v1cs.LGcs.AIcs.CLcs.CR

TL;DR

FICAL addresses the challenge of training federated LLM agents when private data, heterogeneous clients, and costly model-parameter exchange limit collaboration. It exchanges LLM-generated knowledge compendiums and uses RAG-based tool learning to teach agents from an aggregated global compendium. The paper reports competitive performance against SOTA baselines with a 3.33 × 10^5-times reduction in communication cost.

  • Problem

    Federated LLM-agent training must handle private, heterogeneous data while limiting substantial bandwidth and computation demands from model-parameter exchange.

    Source: Introduction · pp. 1–2

  • Method

    FICAL uses an LLM-enhanced KCG module to generate privacy-preserving tool-use knowledge compendiums and a RAG-based TLU module to teach agents from their aggregation.

    Source: Introduction · p. 2, FICAL · pp. 4–5

  • Results

    3.33 × 10^5 times lower communication costs are reported, with competitive performance against other SOTA baselines.

    Source: Introduction · p. 2, Conclusion · p. 7

  • Takeaways & Limitations

    FICAL demonstrates a federated in-context learning approach for training diverse LLM agents without transmitting model parameters or private data samples.

    Source: Introduction · p. 2, Conclusion · p. 7

    The problem setting assumes each client owns a local dataset of tool-using instances and may access only particular tool domains.

    Source: Problem Statement · p. 4

Abstract

from arXiv · show

Large Language Models (LLMs) have revolutionized intelligent services by enabling logical reasoning, tool use, and interaction with external systems as agents. The advancement of LLMs is frequently hindered by the scarcity of high-quality data, much of which is inherently sensitive. Federated learning (FL) offers a potential solution by facilitating the collaborative training of distributed LLMs while safeguarding private data. However, FL frameworks face significant bandwidth and computational demands, along with challenges from heterogeneous data distributions. The emerging in-context learning capability of LLMs offers a promising approach by aggregating natural language rather than bulky model parameters. Yet, this method risks privacy leakage, as it necessitates the collection and presentation of data samples from various clients during aggregation. In this paper, we propose a novel privacy-preserving Federated In-Context LLM Agent Learning (FICAL) algorithm, which to our best knowledge for the first work unleashes the power of in-context learning to train diverse LLM agents through FL. In our design, knowledge compendiums generated by a novel LLM-enhanced Knowledge Compendiums Generation (KCG) module are transmitted between clients and the server instead of model parameters in previous FL methods. Apart from that, an incredible Retrieval Augmented Generation (RAG) based Tool Learning and Utilizing (TLU) module is designed and we incorporate the aggregated global knowledge compendium as a teacher to teach LLM agents the usage of tools. We conducted extensive experiments and the results show that FICAL has competitive performance compared to other SOTA baselines with a significant communication cost decrease of $\mathbf{3.33\times10^5}$ times.

Introduction

FICAL addresses privacy, bandwidth, computation, and heterogeneous-data challenges in federated learning for LLM agents by exchanging tool-use knowledge rather than model parameters. It combines LLM-enhanced knowledge-compendium generation with retrieval-augmented tool learning to train agents through in-context learning.

Introduction

  • Federated LLM-agent training is constrained by costly parameter transmission, intensive computation, non-IID client data, and privacy risks from sharing in-context samples.These challenges motivate replacing bulky parameter exchange and direct data-sample aggregation with a privacy-preserving knowledge-sharing mechanism.Source: Introduction · pp. 1–2
  • In-context learning can reduce communication by transmitting natural-language context, but prior approaches risk privacy leakage because aggregation may expose clients’ data samples.The paper identifies privacy-preserving access to diverse client knowledge as an unresolved challenge.Source: Introduction · pp. 1–2
  • The RAG-based TLU module retrieves relevant tool-use knowledge from the aggregated long-context compendium to teach agents how to use tools for queries.TLU addresses the long-context scalability problem that can arise as the number of clients increases.Source: Introduction · p. 2
  • The KCG module distills local tool-use information into compendiums, which the server aggregates and returns to clients as knowledge for learning tool invocation.The global compendium describes tool usage rather than synthetic private data, supporting privacy protection during aggregation.Source: Introduction · p. 2
  • FICAL exchanges privacy-preserving knowledge compendiums instead of model parameters, enabling in-context federated learning for diverse LLM agents.The KCG module generates local compendiums, while the server aggregates them into a global compendium used to teach tool usage.Source: Introduction · p. 2
  • FICAL is reported to achieve O(1) communication complexity with respect to model size, improve accuracy by 7.6% through TLU, and reduce communication costs by 3.33 × 10^5 times.The reported results compare FICAL favorably with other SOTA baselines across different scenarios.Source: Introduction · p. 2

Related Work

Related work covers single-agent LLM learning, resource-efficient in-context learning and RAG, and federated learning methods for LLMs. These strands provide the conceptual and technical context for FICAL.

Single LLM Agent Learning

  • Single-agent LLM learning has explored domain-specific planning, memorization prediction, and iterative feedback-based output refinement.The cited approaches use gradient-based prompt optimization, equi-compute memorization analysis, or a single LLM for generation, refinement, and feedback.Source: Single LLM Agent Learning · p. 2

Resource efficient LLM Learning

  • Resource-efficient LLM learning includes fine-tuning on symbolic in-context labels, similarity-based example selection, and retrieval-augmented generation.Related RAG work also studies self-reflection and recursive clarification for reducing unnecessary information or resolving ambiguity.Source: Resource efficient LLM Learning · pp. 2–3

FL with LLMs

  • Federated LLM research has examined parameter-efficient tuning, differential-privacy adaptations, gradient-free prompt optimization, and sub-foundation-model alignment.These methods target resource consumption, privacy, communication cost, or performance under federated settings.Source: FL with LLMs · p. 3

Motivation

The motivation is that LLM parameter growth is outpacing communication and computation advances, making conventional federated training costly. FICAL therefore transmits distilled tool-use knowledge that can teach agents without sharing private samples.

Motivation

  • LLM parameter growth is increasing faster than communication bandwidth and computation power, making conventional federated LLM training difficult to scale.Figure 1 compares parameter-size growth with successive generations of communication technology and computation devices.Source: Motivation · p. 3
  • 6.4 h to 36 h is required to transmit selected popular LLMs’ parameters across devices at a typical 100 Mbps rate.The transmission-time comparison illustrates the communication burden of parameter sharing in federated learning.Source: Motivation · p. 3, Table 1 · p. 3
  • FICAL replaces private data-sample transmission with client-generated knowledge compendiums containing distilled instructions for using tools.The aggregated global compendium serves as a teacher for LLM agents, analogous to teaching tool-use skills rather than supplying raw examples.Source: Motivation · pp. 3–4

Methodology

FICAL replaces repeated model-parameter exchange with privacy-preserving knowledge-compendium sharing, then uses retrieval-augmented context to teach distributed LLM agents tool usage.

Problem Statement

  • FICAL assumes clients hold private tool-using instances from potentially different tool domains, reflecting heterogeneous access across industries.Source: Problem Statement · p. 4

Preliminaries on In-Context Learning

Preliminaries on Retrieval Augmented Generation

Traditional FL

  • Traditional FL repeatedly transmits client model parameters to a server for aggregation and returns the resulting global model across communication rounds.Source: Traditional FL · p. 4

FICAL

  • FICAL generates local knowledge compendiums from private datasets with an LLM-enhanced KCG module, preserving tool-use knowledge while avoiding direct parameter exchange.Source: FICAL · p. 4
  • The server concatenates client compendiums into a global compendium, which clients store in a vector database and query through the RAG-based TLU module.Source: FICAL · p. 5

Experiments and Analyses

The experiments compare FICAL with other methods across client-scale settings, including tool-using accuracy and multiple-toolset data. The supplied materials identify Figure 4 and Table 2 as comparison visuals.

Experiments Settings

The evaluation uses simulated tool-use data with multiple toolsets and resource-conscious LLaMA3-8B training settings. FICAL is compared with established federated optimization and parameter-tuning baselines.

Datasets

  • Experiments use simulated interactions across Public API toolsets, with tool-use traces judged by DeepSeek-v2.The default setting uses five clients and five toolsets with 4-bit NF4 quantization and LoRA-based baselines.Source: Datasets · p. 6
  • All experiments use LLaMA3-8B, while baselines train for 50 communication rounds with five local epochs per round.Source: Datasets · p. 6

Baselines

  • FICAL is evaluated against FedACG, FedDecorr, FedLoRA, FedAdam, FedYogi, and FedProx.These baselines cover federated optimization, representation regularization, and federated language-model tuning approaches.Source: Baselines · pp. 6–7

Performance Evaluation

FICAL substantially reduces communication overhead while maintaining competitive or superior tool-use accuracy across heterogeneous tool ownership, quantization levels, client counts, and RAG settings.

Comparison of Communication Resource Consumption

  • 3.33 × 10^5 times less communication resource consumption distinguishes FICAL from other baselines.FICAL uses one communication round, and its overhead remains constant as model size expands, unlike parameter-sharing methods.Source: Comparison of Communication Resource… · p. 7

Results when each client has one toolset’s data

  • 57.61% accuracy makes FICAL highest under one unique toolset per client, versus 26.09%–43.48% for other baselines.The baselines trail FICAL by 14.13%–31.52% in this setting.Source: Results when each client has one… · p. 7

Results when each client has multiple toolsets’ data

  • When each client owns two toolsets, FICAL achieves accuracy gains of 13.09%, 14.02%, 16.82%, 9.35%, 14.02%, and 13.09% over other baselines, demonstrating robustness across heterogeneous tool-data ownership.Source: Results when each client has multiple… · p. 7

Results at different quantization level

  • 71.74% and 53.24% are FICAL’s reported accuracies under 8-bit quantization with five and eight clients, respectively.Under 4-bit quantization, the corresponding accuracies are 57.61% and 44.60%; overall 8-bit performance is higher.Source: Results at different quantization level · p. 7

Impact of Number of clients

Performance comparison with and without RAG

Conclusion

FICAL combines privacy-preserving knowledge compendiums with RAG-based tool learning for federated LLM agents, achieving competitive performance and sharply lower communication cost.

  • FICAL transmits aggregated knowledge compendiums rather than model parameters and achieves competitive performance with a 3.33 × 10^5-times communication-cost decrease.Its KCG module generates local compendiums, while the RAG-based TLU module teaches agents to use tools.Source: Conclusion · p. 7

AAAI Reproducibility Checklist

The checklist reports that the paper provides formal theoretical descriptions, empirical evaluations, dataset documentation, and implementation details. Source code and novel datasets are planned for public release, although preprocessing code, experimental code appendices, and a data appendix are not included.

Checklist scope

  • The checklist instructs that information in the paper or explicitly referenced technical appendices should be treated as satisfying each question, with further explanations placed in a reproducibility checklist.Source: AAAI Reproducibility Checklist · p. 9

Method documentation

  • The methods include conceptual outlines or pseudocode, distinguish opinions and hypotheses from facts and results, and provide pedagogical background references.Source: AAAI Reproducibility Checklist · p. 9

Theoretical contributions

  • The paper makes theoretical contributions and reports clear assumptions, formal novel claims, proofs, intuitions, citations, and empirical demonstrations.Experimental code used to eliminate or disprove claims is not included.Source: AAAI Reproducibility Checklist · p. 9

Dataset documentation

  • The paper relies on datasets, motivates the selected datasets, cites existing datasets, and indicates that publicly available datasets can be obtained from the literature.Novel datasets are not included in a data appendix but are planned for public release; unavailable datasets are marked not applicable.Source: AAAI Reproducibility Checklist · p. 9

Computational experiments

  • The paper includes computational experiments and states that evaluation metrics, run counts, variation analyses, significance tests, final hyperparameters, and development searches are reported.The checklist confirms computational experiments but does not provide their specific tasks or results.Source: AAAI Reproducibility Checklist · p. 9

Implementation reproducibility

  • The paper specifies computing infrastructure, randomness-seeding procedures, implementation comments, and planned public release of experimental source code, but omits code appendices.Preprocessing code and source code required for conducting and analyzing experiments are not included in the appendices.Source: AAAI Reproducibility Checklist · p. 9
Loading 2412.08054v1…