PaperlayerView exact revision on arXiv

Source-linked AI summary

Federated Agent Optimization

Qiang Yang, Zhiqiang Kou, Xueyi Zhang, Dong-Dong Wu, Hanlin Gu, Jing Guo, Yang Liu, Di Jiang, Qian Xu

arXiv:2610.01195v1cs.AI

TL;DR

Agent experience is valuable but often cannot be shared across private environments, and parameter-focused federated learning does not cover the broader capabilities of agents. The paper formulates FAO as controlled collaboration across agent components, with a framework for transferring experience and examining its challenges. It concludes that FAO can support systems that benefit from collective experience while preserving local specialization and organizational control.

  • Problem

    Agent experience is distributed across private environments, and conventional federated learning focuses on parameters rather than the broader capabilities that agents use.

    Source: §1 · p. 2

  • Method

    The paper formulates FAO as controlled information exchange and organizes collaboration across agent components and stages for turning local experience into transferable capabilities.

    Source: §1 · pp. 2–3, §5.1 · p. 10

  • Results

    The paper establishes a framework, taxonomy, and improvement process for FAO and identifies challenges in objective trade-offs, fairness, and evaluation.

    Source: §1 · p. 3, §6 · p. 12, §8 · p. 14

  • Takeaways & Limitations

    FAO is presented as a foundation for systems that benefit from collective experience while preserving local specialization and organizational control.

    Source: §8 · pp. 14–15

    Task relatedness and knownness are heuristics, not necessary conditions for transfer; distant tasks may share strategies, and known items may add robustness.

    Source: §5.3 · p. 11

Abstract

from arXiv · show

Large language model (LLM) agents increasingly operate in private environments and accumulate valuable experience from task execution, tool use, feedback, and local knowledge. Yet such experience is distributed across organizations and cannot be directly shared because of privacy and proprietary constraints. Conventional federated learning is insufficient for this setting, as agent capabilities extend beyond model parameters to memory, tools, rewards, skills, and structured knowledge. In this paper, we formulate \textbf{Federated Agent Optimization (FAO)}, which studies how distributed agents can collaboratively improve through controlled information exchange while keeping raw data, complete trajectories, and private knowledge local. We define FAO as a multi-objective problem balancing agent utility, privacy leakage, and communication cost, and organize its optimization space across policy, memory, tool use, reward, and structured knowledge and skills. We further characterize how private experience can be abstracted, protected, aggregated, and adapted into transferable capabilities, providing a unified view of how agents can benefit from one another without direct experience sharing. Finally, we identify the key challenges of FAO and outline several promising directions for future research toward trustworthy federated agent systems.

1. Introduction

Agent experience can improve task planning, tool use, and decisions, but privacy and proprietary constraints keep complementary experience isolated. The paper frames Federated Agent Optimization (FAO) as a broader collaborative framework than parameter-focused federated learning.

1. Introduction

  • Private experience—including workflows, trajectories, tool records, and feedback—could complement agents facing related tasks, but often cannot be shared directly.It remains local because it is coupled to user data, internal knowledge, business rules, tools, and operational workflows.Source: §1 · p. 2
  • Conventional federated learning exchanges model updates, but agent capabilities also depend on policies, memory, tool use, feedback, and local knowledge.The paper identifies a need to transform useful experience, skills, and feedback into shareable capabilities without exposing private information.Source: §1 · p. 2
  • FAO studies controlled information exchange that improves agents while keeping raw data, complete trajectories, and private knowledge decentralized.Its objectives are local utility, privacy leakage, and communication and system costs; collaboration should benefit individual participants, not only aggregate performance.Source: §1 · p. 2
  • The paper presents a framework, capability taxonomy, and experience-transfer process, then identifies challenges and future directions for federated agents.Its framework covers abstraction, protection, aggregation, adaptation, participant fairness, and evaluation of federation gains.Source: §1 · pp. 2–3

2. Related Work

Related work spans federated learning, multi-agent and agent-learning systems, and privacy-preserving AI. The paper distinguishes FAO by extending collaboration beyond model parameters while keeping agent experience and knowledge local.

Federated Learning and Federated Large Models

  • Federated learning and federated large-model methods mainly optimize parameters or representations, whereas agents also involve memory, tools, rewards, skills, and external knowledge.Federating a large model is therefore not equivalent to federating an agent.Source: §2 · p. 3

Multi-Agent Systems and Agent Reinforcement learning

  • Multi-agent and agent-learning methods explore coordination and capability improvement, but often assume agents can directly exchange messages, trajectories, or feedback.Privacy constraints and data isolation across organizations or private environments are rarely considered in that work.Source: §2 · pp. 3–4

Privacy-Preserving AI

  • Privacy-preserving AI protects data, parameters, and inference, while agent systems also expose sensitive information through trajectories, memories, tool interactions, rules, and skills.The paper calls for protection mechanisms tailored to collaborative agent optimization.Source: §2 · p. 4

3. Problem Definition: Federated Agent Optimization

The paper defines FAO for clients with local agents, task distributions, private data, and utilities, then specifies a federated protocol and a multi-objective optimization problem. It distinguishes this setting from parameter-only learning through heterogeneous agent artifacts, privacy risks, and related special cases.

3. Problem Definition: Federated Agent Optimization

  • FAO considers K clients drawing tasks from a shared task space, with client-specific distributions, utilities, and private datasets that remain local.The datasets include raw records used to build agent memory and knowledge.Source: §3 · p. 4

3.1 Local Agents and Private Data

  • A local agent comprises a policy, memory, callable tools, evaluator, and structured knowledge such as ontologies, business or environment rules, and skills.Policies may use continuous weights or adapters, or discrete prompts and workflow graphs; structured knowledge is explicit rather than retrieved by similarity.Source: §3.1 · p. 4

3.2 Federated Protocol

  • Each protocol round selects participants, applies local updates, aggregates typed component updates into views, and lets clients adapt their agents.Update blocks can be empty, and only typed artifacts leave clients; aggregation may produce shared or personalized views.Source: §3.2 · p. 5, Figure 1 · p. 5

3.3 Multi-Objective Formulation

  • FAO jointly considers client-level performance, privacy leakage, and communication cost, whose competing objectives require choosing among Pareto-optimal protocols.Leakage is assessed on the upload mechanism through a differential privacy guarantee or strongest-attack advantage under a stated threat model.Source: §3.3 · pp. 5–6
  • Unlike parameter-only federation, FAO updates may be text or structures that must be merged, and readable artifacts can disclose records verbatim.Utility comes from trajectories with sparse feedback, so discrete components lack gradients and continuous components have noisy gradient estimates.Source: §3.3 · p. 6
  • Positive-weight scalarization reaches only supported Pareto-optimal protocols, while the constrained form is the one guaranteed to reach every Pareto-optimal protocol in this nonconvex decision space.The formulation also encompasses experience pooling, agent optimization, multi-agent systems that pool raw experience, and methods federating one component.Source: §3.3 · p. 7, §3.4 · p. 7

4. Taxonomy of Federated Agent Optimization

The taxonomy groups federated methods by the agent component shared, spanning policies, memory, tool use, rewards, and structured knowledge and skills. Its comparison highlights substantial payload variation and the privacy and communication trade-offs of sharing different artifacts.

4. Taxonomy of Federated Agent Optimization

  • The taxonomy classifies methods by the typed update component they share, distinguishing agent-level methods from precursors that federate the same component outside an agent.The categories overlap, and Table 1 presents the taxonomy.Source: §4 · p. 7, Table 1 · p. 8
  • Policy sharing averages adapters with client weights for open models or merges textual prompts for closed backbones, producing one shared view.Prompts are small but readable by receivers and may quote private examples.Source: §4 · pp. 7–8
  • Memory sharing keeps raw trajectories local, distills them into items such as insights or workflows, then condenses and retrieves task-matched items.Text items can quote private content, so uploads may require auditing or differentially private generation.Source: §4 · p. 8
  • Tool-use sharing combines comparable tool documentation with call and success counts, from which the coordinator derives merged documentation and a success rate.Only summed counts are needed for secure aggregation; shared descriptions can disclose internal systems and may carry malicious instructions.Source: §4 · pp. 8–9
  • Reward sharing can jointly train a shared evaluator and local head, but the surveyed methods do not federate evaluators of agent trajectories.Existing methods federate preference or reward models, keep rewards local while federating policies, or use client models as referees.Source: §4 · p. 9
  • Structured-knowledge approaches exchange entity or relation embeddings, categorized rules, or skill-library patches, with personalization in some methods.Embeddings can reveal known entities, while skill exchanges can reveal how a client conducts business.Source: §4 · p. 9

Comparison.

  • ≈76 MB for policy updates, 0.08 to 20 MB for tool knowledge, and 0.1 to 0.3 MB for structured knowledge and skills illustrate payloads differing by roughly three orders of magnitude.The cited examples cover adapters of rank 8 per round, tool knowledge per run or round, and skill patches per run.Source: Comparison · p. 9
  • Text and rule exchanges can keep trajectories local without formal privacy mechanisms, and no surveyed method trades off performance, privacy, and communication across components.The paper identifies this cross-component trade-off as the objective of its FAO formulation.Source: Comparison · p. 9

5. How Agents Mutually Improve Each Other

The section traces how local experience becomes protected, transferable knowledge and how recipient agents may benefit from it. It describes semantic update stages, functional knowledge types, and safeguards and heuristics relevant to mutual improvement.

5.1 The Improvement Loop

  • A semantic update abstracts a sender’s local evidence into an artifact, protects it, and carries it through aggregation and recipient adaptation.The abstraction level trades artifact detail against size, leakage, and reach; the levels are jointly chosen as part of Φ.Source: §5.1 · p. 10
  • The sender-side predicate tests whether evidence exhibits a pattern, nmin sets its support threshold, and sanitization removes identifiers or adds calibrated noise.The section frames abstraction as a central design choice because transferable knowledge includes text, rules, and skills, not only parameters or logits.Source: §5.1 · p. 10

5.2 Five Types of Shared Knowledge

  • The five shared-knowledge types differ by function, can interact through aligned ontologies, and may enter a recipient’s memory, prompt, or evaluator.In the running example, banks use agents to verify identity but cannot share customer records.Source: §5.2 · p. 10, Table 2 · p. 11

5.3 When Is the Improvement Mutual?

  • Conditions favoring transfer are heuristics rather than necessary rules: distant tasks may share strategies, and known items may still add robustness.Heterogeneity can obstruct transfer while also creating value; clustered aggregation addresses relatedness, and federated reinforcement learning tolerates task differences but not differences in environment dynamics.Source: §5.3 · p. 11
  • A negative-transfer gate accepts an item only when its estimated gain exceeds twice an error bound, supporting individual rationality but not guaranteeing pairwise mutual improvement.Contribution-linked incentives can encourage clients to contribute rather than only adopt; clients retain the ability to personalize shared capabilities.Source: §5.3 · p. 11

6. Key Challenges

FAO raises coupled optimization, fairness, and evaluation challenges beyond conventional federated learning. The section argues that credible collaboration requires accounting for heterogeneous clients, privacy and communication costs, and local outcomes over time.

6.1 Performance, Privacy and Communication Trade-offs

  • FAO’s performance, privacy, and communication objectives are coupled, so richer updates may aid transfer while increasing exposure and communication, and stronger protection or compression may constrain optimization.Data locality alone does not eliminate leakage; updates, memories, skills, and structured artifacts can expose information, while secure aggregation can constrain validation.Source: §6 · p. 12, §6.1 · p. 12

6.2 How to Ensure Fairness in Federated Agent Systems?

  • Federation-wide utility gains can coexist with unequal client outcomes, negative transfer, and uneven burdens, making fairness depend on contributions, benefits, and costs.Clients need the ability to personalize, selectively adopt, or reject shared capabilities whose local value does not justify their cost or risk.Source: §6.2 · pp. 12–13

6.3 How to Evaluate Federated Agent Systems?

  • Credible evaluation is difficult because agent outcomes depend on interacting reasoning, retrieval, tools, and environments, while drift and erroneous contributions can undermine transfer over time.The paper calls for protocols combining controlled client heterogeneity, explicit sharing boundaries, repeated optimization, and credible local baselines.Source: §6.3 · p. 13
  • Evaluation should report more than aggregate task utility, including privacy leakage, communication and system costs, client-level gains, contribution fairness, negative-transfer rates, and robustness to drift.The supplied discussion notes that existing agent and federated benchmarks cover different aspects and do not yet provide these dedicated FAO protocols.Source: §6 · p. 12, §6.3 · p. 13

7. Future Works

Trustworthy FAO requires collaboration to be evaluated and designed across execution reliability, privacy and security, fairness, credible evaluation, and lifecycle cost. The paper identifies these as linked research questions for collaborative learning among autonomous agents across organizations.

7. Future Works

  • Transferred capabilities may depend on tools, permissions, task distributions, or workflow contexts that differ across agents, so applicability conditions and execution constraints matter.The paper also calls for modeling capability dependencies and composition rather than assuming transferred capabilities remain valid.Source: §7 · p. 13
  • Raw data locality alone does not eliminate leakage from shared parameters, memories, textual abstractions, tool knowledge, or structured representations; confidentiality, update integrity, and quality validation must be considered together.The paper identifies differential privacy, secure aggregation, and robust aggregation as useful foundations.Source: §7 · pp. 13–14
  • Heterogeneous agents may contribute unequal useful experience, receive unequal benefits, or suffer negative transfer, motivating measures of contribution, collaboration gain, and participation cost.The paper calls for benefits commensurate with the information, privacy, and resources contributed.Source: §7 · p. 14
  • Reliable evaluation should compare recipient agents with credible local baselines and assess recipient-level gains, negative transfer, and robustness under drift across repeated collaboration.The paper notes challenges from sparse feedback, long trajectories, and stochastic environments.Source: §7 · p. 14
  • Communication volume alone misses lifecycle costs from adaptation, privacy protection, validation, inference, tool execution, storage, synchronization, and maintenance.The paper calls for models covering the resources needed to produce, protect, aggregate, validate, deploy, and maintain shared capabilities.Source: §7 · p. 14
  • Trustworthy FAO should jointly address execution reliability, privacy and security, fairness, credible evaluation, and lifecycle cost.The paper presents these questions as a principled basis for collaboration among autonomous agents across organizations.Source: §7 · p. 14

8. Conclusion

The conclusion presents FAO as a framework for collaborative improvement across agent capabilities under distributed data and local constraints. It also points to notation summarized in Table 3 for Sections 3–5.

8. Conclusion

  • FAO frames collaborative improvement across policies, memories, tool use, rewards, and structured knowledge while data and local constraints remain distributed.The paper connects local experience to transferable capabilities and recipient-side gains, and identifies performance, privacy, communication cost, fair allocation, and reliable verification as central challenges.Source: §8 · pp. 14–15

A. Notation

B. Methods that Federate a Component of an Agent

The appendix extends Section 4 with a list of agent-level methods and selected precursors, followed by category descriptions and their single-agent foundations.

B. Methods that Federate a Component of an Agent

B.1 Federated Policy Optimization

The appendix surveys federation across five agent components, describing what is shared and how it is aggregated. The approaches have distinct trade-offs in communication, privacy, compatibility, and evaluation.

B.1 Federated Policy Optimization

  • Policy federation ranges from parameter and adapter aggregation to prompt sharing; adapters remain leakage-prone, while textual prompts are readable and may expose source examples.Agent methods include local-trajectory fine-tuning and reinforcement learning, while prompt sharing can work with frozen API backbones.Source: §B.1 · pp. 37–38

B.2 Federated Memory Optimization

  • Memory federation shares abstractions such as reflections, insights, workflows, or reasoning strategies while raw trajectories stay local; coordinators cluster, deduplicate, merge, and rank textual items.Text is a distinct leakage channel, and privacy protection must act on the shared content.Source: §B.2 · p. 38

B.3 Federated Tool-Use Optimization

  • Tool federation shares compendiums, schema-validated artifacts, or aggregated usage statistics; tool descriptions can expose internal systems and create an attack surface for malicious instructions.The cited typed-artifact study found its simple retrieval baseline outperformed its language-model routing variants and said the benchmark could not measure shared-routing benefits.Source: §B.3 · p. 39

B.4 Federated Reward Optimization

  • Reward federation has no method for federating trajectory evaluators yet; client utilities differ, so a single global evaluator may not suit any client.One proposed low-cost direction has a coordinator propose a reward function or rubric for client scoring against private outcomes.Source: §B.4 · pp. 39–40

B.5 Federated Structured Knowledge and Skill Optimization

  • Structured-knowledge federation covers embeddings, abstract environment rules, and skill-library patches; ontology methods assume entities and relations are aligned beforehand.An aligned ontology can support other categories, but is expensive to build and changes slowly; skills can reveal how a client conducts its business.Source: §B.5 · pp. 40–41

B.6 Comparison

The comparison shows trade-offs among communication, privacy, backbone access, and maturity across federated agent methods. The running example distinguishes transferable knowledge types and explains how they support local improvement and formulation extensions.

B.6 Comparison

  • Reported communication payloads range from tens of megabytes per round for adapters to a few hundred kilobytes per run for skills, but studies use different granularities.The category order is not fixed; accumulated text can grow large, while parameter updates can be compressed to a few kilobytes.Source: §B.6 · p. 41
  • Text- and rule-sharing methods keep raw trajectories local but lack formal privacy mechanisms, shifting concern toward disclosure of shared content.Two methods support this concern with empirical reconstruction-attack audits.Source: §B.6 · p. 41
  • Weight sharing requires open weights and a common architecture, while other federated components can work across different closed models; maturity varies by component.Federated fine-tuning and knowledge-graph embedding are established, while federated rewards for agent trajectories remain open and methods for memory, tools, rules, and skills are early.Source: §B.6 · p. 41

C. Knowledge Types in the Running Example

  • In the running bank-verification example, customer records and conversation logs remain local while agents check documents and registry information.Applications may pass, fail, or be escalated to a human reviewer.Source: C. Knowledge Types in the Running… · pp. 41–42

C.1 The Four Types in the Running Example

  • Experience is shared as generalized patterns rather than cases, while skills are reusable, tool-parameterized procedures that are more executable but disclose more business process.Client-side and coordinator-side distillation turn local trajectories and uploaded summaries into a shared insight library.Source: §C.1 · p. 42
  • Reward knowledge provides trajectory-judgment signals, ontologies align concepts across organizations, and rules encode learned effects of actions.The examples include a non-compliant verification dialogue, conditional concept equivalence, and escalating after repeated failed codes.Source: §C.1 · p. 42
  • Ontology alignment can ease transfer of insights, skills, and rules, while reward evaluation selects sender trajectories and helps receivers assess adopted items.Knowledge types are not ordered by abstraction, and a federation without aligned concepts or a reliable evaluator has more difficulty distinguishing useful knowledge from noise.Source: §C.1 · p. 43

C.2 From Shared Knowledge to Local Improvement

  • A shared artifact can improve multiple receiver components because knowledge types and the components that store or use them have a many-to-many relation.An insight, for example, may enter memory, prompts, policy training, or evaluator examples.Source: §C.2 · p. 43

E. Proofs

The proofs establish a relationship between weighted and constrained formulations of the objectives, then give a probabilistic guarantee for gated adaptation under stated sampling assumptions. The included notation and tables map the framework's symbols, methods, and knowledge types to agent components.

Proofs

  • Every maximizer of the positively weighted objective is Pareto optimal, and every Pareto-optimal protocol maximizes the constrained form at its own privacy and communication budgets.The constrained form bounds each client's leakage, but its converse may yield only weak Pareto optimality.Source: E. Proofs · pp. 43–44
  • Under the stated independent validation-sampling assumptions, gated adaptation yields nonnegative collaboration gain with probability at least 1 − η.The proof bounds validation estimates across at most 2^m candidate agents using Hoeffding's inequality and a union bound.Source: E. Proofs · p. 44
  • The notation table covers symbols for clients, tasks, agents, protocols, objectives, privacy, communication, collaboration gain, adaptation, and evaluation.The listed quantities include the number of validation tasks, error bound, and failure probability.Source: E. Proofs · p. 45
  • Table 4 presents agent-level FAO methods and selected precursors as instances of the typed update of Eq. (5), with an asterisk indicating a preprint.Source: Table 4 · p. 46
  • Table 5 maps knowledge types in rows to receiving-agent components in columns, and one shared artifact may improve several components.The relation between shared knowledge and receiver components is many-to-many.Source: Table 5 · p. 47

Related papers

Loading 2610.01195v1…