Source-linked AI summary
AgensFlow: A Coordination-Policy Substrate for Multi-Agent Systems
Nicole Koenigstein
TL;DR
Static multi-agent pipelines cannot reliably handle interacting coordination choices that vary across task regimes and operating constraints. AgensFlow treats coordination as online policy learning under partial observability, making routing decisions observable and auditable across repeated trajectories. Learned coordination improves quality on coordination-heavy classes, reduces exploration cost with warm starts while preserving plateau quality, and remains bounded by the evaluated topology and shared feature taxonomy.
Problem
Interacting choices about skills, models, topology, retrieval, verification, and omission are difficult to fix a priori, while static pipelines and isolated comparisons cover only a limited design space.
Method
AgensFlow learns an inspectable routing policy over abstract folded signatures and actions including skills, model-role bindings, topology choices, and termination under partial observability.
Results
Learned routing improves quality on coordination-heavy classes, while warm-started coordination priors reduce exploration cost without degrading plateau quality.
Takeaways & Limitations
Learned, auditable routing makes coordination choices observable and improvable across repeated task regimes rather than hiding them inside fixed workflows.
Takeaways & Limitations
The reported evaluation covers only a linear-with-skip topology and transfer across two corpora sharing a common feature taxonomy.
Abstract
from arXiv · showhide
Multi-agent systems built on large language models (LLMs) require many coordination choices that are difficult to fix a priori: which skill protocol to invoke, which agent role should perform a subtask, which model to bind to each role, how roles should interact, when to use retrieval or verification, and when to omit a step entirely. These choices interact with task regime and operational constraints, so static pipelines and one-off model comparisons provide only a limited view of the design space. This paper introduces AgensFlow, an open-source framework that treats multi-agent coordination as an online policy-learning problem under partial observability. The framework makes coordination decisions observable and learnable from repeated trajectories, rather than treating skill, role, model, topology, and evaluation choices as fixed pipeline design. AgensFlow is evaluated on two corpora: distributed-systems incident tasks and security-advisory tasks. The evaluation shows three main results: learned routing reaches a higher-quality operating point than a fixed pipeline baseline on coordination-heavy classes; skip:X isolates topology compression as a meaningful part of the substrate; and warm-started policy graphs can reduce exploration cost while preserving plateau quality. Overall, the results support that learned, auditable routing can improve coordination-heavy multi-agent workflows over static wiring.
1 Introduction
Multi-agent systems require dynamic coordination across task signatures, skills, models, and topology because these choices interact and vary by regime. AgensFlow addresses this by learning inspectable routing policies from repeated trajectories rather than relying on static pipelines.
- Multi-agent systems distribute work across specialized roles, parallel exploration, task decomposition, and verification steps.
- Learned routing improves most on coordination-heavy classes while producing ties or narrow trades on procedural or out-of-corpus cases.
- Coordination choices span task signatures, skill protocols, model bindings, and topology decisions, including whether to omit steps.
- These axes interact across task regimes, making isolated model comparisons and fixed benchmarks inadequate for learning a reliable operating policy.
- AgensFlow makes skill selection, model-role assignment, topology choice, and reward audit observable and learnable from repeated trajectories.
2 Related Work
Prior work studies reasoning traces, conversation-based coordination, tool selection, context-dependent module activation, and trajectory evaluation. AgensFlow differs by learning a persistent, auditable routing policy over a joint coordination action surface.
- Reasoning-oriented methods connect intermediate reasoning with tool and environment actions, but AgensFlow treats reasoning patterns as selectable skill cards rather than universally preferred execution modes.
- Conversation-based systems mediate coordination through free-form transcripts, whereas AgensFlow uses structured handoffs over a typed schema to support reusable, inspectable experience.
- Learned tool-use research selects APIs, memory access, or execution strategies; AgensFlow instead learns a persistent routing policy over broader coordination choices.
- The policy jointly spans skill protocols, model-role bindings, optional topology cells, and termination, with skip:X making omission a first-class topology decision.
- AgensFlow extends context-dependent expert coordination from specialized modules toward task-regime-conditioned multi-agent routing.
- Relative trajectory evaluation compares multiple rollouts against an explicit rubric to provide a reward signal sensitive to coordination quality, recovery, and evidence use.
3 Preliminaries: Coordination as Policy Learning
AgensFlow models multi-agent orchestration as sequential policy learning over an observable, belief-conditioned abstraction of partially observed task states. Its router chooses state-dependent coordination actions, learns from delayed trajectory rewards combining judged quality with operational penalties, and stores reusable, auditable policy statistics.
- State as Folded Task Signatures: AgensFlow constructs a folded signature from task context, structured handoff state, and belief estimates because key task properties are not directly observable.The hidden state includes intent, difficulty, evidence quality, model reliability, and intermediate reasoning quality.
- State as Folded Task Signatures: The signature includes a regime label, a binary handoff mask, and discretized estimates of correctness, uncertainty, contradiction risk, and evidence sufficiency.The handoff mask tracks seven fields, while handoff quality is monitored but excluded from the signature to keep the policy graph compact.
- State as Folded Task Signatures: Coarser belief bins increase value sharing but risk state aliasing, whereas finer bins preserve distinctions while requiring more data for reliable routing.Repeated trajectories sharing a signature allow the policy graph to learn reusable behavior across related task regimes rather than memorizing prompts.
- Actions: Skills, Models, and Topology: At each step, the router selects one legal action, observes the resulting handoff update, recomputes the signature, and continues until runtime termination conditions are met.The policy does not commit to a complete trajectory upfront; its action set depends on the activation plan, handoff state, budget, and termination rules.
- Actions: Skills, Models, and Topology: Topology is part of the learned action space because the policy can invoke skill-model pairs, skip scheduled cells, or indirectly cause termination through evaluator invocation.The runtime does not select terminate directly; completion, budget exhaustion, unavailable actions, or governance halts trigger termination implicitly.
- Reward: Quality Under Operational Constraints: The reward combines RelativeJudge trajectory quality with normalized token cost and retry or failure penalties rather than optimizing quality or cost alone.Reported default weights are wq = 1.0, wc = 0.3, and wρ = 0.15, with cost normalized by an 8,000-token cap.
- Learned Object: An Auditable Policy Graph: The learned policy graph records visits, mean reward, reward variance, token statistics, and failure counts for each signature-action pair.Action selection uses reliability-aware UCB1 with annealed exploration, while recorded failure rates downweight repeatedly unreliable actions.
4 Method: The AgensFlow Coordination Substrate
AgensFlow implements coordination as an auditable, online policy over task signatures, variant actions, topology choices, and reliability-aware rewards. Its runtime substrate routes each task, records trajectories, scores outcomes, and updates a folded policy graph.
- Task representation: Task signatures fold regime labels, handoff masks, and bucketed beliefs so recurring task classes can share learned coordination decisions.The framework treats structural features such as ambiguity and verification need as inputs to regime detection rather than separate signature coordinates.
- Action space: The variant pool combines skill protocols with model bindings, while each routing step also permits skip:X actions for optional cells.This makes skill selection, model-role assignment, and topology compression part of the learnable action space.
- Policy learning: Reliability-aware UCB1 selects legal actions by trading off mean reward, exploration, and observed per-edge failure rates.The acquisition rule uses annealed exploration and a failure penalty.
- Auditability: The folded policy graph records selected actions by signature together with visit counts, reward estimates, token statistics, and failure information.This provenance supports post-learning inspection and enables saved graphs to warm-start related domains while retaining adaptation from new trajectories.
- Runtime lifecycle: Each task is gated, represented, routed, executed, state-updated, governed, scored, and backed up into the policy graph before a structured report is emitted.RelativeJudge supplies trajectory scoring, while trace collection and governance provide operational observability.
- Design advantages: The substrate differs from static pipelines by conditioning coordination on task regimes, learning trajectory shape through skips, and exposing skill, model, and reward provenance.These properties are presented as advantages over fixed wiring and one-off model-selection sweeps.
5 Reward and Audit
AgensFlow makes evaluation part of the coordination substrate by producing an online reward and audit signal from relative, rubric-based trajectory comparisons. Cross-judge averaging, axis decomposition, and confidence weighting expose and temper evaluator variability.
- Reward role: The reward layer both supplies a scalar signal for policy-graph backup and exposes information for detecting biased or unstable evaluation.Evaluation is treated as part of the system rather than only an external reporting step.
- Relative evaluation: For each completed trajectory, an LLM judge compares same-class peers side-by-side against an explicit rubric and emits scalar and per-axis scores.Relative ranking is intended to reduce reward-hacking pressure compared with isolated absolute scoring.
- Judge aggregation: Cross-judge averaging combines scores from judges supplied by different model providers and surfaces disagreement as confidence telemetry.The design targets single-judge family bias while retaining per-judge diagnostics.
- Score decomposition: The default rubric decomposes reward across goal achievement, grounding, coordination, and recovery using weighted axis scores.The axis weights compose the scalar score used by the substrate.
- Uncertainty handling: Confidence weighting multiplies reward backups by judge confidence, reducing the influence of noisy observations on value estimates.Confidence is computed from disagreement standard deviation.
6 Empirical Findings
AgensFlow is evaluated across distributed-systems and security-advisory corpora using fixed action and reward interfaces, with cold-start, no-skip, baseline, and warm-start arms. Learned routing outperforms fixed wiring on coordination-heavy classes, while warm-starting reduces exploration cost and reward audits qualify single-judge interpretations.
- Experimental setup: The evaluation spans two 60-task corpora with shared feature, action, and reward interfaces, enabling cross-domain transfer testing.The corpora cover distributed-systems incidents and synthetic security advisories.
- Experimental setup: Four security-advisory arms compare a fixed seven-cell pipeline, no-skip learning, cold-start learning, and warm-start learning from a 443-node policy graph.The warm-start graph was learned on the distributed-systems corpus.
- Topology and reward: The cold-start trajectory initially compresses cost as skip:X usage rises, then rebalances after quality drops, consistent with reward-adjusted skip decisions.The skip:X action makes omission a topology choice, while cross-judge auditing tests reward sensitivity.
- Main comparison: Under cross-family audit, the main substrate beats the fixed baseline at plateau epochs 6 to 8 and wins on 5 of 8 scenario classes.It ties on C6 and trades narrowly on C1 and C5.
- Per-class results: The largest audited gains occur on C3 (+0.18), C8 (+0.16), and C7 (+0.13), where cross-document synthesis and selective verification matter most.Procedural classes C1 and C6 are statistical ties because running every cell is adequate.
- Routing behavior: The learned policy moves off the default solver on 7 of 8 classes, adapting variants to task structure and risk rather than choosing cheaper variants uniformly.It retains the default on C7, where incorrect answers carry the highest cost.
- Warm-start transfer: Warm-starting reduces exploration cost while preserving audited plateau quality, with approximately 10% fewer plateau tokens and 21% fewer full-run tokens.Audited quality lifts are smaller than single-judge lifts: +0.012 at plateau and +0.003 across the full run.
7 Discussion
The discussion frames AgensFlow’s central object as an auditable routing policy over repeated task regimes rather than a fixed workflow. This reframes production selection around coordination policies under observable regimes and operating constraints, while retaining the need to audit rewards.
- Implication: The coordination surface is too large, regime-dependent, and judge-sensitive to navigate reliably by intuition alone.Skills, models, retrieval, verification, skipping, reward design, and cost constraints interact across task regimes.
- Implication: AgensFlow makes routing choices observable and improvable from reward without replacing product judgment or removing the need for reward auditing.The framework logs trajectories, aggregates repeated observations, and exposes skill, model, topology, and evaluation choices.
8 Limitations
The reported evaluation covers only a linear-with-skip topology with single-judge live reward. Broader topology classes, feature taxonomies, planning regimes, and cross-judge reward settings remain for future study.
- The experiments exercise only a linear-with-skip topology with single-judge live reward.
- Alternative parallel, branching, swarm-style, and hierarchical coordination topologies were not exercised.
- Transfer was tested across two corpora sharing a common feature taxonomy, leaving transfer under different taxonomies untested.
- Future work should evaluate broader planning regimes, alternative topology classes, live cross-judge reward, and reward-signal effects under different deployment conditions.
9 Conclusion
AgensFlow learns multi-agent coordination policies online across interacting task, skill, model, and topology choices, with validation across domains. The results indicate that learned coordination transfers to a structurally novel corpus, while audited warm-start priors reduce exploration cost without degrading plateau quality.
- AgensFlow learns coordination policies online from repeated reward signals over task signatures, skills, models, and topology choices.
- Cross-domain validation shows substrate-versus-baseline quality gains concentrated in classes where coordination matters most.
- Cross-family auditing changes the magnitude of warm-start quality advantages while preserving reduced exploration cost and plateau quality.
- The framework and reported evaluation policy graphs are released open-source and reproducible end-to-end from the repository.