PaperlayerView exact revision on arXiv

Source-linked AI summary

EMMA: Structured Episodic Memory for Multi-Agent Workflow Synthesis

Ruijia Niu, Kaiwen Liu, Rose Yu, Kristina Klinkner

arXiv:2610.35966v1cs.MAcs.AI

TL;DR

Specialized-domain agents need procedures beyond general model capability, but existing approaches either incur costly, hard-to-inspect training, require expert system design, or leave multi-agent update scope to the backbone. EMMA treats multi-agent configurations as episodic memory and assigns each diagnosed failure a single guarded edit within its owning scope. It reports the highest mean performance on every benchmark and split, with first place in 17 of 20 settings and the fewest training executions.

  • Problem

    Existing multi-agent evolution lets the backbone choose where and how extensively to change a system, while specialized tasks require procedures that general models lack.

    Source: §1 · pp. 1–2

  • Method

    EMMA treats routing memory, inheriting agents, and procedural skills as episodic memory, assigning each diagnosed failure to a scope for one guarded edit.

    Source: §1 · p. 2

  • Results

    EMMA achieves the highest mean on every benchmark and split across five backbones, ranks first in 17 of 20 settings, and uses the fewest training executions.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    Scoped, guarded updates keep reflection focused on information provided by task experience rather than superficial structural modifications or destructive rewrites.

    Source: §1 · p. 2

    Accepted edits persist even after unsolved attempt chains, so ineffective or harmful entries may remain despite no observed systematic degradation.

    Source: §6 · p. 9

Abstract

from arXiv · show

Deploying LLM agents in specialized domains often require fine-grained task-specific procedures and knowledge a general model lacks, yet parameter training is costly and non-interpretable, and hand-engineered agent systems shift the cost to expert labor. Episodic memory updates at context level, but single agent adaptation struggles to alleviate cross-task interference, especially with complex tasks such as multi-application navigation or data pool analysis. Although multi-agent evolution alleviates this problem, existing methods lack clear definitions on update rules and rely on the backbone model to decide where and how much the system changes. This ill-scoped evolution makes learning follow the backbone's structural preferences rather than the observed failures: one backbone accumulates skills while another spawns agents. We present EMMA (structured Episodic Memory for Multi-Agent workflow synthesis), which treats the configuration of a multi-agent system (controller routing memory, specialists inheriting a generalist prompt, and per-step procedural skills) as episodic memory. EMMA makes the ownership and admissibility of updates explicit: a reflector assigns each diagnosed failure to the scope owning its correction and proposes one edit within it, and backbone-independent guards keep every accepted edit scoped, incremental and bounded. Across KramaBench, AppWorld and BBEH with five backbones and four baselines (ACE, GEPA, EvoMAS and ADAS), EMMA attains the best overall performance on every benchmark, up to 32.5% over the strongest baseline, maintains consistent improvements across backbones, and uses 17-38% fewer training executions than the most efficient baseline.

1 Introduction

Specialized agent tasks need procedures that general models lack, while parameter training is costly and hand-engineered systems require expert labor. EMMA treats multi-agent configurations as episodic memory and constrains failure-driven updates; it reports the best mean performance on every benchmark and fewer training executions than the baselines.

1 Introduction

  • Parameter training is costly and difficult to inspect or selectively correct, while hand-specified agent systems transfer the cost to expert labor.Episodic memory offers an alternative by persisting task experience as text while keeping model weights fixed.Source: §1 · p. 1
  • Multi-agent evolution leaves the choice of where and how extensively to change the system to the backbone, producing backbone-dependent structural preferences.The paper contrasts skill accumulation with repeated agent creation when updates are unconstrained.Source: §1 · p. 2, Figure 1 · p. 2
  • EMMA formulates multi-agent workflow synthesis as episodic learning over controller routing knowledge, inheriting agents, and procedural skills.The controller selects planning knowledge, agents, and skills while executing one step at a time.Source: §1 · p. 2
  • Each diagnosed failure is assigned to an owning scope and corrected by one guarded edit, keeping updates scoped, incremental, and bounded.The guards preserve existing prompt content and bound memory growth.Source: §1 · p. 2
  • EMMA achieves the best mean performance on every benchmark, ranks first in 17 of 20 settings, improves over the strongest baseline by up to 32.5%, and uses the fewest training executions.The results are reported across model families.Source: §1 · p. 2

2 Preliminaries

The preliminaries define episodic learning with fixed backbone parameters and a memory updated from task execution and evaluation feedback. In the orchestrated workflow, a controller plans and dispatches individual steps to sub-agents, then replans using their outputs.

2 Preliminaries

  • The task setup holds backbone parameters fixed and uses a gate to return a binary verdict and a short digest explaining a failure.The controller, sub-agents, and reflector share the same backbone parameters.Source: §2 · p. 2

2.1 Episodic Learning

  • Episodic learning updates an external textual memory from task experience rather than updating model parameters, and the memory is frozen during held-out evaluation.Where and how much the update changes memory is a design choice when learning uses no gradient.Source: §2.1 · p. 3

2.2 Multi-Agent Workflow with Orchestration

  • The controller interleaves planning and execution, assigning each step a subtask, agent, skill subset, and curated context before replanning from the returned output.Because each step depends on prior history, the full plan is not committed in advance, and skills are selected per step.Source: §2.2 · p. 3

3 EMMA: Structured Episodic Memory for Multi-Agent Workflow Synthesis

EMMA treats a multi-agent configuration as episodic memory, with separate levels for routing, agent behavior, and procedures. Its controller builds step-wise plans, while training diagnoses failures and applies one guarded edit at a time.

3 EMMA: Structured Episodic Memory for Multi-Agent Workflow Synthesis

  • EMMA stores the multi-agent configuration in persistent memory, writes it during training, and uses it to guide task execution.Training and inference procedures are described in Sections 3.2 and 3.3.Source: §3 · p. 4, §3.1 · p. 4

3.1.1 Hierarchical Episodic Memory

  • The controller memory stores indexed rules for task decomposition, agent and skill dispatch, and verification; the controller selects relevant entries rather than retrieving by similarity.Loading no entries is valid because irrelevant memory can degrade planning.Source: §3.1.1 · p. 4
  • Memory separates task routing, generalist-inheriting agent prompts, and short procedures for classes of sub-problems into distinct levels.The controller sees skill indexes and assigns skills per step; sub-agents receive the full skill text.Source: §3.1.1 · p. 4

3.1.2 Step-Wise Synthesis with Overflow Decomposition

  • The controller dispatches an ordered plan one step at a time, revising remaining steps from outputs and passing each sub-agent a curated summary of prior work.When a step overflows its context or round budget, the controller splits it into smaller steps over existing agents, preserves completed work, and records the overflow for reflection.Source: §3.1.2 · p. 4

3.2 Training Procedure

  • Training runs each task through a rollout, correctness gate, failure diagnosis, one edit, and a re-run before moving to the next task.A correct answer ends the chain; after the revision budget is spent, no untested final edit is made.Source: §3.2 · p. 4, §3.2.1 · pp. 4–5, Figure 3 · p. 5

3.2.1 Structured Reflection

  • The reflector localizes and classifies a failure, generalizes its correction, then minimizes it to one action within the scope that owns it.One-off execution slips are not stored; behavior changes go to prompts, procedural corrections to skills, and routing corrections to controller memory.Source: §3.2.1 · p. 5

3.2.2 Guarded Update

  • Backbone-independent guards accept only scoped, incremental edits, preserve most existing prompts, bound memory growth, prevent answer leakage, and constrain repeated changes.A refused proposal is retried, with persistent refusals defaulting to no change; revised skills and agents must be exercised in the re-run.Source: §3.2.2 · p. 6

3.3 Training and Inference

  • Training makes one fixed-order pass over the training set, then freezes one memory artifact per benchmark and backbone for held-out inference.Inference retains memory selection, replanning, and overflow decomposition, but does not call the reflector or write memory.Source: §3.3 · p. 6

4 Related Work

Related work covers hand-designed multi-agent systems, automated system design and evolution, and episodic or procedural memory. EMMA differs by storing a multi-agent configuration and constraining updates to one guarded edit within a diagnosed scope.

4 Related Work

  • Hand-designed systems establish role specialization and orchestration, but remain task-specific and carry coordination costs.Examples include ChatDev [Qian et al., 2024], MetaGPT [Hong et al., 2024], AutoGen [Wu et al., 2024], and Magentic-One [Fourney et al., 2024].Source: §4 · p. 6
  • Automated design and evolution methods search programs or configurations, but let the backbone propose whole candidates rather than scope each change to a diagnosed correction.The passage names ADAS [Hu et al., 2025], AFlow [Zhang et al., 2025a], EvoAgent [Yuan et al., 2025], and EvoMAS [Hu et al., 2026].Source: §4 · p. 6
  • Episodic and procedural memory methods improve agents through feedback, stored entries, or skill libraries, usually for a fixed single ReAct agent.EMMA instead stores a multi-agent configuration and adds a level hierarchy with deterministic guards.Source: §4 · p. 6

5 Experiments

Experiments compare EMMA with four persistent-artifact baselines across three benchmarks and five backbones. EMMA has the highest mean on every benchmark and ranks first in 17 of 20 benchmark–backbone settings.

5.1 Experimental Setup

  • Evaluation covers KramaBench and BBEH-mini accuracy and AppWorld task-goal and scenario-goal completion across normal and challenge splits; five backbones share a backbone within each run.The baselines are ACE, GEPA, EvoMAS and ADAS.Source: §5.1 · pp. 6–7

5.2 Experimental Results

  • EMMA leads all three benchmarks: +7.0 accuracy points on KramaBench, +5.8 on BBEH-mini, and +11.9/+17.4 TGC/SGC on AppWorld normal and +6.4/+5.3 on challenge.The AppWorld gains are over ACE; the KramaBench and BBEH-mini gains are over GEPA.Source: Table 2 · p. 7, Table 3 · p. 7, §5.2 · p. 7

5.2.1 Consistency Across Model Families

  • Across every benchmark, EMMA’s weakest-backbone result exceeds every baseline’s weakest-backbone result; on KramaBench and AppWorld normal, it also exceeds every baseline’s five-backbone mean.The weakest-backbone scores are 58.5 on KramaBench, 61.2 TGC on AppWorld challenge, and 56.3 on BBEH. On AppWorld challenge, EMMA’s margin over the strongest baseline per backbone is +0.2 TGC with Opus 5 and +15.4/+16.6 with GLM-5/DeepSeek v3.2; on BBEH it is +2.4 with Opus 5 and +9.0/+10.9 with GLM-5/Kimi-K2.5.Source: §5.2.1 · pp. 7–8

5.2.2 Update Efficiency

  • EMMA achieves the best mean performance with the fewest training executions on every benchmark, using 17–38% fewer executions than the most efficient baseline.EMMA re-runs only failed tasks, with each re-run testing one localized, guarded edit.Source: Table 4 · p. 8, §5.2.2 · p. 8

5.3 Ablation Studies

  • Learning procedural skills accounts for the larger memory-level gain; learning agents adds a further gain from 0.591 to 0.647, while controller memory’s contribution is not isolated.All ablations use KramaBench with Claude Opus 4.5; full EMMA attains 0.647.Source: §5.3 · p. 8
  • Test-time reflection and cross-task self-evolution variants remain well below EMMA, but their differing feedback access and adaptation splits prevent isolating persistence alone.The comparison supports transfer from learned memory rather than persistence by itself.Source: §5.3 · pp. 8–9
  • In the BBEH Sonnet 5 case study, one failed zebra puzzle yielded a constraint-satisfaction skill, a specialist agent and a controller routing rule; subsequent and held-out puzzles were solved.A held-out puzzle was solved and verified unique in a single dispatched step.Source: §5.3 · p. 9

6 Conclusion and Limitations

EMMA treats multi-agent configuration as episodic memory and updates it through scoped, guarded edits. It achieves the highest mean performance across the reported benchmarks and splits, while retaining limitations across backbone families and unsolved attempts.

6 Conclusion and Limitations

  • EMMA achieves the highest mean performance on every benchmark and split across five backbones, ranks first in 17 of 20 settings, and uses the fewest training executions.The results indicate that learning follows training feedback rather than backbone structural preferences.Source: §6 · p. 9
  • EMMA treats controller routing memory, inheriting agents, and procedural skills as episodic memory, updating them through one guarded edit within each diagnosed failure’s scope.The guards are backbone-independent.Source: §6 · p. 9
  • The performance gap between backbone families remains, and accepted edits may leave ineffective or harmful entries when an attempt chain ends unsolved.The authors observed no systematic degradation and identify cross-verification as future work.Source: §6 · p. 9

A Algorithms

The algorithms keep the backbone fixed while training updates external memory through task rollouts and guarded reflection. The loop also records overflow and can revise execution plans before reflecting on failures.

A Algorithms

  • The training loop changes only memory M while keeping θ fixed, with one pass and one attempt chain per task and at most one guarded edit per failed attempt.The seed memory contains one generalist and empty controller and skill stores; tasks are processed in fixed order and memory is snapshotted for evaluation.Source: A Algorithms · p. 13
  • For each task, the loop rolls out and gates attempts, reflects on failures, applies an edit and directive, then updates the controller and snapshots memory.The controller update is capped at two entries per task, whether the attempt passes or fails.Source: A Algorithms · p. 13
  • ROLLOUT selects a controller, plans agent-subtask-skill steps, and returns an answer, trace, and overflow evidence; the plan can be re-split and resumed at the same step after overflow.The plan may also be revised as execution proceeds.Source: A Algorithms · p. 13
  • REFLECTGUARDED receives task and execution context and returns an action, payload, and scope; reflection localizes, classifies, generalizes, and minimizes a correction before guard checks.After up to three rejected proposals, it returns no change.Source: A Algorithms · pp. 13–14

B Configuration

The configuration section distinguishes fixed, pre-registered memory-editing and execution-budget settings from the reflector’s action vocabulary and ordered guards.

B Configuration

  • The configuration table lists pre-registered constants governing memory editing and per-task execution budgets; changing a constant defines a new condition rather than a tuning.Model weights are never updated.Source: B Configuration · p. 14, Table 6 · p. 14
  • The reflector’s action vocabulary is listed separately in Table 7.Source: B Configuration · p. 14, Table 7 · p. 14
  • Guards are checked in a fixed order, covering scope-action compatibility, strategy changes, leakage, prompt preservation, and bounded memory growth.Source: B Configuration · p. 14

C Benchmarks and Splits

The evaluation uses disjoint training and evaluation tasks, with benchmark-specific split procedures for KramaBench, AppWorld, and BBEH-mini.

C Benchmarks and Splits

  • All training tasks are disjoint from evaluation tasks, and systems use the same split; a fingerprint-verifying loader checks pinned split sources.Table 8 summarizes the splits.Source: C Benchmarks and Splits · p. 15, Table 8 · p. 15
  • KramaBench’s 104 tasks are divided within each domain and difficulty group by assigning every fifth identifier-sorted task to training.The benchmark defines no split, and Table 9 gives per-domain counts.Source: C Benchmarks and Splits · p. 15, Table 9 · p. 15
  • AppWorld uses its unmodified test_normal and test_challenge splits, trains on 90 tasks, and evaluates TGC and SGC with state-based unit tests.Source: C Benchmarks and Splits · p. 15
  • BBEH-mini splits 23 categories of 20 examples into four training and 16 evaluation examples per category under seed 42, making macro and micro accuracy coincide.Source: C Benchmarks and Splits · p. 15

D Baseline Methods

The baselines learn persistent artifacts from training feedback and freeze them for evaluation, but differ in what they evolve and how they select updates.

D Baseline Methods

  • All four main-comparison baselines learn persistent artifacts from training feedback and freeze them for evaluation.DACode, smolagents DeepResearch, and Reflexion appear only in the Section 5.3 ablation.Source: D Baseline Methods · p. 15
  • ACE incrementally edits a playbook for a fixed ReAct agent, using helpful or harmful bullet tags and validation accuracy to select the frozen version.Its feedback is answer correctness on KramaBench and, on AppWorld, unit-test pass/fail and the trace without ground truth.Source: D Baseline Methods · p. 15
  • GEPA rewrites prompt components of a compound system from training mini-batch evaluations and selects survivors using a validation-task Pareto front.Its feedback combines the benchmark metric with execution traces.Source: D Baseline Methods · p. 15
  • EvoMAS searches complete multi-agent configurations through meta-model selection, mutation, and crossover, retaining successful configurations in an experience memory.Its main-comparison reward subtracts a token-and-latency cost from the dataset evaluator, and its best configuration and memory are frozen.Source: D Baseline Methods · p. 15
  • ADAS uses a meta-agent to write complete agent programs, evaluates them on the training split, and archives programs to condition later proposals.It keeps the best archived program, or reports the hand-written seed when no discovered program beats it for the specified AppWorld backbones.Source: D Baseline Methods · p. 15

D.1 Update Constraints in Learned Baselines

The baseline comparison distinguishes EMMA's failure-typed, bounded edits from learned systems that leave the update's locus or magnitude largely to the backbone. The reflector prompt operationalizes diagnosis, scope assignment, and minimal generalization before selecting an action.

D.1 Update Constraints in Learned Baselines

  • ADAS checks proposals for novelty and executability, while EvoMAS limits mutations to one component type; neither bounds change magnitude or types edits by diagnosed failure.The comparison also reports that neither method checks for leakage, while ACE's incremental delta bullets and embedding de-duplication operate at a single context level.Source: §D.1 · p. 16, Table 10 · p. 16

E Reflector Prompt

  • The prompt assigns distinct homes to controller routing knowledge, agent working habits, procedural skills, and new agent roles, with different actions for each.Plan revisions are non-durable and apply only to the current task attempt.Source: E Reflector Prompt · p. 16
  • The reflector first localizes the failure and classifies it into one scope, mapping execution slips or routing failures to plan or memory actions and procedural gaps to skills.Missing capabilities can call for a skill or, when its own method and context are needed, an agent.Source: E Reflector Prompt · p. 16
  • Corrections must generalize beyond the current task and use the smallest action at the lowest level that owns the lesson.Artifacts and plan directives must not contain the task's answer or verbatim task spans.Source: E Reflector Prompt · p. 16
  • Overflow in two consecutive attempts requires a structural response rather than a prompt or skill edit.The specified choices are creating an agent or revising the plan to change how work is split.Source: E Reflector Prompt · p. 17

F What Training Produces: Learned Artifacts

A Sonnet 5 BBEH failure chain produced a procedural skill, a specialist, and a routing rule; the resulting workflow solved a held-out zebra puzzle with verified uniqueness. The example shows how task-specific corrections can be retained across memory levels and reused on later tasks.

F What Training Produces: Learned Artifacts

  • On a held-out 52-clue zebra puzzle, the controller selected 8 of 101 entries and one csp_solver step with the skill produced a solution verified unique in 9 rounds.The task completed in 82.3 seconds after all 52 clues were checked.Source: F What Training Produces: Learned… · pp. 18–19, Table 11 · p. 19
  • A failed training puzzle led to the zebra-csp-encode-and-verify skill, then overflow prompted creation of csp_solver, and an end-of-task update added a routing rule.The next three zebra training tasks passed on their first attempt, and none of the three entries named a training-puzzle entity or answer.Source: F What Training Produces: Learned… · pp. 18–19
  • The csp_solver specialization's recorded prompt instead directs large CSP work into checkpointed phases, including batches of constraints saved between phases.This contrasts with the held-out example's single-step routing rule for large zebra puzzles.Source: F What Training Produces: Learned… · p. 18
  • The learned procedural skill calls for uniquely named category values and verified helper functions for relational patterns before encoding clues.It directs the agent to test helpers on a small example before applying them to all clues.Source: F What Training Produces: Learned… · p. 19
  • For an infeasible model, the skill treats the result as a possible encoding bug and calls for auditing the unsatisfiable core before re-solving the full model.After a satisfiable solve, it verifies uniqueness by excluding the found assignment and solving again.Source: F What Training Produces: Learned… · p. 19
Loading 2610.35966v1…