PaperlayerView exact revision on arXiv

Source-linked AI summary

EpiCon: Collective Agent Learning through Co-Evolving Multimodal Memory

Ziyun Zeng, Hang Hua, Shaden Alshammari, Rogerio Feris, William T. Freeman, Jiebo Luo

arXiv:2609.37923v1cs.CV

TL;DR

Agents can learn from past executions, but sharing and building on experience across agents remains challenging. EpiCon links multimodal memory refinement within a question to a shared bank of reusable lessons. Across systems, shared experience supports improved solving while host model parameters remain fixed.

  • Problem

    Agents produce useful experience, but enabling different systems to reuse and build on it remains challenging.

    Source: §1 · p. 1

  • Method

    EpiCon uses a memory controller to co-evolve textual guidance and visual evidence, and a tree self-organizer to accumulate and retrieve shared experience without changing host parameters.

    Source: §1 · pp. 1–2

  • Results

    Contributions from another harness raise the original system’s macro-average score by 2.1–3.2 points, while frozen-bank reuse improves other systems with a single attempt.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Agents can build on one another’s experience through shared external multimodal memory while host model parameters remain fixed.

    Source: §6 · p. 10

Abstract

from arXiv · show

Agents can learn from past executions, but enabling different agents to reuse and build on one another's experience remains challenging. We introduce EpiCon, a shared multimodal memory framework for agent collective learning without updating host model parameters. EpiCon links question-level memory evolution to a persistent experience bank through two independently trained 2B models: a memory controller and a tree self-organizer. The controller jointly refines textual guidance and visual evidence across attempts and selectively includes visual memory. The self-organizer consolidates lessons hierarchically and retrieves experience and rules for new problems. We evaluate EpiCon on eleven benchmarks spanning four multimodal task domains, using two harnesses and multiple backbones. A frozen bank improves other systems even with a single solving attempt. A second harness raises the original system's macro-average score by 2.6 points across eleven benchmarks. Across four host configurations, EpiCon improves macro-average scores by 1.7 to 4.9 points over No Memory and reduces memory-operation time by 67\% to 74\% relative to backbone-sized memory models.

1 INTRODUCTION

EpiCon addresses how agents can retain and share useful experience across tasks and systems without changing host model parameters. It couples question-level multimodal memory refinement with a shared bank of reusable lessons.

1 INTRODUCTION

  • EpiCon responds to the challenge of making experience useful across agents and systems as tasks accumulate, while leaving host model parameters unchanged.The shared memory is intended to support reuse across different harnesses and backbones.Source: §1 · p. 1
  • 2.1–3.2 points: contributions from another harness raise the original system’s macro-average score, while frozen-bank reuse improves other systems with a single attempt.The evaluation covers two harnesses and multiple backbones.Source: §1 · p. 2
  • The Memory Controller jointly refines textual guidance and visual evidence across attempts and selectively includes visual memory, while the Tree Self-Organizer consolidates and retrieves lessons.The two modules operate through a dedicated memory harness and are independently trained 2B models.Source: §1 · pp. 1–2
  • 1.7–4.9 points: the 2B variant improves macro-average scores over No Memory across four host configurations, while reducing memory-operation time by 67–74% versus backbone-sized memory models.The 2B models are independently trained; the reported results cover eleven benchmarks across four multimodal domains.Source: §1 · p. 2

2 RELATED WORK

Prior work studies multi-agent coordination, increasingly persistent and structured agent memories, and reflection or self-improvement from execution outcomes. EpiCon connects multimodal memory evolution within a question to reuse across questions and systems.

2.1 MULTI-AGENT SYSTEMS

  • Multi-agent research emphasizes communication and coordination, while persistent memory addresses what agents retain beyond the current exchange.The section distinguishes information flow among agents from information preserved over time.Source: §2.1 · p. 2

2.2 AGENT MEMORY AND MULTIMODAL EXPERIENCE

  • Agent memory has progressed from stored interaction histories to consolidated, hierarchical, and reusable experience, while multimodal agents preserve perceptual evidence alongside higher-level knowledge.EpiCon jointly refines textual guidance and visual evidence and shares the resulting experience across systems.Source: §2.2 · pp. 2–3

2.3 REFLECTION AND SELF-EVOLVING AGENTS

  • Reflection and self-evolving work spans retries, critique, and the consolidation of successful or failed experience into reusable lessons or persistent memory.EpiCon separates multimodal memory evolution within a question from historical experience reuse, which it evaluates with a single solving attempt.Source: §2.3 · p. 3

3 DATA CURATION

The curation process builds supervision for EpiCon’s memory controller and tree self-organizer from multi-agent task executions. It screens proposed memory updates and tree-operation demonstrations before training.

3 DATA CURATION

  • Qwen3.8-27B proposes memory updates, lessons, and tree-operation demonstrations; Qwen3.8-Flash-Next refines unsuccessful updates and corrects tree-operation targets.The refinement stage also improves lesson quality.Source: §3 · p. 3
  • Training tasks draw on six datasets and cover visual mathematics, document understanding, and reasoning over charts and infographics.MAS executions provide records used to construct memory updates, lessons, and tree-operation supervision.Source: §3 · pp. 3–4
  • Approximately 25K controller examples come from joint text–visual updates screened by replaying previous and proposed memories with the same host.Repair transformations turn failed attempts into successful ones; Compress transformations preserve success while shortening text without increasing crop area.Source: §3 · p. 4
  • Approximately 6.5K tree-operation demonstrations pass schema, node-reference, partition, and provenance checks, but individual operations are not validated by downstream replay.The demonstrations cover placement, merging, splitting, consolidation, and retrieval, including valid empty retrievals.Source: §3 · p. 4

4 METHODOLOGY

EpiCon uses a dedicated memory harness and two independently trained 2B models to connect question-level multimodal memory updates with a persistent experience bank. The bank accumulates, organizes, and retrieves lessons for reuse across tasks and agent systems.

4 METHODOLOGY

  • EpiCon pairs a dedicated memory harness with a Memory Controller and Tree Self-Organizer, while leaving the host MAS orchestration and backbone parameters unchanged.The harness coordinates memory updates, retrieval, visual cropping, and storage.Source: §4 · p. 4

4.1 SHARED MULTIMODAL MEMORY

  • A persistent bank accumulates accepted lessons and revisions for reuse within or across MAS, and historical-memory evaluations freeze the bank and disable question-level updates.At maintenance steps, the harness validates and executes the organizer’s proposals; each step may process multiple lessons together.Source: §4.1 · pp. 4–5

4.2 QUESTION-LEVEL MULTIMODAL MEMORY EVOLUTION

  • The controller jointly revises actionable text and supporting image regions from prior memory, the current question and images, the latest answer, and its execution trace.It can retain, replace, or remove visual evidence; a gate controls whether available visual memory enters the next attempt, without deleting stored evidence.Source: §4.2 · pp. 5–6

4.3 SHARED EXPERIENCE CONSOLIDATION AND REUSE

  • The Tree Self-Organizer groups lessons hierarchically, consolidates supported shared guidance into reusable rules, and selects applicable rules, concrete experiences, or no memory for a new task.The harness validates and executes proposed placement, merging, splitting, and consolidation operations; retrieval uses a frozen text encoder to find candidates.Source: §4.3 · p. 6

5 EXPERIMENTS

EpiCon’s experiments test performance, experience transfer, memory ablations, and generalization across multimodal benchmarks, harnesses, and backbones. Results report gains from shared memory, including transfer between systems and benefits from compact memory models.

5.1 EXPERIMENTAL SETUP

  • Evaluation covers 4,538 questions from 11 benchmarks across four multimodal domains, using two harnesses and multiple backbones.Historical-memory evaluations use one solving attempt, with model parameters and banks frozen; construction questions are disjoint from evaluation questions under the project-defined splits.Source: §5.1 · p. 7

5.2 MAIN RESULTS

  • Backbone-sized EpiCon achieves the highest eleven-benchmark macro-average in all four host configurations, beating each strongest external memory baseline by 1.9 to 5.9 points.The reported comparison uses a single solving attempt per question.Source: §5.2 · p. 7
  • The 2B variant improves macro-average scores over No Memory by 1.7 to 4.9 points while reducing memory-operation time by 67% to 74%.Compared with backbone-sized EpiCon, its macro-average scores are lower by 0.9 to 3.6 points; total MAS-plus-memory time falls by 25% to 35%.Source: §5.2 · p. 8

5.3 EXPERIENCE TRANSFER AND MEMORY EXPANSION

  • A frozen bank improves macro-average scores by 3.2 points in backbone transfer and 4.2 points in harness transfer, without first building a local bank.The bank built with Codex and Qwen3.8-27B is reused with Codex and Gemma4-31B or DeepSeek-Harness and Qwen3.8-27B.Source: §5.3 · p. 8
  • Cross-harness bank evolution raises macro-average scores from 53.5 to 55.7 for Codex and from 53.4 to 56.6 for DeepSeek-Harness.The evolved banks improve eight and nine of eleven benchmark scores, respectively, though individual task results vary with the contributing system.Source: §5.3 · p. 8

5.4 MEMORY ABLATIONS

  • With historical retrieval disabled, the trained controller improves all four scores under both harnesses, while evolving visual memory improves seven scores and ties one.Adaptive injection improves seven of eight scores with either frozen or evolving visual memory; with evolving memory it also reduces MAS execution time by approximately 25% for Codex and 17% for DeepSeek-Harness.Source: §5.4 · pp. 8–9, Table 5 · p. 9
  • Tree memory, which organizes and consolidates lessons, improves all scores over a flat bank built from the same source questions.The comparison holds the harness, backbone, attempt budget, and memory-context budget constant.Source: §5.4 · p. 9, Table 6 · p. 10

5.5 GENERALIZATION TO A STRONGER SOLVER

  • With the memory components and bank fixed, EpiCon improves all four benchmark scores for Codex with GPT-5.6-Luna in a single attempt.ParseBench rises from 63.8 to 66.5 without target-specific retraining.Source: §5.5 · p. 9, Table 7 · p. 10

6 CONCLUSION

EpiCon links question-level textual and visual memory co-evolution with shared experience accumulation and reuse. Experiments support cross-system learning through shared external multimodal memory while host parameters remain fixed.

6 CONCLUSION

  • EpiCon connects question-level textual and visual memory co-evolution with the accumulation and reuse of shared experience.The framework is presented as enabling systems to build on one another’s experience without updating host model parameters.Source: §6 · p. 10
  • Another harness can benefit from an existing bank and contribute experience that improves subsequent solving by the original contributor.The conclusion frames these findings as support for agent collective learning through shared external multimodal memory.Source: §6 · p. 10
  • The reported collective learning leaves host model parameters fixed.Source: §6 · p. 10

A.1 SOURCE TASKS AND DATA COLLECTION

The section builds supervision from multimodal task executions, replay-checked memory revisions, and demonstrations of tree operations. It describes how the resulting memory models and benchmark-derived questions support construction of frozen experience banks.

A.1 SOURCE TASKS AND DATA COLLECTION

  • Six source datasets supply questions and images across three domains, while executions from three harnesses provide records for memory supervision.Qwen3.8-27B proposes updates and operation demonstrations; Qwen3.8-Flash-Next selectively revises unsuccessful updates, lesson quality, and tree-operation targets.Source: §A.1 · p. 15

A.2 REPLAY-VERIFIED JOINT MEMORY TRANSFORMATIONS

  • Joint targets pair revised guidance with visual-use choices and image regions, using the question, images, prior memory, attempt, and correctness feedback as inputs.Visual-reference fields are null when visual use is disabled.Source: §A.2 · p. 15
  • Replay retains repairs that turn a failed attempt into success and compressions that preserve success while shortening text without increasing crop area.The original task images remain available during replay, and the final accepted memory, including teacher revisions, becomes the target.Source: §A.2 · p. 15
  • The approximately 25K memory transformations are supervision examples, not independent tasks, and replay checks source-task transitions rather than guaranteeing later-task gains.Each example pairs update inputs with accepted memory JSON.Source: §A.2 · p. 15

A.3 TREE ORGANIZATION AND RETRIEVAL DEMONSTRATIONS

  • Tree-operation demonstrations cover placement, merging, splitting, consolidation, and retrieval; their approximately 6.5K targets receive structural checks but no downstream MAS replay verification.Valid empty retrievals are retained when no candidate applies.Source: §A.3 · p. 15
  • Benchmark-derived construction questions add content to external banks with the models fixed, while evaluation uses separate questions and frozen historical banks.Seed lessons come from controller memories and are organized into small banks before operation demonstrations are generated.Source: §A.3 · pp. 15–16

B TRAINING THE TWO MEMORY POLICIES

Two independently fine-tuned memory models learn separate controller and tree-organization policies from validated JSON targets. The controller handles joint text–visual updates, while the tree model handles organization and retrieval operations.

B TRAINING THE TWO MEMORY POLICIES

  • Two Qwen3.5-2B models are independently fine-tuned on separate supervision datasets for memory control and tree organization.Controller targets cover repair and compression of joint text–visual memory; tree targets cover placement, merging, splitting, consolidation, and retrieval, with images only in retrieval examples.Source: B TRAINING THE TWO MEMORY POLICIES · p. 16
  • Both models use LoRA rank 16 and scaling 32, with three epochs, learning rate 10^-4, sequence length 8,192, batch size 2, and accumulation over 8 steps.The visual encoder and aligner are frozen; validation-loss checkpoints are selected at step 1,000 for the controller and step 258 for the tree organizer.Source: B TRAINING THE TWO MEMORY POLICIES · p. 16
  • At evaluation, the controller revises memory from the question, images, existing memory, answer, and execution trace, while adaptive visual injection controls what enters the next solver input.Omitting visual memory from an input does not delete it from the stored state.Source: B TRAINING THE TWO MEMORY POLICIES · p. 16

C EXPERIMENTAL PROTOCOL AND DATA SPLITS

The protocol separates memory construction from evaluation and specifies controls for transfer, solving attempts, scoring, and cost reporting. Cross-harness evolution compares original and evolved banks on the same questions under a single-attempt setting.

C EXPERIMENTAL PROTOCOL AND DATA SPLITS

  • The standard evaluation split has 4,538 questions across eleven benchmarks, disjoint from project-defined construction splits; ReasonMap maps stay wholly on one side of the split.Historical-memory experiments share 1,000 initial construction questions; counts refer to source questions, not accepted episodes.Source: C EXPERIMENTAL PROTOCOL AND DATA SPLITS · p. 16
  • Cross-harness evolution adds a non-overlapping 1,000-question construction set, excludes both sets from its 4,388-question evaluation, and compares original and evolved banks on identical questions.Codex and DeepSeek-Harness exchange construction and evolution roles, with memory models fixed; Table 8 gives question distributions for both stages.Source: C EXPERIMENTAL PROTOCOL AND DATA SPLITS · p. 16, Table 8 · p. 17
  • Main comparisons and historical-memory evaluations use one solving attempt with question-level updates disabled, and model parameters and historical banks remain frozen.Reference answers and official scores are reserved for offline evaluation and do not enter memory updates, stopping decisions, or answer selection.Source: C EXPERIMENTAL PROTOCOL AND DATA SPLITS · pp. 16–17
  • Benchmark scores receive equal weight in macro-averages, while MAS and memory-operation time and token usage are reported separately and normalized to matching No Memory MAS costs.No Memory MAS time and token usage each equal 1×, and its memory-operation costs are zero; experiments use NVIDIA H100 GPUs, with GPT-5.6-Luna accessed through its API.Source: C EXPERIMENTAL PROTOCOL AND DATA SPLITS · p. 17

D ADDITIONAL PERFORMANCE AND COST COMPARISONS

With Gemma4-31B, EpiCon improves the eleven-benchmark mean over No Memory, while the trained 2B variant uses fewer tokens than backbone-sized EpiCon on DeepSeek-Harness. The comparisons show a performance–cost trade-off across the two memory-model sizes.

D ADDITIONAL PERFORMANCE AND COST COMPARISONS

  • For Gemma4-31B, Figure 4 reports performance–time trade-offs: performance is the equally weighted mean across eleven benchmarks, while total time sums normalized MAS and memory costs within each harness.Source: Figure 4 · p. 18
  • 25.1% fewer total tokens with Qwen3.8-27B and 16.8% fewer with Gemma4-31B accompany lower mean benchmark scores for the trained 2B variant versus backbone-sized EpiCon.Figure 5 compares performance and total token usage on DeepSeek-Harness, with tokens normalized by corresponding No Memory MAS usage.Source: Figure 5 · p. 18, D ADDITIONAL PERFORMANCE AND COST… · p. 18

Related papers

Loading 2609.37923v1…