Source-linked AI summary
Muffin - A Shared Memory Buffer for Asynchronous AI Agents
Ping-Lin Kuo, Suman Navaratnarajah, Christian Frey, Gresa Shala, Josif Grabocka
TL;DR
Many multi-agent systems exchange complete outputs only after inference, leaving intermediate findings unavailable and adding verbose-trace overhead. Muffin routes queries by reasoning demand and lets agents asynchronously share selected findings through a shared buffer while keeping private reasoning local. Across four reasoning benchmarks and four language models, it achieved the best accuracy in 17 of 20 settings and a better accuracy-token trade-off than multi-agent debate.
Problem
Many multi-agent systems communicate after complete outputs, leaving partial findings unavailable to peers while verbose, task-irrelevant reasoning traces add context overhead.
Source: §1 · p. 1Method
Muffin routes queries by expected reasoning demand and, for demanding queries, has agents asynchronously exchange selected findings through a shared buffer while keeping private reasoning local.
Source: §3.2 · p. 4, §5 · p. 9Results
Takeaways & Limitations
Shared intermediate findings can improve multiagent reasoning without agents exchanging or using full reasoning traces.
Source: §4.4 · p. 9Evaluation focuses on question-answering and reasoning benchmarks, leaving tool use, planning, environment interaction, and long-horizon execution for future evaluation.
Source: §4.4 · p. 9
Abstract
from arXiv · showhide
The rise of Agentic AI systems has changed the landscape of artificial intelligence by enabling strong reasoning capabilities through specialized multi-agent interaction with structured self-reflection. However, in target-driven reasoning tasks, single- and multi-agent systems are often on par in terms of performance gains. A major challenge is that failures often arise from an ineffective exchange of critical information during inter-agent interaction and coordination. Different from existing multi-agent debate paradigms that rely on synchronous turn-taking and sharing information across different agents, we propose an inter-agent protocol based on shared Memory bUFFer INference - referred to as MUFFIN - in an asynchronous agentic framework. In the first step, an orchestrator routes queries based on their reasoning demands and delegates demanding ones to a multi-agent setting, where a shared memory buffer provides asynchronous exchange of intermediate findings while an agent's private reasoning remains local. Our experiments show that our framework Muffin outperforms established competitors, achieving the best accuracy in 17 out of 20 of our benchmark settings, and provides a better quality-cost trade-off than multi-agent debate: Muffin improves accuracy by 19.2 accuracy points while using 77.4% fewer tokens, without requiring full-trace sharing.
1 INTRODUCTION
Complex tasks call for decomposing problems, checking partial information, and integrating conclusions, yet multi-agent systems can lose critical information when coordination waits for complete outputs. Muffin addresses this with demand-based routing and asynchronous sharing of selected findings while keeping private reasoning local.
1 INTRODUCTION
- Complex tasks can benefit from collaboration that decomposes problems, checks partial information, and integrates conclusions generated by different reasoning parts.The section situates this motivation amid goal-oriented multi-agent behavior supported by memory, reasoning, or environmental interaction.Source: §1 · p. 1
- Many multi-agent systems communicate only after complete outputs, leaving partial findings unavailable to peers while verbose, task-irrelevant traces increase context overhead.The paper frames asynchronous sharing of intermediate findings during inference as its central question.Source: §1 · p. 1
- Muffin routes queries by expected reasoning demand, assigns specialized roles and subtasks for demanding queries, and lets agents share selected findings asynchronously while keeping private reasoning local.A summarizer uses the agents’ final messages, their meta information, and the initial query to produce the response.Source: §1 · p. 2
- 19.2 accuracy points with 77.4% fewer tokens than multi-agent debate, while Muffin lies on the accuracy-token Pareto frontier across four reasoning benchmarks.The reported average improvement over the strongest baseline is 4.4 accuracy points, rising to 11.8 on OlymMATH-E.Source: §1 · p. 2
2 RELATED WORK
Prior work explores specialized-agent coordination, aggregation, and fine-grained streaming, but leaves open how agents can select, maintain, and reuse partial findings asynchronously without exposing full traces.
2 RELATED WORK
- Multi-agent frameworks commonly use specialized roles or task decomposition, but often integrate outputs only after complete messages, subtasks, or rounds.The delayed integration limits adaptation to other agents’ reasoning while inference is unfinished.Source: §2 · pp. 2–3
- Aggregation and debate share complete responses, critiques, or round outputs, leaving open how agents should share and use partial information before finishing an answer.Prior work supports the value of shared intermediate information, but typically does not let the LLM select which information is useful enough to share.Source: §2 · p. 3
- Token-level and streaming collaboration expose generation progress earlier, while asynchronous sharing still needs to select, maintain, and jointly reuse findings without exposing full traces.This direction is framed as a middle ground between isolated final-answer aggregation and full-trace exchange.Source: §2 · p. 3
3 METHODOLOGY
The methodology frames multi-agent inference around agents with assigned roles and subtasks, then introduces MUFFIN’s routed, asynchronous collaboration through a shared memory buffer. Agents retrieve recent peer findings while retaining private histories, and a summarizer produces the final answer from their summaries.
3.1 GENERAL MULTI-AGENT PROBLEM SETTING
- Agents receive agent-specific roles and subtasks, reason for multiple steps, and contribute completed messages to a final aggregation or selection.The standard setting treats communication mainly as completed messages rather than explicitly modeling how information is organized and reused across later inference steps.Source: §3.1 · p. 3
3.2 MUFFIN - SHARED MEMORY BUFFER INFERENCE
- The orchestrator maps a query to a plan that selects direct or multi-agent inference and specifies agent roles, non-overlapping subtasks, and a collaboration protocol.The protocol specifies what information is shared, what stays local, and whether new findings can trigger peer reasoning steps.Source: §3.2 · p. 4
- The shared buffer is append-only, stores agent findings with source and step identifiers, and lets agents retrieve a bounded window of recent findings from peers.Retrieval scans backward and skips the agent’s own shared findings, which are available in its private history.Source: §3.2 · p. 4
- Agents run concurrently without a global round index, and one agent’s committed findings become visible to peers only after they are committed.When re-activation is enabled, an agent performs exactly one additional reasoning step after all agents have completed at least T steps.Source: §3.2 · p. 5
- After agents finish their reasoning, the summarizer receives the query, assignments, and each agent’s last-step summary rather than the full interaction trace.The buffer’s findings reach the summarizer through agent summaries produced after reading the buffer.Source: §3.2 · p. 5
4 EXPERIMENTS
Across four reasoning benchmarks and four models, the experiments compare Muffin with single-agent, majority-vote, and debate baselines, then examine accuracy-cost trade-offs, shared findings, and design choices.
4.2 EFFECTIVENESS AND EFFICIENCY
- Muffin achieves the best accuracy in 17 of 20 evaluation settings and is second-best in the remaining three.Against the strongest baseline, it gains 11.8 accuracy points on OlymMATH-E and 2.6 points on OlymMATH-H on average.Source: §4.2 · p. 6, Table 1 · p. 7
- Muffin reaches 59.2% after the final summarizer, up from 44.9%, and beats multi-agent debate by 19.2 accuracy points with 77.4% fewer tokens.The comparison tracks accuracy and token use over ten reasoning steps on OlymMATH-Easy.Source: §4.2 · pp. 6–7, Figure 3 · p. 7
- With three agents, accuracy rises from 50.6% at two agents to 59.3%; four or five agents yield only slight changes as token use increases.At four and five agents, accuracy reaches 59.7% and 58.7%, respectively.Source: Figure 4 · p. 7, §4.2 · p. 7
- At the final step, full-trace sharing raises accuracy from 85.0% to 87.3%, while Muffin reduces token use from 160.6K to 41.9K.This is 73.9% fewer tokens for Muffin, which the authors describe as a better quality-cost trade-off than the full-trace variant.Source: Figure 5 · p. 8, §4.2 · p. 8
- On Chess, 20.5% of shared findings change answers; early changes are mostly wrong-to-correct (51.6% vs. 14.2%), shifting toward 34.8% vs. 21.9% later.The answer-change share falls from 43.2% at the first transition to 15.8% in the last recorded step.Source: Figure 7 · p. 8, Figure 8 · p. 8, §4.2 · p. 8
4.3 ABLATION STUDIES
- For gemma-4-E4B-it, k = 10 improves over k = 5 by 5.0 accuracy points, but more findings reduce performance; gemma-4-31B-it changes by only 2.0 points.The authors suggest a moderate shared-finding budget is sufficient and that additional findings can introduce redundant or outdated information.Source: Figure 6 · p. 8, §4.3 · p. 9
- The summarizer changes the last agent answer in 11.14% of cases; among those changes, wrong-to-correct outcomes exceed correct-to-wrong by 10.85 percentage points.The comparison spans all benchmarks and two Gemma-4 model sizes.Source: Table 2 · p. 9, §4.3 · p. 9
4.4 LIMITATIONS AND DISCUSSION
- Evaluation is limited to question-answering and reasoning benchmarks, with compute costs and incomplete coverage of open- and closed-weight models and settings.The authors call for evaluation on tool use, planning, environment interaction, and long-horizon execution.Source: §4.4 · p. 9
5 CONCLUSION
Muffin exchanges selected intermediate findings asynchronously while keeping agents’ private reasoning local, and the conclusion reports strong benchmark performance with a better accuracy-token trade-off than multi-agent debate.
5 CONCLUSION
- Muffin achieves the best accuracy in 17 of 20 benchmark settings and a better accuracy-token trade-off than multi-agent debate, using 77.4% fewer tokens for a 19.2-point accuracy gain.The conclusion also identifies dynamic agent scaling and extension to tool-using and long-horizon tasks as future work.Source: §5 · p. 9
AI USE STATEMENT
- The authors report using generative AI for code modifications, plotting scripts, prompt formulation, and sentence fluency, but not for research ideas, core methodology, or initial text.They state that AI-assisted code, figures, and text were reviewed by the authors.Source: AI USE STATEMENT · p. 10
REPRODUCIBILITY STATEMENT
- The authors state that the implementation, experiment scripts, benchmark and model settings, prompts, and additional results are provided to support reproduction of the reported results.The reproducibility statement points to Section 4, Appendices A and B, and an anonymous code repository.Source: REPRODUCIBILITY STATEMENT · p. 10
A PROMPT TEMPLATES
The appendix provides Muffin’s prompt templates, describing how queries are routed, agents collaborate through shared findings, and a summarizer selects the final answer.
A.1 ORCHESTRATOR PROMPT
- The orchestrator routes queries to a direct-answer or multi-agent workflow and, for multi-agent tasks, assigns agent roles, subtasks, and collaboration rules.The appendix presents the main prompt templates used by Muffin.Source: §A.1 · p. 15, A PROMPT TEMPLATES · p. 15
- The orchestrator prompt defaults to multi-agent routing and reserves direct mode for answers judged fully certain from prompt evidence or a deterministic trivial fact.Multi-agent routing is specified for uncertainty, ambiguity, calculation risk, domain reasoning, verification, critique, or multiple viewpoints.Source: §A.1 · p. 15
- In multi-agent mode, the orchestrator assigns distinct roles and non-overlapping subtasks, with at least one agent tasked to challenge assumptions and contradictions.The prompt also specifies shared event types and reactive steps for collaboration.Source: §A.1 · p. 15
A.2 AGENT REASONING PROMPT
- Agents receive the task, role, subtask, private notes, and current shared findings, then share concise findings while keeping private reasoning local.They are instructed to verify shared findings before adopting them and to challenge assumptions when assigned as a critical agent.Source: §A.2 · p. 16
A.3 SUMMARIZER PROMPT
- After the reasoning steps, the summarizer selects the final answer from agents’ last summaries and extracted candidates rather than solving the task anew.When review is needed, it audits candidate reasons; it treats weak majorities, ties, and single-candidate cases as requiring evidence review.Source: §A.3 · p. 17
A.4 DIRECT-ANSWER PROMPT
- When the orchestrator selects the direct route, the system bypasses multi-agent collaboration and asks the model to answer the task directly.The prompt calls for one concise response.Source: §A.4 · pp. 17–18
B.1 STEP-WISE ACCURACY AND TOKEN USAGE
The appendix figures show five-step accuracy and token usage across benchmarks for Gemma-4-31B-it and Qwen3.8-27B.
B.1 STEP-WISE ACCURACY AND TOKEN USAGE
- Figures 9–18 show five-step accuracy and token usage across all benchmarks for Gemma-4-31B-it and Qwen3.8-27B.The passages identify the figure scope but do not provide plotted values or comparisons.Source: §B.1 · p. 18
- For Gemma-4-31B-it, the figure captions cover Chess, GPQA, MMLU-Pro, OlymMATH-Easy, and OlymMATH-Hard.The captions identify Figures 9–13 as the Gemma-4-31B-it set.Source: Figure 9 · p. 18, Figure 10 · p. 18, Figure 11 · p. 18
- The supplied captions report step-wise accuracy and token usage for Qwen3.8-27B on Chess, GPQA, and OlymMATH-Easy, with each benchmark named in its caption.Source: Figure 14 · p. 18, Figure 15 · p. 19, Figure 17 · p. 19
B.2 SHARED FINDINGS INSIGHTS
The appendix figures report shared-finding answer-change analyses for Gemma-4-31B-it and Qwen3.8-27B across several benchmarks.
B.2 SHARED FINDINGS INSIGHTS
- The passage states that Figures 19–26 show the shared-finding answer-change analysis for the remaining benchmarks with gemma-4-31B-it, and that Figures 27–36 provide corresponding results for Qwen3.8-27B.Source: §B.2 · p. 19
- For Gemma-4-31B-it, the captions cover mean counts and outcome breakdowns on GPQA, MMLU-Pro, OlymMATH-Easy, and OlymMATH-Hard.The figures do not provide outcome values in the supplied passages.Source: Figure 19 · p. 19, Figure 20 · p. 19, Figure 21 · p. 20
- For Qwen3.8-27B, the captions cover mean counts and outcome breakdowns on Chess, GPQA, MMLU-Pro, OlymMATH-Easy, and OlymMATH-Hard.The figures do not provide outcome values in the supplied passages.Source: Figure 27 · p. 21, Figure 28 · p. 21, Figure 29 · p. 21