Source-linked AI summary
TALM: Dynamic Tree-Structured Multi-Agent Framework with Long-Term Memory for Scalable Code Generation
Ming-Tung Shen, Yuh-Jzer Joung
TL;DR
Single-path prompting can struggle with complex requirements and limited context, while multi-agent systems may rely on rigid workflows and costly recovery. TALM combines tree-based task decomposition, localized re-reasoning, and long-term memory. It consistently outperforms alternatives across three code-generation benchmarks while maintaining high token efficiency.
Problem
Single-path prompting struggles with complex requirements or context beyond the model window, and multi-agent frameworks often use rigid workflows.
Source: §1 · p. 1Method
TALM organizes code agents in a task-decomposition tree, supports subtree-level re-reasoning, and retrieves prior experiences through long-term memory.
Source: §1 · p. 1Results
Takeaways & Limitations
The results are presented as evidence of TALM’s robustness and efficiency for complex code-generation tasks.
Source: §5 · p. 8The benchmarks are modest in scale and limited to class-level tasks, leaving performance on large interconnected codebases unresolved.
Source: §6 · p. 8
Abstract
from arXiv · showhide
Agentic code generation requires large language models (LLMs) capable of complex context management and multi-step reasoning. Prior multi-agent frameworks attempt to address these challenges through collaboration, yet they often suffer from rigid workflows and high reasoning recovery costs. To overcome these limitations, we propose TALM (Tree-Structured Multi-Agent Framework with Long-Term Memory), a dynamic framework that integrates structured task decomposition, localized re-reasoning, and long-term memory mechanisms. TALM employs an extensible tree-based collaboration structure. The parent-child relationships, when combined with a divide-and-conquer strategy, enhance reasoning flexibility and enable efficient error correction across diverse task scopes. Furthermore, a long-term memory module enables semantic querying and integration of prior knowledge, supporting implicit self-improvement through experience reuse. Experimental results on HumanEval, BigCodeBench, and ClassEval benchmarks demonstrate that TALM consistently delivers strong reasoning performance and high token efficiency, highlighting its robustness and practical utility in complex code generation tasks.
1 Introduction
TALM addresses rigid multi-agent workflows and costly reasoning recovery with tree-based task decomposition, localized re-reasoning, and long-term memory. The paper reports that it outperforms existing approaches while maintaining high token efficiency on three code-generation benchmarks.
1 Introduction
- Prior multi-agent systems often use rigid pipelines, while single-path prompting struggles with complex requirements or context beyond the model window.Existing systems also either allow reasoning flaws to propagate or restart the full process, consuming resources.Source: §1 · p. 1
- Its tree structure decomposes tasks into subtasks and supports re-reasoning within an affected subtree when flaws are detected.This is intended to handle tasks at different granularities.Source: §1 · p. 1
- TALM stores and retrieves prior experiences to support implicit self-improvement without additional retraining.The memory module reuses past successes and avoids previously observed mistakes.Source: §1 · p. 1
- TALM outperforms existing approaches while maintaining high token efficiency on HumanEval, BigCodeBench, and ClassEval.The introduction reports this overall comparison without providing metric values.Source: §1 · pp. 1–2
2 Related Work
Prior work extends LLM reasoning through prompting strategies and multi-agent task decomposition. TALM is proposed in response to remaining challenges in workflow rigidity, correction costs, and cumulative memory.
2 Related Work
- Prompting methods progress from step-by-step reasoning and self-planning to tool use, self-reflection, and tree-based search [18] [8] [22] [15] [21].These approaches are described as cost-efficient alternatives to fine-tuning for increasingly complex reasoning tasks.Source: §2 · p. 2
- Multi-agent frameworks divide work among specialized agents, with examples spanning centralized coordination, dynamic decomposition, evolving DAGs, and simulated software-development roles [19] [17] [20] [14] [5].MapCoder adds retrieval of semantically similar problem–solution examples for few-shot prompting.Source: §2 · p. 2
- TALM targets three stated challenges in code-generation collaboration: rigid workflows, high reasoning-correction costs, and absent cumulative memory.Multi-agent decomposition is presented as one way to address single-agent context-window limitations.Source: §2 · p. 2
3 TALM Framework
TALM combines tree-structured collaboration, localized re-reasoning, and long-term memory. Agents decompose and validate work through a hierarchy, recover from errors within affected subtrees, and retrieve or consolidate prior task experience.
3 TALM Framework
- TALM combines tree collaboration, localized re-reasoning, and long-term memory to coordinate agents, correct errors, and accumulate experience.Its workflow decomposes tasks among specialized agents, validates outputs with sandboxed tests, and merges results bottom-up.Source: §3 · p. 2
3.1 Tree-Structured Collaboration
- Agents may solve a task independently or delegate subtasks; validated child results support the parent’s solution in a bottom-up divide-and-conquer process.The tree is bounded by maximum depth, branching factor, and a branching decay rate; the design assumes decomposition generally reduces complexity, while difficult subtasks may remain with one agent.Source: §3.1 · pp. 2–3
3.2 Localized Re-Reasoning
- Localized re-reasoning recomputes only the affected subtree, preserving the rest of the workflow and verified reasoning chains.Children request clarification for underspecified instructions; parents replace a flawed subtree when their decomposition proves unsuitable.Source: §3.2 · p. 4
3.3 Long-Term Memory
- TALM retrieves semantically similar task records, filters candidates by metadata such as tree depth, and uses them as contextual references.After validation, it stores task descriptions, reasoning, code, and metadata; overlapping records can be synthesized into a consolidated memory entry.Source: §3.3 · pp. 4–5
- The framework’s combined mechanisms are presented as balancing flexibility, efficiency, and continual improvement for complex code generation.The memory update process consolidates overlapping entries to limit redundancy and uncontrolled memory growth.Source: §3.3 · p. 5
4.1 Experimental Settings
The evaluation uses three datasets spanning function-level code generation to class-level tasks, and compares TALM with prompting, reasoning, and multi-agent baselines. Experiments use two GPT-4 models, Pass@1, and specified controls and hyperparameters.
4.1 Experimental Settings
- The evaluation uses HumanEval’s 164 problems, BigCodeBench’s 1,140 function-level problems, and ClassEval’s 100 class-level tasks.The datasets cover single-function synthesis, harder function-level tasks, and class-level structural consistency and context handling.Source: §4.1.1 · p. 5
4.1.2 Baselines.
- Baselines include direct prompting, Chain-of-Thought, self-planning, Reflexion, and MapCoder across prompting, reasoning, and multi-agent paradigms.The set is described as spanning different approaches, though MetaGPT and AgentNet are not included.Source: §4.1.2 · p. 5, §4.1.3 · p. 5
4.1.3 Foundation Models, Evaluation Metric, & Hyperparameters.
- Main results use GPT-4o-mini and GPT-4o, while ablations use GPT-4o-mini because of cost constraints.The authors limit evaluation to these two models and argue for comparisons across capability tiers within one model family.Source: §4.1.3 · pp. 5–6
- TALM uses maximum tree depth 3, initial branching factor 3, decay rate 1, and up to 3 validation correction attempts.Memory retrieval returns up to three records; MapCoder retrieval size and retries are 5 for HumanEval and 3 for the other datasets.Source: §4.1.3 · p. 6
- Pass@1 is the evaluation metric, and baseline methods are re-implemented with the same foundation models for comparison.Temperature is fixed at zero; TALM memory is cleared each round and dataset order randomized.Source: §4.1.3 · p. 6
4.2 Main Results
TALM outperforms prompting baselines and MapCoder across the evaluated benchmarks, while using fewer tokens than MapCoder. Its gains are especially pronounced on more demanding code-generation tasks.
4.2 Main Results
- TALM consistently outperforms baseline prompting methods and MapCoder on Pass@1 across HumanEval, BigCodeBench, and ClassEval.Source: §4.2.1 · p. 6
4.2.1 Performance on Code Generation.
- On HumanEval, TALM provides modest, consistent gains over direct prompting and chain-of-thought prompting despite strong accuracy across methods.Source: §4.2.1 · p. 6
- On BigCodeBench, TALM improves roughly 2–3% over direct prompting, while most prompting methods show limited gains or degrade.Source: §4.2.1 · p. 6
- On ClassEval, TALM improves testcase-level performance by over 6% versus direct prompting and 2–3% versus the next-best baseline, with the highest class-level score.Its class-level score exceeds other methods by around three percentage points.Source: §4.2.1 · p. 6
- TALM uses significantly fewer input and output tokens than MapCoder, with Figure 4 comparing their token consumption.The text attributes this to simplified agent roles and hierarchical collaboration, plus memory retrieval from a vector store.Source: §4.2.2 · p. 6, Figure 4 · p. 6
4.2.2 Token Usage Efficiency.
- Enabling long-term memory causes only a marginal increase in token usage, while TALM achieves accuracy gains with few additional tokens.Source: §4.2.2 · p. 6
4.3 Ablation Studies and Analyses
Long-term memory yields more pronounced gains on longer, structurally complex benchmarks, and its benefits on ClassEval grow as experience accumulates across tasks.
4.3 Ablation Studies and Analyses
- Memory gains are more pronounced on BigCodeBench and ClassEval than HumanEval, suggesting greater benefit where cross-task knowledge reuse matters.Source: §4.3.1 · p. 6
4.3.1 Impact of Long-Term Memory.
- On ClassEval, performance stays near baseline while memory is sparse early in the sequence, then improves stably in later segments as knowledge accumulates.Source: §4.3.1 · p. 6
GPT-4o
The analyses report performance drops when localized re-reasoning is removed and show that tree height and branching should be tuned to task complexity.
GPT-4o
- Table 1 reports overall Pass@1 results and marks the best score within each foundation model.Source: Table 1 · p. 7
- Figure 5 concerns the long-term memory module's effect over the ClassEval task sequence.Source: Figure 5 · p. 7
4.3.2 Impact of Localized Re-Reasoning.
- Removing localized re-reasoning lowers performance across all datasets, with particularly notable effects on BigCodeBench and ClassEval.The mechanism corrects flaws at the subtree level, avoiding costly restarts while preserving verified outputs.Source: §4.3.2 · p. 7, Table 2 · p. 7
4.3.3 Impact of Tree Height.
- The best maximum tree height is m=2 for HumanEval and m=3 for BigCodeBench and ClassEval; further increases cause over-decomposition.Excessive splitting consumes resources and adds unnecessary reasoning steps.Source: §4.3.3 · p. 7, Table 3 · p. 7
4.3.4 Impact of Branching Factor.
- A small branching factor n=1 suffices on HumanEval, while values around n=3 work best on BigCodeBench and ClassEval.Too-small branching can limit decomposition flexibility on complex tasks; increasing n does not significantly degrade performance because agents decide whether to spawn children.Source: §4.3.4 · p. 7, Table 4 · p. 8
5 Conclusions and Future Work
TALM is reported to outperform alternatives across three code-generation benchmarks, with ablations supporting the value of localized re-reasoning and long-term memory. The conclusion also identifies a decomposition trade-off and future work on adaptive tree parameters, memory quality, and broader task coverage.
5 Conclusions and Future Work
- TALM uses an expandable agent tree, localized subtree error correction, and long-term memory to address flexibility, recovery, and knowledge retention.The memory stores prior problem–solution traces in a vector database for retrieval on future tasks.Source: §5 · p. 8
- TALM consistently outperforms prompting baselines and MapCoder on HumanEval, BigCodeBench, and ClassEval, particularly in complex and long-context settings.Ablations also support the importance of localized re-reasoning, long-term memory, maximum depth, and branching factor.Source: §5 · p. 8
- For simple tasks, decomposition may add token overhead or occasional failures, while it improves solvability on more challenging problems.The paper characterizes this as lowering the performance floor while raising the ceiling, with the latter effect viewed as more important in realistic software engineering.Source: §5 · p. 8
- Future work may replace fixed tree hyperparameters with runtime adjustment and improve automated memory updates through human feedback or lightweight verification.The proposed directions include adaptively tuning tree height and branching during runtime.Source: §5 · p. 8
- TALM has so far been evaluated only on natural-language-to-code tasks; extending it to code completion, refactoring, and debugging could broaden its applicability.Source: §5 · p. 8
6 Limitations
The evaluation is constrained by modest dataset scale and class-level task scope. The memory was reset between experimental rounds, and benchmarks did not cover large, interconnected codebases.
6 Limitations
- The modest benchmark sizes and resetting memory at each experimental round limit evidence for the memory module’s long-term benefits in deployments where memory accumulates.HumanEval and ClassEval each contain fewer than 200 problems, while BigCodeBench contains 1,140.Source: §6 · p. 8
- ClassEval remains limited to class-level tasks, leaving performance on system-level projects spanning multiple classes, modules, or packages untested.Such large-scale contexts were unavailable in the benchmarks discussed.Source: §6 · p. 8
- The paper summarizes TALM’s evaluation as constrained by dataset scale and scope.Source: §6 · p. 8
A Code Generation Sample by TALM
The appendix presents a representative BigCodeBench code sample as qualitative evidence of TALM’s decomposition and structured code generation. The sample implements an HTTP email handler that validates request data and sends email.
A Code Generation Sample by TALM
- The BigCodeBench sample offers qualitative insight into TALM’s decomposition and structured, executable code generation beyond the paper’s quantitative evaluation.Source: A Code Generation Sample by TALM · p. 9
- The sample defines an email request handler that reads and parses JSON and validates required email fields, including subject, message, and recipient.It checks the content type and reports missing required keys.Source: A Code Generation Sample by TALM · pp. 9–10
- The handler builds a plain-text MIME message and sends it through an SMTP connection using the configured server and credentials.Source: A Code Generation Sample by TALM · p. 10