Source-linked AI summary
HyMem: Hierarchical Context Management for Long-Horizon Agents via Information Isolation
XinQi Wang, Jinwei Xiao, Sijia Cui, Hongming Zhang, Yanna Wang, Qingyang Zhang, Bo Xu
TL;DR
Long-horizon LLM agents struggle when dense execution traces obscure planning signals in flat contexts. HyMem uses typed context isolation and structured memory, improving average Pass@1 over the strongest baseline by 6.1 points on GAIA and 4.7 points on Browsecomp-plus.
Problem
Long-horizon agents face context dilution because dense execution traces can obscure sparse planning signals in flat interaction histories.
Method
HyMem is a training-free inference-time framework that isolates planning, execution, subtask reasoning, and memory consolidation into distinct context spaces.
Results
HyMem improves average Pass@1 over the strongest baseline by 6.1 percentage points on GAIA and 4.7 percentage points on Browsecomp-plus.
Takeaways & Limitations
With DeepSeek-V4, HyMem achieves the best average Pass@1 on both benchmarks while maintaining comparable token consumption and tool-call counts.
Takeaways & Limitations
HyMem’s effectiveness depends on reliable adherence to typed control instructions and structured returns, and its gains are not uniform across backbones and benchmarks.
Abstract
from arXiv · showhide
Large language model (LLM) agents often perform poorly on complex, long-horizon tasks because their context becomes increasingly cluttered over time. As interactions accumulate, detailed execution traces and intermediate outputs dominate the context, making it difficult for the model to retain and use high-level planning information. Most existing methods address this issue through compression or retrieval applied to a single, flat context, which does not clearly separate different types of context information and often leads to degraded reasoning. To address this challenge, we propose HyMem, a hierarchical framework that explicitly separates the agent's context into distinct functional layers. HyMem organizes context by function to separate high-level planning from execution and complex analysis. Its isolated reasoning module handles complex subtasks without adding intermediate reasoning traces to the persistent planning context, while its memory management module preserves task progress across context refreshes through structured summaries. These components reduce redundant context accumulation, retain task-critical information, and support coherent long-horizon reasoning within a limited context window. Experiments on GAIA and Browsecomp-plus show that, with DeepSeek-V4, HyMem achieves average Pass@1 scores of 66.7% and 61.3%, outperforming the strongest baseline by 6.1 and 4.7 percentage points, respectively. Further analysis indicates that HyMem effectively controls the growth of the reasoning context, allowing the model to maintain focus and accuracy across complex, long-horizon tasks.
1 Introduction
HyMem addresses context dilution in long-horizon agents by isolating planning, execution, reasoning, and memory into hierarchical functional layers. It improves benchmark performance while preserving comparable token consumption and tool-call counts.
- Problem: Flat context architectures interleave planning and execution, allowing redundant execution traces to dilute sparse strategic signals and obscure the agent’s original goal.This information-density mismatch motivates hierarchical context isolation.
- Problem: Prior compression and retrieval methods operate on premixed sequences, risking loss of reasoning chains, temporal logic, and distinctions between reasoning signals and execution noise.The paper identifies these limitations as consequences of applying mitigation to a flat context.
- Method: HyMem enforces hierarchical isolation by decoupling high-level planning from low-level execution and managing context to retain decision-useful information.Its Action Layer encapsulates noisy interaction loops, while explicit boundaries separate planner, executor, isolated subtask reasoning, and structured memory management.
- Results: 66.7% average Pass@1 on GAIA and 61.3% on Browsecomp-plus exceed the strongest baseline by 6.1 and 4.7 percentage points, respectively.The evaluations cover complex information retrieval and general-purpose agent reasoning tasks.
- Results: 30.0% Pass@1 on hard Browsecomp-plus tasks rises from 14.0%, while overall performance is achieved with comparable token consumption and tool-call counts.Ablation studies also indicate that isolated reasoning is important for long-horizon tasks.
2 Related Work
Prior work develops LLM-based agents by combining language-model reasoning with tool and environmental interaction, while another line addresses long-horizon tasks through external memory and intra-task context management. Representative methods make reasoning, planning, action selection, self-reflection, memory retrieval, or trajectory compression explicit.
- LLM-based Agent Frameworks: LLM-based agent frameworks combine language-model reasoning with tool use and environmental interaction.Representative frameworks include ReAct, Plan-and-Solve, and Reflexion.
- LLM-based Agent Frameworks: ReAct, Plan-and-Solve, and Reflexion improve task solving by making intermediate reasoning, action selection, planning, or self-reflection explicit.Recent systems extend this paradigm to web search, GUI control, and long-horizon information-seeking tasks.
- Agent Memory and Context Management: Agent memory and context-management methods improve long-horizon agents through memory construction, retrieval, summarization, or learned context management.External memory systems organize reusable information outside the immediate context and retrieve relevant memories when needed.
- Agent Memory and Context Management: Intra-task methods compress or learn to retain useful trajectory information during long tasks, reducing context pressure.Examples include ReSum, MemAgent, MEM1, AgentFold, and context-folding approaches.
3 Methodology
HyMem is a training-free inference-time controller that uses typed context isolation to separate planning, execution, subtask reasoning, and memory consolidation. By admitting only schema-constrained returns into the persistent planner, it limits context dilution while preserving task continuity.
- Context isolation: HyMem maintains separate context spaces for planning, tool execution, isolated reasoning, and memory consolidation, with only schema-constrained messages crossing boundaries.Raw execution traces and intermediate deliberation tokens are not directly appended to the main planning context.
- Context isolation: In long-horizon tasks, execution-level traces grow faster than planning-relevant information, reducing the relative density of goals, verified facts, constraints, and unresolved sub-goals.HyMem refers to this reduction as context dilution.
- Executor: The executor distills raw observations by removing navigation text, duplication, formatting artifacts, irrelevant snippets, and error boilerplate while preserving entities, dates, numbers, sources, evidence, and answer candidates.The executor then synthesizes typed findings, evidence, and sources before discarding its private context and raw observations.
- Isolated reasoning: Isolated reasoning supports multi-hop analysis, hypothesis checking, and conflicting-evidence reconciliation while keeping intermediate reasoning traces private and returning only structured conclusions, facts, sources, and confidence.The isolated context is discarded after return.
- Structured memories: HyMem rewrites episode, working, and tool memories, injecting compact procedural tool summaries into the planner instead of raw tool logs.The planner grows through bounded typed returns and rewritten memories rather than full tool traces or sub-task deliberations.
4 Experiments
HyMem achieves strong performance across GAIA and Browsecomp-plus by isolating reasoning and preserving structured task memory. Its benefits are largest for difficult, long-horizon tasks, while resource use remains comparable and cross-backbone gains are not uniform.
- Main Results: 66.7 average Pass@1 on GAIA and 61.3 on Browsecomp-plus make DeepSeek-V4 with HyMem the strongest overall configuration.HyMem improves over ReAct by 10.1 and 4.7 percentage points on GAIA and Browsecomp-plus, respectively.
- Difficulty-Wise Results: 30.0 Pass@1 on hard Browsecomp-plus questions rises from 14.0, although HyMem is slightly lower than ReAct on easy questions.The gains are concentrated on medium and hard questions, where sustained planning, evidence accumulation, and multi-step verification matter most.
- Backbone Analysis: 33.0 average Pass@1 on GAIA improves over ReAct’s 11.8 and the summarization workflow’s 22.0 for Qwen3-32B, but HyMem does not outperform ReAct on Browsecomp-plus.These results indicate promising but non-uniform cross-backbone robustness.
- Resource Efficiency: Higher task performance than ReAct, ReSum, A-MEM, and THREAD is achieved with resource usage in a comparable range.The performance gain is attributed to structured context isolation directing computation toward task-relevant reasoning and evidence synthesis rather than unbounded context growth or excessive tool use.
- Ablation Study: 18.6 percentage-point drops on GAIA follow removal of either isolated reasoning or memory management, while Browsecomp-plus drops by 5.0 and 9.0 points, respectively.Isolated reasoning protects the planner from intermediate deliberations, while structured memories preserve milestones, goals, and reusable tool experience across context refreshes.
5 Conclusion
HyMem addresses context dilution in long-horizon LLM agents through typed context isolation, structured returns, and asymmetric memory injection that preserve continuity while focusing planning on verified information. Experiments show its strongest benefits with DeepSeek-V4, while also revealing dependence on instruction following, structured returns, and backbone–benchmark combinations.
- Core contribution: HyMem separates planning, tool execution, isolated sub-task reasoning, and memory consolidation into distinct context spaces.This prevents raw observations and intermediate deliberation traces from directly entering the main planning context.
- Core contribution: Structured returns and asymmetric memory injection preserve task continuity while keeping the planner focused on verified facts, current goals, and unresolved gaps.
- Empirical findings: With DeepSeek-V4, HyMem achieves the best average Pass@1 on both GAIA and Browsecomp-plus, with clear gains on harder Browsecomp-plus questions.Resource analysis suggests these gains do not simply result from using more tokens or tool calls, while context-growth analysis supports the intended mechanism.
- Limitations and future work: HyMem’s effectiveness depends on reliable instruction following and structured returns, and its gains are not uniform across backbone and benchmark combinations.Future work includes adaptive routing, memory-folding policies, lower-cost auxiliary distillation, and interactive context isolation under changing user intent.
Limitations
Long-horizon LLM agents are limited by context dilution, as growing interaction histories can obscure the planning signals needed for global task control. Existing compression and retrieval methods typically operate on flat trajectories after planning and execution information have already been mixed.
- Context dilution: Growing interaction histories can obscure sparse planning signals with raw tool outputs, retries, and intermediate deliberations.This context dilution limits performance on long-horizon tasks requiring iterative planning, tool use, and information gathering.
- Flat context: Existing compression and retrieval methods usually operate after planning and execution information has already been mixed into a flat trajectory.
- Typed context isolation: HyMem addresses this limitation by isolating planning, tool execution, sub-task reasoning, and memory consolidation into distinct context spaces.Only schema-constrained results and structured memories enter the main planner context.
A Experiment setup · A.1 Benchmarks and Data Selection
The evaluation uses curated BrowseComp-Plus and text-only GAIA subsets, with ablations preserving the same test selections. Results are averaged across at least three independent trials for reliability and consistency.
- A.1 Benchmarks and Data Selection: 150 BrowseComp-Plus tasks form the primary evaluation subset.The subset is curated from the BrowseComp-Plus benchmark.
- A.1 Benchmarks and Data Selection: BrowseComp-Plus tasks are balanced evenly across easy, medium, and hard difficulty levels.The difficulty allocation follows a 1:1:1 ratio.
- A.1 Benchmarks and Data Selection: GAIA evaluation uses its 165-task validation set.The study focuses on the validation set for GAIA.
- A.1 Benchmarks and Data Selection: Only text-only GAIA instances are filtered for and evaluated.This text-only GAIA subset is also used in ablation testing.
- A.1 Benchmarks and Data Selection: Ablation studies retain the same 150 BrowseComp-Plus tasks and text-only GAIA subset.The ablation test selections match those used for evaluation.
- A.1 Benchmarks and Data Selection: Results are reported as means from at least three independent trials.The repeated trials are intended to ensure statistical reliability and consistency.
A.2 Model Configurations & Hyperparameters
The experiments use Qwen3-32B and DeepSeek-V4 as primary backbones, with DeepSeek-V4 handling auxiliary language modeling tasks across frameworks. HyMem constrains planner, execution, and isolated-reasoning turns to regulate progression and computational efficiency.
- Backbone Models: Qwen3-32B and DeepSeek-V4 serve as the primary backbone models, while DeepSeek-V4 uniformly handles auxiliary language modeling tasks.This setup supports procedural consistency and experimental stability across frameworks.
- Hyperparameter Constraints: HyMem uses hyperparameter constraints to regulate task progression and optimize computational efficiency.The constraints govern interaction limits across planning, execution, and isolated reasoning.
- Turn Limits: Planner_turns=12, execute_turns=10, max_iso_reasoning=5, and max_iso_reasoning_turn=3 cap planning, execution, reasoning sessions, and per-session reasoning steps.These limits apply to the planner Layer, Actor Layer, total isolated reasoning sessions, and each individual isolated reasoning session, respectively.
A.3 Context Analysis · B Case Study
HyMem’s context analysis compares how frameworks manage context volume as tasks progress, while the case study examines hierarchical isolation in Planner–Executor coordination. Together, the passages emphasize reducing redundant information while preserving high-value strategic signals for complex, long-horizon reasoning.
- A.3 Context Analysis: Figure 4 evaluates global context-management efficiency using historical trajectories from identical tasks.The comparison includes HyMem, ReAct, ReSum, and AMEM.
- A.3 Context Analysis: Task progress forms Figure 4’s horizontal axis, while context volume forms its vertical axis.This visualization directly relates progress over time to accumulated context.
- A.3 Context Analysis: The aggregate comparison concatenates context-change data from all evaluated tasks into one continuous curve.This provides a holistic view of information-density management over time.
- B Case Study: The case study demonstrates how hierarchical isolation supports context management in complex, long-horizon tasks.Table 3 provides the detailed case-study analysis.
- B Case Study: Planner and Executor layers use architectural decoupling to filter redundant information.The case study frames this decoupling as part of HyMem’s hierarchical context-management design.
- B Case Study: The decoupled layers facilitate exchange of high-value strategic signals during task execution.The passage presents this exchange alongside redundant-information filtering.
- B Case Study: The case study also highlights isolated reasoning and memory management modules.The supplied passage introduces these modules but ends before describing their specific operation.
C Prompt Engineering
HyMem’s prompt engineering specifies dedicated templates for planning, execution, isolated reasoning, and memory management. An execution trace illustrates how these components support staged identification, verification, retrieval, refinement, and final answering.
- Prompt Templates: HyMem defines primary prompt templates for the Planner Layer, Executor Layer, isolated reasoning sessions, and memory management components.These templates are detailed in Tables 4–7.
- Execution Trace: In the example, the agent identifies Tegla Loroupe, verifies constraints, retrieves her 10,000-meter personal-best time, and confirms the date as 26 August 1999.After direct date retrieval fails for 30:32.03, the strategy shifts to searching the exact time and identifies the 1999 World Championships in Sevilla.
- Planner Layer: The Planner Layer prompt directs complex-question analysis, decomposition, targeted retrieval, delegated isoReasons, and memory-guided iteration.It frames the planner as an expert research planner working with a specialized search actor.
- Executor Layer: The Executor Layer prompt assigns a searching agent to locate, extract, and synthesize information through careful query design and iterative reasoning.The executor is equipped with multiple search tools and file-parsing capabilities.
- Isolated Reasoning: The isolated reasoning module supports recursive sub-problem decomposition and independent analysis for verification.Its prompt is presented as the full template for the Isolated Reasoning Module.
- Memory Management: The memory management prompt compresses, structures, and records interaction histories into concise machine-readable representations across episode, working, and Tool memory.The module is designed to support long-term reasoning, short-term planning, and tool-use optimization.