Source-linked AI summary
Scaling Long-Horizon LLM Agent via Context-Folding
Weiwei Sun, Miao Lu, Zhan Ling, Kang Liu, Xuesong Yao, Yiming Yang, Jiecao Chen
TL;DR
Long-horizon LLM agents face degraded performance and poor efficiency as interaction histories grow. Context Folding lets agents branch into localized subtasks and fold completed branches, while FoldGRPO trains this behavior with process rewards; on two long-context tasks, the approach matches or exceeds larger-context baselines and improves efficiency relative to summary-based condensation.
Problem
Long-horizon agent frameworks accumulate interaction histories, causing degraded performance, poor efficiency, and context-limit failures as trajectories expand.
Method
Context Folding uses branch and return actions to handle localized subtasks and remove intermediate branch steps, while FoldGRPO trains folding with dynamic contexts and token-level process rewards.
Results
0.620 pass@1 on BrowseComp-Plus (+20%) and 0.580 on SWE-Bench Verified (+8.8%); after reinforcement learning, the agent outperforms the reported baselines, including same-length long-context ReAct.
Takeaways & Limitations
Context folding allows agents to match or exceed larger-context baselines while improving efficiency and stability relative to summary-based condensation.
Takeaways & Limitations
The evaluated parallel-branching setting may favor depth-first tasks; breadth-first tasks such as WideSearch may be more promising for studying parallelism.
Abstract
from arXiv · showhide
Large language model (LLM) agents are fundamentally constrained by context length on long-horizon tasks. We introduce Context-Folding, a framework that empowers agents to actively manage their working context. An agent can procedurally branch into a sub-trajectory to handle a subtask and then fold it upon completion, collapsing the intermediate steps while retaining a concise summary of the outcome. To make this behavior learnable, we develop an end-to-end reinforcement learning framework FoldGRPO with specific process rewards to encourage effective task decomposition and context management. On complex long-horizon tasks (Deep Research and SWE), our folding agent matches or outperforms the ReAct baselines while using an active context 10$\times$ smaller and significantly outperforms models that rely on summarization-based context management.
1 Introduction
Long-horizon LLM agents are constrained by growing interaction histories that degrade performance and efficiency. Context Folding lets agents branch into localized subtasks, fold completed trajectories into concise summaries, and learn this behavior through FoldGRPO.
- Motivation: Growing interaction histories create long-context challenges, including degraded performance and inefficient attention and KV-cache management.Existing frameworks linearly accumulate reasoning, tool calls, and observations in one context as task horizons increase.
- Context Folding: Context Folding uses branch and return actions to offload localized subtasks and retain only concise, decision-relevant summaries.Intermediate branch steps are removed from the working context after the agent returns to the main thread.
- Training: FoldGRPO trains context folding with dynamic folded contexts and dense process rewards for decomposition, focus, and summary preservation.Its rewards include penalties for unfolded token-heavy operations and out-of-scope branch behavior.
- Results: 62.0% and 58.0% pass@1 scores were achieved on BrowseComp-Plus and SWE-Bench Verified with a compact 32K active-token budget and at most 10 branches.The Folding Agent surpassed baselines requiring a 327K context window and outperformed context-summarization methods.
- Results: The results support active context management as a scalable path for long-horizon agency across deep-research and coding benchmarks.The paper reports that learning to manage context is preferable to merely extending or heuristically compressing it.
2 Methodology
The method replaces ever-growing ReAct histories with actively managed branches whose intermediate interactions are folded into concise return messages. FoldGRPO jointly trains this behavior using folded contexts and token-level process rewards.
- Vanilla Formulation: Vanilla ReAct appends every action-observation pair, causing long-horizon contexts to exceed limits and weakening reasoning and instruction following.The growing history also creates long-context efficiency challenges.
- Context Folding: Context Folding uses branch to create a separate subtask trajectory and return to fold its intermediate steps into the main thread.The return appends the branch outcome while removing the branch’s intermediate context.
- Context Folding: The context manager F folds action-observation history between branch and return calls before generating subsequent tokens.The folded history replaces the complete prior interaction history in the agent model.
- Context Folding: Context Folding supports a plan-execution design in which planning decomposes tasks and execution handles localized subtasks without nested branching.Token-intensive operations are discouraged in the planning state, while new branches are disabled during execution.
- FoldGRPO: FoldGRPO jointly optimizes main-thread and branch trajectories while maintaining compact folded contexts during reinforcement learning.It augments GRPO with dynamic folded contexts and dense process rewards for branching behavior.
- FoldGRPO: Token-level rewards penalize unfolded main-thread operations, out-of-scope branch actions, and failed tool-call turns.The unfolded-token penalty begins when the main thread exceeds 50% of the working-context limit.
- Method Comparisons: Unlike predefined multi-agent workflows or arbitrary summarization, context folding creates sub-agents on demand and compacts reasoning at subtask boundaries.Agents share a context prefix, and reasoning is preserved until its utility is realized.
3 Experiment
The experiments evaluate the folding agent on deep research and agentic software engineering using matched training infrastructure and context-managed baselines. They also examine performance across task difficulty levels.
- Evaluation Setup: The evaluation uses BrowseComp-Plus for deep research and SWE-Bench Verified for agentic software engineering.The benchmarks use different task-specific tools and verifiable reward procedures.
- Evaluation Setup: BrowseComp-Plus is split into 680 training instances and 150 evaluation instances, while SWE training uses 740 collected instances.SWE-Bench Verified serves as the evaluation set for software engineering.
- Evaluation Setup: Test instances are grouped into easy, medium, and hard levels using ReAct acc@8 for BrowseComp-Plus and time-to-resolve for SWE-Bench Verified.SWE difficulty thresholds are ≤15 minutes, 15 minutes–1 hour, and ≥1 hour.
- Baselines: The comparisons include ReAct agents with 32K, intermediate, and 327K contexts, plus summary agents with a 32K context and 10 summary sessions.The directly comparable baselines use the same base model, data, infrastructure, and RL hyperparameters.
- Results: Table 1 reports performance on BrowseComp-Plus (N=150) and SWE-Bench Verified (N=500), with parentheses comparing against the 327K ReAct baseline.Boldface identifies the best-performing 36B models.
- Results: RL training produces consistent performance gains across easy, medium, and hard instances, with significantly larger improvements on medium and hard subsets.The reported pattern highlights stronger gains on more difficult problems.
4 Experimental Results
Across BrowseComp-Plus and SWE-Bench Verified, reinforcement learning improves context folding while the agent learns more extensive interaction and compact context management. Folding maintains strong performance across increasing difficulty and context demands, with over 90% compression in analyzed behavior.
- Main results: +20% and +8.8% pass@1 after RL training on BrowseComp-Plus and SWE-Bench Verified, respectively.The RL-trained agent outperforms all baselines, including long-context ReAct with the same 327K maximum length.
- RL ablation: +7.7% on BrowseComp and +1.6% on SWE-Bench from FoldGRPO over baseline GRPO.The analysis associates performance gains with increased tool-call frequency encouraged by RL training.
- Task difficulty: RL training yields consistent gains across easy, medium, and hard instances, with significantly larger improvements on medium and hard subsets.The difficulty breakdown compares performance before and after reinforcement learning.
- RL ablation: Over 90% context compression results from reducing the main trajectory to about 8K tokens while processing over 100K in total.FoldGRPO improves scope accuracy and finish rate by encouraging focused branching and condensed main context.
- Scaling with context: Pass@1 consistently surpasses ReAct as context length increases, but performance plateaus beyond 320K tokens.The plateau occurs because most task instances are already completed and additional context provides limited benefit.
- Case study: 107K tokens compress to 6K through 4 branches in a BrowseComp-Plus case study.The agent explores, verifies, expands the search, and finds the correct answer across the folded branches.
- Parallel branching: Parallel branching achieves 0.6133 Pass@1 on BrowseComp-Plus, performing similarly to single-branch folding while reading 110 versus 80 web pages.The parallel-branch agent creates about 2.3 parallel branches on average.
5 Related Work
Prior context-management methods mainly summarize histories or distribute work across specialized agents. This paper frames context management as a learned cognitive skill trained directly with process-level rewards.
- Existing paradigms: Context summarization and multi-agent collaboration are the two main paradigms for managing agent context.Summarization uses external memory stores, while collaboration divides tasks among specialized agents with focused contexts.
- Research gap: These paradigms leave a gap for an integrated approach that treats context management as a learned cognitive skill rather than an external feature.The contrast is between learned internal management and architectural or retrieval-based mechanisms.
- Intrinsic skills: RL research has focused mainly on extrinsic task success, while training intrinsic skills such as working-memory management remains underexplored.The paper addresses this gap by using process-level rewards to teach context management directly.
6 Conclusions and Future Work
The paper concludes that context folding selectively removes ephemeral sub-trajectories while preserving essential decision-relevant information. Combined with RL, it supports efficient long-horizon agents and motivates hierarchical folding as future work.
- Conclusion: Context folding selectively folds ephemeral sub-trajectories while preserving essential decision-relevant information.The mechanism manages long-horizon trajectories through selective condensation.
- Conclusion: On two long-context tasks, folding matches or exceeds larger-context baselines while improving efficiency and stability relative to summary-based condensation.The conclusion attributes these outcomes to context folding coupled with reinforcement learning.
- Future work: Multi-layer context folding is proposed as a future direction for hierarchical strategies in which folds can be folded again for deeper compression.This extends the folding mechanism recursively.
A.1 Multi-Trajectories Collection
For practical training, folded sub-trajectories are retained as separate causally conditioned sequences rather than concatenated into one sequence. This makes context-folding training incompatible with existing infrastructures such as Verl.
- Implementation: Folded sub-trajectories are kept as separate causally conditioned sequences instead of being concatenated into one sequence.This implementation choice supports practical model training under the context-folding formulation.
- Implementation: Context-folding training is not directly compatible with existing training infrastructures such as Verl.The incompatibility follows from the separate-sequence representation.
A.2 Asynchronous Long-Horizon Agent Rollout
Asynchronous rollout mitigates computation bubbles caused by imbalanced long-horizon job durations. The setup allows limited off-policy updates without observed performance degradation.
- 95% of prompts are completed by the main rollout process before remaining jobs are handled by a standalone process.The threshold is adjusted based on GPU configuration to reduce waiting caused by slower jobs.
- Up to 5 off-policy steps are permitted, with no observed performance degradation versus fully on-policy training.
B Prompt Engineering
The prompt workflows organize BrowseComp-Plus research and SWE-Bench coding into staged processes for decomposition, iterative investigation, verification, implementation, and review.
- B.1 BrowseComp-Plus Workflow: BrowseComp-Plus begins by deconstructing the query, identifying entities and constraints, and brainstorming search vectors.
- B.1 BrowseComp-Plus Workflow: The BrowseComp-Plus research phase uses iterative search and page opening, with queries refined across multiple tool calls.Search snippets are treated as insufficient, requiring analysis of promising source documents.
- B.1 BrowseComp-Plus Workflow: BrowseComp-Plus follows an OODA loop that repeatedly observes information gaps, orients, chooses the next action, and acts with tools.
- B.1 BrowseComp-Plus Workflow: The workflow continuously synthesizes information, triangulates critical claims across independent sources, and pursues alternatives when research reaches dead ends.
- B.1 BrowseComp-Plus Workflow: Before final reporting, the agent verifies every checklist item, conducts mandatory re-research for weak evidence, and constructs a supported answer.
- B.2 SWE-Bench Workflow: SWE-Bench prompts follow phases for reading the issue, running tests, exploring relevant files, and creating a reproduction script.
- B.2 SWE-Bench Workflow: The SWE-Bench workflow then analyzes the fix, implements minimal focused source changes, verifies them with tests, and performs a final review.
- B.2 SWE-Bench Workflow: Final SWE-Bench review checks the problem requirements, modified code, and related tests, revising the implementation if tests fail.
C.1 BrowseComp-Plus
The evaluation workflows equip BrowseComp-Plus and SWE-Bench agents with search, document access, coding, execution, reasoning, and context-folding tools. Their finish actions return task results that are evaluated against retrieved evidence or unit-test outcomes.
- C.1 BrowseComp-Plus: BrowseComp-Plus provides search and open_page tools for retrieving ranked documents, snippets, and fuller document contents.Search returns document identifiers, URLs, and content; open_page fetches a document by identifier or URL from prior search results.
- C.1 BrowseComp-Plus: The BrowseComp-Plus finish tool returns a succinct answer, evidence-grounded explanation, and confidence score for evaluation.
- C.2 SWE-Bench: SWE-Bench agents use execute_bash, str_replace_editor, and think, while reward is based on unit tests run in an instance-specific sandbox.
- C.2 SWE-Bench: The str_replace_editor supports viewing, creating, replacing, inserting, and undoing edits in plain-text files, with exact-match requirements for replacements.
- C.2 SWE-Bench: The SWE-Bench finish process evaluates the resulting git diff by applying it in another Docker environment and running unit tests.
- C.3 Context Folding: Context folding adds branch and return tools so agents can execute subtasks and return a comprehensive subtask outcome.