ContextBudget: Budget-Aware Context Management for Long-Horizon Search Agents

1 Introduction

• Context compression summarizes or rewrites past observations, traces, and retrieved evidence to retain task-critical information without external memory or architectural changes.The approach has gained prominence for its empirical effectiveness and conceptual simplicity.Source: §1 · p. 1
• Budget-free compression can over-compress under relaxed budgets or under-compress under tight budgets, causing information loss or context overflow.Prior budget-aware agent methods regulate computation or action counts rather than proactively compressing history for context-window limits.Source: §1 · p. 2
• BACM formulates context compression as a budget-constrained sequential decision, adapting when and how much history to compress as capacity changes.The budget signal also guides which information to preserve for future reasoning.Source: §1 · p. 2
• BACM-RL extends GRPO to optimize compression strategies against downstream task success, using a tightening budget curriculum and a budget-violation penalty.The approach avoids handcrafted compression heuristics.Source: §1 · p. 2
• BACM-RL consistently improves reasoning robustness on compositional multi-objective QA and long-horizon web browsing, with the largest gains under stringent context budgets.The approach uses a budget-constrained RL method with a progressively tightened budget curriculum and overflow-sensitive regularization.Source: §1 · p. 2

2 Related Work

• External-memory approaches extend accessible context but do not update in-context representations during reasoning.They represent context as tiered storage and retrieve relevant history.Source: §2 · p. 2
• The framework observes a budget-conditioned state before loading a pending observation, then selects NULL, PARTIAL, or FULL aggregation; training uses a progressively tightened budget curriculum.Only trajectories satisfying the stage-specific context budget contribute reward and optimization.Source: Figure 1 · p. 3, §2 · p. 3
• Context compression spans single-turn input compression and multi-turn state management, but existing methods treat compression as static or periodically triggered rather than budget-conditioned.This work instead models compression dynamically to adapt the trade-off between retained information and resource constraints.Source: §2 · p. 3
• Budget-aware inference methods regulate output length, computation, tool use, or interaction behavior, but do not adapt context management to context-window limits.When context is the bottleneck, the cited approaches lack mechanisms to adjust history representations to the available budget.Source: §2 · p. 3

3 Budget-Aware Context Management

• The framework combines deferred observation loading, adaptive commit-block aggregation, and a progressively tightened context-window curriculum for long-horizon context allocation.Its unified policy coordinates compression decisions with reasoning actions under changing budget conditions.Source: §3 · p. 4, §3.2 · p. 5

3.1 Budget-Conditioned State with Deferred Loading

• Before loading an observation, the agent uses remaining budget and its token length to refine context so the updated buffer leaves room under the limit.The observation is appended only after the refined context satisfies the space constraint.Source: §3.1 · p. 4

3.2 Commit-Block Aggregation for Budget-Aware Context Compression

• Commit-block actions choose no, partial, or full aggregation, enabling budget-dependent control of compression timing and intensity while coordinating context reduction with task reasoning.The policy preserves full history under high budgets, selectively aggregates under moderate budgets, and fully aggregates under low budgets.Source: §3.2 · pp. 4–5

3.3 Budget-Aware GRPO Objective with Progressive Context Curricula

• GRPO trains across progressively decreasing context budgets, rewarding rollouts by answer F1 subject to stage-specific budget compliance and broadcasting trajectory advantages to tokens.The method uses a multi-turn objective and group-relative normalization to stabilize optimization under tightening constraints.Source: §3.3 · p. 5

4.1 Datasets

• Evaluation combines composite QA tasks, formed from independent questions to lengthen trajectories, with BrowseComp-Plus; QA uses summed token-level F1 and browsing uses average judged accuracy.Training compositions use two objectives, with generalization to larger compositions assessed at test time.Source: §4.1 · p. 6, §4.2 · p. 6

4.3 Compared Methods

• Baselines span unmanaged ReAct and Search-R1, reactive Summary Agent compression, and proactive MEM1, which consolidates history into a fixed-size state.Comparisons use Qwen2.5-7B-Instruct and Qwen3-30B-A3B-Instruct; BACM-RL’s training curriculum tightens budgets from 8k to 4k tokens.Source: §4.3 · pp. 6–7

4.4 Main Results

• BACM-RL achieves the best average performance on BrowseComp-Plus and Multi-objective QA across 7B and 30B backbones, with an approximately 5.0× gain over MEM1 on 32-objective tasks.On BrowseComp-Plus, the 30B variant at 8k achieves 0.147 accuracy versus 0.136 for Qwen3-Inst 235B at 128k.Source: §4.4 · p. 7, Table 1 · p. 7
• At 4k in the 32-objective setting, BACM-RL gains 1.7× cumulative F1 over the best baseline (2.06 vs. 1.21) and remains nearly invariant as budgets fall from 16k to 4k.The comparison covers varying objective counts and context-window budgets.Source: §4.4 · p. 8, Figure 2 · p. 8
• Under a fixed 8k budget, F1 gains accompany fewer compression calls on 2/8-objective tasks but more calls on 16/32-objective tasks.Against MEM1, F1 improves by 8.3%/18.7% at 2/8 objectives while calls fall by 41.7%/35.8%; at 16/32 objectives, F1 improves by 67%/143% while calls rise by 43%/109%.Source: §4.4 · p. 8, Figure 3 · p. 8

4.5 Ablation Study

• Removing budget metadata degrades performance, adding it alone to Search-R1 yields only marginal gains, and budget-conditioned compression achieves the highest cumulative F1 across budgets and objectives.Deferred loading, which accounts for upcoming observation size before loading, consistently outperforms pre-loading.Source: §4.5 · pp. 8–9, Figure 4 · p. 9, Table 3 · p. 9
• With a common 8k evaluation budget, the progressive 8k→4k curriculum achieves the best overall performance (2.662 average), outperforming static and randomized schedules, especially on complex tasks.Randomized training performs worst; progressive training scores 4.011 and 2.938 on 16- and 32-objective tasks.Source: §4.5 · p. 9, Figure 5 · p. 9

4.6 Case Study: Analysis of Model Behavior

• The case study shows the policy preserving commit blocks with sufficient budget, selectively merging earlier blocks under moderate pressure, and compressing all history when budget is tight.The examples correspond to approximately 45.7%, 30.6%, and 28.7% remaining budget, respectively.Source: §4.6 · pp. 9–10

• BACM formulates context compression for long-horizon LLM agents as a budget-conditioned sequential decision problem.Source: §5 · p. 10
• BACM-RL optimizes this context-management framework through reinforcement learning.Source: §5 · p. 10
• Experiments show consistent robustness improvements across context budgets, especially under tight constraints.Source: §5 · p. 10

A.1 Limitations and future work

• Trajectory-level reinforcement learning relies on sparse, delayed rewards, which may limit process-level supervision for context-management decisions.Suggested directions include intermediate supervision, hierarchical reward design, and explicit modeling of long-term information utility.Source: §A.1 · p. 13
• Commit-block aggregation operates at a coarse segment level, potentially limiting preservation of critical information in high-factual-precision tasks.The authors suggest exploring token- or span-level saliency estimation.Source: §A.1 · p. 13
• Evaluation covers multi-objective QA and long-horizon browsing, while open-ended tool use, multimodal reasoning, and human-agent interaction remain future assessment settings.Source: §A.1 · p. 13
• The authors describe budget-aware context management as a promising direction for robust long-horizon reasoning under strict resource constraints.Source: §A.1 · p. 13

A.2 Case study of budget-aware context management

• The agent selects among [NONE], [Selective], and [ALL] as budget conditions vary.Source: §A.2 · p. 13, Figure 6 · p. 14
• As the budget tightens, the illustrated strategy transitions from full retention to partial and then full aggregation.Source: Figure 6 · p. 14
• The case study describes these adaptive choices as preserving task-relevant information.Source: Figure 6 · p. 14

A.3 Efficiency–Performance Trade-offs with Improved Context Utilization

• The proposed method consistently achieves the highest performance across 4k, 8k, and 16k context budgets and varying objective counts.Table 4 reports mean dependent cost (D), mean peak tokens (P), and summed F1 (S).Source: §A.3 · p. 15, Table 4 · p. 15
• At 16 objectives and 16k context, Peak tokens are 3538 for the proposed method versus 4939 for Summary.At 8k, the corresponding Peak tokens are 3443 versus 4161.Source: §A.3 · p. 15
• For 32 objectives, Peak tokens rise from 4339 at 4k to 8795 at 16k, alongside consistent performance improvements.Source: §A.3 · p. 15
• Effective context management requires adaptive token allocation rather than aggressive compression or delayed summarization.Source: §A.3 · p. 15

A.4 Answer rate analysis across context lengths

• Across 2–8 objectives, all methods achieve answer rates above 90%.Source: §A.4 · p. 15
• At 16–32 objectives, baseline answer rates fall, particularly at shorter context lengths, while Mem1 reaches 61.7% at 32 objectives.Figure 7 compares answer rates across 4K–16K context lengths and 2/8/16/32 objectives on the 7B model.Source: §A.4 · p. 15, Figure 7 · p. 16
• At 32 objectives, BACM has lower answer rate than Mem1 but higher task performance, while maintaining higher answer rates than other baselines.The passage attributes the higher task performance to retaining more information; task-performance results are reported in the main experiments.Source: §A.4 · p. 15, Figure 7 · p. 16

A.5.1 Training infrastructure and Core Hyperparameters

• Training uses the verl framework and GRPO across 16 nodes with 8 GPUs each; the global batch size is 128.Table 6 is identified as containing the global RL training hyperparameters.Source: §A.5.1 · p. 16, Table 6 · p. 17

A.5.2 Curriculum Reinforcement Learning Implementation

• BACM-RL uses five 60-step training stages that progressively reduce the maximum context length from 8,192 to 4,096 tokens; all methods receive 300 training steps.The stated schedule decreases the limit by 1,024 tokens per stage.Source: §A.5.2 · p. 16

A.5.3 Search-R1 Implementation

• Search-R1 follows its official implementation and data construction, shares the agent interface, and trains for 300 steps with a fixed 8,192-token limit.Source: §A.5.3 · pp. 16–17

A.5.4 MEM1 Implementation

• The 30B-A3B MEM1 baseline uses <internal state> tags instead of standard <think> tags to trigger its reasoning and memory behaviors more effectively.Source: §A.5.4 · p. 17

A.6 Budget-Aware Context Management Prompt

• Before a deferred tool response is loaded, the budget prompt estimates remaining capacity from current context length, response length, and a 1,000-token safety margin.The prompt asks whether to keep, fold all, or selectively fold prior commits, and says not to fold unless necessary.Source: §A.6 · p. 17

A.7 LLM-as-a-Judge Evaluation Details

• Two independent judges using identical prompts agree on answer correctness within 1%, and BrowseComp-Plus reports close agreement between LLM judgments and human annotations.The evaluation judge receives the question, generated response, and ground-truth answer, then assesses semantic equivalence.Source: §A.7 · p. 18

A.8 Training Dynamics Analysis

• The curriculum maintains reward gains during steps 200–300 as budgets tighten, while the random 4k/8k schedule collapses after step 180 with KL divergence spikes.Static 8k training shows near-zero compression after step 100; curriculum training increases summary-tool calls as the budget falls from 8k to 4k.Source: §A.8 · p. 19, Figure 8 · p. 19