Source-linked AI summary
StateComp: Learning When to Compress History in Long Horizon Agents
Mingxuan Wang, Hongyue Chen, Yinglong Guo, Fei Luo, Chao Ning, Bo Wang, Guorun Yao, Yanbiao Ma, Jungong Han
TL;DR
Long-horizon agents face growing context costs, and the need to retain past interactions changes with the current state. StateComp reassesses interactions for safe compression and separates readiness prediction from executing a rewrite. On WorkBuddyBench, it reduces agent and summarization tokens by 52.27% while mean reward changes from 0.6987 to 0.7026.
Problem
Long-horizon agents accumulate interaction histories whose future importance changes, creating the challenge of deciding which interactions are obsolete while histories also impose substantial token and inference costs.
Source: §1 · p. 1Method
Results
On 260 WorkBuddyBench tasks, total agent and summarization tokens decreased from 698.17M to 333.24M, a reduction of 52.27%, while mean reward changed from 0.6987 to 0.7026.
Source: §1 · p. 2Takeaways & Limitations
The findings support separating prediction of compression opportunities from the decision to rewrite a sufficiently large span.
Source: §6 · p. 11The experiments do not isolate span gates’ effect on cache reuse, and deferred rewrites do not preserve rewritten-suffix KV states or guarantee higher cache-hit rates.
Source: §4.2 · p. 7
Abstract
from arXiv · showhide
Long-horizon agents continuously accumulate interaction history during task execution, yet the importance of past interactions changes as the agent state evolves. Existing context management methods largely compress history based on fixed windows, periodic schedules, or current relevance, overlooking a more fundamental question: when has a past interaction become safe to replace? Premature compression may remove information still needed for future actions, while overly conservative retention leads to substantial context overhead. To address this, we propose State Conditioned Compression (StateComp), a framework that determines when historical interactions can be safely compressed according to the current agent state. StateComp constructs KEEP and READY supervision through a two-stage annotation procedure and trains an imbalance-aware router on hidden representations from a frozen language model. A bounded state representation further reduces the cost of evaluating long histories, while adjacent READY interactions are grouped into continuous spans and replaced with compact summaries during execution. Experiments on WorkBuddyBench show that StateComp reduces total agent and summarization tokens by 52.27% while maintaining task performance, and achieves a 12.67-fold speedup in representation extraction.
1 Introduction
StateComp decides when historical interactions can be compressed by reassessing them against the current agent state. Its WorkBuddyBench evaluation reduces agent and summarization tokens by 52.27% while mean reward changes from 0.6987 to 0.7026.
1 Introduction
- Compression timing depends on the current state because an interaction’s retention needs can change as task execution proceeds.An error trace may be needed during diagnosis but become replaceable after its cause and conclusion are recorded, while an early constraint may remain important.Source: §1 · pp. 1–2
- StateComp determines when historical interactions can be safely compressed according to the current agent state.It separates predicting compressibility from deciding whether to execute a compression operation.Source: §1 · p. 2
- Two-stage annotations identify supported KEEP and READY boundaries, and an imbalance-aware router favors precise selection of compressible history.This addresses asymmetric errors: unnecessary retention costs context, while deleting needed information can harm later actions when raw history cannot be retrieved.Source: §1 · p. 2
- The router uses hidden representations from a frozen language model, with a bounded state representation to reduce representation-extraction cost.Adjacent READY interactions are grouped into continuous spans and summarized to replace raw interactions.Source: §1 · p. 2
- 52.27% fewer total agent and summarization tokens (698.17M to 333.24M) accompany mean reward changing from 0.6987 to 0.7026 across 260 WorkBuddyBench tasks.The paper also examines Eval40, representation efficiency, and multiple acting models.Source: §1 · p. 2
2 Related Work
Related work addresses prompt compression, agent context management, and adaptive selection of historical information. StateComp focuses on deciding when a complete interaction can leave its original form at a particular checkpoint.
2.1 Prompt Compression
- Prompt compression can shorten inputs by selecting information, but does not by itself specify when original interaction details can be replaced during task execution.StateComp targets this temporal decision for complete interactions, using the current state to determine replaceability and a separate summarizer to construct retained content.Source: §2.1 · pp. 2–3
2.2 Context Management for Long Horizon Agents
- Long-horizon context-management approaches operate during agent execution, while StateComp separates a positive readiness prediction from actually triggering summarization.Committed summaries become part of the effective history used by later actions and routing decisions.Source: §2.2 · p. 3
2.3 Adaptive Selection of Historical Information
- StateComp asks whether an interaction still needs its original form at the current checkpoint, rather than whether it is relevant to the task.Its annotation checks evidence available in the prefix and uses future dependencies to reject unsafe candidates; removed originals are not retrieved online.Source: §2.3 · p. 3
3 Supervision for State Dependent Compression
The supervision treats compression as a state-dependent judgment about whether a complete past interaction can be replaced without losing information needed later. It identifies the earliest supported safe point and uses two annotation stages to ground positive labels in evidence available at that checkpoint.
3.1 Compression Depends on the Current State
- A complete interaction includes reasoning, an assistant action, and its observation, and its KEEP or READY label depends jointly on the interaction and the current state.The same interaction may become READY as later observations resolve uncertainty or make its original details redundant; similar-aged interactions can still have different retention needs.Source: §3.1 · pp. 3–4
3.2 The Earliest Safe Compression Point
- An interaction’s earliest supported compression point is the first checkpoint where its full details are judged unnecessary; absent such a point, it remains KEEP throughout the trajectory.The boundary is supported for the observed continuation, not guaranteed for every possible future continuation; the dataset has 244,526 interaction-state pairs, 15,633 positive labels, and 1,832 interactions with a supported point.Source: §3.2 · p. 4
3.3 Two Stage Annotation with Supporting Evidence
- Positive labels require evidence already present in the checkpoint prefix, since later outcomes cannot establish that earlier deletion was safe.Stage 1 records checkpoint-specific judgments, evidence, and future-dependency vetoes; Stage 2 reviews those records across checkpoints to select the earliest supported boundary, leaving unresolved cases KEEP.Source: §3.3 · p. 4
- The annotation favors precision because retaining replaceable history mainly costs context, whereas removing still-needed information can harm later actions when raw history is not retrievable.The overview links structured annotations and boundary review to router training and gated online replacement.Source: §3.3 · p. 5, Figure 1 · p. 5
4 StateComp Methodology
StateComp predicts whether historical interactions are ready for compression using representations conditioned on the current state, then applies separate gates before rewriting history. Committed summaries update the effective history used in subsequent actions and routing.
4 StateComp Methodology
- StateComp separates interaction-level readiness predictions from compression execution, allowing local opportunities to accumulate before a larger rewrite.This is intended to reduce unnecessary summary calls and fragmented updates to active history.Source: §4 · p. 5
4.1 Hidden-State Representations and Router Training
- A frozen language model supplies historical-interaction and current-state representations for the router’s checkpoint predictions.Full-context extraction becomes expensive as histories grow, including recorded inputs around 128K tokens and a largest input of 203,675 tokens; bounded views combine a target interaction, current state, and limited effective history.Source: §4.1 · pp. 5–6
4.2 Interaction Routing and Span Execution Gates
- Selected adjacent interactions form continuous candidate spans, and span-length and source-token gates determine whether a prediction warrants summarization.The gates are intended to avoid summary calls whose cost could exceed savings and to reduce fragmented replacements; rewriting earlier history can invalidate KV reuse from the first changed position.Source: §4.2 · pp. 6–7
4.3 Updating the Effective Agent History
- After a summary is committed, later actions and router decisions use the updated effective history, and remaining interactions are reconsidered at later checkpoints.The original trajectory is retained separately for auditing, not as an online retrieval store.Source: §4.3 · p. 7
5 Experiments
Experiments show compression opportunities generally increase over trajectory, frozen hidden states contain predictive information about readiness, and StateComp substantially reduces token use while keeping aggregate reward close to the baseline.
5.2 Hidden-State Prediction and Representation Efficiency
- 76.40 to 6.03 seconds total extraction time, with bounded-representation F1 0.7818 versus 0.7000 for the full-context reference.Runtime is measured on seven paired states; the bounded and full representations are not information-equivalent.Source: Table 1 · p. 8, §5.2 · p. 9
5.1 Experimental Setup
- WorkBuddyBench covers 260 tasks across four domains, and router evaluation uses five grouped folds with all pairs from each trajectory kept in one split.Each fold rotation has 200 training, 50 validation, and 50 test trajectories.Source: §5.1 · p. 8
5.2 Hidden-State Prediction and Representation Efficiency
- READY labels are scarce near trajectory beginnings and generally become more common across all four domains as execution proceeds.The domains follow different trajectories, so task progress alone is not a sufficient decision rule.Source: Figure 3 · p. 8, §5.2 · p. 8
- Grouped evaluation yields AUROC 0.7096 and PR AUC 0.1736; a selective configuration reaches 72.22% external precision at 0.11% recall.The results indicate predictive information in frozen hidden states, but not sufficient signal for unrestricted compression.Source: §5.2 · p. 9
5.3 Task Performance and Context Cost
- Token reductions span all four domains, ranging from 38.89% in Code to 55.75% in Office, while reward changes vary by domain.Source: §5.3 · p. 9
- 52.27% fewer total agent and summary tokens, with mean reward changing from 0.6987 to 0.7026 on WorkBuddyBench.The full comparison covers 260 tasks and includes main-agent and summary-call tokens.Source: §5.3 · p. 9, Table 2 · p. 10
- On Eval40, StateComp obtains reward 0.7154 with 43.76M tokens, the highest mean reward in the comparison group.Sliding windows, periodic summarization, and PACE show different reward and token tradeoffs.Source: Table 3 · p. 10, §5.3 · p. 10
5.4 Cross-Model Evaluation
- Tokens decrease for all four acting models on Eval40, while the magnitude and domain distribution of reward changes remain model dependent.Reported examples include GLM-5.3-Flash reward 0.6170 to 0.6492 with tokens 47.58M to 38.49M, and Qwen3.8-Flash reward 0.7249 to 0.7784 with tokens 49.75M to 38.23M.Source: Table 4 · p. 11, §5.4 · p. 11
6 Conclusion
STATECOMP predicts whether an interaction can be compressed at the current agent state, using two-stage supervision to identify supported boundaries. The conclusion highlights span-based replacement and reports roughly half the agent and summary tokens while maintaining average reward.
6 Conclusion
- The full benchmark comparison reduces agent and summary tokens by approximately half while maintaining average reward.The conclusion supports separating readiness prediction from the decision to rewrite a sufficiently large span; experiments measure complete-task token usage but do not isolate span gates’ effect on cache reuse.Source: §6 · p. 11
AI Use Statement
- Generative AI tools assisted with language editing, literature discovery, and annotation, with the authors reviewing the procedure, labels, and AI-assisted content.Source: AI Use Statement · pp. 11–12
Reproducibility Statement
- Appendices describe supervision, grouped evaluation, representation construction, router objectives, analysis, online execution, benchmark protocols, and token-accounting conventions.Source: Reproducibility Statement · p. 12
A.1 Interactions and Effective History
- An interaction comprises reasoning, an assistant action, and an observation, with tool calls and responses kept within complete interaction boundaries.Online decisions use the effective history after committed replacements; stored original trajectories are not available as an online retrieval channel.Source: §A.1 · p. 17
- READY means replacing an interaction’s details with an accurate summary, while facts still needed for the task must remain available in the summary or effective history.Source: §A.1 · p. 17
A.2 Two-Stage Annotation
- Stage 1 judges each historical interaction at each checkpoint using task-state evidence; future interactions may veto a replacement but cannot justify it using unavailable future information.Source: §A.2 · pp. 17–18
- Stage 2 reviews proposed boundaries across checkpoints for the same interaction, selects the earliest supported compression point, and leaves unresolved cases KEEP.The ∞ label denotes no supported boundary; monotonicity applies to the annotated continuation, not every possible future trajectory.Source: §A.2 · p. 18
A.3 Data and Grouped Evaluation
The appendix describes trajectory-grouped router evaluation and the design choices for feature construction, class weighting, operating thresholds, bounded inputs, and decision stability. Its bounded-input stability result is conditional on a fixed router and stated perturbation limits.
A.3 Data and Grouped Evaluation
- An always-KEEP classifier reaches approximately 93.61% accuracy without identifying any compression opportunities, so the study uses ranking and precision–recall metrics rather than accuracy alone.All pairs from each trajectory stay in one partition, evaluating generalization across trajectories rather than randomly separated pairs from the same trajectory.Source: §A.3 · p. 18
B.1 Historical and Current-State Features
- Router features combine the target interaction, current-state representation, and available historical features; these operations construct predictions but do not restore removed raw interactions.The feature notation includes a target vector, aggregated historical vectors, and a router score; state dependence comes from current-state information or freshly constructed inputs containing current context.Source: §B.1 · p. 19
B.2 Weighted Classification
- Class weighting changes score interpretation, but the population identity does not establish calibration of the fitted router.The weighted-BCE experiments vary the READY-class weight while keeping the representation model frozen.Source: §B.2 · p. 20
B.3 Asymmetric Costs and Operating Thresholds
- Class weighting changes the training loss, while validation data determine operating choices rather than a known posterior or assumed numerical cost ratio.Source: §B.3 · p. 21
C.1 Input Construction and Computational Cost
- Bounded extraction uses target-specific views of at most 5,120 tokens, but total work still depends on the number of surviving historical targets.One full-context pass can supply all historical readouts; bounded extraction instead uses multiple short views, which may be batched.Source: §C.1 · p. 21
C.2 Stability of Decisions and Selected Spans
- For a fixed router, decisions and selected spans remain unchanged when each feature perturbation stays within the stated threshold margin and source history and execution gates are unchanged.The empirical ablations compare separately fitted variants and measure performance; they do not establish the proposition’s perturbation bound.Source: §C.2 · pp. 21–22
C.3 Representation Measurements
Bounded representation extraction reduces measured runtime, while representation-quality comparisons show that feature variants differ across metrics. The appendix also characterizes variation in readiness timing and selected-span behavior across trajectories.
C.3 Representation Measurements
- Across 2,558 states from 110 trajectories, mean bounded input was 4,181.5 tokens; 73 reached the 5,120-token limit and 178 current-step inputs were truncated.Source: §C.3 · p. 22
- 12.67×: total representation extraction time falls from 76.40 to 6.03 seconds on seven paired states, not complete-task execution.The paired timing includes states where both implementations completed on the same device.Source: §C.3 · p. 22, Table A3 · p. 22
- The capped hybrid achieves the highest F1, while capped same-prefix selection achieves the highest balanced accuracy; adding recurrent or historical features is not uniformly beneficial.These are comparisons of particular feature constructions, not evidence that bounded inputs preserve all information in the full history.Source: §C.3 · p. 22, Table A4 · p. 23
D.1 Readiness and Compression Delay
- Readiness fractions vary across trajectories; percentile bands describe that variation, while normalized delay, span count, span length, and coverage track distinct timing and selection properties.Figure A1 summarizes READY-fraction variation; Figure A2 tracks normalized delay and span dynamics over task progress.Source: Figure A1 · p. 23, Figure A2 · p. 24, §D.1 · p. 24
D.2 Router Configurations
The router analysis reports ranking and selective operating-point results, then describes calibration diagnostics and the online span-compression procedure. Accepted replacements preserve protocol validity and do not expand history at a checkpoint.
D.2 Router Configurations
- Weighted BCE with w1 = 10 reaches AUROC 0.7096 and PR AUC 0.1736, while a separate operating point has 72.222% precision at 0.111% recall.The operating-point measurements are not attributed to a single fitted configuration and do not guarantee safe online compression.Source: §D.2 · p. 24
D.3 Precision, Coverage, and Calibration
- The calibration figures report ECE values of 0.035, 0.038, and 0.011 for raw, temperature-scaled, and Platt-calibrated scores, while the curves establish no high-coverage 98% precision guarantee.The 98% line is a reference target, not an attained guarantee.Source: §D.3 · p. 25, Figure A5 · p. 26, Figure A4 · p. 26
E.1 Candidate Spans and Execution
- The controller scores surviving complete interactions, groups adjacent selected interactions into maximal spans, and summarizes only candidates meeting the configured thresholds.Summaries retain facts, constraints, locations, confirmed conclusions, failure causes, and unresolved items; replacements become effective context for later routing and actions.Source: §E.1 · p. 27
E.2 Structural Preservation and Call Bounds
- Each checkpoint returns a protocol-valid history no longer than its input, becoming strictly shorter when a nonempty replacement is committed.The proposal bound applies to the one-call-per-span algorithm and excludes transport retries.Source: §E.2 · p. 27
Algorithm 1 One checkpoint of STATECOMP
- The checkpoint algorithm preserves the input as fallback, forms eligible spans, summarizes them, and accepts replacements only when the trial history is valid and shorter.Feature-extraction, routing, or summary-generation failures leave the history unchanged; final protocol checks precede commitment.Source: Algorithm 1 One checkpoint of STATECOMP · p. 28
E.3 Interaction Errors and Summary Fidelity
The paper formalizes risks from interaction-label errors and summary fidelity, then gives token break-even conditions and evaluates fixed task sets. Full260 results show large token reductions with a small aggregate reward change, while Eval40 controls expose summary overhead and short-window performance losses.
E.3 Interaction Errors and Summary Fidelity
- The interaction-error bound separates incorrect READY labels from harmful joint replacements despite correct labels; its error quantities are assumptions, not calibrated estimates from aggregate precision.The joint-replacement term includes summary omissions and dependencies, such as replacing both copies of a necessary identifier.Source: §E.3 · pp. 28–29
E.4 Amortized Tokens and Prefix Caching
- For constant per-request savings ∆, a committed replacement saves tokens exactly when R∆ > Csum under an otherwise unchanged request sequence.Actual online runs can change actions, retries, and termination times, so complete-task token use is measured separately; token reductions do not imply monetary savings.Source: §E.4 · p. 29, §F.1 · p. 30
F.1 Task Sets and Metrics
- WorkBuddyBench contains 80 Code, 50 Office, 60 Security, and 70 Web tasks; the overall score is a task-weighted aggregate, and failed outcomes remain in fixed-set comparisons.Eval40 has ten tasks per domain.Source: §F.1 · p. 29
F.2 Full260 Results
- 52.2694%: combined tokens decrease from 698,167,232 to 333,239,366, while mean reward changes from 0.698701 to 0.702556.Security reward decreases slightly while the other three domain rewards increase.Source: §F.2 · p. 30, Table A6 · p. 30, Table A7 · p. 30
F.3 Eval40 Controls and Summary Overhead
- The Eval40 controls compare methods on the same 40 tasks and report summary tokens within total tokens; the StateComp run uses approximately 0.077M summary tokens versus 3.01M for periodic summarization at n = 5.The full control table also shows performance losses for short windows and overhead from repeated summarization.Source: §F.3 · p. 30, Table A8 · p. 31, Table A9 · p. 31
F.4 Cross-Model Results
Across four acting models on the fixed Eval40 set, compression lowers total token use, while reward changes vary by model. The operating settings specify thresholds for routing, span formation, and source size, with a bounded representation budget.
F.4 Cross-Model Results
- All four acting models use fewer total tokens with compression, while reward changes and their domain distribution remain model dependent.The comparisons use the same fixed Eval40 tasks, with ten tasks in each domain; summary usage is included in total tokens.Source: §F.4 · p. 31, Table A11 · p. 32
G.1 Operating Settings
- Full260, Eval40, and cross-model comparisons use τ = 0.60, κ = 3, and Bmin = 1,000 tokens, requiring at least four complete interactions per span.The representation budget is at most 5,120 tokens per target view.Source: §G.1 · p. 32
G.2 Fixed Eval40 Task Identifiers
Eval40 is a fixed benchmark subset containing ten tasks from each domain; the task-identifier table specifies which tasks comprise that subset.
G.2 Fixed Eval40 Task Identifiers
- The fixed task list contains ten tasks from each domain and specifies the benchmark subset used for Eval40 comparisons.Table A12 lists the fixed Eval40 task identifiers.Source: §G.2 · p. 32, Table A12 · p. 33