PaperlayerView exact revision on arXiv

Source-linked AI summary

Memory Control Signals Emerge Before Action in Long Horizon Agents

Mingxuan Wang, Guorun Yao, Fei Luo, Yinglong Guo, Chao Ning, Bo Wang, Hongyue Chen, Yanbiao Ma, Jungong Han

arXiv:2609.27286v1cs.AI

TL;DR

Long-horizon agents face growing histories, and it is unclear whether models already represent compression and recall needs before memory operations occur. The paper probes preaction states and proposes PaMER and PaMER+ to combine state-guided compression with selective evidence retrieval. Across WorkBuddyBench evaluations, the systems substantially reduce context consumption while maintaining competitive task performance.

  • Problem

    Growing interaction histories increase computational cost and complicate preserving and reusing relevant information, while whether models represent memory needs before operations remains unresolved.

    Source: §1 · pp. 1–2

  • Method

    The paper probes preaction hidden states and proposes PaMER for state-guided compression with external evidence retrieval, with PaMER+ selecting historical steps for restoration.

    Source: §1 · p. 2

  • Results

    On WorkBuddyBench, the framework substantially reduces context consumption while maintaining competitive task performance across multiple baselines and model backbones.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Recent context preserves most predictive information for memory decisions, while selected historical evidence recovers long-range information it misses.

    Source: §1 · p. 2

    The probing results are predictive and do not identify the model’s causal mechanism.

    Source: §3.2 · p. 5

Abstract

from arXiv · show

Long horizon language model agents continuously accumulate interaction history, increasing computational cost while making relevant information harder to preserve and reuse. Existing context management methods mainly focus on how to compress or retrieve history, but largely leave open whether the model itself already represents the need for these memory operations before they occur. We study the hidden state immediately before each agent action and find that compression and recall needs are already encoded in the model's internal representations. These signals cannot be explained by simple context length or interaction progress, and they exhibit distinct formation patterns across model depth. We further show that most memory decision information is preserved in a compact recent context, while selectively restored historical evidence complements the long range dependencies that recent context misses. Based on these findings, we propose Preaction Memory with Evidence Retrieval (PaMER), which combines state guided compression with external evidence retrieval. PaMER+ further introduces step level evidence selection to recover only the historical information required by the current task. Experiments on WorkBuddyBench, across multiple context management baselines and model backbones, show that our framework substantially reduces context consumption while maintaining competitive task performance.

1 Introduction

Long-horizon agents must decide both when to compress growing interaction histories and when to recover earlier evidence, yet it is unclear whether models represent those needs before acting. The paper finds predictive memory signals in preaction states, identifies what history preserves them, and uses these findings to motivate PaMER and PaMER+.

1 Introduction

  • Growing histories raise computation, latency, and memory costs and may make relevant information harder to use; compression and recall are separate, task-dependent decisions.Trajectories of equal length can differ in whether they contain critical evidence or redundant and outdated material.Source: §1 · pp. 1–2
  • Preaction hidden states predict compression and recall needs beyond context length, turn count, and tool type, with signals developing differently across model depth.Compression information generally strengthens with depth, while recall emerges earlier and peaks at an intermediate layer.Source: §1 · p. 2
  • Approximately 26% of Full Context—the task prefix and two most recent complete interaction blocks—preserves most predictive performance for both memory decisions.Selected raw historical evidence recovers additional predictive information when long-range dependencies matter again.Source: §1 · p. 2
  • PaMER uses preaction representations to decide when to compress, moves older history to retrievable external memory, and PaMER+ selectively restores task-relevant historical steps.The approach is evaluated on WorkBuddyBench against multiple context-management baselines and across several model backbones.Source: §1 · p. 2

2 Related Work

Prior work addresses context compression, historical retrieval, and probing hidden representations, but the paper distinguishes its focus on predicting both memory needs within long agent trajectories.

2.1 Context Compression for Long Horizon Agents

  • Long-horizon context management has moved beyond fixed length thresholds toward task-state-dependent compression and adaptive context operations.The cited approaches include adaptive trajectory compression and systems that organize, restore, or select context [15, 23, 24, 4, 25, 5, 26].Source: §2.1 · p. 3

2.2 Memory Retrieval and Historical Evidence

  • Memory systems manage historical evidence and retrieval dynamically, while this paper studies compression and renewed need for evidence previously removed from active context.Probing RAG examines external-document retrieval for question answering; this work instead examines two memory needs in long agent trajectories [40].Source: §2.2 · p. 3

2.3 Probing Internal Representations

  • Representation probes can reveal encoded information, but probe accuracy alone does not establish the model’s causal mechanism [41, 42, 43].Here, the probe target is preaction information about upcoming compression or recall needs, so the claims are predictive rather than causal.Source: §2.3 · pp. 3–4

3 Understanding Memory Needs in Long Horizon Agents

The section tests whether preaction states encode compression and recall needs, how those signals develop across layers, and how much history is needed to preserve them. It finds that hidden states outperform observable controls, while compact recent context retains much of the decision information and selected older evidence restores dependencies beyond that window.

3.1 Memory Decision Data and Probing

  • The analysis defines compression as condensing older context while preserving task-relevant information, and recall as needing earlier evidence no longer in active context.Recall is evaluated only when valid external memory exists; the study collects 72,912 decision points from 2,521 trajectories covering 600 questions.Source: §3.1 · p. 4

3.2 Memory Signals beyond Observable Context

  • Final hidden states reach 0.831 AUROC for compression and 0.765 for recall, exceeding metadata’s 0.758 and 0.549, respectively.A nonlinear MLP adds almost no gain over the linear readout.Source: §3.2 · p. 5, Table 1 · p. 5

3.3 How Memory Signals Form across Depth

  • Compression prediction strengthens with depth, from AUROC around 0.57 in the earliest measured representation to 0.831 at the final state, while recall peaks at block 16 at 0.780.Final-state recall AUROC is 0.765; the different patterns indicate that the two memory signals develop differently across the network.Source: §3.3 · p. 6, Figure 1 · p. 6
  • The final input position is more informative than mean pooling: compression AUROC is 0.831 versus 0.712, and recall AUROC is 0.765 versus 0.696.The final state is used for the online controller to provide a consistent interface across model depths, although it is not the best layer for every target.Source: §3.3 · p. 6

3.4 What Historical Information Is Needed

  • Recent2 uses 26.0% of the Full Context token budget and retains compression and recall AUROCs of 0.776 and 0.761, versus 0.829 and 0.780 with Full Context.Selected Raw Top5 reaches 0.832 compression AUROC and 0.769 recall AUROC; Hidden Bank Recent5 uses the smallest budget but loses more decision information.Source: §3.4 · pp. 6–7, Table 2 · p. 7

4 PAMER: Preaction Memory with Evidence Retrieval

PaMER uses the model’s current state to guide history compression and external memory access; PaMER+ selects individual historical steps for restoration.

4 PAMER: Preaction Memory with Evidence Retrieval

  • PaMER uses the current model state to decide when to compress history, retaining information externally for later access; PaMER+ selects individual historical steps for restoration.A short recent context supports compression decisions, while selected older evidence remains available when information has left active history.Source: §4 · p. 8

4.1 State Guided Compression and Retrieval

  • Compression operates on earlier complete interaction blocks, preserving the assistant and tool-call structure while retaining the task prefix and two most recent blocks.For candidates of at least 1,024 tokens, five trained heads read the final state from the prefix and recent blocks; compression triggers when at least three exceed their thresholds.Source: §4.1 · p. 8

4.2 PAMER+: Step Level Evidence Selection

  • On later requests, the controller retrieves three summaries by similarity; PAMER+ uses a selected summary to identify candidate memory and restores only selected complete historical blocks.The summary locates relevant memory, while selected raw steps provide supporting evidence; the current implementation does not use the learned recall probe as a retrieval gate.Source: §4.1 · p. 8, §4.2 · p. 8

5 Experiments

Experiments compare task performance and token use across WorkBuddyBench, context-management baselines, model backbones, and online behavior. PAMER reduces token use with a similar full-benchmark score, while PAMER+ raises the reported overall score at a slightly higher token budget than PAMER.

5 Experiments

  • The evaluation covers the full WorkBuddyBench benchmark, Pilot40 comparisons on a fixed task set, and results across several model backbones.Reported tokens are mean total input and output tokens per task, including the main agent and auxiliary memory calls.Source: §5 · p. 9, §5.1 · p. 9

5.2 Main Results

  • On all 260 WorkBuddyBench tasks, PAMER reduces token use while retaining a similar overall score, and PAMER+ raises the reported overall score at a slightly higher token budget than PAMER.Effects differ across domains, so the aggregate result does not imply improvement on every task type.Source: §5.2 · p. 9, Table 3 · p. 9, §5.1 · p. 9

5.3 Baselines and Cross Model Evaluation

  • On Pilot40, periodic summarization scores higher but uses more tokens, CoMem uses fewer tokens but scores substantially lower, and PAMER variants pair relatively strong performance with a smaller budget than the base agent.Table 4 compares methods on the same fixed 40 tasks.Source: §5.3 · p. 10, Table 4 · p. 10
  • Across backbones, PAMER+ improves MiMo-V2.5 and Qwen3.8-Flash scores, while GPT-5.6-Luna loses performance under both variants; token consumption falls, with quality changes depending on the model.The controller uses the same frozen Qwen3.5-9B feature extractor across these evaluations.Source: §5.3 · p. 10, Table 5 · p. 11

5.4 Ablation and Online Behavior

  • Compression repeatedly reduces active history before context grows again, and removing compression raises token use while removing recall lowers task reward.The component patterns are consistent with complementary roles for managing accumulated history and restoring information that left active context.Source: §5.4 · p. 11

6 Conclusion

The conclusion connects preaction prediction of memory needs with practical context management, while noting that task-performance changes depend on model and domain.

6 Conclusion

  • Preaction hidden representations contain memory-decision information beyond context length and interaction metadata, with distinct patterns across model depth.Recent context preserves much of this predictive signal, while selected historical evidence recovers information that recency alone misses.Source: §6 · pp. 11–12

AI Use Statement

  • The authors report using generative AI for language editing and literature discovery, and an LLM annotator for memory-decision labels; authors reviewed the annotation and AI-assisted content.The authors take responsibility for the final work.Source: AI Use Statement · p. 12

Reproducibility Statement

  • The paper and appendices report data construction, splits, probe and evaluation protocols, retrieval procedures, controller configuration, token accounting, and benchmark task sets for reproduction.Hyperparameters and implementation settings are reported where applicable.Source: Reproducibility Statement · p. 12

A Memory-Decision Data and Probe Analysis

The appendix describes a memory-decision corpus and probes of preaction states, with labels marking when compression or recall is needed. Its analysis uses grouped splits and notes that probe results measure predictive information, not a causal circuit.

A.1 Annotated Memory Decisions and Decision-State Reconstruction

  • The corpus contains 72,912 decision points from 2,521 trajectories covering 600 questions, separate from WorkBuddyBench evaluation tasks.The appendix frames its corpus, representation probes, and supplementary evaluations as measurements of predictive information rather than identification of a causal circuit.Source: §A.1 · p. 17, A Memory-Decision Data and Probe… · p. 17
  • An LLM annotator marks decision states for compression or recall, and a trajectory may contain multiple positive decision points.Compression means older context should be condensed while preserving task-relevant information; recall means earlier evidence absent from active context is needed again, and is evaluated only when valid external memory exists.Source: §A.1 · p. 17
  • Training, validation, and test use grouped 70/15/15 partitions, keeping states and trajectory replicas with the same query_id together.Within-trajectory states remain statistically dependent.Source: §A.1 · p. 17

A.2 Probe Architecture and Training

This section specifies the frozen-model readouts and training protocol, then describes controls and layerwise diagnostics for memory-decision signals. It also qualifies weighted-score interpretation and the limits of five-split comparisons.

A.2 Probe Architecture and Training

  • The primary feature is the final input-token state after output RMSNorm from frozen Qwen3.5-9B, and only readout parameters are trained.The online feature extractor is separate from the task-executing model, so the controller does not require hidden-state access to an API-backed acting model.Source: §A.2 · p. 17

A.4 Observable Controls, Probe Capacity, and Memory Hints

  • The Full Context faithful-hint MLP is the compact-context reference, and its reported F1 is highest among Table 8 hint/capacity settings for both targets, though not every ranking metric.The faithful-hint linear compression probe has higher AUROC; AUROC is the primary ranking metric and AUPRC complements it for imbalanced targets.Source: §A.2 · pp. 17–18, Table 8 · p. 20

A.3 Properties of the Weighted Readout

  • Class weighting changes the optimal score’s probability interpretation and does not certify calibration of a finite fitted probe.Threshold selection by validation F1 remains part of the protocol.Source: §A.3 · pp. 18–19

A.5 Layerwise Diagnostic Measurements

  • At every measured intermediate checkpoint with both pooling measurements, last-token linear readouts outperform mean pooling; recall peaks at block 16, while compression AUROC rises across listed intermediate checkpoints.The final state is used for online compression; the compact-context Full Context reference is separate from the layerwise comparison.Source: §A.5 · p. 19, Table 9 · p. 20

A.6 Paired Compact-Context Comparisons

  • The five-split paired analysis requires sign exchangeability and cannot establish a two-sided p < 0.05 result under its calculation.With five splits, the observed sign pattern and its global negation yield pflip ≥ 2/25 = 0.0625.Source: §A.6 · p. 21, Table 10 · p. 21

B.1 Context Construction and Evaluation

The appendix compares compact contexts, distilled decision probes, and retrieved historical evidence, then details controller and step-selection safeguards. It distinguishes offline input and agreement measurements from online task outcomes and notes limits on what structural guarantees establish.

B.1 Context Construction and Evaluation

  • Recent2 uses 26.0% of Full Context textual input and retains much of the memory-decision signal; selected raw evidence outperforms evaluated hidden-bank alternatives, while other additions do not consistently improve over Recent2.Token budgets differ across conditions and are reported alongside predictive performance.Source: §B.1 · p. 22

B.2 Decision Distillation

  • Recent2 compression heads are trained with a loss combining annotation supervision and a fixed Full Context teacher score, but distillation does not improve every metric.Compression F1 improves more than compression AUROC, while recall AUROC decreases in the Recent2 KD condition; the teacher is used only during training.Source: §B.2 · pp. 22–24

B.3 Decision and Ranking Stability

  • The decision and ranking stability conditions assume the same readout and a stated representation bound, so they are not numerical guarantees for the separately trained experimental probes.The sufficient conditions concern agreement under compact-context perturbations.Source: §B.3 · p. 24

B.4 Retrieval Controls and Evidence Budgets

  • Top3 is an intermediate retrieval operating point, not a measured AUROC maximizer; the 16K offline evidence cap keeps agreement close to uncapped, whereas 8K reduces it.These are offline evidence-budget measurements, not a sweep of online task reward; the offline Qwen retrieval returns raw text, unlike the canonical online controller’s stored summaries.Source: §B.4 · p. 25, Table 13 · p. 25

C.1 Compression Voting and Deferred Retrieval

  • PAMER’s compression heads read the Recent2 state, and stored summaries become available for retrieval on later requests rather than the request that compressed them.The learned recall probe does not gate retrieval, and a successful compression request does not perform same-request recall.Source: §C.1 · p. 26

C.2 Stability of Voting and Retrieval

  • The five-head majority bound permits dependent head errors but does not claim the ensemble must outperform every individual head.The controller also preserves the original request when a memory operation fails.Source: §C.2 · p. 27

C.3 Step-Level Evidence in PAMER+

  • PAMER+ selects complete raw blocks explicitly named by the step selector; selecting a summary alone does not authorize restoring every covered step.The implementation snapshot places step selection in the LLM-router integration, and its historical system comparisons do not isolate that component’s effect.Source: §C.3 · p. 28

C.4 Storage, Evidence Selection, and Token Accounting

  • Storage and provenance validation protect against failed updates removing active history, but they do not guarantee summaries retain all facts or selected evidence suffices for the task.Raw-evidence volume reduction does not include possible selection-call and serialization costs.Source: §C.4 · p. 29

C.5 Implementation Settings and Failure Handling

The implementation and evaluation appendix distinguishes token accounting from active-context measurements and defines fixed benchmark task sets. It also describes cross-model evaluation using a shared frozen feature extractor.

D.1 Benchmark and Task Aggregation

  • Full-benchmark comparisons use the same fixed 260 WorkBuddyBench tasks, while Pilot40 comparisons use the same fixed 40 tasks with ten from each domain.No method-specific task filtering is applied, and the recorded 256-turn outer-agent limit is not a task count.Source: §D.1 · p. 31

D.2 Token and Context Measurements

  • Pilot40 token usage is aggregated over its fixed 40 tasks, with per-task usage calculated by dividing aggregate totals by 40.Percent changes are calculated from unrounded totals before display rounding; cached input and auxiliary operations are counted as specified in the accounting protocol.Source: §D.2 · p. 32
  • The DeepSeek-V4-Flash Full260 baseline contains 698,167,232 total tokens, approximately 2.69M tokens per task.Per-turn active context differs from cumulative task tokens, and token reduction is not identified with an equal percentage reduction in billed cost.Source: §D.2 · p. 32

D.3 Cross-Model Scores on Fixed Pilot40

  • Cross-model comparisons evaluate five acting-model backbones on the same fixed Pilot40 tasks, and use the same frozen Qwen3.5-9B feature extractor across models.Cross-model deployment therefore does not require transferring a probe between different acting-model hidden spaces.Source: §D.3 · p. 32

D.4 Component Ablation

The component ablation reports distinct reward and token-use changes when recall or compression is removed. The accompanying context-dynamics analysis describes per-turn averaging, and a single trajectory illustrates historical evidence becoming accessible again.

D.4 Component Ablation

  • Removing recall lowers reward from 0.7000 to 0.6370, while removing compression increases tokens per task from 0.899M to 3.180M.The authors describe these patterns as consistent with complementary roles for context reduction and historical access, without establishing a universal causal effect.Source: §D.4 · p. 32, Table 18 · p. 34

E.1 Context Dynamics

  • Per-turn context means include only tasks with observed active-token values at that turn, so the number of contributing trajectories can decrease as tasks end.The 128K line is an analysis reference, not a controller-enforced limit.Source: §E.1 · p. 34

E.2 Compression and Recall in One Trajectory

  • In the illustrative 94-turn Question 558 trajectory, the final answer matches the recorded answer, and selected events show old evidence becoming accessible after leaving active context.The trace includes 81 searches, 16 document retrievals, four compression calls, and two memory queries; it is qualitative, not an additional evaluation cohort.Source: §E.2 · p. 34, Table 19 · p. 34

E.3 Fixed Pilot40 Task Identifiers

The paper defines fixed task manifests for Pilot40 and WorkBuddyBench, distinguishing the 40-task comparisons from the full-benchmark evaluation.

E.3 Fixed Pilot40 Task Identifiers

  • Pilot40 contains 40 tasks, with ten tasks per domain, and uses the same manifest for method comparison and cross-model evaluation.The full-benchmark evaluation uses a fixed 260-task WorkBuddyBench list.Source: §E.3 · p. 34, Table 20 · p. 35
  • The fixed Pilot40 identifiers correspond to the same task set across both the method comparison and cross-model evaluation.The identifier table defines that shared manifest.Source: §E.3 · p. 34, Table 20 · p. 35

Related papers

Loading 2609.27286v1…