Source-linked AI summary
Plans Don't Persist: Why Context Management Is Load Bearing for LLM Agents
Aman Mehta, Anupam Datta
TL;DR
Long-horizon agents may lose critical plans when context is compressed because plans can remain context-resident rather than become persistent internal state. The paper introduces replay pairing and strict stripping to measure this persistence, finding rapid plan-signal decay, reasoning-trace measurement confounds, and substantial performance loss after naive eviction. Its broader conclusion is methodological: context management is load bearing, and plan protection alone is insufficient.
Problem
The paper asks whether agent plans persist in hidden state or remain context-time text, because context compression and eviction are safe only when dropped information is no longer needed.
Method
The paper uses replay pairing to compare identical trajectories with and without the plan, measures hidden-state cosine distance, and applies strict stripping to remove prior reasoning traces from the stripped run.
Results
The plan signal peaks at 0.453 one step after elicitation and falls 4.1× on ALFWorld and 12.4× on HotpotQA; strict stripping recovers +163% in-sample and +153% held out on R1.
Takeaways & Limitations
Agent-critical information can be context-resident rather than persistent, making context management load bearing and showing that plan protection alone is not enough.
Takeaways & Limitations
The plan-fidelity signal conflates plan content with position, length, and discourse, while stronger content-specific and causal controls remain open.
Abstract
from arXiv · showhide
Long-horizon agents depend on context management: systems compress, summarize, and evict old tokens so tasks can continue beyond finite windows. That is safe only when dropped information is no longer needed or has been internalized. Plans are the stress case: they are written early, used for many steps, and first to be evicted. We introduce replay pairing, a diagnostic that runs the same trajectory with and without the plan in history and measures hidden-state cosine distance. On Llama-3.1-70B, plan signal spikes to 0.453 one step after the plan, then falls 4.1x in a single action-observation step; HotpotQA falls 12.4x. This is evidence that standard LLM agents do not carry plans forward as persistent state, and instead depend on the plan remaining in context. A layer-L32 probe detects this decay as a diagnostic, not as proof that it reads plan content itself. Reasoning models add a measurement confound: their `<think>` traces re-derive plan content, so standard stripping leaves plan evidence in the stripped condition. We name this the reasoning-trace confound and fix it with strict stripping, which removes prior `<think>` blocks from the stripped run only. It recovers +163% of the step+1 signal in-sample and +153% held out, while not meaningfully changing non-reasoning Llama (+4.8%). On DeepSeek-R1-Distill-Llama-70B, a Llama-trained probe transfers at AUROC 0.748 (p=6e-4), while R1-specific probes reach 1.000, suggesting R1 encodes plan signal in a different hidden-state direction. Finally, a compression stress test shows the practical cost: naive plan eviction cuts ALFWorld success by 34.7pp, while probe-gated re-surfacing does not recover it. The contribution is a measurement and stress-test framework showing that agent-critical information can be context-resident rather than persistent. Context management is load bearing, but plan protection alone is not enough.
1 Introduction
The paper asks whether plans persist in agents’ hidden states or remain context-time text that must be reread. Replay pairing finds rapidly fading plan-related signal, while reasoning traces complicate measurement and strict stripping partly addresses that confound.
- Motivation: Context management is risky when plans are needed after their text is compressed, summarized, or evicted.Plans are written early and can be dropped before later actions need them.
- Motivation: Behavioral success cannot distinguish plans internalized in hidden state from plans reread as text from the context window.The mechanisms produce the same actions while the plan remains visible.
- Method: Replay pairing compares matched trajectories with and without the plan and measures per-step hidden-state cosine distance.The diagnostic estimates how much hidden states change when the plan is removed, though controls are needed for non-content differences.
- Findings: 0.453 at step+1 fell 4.1× in one cycle on ALFWorld, while HotpotQA fell 12.4×, indicating rapidly decaying plan-presence signal.The result supports a context-window effect rather than a persistent state, with the caveat that step phase also contributes to probe performance.
- Reasoning-model confound: Reasoning traces can restate plans in prior <think> blocks, causing standard stripping to underestimate the plan signal.Strict stripping removes prior traces from the stripped run and recovers +163% in-sample and +153% held out, while changing Llama by only +4.8%.
- Implications: The paper’s contribution is a representational measurement and compression-stress framework, not evidence that plan protection alone restores behavior.Naive eviction cuts ALFWorld success by 34.7pp, and probe-gated resurfacing does not recover it.
2 Related Work
Related work studies where information resides, how planning affects behavior, and how hidden states encode concepts. This paper complements those lines by measuring plan persistence across forward passes in multi-step agents.
- Context-time memory: Prior context-memory work distinguishes information supplied in the window from information retained in model parameters.Retrieval-head and multi-turn studies emphasize copying from context and failures when relevant content is distributed across turns.
- Planning and faithfulness: Planning frameworks assume explicit plans guide behavior, but behavioral metrics do not identify the mechanism by which plans influence later steps.This paper therefore asks whether plan information persists internally or remains dependent on visible text.
- Hidden-state representations: Hidden-state probing commonly studies semantic encoding across layers within one forward pass, whereas replay pairing studies persistence across time.The paper applies this temporal perspective to multi-step agent behavior.
- Interventions: Intervention and reasoning-model probe studies target hidden states or context, providing neighboring methods for analyzing and modifying agent representations.The present work focuses specifically on plan persistence across agent turns.
3 Method
The method creates matched plan-present and plan-stripped replays, compares their hidden states over time, and adds controls for replay validity and reasoning-trace contamination. Probes regress the resulting signal, while the authors explicitly bound what the diagnostic establishes.
- Models and environments: The study evaluates Llama-3.1-70B, a within-family Llama replication, and DeepSeek-R1-Distill-Llama-70B as ReAct agents.ALFWorld is primary and HotpotQA supplies a direct cross-domain comparison.
- Plan elicitation: A guard elicits a sequential plan, which remains in history while the agent continues the task.The plan is injected at step 2 in ALFWorld and step 4 in HotpotQA.
- Replay pairing: Condition B replays Condition A’s exact actions and observations while removing the plan exchange from history.B’s output is discarded, so both conditions share the same external trajectory.
- Signal definition: The plan-fidelity signal is the cosine distance between plan-present and plan-stripped hidden states at each step and layer.Step +1 follows the guard, and pre-plan steps provide a validation baseline below 10^-3 in practice.
- Diagnostic limits: The signal is a composite that also reflects history length, token positions, and discourse structure, so content-specific controls remain necessary.The pre-plan baseline rules out replay-mechanics confounds but does not isolate plan content.
- Reasoning-model protocol: Strict stripping removes prior complete <think> blocks from B while retaining post-trace Thought and Action lines.It should be a no-op for non-reasoning models and increase the signal for reasoning models.
- Methodological caveat: Strict stripping remains asymmetric and removes whole reasoning blocks, so recovered signal may reflect contamination removal or distribution shift.The paper proposes symmetric stripping, neutral replacement, and plan-span redaction as follow-up controls.
- Probe: A Ridge probe regresses scalar signal magnitude at the peak layer, with a threshold converting predictions into a fire/no-fire decision.Regression is used because step index is trivially decodable from the residual stream.
4 Experiments: Plan Signal Decay
Plan signal peaks immediately after elicitation and decays sharply within one action-observation cycle. The pattern is consistent across ALFWorld task types, including plans of different lengths.
- Overall Decay Curve: 0.453 ± 0.039 at step +1 fell to 0.110 ± 0.032 at step +2, a 4.1× drop in one cycle.The signal settled near ∼0.027 by step +5.
- Task-type analysis: 0.445-0.474 spikes and 4.0-4.4× decay rates were closely matched across all six ALFWorld task types.Complex 6-8-step plans decayed as fully as simple 3-4-step plans.
- Layer analysis: The step +1 peak was largest at layer 32, with signal 0.686 ± 0.024, motivating L32 as the peak layer.L28 and L36 formed a broader mid-network peak, while early and late layers were lower.
5 Detection: Probe for Plan Decay
A Ridge probe uses layer-32 hidden states to estimate plan-signal magnitude and detect active versus decayed plans. It transfers across domains, but step-index leakage and domain-specific calibration limit interpretation.
- Probe design: The probe regresses plan-signal magnitude from layer-32 hidden states and thresholds predictions into fire/no-fire decisions.Training uses 5-fold stratified cross-validation on 80 ALFWorld tasks, with zero-shot transfer to HotpotQA.
- Probe design: L32 is the peak layer, reaching signal 0.686, with all six ALFWorld task types converging to the same residual.
- In-domain performance: R2 = 0.875 and AUROC 0.999 show strong ALFWorld prediction of plan-signal magnitude and active-versus-decayed status.The binary threshold is τ = 0.15, with precision 0.985 and recall 0.825.
- Cross-domain performance: AUROC 1.000 transfers to HotpotQA at both L32 and L40, while L40 achieves F1 0.968 at the ALFWorld cutoff.At L32, precision is 0.513 before isotonic recalibration, which lifts F1 to 0.920.
- Caveats: Step-index leakage complicates interpretation because step index is decodable at R2 = 0.978 and correlates with plan-signal magnitude.A mixed-effects control holding step index and other factors fixed remains unrun.
- Early warning: The probe fires 4.45 steps before deviation on average and leads deviation in 74.2% of deviating tasks.Among 80 tasks, 31 deviated; the median lead was 5 steps.
6 Cross-Domain and Cross-Model Validation
Validation across domains and model scale preserves the qualitative plan-signal pattern while shifting decay rate and peak depth. The probe direction generalizes, but operating points require recalibration.
- Cross-domain validation: On HotpotQA, plan signal is 0.445 at step +1 and drops 12.4× by step +2, while the peak moves from L32 to L34.The initial magnitude nearly matches ALFWorld’s 0.453, but decay is faster when observations are unrelated to the plan.
- Cross-domain validation: The ALFWorld-trained probe transfers zero-shot to HotpotQA with AUROC 1.000 and F1 = 0.920 after isotonic recalibration.The direction generalizes across domains, while only the operating point needs recalibration.
- Cross-scale validation: On Llama-3.1-8B, plan signal peaks at 0.382 and decays 10.9× by step +5, matching the 70B qualitative shape.The peak occurs at L12 of 32, approximately 37.5% relative depth, versus approximately 40% for 70B.
7 Plan Maintenance in Reasoning Models
Reasoning traces contaminate standard replay pairing by reintroducing plan content into the stripped condition. Strict stripping restores much of the measured signal, while probe transfer indicates reasoning models encode it along different directions.
- Reasoning-trace confound: Standard replay pairing makes R1 appear 4.5× weaker than Llama, with step +1 signal 0.022 versus 0.099.Prior <think> blocks re-state plan content, so both replay conditions retain evidence of the plan.
- Strict stripping: Strict stripping raises R1 step +1 signal from 0.022 to 0.058, a +163% change, shrinking the Llama-R1 gap from 4.5× to 1.7×.Held-out tasks show +153% recovery, while non-reasoning Llama changes by only +4.8%.
- Strict stripping: The residual 1.7× gap may reflect plan content retained in Thought:/Action: text, which strict stripping does not remove.Redacting plan spans wherever they occur is identified as the clean follow-up.
- Probe transfer: A Llama-trained probe reaches AUROC 0.748 on strict-stripped R1 states, whereas R1 self-probes reach 1.000 along a direction 89.3° from Llama’s.Qwen3-native instead shows persistent plan-conditional drift, indicating spike-decay timing is specific to distilled R1.
- Behavioral check: Behavioral effects remain small and nonsignificant across reasoning and non-reasoning models, supporting a representational rather than behavioral claim.Across 25 tasks, strict stripping changes Qwen3-thinking to 2/25 and leaves R1 at 0/25, with Fisher p=0.49.
8 When Context Management Fails: A Compression Stress Test
The compression stress test shows that evicting context sharply harms ALFWorld success, while protecting or re-surfacing the plan does not restore performance at the same budget.
- Stress-test design: 150 runs across 30 ALFWorld tasks tested four matched-budget context policies, including naive eviction, plan protection, and probe-gated re-surfacing.The budget kept the system prompt and four recent messages.
- Performance cost: 56.7% →22.0% (−34.7pp, p < 0.001): naive eviction nearly thirds task success.This result supports the predicted cost of removing plan tokens from the context window.
- Policy comparison: Neither plan-aware policy recovered performance versus naive eviction, with p=0.89 and 0.67 comparisons.The figure caption reports that both policies failed to recover performance at the same context budget.
- Interpretation: Probe-gated re-surfacing fired 6.1 times per run, yet restoring the plan did not recover the loss.The omitted recent observations and actions appear to be the more relevant working state under this budget.
- Interpretation: The stress test therefore indicates that plan protection alone is not a sufficient compression fix, making plan-fidelity monitoring diagnostic rather than turnkey.The tested policies did not restore the context needed for successful task execution.
9 Discussion and Limitations
The discussion generalizes replay pairing beyond plans while emphasizing that the evidence remains representational, model- and domain-limited, and vulnerable to probe and measurement confounds.
- Implications: Replay pairing can test whether safety instructions, constraints, and tool schemas also live only in context, while reasoning traces may partially substitute for retained plans.The authors identify self-refreshing scratchpads as an open compression question after probe-gated re-surfacing failed to recover static-plan loss.
- Limitations: The contribution is representational rather than behavioral: strict stripping had small, nonsignificant task-success effects, and uniform L32 steering was null.Content-specific controls and stronger causal tests remain future work.
- Reasoning-model controls: Strict stripping removes prior <think> blocks only from condition B while retaining externally visible action-observation content; it is a +4.8% no-op on Llama.This protocol addresses reasoning-trace contamination without changing the non-reasoning comparison materially.
- Limitations: Existing archival Phi-3-Medium and Qwen2.5-72B states cannot support plan-signal claims because paired contrasts are undefined when rerun activations are bitidentical.Fresh replay-paired data are needed for other architectures, including mixture-of-experts and RL-tuned reasoning models.
- Probe validity: L32 probe validity remains limited by step-index leakage, although graded residual structure, shuffled-label controls, and monotonic confusion provide supporting checks.The unresolved mixed-effects control would hold step index, task type, plan length, and observation-plan overlap fixed.
- Probe calibration: At L32, the main binary probe is near-perfectly calibrated, whereas the harder +2 versus +3 boundary has ECE = 0.066 and requires isotonic recalibration.The main probe reports Brier = 2.5× 10−6 and ECE = 0.0002.
- Cross-model scope: The Llama-trained probe transfers to R1 at AUROC 0.748, while R1 self-probes reach 1.000 along a direction 89.3◦ from Llama’s L32.R1-Distill-Qwen-32B shows the same self-probe ceiling; Qwen3-native instead shows R2=0.997 but AUROC 0.616.
- Cross-model scope: The per-layer dilution test rules out strong-form uniform dilution: the R1/Llama ratio varies 4.4× and the models peak at L72 and L32, respectively.This comparison uses a five-task ALFWorld subset.
A.8 Intervention Sweeps and Head-Level Analysis
Intervention sweeps find that measuring plan-related hidden-state structure does not reliably provide a behavioral control knob. The signal is distributed across heads and varies across tasks and model scale, with faster decay when observations are less plan-relevant.
- Cross-model comparison: Llama-3.1-8B shows the same decay shape at smaller scale, with peak 0.382 and 10.9× decay at matched relative depth around 33%.Phi-3 and Qwen2.5 cannot be assessed because their archival data are not replay-paired.
- Head-level structure: Head-aware interventions also fail to reliably move task success despite respecting mixed-sign head contributions.Low-magnitude intervention gives +3.3 pp within noise, while β=2.0 flips to −4.4 pp and larger scaling destabilizes generation.
- Head-level structure: The plan direction is distributed across L32 heads: top-3 captures 23% and top-10 captures 55% of the absolute contribution sum.The mixed-sign structure motivates treating the direction as a measurement of productive and antagonistic plan signal rather than an arbitrarily scalable intervention.
- Task comparison: HotpotQA decays 12.4× versus ALFWorld’s 4.1× despite near-identical initial signals, plausibly because its observations are less semantically aligned with the plan.ALFWorld observations directly relate to the plan, whereas HotpotQA passages can be informationally orthogonal and overwrite plan-relevant features faster.
B Phase 1 Cross-Condition Plan-Signal Post-Processing
The cross-condition analysis separates independent forward-agent loops from replay-paired post-processing and verifies behavioral results from directly saved trajectories. In the context-compression stress test, naive eviction is harmful, while plan-aware policies do not recover performance.
- Cross-condition setup: Phase 1 uses independent forward-agent loops for six model and reasoning-condition configurations, while replay-pair semantics belong to a separate analysis.Behavioral task-success results are reported in Table 8 for the independent-trajectory conditions.
- Behavioral results: Strict stripping moves Qwen3-thinking from 0% to 8%, matching its no-thinking baseline, but does not move R1-Distill-Llama-70B.These are Phase 1 behavioral task-success results on n=25 ALFWorld tasks.
- Behavioral measurement: Behavioral success is computed directly from saved trajectory results using max_t score(t) > 0, independently of the post-processor.The raw per-run success flag is decoupled from goal completion, so real success is explicitly redefined by maximum score.
- Stress-test design: The compression stress test evaluates four context policies on 30 ALFWorld tasks, with 150 runs per policy under a 20-step cap.All policies retain four recent messages plus the system prompt; probe_gated re-injects the pinned plan when the L32 Ridge probe fires at τ=0.15.
- Stress-test results: Naive plan eviction is catastrophic, while neither plan-aware policy recovers performance.Table 9 reports p < 0.001 for naive eviction, and both plan-aware policy confidence intervals cross zero.