Source-linked AI summary
When Goals Leak Across Relationships: How Cached Goal Representations Shape Cross-Client Decisions
Saloni Gupta, Sébastien Krier, Glenn Matlin
TL;DR
When clients with conflicting goals share an agent’s persistent state, information associated with one relationship may affect decisions for another. The paper tests this in a two-client market setting, probes KV representations, and selectively ablates goal state. It reports cross-client deviations and finds that changing cached goal representations changes which client’s objective the model follows.
Problem
When an agent retains context across relationships, it is unclear what should remain shared versus client-specific while decisions stay aligned with the current client.
Source: §1 · p. 1Method
Results
Takeaways & Limitations
The findings motivate separating relationship-specific state in persistent agents while preserving information that should remain shared across interactions.
Source: §8 · p. 8Recomputation and persistent execution use different inference paths without established token-level equivalence, so rate differences cannot be attributed solely to cache persistence.
Source: §4 · p. 4
Abstract
from arXiv · showhide
As LLM agents take on more autonomous roles, they may interact with multiple clients and counterparties whose goals and incentives conflict. If an agent maintains shared inference state across these interactions, information associated with one relationship may influence decisions made for another. We test OLMo-3-7B-Instruct in a controlled two-client environment with 20 market scenarios, where each client is assigned an opposing objective. With shared conversational memory, the model deviates from the current client's objective in 22% of eligible decisions. We then investigate how these goals are represented in the KV cache. Linear probes show that client-goal ownership becomes highly recoverable in middle layers and is concentrated in particular attention heads. We use these results to identify where to intervene in the cache and test whether these representations affect the model's decisions. Ablating the competitor's cached goal representations at these locations shifts the model toward the current client. The effect is larger for heads identified by probing than for randomly selected heads, and larger when ablating goal tokens than matched non-goal tokens from the same client. Our results show that client-specific goal representations persist in the KV cache and can causally influence decisions made for other clients, highlighting the importance of relationship-specific state isolation in persistent LLM agents.
1 Introduction
The paper asks how an agent can retain shared context across client relationships without letting one client’s goals shape decisions for another. It tests cross-client interference in a shared KV cache, locates goal-ownership representations, and intervenes on them to examine their effects.
1 Introduction
- Shared persistent state leaves conflicting client goals available to later attention, creating a potential memory-isolation problem.The motivating challenge is to retain useful context while keeping decisions aligned with the client currently served.Source: §1 · p. 1
- 22.0% of eligible decisions under persistent KV reuse deviate from the current client’s objective, and cached competitor-goal representations can shift decisions when ablated.The interference also appears with recomputation from text; probe-selected heads and goal tokens have larger effects than their matched controls.Source: §1 · p. 2
- Linear probes find client-goal ownership recoverable in middle-layer K/V state and concentrated in particular attention heads.Selective ablations test whether those representations influence later choices.Source: §1 · p. 2
- Because client identities and goals are explicit inputs, the findings do not show that the model discovers latent social relationships.The supported claim concerns explicit relationship–goal assignments structured in persistent KV state and able to transmit interference.Source: §1 · p. 2
2 Related Work
Prior work studies persistent agent memory, competing context, KV-state sharing and privacy, and methods for probing and intervening on internal representations. This paper examines whether client goals can leak naturally across relationships that share conversational memory.
Agent memory and multi-principal interaction
- Prior research connects persistent retrieval and KV memory [1] [2] [3] with competing-context distraction and interference between conflicting objectives [4] [5].The paper focuses on the resulting multi-client question: whether goals leak when relationships share conversational memory.Source: §2 · p. 2
KV-cache persistence and latent sharing
- Shared KV and latent-memory work shows that state can carry useful information across interactions [8], while studies report privacy risks and behavior changes from modifying transported KV state [11].CachePrune also identifies cache segments for safe reuse based on sensitivity and contextual dependence.Source: §2 · pp. 2–3
Mechanistic probing and causal interventions
- Linear probes test what information is recoverable [13] [14] [15], while causal interventions test whether manipulating representations changes behavior [18] [19].Work on entity binding and retrieval motivates separating ownership representations from downstream use [22]; the paper does not identify a complete binding or retrieval mechanism.Source: §2 · p. 3
3 Experimental Setup
The setup uses opposing client goals in market-style scenarios, alternates client turns, and evaluates each client’s choices using shared persistent KV state. It combines behavioral choice rates with a continuous target-versus-competitor preference measure for interventions.
3 Experimental Setup
- Twenty market-style goal-conflict families receive BASE and SWAP assignments, yielding 40 configurations and 320 randomized episodes for mechanistic experiments.Each configuration uses eight randomized six-turn A/B orders, totaling 1,920 client turns.Source: §3 · p. 3
- The schematic liquidity-versus-growth example illustrates opposing client objectives and decision options, while experimental scenarios use fixed prompt templates.Base/swap assignments invert which client owns each goal.Source: Table 1 · p. 3, §3 · p. 3
- The experiment uses frozen OLMo-3-7B-Instruct [23] with deterministic decoding and reuses retained past key values as client turns are appended.Earlier goal-token K/V entries persist across subsequent A/B switches rather than being reconstructed from the full transcript.Source: §3 · p. 3
- Behavioral outcomes count target-aligned, opposite-client-aligned, and mixed choices; intervention preference is M = log P(target) − log P(opposite), with ΔM = M_intervention − M_baseline.Positive ΔM indicates a shift toward the current client’s objective relative to the competitor’s.Source: §3 · p. 4
4 Behavioral Interference Persists Under KV Reuse
Under persistent KV reuse, the model sometimes departs from the current client’s objective, counting only cases where isolated-client controls succeed. Interference also appears under transcript recomputation, but differences between the two implementations cannot be attributed solely to cache persistence.
4 Behavioral Interference Persists Under KV Reuse
- 22.0% of 150 eligible persistent-KV decisions deviate from the current client’s objective (95% CI [16.1, 29.3]%).11.3% choose the opposite client’s objective outright; the remaining deviations choose the mixed option. Eligibility requires the corresponding isolated-client control to follow the target.Source: §4 · p. 4, Table 2 · p. 4
- The behavioral result spans 20 scenario families and counts failures only when isolated-client controls show the model can follow the target objective without the competing client’s context.Table 2’s target-deviation measure comprises mixed and opposite-client choices and reports Wilson 95% confidence intervals.Source: §4 · p. 4, Table 2 · p. 4
- Cross-client interference also occurs with transcript recomputation, but its rates cannot be compared as a cache-only effect because the inference paths differ and token-level equivalence is unestablished.Recomputation applies the chat template to the accumulated transcript; persistent execution incrementally appends tokens to a retained cache.Source: §4 · p. 4
5 Goal Ownership Is Localized in Contextualized KV State
Linear probes test whether client ownership is recoverable from contextualized K/V features of goal tokens and identify where that information becomes accessible. Auxiliary probes distinguish goal ownership from explicit client markers.
5 Goal Ownership Is Localized in Contextualized KV State
- Probes used span-mean-pooled concatenated K/V features from goal tokens, excluding client-label tokens, with scenario-family-grouped cross-validation.A 100-shuffle ownership-label permutation baseline defined the empirical null.Source: §5 · pp. 4–5
- Ownership decodability rose from near chance at layer 0 (AUROC 0.481) to 0.982 at layer 15, with a sharp rise around layers 9–11.The layer-15 AUROC exceeded the null 95th percentile of 0.521; no permutation shuffle exceeded it.Source: Figure 2 · p. 5, §5 · p. 5
- Client markers were decodable at layer 0, but goal ownership became strongly decodable only in middle layers, peaking at layer 15.The relationship-history probe was an auxiliary control, not evidence for the primary mechanistic claim.Source: §5 · p. 5
6 Causal Ablation of Relationship-Conditioned Cache State
The section tests whether cached competitor-goal representations affect checkpoint decisions by selectively zeroing K or V state and measuring changes in preference. It compares layer windows, probe-selected heads, matched non-goal tokens, and removal of the current client’s own goal state.
6.1 Layer localization
- Across layers 12–20, competitor-goal ablation shifted preference toward the current client (∆M +0.464 for K; +0.704 for V).The strongest causal effects coincide with the middle layers where goal-ownership AUROC was highest.Source: Figure 3 · p. 6, §6.1 · p. 6
6.2 Probe-selected heads beat matched random heads
- Top3 outperformed Random3: K shifted ∆M by +0.224 vs +0.087, and V by +0.138 vs +0.030; paired gains were +0.137 and +0.107, both with confidence intervals excluding zero.Top3 heads were selected using probing results; Random3 matched the layers, tokens, head count, and intervention type.Source: §6.2 · pp. 6–7
6.3 Goal state exceeds matched non-goal context
- Goal-minus-non-goal shifts favored goal ablation in all four comparisons: +0.122 for K and +0.283 for V in layers 12–20, and +0.111 for K and +0.222 for V at Top3 heads.The comparison held layers, heads, and token-position count fixed, changing only which competitor tokens were ablated.Source: §6.3 · p. 7
6.4 Relationship-specific directionality
- Removing the current client’s own goal state reversed direction: ∆M was −0.363 for K and −0.511 for V, negative across all 20 scenario families.This contrasts with competitor-goal ablation shifting preference toward the current client, countering the explanation that cache perturbation alone favors the target.Source: §6.4 · p. 7
- Competitor-V ablation corrected 23 of 166 initially non-target-aligned decisions (13.9%) without reversing any aligned decisions; K corrected 13 but reversed two.Action changes were less frequent than shifts in the log-probability margin.Source: §6.4 · p. 7
7 Discussion and Limitations
The experiments support a narrow causal claim: modifying goal-derived state in specific K/V regions changes which client objective the model follows. The findings are bounded by the controlled setting and the zero-ablation intervention.
7 Discussion and Limitations
- A next step is separating relationship-specific state while preserving useful shared memory.Source: §7 · p. 7
- Experiments use one 7B model and controlled two-client market scenarios with clearly opposing objectives, rather than more realistic multi-client settings.Client ownership is explicitly provided, and zero-ablation directly modifies cached state rather than modeling a naturally occurring inference change.Source: §7 · p. 7
- The interventions do not determine whether interference comes from incorrect goal-client binding, inappropriate retrieval, or competition between correctly represented objectives.Identifying the responsible mechanism remains future work.Source: §7 · pp. 7–8
- Probe-selected heads shift preference toward the current client more than matched Random3 ablation; ablating the current client’s own state reverses the direction.Goal tokens matter more than matched non-goal context, and reversing which client’s goal is ablated reverses the behavioral effect.Source: §7 · p. 7, Figure 4 · p. 8
8 Conclusion
Conflicting client goals can interfere across interactions when an agent uses shared persistent state. Goal ownership is strongly decodable from middle-layer K/V representations, and selectively modifying that state changes which client objective the model follows.
8 Conclusion
- Conflicting client goals can interfere across interactions in an agent with shared persistent state.Source: §8 · p. 8
- Goal ownership is strongly decodable from middle-layer K/V representations, and selectively modifying this state changes which client objective the model follows.Source: §8 · p. 8
- The findings motivate separating relationship-specific state while preserving information that should remain shared across interactions.Source: §8 · p. 8