Source-linked AI summary
The Memory Curse: How Expanded Recall Erodes Cooperative Intent in LLM Agents
Jiayuan Liu, Tianqin Li, Shiyi Du, Xin Luo, Haoxuan Zeng, Emanuel Tewolde, Tai Sing Lee, Tonghan Wang, Carl Kingsford, Vincent Conitzer
TL;DR
The paper asks whether expanding LLM agents’ memory improves cooperation or instead destabilizes it in repeated social dilemmas. Through large-scale evaluations and interventions targeting memory content and reasoning, it identifies a memory curse in which longer recall often erodes cooperation, while forward-looking reasoning can mitigate the decline.
Problem
It remains unknown how bounded interaction-history lengths shape cooperation, including when longer history supports cooperation and when it amplifies noise and collapse.
Method
The study combines large-scale multi-game evaluations with memory sanitization, reasoning ablation, and forward-looking fine-tuning interventions.
Results
Across diverse social dilemmas, extended histories frequently produce a memory curse, while sanitizing memory content, removing explicit Chain-of-Thought, or fine-tuning forward-looking reasoning can mitigate the collapse.
Takeaways & Limitations
Memory is an active determinant of multi-agent behavior: longer recall can destabilize or support cooperation depending on the reasoning patterns it elicits.
Takeaways & Limitations
The intervention relies on static fine-tuning, leaving dynamic memory curation mechanisms such as selective forgetting and strategic summarization for future work.
Abstract
from arXiv · showhide
Context window expansion is often treated as a straightforward capability upgrade for LLMs, but we find it systematically fails in multi-agent social dilemmas. Across 7 LLMs and 4 games over 500 rounds, expanding accessible history degrades cooperation in 18 of 28 model--game settings, a pattern we term the memory curse. We isolate the underlying mechanism through three analyses. First, lexical analysis of 378,000 reasoning traces associates this breakdown with eroding forward-looking intent rather than rising paranoia. We validate this using targeted fine-tuning as a cognitive probe: a LoRA adapter trained exclusively on forward-looking traces mitigates the decay and transfers zero-shot to distinct games. Second, memory sanitization holds prompt length fixed while replacing visible history with synthetic cooperative records, which restores cooperation substantially, proving the trigger is memory content, not length alone. Finally, ablating explicit Chain-of-Thought reasoning often reduces the collapse, showing that deliberation paradoxically amplifies the memory curse. Together, these results recast memory as an active determinant of multi-agent behavior: longer recall can either destabilize or support cooperation depending on the reasoning patterns it elicits.
1 Introduction
This paper examines how expanding LLM agents’ accessible interaction history affects cooperation in long-horizon social dilemmas. Across distinct behavioral regimes, longer memory can sustain cooperation or produce a memory curse involving defensive, history-following behavior.
- Motivation: The study tests whether expanding interaction history improves or harms cooperation in multi-agent LLM social dilemmas.It targets a gap left by evaluations that typically use no more than 10 rounds.
- Study scope: 500-round repeated interactions expose effects of long histories that shorter evaluations can obscure.
- Study scope: 7 LLMs × 4 games × 9 history-length settings × 3 seeds generate over 378,000 reasoning traces.The evaluation spans repeated social dilemmas with 500 rounds per interaction.
- Main findings: Expanded memory often harms cooperation by making agents less forgiving and more defensive, while some settings retain stable cooperation.
- Main findings: Memory effects form distinct regimes shaped by model capability, game structure, and reasoning style rather than context length alone.
- Mechanism and intervention: Forward-looking LoRA fine-tuning mitigates cooperation decay at extended history lengths and transfers zero-shot to untrained games.The intervention does not compromise general task ability on GSM8K, TriviaQA, HumanEval, and MBPP.
2 Related Work
Prior work shows that LLM cooperation in repeated interactions is fragile and path-dependent, while the role of bounded interaction-history length remains unresolved. This paper positions its study around when longer histories support cooperation versus amplify noise and collapse.
- LLMs in social dilemmas: Prior studies find that LLM cooperation in repeated games is fragile, path-dependent, and often difficult to sustain after defection.
- Reasoning and strategic behavior: Strategic competence varies across game environments, and explicit reasoning methods do not reliably improve performance.
- Memory effects: Memory research has treated memory as an orthogonal route to improving AI robustness and generalization beyond parameter optimization.
- Open question: Different bounded interaction-history lengths remain understudied, including when longer history supports cooperation or drives cooperation collapse.
3 Experiment Design and Settings
The experiments evaluate repeated social dilemmas with shared histories across multiple LLM backbones, games, history lengths, and repeated trials. Agents play long-horizon interactions while pursuing cumulative rewards against the same opponents.
- Models and games: The study evaluates 7 LLMs across Prisoner’s Dilemma, Traveler’s Dilemma, Public Goods Game, and Trust Game.
- Interaction design: Two or three same-backbone agents repeatedly play one game while sharing all players’ past decisions.
- Experimental settings: Agents are instructed to pursue long-term cumulative rewards in interactions that continue with probability 0.99 after each round.
- Experimental settings: History length varies over {0, 1, 2, 3, 5, 10, 20, 40, 80} rounds, with 3 repetitions per setting and up to 500 rounds per interaction.
4 Mechanisms of the Memory Curse: How Expanded Recall Erodes Trust
Expanded history produces distinct cooperation regimes: some model–game settings remain cooperative, while others follow a memory-bounded pattern in which cooperation peaks at short histories and then declines. The decline is associated with reduced forward-looking reasoning, can spread through mixed-memory groups, and is mitigated by targeted forward-looking fine-tuning.
- Cooperation regimes: At short histories, some models sustain cooperation, while memory-bounded agents typically peak at HL ≤5 before cooperation decreases as history expands.The memory-bounded pattern suggests that too little history promotes betrayal forgiveness, whereas too much history slows trust recovery.
- Cooperation regimes: 51.2% to 9.5%: Gemma-3-12B’s Trust Game cooperation falls from HL=2 to HL=80, while cumulative reward drops from 8.59 to 5.19.This illustrates that expanding memory can materially reduce long-run welfare.
- Cooperation regimes: 92.1% to 20.6%: GPT-OSS-20B’s Prisoner’s Dilemma cooperation declines from HL=2 to HL=80, while Llama-4-Scout-17B’s Public Goods cooperation falls from 82.6% to 45.8%.Variance also increases at long histories, reaching ±24.0% for Llama-4-Scout-17B at HL=80.
- Cognitive mechanism: At HL=80, Memory Immune settings retain a mean forward-looking ratio of 0.504 versus 0.340 for Memory Cursed settings.Across history expansion, reduced cooperative language explains the collapse better than a large increase in absolute defensive language, although the relative Paranoia Ratio rises.
- Cognitive mechanism: +14.7 to +79.3 percentage points: forward-looking fine-tuning raises HL=80 cooperation across all four games and transfers zero-shot beyond the Public Goods Game training traces.The fine-tuned model reaches near 100% cooperation in Public Goods, Trust, and Prisoner’s Dilemma while improving substantially in Traveler’s Dilemma, without significant general-task degradation.
- Asymmetric memory: In asymmetric interactions, HL=80 agents suppress cooperation relative to HL=2 partners, degrade group welfare, and remain less cooperative even when HL=2 agents are outnumbered.The reported gaps include +28 percentage points in individual Trust Game cooperation and +33 percentage points for a lone forgiver in Public Goods.
5 Conclusion
The paper finds that expanded interaction histories can destabilize cooperation in LLM social dilemmas, while forward-looking reasoning and suitable memory content can preserve it. Its evaluation spans long-horizon repeated games, and future work targets dynamic memory curation and broader societies.
- Extended histories frequently trap agents in retaliatory cycles, while forward-looking reasoning and memory sanitization help preserve cooperation.The authors attribute the collapse primarily to accumulated negative content and report that explicit Chain-of-Thought deliberation can amplify it.
- Future research should examine dynamic memory curation, open-ended N-player societies, heterogeneous architectures, and agents’ capacity to forgive.The authors specifically propose selective forgetting, strategic summarization, and specialized retrieval-augmented generation.
- The repeated-game environments separate individual short-term incentives from collective welfare and use a 99% continuation probability to emphasize long-run payoffs.Agents observe the most recent HL rounds and are instructed to maximize total payoff over repeated interaction.
B Analysis of Cooperation Dynamics
Cooperation is sustained when models use memory to support trust repair, reciprocity, and long-run reasoning, but excessive history can make earlier defections dominate. The resulting trajectories range from stable cooperation to rigid defensive lock-in.
- Consistent cooperation: Consistently cooperative models use cooperative actions as signals to preserve or repair trust rather than relying on passive cooperation.Their reasoning explicitly considers retaliation, reciprocal incentives, and opponent strategy.
- Consistent cooperation: At HL = 2, models reason from recent events and future consequences; at HL = 80, they justify similar actions through broader historical patterns.The cooperative policy can remain unchanged while the evidential basis shifts from local trust repair to aggregate pattern recognition.
- Forward-looking reasoning: Forward-looking reasoning supports cooperation by weighing immediate temptation against the long-term benefits of continued interaction.This mechanism explains why cooperation can persist through either short recent histories or richer empirical histories.
- Memory-bounded behavior: Memory-bounded models often peak at HL ≤5, most often HL = 2, before cooperation declines as the history window expands.Too little history limits reciprocity, whereas too much can make earlier defections overly salient and impede trust recovery.
- Memory-bounded behavior: Gemma-3-12B progresses from myopic defection to local repair and finally to long-history lock-in.At HL = 80, accumulated history is compressed into a dominant conclusion that the opponent is persistently non-cooperative.
C.2 Intra-model trajectory 2: GPT-OSS-20B (Public Goods)
GPT-OSS-20B shifts from forward-looking reciprocity to defensive thresholds and shallow pattern matching as memory expands. Across the broader evaluation, memory effects are common but depend on the model–game combination.
- GPT-OSS-20B trajectory: GPT-OSS-20B’s reasoning shifts from forward-looking coordination to defensive threshold policies and eventually shallow pattern-matching as memory grows.The trajectory illustrates memory-bounded degradation rather than a uniform improvement from additional context.
- GPT-OSS-20B trajectory: At HL = 2, the model reasons that encouraging A0 can increase long-term average payoff for everyone.This reflects forward-looking reciprocity under a short memory window.
- GPT-OSS-20B trajectory: At HL = 20, the model requires both opponents to play A0 before switching, creating a conservative entry threshold for cooperation.The policy emphasizes predictability and avoiding exploitation, making trust repair difficult.
- GPT-OSS-20B trajectory: At HL = 80, reasoning is reduced to direct recitation of the recent historical pattern rather than strategic analysis.The authors describe this as compressed pattern-matching at the longest memory setting.
- Cross-setting boundary: 18 of 28 model–game settings are memory cursed, while 10 maintain at least 95% cooperation across all history lengths.A single model can be immune in one game and severely cursed in another, indicating game-specific strategic comprehension.
- Cognitive analysis: Lexical analysis of over 378,000 reasoning traces tracks proactive versus reactive intent using forward-looking and history-following dictionaries.The Forward-Looking Ratio divides forward-looking keyword frequency by all matched forward-looking and history-following keywords.
D.4 Cognitive Contrast: Immune vs. Cursed Regimes
Memory-immune and memory-cursed settings differ in the extent to which reasoning remains oriented toward future outcomes. At long history, immune settings retain a substantially higher forward-looking ratio than cursed settings.
- Memory-immune settings sustain a significantly higher forward-looking reasoning ratio than memory-cursed settings across history lengths.The contrast remains robust even under an 80-round context window.
- 0.504 is the mean forward-looking ratio for memory-immune model–game settings at HL=80.This ratio indicates greater weighting toward future outcomes within the paper’s lexical metric.
- 0.340 is the forward-looking ratio for memory-cursed model–game settings at HL=80.The lower ratio reflects reasoning more dominated by historical analysis and defensive calculation.
- The immune regime maintains future-oriented reasoning rather than merely ignoring history, whereas the cursed regime over-indexes on historical noise.The paper connects this contrast to collective welfare, the shadow of the future, and persistent punishment cycles.
- The analysis processes over 378,000 CoT reasoning texts and normalizes raw word counts per 1,000 CoT words.The authors construct semantic clusters for defensive words and positive cooperative words to profile strategic priorities.
E.2 Aggregated Results
Across all evaluated games, expanding history shifts reasoning from cooperation toward defensive calculation. This increase in the Paranoia Ratio appears consistently across structurally diverse social dilemmas.
- Longer history produces a monotonic shift from cooperative-leaning to defensive-leaning reasoning across evaluated games.The aggregated analysis compares HL=2 with HL=80 using normalized semantic frequencies and the Paranoia Ratio.
- The Paranoia Ratio increases consistently across structurally diverse games, including the Trust Game and Public Goods Game.The per-game breakdown is presented as evidence that the semantic shift is not specific to one dilemma.
F Detailed Results for Asymmetric Memory Evaluation
Asymmetric-memory evaluations show that long-memory agents cooperate less than short-memory partners across models, while the effect varies by model and game. The resulting gaps quantify a “grudge-holder” disadvantage and reveal a robust cross-game defensive shift.
- Across all models, long-memory grudge-holders cooperate less than short-memory forgivers, with gaps ranging from +10.3pp to +28.1pp.The comparison uses P1 with HL=2 and P2 with HL=80 in the Trust Game.
- The per-game Paranoia Ratio shows a monotonic transition from cooperative-leaning to defensive-leaning reasoning across diverse social dilemmas.This pattern is described as a robust structural pathology rather than a single-game artifact.
- In the adversarial Public Goods configuration, GPT-OSS-20B shows 37.7% cooperation for the lone forgiver versus 5.0% for grudge-holders, a +32.7pp gap.The setting contains two HL=80 grudge-holders and one HL=2 forgiver.
- Llama-3.3-70B cooperates universally across all configurations in the specific Public Goods game examined.This is reported as a model-specific ceiling effect in that game.
G Detailed Analysis of the Reasoning Ablation and Ceiling Effects
Reasoning ablations show that explicit Chain-of-Thought can amplify memory-driven cooperation collapse, but aggregate averages can hide this effect because capable models often reach cooperation ceilings in simpler games.
- Chain-of-Thought exacerbates the memory curse when compared with the no-reasoning ablation.The comparison uses the main explicit-reasoning results and corresponding no-reasoning results.
- A naive average across four dilemmas can dilute the effect because highly capable models often maintain 100% cooperation in simpler games.The cited examples are Llama-3.3-70B and Qwen2.5-Coder-32B in Prisoner’s Dilemma or Public Goods.
- At cognitive boundaries, deliberation causes a −93.1 pp cooperation drop for Llama-3.3-70B in the Trust Game.The maximum deliberation penalty isolates each model’s most severely affected game at HL=80.
- The no-reasoning ablation instructs agents to output only their final action while retaining the same game rules and prompt structure.This design removes the reasoning requirement rather than changing the game setup.
H Memory Sanitization: Full Experimental Details
Memory sanitization holds the 80-round prompt length fixed while replacing older history with synthetic records. Cooperation recovers when harmful history content is replaced by cooperative content, supporting a content-driven account of the memory curse.
- The sanitization experiment fixes HL=80 and replaces 80−X visible rounds with synthetic mutual-cooperation records, leaving X recent rounds real.This design isolates history content from prompt length.
- The cooperative synthetic records use mutual A0 actions with payoff (10, 10), while X=80 is the unsanitized baseline and X=0 is entirely synthetic.X is the only free parameter in the Trust Game sanitization setup.
- Smaller real-history windows produce greater cooperation recovery for most models, with strong models recovering to 87%–97% at X=2.The sweep evaluates X values from 2 through 80 and reports mean±std over three runs of 500 rounds.
- POLAR sanitization injects explicit defection signals, whereas IDEAL uses purely cooperative synthetic history; some models remain near 100% in both modes.The contrast supports a content-driven interpretation of the effect.
- Holding prompt size constant, sanitization can restore cooperation to near-ceiling levels, including 95.87% for Llama-3.3-70B.The result is presented as evidence against a purely length-based explanation.
I.1 Training Data Construction and Trace Extraction
The study constructs a filtered forward-looking reasoning-trace dataset and applies weighted LoRA fine-tuning to Mistral-7B-Instruct-v0.1 for zero-shot evaluation across four games.
- Trace Extraction: Approximately 28% of raw traces were discarded before evaluating a balanced subset across six source models and seven history-length conditions.The retained traces contained forward-looking keywords such as future, long-term, signal, and mutual benefit.
- Training Data Construction: 5,124 traces passed judge thresholds, and 4,285 remained after an independent anti-cheat substring-match filter.The filters required high forward-looking and specificity scores, then checked forward-looking keyword counts against reactive terms.
- Training Data Construction: The final dataset contained 1,751 training traces and 92 validation traces after a family-level split prevented near-duplicate leakage.Without family-level splitting, 54% of validation responses shared a prefix with a training response.
- Fine-Tuning Procedure: LoRA fine-tuning targeted all attention and MLP projections with rank r = 16 while freezing the base model weights.The adapter used α = 32 and dropout 0.05.
- Fine-Tuning Procedure: Training optimized a per-example weighted cross-entropy loss, with weights defined as wi = sfwd · sspec/100 and Ti-normalization limiting dominance by long traces.The adapter was trained for three epochs and merged into the base model for zero-shot evaluation across all four games.