PaperlayerView exact revision on arXiv

Source-linked AI summary

Can Agents Truly Forget? Explore Machine Unlearning for LLM Agents

Junkai Lin, Ge Xu, Zheng Xu, Junkai Chen, Xiangyuan Lan

arXiv:2610.17659v1cs.AI

TL;DR

LLM agents can expose private knowledge through parameters, persistent memory, or tool calls, while parameter-level unlearning does not cover all those routes. The paper formulates agent unlearning across these channels and introduces AUBench to evaluate it in a controlled environment. Its results show that parameter-level forgetting does not reliably predict complete-agent behavior.

  • Problem

    Agents may expose private knowledge through parameters, memory, or tool access, and existing evaluations lack environmental control to distinguish these routes while assessing retained capabilities.

    Source: §1 · pp. 1–2

  • Method

    The paper formulates agent unlearning across parametric knowledge, persistent memory, and tool-call acquisition, and introduces AUBench, a controlled benchmark with provenance-tracked agent interactions.

    Source: §1 · p. 2, §3.1 · pp. 3–4

  • Results

    Across the evaluated methods, parametric forgetting does not reliably predict complete-agent behavior; memory raises retain-set F1 for all six checkpoints and forget-set F1 in five of six.

    Source: §6 · p. 8

  • Takeaways & Limitations

    The results indicate that unlearning should be evaluated over all information channels available to the deployed agent.

    Source: §6 · p. 8

    Token-level F1 measures lexical overlap rather than information removal, so the results do not establish erasure from model parameters or external storage.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large language model (LLM) agents have shown strong capabilities in interactive problem solving, tool use, and long-horizon task execution. However, they may also acquire sensitive or private knowledge during training and expose it during deployment. Retraining a model from scratch to remove such knowledge is prohibitively costly, making machine unlearning a more practical approach. Machine unlearning aims to remove targeted undesirable knowledge from trained models without retraining from scratch while preserving general utility. When language models are deployed as agents, parameter-level unlearning becomes insufficient: forgotten information may still persist in memory artifacts or be reacquired via tool calls. In this paper, we formulate the problem of agent unlearning as removing private information from an LLM-based agent across three information channels: parametric knowledge, persistent memory, and tool-call acquisition. To study this problem, we introduce we introduce Agent Unlearning Benchmark (AUBench), a controlled benchmark that extends the fictitious-author setting of TOFU into an agent environment with persistent memory, an offline web snapshot, tool-accessible documents, and provenance-tracked interaction trajectories. By evaluating existing parameter-level and memory-level unlearning methods in this setting, AUBench reveals the gap between model-level forgetting and system-level agent unlearning, and provides a foundation for developing unlearning methods that operate across the full lifecycle of LLM agents.

1 Introduction

LLM agents can retain or reacquire private information through model parameters, persistent memory, and tools, so parameter-level unlearning alone does not address the full system. AUBench provides a controlled environment for evaluating forgetting across these channels while tracking retained utility and authorized capabilities.

1 Introduction

  • Agents may expose private knowledge acquired during training, and persistent memory or external tools create additional routes for that information to persist or resurface.Removing information from one component may leave the agent able to retrieve or reacquire it through another.Source: §1 · pp. 1–2
  • Existing evaluations can confuse forgetting with indiscriminate refusal or tool disabling, which may damage legitimate retained-knowledge and authorized-task capabilities.Open-web variation and incomplete provenance also make it difficult to attribute failures to parameters, memory, or tool-mediated reacquisition.Source: §1 · p. 2
  • AUBench provides synthetic subjects, an offline environment, persistent memory, searchable documents, interaction trajectories, and fact-level provenance for controlled agent-unlearning evaluation.It also supplies a matched retain-only oracle and adaptive evaluations beyond exact queries and fixed tool interfaces.Source: §1 · p. 2
  • The paper formulates agent unlearning as removing sensitive information across parametric knowledge, persistent memory, and tool-call acquisition.This extends the focus beyond information encoded in model parameters alone.Source: §1 · pp. 1–2
  • AUBench evaluates privacy leakage, retained utility, authorized tool use, adaptive-attack robustness, and counterfactual behavioral equivalence.These evaluations are intended to identify the strengths and limitations of existing unlearning methods.Source: §1 · p. 2

2 Related Work

Prior LLM-unlearning work and benchmarks mainly address standalone models, while agent unlearning raises additional questions about information persisting or re-emerging through memory, tools, and interaction trajectories. Agent-safety research studies execution-time risks, but the cited work leaves agent machine unlearning comparatively underexplored.

2.1 Machine Unlearning for Large language models

  • LLM-unlearning methods include gradient ascent and difference, KL minimization, preference-based approaches such as NPO, representation misalignment, and adjusted-logit self-distillation.Existing benchmarks cover privacy-oriented, safety-critical, and copyright-related removal, but mainly target standalone LLMs.Source: §2.1 · p. 2

2.2 Agent Safety

  • Agent-safety methods address unsafe actions through instruction hierarchy, prompt-injection defenses, trusted-data separation, and external enforcement, while related benchmarks assess privacy, tool use, and harmful behavior.These efforts primarily address execution-time safety risks; machine unlearning for LLM agents remains limited.Source: §2.2 · p. 3

3 Agent Unlearning

Agent unlearning must address a system whose target information can persist in model parameters or memory and be reacquired through tools. The paper frames it as reducing target recovery across evaluated pathways and strategies while bounding losses in retained utility.

3 Agent Unlearning

  • Agent unlearning spans three coupled channels: parametric knowledge, persistent memory, and tool-call acquisition, so removing information from one component does not ensure system-level forgetting.An agent may recover or use target information through another component.Source: §3 · p. 3
  • Parameter-level forgetting addresses only information encoded in model weights; memory artifacts can preserve explicit facts or partial evidence and propagate them into summaries, indexes, or profiles.Deleting only the original memory record is insufficient when downstream artifacts preserve or point back to the forgotten information.Source: §3 · p. 3
  • Tool-call unlearning concerns whether an agent has unlearned seeking, reconstructing, or using target information through external resources, not merely whether a guardrail blocks a specific call.Privacy-seeking trajectories can shape future tool selection, query formulation, and action planning.Source: §3 · p. 3
  • AUBench evaluates recovery attempts across all three channels while measuring utility on retained knowledge and authorized tasks.The objective is system-level removal across parametric recall, memory retrieval, and tool-call acquisition.Source: §3 · p. 3

3.1 Problem Formulation

  • The formulation represents an agent as a parametric LLM, persistent memory, accessible tools, and a policy governing responses, memory access and updates, and tool use.Unlearning transforms the agent with respect to target knowledge while preserving utility on non-target knowledge and retain tasks.Source: §3.1 · pp. 3–4
  • Worst-case recoverability takes the supremum of expected recovery scores across evaluated response conditions and permitted prompt, interaction, and tool-use strategies.The recovery score applies to the complete trajectory, including memory retrievals, tool calls, memory updates, and the final response.Source: §3.1 · p. 4
  • Agent unlearning requires target knowledge to be behaviorally non-recoverable within the declared access boundary while bounding degradation of non-target capabilities.The formulation specifies thresholds for target recoverability and retained-utility loss.Source: §3.1 · p. 4

4 Dataset Construction

AUBench extends TOFU’s fictitious-author setting into an agent benchmark spanning model parameters, persistent memory, and tool-accessible documents. It uses shared source facts to construct channel-specific artifacts and evaluates both forgetting and retained capability.

4.1 Benchmark Overview

  • AUBench extends TOFU’s author-level forget/retain setting with provenance-aware memories, an offline web snapshot, deterministic tools, and complete agent trajectories.Trajectories trace source facts through memories, documents, and tool observations to final responses; evaluation covers leakage and capability preservation.Source: §4.1 · pp. 4–5

4.2 Dataset Construction

  • Shared TOFU source facts are organized into author-level profiles with 20 QA pairs and provenance references, keeping forget/retain assignments consistent across derived artifacts.The full dataset has 4,000 question–answer pairs covering 200 fictitious authors.Source: §4.2 · p. 5
  • Forget-author QA pairs define the parametric target, while retain-author facts remain usable; author-level grouping prevents an author’s artifacts from crossing partitions.This supports subject-level deletion and tests direct recall reduction alongside retained-query performance.Source: §4.2 · p. 5
  • The tool channel uses a static offline web snapshot and deterministic search to make privacy reacquisition trajectories reproducible.The snapshot contains 800 local HTML pages and a read-only SQLite FTS5/BM25 index.Source: §4.2 · p. 5

4.3 Metrics

  • Token-level F1 is computed from multiset overlap after lowercasing and replacing non-alphanumeric characters with spaces.Forget- and retain-set scores are reported separately: lower forget scores indicate reduced target-answer recovery, while higher retain scores indicate preserved QA performance.Source: §4.3 · p. 5

5 Experiments

The experiments compare parameter-level unlearning methods across parametric, memory, tool, and complete-agent conditions on two model families. Results show that adding memory or tools can change both forgetting and retained capability, and complete-agent outcomes do not reliably follow checkpoint-level forgetting.

5.1 Experimental Setup and Baselines

  • The setup evaluates Llama-3.1-8B and Qwen3-8B with GD, KL, and NPO under parametric, shared-memory, offline-tool, and combined Final conditions.The forget05 and retain95 splits contain 200 and 3,800 questions; retrieved Mem0 records are reused across checkpoints, and tool access uses a local corpus.Source: §5.1 · pp. 5–6

5.2 Main Results

  • Across both model families, the largest parametric forget-set reduction accompanies the largest retain-set loss.For Llama-3.1-8B, NPO reduces forget-set F1 from 49.8% to 27.0% while retain-set F1 falls from 50.2% to 34.4%; for Qwen3-8B, KL lowers the respective scores from 43.6% to 16.8% and 43.4% to 19.1%.Source: §5.2 · p. 6
  • Adding Mem0 raises average retain-set F1 across the six checkpoints from 34.9% to 56.8%, but also raises average forget-set F1 for both models.The forget-set average rises from 39.5% to 40.9% for Llama-3.1-8B and from 19.9% to 26.5% for Qwen3-8B; KL’s Qwen3-8B forget-set F1 rises from 16.8% to 30.1%.Source: §5.2 · pp. 6–7
  • 14.5% forget-set and 45.3% retain-set F1 for Qwen3-8B GD, versus 18.3% and 52.8% for NPO, show model-dependent outcomes in the complete agent.For Qwen3-8B, KL raises forget-set F1 from 16.8% to 36.9% and retain-set F1 from 19.1% to 63.0%, reversing its parametric forgetting ranking; Llama-3.1-8B complete-agent scores remain below parametric and memory-only scores. Table 1 reports forget- and retain-set token-level F1 across Para., Mem., Tool, and Final conditions.Source: §5.2 · pp. 6, 8, Table 1 · p. 7
  • Tool access lowers average forget-set F1 from 29.7% to 12.9% and retain-set F1 from 34.9% to 17.3% across six model–method combinations.On Qwen3-8B, GD reaches 4.1% forget-set and 5.3% retain-set F1, while KL remains at 14.9% and 18.7%; the paired utility loss means lower forget scores alone do not establish better unlearning.Source: §5.2 · p. 7
  • Qwen3-8B produces empty outputs for 23.6%–58.0% of examples under Tool and up to 30.0% under Final, while Llama-3.1-8B empty-output rates stay at or below 1.0%.Qwen3-8B tool invocation under Final falls to 0.5% on forget and 0.1% on retain with KL, so F1 should be read alongside invocation and empty-output rates.Source: Table 2 · p. 7, §5.2 · p. 8

6 Conclusion

The conclusion frames agent unlearning as a system-level problem because private information can persist across model parameters, memory, and tools. Experiments show that checkpoint-level forgetting does not reliably predict complete-agent behavior, motivating evaluation across all available information channels alongside utility.

6 Conclusion

  • LLM agents can access information through parameters, persistent memory, and external tools, so parameter updates alone do not control every source available to the system.Target information may remain accessible through memory retrieval or tool calls; the paper therefore evaluates forgetting on the complete system.Source: §6 · p. 8
  • Across three methods, parametric forgetting does not reliably predict complete-agent behavior.Memory raises retain-set F1 for all six checkpoints and forget-set F1 in five of six; tool access lowers F1 on both splits, and combined-agent outcomes vary by model and method.Source: §6 · p. 8
  • The results support evaluating unlearning across all information channels while assessing forgetting jointly with end-to-end utility.Experiments use two 8B model families, three unlearning methods, shared memory artifacts, and offline tools.Source: §6 · p. 8

Limitations

The evaluation covers two 8B model families, three parameter-level methods, and TOFU’s 5% forget setting. Its controlled QA setting, fixed external sources, and lexical-overlap metric constrain what the findings establish.

Limitations

  • The evaluation covers two 8B model families, three parameter-level unlearning methods, and TOFU’s 5% forget setting.The findings may not generalize to other model scales, datasets, forget-set sizes, or tasks requiring multi-step reasoning and interaction.Source: Limitations · pp. 8–9
  • Fixed memory artifacts and an offline web corpus do not capture continuously updated memories, heterogeneous sources, or live web search.Low retain-set scores under tool access also limit conclusions about tool-mediated answer recovery, and aggregate scores cannot separate invocation, retrieval, and generation failures.Source: Limitations · p. 9
  • Token-level F1 measures lexical overlap rather than information removal, can miss semantically equivalent answers, and establishes neither adversarial-extraction resistance nor a formal privacy guarantee.The results characterize answer recovery under the evaluated conditions; they do not establish erasure from model parameters or external storage.Source: Limitations · p. 9

A.1 Unlearning Methods

The appendix describes three parameter-level unlearning objectives: Gradient Difference, KL Minimization, and Negative Preference Optimization. Each modifies forget-answer likelihood while using a different strategy to preserve or reference the original model’s behavior.

A.1 Unlearning Methods

  • The answer-token cross-entropy objective is initialized from the TOFU-finetuned checkpoint, and question and padding tokens are excluded from the loss.The forget and retain sets are denoted Df and Dr, respectively.Source: §A.1 · p. 11
  • Gradient Difference (GD) combines gradient ascent on forget examples with gradient descent on retain examples.Its first term reduces target-answer likelihood and its second encourages correct retained predictions; training uses AdamW at 10−5 for five epochs, with LoRA disabled by default.Source: §A.1 · p. 11
  • KL Minimization (KL) replaces the retain cross-entropy term with a regularizer preserving the original model’s next-token distributions.Each forget example is paired with a randomly sampled retain example, both loss terms receive unit weights, and training runs for five epochs with LoRA disabled by default.Source: §A.1 · p. 11
  • Negative Preference Optimization (NPO) treats forget-set answers as negative responses and suppresses their likelihood relative to a frozen reference model.The sequence-level likelihood-ratio weighting uses β = 0.1; training uses AdamW at 10−5 for 10 epochs.Source: §A.1 · p. 11

A.2 Forgetting–Utility Trade-offs of Parametric Unlearning

Parametric unlearning reduces forget-set scores, but the size of the reduction and the accompanying retain-set loss vary across methods and backbones. The reported trade-offs are relative to each backbone’s own baseline.

A.2 Forgetting–Utility Trade-offs of Parametric Unlearning

  • On Llama-3.1-8B, KL produces a 2.0% forget-set reduction, while its retain-set change is +0.3% with a 95% confidence interval of [0.0%, 0.6%].The reported retain-set change is statistically indistinguishable from zero, indicating limited forgetting.Source: §A.2 · p. 11, Table 3 · p. 12
  • On Llama-3.1-8B, GD reduces forget-set by 6.3% and retain-set by 1.5%, while NPO reduces them by 22.9% and 15.8%; all Qwen3-8B methods reduce both sets.On Qwen3-8B, forget-set reductions range from 21.7%–26.8% and retain-set reductions from 11.3%–24.3%. Changes are relative to each corresponding backbone’s baseline.Source: §A.2 · pp. 11–12, Table 3 · p. 12
  • Table 4 partitions forget-set token-level F1 by question type; types with small sample sizes have wide confidence intervals, and the table is descriptive rather than a significance test.The table covers 200 forget-set questions across ten target authors.Source: Table 4 · p. 12

A.3 Fine-Grained Analysis of Parametric Unlearning

Parametric unlearning varies by backbone, method, and question type, with some fact categories remaining more resistant than others. A case study also shows how memory and tool access can change factual coverage and attribution across agent conditions.

A.3 Fine-Grained Analysis of Parametric Unlearning

  • Before unlearning, name and genre questions scored highest and career/influence and style/theme lowest across both backbones; GD and KL changed Llama-3.1-8B F1 by under 9% per type.KL slightly increased name and identity scores for Llama-3.1-8B.Source: §A.3 · p. 12
  • Per-type F1 changes ranged from −33% to −18% for Llama-3.1-8B NPO, −32% to −18% for Qwen3-8B GD, −31% to −24% for Qwen3-8B KL, and −49% to −14% for Qwen3-8B NPO.Source: §A.3 · p. 12
  • Qwen3-8B NPO lowered name-question F1 from 51.0% to 1.7% and genre-question F1 from 52.6% to 3.8%, while parents/family and birth/origin retained 31.0% and 28.5%.Source: §A.3 · p. 12
  • Across six configurations, books/works retained 66% of their base score, the highest proportion, while genre retained 54%, the lowest.Source: §A.3 · p. 12

A.4 Case Study of Responses across Agent Conditions

  • In the Qwen3-8B NPO case study, memory responses better matched the reference, the tool response discussed Simon Makoni, and the final response added unsupported Bangkok.The figure compares Para, Mem., Tool, and Final against the same reference answer.Source: §A.4 · pp. 12–13, Figure 2 · p. 13

Related papers

Loading 2610.17659v1…