PaperlayerView exact revision on arXiv

Source-linked AI summary

Confess What You Know: Forget-Set Misalignment with Model Knowledge in LLM Unlearning

Miso Kim, Georu Lee, Seungwon Jeong, Woojin Lee

arXiv:2609.00605v1cs.LGcs.AIcs.CL

TL;DR

LLM unlearning often assumes that a pre-defined forget set matches the model’s memorized knowledge, but users cannot observe that hidden boundary when requesting deletion. The paper analyzes the resulting Under and Out-of-Knowledge failure modes and introduces data-blind CONFS, which elicits and structures model knowledge into aligned forget sets. Across benchmarks, CONFS approaches Gold-standard performance on several metrics and offers a competitive forgetting-utility balance while better preserving utility than alternative data-blind constructions.

  • Problem

    Users cannot observe which personal information an LLM memorized, so requested forget sets may differ from the model’s hidden memorized knowledge.

  • Method

    CONFS elicits the model’s memorized knowledge, formalizes it into Subject-Relation-Object triplets, and uses consistency checking to construct a model-aligned forget set without pre-training data.

  • Results

    Across synthetic, multimodal, and real-world benchmarks, CONFS approaches Gold-standard performance on several metrics and achieves a competitive forgetting-utility balance while better preserving utility than alternative data-blind constructions.

  • Takeaways & Limitations

    Reliable unlearning depends on whether the forget set matches what the model actually knows, in addition to the optimization objective.

  • Takeaways & Limitations

    CONFS focuses on factual entity knowledge structured as Subject-Relation-Object units; broader knowledge forms such as narratives and procedural knowledge may require additional formulation.

Abstract

from arXiv · show

Machine unlearning for large language models (LLMs) often assumes that a pre-defined forget set matches what the model has memorized, but this frequently breaks in realistic privacy settings where the original training data is inaccessible. We term this gap forget-set misalignment and identify two cases. In Under Unlearning, the forget set omits memorized information and leakage persists. In Out-of-Knowledge Unlearning, the algorithm is driven to "forget" knowledge the model never learned, perturbing parameters and degrading utility. Using gradient-level analysis, we show these behaviors arise from misaligned unlearning targets rather than specific optimization choices. We then propose CONfession-to-Forget-Set (CONFS), a data-blind framework that constructs model-aligned forget sets by eliciting and formalizing the model's memorized knowledge. Across synthetic, multimodal, and real-world benchmarks, CONFS approaches Gold-standard performance on several metrics and achieves a competitive forgetting-utility balance, while preserving utility better than other data-blind forget-set constructions.

1 Introduction

The paper identifies forget-set misalignment as a realistic privacy problem: users cannot see which personal details an LLM memorized, so deletion requests may target the wrong information. It proposes CONFS, which elicits and formalizes the model’s memorized knowledge to construct an aligned forget set.

  • Motivation: Users can specify personal details for deletion but cannot observe which details the model memorized, creating forget-set misalignment.The mismatch arises because the requested forget set may differ from the model’s hidden memorized knowledge.
  • Failure modes: Under Unlearning omits memorized sensitive information, leaving associated entity-level knowledge and privacy risks insufficiently addressed.The requested facts may be forgotten while omitted leaked information remains unaffected.
  • Failure modes: Out-of-Knowledge Unlearning targets information the model never encountered, unnecessarily perturbing parameters and degrading general utility.
  • CONFS: CONFS prompts the model to confess memorized knowledge, formalizes it as Subject-Relation-Object triplets, and recursively probes for hidden details.The recursive reconfession process is intended to reduce gaps in the resulting forget set.
  • Results: In a fully data-blind setting, CONFS approaches Gold-standard performance on several metrics and achieves a competitive forgetting-utility balance while preserving utility better than alternative constructions.The results support the importance of matching the forget set to what the model actually knows, not only selecting an optimization objective.

2 Related Work

Prior work frames LLM unlearning as post-hoc fine-tuning with a pre-defined forget set, while benchmarks often construct aligned synthetic targets or use less precise data-blind surrogates. The paper situates forget-set misalignment as a broader source of instability and degradation, including non-adversarial settings.

  • LLM unlearning: LLM unlearning commonly applies post-hoc fine-tuning through gradient ascent, preference-based losses, or reinforcement learning while auxiliary retain objectives preserve utility.
  • Benchmarks: Synthetic benchmarks inject fictitious persona information and define forget sets from that content, enabling reproducible evaluation but departing from realistic privacy leakage.
  • Benchmarks: RWKU operates without the original forget or retain corpora, but its surrogate forget set uses unstructured generated text that makes memorized personal information difficult to isolate.
  • Adverse effects: Improper forget sets can induce abnormal or large gradients, causing indiscriminate parameter updates and broad degradation, including when information was already partly forgotten.
  • Adverse effects: Real-world forget-set misalignment can destabilize unlearning and degrade overall performance even without malicious intent.

3 Forget-Set Misalignment

Forget-set misalignment occurs when a requested forget set differs from the knowledge an LLM memorized, producing either incomplete forgetting or unnecessary utility damage. The section studies these effects with controlled partitions and gradient-based diagnostics.

  • 3 Forget-Set Misalignment: Existing evaluations often assume that the predefined forget set matches the model’s memorized knowledge, although users cannot observe inaccessible pre-training data.This gap is called forget-set misalignment.
  • 3.1 Constructing Misaligned Forget Sets: The controlled setup partitions QA pairs into requested leaked data DL+, omitted leaked data DL−, and requested never-seen data DN+.DL− represents Under Unlearning, whereas DN+ represents Out-of-Knowledge Unlearning.
  • 3.2 Unlearning Settings: The analysis uses loss increments and gradient inner products to estimate how an unlearning update affects target, omitted-leaked, and retain sets.A positive gradient inner product indicates that the update raises the evaluated set’s loss; near-zero values indicate little effect.
  • 3.3 Analysis of the Under Unlearning Setting: In Under Unlearning, targeted forgetting on DL+ rises sharply while Entity Generalization Effect remains near zero, so omitted leaked data degrades only with global utility decay.This indicates that forgetting does not generalize from the requested facts to omitted leaked facts.
  • 3.4 Analysis of the Out-of-Knowledge Unlearning Setting: In Out-of-Knowledge Unlearning, gradients on never-seen DN+ contribute collateral damage without removing memorized data, severely degrading retained utility.The retain-set degradation from DN+ dominates the contribution from memorized DL+ across unlearning steps.

4 Method

CONFS builds a model-aligned forget set in a data-blind setting by eliciting, structuring, expanding, verifying, and converting memorized knowledge into competency questions.

  • CONFS framework: CONFS identifies a model’s memorized knowledge about target entity E and constructs a model-aligned forget set without pre-training data.It operates when the information to remove is not specified in advance.
  • Memorized knowledge: CONFS defines memorization behaviorally: the target model must reproduce a fact under elicitation consistently across stochastic samples.This criterion concerns disclosure behavior rather than training-data provenance.
  • Reconfession: Reconfession selectively probes sub-entity attributes to recover details missed by an initial query and expand the forget set’s coverage.A concrete value is retained as exposed knowledge, while UNKNOWN terminates expansion; leaves become base or attribute-expanded forms.
  • Confession: The confession stage prompts the model with only entity name E to obtain raw natural-language claims reflecting its memorized knowledge.These claims may be latent, distributed, unstructured, and incomplete.
  • Triplet Extraction and Subtriplet Decomposition: CONFS converts claims into SRO triplets with entity E as subject, noun-based relation r, and explicit entity-specific value o.Triplets are decomposed when composite objects encode multiple factual attributes, producing atomic preliminary leaf candidates.
  • Competency Question Generation: Finalized leaves become competency questions, each targeting one leaf-level knowledge unit and using only explicitly represented information.Base leaves generate one question, while attribute-expanded leaves generate two questions targeting the object and attribute value.
  • Hallucination Verification: Hallucination verification retains QA pairs only when sampled answers have average contradiction probability below τ=0.7.The check uses multiple stochastic answers and DeBERTav3-large NLI scoring.

5 Experiments

The experiments evaluate data-blind forget-set constructions across synthetic, multimodal, and real-entity benchmarks with several unlearning objectives. CONFS generally improves the forgetting-utility balance and reduces misalignment-related collateral damage.

  • Benchmarks: CONFS is evaluated on TOFU, CLEAR, and RWKU, covering synthetic, multimodal, and real-world entity unlearning settings.TOFU and CLEAR use 10% target protocols; RWKU averages results over the first 10 real-entity targets.
  • Forget-set constructions: The comparison includes Gold-standard and data-blind constructions FreeRecall-QA, RWKU-style, and CONFS without reconfession.Gold-standard uses original benchmark-provided target data corresponding to injected knowledge.
  • Misalignment diagnostics: CONFS shows the lowest out-of-knowledge interference while maintaining targeted forgetting in the TOFU misalignment diagnostics.Out-of-Knowledge collateral damage increases with DN+ share, while Under-setting forgetting remains confined to requested facts.
  • Unlearning objectives: Four representative objectives—GA, GD, NPO, and RT—are used with the different forget sets under official benchmark protocols.The objectives are Gradient Ascent, Gradient Difference, Negative Preference Optimization, and Rejection Tuning.
  • TOFU: On TOFU, CONFS achieves the best forgetting-utility balance among data-blind constructions while better preserving Retain, Real Authors, and World Facts than FreeRecall-QA and RWKU-style.Reconfession raises Recall and forget-set size, while hallucination verification raises Precision and F1 and shrinks the set.
  • CLEAR: On CLEAR, CONFS maintains a favorable forgetting-utility balance in the multimodal setting.RWKU-style is omitted from the CLEAR comparison.
  • RWKU: On RWKU targets 1-10, CONFS improves forgetting across GA, NPO, and RT while maintaining comparable Neighbor-set performance.Downstream utility and membership inference metrics are maintained or improved.

6 Conclusion

The paper identifies forget-set misalignment as a cause of unlearning failures and proposes CONFS to construct model-aligned forget sets. CONFS is data-blind and approaches Gold-standard performance while preserving utility.

  • Forget-set misalignment causes unlearning failures when the requested forget set differs from the model’s memorized knowledge.
  • The paper identifies Under Unlearning and Out-of-Knowledge Unlearning as two distinct failure modes.
  • CONFS constructs model-aligned forget sets for targeted forgetting without access to pre-training data.
  • CONFS approaches the Gold-standard baseline on several metrics and achieves a competitive forgetting-utility balance.

Limitations

The work focuses on factual entity knowledge that can be represented as Subject-Relation-Object units and converted into competency questions. Broader memorized knowledge requires further formulation.

  • The method focuses on factual knowledge about entities represented as Subject-Relation-Object units and competency questions.
  • Long narratives, procedural knowledge, and relational or contextual memorization may require additional formulation beyond the current scope.
  • Extending CONFS to these broader memorization settings remains future work.

Ethical Considerations

CONFS operates on target-model outputs to surface already-memorized privacy-sensitive knowledge for removal. Its construction procedure uses sampled confessions, structured triplets, reconfession, and verification.

  • CONFS operates only on the target model’s outputs and introduces no new information.
  • The framework surfaces knowledge the model has already memorized so that it can be removed.
  • CONFS constructs a model-aligned forget set from a target entity, target model, structuring model, confession samples, verification samples, and an NLI model.
  • The algorithm samples claims, extracts and decomposes triplets, probes reconfessions, and retains consistently reproduced facts.

B Implementation Details

The experiments evaluate multiple unlearning objectives, model settings, benchmark splits, and metrics across text-only and multimodal tasks. CONFS construction uses model sampling, GPT-4o processing, NLI verification, and measured runtime stages.

  • B.1 Controlled analysis: The controlled analysis injects leaked QA pairs during LLaMA-2-7B-Chat pre-training while holding out other QA pairs and retaining knowledge for 180 non-target entities.
  • B.2 Unlearning objectives: GA reduces likelihood on forget-set responses, while GD adds retain-data regularization and NPO uses a reference-model preference objective.
  • B.2 Unlearning objectives: GA can severely degrade non-targeted knowledge, GD improves retention relative to GA, and forget loss can dominate GD optimization.
  • B.2 Unlearning objectives: RT trains the model to produce rejection-style outputs for forget-set queries.
  • B.3 Model settings: TOFU and RWKU use full-parameter unlearning on LLaMA-2-7B-Chat, while CLEAR uses LoRA-based unlearning on LLaVA-1.5-7B.
  • B.4 Seed reporting: Results for TOFU and CLEAR are means ± standard deviations over three unlearning seeds, with broadly consistent performance trends.

C RWKU Results

On RWKU, CONFS consistently outperforms the benchmark’s forget-set baseline across GA, NPO, and RT, strengthening forgetting while maintaining comparable Neighbor-set and utility performance.

  • Across GA, NPO, and RT, CONFS achieves stronger forgetting than the RWKU-style baseline while maintaining comparable Neighbor-set and utility performance.

D Forget-Set Size and Quality

The evaluation measures how closely data-blind forget-set constructions recover injected TOFU facts, showing that alignment depends on fact quality rather than set size. CONFS with hallucination verification performs best among data-blind constructions, though its F1 remains below the Gold-standard under a conservative matching protocol.

  • Setup: The study evaluates recovery of injected pre-training QA pairs for 20 TOFU target authors against a 400-QA Gold-standard forget set.
  • Evaluation protocol: Fact-level matching requires both attribute and value agreement; Recall, Precision, and F1 quantify agreement, with Gold self-comparison reaching F1 = 0.983 under judge noise.
  • Results: 4000 QA yields F1 = 0.195 for the RWKU-style construction, showing that a larger forget set does not necessarily provide better alignment.
  • Results: Hallucination verification reduces one construction from 757 to 300 QA while increasing F1, and CONFS with verification achieves the best F1 among data-blind constructions.
  • Interpreting the Gap to the Gold-standard: CONFS reaches F1 = 0.402 versus 0.983 for Gold-standard, but bundled Gold answers and atomic CONFS facts make this F1 a conservative coverage estimate.

E.1 TOFU Benchmark

In TOFU, CONFS produces attribute-specific forget-set questions resembling the Gold target, whereas RWKU-style questions are broader. Post-unlearning generations show that only full CONFS cleanly disavows the memorized fact among the reported CONFS variants.

  • E.1 TOFU Benchmark: TOFU’s benchmark forget set matches the pre-training deletion target, while CONFS expresses highly similar content in a more attribute-specific form.
  • E.1 TOFU Benchmark: TOFU examples contrast benchmark questions about Jaime Vasquez’s biography with CONFS questions targeting individual attributes such as birth date, parent profession, award, and residence.
  • E.1 TOFU Benchmark: RWKU-style examples instead ask broad questions about writing style, influences, career evolution, and inspiration.
  • E.1 TOFU Benchmark: On a TOFU question, the LLaMA-2-7B-Chat reference reproduces the injected fact, confirming that the target knowledge is memorized.
  • E.1 TOFU Benchmark: Among CONFS variants, CONFS w/o Recon. reproduces “LGBT”, CONFS w/o Halluc. fabricates a wrong attribute, and full CONFS produces an uncertainty response.
  • E.1 TOFU Benchmark: CONFS constructs model-grounded forget sets by eliciting recall, extracting factual SRO triplets, decomposing them when possible, reconfessing additional attributes, and generating exact-answer questions.
Loading 2609.00605v1…