PaperlayerView exact revision on arXiv

Source-linked AI summary

Can Scientific Claims Be Removed from Large Language Models? A Systematic Evaluation of Claim-Level Unlearning

Snigdha Paul, Manasi Patwardhan, Arman Cohan

arXiv:2608.20960v1cs.AI

TL;DR

Scientific claims in LLMs can become outdated, invalidated, or harmful, but existing unlearning research largely addresses instance-level forgetting rather than interconnected claims. This paper defines scientific claim unlearning and introduces SciUnlearn to evaluate it across paraphrased claims and retain sets. Experiments find that current methods mainly suppress unlearned instances, with limited transfer to paraphrased or complementary sets, motivating methods that target structured conceptual knowledge.

  • Problem

    LLMs trained on static scientific corpora may retain obsolete or harmful claims, while existing unlearning research lacks claim-level methods and systematic benchmarks.

  • Method

    The paper defines scientific claim unlearning and introduces SciUnlearn, using disjoint paraphrased forget sets and retain sets to evaluate claim-level forgetting and knowledge retention.

  • Results

    Existing unlearning methods mainly suppress specific training instances, with limited transfer to paraphrased or complementary forget sets.

  • Takeaways & Limitations

    Scientific claim unlearning requires algorithms and evaluations that go beyond surface-form suppression and target structured conceptual knowledge.

  • Takeaways & Limitations

    The study evaluates 7B–8B models, omits mechanistic or representation-level methods, and includes only a small real-world retracted-paper subset.

Abstract

from arXiv · show

Language models (LMs) are trained on static scientific corpora, whereas scientific knowledge continuously evolves through correction and revision. Scientific claims encoded within these models may later become retracted, disproven, or updated by subsequent research, creating the risk of disseminating outdated information in scientific workflows. This creates a need for LMs to forget obsolete scientific claims. Machine unlearning offers a promising solution by enabling knowledge removal while maintaining overall model utility. Existing studies primarily investigate instance-level forgetting; however, scientific claims introduce additional challenges because they are interconnected, and continually evolving. To address this gap, we introduce the task of Scientific Claim Unlearning and present a new benchmark, SciUnlearn. We show that current unlearning approaches are unable to effectively eliminate claim-level knowledge and often achieve only superficial suppression, highlighting the need for specialized methods designed for structured knowledge removal.

1 Introduction

LLMs support scientific work but are trained on static corpora, so they may retain outdated, invalidated, or harmful scientific claims. The paper motivates scientific claim unlearning and introduces a benchmark and evaluation framework for this gap.

  • LLMs assist with hypothesis generation, literature reviews, and scientific discovery, despite relying on fixed-cutoff training corpora.
  • Retracted or erroneous claims can persist in LLM outputs, affecting ideation, claim verification, healthcare, and other scientific workflows.
  • Some embedded scientific knowledge later becomes harmful, sensitive, or dual-use, creating additional regulatory concerns and removal challenges.
  • Scientific claim unlearning remains largely unexplored, and existing work lacks claim-level methods and systematic benchmarks.
  • The paper defines scientific claim unlearning, introduces a benchmark, adapts baseline algorithms, and analyzes forgetting–retention trade-offs.

2 Related Works

Prior research structures and tracks scientific claims, while machine unlearning targets removal of training influence. However, existing work does not directly address scientific claim-level unlearning.

  • Scientific claim extraction and tracking systems organize claims across domains and temporal evolution but do not focus on unlearning invalid claims.
  • Machine unlearning removes targeted training influence while attempting to preserve model utility, including applications to privacy, copyright, and harmful knowledge.
  • Existing unlearning methods include gradient-based updates, preference optimization, and parameter-efficient tuning, but removed knowledge may persist and re-emerge.

3 Scientific Claim Unlearning Task Definition

The task tests whether unlearning removes an underlying scientific claim rather than only particular question–answer forms. It separates paraphrased forget sets from external and internal retain sets to measure forgetting and utility preservation.

  • The dataset contains two disjoint forget sets of paraphrased question–answer pairs derived from shared underlying scientific claims.
  • Unlearning one forget set should also affect its counterpart, testing claim-level removal beyond superficial form-specific forgetting.
  • External retain data contains cited prior knowledge, while internal retain data contains non-claim-specific context from the same papers.
  • The algorithm updates model parameters using a forget set and optionally selected retain data to balance forgetting and retention losses.
  • Performance is evaluated on a disjoint forget set to measure whether forgetting transfers beyond the unlearned question–answer forms.

4 SciUnlearn Dataset

SciUnlearn builds computer-science, medical, and retracted-paper datasets from Dolma papers, extracting and validating claims before generating diverse QA pairs and retain sets. Its design supports evaluation of claim forgetting, paraphrase transfer, and knowledge retention.

  • SciUnlearn comprises computer-science, medical, and post-2024 retracted-paper datasets built from academic papers in the Dolma corpus.
  • The computer-science subset samples 100 open-access papers from 2018–2022 and extracts 1–5 claims per paper from selected sections.
  • Three language models judge claim accuracy, independence, and clarity on a 0–5 scale, with Table 1 reporting majority consensus rates for scores of at least 4.
  • Each claim generates eight QA pairs spanning multiple-choice, true/false, fill-in-the-blank, and assertion–reason formats.
  • QA pairs are validated with OLMO-3-7B-Instruct using self-consistency checks and evaluated through exact match, ROUGE, and semantic similarity.
  • External retain sets use claims from semantically similar cited papers, while internal retain sets use contextual QA pairs from the same papers without claim overlap.

5 Experimental Set-up

The experiments evaluate representative unlearning methods on OLMO and LLAMA models using domain-specific, retracted-paper, and general-knowledge measures.

  • OLMo-3-7B-Instruct and LLaMA-3-8B-Instruct are evaluated with full-parameter and rank-8 LoRA training.Training uses a single 80 GB A100 GPU.
  • The benchmark compares Gradient Difference, NPO, NPO+RT, SimNPO, and SimNPO with a retain set.All methods are applied using either forget set F1 or F2.
  • Exact Match and ROUGE-L F1 measure task-specific performance, while MMLU, ARC-Challenge, and HellaSwag measure general knowledge retention.Exact Match captures strict correctness, whereas ROUGE-L F1 is paraphrase-tolerant.

6 Results

Unlearning substantially suppresses performance on targeted forget sets but transfers weakly to complementary paraphrases, while retain-objective methods better preserve useful knowledge.

  • Table 6 reports LoRA unlearning results for GD and NPO+RT on the retracted-paper dataset across OLMO and LLAMA.Downward arrows denote forgetting, while upward arrows denote retention and generalization.
  • Forgetting is largely localized to the unlearned forget set, with minimal transfer to the complementary set.This pattern appears across the evaluated representative methods.
  • All representative methods significantly reduce targeted-set performance while leaving the counterpart largely unaffected.The evaluation uses unlearning on either F1 or F2.
  • Current methods mainly suppress surface-level patterns rather than removing underlying claim-level knowledge.The conclusion follows from forgetting remaining subset-specific across paraphrased question-answer sets.
  • GD, NPO+RT, and SimNPO+RT preserve strong performance on external and internal retain sets, unlike NPO and SimNPO.NPO and SimNPO show larger retention drops because they lack an explicit retain objective.

7 Conclusion

The paper introduces Scientific Claim Unlearning and SciUnlearn to test whether language models forget scientific claims rather than only individual training instances.

  • Scientific Claim Unlearning evaluates claim-level forgetting using the SciUnlearn benchmark, including a real-world retracted-claim subset.The benchmark tests whether unlearning transfers beyond the specific instances used during training.
  • Existing unlearning methods mainly suppress specific training instances and transfer only limitedly to paraphrased or complementary forget sets.The paper concludes that specialized algorithms are needed for scientific claim-level removal.
  • The findings suggest that scientific claim unlearning requires algorithms designed to remove underlying claim knowledge.

Limitations

The study is limited by its method coverage, the small retracted-paper subset, and evaluation restricted to 7B–8B models.

  • The evaluation covers optimization-based unlearning methods but excludes mechanistic and representation-level approaches.Those approaches may better target internal claim representations, according to the authors.
  • The real-world retracted-paper subset is small, limiting the breadth of conclusions drawn from that setting.The authors propose expanding it across more retracted or explicitly falsified findings and scientific domains.
  • Experiments are restricted to 7B–8B models because full-parameter unlearning is computationally costly.The behavior of larger frontier-scale models remains unexamined.

H Distinguishing Memorization from General Reasoning

The evaluation combines likelihood-based membership signals with answer-order perturbations to test whether SciUnlearn measures pretrained knowledge rather than reasoning or positional cues alone.

  • Likelihood-based membership signals: Member samples have lower average NLL than non-members (0.24 vs. 3.88) and higher Min-K% probability at K = 20% (0.73 vs. 0.47).The separation provides evidence that OLMo encountered benchmark content during pretraining.
  • Robustness to answer-order perturbations: The answer-order test enumerates 24 permutations for each of 10 four-option questions, producing 240 variants.This intervention tests whether performance depends on answer-position cues.
  • Robustness to answer-order perturbations: Before unlearning, the model answers 227 of 240 variants correctly (94.6%), consistently selecting the same answer content across position changes.This rules out a fixed answer-position preference and indicates stable access to question-specific knowledge.
  • Robustness to answer-order perturbations: After unlearning, accuracy falls to 114 of 240 variants (47.5%), a decrease of 47.1 percentage points.The reduction persists across answer positions, making a purely positional explanation unlikely.
  • Combined interpretation: Together, likelihood separation and the order-invariant decline provide independent evidence that SciUnlearn captures knowledge acquired from pretraining.Neither analysis alone definitively separates memorization from every form of generalization.

I Free-Form Generation Evaluation

The free-form experiment tests whether GD unlearning affects open-ended generation, while additional analyses examine locality, question-format sensitivity, and recurring dataset fragility.

  • I Free-Form Generation Evaluation: The experiment constructs 50 open-ended questions from 50 computer-science claims and evaluates target-answer likelihood with NLL and Min-K% statistics.Exact-match and ROUGE are considered poorly suited to responses with multiple valid surface forms.
  • I Free-Form Generation Evaluation: After GD unlearning, mean NLL rises from 1.39 to 2.59 while mean Min-K% probability falls from 0.58 to 0.47.These changes indicate lower likelihood assigned to the targeted information.
  • I Free-Form Generation Evaluation: The free-form result is preliminary because it uses only 50 questions and one model–algorithm configuration.It nevertheless provides evidence that the effect extends beyond structured QA formats.
  • J Reference Score Drift and Membership Inference Attack: Direct members show stronger forgetting signatures than semantic members, with forgetting mostly localized to F1 and weak transfer to semantically related claims in F2.The analysis uses direct members, semantic members, and non-members with Min-K% probability and reference-model score signals.
  • N.1 Error Distribution Across Question Formats: Across methods, error rates are higher on the trained forget set but drop sharply on a disjoint forget set, indicating weak generalization to differently phrased equivalents.The pattern suggests forgetting remains largely confined to exact training instances.
  • N.1 Error Distribution Across Question Formats: Structured True/False and Assertion/Reason formats show stronger forgetting, whereas MCQ and Fill-in-the-blank show lower degradation.The authors interpret current methods as inducing surface-level suppression rather than removing underlying claim-level knowledge.
  • N.1 Error Distribution Across Question Formats: On retain sets, Fill-in-the-blank questions degrade most, MCQs are retained most, and True/False falls between them.Retention is strongest for formats requiring recognition rather than generation.
  • N.1 Error Distribution Across Question Formats: Assertion–Reason errors often preserve individual factual judgments while degrading whether the reason logically explains the assertion.This suggests factual knowledge is retained while relational structure between concepts is degraded.

N.3 Question Density Analysis of Error-Prone Papers

Error-prone papers are disproportionately dense in evaluation questions, while cross-forget-set spillover varies by paper and is associated with citation-based embeddedness. Lower-citation papers tend to show broader spillover, whereas highly cited papers appear more resistant.

  • Question Density: 27.6 questions per paper was the average among the 10 most error-prone papers, with all selected papers above the dataset median.Nine of ten exceeded the 80th percentile in question coverage.
  • Question Density: Math-word embedding in math search and semantic extraction had the highest density, with 38 questions at the 100th percentile.
  • Question Density: Highly represented papers are more likely to be among the most error-prone cases, suggesting that exposure frequency and claim density contribute to unlearning difficulty.
  • Cross-Forget-Set Spillover: Spillover across paraphrased forget sets is inconsistent across papers but follows the same pattern across all six algorithm–configuration combinations.This consistency suggests that susceptibility is a property of the papers rather than an artifact of a particular unlearning algorithm.
  • Citation-Based Embeddedness: Papers forgotten by both forget sets tend to have lower normal and connected citation counts than papers affected only by the active forget set.The analysis uses KDE-smoothed distributions of both citation measures for SINGLE and OVERLAPPED paper groups.
  • Citation-Based Embeddedness: Lower-citation papers may be more vulnerable to broad spillover, while highly cited papers may require stronger or more comprehensive signals to remove claim-level knowledge beyond surface forms.The interpretation attributes this difference to the breadth and redundancy of direct and derivative textual contexts in pretraining corpora.
Loading 2608.20960v1…