PaperlayerView exact revision on arXiv

Source-linked AI summary

How Do LLMs Change Predictions Under Negation?

Jongwook Yoon, Jongwon Lim, Sungjib Lim, Woojin Cho, Yohan Jo

arXiv:2610.09571v1cs.CLcs.AI

TL;DR

LLMs often fail to reject a correct answer under negation. The paper evaluates this failure and mechanistically examines how models process negation, finding that they suppress the original answer while promoting favored alternatives. It uses this diagnosis to motivate confidence-adaptive training that reduces negation failures while largely preserving general performance.

  • Problem

    Whether LLMs use knowledge of the original answer to determine what negation excludes remains unclear, despite persistent difficulty processing negated prompts.

    Source: §1 · p. 1

  • Method

    The paper evaluates negation behavior and traces its mechanism, identifying attention heads and MLP neurons involved before proposing stronger corrections for more confident original predictions.

    Source: §1 · pp. 1–2

  • Results

    In 37.4–71.1% of originally correct cases, models repeat the same answer under negation; ALiT improves negation while largely preserving general performance.

    Source: §1 · p. 2, §6 · p. 10

  • Takeaways & Limitations

    LLM negation relies on suppressing the original prediction and promoting a favored same-category alternative, leaving failures when correction is insufficient or candidate preferences conflict.

    Source: §1 · pp. 2–3

Abstract

from arXiv · show

Negation is an essential feature of human language, yet large language models (LLMs) remain unreliable in processing it. We evaluate recent open-source and closed-source LLMs on our negation benchmark and find that, in 37-71% of cases, they repeat the same answer under negation (e.g., "Madrid" for "What is not the capital of Spain?"). To understand and address this brittleness, we mechanistically examine how models operate under negation. Our main finding is that specialized attention heads and MLP neurons jointly implement negation by (1) suppressing retrieval of the original answer (e.g., "Madrid") while (2) promoting a favored candidate within the answer category (e.g., "Paris"). This contrasts with accounts of human negation processing, in which information about the original answer helps to determine what should be excluded. Furthermore, we find that this difference from human processing is a key source of negation failures: the model's mechanism relies on suppressing the original answer rather than using it to determine what to exclude, so the model can repeat the original answer when suppression is too weak or when a bias toward particular answers prevents it from selecting an alternative. To address this weakness in the model's negation mechanism, we propose a training objective that requires larger shifts in answer preference for more confident original predictions, and show that it reduces negation failures with less degradation of general capabilities than standard fine-tuning baselines. Together, our results demonstrate how mechanistic analysis can reveal why a linguistic capability fails and guide training that targets the underlying limitation.

1 INTRODUCTION

The paper investigates why LLMs often repeat an answer under negation and finds that their mechanism suppresses the original answer while promoting a favored alternative. This mechanism can fail when suppression is insufficient, motivating a confidence-adaptive training objective.

1 INTRODUCTION

  • In 37.4–71.1% of originally correct cases, models repeat the same answer under negation.The evaluation covers recent open- and closed-source models on six benchmarks spanning factual QA, visual QA, and logical reasoning.Source: §1 · p. 2
  • Negation weakens attention heads retrieving the original answer, strengthens heads retrieving category information, and activates MLP neurons promoting category-specific candidates.The mechanism largely preserves answer-category information while changing candidate preferences.Source: §1 · p. 2, Figure 1 · p. 2
  • Because negation relies on suppressing the original answer rather than using it to determine what to exclude, failures arise when suppression is too weak relative to its support.That support correlates with confidence in the original answer.Source: §1 · p. 2
  • The proposed training objective requires stronger corrections for more confident original predictions and reduces negation failures while preserving performance on non-negated prompts.The authors report better results than multiple baselines.Source: §1 · pp. 2–3

2 EVALUATING NEGATION PROCESSING IN LLMS

The evaluation measures whether models change a correct answer under negation while keeping alternatives within the requested answer category. Models often produce category-appropriate alternatives when they change answers, but frequently repeat the original answer, especially when confidence or negation bias favors it.

2 EVALUATING NEGATION PROCESSING IN LLMS

  • The benchmark pairs prompts that differ by negation across six datasets covering factual association, logical reasoning, and visual question answering.Four negation forms are constructed for each example.Source: §2 · p. 3
  • The dataset contains 105,048 examples, split into training, validation, and test sets at a 4:1:5 ratio.Training and validation support mechanistic analyses; the test split is reserved for final training-method evaluation.Source: §2 · p. 3
  • Across models, when models change their answers, category-preservation rates are high (87.2–97.8%), yet models often repeat original answers under negation, with answer-change rates of only 28.9–62.6%.Source: §2 · p. 4
  • Higher confidence in the original answer is associated with more frequent repetition, and answer changes drop when the most common negated response is the original answer.The paper calls this systematic preference negation bias.Source: §2 · p. 4, Figure 2 · p. 4

3 LOCALIZING THE EFFECT OF NEGATION

The analysis traces how negation reaches the final prompt token and tests whether a shared negation-related direction causally changes downstream computation. It finds that negation is transmitted mainly through other prompt tokens, and removing the direction can restore the original answer.

3 LOCALIZING THE EFFECT OF NEGATION

  • The analysis focuses on four open-source models, uses Gemma 3-12B-IT for its main results, and examines cases with an appropriate alternative under negation.It distinguishes answer, answer-category, negation, and final-prompt tokens.Source: §3 · p. 4

3.1 TRACING THE INFORMATION FLOW OF NEGATION

  • Negation reaches the final prompt token mainly through other prompt tokens: category paths peak at L8–11, then category and other-token paths at L22–25; the direct path has a 1.3% increase.Answer-token path effects peak later and mainly reflect changed attention weights rather than changed value-vector information.Source: §3.1 · p. 5

3.2 A CAUSAL NEGATION SIGNAL AT THE FINAL PROMPT TOKEN

  • Removing the negation direction around L25 restores the original answer in up to 48.6% of cases and reverses downstream attention and MLP changes.The effect is measured across six benchmarks in Figure 3.Source: Figure 3 · p. 5, §3.2 · p. 6

4 HOW LLMS GENERATE ALTERNATIVES UNDER NEGATION

The section identifies attention heads and MLP neurons involved in generating alternatives under negation, then examines how their effects reshape the final-token representation. The model largely preserves answer-category information while shifting preference toward particular candidates.

4.1 COMPONENTS THAT IMPLEMENT NEGATION

  • The top 1% of downstream neurons (3,533 neurons) accounts for 39.7% of positive attribution, and restoring selected components changes answer preference more than matched random controls.Figure 4 summarizes cumulative component scores and restoration effects.Source: §4.1 · p. 6, Figure 4 · p. 7
  • Some heads reduce attention to answer tokens, weakening retrieval of the original answer; restoring their non-negated outputs increases preference for it.The example head L31H13 shows this reduction under negation.Source: §4.1 · p. 6
  • Other heads promote the answer category, while high-attribution MLP neurons favor particular candidates without reliably adjusting their preferences to exclude the original answer.The combined mechanism can fail when the original answer is itself among the favored candidates.Source: §4.1 · p. 7

4.2 HOW NEGATION RESHAPES THE REPRESENTATION

  • Negation largely preserves answer-category information while shifting candidate preference in the learned spaces, with changes tending toward the most frequent negated answer in each category.The category-space and candidate-preference-space findings are shown in Figure 5.Source: Figure 5 · p. 7, §4.2 · p. 8

5 WHY NEGATION FAILS AND HOW TO IMPROVE IT

The section diagnoses repeated answers under negation as cases where active correction is too weak to overcome original-answer support, then introduces confidence-adaptive training. ALiT raises answer change in- and out-of-domain while largely preserving category consistency and general performance.

5.1 DIAGNOSING FAILURE CASES

  • Negation effects remain present but weaker in repeated-answer failures; strengthening selected changes lowers the original-answer match rate from 100% to 57.6% when heads and neurons are combined.The association between original-answer confidence and repetition motivates stronger corrections for more confident predictions.Source: §5.1 · p. 8

5.2 IMPROVING NEGATION THROUGH CONFIDENCE-ADAPTIVE TRAINING

  • ALiT extrapolates the model’s negation-induced logit change until an alternative beats the original by a confidence-dependent margin, then trains on the resulting distribution.The target uses the smallest β ≥1 that makes another token exceed the original top token by at least αm_i.Source: §5.2 · pp. 8–9
  • Answer change with ALiT rises from 58.1% to 96.6% in-domain and from 46.9% to 85.0% out-of-domain, while category preservation remains above 97%.Original-question accuracy and general-capability performance remain close to the base model; UL and DPO reduce category preservation, while SFT sharply reduces answer diversity.Source: §5.2 · p. 9, Table 2 · p. 9
  • ALiT improves answer change both when the original differs from the most frequent negated answer (67.1%) and when they match (40.2%).SFT reaches 94.8% in the differing-answer group but 18.8% in the matching-answer group.Source: §5.2 · p. 9

6 CONCLUSION

The paper explains negation failures through mechanisms that suppress original-answer retrieval while promoting alternatives, and uses this diagnosis to guide adaptive training. It also describes the benchmark construction, related work, mechanistic methods, AI use, and reproducibility materials.

6 CONCLUSION

  • ALiT improves negation while largely preserving general performance and avoiding the sharp loss of answer diversity observed with SFT.The conclusion attributes failures to insufficient correction or candidate preferences that do not reliably adjust to which answer must be excluded.Source: §6 · p. 10

AI USE STATEMENT

  • The authors used generative AI for research feedback, writing, translation, analysis, and interpretation, while independently reviewing research decisions and validating AI-assisted work.They report not using these tools to implement methods, formulate hypotheses, or generate synthetic datasets.Source: AI USE STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • The paper directs readers to appendices for benchmark procedures, evaluation metrics, mechanistic-analysis settings, ALiT and baseline definitions, and training hyperparameters.Additional mechanistic results are also provided in Appendix E.Source: REPRODUCIBILITY STATEMENT · p. 10

A.1 NEGATION PROCESSING IN LARGE LANGUAGE MODELS AND HUMANS

  • Unlike Zhou et al. (2026), this analysis traces original-answer retrieval and separates preserved answer-category information from shifts in candidate preference.The authors describe their analysis as finer-grained across domains and negation forms.Source: §A.1 · p. 16
  • Human accounts propose representing non-negated content before verifying or revising it, while later work allows context and alternatives to interact during meaning formation.These accounts motivate examining how LLMs use information supporting the original answer without assuming the same mechanism.Source: §A.1 · p. 16

A.2 MECHANISTIC INTERPRETABILITY

  • The mechanistic analysis combines information-flow tracing and component identification with DAS-learned subspaces that test causal roles on held-out predictions.It distinguishes answer-category information from candidate preference because geometric separation alone does not establish different causal roles.Source: §A.2 · p. 16

B.1 BENCHMARK CONSTRUCTION

  • 105,048 reviewed pairs remain from 126,756 constructed pairs, split into 41,957 training, 10,348 validation, and 52,743 test pairs.The benchmark draws on six source benchmarks; construction combines task structure, syntactic analysis, and deterministic transformations, with supporting contexts or images preserved.Source: §B.1 · pp. 16–17

B.2 BENCHMARK VALIDATION

The benchmark was filtered using an LLM judge that assessed grammaticality, unambiguity, and negation-scope fidelity, with a placeholder-based adaptation for RippleEdits completion inputs. Filtering primarily removed examples judged to have scope-fidelity problems, and human checks assessed the reliability of those judgments.

B.2 BENCHMARK VALIDATION

  • The judge labeled each example against three independent criteria and excluded any example failing at least one: grammaticality, unambiguity, and scope fidelity.Scope fidelity required negation to apply to the entire original proposition while preserving its other semantic content.Source: §B.2 · pp. 19, 20, 22
  • For RippleEdits, a completion-task benchmark, the authors inserted a placeholder into the blank so the input could be evaluated as a complete sentence.They used a specific validation prompt for this source benchmark.Source: §B.2 · pp. 20, 23, 24
  • 17.13% of examples failed at least one criterion, leaving 105,048 of 126,756; scope-fidelity failures accounted for 16.90%.Unambiguity failures accounted for 5.07% of examples, and PASS rates were consistent across the six source benchmarks.Source: §B.2 · p. 20, Table 4 · p. 20, Table 5 · p. 20
  • 84.29% of scope-fidelity failures used never, which the judge often interpreted as adding temporal meaning that altered the original question’s semantics.The authors attributed these judgments primarily to never’s temporal meaning.Source: §B.2 · p. 20
  • Human annotators unanimously agreed on grammaticality and unambiguity in over 98% of cases, but unanimous agreement was 77.08% for scope fidelity and κ was 0.440.For examples using never, unanimous scope-fidelity agreement was 71.11%.Source: §B.2 · p. 21, Table 6 · p. 21
  • Human–LLM agreement exceeded 90% for grammaticality and unambiguity and was 84.58% for scope fidelity, with Cohen’s κ of 0.334.The authors noted that GPT-5.6 Luna applied a particularly strict scope-fidelity interpretation, especially to examples involving never.Source: §B.2 · p. 21, Table 7 · p. 21

C EVALUATION DETAILS

The appendix details the evaluation cascade and validation, then specifies the interventions used to trace negation information flow, remove the negation signal, and identify components involved in answer changes.

C EVALUATION DETAILS

  • Three isolated judge calls assess original-answer correctness, repetition under negation, and whether a changed negated answer has the expected answer type.The cascade evaluates repetition only after a correct original answer, and answer type only when the negated answer does not repeat it.Source: C EVALUATION DETAILS · pp. 24, 26
  • Human–LLM agreement was 84.62%–98.04% across criteria and 96.06% overall, with Cohen’s κ of 0.9212.Agreement percentages use non-null human–LLM decision pairs; cases outside a criterion’s applicable cascade subset are excluded.Source: C EVALUATION DETAILS · p. 25, Table 9 · p. 27

D.1 DETAILS ON THE INFORMATION FLOW EXPERIMENT

  • Information-flow interventions replace a token group’s attention contribution at a destination position with its paired-prompt contribution.Contributions retain the model’s attention weights without renormalizing within the selected group, so effects can reflect attention changes, value-vector information, or both.Source: §D.1 · p. 28

D.2 DETAILS ON THE NEGATION SIGNAL EXPERIMENT

  • The negation signal is estimated by averaging final-token differences within each benchmark, then averaging benchmark means with equal weights.Steering subtracts that mean difference from the negated prompt’s final-token representation after a selected block.Source: §D.2 · p. 29

D.3 DETAILS ON THE COMPONENT IDENTIFICATION EXPERIMENT

  • Component identification ranks attention heads by attention-weight changes and MLP neurons by signed attribution, then tests selected components against random sets.Components are selected on training examples and fixed before validation; interventions replace their final-prompt-token outputs or activations with non-negated values.Source: §D.3 · p. 30

D.4 DETAILS ON THE SUBSPACE EXPERIMENT

The appendix describes subspaces for answer-category and candidate preferences, how they are learned and evaluated, and extended negation-evaluation patterns across models and benchmarks.

D.4 DETAILS ON THE SUBSPACE EXPERIMENT

  • Distributed Alignment Search identifies a category space C that changes preferred answer category and a candidate-preference space Ls that changes preference within a category.The two spaces are learned with separate objectives.Source: §D.4 · p. 31
  • Subspace interventions replace the residual representation’s component along an orthonormal basis with another prompt’s component, leaving the orthogonal component unchanged.The model continues computation from the modified representation.Source: §D.4 · p. 31
  • Training pairs within a category target a particular alternative answer, whereas cross-category pairs target the answer category without requiring a particular answer.The candidate-preference objective increases the full generated answer’s probability; the category objective increases probability for answers from the target category.Source: §D.4 · p. 31
  • Causal validation tests Ls on held-out same-category pairs and C on held-out different-category pairs, with category predictions based on four fixed reference answers.The category evaluation does not cover all possible answers in each category; learned spaces are also compared with random spaces and residual-representation interventions.Source: §D.4 · p. 32

E.1 EXTENDED RESULTS ON NEGATION EVALUATION

  • 42 of 44 evaluated model–benchmark combinations show a lower answer-change rate for the matching negation-bias group, though the pattern is not universal.The strength and shape of the confidence–answer-change association vary across models rather than showing a uniform decrease.Source: §E.1 · p. 33, Figure 7 · p. 33, Figure 8 · p. 34

E.2 EXTENDED RESULTS ON THE INFORMATION FLOW EXPERIMENT

Extended experiments trace information flow and remove negation-induced directions across several models. The results show indirect information-flow peaks and model-dependent layers for negation-direction removal.

E.2 EXTENDED RESULTS ON THE INFORMATION FLOW EXPERIMENT

  • 15.1% is the largest normalized log-probability gap shift from negation-token contributions to answer-category tokens, at layers 8–11 in Gemma 3-12B-IT.Paths from answer-category and other tokens to the final prompt token peak later at 21.9% each; the direct negation-to-final-token path reaches 1.3%.Source: §E.2 · p. 35
  • The peaks support an indirect route through intervening prompt tokens, but separate interventions do not establish a unique sequential circuit.The answer-token curve also combines attention-weight and value changes.Source: §E.2 · p. 35
  • Qwen3.5-9B’s information-flow analysis covers 2,304 validation pairs, but generation-time interventions make its effects non-comparable with prompt-only results.Its mixed softmax and linear attention require adapted contribution decompositions.Source: §E.2 · p. 36, Figure 11 · p. 38

E.3 EXTENDED RESULTS ON THE NEGATION SIGNAL EXPERIMENT

  • The fixed-dose negation-direction-removal experiments intervene only at the final prompt token, and their selected layers are not uniformly the layerwise maxima.The experiments cover Gemma 3-4B-IT, Gemma 3-12B-IT, Llama 3.1-8B-Instruct, and Qwen3.5-9B.Source: §E.3 · p. 39, Table 10 · p. 39
  • The exploratory β = 1 sweep’s peak negation-direction-removal layer is L18 for Gemma 3-4B-IT, L25 for Gemma 3-12B-IT, L13 for Llama 3.1-8B-Instruct, and L18 for Qwen3.5-9B.The most effective layer differs across models.Source: §E.3 · p. 39, Figure 12 · p. 40, Figure 13 · p. 41

E.4 ANALYSIS ON THE ORIGINAL ANSWER HEADS

Restoring selected attention-head outputs shifts answer preference toward the original prompt in Gemma models, while an individual head can affect the original answer and alternative differently.

E.4 ANALYSIS ON THE ORIGINAL ANSWER HEADS

  • For L31H13, restoring only the answer-token contribution increases the original answer’s full-sequence log-probability by 0.129 nats (95% interval [0.102, 0.158]) and decreases the alternative’s by 0.028 nats.Source: §E.4 · p. 43
  • 21.4% is the normalized log-probability gap shift from jointly restoring five selected Gemma 3-12B-IT heads, versus 2.1% for matched random sets on average.Joint restoration includes contributions from all input tokens.Source: §E.4 · p. 43
  • 29.5% is the gap shift from restoring five selected Gemma 3-4B-IT heads, compared with 12.0% for random heads.Head indices were selected separately for each model, supporting a contribution of selected heads without implying identical roles.Source: §E.4 · p. 43

E.5 ANALYSIS ON THE ANSWER CATEGORY HEADS

The analyzed heads support answer-category information under negation, but do not reliably identify which alternative the model chooses. Interventions show that these heads can shift preference away from an alternative and toward the original answer.

E.5 ANALYSIS ON THE ANSWER CATEGORY HEADS

  • Across 36 validation examples, L25H8’s mean candidate-pool uplift is 0.081 without negation and 0.069 with negation.Source: §E.5 · p. 44
  • L31H12 raises candidate-pool scores by 0.180 on average while reducing preference for the original answer, but its output does not identify the chosen alternative.Source: §E.5 · p. 44
  • Removing L25H8 raises the original answer’s full-sequence log-probability by 1.468 nats and lowers the alternative’s by 0.489 nats.Source: §E.5 · p. 44
  • Restoring L31H12 and L26H3 to their non-negated outputs reduces the alternative-minus-original first-token logit gap by 0.378 and 0.125, respectively.The 95% bootstrap intervals are [0.125, 0.682] and [0.047, 0.208].Source: §E.5 · p. 44

E.6 ANALYSIS ON THE NEGATION-SPECIFIC NEURONS

The analyses trace negation-related effects to concentrated neuron contributions, answer-category subspaces, and changes in attention that can be strengthened to overturn some repeated answers. They also report boundaries on the evidence and describe the training baselines and general-capability measures used for comparison.

E.6 ANALYSIS ON THE NEGATION-SPECIFIC NEURONS

  • Among 353,280 downstream neurons, the top 1% accounts for 39.7% of positive attribution; restoring 256 selected neurons shifts the normalized log-probability gap by 24.7%, versus 0.1% for matched random neurons.Doubling natural activation differences increases the alternative-over-original gap by 7.20 nats mainly through suppression of the original answer; the alternative’s absolute log-probability also decreases.Source: §E.6 · p. 45
  • Joint restoration reduces original-answer matching from 56.6% to 17.4% in a 612-pair follow-up, and natural-value restoration reduces matching relative to random restoration in all four models.The 72 benchmark–relation groups show 21.5% average overlap between relation-specific and global top-256 neurons; joint head-and-neuron restoration yields a 35.2% normalized gap shift, larger than either alone but smaller than their sum.Source: §E.6 · p. 45, Figure 15 · p. 46

E.7 EXTENDED RESULTS ON THE SUBSPACE EXPERIMENT

  • Learned candidate-preference spaces yield larger normalized gap shifts than rank-matched random spaces: 29.9%–38.7% for Gemma, 9.1%–13.6% for Llama, and 12.4%–45.1% for Qwen.Source: §E.7 · p. 47
  • Learned category spaces shift category scores more than random controls, while within-category answer-identity shifts range from 1.0% to 4.6%; these are different metrics, not a separation ratio.The experiments establish causal effects of learned coordinates under the tested replacements.Source: §E.7 · pp. 47–48

E.8 FAILURE DIAGNOSIS

  • Negated answer-token attention remains higher in failing forms than successful ones within the same 150 questions: 16.2% versus 8.5%.Strengthening the identified attention and neuron changes reduces repetition, but the original-prompt path-support test does not establish stronger original-answer retrieval in failures for any of the four models.Source: §E.8 · pp. 49–50
  • Joint intervention lowers original-answer matching to 57.6%, 66.7% in Gemma 3-4B-IT, 64.0% in Llama 3.1-8B-Instruct, and 54.3% in Qwen3.5-9B.For Gemma 3-12B-IT, the joint intervention is 40.9 percentage points below layer-count-matched random controls; the effect exceeds matched random controls in all four models.Source: §E.8 · pp. 49–50, Table 16 · p. 50, Figure 16 · p. 51

E.9 TRAINING RESULTS

  • General-capability evaluation reports MMLU-Pro 5-shot exact match, HellaSwag 10-shot normalized accuracy, GSM8K 8-shot CoT exact match, and IFEval 0-shot prompt-level strict accuracy.Source: §E.9 · p. 52

F.1 BASELINES

  • The baselines test increasing a specified alternative with SFT, reducing the original answer’s probability with UL, and preferring the alternative over the original with DPO.DPO is a general preference-learning method, not a negation-specific algorithm.Source: §F.1 · p. 53

F.2 TRAINING SETUP

Training uses paired negation prompts from PopQA to post-train Gemma-3-12B-IT, with a shared objective to preserve original-prompt predictions. ALiT targets depend on original-answer confidence, and evaluation covers in-domain, pooled out-of-domain, and general-capability performance.

F.2 TRAINING SETUP

  • Gemma-3-12B-IT is post-trained on 14,951 paired prompts from 4,534 PopQA source questions spanning 15 relations, for two epochs.Validation and test examples are not used to construct training targets.Source: §F.2 · p. 54
  • All methods include a shared objective preserving original-prompt predictions using a full-vocabulary forward KL at answer positions and coefficient λ = 4.The model distributions are compared while conditioned on the same stored answer prefix.Source: §F.2 · p. 54
  • ALiT chooses the smallest extrapolation coefficient satisfying its confidence-margin condition, bounded below by 1 and capped at 20; capped or infeasible targets need not meet the margin.When the natural negated prediction already meets the condition, or no feasible direction exists, β = 1 is used.Source: §F.2 · p. 54
  • Evaluation uses PopQA test examples in-domain, five other benchmarks pooled out-of-domain, and the arithmetic mean of four benchmark scores for general capability.The four general-capability benchmarks are MMLU-Pro, HellaSwag, GSM8K, and IFEval.Source: §F.2 · p. 54
Loading 2610.09571v1…